Message ID | 7F861DC0615E0C47A872E6F3C5FCDDBD011A99C6@BPXM14GP.gisp.nec.co.jp (mailing list archive) |
---|---|
State | Superseded, archived |
Headers |
Return-Path: <dev-bounces@dpdk.org> X-Original-To: patchwork@dpdk.org Delivered-To: patchwork@dpdk.org Received: from [92.243.14.124] (localhost [IPv6:::1]) by dpdk.org (Postfix) with ESMTP id 25AA868B7; Thu, 11 Sep 2014 09:55:05 +0200 (CEST) Received: from tyo202.gate.nec.co.jp (TYO202.gate.nec.co.jp [210.143.35.52]) by dpdk.org (Postfix) with ESMTP id AD4415901 for <dev@dpdk.org>; Thu, 11 Sep 2014 09:55:02 +0200 (CEST) Received: from mailgate3.nec.co.jp ([10.7.69.192]) by tyo202.gate.nec.co.jp (8.13.8/8.13.4) with ESMTP id s8B80ETD015095 for <dev@dpdk.org>; Thu, 11 Sep 2014 17:00:14 +0900 (JST) Received: from mailsv.nec.co.jp (imss63.nec.co.jp [10.7.69.158]) by mailgate3.nec.co.jp (8.11.7/3.7W-MAILGATE-NEC) with ESMTP id s8B80C326629 for <dev@dpdk.org>; Thu, 11 Sep 2014 17:00:12 +0900 (JST) Received: from mail03.kamome.nec.co.jp (mail03.kamome.nec.co.jp [10.25.43.7]) by mailsv.nec.co.jp (8.13.8/8.13.4) with ESMTP id s8B80CTs023011 for <dev@dpdk.org>; Thu, 11 Sep 2014 17:00:12 +0900 (JST) Received: from bpxc99gp.gisp.nec.co.jp ([10.38.151.144] [10.38.151.144]) by mail03.kamome.nec.co.jp with ESMTP id BT-MMP-1866868; Thu, 11 Sep 2014 16:52:27 +0900 Received: from BPXM14GP.gisp.nec.co.jp ([169.254.1.238]) by BPXC16GP.gisp.nec.co.jp ([10.38.151.144]) with mapi id 14.02.0328.011; Thu, 11 Sep 2014 16:52:27 +0900 From: Hiroshi Shimamoto <h-shimamoto@ct.jp.nec.com> To: "dev@dpdk.org" <dev@dpdk.org> Thread-Topic: [memnic PATCH 7/7] pmd: split calling mbuf free Thread-Index: Ac/NlUxI5nJ3s/X2TP+cHArf+IKJdA== Date: Thu, 11 Sep 2014 07:52:26 +0000 Message-ID: <7F861DC0615E0C47A872E6F3C5FCDDBD011A99C6@BPXM14GP.gisp.nec.co.jp> Accept-Language: ja-JP, en-US Content-Language: ja-JP X-MS-Has-Attach: X-MS-TNEF-Correlator: x-originating-ip: [10.205.5.123] Content-Type: text/plain; charset="iso-2022-jp" Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 Cc: Hayato Momma <h-momma@ce.jp.nec.com> Subject: [dpdk-dev] [memnic PATCH 7/7] pmd: split calling mbuf free X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: patches and discussions about DPDK <dev.dpdk.org> List-Unsubscribe: <http://dpdk.org/ml/options/dev>, <mailto:dev-request@dpdk.org?subject=unsubscribe> List-Archive: <http://dpdk.org/ml/archives/dev/> List-Post: <mailto:dev@dpdk.org> List-Help: <mailto:dev-request@dpdk.org?subject=help> List-Subscribe: <http://dpdk.org/ml/listinfo/dev>, <mailto:dev-request@dpdk.org?subject=subscribe> Errors-To: dev-bounces@dpdk.org Sender: "dev" <dev-bounces@dpdk.org> |
Commit Message
Hiroshi Shimamoto
Sept. 11, 2014, 7:52 a.m. UTC
From: Hiroshi Shimamoto <h-shimamoto@ct.jp.nec.com> In rte_pktmbuf_free(), there might be cache miss/memory stall issue. In small packet case, it could harm the performance. From the result of memnic-tester, in less than 1024 frame size the performance could be improved. Using Xeon E5-2697 v2 @ 2.70GHz, 4 vCPU. size | before | after 64 | 5.55Mpps | 5.83Mpps 128 | 5.44Mpps | 5.71Mpps 256 | 5.22Mpps | 5.40Mpps 512 | 4.52Mpps | 4.64Mpps 1024 | 3.73Mpps | 3.68Mpps 1280 | 3.22Mpps | 3.17Mpps 1518 | 2.93Mpps | 2.90Mpps Signed-off-by: Hiroshi Shimamoto <h-shimamoto@ct.jp.nec.com> Reviewed-by: Hayato Momma <h-momma@ce.jp.nec.com> --- pmd/pmd_memnic.c | 6 +++--- 1 file changed, 3 insertions(+), 3 deletions(-)
Comments
2014-09-11 07:52, Hiroshi Shimamoto: > @@ -408,9 +408,9 @@ retry: > > rte_compiler_barrier(); > p->status = MEMNIC_PKT_ST_FILLED; > - > - rte_pktmbuf_free(tx_pkts[nr]); > } > + for (i = 0; i < nr; i++) > + rte_pktmbuf_free(tx_pkts[i]); > > /* stats */ > st->opackets += pkts; > You are bursting mbuf freeing. Why title is about "split"?
On Sep 24, 2014, at 10:20 AM, Thomas Monjalon <thomas.monjalon@6wind.com> wrote: > 2014-09-11 07:52, Hiroshi Shimamoto: >> @@ -408,9 +408,9 @@ retry: >> >> rte_compiler_barrier(); >> p->status = MEMNIC_PKT_ST_FILLED; >> - >> - rte_pktmbuf_free(tx_pkts[nr]); >> } >> + for (i = 0; i < nr; i++) >> + rte_pktmbuf_free(tx_pkts[i]); >> >> /* stats */ >> st->opackets += pkts; >> > > You are bursting mbuf freeing. Why title is about "split”? Maybe this should be a new API as in rte_pktmbuf_bulk_free(tx_pkts, nr); ?? This would remove the loop in the application and I know I have done the same thing for Pktgen too. > > -- > Thomas Keith Wiles, Principal Technologist with CTO office, Wind River mobile 972-213-5533
Hi Thomas, Keith, > Subject: Re: [dpdk-dev] [memnic PATCH 7/7] pmd: split calling mbuf free > > > On Sep 24, 2014, at 10:20 AM, Thomas Monjalon <thomas.monjalon@6wind.com> wrote: > > > 2014-09-11 07:52, Hiroshi Shimamoto: > >> @@ -408,9 +408,9 @@ retry: > >> > >> rte_compiler_barrier(); > >> p->status = MEMNIC_PKT_ST_FILLED; > >> - > >> - rte_pktmbuf_free(tx_pkts[nr]); > >> } > >> + for (i = 0; i < nr; i++) > >> + rte_pktmbuf_free(tx_pkts[i]); > >> > >> /* stats */ > >> st->opackets += pkts; > >> > > > > You are bursting mbuf freeing. Why title is about "split”? I thought that in this patch splits main loop operations to putting content and freeing mbuf, then took work "split", but I see "burst mbuf freeing" is preferable. > > Maybe this should be a new API as in rte_pktmbuf_bulk_free(tx_pkts, nr); ?? > This would remove the loop in the application and I know I have done the same thing for Pktgen too. Good point, yes, I'm thinking that having new API like rte_pktmbuf_(alloc|free)_bulk() is good to reduce TLS access and gain performance. I put that on my stack, but haven't had a time yet. Do you have any plan to do such thing? thanks, Hiroshi > > > > -- > > Thomas > > Keith Wiles, Principal Technologist with CTO office, Wind River mobile 972-213-5533
On Sep 24, 2014, at 8:12 PM, Hiroshi Shimamoto <h-shimamoto@ct.jp.nec.com> wrote: > Hi Thomas, Keith, > >> Subject: Re: [dpdk-dev] [memnic PATCH 7/7] pmd: split calling mbuf free >> >> >> On Sep 24, 2014, at 10:20 AM, Thomas Monjalon <thomas.monjalon@6wind.com> wrote: >> >>> 2014-09-11 07:52, Hiroshi Shimamoto: >>>> @@ -408,9 +408,9 @@ retry: >>>> >>>> rte_compiler_barrier(); >>>> p->status = MEMNIC_PKT_ST_FILLED; >>>> - >>>> - rte_pktmbuf_free(tx_pkts[nr]); >>>> } >>>> + for (i = 0; i < nr; i++) >>>> + rte_pktmbuf_free(tx_pkts[i]); >>>> >>>> /* stats */ >>>> st->opackets += pkts; >>>> >>> >>> You are bursting mbuf freeing. Why title is about "split”? > > I thought that in this patch splits main loop operations to putting content and > freeing mbuf, then took work "split", but I see "burst mbuf freeing" is preferable. > >> >> Maybe this should be a new API as in rte_pktmbuf_bulk_free(tx_pkts, nr); ?? >> This would remove the loop in the application and I know I have done the same thing for Pktgen too. > > Good point, yes, I'm thinking that having new API like rte_pktmbuf_(alloc|free)_bulk() > is good to reduce TLS access and gain performance. > I put that on my stack, but haven't had a time yet. > > Do you have any plan to do such thing? I do not have any plans, but the alloc would be good too. > > thanks, > Hiroshi > >>> >>> -- >>> Thomas >> >> Keith Wiles, Principal Technologist with CTO office, Wind River mobile 972-213-5533 Keith Wiles, Principal Technologist with CTO office, Wind River mobile 972-213-5533
diff --git a/pmd/pmd_memnic.c b/pmd/pmd_memnic.c index cc0ae25..1db065f 100644 --- a/pmd/pmd_memnic.c +++ b/pmd/pmd_memnic.c @@ -344,7 +344,7 @@ static uint16_t memnic_xmit_pkts(void *tx_queue, struct memnic_adapter *adapter = q->adapter; struct memnic_data *data = &adapter->nic->down; struct memnic_packet *p; - uint16_t nr; + uint16_t i, nr; int idx; struct rte_eth_stats *st = &adapter->stats[rte_lcore_id()]; uint64_t pkts, bytes, errs; @@ -408,9 +408,9 @@ retry: rte_compiler_barrier(); p->status = MEMNIC_PKT_ST_FILLED; - - rte_pktmbuf_free(tx_pkts[nr]); } + for (i = 0; i < nr; i++) + rte_pktmbuf_free(tx_pkts[i]); /* stats */ st->opackets += pkts;