From patchwork Tue Feb 27 05:41:17 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Tyler Retzlaff X-Patchwork-Id: 137295 X-Patchwork-Delegate: thomas@monjalon.net Return-Path: X-Original-To: patchwork@inbox.dpdk.org Delivered-To: patchwork@inbox.dpdk.org Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id A3D9B43C03; Tue, 27 Feb 2024 06:42:11 +0100 (CET) Received: from mails.dpdk.org (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id BDB6840295; Tue, 27 Feb 2024 06:42:10 +0100 (CET) Received: from linux.microsoft.com (linux.microsoft.com [13.77.154.182]) by mails.dpdk.org (Postfix) with ESMTP id 958BF402A7 for ; Tue, 27 Feb 2024 06:41:41 +0100 (CET) Received: by linux.microsoft.com (Postfix, from userid 1086) id B234720B74C1; Mon, 26 Feb 2024 21:41:40 -0800 (PST) DKIM-Filter: OpenDKIM Filter v2.11.0 linux.microsoft.com B234720B74C1 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linux.microsoft.com; s=default; t=1709012500; bh=v1tKUXog76Mxk6grlZIt7+LwqsEUivznLL61OoHb5T0=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=kIz4no/XMIXhlJTjSbjRINvMJXjS0Oy+seagqifOeGCeLw4+YIQyAsfaBkDlq23I8 2T785h68fa4m7LMJ6dRpWtq90nBfocA/IHOcxrDtmtagKi/gETpzPD/jPnOpxZGJ7u LG40pfBreqyBJltMKBDxoV0Ux0nU+F4sy7wcwkj0= From: Tyler Retzlaff To: dev@dpdk.org Cc: Ajit Khaparde , Andrew Boyer , Andrew Rybchenko , Bruce Richardson , Chenbo Xia , Chengwen Feng , Dariusz Sosnowski , David Christensen , Hyong Youb Kim , Jerin Jacob , Jie Hai , Jingjing Wu , John Daley , Kevin Laatz , Kiran Kumar K , Konstantin Ananyev , Maciej Czekaj , Matan Azrad , Maxime Coquelin , Nithin Dabilpuram , Ori Kam , Ruifeng Wang , Satha Rao , Somnath Kotur , Suanming Mou , Sunil Kumar Kori , Viacheslav Ovsiienko , Yisen Zhuang , Yuying Zhang , mb@smartsharesystems.com, Tyler Retzlaff Subject: [PATCH v6 01/23] mbuf: add accessors for rearm and Rx descriptor fields Date: Mon, 26 Feb 2024 21:41:17 -0800 Message-Id: <1709012499-12813-2-git-send-email-roretzla@linux.microsoft.com> X-Mailer: git-send-email 1.8.3.1 In-Reply-To: <1709012499-12813-1-git-send-email-roretzla@linux.microsoft.com> References: <1706657173-26166-1-git-send-email-roretzla@linux.microsoft.com> <1709012499-12813-1-git-send-email-roretzla@linux.microsoft.com> X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org RTE_MARKER typedefs are a GCC extension unsupported by MSVC. Provide inline functions to access compatible type pointer to rearm_data and rx_descriptor_fields1 which will allow direct references on the rte marker fields to be removed. Signed-off-by: Tyler Retzlaff --- lib/mbuf/rte_mbuf.h | 13 +++++++++++++ lib/mbuf/rte_mbuf_core.h | 11 ++++++++++- 2 files changed, 23 insertions(+), 1 deletion(-) diff --git a/lib/mbuf/rte_mbuf.h b/lib/mbuf/rte_mbuf.h index 286b32b..aa7495b 100644 --- a/lib/mbuf/rte_mbuf.h +++ b/lib/mbuf/rte_mbuf.h @@ -132,6 +132,19 @@ #endif } +static inline +uint64_t * +rte_mbuf_rearm_data(struct rte_mbuf *m) +{ + return (uint64_t *)&m->data_off; +} + +static inline +void * +rte_mbuf_rx_descriptor_fields1(struct rte_mbuf *m) +{ + return &m->packet_type; +} static inline uint16_t rte_pktmbuf_priv_size(struct rte_mempool *mp); diff --git a/lib/mbuf/rte_mbuf_core.h b/lib/mbuf/rte_mbuf_core.h index 5688683..7000c04 100644 --- a/lib/mbuf/rte_mbuf_core.h +++ b/lib/mbuf/rte_mbuf_core.h @@ -486,7 +486,12 @@ struct rte_mbuf { struct rte_mbuf *next; #endif - /* next 8 bytes are initialised on RX descriptor rearm */ + /** + * next 8 bytes are initialised on RX descriptor rearm + * + * To obtain a pointer to rearm_data use the rte_mbuf_rearm_data() + * accessor instead of directly referencing through the data_off field. + */ RTE_MARKER64 rearm_data; uint16_t data_off; @@ -522,6 +527,10 @@ struct rte_mbuf { * mbuf. Example: if vlan stripping is enabled, a received vlan packet * would have RTE_PTYPE_L2_ETHER and not RTE_PTYPE_L2_VLAN because the * vlan is stripped from the data. + * + * To obtain a pointer to rx_descriptor_fields1 use the + * rte_mbuf_rx_descriptor_fields1() accessor instead of directly + * referencing through the the anonymous union fields. */ union { uint32_t packet_type; /**< L2/L3/L4 and tunnel information. */ From patchwork Tue Feb 27 05:41:18 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Tyler Retzlaff X-Patchwork-Id: 137297 X-Patchwork-Delegate: thomas@monjalon.net Return-Path: X-Original-To: patchwork@inbox.dpdk.org Delivered-To: patchwork@inbox.dpdk.org Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id 7028843C03; Tue, 27 Feb 2024 06:42:27 +0100 (CET) Received: from mails.dpdk.org (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id 4002D42E59; Tue, 27 Feb 2024 06:42:14 +0100 (CET) Received: from linux.microsoft.com (linux.microsoft.com [13.77.154.182]) by mails.dpdk.org (Postfix) with ESMTP id BD5C442E28 for ; Tue, 27 Feb 2024 06:41:41 +0100 (CET) Received: by linux.microsoft.com (Postfix, from userid 1086) id C0D8320B74C2; Mon, 26 Feb 2024 21:41:40 -0800 (PST) DKIM-Filter: OpenDKIM Filter v2.11.0 linux.microsoft.com C0D8320B74C2 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linux.microsoft.com; s=default; t=1709012500; bh=Yy8P6Wt0r/jg+qdObE8aOhUH7XBv9xmNrQlsx890pro=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=gZHj7XRf/tPYlMtCyjcflobhOHw3YZgqXc+beW42Y3ERwwdDI9gcOSTYVvW1QVtql KgA6IWhEsm5y7HAB0qQVWO+p3w7UNYZvR25IEp79bD5PSMrldXNz5Khn0TsSE1N8Sd piYfw/vqJeNefe4JK/y1WXMlfQHUecgr1nWqKDgA= From: Tyler Retzlaff To: dev@dpdk.org Cc: Ajit Khaparde , Andrew Boyer , Andrew Rybchenko , Bruce Richardson , Chenbo Xia , Chengwen Feng , Dariusz Sosnowski , David Christensen , Hyong Youb Kim , Jerin Jacob , Jie Hai , Jingjing Wu , John Daley , Kevin Laatz , Kiran Kumar K , Konstantin Ananyev , Maciej Czekaj , Matan Azrad , Maxime Coquelin , Nithin Dabilpuram , Ori Kam , Ruifeng Wang , Satha Rao , Somnath Kotur , Suanming Mou , Sunil Kumar Kori , Viacheslav Ovsiienko , Yisen Zhuang , Yuying Zhang , mb@smartsharesystems.com, Tyler Retzlaff Subject: [PATCH v6 02/23] mbuf: consolidate driver asserts for mbuf struct Date: Mon, 26 Feb 2024 21:41:18 -0800 Message-Id: <1709012499-12813-3-git-send-email-roretzla@linux.microsoft.com> X-Mailer: git-send-email 1.8.3.1 In-Reply-To: <1709012499-12813-1-git-send-email-roretzla@linux.microsoft.com> References: <1706657173-26166-1-git-send-email-roretzla@linux.microsoft.com> <1709012499-12813-1-git-send-email-roretzla@linux.microsoft.com> X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Collect duplicated RTE_BUILD_BUG_ON checks from drivers and place them at global scope with struct rte_mbuf definition using static_assert. Signed-off-by: Tyler Retzlaff Acked-by: Konstantin Ananyev --- lib/mbuf/rte_mbuf_core.h | 34 ++++++++++++++++++++++++++++++++++ 1 file changed, 34 insertions(+) diff --git a/lib/mbuf/rte_mbuf_core.h b/lib/mbuf/rte_mbuf_core.h index 7000c04..36551c2 100644 --- a/lib/mbuf/rte_mbuf_core.h +++ b/lib/mbuf/rte_mbuf_core.h @@ -16,8 +16,11 @@ * New fields and flags should fit in the "dynamic space". */ +#include +#include #include +#include #include #include @@ -673,6 +676,37 @@ struct rte_mbuf { uint32_t dynfield1[9]; /**< Reserved for dynamic fields. */ } __rte_cache_aligned; +static_assert(!(offsetof(struct rte_mbuf, ol_flags) != + offsetof(struct rte_mbuf, rearm_data) + 8), "ol_flags"); +static_assert(!(offsetof(struct rte_mbuf, rearm_data) != + RTE_ALIGN(offsetof(struct rte_mbuf, rearm_data), 16)), "rearm_data"); +static_assert(!(offsetof(struct rte_mbuf, data_off) != + offsetof(struct rte_mbuf, rearm_data)), "data_off"); +static_assert(!(offsetof(struct rte_mbuf, data_off) < + offsetof(struct rte_mbuf, rearm_data)), "data_off"); +static_assert(!(offsetof(struct rte_mbuf, refcnt) < + offsetof(struct rte_mbuf, rearm_data)), "refcnt"); +static_assert(!(offsetof(struct rte_mbuf, nb_segs) < + offsetof(struct rte_mbuf, rearm_data)), "nb_segs"); +static_assert(!(offsetof(struct rte_mbuf, port) < + offsetof(struct rte_mbuf, rearm_data)), "port"); +static_assert(!(offsetof(struct rte_mbuf, data_off) - + offsetof(struct rte_mbuf, rearm_data) > 6), "data_off"); +static_assert(!(offsetof(struct rte_mbuf, refcnt) - + offsetof(struct rte_mbuf, rearm_data) > 6), "refcnt"); +static_assert(!(offsetof(struct rte_mbuf, nb_segs) - + offsetof(struct rte_mbuf, rearm_data) > 6), "nb_segs"); +static_assert(!(offsetof(struct rte_mbuf, port) - + offsetof(struct rte_mbuf, rearm_data) > 6), "port"); +static_assert(!(offsetof(struct rte_mbuf, pkt_len) != + offsetof(struct rte_mbuf, rx_descriptor_fields1) + 4), "pkt_len"); +static_assert(!(offsetof(struct rte_mbuf, data_len) != + offsetof(struct rte_mbuf, rx_descriptor_fields1) + 8), "data_len"); +static_assert(!(offsetof(struct rte_mbuf, vlan_tci) != + offsetof(struct rte_mbuf, rx_descriptor_fields1) + 10), "vlan_tci"); +static_assert(!(offsetof(struct rte_mbuf, hash) != + offsetof(struct rte_mbuf, rx_descriptor_fields1) + 12), "hash"); + /** * Function typedef of callback to free externally attached buffer. */ From patchwork Tue Feb 27 05:41:19 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Tyler Retzlaff X-Patchwork-Id: 137296 X-Patchwork-Delegate: thomas@monjalon.net Return-Path: X-Original-To: patchwork@inbox.dpdk.org Delivered-To: patchwork@inbox.dpdk.org Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id E355B43C03; Tue, 27 Feb 2024 06:42:21 +0100 (CET) Received: from mails.dpdk.org (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id 20352402B2; Tue, 27 Feb 2024 06:42:13 +0100 (CET) Received: from linux.microsoft.com (linux.microsoft.com [13.77.154.182]) by mails.dpdk.org (Postfix) with ESMTP id B0146402B2 for ; Tue, 27 Feb 2024 06:41:41 +0100 (CET) Received: by linux.microsoft.com (Postfix, from userid 1086) id CEA5020B74C3; Mon, 26 Feb 2024 21:41:40 -0800 (PST) DKIM-Filter: OpenDKIM Filter v2.11.0 linux.microsoft.com CEA5020B74C3 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linux.microsoft.com; s=default; t=1709012500; bh=3zEzVolRJmbWvBEOd3sMBl19ZxVX27jSMrbHklt/Aag=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=hFH2+CLSJTmjz5pzWh2Ok7cDr1NT8wisu2wsfpqNiT42YWby4OgUBEhOWVEYnuv6U YEGm0NbP/vRMCXrMABIoTIRPAM9T9sAukjry7LFusIIBVRMdcIQ7ILlLQGtdNv787A 3mmuecXbVJTpFaxbLnBhi1LZyhkwF/pAyT/B0+FI= From: Tyler Retzlaff To: dev@dpdk.org Cc: Ajit Khaparde , Andrew Boyer , Andrew Rybchenko , Bruce Richardson , Chenbo Xia , Chengwen Feng , Dariusz Sosnowski , David Christensen , Hyong Youb Kim , Jerin Jacob , Jie Hai , Jingjing Wu , John Daley , Kevin Laatz , Kiran Kumar K , Konstantin Ananyev , Maciej Czekaj , Matan Azrad , Maxime Coquelin , Nithin Dabilpuram , Ori Kam , Ruifeng Wang , Satha Rao , Somnath Kotur , Suanming Mou , Sunil Kumar Kori , Viacheslav Ovsiienko , Yisen Zhuang , Yuying Zhang , mb@smartsharesystems.com, Tyler Retzlaff Subject: [PATCH v6 03/23] common/idpf: use mbuf descriptor accessors Date: Mon, 26 Feb 2024 21:41:19 -0800 Message-Id: <1709012499-12813-4-git-send-email-roretzla@linux.microsoft.com> X-Mailer: git-send-email 1.8.3.1 In-Reply-To: <1709012499-12813-1-git-send-email-roretzla@linux.microsoft.com> References: <1706657173-26166-1-git-send-email-roretzla@linux.microsoft.com> <1709012499-12813-1-git-send-email-roretzla@linux.microsoft.com> X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org RTE_MARKER typedefs are a GCC extension unsupported by MSVC. Use new rte_mbuf_rearm_data and rte_mbuf_rx_descriptor_fields1 accessors that provide a compatible type pointer without using the marker fields. Signed-off-by: Tyler Retzlaff --- drivers/common/idpf/idpf_common_rxtx.c | 4 +- drivers/common/idpf/idpf_common_rxtx_avx512.c | 73 +++++++-------------------- 2 files changed, 18 insertions(+), 59 deletions(-) diff --git a/drivers/common/idpf/idpf_common_rxtx.c b/drivers/common/idpf/idpf_common_rxtx.c index 83b131e..62ddf2e 100644 --- a/drivers/common/idpf/idpf_common_rxtx.c +++ b/drivers/common/idpf/idpf_common_rxtx.c @@ -1595,7 +1595,6 @@ static inline int idpf_rxq_vec_setup_default(struct idpf_rx_queue *rxq) { - uintptr_t p; struct rte_mbuf mb_def = { .buf_addr = 0 }; /* zeroed mbuf */ mb_def.nb_segs = 1; @@ -1605,8 +1604,7 @@ /* prevent compiler reordering: rearm_data covers previous fields */ rte_compiler_barrier(); - p = (uintptr_t)&mb_def.rearm_data; - rxq->mbuf_initializer = *(uint64_t *)p; + rxq->mbuf_initializer = *rte_mbuf_rearm_data(&mb_def); return 0; } diff --git a/drivers/common/idpf/idpf_common_rxtx_avx512.c b/drivers/common/idpf/idpf_common_rxtx_avx512.c index f65e8d5..f9e2939 100644 --- a/drivers/common/idpf/idpf_common_rxtx_avx512.c +++ b/drivers/common/idpf/idpf_common_rxtx_avx512.c @@ -307,19 +307,6 @@ /* octet 15~14, low 16 bits pkt_len */ 0xFFFFFFFF /* pkt_type set as unknown */ ); - /** - * compile-time check the shuffle layout is correct. - * NOTE: the first field (lowest address) is given last in set_epi - * calls above. - */ - RTE_BUILD_BUG_ON(offsetof(struct rte_mbuf, pkt_len) != - offsetof(struct rte_mbuf, rx_descriptor_fields1) + 4); - RTE_BUILD_BUG_ON(offsetof(struct rte_mbuf, data_len) != - offsetof(struct rte_mbuf, rx_descriptor_fields1) + 8); - RTE_BUILD_BUG_ON(offsetof(struct rte_mbuf, vlan_tci) != - offsetof(struct rte_mbuf, rx_descriptor_fields1) + 10); - RTE_BUILD_BUG_ON(offsetof(struct rte_mbuf, hash) != - offsetof(struct rte_mbuf, rx_descriptor_fields1) + 12); uint16_t i, received; @@ -455,13 +442,7 @@ * add in the previously computed rx_descriptor fields to * make a single 256-bit write per mbuf */ - /* check the structure matches expectations */ - RTE_BUILD_BUG_ON(offsetof(struct rte_mbuf, ol_flags) != - offsetof(struct rte_mbuf, rearm_data) + 8); - RTE_BUILD_BUG_ON(offsetof(struct rte_mbuf, rearm_data) != - RTE_ALIGN(offsetof(struct rte_mbuf, - rearm_data), - 16)); + /* build up data and do writes */ __m256i rearm0, rearm1, rearm2, rearm3, rearm4, rearm5, rearm6, rearm7; @@ -476,13 +457,13 @@ rearm0 = _mm256_permute2f128_si256(mbuf_init, mb0_1, 0x20); /* write to mbuf */ - _mm256_storeu_si256((__m256i *)&rx_pkts[i + 6]->rearm_data, + _mm256_storeu_si256((__m256i *)rte_mbuf_rearm_data(rx_pkts[i + 6]), rearm6); - _mm256_storeu_si256((__m256i *)&rx_pkts[i + 4]->rearm_data, + _mm256_storeu_si256((__m256i *)rte_mbuf_rearm_data(rx_pkts[i + 4]), rearm4); - _mm256_storeu_si256((__m256i *)&rx_pkts[i + 2]->rearm_data, + _mm256_storeu_si256((__m256i *)rte_mbuf_rearm_data(rx_pkts[i + 2]), rearm2); - _mm256_storeu_si256((__m256i *)&rx_pkts[i + 0]->rearm_data, + _mm256_storeu_si256((__m256i *)rte_mbuf_rearm_data(rx_pkts[i + 0]), rearm0); rearm7 = _mm256_blend_epi32(mbuf_init, mb6_7, 0xF0); @@ -491,13 +472,13 @@ rearm1 = _mm256_blend_epi32(mbuf_init, mb0_1, 0xF0); /* again write to mbufs */ - _mm256_storeu_si256((__m256i *)&rx_pkts[i + 7]->rearm_data, + _mm256_storeu_si256((__m256i *)rte_mbuf_rearm_data(rx_pkts[i + 7]), rearm7); - _mm256_storeu_si256((__m256i *)&rx_pkts[i + 5]->rearm_data, + _mm256_storeu_si256((__m256i *)rte_mbuf_rearm_data(rx_pkts[i + 5]), rearm5); - _mm256_storeu_si256((__m256i *)&rx_pkts[i + 3]->rearm_data, + _mm256_storeu_si256((__m256i *)rte_mbuf_rearm_data(rx_pkts[i + 3]), rearm3); - _mm256_storeu_si256((__m256i *)&rx_pkts[i + 1]->rearm_data, + _mm256_storeu_si256((__m256i *)rte_mbuf_rearm_data(rx_pkts[i + 1]), rearm1); /* perform dd_check */ @@ -768,19 +749,6 @@ /* octet 15~14, low 16 bits pkt_len */ 0xFFFFFFFF /* pkt_type set as unknown */ ); - /** - * compile-time check the above crc and shuffle layout is correct. - * NOTE: the first field (lowest address) is given last in set_epi - * calls above. - */ - RTE_BUILD_BUG_ON(offsetof(struct rte_mbuf, pkt_len) != - offsetof(struct rte_mbuf, rx_descriptor_fields1) + 4); - RTE_BUILD_BUG_ON(offsetof(struct rte_mbuf, data_len) != - offsetof(struct rte_mbuf, rx_descriptor_fields1) + 8); - RTE_BUILD_BUG_ON(offsetof(struct rte_mbuf, vlan_tci) != - offsetof(struct rte_mbuf, rx_descriptor_fields1) + 10); - RTE_BUILD_BUG_ON(offsetof(struct rte_mbuf, hash) != - offsetof(struct rte_mbuf, rx_descriptor_fields1) + 12); uint16_t i, received; @@ -915,13 +883,6 @@ * add in the previously computed rx_descriptor fields to * make a single 256-bit write per mbuf */ - /* check the structure matches expectations */ - RTE_BUILD_BUG_ON(offsetof(struct rte_mbuf, ol_flags) != - offsetof(struct rte_mbuf, rearm_data) + 8); - RTE_BUILD_BUG_ON(offsetof(struct rte_mbuf, rearm_data) != - RTE_ALIGN(offsetof(struct rte_mbuf, - rearm_data), - 16)); /* build up data and do writes */ __m256i rearm0, rearm1, rearm2, rearm3, rearm4, rearm5, rearm6, rearm7; @@ -936,13 +897,13 @@ rearm0 = _mm256_permute2f128_si256(mbuf_init, mb0_1, 0x20); /* write to mbuf */ - _mm256_storeu_si256((__m256i *)&rx_pkts[i + 6]->rearm_data, + _mm256_storeu_si256((__m256i *)rte_mbuf_rearm_data(rx_pkts[i + 6]), rearm6); - _mm256_storeu_si256((__m256i *)&rx_pkts[i + 4]->rearm_data, + _mm256_storeu_si256((__m256i *)rte_mbuf_rearm_data(rx_pkts[i + 4]), rearm4); - _mm256_storeu_si256((__m256i *)&rx_pkts[i + 2]->rearm_data, + _mm256_storeu_si256((__m256i *)rte_mbuf_rearm_data(rx_pkts[i + 2]), rearm2); - _mm256_storeu_si256((__m256i *)&rx_pkts[i + 0]->rearm_data, + _mm256_storeu_si256((__m256i *)rte_mbuf_rearm_data(rx_pkts[i + 0]), rearm0); rearm7 = _mm256_blend_epi32(mbuf_init, mb6_7, 0xF0); @@ -951,13 +912,13 @@ rearm1 = _mm256_blend_epi32(mbuf_init, mb0_1, 0xF0); /* again write to mbufs */ - _mm256_storeu_si256((__m256i *)&rx_pkts[i + 7]->rearm_data, + _mm256_storeu_si256((__m256i *)rte_mbuf_rearm_data(rx_pkts[i + 7]), rearm7); - _mm256_storeu_si256((__m256i *)&rx_pkts[i + 5]->rearm_data, + _mm256_storeu_si256((__m256i *)rte_mbuf_rearm_data(rx_pkts[i + 5]), rearm5); - _mm256_storeu_si256((__m256i *)&rx_pkts[i + 3]->rearm_data, + _mm256_storeu_si256((__m256i *)rte_mbuf_rearm_data(rx_pkts[i + 3]), rearm3); - _mm256_storeu_si256((__m256i *)&rx_pkts[i + 1]->rearm_data, + _mm256_storeu_si256((__m256i *)rte_mbuf_rearm_data(rx_pkts[i + 1]), rearm1); const __mmask8 dd_mask = _mm512_cmpeq_epi64_mask( From patchwork Tue Feb 27 05:41:20 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Tyler Retzlaff X-Patchwork-Id: 137298 X-Patchwork-Delegate: thomas@monjalon.net Return-Path: X-Original-To: patchwork@inbox.dpdk.org Delivered-To: patchwork@inbox.dpdk.org Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id 0F3AC43C03; Tue, 27 Feb 2024 06:42:33 +0100 (CET) Received: from mails.dpdk.org (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id 65BFE42E57; Tue, 27 Feb 2024 06:42:15 +0100 (CET) Received: from linux.microsoft.com (linux.microsoft.com [13.77.154.182]) by mails.dpdk.org (Postfix) with ESMTP id C3AAC42E29 for ; Tue, 27 Feb 2024 06:41:41 +0100 (CET) Received: by linux.microsoft.com (Postfix, from userid 1086) id DC83C20B74C4; Mon, 26 Feb 2024 21:41:40 -0800 (PST) DKIM-Filter: OpenDKIM Filter v2.11.0 linux.microsoft.com DC83C20B74C4 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linux.microsoft.com; s=default; t=1709012500; bh=/p13L+GCTe/3iDQwTFqo74AvrlDV/3U7d7cjfEIPE0M=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=YmAgkRtEsCSLcBDL3yHR/9wJfJc7mlRhS+7fk4SisuSOkPXm4z4XS4D0NdzpNMfsb HYw/48RViyf9jM1nHc/5CzYcRjYgsRUe1rPdqbKXMD5n/hicLFEhosOcMAIWQ8lfx7 oW6vgAdaIe+nf4MWLgLpsskl7eq0e4uBGjjjlqcQ= From: Tyler Retzlaff To: dev@dpdk.org Cc: Ajit Khaparde , Andrew Boyer , Andrew Rybchenko , Bruce Richardson , Chenbo Xia , Chengwen Feng , Dariusz Sosnowski , David Christensen , Hyong Youb Kim , Jerin Jacob , Jie Hai , Jingjing Wu , John Daley , Kevin Laatz , Kiran Kumar K , Konstantin Ananyev , Maciej Czekaj , Matan Azrad , Maxime Coquelin , Nithin Dabilpuram , Ori Kam , Ruifeng Wang , Satha Rao , Somnath Kotur , Suanming Mou , Sunil Kumar Kori , Viacheslav Ovsiienko , Yisen Zhuang , Yuying Zhang , mb@smartsharesystems.com, Tyler Retzlaff Subject: [PATCH v6 04/23] net/bnxt: use mbuf descriptor accessors Date: Mon, 26 Feb 2024 21:41:20 -0800 Message-Id: <1709012499-12813-5-git-send-email-roretzla@linux.microsoft.com> X-Mailer: git-send-email 1.8.3.1 In-Reply-To: <1709012499-12813-1-git-send-email-roretzla@linux.microsoft.com> References: <1706657173-26166-1-git-send-email-roretzla@linux.microsoft.com> <1709012499-12813-1-git-send-email-roretzla@linux.microsoft.com> X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org RTE_MARKER typedefs are a GCC extension unsupported by MSVC. Use new rte_mbuf_rearm_data and rte_mbuf_rx_descriptor_fields1 accessors that provide a compatible type pointer without using the marker fields. Signed-off-by: Tyler Retzlaff --- drivers/net/bnxt/bnxt_rxtx_vec_avx2.c | 32 ++++++++++++++++---------------- drivers/net/bnxt/bnxt_rxtx_vec_common.h | 4 +--- drivers/net/bnxt/bnxt_rxtx_vec_neon.c | 16 ++++++++-------- drivers/net/bnxt/bnxt_rxtx_vec_sse.c | 16 ++++++++-------- 4 files changed, 33 insertions(+), 35 deletions(-) diff --git a/drivers/net/bnxt/bnxt_rxtx_vec_avx2.c b/drivers/net/bnxt/bnxt_rxtx_vec_avx2.c index ce6b597..5982d12 100644 --- a/drivers/net/bnxt/bnxt_rxtx_vec_avx2.c +++ b/drivers/net/bnxt/bnxt_rxtx_vec_avx2.c @@ -310,13 +310,13 @@ 0x04); /* Store all mbuf fields for first four packets. */ - _mm256_storeu_si256((void *)&rx_pkts[i + 0]->rearm_data, + _mm256_storeu_si256((void *)rte_mbuf_rearm_data(rx_pkts[i + 0]), rearm0); - _mm256_storeu_si256((void *)&rx_pkts[i + 1]->rearm_data, + _mm256_storeu_si256((void *)rte_mbuf_rearm_data(rx_pkts[i + 1]), rearm1); - _mm256_storeu_si256((void *)&rx_pkts[i + 2]->rearm_data, + _mm256_storeu_si256((void *)rte_mbuf_rearm_data(rx_pkts[i + 2]), rearm2); - _mm256_storeu_si256((void *)&rx_pkts[i + 3]->rearm_data, + _mm256_storeu_si256((void *)rte_mbuf_rearm_data(rx_pkts[i + 3]), rearm3); /* Unpack rearm data, set fixed fields for final four mbufs. */ @@ -336,13 +336,13 @@ 0x04); /* Store all mbuf fields for final four packets. */ - _mm256_storeu_si256((void *)&rx_pkts[i + 4]->rearm_data, + _mm256_storeu_si256((void *)rte_mbuf_rearm_data(rx_pkts[i + 4]), rearm4); - _mm256_storeu_si256((void *)&rx_pkts[i + 5]->rearm_data, + _mm256_storeu_si256((void *)rte_mbuf_rearm_data(rx_pkts[i + 5]), rearm5); - _mm256_storeu_si256((void *)&rx_pkts[i + 6]->rearm_data, + _mm256_storeu_si256((void *)rte_mbuf_rearm_data(rx_pkts[i + 6]), rearm6); - _mm256_storeu_si256((void *)&rx_pkts[i + 7]->rearm_data, + _mm256_storeu_si256((void *)rte_mbuf_rearm_data(rx_pkts[i + 7]), rearm7); nb_rx_pkts += num_valid; @@ -598,13 +598,13 @@ 0x04); /* Store all mbuf fields for first four packets. */ - _mm256_storeu_si256((void *)&rx_pkts[i + 0]->rearm_data, + _mm256_storeu_si256((void *)rte_mbuf_rearm_data(rx_pkts[i + 0]), rearm0); - _mm256_storeu_si256((void *)&rx_pkts[i + 1]->rearm_data, + _mm256_storeu_si256((void *)rte_mbuf_rearm_data(rx_pkts[i + 1]), rearm1); - _mm256_storeu_si256((void *)&rx_pkts[i + 2]->rearm_data, + _mm256_storeu_si256((void *)rte_mbuf_rearm_data(rx_pkts[i + 2]), rearm2); - _mm256_storeu_si256((void *)&rx_pkts[i + 3]->rearm_data, + _mm256_storeu_si256((void *)rte_mbuf_rearm_data(rx_pkts[i + 3]), rearm3); /* Unpack rearm data, set fixed fields for final four mbufs. */ @@ -624,13 +624,13 @@ 0x04); /* Store all mbuf fields for final four packets. */ - _mm256_storeu_si256((void *)&rx_pkts[i + 4]->rearm_data, + _mm256_storeu_si256((void *)rte_mbuf_rearm_data(rx_pkts[i + 4]), rearm4); - _mm256_storeu_si256((void *)&rx_pkts[i + 5]->rearm_data, + _mm256_storeu_si256((void *)rte_mbuf_rearm_data(rx_pkts[i + 5]), rearm5); - _mm256_storeu_si256((void *)&rx_pkts[i + 6]->rearm_data, + _mm256_storeu_si256((void *)rte_mbuf_rearm_data(rx_pkts[i + 6]), rearm6); - _mm256_storeu_si256((void *)&rx_pkts[i + 7]->rearm_data, + _mm256_storeu_si256((void *)rte_mbuf_rearm_data(rx_pkts[i + 7]), rearm7); nb_rx_pkts += num_valid; diff --git a/drivers/net/bnxt/bnxt_rxtx_vec_common.h b/drivers/net/bnxt/bnxt_rxtx_vec_common.h index 2294f0a..fb0b1c1 100644 --- a/drivers/net/bnxt/bnxt_rxtx_vec_common.h +++ b/drivers/net/bnxt/bnxt_rxtx_vec_common.h @@ -36,7 +36,6 @@ static inline int bnxt_rxq_vec_setup_common(struct bnxt_rx_queue *rxq) { - uintptr_t p; struct rte_mbuf mb_def = { .buf_addr = 0 }; /* zeroed mbuf */ mb_def.nb_segs = 1; @@ -46,8 +45,7 @@ /* prevent compiler reordering: rearm_data covers previous fields */ rte_compiler_barrier(); - p = (uintptr_t)&mb_def.rearm_data; - rxq->mbuf_initializer = *(uint64_t *)p; + rxq->mbuf_initializer = *rte_mbuf_rearm_data(&mb_def); rxq->rxrearm_nb = 0; rxq->rxrearm_start = 0; return 0; diff --git a/drivers/net/bnxt/bnxt_rxtx_vec_neon.c b/drivers/net/bnxt/bnxt_rxtx_vec_neon.c index 775400f..9ca9903 100644 --- a/drivers/net/bnxt/bnxt_rxtx_vec_neon.c +++ b/drivers/net/bnxt/bnxt_rxtx_vec_neon.c @@ -135,27 +135,27 @@ /* Update mbuf rearm_data for four packets. */ GET_OL_FLAGS(rss_flags, index, errors, 0, ol_flags); - vst1q_u32((uint32_t *)&mbuf[0]->rearm_data, + vst1q_u32((uint32_t *)rte_mbuf_rearm_data(mbuf[0]), vsetq_lane_u32(ol_flags, vreinterpretq_u32_u64(mb_init), 2)); GET_OL_FLAGS(rss_flags, index, errors, 1, ol_flags); - vst1q_u32((uint32_t *)&mbuf[1]->rearm_data, + vst1q_u32((uint32_t *)rte_mbuf_rearm_data(mbuf[1]), vsetq_lane_u32(ol_flags, vreinterpretq_u32_u64(mb_init), 2)); GET_OL_FLAGS(rss_flags, index, errors, 2, ol_flags); - vst1q_u32((uint32_t *)&mbuf[2]->rearm_data, + vst1q_u32((uint32_t *)rte_mbuf_rearm_data(mbuf[2]), vsetq_lane_u32(ol_flags, vreinterpretq_u32_u64(mb_init), 2)); GET_OL_FLAGS(rss_flags, index, errors, 3, ol_flags); - vst1q_u32((uint32_t *)&mbuf[3]->rearm_data, + vst1q_u32((uint32_t *)rte_mbuf_rearm_data(mbuf[3]), vsetq_lane_u32(ol_flags, vreinterpretq_u32_u64(mb_init), 2)); /* Update mbuf rx_descriptor_fields1 for four packets. */ GET_DESC_FIELDS(mm_rxcmp[0], mm_rxcmp1[0], shuf_msk, ptype_idx, 0, tmp); - vst1q_u32((uint32_t *)&mbuf[0]->rx_descriptor_fields1, tmp); + vst1q_u32((uint32_t *)rte_mbuf_rx_descriptor_fields1(mbuf[0]), tmp); GET_DESC_FIELDS(mm_rxcmp[1], mm_rxcmp1[1], shuf_msk, ptype_idx, 1, tmp); - vst1q_u32((uint32_t *)&mbuf[1]->rx_descriptor_fields1, tmp); + vst1q_u32((uint32_t *)rte_mbuf_rx_descriptor_fields1(mbuf[1]), tmp); GET_DESC_FIELDS(mm_rxcmp[2], mm_rxcmp1[2], shuf_msk, ptype_idx, 2, tmp); - vst1q_u32((uint32_t *)&mbuf[2]->rx_descriptor_fields1, tmp); + vst1q_u32((uint32_t *)rte_mbuf_rx_descriptor_fields1(mbuf[2]), tmp); GET_DESC_FIELDS(mm_rxcmp[3], mm_rxcmp1[3], shuf_msk, ptype_idx, 3, tmp); - vst1q_u32((uint32_t *)&mbuf[3]->rx_descriptor_fields1, tmp); + vst1q_u32((uint32_t *)rte_mbuf_rx_descriptor_fields1(mbuf[3]), tmp); } static uint16_t diff --git a/drivers/net/bnxt/bnxt_rxtx_vec_sse.c b/drivers/net/bnxt/bnxt_rxtx_vec_sse.c index 6c0e332..080e990 100644 --- a/drivers/net/bnxt/bnxt_rxtx_vec_sse.c +++ b/drivers/net/bnxt/bnxt_rxtx_vec_sse.c @@ -114,33 +114,33 @@ /* Update mbuf rearm_data for four packets. */ GET_OL_FLAGS(rss_flags, index, errors, 0, ol_flags); - _mm_store_si128((void *)&mbuf[0]->rearm_data, + _mm_store_si128((void *)rte_mbuf_rearm_data(mbuf[0]), _mm_or_si128(mbuf_init, _mm_set_epi64x(ol_flags, 0))); GET_OL_FLAGS(rss_flags, index, errors, 1, ol_flags); - _mm_store_si128((void *)&mbuf[1]->rearm_data, + _mm_store_si128((void *)rte_mbuf_rearm_data(mbuf[1]), _mm_or_si128(mbuf_init, _mm_set_epi64x(ol_flags, 0))); GET_OL_FLAGS(rss_flags, index, errors, 2, ol_flags); - _mm_store_si128((void *)&mbuf[2]->rearm_data, + _mm_store_si128((void *)rte_mbuf_rearm_data(mbuf[2]), _mm_or_si128(mbuf_init, _mm_set_epi64x(ol_flags, 0))); GET_OL_FLAGS(rss_flags, index, errors, 3, ol_flags); - _mm_store_si128((void *)&mbuf[3]->rearm_data, + _mm_store_si128((void *)rte_mbuf_rearm_data(mbuf[3]), _mm_or_si128(mbuf_init, _mm_set_epi64x(ol_flags, 0))); /* Update mbuf rx_descriptor_fields1 for four packes. */ GET_DESC_FIELDS(mm_rxcmp[0], mm_rxcmp1[0], shuf_msk, ptype_idx, 0, t0); - _mm_store_si128((void *)&mbuf[0]->rx_descriptor_fields1, t0); + _mm_store_si128(rte_mbuf_rx_descriptor_fields1(mbuf[0]), t0); GET_DESC_FIELDS(mm_rxcmp[1], mm_rxcmp1[1], shuf_msk, ptype_idx, 1, t0); - _mm_store_si128((void *)&mbuf[1]->rx_descriptor_fields1, t0); + _mm_store_si128(rte_mbuf_rx_descriptor_fields1(mbuf[1]), t0); GET_DESC_FIELDS(mm_rxcmp[2], mm_rxcmp1[2], shuf_msk, ptype_idx, 2, t0); - _mm_store_si128((void *)&mbuf[2]->rx_descriptor_fields1, t0); + _mm_store_si128(rte_mbuf_rx_descriptor_fields1(mbuf[2]), t0); GET_DESC_FIELDS(mm_rxcmp[3], mm_rxcmp1[3], shuf_msk, ptype_idx, 3, t0); - _mm_store_si128((void *)&mbuf[3]->rx_descriptor_fields1, t0); + _mm_store_si128(rte_mbuf_rx_descriptor_fields1(mbuf[3]), t0); } static uint16_t From patchwork Tue Feb 27 05:41:21 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Tyler Retzlaff X-Patchwork-Id: 137304 X-Patchwork-Delegate: thomas@monjalon.net Return-Path: X-Original-To: patchwork@inbox.dpdk.org Delivered-To: patchwork@inbox.dpdk.org Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id 684F543C03; Tue, 27 Feb 2024 06:43:11 +0100 (CET) Received: from mails.dpdk.org (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id EBE2C42EB2; Tue, 27 Feb 2024 06:42:24 +0100 (CET) Received: from linux.microsoft.com (linux.microsoft.com [13.77.154.182]) by mails.dpdk.org (Postfix) with ESMTP id B63BA42E2F for ; Tue, 27 Feb 2024 06:41:42 +0100 (CET) Received: by linux.microsoft.com (Postfix, from userid 1086) id EB32520B74C5; Mon, 26 Feb 2024 21:41:40 -0800 (PST) DKIM-Filter: OpenDKIM Filter v2.11.0 linux.microsoft.com EB32520B74C5 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linux.microsoft.com; s=default; t=1709012500; bh=lTb/MuC4tCyMyqvfN1bTbytcn6b1tLOZPYxamh2q+Io=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=XktjM8H+5FcbYG4sSt4eUTgCkZla9zDO7fCtYUXRwmbnZIsonuKu74FWUZzScyo4s 2nR8XHm3R6IWCUHqWr2+2fGhDfer3ZPC2buc+WXKo2/kVCAIznSuGD8/gmqFzCP00p xgDrRe2ZzFnSv6lXBdqtpM9ngHitAXQG1IOSIZzA= From: Tyler Retzlaff To: dev@dpdk.org Cc: Ajit Khaparde , Andrew Boyer , Andrew Rybchenko , Bruce Richardson , Chenbo Xia , Chengwen Feng , Dariusz Sosnowski , David Christensen , Hyong Youb Kim , Jerin Jacob , Jie Hai , Jingjing Wu , John Daley , Kevin Laatz , Kiran Kumar K , Konstantin Ananyev , Maciej Czekaj , Matan Azrad , Maxime Coquelin , Nithin Dabilpuram , Ori Kam , Ruifeng Wang , Satha Rao , Somnath Kotur , Suanming Mou , Sunil Kumar Kori , Viacheslav Ovsiienko , Yisen Zhuang , Yuying Zhang , mb@smartsharesystems.com, Tyler Retzlaff Subject: [PATCH v6 05/23] net/cnxk: use mbuf descriptor accessors Date: Mon, 26 Feb 2024 21:41:21 -0800 Message-Id: <1709012499-12813-6-git-send-email-roretzla@linux.microsoft.com> X-Mailer: git-send-email 1.8.3.1 In-Reply-To: <1709012499-12813-1-git-send-email-roretzla@linux.microsoft.com> References: <1706657173-26166-1-git-send-email-roretzla@linux.microsoft.com> <1709012499-12813-1-git-send-email-roretzla@linux.microsoft.com> X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org RTE_MARKER typedefs are a GCC extension unsupported by MSVC. Use new rte_mbuf_rearm_data and rte_mbuf_rx_descriptor_fields1 accessors that provide a compatible type pointer without using the marker fields. Signed-off-by: Tyler Retzlaff --- drivers/net/cnxk/cn10k_rx.h | 36 ++++++++++++++++++------------------ drivers/net/cnxk/cn9k_rx.h | 20 ++++++++++---------- drivers/net/cnxk/cnxk_ethdev.c | 2 +- 3 files changed, 29 insertions(+), 29 deletions(-) diff --git a/drivers/net/cnxk/cn10k_rx.h b/drivers/net/cnxk/cn10k_rx.h index 7bb4c86..4bfb323 100644 --- a/drivers/net/cnxk/cn10k_rx.h +++ b/drivers/net/cnxk/cn10k_rx.h @@ -309,7 +309,7 @@ frag_rx = (union nix_rx_parse_u *)(wqe + 1); frag_size = rlen + frag_rx->lcptr - frag_rx->laptr; - *(uint64_t *)(&mbuf->rearm_data) = mbuf_init; + *rte_mbuf_rearm_data(mbuf) = mbuf_init; mbuf->data_len = frag_size; mbuf->pkt_len = frag_size; mbuf->ol_flags = ol_flags; @@ -368,7 +368,7 @@ fsz_w1 >>= 16; data_off = b_off + frag_rx->lcptr + l3_hdr_size; - *(uint64_t *)(&mbuf->rearm_data) = mbuf_init | data_off; + *rte_mbuf_rearm_data(mbuf) = mbuf_init | data_off; mbuf->data_len = frag_size; /* Mark frag as get */ @@ -384,7 +384,7 @@ fsz_w1 >>= 16; data_off = b_off + frag_rx->lcptr + l3_hdr_size; - *(uint64_t *)(&mbuf->rearm_data) = mbuf_init | data_off; + *rte_mbuf_rearm_data(mbuf) = mbuf_init | data_off; mbuf->data_len = frag_size; /* Mark frag as get */ @@ -401,7 +401,7 @@ fsz_w1 >>= 16; data_off = b_off + frag_rx->lcptr + l3_hdr_size; - *(uint64_t *)(&mbuf->rearm_data) = mbuf_init | data_off; + *rte_mbuf_rearm_data(mbuf) = mbuf_init | data_off; mbuf->data_len = frag_size; /* Mark frag as get */ @@ -502,7 +502,7 @@ inner->pkt_len = len; inner->data_len = len; - *(uint64_t *)(&inner->rearm_data) = mbuf_init; + *rte_mbuf_rearm_data(inner) = mbuf_init; inner->ol_flags = ((CPT_COMP_HWGOOD_MASK & (1U << ucc)) ? RTE_MBUF_F_RX_SEC_OFFLOAD : @@ -584,7 +584,7 @@ /* First frag len */ inner->pkt_len = vgetq_lane_u16(*rx_desc_field1, 2); inner->data_len = vgetq_lane_u16(*rx_desc_field1, 4); - p = (uintptr_t)&inner->rearm_data; + p = (uintptr_t)rte_mbuf_rearm_data(inner); *(uint64_t *)p = mbuf_init; /* Reassembly success */ @@ -774,7 +774,7 @@ mbuf->data_len = sg_len; sg = sg >> 16; - p = (uintptr_t)&mbuf->rearm_data; + p = (uintptr_t)rte_mbuf_rearm_data(mbuf); *(uint64_t *)p = rearm & ~0xFFFF; nb_segs--; iova_list++; @@ -825,7 +825,7 @@ head->nb_segs = nb_segs; } mbuf = next_frag; - p = (uintptr_t)&mbuf->rearm_data; + p = (uintptr_t)rte_mbuf_rearm_data(mbuf); *(uint64_t *)p = rearm + ldptr; mbuf->data_len = (sg & 0xFFFF) - ldptr - (flags & NIX_RX_OFFLOAD_TSTAMP_F ? @@ -849,7 +849,7 @@ len = mbuf->pkt_len; - p = (uintptr_t)&mbuf->rearm_data; + p = (uintptr_t)rte_mbuf_rearm_data(mbuf); *(uint64_t *)p = rearm; mbuf->data_len = (sg & 0xFFFF) - (flags & NIX_RX_OFFLOAD_TSTAMP_F ? @@ -917,7 +917,7 @@ mbuf->ol_flags = ol_flags; mbuf->pkt_len = len; mbuf->data_len = len; - p = (uintptr_t)&mbuf->rearm_data; + p = (uintptr_t)rte_mbuf_rearm_data(mbuf); *(uint64_t *)p = val; } @@ -1966,16 +1966,16 @@ rearm3 = vsetq_lane_u64(ol_flags3, rearm3, 1); /* Update rx_descriptor_fields1 */ - vst1q_u64((uint64_t *)mbuf0->rx_descriptor_fields1, f0); - vst1q_u64((uint64_t *)mbuf1->rx_descriptor_fields1, f1); - vst1q_u64((uint64_t *)mbuf2->rx_descriptor_fields1, f2); - vst1q_u64((uint64_t *)mbuf3->rx_descriptor_fields1, f3); + vst1q_u64((uint64_t *)rte_mbuf_rx_descriptor_fields1(mbuf0), f0); + vst1q_u64((uint64_t *)rte_mbuf_rx_descriptor_fields1(mbuf1), f1); + vst1q_u64((uint64_t *)rte_mbuf_rx_descriptor_fields1(mbuf2), f2); + vst1q_u64((uint64_t *)rte_mbuf_rx_descriptor_fields1(mbuf3), f3); /* Update rearm_data */ - vst1q_u64((uint64_t *)mbuf0->rearm_data, rearm0); - vst1q_u64((uint64_t *)mbuf1->rearm_data, rearm1); - vst1q_u64((uint64_t *)mbuf2->rearm_data, rearm2); - vst1q_u64((uint64_t *)mbuf3->rearm_data, rearm3); + vst1q_u64(rte_mbuf_rearm_data(mbuf0), rearm0); + vst1q_u64(rte_mbuf_rearm_data(mbuf1), rearm1); + vst1q_u64(rte_mbuf_rearm_data(mbuf2), rearm2); + vst1q_u64(rte_mbuf_rearm_data(mbuf3), rearm3); if (flags & NIX_RX_MULTI_SEG_F) { /* Multi segment is enable build mseg list for diff --git a/drivers/net/cnxk/cn9k_rx.h b/drivers/net/cnxk/cn9k_rx.h index d8bb65c..f1212f8 100644 --- a/drivers/net/cnxk/cn9k_rx.h +++ b/drivers/net/cnxk/cn9k_rx.h @@ -155,7 +155,7 @@ mbuf->data_len = sg & 0xFFFF; sg = sg >> 16; - *(uint64_t *)(&mbuf->rearm_data) = rearm; + *rte_mbuf_rearm_data(mbuf) = rearm; nb_segs--; iova_list++; @@ -398,7 +398,7 @@ nix_update_match_id(rx->cn9k.match_id, ol_flags, mbuf); mbuf->ol_flags = ol_flags; - *(uint64_t *)(&mbuf->rearm_data) = val; + *rte_mbuf_rearm_data(mbuf) = val; mbuf->pkt_len = len; mbuf->data_len = len; @@ -799,16 +799,16 @@ rearm3 = vsetq_lane_u64(ol_flags3, rearm3, 1); /* Update rx_descriptor_fields1 */ - vst1q_u64((uint64_t *)mbuf0->rx_descriptor_fields1, f0); - vst1q_u64((uint64_t *)mbuf1->rx_descriptor_fields1, f1); - vst1q_u64((uint64_t *)mbuf2->rx_descriptor_fields1, f2); - vst1q_u64((uint64_t *)mbuf3->rx_descriptor_fields1, f3); + vst1q_u64((uint64_t *)rte_mbuf_rx_descriptor_fields1(mbuf0), f0); + vst1q_u64((uint64_t *)rte_mbuf_rx_descriptor_fields1(mbuf1), f1); + vst1q_u64((uint64_t *)rte_mbuf_rx_descriptor_fields1(mbuf2), f2); + vst1q_u64((uint64_t *)rte_mbuf_rx_descriptor_fields1(mbuf3), f3); /* Update rearm_data */ - vst1q_u64((uint64_t *)mbuf0->rearm_data, rearm0); - vst1q_u64((uint64_t *)mbuf1->rearm_data, rearm1); - vst1q_u64((uint64_t *)mbuf2->rearm_data, rearm2); - vst1q_u64((uint64_t *)mbuf3->rearm_data, rearm3); + vst1q_u64(rte_mbuf_rearm_data(mbuf0), rearm0); + vst1q_u64(rte_mbuf_rearm_data(mbuf1), rearm1); + vst1q_u64(rte_mbuf_rearm_data(mbuf2), rearm2); + vst1q_u64(rte_mbuf_rearm_data(mbuf3), rearm3); if (flags & NIX_RX_MULTI_SEG_F) { /* Multi segment is enable build mseg list for diff --git a/drivers/net/cnxk/cnxk_ethdev.c b/drivers/net/cnxk/cnxk_ethdev.c index 2372a4e..14f6b7c 100644 --- a/drivers/net/cnxk/cnxk_ethdev.c +++ b/drivers/net/cnxk/cnxk_ethdev.c @@ -437,7 +437,7 @@ /* Prevent compiler reordering: rearm_data covers previous fields */ rte_compiler_barrier(); - tmp = (uint64_t *)&mb_def.rearm_data; + tmp = rte_mbuf_rearm_data(&mb_def); return *tmp; } From patchwork Tue Feb 27 05:41:22 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Tyler Retzlaff X-Patchwork-Id: 137305 X-Patchwork-Delegate: thomas@monjalon.net Return-Path: X-Original-To: patchwork@inbox.dpdk.org Delivered-To: patchwork@inbox.dpdk.org Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id D4B4943C03; Tue, 27 Feb 2024 06:43:17 +0100 (CET) Received: from mails.dpdk.org (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id 73C9742E84; Tue, 27 Feb 2024 06:42:26 +0100 (CET) Received: from linux.microsoft.com (linux.microsoft.com [13.77.154.182]) by mails.dpdk.org (Postfix) with ESMTP id C62DC42E42 for ; Tue, 27 Feb 2024 06:41:42 +0100 (CET) Received: by linux.microsoft.com (Postfix, from userid 1086) id 0514820B74C6; Mon, 26 Feb 2024 21:41:40 -0800 (PST) DKIM-Filter: OpenDKIM Filter v2.11.0 linux.microsoft.com 0514820B74C6 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linux.microsoft.com; s=default; t=1709012501; bh=VUNPZFVgLOLwP7BjpG3i+V4rN2X3ZMOQHEmfdOt2UEI=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=QYHNzYPsjJbgjwCrF82I20iXzt4Xq+7x1KgOiLk5xmEwGzAWLznSPOGz/sNYEyWWY Dj+RDKEXghLi6Yxv/oKfFs2A2A5RLJCKUabOgpD34gtLx0REUbfv2UdJmh2gGJXMwL arlM3wlYcTERk9BqWT3O363Jm7jZE8WY2SWaEFoo= From: Tyler Retzlaff To: dev@dpdk.org Cc: Ajit Khaparde , Andrew Boyer , Andrew Rybchenko , Bruce Richardson , Chenbo Xia , Chengwen Feng , Dariusz Sosnowski , David Christensen , Hyong Youb Kim , Jerin Jacob , Jie Hai , Jingjing Wu , John Daley , Kevin Laatz , Kiran Kumar K , Konstantin Ananyev , Maciej Czekaj , Matan Azrad , Maxime Coquelin , Nithin Dabilpuram , Ori Kam , Ruifeng Wang , Satha Rao , Somnath Kotur , Suanming Mou , Sunil Kumar Kori , Viacheslav Ovsiienko , Yisen Zhuang , Yuying Zhang , mb@smartsharesystems.com, Tyler Retzlaff Subject: [PATCH v6 06/23] net/enic: use mbuf descriptor accessors Date: Mon, 26 Feb 2024 21:41:22 -0800 Message-Id: <1709012499-12813-7-git-send-email-roretzla@linux.microsoft.com> X-Mailer: git-send-email 1.8.3.1 In-Reply-To: <1709012499-12813-1-git-send-email-roretzla@linux.microsoft.com> References: <1706657173-26166-1-git-send-email-roretzla@linux.microsoft.com> <1709012499-12813-1-git-send-email-roretzla@linux.microsoft.com> X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org RTE_MARKER typedefs are a GCC extension unsupported by MSVC. Use new rte_mbuf_rearm_data and rte_mbuf_rx_descriptor_fields1 accessors that provide a compatible type pointer without using the marker fields. Signed-off-by: Tyler Retzlaff --- drivers/net/enic/enic_main.c | 4 +--- drivers/net/enic/enic_rxtx_vec_avx2.c | 18 +++++++++--------- 2 files changed, 10 insertions(+), 12 deletions(-) diff --git a/drivers/net/enic/enic_main.c b/drivers/net/enic/enic_main.c index a6aaa76..59e0be4 100644 --- a/drivers/net/enic/enic_main.c +++ b/drivers/net/enic/enic_main.c @@ -577,7 +577,6 @@ int enic_enable(struct enic *enic) int err; struct rte_eth_dev *eth_dev = enic->rte_dev; uint64_t simple_tx_offloads; - uintptr_t p; if (enic->enable_avx2_rx) { struct rte_mbuf mb_def = { .buf_addr = 0 }; @@ -592,8 +591,7 @@ int enic_enable(struct enic *enic) mb_def.port = enic->port_id; rte_mbuf_refcnt_set(&mb_def, 1); rte_compiler_barrier(); - p = (uintptr_t)&mb_def.rearm_data; - enic->mbuf_initializer = *(uint64_t *)p; + enic->mbuf_initializer = *rte_mbuf_rearm_data(&mb_def); } eth_dev->data->dev_link.link_speed = vnic_dev_port_speed(enic->vdev); diff --git a/drivers/net/enic/enic_rxtx_vec_avx2.c b/drivers/net/enic/enic_rxtx_vec_avx2.c index 600efff..f6ab6e1 100644 --- a/drivers/net/enic/enic_rxtx_vec_avx2.c +++ b/drivers/net/enic/enic_rxtx_vec_avx2.c @@ -19,7 +19,7 @@ { bool tnl; - *(uint64_t *)&mb->rearm_data = enic->mbuf_initializer; + *rte_mbuf_rearm_data(mb) = enic->mbuf_initializer; mb->data_len = cqd->bytes_written_flags & CQ_ENET_RQ_DESC_BYTES_WRITTEN_MASK; mb->pkt_len = mb->data_len; @@ -737,14 +737,14 @@ * vlan_tci - 26 (from cqd) * rss - 28 (from cqd) */ - _mm256_storeu_si256((__m256i *)&rxmb[0]->rearm_data, rearm0); - _mm256_storeu_si256((__m256i *)&rxmb[1]->rearm_data, rearm1); - _mm256_storeu_si256((__m256i *)&rxmb[2]->rearm_data, rearm2); - _mm256_storeu_si256((__m256i *)&rxmb[3]->rearm_data, rearm3); - _mm256_storeu_si256((__m256i *)&rxmb[4]->rearm_data, rearm4); - _mm256_storeu_si256((__m256i *)&rxmb[5]->rearm_data, rearm5); - _mm256_storeu_si256((__m256i *)&rxmb[6]->rearm_data, rearm6); - _mm256_storeu_si256((__m256i *)&rxmb[7]->rearm_data, rearm7); + _mm256_storeu_si256((__m256i *)rte_mbuf_rearm_data(rxmb[0]), rearm0); + _mm256_storeu_si256((__m256i *)rte_mbuf_rearm_data(rxmb[1]), rearm1); + _mm256_storeu_si256((__m256i *)rte_mbuf_rearm_data(rxmb[2]), rearm2); + _mm256_storeu_si256((__m256i *)rte_mbuf_rearm_data(rxmb[3]), rearm3); + _mm256_storeu_si256((__m256i *)rte_mbuf_rearm_data(rxmb[4]), rearm4); + _mm256_storeu_si256((__m256i *)rte_mbuf_rearm_data(rxmb[5]), rearm5); + _mm256_storeu_si256((__m256i *)rte_mbuf_rearm_data(rxmb[6]), rearm6); + _mm256_storeu_si256((__m256i *)rte_mbuf_rearm_data(rxmb[7]), rearm7); max_rx -= 8; cqd += 8; From patchwork Tue Feb 27 05:41:23 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Tyler Retzlaff X-Patchwork-Id: 137299 X-Patchwork-Delegate: thomas@monjalon.net Return-Path: X-Original-To: patchwork@inbox.dpdk.org Delivered-To: patchwork@inbox.dpdk.org Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id D5E8643C03; Tue, 27 Feb 2024 06:42:38 +0100 (CET) Received: from mails.dpdk.org (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id 31C1642E76; Tue, 27 Feb 2024 06:42:18 +0100 (CET) Received: from linux.microsoft.com (linux.microsoft.com [13.77.154.182]) by mails.dpdk.org (Postfix) with ESMTP id A8AAF42E29 for ; Tue, 27 Feb 2024 06:41:42 +0100 (CET) Received: by linux.microsoft.com (Postfix, from userid 1086) id 13B6520B74C7; Mon, 26 Feb 2024 21:41:40 -0800 (PST) DKIM-Filter: OpenDKIM Filter v2.11.0 linux.microsoft.com 13B6520B74C7 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linux.microsoft.com; s=default; t=1709012501; bh=YedPAA8j4yEoaTYfSvG0Z6DV5KKWmBGXIT7RT7OiqQ0=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=hNCT3DMSIM0nkAWMcCesjTUJU7lK+oNROTLo3aGU7YZSLmyn4hhn6QE4q1lzhilOX P+EUrRhpT9GbzdhAooHpK2yZmw4GVE97MtRvKjj6BKemn+7ApwBmx0HiOHnal0up4c D3EEGR8+hbcUxQoPs64V7LSjOwaAFyVpJA6kxQMY= From: Tyler Retzlaff To: dev@dpdk.org Cc: Ajit Khaparde , Andrew Boyer , Andrew Rybchenko , Bruce Richardson , Chenbo Xia , Chengwen Feng , Dariusz Sosnowski , David Christensen , Hyong Youb Kim , Jerin Jacob , Jie Hai , Jingjing Wu , John Daley , Kevin Laatz , Kiran Kumar K , Konstantin Ananyev , Maciej Czekaj , Matan Azrad , Maxime Coquelin , Nithin Dabilpuram , Ori Kam , Ruifeng Wang , Satha Rao , Somnath Kotur , Suanming Mou , Sunil Kumar Kori , Viacheslav Ovsiienko , Yisen Zhuang , Yuying Zhang , mb@smartsharesystems.com, Tyler Retzlaff Subject: [PATCH v6 07/23] net/fm10k: use mbuf descriptor accessors Date: Mon, 26 Feb 2024 21:41:23 -0800 Message-Id: <1709012499-12813-8-git-send-email-roretzla@linux.microsoft.com> X-Mailer: git-send-email 1.8.3.1 In-Reply-To: <1709012499-12813-1-git-send-email-roretzla@linux.microsoft.com> References: <1706657173-26166-1-git-send-email-roretzla@linux.microsoft.com> <1709012499-12813-1-git-send-email-roretzla@linux.microsoft.com> X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org RTE_MARKER typedefs are a GCC extension unsupported by MSVC. Use new rte_mbuf_rearm_data and rte_mbuf_rx_descriptor_fields1 accessors that provide a compatible type pointer without using the marker fields. Signed-off-by: Tyler Retzlaff --- drivers/net/fm10k/fm10k_rxtx_vec.c | 32 +++++++------------------------- 1 file changed, 7 insertions(+), 25 deletions(-) diff --git a/drivers/net/fm10k/fm10k_rxtx_vec.c b/drivers/net/fm10k/fm10k_rxtx_vec.c index 2b6914b..795e70a 100644 --- a/drivers/net/fm10k/fm10k_rxtx_vec.c +++ b/drivers/net/fm10k/fm10k_rxtx_vec.c @@ -226,7 +226,6 @@ int __rte_cold fm10k_rxq_vec_setup(struct fm10k_rx_queue *rxq) { - uintptr_t p; struct rte_mbuf mb_def = { .buf_addr = 0 }; /* zeroed mbuf */ mb_def.nb_segs = 1; @@ -239,8 +238,7 @@ /* prevent compiler reordering: rearm_data covers previous fields */ rte_compiler_barrier(); - p = (uintptr_t)&mb_def.rearm_data; - rxq->mbuf_initializer = *(uint64_t *)p; + rxq->mbuf_initializer = *rte_mbuf_rearm_data(&mb_def); return 0; } @@ -282,7 +280,6 @@ /* Initialize the mbufs in vector, process 2 mbufs in one loop */ for (i = 0; i < RTE_FM10K_RXQ_REARM_THRESH; i += 2, mb_alloc += 2) { __m128i vaddr0, vaddr1; - uintptr_t p0, p1; mb0 = mb_alloc[0]; mb1 = mb_alloc[1]; @@ -290,10 +287,8 @@ /* Flush mbuf with pkt template. * Data to be rearmed is 6 bytes long. */ - p0 = (uintptr_t)&mb0->rearm_data; - *(uint64_t *)p0 = rxq->mbuf_initializer; - p1 = (uintptr_t)&mb1->rearm_data; - *(uint64_t *)p1 = rxq->mbuf_initializer; + *rte_mbuf_rearm_data(mb0) = rxq->mbuf_initializer; + *rte_mbuf_rearm_data(mb1) = rxq->mbuf_initializer; /* load buf_addr(lo 64bit) and buf_iova(hi 64bit) */ RTE_BUILD_BUG_ON(offsetof(struct rte_mbuf, buf_iova) != @@ -422,19 +417,6 @@ 0xFF, 0xFF, /* skip high 16 bits pkt_type */ 0xFF, 0xFF /* Skip pkt_type field in shuffle operation */ ); - /* - * Compile-time verify the shuffle mask - * NOTE: some field positions already verified above, but duplicated - * here for completeness in case of future modifications. - */ - RTE_BUILD_BUG_ON(offsetof(struct rte_mbuf, pkt_len) != - offsetof(struct rte_mbuf, rx_descriptor_fields1) + 4); - RTE_BUILD_BUG_ON(offsetof(struct rte_mbuf, data_len) != - offsetof(struct rte_mbuf, rx_descriptor_fields1) + 8); - RTE_BUILD_BUG_ON(offsetof(struct rte_mbuf, vlan_tci) != - offsetof(struct rte_mbuf, rx_descriptor_fields1) + 10); - RTE_BUILD_BUG_ON(offsetof(struct rte_mbuf, hash) != - offsetof(struct rte_mbuf, rx_descriptor_fields1) + 12); /* Cache is empty -> need to scan the buffer rings, but first move * the next 'n' mbufs into the cache @@ -519,9 +501,9 @@ staterr = _mm_unpacklo_epi32(sterr_tmp1, sterr_tmp2); /* D.3 copy final 3,4 data to rx_pkts */ - _mm_storeu_si128((void *)&rx_pkts[pos+3]->rx_descriptor_fields1, + _mm_storeu_si128(rte_mbuf_rx_descriptor_fields1(rx_pkts[pos + 3]), pkt_mb4); - _mm_storeu_si128((void *)&rx_pkts[pos+2]->rx_descriptor_fields1, + _mm_storeu_si128(rte_mbuf_rx_descriptor_fields1(rx_pkts[pos + 2]), pkt_mb3); /* C* extract and record EOP bit */ @@ -557,9 +539,9 @@ staterr = _mm_packs_epi32(staterr, zero); /* D.3 copy final 1,2 data to rx_pkts */ - _mm_storeu_si128((void *)&rx_pkts[pos+1]->rx_descriptor_fields1, + _mm_storeu_si128(rte_mbuf_rx_descriptor_fields1(rx_pkts[pos + 1]), pkt_mb2); - _mm_storeu_si128((void *)&rx_pkts[pos]->rx_descriptor_fields1, + _mm_storeu_si128(rte_mbuf_rx_descriptor_fields1(rx_pkts[pos]), pkt_mb1); fm10k_desc_to_pktype_v(descs0, &rx_pkts[pos]); From patchwork Tue Feb 27 05:41:24 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Tyler Retzlaff X-Patchwork-Id: 137306 X-Patchwork-Delegate: thomas@monjalon.net Return-Path: X-Original-To: patchwork@inbox.dpdk.org Delivered-To: patchwork@inbox.dpdk.org Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id 31B5043C03; Tue, 27 Feb 2024 06:43:24 +0100 (CET) Received: from mails.dpdk.org (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id C40F542EBF; Tue, 27 Feb 2024 06:42:27 +0100 (CET) Received: from linux.microsoft.com (linux.microsoft.com [13.77.154.182]) by mails.dpdk.org (Postfix) with ESMTP id C135F42E41 for ; Tue, 27 Feb 2024 06:41:42 +0100 (CET) Received: by linux.microsoft.com (Postfix, from userid 1086) id 22A4320B74C8; Mon, 26 Feb 2024 21:41:40 -0800 (PST) DKIM-Filter: OpenDKIM Filter v2.11.0 linux.microsoft.com 22A4320B74C8 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linux.microsoft.com; s=default; t=1709012501; bh=yTftrNn5EzqwSkfgdh2/JNeoGLzIoWQcsQFehRUEk7s=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=N31xheo+PaSIh9OqzRqEEgxEDmZv/57NKxgTRoSVASDlGBaluOE87hH4SSx2Sqga0 VGUHgFHFG40VB6Xy4/Dh+9azIy7vRGlfdUg6BOodq1mAMREDMfUnH0s5nlkZqz3tVz jTtOIehXjIiT4VrBelx+i3nv9wlIBc8To7z9MVbo= From: Tyler Retzlaff To: dev@dpdk.org Cc: Ajit Khaparde , Andrew Boyer , Andrew Rybchenko , Bruce Richardson , Chenbo Xia , Chengwen Feng , Dariusz Sosnowski , David Christensen , Hyong Youb Kim , Jerin Jacob , Jie Hai , Jingjing Wu , John Daley , Kevin Laatz , Kiran Kumar K , Konstantin Ananyev , Maciej Czekaj , Matan Azrad , Maxime Coquelin , Nithin Dabilpuram , Ori Kam , Ruifeng Wang , Satha Rao , Somnath Kotur , Suanming Mou , Sunil Kumar Kori , Viacheslav Ovsiienko , Yisen Zhuang , Yuying Zhang , mb@smartsharesystems.com, Tyler Retzlaff Subject: [PATCH v6 08/23] net/hns3: use mbuf descriptor accessors Date: Mon, 26 Feb 2024 21:41:24 -0800 Message-Id: <1709012499-12813-9-git-send-email-roretzla@linux.microsoft.com> X-Mailer: git-send-email 1.8.3.1 In-Reply-To: <1709012499-12813-1-git-send-email-roretzla@linux.microsoft.com> References: <1706657173-26166-1-git-send-email-roretzla@linux.microsoft.com> <1709012499-12813-1-git-send-email-roretzla@linux.microsoft.com> X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org RTE_MARKER typedefs are a GCC extension unsupported by MSVC. Use new rte_mbuf_rearm_data and rte_mbuf_rx_descriptor_fields1 accessors that provide a compatible type pointer without using the marker fields. Signed-off-by: Tyler Retzlaff --- drivers/net/hns3/hns3_rxtx_vec.c | 22 +--------------------- drivers/net/hns3/hns3_rxtx_vec_neon.h | 24 ++++++++---------------- drivers/net/hns3/hns3_rxtx_vec_sve.c | 4 ++-- 3 files changed, 11 insertions(+), 39 deletions(-) diff --git a/drivers/net/hns3/hns3_rxtx_vec.c b/drivers/net/hns3/hns3_rxtx_vec.c index 9708ec6..d6c9e80 100644 --- a/drivers/net/hns3/hns3_rxtx_vec.c +++ b/drivers/net/hns3/hns3_rxtx_vec.c @@ -113,7 +113,6 @@ static void hns3_rxq_vec_setup_rearm_data(struct hns3_rx_queue *rxq) { - uintptr_t p; struct rte_mbuf mb_def = { .buf_addr = 0 }; /* zeroed mbuf */ mb_def.nb_segs = 1; @@ -121,28 +120,9 @@ mb_def.port = rxq->port_id; rte_mbuf_refcnt_set(&mb_def, 1); - /* compile-time verifies the rearm_data first 8bytes */ - RTE_BUILD_BUG_ON(offsetof(struct rte_mbuf, data_off) < - offsetof(struct rte_mbuf, rearm_data)); - RTE_BUILD_BUG_ON(offsetof(struct rte_mbuf, refcnt) < - offsetof(struct rte_mbuf, rearm_data)); - RTE_BUILD_BUG_ON(offsetof(struct rte_mbuf, nb_segs) < - offsetof(struct rte_mbuf, rearm_data)); - RTE_BUILD_BUG_ON(offsetof(struct rte_mbuf, port) < - offsetof(struct rte_mbuf, rearm_data)); - RTE_BUILD_BUG_ON(offsetof(struct rte_mbuf, data_off) - - offsetof(struct rte_mbuf, rearm_data) > 6); - RTE_BUILD_BUG_ON(offsetof(struct rte_mbuf, refcnt) - - offsetof(struct rte_mbuf, rearm_data) > 6); - RTE_BUILD_BUG_ON(offsetof(struct rte_mbuf, nb_segs) - - offsetof(struct rte_mbuf, rearm_data) > 6); - RTE_BUILD_BUG_ON(offsetof(struct rte_mbuf, port) - - offsetof(struct rte_mbuf, rearm_data) > 6); - /* prevent compiler reordering: rearm_data covers previous fields */ rte_compiler_barrier(); - p = (uintptr_t)&mb_def.rearm_data; - rxq->mbuf_initializer = *(uint64_t *)p; + rxq->mbuf_initializer = *rte_mbuf_rearm_data(&mb_def); } void diff --git a/drivers/net/hns3/hns3_rxtx_vec_neon.h b/drivers/net/hns3/hns3_rxtx_vec_neon.h index 0dc6b9f..6f46131 100644 --- a/drivers/net/hns3/hns3_rxtx_vec_neon.h +++ b/drivers/net/hns3/hns3_rxtx_vec_neon.h @@ -156,14 +156,6 @@ 0, 0, 0, /* ignore non-length fields */ }; - /* compile-time verifies the shuffle mask */ - RTE_BUILD_BUG_ON(offsetof(struct rte_mbuf, pkt_len) != - offsetof(struct rte_mbuf, rx_descriptor_fields1) + 4); - RTE_BUILD_BUG_ON(offsetof(struct rte_mbuf, data_len) != - offsetof(struct rte_mbuf, rx_descriptor_fields1) + 8); - RTE_BUILD_BUG_ON(offsetof(struct rte_mbuf, hash.rss) != - offsetof(struct rte_mbuf, rx_descriptor_fields1) + 12); - for (pos = 0; pos < nb_pkts; pos += HNS3_DEFAULT_DESCS_PER_LOOP, rxdp += HNS3_DEFAULT_DESCS_PER_LOOP) { uint64x2x2_t descs[HNS3_DEFAULT_DESCS_PER_LOOP]; @@ -236,23 +228,23 @@ pkt_mb4 = vreinterpretq_u8_u16(tmp); /* save packet info to rx_pkts mbuf */ - vst1q_u8((void *)&sw_ring[pos + 0].mbuf->rx_descriptor_fields1, + vst1q_u8(rte_mbuf_rx_descriptor_fields1(sw_ring[pos + 0].mbuf), pkt_mb1); - vst1q_u8((void *)&sw_ring[pos + 1].mbuf->rx_descriptor_fields1, + vst1q_u8(rte_mbuf_rx_descriptor_fields1(sw_ring[pos + 1].mbuf), pkt_mb2); - vst1q_u8((void *)&sw_ring[pos + 2].mbuf->rx_descriptor_fields1, + vst1q_u8(rte_mbuf_rx_descriptor_fields1(sw_ring[pos + 2].mbuf), pkt_mb3); - vst1q_u8((void *)&sw_ring[pos + 3].mbuf->rx_descriptor_fields1, + vst1q_u8(rte_mbuf_rx_descriptor_fields1(sw_ring[pos + 3].mbuf), pkt_mb4); /* store the first 8 bytes of packets mbuf's rearm_data */ - *(uint64_t *)&sw_ring[pos + 0].mbuf->rearm_data = + *rte_mbuf_rearm_data(sw_ring[pos + 0].mbuf) = rxq->mbuf_initializer; - *(uint64_t *)&sw_ring[pos + 1].mbuf->rearm_data = + *rte_mbuf_rearm_data(sw_ring[pos + 1].mbuf) = rxq->mbuf_initializer; - *(uint64_t *)&sw_ring[pos + 2].mbuf->rearm_data = + *rte_mbuf_rearm_data(sw_ring[pos + 2].mbuf) = rxq->mbuf_initializer; - *(uint64_t *)&sw_ring[pos + 3].mbuf->rearm_data = + *rte_mbuf_rearm_data(sw_ring[pos + 3].mbuf) = rxq->mbuf_initializer; rte_prefetch_non_temporal(rxdp + HNS3_DEFAULT_DESCS_PER_LOOP); diff --git a/drivers/net/hns3/hns3_rxtx_vec_sve.c b/drivers/net/hns3/hns3_rxtx_vec_sve.c index 8aa4448..f6259d1 100644 --- a/drivers/net/hns3/hns3_rxtx_vec_sve.c +++ b/drivers/net/hns3/hns3_rxtx_vec_sve.c @@ -123,9 +123,9 @@ mbuf_init = svdup_n_u64(rxq->mbuf_initializer); /* save mbuf_initializer */ svst1_scatter_u64base_offset_u64(PG64_256BIT, mbp1st, - offsetof(struct rte_mbuf, rearm_data), mbuf_init); + offsetof(struct rte_mbuf, data_off), mbuf_init); svst1_scatter_u64base_offset_u64(PG64_256BIT, mbp2st, - offsetof(struct rte_mbuf, rearm_data), mbuf_init); + offsetof(struct rte_mbuf, data_off), mbuf_init); next_rxdp = rxdp + HNS3_SVE_DEFAULT_DESCS_PER_LOOP; rte_prefetch_non_temporal(next_rxdp); From patchwork Tue Feb 27 05:41:25 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Tyler Retzlaff X-Patchwork-Id: 137300 X-Patchwork-Delegate: thomas@monjalon.net Return-Path: X-Original-To: patchwork@inbox.dpdk.org Delivered-To: patchwork@inbox.dpdk.org Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id 64EC443C03; Tue, 27 Feb 2024 06:42:45 +0100 (CET) Received: from mails.dpdk.org (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id 8C95842E87; Tue, 27 Feb 2024 06:42:19 +0100 (CET) Received: from linux.microsoft.com (linux.microsoft.com [13.77.154.182]) by mails.dpdk.org (Postfix) with ESMTP id 91E3E402A7 for ; Tue, 27 Feb 2024 06:41:42 +0100 (CET) Received: by linux.microsoft.com (Postfix, from userid 1086) id 322E320B74C9; Mon, 26 Feb 2024 21:41:40 -0800 (PST) DKIM-Filter: OpenDKIM Filter v2.11.0 linux.microsoft.com 322E320B74C9 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linux.microsoft.com; s=default; t=1709012501; bh=KVp3GaPTlA6IhRX2/cqo/lCeuFijymdaCpdHvBKfDFs=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=XedJP2uPq9TLkqjdCpGPJjkq4wO58DS6brQkSmKqg0cAke/qSypXd88NaFEsdnM76 onX1YKbBEJ/oExXh5KbNqzletGpHOYI6mHJv25aZNef+ALDnG0cbcwAVwPa7OaWSHl KloPqOTszPOWikswgd59zV7z+5xVMmjK4RgSWYyQ= From: Tyler Retzlaff To: dev@dpdk.org Cc: Ajit Khaparde , Andrew Boyer , Andrew Rybchenko , Bruce Richardson , Chenbo Xia , Chengwen Feng , Dariusz Sosnowski , David Christensen , Hyong Youb Kim , Jerin Jacob , Jie Hai , Jingjing Wu , John Daley , Kevin Laatz , Kiran Kumar K , Konstantin Ananyev , Maciej Czekaj , Matan Azrad , Maxime Coquelin , Nithin Dabilpuram , Ori Kam , Ruifeng Wang , Satha Rao , Somnath Kotur , Suanming Mou , Sunil Kumar Kori , Viacheslav Ovsiienko , Yisen Zhuang , Yuying Zhang , mb@smartsharesystems.com, Tyler Retzlaff Subject: [PATCH v6 09/23] net/i40e: use mbuf descriptor accessors Date: Mon, 26 Feb 2024 21:41:25 -0800 Message-Id: <1709012499-12813-10-git-send-email-roretzla@linux.microsoft.com> X-Mailer: git-send-email 1.8.3.1 In-Reply-To: <1709012499-12813-1-git-send-email-roretzla@linux.microsoft.com> References: <1706657173-26166-1-git-send-email-roretzla@linux.microsoft.com> <1709012499-12813-1-git-send-email-roretzla@linux.microsoft.com> X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org RTE_MARKER typedefs are a GCC extension unsupported by MSVC. Use new rte_mbuf_rearm_data and rte_mbuf_rx_descriptor_fields1 accessors that provide a compatible type pointer without using the marker fields. Use rte_mbuf_prefetch_part2() to prefetch cacheline1 and remove reference to rte marker field. Signed-off-by: Tyler Retzlaff --- drivers/net/i40e/i40e_rxtx_vec_altivec.c | 18 +++++-------- drivers/net/i40e/i40e_rxtx_vec_avx2.c | 34 ++++++------------------- drivers/net/i40e/i40e_rxtx_vec_avx512.c | 35 +++++++------------------- drivers/net/i40e/i40e_rxtx_vec_common.h | 4 +-- drivers/net/i40e/i40e_rxtx_vec_neon.c | 16 ++++++------ drivers/net/i40e/i40e_rxtx_vec_sse.c | 43 +++++++------------------------- 6 files changed, 41 insertions(+), 109 deletions(-) diff --git a/drivers/net/i40e/i40e_rxtx_vec_altivec.c b/drivers/net/i40e/i40e_rxtx_vec_altivec.c index b6b0d38..3e065ee 100644 --- a/drivers/net/i40e/i40e_rxtx_vec_altivec.c +++ b/drivers/net/i40e/i40e_rxtx_vec_altivec.c @@ -55,7 +55,6 @@ /* Initialize the mbufs in vector, process 2 mbufs in one loop */ for (i = 0; i < RTE_I40E_RXQ_REARM_THRESH; i += 2, rxep += 2) { __vector unsigned long vaddr0, vaddr1; - uintptr_t p0, p1; mb0 = rxep[0].mbuf; mb1 = rxep[1].mbuf; @@ -66,10 +65,8 @@ * anyway. So overwrite whole 8 bytes with one load: * 6 bytes of rearm_data plus first 2 bytes of ol_flags. */ - p0 = (uintptr_t)&mb0->rearm_data; - *(uint64_t *)p0 = rxq->mbuf_initializer; - p1 = (uintptr_t)&mb1->rearm_data; - *(uint64_t *)p1 = rxq->mbuf_initializer; + *rte_mbuf_rearm_data(mb0) = rxq->mbuf_initializer; + *rte_mbuf_rearm_data(mb1) = rxq->mbuf_initializer; /* load buf_addr(lo 64bit) and buf_iova(hi 64bit) */ vaddr0 = vec_ld(0, (__vector unsigned long *)&mb0->buf_addr); @@ -370,12 +367,10 @@ /* D.3 copy final 3,4 data to rx_pkts */ vec_st(pkt_mb4, 0, - (__vector unsigned char *)&rx_pkts[pos + 3] - ->rx_descriptor_fields1 + (__vector unsigned char *)rte_mbuf_rx_descriptor_fields1(rx_pkts[pos + 3]) ); vec_st(pkt_mb3, 0, - (__vector unsigned char *)&rx_pkts[pos + 2] - ->rx_descriptor_fields1 + (__vector unsigned char *)rte_mbuf_rx_descriptor_fields1(rx_pkts[pos + 2]) ); /* D.2 pkt 1,2 set in_port/nb_seg and remove crc */ @@ -422,11 +417,10 @@ /* D.3 copy final 1,2 data to rx_pkts */ vec_st(pkt_mb2, 0, - (__vector unsigned char *)&rx_pkts[pos + 1] - ->rx_descriptor_fields1 + (__vector unsigned char *)rte_mbuf_rx_descriptor_fields1(rx_pkts[pos + 1]) ); vec_st(pkt_mb1, 0, - (__vector unsigned char *)&rx_pkts[pos]->rx_descriptor_fields1 + (__vector unsigned char *)rte_mbuf_rx_descriptor_fields1(rx_pkts[pos]) ); desc_to_ptype_v(descs, &rx_pkts[pos], ptype_tbl); desc_to_olflags_v(descs, &rx_pkts[pos]); diff --git a/drivers/net/i40e/i40e_rxtx_vec_avx2.c b/drivers/net/i40e/i40e_rxtx_vec_avx2.c index f468c1f..360d80f 100644 --- a/drivers/net/i40e/i40e_rxtx_vec_avx2.c +++ b/drivers/net/i40e/i40e_rxtx_vec_avx2.c @@ -180,19 +180,6 @@ 0xFF, 0xFF, /* pkt_type set as unknown */ 0xFF, 0xFF /*pkt_type set as unknown */ ); - /* - * compile-time check the above crc and shuffle layout is correct. - * NOTE: the first field (lowest address) is given last in set_epi - * calls above. - */ - RTE_BUILD_BUG_ON(offsetof(struct rte_mbuf, pkt_len) != - offsetof(struct rte_mbuf, rx_descriptor_fields1) + 4); - RTE_BUILD_BUG_ON(offsetof(struct rte_mbuf, data_len) != - offsetof(struct rte_mbuf, rx_descriptor_fields1) + 8); - RTE_BUILD_BUG_ON(offsetof(struct rte_mbuf, vlan_tci) != - offsetof(struct rte_mbuf, rx_descriptor_fields1) + 10); - RTE_BUILD_BUG_ON(offsetof(struct rte_mbuf, hash) != - offsetof(struct rte_mbuf, rx_descriptor_fields1) + 12); /* Status/Error flag masks */ /* @@ -525,11 +512,6 @@ * add in the previously computed rx_descriptor fields to * make a single 256-bit write per mbuf */ - /* check the structure matches expectations */ - RTE_BUILD_BUG_ON(offsetof(struct rte_mbuf, ol_flags) != - offsetof(struct rte_mbuf, rearm_data) + 8); - RTE_BUILD_BUG_ON(offsetof(struct rte_mbuf, rearm_data) != - RTE_ALIGN(offsetof(struct rte_mbuf, rearm_data), 16)); /* build up data and do writes */ __m256i rearm0, rearm1, rearm2, rearm3, rearm4, rearm5, rearm6, rearm7; @@ -543,10 +525,10 @@ rearm2 = _mm256_permute2f128_si256(rearm2, mb2_3, 0x20); rearm0 = _mm256_permute2f128_si256(rearm0, mb0_1, 0x20); /* write to mbuf */ - _mm256_storeu_si256((__m256i *)&rx_pkts[i + 6]->rearm_data, rearm6); - _mm256_storeu_si256((__m256i *)&rx_pkts[i + 4]->rearm_data, rearm4); - _mm256_storeu_si256((__m256i *)&rx_pkts[i + 2]->rearm_data, rearm2); - _mm256_storeu_si256((__m256i *)&rx_pkts[i + 0]->rearm_data, rearm0); + _mm256_storeu_si256((__m256i *)rte_mbuf_rearm_data(rx_pkts[i + 6]), rearm6); + _mm256_storeu_si256((__m256i *)rte_mbuf_rearm_data(rx_pkts[i + 4]), rearm4); + _mm256_storeu_si256((__m256i *)rte_mbuf_rearm_data(rx_pkts[i + 2]), rearm2); + _mm256_storeu_si256((__m256i *)rte_mbuf_rearm_data(rx_pkts[i + 0]), rearm0); /* repeat for the odd mbufs */ const __m256i odd_flags = _mm256_castsi128_si256( @@ -561,10 +543,10 @@ rearm3 = _mm256_blend_epi32(rearm3, mb2_3, 0xF0); rearm1 = _mm256_blend_epi32(rearm1, mb0_1, 0xF0); /* again write to mbufs */ - _mm256_storeu_si256((__m256i *)&rx_pkts[i + 7]->rearm_data, rearm7); - _mm256_storeu_si256((__m256i *)&rx_pkts[i + 5]->rearm_data, rearm5); - _mm256_storeu_si256((__m256i *)&rx_pkts[i + 3]->rearm_data, rearm3); - _mm256_storeu_si256((__m256i *)&rx_pkts[i + 1]->rearm_data, rearm1); + _mm256_storeu_si256((__m256i *)rte_mbuf_rearm_data(rx_pkts[i + 7]), rearm7); + _mm256_storeu_si256((__m256i *)rte_mbuf_rearm_data(rx_pkts[i + 5]), rearm5); + _mm256_storeu_si256((__m256i *)rte_mbuf_rearm_data(rx_pkts[i + 3]), rearm3); + _mm256_storeu_si256((__m256i *)rte_mbuf_rearm_data(rx_pkts[i + 1]), rearm1); /* extract and record EOP bit */ if (split_packet) { diff --git a/drivers/net/i40e/i40e_rxtx_vec_avx512.c b/drivers/net/i40e/i40e_rxtx_vec_avx512.c index f3050cd..e13bd2f 100644 --- a/drivers/net/i40e/i40e_rxtx_vec_avx512.c +++ b/drivers/net/i40e/i40e_rxtx_vec_avx512.c @@ -170,18 +170,6 @@ /* pkt_type set as unknown */ 0xFFFFFFFF ); - /* compile-time check the above crc and shuffle layout is correct. - * NOTE: the first field (lowest address) is given last in set_epi - * calls above. - */ - RTE_BUILD_BUG_ON(offsetof(struct rte_mbuf, pkt_len) != - offsetof(struct rte_mbuf, rx_descriptor_fields1) + 4); - RTE_BUILD_BUG_ON(offsetof(struct rte_mbuf, data_len) != - offsetof(struct rte_mbuf, rx_descriptor_fields1) + 8); - RTE_BUILD_BUG_ON(offsetof(struct rte_mbuf, vlan_tci) != - offsetof(struct rte_mbuf, rx_descriptor_fields1) + 10); - RTE_BUILD_BUG_ON(offsetof(struct rte_mbuf, hash) != - offsetof(struct rte_mbuf, rx_descriptor_fields1) + 12); /* Status/Error flag masks */ /* mask everything except RSS, flow director and VLAN flags @@ -557,11 +545,6 @@ * add in the previously computed rx_descriptor fields to * make a single 256-bit write per mbuf */ - /* check the structure matches expectations */ - RTE_BUILD_BUG_ON(offsetof(struct rte_mbuf, ol_flags) != - offsetof(struct rte_mbuf, rearm_data) + 8); - RTE_BUILD_BUG_ON(offsetof(struct rte_mbuf, rearm_data) != - RTE_ALIGN(offsetof(struct rte_mbuf, rearm_data), 16)); /* build up data and do writes */ __m256i rearm0, rearm1, rearm2, rearm3, rearm4, rearm5, rearm6, rearm7; @@ -580,13 +563,13 @@ rearm0 = _mm256_permute2f128_si256(rearm0, mb0_1, 0x20); /* write to mbuf */ _mm256_storeu_si256 - ((__m256i *)&rx_pkts[i + 6]->rearm_data, rearm6); + ((__m256i *)rte_mbuf_rearm_data(rx_pkts[i + 6]), rearm6); _mm256_storeu_si256 - ((__m256i *)&rx_pkts[i + 4]->rearm_data, rearm4); + ((__m256i *)rte_mbuf_rearm_data(rx_pkts[i + 4]), rearm4); _mm256_storeu_si256 - ((__m256i *)&rx_pkts[i + 2]->rearm_data, rearm2); + ((__m256i *)rte_mbuf_rearm_data(rx_pkts[i + 2]), rearm2); _mm256_storeu_si256 - ((__m256i *)&rx_pkts[i + 0]->rearm_data, rearm0); + ((__m256i *)rte_mbuf_rearm_data(rx_pkts[i + 0]), rearm0); /* repeat for the odd mbufs */ const __m256i odd_flags = _mm256_castsi128_si256 @@ -606,13 +589,13 @@ rearm1 = _mm256_blend_epi32(rearm1, mb0_1, 0xF0); /* again write to mbufs */ _mm256_storeu_si256 - ((__m256i *)&rx_pkts[i + 7]->rearm_data, rearm7); + ((__m256i *)rte_mbuf_rearm_data(rx_pkts[i + 7]), rearm7); _mm256_storeu_si256 - ((__m256i *)&rx_pkts[i + 5]->rearm_data, rearm5); + ((__m256i *)rte_mbuf_rearm_data(rx_pkts[i + 5]), rearm5); _mm256_storeu_si256 - ((__m256i *)&rx_pkts[i + 3]->rearm_data, rearm3); + ((__m256i *)rte_mbuf_rearm_data(rx_pkts[i + 3]), rearm3); _mm256_storeu_si256 - ((__m256i *)&rx_pkts[i + 1]->rearm_data, rearm1); + ((__m256i *)rte_mbuf_rearm_data(rx_pkts[i + 1]), rearm1); /* extract and record EOP bit */ if (split_packet) { @@ -826,7 +809,7 @@ free[0] = m; nb_free = 1; for (i = 1; i < n; i++) { - rte_prefetch0(&txep[i + 3].mbuf->cacheline1); + rte_mbuf_prefetch_part2(txep[i + 3].mbuf); m = rte_pktmbuf_prefree_seg(txep[i].mbuf); if (likely(m)) { if (likely(m->pool == free[0]->pool)) { diff --git a/drivers/net/i40e/i40e_rxtx_vec_common.h b/drivers/net/i40e/i40e_rxtx_vec_common.h index 8b74563..5633268 100644 --- a/drivers/net/i40e/i40e_rxtx_vec_common.h +++ b/drivers/net/i40e/i40e_rxtx_vec_common.h @@ -189,7 +189,6 @@ static inline int i40e_rxq_vec_setup_default(struct i40e_rx_queue *rxq) { - uintptr_t p; struct rte_mbuf mb_def = { .buf_addr = 0 }; /* zeroed mbuf */ mb_def.nb_segs = 1; @@ -199,8 +198,7 @@ /* prevent compiler reordering: rearm_data covers previous fields */ rte_compiler_barrier(); - p = (uintptr_t)&mb_def.rearm_data; - rxq->mbuf_initializer = *(uint64_t *)p; + rxq->mbuf_initializer = *rte_mbuf_rearm_data(&mb_def); rxq->rx_using_sse = 1; return 0; } diff --git a/drivers/net/i40e/i40e_rxtx_vec_neon.c b/drivers/net/i40e/i40e_rxtx_vec_neon.c index d873e30..29dfd92 100644 --- a/drivers/net/i40e/i40e_rxtx_vec_neon.c +++ b/drivers/net/i40e/i40e_rxtx_vec_neon.c @@ -300,10 +300,10 @@ rearm2 = vsetq_lane_u64(vgetq_lane_u32(vlan0, 2), mbuf_init, 1); rearm3 = vsetq_lane_u64(vgetq_lane_u32(vlan0, 3), mbuf_init, 1); - vst1q_u64((uint64_t *)&rx_pkts[0]->rearm_data, rearm0); - vst1q_u64((uint64_t *)&rx_pkts[1]->rearm_data, rearm1); - vst1q_u64((uint64_t *)&rx_pkts[2]->rearm_data, rearm2); - vst1q_u64((uint64_t *)&rx_pkts[3]->rearm_data, rearm3); + vst1q_u64(rte_mbuf_rearm_data(rx_pkts[0]), rearm0); + vst1q_u64(rte_mbuf_rearm_data(rx_pkts[1]), rearm1); + vst1q_u64(rte_mbuf_rearm_data(rx_pkts[2]), rearm2); + vst1q_u64(rte_mbuf_rearm_data(rx_pkts[3]), rearm3); } #define PKTLEN_SHIFT 10 @@ -492,13 +492,13 @@ pkt_mb1 = vreinterpretq_u8_u16(tmp); /* D.3 copy final data to rx_pkts */ - vst1q_u8((void *)&rx_pkts[pos + 3]->rx_descriptor_fields1, + vst1q_u8(rte_mbuf_rx_descriptor_fields1(rx_pkts[pos + 3]), pkt_mb4); - vst1q_u8((void *)&rx_pkts[pos + 2]->rx_descriptor_fields1, + vst1q_u8(rte_mbuf_rx_descriptor_fields1(rx_pkts[pos + 2]), pkt_mb3); - vst1q_u8((void *)&rx_pkts[pos + 1]->rx_descriptor_fields1, + vst1q_u8(rte_mbuf_rx_descriptor_fields1(rx_pkts[pos + 1]), pkt_mb2); - vst1q_u8((void *)&rx_pkts[pos]->rx_descriptor_fields1, + vst1q_u8(rte_mbuf_rx_descriptor_fields1(rx_pkts[pos]), pkt_mb1); desc_to_ptype_v(descs, &rx_pkts[pos], ptype_tbl); diff --git a/drivers/net/i40e/i40e_rxtx_vec_sse.c b/drivers/net/i40e/i40e_rxtx_vec_sse.c index 2d4480a..994c5e1 100644 --- a/drivers/net/i40e/i40e_rxtx_vec_sse.c +++ b/drivers/net/i40e/i40e_rxtx_vec_sse.c @@ -315,14 +315,10 @@ rearm3 = _mm_blend_epi16(mbuf_init, _mm_srli_si128(vlan0, 4), 0x10); /* write the rearm data and the olflags in one write */ - RTE_BUILD_BUG_ON(offsetof(struct rte_mbuf, ol_flags) != - offsetof(struct rte_mbuf, rearm_data) + 8); - RTE_BUILD_BUG_ON(offsetof(struct rte_mbuf, rearm_data) != - RTE_ALIGN(offsetof(struct rte_mbuf, rearm_data), 16)); - _mm_store_si128((__m128i *)&rx_pkts[0]->rearm_data, rearm0); - _mm_store_si128((__m128i *)&rx_pkts[1]->rearm_data, rearm1); - _mm_store_si128((__m128i *)&rx_pkts[2]->rearm_data, rearm2); - _mm_store_si128((__m128i *)&rx_pkts[3]->rearm_data, rearm3); + _mm_store_si128((__m128i *)rte_mbuf_rearm_data(rx_pkts[0]), rearm0); + _mm_store_si128((__m128i *)rte_mbuf_rearm_data(rx_pkts[1]), rearm1); + _mm_store_si128((__m128i *)rte_mbuf_rearm_data(rx_pkts[2]), rearm2); + _mm_store_si128((__m128i *)rte_mbuf_rearm_data(rx_pkts[3]), rearm3); } #define PKTLEN_SHIFT 10 @@ -369,15 +365,7 @@ -rxq->crc_len, /* sub crc on pkt_len */ 0, 0 /* ignore pkt_type field */ ); - /* - * compile-time check the above crc_adjust layout is correct. - * NOTE: the first field (lowest address) is given last in set_epi16 - * call above. - */ - RTE_BUILD_BUG_ON(offsetof(struct rte_mbuf, pkt_len) != - offsetof(struct rte_mbuf, rx_descriptor_fields1) + 4); - RTE_BUILD_BUG_ON(offsetof(struct rte_mbuf, data_len) != - offsetof(struct rte_mbuf, rx_descriptor_fields1) + 8); + __m128i dd_check, eop_check; /* nb_pkts has to be floor-aligned to RTE_I40E_DESCS_PER_LOOP */ @@ -419,19 +407,6 @@ 0xFF, 0xFF, /* pkt_type set as unknown */ 0xFF, 0xFF /*pkt_type set as unknown */ ); - /* - * Compile-time verify the shuffle mask - * NOTE: some field positions already verified above, but duplicated - * here for completeness in case of future modifications. - */ - RTE_BUILD_BUG_ON(offsetof(struct rte_mbuf, pkt_len) != - offsetof(struct rte_mbuf, rx_descriptor_fields1) + 4); - RTE_BUILD_BUG_ON(offsetof(struct rte_mbuf, data_len) != - offsetof(struct rte_mbuf, rx_descriptor_fields1) + 8); - RTE_BUILD_BUG_ON(offsetof(struct rte_mbuf, vlan_tci) != - offsetof(struct rte_mbuf, rx_descriptor_fields1) + 10); - RTE_BUILD_BUG_ON(offsetof(struct rte_mbuf, hash) != - offsetof(struct rte_mbuf, rx_descriptor_fields1) + 12); /* Cache is empty -> need to scan the buffer rings, but first move * the next 'n' mbufs into the cache @@ -535,9 +510,9 @@ staterr = _mm_unpacklo_epi32(sterr_tmp1, sterr_tmp2); /* D.3 copy final 3,4 data to rx_pkts */ - _mm_storeu_si128((void *)&rx_pkts[pos+3]->rx_descriptor_fields1, + _mm_storeu_si128(rte_mbuf_rx_descriptor_fields1(rx_pkts[pos + 3]), pkt_mb4); - _mm_storeu_si128((void *)&rx_pkts[pos+2]->rx_descriptor_fields1, + _mm_storeu_si128(rte_mbuf_rx_descriptor_fields1(rx_pkts[pos + 2]), pkt_mb3); /* D.2 pkt 1,2 set in_port/nb_seg and remove crc */ @@ -571,9 +546,9 @@ staterr = _mm_packs_epi32(staterr, zero); /* D.3 copy final 1,2 data to rx_pkts */ - _mm_storeu_si128((void *)&rx_pkts[pos+1]->rx_descriptor_fields1, + _mm_storeu_si128(rte_mbuf_rx_descriptor_fields1(rx_pkts[pos + 1]), pkt_mb2); - _mm_storeu_si128((void *)&rx_pkts[pos]->rx_descriptor_fields1, + _mm_storeu_si128(rte_mbuf_rx_descriptor_fields1(rx_pkts[pos]), pkt_mb1); desc_to_ptype_v(descs, &rx_pkts[pos], ptype_tbl); /* C.4 calc available number of desc */ From patchwork Tue Feb 27 05:41:26 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Tyler Retzlaff X-Patchwork-Id: 137302 X-Patchwork-Delegate: thomas@monjalon.net Return-Path: X-Original-To: patchwork@inbox.dpdk.org Delivered-To: patchwork@inbox.dpdk.org Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id 2BCD243C03; Tue, 27 Feb 2024 06:42:58 +0100 (CET) Received: from mails.dpdk.org (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id 4AFD342E9F; Tue, 27 Feb 2024 06:42:22 +0100 (CET) Received: from linux.microsoft.com (linux.microsoft.com [13.77.154.182]) by mails.dpdk.org (Postfix) with ESMTP id 91AD040295 for ; Tue, 27 Feb 2024 06:41:42 +0100 (CET) Received: by linux.microsoft.com (Postfix, from userid 1086) id 403FB20B74CA; Mon, 26 Feb 2024 21:41:40 -0800 (PST) DKIM-Filter: OpenDKIM Filter v2.11.0 linux.microsoft.com 403FB20B74CA DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linux.microsoft.com; s=default; t=1709012501; bh=BeEEZxgfyYklvMVnTPlxRiS5ANFInBF/rgMXXjDyXvY=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=sssfvs12wbLvIUOktEyG1PVqLfD5K00NrgnB0ZL2ndMacG3vhlZmeAysyPVJ2+Elt hb2Ad/yqQXaEBXICOKeN/+oUsOEZ2zNA89rC5nvHapM2ik0LhNOvTGmqjDP0YvcHiu 1lFoN+D8H3JJQpWuzDUA84unYkeZ28p45B1K0rJI= From: Tyler Retzlaff To: dev@dpdk.org Cc: Ajit Khaparde , Andrew Boyer , Andrew Rybchenko , Bruce Richardson , Chenbo Xia , Chengwen Feng , Dariusz Sosnowski , David Christensen , Hyong Youb Kim , Jerin Jacob , Jie Hai , Jingjing Wu , John Daley , Kevin Laatz , Kiran Kumar K , Konstantin Ananyev , Maciej Czekaj , Matan Azrad , Maxime Coquelin , Nithin Dabilpuram , Ori Kam , Ruifeng Wang , Satha Rao , Somnath Kotur , Suanming Mou , Sunil Kumar Kori , Viacheslav Ovsiienko , Yisen Zhuang , Yuying Zhang , mb@smartsharesystems.com, Tyler Retzlaff Subject: [PATCH v6 10/23] net/iavf: use mbuf descriptor accessors Date: Mon, 26 Feb 2024 21:41:26 -0800 Message-Id: <1709012499-12813-11-git-send-email-roretzla@linux.microsoft.com> X-Mailer: git-send-email 1.8.3.1 In-Reply-To: <1709012499-12813-1-git-send-email-roretzla@linux.microsoft.com> References: <1706657173-26166-1-git-send-email-roretzla@linux.microsoft.com> <1709012499-12813-1-git-send-email-roretzla@linux.microsoft.com> X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org RTE_MARKER typedefs are a GCC extension unsupported by MSVC. Use new rte_mbuf_rearm_data and rte_mbuf_rx_descriptor_fields1 accessors that provide a compatible type pointer without using the marker fields. Signed-off-by: Tyler Retzlaff --- drivers/net/iavf/iavf_rxtx_vec_avx2.c | 72 +++++++--------------------- drivers/net/iavf/iavf_rxtx_vec_avx512.c | 72 +++++++--------------------- drivers/net/iavf/iavf_rxtx_vec_common.h | 4 +- drivers/net/iavf/iavf_rxtx_vec_neon.c | 16 +++---- drivers/net/iavf/iavf_rxtx_vec_sse.c | 85 +++++++-------------------------- 5 files changed, 58 insertions(+), 191 deletions(-) diff --git a/drivers/net/iavf/iavf_rxtx_vec_avx2.c b/drivers/net/iavf/iavf_rxtx_vec_avx2.c index 510b4d8..33f2850 100644 --- a/drivers/net/iavf/iavf_rxtx_vec_avx2.c +++ b/drivers/net/iavf/iavf_rxtx_vec_avx2.c @@ -98,19 +98,6 @@ 0xFF, 0xFF, /* pkt_type set as unknown */ 0xFF, 0xFF /*pkt_type set as unknown */ ); - /** - * compile-time check the above crc and shuffle layout is correct. - * NOTE: the first field (lowest address) is given last in set_epi - * calls above. - */ - RTE_BUILD_BUG_ON(offsetof(struct rte_mbuf, pkt_len) != - offsetof(struct rte_mbuf, rx_descriptor_fields1) + 4); - RTE_BUILD_BUG_ON(offsetof(struct rte_mbuf, data_len) != - offsetof(struct rte_mbuf, rx_descriptor_fields1) + 8); - RTE_BUILD_BUG_ON(offsetof(struct rte_mbuf, vlan_tci) != - offsetof(struct rte_mbuf, rx_descriptor_fields1) + 10); - RTE_BUILD_BUG_ON(offsetof(struct rte_mbuf, hash) != - offsetof(struct rte_mbuf, rx_descriptor_fields1) + 12); /* Status/Error flag masks */ /** @@ -372,13 +359,6 @@ * add in the previously computed rx_descriptor fields to * make a single 256-bit write per mbuf */ - /* check the structure matches expectations */ - RTE_BUILD_BUG_ON(offsetof(struct rte_mbuf, ol_flags) != - offsetof(struct rte_mbuf, rearm_data) + 8); - RTE_BUILD_BUG_ON(offsetof(struct rte_mbuf, rearm_data) != - RTE_ALIGN(offsetof(struct rte_mbuf, - rearm_data), - 16)); /* build up data and do writes */ __m256i rearm0, rearm1, rearm2, rearm3, rearm4, rearm5, rearm6, rearm7; @@ -398,13 +378,13 @@ rearm2 = _mm256_permute2f128_si256(rearm2, mb2_3, 0x20); rearm0 = _mm256_permute2f128_si256(rearm0, mb0_1, 0x20); /* write to mbuf */ - _mm256_storeu_si256((__m256i *)&rx_pkts[i + 6]->rearm_data, + _mm256_storeu_si256((__m256i *)rte_mbuf_rearm_data(rx_pkts[i + 6]), rearm6); - _mm256_storeu_si256((__m256i *)&rx_pkts[i + 4]->rearm_data, + _mm256_storeu_si256((__m256i *)rte_mbuf_rearm_data(rx_pkts[i + 4]), rearm4); - _mm256_storeu_si256((__m256i *)&rx_pkts[i + 2]->rearm_data, + _mm256_storeu_si256((__m256i *)rte_mbuf_rearm_data(rx_pkts[i + 2]), rearm2); - _mm256_storeu_si256((__m256i *)&rx_pkts[i + 0]->rearm_data, + _mm256_storeu_si256((__m256i *)rte_mbuf_rearm_data(rx_pkts[i + 0]), rearm0); /* repeat for the odd mbufs */ @@ -427,13 +407,13 @@ rearm3 = _mm256_blend_epi32(rearm3, mb2_3, 0xF0); rearm1 = _mm256_blend_epi32(rearm1, mb0_1, 0xF0); /* again write to mbufs */ - _mm256_storeu_si256((__m256i *)&rx_pkts[i + 7]->rearm_data, + _mm256_storeu_si256((__m256i *)rte_mbuf_rearm_data(rx_pkts[i + 7]), rearm7); - _mm256_storeu_si256((__m256i *)&rx_pkts[i + 5]->rearm_data, + _mm256_storeu_si256((__m256i *)rte_mbuf_rearm_data(rx_pkts[i + 5]), rearm5); - _mm256_storeu_si256((__m256i *)&rx_pkts[i + 3]->rearm_data, + _mm256_storeu_si256((__m256i *)rte_mbuf_rearm_data(rx_pkts[i + 3]), rearm3); - _mm256_storeu_si256((__m256i *)&rx_pkts[i + 1]->rearm_data, + _mm256_storeu_si256((__m256i *)rte_mbuf_rearm_data(rx_pkts[i + 1]), rearm1); /* extract and record EOP bit */ @@ -622,19 +602,6 @@ 0xFF, 0xFF, /* pkt_type set as unknown */ 0xFF, 0xFF /*pkt_type set as unknown */ ); - /** - * compile-time check the above crc and shuffle layout is correct. - * NOTE: the first field (lowest address) is given last in set_epi - * calls above. - */ - RTE_BUILD_BUG_ON(offsetof(struct rte_mbuf, pkt_len) != - offsetof(struct rte_mbuf, rx_descriptor_fields1) + 4); - RTE_BUILD_BUG_ON(offsetof(struct rte_mbuf, data_len) != - offsetof(struct rte_mbuf, rx_descriptor_fields1) + 8); - RTE_BUILD_BUG_ON(offsetof(struct rte_mbuf, vlan_tci) != - offsetof(struct rte_mbuf, rx_descriptor_fields1) + 10); - RTE_BUILD_BUG_ON(offsetof(struct rte_mbuf, hash) != - offsetof(struct rte_mbuf, rx_descriptor_fields1) + 12); /* Status/Error flag masks */ /** @@ -1279,13 +1246,6 @@ * add in the previously computed rx_descriptor fields to * make a single 256-bit write per mbuf */ - /* check the structure matches expectations */ - RTE_BUILD_BUG_ON(offsetof(struct rte_mbuf, ol_flags) != - offsetof(struct rte_mbuf, rearm_data) + 8); - RTE_BUILD_BUG_ON(offsetof(struct rte_mbuf, rearm_data) != - RTE_ALIGN(offsetof(struct rte_mbuf, - rearm_data), - 16)); /* build up data and do writes */ __m256i rearm0, rearm1, rearm2, rearm3, rearm4, rearm5, rearm6, rearm7; @@ -1305,13 +1265,13 @@ rearm2 = _mm256_permute2f128_si256(rearm2, mb2_3, 0x20); rearm0 = _mm256_permute2f128_si256(rearm0, mb0_1, 0x20); /* write to mbuf */ - _mm256_storeu_si256((__m256i *)&rx_pkts[i + 6]->rearm_data, + _mm256_storeu_si256((__m256i *)rte_mbuf_rearm_data(rx_pkts[i + 6]), rearm6); - _mm256_storeu_si256((__m256i *)&rx_pkts[i + 4]->rearm_data, + _mm256_storeu_si256((__m256i *)rte_mbuf_rearm_data(rx_pkts[i + 4]), rearm4); - _mm256_storeu_si256((__m256i *)&rx_pkts[i + 2]->rearm_data, + _mm256_storeu_si256((__m256i *)rte_mbuf_rearm_data(rx_pkts[i + 2]), rearm2); - _mm256_storeu_si256((__m256i *)&rx_pkts[i + 0]->rearm_data, + _mm256_storeu_si256((__m256i *)rte_mbuf_rearm_data(rx_pkts[i + 0]), rearm0); /* repeat for the odd mbufs */ @@ -1334,13 +1294,13 @@ rearm3 = _mm256_blend_epi32(rearm3, mb2_3, 0xF0); rearm1 = _mm256_blend_epi32(rearm1, mb0_1, 0xF0); /* again write to mbufs */ - _mm256_storeu_si256((__m256i *)&rx_pkts[i + 7]->rearm_data, + _mm256_storeu_si256((__m256i *)rte_mbuf_rearm_data(rx_pkts[i + 7]), rearm7); - _mm256_storeu_si256((__m256i *)&rx_pkts[i + 5]->rearm_data, + _mm256_storeu_si256((__m256i *)rte_mbuf_rearm_data(rx_pkts[i + 5]), rearm5); - _mm256_storeu_si256((__m256i *)&rx_pkts[i + 3]->rearm_data, + _mm256_storeu_si256((__m256i *)rte_mbuf_rearm_data(rx_pkts[i + 3]), rearm3); - _mm256_storeu_si256((__m256i *)&rx_pkts[i + 1]->rearm_data, + _mm256_storeu_si256((__m256i *)rte_mbuf_rearm_data(rx_pkts[i + 1]), rearm1); /* extract and record EOP bit */ diff --git a/drivers/net/iavf/iavf_rxtx_vec_avx512.c b/drivers/net/iavf/iavf_rxtx_vec_avx512.c index 3bb6f30..38d0669 100644 --- a/drivers/net/iavf/iavf_rxtx_vec_avx512.c +++ b/drivers/net/iavf/iavf_rxtx_vec_avx512.c @@ -135,19 +135,6 @@ /* octet 15~14, low 16 bits pkt_len */ 0xFFFFFFFF /* pkt_type set as unknown */ ); - /** - * compile-time check the above crc and shuffle layout is correct. - * NOTE: the first field (lowest address) is given last in set_epi - * calls above. - */ - RTE_BUILD_BUG_ON(offsetof(struct rte_mbuf, pkt_len) != - offsetof(struct rte_mbuf, rx_descriptor_fields1) + 4); - RTE_BUILD_BUG_ON(offsetof(struct rte_mbuf, data_len) != - offsetof(struct rte_mbuf, rx_descriptor_fields1) + 8); - RTE_BUILD_BUG_ON(offsetof(struct rte_mbuf, vlan_tci) != - offsetof(struct rte_mbuf, rx_descriptor_fields1) + 10); - RTE_BUILD_BUG_ON(offsetof(struct rte_mbuf, hash) != - offsetof(struct rte_mbuf, rx_descriptor_fields1) + 12); uint16_t i, received; @@ -412,13 +399,6 @@ * add in the previously computed rx_descriptor fields to * make a single 256-bit write per mbuf */ - /* check the structure matches expectations */ - RTE_BUILD_BUG_ON(offsetof(struct rte_mbuf, ol_flags) != - offsetof(struct rte_mbuf, rearm_data) + 8); - RTE_BUILD_BUG_ON(offsetof(struct rte_mbuf, rearm_data) != - RTE_ALIGN(offsetof(struct rte_mbuf, - rearm_data), - 16)); /* build up data and do writes */ __m256i rearm0, rearm1, rearm2, rearm3, rearm4, rearm5, rearm6, rearm7; @@ -450,13 +430,13 @@ rearm0 = _mm256_permute2f128_si256(mbuf_init, mb0_1, 0x20); } /* write to mbuf */ - _mm256_storeu_si256((__m256i *)&rx_pkts[i + 6]->rearm_data, + _mm256_storeu_si256((__m256i *)rte_mbuf_rearm_data(rx_pkts[i + 6]), rearm6); - _mm256_storeu_si256((__m256i *)&rx_pkts[i + 4]->rearm_data, + _mm256_storeu_si256((__m256i *)rte_mbuf_rearm_data(rx_pkts[i + 4]), rearm4); - _mm256_storeu_si256((__m256i *)&rx_pkts[i + 2]->rearm_data, + _mm256_storeu_si256((__m256i *)rte_mbuf_rearm_data(rx_pkts[i + 2]), rearm2); - _mm256_storeu_si256((__m256i *)&rx_pkts[i + 0]->rearm_data, + _mm256_storeu_si256((__m256i *)rte_mbuf_rearm_data(rx_pkts[i + 0]), rearm0); /* repeat for the odd mbufs */ @@ -486,13 +466,13 @@ rearm1 = _mm256_blend_epi32(mbuf_init, mb0_1, 0xF0); } /* again write to mbufs */ - _mm256_storeu_si256((__m256i *)&rx_pkts[i + 7]->rearm_data, + _mm256_storeu_si256((__m256i *)rte_mbuf_rearm_data(rx_pkts[i + 7]), rearm7); - _mm256_storeu_si256((__m256i *)&rx_pkts[i + 5]->rearm_data, + _mm256_storeu_si256((__m256i *)rte_mbuf_rearm_data(rx_pkts[i + 5]), rearm5); - _mm256_storeu_si256((__m256i *)&rx_pkts[i + 3]->rearm_data, + _mm256_storeu_si256((__m256i *)rte_mbuf_rearm_data(rx_pkts[i + 3]), rearm3); - _mm256_storeu_si256((__m256i *)&rx_pkts[i + 1]->rearm_data, + _mm256_storeu_si256((__m256i *)rte_mbuf_rearm_data(rx_pkts[i + 1]), rearm1); /* extract and record EOP bit */ @@ -703,19 +683,6 @@ /* octet 4~5, 16 bits pkt_len */ 0xFFFFFFFF /* pkt_type set as unknown */ ); - /** - * compile-time check the above crc and shuffle layout is correct. - * NOTE: the first field (lowest address) is given last in set_epi - * calls above. - */ - RTE_BUILD_BUG_ON(offsetof(struct rte_mbuf, pkt_len) != - offsetof(struct rte_mbuf, rx_descriptor_fields1) + 4); - RTE_BUILD_BUG_ON(offsetof(struct rte_mbuf, data_len) != - offsetof(struct rte_mbuf, rx_descriptor_fields1) + 8); - RTE_BUILD_BUG_ON(offsetof(struct rte_mbuf, vlan_tci) != - offsetof(struct rte_mbuf, rx_descriptor_fields1) + 10); - RTE_BUILD_BUG_ON(offsetof(struct rte_mbuf, hash) != - offsetof(struct rte_mbuf, rx_descriptor_fields1) + 12); uint16_t i, received; @@ -1435,13 +1402,6 @@ * add in the previously computed rx_descriptor fields to * make a single 256-bit write per mbuf */ - /* check the structure matches expectations */ - RTE_BUILD_BUG_ON(offsetof(struct rte_mbuf, ol_flags) != - offsetof(struct rte_mbuf, rearm_data) + 8); - RTE_BUILD_BUG_ON(offsetof(struct rte_mbuf, rearm_data) != - RTE_ALIGN(offsetof(struct rte_mbuf, - rearm_data), - 16)); /* build up data and do writes */ __m256i rearm0, rearm1, rearm2, rearm3, rearm4, rearm5, rearm6, rearm7; @@ -1461,13 +1421,13 @@ rearm2 = _mm256_permute2f128_si256(rearm2, mb2_3, 0x20); rearm0 = _mm256_permute2f128_si256(rearm0, mb0_1, 0x20); /* write to mbuf */ - _mm256_storeu_si256((__m256i *)&rx_pkts[i + 6]->rearm_data, + _mm256_storeu_si256((__m256i *)rte_mbuf_rearm_data(rx_pkts[i + 6]), rearm6); - _mm256_storeu_si256((__m256i *)&rx_pkts[i + 4]->rearm_data, + _mm256_storeu_si256((__m256i *)rte_mbuf_rearm_data(rx_pkts[i + 4]), rearm4); - _mm256_storeu_si256((__m256i *)&rx_pkts[i + 2]->rearm_data, + _mm256_storeu_si256((__m256i *)rte_mbuf_rearm_data(rx_pkts[i + 2]), rearm2); - _mm256_storeu_si256((__m256i *)&rx_pkts[i + 0]->rearm_data, + _mm256_storeu_si256((__m256i *)rte_mbuf_rearm_data(rx_pkts[i + 0]), rearm0); /* repeat for the odd mbufs */ @@ -1490,13 +1450,13 @@ rearm3 = _mm256_blend_epi32(rearm3, mb2_3, 0xF0); rearm1 = _mm256_blend_epi32(rearm1, mb0_1, 0xF0); /* again write to mbufs */ - _mm256_storeu_si256((__m256i *)&rx_pkts[i + 7]->rearm_data, + _mm256_storeu_si256((__m256i *)rte_mbuf_rearm_data(rx_pkts[i + 7]), rearm7); - _mm256_storeu_si256((__m256i *)&rx_pkts[i + 5]->rearm_data, + _mm256_storeu_si256((__m256i *)rte_mbuf_rearm_data(rx_pkts[i + 5]), rearm5); - _mm256_storeu_si256((__m256i *)&rx_pkts[i + 3]->rearm_data, + _mm256_storeu_si256((__m256i *)rte_mbuf_rearm_data(rx_pkts[i + 3]), rearm3); - _mm256_storeu_si256((__m256i *)&rx_pkts[i + 1]->rearm_data, + _mm256_storeu_si256((__m256i *)rte_mbuf_rearm_data(rx_pkts[i + 1]), rearm1); /* extract and record EOP bit */ diff --git a/drivers/net/iavf/iavf_rxtx_vec_common.h b/drivers/net/iavf/iavf_rxtx_vec_common.h index 5c52200..71e3644 100644 --- a/drivers/net/iavf/iavf_rxtx_vec_common.h +++ b/drivers/net/iavf/iavf_rxtx_vec_common.h @@ -197,7 +197,6 @@ static inline int iavf_rxq_vec_setup_default(struct iavf_rx_queue *rxq) { - uintptr_t p; struct rte_mbuf mb_def = { .buf_addr = 0 }; /* zeroed mbuf */ mb_def.nb_segs = 1; @@ -207,8 +206,7 @@ /* prevent compiler reordering: rearm_data covers previous fields */ rte_compiler_barrier(); - p = (uintptr_t)&mb_def.rearm_data; - rxq->mbuf_initializer = *(uint64_t *)p; + rxq->mbuf_initializer = *rte_mbuf_rearm_data(&mb_def); return 0; } diff --git a/drivers/net/iavf/iavf_rxtx_vec_neon.c b/drivers/net/iavf/iavf_rxtx_vec_neon.c index 83825aa..d7ea940 100644 --- a/drivers/net/iavf/iavf_rxtx_vec_neon.c +++ b/drivers/net/iavf/iavf_rxtx_vec_neon.c @@ -159,10 +159,10 @@ rearm2 = vsetq_lane_u64(vgetq_lane_u32(vlan0, 2), mbuf_init, 1); rearm3 = vsetq_lane_u64(vgetq_lane_u32(vlan0, 3), mbuf_init, 1); - vst1q_u64((uint64_t *)&rx_pkts[0]->rearm_data, rearm0); - vst1q_u64((uint64_t *)&rx_pkts[1]->rearm_data, rearm1); - vst1q_u64((uint64_t *)&rx_pkts[2]->rearm_data, rearm2); - vst1q_u64((uint64_t *)&rx_pkts[3]->rearm_data, rearm3); + vst1q_u64(rte_mbuf_rearm_data(rx_pkts[0]), rearm0); + vst1q_u64(rte_mbuf_rearm_data(rx_pkts[1]), rearm1); + vst1q_u64(rte_mbuf_rearm_data(rx_pkts[2]), rearm2); + vst1q_u64(rte_mbuf_rearm_data(rx_pkts[3]), rearm3); } #define PKTLEN_SHIFT 10 @@ -332,13 +332,13 @@ pkt_mb1 = vreinterpretq_u8_u16(tmp); /* D.3 copy final data to rx_pkts */ - vst1q_u8((void *)&rx_pkts[pos + 3]->rx_descriptor_fields1, + vst1q_u8(rte_mbuf_rx_descriptor_fields1(rx_pkts[pos + 3]), pkt_mb4); - vst1q_u8((void *)&rx_pkts[pos + 2]->rx_descriptor_fields1, + vst1q_u8(rte_mbuf_rx_descriptor_fields1(rx_pkts[pos + 2]), pkt_mb3); - vst1q_u8((void *)&rx_pkts[pos + 1]->rx_descriptor_fields1, + vst1q_u8(rte_mbuf_rx_descriptor_fields1(rx_pkts[pos + 1]), pkt_mb2); - vst1q_u8((void *)&rx_pkts[pos]->rx_descriptor_fields1, + vst1q_u8(rte_mbuf_rx_descriptor_fields1(rx_pkts[pos]), pkt_mb1); desc_to_ptype_v(descs, &rx_pkts[pos], ptype_tbl); diff --git a/drivers/net/iavf/iavf_rxtx_vec_sse.c b/drivers/net/iavf/iavf_rxtx_vec_sse.c index 96f187f..9d6a453 100644 --- a/drivers/net/iavf/iavf_rxtx_vec_sse.c +++ b/drivers/net/iavf/iavf_rxtx_vec_sse.c @@ -179,14 +179,10 @@ rearm3 = _mm_blend_epi16(mbuf_init, _mm_srli_si128(vlan0, 4), 0x10); /* write the rearm data and the olflags in one write */ - RTE_BUILD_BUG_ON(offsetof(struct rte_mbuf, ol_flags) != - offsetof(struct rte_mbuf, rearm_data) + 8); - RTE_BUILD_BUG_ON(offsetof(struct rte_mbuf, rearm_data) != - RTE_ALIGN(offsetof(struct rte_mbuf, rearm_data), 16)); - _mm_store_si128((__m128i *)&rx_pkts[0]->rearm_data, rearm0); - _mm_store_si128((__m128i *)&rx_pkts[1]->rearm_data, rearm1); - _mm_store_si128((__m128i *)&rx_pkts[2]->rearm_data, rearm2); - _mm_store_si128((__m128i *)&rx_pkts[3]->rearm_data, rearm3); + _mm_store_si128((__m128i *)rte_mbuf_rearm_data(rx_pkts[0]), rearm0); + _mm_store_si128((__m128i *)rte_mbuf_rearm_data(rx_pkts[1]), rearm1); + _mm_store_si128((__m128i *)rte_mbuf_rearm_data(rx_pkts[2]), rearm2); + _mm_store_si128((__m128i *)rte_mbuf_rearm_data(rx_pkts[3]), rearm3); } static inline __m128i @@ -412,14 +408,10 @@ rearm3 = _mm_blend_epi16(mbuf_init, _mm_srli_si128(flags, 4), 0x30); /* write the rearm data and the olflags in one write */ - RTE_BUILD_BUG_ON(offsetof(struct rte_mbuf, ol_flags) != - offsetof(struct rte_mbuf, rearm_data) + 8); - RTE_BUILD_BUG_ON(offsetof(struct rte_mbuf, rearm_data) != - RTE_ALIGN(offsetof(struct rte_mbuf, rearm_data), 16)); - _mm_store_si128((__m128i *)&rx_pkts[0]->rearm_data, rearm0); - _mm_store_si128((__m128i *)&rx_pkts[1]->rearm_data, rearm1); - _mm_store_si128((__m128i *)&rx_pkts[2]->rearm_data, rearm2); - _mm_store_si128((__m128i *)&rx_pkts[3]->rearm_data, rearm3); + _mm_store_si128((__m128i *)rte_mbuf_rearm_data(rx_pkts[0]), rearm0); + _mm_store_si128((__m128i *)rte_mbuf_rearm_data(rx_pkts[1]), rearm1); + _mm_store_si128((__m128i *)rte_mbuf_rearm_data(rx_pkts[2]), rearm2); + _mm_store_si128((__m128i *)rte_mbuf_rearm_data(rx_pkts[3]), rearm3); } #define PKTLEN_SHIFT 10 @@ -488,14 +480,7 @@ -rxq->crc_len, /* sub crc on pkt_len */ 0, 0 /* ignore pkt_type field */ ); - /* compile-time check the above crc_adjust layout is correct. - * NOTE: the first field (lowest address) is given last in set_epi16 - * call above. - */ - RTE_BUILD_BUG_ON(offsetof(struct rte_mbuf, pkt_len) != - offsetof(struct rte_mbuf, rx_descriptor_fields1) + 4); - RTE_BUILD_BUG_ON(offsetof(struct rte_mbuf, data_len) != - offsetof(struct rte_mbuf, rx_descriptor_fields1) + 8); + __m128i dd_check, eop_check; /* nb_pkts has to be floor-aligned to IAVF_VPMD_DESCS_PER_LOOP */ @@ -536,18 +521,6 @@ 15, 14, /* octet 15~14, low 16 bits pkt_len */ 0xFF, 0xFF, 0xFF, 0xFF /* pkt_type set as unknown */ ); - /* Compile-time verify the shuffle mask - * NOTE: some field positions already verified above, but duplicated - * here for completeness in case of future modifications. - */ - RTE_BUILD_BUG_ON(offsetof(struct rte_mbuf, pkt_len) != - offsetof(struct rte_mbuf, rx_descriptor_fields1) + 4); - RTE_BUILD_BUG_ON(offsetof(struct rte_mbuf, data_len) != - offsetof(struct rte_mbuf, rx_descriptor_fields1) + 8); - RTE_BUILD_BUG_ON(offsetof(struct rte_mbuf, vlan_tci) != - offsetof(struct rte_mbuf, rx_descriptor_fields1) + 10); - RTE_BUILD_BUG_ON(offsetof(struct rte_mbuf, hash) != - offsetof(struct rte_mbuf, rx_descriptor_fields1) + 12); /* Cache is empty -> need to scan the buffer rings, but first move * the next 'n' mbufs into the cache @@ -651,10 +624,10 @@ /* D.3 copy final 3,4 data to rx_pkts */ _mm_storeu_si128( - (void *)&rx_pkts[pos + 3]->rx_descriptor_fields1, + rte_mbuf_rx_descriptor_fields1(rx_pkts[pos + 3]), pkt_mb4); _mm_storeu_si128( - (void *)&rx_pkts[pos + 2]->rx_descriptor_fields1, + rte_mbuf_rx_descriptor_fields1(rx_pkts[pos + 2]), pkt_mb3); /* D.2 pkt 1,2 remove crc */ @@ -689,9 +662,9 @@ /* D.3 copy final 1,2 data to rx_pkts */ _mm_storeu_si128( - (void *)&rx_pkts[pos + 1]->rx_descriptor_fields1, + rte_mbuf_rx_descriptor_fields1(rx_pkts[pos + 1]), pkt_mb2); - _mm_storeu_si128((void *)&rx_pkts[pos]->rx_descriptor_fields1, + _mm_storeu_si128(rte_mbuf_rx_descriptor_fields1(rx_pkts[pos]), pkt_mb1); desc_to_ptype_v(descs, &rx_pkts[pos], ptype_tbl); /* C.4 calc available number of desc */ @@ -760,16 +733,6 @@ 0x04, 0x0C, 0x00, 0x08); - /** - * compile-time check the above crc_adjust layout is correct. - * NOTE: the first field (lowest address) is given last in set_epi16 - * call above. - */ - RTE_BUILD_BUG_ON(offsetof(struct rte_mbuf, pkt_len) != - offsetof(struct rte_mbuf, rx_descriptor_fields1) + 4); - RTE_BUILD_BUG_ON(offsetof(struct rte_mbuf, data_len) != - offsetof(struct rte_mbuf, rx_descriptor_fields1) + 8); - /* 4 packets DD mask */ const __m128i dd_check = _mm_set_epi64x(0x0000000100000001LL, 0x0000000100000001LL); @@ -818,20 +781,6 @@ #endif - /** - * Compile-time verify the shuffle mask - * NOTE: some field positions already verified above, but duplicated - * here for completeness in case of future modifications. - */ - RTE_BUILD_BUG_ON(offsetof(struct rte_mbuf, pkt_len) != - offsetof(struct rte_mbuf, rx_descriptor_fields1) + 4); - RTE_BUILD_BUG_ON(offsetof(struct rte_mbuf, data_len) != - offsetof(struct rte_mbuf, rx_descriptor_fields1) + 8); - RTE_BUILD_BUG_ON(offsetof(struct rte_mbuf, vlan_tci) != - offsetof(struct rte_mbuf, rx_descriptor_fields1) + 10); - RTE_BUILD_BUG_ON(offsetof(struct rte_mbuf, hash) != - offsetof(struct rte_mbuf, rx_descriptor_fields1) + 12); - /* Cache is empty -> need to scan the buffer rings, but first move * the next 'n' mbufs into the cache */ @@ -1089,10 +1038,10 @@ /* D.3 copy final 3,4 data to rx_pkts */ _mm_storeu_si128 - ((void *)&rx_pkts[pos + 3]->rx_descriptor_fields1, + (rte_mbuf_rx_descriptor_fields1(rx_pkts[pos + 3]), pkt_mb3); _mm_storeu_si128 - ((void *)&rx_pkts[pos + 2]->rx_descriptor_fields1, + (rte_mbuf_rx_descriptor_fields1(rx_pkts[pos + 2]), pkt_mb2); /* C* extract and record EOP bit */ @@ -1116,9 +1065,9 @@ /* D.3 copy final 1,2 data to rx_pkts */ _mm_storeu_si128 - ((void *)&rx_pkts[pos + 1]->rx_descriptor_fields1, + (rte_mbuf_rx_descriptor_fields1(rx_pkts[pos + 1]), pkt_mb1); - _mm_storeu_si128((void *)&rx_pkts[pos]->rx_descriptor_fields1, + _mm_storeu_si128(rte_mbuf_rx_descriptor_fields1(rx_pkts[pos]), pkt_mb0); flex_desc_to_ptype_v(descs, &rx_pkts[pos], ptype_tbl); /* C.4 calc available number of desc */ From patchwork Tue Feb 27 05:41:27 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Tyler Retzlaff X-Patchwork-Id: 137303 X-Patchwork-Delegate: thomas@monjalon.net Return-Path: X-Original-To: patchwork@inbox.dpdk.org Delivered-To: patchwork@inbox.dpdk.org Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id EC74643C03; Tue, 27 Feb 2024 06:43:04 +0100 (CET) Received: from mails.dpdk.org (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id A344442EAB; Tue, 27 Feb 2024 06:42:23 +0100 (CET) Received: from linux.microsoft.com (linux.microsoft.com [13.77.154.182]) by mails.dpdk.org (Postfix) with ESMTP id A49E642E28 for ; Tue, 27 Feb 2024 06:41:42 +0100 (CET) Received: by linux.microsoft.com (Postfix, from userid 1086) id 4F70720B74CB; Mon, 26 Feb 2024 21:41:40 -0800 (PST) DKIM-Filter: OpenDKIM Filter v2.11.0 linux.microsoft.com 4F70720B74CB DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linux.microsoft.com; s=default; t=1709012501; bh=QXUSjsa1O426MmBxQKznXdqfLwRtkEq0lm/TN3DZ2MA=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=L2/Ejl00jHZVG/6Xkj0eKa0TadF/PWa2jZi6vibWDZNnm+EeAGrxEJH5CWEY0nmhR vdOy6E85AFIOl2FvhPM+tgaajP0BpfLTdzS05OBWAklF+Kwj4vNWFGct70wDvVyrqX Nz7uAJIAxk2XNjOExna5wJPWqPJmVM6QW95jWRDs= From: Tyler Retzlaff To: dev@dpdk.org Cc: Ajit Khaparde , Andrew Boyer , Andrew Rybchenko , Bruce Richardson , Chenbo Xia , Chengwen Feng , Dariusz Sosnowski , David Christensen , Hyong Youb Kim , Jerin Jacob , Jie Hai , Jingjing Wu , John Daley , Kevin Laatz , Kiran Kumar K , Konstantin Ananyev , Maciej Czekaj , Matan Azrad , Maxime Coquelin , Nithin Dabilpuram , Ori Kam , Ruifeng Wang , Satha Rao , Somnath Kotur , Suanming Mou , Sunil Kumar Kori , Viacheslav Ovsiienko , Yisen Zhuang , Yuying Zhang , mb@smartsharesystems.com, Tyler Retzlaff Subject: [PATCH v6 11/23] net/ice: use mbuf descriptor accessors Date: Mon, 26 Feb 2024 21:41:27 -0800 Message-Id: <1709012499-12813-12-git-send-email-roretzla@linux.microsoft.com> X-Mailer: git-send-email 1.8.3.1 In-Reply-To: <1709012499-12813-1-git-send-email-roretzla@linux.microsoft.com> References: <1706657173-26166-1-git-send-email-roretzla@linux.microsoft.com> <1709012499-12813-1-git-send-email-roretzla@linux.microsoft.com> X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org RTE_MARKER typedefs are a GCC extension unsupported by MSVC. Use new rte_mbuf_rearm_data and rte_mbuf_rx_descriptor_fields1 accessors that provide a compatible type pointer without using the marker fields. Signed-off-by: Tyler Retzlaff --- drivers/net/ice/ice_rxtx_vec_avx2.c | 36 +++++++--------------------- drivers/net/ice/ice_rxtx_vec_avx512.c | 37 +++++++---------------------- drivers/net/ice/ice_rxtx_vec_common.h | 4 +--- drivers/net/ice/ice_rxtx_vec_sse.c | 44 +++++++---------------------------- 4 files changed, 25 insertions(+), 96 deletions(-) diff --git a/drivers/net/ice/ice_rxtx_vec_avx2.c b/drivers/net/ice/ice_rxtx_vec_avx2.c index 6f6d790..853f99b 100644 --- a/drivers/net/ice/ice_rxtx_vec_avx2.c +++ b/drivers/net/ice/ice_rxtx_vec_avx2.c @@ -114,19 +114,6 @@ 0xFF, 0xFF, /* pkt_type set as unknown */ 0xFF, 0xFF /*pkt_type set as unknown */ ); - /** - * compile-time check the above crc and shuffle layout is correct. - * NOTE: the first field (lowest address) is given last in set_epi - * calls above. - */ - RTE_BUILD_BUG_ON(offsetof(struct rte_mbuf, pkt_len) != - offsetof(struct rte_mbuf, rx_descriptor_fields1) + 4); - RTE_BUILD_BUG_ON(offsetof(struct rte_mbuf, data_len) != - offsetof(struct rte_mbuf, rx_descriptor_fields1) + 8); - RTE_BUILD_BUG_ON(offsetof(struct rte_mbuf, vlan_tci) != - offsetof(struct rte_mbuf, rx_descriptor_fields1) + 10); - RTE_BUILD_BUG_ON(offsetof(struct rte_mbuf, hash) != - offsetof(struct rte_mbuf, rx_descriptor_fields1) + 12); /* Status/Error flag masks */ /** @@ -570,13 +557,6 @@ * add in the previously computed rx_descriptor fields to * make a single 256-bit write per mbuf */ - /* check the structure matches expectations */ - RTE_BUILD_BUG_ON(offsetof(struct rte_mbuf, ol_flags) != - offsetof(struct rte_mbuf, rearm_data) + 8); - RTE_BUILD_BUG_ON(offsetof(struct rte_mbuf, rearm_data) != - RTE_ALIGN(offsetof(struct rte_mbuf, - rearm_data), - 16)); /* build up data and do writes */ __m256i rearm0, rearm1, rearm2, rearm3, rearm4, rearm5, rearm6, rearm7; @@ -596,13 +576,13 @@ rearm2 = _mm256_permute2f128_si256(rearm2, mb2_3, 0x20); rearm0 = _mm256_permute2f128_si256(rearm0, mb0_1, 0x20); /* write to mbuf */ - _mm256_storeu_si256((__m256i *)&rx_pkts[i + 6]->rearm_data, + _mm256_storeu_si256((__m256i *)rte_mbuf_rearm_data(rx_pkts[i + 6]), rearm6); - _mm256_storeu_si256((__m256i *)&rx_pkts[i + 4]->rearm_data, + _mm256_storeu_si256((__m256i *)rte_mbuf_rearm_data(rx_pkts[i + 4]), rearm4); - _mm256_storeu_si256((__m256i *)&rx_pkts[i + 2]->rearm_data, + _mm256_storeu_si256((__m256i *)rte_mbuf_rearm_data(rx_pkts[i + 2]), rearm2); - _mm256_storeu_si256((__m256i *)&rx_pkts[i + 0]->rearm_data, + _mm256_storeu_si256((__m256i *)rte_mbuf_rearm_data(rx_pkts[i + 0]), rearm0); /* repeat for the odd mbufs */ @@ -625,13 +605,13 @@ rearm3 = _mm256_blend_epi32(rearm3, mb2_3, 0xF0); rearm1 = _mm256_blend_epi32(rearm1, mb0_1, 0xF0); /* again write to mbufs */ - _mm256_storeu_si256((__m256i *)&rx_pkts[i + 7]->rearm_data, + _mm256_storeu_si256((__m256i *)rte_mbuf_rearm_data(rx_pkts[i + 7]), rearm7); - _mm256_storeu_si256((__m256i *)&rx_pkts[i + 5]->rearm_data, + _mm256_storeu_si256((__m256i *)rte_mbuf_rearm_data(rx_pkts[i + 5]), rearm5); - _mm256_storeu_si256((__m256i *)&rx_pkts[i + 3]->rearm_data, + _mm256_storeu_si256((__m256i *)rte_mbuf_rearm_data(rx_pkts[i + 3]), rearm3); - _mm256_storeu_si256((__m256i *)&rx_pkts[i + 1]->rearm_data, + _mm256_storeu_si256((__m256i *)rte_mbuf_rearm_data(rx_pkts[i + 1]), rearm1); /* extract and record EOP bit */ diff --git a/drivers/net/ice/ice_rxtx_vec_avx512.c b/drivers/net/ice/ice_rxtx_vec_avx512.c index 04148e8..6d04bf7 100644 --- a/drivers/net/ice/ice_rxtx_vec_avx512.c +++ b/drivers/net/ice/ice_rxtx_vec_avx512.c @@ -100,20 +100,6 @@ 0xFFFFFFFF ); - /** - * compile-time check the above crc and shuffle layout is correct. - * NOTE: the first field (lowest address) is given last in set_epi - * calls above. - */ - RTE_BUILD_BUG_ON(offsetof(struct rte_mbuf, pkt_len) != - offsetof(struct rte_mbuf, rx_descriptor_fields1) + 4); - RTE_BUILD_BUG_ON(offsetof(struct rte_mbuf, data_len) != - offsetof(struct rte_mbuf, rx_descriptor_fields1) + 8); - RTE_BUILD_BUG_ON(offsetof(struct rte_mbuf, vlan_tci) != - offsetof(struct rte_mbuf, rx_descriptor_fields1) + 10); - RTE_BUILD_BUG_ON(offsetof(struct rte_mbuf, hash) != - offsetof(struct rte_mbuf, rx_descriptor_fields1) + 12); - /* following code block is for Rx Checksum Offload */ /* Status/Error flag masks */ /** @@ -568,13 +554,6 @@ * add in the previously computed rx_descriptor fields to * make a single 256-bit write per mbuf */ - /* check the structure matches expectations */ - RTE_BUILD_BUG_ON(offsetof(struct rte_mbuf, ol_flags) != - offsetof(struct rte_mbuf, rearm_data) + 8); - RTE_BUILD_BUG_ON(offsetof(struct rte_mbuf, rearm_data) != - RTE_ALIGN(offsetof(struct rte_mbuf, - rearm_data), - 16)); /* build up data and do writes */ __m256i rearm0, rearm1, rearm2, rearm3, rearm4, rearm5, rearm6, rearm7; @@ -597,13 +576,13 @@ rearm0 = _mm256_permute2f128_si256(rearm0, mb0_1, 0x20); /* write to mbuf */ - _mm256_storeu_si256((__m256i *)&rx_pkts[i + 6]->rearm_data, + _mm256_storeu_si256((__m256i *)rte_mbuf_rearm_data(rx_pkts[i + 6]), rearm6); - _mm256_storeu_si256((__m256i *)&rx_pkts[i + 4]->rearm_data, + _mm256_storeu_si256((__m256i *)rte_mbuf_rearm_data(rx_pkts[i + 4]), rearm4); - _mm256_storeu_si256((__m256i *)&rx_pkts[i + 2]->rearm_data, + _mm256_storeu_si256((__m256i *)rte_mbuf_rearm_data(rx_pkts[i + 2]), rearm2); - _mm256_storeu_si256((__m256i *)&rx_pkts[i + 0]->rearm_data, + _mm256_storeu_si256((__m256i *)rte_mbuf_rearm_data(rx_pkts[i + 0]), rearm0); /* repeat for the odd mbufs */ @@ -627,13 +606,13 @@ rearm3 = _mm256_blend_epi32(rearm3, mb2_3, 0xF0); rearm1 = _mm256_blend_epi32(rearm1, mb0_1, 0xF0); /* again write to mbufs */ - _mm256_storeu_si256((__m256i *)&rx_pkts[i + 7]->rearm_data, + _mm256_storeu_si256((__m256i *)rte_mbuf_rearm_data(rx_pkts[i + 7]), rearm7); - _mm256_storeu_si256((__m256i *)&rx_pkts[i + 5]->rearm_data, + _mm256_storeu_si256((__m256i *)rte_mbuf_rearm_data(rx_pkts[i + 5]), rearm5); - _mm256_storeu_si256((__m256i *)&rx_pkts[i + 3]->rearm_data, + _mm256_storeu_si256((__m256i *)rte_mbuf_rearm_data(rx_pkts[i + 3]), rearm3); - _mm256_storeu_si256((__m256i *)&rx_pkts[i + 1]->rearm_data, + _mm256_storeu_si256((__m256i *)rte_mbuf_rearm_data(rx_pkts[i + 1]), rearm1); /* extract and record EOP bit */ diff --git a/drivers/net/ice/ice_rxtx_vec_common.h b/drivers/net/ice/ice_rxtx_vec_common.h index 4b73465..c284d2d 100644 --- a/drivers/net/ice/ice_rxtx_vec_common.h +++ b/drivers/net/ice/ice_rxtx_vec_common.h @@ -232,7 +232,6 @@ static inline int ice_rxq_vec_setup_default(struct ice_rx_queue *rxq) { - uintptr_t p; struct rte_mbuf mb_def = { .buf_addr = 0 }; /* zeroed mbuf */ mb_def.nb_segs = 1; @@ -242,8 +241,7 @@ /* prevent compiler reordering: rearm_data covers previous fields */ rte_compiler_barrier(); - p = (uintptr_t)&mb_def.rearm_data; - rxq->mbuf_initializer = *(uint64_t *)p; + rxq->mbuf_initializer = *rte_mbuf_rearm_data(&mb_def); return 0; } diff --git a/drivers/net/ice/ice_rxtx_vec_sse.c b/drivers/net/ice/ice_rxtx_vec_sse.c index 9a1b7e3..ed9928b 100644 --- a/drivers/net/ice/ice_rxtx_vec_sse.c +++ b/drivers/net/ice/ice_rxtx_vec_sse.c @@ -267,14 +267,10 @@ rearm3 = _mm_blend_epi16(mbuf_init, _mm_srli_si128(flags, 4), 0x30); /* write the rearm data and the olflags in one write */ - RTE_BUILD_BUG_ON(offsetof(struct rte_mbuf, ol_flags) != - offsetof(struct rte_mbuf, rearm_data) + 8); - RTE_BUILD_BUG_ON(offsetof(struct rte_mbuf, rearm_data) != - RTE_ALIGN(offsetof(struct rte_mbuf, rearm_data), 16)); - _mm_store_si128((__m128i *)&rx_pkts[0]->rearm_data, rearm0); - _mm_store_si128((__m128i *)&rx_pkts[1]->rearm_data, rearm1); - _mm_store_si128((__m128i *)&rx_pkts[2]->rearm_data, rearm2); - _mm_store_si128((__m128i *)&rx_pkts[3]->rearm_data, rearm3); + _mm_store_si128((__m128i *)rte_mbuf_rearm_data(rx_pkts[0]), rearm0); + _mm_store_si128((__m128i *)rte_mbuf_rearm_data(rx_pkts[1]), rearm1); + _mm_store_si128((__m128i *)rte_mbuf_rearm_data(rx_pkts[2]), rearm2); + _mm_store_si128((__m128i *)rte_mbuf_rearm_data(rx_pkts[3]), rearm3); } static inline void @@ -342,16 +338,6 @@ 0x04, 0x0C, 0x00, 0x08); - /** - * compile-time check the above crc_adjust layout is correct. - * NOTE: the first field (lowest address) is given last in set_epi16 - * call above. - */ - RTE_BUILD_BUG_ON(offsetof(struct rte_mbuf, pkt_len) != - offsetof(struct rte_mbuf, rx_descriptor_fields1) + 4); - RTE_BUILD_BUG_ON(offsetof(struct rte_mbuf, data_len) != - offsetof(struct rte_mbuf, rx_descriptor_fields1) + 8); - /* 4 packets DD mask */ const __m128i dd_check = _mm_set_epi64x(0x0000000100000001LL, 0x0000000100000001LL); @@ -382,20 +368,6 @@ rte_cpu_to_le_32(1 << ICE_RX_FLEX_DESC_STATUS0_DD_S))) return 0; - /** - * Compile-time verify the shuffle mask - * NOTE: some field positions already verified above, but duplicated - * here for completeness in case of future modifications. - */ - RTE_BUILD_BUG_ON(offsetof(struct rte_mbuf, pkt_len) != - offsetof(struct rte_mbuf, rx_descriptor_fields1) + 4); - RTE_BUILD_BUG_ON(offsetof(struct rte_mbuf, data_len) != - offsetof(struct rte_mbuf, rx_descriptor_fields1) + 8); - RTE_BUILD_BUG_ON(offsetof(struct rte_mbuf, vlan_tci) != - offsetof(struct rte_mbuf, rx_descriptor_fields1) + 10); - RTE_BUILD_BUG_ON(offsetof(struct rte_mbuf, hash) != - offsetof(struct rte_mbuf, rx_descriptor_fields1) + 12); - /* Cache is empty -> need to scan the buffer rings, but first move * the next 'n' mbufs into the cache */ @@ -542,10 +514,10 @@ /* D.3 copy final 3,4 data to rx_pkts */ _mm_storeu_si128 - ((void *)&rx_pkts[pos + 3]->rx_descriptor_fields1, + (rte_mbuf_rx_descriptor_fields1(rx_pkts[pos + 3]), pkt_mb3); _mm_storeu_si128 - ((void *)&rx_pkts[pos + 2]->rx_descriptor_fields1, + (rte_mbuf_rx_descriptor_fields1(rx_pkts[pos + 2]), pkt_mb2); /* C* extract and record EOP bit */ @@ -569,9 +541,9 @@ /* D.3 copy final 1,2 data to rx_pkts */ _mm_storeu_si128 - ((void *)&rx_pkts[pos + 1]->rx_descriptor_fields1, + (rte_mbuf_rx_descriptor_fields1(rx_pkts[pos + 1]), pkt_mb1); - _mm_storeu_si128((void *)&rx_pkts[pos]->rx_descriptor_fields1, + _mm_storeu_si128(rte_mbuf_rx_descriptor_fields1(rx_pkts[pos]), pkt_mb0); ice_rx_desc_to_ptype_v(descs, &rx_pkts[pos], ptype_tbl); /* C.4 calc available number of desc */ From patchwork Tue Feb 27 05:41:28 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Tyler Retzlaff X-Patchwork-Id: 137301 X-Patchwork-Delegate: thomas@monjalon.net Return-Path: X-Original-To: patchwork@inbox.dpdk.org Delivered-To: patchwork@inbox.dpdk.org Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id 1CF7043C03; Tue, 27 Feb 2024 06:42:52 +0100 (CET) Received: from mails.dpdk.org (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id 0170842E8F; Tue, 27 Feb 2024 06:42:21 +0100 (CET) Received: from linux.microsoft.com (linux.microsoft.com [13.77.154.182]) by mails.dpdk.org (Postfix) with ESMTP id A48BE402B2 for ; Tue, 27 Feb 2024 06:41:42 +0100 (CET) Received: by linux.microsoft.com (Postfix, from userid 1086) id 5DDC320B74CC; Mon, 26 Feb 2024 21:41:40 -0800 (PST) DKIM-Filter: OpenDKIM Filter v2.11.0 linux.microsoft.com 5DDC320B74CC DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linux.microsoft.com; s=default; t=1709012501; bh=mV+JKGOyhBQnNtti08RCsMbmQ/psfivyOCNTk9CbK9s=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=DTabpdJwtRFqWM7lAymht65IpqrJ6AIegbApLt7F39xBQX8DgGL0hU5uicjvsLE42 5IPSg8ZXvyBYvGWo4JZLr1rRyFzOF0KzEUcwkgnB/icpWZfuaxuUoTGn9MOQKOotuP fdBXVbB4UkPANBwOkxAeo5t3LBEM7vMw0ErA5hfo= From: Tyler Retzlaff To: dev@dpdk.org Cc: Ajit Khaparde , Andrew Boyer , Andrew Rybchenko , Bruce Richardson , Chenbo Xia , Chengwen Feng , Dariusz Sosnowski , David Christensen , Hyong Youb Kim , Jerin Jacob , Jie Hai , Jingjing Wu , John Daley , Kevin Laatz , Kiran Kumar K , Konstantin Ananyev , Maciej Czekaj , Matan Azrad , Maxime Coquelin , Nithin Dabilpuram , Ori Kam , Ruifeng Wang , Satha Rao , Somnath Kotur , Suanming Mou , Sunil Kumar Kori , Viacheslav Ovsiienko , Yisen Zhuang , Yuying Zhang , mb@smartsharesystems.com, Tyler Retzlaff Subject: [PATCH v6 12/23] net/ionic: use mbuf descriptor accessors Date: Mon, 26 Feb 2024 21:41:28 -0800 Message-Id: <1709012499-12813-13-git-send-email-roretzla@linux.microsoft.com> X-Mailer: git-send-email 1.8.3.1 In-Reply-To: <1709012499-12813-1-git-send-email-roretzla@linux.microsoft.com> References: <1706657173-26166-1-git-send-email-roretzla@linux.microsoft.com> <1709012499-12813-1-git-send-email-roretzla@linux.microsoft.com> X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org RTE_MARKER typedefs are a GCC extension unsupported by MSVC. Use new rte_mbuf_rearm_data and rte_mbuf_rx_descriptor_fields1 accessors that provide a compatible type pointer without using the marker fields. Signed-off-by: Tyler Retzlaff --- drivers/net/ionic/ionic_lif.c | 6 ++---- drivers/net/ionic/ionic_rxtx_sg.c | 4 ++-- drivers/net/ionic/ionic_rxtx_simple.c | 2 +- 3 files changed, 5 insertions(+), 7 deletions(-) diff --git a/drivers/net/ionic/ionic_lif.c b/drivers/net/ionic/ionic_lif.c index 93a1011..4a5aa1e 100644 --- a/drivers/net/ionic/ionic_lif.c +++ b/drivers/net/ionic/ionic_lif.c @@ -744,8 +744,7 @@ rte_compiler_barrier(); - RTE_BUILD_BUG_ON(sizeof(rxm.rearm_data[0]) != sizeof(uint64_t)); - return rxm.rearm_data[0]; + return *rte_mbuf_rearm_data(&rxm); } static uint64_t @@ -762,8 +761,7 @@ rte_compiler_barrier(); - RTE_BUILD_BUG_ON(sizeof(rxm.rearm_data[0]) != sizeof(uint64_t)); - return rxm.rearm_data[0]; + return *rte_mbuf_rearm_data(&rxm); } int diff --git a/drivers/net/ionic/ionic_rxtx_sg.c b/drivers/net/ionic/ionic_rxtx_sg.c index 92e1d6e..4a6f3c2 100644 --- a/drivers/net/ionic/ionic_rxtx_sg.c +++ b/drivers/net/ionic/ionic_rxtx_sg.c @@ -286,7 +286,7 @@ info[0] = NULL; /* Set the mbuf metadata based on the cq entry */ - rxm->rearm_data[0] = rxq->rearm_data; + *rte_mbuf_rearm_data(rxm) = rxq->rearm_data; rxm->pkt_len = cq_desc_len; rxm->data_len = RTE_MIN(rxq->hdr_seg_size, cq_desc_len); left = cq_desc_len - rxm->data_len; @@ -299,7 +299,7 @@ info[i] = NULL; /* Set the chained mbuf metadata */ - rxm_seg->rearm_data[0] = rxq->rearm_seg_data; + *rte_mbuf_rearm_data(rxm_seg) = rxq->rearm_seg_data; rxm_seg->data_len = RTE_MIN(rxq->seg_size, left); left -= rxm_seg->data_len; diff --git a/drivers/net/ionic/ionic_rxtx_simple.c b/drivers/net/ionic/ionic_rxtx_simple.c index f12f66f..02528e8 100644 --- a/drivers/net/ionic/ionic_rxtx_simple.c +++ b/drivers/net/ionic/ionic_rxtx_simple.c @@ -257,7 +257,7 @@ info[0] = NULL; /* Set the mbuf metadata based on the cq entry */ - rxm->rearm_data[0] = rxq->rearm_data; + *rte_mbuf_rearm_data(rxm) = rxq->rearm_data; rxm->pkt_len = cq_desc_len; rxm->data_len = cq_desc_len; From patchwork Tue Feb 27 05:41:29 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Tyler Retzlaff X-Patchwork-Id: 137308 X-Patchwork-Delegate: thomas@monjalon.net Return-Path: X-Original-To: patchwork@inbox.dpdk.org Delivered-To: patchwork@inbox.dpdk.org Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id AEB1D43C03; Tue, 27 Feb 2024 06:43:37 +0100 (CET) Received: from mails.dpdk.org (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id 53AA842ECF; Tue, 27 Feb 2024 06:42:30 +0100 (CET) Received: from linux.microsoft.com (linux.microsoft.com [13.77.154.182]) by mails.dpdk.org (Postfix) with ESMTP id BBE7142E3E for ; Tue, 27 Feb 2024 06:41:42 +0100 (CET) Received: by linux.microsoft.com (Postfix, from userid 1086) id 6C99D20B74CD; Mon, 26 Feb 2024 21:41:40 -0800 (PST) DKIM-Filter: OpenDKIM Filter v2.11.0 linux.microsoft.com 6C99D20B74CD DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linux.microsoft.com; s=default; t=1709012501; bh=v8P9hR2ueLoRiGEJMITmdEn1Ro+xEEstShCqsrwD72E=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=HjO8EHFnS8It6JdCuRRf4aS3TKS/s1tiK+ADwd8LMt8SnfPU8oaYjHuGPHpygMxGG EHaBVtv11y4jogWPVFxSGpjemU3hUfgLR960jxlwsODL0nKNnKv/81b9DhpaXK0NhM rm1jv3lANldE8mYtFMchC4FkiE5Rx2coyWD01tvI= From: Tyler Retzlaff To: dev@dpdk.org Cc: Ajit Khaparde , Andrew Boyer , Andrew Rybchenko , Bruce Richardson , Chenbo Xia , Chengwen Feng , Dariusz Sosnowski , David Christensen , Hyong Youb Kim , Jerin Jacob , Jie Hai , Jingjing Wu , John Daley , Kevin Laatz , Kiran Kumar K , Konstantin Ananyev , Maciej Czekaj , Matan Azrad , Maxime Coquelin , Nithin Dabilpuram , Ori Kam , Ruifeng Wang , Satha Rao , Somnath Kotur , Suanming Mou , Sunil Kumar Kori , Viacheslav Ovsiienko , Yisen Zhuang , Yuying Zhang , mb@smartsharesystems.com, Tyler Retzlaff Subject: [PATCH v6 13/23] net/ixgbe: use mbuf descriptor accessors Date: Mon, 26 Feb 2024 21:41:29 -0800 Message-Id: <1709012499-12813-14-git-send-email-roretzla@linux.microsoft.com> X-Mailer: git-send-email 1.8.3.1 In-Reply-To: <1709012499-12813-1-git-send-email-roretzla@linux.microsoft.com> References: <1706657173-26166-1-git-send-email-roretzla@linux.microsoft.com> <1709012499-12813-1-git-send-email-roretzla@linux.microsoft.com> X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org RTE_MARKER typedefs are a GCC extension unsupported by MSVC. Use new rte_mbuf_rearm_data and rte_mbuf_rx_descriptor_fields1 accessors that provide a compatible type pointer without using the marker fields. Signed-off-by: Tyler Retzlaff --- drivers/net/ixgbe/ixgbe_rxtx_vec_common.h | 4 +-- drivers/net/ixgbe/ixgbe_rxtx_vec_neon.c | 12 ++++---- drivers/net/ixgbe/ixgbe_rxtx_vec_sse.c | 51 ++++++++----------------------- 3 files changed, 20 insertions(+), 47 deletions(-) diff --git a/drivers/net/ixgbe/ixgbe_rxtx_vec_common.h b/drivers/net/ixgbe/ixgbe_rxtx_vec_common.h index a4d9ec9..78ab168 100644 --- a/drivers/net/ixgbe/ixgbe_rxtx_vec_common.h +++ b/drivers/net/ixgbe/ixgbe_rxtx_vec_common.h @@ -244,7 +244,6 @@ static inline int ixgbe_rxq_vec_setup_default(struct ixgbe_rx_queue *rxq) { - uintptr_t p; struct rte_mbuf mb_def = { .buf_addr = 0 }; /* zeroed mbuf */ mb_def.nb_segs = 1; @@ -254,8 +253,7 @@ /* prevent compiler reordering: rearm_data covers previous fields */ rte_compiler_barrier(); - p = (uintptr_t)&mb_def.rearm_data; - rxq->mbuf_initializer = *(uint64_t *)p; + rxq->mbuf_initializer = *rte_mbuf_rearm_data(&mb_def); return 0; } diff --git a/drivers/net/ixgbe/ixgbe_rxtx_vec_neon.c b/drivers/net/ixgbe/ixgbe_rxtx_vec_neon.c index 952b032..2e1d903 100644 --- a/drivers/net/ixgbe/ixgbe_rxtx_vec_neon.c +++ b/drivers/net/ixgbe/ixgbe_rxtx_vec_neon.c @@ -56,13 +56,13 @@ * Flush mbuf with pkt template. * Data to be rearmed is 6 bytes long. */ - vst1_u8((uint8_t *)&mb0->rearm_data, p); + vst1_u8((uint8_t *)rte_mbuf_rearm_data(mb0), p); paddr = mb0->buf_iova + RTE_PKTMBUF_HEADROOM; dma_addr0 = vsetq_lane_u64(paddr, zero, 0); /* flush desc with pa dma_addr */ vst1q_u64((uint64_t *)&rxdp++->read, dma_addr0); - vst1_u8((uint8_t *)&mb1->rearm_data, p); + vst1_u8((uint8_t *)rte_mbuf_rearm_data(mb1), p); paddr = mb1->buf_iova + RTE_PKTMBUF_HEADROOM; dma_addr1 = vsetq_lane_u64(paddr, zero, 0); vst1q_u64((uint64_t *)&rxdp++->read, dma_addr1); @@ -411,9 +411,9 @@ pkt_mb3 = vreinterpretq_u8_u16(tmp); /* D.3 copy final 3,4 data to rx_pkts */ - vst1q_u8((void *)&rx_pkts[pos + 3]->rx_descriptor_fields1, + vst1q_u8(rte_mbuf_rx_descriptor_fields1(rx_pkts[pos + 3]), pkt_mb4); - vst1q_u8((void *)&rx_pkts[pos + 2]->rx_descriptor_fields1, + vst1q_u8(rte_mbuf_rx_descriptor_fields1(rx_pkts[pos + 2]), pkt_mb3); /* D.2 pkt 1,2 set in_port/nb_seg and remove crc */ @@ -441,9 +441,9 @@ rte_prefetch_non_temporal(rxdp + RTE_IXGBE_DESCS_PER_LOOP); /* D.3 copy final 1,2 data to rx_pkts */ - vst1q_u8((uint8_t *)&rx_pkts[pos + 1]->rx_descriptor_fields1, + vst1q_u8((uint8_t *)rte_mbuf_rx_descriptor_fields1(rx_pkts[pos + 1]), pkt_mb2); - vst1q_u8((uint8_t *)&rx_pkts[pos]->rx_descriptor_fields1, + vst1q_u8((uint8_t *)rte_mbuf_rx_descriptor_fields1(rx_pkts[pos]), pkt_mb1); desc_to_ptype_v(descs, rxq->pkt_type_mask, &rx_pkts[pos]); diff --git a/drivers/net/ixgbe/ixgbe_rxtx_vec_sse.c b/drivers/net/ixgbe/ixgbe_rxtx_vec_sse.c index f60808d..e03daae 100644 --- a/drivers/net/ixgbe/ixgbe_rxtx_vec_sse.c +++ b/drivers/net/ixgbe/ixgbe_rxtx_vec_sse.c @@ -98,10 +98,10 @@ desc_to_olflags_v_ipsec(__m128i descs[4], struct rte_mbuf **rx_pkts) { __m128i sterr, rearm, tmp_e, tmp_p; - uint32_t *rearm0 = (uint32_t *)rx_pkts[0]->rearm_data + 2; - uint32_t *rearm1 = (uint32_t *)rx_pkts[1]->rearm_data + 2; - uint32_t *rearm2 = (uint32_t *)rx_pkts[2]->rearm_data + 2; - uint32_t *rearm3 = (uint32_t *)rx_pkts[3]->rearm_data + 2; + uint32_t *rearm0 = (uint32_t *)rte_mbuf_rearm_data(rx_pkts[0]) + 2; + uint32_t *rearm1 = (uint32_t *)rte_mbuf_rearm_data(rx_pkts[1]) + 2; + uint32_t *rearm2 = (uint32_t *)rte_mbuf_rearm_data(rx_pkts[2]) + 2; + uint32_t *rearm3 = (uint32_t *)rte_mbuf_rearm_data(rx_pkts[3]) + 2; const __m128i ipsec_sterr_msk = _mm_set1_epi32(IXGBE_RXDADV_IPSEC_STATUS_SECP | IXGBE_RXDADV_IPSEC_ERROR_AUTH_FAILED); @@ -251,14 +251,10 @@ rearm3 = _mm_blend_epi16(mbuf_init, _mm_slli_si128(vtag1, 2), 0x10); /* write the rearm data and the olflags in one write */ - RTE_BUILD_BUG_ON(offsetof(struct rte_mbuf, ol_flags) != - offsetof(struct rte_mbuf, rearm_data) + 8); - RTE_BUILD_BUG_ON(offsetof(struct rte_mbuf, rearm_data) != - RTE_ALIGN(offsetof(struct rte_mbuf, rearm_data), 16)); - _mm_store_si128((__m128i *)&rx_pkts[0]->rearm_data, rearm0); - _mm_store_si128((__m128i *)&rx_pkts[1]->rearm_data, rearm1); - _mm_store_si128((__m128i *)&rx_pkts[2]->rearm_data, rearm2); - _mm_store_si128((__m128i *)&rx_pkts[3]->rearm_data, rearm3); + _mm_store_si128((__m128i *)rte_mbuf_rearm_data(rx_pkts[0]), rearm0); + _mm_store_si128((__m128i *)rte_mbuf_rearm_data(rx_pkts[1]), rearm1); + _mm_store_si128((__m128i *)rte_mbuf_rearm_data(rx_pkts[2]), rearm2); + _mm_store_si128((__m128i *)rte_mbuf_rearm_data(rx_pkts[3]), rearm3); } static inline uint32_t get_packet_type(int index, @@ -350,15 +346,7 @@ static inline uint32_t get_packet_type(int index, -rxq->crc_len, /* sub crc on pkt_len */ 0, 0 /* ignore pkt_type field */ ); - /* - * compile-time check the above crc_adjust layout is correct. - * NOTE: the first field (lowest address) is given last in set_epi16 - * call above. - */ - RTE_BUILD_BUG_ON(offsetof(struct rte_mbuf, pkt_len) != - offsetof(struct rte_mbuf, rx_descriptor_fields1) + 4); - RTE_BUILD_BUG_ON(offsetof(struct rte_mbuf, data_len) != - offsetof(struct rte_mbuf, rx_descriptor_fields1) + 8); + __m128i dd_check, eop_check; __m128i mbuf_init; uint8_t vlan_flags; @@ -417,19 +405,6 @@ static inline uint32_t get_packet_type(int index, 0xFF, 0xFF, /* skip 32 bit pkt_type */ 0xFF, 0xFF ); - /* - * Compile-time verify the shuffle mask - * NOTE: some field positions already verified above, but duplicated - * here for completeness in case of future modifications. - */ - RTE_BUILD_BUG_ON(offsetof(struct rte_mbuf, pkt_len) != - offsetof(struct rte_mbuf, rx_descriptor_fields1) + 4); - RTE_BUILD_BUG_ON(offsetof(struct rte_mbuf, data_len) != - offsetof(struct rte_mbuf, rx_descriptor_fields1) + 8); - RTE_BUILD_BUG_ON(offsetof(struct rte_mbuf, vlan_tci) != - offsetof(struct rte_mbuf, rx_descriptor_fields1) + 10); - RTE_BUILD_BUG_ON(offsetof(struct rte_mbuf, hash) != - offsetof(struct rte_mbuf, rx_descriptor_fields1) + 12); mbuf_init = _mm_set_epi64x(0, rxq->mbuf_initializer); @@ -530,9 +505,9 @@ static inline uint32_t get_packet_type(int index, staterr = _mm_unpacklo_epi32(sterr_tmp1, sterr_tmp2); /* D.3 copy final 3,4 data to rx_pkts */ - _mm_storeu_si128((void *)&rx_pkts[pos+3]->rx_descriptor_fields1, + _mm_storeu_si128(rte_mbuf_rx_descriptor_fields1(rx_pkts[pos + 3]), pkt_mb4); - _mm_storeu_si128((void *)&rx_pkts[pos+2]->rx_descriptor_fields1, + _mm_storeu_si128(rte_mbuf_rx_descriptor_fields1(rx_pkts[pos + 2]), pkt_mb3); /* D.2 pkt 1,2 set in_port/nb_seg and remove crc */ @@ -566,9 +541,9 @@ static inline uint32_t get_packet_type(int index, staterr = _mm_packs_epi32(staterr, zero); /* D.3 copy final 1,2 data to rx_pkts */ - _mm_storeu_si128((void *)&rx_pkts[pos+1]->rx_descriptor_fields1, + _mm_storeu_si128(rte_mbuf_rx_descriptor_fields1(rx_pkts[pos + 1]), pkt_mb2); - _mm_storeu_si128((void *)&rx_pkts[pos]->rx_descriptor_fields1, + _mm_storeu_si128(rte_mbuf_rx_descriptor_fields1(rx_pkts[pos]), pkt_mb1); desc_to_ptype_v(descs, rxq->pkt_type_mask, &rx_pkts[pos]); From patchwork Tue Feb 27 05:41:30 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Tyler Retzlaff X-Patchwork-Id: 137307 X-Patchwork-Delegate: thomas@monjalon.net Return-Path: X-Original-To: patchwork@inbox.dpdk.org Delivered-To: patchwork@inbox.dpdk.org Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id E481043C03; Tue, 27 Feb 2024 06:43:30 +0100 (CET) Received: from mails.dpdk.org (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id 1E77E42EC6; Tue, 27 Feb 2024 06:42:29 +0100 (CET) Received: from linux.microsoft.com (linux.microsoft.com [13.77.154.182]) by mails.dpdk.org (Postfix) with ESMTP id 0381B42E46 for ; Tue, 27 Feb 2024 06:41:42 +0100 (CET) Received: by linux.microsoft.com (Postfix, from userid 1086) id 7B10120B74CE; Mon, 26 Feb 2024 21:41:40 -0800 (PST) DKIM-Filter: OpenDKIM Filter v2.11.0 linux.microsoft.com 7B10120B74CE DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linux.microsoft.com; s=default; t=1709012501; bh=b2gz+pp38CCrlc4dzML+CQBDUe/9prUU0+s1hTbGtl0=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=MjhClYTz0LHK+dTVi8TYTUYTTTEtFjGg9RKZ1Fc02HJ4JvHqmjjLHzV5jExUdU2Qt wAlbSEK6Py3MGt1kefgNNICgTDf8eC5H7rpn3UF+buoCcHWp1tqqTN/0g8zhZrs3aK cuaD0NgTXU/VM7sSXnf67WNvVfpszrhUs9OabdK8= From: Tyler Retzlaff To: dev@dpdk.org Cc: Ajit Khaparde , Andrew Boyer , Andrew Rybchenko , Bruce Richardson , Chenbo Xia , Chengwen Feng , Dariusz Sosnowski , David Christensen , Hyong Youb Kim , Jerin Jacob , Jie Hai , Jingjing Wu , John Daley , Kevin Laatz , Kiran Kumar K , Konstantin Ananyev , Maciej Czekaj , Matan Azrad , Maxime Coquelin , Nithin Dabilpuram , Ori Kam , Ruifeng Wang , Satha Rao , Somnath Kotur , Suanming Mou , Sunil Kumar Kori , Viacheslav Ovsiienko , Yisen Zhuang , Yuying Zhang , mb@smartsharesystems.com, Tyler Retzlaff Subject: [PATCH v6 14/23] net/mlx5: use mbuf descriptor accessors Date: Mon, 26 Feb 2024 21:41:30 -0800 Message-Id: <1709012499-12813-15-git-send-email-roretzla@linux.microsoft.com> X-Mailer: git-send-email 1.8.3.1 In-Reply-To: <1709012499-12813-1-git-send-email-roretzla@linux.microsoft.com> References: <1706657173-26166-1-git-send-email-roretzla@linux.microsoft.com> <1709012499-12813-1-git-send-email-roretzla@linux.microsoft.com> X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org RTE_MARKER typedefs are a GCC extension unsupported by MSVC. Use new rte_mbuf_rearm_data and rte_mbuf_rx_descriptor_fields1 accessors that provide a compatible type pointer without using the marker fields. Signed-off-by: Tyler Retzlaff --- drivers/net/mlx5/mlx5_rxq.c | 2 +- drivers/net/mlx5/mlx5_rxtx_vec.h | 19 ------------------- drivers/net/mlx5/mlx5_rxtx_vec_altivec.h | 28 ++++++++++++++-------------- drivers/net/mlx5/mlx5_rxtx_vec_neon.h | 20 ++++++++++---------- drivers/net/mlx5/mlx5_rxtx_vec_sse.h | 28 ++++++++++++++-------------- 5 files changed, 39 insertions(+), 58 deletions(-) diff --git a/drivers/net/mlx5/mlx5_rxq.c b/drivers/net/mlx5/mlx5_rxq.c index ca2eeed..b854418 100644 --- a/drivers/net/mlx5/mlx5_rxq.c +++ b/drivers/net/mlx5/mlx5_rxq.c @@ -200,7 +200,7 @@ */ rte_compiler_barrier(); rxq->mbuf_initializer = - *(rte_xmm_t *)&mbuf_init->rearm_data; + *(rte_xmm_t *)rte_mbuf_rearm_data(mbuf_init); /* Padding with a fake mbuf for vectorized Rx. */ for (j = 0; j < MLX5_VPMD_DESCS_PER_LOOP; ++j) (*rxq->elts)[elts_n + j] = &rxq->fake_mbuf; diff --git a/drivers/net/mlx5/mlx5_rxtx_vec.h b/drivers/net/mlx5/mlx5_rxtx_vec.h index 77c3f4e..43518c4 100644 --- a/drivers/net/mlx5/mlx5_rxtx_vec.h +++ b/drivers/net/mlx5/mlx5_rxtx_vec.h @@ -29,25 +29,6 @@ #define S_ASSERT_MLX5_CQE(s) \ static_assert(s, "A field of struct mlx5_cqe is changed") -/* rxq_cq_decompress_v() */ -S_ASSERT_RTE_MBUF(offsetof(struct rte_mbuf, pkt_len) == - offsetof(struct rte_mbuf, rx_descriptor_fields1) + 4); -S_ASSERT_RTE_MBUF(offsetof(struct rte_mbuf, data_len) == - offsetof(struct rte_mbuf, rx_descriptor_fields1) + 8); -S_ASSERT_RTE_MBUF(offsetof(struct rte_mbuf, hash) == - offsetof(struct rte_mbuf, rx_descriptor_fields1) + 12); - -/* rxq_cq_to_ptype_oflags_v() */ -S_ASSERT_RTE_MBUF(offsetof(struct rte_mbuf, ol_flags) == - offsetof(struct rte_mbuf, rearm_data) + 8); -S_ASSERT_RTE_MBUF(offsetof(struct rte_mbuf, rearm_data) == - RTE_ALIGN(offsetof(struct rte_mbuf, rearm_data), 16)); - -/* rxq_burst_v() */ -S_ASSERT_RTE_MBUF(offsetof(struct rte_mbuf, pkt_len) == - offsetof(struct rte_mbuf, rx_descriptor_fields1) + 4); -S_ASSERT_RTE_MBUF(offsetof(struct rte_mbuf, data_len) == - offsetof(struct rte_mbuf, rx_descriptor_fields1) + 8); #if (RTE_CACHE_LINE_SIZE == 128) S_ASSERT_MLX5_CQE(offsetof(struct mlx5_cqe, pkt_info) == 64); #else diff --git a/drivers/net/mlx5/mlx5_rxtx_vec_altivec.h b/drivers/net/mlx5/mlx5_rxtx_vec_altivec.h index cccfa7f..9349c21 100644 --- a/drivers/net/mlx5/mlx5_rxtx_vec_altivec.h +++ b/drivers/net/mlx5/mlx5_rxtx_vec_altivec.h @@ -101,10 +101,10 @@ uint16_t pkts_n = mcqe_n; const __vector unsigned char rearm = (__vector unsigned char)vec_vsx_ld(0, - (signed int const *)&t_pkt->rearm_data); + (signed int const *)rte_mbuf_rearm_data(t_pkt)); const __vector unsigned char rxdf = (__vector unsigned char)vec_vsx_ld(0, - (signed int const *)&t_pkt->rx_descriptor_fields1); + (signed int const *)rte_mbuf_rx_descriptor_fields1(t_pkt)); const __vector unsigned char crc_adj = (__vector unsigned char)(__vector unsigned short){ 0, 0, rxq->crc_present * RTE_ETHER_CRC_LEN, 0, @@ -173,9 +173,9 @@ /* B.1 store rearm data to mbuf. */ *(__vector unsigned char *) - &elts[pos]->rearm_data = rearm; + rte_mbuf_rearm_data(elts[pos]) = rearm; *(__vector unsigned char *) - &elts[pos + 1]->rearm_data = rearm; + rte_mbuf_rearm_data(elts[pos + 1]) = rearm; /* C.1 combine data from mCQEs with rx_descriptor_fields1. */ rxdf1 = vec_perm(mcqe1, zero, shuf_mask1); @@ -195,15 +195,15 @@ /* D.1 store rx_descriptor_fields1. */ *(__vector unsigned char *) - &elts[pos]->rx_descriptor_fields1 = rxdf1; + rte_mbuf_rx_descriptor_fields1(elts[pos]) = rxdf1; *(__vector unsigned char *) - &elts[pos + 1]->rx_descriptor_fields1 = rxdf2; + rte_mbuf_rx_descriptor_fields1(elts[pos + 1]) = rxdf2; /* B.1 store rearm data to mbuf. */ *(__vector unsigned char *) - &elts[pos + 2]->rearm_data = rearm; + rte_mbuf_rearm_data(elts[pos + 2]) = rearm; *(__vector unsigned char *) - &elts[pos + 3]->rearm_data = rearm; + rte_mbuf_rearm_data(elts[pos + 3]) = rearm; /* C.1 combine data from mCQEs with rx_descriptor_fields1. */ rxdf1 = vec_perm(mcqe2, zero, shuf_mask1); @@ -223,9 +223,9 @@ /* D.1 store rx_descriptor_fields1. */ *(__vector unsigned char *) - &elts[pos + 2]->rx_descriptor_fields1 = rxdf1; + rte_mbuf_rx_descriptor_fields1(elts[pos + 2]) = rxdf1; *(__vector unsigned char *) - &elts[pos + 3]->rx_descriptor_fields1 = rxdf2; + rte_mbuf_rx_descriptor_fields1(elts[pos + 3]) = rxdf2; #ifdef MLX5_PMD_SOFT_COUNTERS invalid_mask = (__vector unsigned char)(__vector unsigned long){ @@ -769,13 +769,13 @@ /* Write 8B rearm_data and 8B ol_flags. */ vec_vsx_st(rearm0, 0, - (__vector unsigned char *)&pkts[0]->rearm_data); + (__vector unsigned char *)rte_mbuf_rearm_data(pkts[0])); vec_vsx_st(rearm1, 0, - (__vector unsigned char *)&pkts[1]->rearm_data); + (__vector unsigned char *)rte_mbuf_rearm_data(pkts[1])); vec_vsx_st(rearm2, 0, - (__vector unsigned char *)&pkts[2]->rearm_data); + (__vector unsigned char *)rte_mbuf_rearm_data(pkts[2])); vec_vsx_st(rearm3, 0, - (__vector unsigned char *)&pkts[3]->rearm_data); + (__vector unsigned char *)rte_mbuf_rearm_data(pkts[3])); } /** diff --git a/drivers/net/mlx5/mlx5_rxtx_vec_neon.h b/drivers/net/mlx5/mlx5_rxtx_vec_neon.h index 3ed6881..97ea620 100644 --- a/drivers/net/mlx5/mlx5_rxtx_vec_neon.h +++ b/drivers/net/mlx5/mlx5_rxtx_vec_neon.h @@ -99,7 +99,7 @@ t_pkt->data_len + (rxq->crc_present * RTE_ETHER_CRC_LEN); uint16_t pkts_n = mcqe_n; const uint64x2_t rearm = - vld1q_u64((void *)&t_pkt->rearm_data); + vld1q_u64((void *)rte_mbuf_rearm_data(t_pkt)); const uint32x4_t rxdf_mask = { 0xffffffff, /* packet_type */ 0, /* skip pkt_len */ @@ -107,7 +107,7 @@ 0, /* skip hash.rss */ }; const uint8x16_t rxdf = - vandq_u8(vld1q_u8((void *)&t_pkt->rx_descriptor_fields1), + vandq_u8(vld1q_u8(rte_mbuf_rx_descriptor_fields1(t_pkt)), vreinterpretq_u8_u32(rxdf_mask)); const uint16x8_t crc_adj = { 0, 0, @@ -140,10 +140,10 @@ rte_prefetch0((void *)(cq + mcqe_n)); for (pos = 0; pos < mcqe_n; ) { uint8_t *p = (void *)&mcq[pos % 8]; - uint8_t *e0 = (void *)&elts[pos]->rearm_data; - uint8_t *e1 = (void *)&elts[pos + 1]->rearm_data; - uint8_t *e2 = (void *)&elts[pos + 2]->rearm_data; - uint8_t *e3 = (void *)&elts[pos + 3]->rearm_data; + uint8_t *e0 = (void *)rte_mbuf_rearm_data(elts[pos]); + uint8_t *e1 = (void *)rte_mbuf_rearm_data(elts[pos + 1]); + uint8_t *e2 = (void *)rte_mbuf_rearm_data(elts[pos + 2]); + uint8_t *e3 = (void *)rte_mbuf_rearm_data(elts[pos + 3]); uint16x4_t byte_cnt; #ifdef MLX5_PMD_SOFT_COUNTERS uint16x4_t invalid_mask = @@ -513,10 +513,10 @@ (vgetq_lane_u32(ol_flags, 0), vreinterpretq_u32_u64(mbuf_init), 2)); - vst1q_u64((void *)&pkts[0]->rearm_data, rearm0); - vst1q_u64((void *)&pkts[1]->rearm_data, rearm1); - vst1q_u64((void *)&pkts[2]->rearm_data, rearm2); - vst1q_u64((void *)&pkts[3]->rearm_data, rearm3); + vst1q_u64((void *)rte_mbuf_rearm_data(pkts[0]), rearm0); + vst1q_u64((void *)rte_mbuf_rearm_data(pkts[1]), rearm1); + vst1q_u64((void *)rte_mbuf_rearm_data(pkts[2]), rearm2); + vst1q_u64((void *)rte_mbuf_rearm_data(pkts[3]), rearm3); } /** diff --git a/drivers/net/mlx5/mlx5_rxtx_vec_sse.h b/drivers/net/mlx5/mlx5_rxtx_vec_sse.h index 2bdd1f6..088ce37 100644 --- a/drivers/net/mlx5/mlx5_rxtx_vec_sse.h +++ b/drivers/net/mlx5/mlx5_rxtx_vec_sse.h @@ -98,9 +98,9 @@ t_pkt->data_len + (rxq->crc_present * RTE_ETHER_CRC_LEN); uint16_t pkts_n = mcqe_n; const __m128i rearm = - _mm_loadu_si128((__m128i *)&t_pkt->rearm_data); + _mm_loadu_si128((__m128i *)rte_mbuf_rearm_data(t_pkt)); const __m128i rxdf = - _mm_loadu_si128((__m128i *)&t_pkt->rx_descriptor_fields1); + _mm_loadu_si128((__m128i *)rte_mbuf_rx_descriptor_fields1(t_pkt)); const __m128i crc_adj = _mm_set_epi16(0, 0, 0, rxq->crc_present * RTE_ETHER_CRC_LEN, @@ -145,8 +145,8 @@ mcqe1 = _mm_loadu_si128((__m128i *)&mcq[pos % 8]); mcqe2 = _mm_loadu_si128((__m128i *)&mcq[pos % 8 + 2]); /* B.1 store rearm data to mbuf. */ - _mm_storeu_si128((__m128i *)&elts[pos]->rearm_data, rearm); - _mm_storeu_si128((__m128i *)&elts[pos + 1]->rearm_data, rearm); + _mm_storeu_si128((__m128i *)rte_mbuf_rearm_data(elts[pos]), rearm); + _mm_storeu_si128((__m128i *)rte_mbuf_rearm_data(elts[pos + 1]), rearm); /* C.1 combine data from mCQEs with rx_descriptor_fields1. */ rxdf1 = _mm_shuffle_epi8(mcqe1, shuf_mask1); rxdf2 = _mm_shuffle_epi8(mcqe1, shuf_mask2); @@ -156,14 +156,14 @@ rxdf2 = _mm_blend_epi16(rxdf2, rxdf, 0x23); /* D.1 store rx_descriptor_fields1. */ _mm_storeu_si128((__m128i *) - &elts[pos]->rx_descriptor_fields1, + rte_mbuf_rx_descriptor_fields1(elts[pos]), rxdf1); _mm_storeu_si128((__m128i *) - &elts[pos + 1]->rx_descriptor_fields1, + rte_mbuf_rx_descriptor_fields1(elts[pos + 1]), rxdf2); /* B.1 store rearm data to mbuf. */ - _mm_storeu_si128((__m128i *)&elts[pos + 2]->rearm_data, rearm); - _mm_storeu_si128((__m128i *)&elts[pos + 3]->rearm_data, rearm); + _mm_storeu_si128((__m128i *)rte_mbuf_rearm_data(elts[pos + 2]), rearm); + _mm_storeu_si128((__m128i *)rte_mbuf_rearm_data(elts[pos + 3]), rearm); /* C.1 combine data from mCQEs with rx_descriptor_fields1. */ rxdf1 = _mm_shuffle_epi8(mcqe2, shuf_mask1); rxdf2 = _mm_shuffle_epi8(mcqe2, shuf_mask2); @@ -173,10 +173,10 @@ rxdf2 = _mm_blend_epi16(rxdf2, rxdf, 0x23); /* D.1 store rx_descriptor_fields1. */ _mm_storeu_si128((__m128i *) - &elts[pos + 2]->rx_descriptor_fields1, + rte_mbuf_rx_descriptor_fields1(elts[pos + 2]), rxdf1); _mm_storeu_si128((__m128i *) - &elts[pos + 3]->rx_descriptor_fields1, + rte_mbuf_rx_descriptor_fields1(elts[pos + 3]), rxdf2); #ifdef MLX5_PMD_SOFT_COUNTERS invalid_mask = _mm_set_epi64x(0, @@ -511,10 +511,10 @@ rearm2 = _mm_blend_epi16(mbuf_init, ol_flags, 0x30); rearm3 = _mm_blend_epi16(mbuf_init, _mm_srli_si128(ol_flags, 4), 0x30); /* Write 8B rearm_data and 8B ol_flags. */ - _mm_store_si128((__m128i *)&pkts[0]->rearm_data, rearm0); - _mm_store_si128((__m128i *)&pkts[1]->rearm_data, rearm1); - _mm_store_si128((__m128i *)&pkts[2]->rearm_data, rearm2); - _mm_store_si128((__m128i *)&pkts[3]->rearm_data, rearm3); + _mm_store_si128((__m128i *)rte_mbuf_rearm_data(pkts[0]), rearm0); + _mm_store_si128((__m128i *)rte_mbuf_rearm_data(pkts[1]), rearm1); + _mm_store_si128((__m128i *)rte_mbuf_rearm_data(pkts[2]), rearm2); + _mm_store_si128((__m128i *)rte_mbuf_rearm_data(pkts[3]), rearm3); } /** From patchwork Tue Feb 27 05:41:31 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Tyler Retzlaff X-Patchwork-Id: 137310 X-Patchwork-Delegate: thomas@monjalon.net Return-Path: X-Original-To: patchwork@inbox.dpdk.org Delivered-To: patchwork@inbox.dpdk.org Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id 535F243C03; Tue, 27 Feb 2024 06:43:49 +0100 (CET) Received: from mails.dpdk.org (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id CD9C542EE2; Tue, 27 Feb 2024 06:42:32 +0100 (CET) Received: from linux.microsoft.com (linux.microsoft.com [13.77.154.182]) by mails.dpdk.org (Postfix) with ESMTP id 5295542E56 for ; Tue, 27 Feb 2024 06:41:43 +0100 (CET) Received: by linux.microsoft.com (Postfix, from userid 1086) id 8AB1320B74CF; Mon, 26 Feb 2024 21:41:41 -0800 (PST) DKIM-Filter: OpenDKIM Filter v2.11.0 linux.microsoft.com 8AB1320B74CF DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linux.microsoft.com; s=default; t=1709012501; bh=zNWJvX6I4yGoJDSJ1k+o8LChAhF9YjRc60nIkCTrlqc=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=Ij5s0kA6ltS79pIy24jCDl9jf4FZyoHBUJdAjUFWQAVzJUHC2NI/G0TCgi1GtzNd5 xPToYaiyk29sCyk3eWmxU00J5V/2aYZ6t0w0V+cTO1LklK/s6LcV241qHJ2UqP1j+f 0qSkfyXdob8YJLrvEBWHzum3YbEsULSLCwUsMNPk= From: Tyler Retzlaff To: dev@dpdk.org Cc: Ajit Khaparde , Andrew Boyer , Andrew Rybchenko , Bruce Richardson , Chenbo Xia , Chengwen Feng , Dariusz Sosnowski , David Christensen , Hyong Youb Kim , Jerin Jacob , Jie Hai , Jingjing Wu , John Daley , Kevin Laatz , Kiran Kumar K , Konstantin Ananyev , Maciej Czekaj , Matan Azrad , Maxime Coquelin , Nithin Dabilpuram , Ori Kam , Ruifeng Wang , Satha Rao , Somnath Kotur , Suanming Mou , Sunil Kumar Kori , Viacheslav Ovsiienko , Yisen Zhuang , Yuying Zhang , mb@smartsharesystems.com, Tyler Retzlaff Subject: [PATCH v6 15/23] net/octeon_ep: use mbuf descriptor accessors Date: Mon, 26 Feb 2024 21:41:31 -0800 Message-Id: <1709012499-12813-16-git-send-email-roretzla@linux.microsoft.com> X-Mailer: git-send-email 1.8.3.1 In-Reply-To: <1709012499-12813-1-git-send-email-roretzla@linux.microsoft.com> References: <1706657173-26166-1-git-send-email-roretzla@linux.microsoft.com> <1709012499-12813-1-git-send-email-roretzla@linux.microsoft.com> X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org RTE_MARKER typedefs are a GCC extension unsupported by MSVC. Use new rte_mbuf_rearm_data and rte_mbuf_rx_descriptor_fields1 accessors that provide a compatible type pointer without using the marker fields. Signed-off-by: Tyler Retzlaff --- drivers/net/octeon_ep/cnxk_ep_rx.h | 2 +- drivers/net/octeon_ep/cnxk_ep_rx_avx.c | 2 +- drivers/net/octeon_ep/cnxk_ep_rx_neon.c | 8 ++++---- drivers/net/octeon_ep/cnxk_ep_rx_sse.c | 8 ++++---- drivers/net/octeon_ep/otx_ep_rxtx.c | 5 +---- 5 files changed, 11 insertions(+), 14 deletions(-) diff --git a/drivers/net/octeon_ep/cnxk_ep_rx.h b/drivers/net/octeon_ep/cnxk_ep_rx.h index 61263e6..1d706e2 100644 --- a/drivers/net/octeon_ep/cnxk_ep_rx.h +++ b/drivers/net/octeon_ep/cnxk_ep_rx.h @@ -158,7 +158,7 @@ mbuf->pkt_len = pkt_len; mbuf->data_len = pkt_len; - *(uint64_t *)&mbuf->rearm_data = droq->rearm_data; + *rte_mbuf_rearm_data(mbuf) = droq->rearm_data; rx_pkts[pkts] = mbuf; bytes_rsvd += pkt_len; } diff --git a/drivers/net/octeon_ep/cnxk_ep_rx_avx.c b/drivers/net/octeon_ep/cnxk_ep_rx_avx.c index 47eb1d2..3b8241c 100644 --- a/drivers/net/octeon_ep/cnxk_ep_rx_avx.c +++ b/drivers/net/octeon_ep/cnxk_ep_rx_avx.c @@ -60,7 +60,7 @@ /* Store the 256bit data to the mbuf. */ for (i = 0; i < CNXK_EP_OQ_DESC_PER_LOOP_AVX; i++) - _mm256_storeu_si256((__m256i *)&m[i]->rearm_data, data[i]); + _mm256_storeu_si256((__m256i *)rte_mbuf_rearm_data(m[i]), data[i]); for (i = 0; i < CNXK_EP_OQ_DESC_PER_LOOP_AVX; i++) rx_pkts[pkts++] = m[i]; diff --git a/drivers/net/octeon_ep/cnxk_ep_rx_neon.c b/drivers/net/octeon_ep/cnxk_ep_rx_neon.c index 4c46a7e..8797252 100644 --- a/drivers/net/octeon_ep/cnxk_ep_rx_neon.c +++ b/drivers/net/octeon_ep/cnxk_ep_rx_neon.c @@ -72,10 +72,10 @@ *(uint64_t *)&m3->pkt_len = vgetq_lane_u64(s23, 1); /* Reset rearm data. */ - *(uint64_t *)&m0->rearm_data = droq->rearm_data; - *(uint64_t *)&m1->rearm_data = droq->rearm_data; - *(uint64_t *)&m2->rearm_data = droq->rearm_data; - *(uint64_t *)&m3->rearm_data = droq->rearm_data; + *rte_mbuf_rearm_data(m0) = droq->rearm_data; + *rte_mbuf_rearm_data(m1) = droq->rearm_data; + *rte_mbuf_rearm_data(m2) = droq->rearm_data; + *rte_mbuf_rearm_data(m3) = droq->rearm_data; rx_pkts[pkts++] = m0; rx_pkts[pkts++] = m1; diff --git a/drivers/net/octeon_ep/cnxk_ep_rx_sse.c b/drivers/net/octeon_ep/cnxk_ep_rx_sse.c index 308c8b2..1466217 100644 --- a/drivers/net/octeon_ep/cnxk_ep_rx_sse.c +++ b/drivers/net/octeon_ep/cnxk_ep_rx_sse.c @@ -64,10 +64,10 @@ *(uint64_t *)&m3->pkt_len = ((rte_xmm_t)s23).u64[1]; /* Reset rearm data. */ - *(uint64_t *)&m0->rearm_data = droq->rearm_data; - *(uint64_t *)&m1->rearm_data = droq->rearm_data; - *(uint64_t *)&m2->rearm_data = droq->rearm_data; - *(uint64_t *)&m3->rearm_data = droq->rearm_data; + *rte_mbuf_rearm_data(m0) = droq->rearm_data; + *rte_mbuf_rearm_data(m1) = droq->rearm_data; + *rte_mbuf_rearm_data(m2) = droq->rearm_data; + *rte_mbuf_rearm_data(m3) = droq->rearm_data; rx_pkts[pkts++] = m0; rx_pkts[pkts++] = m1; diff --git a/drivers/net/octeon_ep/otx_ep_rxtx.c b/drivers/net/octeon_ep/otx_ep_rxtx.c index aea148e..4470599 100644 --- a/drivers/net/octeon_ep/otx_ep_rxtx.c +++ b/drivers/net/octeon_ep/otx_ep_rxtx.c @@ -289,7 +289,6 @@ { uint16_t port_id = otx_ep->port_id; struct rte_mbuf mb_def; - uint64_t *tmp; RTE_BUILD_BUG_ON(offsetof(struct rte_mbuf, data_off) % 8 != 0); RTE_BUILD_BUG_ON(offsetof(struct rte_mbuf, refcnt) - offsetof(struct rte_mbuf, data_off) != @@ -305,9 +304,7 @@ /* Prevent compiler reordering: rearm_data covers previous fields */ rte_compiler_barrier(); - tmp = (uint64_t *)&mb_def.rearm_data; - - return *tmp; + return *rte_mbuf_rearm_data(&mb_def); } /* OQ initialization */ From patchwork Tue Feb 27 05:41:32 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Tyler Retzlaff X-Patchwork-Id: 137313 X-Patchwork-Delegate: thomas@monjalon.net Return-Path: X-Original-To: patchwork@inbox.dpdk.org Delivered-To: patchwork@inbox.dpdk.org Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id F29FC43C03; Tue, 27 Feb 2024 06:44:04 +0100 (CET) Received: from mails.dpdk.org (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id A9F9642EF0; Tue, 27 Feb 2024 06:42:36 +0100 (CET) Received: from linux.microsoft.com (linux.microsoft.com [13.77.154.182]) by mails.dpdk.org (Postfix) with ESMTP id 5248A42E54 for ; Tue, 27 Feb 2024 06:41:43 +0100 (CET) Received: by linux.microsoft.com (Postfix, from userid 1086) id 98B6120B74D0; Mon, 26 Feb 2024 21:41:41 -0800 (PST) DKIM-Filter: OpenDKIM Filter v2.11.0 linux.microsoft.com 98B6120B74D0 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linux.microsoft.com; s=default; t=1709012501; bh=8tw0oToDLDUsQ5KZ4tGaFL1Uq9a+Bb1PhVNBK/M4mI0=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=GHFlfPog7YpekmOhnhc/1fHTtRoslOb+p07D3wU/SdFNYL4mxqa+4B+bp/xOFTPD8 /t+j5AfT6OKzBNz3CQNLLCAtTJ1/e6rCdkcmm7PnM23mKUPhQOiklGbxXECmDrvisA OO8tz2K/Sc7n8HvyMCbQU795KhMbAOR1y3fBn3hc= From: Tyler Retzlaff To: dev@dpdk.org Cc: Ajit Khaparde , Andrew Boyer , Andrew Rybchenko , Bruce Richardson , Chenbo Xia , Chengwen Feng , Dariusz Sosnowski , David Christensen , Hyong Youb Kim , Jerin Jacob , Jie Hai , Jingjing Wu , John Daley , Kevin Laatz , Kiran Kumar K , Konstantin Ananyev , Maciej Czekaj , Matan Azrad , Maxime Coquelin , Nithin Dabilpuram , Ori Kam , Ruifeng Wang , Satha Rao , Somnath Kotur , Suanming Mou , Sunil Kumar Kori , Viacheslav Ovsiienko , Yisen Zhuang , Yuying Zhang , mb@smartsharesystems.com, Tyler Retzlaff Subject: [PATCH v6 16/23] net/sfc: use mbuf descriptor accessors Date: Mon, 26 Feb 2024 21:41:32 -0800 Message-Id: <1709012499-12813-17-git-send-email-roretzla@linux.microsoft.com> X-Mailer: git-send-email 1.8.3.1 In-Reply-To: <1709012499-12813-1-git-send-email-roretzla@linux.microsoft.com> References: <1706657173-26166-1-git-send-email-roretzla@linux.microsoft.com> <1709012499-12813-1-git-send-email-roretzla@linux.microsoft.com> X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org RTE_MARKER typedefs are a GCC extension unsupported by MSVC. Use new rte_mbuf_rearm_data and rte_mbuf_rx_descriptor_fields1 accessors that provide a compatible type pointer without using the marker fields. Signed-off-by: Tyler Retzlaff --- drivers/net/sfc/sfc_ef100_rx.c | 7 ++----- drivers/net/sfc/sfc_ef10_rx.c | 10 +++------- 2 files changed, 5 insertions(+), 12 deletions(-) diff --git a/drivers/net/sfc/sfc_ef100_rx.c b/drivers/net/sfc/sfc_ef100_rx.c index e283879..51c7da0 100644 --- a/drivers/net/sfc/sfc_ef100_rx.c +++ b/drivers/net/sfc/sfc_ef100_rx.c @@ -553,9 +553,7 @@ struct sfc_ef100_rxq { pkt = sfc_ef100_rx_next_mbuf(rxq); __rte_mbuf_raw_sanity_check(pkt); - RTE_BUILD_BUG_ON(sizeof(pkt->rearm_data[0]) != - sizeof(rxq->rearm_data)); - pkt->rearm_data[0] = rxq->rearm_data; + *rte_mbuf_rearm_data(pkt) = rxq->rearm_data; /* data_off already moved past Rx prefix */ rx_prefix = (const efx_xword_t *)sfc_ef100_rx_pkt_prefix(pkt); @@ -760,8 +758,7 @@ struct sfc_ef100_rxq { /* rearm_data covers structure members filled in above */ rte_compiler_barrier(); - RTE_BUILD_BUG_ON(sizeof(m.rearm_data[0]) != sizeof(uint64_t)); - return m.rearm_data[0]; + return *rte_mbuf_rearm_data(&m); } static sfc_dp_rx_qcreate_t sfc_ef100_rx_qcreate; diff --git a/drivers/net/sfc/sfc_ef10_rx.c b/drivers/net/sfc/sfc_ef10_rx.c index 6044293..2ac0203 100644 --- a/drivers/net/sfc/sfc_ef10_rx.c +++ b/drivers/net/sfc/sfc_ef10_rx.c @@ -322,8 +322,7 @@ struct sfc_ef10_rxq { m = rxd->mbuf; - RTE_BUILD_BUG_ON(sizeof(m->rearm_data[0]) != sizeof(rxq->rearm_data)); - m->rearm_data[0] = rxq->rearm_data; + *rte_mbuf_rearm_data(m) = rxq->rearm_data; /* Classify packet based on Rx event */ /* Mask RSS hash offload flag if RSS is not enabled */ @@ -377,9 +376,7 @@ struct sfc_ef10_rxq { rxq->completed = pending; } - RTE_BUILD_BUG_ON(sizeof(m->rearm_data[0]) != - sizeof(rxq->rearm_data)); - m->rearm_data[0] = rxq->rearm_data; + *rte_mbuf_rearm_data(m) = rxq->rearm_data; /* Event-dependent information is the same */ m->ol_flags = m0->ol_flags; @@ -633,8 +630,7 @@ struct sfc_ef10_rxq { /* rearm_data covers structure members filled in above */ rte_compiler_barrier(); - RTE_BUILD_BUG_ON(sizeof(m.rearm_data[0]) != sizeof(uint64_t)); - return m.rearm_data[0]; + return *rte_mbuf_rearm_data(&m); } static sfc_dp_rx_qcreate_t sfc_ef10_rx_qcreate; From patchwork Tue Feb 27 05:41:33 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Tyler Retzlaff X-Patchwork-Id: 137316 X-Patchwork-Delegate: thomas@monjalon.net Return-Path: X-Original-To: patchwork@inbox.dpdk.org Delivered-To: patchwork@inbox.dpdk.org Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id 99E2143C03; Tue, 27 Feb 2024 06:44:20 +0100 (CET) Received: from mails.dpdk.org (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id 626ED42F0C; Tue, 27 Feb 2024 06:42:40 +0100 (CET) Received: from linux.microsoft.com (linux.microsoft.com [13.77.154.182]) by mails.dpdk.org (Postfix) with ESMTP id 4D9CE42E4E for ; Tue, 27 Feb 2024 06:41:43 +0100 (CET) Received: by linux.microsoft.com (Postfix, from userid 1086) id A889020B74D1; Mon, 26 Feb 2024 21:41:41 -0800 (PST) DKIM-Filter: OpenDKIM Filter v2.11.0 linux.microsoft.com A889020B74D1 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linux.microsoft.com; s=default; t=1709012501; bh=43J41FUN8U97ef9HAWZHV2AQLsuA5wCPXesxdNMpgMw=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=MzcKe6Y+nlKB6DGivU47xnSAnhoVx8XY7i3tUPdxBXiTKF7bb19gIWNMuWQArnnEd Tm59SPRtr+JOTu0M79ALvEao26Vq3S3TRsQwEbJzb3pb/5jgJBXbwOkb3oznqsmZlz I8ajPzLPuviEVDv9jE5xopx4Smx+7PnNGRbDf2UM= From: Tyler Retzlaff To: dev@dpdk.org Cc: Ajit Khaparde , Andrew Boyer , Andrew Rybchenko , Bruce Richardson , Chenbo Xia , Chengwen Feng , Dariusz Sosnowski , David Christensen , Hyong Youb Kim , Jerin Jacob , Jie Hai , Jingjing Wu , John Daley , Kevin Laatz , Kiran Kumar K , Konstantin Ananyev , Maciej Czekaj , Matan Azrad , Maxime Coquelin , Nithin Dabilpuram , Ori Kam , Ruifeng Wang , Satha Rao , Somnath Kotur , Suanming Mou , Sunil Kumar Kori , Viacheslav Ovsiienko , Yisen Zhuang , Yuying Zhang , mb@smartsharesystems.com, Tyler Retzlaff Subject: [PATCH v6 17/23] net/thunderx: use mbuf descriptor accessors Date: Mon, 26 Feb 2024 21:41:33 -0800 Message-Id: <1709012499-12813-18-git-send-email-roretzla@linux.microsoft.com> X-Mailer: git-send-email 1.8.3.1 In-Reply-To: <1709012499-12813-1-git-send-email-roretzla@linux.microsoft.com> References: <1706657173-26166-1-git-send-email-roretzla@linux.microsoft.com> <1709012499-12813-1-git-send-email-roretzla@linux.microsoft.com> X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org RTE_MARKER typedefs are a GCC extension unsupported by MSVC. Use new rte_mbuf_rearm_data and rte_mbuf_rx_descriptor_fields1 accessors that provide a compatible type pointer without using the marker fields. Signed-off-by: Tyler Retzlaff --- drivers/net/thunderx/nicvf_ethdev.c | 4 +--- drivers/net/thunderx/nicvf_rxtx.h | 4 ++-- 2 files changed, 3 insertions(+), 5 deletions(-) diff --git a/drivers/net/thunderx/nicvf_ethdev.c b/drivers/net/thunderx/nicvf_ethdev.c index 609d95d..722751c 100644 --- a/drivers/net/thunderx/nicvf_ethdev.c +++ b/drivers/net/thunderx/nicvf_ethdev.c @@ -1306,7 +1306,6 @@ enum nicvf_link_speed { static inline void nicvf_rxq_mbuf_setup(struct nicvf_rxq *rxq) { - uintptr_t p; struct rte_mbuf mb_def; struct nicvf *nic = rxq->nic; @@ -1328,8 +1327,7 @@ enum nicvf_link_speed { /* Prevent compiler reordering: rearm_data covers previous fields */ rte_compiler_barrier(); - p = (uintptr_t)&mb_def.rearm_data; - rxq->mbuf_initializer.value = *(uint64_t *)p; + rxq->mbuf_initializer.value = *rte_mbuf_rearm_data(&mb_def); } static int diff --git a/drivers/net/thunderx/nicvf_rxtx.h b/drivers/net/thunderx/nicvf_rxtx.h index 4b83e33..7c07cde 100644 --- a/drivers/net/thunderx/nicvf_rxtx.h +++ b/drivers/net/thunderx/nicvf_rxtx.h @@ -66,7 +66,7 @@ static inline uint16_t __attribute__((const)) #else init.value += apad; #endif - *(uint64_t *)(&pkt->rearm_data) = init.value; + *rte_mbuf_rearm_data(pkt) = init.value; } static inline void @@ -80,7 +80,7 @@ static inline uint16_t __attribute__((const)) init.value += apad; #endif init.fields.nb_segs = nb_segs; - *(uint64_t *)(&pkt->rearm_data) = init.value; + *rte_mbuf_rearm_data(pkt) = init.value; } uint32_t nicvf_dev_rx_queue_count(void *rx_queue); From patchwork Tue Feb 27 05:41:34 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Tyler Retzlaff X-Patchwork-Id: 137317 X-Patchwork-Delegate: thomas@monjalon.net Return-Path: X-Original-To: patchwork@inbox.dpdk.org Delivered-To: patchwork@inbox.dpdk.org Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id AFE6743C03; Tue, 27 Feb 2024 06:44:25 +0100 (CET) Received: from mails.dpdk.org (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id 9A16942F0F; Tue, 27 Feb 2024 06:42:41 +0100 (CET) Received: from linux.microsoft.com (linux.microsoft.com [13.77.154.182]) by mails.dpdk.org (Postfix) with ESMTP id 4D77E42E4B for ; Tue, 27 Feb 2024 06:41:43 +0100 (CET) Received: by linux.microsoft.com (Postfix, from userid 1086) id B88C520B74D2; Mon, 26 Feb 2024 21:41:41 -0800 (PST) DKIM-Filter: OpenDKIM Filter v2.11.0 linux.microsoft.com B88C520B74D2 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linux.microsoft.com; s=default; t=1709012501; bh=g5QA1cn6H2XRCnwto8cFpq1UfgeAarDzD+gzpktDtNs=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=eshULGuxLBWOn7+WkS+kjdaZWhr6Lposf16XP4P34Zg42UZiu1muLtGaIHBPlnQ6a CSR6bTGHD5uIkvaAjQmmJBlEPlXtB6tuyGdcg63K19BuCziC3vjtokajJaQ3x9PzNg nQWxJiEU0OG9Mug0fK53502UuMSbA8zrmC0E5W20= From: Tyler Retzlaff To: dev@dpdk.org Cc: Ajit Khaparde , Andrew Boyer , Andrew Rybchenko , Bruce Richardson , Chenbo Xia , Chengwen Feng , Dariusz Sosnowski , David Christensen , Hyong Youb Kim , Jerin Jacob , Jie Hai , Jingjing Wu , John Daley , Kevin Laatz , Kiran Kumar K , Konstantin Ananyev , Maciej Czekaj , Matan Azrad , Maxime Coquelin , Nithin Dabilpuram , Ori Kam , Ruifeng Wang , Satha Rao , Somnath Kotur , Suanming Mou , Sunil Kumar Kori , Viacheslav Ovsiienko , Yisen Zhuang , Yuying Zhang , mb@smartsharesystems.com, Tyler Retzlaff Subject: [PATCH v6 18/23] net/virtio: use mbuf descriptor accessors Date: Mon, 26 Feb 2024 21:41:34 -0800 Message-Id: <1709012499-12813-19-git-send-email-roretzla@linux.microsoft.com> X-Mailer: git-send-email 1.8.3.1 In-Reply-To: <1709012499-12813-1-git-send-email-roretzla@linux.microsoft.com> References: <1706657173-26166-1-git-send-email-roretzla@linux.microsoft.com> <1709012499-12813-1-git-send-email-roretzla@linux.microsoft.com> X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org RTE_MARKER typedefs are a GCC extension unsupported by MSVC. Use new rte_mbuf_rearm_data and rte_mbuf_rx_descriptor_fields1 accessors that provide a compatible type pointer without using the marker fields. Signed-off-by: Tyler Retzlaff --- drivers/net/virtio/virtio_rxtx_packed.h | 4 ++-- drivers/net/virtio/virtio_rxtx_packed_avx.h | 16 +++++----------- drivers/net/virtio/virtio_rxtx_packed_neon.h | 18 ++++++++---------- drivers/net/virtio/virtio_rxtx_simple.c | 4 +--- drivers/net/virtio/virtio_rxtx_simple.h | 5 +---- drivers/net/virtio/virtio_rxtx_simple_altivec.c | 16 ++++++++-------- drivers/net/virtio/virtio_rxtx_simple_neon.c | 24 ++++++++---------------- drivers/net/virtio/virtio_rxtx_simple_sse.c | 16 ++++++++-------- 8 files changed, 41 insertions(+), 62 deletions(-) diff --git a/drivers/net/virtio/virtio_rxtx_packed.h b/drivers/net/virtio/virtio_rxtx_packed.h index 5361129..0a8c9f8 100644 --- a/drivers/net/virtio/virtio_rxtx_packed.h +++ b/drivers/net/virtio/virtio_rxtx_packed.h @@ -37,12 +37,12 @@ /* reference count offset in mbuf rearm data */ #define REFCNT_BITS_OFFSET ((offsetof(struct rte_mbuf, refcnt) - \ - offsetof(struct rte_mbuf, rearm_data)) * BYTE_SIZE) + offsetof(struct rte_mbuf, data_off)) * BYTE_SIZE) #ifdef CC_AVX512_SUPPORT /* segment number offset in mbuf rearm data */ #define SEG_NUM_BITS_OFFSET ((offsetof(struct rte_mbuf, nb_segs) - \ - offsetof(struct rte_mbuf, rearm_data)) * BYTE_SIZE) + offsetof(struct rte_mbuf, data_off)) * BYTE_SIZE) /* default rearm data */ #define DEFAULT_REARM_DATA (1ULL << SEG_NUM_BITS_OFFSET | \ 1ULL << REFCNT_BITS_OFFSET) diff --git a/drivers/net/virtio/virtio_rxtx_packed_avx.h b/drivers/net/virtio/virtio_rxtx_packed_avx.h index 584ac72..5d30e2a 100644 --- a/drivers/net/virtio/virtio_rxtx_packed_avx.h +++ b/drivers/net/virtio/virtio_rxtx_packed_avx.h @@ -36,10 +36,10 @@ /* Load four mbufs rearm data */ RTE_BUILD_BUG_ON(REFCNT_BITS_OFFSET >= 64); RTE_BUILD_BUG_ON(SEG_NUM_BITS_OFFSET >= 64); - __m256i mbufs = _mm256_set_epi64x(*tx_pkts[3]->rearm_data, - *tx_pkts[2]->rearm_data, - *tx_pkts[1]->rearm_data, - *tx_pkts[0]->rearm_data); + __m256i mbufs = _mm256_set_epi64x(*rte_mbuf_rearm_data(tx_pkts[3]), + *rte_mbuf_rearm_data(tx_pkts[2]), + *rte_mbuf_rearm_data(tx_pkts[1]), + *rte_mbuf_rearm_data(tx_pkts[0])); /* refcnt=1 and nb_segs=1 */ __m256i mbuf_ref = _mm256_set1_epi64x(DEFAULT_REARM_DATA); @@ -53,8 +53,6 @@ /* Check headroom is enough */ const __mmask16 data_mask = 0x1 | 0x1 << 4 | 0x1 << 8 | 0x1 << 12; - RTE_BUILD_BUG_ON(offsetof(struct rte_mbuf, data_off) != - offsetof(struct rte_mbuf, rearm_data)); cmp = _mm256_mask_cmplt_epu16_mask(data_mask, mbufs, head_rooms); if (unlikely(cmp)) return -1; @@ -187,7 +185,7 @@ rx_pkts[i] = (struct rte_mbuf *)vq->vq_descx[id + i].cookie; rte_packet_prefetch(rte_pktmbuf_mtod(rx_pkts[i], void *)); - addrs[i] = (uintptr_t)rx_pkts[i]->rx_descriptor_fields1; + addrs[i] = (uintptr_t)rte_mbuf_rx_descriptor_fields1(rx_pkts[i]); } /* @@ -203,10 +201,6 @@ __m512i v_value = _mm512_add_epi32(values, mbuf_len_offset); - /* assert offset of data_len */ - RTE_BUILD_BUG_ON(offsetof(struct rte_mbuf, data_len) != - offsetof(struct rte_mbuf, rx_descriptor_fields1) + 8); - __m512i v_index = _mm512_set_epi64(addrs[3] + 8, addrs[3], addrs[2] + 8, addrs[2], addrs[1] + 8, addrs[1], diff --git a/drivers/net/virtio/virtio_rxtx_packed_neon.h b/drivers/net/virtio/virtio_rxtx_packed_neon.h index c222ebf..c172fad 100644 --- a/drivers/net/virtio/virtio_rxtx_packed_neon.h +++ b/drivers/net/virtio/virtio_rxtx_packed_neon.h @@ -59,10 +59,10 @@ uint8x16x2_t mbuf; /* Load four mbufs rearm data. */ RTE_BUILD_BUG_ON(REFCNT_BITS_OFFSET >= 64); - pkts[0] = vld1_u16((uint16_t *)&tx_pkts[0]->rearm_data); - pkts[1] = vld1_u16((uint16_t *)&tx_pkts[1]->rearm_data); - pkts[2] = vld1_u16((uint16_t *)&tx_pkts[2]->rearm_data); - pkts[3] = vld1_u16((uint16_t *)&tx_pkts[3]->rearm_data); + pkts[0] = vld1_u16((uint16_t *)rte_mbuf_rearm_data(tx_pkts[0])); + pkts[1] = vld1_u16((uint16_t *)rte_mbuf_rearm_data(tx_pkts[1])); + pkts[2] = vld1_u16((uint16_t *)rte_mbuf_rearm_data(tx_pkts[2])); + pkts[3] = vld1_u16((uint16_t *)rte_mbuf_rearm_data(tx_pkts[3])); mbuf.val[0] = vreinterpretq_u8_u16(vcombine_u16(pkts[0], pkts[1])); mbuf.val[1] = vreinterpretq_u8_u16(vcombine_u16(pkts[2], pkts[3])); @@ -77,8 +77,6 @@ /* Check headroom is enough. */ uint16x4_t head_rooms = vdup_n_u16(head_size); - RTE_BUILD_BUG_ON(offsetof(struct rte_mbuf, data_off) != - offsetof(struct rte_mbuf, rearm_data)); uint16x4_t data_offset = vreinterpret_u16_u8(vqtbl2_u8(mbuf, data_msk)); uint64x1_t cmp2 = vreinterpret_u64_u16(vclt_u16(data_offset, head_rooms)); if (unlikely(vget_lane_u64(cmp2, 0))) @@ -263,10 +261,10 @@ pkt_mb[3] = vreinterpretq_u64_u16(vsubq_u16( vreinterpretq_u16_u64(pkt_mb[3]), len_adjust)); - vst1q_u64((void *)&rx_pkts[0]->rx_descriptor_fields1, pkt_mb[0]); - vst1q_u64((void *)&rx_pkts[1]->rx_descriptor_fields1, pkt_mb[1]); - vst1q_u64((void *)&rx_pkts[2]->rx_descriptor_fields1, pkt_mb[2]); - vst1q_u64((void *)&rx_pkts[3]->rx_descriptor_fields1, pkt_mb[3]); + vst1q_u64(rte_mbuf_rx_descriptor_fields1(rx_pkts[0]), pkt_mb[0]); + vst1q_u64(rte_mbuf_rx_descriptor_fields1(rx_pkts[1]), pkt_mb[1]); + vst1q_u64(rte_mbuf_rx_descriptor_fields1(rx_pkts[2]), pkt_mb[2]); + vst1q_u64(rte_mbuf_rx_descriptor_fields1(rx_pkts[3]), pkt_mb[3]); if (hw->has_rx_offload) { virtio_for_each_try_unroll(i, 0, PACKED_BATCH_SIZE) { diff --git a/drivers/net/virtio/virtio_rxtx_simple.c b/drivers/net/virtio/virtio_rxtx_simple.c index 4382569..e6a82e5 100644 --- a/drivers/net/virtio/virtio_rxtx_simple.c +++ b/drivers/net/virtio/virtio_rxtx_simple.c @@ -31,7 +31,6 @@ virtio_rxq_vec_setup(struct virtnet_rx *rxq) { struct virtqueue *vq = virtnet_rxq_to_vq(rxq); - uintptr_t p; struct rte_mbuf mb_def = { .buf_addr = 0 }; /* zeroed mbuf */ mb_def.nb_segs = 1; @@ -41,8 +40,7 @@ /* prevent compiler reordering: rearm_data covers previous fields */ rte_compiler_barrier(); - p = (uintptr_t)&mb_def.rearm_data; - rxq->mbuf_initializer = *(uint64_t *)p; + rxq->mbuf_initializer = *rte_mbuf_rearm_data(&mb_def); return 0; } diff --git a/drivers/net/virtio/virtio_rxtx_simple.h b/drivers/net/virtio/virtio_rxtx_simple.h index 79196ed..16fef0e 100644 --- a/drivers/net/virtio/virtio_rxtx_simple.h +++ b/drivers/net/virtio/virtio_rxtx_simple.h @@ -39,10 +39,7 @@ } for (i = 0; i < RTE_VIRTIO_VPMD_RX_REARM_THRESH; i++) { - uintptr_t p; - - p = (uintptr_t)&sw_ring[i]->rearm_data; - *(uint64_t *)p = rxvq->mbuf_initializer; + *rte_mbuf_rearm_data(sw_ring[i]) = rxvq->mbuf_initializer; start_dp[i].addr = VIRTIO_MBUF_ADDR(sw_ring[i], vq) + RTE_PKTMBUF_HEADROOM - vq->hw->vtnet_hdr_size; diff --git a/drivers/net/virtio/virtio_rxtx_simple_altivec.c b/drivers/net/virtio/virtio_rxtx_simple_altivec.c index 542ec3d..39ec1ca 100644 --- a/drivers/net/virtio/virtio_rxtx_simple_altivec.c +++ b/drivers/net/virtio/virtio_rxtx_simple_altivec.c @@ -138,9 +138,9 @@ ((__vector unsigned short)pkt_mb[0] + len_adjust); pkt_mb[1] = (__vector unsigned char) ((__vector unsigned short)pkt_mb[1] + len_adjust); - *(__vector unsigned char *)&rx_pkts[0]->rx_descriptor_fields1 = + *(__vector unsigned char *)rte_mbuf_rx_descriptor_fields1(rx_pkts[0]) = pkt_mb[0]; - *(__vector unsigned char *)&rx_pkts[1]->rx_descriptor_fields1 = + *(__vector unsigned char *)rte_mbuf_rx_descriptor_fields1(rx_pkts[1]) = pkt_mb[1]; pkt_mb[2] = vec_perm(desc[1], zero, shuf_msk1); @@ -149,9 +149,9 @@ ((__vector unsigned short)pkt_mb[2] + len_adjust); pkt_mb[3] = (__vector unsigned char) ((__vector unsigned short)pkt_mb[3] + len_adjust); - *(__vector unsigned char *)&rx_pkts[2]->rx_descriptor_fields1 = + *(__vector unsigned char *)rte_mbuf_rx_descriptor_fields1(rx_pkts[2]) = pkt_mb[2]; - *(__vector unsigned char *)&rx_pkts[3]->rx_descriptor_fields1 = + *(__vector unsigned char *)rte_mbuf_rx_descriptor_fields1(rx_pkts[3]) = pkt_mb[3]; pkt_mb[4] = vec_perm(desc[2], zero, shuf_msk1); @@ -160,9 +160,9 @@ ((__vector unsigned short)pkt_mb[4] + len_adjust); pkt_mb[5] = (__vector unsigned char) ((__vector unsigned short)pkt_mb[5] + len_adjust); - *(__vector unsigned char *)&rx_pkts[4]->rx_descriptor_fields1 = + *(__vector unsigned char *)rte_mbuf_rx_descriptor_fields1(rx_pkts[4]) = pkt_mb[4]; - *(__vector unsigned char *)&rx_pkts[5]->rx_descriptor_fields1 = + *(__vector unsigned char *)rte_mbuf_rx_descriptor_fields1(rx_pkts[5]) = pkt_mb[5]; pkt_mb[6] = vec_perm(desc[3], zero, shuf_msk1); @@ -171,9 +171,9 @@ ((__vector unsigned short)pkt_mb[6] + len_adjust); pkt_mb[7] = (__vector unsigned char) ((__vector unsigned short)pkt_mb[7] + len_adjust); - *(__vector unsigned char *)&rx_pkts[6]->rx_descriptor_fields1 = + *(__vector unsigned char *)rte_mbuf_rx_descriptor_fields1(rx_pkts[6]) = pkt_mb[6]; - *(__vector unsigned char *)&rx_pkts[7]->rx_descriptor_fields1 = + *(__vector unsigned char *)rte_mbuf_rx_descriptor_fields1(rx_pkts[7]) = pkt_mb[7]; if (unlikely(nb_used <= RTE_VIRTIO_DESC_PER_LOOP)) { diff --git a/drivers/net/virtio/virtio_rxtx_simple_neon.c b/drivers/net/virtio/virtio_rxtx_simple_neon.c index 7139b31..873b42c 100644 --- a/drivers/net/virtio/virtio_rxtx_simple_neon.c +++ b/drivers/net/virtio/virtio_rxtx_simple_neon.c @@ -138,10 +138,8 @@ vreinterpretq_u16_u64(pkt_mb[1]), len_adjust)); pkt_mb[0] = vreinterpretq_u64_u16(vsubq_u16( vreinterpretq_u16_u64(pkt_mb[0]), len_adjust)); - vst1q_u64((void *)&rx_pkts[1]->rx_descriptor_fields1, - pkt_mb[1]); - vst1q_u64((void *)&rx_pkts[0]->rx_descriptor_fields1, - pkt_mb[0]); + vst1q_u64(rte_mbuf_rx_descriptor_fields1(rx_pkts[1]), pkt_mb[1]); + vst1q_u64(rte_mbuf_rx_descriptor_fields1(rx_pkts[0]), pkt_mb[0]); pkt_mb[3] = vreinterpretq_u64_u8(vqtbl1q_u8( vreinterpretq_u8_u64(desc[1]), shuf_msk2)); @@ -151,10 +149,8 @@ vreinterpretq_u16_u64(pkt_mb[3]), len_adjust)); pkt_mb[2] = vreinterpretq_u64_u16(vsubq_u16( vreinterpretq_u16_u64(pkt_mb[2]), len_adjust)); - vst1q_u64((void *)&rx_pkts[3]->rx_descriptor_fields1, - pkt_mb[3]); - vst1q_u64((void *)&rx_pkts[2]->rx_descriptor_fields1, - pkt_mb[2]); + vst1q_u64(rte_mbuf_rx_descriptor_fields1(rx_pkts[3]), pkt_mb[3]); + vst1q_u64(rte_mbuf_rx_descriptor_fields1(rx_pkts[2]), pkt_mb[2]); pkt_mb[5] = vreinterpretq_u64_u8(vqtbl1q_u8( vreinterpretq_u8_u64(desc[2]), shuf_msk2)); @@ -164,10 +160,8 @@ vreinterpretq_u16_u64(pkt_mb[5]), len_adjust)); pkt_mb[4] = vreinterpretq_u64_u16(vsubq_u16( vreinterpretq_u16_u64(pkt_mb[4]), len_adjust)); - vst1q_u64((void *)&rx_pkts[5]->rx_descriptor_fields1, - pkt_mb[5]); - vst1q_u64((void *)&rx_pkts[4]->rx_descriptor_fields1, - pkt_mb[4]); + vst1q_u64(rte_mbuf_rx_descriptor_fields1(rx_pkts[5]), pkt_mb[5]); + vst1q_u64(rte_mbuf_rx_descriptor_fields1(rx_pkts[4]), pkt_mb[4]); pkt_mb[7] = vreinterpretq_u64_u8(vqtbl1q_u8( vreinterpretq_u8_u64(desc[3]), shuf_msk2)); @@ -177,10 +171,8 @@ vreinterpretq_u16_u64(pkt_mb[7]), len_adjust)); pkt_mb[6] = vreinterpretq_u64_u16(vsubq_u16( vreinterpretq_u16_u64(pkt_mb[6]), len_adjust)); - vst1q_u64((void *)&rx_pkts[7]->rx_descriptor_fields1, - pkt_mb[7]); - vst1q_u64((void *)&rx_pkts[6]->rx_descriptor_fields1, - pkt_mb[6]); + vst1q_u64(rte_mbuf_rx_descriptor_fields1(rx_pkts[7]), pkt_mb[7]); + vst1q_u64(rte_mbuf_rx_descriptor_fields1(rx_pkts[6]), pkt_mb[6]); if (unlikely(nb_used <= RTE_VIRTIO_DESC_PER_LOOP)) { if (sw_ring + nb_used <= sw_ring_end) diff --git a/drivers/net/virtio/virtio_rxtx_simple_sse.c b/drivers/net/virtio/virtio_rxtx_simple_sse.c index 6a18741..25db486 100644 --- a/drivers/net/virtio/virtio_rxtx_simple_sse.c +++ b/drivers/net/virtio/virtio_rxtx_simple_sse.c @@ -134,36 +134,36 @@ pkt_mb[0] = _mm_shuffle_epi8(desc[0], shuf_msk1); pkt_mb[1] = _mm_add_epi16(pkt_mb[1], len_adjust); pkt_mb[0] = _mm_add_epi16(pkt_mb[0], len_adjust); - _mm_storeu_si128((void *)&rx_pkts[1]->rx_descriptor_fields1, + _mm_storeu_si128(rte_mbuf_rx_descriptor_fields1(rx_pkts[1]), pkt_mb[1]); - _mm_storeu_si128((void *)&rx_pkts[0]->rx_descriptor_fields1, + _mm_storeu_si128(rte_mbuf_rx_descriptor_fields1(rx_pkts[0]), pkt_mb[0]); pkt_mb[3] = _mm_shuffle_epi8(desc[1], shuf_msk2); pkt_mb[2] = _mm_shuffle_epi8(desc[1], shuf_msk1); pkt_mb[3] = _mm_add_epi16(pkt_mb[3], len_adjust); pkt_mb[2] = _mm_add_epi16(pkt_mb[2], len_adjust); - _mm_storeu_si128((void *)&rx_pkts[3]->rx_descriptor_fields1, + _mm_storeu_si128(rte_mbuf_rx_descriptor_fields1(rx_pkts[3]), pkt_mb[3]); - _mm_storeu_si128((void *)&rx_pkts[2]->rx_descriptor_fields1, + _mm_storeu_si128(rte_mbuf_rx_descriptor_fields1(rx_pkts[2]), pkt_mb[2]); pkt_mb[5] = _mm_shuffle_epi8(desc[2], shuf_msk2); pkt_mb[4] = _mm_shuffle_epi8(desc[2], shuf_msk1); pkt_mb[5] = _mm_add_epi16(pkt_mb[5], len_adjust); pkt_mb[4] = _mm_add_epi16(pkt_mb[4], len_adjust); - _mm_storeu_si128((void *)&rx_pkts[5]->rx_descriptor_fields1, + _mm_storeu_si128(rte_mbuf_rx_descriptor_fields1(rx_pkts[5]), pkt_mb[5]); - _mm_storeu_si128((void *)&rx_pkts[4]->rx_descriptor_fields1, + _mm_storeu_si128(rte_mbuf_rx_descriptor_fields1(rx_pkts[4]), pkt_mb[4]); pkt_mb[7] = _mm_shuffle_epi8(desc[3], shuf_msk2); pkt_mb[6] = _mm_shuffle_epi8(desc[3], shuf_msk1); pkt_mb[7] = _mm_add_epi16(pkt_mb[7], len_adjust); pkt_mb[6] = _mm_add_epi16(pkt_mb[6], len_adjust); - _mm_storeu_si128((void *)&rx_pkts[7]->rx_descriptor_fields1, + _mm_storeu_si128(rte_mbuf_rx_descriptor_fields1(rx_pkts[7]), pkt_mb[7]); - _mm_storeu_si128((void *)&rx_pkts[6]->rx_descriptor_fields1, + _mm_storeu_si128(rte_mbuf_rx_descriptor_fields1(rx_pkts[6]), pkt_mb[6]); if (unlikely(nb_used <= RTE_VIRTIO_DESC_PER_LOOP)) { From patchwork Tue Feb 27 05:41:35 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Tyler Retzlaff X-Patchwork-Id: 137309 X-Patchwork-Delegate: thomas@monjalon.net Return-Path: X-Original-To: patchwork@inbox.dpdk.org Delivered-To: patchwork@inbox.dpdk.org Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id 9C2EC43C03; Tue, 27 Feb 2024 06:43:43 +0100 (CET) Received: from mails.dpdk.org (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id 8DB8842ED7; Tue, 27 Feb 2024 06:42:31 +0100 (CET) Received: from linux.microsoft.com (linux.microsoft.com [13.77.154.182]) by mails.dpdk.org (Postfix) with ESMTP id 594C642E29 for ; Tue, 27 Feb 2024 06:41:43 +0100 (CET) Received: by linux.microsoft.com (Postfix, from userid 1086) id C7ADA20B74D3; Mon, 26 Feb 2024 21:41:41 -0800 (PST) DKIM-Filter: OpenDKIM Filter v2.11.0 linux.microsoft.com C7ADA20B74D3 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linux.microsoft.com; s=default; t=1709012501; bh=u4Hpx6TXmnVLV5OjQVFA1oRAHgS0KFkdXS6GeSBxts8=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=jQKDUhVp+sEyotb1KoPguExni3oj9MEJwqOFFVt4eCJKHuPuUCpj0AcxEE9O1X558 nQ30vQ4eQxzYp0xDLM7KO0Xh5wZ0J0FpLLAyh5mkmZ5rU9FE/OmTsM6L/bSkL+mVnz pkbrxsdA+qOOCqlzMsQAB3+uOgLsGHLxI9woRFOo= From: Tyler Retzlaff To: dev@dpdk.org Cc: Ajit Khaparde , Andrew Boyer , Andrew Rybchenko , Bruce Richardson , Chenbo Xia , Chengwen Feng , Dariusz Sosnowski , David Christensen , Hyong Youb Kim , Jerin Jacob , Jie Hai , Jingjing Wu , John Daley , Kevin Laatz , Kiran Kumar K , Konstantin Ananyev , Maciej Czekaj , Matan Azrad , Maxime Coquelin , Nithin Dabilpuram , Ori Kam , Ruifeng Wang , Satha Rao , Somnath Kotur , Suanming Mou , Sunil Kumar Kori , Viacheslav Ovsiienko , Yisen Zhuang , Yuying Zhang , mb@smartsharesystems.com, Tyler Retzlaff Subject: [PATCH v6 19/23] examples/dma: use mbuf descriptor accessor Date: Mon, 26 Feb 2024 21:41:35 -0800 Message-Id: <1709012499-12813-20-git-send-email-roretzla@linux.microsoft.com> X-Mailer: git-send-email 1.8.3.1 In-Reply-To: <1709012499-12813-1-git-send-email-roretzla@linux.microsoft.com> References: <1706657173-26166-1-git-send-email-roretzla@linux.microsoft.com> <1709012499-12813-1-git-send-email-roretzla@linux.microsoft.com> X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org RTE_MARKER typedefs are a GCC extension unsupported by MSVC. Use new rte_mbuf_rx_descriptor_fields1 accessor that provides a compatible type pointer without using the marker field. Signed-off-by: Tyler Retzlaff --- examples/dma/dmafwd.c | 5 +++-- 1 file changed, 3 insertions(+), 2 deletions(-) diff --git a/examples/dma/dmafwd.c b/examples/dma/dmafwd.c index f4a0bff..54991f6 100644 --- a/examples/dma/dmafwd.c +++ b/examples/dma/dmafwd.c @@ -294,9 +294,10 @@ struct dma_bufs { pktmbuf_metadata_copy(const struct rte_mbuf *src, struct rte_mbuf *dst) { dst->data_off = src->data_off; - memcpy(&dst->rx_descriptor_fields1, &src->rx_descriptor_fields1, + memcpy(rte_mbuf_rx_descriptor_fields1(dst), + rte_mbuf_rx_descriptor_fields1((struct rte_mbuf *)(uintptr_t)src), offsetof(struct rte_mbuf, buf_len) - - offsetof(struct rte_mbuf, rx_descriptor_fields1)); + offsetof(struct rte_mbuf, data_off)); } /* Copy packet data */ From patchwork Tue Feb 27 05:41:36 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Tyler Retzlaff X-Patchwork-Id: 137314 X-Patchwork-Delegate: thomas@monjalon.net Return-Path: X-Original-To: patchwork@inbox.dpdk.org Delivered-To: patchwork@inbox.dpdk.org Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id 4114F43C03; Tue, 27 Feb 2024 06:44:10 +0100 (CET) Received: from mails.dpdk.org (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id DE36842EFA; Tue, 27 Feb 2024 06:42:37 +0100 (CET) Received: from linux.microsoft.com (linux.microsoft.com [13.77.154.182]) by mails.dpdk.org (Postfix) with ESMTP id 799F640295 for ; Tue, 27 Feb 2024 06:41:43 +0100 (CET) Received: by linux.microsoft.com (Postfix, from userid 1086) id D5FB720B74D4; Mon, 26 Feb 2024 21:41:41 -0800 (PST) DKIM-Filter: OpenDKIM Filter v2.11.0 linux.microsoft.com D5FB720B74D4 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linux.microsoft.com; s=default; t=1709012501; bh=aIAjZY6el0OtlUq+PW7VTBpFXg14OTbvRZVp4+nqP+0=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=FR3eGCpPoeMwgUP+ab5Bm4mXpKyDbGcSrZiqIqHqKVUHGx5u9AyUDcmC1cfdP1R0k lI3ga3/VxMhIEAxr/OcTvHDWawW9xbxn6NZM9FVaPb56owQEcMKiPYjjFYNgax8ya3 Qtuf+QtR41k6IwcyCOIM+mPaRUfSN7cuO/a8Fy8A= From: Tyler Retzlaff To: dev@dpdk.org Cc: Ajit Khaparde , Andrew Boyer , Andrew Rybchenko , Bruce Richardson , Chenbo Xia , Chengwen Feng , Dariusz Sosnowski , David Christensen , Hyong Youb Kim , Jerin Jacob , Jie Hai , Jingjing Wu , John Daley , Kevin Laatz , Kiran Kumar K , Konstantin Ananyev , Maciej Czekaj , Matan Azrad , Maxime Coquelin , Nithin Dabilpuram , Ori Kam , Ruifeng Wang , Satha Rao , Somnath Kotur , Suanming Mou , Sunil Kumar Kori , Viacheslav Ovsiienko , Yisen Zhuang , Yuying Zhang , mb@smartsharesystems.com, Tyler Retzlaff Subject: [PATCH v6 20/23] mbuf: remove and stop using rte marker fields Date: Mon, 26 Feb 2024 21:41:36 -0800 Message-Id: <1709012499-12813-21-git-send-email-roretzla@linux.microsoft.com> X-Mailer: git-send-email 1.8.3.1 In-Reply-To: <1709012499-12813-1-git-send-email-roretzla@linux.microsoft.com> References: <1706657173-26166-1-git-send-email-roretzla@linux.microsoft.com> <1709012499-12813-1-git-send-email-roretzla@linux.microsoft.com> X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org RTE_MARKER typedefs are a GCC extension unsupported by MSVC. Remove RTE_MARKER fields from rte_mbuf struct. Maintain alignment of fields after removed cacheline1 marker by placing C11 alignas(RTE_CACHE_LINE_MIN_SIZE). Update implementation of rte_mbuf_prefetch_part1() and rte_mbuf_prefetch_part2() inline functions calculate pointer for prefetch of cachline0 and cachline1 without using removed markers. Update static_assert of rte_mbuf struct fields to reference data_off and packet_type fields that occupy the original offsets of the marker fields. Signed-off-by: Tyler Retzlaff Acked-by: Konstantin Ananyev --- doc/guides/rel_notes/release_24_03.rst | 9 ++++++++ lib/mbuf/rte_mbuf.h | 4 ++-- lib/mbuf/rte_mbuf_core.h | 39 +++++++++++++--------------------- 3 files changed, 26 insertions(+), 26 deletions(-) diff --git a/doc/guides/rel_notes/release_24_03.rst b/doc/guides/rel_notes/release_24_03.rst index 879bb49..67750f2 100644 --- a/doc/guides/rel_notes/release_24_03.rst +++ b/doc/guides/rel_notes/release_24_03.rst @@ -156,6 +156,15 @@ Removed Items The application reserved statically defined logtypes ``RTE_LOGTYPE_USER1..RTE_LOGTYPE_USER8`` are still defined. +* mbuf: ``RTE_MARKER`` fields ``cacheline0`` ``cacheline1`` + ``rx_descriptor_fields1`` and ``RTE_MARKER64`` field ``rearm_data`` + have been removed from ``struct rte_mbuf``. + Prefetch of ``cacheline0`` and ``cacheline1`` may be achieved through + ``rte_mbuf_prefetch_part1()`` and ``rte_mbuf_prefetch_part2()`` inline + functions respectively. + Access to ``rearm_data`` and ``rx_descriptor_fields1`` should be + through new inline functions ``rte_mbuf_rearm_data()`` and + ``rte_mbuf_rx_descriptor_fields1()`` respectively. API Changes ----------- diff --git a/lib/mbuf/rte_mbuf.h b/lib/mbuf/rte_mbuf.h index aa7495b..61cda20 100644 --- a/lib/mbuf/rte_mbuf.h +++ b/lib/mbuf/rte_mbuf.h @@ -108,7 +108,7 @@ static inline void rte_mbuf_prefetch_part1(struct rte_mbuf *m) { - rte_prefetch0(&m->cacheline0); + rte_prefetch0(m); } /** @@ -126,7 +126,7 @@ rte_mbuf_prefetch_part2(struct rte_mbuf *m) { #if RTE_CACHE_LINE_SIZE == 64 - rte_prefetch0(&m->cacheline1); + rte_prefetch0(RTE_PTR_ADD(m, RTE_CACHE_LINE_MIN_SIZE)); #else RTE_SET_USED(m); #endif diff --git a/lib/mbuf/rte_mbuf_core.h b/lib/mbuf/rte_mbuf_core.h index 36551c2..4e06f15 100644 --- a/lib/mbuf/rte_mbuf_core.h +++ b/lib/mbuf/rte_mbuf_core.h @@ -18,6 +18,7 @@ #include #include +#include #include #include @@ -467,8 +468,6 @@ enum { * The generic rte_mbuf, containing a packet mbuf. */ struct rte_mbuf { - RTE_MARKER cacheline0; - void *buf_addr; /**< Virtual address of segment buffer. */ #if RTE_IOVA_IN_MBUF /** @@ -495,7 +494,6 @@ struct rte_mbuf { * To obtain a pointer to rearm_data use the rte_mbuf_rearm_data() * accessor instead of directly referencing through the data_off field. */ - RTE_MARKER64 rearm_data; uint16_t data_off; /** @@ -522,8 +520,6 @@ struct rte_mbuf { uint64_t ol_flags; /**< Offload features. */ /* remaining bytes are set on RX when pulling packet from descriptor */ - RTE_MARKER rx_descriptor_fields1; - /* * The packet type, which is the combination of outer/inner L2, L3, L4 * and tunnel types. The packet_type is about data really present in the @@ -607,8 +603,7 @@ struct rte_mbuf { struct rte_mempool *pool; /**< Pool from which mbuf was allocated. */ /* second cache line - fields only used in slow path or on TX */ - RTE_MARKER cacheline1 __rte_cache_min_aligned; - + alignas(RTE_CACHE_LINE_MIN_SIZE) #if RTE_IOVA_IN_MBUF /** * Next segment of scattered packet. Must be NULL in the last @@ -677,35 +672,31 @@ struct rte_mbuf { } __rte_cache_aligned; static_assert(!(offsetof(struct rte_mbuf, ol_flags) != - offsetof(struct rte_mbuf, rearm_data) + 8), "ol_flags"); -static_assert(!(offsetof(struct rte_mbuf, rearm_data) != - RTE_ALIGN(offsetof(struct rte_mbuf, rearm_data), 16)), "rearm_data"); + offsetof(struct rte_mbuf, data_off) + 8), "ol_flags"); static_assert(!(offsetof(struct rte_mbuf, data_off) != - offsetof(struct rte_mbuf, rearm_data)), "data_off"); -static_assert(!(offsetof(struct rte_mbuf, data_off) < - offsetof(struct rte_mbuf, rearm_data)), "data_off"); + RTE_ALIGN(offsetof(struct rte_mbuf, data_off), 16)), "data_off"); static_assert(!(offsetof(struct rte_mbuf, refcnt) < - offsetof(struct rte_mbuf, rearm_data)), "refcnt"); + offsetof(struct rte_mbuf, data_off)), "refcnt"); static_assert(!(offsetof(struct rte_mbuf, nb_segs) < - offsetof(struct rte_mbuf, rearm_data)), "nb_segs"); + offsetof(struct rte_mbuf, data_off)), "nb_segs"); static_assert(!(offsetof(struct rte_mbuf, port) < - offsetof(struct rte_mbuf, rearm_data)), "port"); + offsetof(struct rte_mbuf, data_off)), "port"); static_assert(!(offsetof(struct rte_mbuf, data_off) - - offsetof(struct rte_mbuf, rearm_data) > 6), "data_off"); + offsetof(struct rte_mbuf, data_off) > 6), "data_off"); static_assert(!(offsetof(struct rte_mbuf, refcnt) - - offsetof(struct rte_mbuf, rearm_data) > 6), "refcnt"); + offsetof(struct rte_mbuf, data_off) > 6), "refcnt"); static_assert(!(offsetof(struct rte_mbuf, nb_segs) - - offsetof(struct rte_mbuf, rearm_data) > 6), "nb_segs"); + offsetof(struct rte_mbuf, data_off) > 6), "nb_segs"); static_assert(!(offsetof(struct rte_mbuf, port) - - offsetof(struct rte_mbuf, rearm_data) > 6), "port"); + offsetof(struct rte_mbuf, data_off) > 6), "port"); static_assert(!(offsetof(struct rte_mbuf, pkt_len) != - offsetof(struct rte_mbuf, rx_descriptor_fields1) + 4), "pkt_len"); + offsetof(struct rte_mbuf, packet_type) + 4), "pkt_len"); static_assert(!(offsetof(struct rte_mbuf, data_len) != - offsetof(struct rte_mbuf, rx_descriptor_fields1) + 8), "data_len"); + offsetof(struct rte_mbuf, packet_type) + 8), "data_len"); static_assert(!(offsetof(struct rte_mbuf, vlan_tci) != - offsetof(struct rte_mbuf, rx_descriptor_fields1) + 10), "vlan_tci"); + offsetof(struct rte_mbuf, packet_type) + 10), "vlan_tci"); static_assert(!(offsetof(struct rte_mbuf, hash) != - offsetof(struct rte_mbuf, rx_descriptor_fields1) + 12), "hash"); + offsetof(struct rte_mbuf, packet_type) + 12), "hash"); /** * Function typedef of callback to free externally attached buffer. From patchwork Tue Feb 27 05:41:37 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Tyler Retzlaff X-Patchwork-Id: 137312 X-Patchwork-Delegate: thomas@monjalon.net Return-Path: X-Original-To: patchwork@inbox.dpdk.org Delivered-To: patchwork@inbox.dpdk.org Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id EF5A643C03; Tue, 27 Feb 2024 06:43:59 +0100 (CET) Received: from mails.dpdk.org (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id 5337342EEC; Tue, 27 Feb 2024 06:42:35 +0100 (CET) Received: from linux.microsoft.com (linux.microsoft.com [13.77.154.182]) by mails.dpdk.org (Postfix) with ESMTP id 513ED42E52 for ; Tue, 27 Feb 2024 06:41:43 +0100 (CET) Received: by linux.microsoft.com (Postfix, from userid 1086) id E59E820B74D5; Mon, 26 Feb 2024 21:41:41 -0800 (PST) DKIM-Filter: OpenDKIM Filter v2.11.0 linux.microsoft.com E59E820B74D5 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linux.microsoft.com; s=default; t=1709012501; bh=RI+sPyehAaCr1nyx2ukODaS7H5avnqu3oC6TRn9Afb4=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=G1vYEeFJyDLrO5Uiy/eKZ6ILsVeAuP9oiWCIN4C1qDbKdS0MxhXXJRpSGo5fyu72C YdSLeCSdoVPyH/cOe1NfjOKJySke+Y/fuGZF7c1/rjQLjpNCgM1OneQWnGSBtrWRV8 iJLcM3dfvOzBtXzj7fghMo3r/5i1kJLAdV+T2mS0= From: Tyler Retzlaff To: dev@dpdk.org Cc: Ajit Khaparde , Andrew Boyer , Andrew Rybchenko , Bruce Richardson , Chenbo Xia , Chengwen Feng , Dariusz Sosnowski , David Christensen , Hyong Youb Kim , Jerin Jacob , Jie Hai , Jingjing Wu , John Daley , Kevin Laatz , Kiran Kumar K , Konstantin Ananyev , Maciej Czekaj , Matan Azrad , Maxime Coquelin , Nithin Dabilpuram , Ori Kam , Ruifeng Wang , Satha Rao , Somnath Kotur , Suanming Mou , Sunil Kumar Kori , Viacheslav Ovsiienko , Yisen Zhuang , Yuying Zhang , mb@smartsharesystems.com, Tyler Retzlaff Subject: [PATCH v6 21/23] security: remove rte marker fields Date: Mon, 26 Feb 2024 21:41:37 -0800 Message-Id: <1709012499-12813-22-git-send-email-roretzla@linux.microsoft.com> X-Mailer: git-send-email 1.8.3.1 In-Reply-To: <1709012499-12813-1-git-send-email-roretzla@linux.microsoft.com> References: <1706657173-26166-1-git-send-email-roretzla@linux.microsoft.com> <1709012499-12813-1-git-send-email-roretzla@linux.microsoft.com> X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org RTE_MARKER typedefs are a GCC extension unsupported by MSVC. Remove RTE_MARKER fields from rte_mbuf struct. Maintain alignment of fields after removed cacheline1 marker by placing C11 alignas(RTE_CACHE_LINE_MIN_SIZE). Signed-off-by: Tyler Retzlaff --- doc/guides/rel_notes/release_24_03.rst | 3 +++ lib/security/rte_security_driver.h | 5 +++-- 2 files changed, 6 insertions(+), 2 deletions(-) diff --git a/doc/guides/rel_notes/release_24_03.rst b/doc/guides/rel_notes/release_24_03.rst index 67750f2..9469b92 100644 --- a/doc/guides/rel_notes/release_24_03.rst +++ b/doc/guides/rel_notes/release_24_03.rst @@ -166,6 +166,9 @@ Removed Items through new inline functions ``rte_mbuf_rearm_data()`` and ``rte_mbuf_rx_descriptor_fields1()`` respectively. +* security: ``RTE_MARKER`` fields ``cacheline0`` and ``cacheline1`` + have been removed from ``struct rte_security_session``. + API Changes ----------- diff --git a/lib/security/rte_security_driver.h b/lib/security/rte_security_driver.h index faa4074..18a1e3c 100644 --- a/lib/security/rte_security_driver.h +++ b/lib/security/rte_security_driver.h @@ -12,6 +12,8 @@ * RTE Security Common Definitions */ +#include + #ifdef __cplusplus extern "C" { #endif @@ -24,7 +26,6 @@ * Security session to be used by library for internal usage */ struct rte_security_session { - RTE_MARKER cacheline0; uint64_t opaque_data; /**< Opaque user defined data */ uint64_t fast_mdata; @@ -32,7 +33,7 @@ struct rte_security_session { rte_iova_t driver_priv_data_iova; /**< session private data IOVA address */ - RTE_MARKER cacheline1 __rte_cache_min_aligned; + alignas(RTE_CACHE_LINE_MIN_SIZE) uint8_t driver_priv_data[]; /**< Private session material, variable size (depends on driver) */ }; From patchwork Tue Feb 27 05:41:38 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Tyler Retzlaff X-Patchwork-Id: 137315 X-Patchwork-Delegate: thomas@monjalon.net Return-Path: X-Original-To: patchwork@inbox.dpdk.org Delivered-To: patchwork@inbox.dpdk.org Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id 675F443C03; Tue, 27 Feb 2024 06:44:15 +0100 (CET) Received: from mails.dpdk.org (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id 340B242EFF; Tue, 27 Feb 2024 06:42:39 +0100 (CET) Received: from linux.microsoft.com (linux.microsoft.com [13.77.154.182]) by mails.dpdk.org (Postfix) with ESMTP id 589FC402A7 for ; Tue, 27 Feb 2024 06:41:43 +0100 (CET) Received: by linux.microsoft.com (Postfix, from userid 1086) id 00BF020B74D6; Mon, 26 Feb 2024 21:41:41 -0800 (PST) DKIM-Filter: OpenDKIM Filter v2.11.0 linux.microsoft.com 00BF020B74D6 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linux.microsoft.com; s=default; t=1709012502; bh=PSGbNlsOKx0z5xok44avDsewOHyuQBSknxSR0S7SFJ4=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=pDXrqSxF6qJ3+D3XjuNdA++H8FYBIWJcpIjGU1Z1wSMgo1nl6X+/THMkf8SUbyneU ygroRs2S0BcYiljIB1vLe/epoCESdO3aZmQd0GUXrd4facGcA1iKnTR4JcVQgC5Rpw uLx2hReSncL7UBlLhUDFO6d+eBxG54ApcKRp30v4= From: Tyler Retzlaff To: dev@dpdk.org Cc: Ajit Khaparde , Andrew Boyer , Andrew Rybchenko , Bruce Richardson , Chenbo Xia , Chengwen Feng , Dariusz Sosnowski , David Christensen , Hyong Youb Kim , Jerin Jacob , Jie Hai , Jingjing Wu , John Daley , Kevin Laatz , Kiran Kumar K , Konstantin Ananyev , Maciej Czekaj , Matan Azrad , Maxime Coquelin , Nithin Dabilpuram , Ori Kam , Ruifeng Wang , Satha Rao , Somnath Kotur , Suanming Mou , Sunil Kumar Kori , Viacheslav Ovsiienko , Yisen Zhuang , Yuying Zhang , mb@smartsharesystems.com, Tyler Retzlaff Subject: [PATCH v6 22/23] cryptodev: remove rte marker fields Date: Mon, 26 Feb 2024 21:41:38 -0800 Message-Id: <1709012499-12813-23-git-send-email-roretzla@linux.microsoft.com> X-Mailer: git-send-email 1.8.3.1 In-Reply-To: <1709012499-12813-1-git-send-email-roretzla@linux.microsoft.com> References: <1706657173-26166-1-git-send-email-roretzla@linux.microsoft.com> <1709012499-12813-1-git-send-email-roretzla@linux.microsoft.com> X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org RTE_MARKER typedefs are a GCC extension unsupported by MSVC. Remove RTE_MARKER fields from rte_mbuf struct. Maintain alignment of fields after removed cacheline1 marker by placing C11 alignas(RTE_CACHE_LINE_MIN_SIZE). Signed-off-by: Tyler Retzlaff --- doc/guides/rel_notes/release_24_03.rst | 3 +++ lib/cryptodev/cryptodev_pmd.h | 5 +++-- 2 files changed, 6 insertions(+), 2 deletions(-) diff --git a/doc/guides/rel_notes/release_24_03.rst b/doc/guides/rel_notes/release_24_03.rst index 9469b92..2903f7e 100644 --- a/doc/guides/rel_notes/release_24_03.rst +++ b/doc/guides/rel_notes/release_24_03.rst @@ -169,6 +169,9 @@ Removed Items * security: ``RTE_MARKER`` fields ``cacheline0`` and ``cacheline1`` have been removed from ``struct rte_security_session``. +* cryptodev: ``RTE_MARKER`` fields ``cacheline0`` and ``cacheline1`` + have been removed from ``struct cryptodev_driver``. + API Changes ----------- diff --git a/lib/cryptodev/cryptodev_pmd.h b/lib/cryptodev/cryptodev_pmd.h index 0732b35..e676ef7 100644 --- a/lib/cryptodev/cryptodev_pmd.h +++ b/lib/cryptodev/cryptodev_pmd.h @@ -5,6 +5,8 @@ #ifndef _CRYPTODEV_PMD_H_ #define _CRYPTODEV_PMD_H_ +#include + #ifdef __cplusplus extern "C" { #endif @@ -139,7 +141,6 @@ struct cryptodev_driver { * has a fixed algo, key, op-type, digest_len etc. */ struct rte_cryptodev_sym_session { - RTE_MARKER cacheline0; uint64_t opaque_data; /**< Can be used for external metadata */ uint32_t sess_data_sz; @@ -151,7 +152,7 @@ struct rte_cryptodev_sym_session { rte_iova_t driver_priv_data_iova; /**< Session driver data IOVA address */ - RTE_MARKER cacheline1 __rte_cache_min_aligned; + alignas(RTE_CACHE_LINE_MIN_SIZE) /**< Second cache line - start of the driver session data */ uint8_t driver_priv_data[]; /**< Driver specific session data, variable size */ From patchwork Tue Feb 27 05:41:39 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Tyler Retzlaff X-Patchwork-Id: 137311 X-Patchwork-Delegate: thomas@monjalon.net Return-Path: X-Original-To: patchwork@inbox.dpdk.org Delivered-To: patchwork@inbox.dpdk.org Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id A9A8543C03; Tue, 27 Feb 2024 06:43:54 +0100 (CET) Received: from mails.dpdk.org (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id 1F9AC42EE4; Tue, 27 Feb 2024 06:42:34 +0100 (CET) Received: from linux.microsoft.com (linux.microsoft.com [13.77.154.182]) by mails.dpdk.org (Postfix) with ESMTP id 63D5C402B2 for ; Tue, 27 Feb 2024 06:41:43 +0100 (CET) Received: by linux.microsoft.com (Postfix, from userid 1086) id 0F00A20B74D7; Mon, 26 Feb 2024 21:41:41 -0800 (PST) DKIM-Filter: OpenDKIM Filter v2.11.0 linux.microsoft.com 0F00A20B74D7 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linux.microsoft.com; s=default; t=1709012502; bh=C9P2pYSYgt5nxhsfAnVBAKnM066HTxNkrTK90D5lQSw=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=i88H08O/bIhcIX8EbkydGSA+tiq7FKSOuPl2NH6r5MuX6FOumrR7fpazfJLjcYMgB uhK2OGcjAEBWOOoTAUtYpRMIFhIguDBUpW0TSLkGvm4VYR3DNv/SVYhoFwcot6NUez fut9dj3LE/EUiq63mN9JaZWIc93PYFSPfBmRXoN8= From: Tyler Retzlaff To: dev@dpdk.org Cc: Ajit Khaparde , Andrew Boyer , Andrew Rybchenko , Bruce Richardson , Chenbo Xia , Chengwen Feng , Dariusz Sosnowski , David Christensen , Hyong Youb Kim , Jerin Jacob , Jie Hai , Jingjing Wu , John Daley , Kevin Laatz , Kiran Kumar K , Konstantin Ananyev , Maciej Czekaj , Matan Azrad , Maxime Coquelin , Nithin Dabilpuram , Ori Kam , Ruifeng Wang , Satha Rao , Somnath Kotur , Suanming Mou , Sunil Kumar Kori , Viacheslav Ovsiienko , Yisen Zhuang , Yuying Zhang , mb@smartsharesystems.com, Tyler Retzlaff Subject: [PATCH v6 23/23] devtools: forbid new use of rte marker typedefs Date: Mon, 26 Feb 2024 21:41:39 -0800 Message-Id: <1709012499-12813-24-git-send-email-roretzla@linux.microsoft.com> X-Mailer: git-send-email 1.8.3.1 In-Reply-To: <1709012499-12813-1-git-send-email-roretzla@linux.microsoft.com> References: <1706657173-26166-1-git-send-email-roretzla@linux.microsoft.com> <1709012499-12813-1-git-send-email-roretzla@linux.microsoft.com> X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Add a check for RTE_MARKER{,8,16,32,64} forbidding new use. Signed-off-by: Tyler Retzlaff --- devtools/checkpatches.sh | 8 ++++++++ 1 file changed, 8 insertions(+) diff --git a/devtools/checkpatches.sh b/devtools/checkpatches.sh index e379700..ea31ba4 100755 --- a/devtools/checkpatches.sh +++ b/devtools/checkpatches.sh @@ -61,6 +61,14 @@ check_forbidden_additions() { # -f $(dirname $(readlink -f $0))/check-forbidden-tokens.awk \ "$1" || res=1 + # refrain from new uses of RTE_MARKER + awk -v FOLDERS="lib drivers" \ + -v EXPRESSIONS="RTE_MARKER(8|16|32|64)?" \ + -v RET_ON_FAIL=1 \ + -v MESSAGE='Using RTE_MARKER' \ + -f $(dirname $(readlink -f $0))/check-forbidden-tokens.awk \ + "$1" || res=1 + # no output on stdout or stderr awk -v FOLDERS="lib drivers" \ -v EXPRESSIONS="\\\ \\\