From patchwork Fri Apr 16 11:04:30 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Chengchang Tang X-Patchwork-Id: 91642 X-Patchwork-Delegate: ferruh.yigit@amd.com Return-Path: X-Original-To: patchwork@inbox.dpdk.org Delivered-To: patchwork@inbox.dpdk.org Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id 1FC0CA0C43; Fri, 16 Apr 2021 13:04:37 +0200 (CEST) Received: from [217.70.189.124] (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id C2F9C1415FB; Fri, 16 Apr 2021 13:04:25 +0200 (CEST) Received: from szxga07-in.huawei.com (szxga07-in.huawei.com [45.249.212.35]) by mails.dpdk.org (Postfix) with ESMTP id 1B0F8141163 for ; Fri, 16 Apr 2021 13:04:21 +0200 (CEST) Received: from DGGEMS414-HUB.china.huawei.com (unknown [172.30.72.60]) by szxga07-in.huawei.com (SkyGuard) with ESMTP id 4FMCvr1CWXzB1jR; Fri, 16 Apr 2021 19:02:00 +0800 (CST) Received: from localhost.localdomain (10.69.192.56) by DGGEMS414-HUB.china.huawei.com (10.3.19.214) with Microsoft SMTP Server id 14.3.498.0; Fri, 16 Apr 2021 19:04:11 +0800 From: Chengchang Tang To: CC: , , , Date: Fri, 16 Apr 2021 19:04:30 +0800 Message-ID: <1618571071-5927-2-git-send-email-tangchengchang@huawei.com> X-Mailer: git-send-email 2.7.4 In-Reply-To: <1618571071-5927-1-git-send-email-tangchengchang@huawei.com> References: <1618571071-5927-1-git-send-email-tangchengchang@huawei.com> MIME-Version: 1.0 X-Originating-IP: [10.69.192.56] X-CFilter-Loop: Reflected Subject: [dpdk-dev] [RFC 1/2] net/bonding: add Tx prepare for bonding X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Sender: "dev" To use the HW offloads capability (e.g. checksum and TSO) in the Tx direction, the upper-layer users need to call rte_eth_dev_prepare to do some adjustment to the packets before sending them (e.g. processing pseudo headers when Tx checksum offoad enabled). But, the tx_prepare callback of the bond driver is not implemented. Therefore, related offloads can not be used unless the upper layer users process the packet properly in their own application. But it is bad for the transplantability. However, it is difficult to design the tx_prepare callback for bonding driver. Because when a bonded device sends packets, the bonded device allocates the packets to different slave devices based on the real-time link status and bonding mode. That is, it is very difficult for the bonding device to determine which slave device's prepare function should be invoked. In addition, if the link status changes after the packets are prepared, the packets may fail to be sent because packets allocation may change. So, in this patch, the tx_prepare callback of bonding driver is not implemented. Instead, the prepare function of the slave device is added to the tx_burst callback. And a global variable is introduced to control whether the bonded device need call the rte_eth_tx_prepare. If upper-layer users need to use related offloads, they should enable the preparation function. In this way, the bonded device will call the rte_eth_tx_prepare for the fast path packets in the tx_burst callback. Note: The rte_eth_tx_prepare is not added to bond mode 3(Broadcast). This is because in broadcast mode, a packet needs to be sent by all slave ports. Different PMDs process the packets differently in tx_prepare. As a result, the sent packet may be incorrect. Signed-off-by: Chengchang Tang --- drivers/net/bonding/eth_bond_private.h | 1 + drivers/net/bonding/rte_eth_bond.h | 29 +++++++++++++++++++++++++++++ drivers/net/bonding/rte_eth_bond_api.c | 28 ++++++++++++++++++++++++++++ drivers/net/bonding/rte_eth_bond_pmd.c | 33 +++++++++++++++++++++++++++++---- drivers/net/bonding/version.map | 5 +++++ 5 files changed, 92 insertions(+), 4 deletions(-) -- 2.7.4 diff --git a/drivers/net/bonding/eth_bond_private.h b/drivers/net/bonding/eth_bond_private.h index 75fb8dc..72ec4a0 100644 --- a/drivers/net/bonding/eth_bond_private.h +++ b/drivers/net/bonding/eth_bond_private.h @@ -126,6 +126,7 @@ struct bond_dev_private { /**< Flag for whether MAC address is user defined or not */ uint8_t link_status_polling_enabled; + uint8_t tx_prepare_enabled; uint32_t link_status_polling_interval_ms; uint32_t link_down_delay_ms; diff --git a/drivers/net/bonding/rte_eth_bond.h b/drivers/net/bonding/rte_eth_bond.h index 874aa91..8ec09eb 100644 --- a/drivers/net/bonding/rte_eth_bond.h +++ b/drivers/net/bonding/rte_eth_bond.h @@ -343,6 +343,35 @@ rte_eth_bond_link_up_prop_delay_set(uint16_t bonded_port_id, int rte_eth_bond_link_up_prop_delay_get(uint16_t bonded_port_id); +/** + * Enable Tx prepare for bonded port + * + * To perform some HW offloads in the Tx direction, some PMDs need to call + * rte_eth_tx_prepare to do some adjustment for packets. This function + * enables packets preparation in the fast path for bonded device. + * + * @param bonded_port_id Bonded device id + * + * @return + * 0 on success, negative value otherwise. + */ +__rte_experimental +int +rte_eth_bond_tx_prepare_enable(uint16_t bonded_port_id); + +/** + * Disable Tx prepare for bonded port + * + * This function disables Tx prepare for the fast path packets. + * + * @param bonded_port_id Bonded device id + * + * @return + * 0 on success, negative value otherwise. + */ +__rte_experimental +int +rte_eth_bond_tx_prepare_disable(uint16_t bonded_port_id); #ifdef __cplusplus } diff --git a/drivers/net/bonding/rte_eth_bond_api.c b/drivers/net/bonding/rte_eth_bond_api.c index 17e6ff8..b04806a 100644 --- a/drivers/net/bonding/rte_eth_bond_api.c +++ b/drivers/net/bonding/rte_eth_bond_api.c @@ -1050,3 +1050,31 @@ rte_eth_bond_link_up_prop_delay_get(uint16_t bonded_port_id) return internals->link_up_delay_ms; } + +int +rte_eth_bond_tx_prepare_enable(uint16_t bonded_port_id) +{ + struct bond_dev_private *internals; + + if (valid_bonded_port_id(bonded_port_id) != 0) + return -1; + + internals = rte_eth_devices[bonded_port_id].data->dev_private; + internals->tx_prepare_enabled = 1; + + return 0; +} + +int +rte_eth_bond_tx_prepare_disable(uint16_t bonded_port_id) +{ + struct bond_dev_private *internals; + + if (valid_bonded_port_id(bonded_port_id) != 0) + return -1; + + internals = rte_eth_devices[bonded_port_id].data->dev_private; + internals->tx_prepare_enabled = 0; + + return 0; +} diff --git a/drivers/net/bonding/rte_eth_bond_pmd.c b/drivers/net/bonding/rte_eth_bond_pmd.c index 2e9cea5..3b7870f 100644 --- a/drivers/net/bonding/rte_eth_bond_pmd.c +++ b/drivers/net/bonding/rte_eth_bond_pmd.c @@ -606,8 +606,14 @@ bond_ethdev_tx_burst_round_robin(void *queue, struct rte_mbuf **bufs, /* Send packet burst on each slave device */ for (i = 0; i < num_of_slaves; i++) { if (slave_nb_pkts[i] > 0) { + int nb_prep_pkts = slave_nb_pkts[i]; + if (internals->tx_prepare_enabled) + nb_prep_pkts = rte_eth_tx_prepare(slaves[i], + bd_tx_q->queue_id, + slave_bufs[i], nb_prep_pkts); + num_tx_slave = rte_eth_tx_burst(slaves[i], bd_tx_q->queue_id, - slave_bufs[i], slave_nb_pkts[i]); + slave_bufs[i], nb_prep_pkts); /* if tx burst fails move packets to end of bufs */ if (unlikely(num_tx_slave < slave_nb_pkts[i])) { @@ -632,6 +638,7 @@ bond_ethdev_tx_burst_active_backup(void *queue, { struct bond_dev_private *internals; struct bond_tx_queue *bd_tx_q; + int nb_prep_pkts = nb_pkts; bd_tx_q = (struct bond_tx_queue *)queue; internals = bd_tx_q->dev_private; @@ -639,8 +646,13 @@ bond_ethdev_tx_burst_active_backup(void *queue, if (internals->active_slave_count < 1) return 0; + if (internals->tx_prepare_enabled) + nb_prep_pkts = + rte_eth_tx_prepare(internals->current_primary_port, + bd_tx_q->queue_id, bufs, nb_prep_pkts); + return rte_eth_tx_burst(internals->current_primary_port, bd_tx_q->queue_id, - bufs, nb_pkts); + bufs, nb_prep_pkts); } static inline uint16_t @@ -939,6 +951,7 @@ bond_ethdev_tx_burst_tlb(void *queue, struct rte_mbuf **bufs, uint16_t nb_pkts) } for (i = 0; i < num_of_slaves; i++) { + int nb_prep_pkts; rte_eth_macaddr_get(slaves[i], &active_slave_addr); for (j = num_tx_total; j < nb_pkts; j++) { if (j + 3 < nb_pkts) @@ -955,8 +968,14 @@ bond_ethdev_tx_burst_tlb(void *queue, struct rte_mbuf **bufs, uint16_t nb_pkts) #endif } + nb_prep_pkts = nb_pkts - num_tx_total; + if (internals->tx_prepare_enabled) + nb_prep_pkts = rte_eth_tx_prepare(slaves[i], + bd_tx_q->queue_id, bufs + num_tx_total, + nb_prep_pkts); + num_tx_total += rte_eth_tx_burst(slaves[i], bd_tx_q->queue_id, - bufs + num_tx_total, nb_pkts - num_tx_total); + bufs + num_tx_total, nb_prep_pkts); if (num_tx_total == nb_pkts) break; @@ -1159,12 +1178,18 @@ tx_burst_balance(void *queue, struct rte_mbuf **bufs, uint16_t nb_bufs, /* Send packet burst on each slave device */ for (i = 0; i < slave_count; i++) { + int nb_prep_pkts; if (slave_nb_bufs[i] == 0) continue; + nb_prep_pkts = slave_nb_bufs[i]; + if (internals->tx_prepare_enabled) + nb_prep_pkts = rte_eth_tx_prepare(slave_port_ids[i], + bd_tx_q->queue_id, slave_bufs[i], + nb_prep_pkts); slave_tx_count = rte_eth_tx_burst(slave_port_ids[i], bd_tx_q->queue_id, slave_bufs[i], - slave_nb_bufs[i]); + nb_prep_pkts); total_tx_count += slave_tx_count; diff --git a/drivers/net/bonding/version.map b/drivers/net/bonding/version.map index df81ee7..b642729 100644 --- a/drivers/net/bonding/version.map +++ b/drivers/net/bonding/version.map @@ -31,3 +31,8 @@ DPDK_21 { local: *; }; + +EXPERIMENTAL { + rte_eth_bond_tx_prepare_disable; + rte_eth_bond_tx_prepare_enable; +}; From patchwork Fri Apr 16 11:04:31 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Chengchang Tang X-Patchwork-Id: 91641 X-Patchwork-Delegate: ferruh.yigit@amd.com Return-Path: X-Original-To: patchwork@inbox.dpdk.org Delivered-To: patchwork@inbox.dpdk.org Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id BAC7BA0C43; Fri, 16 Apr 2021 13:04:30 +0200 (CEST) Received: from [217.70.189.124] (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id 6D9A71414D6; Fri, 16 Apr 2021 13:04:24 +0200 (CEST) Received: from szxga07-in.huawei.com (szxga07-in.huawei.com [45.249.212.35]) by mails.dpdk.org (Postfix) with ESMTP id 1800B141116 for ; Fri, 16 Apr 2021 13:04:21 +0200 (CEST) Received: from DGGEMS414-HUB.china.huawei.com (unknown [172.30.72.60]) by szxga07-in.huawei.com (SkyGuard) with ESMTP id 4FMCvr1QPVzB1jW; Fri, 16 Apr 2021 19:02:00 +0800 (CST) Received: from localhost.localdomain (10.69.192.56) by DGGEMS414-HUB.china.huawei.com (10.3.19.214) with Microsoft SMTP Server id 14.3.498.0; Fri, 16 Apr 2021 19:04:11 +0800 From: Chengchang Tang To: CC: , , , Date: Fri, 16 Apr 2021 19:04:31 +0800 Message-ID: <1618571071-5927-3-git-send-email-tangchengchang@huawei.com> X-Mailer: git-send-email 2.7.4 In-Reply-To: <1618571071-5927-1-git-send-email-tangchengchang@huawei.com> References: <1618571071-5927-1-git-send-email-tangchengchang@huawei.com> MIME-Version: 1.0 X-Originating-IP: [10.69.192.56] X-CFilter-Loop: Reflected Subject: [dpdk-dev] [RFC 2/2] app/testpmd: add cmd for bonding Tx prepare X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Sender: "dev" Add new command to support enable/disable Tx prepare on each slave of a bonded device. This helps to test some Tx HW offloads (e.g. checksum and TSO) for boned devices in testpmd. The related commands are as follows: set bonding tx_prepare [enable|disable] When this option is enabled, bonding driver would call rte_eth_dev_prepare to do some adjustment to the packets in the fast path to meet the device's requirement to turn on some HW offload(e.g. processing pseudo headers when Tx checksum offload enabled). This help bonded device to use more Tx offloads. Signed-off-by: Chengchang Tang --- app/test-pmd/cmdline.c | 66 +++++++++++++++++++++++++++++ doc/guides/testpmd_app_ug/testpmd_funcs.rst | 9 ++++ 2 files changed, 75 insertions(+) -- 2.7.4 diff --git a/app/test-pmd/cmdline.c b/app/test-pmd/cmdline.c index f44116b..2d1b3b6 100644 --- a/app/test-pmd/cmdline.c +++ b/app/test-pmd/cmdline.c @@ -647,6 +647,9 @@ static void cmd_help_long_parsed(void *parsed_result, "set bonding lacp dedicated_queues (enable|disable)\n" " Enable/disable dedicated queues for LACP control traffic.\n\n" + "set bonding tx_prepare (enable|disable)\n" + " Enable/disable tx_prepare for fast path traffic.\n\n" + #endif "set link-up port (port_id)\n" " Set link up for a port.\n\n" @@ -5886,6 +5889,68 @@ cmdline_parse_inst_t cmd_set_lacp_dedicated_queues = { } }; +/* *** SET BONDING TX_PREPARE *** */ +struct cmd_set_bonding_tx_prepare_result { + cmdline_fixed_string_t set; + cmdline_fixed_string_t bonding; + cmdline_fixed_string_t tx_prepare; + portid_t port_id; + cmdline_fixed_string_t mode; +}; + +static void cmd_set_bonding_tx_prepare_parsed(void *parsed_result, + __rte_unused struct cmdline *cl, + __rte_unused void *data) +{ + struct cmd_set_bonding_tx_prepare_result *res = parsed_result; + portid_t port_id = res->port_id; + + if (!strcmp(res->mode, "enable")) { + if (rte_eth_bond_tx_prepare_enable(port_id) == 0) + printf("Tx prepare for bonding device enabled\n"); + else + printf("Enabling bonding device Tx prepare " + "on port %d failed\n", port_id); + } else if (!strcmp(res->mode, "disable")) { + if (rte_eth_bond_tx_prepare_disable(port_id) == 0) + printf("Tx prepare for bonding device disabled\n"); + else + printf("Disabling bonding device Tx prepare " + "on port %d failed\n", port_id); + } +} + +cmdline_parse_token_string_t cmd_setbonding_tx_prepare_set = +TOKEN_STRING_INITIALIZER(struct cmd_set_bonding_tx_prepare_result, + set, "set"); +cmdline_parse_token_string_t cmd_setbonding_tx_prepare_bonding = +TOKEN_STRING_INITIALIZER(struct cmd_set_bonding_tx_prepare_result, + bonding, "bonding"); +cmdline_parse_token_string_t cmd_setbonding_tx_prepare_tx_prepare = +TOKEN_STRING_INITIALIZER(struct cmd_set_bonding_tx_prepare_result, + tx_prepare, "tx_prepare"); +cmdline_parse_token_num_t cmd_setbonding_tx_prepare_port_id = +TOKEN_NUM_INITIALIZER(struct cmd_set_bonding_tx_prepare_result, + port_id, RTE_UINT16); +cmdline_parse_token_string_t cmd_setbonding_tx_prepare_mode = +TOKEN_STRING_INITIALIZER(struct cmd_set_bonding_tx_prepare_result, + mode, "enable#disable"); + +cmdline_parse_inst_t cmd_set_bond_tx_prepare = { + .f = cmd_set_bonding_tx_prepare_parsed, + .help_str = "set bonding tx_prepare enable|disable: " + "Enable/disable tx_prepare for port_id", + .data = NULL, + .tokens = { + (void *)&cmd_setbonding_tx_prepare_set, + (void *)&cmd_setbonding_tx_prepare_bonding, + (void *)&cmd_setbonding_tx_prepare_tx_prepare, + (void *)&cmd_setbonding_tx_prepare_port_id, + (void *)&cmd_setbonding_tx_prepare_mode, + NULL + } +}; + /* *** SET BALANCE XMIT POLICY *** */ struct cmd_set_bonding_balance_xmit_policy_result { cmdline_fixed_string_t set; @@ -16966,6 +17031,7 @@ cmdline_parse_ctx_t main_ctx[] = { (cmdline_parse_inst_t *) &cmd_set_balance_xmit_policy, (cmdline_parse_inst_t *) &cmd_set_bond_mon_period, (cmdline_parse_inst_t *) &cmd_set_lacp_dedicated_queues, + (cmdline_parse_inst_t *) &cmd_set_bond_tx_prepare, (cmdline_parse_inst_t *) &cmd_set_bonding_agg_mode_policy, #endif (cmdline_parse_inst_t *)&cmd_vlan_offload, diff --git a/doc/guides/testpmd_app_ug/testpmd_funcs.rst b/doc/guides/testpmd_app_ug/testpmd_funcs.rst index 36f0a32..bdbf1ea 100644 --- a/doc/guides/testpmd_app_ug/testpmd_funcs.rst +++ b/doc/guides/testpmd_app_ug/testpmd_funcs.rst @@ -2590,6 +2590,15 @@ when in mode 4 (link-aggregation-802.3ad):: testpmd> set bonding lacp dedicated_queues (port_id) (enable|disable) +set bonding tx_prepare +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ + +Enable Tx prepare on bonding devices to help the slave devices prepare the +packets for some HW offloading (e.g. checksum and TSO):: + + testpmd> set bonding tx_prepare (port_id) (enable|disable) + + set bonding agg_mode ~~~~~~~~~~~~~~~~~~~~