From patchwork Tue Sep 22 07:06:30 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Somnath Kotur X-Patchwork-Id: 78237 X-Patchwork-Delegate: ajit.khaparde@broadcom.com Return-Path: X-Original-To: patchwork@inbox.dpdk.org Delivered-To: patchwork@inbox.dpdk.org Received: from dpdk.org (dpdk.org [92.243.14.124]) by inbox.dpdk.org (Postfix) with ESMTP id AB612A04DC; Tue, 22 Sep 2020 09:14:20 +0200 (CEST) Received: from [92.243.14.124] (localhost [127.0.0.1]) by dpdk.org (Postfix) with ESMTP id D63D71D8E4; Tue, 22 Sep 2020 09:13:32 +0200 (CEST) Received: from relay.smtp-ext.broadcom.com (unknown [192.19.221.30]) by dpdk.org (Postfix) with ESMTP id C03251D567 for ; Tue, 22 Sep 2020 09:13:21 +0200 (CEST) Received: from dhcp-10-123-153-55.dhcp.broadcom.net (dhcp-10-123-153-55.dhcp.broadcom.net [10.123.153.55]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (No client certificate requested) by relay.smtp-ext.broadcom.com (Postfix) with ESMTPS id 6230342A1A; Tue, 22 Sep 2020 00:13:20 -0700 (PDT) DKIM-Filter: OpenDKIM Filter v2.11.0 relay.smtp-ext.broadcom.com 6230342A1A DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=broadcom.com; s=dkimrelay; t=1600758800; bh=bKLCoksPXnv9igxZE6zKCNg04icmfny0bMT39LS7vSY=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=RB76LrWwU3asW6/36rclLK7HDf7v+a8CgRMcgdHNtrKnRehPNwsfAz1TQnnh3o8bo 30ep8VeuqO/bpS/2Cj7hALmlFHjONQdIPnzlEFxJ90wVrHXsZKc3vskt3LxfN15m5d PNMKeBfls8P4/9GUt7HigK2XGII5k00nAhhQB+4A= From: Somnath Kotur To: dev@dpdk.org Cc: ferruh.yigit@intel.com, Somnath Kotur , Michael Baucom , Venkat Duvvuru Date: Tue, 22 Sep 2020 12:36:30 +0530 Message-Id: <20200922070632.17706-7-somnath.kotur@broadcom.com> X-Mailer: git-send-email 2.28.0.450.g3a238e5 In-Reply-To: <20200922070632.17706-1-somnath.kotur@broadcom.com> References: <20200922070632.17706-1-somnath.kotur@broadcom.com> MIME-Version: 1.0 Subject: [dpdk-dev] [PATCH 6/8] net/bnxt: support for representors on remote host domain X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Sender: "dev" In the Stingray use case, Representors are conventionally run inside the SoC domain representing functions that are on the X86 domain. In order to support this mechanism of building representors for endpoints that are not in the same host domain, additional dev args have been in the PMD like so: rep-based-pf= rep-is-pf= where `rep-based-pf` specifies the physical index of the base PF that the Representor is derived off of. Since representor(s) can be created for endpoint PFs as well, rename struct bnxt_vf_representor to bnxt_representor and other such dev_ops and function names. devargs have also been extended to specify the exact CoS queue along with flow control enablement to be used for the conduit between the representor and the endpoint function. This is how a sample devargs would look with all the extended devargs -w 0000:06:02.0,host-based-truflow=1,representor=[1],rep-based-pf=8, rep-is-pf=1,rep-q-r2f=1,rep-fc-r2f=0,rep-q-f2r=1,rep-fc-f2r=1 Also, Call CFA_PAIR_ALLOC only in case of Stingray instead of CFA_VFR_ALLOC Signed-off-by: Somnath Kotur Reviewed-by: Michael Baucom Reviewed-by: Venkat Duvvuru --- drivers/net/bnxt/bnxt.h | 21 +- drivers/net/bnxt/bnxt_cpr.c | 4 +- drivers/net/bnxt/bnxt_ethdev.c | 349 ++++++++++++++++++++++++++++++-- drivers/net/bnxt/bnxt_hwrm.c | 95 +++++++++ drivers/net/bnxt/bnxt_hwrm.h | 4 + drivers/net/bnxt/bnxt_reps.c | 192 +++++++++++------- drivers/net/bnxt/bnxt_reps.h | 30 +-- drivers/net/bnxt/hsi_struct_def_dpdk.h | 248 +++++++++++++++++++++++ drivers/net/bnxt/tf_ulp/bnxt_ulp.c | 4 +- drivers/net/bnxt/tf_ulp/ulp_def_rules.c | 4 +- 10 files changed, 834 insertions(+), 117 deletions(-) diff --git a/drivers/net/bnxt/bnxt.h b/drivers/net/bnxt/bnxt.h index bb26599..3dded37 100644 --- a/drivers/net/bnxt/bnxt.h +++ b/drivers/net/bnxt/bnxt.h @@ -815,15 +815,26 @@ struct bnxt { /** * Structure to store private data for each VF representor instance */ -struct bnxt_vf_representor { +struct bnxt_representor { uint16_t switch_domain_id; uint16_t vf_id; +#define BNXT_REP_IS_PF BIT(0) +#define BNXT_REP_Q_R2F_VALID BIT(1) +#define BNXT_REP_Q_F2R_VALID BIT(2) +#define BNXT_REP_FC_R2F_VALID BIT(3) +#define BNXT_REP_FC_F2R_VALID BIT(4) + uint32_t flags; uint16_t fw_fid; #define BNXT_DFLT_VNIC_ID_INVALID 0xFFFF uint16_t dflt_vnic_id; uint16_t svif; uint16_t vfr_tx_cfa_action; uint32_t dpdk_port_id; + uint32_t rep_based_pf; + uint8_t rep_q_r2f; + uint8_t rep_q_f2r; + uint8_t rep_fc_r2f; + uint8_t rep_fc_f2r; /* Private data store of associated PF/Trusted VF */ struct rte_eth_dev *parent_dev; uint8_t mac_addr[RTE_ETHER_ADDR_LEN]; @@ -839,9 +850,11 @@ struct bnxt_vf_representor { uint64_t rx_drop_bytes[BNXT_MAX_VF_REP_RINGS]; }; +#define BNXT_REP_PF(vfr_bp) ((vfr_bp)->flags & BNXT_REP_IS_PF) + struct bnxt_vf_rep_tx_queue { struct bnxt_tx_queue *txq; - struct bnxt_vf_representor *bp; + struct bnxt_representor *bp; }; int bnxt_mtu_set_op(struct rte_eth_dev *eth_dev, uint16_t new_mtu); @@ -900,7 +913,7 @@ void bnxt_ulp_destroy_df_rules(struct bnxt *bp, bool global); int32_t bnxt_ulp_create_vfr_default_rules(struct rte_eth_dev *vfr_ethdev); int32_t -bnxt_ulp_delete_vfr_default_rules(struct bnxt_vf_representor *vfr); +bnxt_ulp_delete_vfr_default_rules(struct bnxt_representor *vfr); uint16_t bnxt_get_vnic_id(uint16_t port, enum bnxt_ulp_intf_type type); uint16_t bnxt_get_svif(uint16_t port_id, bool func_svif, enum bnxt_ulp_intf_type type); @@ -910,7 +923,7 @@ uint16_t bnxt_get_phy_port_id(uint16_t port); uint16_t bnxt_get_vport(uint16_t port); enum bnxt_ulp_intf_type bnxt_get_interface_type(uint16_t port); -int bnxt_vf_rep_dev_start_op(struct rte_eth_dev *eth_dev); +int bnxt_rep_dev_start_op(struct rte_eth_dev *eth_dev); void bnxt_cancel_fc_thread(struct bnxt *bp); void bnxt_flow_cnt_alarm_cb(void *arg); diff --git a/drivers/net/bnxt/bnxt_cpr.c b/drivers/net/bnxt/bnxt_cpr.c index 464ca8b..8311e26 100644 --- a/drivers/net/bnxt/bnxt_cpr.c +++ b/drivers/net/bnxt/bnxt_cpr.c @@ -51,7 +51,7 @@ bnxt_process_default_vnic_change(struct bnxt *bp, struct hwrm_async_event_cmpl *async_cmp) { uint16_t fid, vnic_state, parent_id, vf_fid, vf_id; - struct bnxt_vf_representor *vf_rep_bp; + struct bnxt_representor *vf_rep_bp; struct rte_eth_dev *eth_dev; bool vfr_found = false; uint32_t event_data; @@ -91,7 +91,7 @@ bnxt_process_default_vnic_change(struct bnxt *bp, if (!vfr_found) return; - bnxt_vf_rep_dev_start_op(eth_dev); + bnxt_rep_dev_start_op(eth_dev); } /* diff --git a/drivers/net/bnxt/bnxt_ethdev.c b/drivers/net/bnxt/bnxt_ethdev.c index 84eba0b..d7c8a3a 100644 --- a/drivers/net/bnxt/bnxt_ethdev.c +++ b/drivers/net/bnxt/bnxt_ethdev.c @@ -99,12 +99,24 @@ static const struct rte_pci_id bnxt_pci_id_map[] = { #define BNXT_DEVARG_FLOW_XSTAT "flow-xstat" #define BNXT_DEVARG_MAX_NUM_KFLOWS "max-num-kflows" #define BNXT_DEVARG_REPRESENTOR "representor" +#define BNXT_DEVARG_REP_BASED_PF "rep-based-pf" +#define BNXT_DEVARG_REP_IS_PF "rep-is-pf" +#define BNXT_DEVARG_REP_Q_R2F "rep-q-r2f" +#define BNXT_DEVARG_REP_Q_F2R "rep-q-f2r" +#define BNXT_DEVARG_REP_FC_R2F "rep-fc-r2f" +#define BNXT_DEVARG_REP_FC_F2R "rep-fc-f2r" static const char *const bnxt_dev_args[] = { BNXT_DEVARG_REPRESENTOR, BNXT_DEVARG_TRUFLOW, BNXT_DEVARG_FLOW_XSTAT, BNXT_DEVARG_MAX_NUM_KFLOWS, + BNXT_DEVARG_REP_BASED_PF, + BNXT_DEVARG_REP_IS_PF, + BNXT_DEVARG_REP_Q_R2F, + BNXT_DEVARG_REP_Q_F2R, + BNXT_DEVARG_REP_FC_R2F, + BNXT_DEVARG_REP_FC_F2R, NULL }; @@ -121,6 +133,36 @@ static const char *const bnxt_dev_args[] = { #define BNXT_DEVARG_FLOW_XSTAT_INVALID(flow_xstat) ((flow_xstat) > 1) /* + * rep_is_pf == false to indicate VF representor + * rep_is_pf == true to indicate PF representor + */ +#define BNXT_DEVARG_REP_IS_PF_INVALID(rep_is_pf) ((rep_is_pf) > 1) + +/* + * rep_based_pf == Physical index of the PF + */ +#define BNXT_DEVARG_REP_BASED_PF_INVALID(rep_based_pf) ((rep_based_pf) > 15) +/* + * rep_q_r2f == Logical COS Queue index for the rep to endpoint direction + */ +#define BNXT_DEVARG_REP_Q_R2F_INVALID(rep_q_r2f) ((rep_q_r2f) > 3) + +/* + * rep_q_f2r == Logical COS Queue index for the endpoint to rep direction + */ +#define BNXT_DEVARG_REP_Q_F2R_INVALID(rep_q_f2r) ((rep_q_f2r) > 3) + +/* + * rep_fc_r2f == Flow control for the representor to endpoint direction + */ +#define BNXT_DEVARG_REP_FC_R2F_INVALID(rep_fc_r2f) ((rep_fc_r2f) > 1) + +/* + * rep_fc_f2r == Flow control for the endpoint to representor direction + */ +#define BNXT_DEVARG_REP_FC_F2R_INVALID(rep_fc_f2r) ((rep_fc_f2r) > 1) + +/* * max_num_kflows must be >= 32 * and must be a power-of-2 supported value * return: 1 -> invalid @@ -1316,7 +1358,7 @@ static void bnxt_dev_stop_op(struct rte_eth_dev *eth_dev) rte_intr_disable(intr_handle); /* Stop the child representors for this device */ - bnxt_vf_rep_stop_all(bp); + bnxt_rep_stop_all(bp); /* delete the bnxt ULP port details */ bnxt_ulp_port_deinit(bp); @@ -3730,7 +3772,7 @@ bnxt_filter_ctrl_op(struct rte_eth_dev *dev, return -EIO; if (BNXT_ETH_DEV_IS_REPRESENTOR(dev)) { - struct bnxt_vf_representor *vfr = dev->data->dev_private; + struct bnxt_representor *vfr = dev->data->dev_private; bp = vfr->parent_dev->data->dev_private; /* parent is deleted while children are still valid */ if (!bp) { @@ -5186,7 +5228,7 @@ bnxt_get_svif(uint16_t port_id, bool func_svif, eth_dev = &rte_eth_devices[port_id]; if (BNXT_ETH_DEV_IS_REPRESENTOR(eth_dev)) { - struct bnxt_vf_representor *vfr = eth_dev->data->dev_private; + struct bnxt_representor *vfr = eth_dev->data->dev_private; if (!vfr) return 0; @@ -5210,7 +5252,7 @@ bnxt_get_vnic_id(uint16_t port, enum bnxt_ulp_intf_type type) eth_dev = &rte_eth_devices[port]; if (BNXT_ETH_DEV_IS_REPRESENTOR(eth_dev)) { - struct bnxt_vf_representor *vfr = eth_dev->data->dev_private; + struct bnxt_representor *vfr = eth_dev->data->dev_private; if (!vfr) return 0; @@ -5235,7 +5277,7 @@ bnxt_get_fw_func_id(uint16_t port, enum bnxt_ulp_intf_type type) eth_dev = &rte_eth_devices[port]; if (BNXT_ETH_DEV_IS_REPRESENTOR(eth_dev)) { - struct bnxt_vf_representor *vfr = eth_dev->data->dev_private; + struct bnxt_representor *vfr = eth_dev->data->dev_private; if (!vfr) return 0; @@ -5274,7 +5316,7 @@ bnxt_get_interface_type(uint16_t port) uint16_t bnxt_get_phy_port_id(uint16_t port_id) { - struct bnxt_vf_representor *vfr; + struct bnxt_representor *vfr; struct rte_eth_dev *eth_dev; struct bnxt *bp; @@ -5300,7 +5342,7 @@ bnxt_get_parif(uint16_t port_id, enum bnxt_ulp_intf_type type) eth_dev = &rte_eth_devices[port_id]; if (BNXT_ETH_DEV_IS_REPRESENTOR(eth_dev)) { - struct bnxt_vf_representor *vfr = eth_dev->data->dev_private; + struct bnxt_representor *vfr = eth_dev->data->dev_private; if (!vfr) return 0; @@ -5661,6 +5703,227 @@ bnxt_parse_devarg_max_num_kflows(__rte_unused const char *key, return 0; } +static int +bnxt_parse_devarg_rep_is_pf(__rte_unused const char *key, + const char *value, void *opaque_arg) +{ + struct bnxt_representor *vfr_bp = opaque_arg; + unsigned long rep_is_pf; + char *end = NULL; + + if (!value || !opaque_arg) { + PMD_DRV_LOG(ERR, + "Invalid parameter passed to rep_is_pf devargs.\n"); + return -EINVAL; + } + + rep_is_pf = strtoul(value, &end, 10); + if (end == NULL || *end != '\0' || + (rep_is_pf == ULONG_MAX && errno == ERANGE)) { + PMD_DRV_LOG(ERR, + "Invalid parameter passed to rep_is_pf devargs.\n"); + return -EINVAL; + } + + if (BNXT_DEVARG_REP_IS_PF_INVALID(rep_is_pf)) { + PMD_DRV_LOG(ERR, + "Invalid value passed to rep_is_pf devargs.\n"); + return -EINVAL; + } + + vfr_bp->flags |= rep_is_pf; + if (BNXT_REP_PF(vfr_bp)) + PMD_DRV_LOG(INFO, "PF representor\n"); + else + PMD_DRV_LOG(INFO, "VF representor\n"); + + return 0; +} + +static int +bnxt_parse_devarg_rep_based_pf(__rte_unused const char *key, + const char *value, void *opaque_arg) +{ + struct bnxt_representor *vfr_bp = opaque_arg; + unsigned long rep_based_pf; + char *end = NULL; + + if (!value || !opaque_arg) { + PMD_DRV_LOG(ERR, + "Invalid parameter passed to rep_based_pf " + "devargs.\n"); + return -EINVAL; + } + + rep_based_pf = strtoul(value, &end, 10); + if (end == NULL || *end != '\0' || + (rep_based_pf == ULONG_MAX && errno == ERANGE)) { + PMD_DRV_LOG(ERR, + "Invalid parameter passed to rep_based_pf " + "devargs.\n"); + return -EINVAL; + } + + if (BNXT_DEVARG_REP_BASED_PF_INVALID(rep_based_pf)) { + PMD_DRV_LOG(ERR, + "Invalid value passed to rep_based_pf devargs.\n"); + return -EINVAL; + } + + vfr_bp->rep_based_pf = rep_based_pf; + PMD_DRV_LOG(INFO, "rep-based-pf = %d\n", vfr_bp->rep_based_pf); + + return 0; +} + +static int +bnxt_parse_devarg_rep_q_r2f(__rte_unused const char *key, + const char *value, void *opaque_arg) +{ + struct bnxt_representor *vfr_bp = opaque_arg; + unsigned long rep_q_r2f; + char *end = NULL; + + if (!value || !opaque_arg) { + PMD_DRV_LOG(ERR, + "Invalid parameter passed to rep_q_r2f " + "devargs.\n"); + return -EINVAL; + } + + rep_q_r2f = strtoul(value, &end, 10); + if (end == NULL || *end != '\0' || + (rep_q_r2f == ULONG_MAX && errno == ERANGE)) { + PMD_DRV_LOG(ERR, + "Invalid parameter passed to rep_q_r2f " + "devargs.\n"); + return -EINVAL; + } + + if (BNXT_DEVARG_REP_Q_R2F_INVALID(rep_q_r2f)) { + PMD_DRV_LOG(ERR, + "Invalid value passed to rep_q_r2f devargs.\n"); + return -EINVAL; + } + + vfr_bp->rep_q_r2f = rep_q_r2f; + vfr_bp->flags |= BNXT_REP_Q_R2F_VALID; + PMD_DRV_LOG(INFO, "rep-q-r2f = %d\n", vfr_bp->rep_q_r2f); + + return 0; +} + +static int +bnxt_parse_devarg_rep_q_f2r(__rte_unused const char *key, + const char *value, void *opaque_arg) +{ + struct bnxt_representor *vfr_bp = opaque_arg; + unsigned long rep_q_f2r; + char *end = NULL; + + if (!value || !opaque_arg) { + PMD_DRV_LOG(ERR, + "Invalid parameter passed to rep_q_f2r " + "devargs.\n"); + return -EINVAL; + } + + rep_q_f2r = strtoul(value, &end, 10); + if (end == NULL || *end != '\0' || + (rep_q_f2r == ULONG_MAX && errno == ERANGE)) { + PMD_DRV_LOG(ERR, + "Invalid parameter passed to rep_q_f2r " + "devargs.\n"); + return -EINVAL; + } + + if (BNXT_DEVARG_REP_Q_F2R_INVALID(rep_q_f2r)) { + PMD_DRV_LOG(ERR, + "Invalid value passed to rep_q_f2r devargs.\n"); + return -EINVAL; + } + + vfr_bp->rep_q_f2r = rep_q_f2r; + vfr_bp->flags |= BNXT_REP_Q_F2R_VALID; + PMD_DRV_LOG(INFO, "rep-q-f2r = %d\n", vfr_bp->rep_q_f2r); + + return 0; +} + +static int +bnxt_parse_devarg_rep_fc_r2f(__rte_unused const char *key, + const char *value, void *opaque_arg) +{ + struct bnxt_representor *vfr_bp = opaque_arg; + unsigned long rep_fc_r2f; + char *end = NULL; + + if (!value || !opaque_arg) { + PMD_DRV_LOG(ERR, + "Invalid parameter passed to rep_fc_r2f " + "devargs.\n"); + return -EINVAL; + } + + rep_fc_r2f = strtoul(value, &end, 10); + if (end == NULL || *end != '\0' || + (rep_fc_r2f == ULONG_MAX && errno == ERANGE)) { + PMD_DRV_LOG(ERR, + "Invalid parameter passed to rep_fc_r2f " + "devargs.\n"); + return -EINVAL; + } + + if (BNXT_DEVARG_REP_FC_R2F_INVALID(rep_fc_r2f)) { + PMD_DRV_LOG(ERR, + "Invalid value passed to rep_fc_r2f devargs.\n"); + return -EINVAL; + } + + vfr_bp->flags |= BNXT_REP_FC_R2F_VALID; + vfr_bp->rep_fc_r2f = rep_fc_r2f; + PMD_DRV_LOG(INFO, "rep-fc-r2f = %lu\n", rep_fc_r2f); + + return 0; +} + +static int +bnxt_parse_devarg_rep_fc_f2r(__rte_unused const char *key, + const char *value, void *opaque_arg) +{ + struct bnxt_representor *vfr_bp = opaque_arg; + unsigned long rep_fc_f2r; + char *end = NULL; + + if (!value || !opaque_arg) { + PMD_DRV_LOG(ERR, + "Invalid parameter passed to rep_fc_f2r " + "devargs.\n"); + return -EINVAL; + } + + rep_fc_f2r = strtoul(value, &end, 10); + if (end == NULL || *end != '\0' || + (rep_fc_f2r == ULONG_MAX && errno == ERANGE)) { + PMD_DRV_LOG(ERR, + "Invalid parameter passed to rep_fc_f2r " + "devargs.\n"); + return -EINVAL; + } + + if (BNXT_DEVARG_REP_FC_F2R_INVALID(rep_fc_f2r)) { + PMD_DRV_LOG(ERR, + "Invalid value passed to rep_fc_f2r devargs.\n"); + return -EINVAL; + } + + vfr_bp->flags |= BNXT_REP_FC_F2R_VALID; + vfr_bp->rep_fc_f2r = rep_fc_f2r; + PMD_DRV_LOG(INFO, "rep-fc-f2r = %lu\n", rep_fc_f2r); + + return 0; +} + static void bnxt_parse_dev_args(struct bnxt *bp, struct rte_devargs *devargs) { @@ -5957,7 +6220,7 @@ static int bnxt_pci_remove_dev_with_reps(struct rte_eth_dev *eth_dev) continue; PMD_DRV_LOG(DEBUG, "BNXT Port:%d VFR pci remove\n", vf_rep_eth_dev->data->port_id); - rte_eth_dev_destroy(vf_rep_eth_dev, bnxt_vf_representor_uninit); + rte_eth_dev_destroy(vf_rep_eth_dev, bnxt_representor_uninit); } PMD_DRV_LOG(DEBUG, "BNXT Port:%d pci remove\n", eth_dev->data->port_id); @@ -6019,13 +6282,15 @@ static int bnxt_init_rep_info(struct bnxt *bp) static int bnxt_rep_port_probe(struct rte_pci_device *pci_dev, struct rte_eth_devargs eth_da, - struct rte_eth_dev *backing_eth_dev) + struct rte_eth_dev *backing_eth_dev, + const char *dev_args) { struct rte_eth_dev *vf_rep_eth_dev; char name[RTE_ETH_NAME_MAX_LEN]; struct bnxt *backing_bp; uint16_t num_rep; int i, ret = 0; + struct rte_kvargs *kvlist; num_rep = eth_da.nb_representor_ports; if (num_rep > BNXT_MAX_VF_REPS) { @@ -6056,7 +6321,7 @@ static int bnxt_rep_port_probe(struct rte_pci_device *pci_dev, return 0; for (i = 0; i < num_rep; i++) { - struct bnxt_vf_representor representor = { + struct bnxt_representor representor = { .vf_id = eth_da.representor_ports[i], .switch_domain_id = backing_bp->switch_domain_id, .parent_dev = backing_eth_dev @@ -6072,10 +6337,62 @@ static int bnxt_rep_port_probe(struct rte_pci_device *pci_dev, snprintf(name, sizeof(name), "net_%s_representor_%d", pci_dev->device.name, eth_da.representor_ports[i]); + kvlist = rte_kvargs_parse(dev_args, bnxt_dev_args); + if (kvlist) { + /* + * Handler for "rep_is_pf" devarg. + * Invoked as for ex: "-w 000:00:0d.0, + * rep-based-pf= rep-is-pf=" + */ + rte_kvargs_process(kvlist, BNXT_DEVARG_REP_IS_PF, + bnxt_parse_devarg_rep_is_pf, + (void *)&representor); + /* + * Handler for "rep_based_pf" devarg. + * Invoked as for ex: "-w 000:00:0d.0, + * rep-based-pf= rep-is-pf=" + */ + rte_kvargs_process(kvlist, BNXT_DEVARG_REP_BASED_PF, + bnxt_parse_devarg_rep_based_pf, + (void *)&representor); + /* + * Handler for "rep_based_pf" devarg. + * Invoked as for ex: "-w 000:00:0d.0, + * rep-based-pf= rep-is-pf=" + */ + rte_kvargs_process(kvlist, BNXT_DEVARG_REP_Q_R2F, + bnxt_parse_devarg_rep_q_r2f, + (void *)&representor); + /* + * Handler for "rep_based_pf" devarg. + * Invoked as for ex: "-w 000:00:0d.0, + * rep-based-pf= rep-is-pf=" + */ + rte_kvargs_process(kvlist, BNXT_DEVARG_REP_Q_F2R, + bnxt_parse_devarg_rep_q_f2r, + (void *)&representor); + /* + * Handler for "rep_based_pf" devarg. + * Invoked as for ex: "-w 000:00:0d.0, + * rep-based-pf= rep-is-pf=" + */ + rte_kvargs_process(kvlist, BNXT_DEVARG_REP_FC_R2F, + bnxt_parse_devarg_rep_fc_r2f, + (void *)&representor); + /* + * Handler for "rep_based_pf" devarg. + * Invoked as for ex: "-w 000:00:0d.0, + * rep-based-pf= rep-is-pf=" + */ + rte_kvargs_process(kvlist, BNXT_DEVARG_REP_FC_F2R, + bnxt_parse_devarg_rep_fc_f2r, + (void *)&representor); + } + ret = rte_eth_dev_create(&pci_dev->device, name, - sizeof(struct bnxt_vf_representor), + sizeof(struct bnxt_representor), NULL, NULL, - bnxt_vf_representor_init, + bnxt_representor_init, &representor); if (ret) { PMD_DRV_LOG(ERR, "failed to create bnxt vf " @@ -6092,10 +6409,11 @@ static int bnxt_rep_port_probe(struct rte_pci_device *pci_dev, } PMD_DRV_LOG(DEBUG, "BNXT Port:%d VFR pci probe\n", - backing_eth_dev->data->port_id); + backing_eth_dev->data->port_id); backing_bp->rep_info[representor.vf_id].vfr_eth_dev = vf_rep_eth_dev; backing_bp->num_reps++; + } return 0; @@ -6152,7 +6470,8 @@ static int bnxt_pci_probe(struct rte_pci_driver *pci_drv __rte_unused, return ret; /* probe representor ports now */ - ret = bnxt_rep_port_probe(pci_dev, eth_da, backing_eth_dev); + ret = bnxt_rep_port_probe(pci_dev, eth_da, backing_eth_dev, + pci_dev->device.devargs->args); return ret; } @@ -6173,7 +6492,7 @@ static int bnxt_pci_remove(struct rte_pci_device *pci_dev) if (rte_eal_process_type() == RTE_PROC_PRIMARY) { if (eth_dev->data->dev_flags & RTE_ETH_DEV_REPRESENTOR) return rte_eth_dev_destroy(eth_dev, - bnxt_vf_representor_uninit); + bnxt_representor_uninit); else return rte_eth_dev_destroy(eth_dev, bnxt_dev_uninit); diff --git a/drivers/net/bnxt/bnxt_hwrm.c b/drivers/net/bnxt/bnxt_hwrm.c index 04eb05e..f25cdf0 100644 --- a/drivers/net/bnxt/bnxt_hwrm.c +++ b/drivers/net/bnxt/bnxt_hwrm.c @@ -5556,3 +5556,98 @@ int bnxt_hwrm_cfa_vfr_free(struct bnxt *bp, uint16_t vf_idx) PMD_DRV_LOG(DEBUG, "VFR %d freed\n", vf_idx); return rc; } + +int bnxt_hwrm_first_vf_id_query(struct bnxt *bp, uint16_t fid, + uint16_t *first_vf_id) +{ + int rc = 0; + struct hwrm_func_qcaps_input req = {.req_type = 0 }; + struct hwrm_func_qcaps_output *resp = bp->hwrm_cmd_resp_addr; + + HWRM_PREP(&req, HWRM_FUNC_QCAPS, BNXT_USE_CHIMP_MB); + + req.fid = rte_cpu_to_le_16(fid); + + rc = bnxt_hwrm_send_message(bp, &req, sizeof(req), BNXT_USE_CHIMP_MB); + + HWRM_CHECK_RESULT(); + + if (first_vf_id) + *first_vf_id = rte_le_to_cpu_16(resp->first_vf_id); + + HWRM_UNLOCK(); + + return rc; +} + +int bnxt_hwrm_cfa_pair_alloc(struct bnxt *bp, struct bnxt_representor *rep_bp) +{ + struct hwrm_cfa_pair_alloc_output *resp = bp->hwrm_cmd_resp_addr; + struct hwrm_cfa_pair_alloc_input req = {0}; + int rc; + + if (!(BNXT_PF(bp) || BNXT_VF_IS_TRUSTED(bp))) { + PMD_DRV_LOG(DEBUG, + "Not a PF or trusted VF. Command not supported\n"); + return 0; + } + + HWRM_PREP(&req, HWRM_CFA_PAIR_ALLOC, BNXT_USE_CHIMP_MB); + req.pair_mode = HWRM_CFA_PAIR_FREE_INPUT_PAIR_MODE_REP2FN_TRUFLOW; + snprintf(req.pair_name, sizeof(req.pair_name), "%svfr%d", + bp->eth_dev->data->name, rep_bp->vf_id); + + req.pf_b_id = rte_cpu_to_le_32(rep_bp->rep_based_pf); + req.vf_b_id = rte_cpu_to_le_16(rep_bp->vf_id); + req.vf_a_id = rte_cpu_to_le_16(bp->fw_fid); + req.host_b_id = 1; /* TBD - Confirm if this is OK */ + + req.enables |= rep_bp->flags & BNXT_REP_Q_R2F_VALID ? + HWRM_CFA_PAIR_ALLOC_INPUT_ENABLES_Q_AB_VALID : 0; + req.enables |= rep_bp->flags & BNXT_REP_Q_F2R_VALID ? + HWRM_CFA_PAIR_ALLOC_INPUT_ENABLES_Q_BA_VALID : 0; + req.enables |= rep_bp->flags & BNXT_REP_FC_R2F_VALID ? + HWRM_CFA_PAIR_ALLOC_INPUT_ENABLES_FC_AB_VALID : 0; + req.enables |= rep_bp->flags & BNXT_REP_FC_F2R_VALID ? + HWRM_CFA_PAIR_ALLOC_INPUT_ENABLES_FC_BA_VALID : 0; + + req.q_ab = rep_bp->rep_q_r2f; + req.q_ba = rep_bp->rep_q_f2r; + req.fc_ab = rep_bp->rep_fc_r2f; + req.fc_ba = rep_bp->rep_fc_f2r; + + rc = bnxt_hwrm_send_message(bp, &req, sizeof(req), BNXT_USE_CHIMP_MB); + HWRM_CHECK_RESULT(); + + HWRM_UNLOCK(); + PMD_DRV_LOG(DEBUG, "%s %d allocated\n", + BNXT_REP_PF(rep_bp) ? "PFR" : "VFR", rep_bp->vf_id); + return rc; +} + +int bnxt_hwrm_cfa_pair_free(struct bnxt *bp, struct bnxt_representor *rep_bp) +{ + struct hwrm_cfa_pair_free_output *resp = bp->hwrm_cmd_resp_addr; + struct hwrm_cfa_pair_free_input req = {0}; + int rc; + + if (!(BNXT_PF(bp) || BNXT_VF_IS_TRUSTED(bp))) { + PMD_DRV_LOG(DEBUG, + "Not a PF or trusted VF. Command not supported\n"); + return 0; + } + + HWRM_PREP(&req, HWRM_CFA_PAIR_FREE, BNXT_USE_CHIMP_MB); + snprintf(req.pair_name, sizeof(req.pair_name), "%svfr%d", + bp->eth_dev->data->name, rep_bp->vf_id); + req.pf_b_id = rte_cpu_to_le_32(rep_bp->rep_based_pf); + req.vf_id = rte_cpu_to_le_16(rep_bp->vf_id); + req.pair_mode = HWRM_CFA_PAIR_FREE_INPUT_PAIR_MODE_REP2FN_TRUFLOW; + + rc = bnxt_hwrm_send_message(bp, &req, sizeof(req), BNXT_USE_CHIMP_MB); + HWRM_CHECK_RESULT(); + HWRM_UNLOCK(); + PMD_DRV_LOG(DEBUG, "%s %d freed\n", BNXT_REP_PF(rep_bp) ? "PFR" : "VFR", + rep_bp->vf_id); + return rc; +} diff --git a/drivers/net/bnxt/bnxt_hwrm.h b/drivers/net/bnxt/bnxt_hwrm.h index b5ec23a..77ae0b7 100644 --- a/drivers/net/bnxt/bnxt_hwrm.h +++ b/drivers/net/bnxt/bnxt_hwrm.h @@ -283,4 +283,8 @@ int bnxt_clear_one_vnic_filter(struct bnxt *bp, int bnxt_hwrm_cfa_vfr_alloc(struct bnxt *bp, uint16_t vf_idx); int bnxt_hwrm_cfa_vfr_free(struct bnxt *bp, uint16_t vf_idx); void bnxt_hwrm_free_vf_info(struct bnxt *bp); +int bnxt_hwrm_first_vf_id_query(struct bnxt *bp, uint16_t fid, + uint16_t *first_vf_id); +int bnxt_hwrm_cfa_pair_alloc(struct bnxt *bp, struct bnxt_representor *rep); +int bnxt_hwrm_cfa_pair_free(struct bnxt *bp, struct bnxt_representor *rep); #endif diff --git a/drivers/net/bnxt/bnxt_reps.c b/drivers/net/bnxt/bnxt_reps.c index d4d0a9e..b76487f 100644 --- a/drivers/net/bnxt/bnxt_reps.c +++ b/drivers/net/bnxt/bnxt_reps.c @@ -16,19 +16,19 @@ #include "ulp_port_db.h" #include "ulp_flow_db.h" -static const struct eth_dev_ops bnxt_vf_rep_dev_ops = { - .dev_infos_get = bnxt_vf_rep_dev_info_get_op, - .dev_configure = bnxt_vf_rep_dev_configure_op, - .dev_start = bnxt_vf_rep_dev_start_op, - .rx_queue_setup = bnxt_vf_rep_rx_queue_setup_op, - .rx_queue_release = bnxt_vf_rep_rx_queue_release_op, - .tx_queue_setup = bnxt_vf_rep_tx_queue_setup_op, - .tx_queue_release = bnxt_vf_rep_tx_queue_release_op, - .link_update = bnxt_vf_rep_link_update_op, - .dev_close = bnxt_vf_rep_dev_close_op, - .dev_stop = bnxt_vf_rep_dev_stop_op, - .stats_get = bnxt_vf_rep_stats_get_op, - .stats_reset = bnxt_vf_rep_stats_reset_op, +static const struct eth_dev_ops bnxt_rep_dev_ops = { + .dev_infos_get = bnxt_rep_dev_info_get_op, + .dev_configure = bnxt_rep_dev_configure_op, + .dev_start = bnxt_rep_dev_start_op, + .rx_queue_setup = bnxt_rep_rx_queue_setup_op, + .rx_queue_release = bnxt_rep_rx_queue_release_op, + .tx_queue_setup = bnxt_rep_tx_queue_setup_op, + .tx_queue_release = bnxt_rep_tx_queue_release_op, + .link_update = bnxt_rep_link_update_op, + .dev_close = bnxt_rep_dev_close_op, + .dev_stop = bnxt_rep_dev_stop_op, + .stats_get = bnxt_rep_stats_get_op, + .stats_reset = bnxt_rep_stats_reset_op, .filter_ctrl = bnxt_filter_ctrl_op }; @@ -39,7 +39,7 @@ bnxt_vfr_recv(uint16_t port_id, uint16_t queue_id, struct rte_mbuf *mbuf) struct bnxt_rx_ring_info *rep_rxr; struct bnxt_rx_queue *rep_rxq; struct rte_eth_dev *vfr_eth_dev; - struct bnxt_vf_representor *vfr_bp; + struct bnxt_representor *vfr_bp; uint16_t mask; uint8_t que; @@ -72,7 +72,7 @@ bnxt_vfr_recv(uint16_t port_id, uint16_t queue_id, struct rte_mbuf *mbuf) } static uint16_t -bnxt_vf_rep_rx_burst(void *rx_queue, +bnxt_rep_rx_burst(void *rx_queue, struct rte_mbuf **rx_pkts, uint16_t nb_pkts) { @@ -102,14 +102,14 @@ bnxt_vf_rep_rx_burst(void *rx_queue, } static uint16_t -bnxt_vf_rep_tx_burst(void *tx_queue, +bnxt_rep_tx_burst(void *tx_queue, struct rte_mbuf **tx_pkts, __rte_unused uint16_t nb_pkts) { struct bnxt_vf_rep_tx_queue *vfr_txq = tx_queue; struct bnxt_tx_queue *ptxq; struct bnxt *parent; - struct bnxt_vf_representor *vf_rep_bp; + struct bnxt_representor *vf_rep_bp; int qid; int rc; int i; @@ -138,7 +138,7 @@ bnxt_vf_rep_tx_burst(void *tx_queue, } static int -bnxt_get_dflt_vnic_svif(struct bnxt *bp, struct bnxt_vf_representor *vf_rep_bp) +bnxt_get_dflt_vnic_svif(struct bnxt *bp, struct bnxt_representor *vf_rep_bp) { struct bnxt_rep_info *rep_info; int rc; @@ -163,18 +163,26 @@ bnxt_get_dflt_vnic_svif(struct bnxt *bp, struct bnxt_vf_representor *vf_rep_bp) return rc; } -int bnxt_vf_representor_init(struct rte_eth_dev *eth_dev, void *params) +int bnxt_representor_init(struct rte_eth_dev *eth_dev, void *params) { - struct bnxt_vf_representor *vf_rep_bp = eth_dev->data->dev_private; - struct bnxt_vf_representor *rep_params = - (struct bnxt_vf_representor *)params; + struct bnxt_representor *vf_rep_bp = eth_dev->data->dev_private; + struct bnxt_representor *rep_params = + (struct bnxt_representor *)params; struct rte_eth_link *link; struct bnxt *parent_bp; + uint16_t first_vf_id; + int rc = 0; PMD_DRV_LOG(DEBUG, "BNXT Port:%d VFR init\n", eth_dev->data->port_id); vf_rep_bp->vf_id = rep_params->vf_id; vf_rep_bp->switch_domain_id = rep_params->switch_domain_id; vf_rep_bp->parent_dev = rep_params->parent_dev; + vf_rep_bp->rep_based_pf = rep_params->rep_based_pf; + vf_rep_bp->flags = rep_params->flags; + vf_rep_bp->rep_q_r2f = rep_params->rep_q_r2f; + vf_rep_bp->rep_q_f2r = rep_params->rep_q_f2r; + vf_rep_bp->rep_fc_r2f = rep_params->rep_fc_r2f; + vf_rep_bp->rep_fc_f2r = rep_params->rep_fc_f2r; eth_dev->data->dev_flags |= RTE_ETH_DEV_REPRESENTOR; eth_dev->data->representor_id = rep_params->vf_id; @@ -184,13 +192,13 @@ int bnxt_vf_representor_init(struct rte_eth_dev *eth_dev, void *params) sizeof(vf_rep_bp->mac_addr)); eth_dev->data->mac_addrs = (struct rte_ether_addr *)&vf_rep_bp->mac_addr; - eth_dev->dev_ops = &bnxt_vf_rep_dev_ops; + eth_dev->dev_ops = &bnxt_rep_dev_ops; /* No data-path, but need stub Rx/Tx functions to avoid crash * when testing with ovs-dpdk */ - eth_dev->rx_pkt_burst = bnxt_vf_rep_rx_burst; - eth_dev->tx_pkt_burst = bnxt_vf_rep_tx_burst; + eth_dev->rx_pkt_burst = bnxt_rep_rx_burst; + eth_dev->tx_pkt_burst = bnxt_rep_tx_burst; /* Link state. Inherited from PF or trusted VF */ parent_bp = vf_rep_bp->parent_dev->data->dev_private; link = &parent_bp->eth_dev->data->dev_link; @@ -211,17 +219,39 @@ int bnxt_vf_representor_init(struct rte_eth_dev *eth_dev, void *params) "Switch domain id %d: Representor Device %d init done\n", vf_rep_bp->switch_domain_id, vf_rep_bp->vf_id); - vf_rep_bp->fw_fid = rep_params->vf_id + parent_bp->first_vf_id; + if (vf_rep_bp->rep_based_pf) { + vf_rep_bp->fw_fid = vf_rep_bp->rep_based_pf + 1; + if (!(BNXT_REP_PF(vf_rep_bp))) { + /* VF representor for the remote PF,get first_vf_id */ + rc = bnxt_hwrm_first_vf_id_query(parent_bp, + vf_rep_bp->fw_fid, + &first_vf_id); + if (rc) + return rc; + if (first_vf_id == 0xffff) { + PMD_DRV_LOG(ERR, + "Invalid first_vf_id fid:%x\n", + vf_rep_bp->fw_fid); + return -EINVAL; + } + PMD_DRV_LOG(INFO, "first_vf_id = %x parent_fid:%x\n", + first_vf_id, vf_rep_bp->fw_fid); + vf_rep_bp->fw_fid = rep_params->vf_id + first_vf_id; + } + } else { + vf_rep_bp->fw_fid = rep_params->vf_id + parent_bp->first_vf_id; + } + PMD_DRV_LOG(INFO, "vf_rep->fw_fid = %d\n", vf_rep_bp->fw_fid); return 0; } -int bnxt_vf_representor_uninit(struct rte_eth_dev *eth_dev) +int bnxt_representor_uninit(struct rte_eth_dev *eth_dev) { struct bnxt *parent_bp; - struct bnxt_vf_representor *rep = - (struct bnxt_vf_representor *)eth_dev->data->dev_private; + struct bnxt_representor *rep = + (struct bnxt_representor *)eth_dev->data->dev_private; uint16_t vf_id; PMD_DRV_LOG(DEBUG, "BNXT Port:%d VFR uninit\n", eth_dev->data->port_id); @@ -244,11 +274,11 @@ int bnxt_vf_representor_uninit(struct rte_eth_dev *eth_dev) return 0; } -int bnxt_vf_rep_link_update_op(struct rte_eth_dev *eth_dev, int wait_to_compl) +int bnxt_rep_link_update_op(struct rte_eth_dev *eth_dev, int wait_to_compl) { struct bnxt *parent_bp; - struct bnxt_vf_representor *rep = - (struct bnxt_vf_representor *)eth_dev->data->dev_private; + struct bnxt_representor *rep = + (struct bnxt_representor *)eth_dev->data->dev_private; struct rte_eth_link *link; int rc; @@ -273,7 +303,7 @@ int bnxt_vf_rep_link_update_op(struct rte_eth_dev *eth_dev, int wait_to_compl) static int bnxt_tf_vfr_alloc(struct rte_eth_dev *vfr_ethdev) { int rc; - struct bnxt_vf_representor *vfr = vfr_ethdev->data->dev_private; + struct bnxt_representor *vfr = vfr_ethdev->data->dev_private; struct rte_eth_dev *parent_dev = vfr->parent_dev; struct bnxt *parent_bp = parent_dev->data->dev_private; @@ -299,7 +329,12 @@ static int bnxt_tf_vfr_alloc(struct rte_eth_dev *vfr_ethdev) } /* update the port id so you can backtrack to ethdev */ vfr->dpdk_port_id = vfr_ethdev->data->port_id; - rc = bnxt_hwrm_cfa_vfr_alloc(parent_bp, vfr->vf_id); + + if (BNXT_STINGRAY(parent_bp)) { + rc = bnxt_hwrm_cfa_pair_alloc(parent_bp, vfr); + } else { + rc = bnxt_hwrm_cfa_vfr_alloc(parent_bp, vfr->vf_id); + } if (rc) { BNXT_TF_DBG(ERR, "Failed in hwrm vfr alloc vfr:%u rc=%d\n", vfr->vf_id, rc); @@ -313,7 +348,7 @@ static int bnxt_tf_vfr_alloc(struct rte_eth_dev *vfr_ethdev) static int bnxt_vfr_alloc(struct rte_eth_dev *vfr_ethdev) { int rc = 0; - struct bnxt_vf_representor *vfr = vfr_ethdev->data->dev_private; + struct bnxt_representor *vfr = vfr_ethdev->data->dev_private; struct bnxt *parent_bp; if (!vfr || !vfr->parent_dev) { @@ -350,7 +385,7 @@ static int bnxt_vfr_alloc(struct rte_eth_dev *vfr_ethdev) return rc; } -static void bnxt_vf_rep_free_rx_mbufs(struct bnxt_vf_representor *rep_bp) +static void bnxt_rep_free_rx_mbufs(struct bnxt_representor *rep_bp) { struct bnxt_rx_queue *rxq; unsigned int i; @@ -361,9 +396,9 @@ static void bnxt_vf_rep_free_rx_mbufs(struct bnxt_vf_representor *rep_bp) } } -int bnxt_vf_rep_dev_start_op(struct rte_eth_dev *eth_dev) +int bnxt_rep_dev_start_op(struct rte_eth_dev *eth_dev) { - struct bnxt_vf_representor *rep_bp = eth_dev->data->dev_private; + struct bnxt_representor *rep_bp = eth_dev->data->dev_private; struct bnxt_rep_info *rep_info; struct bnxt *parent_bp; int rc; @@ -385,23 +420,23 @@ int bnxt_vf_rep_dev_start_op(struct rte_eth_dev *eth_dev) rc = bnxt_vfr_alloc(eth_dev); if (rc) { eth_dev->data->dev_link.link_status = 0; - bnxt_vf_rep_free_rx_mbufs(rep_bp); + bnxt_rep_free_rx_mbufs(rep_bp); return rc; } - eth_dev->rx_pkt_burst = &bnxt_vf_rep_rx_burst; - eth_dev->tx_pkt_burst = &bnxt_vf_rep_tx_burst; - bnxt_vf_rep_link_update_op(eth_dev, 1); + eth_dev->rx_pkt_burst = &bnxt_rep_rx_burst; + eth_dev->tx_pkt_burst = &bnxt_rep_tx_burst; + bnxt_rep_link_update_op(eth_dev, 1); return 0; } -static int bnxt_tf_vfr_free(struct bnxt_vf_representor *vfr) +static int bnxt_tf_vfr_free(struct bnxt_representor *vfr) { BNXT_TF_DBG(DEBUG, "BNXT Port:%d VFR ulp free\n", vfr->dpdk_port_id); return bnxt_ulp_delete_vfr_default_rules(vfr); } -static int bnxt_vfr_free(struct bnxt_vf_representor *vfr) +static int bnxt_vfr_free(struct bnxt_representor *vfr) { int rc = 0; struct bnxt *parent_bp; @@ -434,14 +469,17 @@ static int bnxt_vfr_free(struct bnxt_vf_representor *vfr) vfr->vf_id); vfr->vfr_tx_cfa_action = 0; - rc = bnxt_hwrm_cfa_vfr_free(parent_bp, vfr->vf_id); + if (BNXT_STINGRAY(parent_bp)) + rc = bnxt_hwrm_cfa_pair_free(parent_bp, vfr); + else + rc = bnxt_hwrm_cfa_vfr_free(parent_bp, vfr->vf_id); return rc; } -void bnxt_vf_rep_dev_stop_op(struct rte_eth_dev *eth_dev) +void bnxt_rep_dev_stop_op(struct rte_eth_dev *eth_dev) { - struct bnxt_vf_representor *vfr_bp = eth_dev->data->dev_private; + struct bnxt_representor *vfr_bp = eth_dev->data->dev_private; /* Avoid crashes as we are about to free queues */ eth_dev->rx_pkt_burst = &bnxt_dummy_recv_pkts; @@ -454,19 +492,19 @@ void bnxt_vf_rep_dev_stop_op(struct rte_eth_dev *eth_dev) if (eth_dev->data->dev_started) eth_dev->data->dev_link.link_status = 0; - bnxt_vf_rep_free_rx_mbufs(vfr_bp); + bnxt_rep_free_rx_mbufs(vfr_bp); } -void bnxt_vf_rep_dev_close_op(struct rte_eth_dev *eth_dev) +void bnxt_rep_dev_close_op(struct rte_eth_dev *eth_dev) { BNXT_TF_DBG(DEBUG, "BNXT Port:%d VFR close\n", eth_dev->data->port_id); - bnxt_vf_representor_uninit(eth_dev); + bnxt_representor_uninit(eth_dev); } -int bnxt_vf_rep_dev_info_get_op(struct rte_eth_dev *eth_dev, +int bnxt_rep_dev_info_get_op(struct rte_eth_dev *eth_dev, struct rte_eth_dev_info *dev_info) { - struct bnxt_vf_representor *rep_bp = eth_dev->data->dev_private; + struct bnxt_representor *rep_bp = eth_dev->data->dev_private; struct bnxt *parent_bp; unsigned int max_rx_rings; int rc = 0; @@ -510,9 +548,9 @@ int bnxt_vf_rep_dev_info_get_op(struct rte_eth_dev *eth_dev, return 0; } -int bnxt_vf_rep_dev_configure_op(__rte_unused struct rte_eth_dev *eth_dev) +int bnxt_rep_dev_configure_op(__rte_unused struct rte_eth_dev *eth_dev) { - struct bnxt_vf_representor *rep_bp = eth_dev->data->dev_private; + struct bnxt_representor *rep_bp = eth_dev->data->dev_private; PMD_DRV_LOG(DEBUG, "Representor dev_configure_op\n"); rep_bp->rx_queues = (void *)eth_dev->data->rx_queues; @@ -547,14 +585,14 @@ static int bnxt_init_rep_rx_ring(struct bnxt_rx_queue *rxq, return 0; } -int bnxt_vf_rep_rx_queue_setup_op(struct rte_eth_dev *eth_dev, - uint16_t queue_idx, - uint16_t nb_desc, - unsigned int socket_id, - __rte_unused const struct rte_eth_rxconf *rx_conf, - __rte_unused struct rte_mempool *mp) +int bnxt_rep_rx_queue_setup_op(struct rte_eth_dev *eth_dev, + uint16_t queue_idx, + uint16_t nb_desc, + unsigned int socket_id, + __rte_unused const struct rte_eth_rxconf *rx_conf, + __rte_unused struct rte_mempool *mp) { - struct bnxt_vf_representor *rep_bp = eth_dev->data->dev_private; + struct bnxt_representor *rep_bp = eth_dev->data->dev_private; struct bnxt *parent_bp = rep_bp->parent_dev->data->dev_private; struct bnxt_rx_queue *parent_rxq; struct bnxt_rx_queue *rxq; @@ -628,12 +666,12 @@ int bnxt_vf_rep_rx_queue_setup_op(struct rte_eth_dev *eth_dev, out: if (rxq) - bnxt_vf_rep_rx_queue_release_op(rxq); + bnxt_rep_rx_queue_release_op(rxq); return rc; } -void bnxt_vf_rep_rx_queue_release_op(void *rx_queue) +void bnxt_rep_rx_queue_release_op(void *rx_queue) { struct bnxt_rx_queue *rxq = (struct bnxt_rx_queue *)rx_queue; @@ -649,13 +687,13 @@ void bnxt_vf_rep_rx_queue_release_op(void *rx_queue) rte_free(rxq); } -int bnxt_vf_rep_tx_queue_setup_op(struct rte_eth_dev *eth_dev, - uint16_t queue_idx, - uint16_t nb_desc, - unsigned int socket_id, - __rte_unused const struct rte_eth_txconf *tx_conf) +int bnxt_rep_tx_queue_setup_op(struct rte_eth_dev *eth_dev, + uint16_t queue_idx, + uint16_t nb_desc, + unsigned int socket_id, + __rte_unused const struct rte_eth_txconf *tx_conf) { - struct bnxt_vf_representor *rep_bp = eth_dev->data->dev_private; + struct bnxt_representor *rep_bp = eth_dev->data->dev_private; struct bnxt *parent_bp = rep_bp->parent_dev->data->dev_private; struct bnxt_tx_queue *parent_txq, *txq; struct bnxt_vf_rep_tx_queue *vfr_txq; @@ -690,7 +728,7 @@ int bnxt_vf_rep_tx_queue_setup_op(struct rte_eth_dev *eth_dev, if (eth_dev->data->tx_queues) { vfr_txq = eth_dev->data->tx_queues[queue_idx]; - bnxt_vf_rep_tx_queue_release_op(vfr_txq); + bnxt_rep_tx_queue_release_op(vfr_txq); vfr_txq = NULL; } @@ -720,7 +758,7 @@ int bnxt_vf_rep_tx_queue_setup_op(struct rte_eth_dev *eth_dev, return 0; } -void bnxt_vf_rep_tx_queue_release_op(void *tx_queue) +void bnxt_rep_tx_queue_release_op(void *tx_queue) { struct bnxt_vf_rep_tx_queue *vfr_txq = tx_queue; @@ -731,10 +769,10 @@ void bnxt_vf_rep_tx_queue_release_op(void *tx_queue) rte_free(vfr_txq); } -int bnxt_vf_rep_stats_get_op(struct rte_eth_dev *eth_dev, +int bnxt_rep_stats_get_op(struct rte_eth_dev *eth_dev, struct rte_eth_stats *stats) { - struct bnxt_vf_representor *rep_bp = eth_dev->data->dev_private; + struct bnxt_representor *rep_bp = eth_dev->data->dev_private; int i; memset(stats, 0, sizeof(*stats)); @@ -755,9 +793,9 @@ int bnxt_vf_rep_stats_get_op(struct rte_eth_dev *eth_dev, return 0; } -int bnxt_vf_rep_stats_reset_op(struct rte_eth_dev *eth_dev) +int bnxt_rep_stats_reset_op(struct rte_eth_dev *eth_dev) { - struct bnxt_vf_representor *rep_bp = eth_dev->data->dev_private; + struct bnxt_representor *rep_bp = eth_dev->data->dev_private; int i; for (i = 0; i < BNXT_MAX_VF_REP_RINGS; i++) { @@ -770,7 +808,7 @@ int bnxt_vf_rep_stats_reset_op(struct rte_eth_dev *eth_dev) return 0; } -void bnxt_vf_rep_stop_all(struct bnxt *bp) +void bnxt_rep_stop_all(struct bnxt *bp) { uint16_t vf_id; struct rte_eth_dev *rep_eth_dev; @@ -783,6 +821,6 @@ void bnxt_vf_rep_stop_all(struct bnxt *bp) rep_eth_dev = bp->rep_info[vf_id].vfr_eth_dev; if (!rep_eth_dev) continue; - bnxt_vf_rep_dev_stop_op(rep_eth_dev); + bnxt_rep_dev_stop_op(rep_eth_dev); } } diff --git a/drivers/net/bnxt/bnxt_reps.h b/drivers/net/bnxt/bnxt_reps.h index 3239e03..3159f68 100644 --- a/drivers/net/bnxt/bnxt_reps.h +++ b/drivers/net/bnxt/bnxt_reps.h @@ -21,33 +21,33 @@ uint16_t bnxt_vfr_recv(uint16_t port_id, uint16_t queue_id, struct rte_mbuf *mbuf); -int bnxt_vf_representor_init(struct rte_eth_dev *eth_dev, void *params); -int bnxt_vf_representor_uninit(struct rte_eth_dev *eth_dev); -int bnxt_vf_rep_dev_info_get_op(struct rte_eth_dev *eth_dev, +int bnxt_representor_init(struct rte_eth_dev *eth_dev, void *params); +int bnxt_representor_uninit(struct rte_eth_dev *eth_dev); +int bnxt_rep_dev_info_get_op(struct rte_eth_dev *eth_dev, struct rte_eth_dev_info *dev_info); -int bnxt_vf_rep_dev_configure_op(struct rte_eth_dev *eth_dev); +int bnxt_rep_dev_configure_op(struct rte_eth_dev *eth_dev); -int bnxt_vf_rep_link_update_op(struct rte_eth_dev *eth_dev, int wait_to_compl); -int bnxt_vf_rep_dev_start_op(struct rte_eth_dev *eth_dev); -int bnxt_vf_rep_rx_queue_setup_op(struct rte_eth_dev *eth_dev, +int bnxt_rep_link_update_op(struct rte_eth_dev *eth_dev, int wait_to_compl); +int bnxt_rep_dev_start_op(struct rte_eth_dev *eth_dev); +int bnxt_rep_rx_queue_setup_op(struct rte_eth_dev *eth_dev, __rte_unused uint16_t queue_idx, __rte_unused uint16_t nb_desc, __rte_unused unsigned int socket_id, __rte_unused const struct rte_eth_rxconf * rx_conf, __rte_unused struct rte_mempool *mp); -int bnxt_vf_rep_tx_queue_setup_op(struct rte_eth_dev *eth_dev, +int bnxt_rep_tx_queue_setup_op(struct rte_eth_dev *eth_dev, __rte_unused uint16_t queue_idx, __rte_unused uint16_t nb_desc, __rte_unused unsigned int socket_id, __rte_unused const struct rte_eth_txconf * tx_conf); -void bnxt_vf_rep_rx_queue_release_op(void *rx_queue); -void bnxt_vf_rep_tx_queue_release_op(void *tx_queue); -void bnxt_vf_rep_dev_stop_op(struct rte_eth_dev *eth_dev); -void bnxt_vf_rep_dev_close_op(struct rte_eth_dev *eth_dev); -int bnxt_vf_rep_stats_get_op(struct rte_eth_dev *eth_dev, +void bnxt_rep_rx_queue_release_op(void *rx_queue); +void bnxt_rep_tx_queue_release_op(void *tx_queue); +void bnxt_rep_dev_stop_op(struct rte_eth_dev *eth_dev); +void bnxt_rep_dev_close_op(struct rte_eth_dev *eth_dev); +int bnxt_rep_stats_get_op(struct rte_eth_dev *eth_dev, struct rte_eth_stats *stats); -int bnxt_vf_rep_stats_reset_op(struct rte_eth_dev *eth_dev); -void bnxt_vf_rep_stop_all(struct bnxt *bp); +int bnxt_rep_stats_reset_op(struct rte_eth_dev *eth_dev); +void bnxt_rep_stop_all(struct bnxt *bp); #endif /* _BNXT_REPS_H_ */ diff --git a/drivers/net/bnxt/hsi_struct_def_dpdk.h b/drivers/net/bnxt/hsi_struct_def_dpdk.h index 7cb93b7..6ac49ee 100644 --- a/drivers/net/bnxt/hsi_struct_def_dpdk.h +++ b/drivers/net/bnxt/hsi_struct_def_dpdk.h @@ -42895,4 +42895,252 @@ struct hwrm_cfa_counter_qstats_output { uint8_t valid; } __rte_packed; +/*********************** + * hwrm_cfa_pair_alloc * + ***********************/ + + +/* hwrm_cfa_pair_alloc_input (size:576b/72B) */ +struct hwrm_cfa_pair_alloc_input { + /* The HWRM command request type. */ + uint16_t req_type; + /* + * The completion ring to send the completion event on. This should + * be the NQ ID returned from the `nq_alloc` HWRM command. + */ + uint16_t cmpl_ring; + /* + * The sequence ID is used by the driver for tracking multiple + * commands. This ID is treated as opaque data by the firmware and + * the value is returned in the `hwrm_resp_hdr` upon completion. + */ + uint16_t seq_id; + /* + * The target ID of the command: + * * 0x0-0xFFF8 - The function ID + * * 0xFFF8-0xFFFC, 0xFFFE - Reserved for internal processors + * * 0xFFFD - Reserved for user-space HWRM interface + * * 0xFFFF - HWRM + */ + uint16_t target_id; + /* + * A physical address pointer pointing to a host buffer that the + * command's response data will be written. This can be either a host + * physical address (HPA) or a guest physical address (GPA) and must + * point to a physically contiguous block of memory. + */ + uint64_t resp_addr; + /* + * Pair mode (0-vf2fn, 1-rep2fn, 2-rep2rep, 3-proxy, 4-pfpair, + * 5-rep2fn_mod, 6-rep2fn_modall, 7-rep2fn_truflow). + */ + uint16_t pair_mode; + /* Pair between VF on local host with PF or VF on specified host. */ + #define HWRM_CFA_PAIR_ALLOC_INPUT_PAIR_MODE_VF2FN \ + UINT32_C(0x0) + /* Pair between REP on local host with PF or VF on specified host. */ + #define HWRM_CFA_PAIR_ALLOC_INPUT_PAIR_MODE_REP2FN \ + UINT32_C(0x1) + /* Pair between REP on local host with REP on specified host. */ + #define HWRM_CFA_PAIR_ALLOC_INPUT_PAIR_MODE_REP2REP \ + UINT32_C(0x2) + /* Pair for the proxy interface. */ + #define HWRM_CFA_PAIR_ALLOC_INPUT_PAIR_MODE_PROXY \ + UINT32_C(0x3) + /* Pair for the PF interface. */ + #define HWRM_CFA_PAIR_ALLOC_INPUT_PAIR_MODE_PFPAIR \ + UINT32_C(0x4) + /* Modify existing rep2fn pair and move pair to new PF. */ + #define HWRM_CFA_PAIR_ALLOC_INPUT_PAIR_MODE_REP2FN_MOD \ + UINT32_C(0x5) + /* Modify exsiting rep2fn pairs paired with same PF and move pairs to new PF. */ + #define HWRM_CFA_PAIR_ALLOC_INPUT_PAIR_MODE_REP2FN_MODALL \ + UINT32_C(0x6) + /* Truflow pair between REP on local host with PF or VF on specified host. */ + #define HWRM_CFA_PAIR_ALLOC_INPUT_PAIR_MODE_REP2FN_TRUFLOW \ + UINT32_C(0x7) + #define HWRM_CFA_PAIR_ALLOC_INPUT_PAIR_MODE_LAST \ + HWRM_CFA_PAIR_ALLOC_INPUT_PAIR_MODE_REP2FN_TRUFLOW + /* Logical VF number (range: 0 -> MAX_VFS -1). */ + uint16_t vf_a_id; + /* Logical Host (0xff-local host). */ + uint8_t host_b_id; + /* Logical PF (0xff-PF for command channel). */ + uint8_t pf_b_id; + /* Logical VF number (range: 0 -> MAX_VFS -1). */ + uint16_t vf_b_id; + /* Loopback port (0xff-internal loopback), valid for mode-3. */ + uint8_t port_id; + /* Priority used for encap of loopback packets valid for mode-3. */ + uint8_t pri; + /* New PF for rep2fn modify, valid for mode 5. */ + uint16_t new_pf_fid; + uint32_t enables; + /* + * This bit must be '1' for the q_ab field to be + * configured. + */ + #define HWRM_CFA_PAIR_ALLOC_INPUT_ENABLES_Q_AB_VALID UINT32_C(0x1) + /* + * This bit must be '1' for the q_ba field to be + * configured. + */ + #define HWRM_CFA_PAIR_ALLOC_INPUT_ENABLES_Q_BA_VALID UINT32_C(0x2) + /* + * This bit must be '1' for the fc_ab field to be + * configured. + */ + #define HWRM_CFA_PAIR_ALLOC_INPUT_ENABLES_FC_AB_VALID UINT32_C(0x4) + /* + * This bit must be '1' for the fc_ba field to be + * configured. + */ + #define HWRM_CFA_PAIR_ALLOC_INPUT_ENABLES_FC_BA_VALID UINT32_C(0x8) + /* VF Pair name (32 byte string). */ + char pair_name[32]; + /* + * The q_ab value specifies the logical index of the TX/RX CoS + * queue to be assigned for traffic in the A to B direction of + * the interface pair. The default value is 0. + */ + uint8_t q_ab; + /* + * The q_ba value specifies the logical index of the TX/RX CoS + * queue to be assigned for traffic in the B to A direction of + * the interface pair. The default value is 1. + */ + uint8_t q_ba; + /* + * Specifies whether RX ring flow control is disabled (0) or enabled + * (1) in the A to B direction. The default value is 0, meaning that + * packets will be dropped when the B-side RX rings are full. + */ + uint8_t fc_ab; + /* + * Specifies whether RX ring flow control is disabled (0) or enabled + * (1) in the B to A direction. The default value is 1, meaning that + * the RX CoS queue will be flow controlled when the A-side RX rings + * are full. + */ + uint8_t fc_ba; + uint8_t unused_1[4]; +} __rte_packed; + +/* hwrm_cfa_pair_alloc_output (size:192b/24B) */ +struct hwrm_cfa_pair_alloc_output { + /* The specific error status for the command. */ + uint16_t error_code; + /* The HWRM command request type. */ + uint16_t req_type; + /* The sequence ID from the original command. */ + uint16_t seq_id; + /* The length of the response data in number of bytes. */ + uint16_t resp_len; + /* Only valid for modes 1 and 2. */ + uint16_t rx_cfa_code_a; + /* Only valid for modes 1 and 2. */ + uint16_t tx_cfa_action_a; + /* Only valid for mode 2. */ + uint16_t rx_cfa_code_b; + /* Only valid for mode 2. */ + uint16_t tx_cfa_action_b; + uint8_t unused_0[7]; + /* + * This field is used in Output records to indicate that the output + * is completely written to RAM. This field should be read as '1' + * to indicate that the output has been completely written. + * When writing a command completion or response to an internal processor, + * the order of writes has to be such that this field is written last. + */ + uint8_t valid; +} __rte_packed; + +/********************** + * hwrm_cfa_pair_free * + **********************/ + + +/* hwrm_cfa_pair_free_input (size:448b/56B) */ +struct hwrm_cfa_pair_free_input { + /* The HWRM command request type. */ + uint16_t req_type; + /* + * The completion ring to send the completion event on. This should + * be the NQ ID returned from the `nq_alloc` HWRM command. + */ + uint16_t cmpl_ring; + /* + * The sequence ID is used by the driver for tracking multiple + * commands. This ID is treated as opaque data by the firmware and + * the value is returned in the `hwrm_resp_hdr` upon completion. + */ + uint16_t seq_id; + /* + * The target ID of the command: + * * 0x0-0xFFF8 - The function ID + * * 0xFFF8-0xFFFC, 0xFFFE - Reserved for internal processors + * * 0xFFFD - Reserved for user-space HWRM interface + * * 0xFFFF - HWRM + */ + uint16_t target_id; + /* + * A physical address pointer pointing to a host buffer that the + * command's response data will be written. This can be either a host + * physical address (HPA) or a guest physical address (GPA) and must + * point to a physically contiguous block of memory. + */ + uint64_t resp_addr; + /* VF Pair name (32 byte string). */ + char pair_name[32]; + /* Logical PF (0xff-PF for command channel). */ + uint8_t pf_b_id; + uint8_t unused_0[3]; + /* Logical VF number (range: 0 -> MAX_VFS -1). */ + uint16_t vf_id; + /* + * Pair mode (0-vf2fn, 1-rep2fn, 2-rep2rep, 3-proxy, 4-pfpair, + * 5-rep2fn_mod, 6-rep2fn_modall, 7-rep2fn_truflow). + */ + uint16_t pair_mode; + /* Pair between VF on local host with PF or VF on specified host. */ + #define HWRM_CFA_PAIR_FREE_INPUT_PAIR_MODE_VF2FN UINT32_C(0x0) + /* Pair between REP on local host with PF or VF on specified host. */ + #define HWRM_CFA_PAIR_FREE_INPUT_PAIR_MODE_REP2FN UINT32_C(0x1) + /* Pair between REP on local host with REP on specified host. */ + #define HWRM_CFA_PAIR_FREE_INPUT_PAIR_MODE_REP2REP UINT32_C(0x2) + /* Pair for the proxy interface. */ + #define HWRM_CFA_PAIR_FREE_INPUT_PAIR_MODE_PROXY UINT32_C(0x3) + /* Pair for the PF interface. */ + #define HWRM_CFA_PAIR_FREE_INPUT_PAIR_MODE_PFPAIR UINT32_C(0x4) + /* Modify existing rep2fn pair and move pair to new PF. */ + #define HWRM_CFA_PAIR_FREE_INPUT_PAIR_MODE_REP2FN_MOD UINT32_C(0x5) + /* Modify existing rep2fn pairs paired with same PF and move pairs to new PF. */ + #define HWRM_CFA_PAIR_FREE_INPUT_PAIR_MODE_REP2FN_MODALL UINT32_C(0x6) + /* Truflow pair between REP on local host with PF or VF on specified host. */ + #define HWRM_CFA_PAIR_FREE_INPUT_PAIR_MODE_REP2FN_TRUFLOW UINT32_C(0x7) + #define HWRM_CFA_PAIR_FREE_INPUT_PAIR_MODE_LAST \ + HWRM_CFA_PAIR_FREE_INPUT_PAIR_MODE_REP2FN_TRUFLOW +} __rte_packed; + +/* hwrm_cfa_pair_free_output (size:128b/16B) */ +struct hwrm_cfa_pair_free_output { + /* The specific error status for the command. */ + uint16_t error_code; + /* The HWRM command request type. */ + uint16_t req_type; + /* The sequence ID from the original command. */ + uint16_t seq_id; + /* The length of the response data in number of bytes. */ + uint16_t resp_len; + uint8_t unused_0[7]; + /* + * This field is used in Output records to indicate that the output + * is completely written to RAM. This field should be read as '1' + * to indicate that the output has been completely written. + * When writing a command completion or response to an internal processor, + * the order of writes has to be such that this field is written last. + */ + uint8_t valid; +} __rte_packed; + #endif /* _HSI_STRUCT_DEF_DPDK_H_ */ diff --git a/drivers/net/bnxt/tf_ulp/bnxt_ulp.c b/drivers/net/bnxt/tf_ulp/bnxt_ulp.c index e8927f6..762fc0c 100644 --- a/drivers/net/bnxt/tf_ulp/bnxt_ulp.c +++ b/drivers/net/bnxt/tf_ulp/bnxt_ulp.c @@ -648,7 +648,7 @@ bnxt_ulp_destroy_vfr_default_rules(struct bnxt *bp, bool global) struct bnxt_ulp_vfr_rule_info *info; uint8_t port_id; struct rte_eth_dev *vfr_eth_dev; - struct bnxt_vf_representor *vfr_bp; + struct bnxt_representor *vfr_bp; if (!BNXT_TRUFLOW_EN(bp) || BNXT_ETH_DEV_IS_REPRESENTOR(bp->eth_dev)) return; @@ -1139,7 +1139,7 @@ bnxt_ulp_eth_dev_ptr2_cntxt_get(struct rte_eth_dev *dev) struct bnxt *bp = (struct bnxt *)dev->data->dev_private; if (BNXT_ETH_DEV_IS_REPRESENTOR(dev)) { - struct bnxt_vf_representor *vfr = dev->data->dev_private; + struct bnxt_representor *vfr = dev->data->dev_private; bp = vfr->parent_dev->data->dev_private; } diff --git a/drivers/net/bnxt/tf_ulp/ulp_def_rules.c b/drivers/net/bnxt/tf_ulp/ulp_def_rules.c index 2d0c3bc..f421e2e 100644 --- a/drivers/net/bnxt/tf_ulp/ulp_def_rules.c +++ b/drivers/net/bnxt/tf_ulp/ulp_def_rules.c @@ -536,7 +536,7 @@ int32_t bnxt_ulp_create_vfr_default_rules(struct rte_eth_dev *vfr_ethdev) { struct bnxt_ulp_vfr_rule_info *info; - struct bnxt_vf_representor *vfr = vfr_ethdev->data->dev_private; + struct bnxt_representor *vfr = vfr_ethdev->data->dev_private; struct rte_eth_dev *parent_dev = vfr->parent_dev; struct bnxt *bp = parent_dev->data->dev_private; uint16_t vfr_port_id = vfr_ethdev->data->port_id; @@ -596,7 +596,7 @@ bnxt_ulp_create_vfr_default_rules(struct rte_eth_dev *vfr_ethdev) } int32_t -bnxt_ulp_delete_vfr_default_rules(struct bnxt_vf_representor *vfr) +bnxt_ulp_delete_vfr_default_rules(struct bnxt_representor *vfr) { struct bnxt_ulp_vfr_rule_info *info; struct rte_eth_dev *parent_dev = vfr->parent_dev;