From patchwork Sat Jun 6 13:50:57 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Xu, Ting" X-Patchwork-Id: 70919 X-Patchwork-Delegate: qi.z.zhang@intel.com Return-Path: X-Original-To: patchwork@inbox.dpdk.org Delivered-To: patchwork@inbox.dpdk.org Received: from dpdk.org (dpdk.org [92.243.14.124]) by inbox.dpdk.org (Postfix) with ESMTP id 50375A0350; Sat, 6 Jun 2020 07:52:12 +0200 (CEST) Received: from [92.243.14.124] (localhost [127.0.0.1]) by dpdk.org (Postfix) with ESMTP id 01F4E1D530; Sat, 6 Jun 2020 07:51:58 +0200 (CEST) Received: from mga18.intel.com (mga18.intel.com [134.134.136.126]) by dpdk.org (Postfix) with ESMTP id 887511D528 for ; Sat, 6 Jun 2020 07:51:55 +0200 (CEST) IronPort-SDR: Sw1Tsus2FbfvGkK2oA/KogTcgB2U4a07Zijko8n/oph5tY8kNffZPi5bjAov0tc33es9Ber902 ppJv4EEGFztQ== X-Amp-Result: SKIPPED(no attachment in message) X-Amp-File-Uploaded: False Received: from orsmga005.jf.intel.com ([10.7.209.41]) by orsmga106.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 05 Jun 2020 22:51:55 -0700 IronPort-SDR: Jdt2lZL0G4agyL4oYwSduv13EihvO8yvSHXQFFSL4pypntqEmxA1A8OQP4RnSDeXk7a+GMMnZW NPolQza8IDdw== X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="5.73,479,1583222400"; d="scan'208";a="446140818" Received: from dpdk-xuting-main.sh.intel.com ([10.67.117.84]) by orsmga005.jf.intel.com with ESMTP; 05 Jun 2020 22:51:53 -0700 From: Ting Xu To: dev@dpdk.org Cc: qi.z.zhang@intel.com, qiming.yang@intel.com, jingjing.wu@intel.com, beilei.xing@intel.com, marko.kovacevic@intel.com, john.mcnamara@intel.com Date: Sat, 6 Jun 2020 13:50:57 +0000 Message-Id: <20200606135058.112697-3-ting.xu@intel.com> X-Mailer: git-send-email 2.17.1 In-Reply-To: <20200606135058.112697-1-ting.xu@intel.com> References: <20200606135058.112697-1-ting.xu@intel.com> Subject: [dpdk-dev] [RFC PATCH 2/3] drivers/net: support multiple DCF instance X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Sender: "dev" From: Qi Zhang DCF will not explicitly call VIRTCHNL_OP_DCF_DISABLE during uninit if DCF PMD is required to support multiple instances co-exist. This is hinted by devargs "cap=mdcf". NOTE: in kernel PF, it still only take one VF as DCF, so there is only one active DPDK DCF instance has the DCF capability, more specific, when two DPDK DCF instances are probed, the later one will take over the DCF capability from the first one and the only way to disable DCF capability is by turn trust mode off on current active DCF. We assume at this use case, all the SR-IOV drivers are DCF itself, so the VSI update event is no need to be handled. Signed-off-by: Qi Zhang --- doc/guides/nics/ice.rst | 5 ++ drivers/net/iavf/iavf_ethdev.c | 2 +- drivers/net/ice/ice_dcf.c | 6 +- drivers/net/ice/ice_dcf.h | 1 + drivers/net/ice/ice_dcf_ethdev.c | 98 ++++++++++++++++------------- drivers/net/ice/ice_dcf_parent.c | 3 + drivers/net/ice/ice_switch_filter.c | 8 ++- 7 files changed, 74 insertions(+), 49 deletions(-) diff --git a/doc/guides/nics/ice.rst b/doc/guides/nics/ice.rst index 9a9f4a6bb..26f02f6bc 100644 --- a/doc/guides/nics/ice.rst +++ b/doc/guides/nics/ice.rst @@ -270,6 +270,11 @@ responses for the same from PF. 192.168.0.2', dst="192.168.0.3")/TCP(flags='S')/Raw(load='XXXXXXXXXX'), \ iface="enp24s0f0", count=10) +DCF will not explicitly call VIRTCHNL_OP_DCF_DISABLE during uninit if DCF PMD +is required to support multiple instances co-exist. When two DPDK DCF instances +are probed, the later one will take over the DCF capability from the first one. +This is hinted by devargs "cap=mdcf". + Sample Application Notes ------------------------ diff --git a/drivers/net/iavf/iavf_ethdev.c b/drivers/net/iavf/iavf_ethdev.c index e09efffd1..fe734e9c5 100644 --- a/drivers/net/iavf/iavf_ethdev.c +++ b/drivers/net/iavf/iavf_ethdev.c @@ -1495,7 +1495,7 @@ static int iavf_dcf_cap_check_handler(__rte_unused const char *key, const char *value, __rte_unused void *opaque) { - if (strcmp(value, "dcf")) + if (strcmp(value, "dcf") && strcmp(value, "mdcf")) return -1; return 0; diff --git a/drivers/net/ice/ice_dcf.c b/drivers/net/ice/ice_dcf.c index 6ec32d010..7dcf659e5 100644 --- a/drivers/net/ice/ice_dcf.c +++ b/drivers/net/ice/ice_dcf.c @@ -641,7 +641,8 @@ ice_dcf_init_hw(struct rte_eth_dev *eth_dev, struct ice_dcf_hw *hw) if (ice_dcf_get_vf_vsi_map(hw) < 0) { PMD_INIT_LOG(ERR, "Failed to get VF VSI map"); - ice_dcf_mode_disable(hw); + if (!hw->multi_inst) + ice_dcf_mode_disable(hw); goto err_alloc; } @@ -700,7 +701,8 @@ ice_dcf_uninit_hw(struct rte_eth_dev *eth_dev, struct ice_dcf_hw *hw) rte_intr_callback_unregister(intr_handle, ice_dcf_dev_interrupt_handler, hw); - ice_dcf_mode_disable(hw); + if (!hw->multi_inst) + ice_dcf_mode_disable(hw); iavf_shutdown_adminq(&hw->avf); rte_free(hw->arq_buf); diff --git a/drivers/net/ice/ice_dcf.h b/drivers/net/ice/ice_dcf.h index 7e4d48fc5..4d57759f6 100644 --- a/drivers/net/ice/ice_dcf.h +++ b/drivers/net/ice/ice_dcf.h @@ -50,6 +50,7 @@ struct ice_dcf_hw { uint16_t vsi_id; struct rte_eth_dev *eth_dev; + bool multi_inst; uint8_t *rss_lut; uint8_t *rss_key; uint64_t supported_rxdid; diff --git a/drivers/net/ice/ice_dcf_ethdev.c b/drivers/net/ice/ice_dcf_ethdev.c index a1b1ffb56..0ec9bd5c1 100644 --- a/drivers/net/ice/ice_dcf_ethdev.c +++ b/drivers/net/ice/ice_dcf_ethdev.c @@ -19,6 +19,7 @@ #include #include #include +#include #include @@ -878,11 +879,64 @@ static const struct eth_dev_ops ice_dcf_eth_dev_ops = { .filter_ctrl = ice_dcf_dev_filter_ctrl, }; +static int +ice_dcf_cap_check_handler(__rte_unused const char *key, + const char *value, void *opaque) +{ + bool *mi = opaque; + + if (!strcmp(value, "dcf")) { + *mi = 0; + return 0; + } + if (!strcmp(value, "mdcf")) { + *mi = 1; + return 0; + } + + return -1; +} + +static int +ice_dcf_cap_selected(struct ice_dcf_adapter *adapter, + struct rte_devargs *devargs) +{ + struct rte_kvargs *kvlist; + const char *key = "cap"; + int ret = 0; + + if (devargs == NULL) + return 0; + + kvlist = rte_kvargs_parse(devargs->args, NULL); + if (kvlist == NULL) + return 0; + + if (!rte_kvargs_count(kvlist, key)) + goto exit; + + /* dcf capability selected when there's a key-value pair: cap=dcf */ + if (rte_kvargs_process(kvlist, key, + ice_dcf_cap_check_handler, + &adapter->real_hw.multi_inst) < 0) + goto exit; + + ret = 1; + +exit: + rte_kvargs_free(kvlist); + return ret; +} + static int ice_dcf_dev_init(struct rte_eth_dev *eth_dev) { + struct rte_pci_device *pci_dev = RTE_DEV_TO_PCI(eth_dev->device); struct ice_dcf_adapter *adapter = eth_dev->data->dev_private; + if (!ice_dcf_cap_selected(adapter, pci_dev->device.devargs)) + return 1; + eth_dev->dev_ops = &ice_dcf_eth_dev_ops; eth_dev->rx_pkt_burst = ice_dcf_recv_pkts; eth_dev->tx_pkt_burst = ice_dcf_xmit_pkts; @@ -915,51 +969,9 @@ ice_dcf_dev_uninit(struct rte_eth_dev *eth_dev) return 0; } -static int -ice_dcf_cap_check_handler(__rte_unused const char *key, - const char *value, __rte_unused void *opaque) -{ - if (strcmp(value, "dcf")) - return -1; - - return 0; -} - -static int -ice_dcf_cap_selected(struct rte_devargs *devargs) -{ - struct rte_kvargs *kvlist; - const char *key = "cap"; - int ret = 0; - - if (devargs == NULL) - return 0; - - kvlist = rte_kvargs_parse(devargs->args, NULL); - if (kvlist == NULL) - return 0; - - if (!rte_kvargs_count(kvlist, key)) - goto exit; - - /* dcf capability selected when there's a key-value pair: cap=dcf */ - if (rte_kvargs_process(kvlist, key, - ice_dcf_cap_check_handler, NULL) < 0) - goto exit; - - ret = 1; - -exit: - rte_kvargs_free(kvlist); - return ret; -} - static int eth_ice_dcf_pci_probe(__rte_unused struct rte_pci_driver *pci_drv, struct rte_pci_device *pci_dev) { - if (!ice_dcf_cap_selected(pci_dev->device.devargs)) - return 1; - return rte_eth_dev_pci_generic_probe(pci_dev, sizeof(struct ice_dcf_adapter), ice_dcf_dev_init); @@ -985,4 +997,4 @@ static struct rte_pci_driver rte_ice_dcf_pmd = { RTE_PMD_REGISTER_PCI(net_ice_dcf, rte_ice_dcf_pmd); RTE_PMD_REGISTER_PCI_TABLE(net_ice_dcf, pci_id_ice_dcf_map); RTE_PMD_REGISTER_KMOD_DEP(net_ice_dcf, "* igb_uio | vfio-pci"); -RTE_PMD_REGISTER_PARAM_STRING(net_ice_dcf, "cap=dcf"); +RTE_PMD_REGISTER_PARAM_STRING(net_ice_dcf, "cap=dcf|mdcf"); diff --git a/drivers/net/ice/ice_dcf_parent.c b/drivers/net/ice/ice_dcf_parent.c index 8ad8bea1a..b4e200ccb 100644 --- a/drivers/net/ice/ice_dcf_parent.c +++ b/drivers/net/ice/ice_dcf_parent.c @@ -83,6 +83,9 @@ ice_dcf_vsi_update_service_handler(void *param) { struct ice_dcf_hw *hw = param; + if (hw->multi_inst) + return NULL; + usleep(ICE_DCF_VSI_UPDATE_SERVICE_INTERVAL); rte_spinlock_lock(&vsi_update_lock); diff --git a/drivers/net/ice/ice_switch_filter.c b/drivers/net/ice/ice_switch_filter.c index 993044f88..4b5fa6125 100644 --- a/drivers/net/ice/ice_switch_filter.c +++ b/drivers/net/ice/ice_switch_filter.c @@ -24,6 +24,7 @@ #include "ice_ethdev.h" #include "ice_generic_flow.h" #include "ice_dcf_ethdev.h" +#include "ice_dcf.h" #define MAX_QGRP_NUM_TYPE 7 @@ -358,9 +359,10 @@ ice_switch_destroy(struct ice_adapter *ad, struct rte_flow *flow, struct rte_flow_error *error) { + struct ice_dcf_hw *dcf_hw = ad->hw.aq_send_cmd_param; + struct ice_rule_query_data *filter_ptr; struct ice_hw *hw = &ad->hw; int ret; - struct ice_rule_query_data *filter_ptr; filter_ptr = (struct ice_rule_query_data *) flow->rule; @@ -374,7 +376,7 @@ ice_switch_destroy(struct ice_adapter *ad, } ret = ice_rem_adv_rule_by_id(hw, filter_ptr); - if (ret) { + if (ret && !(hw->dcf_enabled && dcf_hw->multi_inst)) { rte_flow_error_set(error, EINVAL, RTE_FLOW_ERROR_TYPE_HANDLE, NULL, "fail to destroy switch filter rule"); @@ -382,7 +384,7 @@ ice_switch_destroy(struct ice_adapter *ad, } rte_free(filter_ptr); - return ret; + return 0; } static void