From patchwork Tue Apr 27 13:39:10 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Wang, Haiyue" X-Patchwork-Id: 92262 X-Patchwork-Delegate: david.marchand@redhat.com Return-Path: X-Original-To: patchwork@inbox.dpdk.org Delivered-To: patchwork@inbox.dpdk.org Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id 12839A0A02; Tue, 27 Apr 2021 16:05:26 +0200 (CEST) Received: from [217.70.189.124] (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id 9AFE441254; Tue, 27 Apr 2021 16:05:23 +0200 (CEST) Received: from mga18.intel.com (mga18.intel.com [134.134.136.126]) by mails.dpdk.org (Postfix) with ESMTP id 27389410FC for ; Tue, 27 Apr 2021 16:05:19 +0200 (CEST) IronPort-SDR: ROhZWyZ7Rnn++F/nQjzQEm+7odU5ap9Lz1jg+pI18Wp8tGWRevRQLuxQhHk9GtaT21JsaQ79Uz SZBvr/s8MEGA== X-IronPort-AV: E=McAfee;i="6200,9189,9967"; a="183994830" X-IronPort-AV: E=Sophos;i="5.82,254,1613462400"; d="scan'208";a="183994830" Received: from fmsmga002.fm.intel.com ([10.253.24.26]) by orsmga106.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 27 Apr 2021 07:05:19 -0700 IronPort-SDR: EzyfrhZ3PVeA/sH3hwxfUFq6pERadpmjVn2NYh8keNLRO168RZGrgyTaJW14W51iRDuK1QUfHZ 07UL/tr2n1NA== X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="5.82,254,1613462400"; d="scan'208";a="457656359" Received: from npg-dpdk-haiyue-2.sh.intel.com ([10.67.119.63]) by fmsmga002.fm.intel.com with ESMTP; 27 Apr 2021 07:05:17 -0700 From: Haiyue Wang To: dev@dpdk.org Cc: qi.z.zhang@intel.com, liang-min.wang@intel.com, david.marchand@redhat.com, Haiyue Wang , Ray Kinsella , Neil Horman , Gaetan Rivet Date: Tue, 27 Apr 2021 21:39:10 +0800 Message-Id: <20210427133912.261993-2-haiyue.wang@intel.com> X-Mailer: git-send-email 2.31.1 In-Reply-To: <20210427133912.261993-1-haiyue.wang@intel.com> References: <20210421050243.130585-1-haiyue.wang@intel.com> <20210427133912.261993-1-haiyue.wang@intel.com> MIME-Version: 1.0 Subject: [dpdk-dev] [PATCH v4 1/3] bus/pci: set PCI master in command register X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Sender: "dev" Add the API to set 'Bus Master Enable' bit to be enabled or disabled in the PCI command register. Signed-off-by: Haiyue Wang Acked-by: Ray Kinsella --- drivers/bus/pci/pci_common.c | 28 ++++++++++++++++++++++++++++ drivers/bus/pci/rte_bus_pci.h | 14 ++++++++++++++ drivers/bus/pci/version.map | 3 +++ lib/pci/rte_pci.h | 4 ++++ 4 files changed, 49 insertions(+) diff --git a/drivers/bus/pci/pci_common.c b/drivers/bus/pci/pci_common.c index ee7f96635..35d7d092d 100644 --- a/drivers/bus/pci/pci_common.c +++ b/drivers/bus/pci/pci_common.c @@ -746,6 +746,34 @@ rte_pci_find_ext_capability(struct rte_pci_device *dev, uint32_t cap) return 0; } +int +rte_pci_set_bus_master(struct rte_pci_device *dev, bool enable) +{ + uint16_t old_cmd, cmd; + + if (rte_pci_read_config(dev, &old_cmd, sizeof(old_cmd), + RTE_PCI_COMMAND) < 0) { + RTE_LOG(ERR, EAL, "error in reading PCI command register\n"); + return -1; + } + + if (enable) + cmd = old_cmd | RTE_PCI_COMMAND_MASTER; + else + cmd = old_cmd & ~RTE_PCI_COMMAND_MASTER; + + if (cmd == old_cmd) + return 0; + + if (rte_pci_write_config(dev, &cmd, sizeof(cmd), + RTE_PCI_COMMAND) < 0) { + RTE_LOG(ERR, EAL, "error in writing PCI command register\n"); + return -1; + } + + return 0; +} + struct rte_pci_bus rte_pci_bus = { .bus = { .scan = rte_pci_scan, diff --git a/drivers/bus/pci/rte_bus_pci.h b/drivers/bus/pci/rte_bus_pci.h index 64886b473..976c33c92 100644 --- a/drivers/bus/pci/rte_bus_pci.h +++ b/drivers/bus/pci/rte_bus_pci.h @@ -249,6 +249,20 @@ void rte_pci_dump(FILE *f); __rte_experimental off_t rte_pci_find_ext_capability(struct rte_pci_device *dev, uint32_t cap); +/** + * Enables/Disables Bus Master for device's PCI command register. + * + * @param dev + * A pointer to rte_pci_device structure. + * @param enable + * Enable or disable Bus Master. + * + * @return + * 0 on success, -1 on error in PCI config space read/write. + */ +__rte_experimental +int rte_pci_set_bus_master(struct rte_pci_device *dev, bool enable); + /** * Register a PCI driver. * diff --git a/drivers/bus/pci/version.map b/drivers/bus/pci/version.map index f33ed0abd..c6e5f797c 100644 --- a/drivers/bus/pci/version.map +++ b/drivers/bus/pci/version.map @@ -21,4 +21,7 @@ EXPERIMENTAL { global: rte_pci_find_ext_capability; + + # added in 21.05 + rte_pci_set_bus_master; }; diff --git a/lib/pci/rte_pci.h b/lib/pci/rte_pci.h index a8f8e404a..1f33d687f 100644 --- a/lib/pci/rte_pci.h +++ b/lib/pci/rte_pci.h @@ -32,6 +32,10 @@ extern "C" { #define RTE_PCI_VENDOR_ID 0x00 /* 16 bits */ #define RTE_PCI_DEVICE_ID 0x02 /* 16 bits */ +#define RTE_PCI_COMMAND 0x04 /* 16 bits */ + +/* PCI Command Register */ +#define RTE_PCI_COMMAND_MASTER 0x4 /* Bus Master Enable */ /* PCI Express capability registers */ #define RTE_PCI_EXP_DEVCTL 8 /* Device Control */ From patchwork Tue Apr 27 13:39:11 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Wang, Haiyue" X-Patchwork-Id: 92263 X-Patchwork-Delegate: david.marchand@redhat.com Return-Path: X-Original-To: patchwork@inbox.dpdk.org Delivered-To: patchwork@inbox.dpdk.org Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id 389A9A0A02; Tue, 27 Apr 2021 16:05:33 +0200 (CEST) Received: from [217.70.189.124] (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id C134F4125F; Tue, 27 Apr 2021 16:05:24 +0200 (CEST) Received: from mga18.intel.com (mga18.intel.com [134.134.136.126]) by mails.dpdk.org (Postfix) with ESMTP id 4E73741253 for ; Tue, 27 Apr 2021 16:05:22 +0200 (CEST) IronPort-SDR: w1yBUz8pW3DRPUIgTzUWaIO9XO3taNJQZ2YGqY/trfUmgpSft8K+hAKJhT0Kr86Ooo/kVPip4E i/9/GT5U88kg== X-IronPort-AV: E=McAfee;i="6200,9189,9967"; a="183994840" X-IronPort-AV: E=Sophos;i="5.82,254,1613462400"; d="scan'208";a="183994840" Received: from fmsmga002.fm.intel.com ([10.253.24.26]) by orsmga106.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 27 Apr 2021 07:05:21 -0700 IronPort-SDR: 8rfGOCutDCc+TIZHjgLD7x7HYNR4tfk0j5ITcCtJ7mnH1KUW6uLWH/8VR8pOgeLSUOCbPxTuw9 xHpmLWtCTaAg== X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="5.82,254,1613462400"; d="scan'208";a="457656370" Received: from npg-dpdk-haiyue-2.sh.intel.com ([10.67.119.63]) by fmsmga002.fm.intel.com with ESMTP; 27 Apr 2021 07:05:19 -0700 From: Haiyue Wang To: dev@dpdk.org Cc: qi.z.zhang@intel.com, liang-min.wang@intel.com, david.marchand@redhat.com, Haiyue Wang , Jingjing Wu , Beilei Xing Date: Tue, 27 Apr 2021 21:39:11 +0800 Message-Id: <20210427133912.261993-3-haiyue.wang@intel.com> X-Mailer: git-send-email 2.31.1 In-Reply-To: <20210427133912.261993-1-haiyue.wang@intel.com> References: <20210421050243.130585-1-haiyue.wang@intel.com> <20210427133912.261993-1-haiyue.wang@intel.com> MIME-Version: 1.0 Subject: [dpdk-dev] [PATCH v4 2/3] net/iavf: enable PCI bus master after reset X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Sender: "dev" The VF reset can be triggerred by the PF reset event, in this case, the PCI bus master will be cleared, then the VF is not allowed to issue any Memory or I/O Requests. So after the reset event is detected, always enable the PCI bus master. Signed-off-by: Haiyue Wang Acked-by: Qi Zhang --- drivers/net/iavf/iavf_ethdev.c | 3 +++ 1 file changed, 3 insertions(+) diff --git a/drivers/net/iavf/iavf_ethdev.c b/drivers/net/iavf/iavf_ethdev.c index d523a0618..9a0a20a29 100644 --- a/drivers/net/iavf/iavf_ethdev.c +++ b/drivers/net/iavf/iavf_ethdev.c @@ -2255,6 +2255,9 @@ iavf_dev_close(struct rte_eth_dev *dev) rte_free(vf->aq_resp); vf->aq_resp = NULL; + if (vf->vf_reset) + rte_pci_set_bus_master(pci_dev, true); + vf->vf_reset = false; return ret; From patchwork Tue Apr 27 13:39:12 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Wang, Haiyue" X-Patchwork-Id: 92264 X-Patchwork-Delegate: david.marchand@redhat.com Return-Path: X-Original-To: patchwork@inbox.dpdk.org Delivered-To: patchwork@inbox.dpdk.org Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id D98DDA0A02; Tue, 27 Apr 2021 16:05:39 +0200 (CEST) Received: from [217.70.189.124] (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id 121C441266; Tue, 27 Apr 2021 16:05:28 +0200 (CEST) Received: from mga18.intel.com (mga18.intel.com [134.134.136.126]) by mails.dpdk.org (Postfix) with ESMTP id 282224125A for ; Tue, 27 Apr 2021 16:05:23 +0200 (CEST) IronPort-SDR: MFW8/DDeUzW1Gff2tTChbOUOWRTaergtlcrNqKnE4ukQSyxidr7lxPRcqYfnGfK6ILIHSh8GQ9 4hu5otSuEjpw== X-IronPort-AV: E=McAfee;i="6200,9189,9967"; a="183994846" X-IronPort-AV: E=Sophos;i="5.82,254,1613462400"; d="scan'208";a="183994846" Received: from fmsmga002.fm.intel.com ([10.253.24.26]) by orsmga106.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 27 Apr 2021 07:05:23 -0700 IronPort-SDR: bJnxJENfUr5Ln/RvqeSzLQkFIUHOz8VXHuF9Vq7v+ARcftwJIakvbb1itL5I5Pph8ByfqbLTOo mV/gIUB7mZgA== X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="5.82,254,1613462400"; d="scan'208";a="457656376" Received: from npg-dpdk-haiyue-2.sh.intel.com ([10.67.119.63]) by fmsmga002.fm.intel.com with ESMTP; 27 Apr 2021 07:05:21 -0700 From: Haiyue Wang To: dev@dpdk.org Cc: qi.z.zhang@intel.com, liang-min.wang@intel.com, david.marchand@redhat.com, Haiyue Wang , Beilei Xing , Jeff Guo Date: Tue, 27 Apr 2021 21:39:12 +0800 Message-Id: <20210427133912.261993-4-haiyue.wang@intel.com> X-Mailer: git-send-email 2.31.1 In-Reply-To: <20210427133912.261993-1-haiyue.wang@intel.com> References: <20210421050243.130585-1-haiyue.wang@intel.com> <20210427133912.261993-1-haiyue.wang@intel.com> MIME-Version: 1.0 Subject: [dpdk-dev] [PATCH v4 3/3] net/i40e: enable PCI bus master after reset X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Sender: "dev" The VF reset can be triggerred by the PF reset event, in this case, the PCI bus master will be cleared, then the VF is not allowed to issue any Memory or I/O Requests. So after the reset event is detected, always enable the PCI bus master. And align the VF reset event handling in device close module as the AVF driver does. Signed-off-by: Haiyue Wang Acked-by: Qi Zhang --- drivers/net/i40e/i40e_ethdev_vf.c | 7 ++++++- 1 file changed, 6 insertions(+), 1 deletion(-) diff --git a/drivers/net/i40e/i40e_ethdev_vf.c b/drivers/net/i40e/i40e_ethdev_vf.c index 3c258ba7c..8b041e94c 100644 --- a/drivers/net/i40e/i40e_ethdev_vf.c +++ b/drivers/net/i40e/i40e_ethdev_vf.c @@ -1212,7 +1212,6 @@ i40evf_check_vf_reset_done(struct rte_eth_dev *dev) if (i >= MAX_RESET_WAIT_CNT) return -1; - vf->vf_reset = false; vf->pend_msg &= ~PFMSG_RESET_IMPENDING; return 0; @@ -1391,6 +1390,7 @@ i40evf_handle_pf_event(struct rte_eth_dev *dev, uint8_t *msg, switch (pf_msg->event) { case VIRTCHNL_EVENT_RESET_IMPENDING: PMD_DRV_LOG(DEBUG, "VIRTCHNL_EVENT_RESET_IMPENDING event"); + vf->vf_reset = true; rte_eth_dev_callback_process(dev, RTE_ETH_EVENT_INTR_RESET, NULL); break; @@ -2487,6 +2487,11 @@ i40evf_dev_close(struct rte_eth_dev *dev) i40e_shutdown_adminq(hw); i40evf_disable_irq0(hw); + if (vf->vf_reset) + rte_pci_set_bus_master(RTE_ETH_DEV_TO_PCI(dev), true); + + vf->vf_reset = false; + rte_free(vf->vf_res); vf->vf_res = NULL; rte_free(vf->aq_resp);