From patchwork Wed Sep 9 13:56:52 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Hyong Youb Kim (hyonkim)" X-Patchwork-Id: 77050 X-Patchwork-Delegate: ferruh.yigit@amd.com Return-Path: X-Original-To: patchwork@inbox.dpdk.org Delivered-To: patchwork@inbox.dpdk.org Received: from dpdk.org (dpdk.org [92.243.14.124]) by inbox.dpdk.org (Postfix) with ESMTP id 6D416A04B1; Wed, 9 Sep 2020 15:57:19 +0200 (CEST) Received: from [92.243.14.124] (localhost [127.0.0.1]) by dpdk.org (Postfix) with ESMTP id 1038E1C0C9; Wed, 9 Sep 2020 15:57:19 +0200 (CEST) Received: from rcdn-iport-6.cisco.com (rcdn-iport-6.cisco.com [173.37.86.77]) by dpdk.org (Postfix) with ESMTP id 5D4CA1C0BE for ; Wed, 9 Sep 2020 15:57:17 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=cisco.com; i=@cisco.com; l=7086; q=dns/txt; s=iport; t=1599659837; x=1600869437; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=09Mr2dckLXEkY+ohW/b5hQOvk4+U8yqSyoQ21WFuvR4=; b=S9kxQ+Ql2zUIyaBl4VpXuoUNkpSIZiMfADE+cnQ8sQRZ1P7PgpEtjRSc 5y5NzmH3XW+fgItBfglM5Gi8HHARzBxnhAJ/PcJbuf3nB/HRaMpN2MxYf 9MolNT+rMXz4vkCya4yizmZNQaeCn+F871p2/5fdXV8G/ZwDuxNxrAG23 8=; X-IronPort-AV: E=Sophos;i="5.76,409,1592870400"; d="scan'208";a="825411486" Received: from rcdn-core-3.cisco.com ([173.37.93.154]) by rcdn-iport-6.cisco.com with ESMTP/TLS/DHE-RSA-SEED-SHA; 09 Sep 2020 13:57:16 +0000 Received: from cisco.com (savbu-usnic-a.cisco.com [10.193.184.48]) by rcdn-core-3.cisco.com (8.15.2/8.15.2) with ESMTP id 089DvG9Z022478; Wed, 9 Sep 2020 13:57:16 GMT Received: by cisco.com (Postfix, from userid 508933) id 04A5D20F2005; Wed, 9 Sep 2020 06:57:16 -0700 (PDT) From: Hyong Youb Kim To: Ferruh Yigit Cc: dev@dpdk.org, Hyong Youb Kim , John Daley Date: Wed, 9 Sep 2020 06:56:52 -0700 Message-Id: <20200909135656.18892-2-hyonkim@cisco.com> X-Mailer: git-send-email 2.26.2 In-Reply-To: <20200909135656.18892-1-hyonkim@cisco.com> References: <20200909135656.18892-1-hyonkim@cisco.com> MIME-Version: 1.0 X-Outbound-SMTP-Client: 10.193.184.48, savbu-usnic-a.cisco.com X-Outbound-Node: rcdn-core-3.cisco.com Subject: [dpdk-dev] [PATCH 1/5] net/enic: extend vnic dev API for VF representors X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Sender: "dev" VF representors need to proxy devcmd through the PF vnic_dev instance. Extend vnic_dev to accommodate them as follows. 1. Add vnic_vf_rep_register() A VF representor creates its own vnic_dev instance via this function and saves VF ID. When performing devcmd, vnic_dev uses the saved VF ID to proxy devcmd through the PF vnic_dev instance. 2. Add vnic_register_lock() As PF and VF representors appear as independent ports to the application, its threads may invoke APIs on them simultaneously, leading to race conditions on the PF vnic_dev. For example, thread A can query stats on PF port, while thread B queries stats on a VF representor. The PF port invokes this function to provide a lock to vnic_dev. This lock is used to serialize devcmd calls from PF and VF representors. 3. Add utility functions to assist VF representor settings vnic_dev_mtu() and vnic_dev_uif() retrieve vnic MTU and UIF number (uplink index), respectively. Signed-off-by: Hyong Youb Kim Reviewed-by: John Daley --- drivers/net/enic/base/vnic_dev.c | 112 ++++++++++++++++++++++++++++++- drivers/net/enic/base/vnic_dev.h | 4 ++ 2 files changed, 113 insertions(+), 3 deletions(-) diff --git a/drivers/net/enic/base/vnic_dev.c b/drivers/net/enic/base/vnic_dev.c index ac03817f4..aaca07ca6 100644 --- a/drivers/net/enic/base/vnic_dev.c +++ b/drivers/net/enic/base/vnic_dev.c @@ -61,6 +61,16 @@ struct vnic_dev { void (*free_consistent)(void *priv, size_t size, void *vaddr, dma_addr_t dma_handle); + /* + * Used to serialize devcmd access, currently from PF and its + * VF representors. When there are no representors, lock is + * not used. + */ + int locked; + void (*lock)(void *priv); + void (*unlock)(void *priv); + struct vnic_dev *pf_vdev; + int vf_id; }; #define VNIC_MAX_RES_HDR_SIZE \ @@ -84,6 +94,14 @@ void vnic_register_cbacks(struct vnic_dev *vdev, vdev->free_consistent = free_consistent; } +void vnic_register_lock(struct vnic_dev *vdev, void (*lock)(void *priv), + void (*unlock)(void *priv)) +{ + vdev->lock = lock; + vdev->unlock = unlock; + vdev->locked = 0; +} + static int vnic_dev_discover_res(struct vnic_dev *vdev, struct vnic_dev_bar *bar, unsigned int num_bars) { @@ -410,12 +428,39 @@ static int vnic_dev_cmd_no_proxy(struct vnic_dev *vdev, return err; } +void vnic_dev_cmd_proxy_by_index_start(struct vnic_dev *vdev, uint16_t index) +{ + vdev->proxy = PROXY_BY_INDEX; + vdev->proxy_index = index; +} + +void vnic_dev_cmd_proxy_end(struct vnic_dev *vdev) +{ + vdev->proxy = PROXY_NONE; + vdev->proxy_index = 0; +} + int vnic_dev_cmd(struct vnic_dev *vdev, enum vnic_devcmd_cmd cmd, uint64_t *a0, uint64_t *a1, int wait) { uint64_t args[2]; + bool vf_rep; + int vf_idx; int err; + vf_rep = false; + if (vdev->pf_vdev) { + vf_rep = true; + vf_idx = vdev->vf_id; + /* Everything below assumes PF vdev */ + vdev = vdev->pf_vdev; + } + if (vdev->lock) + vdev->lock(vdev->priv); + /* For VF representor, proxy devcmd to VF index */ + if (vf_rep) + vnic_dev_cmd_proxy_by_index_start(vdev, vf_idx); + args[0] = *a0; args[1] = *a1; memset(vdev->args, 0, sizeof(vdev->args)); @@ -435,6 +480,10 @@ int vnic_dev_cmd(struct vnic_dev *vdev, enum vnic_devcmd_cmd cmd, break; } + if (vf_rep) + vnic_dev_cmd_proxy_end(vdev); + if (vdev->unlock) + vdev->unlock(vdev->priv); if (err == 0) { *a0 = args[0]; *a1 = args[1]; @@ -446,17 +495,41 @@ int vnic_dev_cmd(struct vnic_dev *vdev, enum vnic_devcmd_cmd cmd, int vnic_dev_cmd_args(struct vnic_dev *vdev, enum vnic_devcmd_cmd cmd, uint64_t *args, int nargs, int wait) { + bool vf_rep; + int vf_idx; + int err; + + vf_rep = false; + if (vdev->pf_vdev) { + vf_rep = true; + vf_idx = vdev->vf_id; + vdev = vdev->pf_vdev; + } + if (vdev->lock) + vdev->lock(vdev->priv); + if (vf_rep) + vnic_dev_cmd_proxy_by_index_start(vdev, vf_idx); + switch (vdev->proxy) { case PROXY_BY_INDEX: - return vnic_dev_cmd_proxy(vdev, CMD_PROXY_BY_INDEX, cmd, + err = vnic_dev_cmd_proxy(vdev, CMD_PROXY_BY_INDEX, cmd, args, nargs, wait); + break; case PROXY_BY_BDF: - return vnic_dev_cmd_proxy(vdev, CMD_PROXY_BY_BDF, cmd, + err = vnic_dev_cmd_proxy(vdev, CMD_PROXY_BY_BDF, cmd, args, nargs, wait); + break; case PROXY_NONE: default: - return vnic_dev_cmd_no_proxy(vdev, cmd, args, nargs, wait); + err = vnic_dev_cmd_no_proxy(vdev, cmd, args, nargs, wait); + break; } + + if (vf_rep) + vnic_dev_cmd_proxy_end(vdev); + if (vdev->unlock) + vdev->unlock(vdev->priv); + return err; } int vnic_dev_fw_info(struct vnic_dev *vdev, @@ -1012,6 +1085,22 @@ uint32_t vnic_dev_port_speed(struct vnic_dev *vdev) return vdev->notify_copy.port_speed; } +uint32_t vnic_dev_mtu(struct vnic_dev *vdev) +{ + if (!vnic_dev_notify_ready(vdev)) + return 0; + + return vdev->notify_copy.mtu; +} + +uint32_t vnic_dev_uif(struct vnic_dev *vdev) +{ + if (!vnic_dev_notify_ready(vdev)) + return 0; + + return vdev->notify_copy.uif; +} + uint32_t vnic_dev_intr_coal_timer_usec_to_hw(struct vnic_dev *vdev, uint32_t usec) { @@ -1100,6 +1189,23 @@ struct vnic_dev *vnic_dev_register(struct vnic_dev *vdev, return NULL; } +struct vnic_dev *vnic_vf_rep_register(void *priv, struct vnic_dev *pf_vdev, + int vf_id) +{ + struct vnic_dev *vdev; + + vdev = (struct vnic_dev *)rte_zmalloc("enic-vf-rep-vdev", + sizeof(struct vnic_dev), RTE_CACHE_LINE_SIZE); + if (!vdev) + return NULL; + vdev->priv = priv; + vdev->pf_vdev = pf_vdev; + vdev->vf_id = vf_id; + vdev->alloc_consistent = pf_vdev->alloc_consistent; + vdev->free_consistent = pf_vdev->free_consistent; + return vdev; +} + /* * vnic_dev_classifier: Add/Delete classifier entries * @vdev: vdev of the device diff --git a/drivers/net/enic/base/vnic_dev.h b/drivers/net/enic/base/vnic_dev.h index 02e19c0b8..30ba57bfc 100644 --- a/drivers/net/enic/base/vnic_dev.h +++ b/drivers/net/enic/base/vnic_dev.h @@ -80,6 +80,8 @@ void vnic_register_cbacks(struct vnic_dev *vdev, void (*free_consistent)(void *priv, size_t size, void *vaddr, dma_addr_t dma_handle)); +void vnic_register_lock(struct vnic_dev *vdev, void (*lock)(void *priv), + void (*unlock)(void *priv)); void __iomem *vnic_dev_get_res(struct vnic_dev *vdev, enum vnic_res_type type, unsigned int index); dma_addr_t vnic_dev_get_res_bus_addr(struct vnic_dev *vdev, @@ -172,6 +174,8 @@ struct vnic_dev *vnic_dev_register(struct vnic_dev *vdev, void *priv, struct rte_pci_device *pdev, struct vnic_dev_bar *bar, unsigned int num_bars); struct rte_pci_device *vnic_dev_get_pdev(struct vnic_dev *vdev); +struct vnic_dev *vnic_vf_rep_register(void *priv, struct vnic_dev *pf_vdev, + int vf_id); int vnic_dev_alloc_stats_mem(struct vnic_dev *vdev); int vnic_dev_cmd_init(struct vnic_dev *vdev, int fallback); int vnic_dev_get_size(void);