From patchwork Mon Jul 9 03:36:48 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Qi Zhang X-Patchwork-Id: 42577 X-Patchwork-Delegate: thomas@monjalon.net Return-Path: X-Original-To: patchwork@dpdk.org Delivered-To: patchwork@dpdk.org Received: from [92.243.14.124] (localhost [127.0.0.1]) by dpdk.org (Postfix) with ESMTP id EC9121B066; Mon, 9 Jul 2018 05:36:42 +0200 (CEST) Received: from mga12.intel.com (mga12.intel.com [192.55.52.136]) by dpdk.org (Postfix) with ESMTP id 0DD111B059 for ; Mon, 9 Jul 2018 05:36:38 +0200 (CEST) X-Amp-Result: SKIPPED(no attachment in message) X-Amp-File-Uploaded: False Received: from fmsmga005.fm.intel.com ([10.253.24.32]) by fmsmga106.fm.intel.com with ESMTP/TLS/DHE-RSA-AES256-GCM-SHA384; 08 Jul 2018 20:36:37 -0700 X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="5.51,328,1526367600"; d="scan'208";a="243632693" Received: from dpdk51.sh.intel.com ([10.67.110.190]) by fmsmga005.fm.intel.com with ESMTP; 08 Jul 2018 20:36:30 -0700 From: Qi Zhang To: thomas@monjalon.net, anatoly.burakov@intel.com Cc: konstantin.ananyev@intel.com, dev@dpdk.org, bruce.richardson@intel.com, ferruh.yigit@intel.com, benjamin.h.shelton@intel.com, narender.vangati@intel.com, Qi Zhang Date: Mon, 9 Jul 2018 11:36:48 +0800 Message-Id: <20180709033706.27858-2-qi.z.zhang@intel.com> X-Mailer: git-send-email 2.13.6 In-Reply-To: <20180709033706.27858-1-qi.z.zhang@intel.com> References: <20180607123849.14439-1-qi.z.zhang@intel.com> <20180709033706.27858-1-qi.z.zhang@intel.com> Subject: [dpdk-dev] [PATCH v10 01/19] ethdev: add function to release port in local process X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Sender: "dev" Add driver API rte_eth_release_port_private to support the case when an ethdev need to be detached on a secondary process. Local state is set to unused and shared data will not be reset so the primary process can still use it. Signed-off-by: Qi Zhang Reviewed-by: Andrew Rybchenko Acked-by: Remy Horton --- lib/librte_ethdev/rte_ethdev.c | 12 ++++++++++++ lib/librte_ethdev/rte_ethdev_driver.h | 16 +++++++++++++++- lib/librte_ethdev/rte_ethdev_pci.h | 8 ++++++++ 3 files changed, 35 insertions(+), 1 deletion(-) diff --git a/lib/librte_ethdev/rte_ethdev.c b/lib/librte_ethdev/rte_ethdev.c index a9977df97..52a97694c 100644 --- a/lib/librte_ethdev/rte_ethdev.c +++ b/lib/librte_ethdev/rte_ethdev.c @@ -359,6 +359,18 @@ rte_eth_dev_attach_secondary(const char *name) } int +rte_eth_dev_release_port_private(struct rte_eth_dev *eth_dev) +{ + if (eth_dev == NULL) + return -EINVAL; + + _rte_eth_dev_callback_process(eth_dev, RTE_ETH_EVENT_DESTROY, NULL); + eth_dev->state = RTE_ETH_DEV_UNUSED; + + return 0; +} + +int rte_eth_dev_release_port(struct rte_eth_dev *eth_dev) { if (eth_dev == NULL) diff --git a/lib/librte_ethdev/rte_ethdev_driver.h b/lib/librte_ethdev/rte_ethdev_driver.h index c9c825e3f..269586d88 100644 --- a/lib/librte_ethdev/rte_ethdev_driver.h +++ b/lib/librte_ethdev/rte_ethdev_driver.h @@ -62,7 +62,7 @@ struct rte_eth_dev *rte_eth_dev_attach_secondary(const char *name); * Release the specified ethdev port. * * @param eth_dev - * The *eth_dev* pointer is the address of the *rte_eth_dev* structure. + * Device to be detached. * @return * - 0 on success, negative on error */ @@ -70,6 +70,20 @@ int rte_eth_dev_release_port(struct rte_eth_dev *eth_dev); /** * @internal + * Release the specified ethdev port in the local process. + * Only set ethdev state to unused, but not reset shared data since + * it assume other processes is still using it. typically it is + * called by a secondary process. + * + * @param eth_dev + * Device to be detached. + * @return + * - 0 on success, negative on error + */ +int rte_eth_dev_release_port_private(struct rte_eth_dev *eth_dev); + +/** + * @internal * Release device queues and clear its configuration to force the user * application to reconfigure it. It is for internal use only. * diff --git a/lib/librte_ethdev/rte_ethdev_pci.h b/lib/librte_ethdev/rte_ethdev_pci.h index 2cfd37274..a46d9e182 100644 --- a/lib/librte_ethdev/rte_ethdev_pci.h +++ b/lib/librte_ethdev/rte_ethdev_pci.h @@ -197,6 +197,14 @@ rte_eth_dev_pci_generic_remove(struct rte_pci_device *pci_dev, if (!eth_dev) return -ENODEV; + /** + * PCI device can only be globally detached directly by a + * primary process. In secondary process, we only need to + * release port. + */ + if (rte_eal_process_type() != RTE_PROC_PRIMARY) + return rte_eth_dev_release_port_private(eth_dev); + if (dev_uninit) { ret = dev_uninit(eth_dev); if (ret) From patchwork Mon Jul 9 03:36:49 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Qi Zhang X-Patchwork-Id: 42580 X-Patchwork-Delegate: thomas@monjalon.net Return-Path: X-Original-To: patchwork@dpdk.org Delivered-To: patchwork@dpdk.org Received: from [92.243.14.124] (localhost [127.0.0.1]) by dpdk.org (Postfix) with ESMTP id 244C31B295; Mon, 9 Jul 2018 05:36:50 +0200 (CEST) Received: from mga12.intel.com (mga12.intel.com [192.55.52.136]) by dpdk.org (Postfix) with ESMTP id 9E7501B059; Mon, 9 Jul 2018 05:36:39 +0200 (CEST) X-Amp-Result: SKIPPED(no attachment in message) X-Amp-File-Uploaded: False Received: from fmsmga005.fm.intel.com ([10.253.24.32]) by fmsmga106.fm.intel.com with ESMTP/TLS/DHE-RSA-AES256-GCM-SHA384; 08 Jul 2018 20:36:37 -0700 X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="5.51,328,1526367600"; d="scan'208";a="243632702" Received: from dpdk51.sh.intel.com ([10.67.110.190]) by fmsmga005.fm.intel.com with ESMTP; 08 Jul 2018 20:36:32 -0700 From: Qi Zhang To: thomas@monjalon.net, anatoly.burakov@intel.com Cc: konstantin.ananyev@intel.com, dev@dpdk.org, bruce.richardson@intel.com, ferruh.yigit@intel.com, benjamin.h.shelton@intel.com, narender.vangati@intel.com, Qi Zhang , stable@dpdk.org Date: Mon, 9 Jul 2018 11:36:49 +0800 Message-Id: <20180709033706.27858-3-qi.z.zhang@intel.com> X-Mailer: git-send-email 2.13.6 In-Reply-To: <20180709033706.27858-1-qi.z.zhang@intel.com> References: <20180607123849.14439-1-qi.z.zhang@intel.com> <20180709033706.27858-1-qi.z.zhang@intel.com> Subject: [dpdk-dev] [PATCH v10 02/19] bus/pci: fix PCI address compare X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Sender: "dev" When use memcmp to compare two PCI address, sizeof(struct rte_pci_addr) is 4 bytes aligned, and it is 8. While only 7 byte of struct rte_pci_addr is valid. So compare the 8th byte will cause the unexpected result, which happens when repeatedly attach/detach a device. Fixes: c752998b5e2e ("pci: introduce library and driver") Cc: stable@dpdk.org Signed-off-by: Qi Zhang --- drivers/bus/pci/linux/pci_vfio.c | 13 ++++++++++++- 1 file changed, 12 insertions(+), 1 deletion(-) diff --git a/drivers/bus/pci/linux/pci_vfio.c b/drivers/bus/pci/linux/pci_vfio.c index aeeaa9ed8..dd25c3542 100644 --- a/drivers/bus/pci/linux/pci_vfio.c +++ b/drivers/bus/pci/linux/pci_vfio.c @@ -43,6 +43,17 @@ static struct rte_tailq_elem rte_vfio_tailq = { }; EAL_REGISTER_TAILQ(rte_vfio_tailq) +/* Compair two pci address */ +static int pci_addr_cmp(struct rte_pci_addr *addr1, struct rte_pci_addr *addr2) +{ + if (addr1->domain == addr2->domain && + addr1->bus == addr2->bus && + addr1->devid == addr2->devid && + addr1->function == addr2->function) + return 0; + return 1; +} + int pci_vfio_read_config(const struct rte_intr_handle *intr_handle, void *buf, size_t len, off_t offs) @@ -642,7 +653,7 @@ pci_vfio_unmap_resource(struct rte_pci_device *dev) vfio_res_list = RTE_TAILQ_CAST(rte_vfio_tailq.head, mapped_pci_res_list); /* Get vfio_res */ TAILQ_FOREACH(vfio_res, vfio_res_list, next) { - if (memcmp(&vfio_res->pci_addr, &dev->addr, sizeof(dev->addr))) + if (pci_addr_cmp(&vfio_res->pci_addr, &dev->addr)) continue; break; } From patchwork Mon Jul 9 03:36:50 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Qi Zhang X-Patchwork-Id: 42578 X-Patchwork-Delegate: thomas@monjalon.net Return-Path: X-Original-To: patchwork@dpdk.org Delivered-To: patchwork@dpdk.org Received: from [92.243.14.124] (localhost [127.0.0.1]) by dpdk.org (Postfix) with ESMTP id E4ABF1B1ED; Mon, 9 Jul 2018 05:36:45 +0200 (CEST) Received: from mga12.intel.com (mga12.intel.com [192.55.52.136]) by dpdk.org (Postfix) with ESMTP id CAF351B05B for ; Mon, 9 Jul 2018 05:36:39 +0200 (CEST) X-Amp-Result: SKIPPED(no attachment in message) X-Amp-File-Uploaded: False Received: from fmsmga005.fm.intel.com ([10.253.24.32]) by fmsmga106.fm.intel.com with ESMTP/TLS/DHE-RSA-AES256-GCM-SHA384; 08 Jul 2018 20:36:37 -0700 X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="5.51,328,1526367600"; d="scan'208";a="243632712" Received: from dpdk51.sh.intel.com ([10.67.110.190]) by fmsmga005.fm.intel.com with ESMTP; 08 Jul 2018 20:36:33 -0700 From: Qi Zhang To: thomas@monjalon.net, anatoly.burakov@intel.com Cc: konstantin.ananyev@intel.com, dev@dpdk.org, bruce.richardson@intel.com, ferruh.yigit@intel.com, benjamin.h.shelton@intel.com, narender.vangati@intel.com, Qi Zhang Date: Mon, 9 Jul 2018 11:36:50 +0800 Message-Id: <20180709033706.27858-4-qi.z.zhang@intel.com> X-Mailer: git-send-email 2.13.6 In-Reply-To: <20180709033706.27858-1-qi.z.zhang@intel.com> References: <20180607123849.14439-1-qi.z.zhang@intel.com> <20180709033706.27858-1-qi.z.zhang@intel.com> Subject: [dpdk-dev] [PATCH v10 03/19] bus/pci: enable vfio unmap resource for secondary X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Sender: "dev" Subroutine to unmap VFIO resource is shared by secondary and primary, and it does not work on the secondary process. The patch adds a dedicate function to handle the situation when a device is unmapped on a secondary process. Signed-off-by: Qi Zhang --- drivers/bus/pci/linux/pci_vfio.c | 75 ++++++++++++++++++++++++++++++++++++++-- 1 file changed, 73 insertions(+), 2 deletions(-) diff --git a/drivers/bus/pci/linux/pci_vfio.c b/drivers/bus/pci/linux/pci_vfio.c index dd25c3542..72481ac45 100644 --- a/drivers/bus/pci/linux/pci_vfio.c +++ b/drivers/bus/pci/linux/pci_vfio.c @@ -595,6 +595,9 @@ pci_vfio_map_resource_secondary(struct rte_pci_device *dev) dev->mem_resource[i].addr = maps[i].addr; } + /* we need save vfio_dev_fd, so it can be used during release */ + dev->intr_handle.vfio_dev_fd = vfio_dev_fd; + return 0; err_vfio_dev_fd: close(vfio_dev_fd); @@ -614,8 +617,8 @@ pci_vfio_map_resource(struct rte_pci_device *dev) return pci_vfio_map_resource_secondary(dev); } -int -pci_vfio_unmap_resource(struct rte_pci_device *dev) +static int +pci_vfio_unmap_resource_primary(struct rte_pci_device *dev) { char pci_addr[PATH_MAX] = {0}; struct rte_pci_addr *loc = &dev->addr; @@ -687,6 +690,74 @@ pci_vfio_unmap_resource(struct rte_pci_device *dev) return 0; } +static int +pci_vfio_unmap_resource_secondary(struct rte_pci_device *dev) +{ + char pci_addr[PATH_MAX] = {0}; + struct rte_pci_addr *loc = &dev->addr; + int i, ret; + struct mapped_pci_resource *vfio_res = NULL; + struct mapped_pci_res_list *vfio_res_list; + + struct pci_map *maps; + + /* store PCI address string */ + snprintf(pci_addr, sizeof(pci_addr), PCI_PRI_FMT, + loc->domain, loc->bus, loc->devid, loc->function); + + ret = rte_vfio_release_device(rte_pci_get_sysfs_path(), pci_addr, + dev->intr_handle.vfio_dev_fd); + if (ret < 0) { + RTE_LOG(ERR, EAL, + "%s(): cannot release device\n", __func__); + return ret; + } + + vfio_res_list = + RTE_TAILQ_CAST(rte_vfio_tailq.head, mapped_pci_res_list); + /* Get vfio_res */ + TAILQ_FOREACH(vfio_res, vfio_res_list, next) { + if (pci_addr_cmp(&vfio_res->pci_addr, &dev->addr)) + continue; + break; + } + /* if we haven't found our tailq entry, something's wrong */ + if (vfio_res == NULL) { + RTE_LOG(ERR, EAL, " %s cannot find TAILQ entry for PCI device!\n", + pci_addr); + return -1; + } + + /* unmap BARs */ + maps = vfio_res->maps; + + RTE_LOG(INFO, EAL, "Releasing pci mapped resource for %s\n", + pci_addr); + for (i = 0; i < (int) vfio_res->nb_maps; i++) { + + /* + * We do not need to be aware of MSI-X table BAR mappings as + * when mapping. Just using current maps array is enough + */ + if (maps[i].addr) { + RTE_LOG(INFO, EAL, "Calling pci_unmap_resource for %s at %p\n", + pci_addr, maps[i].addr); + pci_unmap_resource(maps[i].addr, maps[i].size); + } + } + + return 0; +} + +int +pci_vfio_unmap_resource(struct rte_pci_device *dev) +{ + if (rte_eal_process_type() == RTE_PROC_PRIMARY) + return pci_vfio_unmap_resource_primary(dev); + else + return pci_vfio_unmap_resource_secondary(dev); +} + int pci_vfio_ioport_map(struct rte_pci_device *dev, int bar, struct rte_pci_ioport *p) From patchwork Mon Jul 9 03:36:51 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Qi Zhang X-Patchwork-Id: 42579 X-Patchwork-Delegate: thomas@monjalon.net Return-Path: X-Original-To: patchwork@dpdk.org Delivered-To: patchwork@dpdk.org Received: from [92.243.14.124] (localhost [127.0.0.1]) by dpdk.org (Postfix) with ESMTP id 4865B1B1FB; Mon, 9 Jul 2018 05:36:48 +0200 (CEST) Received: from mga12.intel.com (mga12.intel.com [192.55.52.136]) by dpdk.org (Postfix) with ESMTP id 7FF7F1B05B for ; Mon, 9 Jul 2018 05:36:40 +0200 (CEST) X-Amp-Result: SKIPPED(no attachment in message) X-Amp-File-Uploaded: False Received: from fmsmga005.fm.intel.com ([10.253.24.32]) by fmsmga106.fm.intel.com with ESMTP/TLS/DHE-RSA-AES256-GCM-SHA384; 08 Jul 2018 20:36:38 -0700 X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="5.51,328,1526367600"; d="scan'208";a="243632719" Received: from dpdk51.sh.intel.com ([10.67.110.190]) by fmsmga005.fm.intel.com with ESMTP; 08 Jul 2018 20:36:36 -0700 From: Qi Zhang To: thomas@monjalon.net, anatoly.burakov@intel.com Cc: konstantin.ananyev@intel.com, dev@dpdk.org, bruce.richardson@intel.com, ferruh.yigit@intel.com, benjamin.h.shelton@intel.com, narender.vangati@intel.com, Qi Zhang Date: Mon, 9 Jul 2018 11:36:51 +0800 Message-Id: <20180709033706.27858-5-qi.z.zhang@intel.com> X-Mailer: git-send-email 2.13.6 In-Reply-To: <20180709033706.27858-1-qi.z.zhang@intel.com> References: <20180607123849.14439-1-qi.z.zhang@intel.com> <20180709033706.27858-1-qi.z.zhang@intel.com> Subject: [dpdk-dev] [PATCH v10 04/19] vfio: remove uneccessary IPC for group fd clear X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Sender: "dev" Clear vfio_group_fd is not necessary to involve any IPC. Also, current IPC implementation for SOCKET_CLR_GROUP is not correct. rte_vfio_clear_group on secondary will always fail, that prevent device be detached correctly on a secondary process. The patch simply removes all IPC related stuff in rte_vfio_clear_group. Signed-off-by: Qi Zhang Acked-by: Anatoly Burakov --- lib/librte_eal/linuxapp/eal/eal_vfio.c | 45 +++++--------------------- lib/librte_eal/linuxapp/eal/eal_vfio.h | 1 - lib/librte_eal/linuxapp/eal/eal_vfio_mp_sync.c | 8 ----- 3 files changed, 8 insertions(+), 46 deletions(-) diff --git a/lib/librte_eal/linuxapp/eal/eal_vfio.c b/lib/librte_eal/linuxapp/eal/eal_vfio.c index a2bbdfbf4..c0eccddc3 100644 --- a/lib/librte_eal/linuxapp/eal/eal_vfio.c +++ b/lib/librte_eal/linuxapp/eal/eal_vfio.c @@ -575,10 +575,6 @@ int rte_vfio_clear_group(int vfio_group_fd) { int i; - struct rte_mp_msg mp_req, *mp_rep; - struct rte_mp_reply mp_reply; - struct timespec ts = {.tv_sec = 5, .tv_nsec = 0}; - struct vfio_mp_param *p = (struct vfio_mp_param *)mp_req.param; struct vfio_config *vfio_cfg; vfio_cfg = get_vfio_cfg_by_group_fd(vfio_group_fd); @@ -587,40 +583,15 @@ rte_vfio_clear_group(int vfio_group_fd) return -1; } - if (internal_config.process_type == RTE_PROC_PRIMARY) { - - i = get_vfio_group_idx(vfio_group_fd); - if (i < 0) - return -1; - vfio_cfg->vfio_groups[i].group_num = -1; - vfio_cfg->vfio_groups[i].fd = -1; - vfio_cfg->vfio_groups[i].devices = 0; - vfio_cfg->vfio_active_groups--; - return 0; - } - - p->req = SOCKET_CLR_GROUP; - p->group_num = vfio_group_fd; - strcpy(mp_req.name, EAL_VFIO_MP); - mp_req.len_param = sizeof(*p); - mp_req.num_fds = 0; - - if (rte_mp_request_sync(&mp_req, &mp_reply, &ts) == 0 && - mp_reply.nb_received == 1) { - mp_rep = &mp_reply.msgs[0]; - p = (struct vfio_mp_param *)mp_rep->param; - if (p->result == SOCKET_OK) { - free(mp_reply.msgs); - return 0; - } else if (p->result == SOCKET_NO_FD) - RTE_LOG(ERR, EAL, " BAD VFIO group fd!\n"); - else - RTE_LOG(ERR, EAL, " no such VFIO group fd!\n"); - - free(mp_reply.msgs); - } + i = get_vfio_group_idx(vfio_group_fd); + if (i < 0) + return -1; + vfio_cfg->vfio_groups[i].group_num = -1; + vfio_cfg->vfio_groups[i].fd = -1; + vfio_cfg->vfio_groups[i].devices = 0; + vfio_cfg->vfio_active_groups--; - return -1; + return 0; } int diff --git a/lib/librte_eal/linuxapp/eal/eal_vfio.h b/lib/librte_eal/linuxapp/eal/eal_vfio.h index e65b10374..68d4750a5 100644 --- a/lib/librte_eal/linuxapp/eal/eal_vfio.h +++ b/lib/librte_eal/linuxapp/eal/eal_vfio.h @@ -129,7 +129,6 @@ int vfio_mp_sync_setup(void); #define SOCKET_REQ_CONTAINER 0x100 #define SOCKET_REQ_GROUP 0x200 -#define SOCKET_CLR_GROUP 0x300 #define SOCKET_OK 0x0 #define SOCKET_NO_FD 0x1 #define SOCKET_ERR 0xFF diff --git a/lib/librte_eal/linuxapp/eal/eal_vfio_mp_sync.c b/lib/librte_eal/linuxapp/eal/eal_vfio_mp_sync.c index 9c202bb08..680a24aae 100644 --- a/lib/librte_eal/linuxapp/eal/eal_vfio_mp_sync.c +++ b/lib/librte_eal/linuxapp/eal/eal_vfio_mp_sync.c @@ -55,14 +55,6 @@ vfio_mp_primary(const struct rte_mp_msg *msg, const void *peer) reply.fds[0] = fd; } break; - case SOCKET_CLR_GROUP: - r->req = SOCKET_CLR_GROUP; - r->group_num = m->group_num; - if (rte_vfio_clear_group(m->group_num) < 0) - r->result = SOCKET_NO_FD; - else - r->result = SOCKET_OK; - break; case SOCKET_REQ_CONTAINER: r->req = SOCKET_REQ_CONTAINER; fd = rte_vfio_get_container_fd(); From patchwork Mon Jul 9 03:36:52 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Qi Zhang X-Patchwork-Id: 42581 X-Patchwork-Delegate: thomas@monjalon.net Return-Path: X-Original-To: patchwork@dpdk.org Delivered-To: patchwork@dpdk.org Received: from [92.243.14.124] (localhost [127.0.0.1]) by dpdk.org (Postfix) with ESMTP id 2FCB91B3A3; Mon, 9 Jul 2018 05:36:52 +0200 (CEST) Received: from mga12.intel.com (mga12.intel.com [192.55.52.136]) by dpdk.org (Postfix) with ESMTP id 85E8B1B05B for ; Mon, 9 Jul 2018 05:36:40 +0200 (CEST) X-Amp-Result: SKIPPED(no attachment in message) X-Amp-File-Uploaded: False Received: from fmsmga005.fm.intel.com ([10.253.24.32]) by fmsmga106.fm.intel.com with ESMTP/TLS/DHE-RSA-AES256-GCM-SHA384; 08 Jul 2018 20:36:39 -0700 X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="5.51,328,1526367600"; d="scan'208";a="243632723" Received: from dpdk51.sh.intel.com ([10.67.110.190]) by fmsmga005.fm.intel.com with ESMTP; 08 Jul 2018 20:36:38 -0700 From: Qi Zhang To: thomas@monjalon.net, anatoly.burakov@intel.com Cc: konstantin.ananyev@intel.com, dev@dpdk.org, bruce.richardson@intel.com, ferruh.yigit@intel.com, benjamin.h.shelton@intel.com, narender.vangati@intel.com, Qi Zhang Date: Mon, 9 Jul 2018 11:36:52 +0800 Message-Id: <20180709033706.27858-6-qi.z.zhang@intel.com> X-Mailer: git-send-email 2.13.6 In-Reply-To: <20180709033706.27858-1-qi.z.zhang@intel.com> References: <20180607123849.14439-1-qi.z.zhang@intel.com> <20180709033706.27858-1-qi.z.zhang@intel.com> Subject: [dpdk-dev] [PATCH v10 05/19] eal: enable hotplug on multi-process X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Sender: "dev" We are going to introduce the solution to handle hotplug in multi-process, it includes the below scenario: 1. Attach a device from the primary 2. Detach a device from the primary 3. Attach a device from a secondary 4. Detach a device from a secondary In the primary-secondary process model, we assume devices are shared by default. that means attaches or detaches a device on any process will broadcast to all other processes through mp channel then device information will be synchronized on all processes. Any failure during attaching/detaching process will cause inconsistent status between processes, so proper rollback action should be considered. This patch covers the implementation of case 1,2. Case 3,4 will be implemented on a separate patch. IPC scenario for Case 1, 2: attach a device a) primary attach the new device if failed goto h). b) primary send attach sync request to all secondary. c) secondary receive request and attach the device and send a reply. d) primary check the reply if all success goes to i). e) primary send attach rollback sync request to all secondary. f) secondary receive the request and detach the device and send a reply. g) primary receive the reply and detach device as rollback action. h) attach fail i) attach success detach a device a) primary send detach sync request to all secondary b) secondary detach the device and send reply c) primary check the reply if all success goes to f). d) primary send detach rollback sync request to all secondary. e) secondary receive the request and attach back device. goto g) f) primary detach the device if success goto g), else goto d) g) detach fail. h) detach success. Signed-off-by: Qi Zhang --- lib/librte_eal/bsdapp/eal/Makefile | 1 + lib/librte_eal/common/eal_common_dev.c | 140 +++++++++++++++++++++++- lib/librte_eal/common/eal_private.h | 37 +++++++ lib/librte_eal/common/hotplug_mp.c | 181 ++++++++++++++++++++++++++++++++ lib/librte_eal/common/hotplug_mp.h | 44 ++++++++ lib/librte_eal/common/include/rte_bus.h | 3 + lib/librte_eal/common/include/rte_dev.h | 9 ++ lib/librte_eal/common/meson.build | 1 + lib/librte_eal/linuxapp/eal/Makefile | 1 + lib/librte_eal/linuxapp/eal/eal.c | 6 ++ 10 files changed, 418 insertions(+), 5 deletions(-) create mode 100644 lib/librte_eal/common/hotplug_mp.c create mode 100644 lib/librte_eal/common/hotplug_mp.h diff --git a/lib/librte_eal/bsdapp/eal/Makefile b/lib/librte_eal/bsdapp/eal/Makefile index 3fd33f1e4..4ecc73b42 100644 --- a/lib/librte_eal/bsdapp/eal/Makefile +++ b/lib/librte_eal/bsdapp/eal/Makefile @@ -59,6 +59,7 @@ SRCS-$(CONFIG_RTE_EXEC_ENV_BSDAPP) += eal_common_thread.c SRCS-$(CONFIG_RTE_EXEC_ENV_BSDAPP) += eal_common_proc.c SRCS-$(CONFIG_RTE_EXEC_ENV_BSDAPP) += eal_common_fbarray.c SRCS-$(CONFIG_RTE_EXEC_ENV_BSDAPP) += rte_malloc.c +SRCS-$(CONFIG_RTE_EXEC_ENV_BSDAPP) += hotplug_mp.c SRCS-$(CONFIG_RTE_EXEC_ENV_BSDAPP) += malloc_elem.c SRCS-$(CONFIG_RTE_EXEC_ENV_BSDAPP) += malloc_heap.c SRCS-$(CONFIG_RTE_EXEC_ENV_BSDAPP) += malloc_mp.c diff --git a/lib/librte_eal/common/eal_common_dev.c b/lib/librte_eal/common/eal_common_dev.c index 14c5f05fa..fb1a122ae 100644 --- a/lib/librte_eal/common/eal_common_dev.c +++ b/lib/librte_eal/common/eal_common_dev.c @@ -16,8 +16,10 @@ #include #include #include +#include #include "eal_private.h" +#include "hotplug_mp.h" /** * The device event callback description. @@ -102,8 +104,9 @@ int rte_eal_dev_detach(struct rte_device *dev) return ret; } -int __rte_experimental rte_eal_hotplug_add(const char *busname, const char *devname, - const char *devargs) +int +do_dev_hotplug_add(const char *busname, const char *devname, + const char *devargs) { struct rte_bus *bus; struct rte_device *dev; @@ -168,8 +171,7 @@ int __rte_experimental rte_eal_hotplug_add(const char *busname, const char *devn return ret; } -int __rte_experimental -rte_eal_hotplug_remove(const char *busname, const char *devname) +int do_dev_hotplug_remove(const char *busname, const char *devname) { struct rte_bus *bus; struct rte_device *dev; @@ -197,11 +199,139 @@ rte_eal_hotplug_remove(const char *busname, const char *devname) if (ret) RTE_LOG(ERR, EAL, "Driver cannot detach the device (%s)\n", dev->name); - rte_devargs_remove(busname, devname); + else + rte_devargs_remove(busname, devname); + return ret; } int __rte_experimental +rte_eal_hotplug_add(const char *busname, const char *devname, + const char *devargs) +{ + struct eal_dev_mp_req req; + int ret; + + memset(&req, 0, sizeof(req)); + req.t = EAL_DEV_REQ_TYPE_ATTACH; + strlcpy(req.busname, busname, RTE_BUS_NAME_MAX_LEN); + strlcpy(req.devname, devname, RTE_DEV_NAME_MAX_LEN); + strlcpy(req.devargs, devargs, RTE_DEV_ARGS_MAX_LEN); + + if (rte_eal_process_type() != RTE_PROC_PRIMARY) + return -ENOTSUP; + + /** + * attach a device from primary start from here: + * + * a) primary attach the new device if failed goto h). + * b) primary send attach sync request to all secondary. + * c) secondary receive request and attach the device and send a reply. + * d) primary check the reply if all success goes to i). + * e) primary send attach rollback sync request to all secondary. + * f) secondary receive the request and detach the device and send a + * reply. + * g) primary receive the reply and detach device as rollback action. + * h) attach fail + * i) attach success + */ + + /* step a) */ + ret = do_dev_hotplug_add(busname, devname, devargs); + if (ret) + /* step h) */ + return ret; + + /* step b), c) */ + ret = eal_dev_hotplug_request_to_secondary(&req); + + /* step d) */ + if (ret) { + RTE_LOG(ERR, EAL, + "Failed to send hotplug add request to secondary\n"); + goto rollback; + } + + if (req.result) + goto rollback; + + /* step i */ + return 0; + +rollback: + req.t = EAL_DEV_REQ_TYPE_ATTACH_ROLLBACK; + /* step e), f) */ + eal_dev_hotplug_request_to_secondary(&req); + /* step g) */ + do_dev_hotplug_remove(busname, devname); + /* step h */ + return -ENODEV; +} + +int __rte_experimental +rte_eal_hotplug_remove(const char *busname, const char *devname) +{ + struct eal_dev_mp_req req; + int ret; + + memset(&req, 0, sizeof(req)); + req.t = EAL_DEV_REQ_TYPE_DETACH; + strlcpy(req.busname, busname, RTE_BUS_NAME_MAX_LEN); + strlcpy(req.devname, devname, RTE_DEV_NAME_MAX_LEN); + + if (rte_eal_process_type() != RTE_PROC_PRIMARY) + return -ENOTSUP; + + /** + * detach a device from primary start from here: + * + * a) primary send detach sync request to all secondary + * b) secondary detach the device and send reply + * c) primary check the reply if all success goes to f). + * d) primary send detach rollback sync request to all secondary. + * e) secondary receive the request and attach back device. goto g) + * f) primary detach the device if success goto g), else goto d) + * g) detach fail. + * h) detach success. + */ + + /* step a), b) */ + ret = eal_dev_hotplug_request_to_secondary(&req); + /* step c) */ + if (ret) { + RTE_LOG(ERR, EAL, + "Failed to send device detach request to secondary\n"); + return ret; + } + + if (req.result) { + RTE_LOG(ERR, EAL, + "Failed to detach device on secondary process\n"); + goto rollback; + } + + /* step f) */ + ret = do_dev_hotplug_remove(busname, devname); + if (ret) { + RTE_LOG(ERR, EAL, + "Failed to detach device on primary process\n"); + goto rollback; + } + /* step h */ + return 0; + +rollback: + req.t = EAL_DEV_REQ_TYPE_DETACH_ROLLBACK; + /* step d), e) */ + if (eal_dev_hotplug_request_to_secondary(&req)) + RTE_LOG(ERR, EAL, + "Failed to attach back device on secondary." + "Devices in secondary may not sync with primary\n"); + /* step g) */ + return -ENODEV; +} + +int __rte_experimental rte_dev_event_callback_register(const char *device_name, rte_dev_event_cb_fn cb_fn, void *cb_arg) diff --git a/lib/librte_eal/common/eal_private.h b/lib/librte_eal/common/eal_private.h index bdadc4d50..1883b05d1 100644 --- a/lib/librte_eal/common/eal_private.h +++ b/lib/librte_eal/common/eal_private.h @@ -258,4 +258,41 @@ int rte_mp_channel_init(void); */ void dev_callback_process(char *device_name, enum rte_dev_event_type event); +/** + * Hotplug add a given device to a specific bus. + * + * @param busname + * The bus name the device is added to. + * @param devname + * The device name. Based on this device name, eal will identify a driver + * capable of handling it and pass it to the driver probing function. + * @param devargs + * Device arguments to be passed to the driver. + * @return + * 0 on success, negative on error. + */ +int do_dev_hotplug_add(const char *busname, const char *devname, + const char *devargs); + +/** + * Hotplug remove a given device from a specific bus. + * + * @param busname + * The bus name the device is removed from. + * @param devname + * The device name being removed. + * @return + * 0 on success, negative on error. + */ +int do_dev_hotplug_remove(const char *busname, + const char *devname); + +/** + * Register all mp action callbacks for hotplug. + * + * @return + * 0 on success, negative on error. + */ +int rte_dev_hotplug_mp_init(void); + #endif /* _EAL_PRIVATE_H_ */ diff --git a/lib/librte_eal/common/hotplug_mp.c b/lib/librte_eal/common/hotplug_mp.c new file mode 100644 index 000000000..261d17fe6 --- /dev/null +++ b/lib/librte_eal/common/hotplug_mp.c @@ -0,0 +1,181 @@ +/* SPDX-License-Identifier: BSD-3-Clause + * Copyright(c) 2018 Intel Corporation + */ +#include + +#include +#include +#include + +#include "hotplug_mp.h" +#include "eal_private.h" + +#define MP_TIMEOUT_S 5 /**< 5 seconds timeouts */ + +struct mp_reply_bundle { + struct rte_mp_msg msg; + void *peer; +}; + +static int +handle_secondary_request(const struct rte_mp_msg *msg, const void *peer) +{ + RTE_SET_USED(msg); + RTE_SET_USED(peer); + return -ENOTSUP; +} + +static void __handle_primary_request(void *param) +{ + struct mp_reply_bundle *bundle = param; + struct rte_mp_msg *msg = &bundle->msg; + const struct eal_dev_mp_req *req = + (const struct eal_dev_mp_req *)msg->param; + struct rte_mp_msg mp_resp; + struct eal_dev_mp_req *resp = + (struct eal_dev_mp_req *)mp_resp.param; + int ret = 0; + + memset(&mp_resp, 0, sizeof(mp_resp)); + + switch (req->t) { + case EAL_DEV_REQ_TYPE_ATTACH: + case EAL_DEV_REQ_TYPE_DETACH_ROLLBACK: + ret = do_dev_hotplug_add(req->busname, req->devname, ""); + break; + case EAL_DEV_REQ_TYPE_DETACH: + case EAL_DEV_REQ_TYPE_ATTACH_ROLLBACK: + ret = do_dev_hotplug_remove(req->busname, req->devname); + break; + default: + ret = -EINVAL; + } + + strlcpy(mp_resp.name, EAL_DEV_MP_ACTION_REQUEST, sizeof(mp_resp.name)); + mp_resp.len_param = sizeof(*req); + memcpy(resp, req, sizeof(*resp)); + resp->result = ret; + if (rte_mp_reply(&mp_resp, bundle->peer) < 0) + RTE_LOG(ERR, EAL, "failed to send reply to primary request\n"); + + free(bundle->peer); + free(bundle); +} + +static int +handle_primary_request(const struct rte_mp_msg *msg, const void *peer) +{ + struct rte_mp_msg mp_resp; + const struct eal_dev_mp_req *req = + (const struct eal_dev_mp_req *)msg->param; + struct eal_dev_mp_req *resp = + (struct eal_dev_mp_req *)mp_resp.param; + struct mp_reply_bundle *bundle; + int ret = 0; + + memset(&mp_resp, 0, sizeof(mp_resp)); + strlcpy(mp_resp.name, EAL_DEV_MP_ACTION_REQUEST, sizeof(mp_resp.name)); + mp_resp.len_param = sizeof(*req); + memcpy(resp, req, sizeof(*resp)); + + bundle = calloc(1, sizeof(*bundle)); + if (bundle == NULL) { + resp->result = -ENOMEM; + ret = rte_mp_reply(&mp_resp, peer); + if (ret) { + RTE_LOG(ERR, EAL, "failed to send reply to primary request\n"); + return ret; + } + } + + bundle->msg = *msg; + /** + * We need to send reply on interrupt thread, but peer can't be + * parsed directly, so this is a temporal hack, need to be fixed + * when it is ready. + */ + bundle->peer = (void *)strdup(peer); + + /** + * We are at IPC callback thread, sync IPC is not allowed due to + * dead lock, so we delegate the task to interrupt thread. + */ + ret = rte_eal_alarm_set(1, __handle_primary_request, bundle); + if (ret) { + resp->result = ret; + ret = rte_mp_reply(&mp_resp, peer); + if (ret) { + RTE_LOG(ERR, EAL, "failed to send reply to primary request\n"); + return ret; + } + } + return 0; +} + +int eal_dev_hotplug_request_to_primary(struct eal_dev_mp_req *req) +{ + RTE_SET_USED(req); + return -ENOTSUP; +} + +int eal_dev_hotplug_request_to_secondary(struct eal_dev_mp_req *req) +{ + struct rte_mp_msg mp_req; + struct rte_mp_reply mp_reply; + struct timespec ts = {.tv_sec = MP_TIMEOUT_S, .tv_nsec = 0}; + int ret; + int i; + + memset(&mp_req, 0, sizeof(mp_req)); + memcpy(mp_req.param, req, sizeof(*req)); + mp_req.len_param = sizeof(*req); + strlcpy(mp_req.name, EAL_DEV_MP_ACTION_REQUEST, sizeof(mp_req.name)); + + ret = rte_mp_request_sync(&mp_req, &mp_reply, &ts); + if (ret) { + RTE_LOG(ERR, EAL, "rte_mp_request_sync failed\n"); + return ret; + } + + if (mp_reply.nb_sent != mp_reply.nb_received) { + RTE_LOG(ERR, EAL, "not all secondary reply\n"); + return -1; + } + + req->result = 0; + for (i = 0; i < mp_reply.nb_received; i++) { + struct eal_dev_mp_req *resp = + (struct eal_dev_mp_req *)mp_reply.msgs[i].param; + if (resp->result) { + req->result = resp->result; + break; + } + } + + return 0; +} + +int rte_dev_hotplug_mp_init(void) +{ + int ret; + + if (rte_eal_process_type() == RTE_PROC_PRIMARY) { + ret = rte_mp_action_register(EAL_DEV_MP_ACTION_REQUEST, + handle_secondary_request); + if (ret) { + RTE_LOG(ERR, EAL, "Couldn't register '%s' action\n", + EAL_DEV_MP_ACTION_REQUEST); + return ret; + } + } else { + ret = rte_mp_action_register(EAL_DEV_MP_ACTION_REQUEST, + handle_primary_request); + if (ret) { + RTE_LOG(ERR, EAL, "Couldn't register '%s' action\n", + EAL_DEV_MP_ACTION_REQUEST); + return ret; + } + } + + return 0; +} diff --git a/lib/librte_eal/common/hotplug_mp.h b/lib/librte_eal/common/hotplug_mp.h new file mode 100644 index 000000000..f24ba107b --- /dev/null +++ b/lib/librte_eal/common/hotplug_mp.h @@ -0,0 +1,44 @@ +/* SPDX-License-Identifier: BSD-3-Clause + * Copyright(c) 2018 Intel Corporation + */ + +#ifndef _HOTPLUG_MP_H_ +#define _HOTPLUG_MP_H_ + +#include +#include + +#define EAL_DEV_MP_ACTION_REQUEST "eal_dev_mp_request" +#define EAL_DEV_MP_ACTION_RESPONSE "eal_dev_mp_response" + +enum eal_dev_req_type { + EAL_DEV_REQ_TYPE_ATTACH, + EAL_DEV_REQ_TYPE_DETACH, + EAL_DEV_REQ_TYPE_ATTACH_ROLLBACK, + EAL_DEV_REQ_TYPE_DETACH_ROLLBACK, +}; + +struct eal_dev_mp_req { + enum eal_dev_req_type t; + char busname[RTE_BUS_NAME_MAX_LEN]; + char devname[RTE_DEV_NAME_MAX_LEN]; + char devargs[RTE_DEV_ARGS_MAX_LEN]; + int result; +}; + +/** + * this is a synchronous wrapper for secondary process send + * request to primary process, this is invoked when an attach + * or detach request issued from primary process. + */ +int eal_dev_hotplug_request_to_primary(struct eal_dev_mp_req *req); + +/** + * this is a synchronous wrapper for primary process send + * request to secondary process, this is invoked when an attach + * or detach request issued from secondary process. + */ +int eal_dev_hotplug_request_to_secondary(struct eal_dev_mp_req *req); + + +#endif /* _HOTPLUG_MP_H_ */ diff --git a/lib/librte_eal/common/include/rte_bus.h b/lib/librte_eal/common/include/rte_bus.h index eb9eded4e..720f7c3c8 100644 --- a/lib/librte_eal/common/include/rte_bus.h +++ b/lib/librte_eal/common/include/rte_bus.h @@ -197,6 +197,9 @@ struct rte_bus_conf { typedef enum rte_iova_mode (*rte_bus_get_iommu_class_t)(void); +/* Max length for a bus name */ +#define RTE_BUS_NAME_MAX_LEN 32 + /** * A structure describing a generic bus. */ diff --git a/lib/librte_eal/common/include/rte_dev.h b/lib/librte_eal/common/include/rte_dev.h index 3879ff3ca..667df20f0 100644 --- a/lib/librte_eal/common/include/rte_dev.h +++ b/lib/librte_eal/common/include/rte_dev.h @@ -152,6 +152,9 @@ struct rte_driver { */ #define RTE_DEV_NAME_MAX_LEN 64 +/* Max devargs length be allowed */ +#define RTE_DEV_ARGS_MAX_LEN 128 + /** * A structure describing a generic device. */ @@ -193,6 +196,9 @@ int rte_eal_dev_detach(struct rte_device *dev); * @b EXPERIMENTAL: this API may change without prior notice * * Hotplug add a given device to a specific bus. + * In multi-process, this function will inform all other processes + * to hotplug add the same device. Any failure on other process + * will rollback the action. * * @param busname * The bus name the device is added to. @@ -212,6 +218,9 @@ int __rte_experimental rte_eal_hotplug_add(const char *busname, const char *devn * @b EXPERIMENTAL: this API may change without prior notice * * Hotplug remove a given device from a specific bus. + * In multi-process, this function will inform all other processes + * to hotplug remove the same device. Any failure on other process + * will rollback the action. * * @param busname * The bus name the device is removed from. diff --git a/lib/librte_eal/common/meson.build b/lib/librte_eal/common/meson.build index 8a3dcfee0..0f2cf9a1a 100644 --- a/lib/librte_eal/common/meson.build +++ b/lib/librte_eal/common/meson.build @@ -25,6 +25,7 @@ common_sources = files( 'eal_common_tailqs.c', 'eal_common_thread.c', 'eal_common_timer.c', + 'hotplug_mp.c', 'malloc_elem.c', 'malloc_heap.c', 'malloc_mp.c', diff --git a/lib/librte_eal/linuxapp/eal/Makefile b/lib/librte_eal/linuxapp/eal/Makefile index 3719ec9d7..6c225b54a 100644 --- a/lib/librte_eal/linuxapp/eal/Makefile +++ b/lib/librte_eal/linuxapp/eal/Makefile @@ -67,6 +67,7 @@ SRCS-$(CONFIG_RTE_EXEC_ENV_LINUXAPP) += eal_common_thread.c SRCS-$(CONFIG_RTE_EXEC_ENV_LINUXAPP) += eal_common_proc.c SRCS-$(CONFIG_RTE_EXEC_ENV_LINUXAPP) += eal_common_fbarray.c SRCS-$(CONFIG_RTE_EXEC_ENV_LINUXAPP) += rte_malloc.c +SRCS-$(CONFIG_RTE_EXEC_ENV_LINUXAPP) += hotplug_mp.c SRCS-$(CONFIG_RTE_EXEC_ENV_LINUXAPP) += malloc_elem.c SRCS-$(CONFIG_RTE_EXEC_ENV_LINUXAPP) += malloc_heap.c SRCS-$(CONFIG_RTE_EXEC_ENV_LINUXAPP) += malloc_mp.c diff --git a/lib/librte_eal/linuxapp/eal/eal.c b/lib/librte_eal/linuxapp/eal/eal.c index 8655b8691..0c6984990 100644 --- a/lib/librte_eal/linuxapp/eal/eal.c +++ b/lib/librte_eal/linuxapp/eal/eal.c @@ -850,6 +850,12 @@ rte_eal_init(int argc, char **argv) } } + /* register mp action callbacks for hotplug */ + if (rte_dev_hotplug_mp_init() < 0) { + rte_eal_init_alert("failed to register mp callback for hotplug\n"); + return -1; + } + if (rte_bus_scan()) { rte_eal_init_alert("Cannot scan the buses for devices\n"); rte_errno = ENODEV; From patchwork Mon Jul 9 03:36:53 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Qi Zhang X-Patchwork-Id: 42582 X-Patchwork-Delegate: thomas@monjalon.net Return-Path: X-Original-To: patchwork@dpdk.org Delivered-To: patchwork@dpdk.org Received: from [92.243.14.124] (localhost [127.0.0.1]) by dpdk.org (Postfix) with ESMTP id 629AB1B3AC; Mon, 9 Jul 2018 05:36:54 +0200 (CEST) Received: from mga12.intel.com (mga12.intel.com [192.55.52.136]) by dpdk.org (Postfix) with ESMTP id 42DF21B065 for ; Mon, 9 Jul 2018 05:36:42 +0200 (CEST) X-Amp-Result: SKIPPED(no attachment in message) X-Amp-File-Uploaded: False Received: from fmsmga005.fm.intel.com ([10.253.24.32]) by fmsmga106.fm.intel.com with ESMTP/TLS/DHE-RSA-AES256-GCM-SHA384; 08 Jul 2018 20:36:41 -0700 X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="5.51,328,1526367600"; d="scan'208";a="243632732" Received: from dpdk51.sh.intel.com ([10.67.110.190]) by fmsmga005.fm.intel.com with ESMTP; 08 Jul 2018 20:36:39 -0700 From: Qi Zhang To: thomas@monjalon.net, anatoly.burakov@intel.com Cc: konstantin.ananyev@intel.com, dev@dpdk.org, bruce.richardson@intel.com, ferruh.yigit@intel.com, benjamin.h.shelton@intel.com, narender.vangati@intel.com, Qi Zhang Date: Mon, 9 Jul 2018 11:36:53 +0800 Message-Id: <20180709033706.27858-7-qi.z.zhang@intel.com> X-Mailer: git-send-email 2.13.6 In-Reply-To: <20180709033706.27858-1-qi.z.zhang@intel.com> References: <20180607123849.14439-1-qi.z.zhang@intel.com> <20180709033706.27858-1-qi.z.zhang@intel.com> Subject: [dpdk-dev] [PATCH v10 06/19] eal: support attach or detach share device from secondary X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Sender: "dev" This patch cover the multi-process hotplug case when a device attach/detach request be issued from a secondary process device attach on secondary: a) secondary send sync request to the primary. b) primary receive the request and attach the new device if failed goto i). c) primary forward attach sync request to all secondary. d) secondary receive the request and attach the device and send a reply. e) primary check the reply if all success goes to j). f) primary send attach rollback sync request to all secondary. g) secondary receive the request and detach the device and send a reply. h) primary receive the reply and detach device as rollback action. i) send attach fail to secondary as a reply of step a), goto k). j) send attach success to secondary as a reply of step a). k) secondary receive reply and return. device detach on secondary: a) secondary send sync request to the primary. b) primary send detach sync request to all secondary. c) secondary detach the device and send a reply. d) primary check the reply if all success goes to g). e) primary send detach rollback sync request to all secondary. f) secondary receive the request and attach back device. goto h). g) primary detach the device if success goto i), else goto e). h) primary send detach fail to secondary as a reply of step a), goto j). i) primary send detach success to secondary as a reply of step a). j) secondary receive reply and return. Signed-off-by: Qi Zhang --- lib/librte_eal/common/eal_common_dev.c | 36 ++++++- lib/librte_eal/common/hotplug_mp.c | 175 ++++++++++++++++++++++++++++++++- 2 files changed, 202 insertions(+), 9 deletions(-) diff --git a/lib/librte_eal/common/eal_common_dev.c b/lib/librte_eal/common/eal_common_dev.c index fb1a122ae..195e1fe00 100644 --- a/lib/librte_eal/common/eal_common_dev.c +++ b/lib/librte_eal/common/eal_common_dev.c @@ -218,8 +218,22 @@ rte_eal_hotplug_add(const char *busname, const char *devname, strlcpy(req.devname, devname, RTE_DEV_NAME_MAX_LEN); strlcpy(req.devargs, devargs, RTE_DEV_ARGS_MAX_LEN); - if (rte_eal_process_type() != RTE_PROC_PRIMARY) - return -ENOTSUP; + if (rte_eal_process_type() != RTE_PROC_PRIMARY) { + /** + * If in secondary process, just send IPC request to + * primary process. + */ + ret = eal_dev_hotplug_request_to_primary(&req); + if (ret) { + RTE_LOG(ERR, EAL, + "Failed to send hotplug request to primary\n"); + return ret; + } + if (req.result) + RTE_LOG(ERR, EAL, + "Failed to hotplug add device\n"); + return req.result; + } /** * attach a device from primary start from here: @@ -279,8 +293,22 @@ rte_eal_hotplug_remove(const char *busname, const char *devname) strlcpy(req.busname, busname, RTE_BUS_NAME_MAX_LEN); strlcpy(req.devname, devname, RTE_DEV_NAME_MAX_LEN); - if (rte_eal_process_type() != RTE_PROC_PRIMARY) - return -ENOTSUP; + if (rte_eal_process_type() != RTE_PROC_PRIMARY) { + /** + * If in secondary process, just send IPC request to + * primary process. + */ + ret = eal_dev_hotplug_request_to_primary(&req); + if (ret) { + RTE_LOG(ERR, EAL, + "Failed to send hotplug request to primary\n"); + return ret; + } + if (req.result) + RTE_LOG(ERR, EAL, + "Failed to hotplug remove device\n"); + return req.result; + } /** * detach a device from primary start from here: diff --git a/lib/librte_eal/common/hotplug_mp.c b/lib/librte_eal/common/hotplug_mp.c index 261d17fe6..68ca18bbe 100644 --- a/lib/librte_eal/common/hotplug_mp.c +++ b/lib/librte_eal/common/hotplug_mp.c @@ -17,12 +17,158 @@ struct mp_reply_bundle { void *peer; }; +/** + * Secondary to primary request. + * start from function eal_dev_hotplug_request_to_primary. + * + * device attach on secondary: + * a) secondary send sync request to the primary. + * b) primary receive the request and attach the new device if + * failed goto i). + * c) primary forward attach sync request to all secondary. + * d) secondary receive the request and attach the device and send a reply. + * e) primary check the reply if all success goes to j). + * f) primary send attach rollback sync request to all secondary. + * g) secondary receive the request and detach the device and send a reply. + * h) primary receive the reply and detach device as rollback action. + * i) send attach fail to secondary as a reply of step a), goto k). + * j) send attach success to secondary as a reply of step a). + * k) secondary receive reply and return. + * + * device detach on secondary: + * a) secondary send sync request to the primary. + * b) primary send detach sync request to all secondary. + * c) secondary detach the device and send a reply. + * d) primary check the reply if all success goes to g). + * e) primary send detach rollback sync request to all secondary. + * f) secondary receive the request and attach back device. goto h). + * g) primary detach the device if success goto i), else goto e). + * h) primary send detach fail to secondary as a reply of step a), goto j). + * i) primary send detach success to secondary as a reply of step a). + * j) secondary receive reply and return. + */ +static int +send_response_to_secondary(const struct eal_dev_mp_req *req, + int result, + const void *peer) +{ + struct rte_mp_msg mp_resp; + struct eal_dev_mp_req *resp = + (struct eal_dev_mp_req *)mp_resp.param; + int ret; + + memset(&mp_resp, 0, sizeof(mp_resp)); + mp_resp.len_param = sizeof(*resp); + strlcpy(mp_resp.name, EAL_DEV_MP_ACTION_REQUEST, sizeof(mp_resp.name)); + memcpy(resp, req, sizeof(*req)); + resp->result = result; + + ret = rte_mp_reply(&mp_resp, peer); + if (ret) + RTE_LOG(ERR, EAL, "failed to send response to secondary\n"); + + return ret; +} + +static void +__handle_secondary_request(void *param) +{ + struct mp_reply_bundle *bundle = param; + const struct rte_mp_msg *msg = &bundle->msg; + const struct eal_dev_mp_req *req = + (const struct eal_dev_mp_req *)msg->param; + struct eal_dev_mp_req tmp_req; + int ret = 0; + + tmp_req = *req; + + if (req->t == EAL_DEV_REQ_TYPE_ATTACH) { + ret = do_dev_hotplug_add(req->busname, req->devname, + req->devargs); + if (ret) { + RTE_LOG(ERR, EAL, "Failed to hotplug add device on primary\n"); + goto finish; + } + ret = eal_dev_hotplug_request_to_secondary(&tmp_req); + if (ret) { + RTE_LOG(ERR, EAL, "Failed to send hotplug request to secondary\n"); + goto finish; + } + if (tmp_req.result) { + ret = tmp_req.result; + RTE_LOG(ERR, EAL, "Failed to hotplug add device on secondary\n"); + } + } else if (req->t == EAL_DEV_REQ_TYPE_DETACH) { + ret = eal_dev_hotplug_request_to_secondary(&tmp_req); + if (ret) { + RTE_LOG(ERR, EAL, "Failed to send hotplug request to secondary\n"); + goto finish; + } + if (!tmp_req.result) { + ret = do_dev_hotplug_remove(req->busname, + req->devname); + if (ret) { + RTE_LOG(ERR, EAL, "Failed to hotplug remove device on primary\n"); + goto rollback; + } + } else { + RTE_LOG(ERR, EAL, "Failed to hotplug remove device on secondary\n"); + ret = tmp_req.result; + goto rollback; + } + } else { + RTE_LOG(ERR, EAL, "unsupported secondary to primary request\n"); + ret = -ENOTSUP; + } + goto finish; + +rollback: + tmp_req.t = EAL_DEV_REQ_TYPE_DETACH_ROLLBACK; + ret = eal_dev_hotplug_request_to_secondary(&tmp_req); + if (ret) + RTE_LOG(ERR, EAL, "Failed to send hotplug rollback request to secondary\n"); + +finish: + ret = send_response_to_secondary(&tmp_req, ret, bundle->peer); + if (ret) + RTE_LOG(ERR, EAL, "failed to send response to secondary\n"); + + free(bundle->peer); + free(bundle); +} + static int handle_secondary_request(const struct rte_mp_msg *msg, const void *peer) { - RTE_SET_USED(msg); - RTE_SET_USED(peer); - return -ENOTSUP; + struct mp_reply_bundle *bundle; + const struct eal_dev_mp_req *req = + (const struct eal_dev_mp_req *)msg->param; + int ret = 0; + + bundle = malloc(sizeof(*bundle)); + if (bundle == NULL) { + RTE_LOG(ERR, EAL, "not enough memory\n"); + return send_response_to_secondary(req, -ENOMEM, peer); + } + + bundle->msg = *msg; + /** + * We need to send reply on interrupt thread, but peer can't be + * parsed directly, so this is a temporal hack, need to be fixed + * when it is ready. + */ + bundle->peer = strdup(peer); + + /** + * We are at IPC callback thread, sync IPC is not allowed due to + * dead lock, so we delegate the task to interrupt thread. + */ + ret = rte_eal_alarm_set(1, __handle_secondary_request, bundle); + if (ret) { + RTE_LOG(ERR, EAL, "failed to add mp task\n"); + return send_response_to_secondary(req, ret, peer); + } + return 0; } static void __handle_primary_request(void *param) @@ -114,8 +260,27 @@ handle_primary_request(const struct rte_mp_msg *msg, const void *peer) int eal_dev_hotplug_request_to_primary(struct eal_dev_mp_req *req) { - RTE_SET_USED(req); - return -ENOTSUP; + struct rte_mp_msg mp_req; + struct rte_mp_reply mp_reply; + struct timespec ts = {.tv_sec = MP_TIMEOUT_S, .tv_nsec = 0}; + struct eal_dev_mp_req *resp; + int ret; + + memset(&mp_req, 0, sizeof(mp_req)); + memcpy(mp_req.param, req, sizeof(*req)); + mp_req.len_param = sizeof(*req); + strlcpy(mp_req.name, EAL_DEV_MP_ACTION_REQUEST, sizeof(mp_req.name)); + + ret = rte_mp_request_sync(&mp_req, &mp_reply, &ts); + if (ret) { + RTE_LOG(ERR, EAL, "cannot send request to primary"); + return ret; + } + + resp = (struct eal_dev_mp_req *)mp_reply.msgs[0].param; + req->result = resp->result; + + return ret; } int eal_dev_hotplug_request_to_secondary(struct eal_dev_mp_req *req) From patchwork Mon Jul 9 03:36:54 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Qi Zhang X-Patchwork-Id: 42583 X-Patchwork-Delegate: thomas@monjalon.net Return-Path: X-Original-To: patchwork@dpdk.org Delivered-To: patchwork@dpdk.org Received: from [92.243.14.124] (localhost [127.0.0.1]) by dpdk.org (Postfix) with ESMTP id 08DA41B3B4; Mon, 9 Jul 2018 05:36:56 +0200 (CEST) Received: from mga12.intel.com (mga12.intel.com [192.55.52.136]) by dpdk.org (Postfix) with ESMTP id 274F51B1C3 for ; Mon, 9 Jul 2018 05:36:44 +0200 (CEST) X-Amp-Result: SKIPPED(no attachment in message) X-Amp-File-Uploaded: False Received: from fmsmga005.fm.intel.com ([10.253.24.32]) by fmsmga106.fm.intel.com with ESMTP/TLS/DHE-RSA-AES256-GCM-SHA384; 08 Jul 2018 20:36:43 -0700 X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="5.51,328,1526367600"; d="scan'208";a="243632736" Received: from dpdk51.sh.intel.com ([10.67.110.190]) by fmsmga005.fm.intel.com with ESMTP; 08 Jul 2018 20:36:41 -0700 From: Qi Zhang To: thomas@monjalon.net, anatoly.burakov@intel.com Cc: konstantin.ananyev@intel.com, dev@dpdk.org, bruce.richardson@intel.com, ferruh.yigit@intel.com, benjamin.h.shelton@intel.com, narender.vangati@intel.com, Qi Zhang Date: Mon, 9 Jul 2018 11:36:54 +0800 Message-Id: <20180709033706.27858-8-qi.z.zhang@intel.com> X-Mailer: git-send-email 2.13.6 In-Reply-To: <20180709033706.27858-1-qi.z.zhang@intel.com> References: <20180607123849.14439-1-qi.z.zhang@intel.com> <20180709033706.27858-1-qi.z.zhang@intel.com> Subject: [dpdk-dev] [PATCH v10 07/19] net/i40e: enable hotplug on secondary process X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Sender: "dev" Previously, detach port on a secondary process will mess primary process and cause the same device can't be attached back again. A secondary process should use rte_eth_release_port_private to release a port. Signed-off-by: Qi Zhang --- drivers/net/i40e/i40e_ethdev.c | 2 ++ 1 file changed, 2 insertions(+) diff --git a/drivers/net/i40e/i40e_ethdev.c b/drivers/net/i40e/i40e_ethdev.c index 13c5d3296..7d1f98422 100644 --- a/drivers/net/i40e/i40e_ethdev.c +++ b/drivers/net/i40e/i40e_ethdev.c @@ -678,6 +678,8 @@ static int eth_i40e_pci_remove(struct rte_pci_device *pci_dev) if (!ethdev) return -ENODEV; + if (rte_eal_process_type() != RTE_PROC_PRIMARY) + return rte_eth_dev_release_port_private(ethdev); if (ethdev->data->dev_flags & RTE_ETH_DEV_REPRESENTOR) return rte_eth_dev_destroy(ethdev, i40e_vf_representor_uninit); From patchwork Mon Jul 9 03:36:55 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Qi Zhang X-Patchwork-Id: 42584 X-Patchwork-Delegate: thomas@monjalon.net Return-Path: X-Original-To: patchwork@dpdk.org Delivered-To: patchwork@dpdk.org Received: from [92.243.14.124] (localhost [127.0.0.1]) by dpdk.org (Postfix) with ESMTP id 0755C1B3B9; Mon, 9 Jul 2018 05:36:58 +0200 (CEST) Received: from mga12.intel.com (mga12.intel.com [192.55.52.136]) by dpdk.org (Postfix) with ESMTP id 94B271B1CC for ; Mon, 9 Jul 2018 05:36:45 +0200 (CEST) X-Amp-Result: SKIPPED(no attachment in message) X-Amp-File-Uploaded: False Received: from fmsmga005.fm.intel.com ([10.253.24.32]) by fmsmga106.fm.intel.com with ESMTP/TLS/DHE-RSA-AES256-GCM-SHA384; 08 Jul 2018 20:36:45 -0700 X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="5.51,328,1526367600"; d="scan'208";a="243632742" Received: from dpdk51.sh.intel.com ([10.67.110.190]) by fmsmga005.fm.intel.com with ESMTP; 08 Jul 2018 20:36:43 -0700 From: Qi Zhang To: thomas@monjalon.net, anatoly.burakov@intel.com Cc: konstantin.ananyev@intel.com, dev@dpdk.org, bruce.richardson@intel.com, ferruh.yigit@intel.com, benjamin.h.shelton@intel.com, narender.vangati@intel.com, Qi Zhang Date: Mon, 9 Jul 2018 11:36:55 +0800 Message-Id: <20180709033706.27858-9-qi.z.zhang@intel.com> X-Mailer: git-send-email 2.13.6 In-Reply-To: <20180709033706.27858-1-qi.z.zhang@intel.com> References: <20180607123849.14439-1-qi.z.zhang@intel.com> <20180709033706.27858-1-qi.z.zhang@intel.com> Subject: [dpdk-dev] [PATCH v10 08/19] net/ixgbe: enable hotplug on secondary process X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Sender: "dev" Previously, detach port on a secondary process will mess primary process and cause the same device can't be attached back again. A secondary process should use rte_eth_release_port_private to release a port. Signed-off-by: Qi Zhang --- drivers/net/ixgbe/ixgbe_ethdev.c | 3 +++ 1 file changed, 3 insertions(+) diff --git a/drivers/net/ixgbe/ixgbe_ethdev.c b/drivers/net/ixgbe/ixgbe_ethdev.c index 87d2ad090..161a15f05 100644 --- a/drivers/net/ixgbe/ixgbe_ethdev.c +++ b/drivers/net/ixgbe/ixgbe_ethdev.c @@ -1792,6 +1792,9 @@ static int eth_ixgbe_pci_remove(struct rte_pci_device *pci_dev) if (!ethdev) return -ENODEV; + if (rte_eal_process_type() != RTE_PROC_PRIMARY) + return rte_eth_dev_release_port_private(ethdev); + if (ethdev->data->dev_flags & RTE_ETH_DEV_REPRESENTOR) return rte_eth_dev_destroy(ethdev, ixgbe_vf_representor_uninit); else From patchwork Mon Jul 9 03:36:56 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Qi Zhang X-Patchwork-Id: 42585 X-Patchwork-Delegate: thomas@monjalon.net Return-Path: X-Original-To: patchwork@dpdk.org Delivered-To: patchwork@dpdk.org Received: from [92.243.14.124] (localhost [127.0.0.1]) by dpdk.org (Postfix) with ESMTP id 1D8671B42B; Mon, 9 Jul 2018 05:37:00 +0200 (CEST) Received: from mga12.intel.com (mga12.intel.com [192.55.52.136]) by dpdk.org (Postfix) with ESMTP id 2480F1B1F7 for ; Mon, 9 Jul 2018 05:36:47 +0200 (CEST) X-Amp-Result: SKIPPED(no attachment in message) X-Amp-File-Uploaded: False Received: from fmsmga005.fm.intel.com ([10.253.24.32]) by fmsmga106.fm.intel.com with ESMTP/TLS/DHE-RSA-AES256-GCM-SHA384; 08 Jul 2018 20:36:46 -0700 X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="5.51,328,1526367600"; d="scan'208";a="243632748" Received: from dpdk51.sh.intel.com ([10.67.110.190]) by fmsmga005.fm.intel.com with ESMTP; 08 Jul 2018 20:36:45 -0700 From: Qi Zhang To: thomas@monjalon.net, anatoly.burakov@intel.com Cc: konstantin.ananyev@intel.com, dev@dpdk.org, bruce.richardson@intel.com, ferruh.yigit@intel.com, benjamin.h.shelton@intel.com, narender.vangati@intel.com, Qi Zhang Date: Mon, 9 Jul 2018 11:36:56 +0800 Message-Id: <20180709033706.27858-10-qi.z.zhang@intel.com> X-Mailer: git-send-email 2.13.6 In-Reply-To: <20180709033706.27858-1-qi.z.zhang@intel.com> References: <20180607123849.14439-1-qi.z.zhang@intel.com> <20180709033706.27858-1-qi.z.zhang@intel.com> Subject: [dpdk-dev] [PATCH v10 09/19] net/af_packet: enable hotplug on secondary process X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Sender: "dev" Attach port from secondary should ignore devargs since the private device is not necessary to support. Also previously, detach port on a secondary process will mess primary process and cause the same device can't be attached back again. A secondary process should use rte_eth_release_port_private to release a port. Signed-off-by: Qi Zhang --- drivers/net/af_packet/rte_eth_af_packet.c | 7 +++++-- 1 file changed, 5 insertions(+), 2 deletions(-) diff --git a/drivers/net/af_packet/rte_eth_af_packet.c b/drivers/net/af_packet/rte_eth_af_packet.c index ea47abbf8..9b0dabeeb 100644 --- a/drivers/net/af_packet/rte_eth_af_packet.c +++ b/drivers/net/af_packet/rte_eth_af_packet.c @@ -926,8 +926,7 @@ rte_pmd_af_packet_probe(struct rte_vdev_device *dev) PMD_LOG(INFO, "Initializing pmd_af_packet for %s", name); - if (rte_eal_process_type() == RTE_PROC_SECONDARY && - strlen(rte_vdev_device_args(dev)) == 0) { + if (rte_eal_process_type() == RTE_PROC_SECONDARY) { eth_dev = rte_eth_dev_attach_secondary(name); if (!eth_dev) { PMD_LOG(ERR, "Failed to probe %s", name); @@ -935,6 +934,7 @@ rte_pmd_af_packet_probe(struct rte_vdev_device *dev) } /* TODO: request info from primary to set up Rx and Tx */ eth_dev->dev_ops = &ops; + eth_dev->device = &dev->device; rte_eth_dev_probing_finish(eth_dev); return 0; } @@ -986,6 +986,9 @@ rte_pmd_af_packet_remove(struct rte_vdev_device *dev) if (eth_dev == NULL) return -1; + if (rte_eal_process_type() != RTE_PROC_PRIMARY) + return rte_eth_dev_release_port_private(eth_dev); + internals = eth_dev->data->dev_private; for (q = 0; q < internals->nb_queues; q++) { rte_free(internals->rx_queue[q].rd); From patchwork Mon Jul 9 03:36:57 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Qi Zhang X-Patchwork-Id: 42586 X-Patchwork-Delegate: thomas@monjalon.net Return-Path: X-Original-To: patchwork@dpdk.org Delivered-To: patchwork@dpdk.org Received: from [92.243.14.124] (localhost [127.0.0.1]) by dpdk.org (Postfix) with ESMTP id B2C571B42E; Mon, 9 Jul 2018 05:37:01 +0200 (CEST) Received: from mga12.intel.com (mga12.intel.com [192.55.52.136]) by dpdk.org (Postfix) with ESMTP id DEE1B1B068 for ; Mon, 9 Jul 2018 05:36:48 +0200 (CEST) X-Amp-Result: SKIPPED(no attachment in message) X-Amp-File-Uploaded: False Received: from fmsmga005.fm.intel.com ([10.253.24.32]) by fmsmga106.fm.intel.com with ESMTP/TLS/DHE-RSA-AES256-GCM-SHA384; 08 Jul 2018 20:36:48 -0700 X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="5.51,328,1526367600"; d="scan'208";a="243632762" Received: from dpdk51.sh.intel.com ([10.67.110.190]) by fmsmga005.fm.intel.com with ESMTP; 08 Jul 2018 20:36:47 -0700 From: Qi Zhang To: thomas@monjalon.net, anatoly.burakov@intel.com Cc: konstantin.ananyev@intel.com, dev@dpdk.org, bruce.richardson@intel.com, ferruh.yigit@intel.com, benjamin.h.shelton@intel.com, narender.vangati@intel.com, Qi Zhang Date: Mon, 9 Jul 2018 11:36:57 +0800 Message-Id: <20180709033706.27858-11-qi.z.zhang@intel.com> X-Mailer: git-send-email 2.13.6 In-Reply-To: <20180709033706.27858-1-qi.z.zhang@intel.com> References: <20180607123849.14439-1-qi.z.zhang@intel.com> <20180709033706.27858-1-qi.z.zhang@intel.com> Subject: [dpdk-dev] [PATCH v10 10/19] net/bonding: enable hotplug on secondary process X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Sender: "dev" Attach port from secondary should ignore devargs since the private device is not necessary to support. Also previously, detach port on a secondary process will mess primary process and cause the same device can't be attached back again. A secondary process should use rte_eth_release_port_private to release a port. Signed-off-by: Qi Zhang --- drivers/net/bonding/rte_eth_bond_pmd.c | 7 +++++-- 1 file changed, 5 insertions(+), 2 deletions(-) diff --git a/drivers/net/bonding/rte_eth_bond_pmd.c b/drivers/net/bonding/rte_eth_bond_pmd.c index f155ff779..91c352c4c 100644 --- a/drivers/net/bonding/rte_eth_bond_pmd.c +++ b/drivers/net/bonding/rte_eth_bond_pmd.c @@ -3053,8 +3053,7 @@ bond_probe(struct rte_vdev_device *dev) name = rte_vdev_device_name(dev); RTE_BOND_LOG(INFO, "Initializing pmd_bond for %s", name); - if (rte_eal_process_type() == RTE_PROC_SECONDARY && - strlen(rte_vdev_device_args(dev)) == 0) { + if (rte_eal_process_type() == RTE_PROC_SECONDARY) { eth_dev = rte_eth_dev_attach_secondary(name); if (!eth_dev) { RTE_BOND_LOG(ERR, "Failed to probe %s", name); @@ -3062,6 +3061,7 @@ bond_probe(struct rte_vdev_device *dev) } /* TODO: request info from primary to set up Rx and Tx */ eth_dev->dev_ops = &default_dev_ops; + eth_dev->device = &dev->device; rte_eth_dev_probing_finish(eth_dev); return 0; } @@ -3168,6 +3168,9 @@ bond_remove(struct rte_vdev_device *dev) if (eth_dev == NULL) return -ENODEV; + if (rte_eal_process_type() != RTE_PROC_PRIMARY) + return rte_eth_dev_release_port_private(eth_dev); + RTE_ASSERT(eth_dev->device == &dev->device); internals = eth_dev->data->dev_private; From patchwork Mon Jul 9 03:36:58 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Qi Zhang X-Patchwork-Id: 42587 X-Patchwork-Delegate: thomas@monjalon.net Return-Path: X-Original-To: patchwork@dpdk.org Delivered-To: patchwork@dpdk.org Received: from [92.243.14.124] (localhost [127.0.0.1]) by dpdk.org (Postfix) with ESMTP id 9DEB01B433; Mon, 9 Jul 2018 05:37:03 +0200 (CEST) Received: from mga12.intel.com (mga12.intel.com [192.55.52.136]) by dpdk.org (Postfix) with ESMTP id B39131B39A for ; Mon, 9 Jul 2018 05:36:50 +0200 (CEST) X-Amp-Result: SKIPPED(no attachment in message) X-Amp-File-Uploaded: False Received: from fmsmga005.fm.intel.com ([10.253.24.32]) by fmsmga106.fm.intel.com with ESMTP/TLS/DHE-RSA-AES256-GCM-SHA384; 08 Jul 2018 20:36:50 -0700 X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="5.51,328,1526367600"; d="scan'208";a="243632768" Received: from dpdk51.sh.intel.com ([10.67.110.190]) by fmsmga005.fm.intel.com with ESMTP; 08 Jul 2018 20:36:48 -0700 From: Qi Zhang To: thomas@monjalon.net, anatoly.burakov@intel.com Cc: konstantin.ananyev@intel.com, dev@dpdk.org, bruce.richardson@intel.com, ferruh.yigit@intel.com, benjamin.h.shelton@intel.com, narender.vangati@intel.com, Qi Zhang Date: Mon, 9 Jul 2018 11:36:58 +0800 Message-Id: <20180709033706.27858-12-qi.z.zhang@intel.com> X-Mailer: git-send-email 2.13.6 In-Reply-To: <20180709033706.27858-1-qi.z.zhang@intel.com> References: <20180607123849.14439-1-qi.z.zhang@intel.com> <20180709033706.27858-1-qi.z.zhang@intel.com> Subject: [dpdk-dev] [PATCH v10 11/19] net/kni: enable hotplug on secondary process X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Sender: "dev" Attach port from secondary should ignore devargs since the private device is not necessary to support. Also previously, detach port on a secondary process will mess primary process and cause the same device can't be attached back again. A secondary process should use rte_eth_release_port_private to release a port. Signed-off-by: Qi Zhang --- drivers/net/kni/rte_eth_kni.c | 7 +++++-- 1 file changed, 5 insertions(+), 2 deletions(-) diff --git a/drivers/net/kni/rte_eth_kni.c b/drivers/net/kni/rte_eth_kni.c index ab63ea427..984d0d29a 100644 --- a/drivers/net/kni/rte_eth_kni.c +++ b/drivers/net/kni/rte_eth_kni.c @@ -410,8 +410,7 @@ eth_kni_probe(struct rte_vdev_device *vdev) params = rte_vdev_device_args(vdev); PMD_LOG(INFO, "Initializing eth_kni for %s", name); - if (rte_eal_process_type() == RTE_PROC_SECONDARY && - strlen(params) == 0) { + if (rte_eal_process_type() == RTE_PROC_SECONDARY) { eth_dev = rte_eth_dev_attach_secondary(name); if (!eth_dev) { PMD_LOG(ERR, "Failed to probe %s", name); @@ -419,6 +418,7 @@ eth_kni_probe(struct rte_vdev_device *vdev) } /* TODO: request info from primary to set up Rx and Tx */ eth_dev->dev_ops = ð_kni_ops; + eth_dev->device = &vdev->device; rte_eth_dev_probing_finish(eth_dev); return 0; } @@ -463,6 +463,9 @@ eth_kni_remove(struct rte_vdev_device *vdev) if (eth_dev == NULL) return -1; + if (rte_eal_process_type() != RTE_PROC_PRIMARY) + return rte_eth_dev_release_port_private(eth_dev); + eth_kni_dev_stop(eth_dev); internals = eth_dev->data->dev_private; From patchwork Mon Jul 9 03:36:59 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Qi Zhang X-Patchwork-Id: 42588 X-Patchwork-Delegate: thomas@monjalon.net Return-Path: X-Original-To: patchwork@dpdk.org Delivered-To: patchwork@dpdk.org Received: from [92.243.14.124] (localhost [127.0.0.1]) by dpdk.org (Postfix) with ESMTP id A6BB91B437; Mon, 9 Jul 2018 05:37:06 +0200 (CEST) Received: from mga12.intel.com (mga12.intel.com [192.55.52.136]) by dpdk.org (Postfix) with ESMTP id 2CDDC1B3A2 for ; Mon, 9 Jul 2018 05:36:52 +0200 (CEST) X-Amp-Result: SKIPPED(no attachment in message) X-Amp-File-Uploaded: False Received: from fmsmga005.fm.intel.com ([10.253.24.32]) by fmsmga106.fm.intel.com with ESMTP/TLS/DHE-RSA-AES256-GCM-SHA384; 08 Jul 2018 20:36:51 -0700 X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="5.51,328,1526367600"; d="scan'208";a="243632775" Received: from dpdk51.sh.intel.com ([10.67.110.190]) by fmsmga005.fm.intel.com with ESMTP; 08 Jul 2018 20:36:50 -0700 From: Qi Zhang To: thomas@monjalon.net, anatoly.burakov@intel.com Cc: konstantin.ananyev@intel.com, dev@dpdk.org, bruce.richardson@intel.com, ferruh.yigit@intel.com, benjamin.h.shelton@intel.com, narender.vangati@intel.com, Qi Zhang Date: Mon, 9 Jul 2018 11:36:59 +0800 Message-Id: <20180709033706.27858-13-qi.z.zhang@intel.com> X-Mailer: git-send-email 2.13.6 In-Reply-To: <20180709033706.27858-1-qi.z.zhang@intel.com> References: <20180607123849.14439-1-qi.z.zhang@intel.com> <20180709033706.27858-1-qi.z.zhang@intel.com> Subject: [dpdk-dev] [PATCH v10 12/19] net/null: enable hotplug on secondary process X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Sender: "dev" Attach port from secondary should ignore devargs since the private device is not necessary to support. Also previously, detach port on a secondary process will mess primary process and cause the same device can't be attached back again. A secondary process should use rte_eth_release_port_private to release a port. Signed-off-by: Qi Zhang --- drivers/net/null/rte_eth_null.c | 12 +++++++++--- 1 file changed, 9 insertions(+), 3 deletions(-) diff --git a/drivers/net/null/rte_eth_null.c b/drivers/net/null/rte_eth_null.c index 1d2e6b9e9..4d5989bd1 100644 --- a/drivers/net/null/rte_eth_null.c +++ b/drivers/net/null/rte_eth_null.c @@ -614,8 +614,7 @@ rte_pmd_null_probe(struct rte_vdev_device *dev) params = rte_vdev_device_args(dev); PMD_LOG(INFO, "Initializing pmd_null for %s", name); - if (rte_eal_process_type() == RTE_PROC_SECONDARY && - strlen(params) == 0) { + if (rte_eal_process_type() == RTE_PROC_SECONDARY) { eth_dev = rte_eth_dev_attach_secondary(name); if (!eth_dev) { PMD_LOG(ERR, "Failed to probe %s", name); @@ -623,6 +622,7 @@ rte_pmd_null_probe(struct rte_vdev_device *dev) } /* TODO: request info from primary to set up Rx and Tx */ eth_dev->dev_ops = &ops; + eth_dev->device = &dev->device; rte_eth_dev_probing_finish(eth_dev); return 0; } @@ -667,18 +667,24 @@ static int rte_pmd_null_remove(struct rte_vdev_device *dev) { struct rte_eth_dev *eth_dev = NULL; + const char *name; if (!dev) return -EINVAL; + name = rte_vdev_device_name(dev); + PMD_LOG(INFO, "Closing null ethdev on numa socket %u", rte_socket_id()); /* find the ethdev entry */ - eth_dev = rte_eth_dev_allocated(rte_vdev_device_name(dev)); + eth_dev = rte_eth_dev_allocated(name); if (eth_dev == NULL) return -1; + if (rte_eal_process_type() != RTE_PROC_PRIMARY) + return rte_eth_dev_release_port_private(eth_dev); + rte_free(eth_dev->data->dev_private); rte_eth_dev_release_port(eth_dev); From patchwork Mon Jul 9 03:37:00 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Qi Zhang X-Patchwork-Id: 42589 X-Patchwork-Delegate: thomas@monjalon.net Return-Path: X-Original-To: patchwork@dpdk.org Delivered-To: patchwork@dpdk.org Received: from [92.243.14.124] (localhost [127.0.0.1]) by dpdk.org (Postfix) with ESMTP id 341F51B43D; Mon, 9 Jul 2018 05:37:09 +0200 (CEST) Received: from mga12.intel.com (mga12.intel.com [192.55.52.136]) by dpdk.org (Postfix) with ESMTP id B20A71B05D for ; Mon, 9 Jul 2018 05:36:53 +0200 (CEST) X-Amp-Result: SKIPPED(no attachment in message) X-Amp-File-Uploaded: False Received: from fmsmga005.fm.intel.com ([10.253.24.32]) by fmsmga106.fm.intel.com with ESMTP/TLS/DHE-RSA-AES256-GCM-SHA384; 08 Jul 2018 20:36:53 -0700 X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="5.51,328,1526367600"; d="scan'208";a="243632784" Received: from dpdk51.sh.intel.com ([10.67.110.190]) by fmsmga005.fm.intel.com with ESMTP; 08 Jul 2018 20:36:52 -0700 From: Qi Zhang To: thomas@monjalon.net, anatoly.burakov@intel.com Cc: konstantin.ananyev@intel.com, dev@dpdk.org, bruce.richardson@intel.com, ferruh.yigit@intel.com, benjamin.h.shelton@intel.com, narender.vangati@intel.com, Qi Zhang Date: Mon, 9 Jul 2018 11:37:00 +0800 Message-Id: <20180709033706.27858-14-qi.z.zhang@intel.com> X-Mailer: git-send-email 2.13.6 In-Reply-To: <20180709033706.27858-1-qi.z.zhang@intel.com> References: <20180607123849.14439-1-qi.z.zhang@intel.com> <20180709033706.27858-1-qi.z.zhang@intel.com> Subject: [dpdk-dev] [PATCH v10 13/19] net/octeontx: enable hotplug on secondary process X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Sender: "dev" Attach port from secondary should ignore devargs since the private device is not necessary to support. Also previously, detach port on a secondary process will mess primary process and cause the same device can't be attached back again. A secondary process should use rte_eth_release_port_private to release a port. Signed-off-by: Qi Zhang --- drivers/net/octeontx/octeontx_ethdev.c | 9 +++++++++ 1 file changed, 9 insertions(+) diff --git a/drivers/net/octeontx/octeontx_ethdev.c b/drivers/net/octeontx/octeontx_ethdev.c index 1eb453b21..1b98f4086 100644 --- a/drivers/net/octeontx/octeontx_ethdev.c +++ b/drivers/net/octeontx/octeontx_ethdev.c @@ -1016,6 +1016,7 @@ octeontx_create(struct rte_vdev_device *dev, int port, uint8_t evdev, eth_dev->tx_pkt_burst = octeontx_xmit_pkts; eth_dev->rx_pkt_burst = octeontx_recv_pkts; + eth_dev->device = &dev->device; rte_eth_dev_probing_finish(eth_dev); return 0; } @@ -1138,6 +1139,11 @@ octeontx_remove(struct rte_vdev_device *dev) if (eth_dev == NULL) return -ENODEV; + if (rte_eal_process_type() != RTE_PROC_PRIMARY) { + rte_eth_dev_release_port_private(eth_dev); + continue; + } + nic = octeontx_pmd_priv(eth_dev); rte_event_dev_stop(nic->evdev); PMD_INIT_LOG(INFO, "Closing octeontx device %s", octtx_name); @@ -1148,6 +1154,9 @@ octeontx_remove(struct rte_vdev_device *dev) rte_event_dev_close(nic->evdev); } + if (rte_eal_process_type() != RTE_PROC_PRIMARY) + return 0; + /* Free FC resource */ octeontx_pko_fc_free(); From patchwork Mon Jul 9 03:37:01 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Qi Zhang X-Patchwork-Id: 42590 X-Patchwork-Delegate: thomas@monjalon.net Return-Path: X-Original-To: patchwork@dpdk.org Delivered-To: patchwork@dpdk.org Received: from [92.243.14.124] (localhost [127.0.0.1]) by dpdk.org (Postfix) with ESMTP id 0DAEF1B443; Mon, 9 Jul 2018 05:37:11 +0200 (CEST) Received: from mga12.intel.com (mga12.intel.com [192.55.52.136]) by dpdk.org (Postfix) with ESMTP id AD4F91C0B for ; Mon, 9 Jul 2018 05:36:55 +0200 (CEST) X-Amp-Result: SKIPPED(no attachment in message) X-Amp-File-Uploaded: False Received: from fmsmga005.fm.intel.com ([10.253.24.32]) by fmsmga106.fm.intel.com with ESMTP/TLS/DHE-RSA-AES256-GCM-SHA384; 08 Jul 2018 20:36:55 -0700 X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="5.51,328,1526367600"; d="scan'208";a="243632791" Received: from dpdk51.sh.intel.com ([10.67.110.190]) by fmsmga005.fm.intel.com with ESMTP; 08 Jul 2018 20:36:53 -0700 From: Qi Zhang To: thomas@monjalon.net, anatoly.burakov@intel.com Cc: konstantin.ananyev@intel.com, dev@dpdk.org, bruce.richardson@intel.com, ferruh.yigit@intel.com, benjamin.h.shelton@intel.com, narender.vangati@intel.com, Qi Zhang Date: Mon, 9 Jul 2018 11:37:01 +0800 Message-Id: <20180709033706.27858-15-qi.z.zhang@intel.com> X-Mailer: git-send-email 2.13.6 In-Reply-To: <20180709033706.27858-1-qi.z.zhang@intel.com> References: <20180607123849.14439-1-qi.z.zhang@intel.com> <20180709033706.27858-1-qi.z.zhang@intel.com> Subject: [dpdk-dev] [PATCH v10 14/19] net/pcap: enable hotplug on secondary process X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Sender: "dev" Attach port from secondary should ignore devargs since the private device is not necessary to support. Also previously, detach port on a secondary process will mess primary process and cause the same device can't be attached back again. A secondary process should use rte_eth_release_port_private to release a port. Signed-off-by: Qi Zhang --- drivers/net/pcap/rte_eth_pcap.c | 11 ++++++++--- 1 file changed, 8 insertions(+), 3 deletions(-) diff --git a/drivers/net/pcap/rte_eth_pcap.c b/drivers/net/pcap/rte_eth_pcap.c index 6bd4a7d79..3c27a98df 100644 --- a/drivers/net/pcap/rte_eth_pcap.c +++ b/drivers/net/pcap/rte_eth_pcap.c @@ -916,8 +916,7 @@ pmd_pcap_probe(struct rte_vdev_device *dev) start_cycles = rte_get_timer_cycles(); hz = rte_get_timer_hz(); - if (rte_eal_process_type() == RTE_PROC_SECONDARY && - strlen(rte_vdev_device_args(dev)) == 0) { + if (rte_eal_process_type() == RTE_PROC_SECONDARY) { eth_dev = rte_eth_dev_attach_secondary(name); if (!eth_dev) { PMD_LOG(ERR, "Failed to probe %s", name); @@ -925,6 +924,7 @@ pmd_pcap_probe(struct rte_vdev_device *dev) } /* TODO: request info from primary to set up Rx and Tx */ eth_dev->dev_ops = &ops; + eth_dev->device = &dev->device; rte_eth_dev_probing_finish(eth_dev); return 0; } @@ -1016,6 +1016,7 @@ static int pmd_pcap_remove(struct rte_vdev_device *dev) { struct rte_eth_dev *eth_dev = NULL; + const char *name; PMD_LOG(INFO, "Closing pcap ethdev on numa socket %d", rte_socket_id()); @@ -1023,11 +1024,15 @@ pmd_pcap_remove(struct rte_vdev_device *dev) if (!dev) return -1; + name = rte_vdev_device_name(dev); /* reserve an ethdev entry */ - eth_dev = rte_eth_dev_allocated(rte_vdev_device_name(dev)); + eth_dev = rte_eth_dev_allocated(name); if (eth_dev == NULL) return -1; + if (rte_eal_process_type() != RTE_PROC_PRIMARY) + return rte_eth_dev_release_port_private(eth_dev); + rte_free(eth_dev->data->dev_private); rte_eth_dev_release_port(eth_dev); From patchwork Mon Jul 9 03:37:02 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Qi Zhang X-Patchwork-Id: 42591 X-Patchwork-Delegate: thomas@monjalon.net Return-Path: X-Original-To: patchwork@dpdk.org Delivered-To: patchwork@dpdk.org Received: from [92.243.14.124] (localhost [127.0.0.1]) by dpdk.org (Postfix) with ESMTP id 31F3C1B448; Mon, 9 Jul 2018 05:37:13 +0200 (CEST) Received: from mga12.intel.com (mga12.intel.com [192.55.52.136]) by dpdk.org (Postfix) with ESMTP id 49DB21B3A7 for ; Mon, 9 Jul 2018 05:36:57 +0200 (CEST) X-Amp-Result: SKIPPED(no attachment in message) X-Amp-File-Uploaded: False Received: from fmsmga005.fm.intel.com ([10.253.24.32]) by fmsmga106.fm.intel.com with ESMTP/TLS/DHE-RSA-AES256-GCM-SHA384; 08 Jul 2018 20:36:57 -0700 X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="5.51,328,1526367600"; d="scan'208";a="243632797" Received: from dpdk51.sh.intel.com ([10.67.110.190]) by fmsmga005.fm.intel.com with ESMTP; 08 Jul 2018 20:36:55 -0700 From: Qi Zhang To: thomas@monjalon.net, anatoly.burakov@intel.com Cc: konstantin.ananyev@intel.com, dev@dpdk.org, bruce.richardson@intel.com, ferruh.yigit@intel.com, benjamin.h.shelton@intel.com, narender.vangati@intel.com, Qi Zhang Date: Mon, 9 Jul 2018 11:37:02 +0800 Message-Id: <20180709033706.27858-16-qi.z.zhang@intel.com> X-Mailer: git-send-email 2.13.6 In-Reply-To: <20180709033706.27858-1-qi.z.zhang@intel.com> References: <20180607123849.14439-1-qi.z.zhang@intel.com> <20180709033706.27858-1-qi.z.zhang@intel.com> Subject: [dpdk-dev] [PATCH v10 15/19] net/softnic: enable hotplug on secondary process X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Sender: "dev" Attach port from secondary should ignore devargs since the private device is not necessary to support. Also previously, detach port on a secondary process will mess primary process and cause the same device can't be attached back again. A secondary process should use rte_eth_release_port_private to release a port. Signed-off-by: Qi Zhang --- drivers/net/softnic/rte_eth_softnic.c | 15 ++++++++++----- 1 file changed, 10 insertions(+), 5 deletions(-) diff --git a/drivers/net/softnic/rte_eth_softnic.c b/drivers/net/softnic/rte_eth_softnic.c index 6b3c13e5c..f3b6863ae 100644 --- a/drivers/net/softnic/rte_eth_softnic.c +++ b/drivers/net/softnic/rte_eth_softnic.c @@ -741,8 +741,7 @@ pmd_probe(struct rte_vdev_device *vdev) /* Parse input arguments */ params = rte_vdev_device_args(vdev); - if (rte_eal_process_type() == RTE_PROC_SECONDARY && - strlen(params) == 0) { + if (rte_eal_process_type() == RTE_PROC_SECONDARY) { eth_dev = rte_eth_dev_attach_secondary(name); if (!eth_dev) { PMD_LOG(ERR, "Failed to probe %s", name); @@ -750,6 +749,7 @@ pmd_probe(struct rte_vdev_device *vdev) } /* TODO: request info from primary to set up Rx and Tx */ eth_dev->dev_ops = &pmd_ops; + eth_dev->device = &vdev->device; rte_eth_dev_probing_finish(eth_dev); return 0; } @@ -803,17 +803,22 @@ pmd_remove(struct rte_vdev_device *vdev) { struct rte_eth_dev *dev = NULL; struct pmd_internals *p; + const char *name; if (!vdev) return -EINVAL; - PMD_LOG(INFO, "Removing device \"%s\"", - rte_vdev_device_name(vdev)); + name = rte_vdev_device_name(vdev); + PMD_LOG(INFO, "Removing device \"%s\"", name); /* Find the ethdev entry */ - dev = rte_eth_dev_allocated(rte_vdev_device_name(vdev)); + dev = rte_eth_dev_allocated(name); if (dev == NULL) return -ENODEV; + + if (rte_eal_process_type() != RTE_PROC_PRIMARY) + return rte_eth_dev_release_port_private(dev); + p = dev->data->dev_private; /* Free device data structures*/ From patchwork Mon Jul 9 03:37:03 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Qi Zhang X-Patchwork-Id: 42592 X-Patchwork-Delegate: thomas@monjalon.net Return-Path: X-Original-To: patchwork@dpdk.org Delivered-To: patchwork@dpdk.org Received: from [92.243.14.124] (localhost [127.0.0.1]) by dpdk.org (Postfix) with ESMTP id 3CA021B44F; Mon, 9 Jul 2018 05:37:15 +0200 (CEST) Received: from mga12.intel.com (mga12.intel.com [192.55.52.136]) by dpdk.org (Postfix) with ESMTP id BBEFE1B3BA for ; Mon, 9 Jul 2018 05:36:58 +0200 (CEST) X-Amp-Result: SKIPPED(no attachment in message) X-Amp-File-Uploaded: False Received: from fmsmga005.fm.intel.com ([10.253.24.32]) by fmsmga106.fm.intel.com with ESMTP/TLS/DHE-RSA-AES256-GCM-SHA384; 08 Jul 2018 20:36:58 -0700 X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="5.51,328,1526367600"; d="scan'208";a="243632805" Received: from dpdk51.sh.intel.com ([10.67.110.190]) by fmsmga005.fm.intel.com with ESMTP; 08 Jul 2018 20:36:57 -0700 From: Qi Zhang To: thomas@monjalon.net, anatoly.burakov@intel.com Cc: konstantin.ananyev@intel.com, dev@dpdk.org, bruce.richardson@intel.com, ferruh.yigit@intel.com, benjamin.h.shelton@intel.com, narender.vangati@intel.com, Qi Zhang Date: Mon, 9 Jul 2018 11:37:03 +0800 Message-Id: <20180709033706.27858-17-qi.z.zhang@intel.com> X-Mailer: git-send-email 2.13.6 In-Reply-To: <20180709033706.27858-1-qi.z.zhang@intel.com> References: <20180607123849.14439-1-qi.z.zhang@intel.com> <20180709033706.27858-1-qi.z.zhang@intel.com> Subject: [dpdk-dev] [PATCH v10 16/19] net/tap: enable hotplug on secondary process X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Sender: "dev" Attach port from secondary should ignore devargs since the private device is not necessary to support. Also previously, detach port on a secondary process will mess primary process and cause the same device can't be attached back again. A secondary process should use rte_eth_release_port_private to release a port. Signed-off-by: Qi Zhang Acked-by: Keith Wiles --- drivers/net/tap/rte_eth_tap.c | 13 +++++++++---- 1 file changed, 9 insertions(+), 4 deletions(-) diff --git a/drivers/net/tap/rte_eth_tap.c b/drivers/net/tap/rte_eth_tap.c index df396bfde..a58d5ebe6 100644 --- a/drivers/net/tap/rte_eth_tap.c +++ b/drivers/net/tap/rte_eth_tap.c @@ -1750,8 +1750,7 @@ rte_pmd_tap_probe(struct rte_vdev_device *dev) name = rte_vdev_device_name(dev); params = rte_vdev_device_args(dev); - if (rte_eal_process_type() == RTE_PROC_SECONDARY && - strlen(params) == 0) { + if (rte_eal_process_type() == RTE_PROC_SECONDARY) { eth_dev = rte_eth_dev_attach_secondary(name); if (!eth_dev) { TAP_LOG(ERR, "Failed to probe %s", name); @@ -1759,6 +1758,7 @@ rte_pmd_tap_probe(struct rte_vdev_device *dev) } /* TODO: request info from primary to set up Rx and Tx */ eth_dev->dev_ops = &ops; + eth_dev->device = &dev->device; rte_eth_dev_probing_finish(eth_dev); return 0; } @@ -1827,12 +1827,17 @@ rte_pmd_tap_remove(struct rte_vdev_device *dev) { struct rte_eth_dev *eth_dev = NULL; struct pmd_internals *internals; + const char *name; int i; + name = rte_vdev_device_name(dev); /* find the ethdev entry */ - eth_dev = rte_eth_dev_allocated(rte_vdev_device_name(dev)); + eth_dev = rte_eth_dev_allocated(name); if (!eth_dev) - return 0; + return -ENODEV; + + if (rte_eal_process_type() != RTE_PROC_PRIMARY) + return rte_eth_dev_release_port_private(eth_dev); internals = eth_dev->data->dev_private; From patchwork Mon Jul 9 03:37:04 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Qi Zhang X-Patchwork-Id: 42593 X-Patchwork-Delegate: thomas@monjalon.net Return-Path: X-Original-To: patchwork@dpdk.org Delivered-To: patchwork@dpdk.org Received: from [92.243.14.124] (localhost [127.0.0.1]) by dpdk.org (Postfix) with ESMTP id 4F2111B45D; Mon, 9 Jul 2018 05:37:17 +0200 (CEST) Received: from mga12.intel.com (mga12.intel.com [192.55.52.136]) by dpdk.org (Postfix) with ESMTP id B9C4B1B42E for ; Mon, 9 Jul 2018 05:37:00 +0200 (CEST) X-Amp-Result: SKIPPED(no attachment in message) X-Amp-File-Uploaded: False Received: from fmsmga005.fm.intel.com ([10.253.24.32]) by fmsmga106.fm.intel.com with ESMTP/TLS/DHE-RSA-AES256-GCM-SHA384; 08 Jul 2018 20:37:00 -0700 X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="5.51,328,1526367600"; d="scan'208";a="243632813" Received: from dpdk51.sh.intel.com ([10.67.110.190]) by fmsmga005.fm.intel.com with ESMTP; 08 Jul 2018 20:36:58 -0700 From: Qi Zhang To: thomas@monjalon.net, anatoly.burakov@intel.com Cc: konstantin.ananyev@intel.com, dev@dpdk.org, bruce.richardson@intel.com, ferruh.yigit@intel.com, benjamin.h.shelton@intel.com, narender.vangati@intel.com, Qi Zhang Date: Mon, 9 Jul 2018 11:37:04 +0800 Message-Id: <20180709033706.27858-18-qi.z.zhang@intel.com> X-Mailer: git-send-email 2.13.6 In-Reply-To: <20180709033706.27858-1-qi.z.zhang@intel.com> References: <20180607123849.14439-1-qi.z.zhang@intel.com> <20180709033706.27858-1-qi.z.zhang@intel.com> Subject: [dpdk-dev] [PATCH v10 17/19] net/vhost: enable hotplug on secondary process X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Sender: "dev" Attach port from secondary should ignore devargs since the private device is not necessary to support. Also previously, detach port on a secondary process will mess primary process and cause the same device can't be attached back again. A secondary process should use rte_eth_release_port_private to release a port. Signed-off-by: Qi Zhang --- drivers/net/vhost/rte_eth_vhost.c | 7 +++++-- 1 file changed, 5 insertions(+), 2 deletions(-) diff --git a/drivers/net/vhost/rte_eth_vhost.c b/drivers/net/vhost/rte_eth_vhost.c index ba9d768a0..3a3f8f6cc 100644 --- a/drivers/net/vhost/rte_eth_vhost.c +++ b/drivers/net/vhost/rte_eth_vhost.c @@ -1344,8 +1344,7 @@ rte_pmd_vhost_probe(struct rte_vdev_device *dev) VHOST_LOG(INFO, "Initializing pmd_vhost for %s\n", name); - if (rte_eal_process_type() == RTE_PROC_SECONDARY && - strlen(rte_vdev_device_args(dev)) == 0) { + if (rte_eal_process_type() == RTE_PROC_SECONDARY) { eth_dev = rte_eth_dev_attach_secondary(name); if (!eth_dev) { VHOST_LOG(ERR, "Failed to probe %s\n", name); @@ -1353,6 +1352,7 @@ rte_pmd_vhost_probe(struct rte_vdev_device *dev) } /* TODO: request info from primary to set up Rx and Tx */ eth_dev->dev_ops = &ops; + eth_dev->device = &dev->device; rte_eth_dev_probing_finish(eth_dev); return 0; } @@ -1435,6 +1435,9 @@ rte_pmd_vhost_remove(struct rte_vdev_device *dev) if (eth_dev == NULL) return -ENODEV; + if (rte_eal_process_type() != RTE_PROC_PRIMARY) + return rte_eth_dev_release_port_private(eth_dev); + eth_dev_close(eth_dev); rte_free(vring_states[eth_dev->data->port_id]); From patchwork Mon Jul 9 03:37:05 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Qi Zhang X-Patchwork-Id: 42594 X-Patchwork-Delegate: thomas@monjalon.net Return-Path: X-Original-To: patchwork@dpdk.org Delivered-To: patchwork@dpdk.org Received: from [92.243.14.124] (localhost [127.0.0.1]) by dpdk.org (Postfix) with ESMTP id F152D1B461; Mon, 9 Jul 2018 05:37:20 +0200 (CEST) Received: from mga12.intel.com (mga12.intel.com [192.55.52.136]) by dpdk.org (Postfix) with ESMTP id 823231B1EC for ; Mon, 9 Jul 2018 05:37:02 +0200 (CEST) X-Amp-Result: SKIPPED(no attachment in message) X-Amp-File-Uploaded: False Received: from fmsmga005.fm.intel.com ([10.253.24.32]) by fmsmga106.fm.intel.com with ESMTP/TLS/DHE-RSA-AES256-GCM-SHA384; 08 Jul 2018 20:37:02 -0700 X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="5.51,328,1526367600"; d="scan'208";a="243632824" Received: from dpdk51.sh.intel.com ([10.67.110.190]) by fmsmga005.fm.intel.com with ESMTP; 08 Jul 2018 20:37:00 -0700 From: Qi Zhang To: thomas@monjalon.net, anatoly.burakov@intel.com Cc: konstantin.ananyev@intel.com, dev@dpdk.org, bruce.richardson@intel.com, ferruh.yigit@intel.com, benjamin.h.shelton@intel.com, narender.vangati@intel.com, Qi Zhang Date: Mon, 9 Jul 2018 11:37:05 +0800 Message-Id: <20180709033706.27858-19-qi.z.zhang@intel.com> X-Mailer: git-send-email 2.13.6 In-Reply-To: <20180709033706.27858-1-qi.z.zhang@intel.com> References: <20180607123849.14439-1-qi.z.zhang@intel.com> <20180709033706.27858-1-qi.z.zhang@intel.com> Subject: [dpdk-dev] [PATCH v10 18/19] examples/multi_process: add hotplug sample X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Sender: "dev" The sample code demonstrates device (ethdev only) management at a multi-process environment. The user can attach/detach a device on primary process and see it is synced on secondary process automatically. How to start? ./hotplug_mp --proc-type=auto Command Line Example: >help >list /* attach a af_packet vdev */ >attach net_af_packet,iface=eth0 /* detach port 0 */ >detach 0 Signed-off-by: Qi Zhang --- examples/multi_process/Makefile | 1 + examples/multi_process/hotplug_mp/Makefile | 23 ++++ examples/multi_process/hotplug_mp/commands.c | 197 +++++++++++++++++++++++++++ examples/multi_process/hotplug_mp/commands.h | 10 ++ examples/multi_process/hotplug_mp/main.c | 41 ++++++ 5 files changed, 272 insertions(+) create mode 100644 examples/multi_process/hotplug_mp/Makefile create mode 100644 examples/multi_process/hotplug_mp/commands.c create mode 100644 examples/multi_process/hotplug_mp/commands.h create mode 100644 examples/multi_process/hotplug_mp/main.c diff --git a/examples/multi_process/Makefile b/examples/multi_process/Makefile index a6708b7e4..b76b02fcb 100644 --- a/examples/multi_process/Makefile +++ b/examples/multi_process/Makefile @@ -13,5 +13,6 @@ include $(RTE_SDK)/mk/rte.vars.mk DIRS-$(CONFIG_RTE_EXEC_ENV_LINUXAPP) += client_server_mp DIRS-$(CONFIG_RTE_EXEC_ENV_LINUXAPP) += simple_mp DIRS-$(CONFIG_RTE_EXEC_ENV_LINUXAPP) += symmetric_mp +DIRS-$(CONFIG_RTE_EXEC_ENV_LINUXAPP) += hotplug_mp include $(RTE_SDK)/mk/rte.extsubdir.mk diff --git a/examples/multi_process/hotplug_mp/Makefile b/examples/multi_process/hotplug_mp/Makefile new file mode 100644 index 000000000..bc36aeaed --- /dev/null +++ b/examples/multi_process/hotplug_mp/Makefile @@ -0,0 +1,23 @@ +# SPDX-License-Identifier: BSD-3-Clause +# Copyright(c) 2018 Intel Corporation + +ifeq ($(RTE_SDK),) +$(error "Please define RTE_SDK environment variable") +endif + +# Default target, can be overridden by command line or environment +RTE_TARGET ?= x86_64-native-linuxapp-gcc + +include $(RTE_SDK)/mk/rte.vars.mk + +# binary name +APP = hotplug_mp + +# all source are stored in SRCS-y +SRCS-y := main.c commands.c + +CFLAGS += -O3 +CFLAGS += $(WERROR_FLAGS) +CFLAGS += -DALLOW_EXPERIMENTAL_API + +include $(RTE_SDK)/mk/rte.extapp.mk diff --git a/examples/multi_process/hotplug_mp/commands.c b/examples/multi_process/hotplug_mp/commands.c new file mode 100644 index 000000000..fb7198d51 --- /dev/null +++ b/examples/multi_process/hotplug_mp/commands.c @@ -0,0 +1,197 @@ +/* SPDX-License-Identifier: BSD-3-Clause + * Copyright(c) 2018 Intel Corporation. + */ + +#include +#include +#include +#include +#include +#include +#include + +/**********************************************************/ + +struct cmd_help_result { + cmdline_fixed_string_t help; +}; + +static void cmd_help_parsed(__attribute__((unused)) void *parsed_result, + struct cmdline *cl, + __attribute__((unused)) void *data) +{ + cmdline_printf(cl, + "commands:\n" + "- attach \n" + "- detach \n" + "- list\n\n"); +} + +cmdline_parse_token_string_t cmd_help_help = + TOKEN_STRING_INITIALIZER(struct cmd_help_result, help, "help"); + +cmdline_parse_inst_t cmd_help = { + .f = cmd_help_parsed, /* function to call */ + .data = NULL, /* 2nd arg of func */ + .help_str = "show help", + .tokens = { /* token list, NULL terminated */ + (void *)&cmd_help_help, + NULL, + }, +}; + +/**********************************************************/ + +struct cmd_quit_result { + cmdline_fixed_string_t quit; +}; + +static void cmd_quit_parsed(__attribute__((unused)) void *parsed_result, + struct cmdline *cl, + __attribute__((unused)) void *data) +{ + cmdline_quit(cl); +} + +cmdline_parse_token_string_t cmd_quit_quit = + TOKEN_STRING_INITIALIZER(struct cmd_quit_result, quit, "quit"); + +cmdline_parse_inst_t cmd_quit = { + .f = cmd_quit_parsed, /* function to call */ + .data = NULL, /* 2nd arg of func */ + .help_str = "quit", + .tokens = { /* token list, NULL terminated */ + (void *)&cmd_quit_quit, + NULL, + }, +}; + +/**********************************************************/ + +struct cmd_list_result { + cmdline_fixed_string_t list; +}; + +static void cmd_list_parsed(__attribute__((unused)) void *parsed_result, + struct cmdline *cl, + __attribute__((unused)) void *data) +{ + uint16_t port_id; + char dev_name[RTE_DEV_NAME_MAX_LEN]; + + cmdline_printf(cl, "list all etherdev\n"); + + RTE_ETH_FOREACH_DEV(port_id) { + rte_eth_dev_get_name_by_port(port_id, dev_name); + if (strlen(dev_name) > 0) + cmdline_printf(cl, "%d\t%s\n", port_id, dev_name); + else + printf("empty dev_name is not expected!\n"); + } +} + +cmdline_parse_token_string_t cmd_list_list = + TOKEN_STRING_INITIALIZER(struct cmd_list_result, list, "list"); + +cmdline_parse_inst_t cmd_list = { + .f = cmd_list_parsed, /* function to call */ + .data = NULL, /* 2nd arg of func */ + .help_str = "list all devices", + .tokens = { /* token list, NULL terminated */ + (void *)&cmd_list_list, + NULL, + }, +}; + +/**********************************************************/ + +struct cmd_dev_attach_result { + cmdline_fixed_string_t attach; + cmdline_fixed_string_t device; +}; + +static void cmd_dev_attach_parsed(void *parsed_result, + struct cmdline *cl, + __attribute__((unused)) void *data) +{ + struct cmd_dev_attach_result *res = parsed_result; + uint16_t port_id; + + if (!rte_eth_dev_attach(res->device, &port_id)) + cmdline_printf(cl, "attached device %s at port %d\n", + res->device, port_id); + else + cmdline_printf(cl, "failed to attached device %s\n", + res->device); +} + +cmdline_parse_token_string_t cmd_dev_attach_attach = + TOKEN_STRING_INITIALIZER(struct cmd_dev_attach_result, attach, + "attach"); +cmdline_parse_token_string_t cmd_dev_attach_device = + TOKEN_STRING_INITIALIZER(struct cmd_dev_attach_result, device, NULL); + +cmdline_parse_inst_t cmd_attach_device = { + .f = cmd_dev_attach_parsed, /* function to call */ + .data = NULL, /* 2nd arg of func */ + .help_str = "attach a device", + .tokens = { /* token list, NULL terminated */ + (void *)&cmd_dev_attach_attach, + (void *)&cmd_dev_attach_device, + NULL, + }, +}; + +/**********************************************************/ + +struct cmd_dev_detach_result { + cmdline_fixed_string_t detach; + uint16_t port_id; +}; + +static void cmd_dev_detach_parsed(void *parsed_result, + struct cmdline *cl, + __attribute__((unused)) void *data) +{ + struct cmd_dev_detach_result *res = parsed_result; + uint16_t port_id = res->port_id; + char dev_name[RTE_DEV_NAME_MAX_LEN]; + + printf("detaching...\n"); + if (!rte_eth_dev_detach(port_id, dev_name)) + cmdline_printf(cl, "detached device at port %d\n", + port_id); + else + cmdline_printf(cl, "failed to dettached at port %d\n", + port_id); +} + +cmdline_parse_token_string_t cmd_dev_detach_detach = + TOKEN_STRING_INITIALIZER(struct cmd_dev_detach_result, detach, + "detach"); +cmdline_parse_token_num_t cmd_dev_detach_port_id = + TOKEN_NUM_INITIALIZER(struct cmd_dev_detach_result, port_id, UINT16); + +cmdline_parse_inst_t cmd_detach_device = { + .f = cmd_dev_detach_parsed, /* function to call */ + .data = NULL, /* 2nd arg of func */ + .help_str = "detach a device", + .tokens = { /* token list, NULL terminated */ + (void *)&cmd_dev_detach_detach, + (void *)&cmd_dev_detach_port_id, + NULL, + }, +}; + +/**********************************************************/ +/**********************************************************/ +/****** CONTEXT (list of instruction) */ + +cmdline_parse_ctx_t main_ctx[] = { + (cmdline_parse_inst_t *)&cmd_help, + (cmdline_parse_inst_t *)&cmd_quit, + (cmdline_parse_inst_t *)&cmd_list, + (cmdline_parse_inst_t *)&cmd_attach_device, + (cmdline_parse_inst_t *)&cmd_detach_device, + NULL, +}; diff --git a/examples/multi_process/hotplug_mp/commands.h b/examples/multi_process/hotplug_mp/commands.h new file mode 100644 index 000000000..afcf177db --- /dev/null +++ b/examples/multi_process/hotplug_mp/commands.h @@ -0,0 +1,10 @@ +/* SPDX-License-Identifier: BSD-3-Clause + * Copyright(c) 2018 Intel Corporation + */ + +#ifndef _COMMANDS_H_ +#define _COMMANDS_H_ + +extern cmdline_parse_ctx_t main_ctx[]; + +#endif /* _COMMANDS_H_ */ diff --git a/examples/multi_process/hotplug_mp/main.c b/examples/multi_process/hotplug_mp/main.c new file mode 100644 index 000000000..d66858078 --- /dev/null +++ b/examples/multi_process/hotplug_mp/main.c @@ -0,0 +1,41 @@ +/* SPDX-License-Identifier: BSD-3-Clause + * Copyright(c) 2018 Intel Corporation + */ + +#include +#include +#include +#include +#include +#include + +#include +#include +#include +#include + +#include +#include +#include + +#include "commands.h" + +int main(int argc, char **argv) +{ + int ret; + struct cmdline *cl; + + ret = rte_eal_init(argc, argv); + if (ret < 0) + rte_panic("Cannot init EAL\n"); + + cl = cmdline_stdin_new(main_ctx, "example> "); + if (cl == NULL) + rte_panic("Cannot create cmdline instance\n"); + cmdline_interact(cl); + cmdline_stdin_exit(cl); + + rte_eal_cleanup(); + + return 0; +} From patchwork Mon Jul 9 03:37:06 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Qi Zhang X-Patchwork-Id: 42595 X-Patchwork-Delegate: thomas@monjalon.net Return-Path: X-Original-To: patchwork@dpdk.org Delivered-To: patchwork@dpdk.org Received: from [92.243.14.124] (localhost [127.0.0.1]) by dpdk.org (Postfix) with ESMTP id B9E8E1B466; Mon, 9 Jul 2018 05:37:24 +0200 (CEST) Received: from mga12.intel.com (mga12.intel.com [192.55.52.136]) by dpdk.org (Postfix) with ESMTP id 35FC01B439 for ; Mon, 9 Jul 2018 05:37:05 +0200 (CEST) X-Amp-Result: SKIPPED(no attachment in message) X-Amp-File-Uploaded: False Received: from fmsmga005.fm.intel.com ([10.253.24.32]) by fmsmga106.fm.intel.com with ESMTP/TLS/DHE-RSA-AES256-GCM-SHA384; 08 Jul 2018 20:37:04 -0700 X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="5.51,328,1526367600"; d="scan'208";a="243632841" Received: from dpdk51.sh.intel.com ([10.67.110.190]) by fmsmga005.fm.intel.com with ESMTP; 08 Jul 2018 20:37:02 -0700 From: Qi Zhang To: thomas@monjalon.net, anatoly.burakov@intel.com Cc: konstantin.ananyev@intel.com, dev@dpdk.org, bruce.richardson@intel.com, ferruh.yigit@intel.com, benjamin.h.shelton@intel.com, narender.vangati@intel.com, Qi Zhang Date: Mon, 9 Jul 2018 11:37:06 +0800 Message-Id: <20180709033706.27858-20-qi.z.zhang@intel.com> X-Mailer: git-send-email 2.13.6 In-Reply-To: <20180709033706.27858-1-qi.z.zhang@intel.com> References: <20180607123849.14439-1-qi.z.zhang@intel.com> <20180709033706.27858-1-qi.z.zhang@intel.com> Subject: [dpdk-dev] [PATCH v10 19/19] doc: update release notes for multi process hotplug X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Sender: "dev" Update release notes for the new multi-process hotplug feature. Signed-off-by: Qi Zhang --- doc/guides/rel_notes/release_18_08.rst | 11 +++++++++++ 1 file changed, 11 insertions(+) diff --git a/doc/guides/rel_notes/release_18_08.rst b/doc/guides/rel_notes/release_18_08.rst index bc0124295..1251e4b5b 100644 --- a/doc/guides/rel_notes/release_18_08.rst +++ b/doc/guides/rel_notes/release_18_08.rst @@ -46,6 +46,12 @@ New Features Flow API support has been added to CXGBE Poll Mode Driver to offload flows to Chelsio T5/T6 NICs. +* **Support device multi-process hotplug.** + + Hotplug and hot-unplug for devices will now be supported in multiprocessing + scenario. Any ethdev devices created in the primary process will be regarded + as shared and will be available for all DPDK processes. Synchronization between + processes will be done using DPDK IPC. API Changes ----------- @@ -60,6 +66,11 @@ API Changes Also, make sure to start the actual text at the margin. ========================================================= +* eal: scope of rte_eal_hotplug_add and rte_eal_hotplug_remove is extended. + + In primary-secondary process model, ``rte_eal_hotplug_add`` will guarantee + that device be attached on all processes, while ``rte_eal_hotplug_remove`` + will guarantee device be detached on all processes. ABI Changes -----------