From patchwork Tue Jun 26 07:08:09 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Qi Zhang X-Patchwork-Id: 41508 X-Patchwork-Delegate: thomas@monjalon.net Return-Path: X-Original-To: patchwork@dpdk.org Delivered-To: patchwork@dpdk.org Received: from [92.243.14.124] (localhost [127.0.0.1]) by dpdk.org (Postfix) with ESMTP id 5BC664CBD; Tue, 26 Jun 2018 09:08:08 +0200 (CEST) Received: from mga09.intel.com (mga09.intel.com [134.134.136.24]) by dpdk.org (Postfix) with ESMTP id 06BB64CA6 for ; Tue, 26 Jun 2018 09:08:02 +0200 (CEST) X-Amp-Result: SKIPPED(no attachment in message) X-Amp-File-Uploaded: False Received: from fmsmga006.fm.intel.com ([10.253.24.20]) by orsmga102.jf.intel.com with ESMTP/TLS/DHE-RSA-AES256-GCM-SHA384; 26 Jun 2018 00:08:02 -0700 X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="5.51,273,1526367600"; d="scan'208";a="240659935" Received: from dpdk51.sh.intel.com ([10.67.110.190]) by fmsmga006.fm.intel.com with ESMTP; 26 Jun 2018 00:08:00 -0700 From: Qi Zhang To: thomas@monjalon.net, anatoly.burakov@intel.com Cc: konstantin.ananyev@intel.com, dev@dpdk.org, bruce.richardson@intel.com, ferruh.yigit@intel.com, benjamin.h.shelton@intel.com, narender.vangati@intel.com, Qi Zhang Date: Tue, 26 Jun 2018 15:08:09 +0800 Message-Id: <20180626070832.3055-2-qi.z.zhang@intel.com> X-Mailer: git-send-email 2.13.6 In-Reply-To: <20180626070832.3055-1-qi.z.zhang@intel.com> References: <20180607123849.14439-1-qi.z.zhang@intel.com> <20180626070832.3055-1-qi.z.zhang@intel.com> Subject: [dpdk-dev] [PATCH v4 01/24] eal: introduce one device scan X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Sender: "dev" When hot plug a new device, it is not necessary to scan everything on the bus since the devname and devargs are already there. So new rte_bus ops "scan_one" is introduced, bus driver can implement this function to simplify the hotplug process. Signed-off-by: Qi Zhang Acked-by: Remy Horton --- lib/librte_eal/common/eal_common_dev.c | 17 +++++++++++++---- lib/librte_eal/common/include/rte_bus.h | 16 ++++++++++++++++ 2 files changed, 29 insertions(+), 4 deletions(-) diff --git a/lib/librte_eal/common/eal_common_dev.c b/lib/librte_eal/common/eal_common_dev.c index 61cb3b162..1ad033536 100644 --- a/lib/librte_eal/common/eal_common_dev.c +++ b/lib/librte_eal/common/eal_common_dev.c @@ -147,11 +147,20 @@ int __rte_experimental rte_eal_hotplug_add(const char *busname, const char *devn if (ret) goto err_devarg; - ret = bus->scan(); - if (ret) - goto err_devarg; + /** + * if bus support to scan specific device by devargs, + * we don't need to scan all devices on the bus. + */ + if (bus->scan_one) { + dev = bus->scan_one(da); + } else { + ret = bus->scan(); + if (ret) + goto err_devarg; + + dev = bus->find_device(NULL, cmp_detached_dev_name, devname); + } - dev = bus->find_device(NULL, cmp_detached_dev_name, devname); if (dev == NULL) { RTE_LOG(ERR, EAL, "Cannot find unplugged device (%s)\n", devname); diff --git a/lib/librte_eal/common/include/rte_bus.h b/lib/librte_eal/common/include/rte_bus.h index eb9eded4e..3269ef78b 100644 --- a/lib/librte_eal/common/include/rte_bus.h +++ b/lib/librte_eal/common/include/rte_bus.h @@ -84,6 +84,21 @@ enum rte_iova_mode { typedef int (*rte_bus_scan_t)(void); /** + * Bus specific scan for one specific device attached on the bus. + * For each bus object, the scan would be responsible for finding the specific + * device and adding it to its private device list, and the device object will + * be return also. + * + * @param devargs + * Device arguments be used to identify the device. + * + * @return + * !NULL for successful scan + * NULL for unsuccessful scan + */ +typedef struct rte_device *(*rte_bus_scan_one_t)(struct rte_devargs *devargs); + +/** * Implementation specific probe function which is responsible for linking * devices on that bus with applicable drivers. * @@ -204,6 +219,7 @@ struct rte_bus { TAILQ_ENTRY(rte_bus) next; /**< Next bus object in linked list */ const char *name; /**< Name of the bus */ rte_bus_scan_t scan; /**< Scan for devices attached to bus */ + rte_bus_scan_one_t scan_one; /**< Scan one device using devargs */ rte_bus_probe_t probe; /**< Probe devices on bus */ rte_bus_find_device_t find_device; /**< Find a device on the bus */ rte_bus_plug_t plug; /**< Probe single device for drivers */ From patchwork Tue Jun 26 07:08:10 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Qi Zhang X-Patchwork-Id: 41509 X-Patchwork-Delegate: thomas@monjalon.net Return-Path: X-Original-To: patchwork@dpdk.org Delivered-To: patchwork@dpdk.org Received: from [92.243.14.124] (localhost [127.0.0.1]) by dpdk.org (Postfix) with ESMTP id 7DA404F9C; Tue, 26 Jun 2018 09:08:10 +0200 (CEST) Received: from mga09.intel.com (mga09.intel.com [134.134.136.24]) by dpdk.org (Postfix) with ESMTP id AF5474CC3 for ; Tue, 26 Jun 2018 09:08:04 +0200 (CEST) X-Amp-Result: SKIPPED(no attachment in message) X-Amp-File-Uploaded: False Received: from fmsmga006.fm.intel.com ([10.253.24.20]) by orsmga102.jf.intel.com with ESMTP/TLS/DHE-RSA-AES256-GCM-SHA384; 26 Jun 2018 00:08:04 -0700 X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="5.51,273,1526367600"; d="scan'208";a="240659946" Received: from dpdk51.sh.intel.com ([10.67.110.190]) by fmsmga006.fm.intel.com with ESMTP; 26 Jun 2018 00:08:02 -0700 From: Qi Zhang To: thomas@monjalon.net, anatoly.burakov@intel.com Cc: konstantin.ananyev@intel.com, dev@dpdk.org, bruce.richardson@intel.com, ferruh.yigit@intel.com, benjamin.h.shelton@intel.com, narender.vangati@intel.com, Qi Zhang Date: Tue, 26 Jun 2018 15:08:10 +0800 Message-Id: <20180626070832.3055-3-qi.z.zhang@intel.com> X-Mailer: git-send-email 2.13.6 In-Reply-To: <20180626070832.3055-1-qi.z.zhang@intel.com> References: <20180607123849.14439-1-qi.z.zhang@intel.com> <20180626070832.3055-1-qi.z.zhang@intel.com> Subject: [dpdk-dev] [PATCH v4 02/24] bus/vdev: enable one device scan X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Sender: "dev" The patch implemented the ops scan_one for vdev bus, it gives two benifits 1. Improve scan efficiency when a device is attached as hotplug, since no need to pupulate a new device by iterating all devargs in devargs_list. 2. It also avoid sync IPC invoke (which happens in vdev->scan on secondary process). The benifit is this removes the potential deadlock in the case when secondary process receive a request from primary process to attach a new device, since vdev->scan will be invoked on mp thread itself in that case. Signed-off-by: Qi Zhang Acked-by: Remy Horton --- drivers/bus/vdev/vdev.c | 30 ++++++++++++++++++++++++++++++ 1 file changed, 30 insertions(+) diff --git a/drivers/bus/vdev/vdev.c b/drivers/bus/vdev/vdev.c index 6139dd551..cdbd77df0 100644 --- a/drivers/bus/vdev/vdev.c +++ b/drivers/bus/vdev/vdev.c @@ -467,6 +467,35 @@ vdev_scan(void) return 0; } +static struct rte_device *vdev_scan_one(struct rte_devargs *devargs) +{ + struct rte_vdev_device *dev = NULL; + + dev = calloc(1, sizeof(*dev)); + if (!dev) { + VDEV_LOG(ERR, "failed to allocate memory for new device"); + return NULL; + } + + rte_spinlock_recursive_lock(&vdev_device_list_lock); + + if (find_vdev(devargs->name)) { + VDEV_LOG(ERR, "device %s already exist", devargs->name); + free(dev); + rte_spinlock_recursive_unlock(&vdev_device_list_lock); + return NULL; + } + + dev->device.devargs = devargs; + dev->device.numa_node = SOCKET_ID_ANY; + dev->device.name = devargs->name; + TAILQ_INSERT_TAIL(&vdev_device_list, dev, next); + + rte_spinlock_recursive_unlock(&vdev_device_list_lock); + + return &dev->device; +} + static int vdev_probe(void) { @@ -531,6 +560,7 @@ vdev_unplug(struct rte_device *dev) static struct rte_bus rte_vdev_bus = { .scan = vdev_scan, + .scan_one = vdev_scan_one, .probe = vdev_probe, .find_device = vdev_find_device, .plug = vdev_plug, From patchwork Tue Jun 26 07:08:11 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Qi Zhang X-Patchwork-Id: 41510 X-Patchwork-Delegate: thomas@monjalon.net Return-Path: X-Original-To: patchwork@dpdk.org Delivered-To: patchwork@dpdk.org Received: from [92.243.14.124] (localhost [127.0.0.1]) by dpdk.org (Postfix) with ESMTP id 92FC45592; Tue, 26 Jun 2018 09:08:12 +0200 (CEST) Received: from mga09.intel.com (mga09.intel.com [134.134.136.24]) by dpdk.org (Postfix) with ESMTP id B5BB94CC3 for ; Tue, 26 Jun 2018 09:08:05 +0200 (CEST) X-Amp-Result: SKIPPED(no attachment in message) X-Amp-File-Uploaded: False Received: from fmsmga006.fm.intel.com ([10.253.24.20]) by orsmga102.jf.intel.com with ESMTP/TLS/DHE-RSA-AES256-GCM-SHA384; 26 Jun 2018 00:08:05 -0700 X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="5.51,273,1526367600"; d="scan'208";a="240659955" Received: from dpdk51.sh.intel.com ([10.67.110.190]) by fmsmga006.fm.intel.com with ESMTP; 26 Jun 2018 00:08:03 -0700 From: Qi Zhang To: thomas@monjalon.net, anatoly.burakov@intel.com Cc: konstantin.ananyev@intel.com, dev@dpdk.org, bruce.richardson@intel.com, ferruh.yigit@intel.com, benjamin.h.shelton@intel.com, narender.vangati@intel.com, Qi Zhang Date: Tue, 26 Jun 2018 15:08:11 +0800 Message-Id: <20180626070832.3055-4-qi.z.zhang@intel.com> X-Mailer: git-send-email 2.13.6 In-Reply-To: <20180626070832.3055-1-qi.z.zhang@intel.com> References: <20180607123849.14439-1-qi.z.zhang@intel.com> <20180626070832.3055-1-qi.z.zhang@intel.com> Subject: [dpdk-dev] [PATCH v4 03/24] ethdev: add function to release port in local process X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Sender: "dev" Add driver API rte_eth_release_port_private to support the requirement that an ethdev only be released on secondary process, so only local state be set to unused , share data will not be reset so primary process can still use it. Signed-off-by: Qi Zhang Acked-by: Remy Horton --- lib/librte_ethdev/rte_ethdev.c | 24 +++++++++++++++++++++--- lib/librte_ethdev/rte_ethdev_driver.h | 13 +++++++++++++ 2 files changed, 34 insertions(+), 3 deletions(-) diff --git a/lib/librte_ethdev/rte_ethdev.c b/lib/librte_ethdev/rte_ethdev.c index a9977df97..205b2ee33 100644 --- a/lib/librte_ethdev/rte_ethdev.c +++ b/lib/librte_ethdev/rte_ethdev.c @@ -359,6 +359,23 @@ rte_eth_dev_attach_secondary(const char *name) } int +rte_eth_dev_release_port_private(struct rte_eth_dev *eth_dev) +{ + if (eth_dev == NULL) + return -EINVAL; + + _rte_eth_dev_callback_process(eth_dev, RTE_ETH_EVENT_DESTROY, NULL); + + rte_spinlock_lock(&rte_eth_dev_shared_data->ownership_lock); + + eth_dev->state = RTE_ETH_DEV_UNUSED; + + rte_spinlock_unlock(&rte_eth_dev_shared_data->ownership_lock); + + return 0; +} + +int rte_eth_dev_release_port(struct rte_eth_dev *eth_dev) { if (eth_dev == NULL) @@ -370,9 +387,10 @@ rte_eth_dev_release_port(struct rte_eth_dev *eth_dev) rte_spinlock_lock(&rte_eth_dev_shared_data->ownership_lock); - eth_dev->state = RTE_ETH_DEV_UNUSED; - - memset(eth_dev->data, 0, sizeof(struct rte_eth_dev_data)); + if (eth_dev->state != RTE_ETH_DEV_UNUSED) { + eth_dev->state = RTE_ETH_DEV_UNUSED; + memset(eth_dev->data, 0, sizeof(struct rte_eth_dev_data)); + } rte_spinlock_unlock(&rte_eth_dev_shared_data->ownership_lock); diff --git a/lib/librte_ethdev/rte_ethdev_driver.h b/lib/librte_ethdev/rte_ethdev_driver.h index c9c825e3f..49c27223d 100644 --- a/lib/librte_ethdev/rte_ethdev_driver.h +++ b/lib/librte_ethdev/rte_ethdev_driver.h @@ -70,6 +70,19 @@ int rte_eth_dev_release_port(struct rte_eth_dev *eth_dev); /** * @internal + * Release the specified ethdev port in local process, only set to ethdev + * state to unused, but not reset share data since it assume other process + * is still using it, typically it is called by secondary process. + * + * @param eth_dev + * The *eth_dev* pointer is the address of the *rte_eth_dev* structure. + * @return + * - 0 on success, negative on error + */ +int rte_eth_dev_release_port_private(struct rte_eth_dev *eth_dev); + +/** + * @internal * Release device queues and clear its configuration to force the user * application to reconfigure it. It is for internal use only. * From patchwork Tue Jun 26 07:08:12 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Qi Zhang X-Patchwork-Id: 41511 X-Patchwork-Delegate: thomas@monjalon.net Return-Path: X-Original-To: patchwork@dpdk.org Delivered-To: patchwork@dpdk.org Received: from [92.243.14.124] (localhost [127.0.0.1]) by dpdk.org (Postfix) with ESMTP id B3885569B; Tue, 26 Jun 2018 09:08:14 +0200 (CEST) Received: from mga09.intel.com (mga09.intel.com [134.134.136.24]) by dpdk.org (Postfix) with ESMTP id 42A6F4CBD for ; Tue, 26 Jun 2018 09:08:07 +0200 (CEST) X-Amp-Result: SKIPPED(no attachment in message) X-Amp-File-Uploaded: False Received: from fmsmga006.fm.intel.com ([10.253.24.20]) by orsmga102.jf.intel.com with ESMTP/TLS/DHE-RSA-AES256-GCM-SHA384; 26 Jun 2018 00:08:07 -0700 X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="5.51,273,1526367600"; d="scan'208";a="240659968" Received: from dpdk51.sh.intel.com ([10.67.110.190]) by fmsmga006.fm.intel.com with ESMTP; 26 Jun 2018 00:08:05 -0700 From: Qi Zhang To: thomas@monjalon.net, anatoly.burakov@intel.com Cc: konstantin.ananyev@intel.com, dev@dpdk.org, bruce.richardson@intel.com, ferruh.yigit@intel.com, benjamin.h.shelton@intel.com, narender.vangati@intel.com, Qi Zhang Date: Tue, 26 Jun 2018 15:08:12 +0800 Message-Id: <20180626070832.3055-5-qi.z.zhang@intel.com> X-Mailer: git-send-email 2.13.6 In-Reply-To: <20180626070832.3055-1-qi.z.zhang@intel.com> References: <20180607123849.14439-1-qi.z.zhang@intel.com> <20180626070832.3055-1-qi.z.zhang@intel.com> Subject: [dpdk-dev] [PATCH v4 04/24] eal: enable multi process init callback X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Sender: "dev" Introduce new API rte_eal_register_mp_init that help to register a callback function which will be invoked right after multi-process channel be established (rte_mp_channel_init). Typically the API will be used by other module that want it's mp channel action callbacks can be registered during rte_eal_init automatically. Signed-off-by: Qi Zhang Acked-by: Anatoly Burakov --- lib/librte_eal/common/eal_common_proc.c | 51 ++++++++++++++++++++++++++++++++- lib/librte_eal/common/eal_private.h | 5 ++++ lib/librte_eal/common/include/rte_eal.h | 34 ++++++++++++++++++++++ lib/librte_eal/linuxapp/eal/eal.c | 2 ++ 4 files changed, 91 insertions(+), 1 deletion(-) diff --git a/lib/librte_eal/common/eal_common_proc.c b/lib/librte_eal/common/eal_common_proc.c index 707d8ab30..fc0eb4d17 100644 --- a/lib/librte_eal/common/eal_common_proc.c +++ b/lib/librte_eal/common/eal_common_proc.c @@ -619,6 +619,42 @@ unlink_sockets(const char *filter) return 0; } +struct mp_init_entry { + TAILQ_ENTRY(mp_init_entry) next; + rte_eal_mp_init_callback_t callback; +}; + +TAILQ_HEAD(mp_init_entry_list, mp_init_entry); +static struct mp_init_entry_list mp_init_entry_list = + TAILQ_HEAD_INITIALIZER(mp_init_entry_list); + +static int process_mp_init_callbacks(void) +{ + struct mp_init_entry *entry; + int ret; + + TAILQ_FOREACH(entry, &mp_init_entry_list, next) { + ret = entry->callback(); + if (ret) + return ret; + } + return 0; +} + +int __rte_experimental +rte_eal_register_mp_init(rte_eal_mp_init_callback_t callback) +{ + struct mp_init_entry *entry = calloc(1, sizeof(struct mp_init_entry)); + + if (entry == NULL) + return -ENOMEM; + + entry->callback = callback; + TAILQ_INSERT_TAIL(&mp_init_entry_list, entry, next); + + return 0; +} + int rte_mp_channel_init(void) { @@ -686,7 +722,20 @@ rte_mp_channel_init(void) flock(dir_fd, LOCK_UN); close(dir_fd); - return 0; + return process_mp_init_callbacks(); +} + +void rte_mp_init_callback_cleanup(void) +{ + struct mp_init_entry *entry; + + while (!TAILQ_EMPTY(&mp_init_entry_list)) { + TAILQ_FOREACH(entry, &mp_init_entry_list, next) { + TAILQ_REMOVE(&mp_init_entry_list, entry, next); + free(entry); + break; + } + } } /** diff --git a/lib/librte_eal/common/eal_private.h b/lib/librte_eal/common/eal_private.h index bdadc4d50..bc230ee23 100644 --- a/lib/librte_eal/common/eal_private.h +++ b/lib/librte_eal/common/eal_private.h @@ -247,6 +247,11 @@ struct rte_bus *rte_bus_find_by_device_name(const char *str); int rte_mp_channel_init(void); /** + * Cleanup all mp channel init callbacks. + */ +void rte_mp_init_callback_cleanup(void); + +/** * Internal Executes all the user application registered callbacks for * the specific device. It is for DPDK internal user only. User * application should not call it directly. diff --git a/lib/librte_eal/common/include/rte_eal.h b/lib/librte_eal/common/include/rte_eal.h index 8de5d69e8..506f17f34 100644 --- a/lib/librte_eal/common/include/rte_eal.h +++ b/lib/librte_eal/common/include/rte_eal.h @@ -512,6 +512,40 @@ __rte_deprecated const char * rte_eal_mbuf_default_mempool_ops(void); +/** + * Callback function right after multi-process channel be established. + * Typical implementation of these functions is to register mp channel + * action callbacks + * + * @return + * - 0 on success. + * - (<0) on failure. + */ +typedef int (*rte_eal_mp_init_callback_t)(void); + +/** + * @warning + * @b EXPERIMENTAL: this API may change without prior notice + * + * Register a callback function that will be invoked right after + * multi-process channel be established (rte_mp_channel_init). Typically + * the function is used by other module that want it's mp channel + * action callbacks can be registered during rte_eal_init automatically. + * + * @note + * This function only take effect when be called before rte_eal_init, + * and all registered callback will be clear during rte_eal_cleanup. + * + * @param callback + * function be called at that moment. + * + * @return + * - 0 on success. + * - (<0) on failure. + */ +int __rte_experimental +rte_eal_register_mp_init(rte_eal_mp_init_callback_t callback); + #ifdef __cplusplus } #endif diff --git a/lib/librte_eal/linuxapp/eal/eal.c b/lib/librte_eal/linuxapp/eal/eal.c index 8655b8691..45cccff7e 100644 --- a/lib/librte_eal/linuxapp/eal/eal.c +++ b/lib/librte_eal/linuxapp/eal/eal.c @@ -1048,6 +1048,8 @@ int __rte_experimental rte_eal_cleanup(void) { rte_service_finalize(); + rte_mp_init_callback_cleanup(); + return 0; } From patchwork Tue Jun 26 07:08:13 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Qi Zhang X-Patchwork-Id: 41512 X-Patchwork-Delegate: thomas@monjalon.net Return-Path: X-Original-To: patchwork@dpdk.org Delivered-To: patchwork@dpdk.org Received: from [92.243.14.124] (localhost [127.0.0.1]) by dpdk.org (Postfix) with ESMTP id 75F6258FA; Tue, 26 Jun 2018 09:08:21 +0200 (CEST) Received: from mga09.intel.com (mga09.intel.com [134.134.136.24]) by dpdk.org (Postfix) with ESMTP id E29FE4F98 for ; Tue, 26 Jun 2018 09:08:08 +0200 (CEST) X-Amp-Result: SKIPPED(no attachment in message) X-Amp-File-Uploaded: False Received: from fmsmga006.fm.intel.com ([10.253.24.20]) by orsmga102.jf.intel.com with ESMTP/TLS/DHE-RSA-AES256-GCM-SHA384; 26 Jun 2018 00:08:08 -0700 X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="5.51,273,1526367600"; d="scan'208";a="240659979" Received: from dpdk51.sh.intel.com ([10.67.110.190]) by fmsmga006.fm.intel.com with ESMTP; 26 Jun 2018 00:08:07 -0700 From: Qi Zhang To: thomas@monjalon.net, anatoly.burakov@intel.com Cc: konstantin.ananyev@intel.com, dev@dpdk.org, bruce.richardson@intel.com, ferruh.yigit@intel.com, benjamin.h.shelton@intel.com, narender.vangati@intel.com, Qi Zhang Date: Tue, 26 Jun 2018 15:08:13 +0800 Message-Id: <20180626070832.3055-6-qi.z.zhang@intel.com> X-Mailer: git-send-email 2.13.6 In-Reply-To: <20180626070832.3055-1-qi.z.zhang@intel.com> References: <20180607123849.14439-1-qi.z.zhang@intel.com> <20180626070832.3055-1-qi.z.zhang@intel.com> Subject: [dpdk-dev] [PATCH v4 05/24] eal: support mp task be invoked in a separate task X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Sender: "dev" We know the limitation that sync IPC can't be invoked in mp handler itself which will cause deadlock, the patch introduce new API rte_eal_mp_task_add to support mp handler be delegated in a separate task. Signed-off-by: Qi Zhang --- lib/librte_eal/common/eal_common_proc.c | 67 +++++++++++++++++++++++++++++++++ lib/librte_eal/common/include/rte_eal.h | 31 +++++++++++++++ 2 files changed, 98 insertions(+) diff --git a/lib/librte_eal/common/eal_common_proc.c b/lib/librte_eal/common/eal_common_proc.c index fc0eb4d17..166bb0951 100644 --- a/lib/librte_eal/common/eal_common_proc.c +++ b/lib/librte_eal/common/eal_common_proc.c @@ -27,6 +27,7 @@ #include #include #include +#include #include "eal_private.h" #include "eal_filesystem.h" @@ -738,6 +739,72 @@ void rte_mp_init_callback_cleanup(void) } } +struct mp_task { + TAILQ_ENTRY(mp_task) next; + rte_eal_mp_task task; + void *args; +}; + +TAILQ_HEAD(mp_task_list, mp_task); +static struct mp_task_list mp_task_list = + TAILQ_HEAD_INITIALIZER(mp_task_list); +static rte_spinlock_t mp_task_lock = RTE_SPINLOCK_INITIALIZER; + +static void *schedule_mp_task(void *args __rte_unused) +{ + struct mp_task *task; + + rte_spinlock_lock(&mp_task_lock); + while (!TAILQ_EMPTY(&mp_task_list)) { + + task = TAILQ_FIRST(&mp_task_list); + rte_spinlock_unlock(&mp_task_lock); + + task->task(task->args); + + rte_spinlock_lock(&mp_task_lock); + TAILQ_REMOVE(&mp_task_list, task, next); + if (task->args) + free(task->args); + free(task); + } + + rte_spinlock_unlock(&mp_task_lock); + return NULL; +} + +int __rte_experimental +rte_eal_mp_task_add(rte_eal_mp_task task, void *args) +{ + struct mp_task *t = calloc(1, sizeof(struct mp_task)); + pthread_t tid; + + if (t == NULL) + return -ENOMEM; + + t->task = task; + t->args = args; + + rte_spinlock_lock(&mp_task_lock); + + if (TAILQ_EMPTY(&mp_task_list)) { + TAILQ_INSERT_TAIL(&mp_task_list, t, next); + rte_spinlock_unlock(&mp_task_lock); + + if (rte_ctrl_thread_create(&tid, + "rte_mp_handle", NULL, schedule_mp_task, NULL) < 0) { + RTE_LOG(ERR, EAL, "failed to create mp thead: %s\n", + strerror(errno)); + } + return 0; + } + + TAILQ_INSERT_TAIL(&mp_task_list, t, next); + rte_spinlock_unlock(&mp_task_lock); + + return 0; +} + /** * Return -1, as fail to send message and it's caused by the local side. * Return 0, as fail to send message and it's caused by the remote side. diff --git a/lib/librte_eal/common/include/rte_eal.h b/lib/librte_eal/common/include/rte_eal.h index 506f17f34..0ce49668c 100644 --- a/lib/librte_eal/common/include/rte_eal.h +++ b/lib/librte_eal/common/include/rte_eal.h @@ -546,6 +546,37 @@ typedef int (*rte_eal_mp_init_callback_t)(void); int __rte_experimental rte_eal_register_mp_init(rte_eal_mp_init_callback_t callback); +/** + * Function to perform the task that handle mp request, + * it will be scheduled on a separate task. + * + * @param args + * argument parse to the function point. + */ +typedef void (*rte_eal_mp_task)(void *args); + +/** + * @warning + * @b EXPERIMENTAL: this API may change without prior notice + * + * Add a rte_eal_mp_task into a task list, it will invoked in a + * separate task, the purpose is to prevent deadlock if sync IPC + * is required in the task. + * + * @param task + * function point to perform the task. + * + * @param args + * argument parse to the function point. + * + * @return + * - 0 on success. + * - (<0) on failure. + */ +int __rte_experimental +rte_eal_mp_task_add(rte_eal_mp_task task, void *args); + + #ifdef __cplusplus } #endif From patchwork Tue Jun 26 07:08:14 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Qi Zhang X-Patchwork-Id: 41513 X-Patchwork-Delegate: thomas@monjalon.net Return-Path: X-Original-To: patchwork@dpdk.org Delivered-To: patchwork@dpdk.org Received: from [92.243.14.124] (localhost [127.0.0.1]) by dpdk.org (Postfix) with ESMTP id 89B7B5B1C; Tue, 26 Jun 2018 09:08:23 +0200 (CEST) Received: from mga09.intel.com (mga09.intel.com [134.134.136.24]) by dpdk.org (Postfix) with ESMTP id 9E1915398 for ; Tue, 26 Jun 2018 09:08:10 +0200 (CEST) X-Amp-Result: SKIPPED(no attachment in message) X-Amp-File-Uploaded: False Received: from fmsmga006.fm.intel.com ([10.253.24.20]) by orsmga102.jf.intel.com with ESMTP/TLS/DHE-RSA-AES256-GCM-SHA384; 26 Jun 2018 00:08:10 -0700 X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="5.51,273,1526367600"; d="scan'208";a="240659986" Received: from dpdk51.sh.intel.com ([10.67.110.190]) by fmsmga006.fm.intel.com with ESMTP; 26 Jun 2018 00:08:08 -0700 From: Qi Zhang To: thomas@monjalon.net, anatoly.burakov@intel.com Cc: konstantin.ananyev@intel.com, dev@dpdk.org, bruce.richardson@intel.com, ferruh.yigit@intel.com, benjamin.h.shelton@intel.com, narender.vangati@intel.com, Qi Zhang Date: Tue, 26 Jun 2018 15:08:14 +0800 Message-Id: <20180626070832.3055-7-qi.z.zhang@intel.com> X-Mailer: git-send-email 2.13.6 In-Reply-To: <20180626070832.3055-1-qi.z.zhang@intel.com> References: <20180607123849.14439-1-qi.z.zhang@intel.com> <20180626070832.3055-1-qi.z.zhang@intel.com> Subject: [dpdk-dev] [PATCH v4 06/24] ethdev: enable hotplug on multi-process X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Sender: "dev" We are going to introduce the solution to handle different hotplug cases in multi-process situation, it include below scenario: 1. Attach a share device from primary 2. Detach a share device from primary 3. Attach a share device from secondary 4. Detach a share device from secondary 5. Attach a private device from secondary 6. Detach a private device from secondary 7. Detach a share device from secondary privately 8. Attach a share device from secondary privately In primary-secondary process model, we assume device is shared by default. that means attach or detach a device on any process will broadcast to all other processes through mp channel then device information will be synchronized on all processes. Any failure during attaching process will cause inconsistent status between processes, so proper rollback action should be considered. Also it is not safe to detach a share device when other process still use it, so a handshake mechanism is introduced. This patch covers the implementation of case 1,2,5,6,7,8. Case 3,4 will be implemented on separate patch as well as handshake mechanism. Scenario for Case 1, 2: attach device a) primary attach the new device if failed goto h). b) primary send attach sync request to all secondary. c) secondary receive request and attach device and send reply. d) primary check the reply if all success go to i). e) primary send attach rollback sync request to all secondary. f) secondary receive the request and detach device and send reply. g) primary receive the reply and detach device as rollback action. h) attach fail i) attach success detach device a) primary perform pre-detach check, if device is locked, goto i). b) primary send pre-detach sync request to all secondary. c) secondary perform pre-detach check and send reply. d) primary check the reply if any fail goto i). e) primary send detach sync request to all secondary f) secondary detach the device and send reply (assume no fail) g) primary detach the device. h) detach success i) detach failed Case 5, 6: Secondary process can attach private device which only visible to itself, in this case no IPC is involved, primary process is not allowed to have private device so far. Case 7, 8: Secondary process can also temporally to detach a share device "privately" then attach it back later, this action also not impact other processes. APIs changes: rte_eth_dev_attach and rte_eth_dev_attach are extended to support share device attach/detach in primary-secondary process model, it will be called in case 1,2,3,4. New API rte_eth_dev_attach_private and rte_eth_dev_detach_private are introduced to cover case 5,6,7,8, this API can only be invoked in secondary process. Signed-off-by: Qi Zhang --- lib/librte_ethdev/Makefile | 1 + lib/librte_ethdev/ethdev_mp.c | 248 ++++++++++++++++++++++++++++++++++++ lib/librte_ethdev/ethdev_mp.h | 41 ++++++ lib/librte_ethdev/ethdev_private.h | 39 ++++++ lib/librte_ethdev/meson.build | 1 + lib/librte_ethdev/rte_ethdev.c | 190 ++++++++++++++++++++++++--- lib/librte_ethdev/rte_ethdev.h | 45 +++++++ lib/librte_ethdev/rte_ethdev_core.h | 5 + 8 files changed, 553 insertions(+), 17 deletions(-) create mode 100644 lib/librte_ethdev/ethdev_mp.c create mode 100644 lib/librte_ethdev/ethdev_mp.h create mode 100644 lib/librte_ethdev/ethdev_private.h diff --git a/lib/librte_ethdev/Makefile b/lib/librte_ethdev/Makefile index c2f2f7d82..d0a059b83 100644 --- a/lib/librte_ethdev/Makefile +++ b/lib/librte_ethdev/Makefile @@ -19,6 +19,7 @@ EXPORT_MAP := rte_ethdev_version.map LIBABIVER := 9 SRCS-y += rte_ethdev.c +SRCS-y += ethdev_mp.c SRCS-y += rte_flow.c SRCS-y += rte_tm.c SRCS-y += rte_mtr.c diff --git a/lib/librte_ethdev/ethdev_mp.c b/lib/librte_ethdev/ethdev_mp.c new file mode 100644 index 000000000..87fc430bf --- /dev/null +++ b/lib/librte_ethdev/ethdev_mp.c @@ -0,0 +1,248 @@ +/* SPDX-License-Identifier: BSD-3-Clause + * Copyright(c) 2010-2018 Intel Corporation + */ + +#include +#include "rte_ethdev_driver.h" +#include "ethdev_mp.h" + +#define MP_TIMEOUT_S 5 /**< 5 seconds timeouts */ + +struct mp_reply_bundle { + struct rte_mp_msg msg; + const void *peer; +}; + +static int detach_on_secondary(uint16_t port_id) +{ + struct rte_device *dev; + struct rte_bus *bus; + int ret = 0; + + if (rte_eth_devices[port_id].state == RTE_ETH_DEV_UNUSED) { + ethdev_log(ERR, "detach on secondary: invalid port %d\n", + port_id); + return -ENODEV; + } + + dev = rte_eth_devices[port_id].device; + if (dev == NULL) + return -EINVAL; + + bus = rte_bus_find_by_device(dev); + if (bus == NULL) + return -ENOENT; + + ret = rte_eal_hotplug_remove(bus->name, dev->name); + if (ret) { + ethdev_log(ERR, "failed to hot unplug bus: %s, device:%s\n", + bus->name, dev->name); + return ret; + } + + rte_eth_dev_release_port_private(&rte_eth_devices[port_id]); + return ret; +} + +static int attach_on_secondary(const char *devargs, uint16_t port_id) +{ + struct rte_devargs da; + int ret; + + if (rte_eth_devices[port_id].state != RTE_ETH_DEV_UNUSED) { + ethdev_log(ERR, "port %d already in used, failed to attach\n", + port_id); + return -EINVAL; + } + + memset(&da, 0, sizeof(da)); + + if (rte_devargs_parse(&da, "%s", devargs)) { + ethdev_log(ERR, "failed to parse devargs %s\n", devargs); + return -EINVAL; + } + + ret = rte_eal_hotplug_add(da.bus->name, da.name, ""); + if (ret) { + ethdev_log(ERR, "failed to hotplug bus:%s, device:%s\n", + da.bus->name, da.name); + free(da.args); + return ret; + } + + if (rte_eth_devices[port_id].state == RTE_ETH_DEV_UNUSED) { + ethdev_log(ERR, + "failed to attach to port %d, this is a pmd issue\n", + port_id); + free(da.args); + return -ENODEV; + } + free(da.args); + return 0; +} + +static int +handle_secondary_request(const struct rte_mp_msg *msg, const void *peer) +{ + RTE_SET_USED(msg); + RTE_SET_USED(peer); + return -ENOTSUP; +} + +static void __handle_primary_request(void *param) +{ + struct mp_reply_bundle *bundle = param; + struct rte_mp_msg *msg = &bundle->msg; + const struct eth_dev_mp_req *req = + (const struct eth_dev_mp_req *)msg->param; + struct rte_mp_msg mp_resp; + struct eth_dev_mp_req *resp = + (struct eth_dev_mp_req *)mp_resp.param; + int ret = 0; + + memset(&mp_resp, 0, sizeof(mp_resp)); + + switch (req->t) { + case REQ_TYPE_ATTACH: + ret = attach_on_secondary(req->devargs, req->port_id); + break; + case REQ_TYPE_PRE_DETACH: + ret = 0; + break; + case REQ_TYPE_DETACH: + case REQ_TYPE_ATTACH_ROLLBACK: + ret = detach_on_secondary(req->port_id); + break; + default: + ret = -EINVAL; + } + + strlcpy(mp_resp.name, ETH_DEV_MP_ACTION_REQUEST, sizeof(mp_resp.name)); + mp_resp.len_param = sizeof(*req); + memcpy(resp, req, sizeof(*resp)); + resp->result = ret; + if (rte_mp_reply(&mp_resp, bundle->peer) < 0) + ethdev_log(ERR, "failed to send reply to primary request\n"); +} + +static int +handle_primary_request(const struct rte_mp_msg *msg, const void *peer) +{ + + struct rte_mp_msg mp_resp; + const struct eth_dev_mp_req *req = + (const struct eth_dev_mp_req *)msg->param; + struct eth_dev_mp_req *resp = + (struct eth_dev_mp_req *)mp_resp.param; + struct mp_reply_bundle *bundle; + int ret = 0; + + memset(&mp_resp, 0, sizeof(mp_resp)); + strlcpy(mp_resp.name, ETH_DEV_MP_ACTION_REQUEST, sizeof(mp_resp.name)); + mp_resp.len_param = sizeof(*req); + memcpy(resp, req, sizeof(*resp)); + + bundle = calloc(1, sizeof(*bundle)); + if (bundle == NULL) { + resp->result = -ENOMEM; + ret = rte_mp_reply(&mp_resp, peer); + if (ret) { + ethdev_log(ERR, "failed to send reply to primary request\n"); + return ret; + } + } + + bundle->msg = *msg; + bundle->peer = peer; + + ret = rte_eal_mp_task_add(__handle_primary_request, bundle); + if (ret) { + resp->result = ret; + ret = rte_mp_reply(&mp_resp, peer); + if (ret) { + ethdev_log(ERR, "failed to send reply to primary request\n"); + return ret; + } + } + return 0; +} + +int eth_dev_request_to_primary(struct eth_dev_mp_req *req) +{ + RTE_SET_USED(req); + return -ENOTSUP; +} + +/** + * Request from primary to secondary. + * + * Be invoked when try to attach or detach a share device + * from primary process. + */ +int eth_dev_request_to_secondary(struct eth_dev_mp_req *req) +{ + struct rte_mp_msg mp_req; + struct rte_mp_reply mp_reply; + struct timespec ts = {.tv_sec = MP_TIMEOUT_S, .tv_nsec = 0}; + int ret; + int i; + + memset(&mp_req, 0, sizeof(mp_req)); + memcpy(mp_req.param, req, sizeof(*req)); + mp_req.len_param = sizeof(*req); + strlcpy(mp_req.name, ETH_DEV_MP_ACTION_REQUEST, sizeof(mp_req.name)); + + ret = rte_mp_request_sync(&mp_req, &mp_reply, &ts); + if (ret) { + ethdev_log(ERR, "rte_mp_request_sync failed\n"); + return ret; + } + + if (mp_reply.nb_sent != mp_reply.nb_received) { + ethdev_log(ERR, "not all secondary reply\n"); + return -1; + } + + req->result = 0; + for (i = 0; i < mp_reply.nb_received; i++) { + struct eth_dev_mp_req *resp = + (struct eth_dev_mp_req *)mp_reply.msgs[i].param; + if (resp->result) { + req->result = resp->result; + break; + } + } + + return 0; +} + +static int on_mp_init(void) +{ + int ret; + + if (rte_eal_process_type() == RTE_PROC_PRIMARY) { + ret = rte_mp_action_register(ETH_DEV_MP_ACTION_REQUEST, + handle_secondary_request); + if (ret) { + RTE_LOG(ERR, EAL, "Couldn't register '%s' action\n", + ETH_DEV_MP_ACTION_REQUEST); + return ret; + } + } else { + ret = rte_mp_action_register(ETH_DEV_MP_ACTION_REQUEST, + handle_primary_request); + if (ret) { + RTE_LOG(ERR, EAL, "Couldn't register '%s' action\n", + ETH_DEV_MP_ACTION_REQUEST); + return ret; + } + } + + return 0; +} + +RTE_INIT(ethdev_mp_init) +{ + if (rte_eal_register_mp_init(on_mp_init)) + RTE_LOG(ERR, EAL, "ethdev mp channel init failed\n"); +} diff --git a/lib/librte_ethdev/ethdev_mp.h b/lib/librte_ethdev/ethdev_mp.h new file mode 100644 index 000000000..40be46c89 --- /dev/null +++ b/lib/librte_ethdev/ethdev_mp.h @@ -0,0 +1,41 @@ +/* SPDX-License-Identifier: BSD-3-Clause + * Copyright(c) 2018 Intel Corporation + */ + +#ifndef _RTE_ETHDEV_MP_H_ +#define _RTE_ETHDEV_MP_H_ + +#define MAX_DEV_ARGS_LEN 0x80 + +#define ETH_DEV_MP_ACTION_REQUEST "eth_dev_mp_request" +#define ETH_DEV_MP_ACTION_RESPONSE "eth_dev_mp_response" + +enum eth_dev_req_type { + REQ_TYPE_ATTACH, + REQ_TYPE_PRE_DETACH, + REQ_TYPE_DETACH, + REQ_TYPE_ATTACH_ROLLBACK, +}; + +struct eth_dev_mp_req { + enum eth_dev_req_type t; + char devargs[MAX_DEV_ARGS_LEN]; + uint16_t port_id; + int result; +}; + +/** + * this is a synchronous wrapper for secondary process send + * request to primary process, this is invoked when an attach + * or detach request issued from primary. + */ +int eth_dev_request_to_primary(struct eth_dev_mp_req *req); + +/** + * this is a synchronous wrapper for primary process send + * request to secondary process, this is invoked when an attach + * or detach request issued from secondary process. + */ +int eth_dev_request_to_secondary(struct eth_dev_mp_req *req); + +#endif diff --git a/lib/librte_ethdev/ethdev_private.h b/lib/librte_ethdev/ethdev_private.h new file mode 100644 index 000000000..981e7de8a --- /dev/null +++ b/lib/librte_ethdev/ethdev_private.h @@ -0,0 +1,39 @@ +/* SPDX-License-Identifier: BSD-3-Clause + * Copyright(c) 2010-2018 Intel Corporation + */ + +#ifndef _ETHDEV_PRIVATE_H_ +#define _ETHDEV_PRIVATE_H_ + +/** + * Attach a new Ethernet device in current process. + * + * @param devargs + * A pointer to a strings array describing the new device + * to be attached. The strings should be a pci address like + * '0000:01:00.0' or virtual device name like 'net_pcap0'. + * + * @param port_id + * A pointer to a port identifier actually attached. + * + * @return + * 0 on success and port_id is filled, negative on error + */ +int do_eth_dev_attach(const char *devargs, uint16_t *port_id); + +/** + * Detach a Ethernet device in current process. + * + * @param port_id + * The port identifier of the device to detach. + * + * @param devname + * A pointer to a buffer that will be filled with the device name. + * This buffer must be at least RTE_DEV_NAME_MAX_LEN long. + * + * @return + * 0 on success and devname is filled, negative on error + */ +int do_eth_dev_detach(uint16_t port_id); + +#endif diff --git a/lib/librte_ethdev/meson.build b/lib/librte_ethdev/meson.build index aed5d2265..b60256855 100644 --- a/lib/librte_ethdev/meson.build +++ b/lib/librte_ethdev/meson.build @@ -5,6 +5,7 @@ name = 'ethdev' version = 9 allow_experimental_apis = true sources = files('ethdev_profile.c', + 'ethdev_mp.c' 'rte_ethdev.c', 'rte_flow.c', 'rte_mtr.c', diff --git a/lib/librte_ethdev/rte_ethdev.c b/lib/librte_ethdev/rte_ethdev.c index 205b2ee33..1a5861f30 100644 --- a/lib/librte_ethdev/rte_ethdev.c +++ b/lib/librte_ethdev/rte_ethdev.c @@ -41,11 +41,13 @@ #include "rte_ethdev.h" #include "rte_ethdev_driver.h" #include "ethdev_profile.h" +#include "ethdev_mp.h" +#include "ethdev_private.h" -static int ethdev_logtype; +int ethdev_logtype; -#define ethdev_log(level, fmt, ...) \ - rte_log(RTE_LOG_ ## level, ethdev_logtype, fmt "\n", ## __VA_ARGS__) +#define RTE_ETH_MP_ACTION_REQUEST "rte_eth_mp_request" +#define RTE_ETH_MP_ACTION_RESPONSE "rte_eth_mp_response" static const char *MZ_RTE_ETH_DEV_DATA = "rte_eth_dev_data"; struct rte_eth_dev rte_eth_devices[RTE_MAX_ETHPORTS]; @@ -656,9 +658,8 @@ eth_err(uint16_t port_id, int ret) return ret; } -/* attach the new device, then store port_id of the device */ int -rte_eth_dev_attach(const char *devargs, uint16_t *port_id) +do_eth_dev_attach(const char *devargs, uint16_t *port_id) { int current = rte_eth_dev_count_total(); struct rte_devargs da; @@ -703,14 +704,105 @@ rte_eth_dev_attach(const char *devargs, uint16_t *port_id) return ret; } -/* detach the device, then store the name of the device */ int -rte_eth_dev_detach(uint16_t port_id, char *name __rte_unused) +do_eth_dev_detach(uint16_t port_id) { struct rte_device *dev; struct rte_bus *bus; + int ret = 0; + + dev = rte_eth_devices[port_id].device; + if (dev == NULL) + return -EINVAL; + + bus = rte_bus_find_by_device(dev); + if (bus == NULL) + return -ENOENT; + + ret = rte_eal_hotplug_remove(bus->name, dev->name); + if (ret < 0) + return ret; + + rte_eth_dev_release_port(&rte_eth_devices[port_id]); + return ret; + +} + +/* attach the new device, then store port_id of the device */ +int +rte_eth_dev_attach(const char *devargs, uint16_t *port_id) +{ + struct eth_dev_mp_req req; + int ret; + + if (rte_eal_process_type() != RTE_PROC_PRIMARY) { + + /** + * If secondary process, we just send request to primary + * to start the process. + */ + req.t = REQ_TYPE_ATTACH; + strlcpy(req.devargs, devargs, MAX_DEV_ARGS_LEN); + + ret = eth_dev_request_to_primary(&req); + if (ret) { + ethdev_log(ERR, + "Failed to send device attach request to primary\n"); + return ret; + } + + *port_id = req.port_id; + return req.result; + } + + ret = do_eth_dev_attach(devargs, port_id); + if (ret) + return ret; + + /* send attach request to seoncary */ + req.t = REQ_TYPE_ATTACH; + strlcpy(req.devargs, devargs, MAX_DEV_ARGS_LEN); + req.port_id = *port_id; + ret = eth_dev_request_to_secondary(&req); + if (ret) { + ethdev_log(ERR, "Failed to send device attach request to secondary\n"); + goto rollback; + } + + if (req.result) + goto rollback; + + return 0; + +rollback: + /* send rollback request to secondary since some one fail to attach */ + req.t = REQ_TYPE_ATTACH_ROLLBACK; + req.port_id = *port_id; + eth_dev_request_to_secondary(&req); + + do_eth_dev_detach(*port_id); + + return -ENODEV; +} + +/* attach the new device, then store port_id of the device */ +int +rte_eth_dev_attach_private(const char *devargs, uint16_t *port_id) +{ + + if (rte_eal_process_type() == RTE_PROC_PRIMARY) + return -ENOTSUP; + + return do_eth_dev_attach(devargs, port_id); +} + +/* detach the device, then store the name of the device */ +int +rte_eth_dev_detach(uint16_t port_id, char *name __rte_unused) +{ + struct eth_dev_mp_req req = {0}; + int ret; uint32_t dev_flags; - int ret = -1; RTE_ETH_VALID_PORTID_OR_ERR_RET(port_id, -EINVAL); @@ -721,22 +813,86 @@ rte_eth_dev_detach(uint16_t port_id, char *name __rte_unused) return -ENOTSUP; } - dev = rte_eth_devices[port_id].device; - if (dev == NULL) - return -EINVAL; + if (rte_eal_process_type() != RTE_PROC_PRIMARY) { + req.t = REQ_TYPE_DETACH; + req.port_id = port_id; - bus = rte_bus_find_by_device(dev); - if (bus == NULL) - return -ENOENT; + /** + * If secondary process, we just send request to primary + * to start the process. + */ + ret = eth_dev_request_to_primary(&req); + if (ret) { + ethdev_log(ERR, + "Failed to send device detach request to primary\n"); + return ret; + } - ret = rte_eal_hotplug_remove(bus->name, dev->name); - if (ret < 0) + return req.result; + } + + /* check pre_detach */ + req.t = REQ_TYPE_PRE_DETACH; + req.port_id = port_id; + ret = eth_dev_request_to_secondary(&req); + if (ret) { + ethdev_log(ERR, + "Failed to send device pre-detach request to secondary\n"); + return ret; + } + + if (req.result) { + ethdev_log(ERR, + "Device is busy on secondary, can't be detached\n"); + return req.result; + } + + /* detach on seconary first */ + req.t = REQ_TYPE_DETACH; + ret = eth_dev_request_to_secondary(&req); + if (ret) { + ethdev_log(ERR, + "Failed to send device detach request to secondary\n"); + return ret; + } + + if (req.result) + /** + * this should rarely happen, something wrong in secondary + * process, will not block primary detach. + */ + ethdev_log(ERR, + "Failed to detach device on secondary process\n"); + + /* detach on primary */ + ret = do_eth_dev_detach(port_id); + if (ret) return ret; - rte_eth_dev_release_port(&rte_eth_devices[port_id]); return 0; } +/* detach the device, then store the name of the device */ +int +rte_eth_dev_detach_private(uint16_t port_id, char *name __rte_unused) +{ + uint32_t dev_flags; + + if (rte_eal_process_type() == RTE_PROC_PRIMARY) + return -ENOTSUP; + + RTE_ETH_VALID_PORTID_OR_ERR_RET(port_id, -EINVAL); + + dev_flags = rte_eth_devices[port_id].data->dev_flags; + if (dev_flags & RTE_ETH_DEV_BONDED_SLAVE) { + ethdev_log(ERR, + "Port %" PRIu16 " is bonded, cannot detach", port_id); + return -ENOTSUP; + } + + return do_eth_dev_detach(port_id); +} + static int rte_eth_dev_rx_queue_config(struct rte_eth_dev *dev, uint16_t nb_queues) { diff --git a/lib/librte_ethdev/rte_ethdev.h b/lib/librte_ethdev/rte_ethdev.h index 36e3984ea..813806e3c 100644 --- a/lib/librte_ethdev/rte_ethdev.h +++ b/lib/librte_ethdev/rte_ethdev.h @@ -1462,6 +1462,9 @@ uint16_t __rte_experimental rte_eth_dev_count_total(void); /** * Attach a new Ethernet device specified by arguments. + * In multi-process mode, it will sync with other process + * to make sure all processes attach the device, any + * failure on other process will rollback the action. * * @param devargs * A pointer to a strings array describing the new device @@ -1475,9 +1478,31 @@ uint16_t __rte_experimental rte_eth_dev_count_total(void); int rte_eth_dev_attach(const char *devargs, uint16_t *port_id); /** + * @warning + * @b EXPERIMENTAL: this API may change without prior notice + * + * Attach a private Ethernet device specified by arguments. + * A private device is invisible to other process. + * Can only be invoked in secondary process. + * + * @param devargs + * A pointer to a strings array describing the new device + * to be attached. The strings should be a pci address like + * '0000:01:00.0' or virtual device name like 'net_pcap0'. + * @param port_id + * A pointer to a port identifier actually attached. + * @return + * 0 on success and port_id is filled, negative on error + */ +int __rte_experimental +rte_eth_dev_attach_private(const char *devargs, uint16_t *port_id); + +/** * Detach a Ethernet device specified by port identifier. * This function must be called when the device is in the * closed state. + * In multi-process mode, it will sync with other process + * to detach the device. * * @param port_id * The port identifier of the device to detach. @@ -1490,6 +1515,26 @@ int rte_eth_dev_attach(const char *devargs, uint16_t *port_id); int rte_eth_dev_detach(uint16_t port_id, char *devname); /** + * @warning + * @b EXPERIMENTAL: this API may change without prior notice + * + * Detach a private Ethernet device specified by port identifier + * This function must be called when the device is in the + * closed state. + * Can only be invoked in secondary process. + * + * @param port_id + * The port identifier of the device to detach. + * @param devname + * A pointer to a buffer that will be filled with the device name. + * This buffer must be at least RTE_DEV_NAME_MAX_LEN long. + * @return + * 0 on success and devname is filled, negative on error + */ +int __rte_experimental +rte_eth_dev_detach_private(uint16_t port_id, char *devname); + +/** * Convert a numerical speed in Mbps to a bitmap flag that can be used in * the bitmap link_speeds of the struct rte_eth_conf * diff --git a/lib/librte_ethdev/rte_ethdev_core.h b/lib/librte_ethdev/rte_ethdev_core.h index 33d12b3a2..2cb6de745 100644 --- a/lib/librte_ethdev/rte_ethdev_core.h +++ b/lib/librte_ethdev/rte_ethdev_core.h @@ -622,4 +622,9 @@ struct rte_eth_dev_data { */ extern struct rte_eth_dev rte_eth_devices[]; +extern int ethdev_logtype; +#define ethdev_log(level, fmt, ...) \ + rte_log(RTE_LOG_ ## level, ethdev_logtype, fmt "\n", ## __VA_ARGS__) + + #endif /* _RTE_ETHDEV_CORE_H_ */ From patchwork Tue Jun 26 07:08:15 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Qi Zhang X-Patchwork-Id: 41514 X-Patchwork-Delegate: thomas@monjalon.net Return-Path: X-Original-To: patchwork@dpdk.org Delivered-To: patchwork@dpdk.org Received: from [92.243.14.124] (localhost [127.0.0.1]) by dpdk.org (Postfix) with ESMTP id 060045B26; Tue, 26 Jun 2018 09:08:26 +0200 (CEST) Received: from mga09.intel.com (mga09.intel.com [134.134.136.24]) by dpdk.org (Postfix) with ESMTP id 269045398 for ; Tue, 26 Jun 2018 09:08:11 +0200 (CEST) X-Amp-Result: SKIPPED(no attachment in message) X-Amp-File-Uploaded: False Received: from fmsmga006.fm.intel.com ([10.253.24.20]) by orsmga102.jf.intel.com with ESMTP/TLS/DHE-RSA-AES256-GCM-SHA384; 26 Jun 2018 00:08:12 -0700 X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="5.51,273,1526367600"; d="scan'208";a="240659996" Received: from dpdk51.sh.intel.com ([10.67.110.190]) by fmsmga006.fm.intel.com with ESMTP; 26 Jun 2018 00:08:10 -0700 From: Qi Zhang To: thomas@monjalon.net, anatoly.burakov@intel.com Cc: konstantin.ananyev@intel.com, dev@dpdk.org, bruce.richardson@intel.com, ferruh.yigit@intel.com, benjamin.h.shelton@intel.com, narender.vangati@intel.com, Qi Zhang Date: Tue, 26 Jun 2018 15:08:15 +0800 Message-Id: <20180626070832.3055-8-qi.z.zhang@intel.com> X-Mailer: git-send-email 2.13.6 In-Reply-To: <20180626070832.3055-1-qi.z.zhang@intel.com> References: <20180607123849.14439-1-qi.z.zhang@intel.com> <20180626070832.3055-1-qi.z.zhang@intel.com> Subject: [dpdk-dev] [PATCH v4 07/24] ethdev: introduce device lock X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Sender: "dev" Introduce API rte_eth_dev_lock and rte_eth_dev_unlock to let application lock or unlock on specific ethdev, a locked device can't be detached, this help applicaiton to prevent unexpected device detaching, especially in multi-process envrionment. Aslo introduce the new API rte_eth_dev_lock_with_callback and rte_eth_dev_unlock_with callback to let application to register a callback function which will be invoked before a device is going to be detached, the return value of the function will decide if device will continue be detached or not, this support application to do condition check at runtime. Signed-off-by: Qi Zhang --- lib/librte_ethdev/Makefile | 1 + lib/librte_ethdev/ethdev_lock.c | 140 ++++++++++++++++++++++++++++++++++++++++ lib/librte_ethdev/ethdev_lock.h | 31 +++++++++ lib/librte_ethdev/ethdev_mp.c | 3 +- lib/librte_ethdev/meson.build | 1 + lib/librte_ethdev/rte_ethdev.c | 60 ++++++++++++++++- lib/librte_ethdev/rte_ethdev.h | 124 +++++++++++++++++++++++++++++++++++ 7 files changed, 358 insertions(+), 2 deletions(-) create mode 100644 lib/librte_ethdev/ethdev_lock.c create mode 100644 lib/librte_ethdev/ethdev_lock.h diff --git a/lib/librte_ethdev/Makefile b/lib/librte_ethdev/Makefile index d0a059b83..62bef03fc 100644 --- a/lib/librte_ethdev/Makefile +++ b/lib/librte_ethdev/Makefile @@ -20,6 +20,7 @@ LIBABIVER := 9 SRCS-y += rte_ethdev.c SRCS-y += ethdev_mp.c +SRCS-y += ethdev_lock.c SRCS-y += rte_flow.c SRCS-y += rte_tm.c SRCS-y += rte_mtr.c diff --git a/lib/librte_ethdev/ethdev_lock.c b/lib/librte_ethdev/ethdev_lock.c new file mode 100644 index 000000000..6379519e3 --- /dev/null +++ b/lib/librte_ethdev/ethdev_lock.c @@ -0,0 +1,140 @@ +/* SPDX-License-Identifier: BSD-3-Clause + * Copyright(c) 2018 Intel Corporation + */ +#include "ethdev_lock.h" + +struct lock_entry { + TAILQ_ENTRY(lock_entry) next; + rte_eth_dev_lock_callback_t callback; + uint16_t port_id; + void *user_args; + int ref_count; +}; + +TAILQ_HEAD(lock_entry_list, lock_entry); +static struct lock_entry_list lock_entry_list = + TAILQ_HEAD_INITIALIZER(lock_entry_list); +static rte_spinlock_t lock_entry_lock = RTE_SPINLOCK_INITIALIZER; + +int +register_lock_callback(uint16_t port_id, + rte_eth_dev_lock_callback_t callback, + void *user_args) +{ + struct lock_entry *le; + + rte_spinlock_lock(&lock_entry_lock); + + TAILQ_FOREACH(le, &lock_entry_list, next) { + if (le->port_id == port_id && + le->callback == callback && + le->user_args == user_args) + break; + } + + if (le == NULL) { + le = calloc(1, sizeof(struct lock_entry)); + if (le == NULL) { + rte_spinlock_unlock(&lock_entry_lock); + return -ENOMEM; + } + le->callback = callback; + le->port_id = port_id; + le->user_args = user_args; + TAILQ_INSERT_TAIL(&lock_entry_list, le, next); + } + le->ref_count++; + + rte_spinlock_unlock(&lock_entry_lock); + return 0; +} + +int +unregister_lock_callback(uint16_t port_id, + rte_eth_dev_lock_callback_t callback, + void *user_args) +{ + struct lock_entry *le; + int ret = 0; + + rte_spinlock_lock(&lock_entry_lock); + + TAILQ_FOREACH(le, &lock_entry_list, next) { + if (le->port_id == port_id && + le->callback == callback && + le->user_args == user_args) + break; + } + + if (le != NULL) { + le->ref_count--; + if (le->ref_count == 0) { + TAILQ_REMOVE(&lock_entry_list, le, next); + free(le); + } + } else { + ret = -ENOENT; + } + + rte_spinlock_unlock(&lock_entry_lock); + return ret; +} + +static int clean_lock_callback_one(uint16_t port_id) +{ + struct lock_entry *le; + int ret = 0; + + TAILQ_FOREACH(le, &lock_entry_list, next) { + if (le->port_id == port_id) + break; + } + + if (le != NULL) { + le->ref_count--; + if (le->ref_count == 0) { + TAILQ_REMOVE(&lock_entry_list, le, next); + free(le); + } + } else { + ret = -ENOENT; + } + + return ret; + +} + +void clean_lock_callback(uint16_t port_id) +{ + int ret; + + rte_spinlock_lock(&lock_entry_lock); + + for (;;) { + ret = clean_lock_callback_one(port_id); + if (ret == -ENOENT) + break; + } + + rte_spinlock_unlock(&lock_entry_lock); +} + +int process_lock_callbacks(uint16_t port_id) +{ + struct lock_entry *le; + + rte_spinlock_lock(&lock_entry_lock); + + TAILQ_FOREACH(le, &lock_entry_list, next) { + if (le->port_id != port_id) + continue; + + if (le->callback(port_id, le->user_args)) { + rte_spinlock_unlock(&lock_entry_lock); + return -EBUSY; + } + } + + rte_spinlock_unlock(&lock_entry_lock); + return 0; +} diff --git a/lib/librte_ethdev/ethdev_lock.h b/lib/librte_ethdev/ethdev_lock.h new file mode 100644 index 000000000..82132eb0c --- /dev/null +++ b/lib/librte_ethdev/ethdev_lock.h @@ -0,0 +1,31 @@ +/* SPDX-License-Identifier: BSD-3-Clause + * Copyright(c) 2018 Intel Corporation + */ + +#ifndef _RTE_ETHDEV_LOCK_H_ +#define _RTE_ETHDEV_LOCK_H_ + +#include "rte_ethdev.h" + +/* Register lock callback function on specific port */ +int +register_lock_callback(uint16_t port_id, + rte_eth_dev_lock_callback_t callback, + void *user_args); + +/* Unregister lock callback function on specific port */ +int +unregister_lock_callback(uint16_t port_id, + rte_eth_dev_lock_callback_t callback, + void *user_args); + +/** + * Unregister all callback function on specific port. + * This will be called when a device is detached. + */ +void clean_lock_callback(uint16_t port_id); + +/* Run each callback one by one. */ +int process_lock_callbacks(uint16_t port_id); + +#endif diff --git a/lib/librte_ethdev/ethdev_mp.c b/lib/librte_ethdev/ethdev_mp.c index 87fc430bf..b94bd9501 100644 --- a/lib/librte_ethdev/ethdev_mp.c +++ b/lib/librte_ethdev/ethdev_mp.c @@ -5,6 +5,7 @@ #include #include "rte_ethdev_driver.h" #include "ethdev_mp.h" +#include "ethdev_lock.h" #define MP_TIMEOUT_S 5 /**< 5 seconds timeouts */ @@ -107,7 +108,7 @@ static void __handle_primary_request(void *param) ret = attach_on_secondary(req->devargs, req->port_id); break; case REQ_TYPE_PRE_DETACH: - ret = 0; + ret = process_lock_callbacks(req->port_id); break; case REQ_TYPE_DETACH: case REQ_TYPE_ATTACH_ROLLBACK: diff --git a/lib/librte_ethdev/meson.build b/lib/librte_ethdev/meson.build index b60256855..9bb0aec7f 100644 --- a/lib/librte_ethdev/meson.build +++ b/lib/librte_ethdev/meson.build @@ -6,6 +6,7 @@ version = 9 allow_experimental_apis = true sources = files('ethdev_profile.c', 'ethdev_mp.c' + 'ethdev_lock.c' 'rte_ethdev.c', 'rte_flow.c', 'rte_mtr.c', diff --git a/lib/librte_ethdev/rte_ethdev.c b/lib/librte_ethdev/rte_ethdev.c index 1a5861f30..575dd40ae 100644 --- a/lib/librte_ethdev/rte_ethdev.c +++ b/lib/librte_ethdev/rte_ethdev.c @@ -43,6 +43,7 @@ #include "ethdev_profile.h" #include "ethdev_mp.h" #include "ethdev_private.h" +#include "ethdev_lock.h" int ethdev_logtype; @@ -723,6 +724,7 @@ do_eth_dev_detach(uint16_t port_id) if (ret < 0) return ret; + clean_lock_callback(port_id); rte_eth_dev_release_port(&rte_eth_devices[port_id]); return ret; @@ -789,7 +791,6 @@ rte_eth_dev_attach(const char *devargs, uint16_t *port_id) int rte_eth_dev_attach_private(const char *devargs, uint16_t *port_id) { - if (rte_eal_process_type() == RTE_PROC_PRIMARY) return -ENOTSUP; @@ -831,6 +832,10 @@ rte_eth_dev_detach(uint16_t port_id, char *name __rte_unused) return req.result; } + ret = process_lock_callbacks(port_id); + if (ret) + return ret; + /* check pre_detach */ req.t = REQ_TYPE_PRE_DETACH; req.port_id = port_id; @@ -877,6 +882,7 @@ int rte_eth_dev_detach_private(uint16_t port_id, char *name __rte_unused) { uint32_t dev_flags; + int ret; if (rte_eal_process_type() == RTE_PROC_PRIMARY) return -ENOTSUP; @@ -890,6 +896,10 @@ rte_eth_dev_detach_private(uint16_t port_id, char *name __rte_unused) return -ENOTSUP; } + ret = process_lock_callbacks(port_id); + if (ret) + return ret; + return do_eth_dev_detach(port_id); } @@ -4692,6 +4702,54 @@ rte_eth_devargs_parse(const char *dargs, struct rte_eth_devargs *eth_da) return result; } +static int +dev_is_busy(uint16_t port_id __rte_unused, void *user_args __rte_unused) +{ + return -EBUSY; +} + +int +rte_eth_dev_lock(uint16_t port_id) +{ + RTE_ETH_VALID_PORTID_OR_ERR_RET(port_id, -ENODEV); + + return register_lock_callback(port_id, dev_is_busy, NULL); +} + +int +rte_eth_dev_lock_with_callback(uint16_t port_id, + rte_eth_dev_lock_callback_t callback, + void *user_args) +{ + RTE_ETH_VALID_PORTID_OR_ERR_RET(port_id, -ENODEV); + + if (callback == NULL) + return -EINVAL; + + return register_lock_callback(port_id, callback, user_args); +} + +int +rte_eth_dev_unlock(uint16_t port_id) +{ + RTE_ETH_VALID_PORTID_OR_ERR_RET(port_id, -ENODEV); + + return unregister_lock_callback(port_id, dev_is_busy, NULL); +} + +int +rte_eth_dev_unlock_with_callback(uint16_t port_id, + rte_eth_dev_lock_callback_t callback, + void *user_args) +{ + RTE_ETH_VALID_PORTID_OR_ERR_RET(port_id, -ENODEV); + + if (callback == NULL) + return -EINVAL; + + return unregister_lock_callback(port_id, callback, user_args); +} + RTE_INIT(ethdev_init_log); static void ethdev_init_log(void) diff --git a/lib/librte_ethdev/rte_ethdev.h b/lib/librte_ethdev/rte_ethdev.h index 813806e3c..1596b6e2b 100644 --- a/lib/librte_ethdev/rte_ethdev.h +++ b/lib/librte_ethdev/rte_ethdev.h @@ -4364,6 +4364,130 @@ rte_eth_tx_buffer(uint16_t port_id, uint16_t queue_id, return rte_eth_tx_buffer_flush(port_id, queue_id, buffer); } +/** + * Callback function before device is detached. + * + * This type of function will be added into a function list, and will be + * invoked before device be detached. Application can register a callback + * function so it can be notified and do some cleanup before detach happen. + * Also, any callback function return !0 value will prevent device be + * detached (ref. rte_eth_dev_lock_with_callback and + * rte_eth_dev_unlock_with_callback). + * + * @param port_id + * The port identifier of the Ethernet device. + * @param user_args + * This is parameter "user_args" be saved when callback function is + * registered(rte_dev_eth_lock). + * + * @return + * 0 device is allowed be detached. + * !0 device is not allowed be detached. + */ +typedef int (*rte_eth_dev_lock_callback_t)(uint16_t port_id, void *user_args); + +/** + * @warning + * @b EXPERIMENTAL: this API may change without prior notice. + * + * Lock an Ethernet Device, this help application to prevent a device + * be detached unexpectedly. + * + * @note + * In multi-process situation, any process lock a share device will + * prevent it be detached from all process. Also this is per-process + * lock, which means unlock a device from process A take no effect + * if the device is locked from process B. + * + * @note + * Lock a device multiple times will increase a ref_count, and + * corresponding unlock decrease the ref_count, the device will be + * unlocked when ref_count reach 0. + * + * @param port_id + * The port identifier of the Ethernet device. + * + * @return + * 0 on success, negative on error. + */ +int __rte_experimental rte_eth_dev_lock(uint16_t port_id); + +/** + * @warning + * @b EXPERIMENTAL: this API may change without prior notice. + * + * Lock an Ethernet device base on a callback function which can performs + * condition check at the moment before device be detached. if the + * condition check not pass, the device will not be detached, else, + * continue to detach or not rely on return value of other callbacks + * on the same port. + * + * @note + * Same as rte_eth_dev_lock, it is per-process lock. + * + * @note + * Lock a device with different callback or user_args will add different + * lock entries ( pair) in a list. Lock a device + * multiple times with same callback and args will only increase a + * ref_count of specific lock entry, and corresponding unlock decrease + * the ref_count, an entry will be removed if its ref_count reach 0. + * + * @note + * All callbacks be attached to specific port will be removed + * automatically if the device is detached. + * + * @param port_id + * The port identifier of the Ethernet device. + * @param callback + * the callback function will be added into a pre-detach list, + * it will be invoked when a device is going to be detached. The + * return value will decide if continue detach the device or not. + * @param user_args + * parameter will be parsed to callback function. + * + * @return + * 0 on success, negative on error. + */ +int __rte_experimental +rte_eth_dev_lock_with_callback(uint16_t port_id, + rte_eth_dev_lock_callback_t callback, + void *user_args); + +/** + * @warning + * @b EXPERIMENTAL: this API may change without prior notice. + * + * Reverse operation of rte_eth_dev_lock. + * + * @param port_id + * The port identifier of the Ethernet device. + * + * @return + * 0 on success, negative on error. + */ +int __rte_experimental rte_eth_dev_unlock(uint16_t port_id); + +/** + * @warning + * @b EXPERIMENTAL: this API may change without prior notice. + * + * Reverse operation of rte_eth_dev_lock_with_callback. + * + * @param port_id + * The port identifier of the Ethernet device. + * @param callback + * parameter to match a lock entry. + * @param user_args + * parameter to match a lock entry. + * + * @return + * 0 on success, negative on error. + */ +int __rte_experimental +rte_eth_dev_unlock_with_callback(uint16_t port_id, + rte_eth_dev_lock_callback_t callback, + void *user_args); + #ifdef __cplusplus } #endif From patchwork Tue Jun 26 07:08:16 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Qi Zhang X-Patchwork-Id: 41515 X-Patchwork-Delegate: thomas@monjalon.net Return-Path: X-Original-To: patchwork@dpdk.org Delivered-To: patchwork@dpdk.org Received: from [92.243.14.124] (localhost [127.0.0.1]) by dpdk.org (Postfix) with ESMTP id 3B9F05B38; Tue, 26 Jun 2018 09:08:28 +0200 (CEST) Received: from mga09.intel.com (mga09.intel.com [134.134.136.24]) by dpdk.org (Postfix) with ESMTP id AFCD05398 for ; Tue, 26 Jun 2018 09:08:13 +0200 (CEST) X-Amp-Result: SKIPPED(no attachment in message) X-Amp-File-Uploaded: False Received: from fmsmga006.fm.intel.com ([10.253.24.20]) by orsmga102.jf.intel.com with ESMTP/TLS/DHE-RSA-AES256-GCM-SHA384; 26 Jun 2018 00:08:13 -0700 X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="5.51,273,1526367600"; d="scan'208";a="240660004" Received: from dpdk51.sh.intel.com ([10.67.110.190]) by fmsmga006.fm.intel.com with ESMTP; 26 Jun 2018 00:08:11 -0700 From: Qi Zhang To: thomas@monjalon.net, anatoly.burakov@intel.com Cc: konstantin.ananyev@intel.com, dev@dpdk.org, bruce.richardson@intel.com, ferruh.yigit@intel.com, benjamin.h.shelton@intel.com, narender.vangati@intel.com, Qi Zhang Date: Tue, 26 Jun 2018 15:08:16 +0800 Message-Id: <20180626070832.3055-9-qi.z.zhang@intel.com> X-Mailer: git-send-email 2.13.6 In-Reply-To: <20180626070832.3055-1-qi.z.zhang@intel.com> References: <20180607123849.14439-1-qi.z.zhang@intel.com> <20180626070832.3055-1-qi.z.zhang@intel.com> Subject: [dpdk-dev] [PATCH v4 08/24] ethdev: support attach or detach share device from secondary X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Sender: "dev" This patch cover the multi-process hotplug case when a share device attach/detach request be issued from secondary process device attach on secondary: a) seconary send sync request to primary. b) primary receive the request and attach the new device if failed goto i). c) primary forward attach sync request to all secondary. d) secondary receive request and attach device and send reply. e) primary check the reply if all success go to j). f) primary send attach rollback sync request to all secondary. g) secondary receive the request and detach device and send reply. h) primary receive the reply and detach device as rollback action. i) send fail reply to secondary, goto k). j) send success reply to secondary. k) secondary process receive reply of step a) and return. device detach on secondary: a) secondary send sync request to primary b) primary receive the request and perform pre-detach check, if device is locked, goto j). c) primary send pre-detach sync request to all secondary. d) secondary perform pre-detach check and send reply. e) primary check the reply if any fail goto j). f) primary send detach sync request to all secondary g) secondary detach the device and send reply h) primary detach the device. i) send success reply to secondary, goto k). j) send fail reply to secondary. k) secondary process receive reply of step a) and return. Signed-off-by: Qi Zhang --- lib/librte_ethdev/ethdev_mp.c | 155 ++++++++++++++++++++++++++++++++++++++++-- 1 file changed, 149 insertions(+), 6 deletions(-) diff --git a/lib/librte_ethdev/ethdev_mp.c b/lib/librte_ethdev/ethdev_mp.c index b94bd9501..7afbd4cf2 100644 --- a/lib/librte_ethdev/ethdev_mp.c +++ b/lib/librte_ethdev/ethdev_mp.c @@ -4,8 +4,44 @@ #include #include "rte_ethdev_driver.h" + #include "ethdev_mp.h" #include "ethdev_lock.h" +#include "ethdev_private.h" + +/** + * + * secondary to primary request. + * start from function eth_dev_request_to_primary. + * + * device attach on secondary: + * a) seconary send sycn request to primary + * b) primary receive the request and attach the new device thread, + * if failed goto i). + * c) primary forward attach request to all secondary as sync request + * d) secondary receive request and attach device and send reply. + * e) primary check the reply if all success go to j). + * f) primary send attach rollback sync request to all secondary. + * g) secondary receive the request and detach device and send reply. + * h) primary receive the reply and detach device as rollback action. + * i) send fail sync reply to secondary, goto k). + * j) send success sync reply to secondary. + * k) secondary process receive reply of step a) and return. + * + * device detach on secondary: + * a) secondary send detach sync request to primary + * b) primary receive the request and perform pre-detach check, if device + * is locked, goto j). + * c) primary send pre-detach sync request to all secondary. + * d) secondary perform pre-detach check and send reply. + * e) primary check the reply if any fail goto j). + * f) primary send detach sync request to all secondary + * g) secondary detach the device and send reply + * h) primary detach the device. + * i) send success sync reply to secondary, goto k). + * j) send fail sync reply to secondary. + * k) secondary process receive reply of step a) and return. + */ #define MP_TIMEOUT_S 5 /**< 5 seconds timeouts */ @@ -83,11 +119,98 @@ static int attach_on_secondary(const char *devargs, uint16_t port_id) } static int -handle_secondary_request(const struct rte_mp_msg *msg, const void *peer) +send_response_to_secondary(const struct eth_dev_mp_req *req, + int result, + const void *peer) +{ + struct rte_mp_msg mp_resp; + struct eth_dev_mp_req *resp = + (struct eth_dev_mp_req *)mp_resp.param; + int ret; + + memset(&mp_resp, 0, sizeof(mp_resp)); + mp_resp.len_param = sizeof(*resp); + strcpy(mp_resp.name, ETH_DEV_MP_ACTION_REQUEST); + memcpy(resp, req, sizeof(*req)); + resp->result = result; + + ret = rte_mp_reply(&mp_resp, peer); + if (ret) + ethdev_log(ERR, "failed to send response to secondary\n"); + + return ret; +} + +int eth_dev_request_to_secondary(struct eth_dev_mp_req *req); + +static void +__handle_secondary_request(void *param) { - RTE_SET_USED(msg); - RTE_SET_USED(peer); - return -ENOTSUP; + struct mp_reply_bundle *bundle = param; + const struct rte_mp_msg *msg = &bundle->msg; + const struct eth_dev_mp_req *req = + (const struct eth_dev_mp_req *)msg->param; + struct eth_dev_mp_req tmp_req; + uint16_t port_id; + int ret = 0; + + tmp_req = *req; + + if (req->t == REQ_TYPE_ATTACH) { + ret = do_eth_dev_attach(req->devargs, &port_id); + if (!ret) { + tmp_req.port_id = port_id; + ret = eth_dev_request_to_secondary(&tmp_req); + } + } else if (req->t == REQ_TYPE_DETACH) { + if (!rte_eth_dev_is_valid_port(req->port_id)) + ret = -EINVAL; + if (!ret) + ret = process_lock_callbacks(req->port_id); + if (!ret) { + tmp_req.t = REQ_TYPE_PRE_DETACH; + ret = eth_dev_request_to_secondary(&tmp_req); + } + if (!ret && !tmp_req.result) { + tmp_req.t = REQ_TYPE_DETACH; + ret = eth_dev_request_to_secondary(&tmp_req); + } + if (!ret) + ret = do_eth_dev_detach(req->port_id); + } else { + ethdev_log(ERR, "unsupported secondary to primary request\n"); + ret = -ENOTSUP; + } + + ret = send_response_to_secondary(&tmp_req, ret, bundle->peer); + if (ret) + ethdev_log(ERR, "failed to send response to secondary\n"); +} + +static int +handle_secondary_request(const struct rte_mp_msg *msg, + const void *peer) +{ + struct mp_reply_bundle *bundle; + const struct eth_dev_mp_req *req = + (const struct eth_dev_mp_req *)msg->param; + int ret = 0; + + bundle = malloc(sizeof(*bundle)); + if (bundle == NULL) { + ethdev_log(ERR, "not enough memory\n"); + return send_response_to_secondary(req, -ENOMEM, peer); + } + + bundle->msg = *msg; + bundle->peer = peer; + + ret = rte_eal_mp_task_add(__handle_secondary_request, bundle); + if (ret) { + ethdev_log(ERR, "failed to add mp task\n"); + return send_response_to_secondary(req, ret, peer); + } + return 0; } static void __handle_primary_request(void *param) @@ -170,8 +293,28 @@ handle_primary_request(const struct rte_mp_msg *msg, const void *peer) int eth_dev_request_to_primary(struct eth_dev_mp_req *req) { - RTE_SET_USED(req); - return -ENOTSUP; + struct rte_mp_msg mp_req; + struct rte_mp_reply mp_reply; + struct timespec ts = {.tv_sec = MP_TIMEOUT_S, .tv_nsec = 0}; + struct eth_dev_mp_req *resp; + int ret; + + memset(&mp_req, 0, sizeof(mp_req)); + memcpy(mp_req.param, req, sizeof(*req)); + mp_req.len_param = sizeof(*req); + strlcpy(mp_req.name, ETH_DEV_MP_ACTION_REQUEST, sizeof(mp_req.name)); + + ret = rte_mp_request_sync(&mp_req, &mp_reply, &ts); + if (ret) { + ethdev_log(ERR, "cannot send request to primary"); + return ret; + } + + resp = (struct eth_dev_mp_req *)mp_reply.msgs[0].param; + req->result = resp->result; + req->port_id = resp->port_id; + + return ret; } /** From patchwork Tue Jun 26 07:08:17 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Qi Zhang X-Patchwork-Id: 41516 X-Patchwork-Delegate: thomas@monjalon.net Return-Path: X-Original-To: patchwork@dpdk.org Delivered-To: patchwork@dpdk.org Received: from [92.243.14.124] (localhost [127.0.0.1]) by dpdk.org (Postfix) with ESMTP id 7B0DA5F11; Tue, 26 Jun 2018 09:08:30 +0200 (CEST) Received: from mga09.intel.com (mga09.intel.com [134.134.136.24]) by dpdk.org (Postfix) with ESMTP id 5716458C4 for ; Tue, 26 Jun 2018 09:08:15 +0200 (CEST) X-Amp-Result: SKIPPED(no attachment in message) X-Amp-File-Uploaded: False Received: from fmsmga006.fm.intel.com ([10.253.24.20]) by orsmga102.jf.intel.com with ESMTP/TLS/DHE-RSA-AES256-GCM-SHA384; 26 Jun 2018 00:08:15 -0700 X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="5.51,273,1526367600"; d="scan'208";a="240660011" Received: from dpdk51.sh.intel.com ([10.67.110.190]) by fmsmga006.fm.intel.com with ESMTP; 26 Jun 2018 00:08:13 -0700 From: Qi Zhang To: thomas@monjalon.net, anatoly.burakov@intel.com Cc: konstantin.ananyev@intel.com, dev@dpdk.org, bruce.richardson@intel.com, ferruh.yigit@intel.com, benjamin.h.shelton@intel.com, narender.vangati@intel.com, Qi Zhang Date: Tue, 26 Jun 2018 15:08:17 +0800 Message-Id: <20180626070832.3055-10-qi.z.zhang@intel.com> X-Mailer: git-send-email 2.13.6 In-Reply-To: <20180626070832.3055-1-qi.z.zhang@intel.com> References: <20180607123849.14439-1-qi.z.zhang@intel.com> <20180626070832.3055-1-qi.z.zhang@intel.com> Subject: [dpdk-dev] [PATCH v4 09/24] net/i40e: enable port detach on secondary process X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Sender: "dev" Previously, detach port on secondary process will mess primary process and cause same device can't be attached again, by take advantage of rte_eth_release_port_private, we can support this with minor change. Signed-off-by: Qi Zhang --- drivers/net/i40e/i40e_ethdev.c | 2 ++ drivers/net/i40e/i40e_ethdev_vf.c | 9 +++++++++ 2 files changed, 11 insertions(+) diff --git a/drivers/net/i40e/i40e_ethdev.c b/drivers/net/i40e/i40e_ethdev.c index 13c5d3296..7d1f98422 100644 --- a/drivers/net/i40e/i40e_ethdev.c +++ b/drivers/net/i40e/i40e_ethdev.c @@ -678,6 +678,8 @@ static int eth_i40e_pci_remove(struct rte_pci_device *pci_dev) if (!ethdev) return -ENODEV; + if (rte_eal_process_type() != RTE_PROC_PRIMARY) + return rte_eth_dev_release_port_private(ethdev); if (ethdev->data->dev_flags & RTE_ETH_DEV_REPRESENTOR) return rte_eth_dev_destroy(ethdev, i40e_vf_representor_uninit); diff --git a/drivers/net/i40e/i40e_ethdev_vf.c b/drivers/net/i40e/i40e_ethdev_vf.c index 804e44530..fc6f079d5 100644 --- a/drivers/net/i40e/i40e_ethdev_vf.c +++ b/drivers/net/i40e/i40e_ethdev_vf.c @@ -1500,6 +1500,15 @@ static int eth_i40evf_pci_probe(struct rte_pci_driver *pci_drv __rte_unused, static int eth_i40evf_pci_remove(struct rte_pci_device *pci_dev) { + struct rte_eth_dev *ethdev; + ethdev = rte_eth_dev_allocated(pci_dev->device.name); + + if (!ethdev) + return -ENODEV; + + if (rte_eal_process_type() != RTE_PROC_PRIMARY) + return rte_eth_dev_release_port_private(ethdev); + return rte_eth_dev_pci_generic_remove(pci_dev, i40evf_dev_uninit); } From patchwork Tue Jun 26 07:08:18 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Qi Zhang X-Patchwork-Id: 41517 X-Patchwork-Delegate: thomas@monjalon.net Return-Path: X-Original-To: patchwork@dpdk.org Delivered-To: patchwork@dpdk.org Received: from [92.243.14.124] (localhost [127.0.0.1]) by dpdk.org (Postfix) with ESMTP id 97ADA5F21; Tue, 26 Jun 2018 09:08:32 +0200 (CEST) Received: from mga09.intel.com (mga09.intel.com [134.134.136.24]) by dpdk.org (Postfix) with ESMTP id AECCD559A for ; Tue, 26 Jun 2018 09:08:16 +0200 (CEST) X-Amp-Result: SKIPPED(no attachment in message) X-Amp-File-Uploaded: False Received: from fmsmga006.fm.intel.com ([10.253.24.20]) by orsmga102.jf.intel.com with ESMTP/TLS/DHE-RSA-AES256-GCM-SHA384; 26 Jun 2018 00:08:16 -0700 X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="5.51,273,1526367600"; d="scan'208";a="240660018" Received: from dpdk51.sh.intel.com ([10.67.110.190]) by fmsmga006.fm.intel.com with ESMTP; 26 Jun 2018 00:08:14 -0700 From: Qi Zhang To: thomas@monjalon.net, anatoly.burakov@intel.com Cc: konstantin.ananyev@intel.com, dev@dpdk.org, bruce.richardson@intel.com, ferruh.yigit@intel.com, benjamin.h.shelton@intel.com, narender.vangati@intel.com, Qi Zhang Date: Tue, 26 Jun 2018 15:08:18 +0800 Message-Id: <20180626070832.3055-11-qi.z.zhang@intel.com> X-Mailer: git-send-email 2.13.6 In-Reply-To: <20180626070832.3055-1-qi.z.zhang@intel.com> References: <20180607123849.14439-1-qi.z.zhang@intel.com> <20180626070832.3055-1-qi.z.zhang@intel.com> Subject: [dpdk-dev] [PATCH v4 10/24] net/ixgbe: enable port detach on secondary process X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Sender: "dev" Previously, detach port on secondary process will mess primary process and cause same device can't be attached again, by take advantage of rte_eth_release_port_private, we can support this with minor change. Signed-off-by: Qi Zhang --- drivers/net/ixgbe/ixgbe_ethdev.c | 12 ++++++++++++ 1 file changed, 12 insertions(+) diff --git a/drivers/net/ixgbe/ixgbe_ethdev.c b/drivers/net/ixgbe/ixgbe_ethdev.c index 87d2ad090..f9d560835 100644 --- a/drivers/net/ixgbe/ixgbe_ethdev.c +++ b/drivers/net/ixgbe/ixgbe_ethdev.c @@ -1792,6 +1792,9 @@ static int eth_ixgbe_pci_remove(struct rte_pci_device *pci_dev) if (!ethdev) return -ENODEV; + if (rte_eal_process_type() != RTE_PROC_PRIMARY) + return rte_eth_dev_release_port_private(ethdev); + if (ethdev->data->dev_flags & RTE_ETH_DEV_REPRESENTOR) return rte_eth_dev_destroy(ethdev, ixgbe_vf_representor_uninit); else @@ -1809,6 +1812,15 @@ static struct rte_pci_driver rte_ixgbe_pmd = { static int eth_ixgbevf_pci_probe(struct rte_pci_driver *pci_drv __rte_unused, struct rte_pci_device *pci_dev) { + struct rte_eth_dev *ethdev; + + ethdev = rte_eth_dev_allocated(pci_dev->device.name); + if (!ethdev) + return -ENODEV; + + if (rte_eal_process_type() != RTE_PROC_PRIMARY) + return rte_eth_dev_release_port_private(ethdev); + return rte_eth_dev_pci_generic_probe(pci_dev, sizeof(struct ixgbe_adapter), eth_ixgbevf_dev_init); } From patchwork Tue Jun 26 07:08:19 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Qi Zhang X-Patchwork-Id: 41518 X-Patchwork-Delegate: thomas@monjalon.net Return-Path: X-Original-To: patchwork@dpdk.org Delivered-To: patchwork@dpdk.org Received: from [92.243.14.124] (localhost [127.0.0.1]) by dpdk.org (Postfix) with ESMTP id A002E5F29; Tue, 26 Jun 2018 09:08:35 +0200 (CEST) Received: from mga09.intel.com (mga09.intel.com [134.134.136.24]) by dpdk.org (Postfix) with ESMTP id 7E9F4559A for ; Tue, 26 Jun 2018 09:08:18 +0200 (CEST) X-Amp-Result: SKIPPED(no attachment in message) X-Amp-File-Uploaded: False Received: from fmsmga006.fm.intel.com ([10.253.24.20]) by orsmga102.jf.intel.com with ESMTP/TLS/DHE-RSA-AES256-GCM-SHA384; 26 Jun 2018 00:08:18 -0700 X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="5.51,273,1526367600"; d="scan'208";a="240660024" Received: from dpdk51.sh.intel.com ([10.67.110.190]) by fmsmga006.fm.intel.com with ESMTP; 26 Jun 2018 00:08:16 -0700 From: Qi Zhang To: thomas@monjalon.net, anatoly.burakov@intel.com Cc: konstantin.ananyev@intel.com, dev@dpdk.org, bruce.richardson@intel.com, ferruh.yigit@intel.com, benjamin.h.shelton@intel.com, narender.vangati@intel.com, Qi Zhang Date: Tue, 26 Jun 2018 15:08:19 +0800 Message-Id: <20180626070832.3055-12-qi.z.zhang@intel.com> X-Mailer: git-send-email 2.13.6 In-Reply-To: <20180626070832.3055-1-qi.z.zhang@intel.com> References: <20180607123849.14439-1-qi.z.zhang@intel.com> <20180626070832.3055-1-qi.z.zhang@intel.com> Subject: [dpdk-dev] [PATCH v4 11/24] net/e1000: enable port detach on secondary process X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Sender: "dev" Previously, detach port on secondary process will mess primary process and cause same device can't be attached again, by take advantage of rte_eth_release_port_private, we can support this with minor change. Signed-off-by: Qi Zhang --- drivers/net/e1000/em_ethdev.c | 9 +++++++++ 1 file changed, 9 insertions(+) diff --git a/drivers/net/e1000/em_ethdev.c b/drivers/net/e1000/em_ethdev.c index 7039dc100..e6b7ce63a 100644 --- a/drivers/net/e1000/em_ethdev.c +++ b/drivers/net/e1000/em_ethdev.c @@ -349,6 +349,15 @@ static int eth_em_pci_probe(struct rte_pci_driver *pci_drv __rte_unused, static int eth_em_pci_remove(struct rte_pci_device *pci_dev) { + struct rte_eth_dev *ethdev = + rte_eth_dev_allocated(pci_dev->device.name); + + if (!ethdev) + return -ENODEV; + + if (rte_eal_process_type() != RTE_PROC_PRIMARY) + return rte_eth_dev_release_port_private(ethdev); + return rte_eth_dev_pci_generic_remove(pci_dev, eth_em_dev_uninit); } From patchwork Tue Jun 26 07:08:20 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Qi Zhang X-Patchwork-Id: 41519 X-Patchwork-Delegate: thomas@monjalon.net Return-Path: X-Original-To: patchwork@dpdk.org Delivered-To: patchwork@dpdk.org Received: from [92.243.14.124] (localhost [127.0.0.1]) by dpdk.org (Postfix) with ESMTP id 6A3955B1E; Tue, 26 Jun 2018 09:08:46 +0200 (CEST) Received: from mga09.intel.com (mga09.intel.com [134.134.136.24]) by dpdk.org (Postfix) with ESMTP id F1C4B559A for ; Tue, 26 Jun 2018 09:08:19 +0200 (CEST) X-Amp-Result: SKIPPED(no attachment in message) X-Amp-File-Uploaded: False Received: from fmsmga006.fm.intel.com ([10.253.24.20]) by orsmga102.jf.intel.com with ESMTP/TLS/DHE-RSA-AES256-GCM-SHA384; 26 Jun 2018 00:08:20 -0700 X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="5.51,273,1526367600"; d="scan'208";a="240660030" Received: from dpdk51.sh.intel.com ([10.67.110.190]) by fmsmga006.fm.intel.com with ESMTP; 26 Jun 2018 00:08:18 -0700 From: Qi Zhang To: thomas@monjalon.net, anatoly.burakov@intel.com Cc: konstantin.ananyev@intel.com, dev@dpdk.org, bruce.richardson@intel.com, ferruh.yigit@intel.com, benjamin.h.shelton@intel.com, narender.vangati@intel.com, Qi Zhang Date: Tue, 26 Jun 2018 15:08:20 +0800 Message-Id: <20180626070832.3055-13-qi.z.zhang@intel.com> X-Mailer: git-send-email 2.13.6 In-Reply-To: <20180626070832.3055-1-qi.z.zhang@intel.com> References: <20180607123849.14439-1-qi.z.zhang@intel.com> <20180626070832.3055-1-qi.z.zhang@intel.com> Subject: [dpdk-dev] [PATCH v4 12/24] net/igb: enable port detach on secondary process X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Sender: "dev" Previously, detach port on secondary process will mess primary process and cause same device can't be attached again, by take advantage of rte_eth_release_port_private, we can support this with minor change. Signed-off-by: Qi Zhang --- drivers/net/e1000/igb_ethdev.c | 9 +++++++++ 1 file changed, 9 insertions(+) diff --git a/drivers/net/e1000/igb_ethdev.c b/drivers/net/e1000/igb_ethdev.c index edc7be319..db07a83e3 100644 --- a/drivers/net/e1000/igb_ethdev.c +++ b/drivers/net/e1000/igb_ethdev.c @@ -1089,6 +1089,15 @@ static int eth_igb_pci_probe(struct rte_pci_driver *pci_drv __rte_unused, static int eth_igb_pci_remove(struct rte_pci_device *pci_dev) { + struct rte_eth_dev *ethdev = + rte_eth_dev_allocated(pci_dev->device.name); + + if (!ethdev) + return -ENODEV; + + if (rte_eal_process_type() != RTE_PROC_PRIMARY) + return rte_eth_dev_release_port_private(ethdev); + return rte_eth_dev_pci_generic_remove(pci_dev, eth_igb_dev_uninit); } From patchwork Tue Jun 26 07:08:21 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Qi Zhang X-Patchwork-Id: 41520 X-Patchwork-Delegate: thomas@monjalon.net Return-Path: X-Original-To: patchwork@dpdk.org Delivered-To: patchwork@dpdk.org Received: from [92.243.14.124] (localhost [127.0.0.1]) by dpdk.org (Postfix) with ESMTP id 6B5665F33; Tue, 26 Jun 2018 09:08:48 +0200 (CEST) Received: from mga09.intel.com (mga09.intel.com [134.134.136.24]) by dpdk.org (Postfix) with ESMTP id 9575A5920 for ; Tue, 26 Jun 2018 09:08:21 +0200 (CEST) X-Amp-Result: SKIPPED(no attachment in message) X-Amp-File-Uploaded: False Received: from fmsmga006.fm.intel.com ([10.253.24.20]) by orsmga102.jf.intel.com with ESMTP/TLS/DHE-RSA-AES256-GCM-SHA384; 26 Jun 2018 00:08:21 -0700 X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="5.51,273,1526367600"; d="scan'208";a="240660035" Received: from dpdk51.sh.intel.com ([10.67.110.190]) by fmsmga006.fm.intel.com with ESMTP; 26 Jun 2018 00:08:19 -0700 From: Qi Zhang To: thomas@monjalon.net, anatoly.burakov@intel.com Cc: konstantin.ananyev@intel.com, dev@dpdk.org, bruce.richardson@intel.com, ferruh.yigit@intel.com, benjamin.h.shelton@intel.com, narender.vangati@intel.com, Qi Zhang Date: Tue, 26 Jun 2018 15:08:21 +0800 Message-Id: <20180626070832.3055-14-qi.z.zhang@intel.com> X-Mailer: git-send-email 2.13.6 In-Reply-To: <20180626070832.3055-1-qi.z.zhang@intel.com> References: <20180607123849.14439-1-qi.z.zhang@intel.com> <20180626070832.3055-1-qi.z.zhang@intel.com> Subject: [dpdk-dev] [PATCH v4 13/24] net/fm10k: enable port detach on secondary process X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Sender: "dev" Previously, detach port on secondary process will mess primary process and cause same device can't be attached again, by take advantage of rte_eth_release_port_private, we can support this with minor change. Signed-off-by: Qi Zhang --- drivers/net/fm10k/fm10k_ethdev.c | 9 +++++++++ 1 file changed, 9 insertions(+) diff --git a/drivers/net/fm10k/fm10k_ethdev.c b/drivers/net/fm10k/fm10k_ethdev.c index 3ff1b0e0f..f73301182 100644 --- a/drivers/net/fm10k/fm10k_ethdev.c +++ b/drivers/net/fm10k/fm10k_ethdev.c @@ -3264,6 +3264,15 @@ static int eth_fm10k_pci_probe(struct rte_pci_driver *pci_drv __rte_unused, static int eth_fm10k_pci_remove(struct rte_pci_device *pci_dev) { + struct rte_eth_dev *ethdev = + rte_eth_dev_allocated(pci_dev->device.name); + + if (!ethdev) + return -ENODEV; + + if (rte_eal_process_type() != RTE_PROC_PRIMARY) + return rte_eth_dev_release_port_private(ethdev); + return rte_eth_dev_pci_generic_remove(pci_dev, eth_fm10k_dev_uninit); } From patchwork Tue Jun 26 07:08:22 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Qi Zhang X-Patchwork-Id: 41521 X-Patchwork-Delegate: thomas@monjalon.net Return-Path: X-Original-To: patchwork@dpdk.org Delivered-To: patchwork@dpdk.org Received: from [92.243.14.124] (localhost [127.0.0.1]) by dpdk.org (Postfix) with ESMTP id 596255F44; Tue, 26 Jun 2018 09:08:50 +0200 (CEST) Received: from mga09.intel.com (mga09.intel.com [134.134.136.24]) by dpdk.org (Postfix) with ESMTP id 1B3035B12 for ; Tue, 26 Jun 2018 09:08:22 +0200 (CEST) X-Amp-Result: SKIPPED(no attachment in message) X-Amp-File-Uploaded: False Received: from fmsmga006.fm.intel.com ([10.253.24.20]) by orsmga102.jf.intel.com with ESMTP/TLS/DHE-RSA-AES256-GCM-SHA384; 26 Jun 2018 00:08:23 -0700 X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="5.51,273,1526367600"; d="scan'208";a="240660041" Received: from dpdk51.sh.intel.com ([10.67.110.190]) by fmsmga006.fm.intel.com with ESMTP; 26 Jun 2018 00:08:21 -0700 From: Qi Zhang To: thomas@monjalon.net, anatoly.burakov@intel.com Cc: konstantin.ananyev@intel.com, dev@dpdk.org, bruce.richardson@intel.com, ferruh.yigit@intel.com, benjamin.h.shelton@intel.com, narender.vangati@intel.com, Qi Zhang Date: Tue, 26 Jun 2018 15:08:22 +0800 Message-Id: <20180626070832.3055-15-qi.z.zhang@intel.com> X-Mailer: git-send-email 2.13.6 In-Reply-To: <20180626070832.3055-1-qi.z.zhang@intel.com> References: <20180607123849.14439-1-qi.z.zhang@intel.com> <20180626070832.3055-1-qi.z.zhang@intel.com> Subject: [dpdk-dev] [PATCH v4 14/24] net/af_packet: enable port detach on secondary process X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Sender: "dev" Previously, detach port on secondary process will mess primary process and cause same device can't be attached again, by take advantage of rte_eth_release_port_private, we can support this with minor change. Signed-off-by: Qi Zhang --- drivers/net/af_packet/rte_eth_af_packet.c | 11 +++++++++++ 1 file changed, 11 insertions(+) diff --git a/drivers/net/af_packet/rte_eth_af_packet.c b/drivers/net/af_packet/rte_eth_af_packet.c index ea47abbf8..33ac19de8 100644 --- a/drivers/net/af_packet/rte_eth_af_packet.c +++ b/drivers/net/af_packet/rte_eth_af_packet.c @@ -935,6 +935,7 @@ rte_pmd_af_packet_probe(struct rte_vdev_device *dev) } /* TODO: request info from primary to set up Rx and Tx */ eth_dev->dev_ops = &ops; + eth_dev->device = &dev->device; rte_eth_dev_probing_finish(eth_dev); return 0; } @@ -986,6 +987,16 @@ rte_pmd_af_packet_remove(struct rte_vdev_device *dev) if (eth_dev == NULL) return -1; + if (rte_eal_process_type() != RTE_PROC_PRIMARY) { + /* detach device on local pprocess only */ + if (strlen(rte_vdev_device_args(dev)) == 0) + return rte_eth_dev_release_port_private(eth_dev); + /** + * else this is a private device for current process + * so continue with normal detach scenario + */ + } + internals = eth_dev->data->dev_private; for (q = 0; q < internals->nb_queues; q++) { rte_free(internals->rx_queue[q].rd); From patchwork Tue Jun 26 07:08:23 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Qi Zhang X-Patchwork-Id: 41522 X-Patchwork-Delegate: thomas@monjalon.net Return-Path: X-Original-To: patchwork@dpdk.org Delivered-To: patchwork@dpdk.org Received: from [92.243.14.124] (localhost [127.0.0.1]) by dpdk.org (Postfix) with ESMTP id 6CFF65F54; Tue, 26 Jun 2018 09:08:52 +0200 (CEST) Received: from mga09.intel.com (mga09.intel.com [134.134.136.24]) by dpdk.org (Postfix) with ESMTP id 9AA01568A for ; Tue, 26 Jun 2018 09:08:24 +0200 (CEST) X-Amp-Result: SKIPPED(no attachment in message) X-Amp-File-Uploaded: False Received: from fmsmga006.fm.intel.com ([10.253.24.20]) by orsmga102.jf.intel.com with ESMTP/TLS/DHE-RSA-AES256-GCM-SHA384; 26 Jun 2018 00:08:24 -0700 X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="5.51,273,1526367600"; d="scan'208";a="240660046" Received: from dpdk51.sh.intel.com ([10.67.110.190]) by fmsmga006.fm.intel.com with ESMTP; 26 Jun 2018 00:08:22 -0700 From: Qi Zhang To: thomas@monjalon.net, anatoly.burakov@intel.com Cc: konstantin.ananyev@intel.com, dev@dpdk.org, bruce.richardson@intel.com, ferruh.yigit@intel.com, benjamin.h.shelton@intel.com, narender.vangati@intel.com, Qi Zhang Date: Tue, 26 Jun 2018 15:08:23 +0800 Message-Id: <20180626070832.3055-16-qi.z.zhang@intel.com> X-Mailer: git-send-email 2.13.6 In-Reply-To: <20180626070832.3055-1-qi.z.zhang@intel.com> References: <20180607123849.14439-1-qi.z.zhang@intel.com> <20180626070832.3055-1-qi.z.zhang@intel.com> Subject: [dpdk-dev] [PATCH v4 15/24] net/bonding: enable port detach on secondary process X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Sender: "dev" Previously, detach port on secondary process will mess primary process and cause same device can't be attached again, by take advantage of rte_eth_release_port_private, we can support this with minor change. Signed-off-by: Qi Zhang --- drivers/net/bonding/rte_eth_bond_pmd.c | 11 +++++++++++ 1 file changed, 11 insertions(+) diff --git a/drivers/net/bonding/rte_eth_bond_pmd.c b/drivers/net/bonding/rte_eth_bond_pmd.c index f155ff779..da45ba9ba 100644 --- a/drivers/net/bonding/rte_eth_bond_pmd.c +++ b/drivers/net/bonding/rte_eth_bond_pmd.c @@ -3062,6 +3062,7 @@ bond_probe(struct rte_vdev_device *dev) } /* TODO: request info from primary to set up Rx and Tx */ eth_dev->dev_ops = &default_dev_ops; + eth_dev->device = &dev->device; rte_eth_dev_probing_finish(eth_dev); return 0; } @@ -3168,6 +3169,16 @@ bond_remove(struct rte_vdev_device *dev) if (eth_dev == NULL) return -ENODEV; + if (rte_eal_process_type() != RTE_PROC_PRIMARY) { + /* detach device on local pprocess only */ + if (strlen(rte_vdev_device_args(dev)) == 0) + return rte_eth_dev_release_port_private(eth_dev); + /** + * else this is a private device for current process + * so continue with normal detach scenario + */ + } + RTE_ASSERT(eth_dev->device == &dev->device); internals = eth_dev->data->dev_private; From patchwork Tue Jun 26 07:08:24 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Qi Zhang X-Patchwork-Id: 41523 X-Patchwork-Delegate: thomas@monjalon.net Return-Path: X-Original-To: patchwork@dpdk.org Delivered-To: patchwork@dpdk.org Received: from [92.243.14.124] (localhost [127.0.0.1]) by dpdk.org (Postfix) with ESMTP id 331BB6CC3; Tue, 26 Jun 2018 09:08:54 +0200 (CEST) Received: from mga09.intel.com (mga09.intel.com [134.134.136.24]) by dpdk.org (Postfix) with ESMTP id 239A75B32 for ; Tue, 26 Jun 2018 09:08:25 +0200 (CEST) X-Amp-Result: SKIPPED(no attachment in message) X-Amp-File-Uploaded: False Received: from fmsmga006.fm.intel.com ([10.253.24.20]) by orsmga102.jf.intel.com with ESMTP/TLS/DHE-RSA-AES256-GCM-SHA384; 26 Jun 2018 00:08:26 -0700 X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="5.51,273,1526367600"; d="scan'208";a="240660050" Received: from dpdk51.sh.intel.com ([10.67.110.190]) by fmsmga006.fm.intel.com with ESMTP; 26 Jun 2018 00:08:24 -0700 From: Qi Zhang To: thomas@monjalon.net, anatoly.burakov@intel.com Cc: konstantin.ananyev@intel.com, dev@dpdk.org, bruce.richardson@intel.com, ferruh.yigit@intel.com, benjamin.h.shelton@intel.com, narender.vangati@intel.com, Qi Zhang Date: Tue, 26 Jun 2018 15:08:24 +0800 Message-Id: <20180626070832.3055-17-qi.z.zhang@intel.com> X-Mailer: git-send-email 2.13.6 In-Reply-To: <20180626070832.3055-1-qi.z.zhang@intel.com> References: <20180607123849.14439-1-qi.z.zhang@intel.com> <20180626070832.3055-1-qi.z.zhang@intel.com> Subject: [dpdk-dev] [PATCH v4 16/24] net/failsafe: enable port detach on secondary process X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Sender: "dev" Previously, detach port on secondary process will mess primary process and cause same device can't be attached again, by take advantage of rte_eth_release_port_private, we can support this with minor change. Signed-off-by: Qi Zhang --- drivers/net/failsafe/failsafe.c | 16 ++++++++++++++++ 1 file changed, 16 insertions(+) diff --git a/drivers/net/failsafe/failsafe.c b/drivers/net/failsafe/failsafe.c index eafbb75df..c5e8651f6 100644 --- a/drivers/net/failsafe/failsafe.c +++ b/drivers/net/failsafe/failsafe.c @@ -328,6 +328,7 @@ rte_pmd_failsafe_probe(struct rte_vdev_device *vdev) } /* TODO: request info from primary to set up Rx and Tx */ eth_dev->dev_ops = &failsafe_ops; + eth_dev->device = &vdev->device; rte_eth_dev_probing_finish(eth_dev); return 0; } @@ -338,10 +339,25 @@ rte_pmd_failsafe_probe(struct rte_vdev_device *vdev) static int rte_pmd_failsafe_remove(struct rte_vdev_device *vdev) { + struct rte_eth_dev *eth_dev; const char *name; name = rte_vdev_device_name(vdev); INFO("Uninitializing " FAILSAFE_DRIVER_NAME " for %s", name); + + eth_dev = rte_eth_dev_allocated(name); + if (!eth_dev) + return -ENODEV; + if (rte_eal_process_type() != RTE_PROC_PRIMARY) { + /* detach device on local pprocess only */ + if (strlen(rte_vdev_device_args(vdev)) == 0) + return rte_eth_dev_release_port_private(eth_dev); + /** + * else this is a private device for current process + * so continue with normal detach scenario. + */ + } + return fs_rte_eth_free(name); } From patchwork Tue Jun 26 07:08:25 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Qi Zhang X-Patchwork-Id: 41524 X-Patchwork-Delegate: thomas@monjalon.net Return-Path: X-Original-To: patchwork@dpdk.org Delivered-To: patchwork@dpdk.org Received: from [92.243.14.124] (localhost [127.0.0.1]) by dpdk.org (Postfix) with ESMTP id 172A57CCA; Tue, 26 Jun 2018 09:08:56 +0200 (CEST) Received: from mga09.intel.com (mga09.intel.com [134.134.136.24]) by dpdk.org (Postfix) with ESMTP id BE0E35B32 for ; Tue, 26 Jun 2018 09:08:27 +0200 (CEST) X-Amp-Result: SKIPPED(no attachment in message) X-Amp-File-Uploaded: False Received: from fmsmga006.fm.intel.com ([10.253.24.20]) by orsmga102.jf.intel.com with ESMTP/TLS/DHE-RSA-AES256-GCM-SHA384; 26 Jun 2018 00:08:27 -0700 X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="5.51,273,1526367600"; d="scan'208";a="240660055" Received: from dpdk51.sh.intel.com ([10.67.110.190]) by fmsmga006.fm.intel.com with ESMTP; 26 Jun 2018 00:08:25 -0700 From: Qi Zhang To: thomas@monjalon.net, anatoly.burakov@intel.com Cc: konstantin.ananyev@intel.com, dev@dpdk.org, bruce.richardson@intel.com, ferruh.yigit@intel.com, benjamin.h.shelton@intel.com, narender.vangati@intel.com, Qi Zhang Date: Tue, 26 Jun 2018 15:08:25 +0800 Message-Id: <20180626070832.3055-18-qi.z.zhang@intel.com> X-Mailer: git-send-email 2.13.6 In-Reply-To: <20180626070832.3055-1-qi.z.zhang@intel.com> References: <20180607123849.14439-1-qi.z.zhang@intel.com> <20180626070832.3055-1-qi.z.zhang@intel.com> Subject: [dpdk-dev] [PATCH v4 17/24] net/kni: enable port detach on secondary process X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Sender: "dev" Previously, detach port on secondary process will mess primary process and cause same device can't be attached again, by take advantage of rte_eth_release_port_private, we can support this with minor change. Signed-off-by: Qi Zhang --- drivers/net/kni/rte_eth_kni.c | 11 +++++++++++ 1 file changed, 11 insertions(+) diff --git a/drivers/net/kni/rte_eth_kni.c b/drivers/net/kni/rte_eth_kni.c index ab63ea427..e5679c76a 100644 --- a/drivers/net/kni/rte_eth_kni.c +++ b/drivers/net/kni/rte_eth_kni.c @@ -419,6 +419,7 @@ eth_kni_probe(struct rte_vdev_device *vdev) } /* TODO: request info from primary to set up Rx and Tx */ eth_dev->dev_ops = ð_kni_ops; + eth_dev->device = &vdev->device; rte_eth_dev_probing_finish(eth_dev); return 0; } @@ -463,6 +464,16 @@ eth_kni_remove(struct rte_vdev_device *vdev) if (eth_dev == NULL) return -1; + if (rte_eal_process_type() != RTE_PROC_PRIMARY) { + /* detach device on local pprocess only */ + if (strlen(rte_vdev_device_args(vdev)) == 0) + return rte_eth_dev_release_port_private(eth_dev); + /** + * else this is a private device for current process + * so continue with normal detach scenario + */ + } + eth_kni_dev_stop(eth_dev); internals = eth_dev->data->dev_private; From patchwork Tue Jun 26 07:08:26 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Qi Zhang X-Patchwork-Id: 41525 X-Patchwork-Delegate: thomas@monjalon.net Return-Path: X-Original-To: patchwork@dpdk.org Delivered-To: patchwork@dpdk.org Received: from [92.243.14.124] (localhost [127.0.0.1]) by dpdk.org (Postfix) with ESMTP id D6AB57CEB; Tue, 26 Jun 2018 09:08:57 +0200 (CEST) Received: from mga09.intel.com (mga09.intel.com [134.134.136.24]) by dpdk.org (Postfix) with ESMTP id 43E485B32 for ; Tue, 26 Jun 2018 09:08:29 +0200 (CEST) X-Amp-Result: SKIPPED(no attachment in message) X-Amp-File-Uploaded: False Received: from fmsmga006.fm.intel.com ([10.253.24.20]) by orsmga102.jf.intel.com with ESMTP/TLS/DHE-RSA-AES256-GCM-SHA384; 26 Jun 2018 00:08:29 -0700 X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="5.51,273,1526367600"; d="scan'208";a="240660063" Received: from dpdk51.sh.intel.com ([10.67.110.190]) by fmsmga006.fm.intel.com with ESMTP; 26 Jun 2018 00:08:27 -0700 From: Qi Zhang To: thomas@monjalon.net, anatoly.burakov@intel.com Cc: konstantin.ananyev@intel.com, dev@dpdk.org, bruce.richardson@intel.com, ferruh.yigit@intel.com, benjamin.h.shelton@intel.com, narender.vangati@intel.com, Qi Zhang Date: Tue, 26 Jun 2018 15:08:26 +0800 Message-Id: <20180626070832.3055-19-qi.z.zhang@intel.com> X-Mailer: git-send-email 2.13.6 In-Reply-To: <20180626070832.3055-1-qi.z.zhang@intel.com> References: <20180607123849.14439-1-qi.z.zhang@intel.com> <20180626070832.3055-1-qi.z.zhang@intel.com> Subject: [dpdk-dev] [PATCH v4 18/24] net/null: enable port detach on secondary process X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Sender: "dev" Previously, detach port on secondary process will mess primary process and cause same device can't be attached again, by take advantage of rte_eth_release_port_private, we can support this with minor change. Signed-off-by: Qi Zhang --- drivers/net/null/rte_eth_null.c | 16 +++++++++++++++- 1 file changed, 15 insertions(+), 1 deletion(-) diff --git a/drivers/net/null/rte_eth_null.c b/drivers/net/null/rte_eth_null.c index 1d2e6b9e9..2f040729b 100644 --- a/drivers/net/null/rte_eth_null.c +++ b/drivers/net/null/rte_eth_null.c @@ -623,6 +623,7 @@ rte_pmd_null_probe(struct rte_vdev_device *dev) } /* TODO: request info from primary to set up Rx and Tx */ eth_dev->dev_ops = &ops; + eth_dev->device = &dev->device; rte_eth_dev_probing_finish(eth_dev); return 0; } @@ -667,18 +668,31 @@ static int rte_pmd_null_remove(struct rte_vdev_device *dev) { struct rte_eth_dev *eth_dev = NULL; + const char *name; if (!dev) return -EINVAL; + name = rte_vdev_device_name(dev); + PMD_LOG(INFO, "Closing null ethdev on numa socket %u", rte_socket_id()); /* find the ethdev entry */ - eth_dev = rte_eth_dev_allocated(rte_vdev_device_name(dev)); + eth_dev = rte_eth_dev_allocated(name); if (eth_dev == NULL) return -1; + if (rte_eal_process_type() != RTE_PROC_PRIMARY) { + /* detach device on local pprocess only */ + if (strlen(rte_vdev_device_args(dev)) == 0) + return rte_eth_dev_release_port_private(eth_dev); + /** + * else this is a private device for current process + * so continue with normal detach scenario + */ + } + rte_free(eth_dev->data->dev_private); rte_eth_dev_release_port(eth_dev); From patchwork Tue Jun 26 07:08:27 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Qi Zhang X-Patchwork-Id: 41526 X-Patchwork-Delegate: thomas@monjalon.net Return-Path: X-Original-To: patchwork@dpdk.org Delivered-To: patchwork@dpdk.org Received: from [92.243.14.124] (localhost [127.0.0.1]) by dpdk.org (Postfix) with ESMTP id 891898D4D; Tue, 26 Jun 2018 09:08:59 +0200 (CEST) Received: from mga09.intel.com (mga09.intel.com [134.134.136.24]) by dpdk.org (Postfix) with ESMTP id C5B7D5F17 for ; Tue, 26 Jun 2018 09:08:30 +0200 (CEST) X-Amp-Result: SKIPPED(no attachment in message) X-Amp-File-Uploaded: False Received: from fmsmga006.fm.intel.com ([10.253.24.20]) by orsmga102.jf.intel.com with ESMTP/TLS/DHE-RSA-AES256-GCM-SHA384; 26 Jun 2018 00:08:30 -0700 X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="5.51,273,1526367600"; d="scan'208";a="240660072" Received: from dpdk51.sh.intel.com ([10.67.110.190]) by fmsmga006.fm.intel.com with ESMTP; 26 Jun 2018 00:08:29 -0700 From: Qi Zhang To: thomas@monjalon.net, anatoly.burakov@intel.com Cc: konstantin.ananyev@intel.com, dev@dpdk.org, bruce.richardson@intel.com, ferruh.yigit@intel.com, benjamin.h.shelton@intel.com, narender.vangati@intel.com, Qi Zhang Date: Tue, 26 Jun 2018 15:08:27 +0800 Message-Id: <20180626070832.3055-20-qi.z.zhang@intel.com> X-Mailer: git-send-email 2.13.6 In-Reply-To: <20180626070832.3055-1-qi.z.zhang@intel.com> References: <20180607123849.14439-1-qi.z.zhang@intel.com> <20180626070832.3055-1-qi.z.zhang@intel.com> Subject: [dpdk-dev] [PATCH v4 19/24] net/octeontx: enable port detach on secondary process X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Sender: "dev" Previously, detach port on secondary process will mess primary process and cause same device can't be attached again, by take advantage of rte_eth_release_port_private, we can support this with minor change. Signed-off-by: Qi Zhang --- drivers/net/octeontx/octeontx_ethdev.c | 16 ++++++++++++++++ 1 file changed, 16 insertions(+) diff --git a/drivers/net/octeontx/octeontx_ethdev.c b/drivers/net/octeontx/octeontx_ethdev.c index 1eb453b21..497bacdc6 100644 --- a/drivers/net/octeontx/octeontx_ethdev.c +++ b/drivers/net/octeontx/octeontx_ethdev.c @@ -1016,6 +1016,7 @@ octeontx_create(struct rte_vdev_device *dev, int port, uint8_t evdev, eth_dev->tx_pkt_burst = octeontx_xmit_pkts; eth_dev->rx_pkt_burst = octeontx_recv_pkts; + eth_dev->device = &dev->device; rte_eth_dev_probing_finish(eth_dev); return 0; } @@ -1138,6 +1139,18 @@ octeontx_remove(struct rte_vdev_device *dev) if (eth_dev == NULL) return -ENODEV; + if (rte_eal_process_type() != RTE_PROC_PRIMARY) { + /* detach device on local pprocess only */ + if (strlen(rte_vdev_device_args(dev)) == 0) { + rte_eth_dev_release_port_private(eth_dev); + continue; + } + /** + * else this is a private device for current process + * so continue with normal detach scenario + */ + } + nic = octeontx_pmd_priv(eth_dev); rte_event_dev_stop(nic->evdev); PMD_INIT_LOG(INFO, "Closing octeontx device %s", octtx_name); @@ -1148,6 +1161,9 @@ octeontx_remove(struct rte_vdev_device *dev) rte_event_dev_close(nic->evdev); } + if (rte_eal_process_type() != RTE_PROC_PRIMARY) + return 0; + /* Free FC resource */ octeontx_pko_fc_free(); From patchwork Tue Jun 26 07:08:28 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Qi Zhang X-Patchwork-Id: 41527 X-Patchwork-Delegate: thomas@monjalon.net Return-Path: X-Original-To: patchwork@dpdk.org Delivered-To: patchwork@dpdk.org Received: from [92.243.14.124] (localhost [127.0.0.1]) by dpdk.org (Postfix) with ESMTP id A83AC8D8F; Tue, 26 Jun 2018 09:09:01 +0200 (CEST) Received: from mga09.intel.com (mga09.intel.com [134.134.136.24]) by dpdk.org (Postfix) with ESMTP id 4D1325B1E for ; Tue, 26 Jun 2018 09:08:32 +0200 (CEST) X-Amp-Result: SKIPPED(no attachment in message) X-Amp-File-Uploaded: False Received: from fmsmga006.fm.intel.com ([10.253.24.20]) by orsmga102.jf.intel.com with ESMTP/TLS/DHE-RSA-AES256-GCM-SHA384; 26 Jun 2018 00:08:32 -0700 X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="5.51,273,1526367600"; d="scan'208";a="240660082" Received: from dpdk51.sh.intel.com ([10.67.110.190]) by fmsmga006.fm.intel.com with ESMTP; 26 Jun 2018 00:08:30 -0700 From: Qi Zhang To: thomas@monjalon.net, anatoly.burakov@intel.com Cc: konstantin.ananyev@intel.com, dev@dpdk.org, bruce.richardson@intel.com, ferruh.yigit@intel.com, benjamin.h.shelton@intel.com, narender.vangati@intel.com, Qi Zhang Date: Tue, 26 Jun 2018 15:08:28 +0800 Message-Id: <20180626070832.3055-21-qi.z.zhang@intel.com> X-Mailer: git-send-email 2.13.6 In-Reply-To: <20180626070832.3055-1-qi.z.zhang@intel.com> References: <20180607123849.14439-1-qi.z.zhang@intel.com> <20180626070832.3055-1-qi.z.zhang@intel.com> Subject: [dpdk-dev] [PATCH v4 20/24] net/pcap: enable port detach on secondary process X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Sender: "dev" Previously, detach port on secondary process will mess primary process and cause same device can't be attached again, by take advantage of rte_eth_release_port_private, we can support this with minor change. Signed-off-by: Qi Zhang --- drivers/net/pcap/rte_eth_pcap.c | 15 ++++++++++++++- 1 file changed, 14 insertions(+), 1 deletion(-) diff --git a/drivers/net/pcap/rte_eth_pcap.c b/drivers/net/pcap/rte_eth_pcap.c index 6bd4a7d79..6cc20c2b2 100644 --- a/drivers/net/pcap/rte_eth_pcap.c +++ b/drivers/net/pcap/rte_eth_pcap.c @@ -925,6 +925,7 @@ pmd_pcap_probe(struct rte_vdev_device *dev) } /* TODO: request info from primary to set up Rx and Tx */ eth_dev->dev_ops = &ops; + eth_dev->device = &dev->device; rte_eth_dev_probing_finish(eth_dev); return 0; } @@ -1016,6 +1017,7 @@ static int pmd_pcap_remove(struct rte_vdev_device *dev) { struct rte_eth_dev *eth_dev = NULL; + const char *name; PMD_LOG(INFO, "Closing pcap ethdev on numa socket %d", rte_socket_id()); @@ -1023,11 +1025,22 @@ pmd_pcap_remove(struct rte_vdev_device *dev) if (!dev) return -1; + name = rte_vdev_device_name(dev); /* reserve an ethdev entry */ - eth_dev = rte_eth_dev_allocated(rte_vdev_device_name(dev)); + eth_dev = rte_eth_dev_allocated(name); if (eth_dev == NULL) return -1; + if (rte_eal_process_type() != RTE_PROC_PRIMARY) { + /* detach device on local pprocess only */ + if (strlen(rte_vdev_device_args(dev)) == 0) + return rte_eth_dev_release_port_private(eth_dev); + /** + * else this is a private device for current process + * so continue with normal detach scenario + */ + } + rte_free(eth_dev->data->dev_private); rte_eth_dev_release_port(eth_dev); From patchwork Tue Jun 26 07:08:29 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Qi Zhang X-Patchwork-Id: 41528 X-Patchwork-Delegate: thomas@monjalon.net Return-Path: X-Original-To: patchwork@dpdk.org Delivered-To: patchwork@dpdk.org Received: from [92.243.14.124] (localhost [127.0.0.1]) by dpdk.org (Postfix) with ESMTP id AAA968E65; Tue, 26 Jun 2018 09:09:03 +0200 (CEST) Received: from mga09.intel.com (mga09.intel.com [134.134.136.24]) by dpdk.org (Postfix) with ESMTP id D1BA65F27 for ; Tue, 26 Jun 2018 09:08:33 +0200 (CEST) X-Amp-Result: SKIPPED(no attachment in message) X-Amp-File-Uploaded: False Received: from fmsmga006.fm.intel.com ([10.253.24.20]) by orsmga102.jf.intel.com with ESMTP/TLS/DHE-RSA-AES256-GCM-SHA384; 26 Jun 2018 00:08:33 -0700 X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="5.51,273,1526367600"; d="scan'208";a="240660091" Received: from dpdk51.sh.intel.com ([10.67.110.190]) by fmsmga006.fm.intel.com with ESMTP; 26 Jun 2018 00:08:32 -0700 From: Qi Zhang To: thomas@monjalon.net, anatoly.burakov@intel.com Cc: konstantin.ananyev@intel.com, dev@dpdk.org, bruce.richardson@intel.com, ferruh.yigit@intel.com, benjamin.h.shelton@intel.com, narender.vangati@intel.com, Qi Zhang Date: Tue, 26 Jun 2018 15:08:29 +0800 Message-Id: <20180626070832.3055-22-qi.z.zhang@intel.com> X-Mailer: git-send-email 2.13.6 In-Reply-To: <20180626070832.3055-1-qi.z.zhang@intel.com> References: <20180607123849.14439-1-qi.z.zhang@intel.com> <20180626070832.3055-1-qi.z.zhang@intel.com> Subject: [dpdk-dev] [PATCH v4 21/24] net/softnic: enable port detach on secondary process X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Sender: "dev" Previously, detach port on secondary process will mess primary process and cause same device can't be attached again, by take advantage of rte_eth_release_port_private, we can support this with minor change. Signed-off-by: Qi Zhang --- drivers/net/softnic/rte_eth_softnic.c | 19 ++++++++++++++++--- 1 file changed, 16 insertions(+), 3 deletions(-) diff --git a/drivers/net/softnic/rte_eth_softnic.c b/drivers/net/softnic/rte_eth_softnic.c index 6b3c13e5c..a45a7b0dd 100644 --- a/drivers/net/softnic/rte_eth_softnic.c +++ b/drivers/net/softnic/rte_eth_softnic.c @@ -750,6 +750,7 @@ pmd_probe(struct rte_vdev_device *vdev) } /* TODO: request info from primary to set up Rx and Tx */ eth_dev->dev_ops = &pmd_ops; + eth_dev->device = &vdev->device; rte_eth_dev_probing_finish(eth_dev); return 0; } @@ -803,17 +804,29 @@ pmd_remove(struct rte_vdev_device *vdev) { struct rte_eth_dev *dev = NULL; struct pmd_internals *p; + const char *name; if (!vdev) return -EINVAL; - PMD_LOG(INFO, "Removing device \"%s\"", - rte_vdev_device_name(vdev)); + name = rte_vdev_device_name(vdev); + PMD_LOG(INFO, "Removing device \"%s\"", name); /* Find the ethdev entry */ - dev = rte_eth_dev_allocated(rte_vdev_device_name(vdev)); + dev = rte_eth_dev_allocated(name); if (dev == NULL) return -ENODEV; + + if (rte_eal_process_type() != RTE_PROC_PRIMARY) { + /* detach device on local pprocess only */ + if (strlen(rte_vdev_device_args(vdev)) == 0) + return rte_eth_dev_release_port_private(dev); + /** + * else this is a private device for current process + * so continue with normal detach scenario + */ + } + p = dev->data->dev_private; /* Free device data structures*/ From patchwork Tue Jun 26 07:08:30 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Qi Zhang X-Patchwork-Id: 41529 X-Patchwork-Delegate: thomas@monjalon.net Return-Path: X-Original-To: patchwork@dpdk.org Delivered-To: patchwork@dpdk.org Received: from [92.243.14.124] (localhost [127.0.0.1]) by dpdk.org (Postfix) with ESMTP id 97890AAD5; Tue, 26 Jun 2018 09:09:05 +0200 (CEST) Received: from mga09.intel.com (mga09.intel.com [134.134.136.24]) by dpdk.org (Postfix) with ESMTP id 5B8755F27 for ; Tue, 26 Jun 2018 09:08:35 +0200 (CEST) X-Amp-Result: SKIPPED(no attachment in message) X-Amp-File-Uploaded: False Received: from fmsmga006.fm.intel.com ([10.253.24.20]) by orsmga102.jf.intel.com with ESMTP/TLS/DHE-RSA-AES256-GCM-SHA384; 26 Jun 2018 00:08:35 -0700 X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="5.51,273,1526367600"; d="scan'208";a="240660099" Received: from dpdk51.sh.intel.com ([10.67.110.190]) by fmsmga006.fm.intel.com with ESMTP; 26 Jun 2018 00:08:33 -0700 From: Qi Zhang To: thomas@monjalon.net, anatoly.burakov@intel.com Cc: konstantin.ananyev@intel.com, dev@dpdk.org, bruce.richardson@intel.com, ferruh.yigit@intel.com, benjamin.h.shelton@intel.com, narender.vangati@intel.com, Qi Zhang Date: Tue, 26 Jun 2018 15:08:30 +0800 Message-Id: <20180626070832.3055-23-qi.z.zhang@intel.com> X-Mailer: git-send-email 2.13.6 In-Reply-To: <20180626070832.3055-1-qi.z.zhang@intel.com> References: <20180607123849.14439-1-qi.z.zhang@intel.com> <20180626070832.3055-1-qi.z.zhang@intel.com> Subject: [dpdk-dev] [PATCH v4 22/24] net/tap: enable port detach on secondary process X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Sender: "dev" Previously, detach port on secondary process will mess primary process and cause same device can't be attached again, by take advantage of rte_eth_release_port_private, we can support this with minor change. Signed-off-by: Qi Zhang --- drivers/net/tap/rte_eth_tap.c | 17 +++++++++++++++-- 1 file changed, 15 insertions(+), 2 deletions(-) diff --git a/drivers/net/tap/rte_eth_tap.c b/drivers/net/tap/rte_eth_tap.c index df396bfde..bb5f20b01 100644 --- a/drivers/net/tap/rte_eth_tap.c +++ b/drivers/net/tap/rte_eth_tap.c @@ -1759,6 +1759,7 @@ rte_pmd_tap_probe(struct rte_vdev_device *dev) } /* TODO: request info from primary to set up Rx and Tx */ eth_dev->dev_ops = &ops; + eth_dev->device = &dev->device; rte_eth_dev_probing_finish(eth_dev); return 0; } @@ -1827,12 +1828,24 @@ rte_pmd_tap_remove(struct rte_vdev_device *dev) { struct rte_eth_dev *eth_dev = NULL; struct pmd_internals *internals; + const char *name; int i; + name = rte_vdev_device_name(dev); /* find the ethdev entry */ - eth_dev = rte_eth_dev_allocated(rte_vdev_device_name(dev)); + eth_dev = rte_eth_dev_allocated(name); if (!eth_dev) - return 0; + return -ENODEV; + + if (rte_eal_process_type() != RTE_PROC_PRIMARY) { + /* detach device on local pprocess only */ + if (strlen(rte_vdev_device_args(dev)) == 0) + return rte_eth_dev_release_port_private(eth_dev); + /** + * else this is a private device for current process + * so continue with normal detach scenario + */ + } internals = eth_dev->data->dev_private; From patchwork Tue Jun 26 07:08:31 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Qi Zhang X-Patchwork-Id: 41530 X-Patchwork-Delegate: thomas@monjalon.net Return-Path: X-Original-To: patchwork@dpdk.org Delivered-To: patchwork@dpdk.org Received: from [92.243.14.124] (localhost [127.0.0.1]) by dpdk.org (Postfix) with ESMTP id 9D5731B04D; Tue, 26 Jun 2018 09:09:07 +0200 (CEST) Received: from mga09.intel.com (mga09.intel.com [134.134.136.24]) by dpdk.org (Postfix) with ESMTP id CC75F5F17 for ; Tue, 26 Jun 2018 09:08:36 +0200 (CEST) X-Amp-Result: SKIPPED(no attachment in message) X-Amp-File-Uploaded: False Received: from fmsmga006.fm.intel.com ([10.253.24.20]) by orsmga102.jf.intel.com with ESMTP/TLS/DHE-RSA-AES256-GCM-SHA384; 26 Jun 2018 00:08:36 -0700 X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="5.51,273,1526367600"; d="scan'208";a="240660107" Received: from dpdk51.sh.intel.com ([10.67.110.190]) by fmsmga006.fm.intel.com with ESMTP; 26 Jun 2018 00:08:35 -0700 From: Qi Zhang To: thomas@monjalon.net, anatoly.burakov@intel.com Cc: konstantin.ananyev@intel.com, dev@dpdk.org, bruce.richardson@intel.com, ferruh.yigit@intel.com, benjamin.h.shelton@intel.com, narender.vangati@intel.com, Qi Zhang Date: Tue, 26 Jun 2018 15:08:31 +0800 Message-Id: <20180626070832.3055-24-qi.z.zhang@intel.com> X-Mailer: git-send-email 2.13.6 In-Reply-To: <20180626070832.3055-1-qi.z.zhang@intel.com> References: <20180607123849.14439-1-qi.z.zhang@intel.com> <20180626070832.3055-1-qi.z.zhang@intel.com> Subject: [dpdk-dev] [PATCH v4 23/24] net/vhost: enable port detach on secondary process X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Sender: "dev" Previously, detach port on secondary process will mess primary process and cause same device can't be attached again, by take advantage of rte_eth_release_port_private, we can support this with minor change. Signed-off-by: Qi Zhang --- drivers/net/vhost/rte_eth_vhost.c | 11 +++++++++++ 1 file changed, 11 insertions(+) diff --git a/drivers/net/vhost/rte_eth_vhost.c b/drivers/net/vhost/rte_eth_vhost.c index ba9d768a0..f773711b4 100644 --- a/drivers/net/vhost/rte_eth_vhost.c +++ b/drivers/net/vhost/rte_eth_vhost.c @@ -1353,6 +1353,7 @@ rte_pmd_vhost_probe(struct rte_vdev_device *dev) } /* TODO: request info from primary to set up Rx and Tx */ eth_dev->dev_ops = &ops; + eth_dev->device = &dev->device; rte_eth_dev_probing_finish(eth_dev); return 0; } @@ -1435,6 +1436,16 @@ rte_pmd_vhost_remove(struct rte_vdev_device *dev) if (eth_dev == NULL) return -ENODEV; + if (rte_eal_process_type() != RTE_PROC_PRIMARY) { + /* detach device on local pprocess only */ + if (strlen(rte_vdev_device_args(dev)) == 0) + return rte_eth_dev_release_port_private(eth_dev); + /** + * else this is a private device for current process + * so continue with normal detach scenario + */ + } + eth_dev_close(eth_dev); rte_free(vring_states[eth_dev->data->port_id]); From patchwork Tue Jun 26 07:08:32 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Qi Zhang X-Patchwork-Id: 41531 X-Patchwork-Delegate: thomas@monjalon.net Return-Path: X-Original-To: patchwork@dpdk.org Delivered-To: patchwork@dpdk.org Received: from [92.243.14.124] (localhost [127.0.0.1]) by dpdk.org (Postfix) with ESMTP id 3998A1B05C; Tue, 26 Jun 2018 09:09:09 +0200 (CEST) Received: from mga09.intel.com (mga09.intel.com [134.134.136.24]) by dpdk.org (Postfix) with ESMTP id 681E95F27 for ; Tue, 26 Jun 2018 09:08:38 +0200 (CEST) X-Amp-Result: SKIPPED(no attachment in message) X-Amp-File-Uploaded: False Received: from fmsmga006.fm.intel.com ([10.253.24.20]) by orsmga102.jf.intel.com with ESMTP/TLS/DHE-RSA-AES256-GCM-SHA384; 26 Jun 2018 00:08:38 -0700 X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="5.51,273,1526367600"; d="scan'208";a="240660116" Received: from dpdk51.sh.intel.com ([10.67.110.190]) by fmsmga006.fm.intel.com with ESMTP; 26 Jun 2018 00:08:36 -0700 From: Qi Zhang To: thomas@monjalon.net, anatoly.burakov@intel.com Cc: konstantin.ananyev@intel.com, dev@dpdk.org, bruce.richardson@intel.com, ferruh.yigit@intel.com, benjamin.h.shelton@intel.com, narender.vangati@intel.com, Qi Zhang Date: Tue, 26 Jun 2018 15:08:32 +0800 Message-Id: <20180626070832.3055-25-qi.z.zhang@intel.com> X-Mailer: git-send-email 2.13.6 In-Reply-To: <20180626070832.3055-1-qi.z.zhang@intel.com> References: <20180607123849.14439-1-qi.z.zhang@intel.com> <20180626070832.3055-1-qi.z.zhang@intel.com> Subject: [dpdk-dev] [PATCH v4 24/24] examples/multi_process: add hotplug sample X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Sender: "dev" The sample code demonstrate device (ethdev only) management at multi-process envrionment. User can attach/detach a device on primary process and see it is synced on secondary process automatically, also user can lock a device to prevent it be detached or unlock it to go back to default behaviour. How to start? ./hotplug_mp --proc-type=auto Command Line Example: >help >list /* attach a af_packet vdev */ >attach net_af_packet,iface=eth0 /* detach port 0 */ >detach 0 /* attach a private af_packet vdev (secondary process only)*/ >attachp net_af_packet,iface=eth0 /* detach a private device (secondary process only) */ >detachp 0 /* lock port 0 */ >lock 0 /* unlock port 0 */ >unlock 0 Signed-off-by: Qi Zhang Acked-by: Anatoly Burakov --- examples/multi_process/Makefile | 1 + examples/multi_process/hotplug_mp/Makefile | 23 ++ examples/multi_process/hotplug_mp/commands.c | 356 +++++++++++++++++++++++++++ examples/multi_process/hotplug_mp/commands.h | 10 + examples/multi_process/hotplug_mp/main.c | 41 +++ 5 files changed, 431 insertions(+) create mode 100644 examples/multi_process/hotplug_mp/Makefile create mode 100644 examples/multi_process/hotplug_mp/commands.c create mode 100644 examples/multi_process/hotplug_mp/commands.h create mode 100644 examples/multi_process/hotplug_mp/main.c diff --git a/examples/multi_process/Makefile b/examples/multi_process/Makefile index a6708b7e4..b76b02fcb 100644 --- a/examples/multi_process/Makefile +++ b/examples/multi_process/Makefile @@ -13,5 +13,6 @@ include $(RTE_SDK)/mk/rte.vars.mk DIRS-$(CONFIG_RTE_EXEC_ENV_LINUXAPP) += client_server_mp DIRS-$(CONFIG_RTE_EXEC_ENV_LINUXAPP) += simple_mp DIRS-$(CONFIG_RTE_EXEC_ENV_LINUXAPP) += symmetric_mp +DIRS-$(CONFIG_RTE_EXEC_ENV_LINUXAPP) += hotplug_mp include $(RTE_SDK)/mk/rte.extsubdir.mk diff --git a/examples/multi_process/hotplug_mp/Makefile b/examples/multi_process/hotplug_mp/Makefile new file mode 100644 index 000000000..c09a57bfa --- /dev/null +++ b/examples/multi_process/hotplug_mp/Makefile @@ -0,0 +1,23 @@ +# SPDX-License-Identifier: BSD-3-Clause +# Copyright(c) 2010-2014 Intel Corporation + +ifeq ($(RTE_SDK),) +$(error "Please define RTE_SDK environment variable") +endif + +# Default target, can be overridden by command line or environment +RTE_TARGET ?= x86_64-native-linuxapp-gcc + +include $(RTE_SDK)/mk/rte.vars.mk + +# binary name +APP = hotplug_mp + +# all source are stored in SRCS-y +SRCS-y := main.c commands.c + +CFLAGS += -O3 +CFLAGS += $(WERROR_FLAGS) +CFLAGS += -DALLOW_EXPERIMENTAL_API + +include $(RTE_SDK)/mk/rte.extapp.mk diff --git a/examples/multi_process/hotplug_mp/commands.c b/examples/multi_process/hotplug_mp/commands.c new file mode 100644 index 000000000..31f9e2e15 --- /dev/null +++ b/examples/multi_process/hotplug_mp/commands.c @@ -0,0 +1,356 @@ +/* SPDX-License-Identifier: BSD-3-Clause + * Copyright(c) 2018 Intel Corporation. + */ + +#include +#include +#include +#include +#include +#include +#include + +/**********************************************************/ + +struct cmd_help_result { + cmdline_fixed_string_t help; +}; + +static void cmd_help_parsed(__attribute__((unused)) void *parsed_result, + struct cmdline *cl, + __attribute__((unused)) void *data) +{ + cmdline_printf(cl, + "commands:\n" + "- attach \n" + "- detach \n" + "- attachp \n" + "- detachp \n" + "- lock \n" + "- unlock \n" + "- list\n\n"); +} + +cmdline_parse_token_string_t cmd_help_help = + TOKEN_STRING_INITIALIZER(struct cmd_help_result, help, "help"); + +cmdline_parse_inst_t cmd_help = { + .f = cmd_help_parsed, /* function to call */ + .data = NULL, /* 2nd arg of func */ + .help_str = "show help", + .tokens = { /* token list, NULL terminated */ + (void *)&cmd_help_help, + NULL, + }, +}; + +/**********************************************************/ + +struct cmd_quit_result { + cmdline_fixed_string_t quit; +}; + +static void cmd_quit_parsed(__attribute__((unused)) void *parsed_result, + struct cmdline *cl, + __attribute__((unused)) void *data) +{ + cmdline_quit(cl); +} + +cmdline_parse_token_string_t cmd_quit_quit = + TOKEN_STRING_INITIALIZER(struct cmd_quit_result, quit, "quit"); + +cmdline_parse_inst_t cmd_quit = { + .f = cmd_quit_parsed, /* function to call */ + .data = NULL, /* 2nd arg of func */ + .help_str = "quit", + .tokens = { /* token list, NULL terminated */ + (void *)&cmd_quit_quit, + NULL, + }, +}; + +/**********************************************************/ + +struct cmd_list_result { + cmdline_fixed_string_t list; +}; + +static void cmd_list_parsed(__attribute__((unused)) void *parsed_result, + struct cmdline *cl, + __attribute__((unused)) void *data) +{ + uint16_t port_id; + char dev_name[RTE_DEV_NAME_MAX_LEN]; + + cmdline_printf(cl, "list all etherdev\n"); + + RTE_ETH_FOREACH_DEV(port_id) { + rte_eth_dev_get_name_by_port(port_id, dev_name); + if (strlen(dev_name) > 0) + cmdline_printf(cl, "%d\t%s\n", port_id, dev_name); + else + printf("empty dev_name is not expected!\n"); + } +} + +cmdline_parse_token_string_t cmd_list_list = + TOKEN_STRING_INITIALIZER(struct cmd_list_result, list, "list"); + +cmdline_parse_inst_t cmd_list = { + .f = cmd_list_parsed, /* function to call */ + .data = NULL, /* 2nd arg of func */ + .help_str = "list all devices", + .tokens = { /* token list, NULL terminated */ + (void *)&cmd_list_list, + NULL, + }, +}; + +/**********************************************************/ + +struct cmd_dev_attach_result { + cmdline_fixed_string_t attach; + cmdline_fixed_string_t device; +}; + +static void cmd_dev_attach_parsed(void *parsed_result, + struct cmdline *cl, + __attribute__((unused)) void *data) +{ + struct cmd_dev_attach_result *res = parsed_result; + uint16_t port_id; + + if (!rte_eth_dev_attach(res->device, &port_id)) + cmdline_printf(cl, "attached device %s at port %d\n", + res->device, port_id); + else + cmdline_printf(cl, "failed to attached device %s\n", + res->device); +} + +cmdline_parse_token_string_t cmd_dev_attach_attach = + TOKEN_STRING_INITIALIZER(struct cmd_dev_attach_result, attach, + "attach"); +cmdline_parse_token_string_t cmd_dev_attach_device = + TOKEN_STRING_INITIALIZER(struct cmd_dev_attach_result, device, NULL); + +cmdline_parse_inst_t cmd_attach_device = { + .f = cmd_dev_attach_parsed, /* function to call */ + .data = NULL, /* 2nd arg of func */ + .help_str = "attach a device", + .tokens = { /* token list, NULL terminated */ + (void *)&cmd_dev_attach_attach, + (void *)&cmd_dev_attach_device, + NULL, + }, +}; + +/**********************************************************/ + +struct cmd_dev_attachp_result { + cmdline_fixed_string_t attachp; + cmdline_fixed_string_t device; +}; + +static void cmd_dev_attachp_parsed(void *parsed_result, + struct cmdline *cl, + __attribute__((unused)) void *data) +{ + struct cmd_dev_attachp_result *res = parsed_result; + uint16_t port_id; + + if (!rte_eth_dev_attach_private(res->device, &port_id)) + cmdline_printf(cl, "attached prviate device %s at port %d\n", + res->device, port_id); + else + cmdline_printf(cl, "failed to attached private device %s\n", + res->device); +} + +cmdline_parse_token_string_t cmd_dev_attachp_attachp = + TOKEN_STRING_INITIALIZER(struct cmd_dev_attachp_result, attachp, + "attachp"); +cmdline_parse_token_string_t cmd_dev_attachp_device = + TOKEN_STRING_INITIALIZER(struct cmd_dev_attachp_result, device, NULL); + +cmdline_parse_inst_t cmd_attachp_device = { + .f = cmd_dev_attachp_parsed, /* function to call */ + .data = NULL, /* 2nd arg of func */ + .help_str = "attach a private device", + .tokens = { /* token list, NULL terminated */ + (void *)&cmd_dev_attachp_attachp, + (void *)&cmd_dev_attachp_device, + NULL, + }, +}; + +/**********************************************************/ + +struct cmd_dev_detach_result { + cmdline_fixed_string_t detach; + uint16_t port_id; +}; + +static void cmd_dev_detach_parsed(void *parsed_result, + struct cmdline *cl, + __attribute__((unused)) void *data) +{ + struct cmd_dev_detach_result *res = parsed_result; + uint16_t port_id = res->port_id; + char dev_name[RTE_DEV_NAME_MAX_LEN]; + + printf("detaching...\n"); + if (!rte_eth_dev_detach(port_id, dev_name)) + cmdline_printf(cl, "detached device at port %d\n", + port_id); + else + cmdline_printf(cl, "failed to dettached at port %d\n", + port_id); +} + +cmdline_parse_token_string_t cmd_dev_detach_detach = + TOKEN_STRING_INITIALIZER(struct cmd_dev_detach_result, detach, + "detach"); +cmdline_parse_token_num_t cmd_dev_detach_port_id = + TOKEN_NUM_INITIALIZER(struct cmd_dev_detach_result, port_id, UINT16); + +cmdline_parse_inst_t cmd_detach_device = { + .f = cmd_dev_detach_parsed, /* function to call */ + .data = NULL, /* 2nd arg of func */ + .help_str = "detach a device", + .tokens = { /* token list, NULL terminated */ + (void *)&cmd_dev_detach_detach, + (void *)&cmd_dev_detach_port_id, + NULL, + }, +}; + +/**********************************************************/ + +struct cmd_dev_detachp_result { + cmdline_fixed_string_t detachp; + uint16_t port_id; +}; + +static void cmd_dev_detachp_parsed(void *parsed_result, + struct cmdline *cl, + __attribute__((unused)) void *data) +{ + struct cmd_dev_detachp_result *res = parsed_result; + uint16_t port_id = res->port_id; + char dev_name[RTE_DEV_NAME_MAX_LEN]; + + printf("detaching...\n"); + if (!rte_eth_dev_detach_private(port_id, dev_name)) + cmdline_printf(cl, "detached private device at port %d\n", + port_id); + else + cmdline_printf(cl, "failed to detach private device at port %d\n", + port_id); +} + +cmdline_parse_token_string_t cmd_dev_detachp_detachp = + TOKEN_STRING_INITIALIZER(struct cmd_dev_detachp_result, detachp, + "detachp"); +cmdline_parse_token_num_t cmd_dev_detachp_port_id = + TOKEN_NUM_INITIALIZER(struct cmd_dev_detachp_result, port_id, UINT16); + +cmdline_parse_inst_t cmd_detachp_device = { + .f = cmd_dev_detachp_parsed, /* function to call */ + .data = NULL, /* 2nd arg of func */ + .help_str = "detach a private device", + .tokens = { /* token list, NULL terminated */ + (void *)&cmd_dev_detachp_detachp, + (void *)&cmd_dev_detachp_port_id, + NULL, + }, +}; + +/**********************************************************/ + +struct cmd_dev_lock_result { + cmdline_fixed_string_t lock; + uint16_t port_id; +}; + +static void cmd_dev_lock_parsed(void *parsed_result, + struct cmdline *cl, + __attribute__((unused)) void *data) +{ + struct cmd_dev_lock_result *res = parsed_result; + uint16_t port_id = res->port_id; + int ret = 0; + + ret = rte_eth_dev_lock(res->port_id); + cmdline_printf(cl, "lock port %d, ret = %d\n", port_id, ret); +} + +cmdline_parse_token_string_t cmd_dev_lock_lock = + TOKEN_STRING_INITIALIZER(struct cmd_dev_lock_result, lock, "lock"); +cmdline_parse_token_num_t cmd_dev_lock_port_id = + TOKEN_NUM_INITIALIZER(struct cmd_dev_lock_result, port_id, UINT16); + +cmdline_parse_inst_t cmd_lock_device = { + .f = cmd_dev_lock_parsed, /* function to call */ + .data = NULL, /* 2nd arg of func */ + .help_str = "lock a device", + .tokens = { /* token list, NULL terminated */ + (void *)&cmd_dev_lock_lock, + (void *)&cmd_dev_lock_port_id, + NULL, + }, +}; + +/**********************************************************/ + +struct cmd_dev_unlock_result { + cmdline_fixed_string_t unlock; + uint16_t port_id; +}; + +static void cmd_dev_unlock_parsed(void *parsed_result, + struct cmdline *cl, + __attribute__((unused)) void *data) +{ + struct cmd_dev_unlock_result *res = parsed_result; + uint16_t port_id = res->port_id; + int ret = 0; + + ret = rte_eth_dev_unlock(res->port_id); + cmdline_printf(cl, "unlock port %d, ret = %d\n", port_id, ret); +} + +cmdline_parse_token_string_t cmd_dev_unlock_unlock = + TOKEN_STRING_INITIALIZER(struct cmd_dev_unlock_result, unlock, + "unlock"); +cmdline_parse_token_num_t cmd_dev_unlock_port_id = + TOKEN_NUM_INITIALIZER(struct cmd_dev_unlock_result, port_id, UINT16); + +cmdline_parse_inst_t cmd_unlock_device = { + .f = cmd_dev_unlock_parsed, /* function to call */ + .data = NULL, /* 2nd arg of func */ + .help_str = "unlock a device", + .tokens = { /* token list, NULL terminated */ + (void *)&cmd_dev_unlock_unlock, + (void *)&cmd_dev_unlock_port_id, + NULL, + }, +}; + +/**********************************************************/ +/**********************************************************/ +/****** CONTEXT (list of instruction) */ + +cmdline_parse_ctx_t main_ctx[] = { + (cmdline_parse_inst_t *)&cmd_help, + (cmdline_parse_inst_t *)&cmd_quit, + (cmdline_parse_inst_t *)&cmd_list, + (cmdline_parse_inst_t *)&cmd_attach_device, + (cmdline_parse_inst_t *)&cmd_detach_device, + (cmdline_parse_inst_t *)&cmd_attachp_device, + (cmdline_parse_inst_t *)&cmd_detachp_device, + (cmdline_parse_inst_t *)&cmd_lock_device, + (cmdline_parse_inst_t *)&cmd_unlock_device, + NULL, +}; diff --git a/examples/multi_process/hotplug_mp/commands.h b/examples/multi_process/hotplug_mp/commands.h new file mode 100644 index 000000000..afcf177db --- /dev/null +++ b/examples/multi_process/hotplug_mp/commands.h @@ -0,0 +1,10 @@ +/* SPDX-License-Identifier: BSD-3-Clause + * Copyright(c) 2018 Intel Corporation + */ + +#ifndef _COMMANDS_H_ +#define _COMMANDS_H_ + +extern cmdline_parse_ctx_t main_ctx[]; + +#endif /* _COMMANDS_H_ */ diff --git a/examples/multi_process/hotplug_mp/main.c b/examples/multi_process/hotplug_mp/main.c new file mode 100644 index 000000000..d66858078 --- /dev/null +++ b/examples/multi_process/hotplug_mp/main.c @@ -0,0 +1,41 @@ +/* SPDX-License-Identifier: BSD-3-Clause + * Copyright(c) 2018 Intel Corporation + */ + +#include +#include +#include +#include +#include +#include + +#include +#include +#include +#include + +#include +#include +#include + +#include "commands.h" + +int main(int argc, char **argv) +{ + int ret; + struct cmdline *cl; + + ret = rte_eal_init(argc, argv); + if (ret < 0) + rte_panic("Cannot init EAL\n"); + + cl = cmdline_stdin_new(main_ctx, "example> "); + if (cl == NULL) + rte_panic("Cannot create cmdline instance\n"); + cmdline_interact(cl); + cmdline_stdin_exit(cl); + + rte_eal_cleanup(); + + return 0; +}