From patchwork Fri Sep 28 04:23:21 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Qi Zhang X-Patchwork-Id: 45550 X-Patchwork-Delegate: thomas@monjalon.net Return-Path: X-Original-To: patchwork@dpdk.org Delivered-To: patchwork@dpdk.org Received: from [92.243.14.124] (localhost [127.0.0.1]) by dpdk.org (Postfix) with ESMTP id 7F33A1B13E; Fri, 28 Sep 2018 06:22:55 +0200 (CEST) Received: from mga04.intel.com (mga04.intel.com [192.55.52.120]) by dpdk.org (Postfix) with ESMTP id 0F21D1B11C for ; Fri, 28 Sep 2018 06:22:52 +0200 (CEST) X-Amp-Result: SKIPPED(no attachment in message) X-Amp-File-Uploaded: False Received: from fmsmga005.fm.intel.com ([10.253.24.32]) by fmsmga104.fm.intel.com with ESMTP/TLS/DHE-RSA-AES256-GCM-SHA384; 27 Sep 2018 21:22:52 -0700 X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="5.54,313,1534834800"; d="scan'208";a="266654779" Received: from dpdk51.sh.intel.com ([10.67.110.190]) by fmsmga005.fm.intel.com with ESMTP; 27 Sep 2018 21:22:28 -0700 From: Qi Zhang To: thomas@monjalon.net, gaetan.rivet@6wind.com, anatoly.burakov@intel.com, arybchenko@solarflare.com Cc: konstantin.ananyev@intel.com, dev@dpdk.org, bruce.richardson@intel.com, ferruh.yigit@intel.com, benjamin.h.shelton@intel.com, narender.vangati@intel.com, Qi Zhang Date: Fri, 28 Sep 2018 12:23:21 +0800 Message-Id: <20180928042326.50471-2-qi.z.zhang@intel.com> X-Mailer: git-send-email 2.13.6 In-Reply-To: <20180928042326.50471-1-qi.z.zhang@intel.com> References: <20180607123849.14439-1-qi.z.zhang@intel.com> <20180928042326.50471-1-qi.z.zhang@intel.com> Subject: [dpdk-dev] [PATCH v16 1/6] ethdev: add function to release port in secondary process X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Sender: "dev" Add driver API rte_eth_release_port_secondary to support the case when an ethdev need to be detached on a secondary process. Local state is set to unused and shared data will not be reset so the primary process can still use it. Signed-off-by: Qi Zhang --- lib/librte_ethdev/rte_ethdev.c | 17 +++++++++++++++-- lib/librte_ethdev/rte_ethdev_driver.h | 16 +++++++++++++++- lib/librte_ethdev/rte_ethdev_pci.h | 10 ++++++++-- lib/librte_ethdev/rte_ethdev_version.map | 7 +++++++ 4 files changed, 45 insertions(+), 5 deletions(-) diff --git a/lib/librte_ethdev/rte_ethdev.c b/lib/librte_ethdev/rte_ethdev.c index aa7730ce2..c58b1c6ed 100644 --- a/lib/librte_ethdev/rte_ethdev.c +++ b/lib/librte_ethdev/rte_ethdev.c @@ -360,6 +360,18 @@ rte_eth_dev_attach_secondary(const char *name) } int +rte_eth_dev_release_port_secondary(struct rte_eth_dev *eth_dev) +{ + if (eth_dev == NULL) + return -EINVAL; + + _rte_eth_dev_callback_process(eth_dev, RTE_ETH_EVENT_DESTROY, NULL); + eth_dev->state = RTE_ETH_DEV_UNUSED; + + return 0; +} + +int rte_eth_dev_release_port(struct rte_eth_dev *eth_dev) { if (eth_dev == NULL) @@ -3540,9 +3552,10 @@ rte_eth_dev_destroy(struct rte_eth_dev *ethdev, return ret; } - if (rte_eal_process_type() == RTE_PROC_PRIMARY) - rte_free(ethdev->data->dev_private); + if (rte_eal_process_type() != RTE_PROC_PRIMARY) + return rte_eth_dev_release_port_secondary(ethdev); + rte_free(ethdev->data->dev_private); ethdev->data->dev_private = NULL; return rte_eth_dev_release_port(ethdev); diff --git a/lib/librte_ethdev/rte_ethdev_driver.h b/lib/librte_ethdev/rte_ethdev_driver.h index f158462a0..ca31b5777 100644 --- a/lib/librte_ethdev/rte_ethdev_driver.h +++ b/lib/librte_ethdev/rte_ethdev_driver.h @@ -61,7 +61,7 @@ struct rte_eth_dev *rte_eth_dev_attach_secondary(const char *name); * Release the specified ethdev port. * * @param eth_dev - * The *eth_dev* pointer is the address of the *rte_eth_dev* structure. + * Device to be detached. * @return * - 0 on success, negative on error */ @@ -69,6 +69,20 @@ int rte_eth_dev_release_port(struct rte_eth_dev *eth_dev); /** * @internal + * Release the specified ethdev port in the local process. + * Only set ethdev state to unused, but not reset shared data since + * it assume other processes is still using it. typically it is + * called by a secondary process. + * + * @param eth_dev + * Device to be detached. + * @return + * - 0 on success, negative on error + */ +int rte_eth_dev_release_port_secondary(struct rte_eth_dev *eth_dev); + +/** + * @internal * Release device queues and clear its configuration to force the user * application to reconfigure it. It is for internal use only. * diff --git a/lib/librte_ethdev/rte_ethdev_pci.h b/lib/librte_ethdev/rte_ethdev_pci.h index f652596f4..70d2d2503 100644 --- a/lib/librte_ethdev/rte_ethdev_pci.h +++ b/lib/librte_ethdev/rte_ethdev_pci.h @@ -135,9 +135,15 @@ rte_eth_dev_pci_allocate(struct rte_pci_device *dev, size_t private_data_size) static inline void rte_eth_dev_pci_release(struct rte_eth_dev *eth_dev) { - if (rte_eal_process_type() == RTE_PROC_PRIMARY) - rte_free(eth_dev->data->dev_private); + if (rte_eal_process_type() != RTE_PROC_PRIMARY) { + eth_dev->device = NULL; + eth_dev->intr_handle = NULL; + rte_eth_dev_release_port_secondary(eth_dev); + return; + } + /* primary process */ + rte_free(eth_dev->data->dev_private); eth_dev->data->dev_private = NULL; /* diff --git a/lib/librte_ethdev/rte_ethdev_version.map b/lib/librte_ethdev/rte_ethdev_version.map index 38f117f01..acc407f86 100644 --- a/lib/librte_ethdev/rte_ethdev_version.map +++ b/lib/librte_ethdev/rte_ethdev_version.map @@ -220,6 +220,13 @@ DPDK_18.08 { } DPDK_18.05; +DPDK_18.11 { + global: + + rte_eth_dev_release_port_secondary; + +} DPDK_18.08; + EXPERIMENTAL { global: From patchwork Fri Sep 28 04:23:22 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Qi Zhang X-Patchwork-Id: 45551 X-Patchwork-Delegate: thomas@monjalon.net Return-Path: X-Original-To: patchwork@dpdk.org Delivered-To: patchwork@dpdk.org Received: from [92.243.14.124] (localhost [127.0.0.1]) by dpdk.org (Postfix) with ESMTP id 4F0571B145; Fri, 28 Sep 2018 06:22:58 +0200 (CEST) Received: from mga04.intel.com (mga04.intel.com [192.55.52.120]) by dpdk.org (Postfix) with ESMTP id D635E1B11C for ; Fri, 28 Sep 2018 06:22:53 +0200 (CEST) X-Amp-Result: SKIPPED(no attachment in message) X-Amp-File-Uploaded: False Received: from fmsmga005.fm.intel.com ([10.253.24.32]) by fmsmga104.fm.intel.com with ESMTP/TLS/DHE-RSA-AES256-GCM-SHA384; 27 Sep 2018 21:22:52 -0700 X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="5.54,313,1534834800"; d="scan'208";a="266654788" Received: from dpdk51.sh.intel.com ([10.67.110.190]) by fmsmga005.fm.intel.com with ESMTP; 27 Sep 2018 21:22:30 -0700 From: Qi Zhang To: thomas@monjalon.net, gaetan.rivet@6wind.com, anatoly.burakov@intel.com, arybchenko@solarflare.com Cc: konstantin.ananyev@intel.com, dev@dpdk.org, bruce.richardson@intel.com, ferruh.yigit@intel.com, benjamin.h.shelton@intel.com, narender.vangati@intel.com, Qi Zhang Date: Fri, 28 Sep 2018 12:23:22 +0800 Message-Id: <20180928042326.50471-3-qi.z.zhang@intel.com> X-Mailer: git-send-email 2.13.6 In-Reply-To: <20180928042326.50471-1-qi.z.zhang@intel.com> References: <20180607123849.14439-1-qi.z.zhang@intel.com> <20180928042326.50471-1-qi.z.zhang@intel.com> Subject: [dpdk-dev] [PATCH v16 2/6] eal: enable hotplug on multi-process X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Sender: "dev" We are going to introduce the solution to handle hotplug in multi-process, it includes the below scenario: 1. Attach a device from the primary 2. Detach a device from the primary 3. Attach a device from a secondary 4. Detach a device from a secondary In the primary-secondary process model, we assume devices are shared by default. that means attaches or detaches a device on any process will broadcast to all other processes through mp channel then device information will be synchronized on all processes. Any failure during attaching/detaching process will cause inconsistent status between processes, so proper rollback action should be considered. This patch covers the implementation of case 1,2. Case 3,4 will be implemented on a separate patch. IPC scenario for Case 1, 2: attach a device a) primary attach the new device if failed goto h). b) primary send attach sync request to all secondary. c) secondary receive request and attach the device and send a reply. d) primary check the reply if all success goes to i). e) primary send attach rollback sync request to all secondary. f) secondary receive the request and detach the device and send a reply. g) primary receive the reply and detach device as rollback action. h) attach fail i) attach success detach a device a) primary send detach sync request to all secondary b) secondary detach the device and send reply c) primary check the reply if all success goes to f). d) primary send detach rollback sync request to all secondary. e) secondary receive the request and attach back device. goto g) f) primary detach the device if success goto g), else goto d) g) detach fail. h) detach success. Signed-off-by: Qi Zhang Acked-by: Anatoly Burakov --- doc/guides/rel_notes/release_18_11.rst | 11 ++ lib/librte_eal/bsdapp/eal/Makefile | 1 + lib/librte_eal/common/eal_common_dev.c | 225 ++++++++++++++++++++++++++++++-- lib/librte_eal/common/eal_private.h | 30 +++++ lib/librte_eal/common/hotplug_mp.c | 221 +++++++++++++++++++++++++++++++ lib/librte_eal/common/hotplug_mp.h | 46 +++++++ lib/librte_eal/common/include/rte_dev.h | 9 ++ lib/librte_eal/common/meson.build | 1 + lib/librte_eal/linuxapp/eal/Makefile | 1 + lib/librte_eal/linuxapp/eal/eal.c | 6 + 10 files changed, 542 insertions(+), 9 deletions(-) create mode 100644 lib/librte_eal/common/hotplug_mp.c create mode 100644 lib/librte_eal/common/hotplug_mp.h diff --git a/doc/guides/rel_notes/release_18_11.rst b/doc/guides/rel_notes/release_18_11.rst index bc9b74ec4..f88910c7f 100644 --- a/doc/guides/rel_notes/release_18_11.rst +++ b/doc/guides/rel_notes/release_18_11.rst @@ -67,6 +67,12 @@ New Features SR-IOV option in Hyper-V and Azure. This is an alternative to the previous vdev_netvsc, tap, and failsafe drivers combination. +* **Support device multi-process hotplug.** + + Hotplug and hot-unplug for devices will now be supported in multiprocessing + scenario. Any ethdev devices created in the primary process will be regarded + as shared and will be available for all DPDK processes. Synchronization + between processes will be done using DPDK IPC. API Changes ----------- @@ -91,6 +97,11 @@ API Changes flag the MAC can be properly configured in any case. This is particularly important for bonding. +* eal: scope of rte_eal_hotplug_add and rte_eal_hotplug_remove is extended. + + In primary-secondary process model, ``rte_eal_hotplug_add`` will guarantee + that device be attached on all processes, while ``rte_eal_hotplug_remove`` + will guarantee device be detached on all processes. ABI Changes ----------- diff --git a/lib/librte_eal/bsdapp/eal/Makefile b/lib/librte_eal/bsdapp/eal/Makefile index d27da3d15..4351c6a20 100644 --- a/lib/librte_eal/bsdapp/eal/Makefile +++ b/lib/librte_eal/bsdapp/eal/Makefile @@ -62,6 +62,7 @@ SRCS-$(CONFIG_RTE_EXEC_ENV_BSDAPP) += eal_common_proc.c SRCS-$(CONFIG_RTE_EXEC_ENV_BSDAPP) += eal_common_fbarray.c SRCS-$(CONFIG_RTE_EXEC_ENV_BSDAPP) += eal_common_uuid.c SRCS-$(CONFIG_RTE_EXEC_ENV_BSDAPP) += rte_malloc.c +SRCS-$(CONFIG_RTE_EXEC_ENV_BSDAPP) += hotplug_mp.c SRCS-$(CONFIG_RTE_EXEC_ENV_BSDAPP) += malloc_elem.c SRCS-$(CONFIG_RTE_EXEC_ENV_BSDAPP) += malloc_heap.c SRCS-$(CONFIG_RTE_EXEC_ENV_BSDAPP) += malloc_mp.c diff --git a/lib/librte_eal/common/eal_common_dev.c b/lib/librte_eal/common/eal_common_dev.c index 85eb1569f..314266041 100644 --- a/lib/librte_eal/common/eal_common_dev.c +++ b/lib/librte_eal/common/eal_common_dev.c @@ -19,8 +19,10 @@ #include #include #include +#include #include "eal_private.h" +#include "hotplug_mp.h" /** * The device event callback description. @@ -127,9 +129,10 @@ int rte_eal_dev_detach(struct rte_device *dev) return ret; } -int -rte_eal_hotplug_add(const char *busname, const char *devname, - const char *drvargs) +/* help funciton to build devargs, caller should free the memory */ +static char * +build_devargs(const char *busname, const char *devname, + const char *drvargs) { char *devargs = NULL; int size, length = -1; @@ -140,19 +143,33 @@ rte_eal_hotplug_add(const char *busname, const char *devname, if (length >= size) devargs = malloc(length + 1); if (devargs == NULL) - return -ENOMEM; + break; } while (size == 0); + return devargs; +} + +int +rte_eal_hotplug_add(const char *busname, const char *devname, + const char *drvargs) +{ + char *devargs = build_devargs(busname, devname, drvargs); + + if (devargs == NULL) + return -ENOMEM; + return rte_dev_probe(devargs); } -int __rte_experimental -rte_dev_probe(const char *devargs) +/* probe device at local process. */ +int +local_dev_probe(const char *devargs, struct rte_device **new_dev) { struct rte_device *dev; struct rte_devargs *da; int ret; + *new_dev = NULL; da = calloc(1, sizeof(*da)); if (da == NULL) return -ENOMEM; @@ -195,6 +212,8 @@ rte_dev_probe(const char *devargs) dev->name); goto err_devarg; } + + *new_dev = dev; return 0; err_devarg: @@ -226,8 +245,9 @@ rte_eal_hotplug_remove(const char *busname, const char *devname) return rte_dev_remove(dev); } -int __rte_experimental -rte_dev_remove(struct rte_device *dev) +/* remove device at local process. */ +int +local_dev_remove(struct rte_device *dev) { struct rte_bus *bus; int ret; @@ -248,7 +268,194 @@ rte_dev_remove(struct rte_device *dev) if (ret) RTE_LOG(ERR, EAL, "Driver cannot detach the device (%s)\n", dev->name); - rte_devargs_remove(dev->devargs); + else + rte_devargs_remove(dev->devargs); + + return ret; +} + +int __rte_experimental +rte_dev_probe(const char *devargs) +{ + struct eal_dev_mp_req req; + struct rte_device *dev; + int ret; + + memset(&req, 0, sizeof(req)); + req.t = EAL_DEV_REQ_TYPE_ATTACH; + strlcpy(req.devargs, devargs, EAL_DEV_MP_DEV_ARGS_MAX_LEN); + + if (rte_eal_process_type() != RTE_PROC_PRIMARY) { + /** + * If in secondary process, just send IPC request to + * primary process. + */ + ret = eal_dev_hotplug_request_to_primary(&req); + if (ret) { + RTE_LOG(ERR, EAL, + "Failed to send hotplug request to primary\n"); + return -ENOMSG; + } + if (req.result) + RTE_LOG(ERR, EAL, + "Failed to hotplug add device\n"); + return req.result; + } + + /* attach a shared device from primary start from here: */ + + /* primary attach the new device itself. */ + ret = local_dev_probe(devargs, &dev); + + if (ret) { + RTE_LOG(ERR, EAL, + "Failed to attach device on primary process\n"); + + /** + * it is possible that secondary process failed to attached a + * device that primary process have during initialization, + * so for -EEXIST case, we still need to sync with secondary + * process. + */ + if (ret != -EEXIST) + return ret; + } + + /* primary send attach sync request to secondary. */ + ret = eal_dev_hotplug_request_to_secondary(&req); + + /* if any commnunication error, we need to rollback. */ + if (ret) { + RTE_LOG(ERR, EAL, + "Failed to send hotplug add request to secondary\n"); + ret = -ENOMSG; + goto rollback; + } + + /** + * if any secondary failed to attach, we need to consider if rollback + * is necessary. + */ + if (req.result) { + RTE_LOG(ERR, EAL, + "Failed to attach device on secondary process\n"); + ret = req.result; + + /* for -EEXIST, we don't need to rollback. */ + if (ret == -EEXIST) + return ret; + goto rollback; + } + + return 0; + +rollback: + req.t = EAL_DEV_REQ_TYPE_ATTACH_ROLLBACK; + + /* primary send rollback request to secondary. */ + if (eal_dev_hotplug_request_to_secondary(&req)) + RTE_LOG(WARNING, EAL, + "Failed to rollback device attach on secondary." + "Devices in secondary may not sync with primary\n"); + + /* primary rollback itself. */ + if (local_dev_remove(dev)) + RTE_LOG(WARNING, EAL, + "Failed to rollback device attach on primary." + "Devices in secondary may not sync with primary\n"); + + return ret; +} + +int __rte_experimental +rte_dev_remove(struct rte_device *dev) +{ + struct eal_dev_mp_req req; + char *devargs; + int ret; + + devargs = build_devargs(dev->devargs->bus->name, dev->name, ""); + if (devargs == NULL) + return -ENOMEM; + + memset(&req, 0, sizeof(req)); + req.t = EAL_DEV_REQ_TYPE_DETACH; + strlcpy(req.devargs, devargs, EAL_DEV_MP_DEV_ARGS_MAX_LEN); + free(devargs); + + if (rte_eal_process_type() != RTE_PROC_PRIMARY) { + /** + * If in secondary process, just send IPC request to + * primary process. + */ + ret = eal_dev_hotplug_request_to_primary(&req); + if (ret) { + RTE_LOG(ERR, EAL, + "Failed to send hotplug request to primary\n"); + return -ENOMSG; + } + if (req.result) + RTE_LOG(ERR, EAL, + "Failed to hotplug remove device\n"); + return req.result; + } + + /* detach a device from primary start from here: */ + + /* primary send detach sync request to secondary */ + ret = eal_dev_hotplug_request_to_secondary(&req); + + /** + * if communication error, we need to rollback, because it is possible + * part of the secondary processes still detached it successfully. + */ + if (ret) { + RTE_LOG(ERR, EAL, + "Failed to send device detach request to secondary\n"); + ret = -ENOMSG; + goto rollback; + } + + /** + * if any secondary failed to detach, we need to consider if rollback + * is necessary. + */ + if (req.result) { + RTE_LOG(ERR, EAL, + "Failed to detach device on secondary process\n"); + ret = req.result; + /** + * if -ENOENT, we don't need to rollback, since devices is + * already detached on secondary process. + */ + if (ret != -ENOENT) + goto rollback; + } + + /* primary detach the device itself. */ + ret = local_dev_remove(dev); + + /* if primary failed, still need to consider if rollback is necessary */ + if (ret) { + RTE_LOG(ERR, EAL, + "Failed to detach device on primary process\n"); + /* if -ENOENT, we don't need to rollback */ + if (ret == -ENOENT) + return ret; + goto rollback; + } + + return 0; + +rollback: + req.t = EAL_DEV_REQ_TYPE_DETACH_ROLLBACK; + + /* primary send rollback request to secondary. */ + if (eal_dev_hotplug_request_to_secondary(&req)) + RTE_LOG(WARNING, EAL, + "Failed to rollback device detach on secondary." + "Devices in secondary may not sync with primary\n"); + return ret; } diff --git a/lib/librte_eal/common/eal_private.h b/lib/librte_eal/common/eal_private.h index 4f809a83c..83f10a9f8 100644 --- a/lib/librte_eal/common/eal_private.h +++ b/lib/librte_eal/common/eal_private.h @@ -304,4 +304,34 @@ int rte_devargs_layers_parse(struct rte_devargs *devargs, const char *devstr); +/* + * probe a device at local process. + * + * @param devargs + * Device arguments including bus, class and driver properties. + * @param new_dev + * new device be probed as output. + * @return + * 0 on success, negative on error. + */ +int local_dev_probe(const char *devargs, struct rte_device **new_dev); + +/** + * Hotplug remove a given device from a specific bus at local process. + * + * @param dev + * Data structure of the device to remove. + * @return + * 0 on success, negative on error. + */ +int local_dev_remove(struct rte_device *dev); + +/** + * Register all mp action callbacks for hotplug. + * + * @return + * 0 on success, negative on error. + */ +int rte_dev_hotplug_mp_init(void); + #endif /* _EAL_PRIVATE_H_ */ diff --git a/lib/librte_eal/common/hotplug_mp.c b/lib/librte_eal/common/hotplug_mp.c new file mode 100644 index 000000000..1c92e44cb --- /dev/null +++ b/lib/librte_eal/common/hotplug_mp.c @@ -0,0 +1,221 @@ +/* SPDX-License-Identifier: BSD-3-Clause + * Copyright(c) 2018 Intel Corporation + */ +#include + +#include +#include +#include +#include + +#include "hotplug_mp.h" +#include "eal_private.h" + +#define MP_TIMEOUT_S 5 /**< 5 seconds timeouts */ + +static int cmp_dev_name(const struct rte_device *dev, const void *_name) +{ + const char *name = _name; + + return strcmp(dev->name, name); +} + +struct mp_reply_bundle { + struct rte_mp_msg msg; + void *peer; +}; + +static int +handle_secondary_request(const struct rte_mp_msg *msg, const void *peer) +{ + RTE_SET_USED(msg); + RTE_SET_USED(peer); + return -ENOTSUP; +} + +static void __handle_primary_request(void *param) +{ + struct mp_reply_bundle *bundle = param; + struct rte_mp_msg *msg = &bundle->msg; + const struct eal_dev_mp_req *req = + (const struct eal_dev_mp_req *)msg->param; + struct rte_mp_msg mp_resp; + struct eal_dev_mp_req *resp = + (struct eal_dev_mp_req *)mp_resp.param; + struct rte_devargs *da; + struct rte_device *dev; + struct rte_bus *bus; + int ret = 0; + + memset(&mp_resp, 0, sizeof(mp_resp)); + + switch (req->t) { + case EAL_DEV_REQ_TYPE_ATTACH: + case EAL_DEV_REQ_TYPE_DETACH_ROLLBACK: + ret = local_dev_probe(req->devargs, &dev); + break; + case EAL_DEV_REQ_TYPE_DETACH: + case EAL_DEV_REQ_TYPE_ATTACH_ROLLBACK: + da = calloc(1, sizeof(*da)); + if (da == NULL) { + ret = -ENOMEM; + goto quit; + } + + ret = rte_devargs_parse(da, req->devargs); + if (ret) + goto quit; + + bus = rte_bus_find_by_name(da->bus->name); + if (bus == NULL) { + RTE_LOG(ERR, EAL, "Cannot find bus (%s)\n", da->bus->name); + ret = -ENOENT; + goto quit; + } + + dev = bus->find_device(NULL, cmp_dev_name, da->name); + if (dev == NULL) { + RTE_LOG(ERR, EAL, "Cannot find plugged device (%s)\n", da->name); + ret = -ENOENT; + goto quit; + } + + ret = local_dev_remove(dev); +quit: + break; + default: + ret = -EINVAL; + } + + strlcpy(mp_resp.name, EAL_DEV_MP_ACTION_REQUEST, sizeof(mp_resp.name)); + mp_resp.len_param = sizeof(*req); + memcpy(resp, req, sizeof(*resp)); + resp->result = ret; + if (rte_mp_reply(&mp_resp, bundle->peer) < 0) + RTE_LOG(ERR, EAL, "failed to send reply to primary request\n"); + + free(bundle->peer); + free(bundle); +} + +static int +handle_primary_request(const struct rte_mp_msg *msg, const void *peer) +{ + struct rte_mp_msg mp_resp; + const struct eal_dev_mp_req *req = + (const struct eal_dev_mp_req *)msg->param; + struct eal_dev_mp_req *resp = + (struct eal_dev_mp_req *)mp_resp.param; + struct mp_reply_bundle *bundle; + int ret = 0; + + memset(&mp_resp, 0, sizeof(mp_resp)); + strlcpy(mp_resp.name, EAL_DEV_MP_ACTION_REQUEST, sizeof(mp_resp.name)); + mp_resp.len_param = sizeof(*req); + memcpy(resp, req, sizeof(*resp)); + + bundle = calloc(1, sizeof(*bundle)); + if (bundle == NULL) { + resp->result = -ENOMEM; + ret = rte_mp_reply(&mp_resp, peer); + if (ret) + RTE_LOG(ERR, EAL, "failed to send reply to primary request\n"); + return ret; + } + + bundle->msg = *msg; + /** + * We need to send reply on interrupt thread, but peer can't be + * parsed directly, so this is a temporal hack, need to be fixed + * when it is ready. + */ + bundle->peer = (void *)strdup(peer); + + /** + * We are at IPC callback thread, sync IPC is not allowed due to + * dead lock, so we delegate the task to interrupt thread. + */ + ret = rte_eal_alarm_set(1, __handle_primary_request, bundle); + if (ret) { + resp->result = ret; + ret = rte_mp_reply(&mp_resp, peer); + if (ret) { + RTE_LOG(ERR, EAL, "failed to send reply to primary request\n"); + return ret; + } + } + return 0; +} + +int eal_dev_hotplug_request_to_primary(struct eal_dev_mp_req *req) +{ + RTE_SET_USED(req); + return -ENOTSUP; +} + +int eal_dev_hotplug_request_to_secondary(struct eal_dev_mp_req *req) +{ + struct rte_mp_msg mp_req; + struct rte_mp_reply mp_reply; + struct timespec ts = {.tv_sec = MP_TIMEOUT_S, .tv_nsec = 0}; + int ret; + int i; + + memset(&mp_req, 0, sizeof(mp_req)); + memcpy(mp_req.param, req, sizeof(*req)); + mp_req.len_param = sizeof(*req); + strlcpy(mp_req.name, EAL_DEV_MP_ACTION_REQUEST, sizeof(mp_req.name)); + + ret = rte_mp_request_sync(&mp_req, &mp_reply, &ts); + if (ret) { + RTE_LOG(ERR, EAL, "rte_mp_request_sync failed\n"); + return ret; + } + + if (mp_reply.nb_sent != mp_reply.nb_received) { + RTE_LOG(ERR, EAL, "not all secondary reply\n"); + return -1; + } + + req->result = 0; + for (i = 0; i < mp_reply.nb_received; i++) { + struct eal_dev_mp_req *resp = + (struct eal_dev_mp_req *)mp_reply.msgs[i].param; + if (resp->result) { + req->result = resp->result; + if (req->t == EAL_DEV_REQ_TYPE_ATTACH && + req->result != -EEXIST) + break; + if (req->t == EAL_DEV_REQ_TYPE_DETACH && + req->result != -ENOENT) + break; + } + } + + return 0; +} + +int rte_dev_hotplug_mp_init(void) +{ + int ret; + + if (rte_eal_process_type() == RTE_PROC_PRIMARY) { + ret = rte_mp_action_register(EAL_DEV_MP_ACTION_REQUEST, + handle_secondary_request); + if (ret) { + RTE_LOG(ERR, EAL, "Couldn't register '%s' action\n", + EAL_DEV_MP_ACTION_REQUEST); + return ret; + } + } else { + ret = rte_mp_action_register(EAL_DEV_MP_ACTION_REQUEST, + handle_primary_request); + if (ret) { + RTE_LOG(ERR, EAL, "Couldn't register '%s' action\n", + EAL_DEV_MP_ACTION_REQUEST); + return ret; + } + } + + return 0; +} diff --git a/lib/librte_eal/common/hotplug_mp.h b/lib/librte_eal/common/hotplug_mp.h new file mode 100644 index 000000000..c95c8f1fb --- /dev/null +++ b/lib/librte_eal/common/hotplug_mp.h @@ -0,0 +1,46 @@ +/* SPDX-License-Identifier: BSD-3-Clause + * Copyright(c) 2018 Intel Corporation + */ + +#ifndef _HOTPLUG_MP_H_ +#define _HOTPLUG_MP_H_ + +#include +#include + +#define EAL_DEV_MP_ACTION_REQUEST "eal_dev_mp_request" +#define EAL_DEV_MP_ACTION_RESPONSE "eal_dev_mp_response" + +#define EAL_DEV_MP_DEV_NAME_MAX_LEN RTE_DEV_NAME_MAX_LEN +#define EAL_DEV_MP_BUS_NAME_MAX_LEN 32 +#define EAL_DEV_MP_DEV_ARGS_MAX_LEN 128 + +enum eal_dev_req_type { + EAL_DEV_REQ_TYPE_ATTACH, + EAL_DEV_REQ_TYPE_DETACH, + EAL_DEV_REQ_TYPE_ATTACH_ROLLBACK, + EAL_DEV_REQ_TYPE_DETACH_ROLLBACK, +}; + +struct eal_dev_mp_req { + enum eal_dev_req_type t; + char devargs[EAL_DEV_MP_DEV_ARGS_MAX_LEN]; + int result; +}; + +/** + * this is a synchronous wrapper for secondary process send + * request to primary process, this is invoked when an attach + * or detach request issued from primary process. + */ +int eal_dev_hotplug_request_to_primary(struct eal_dev_mp_req *req); + +/** + * this is a synchronous wrapper for primary process send + * request to secondary process, this is invoked when an attach + * or detach request issued from secondary process. + */ +int eal_dev_hotplug_request_to_secondary(struct eal_dev_mp_req *req); + + +#endif /* _HOTPLUG_MP_H_ */ diff --git a/lib/librte_eal/common/include/rte_dev.h b/lib/librte_eal/common/include/rte_dev.h index 7a30362c0..266331acd 100644 --- a/lib/librte_eal/common/include/rte_dev.h +++ b/lib/librte_eal/common/include/rte_dev.h @@ -190,6 +190,9 @@ int rte_eal_dev_detach(struct rte_device *dev); /** * Hotplug add a given device to a specific bus. + * In multi-process, this function will inform all other processes + * to hotplug add the same device. Any failure on other process rollback + * the action. * * @param busname * The bus name the device is added to. @@ -219,6 +222,9 @@ int __rte_experimental rte_dev_probe(const char *devargs); /** * Hotplug remove a given device from a specific bus. + * In multi-process, this function will inform all other processes + * to hotplug remove the same device. Any failure on other process + * will rollback the action. * * @param busname * The bus name the device is removed from. @@ -234,6 +240,9 @@ int rte_eal_hotplug_remove(const char *busname, const char *devname); * @b EXPERIMENTAL: this API may change without prior notice * * Remove one device. + * In multi-process, this function will inform all other processes + * to hotplug remove the same device. Any failure on other process + * will rollback the action. * * @param dev * Data structure of the device to remove. diff --git a/lib/librte_eal/common/meson.build b/lib/librte_eal/common/meson.build index b7fc98499..04c414356 100644 --- a/lib/librte_eal/common/meson.build +++ b/lib/librte_eal/common/meson.build @@ -28,6 +28,7 @@ common_sources = files( 'eal_common_thread.c', 'eal_common_timer.c', 'eal_common_uuid.c', + 'hotplug_mp.c', 'malloc_elem.c', 'malloc_heap.c', 'malloc_mp.c', diff --git a/lib/librte_eal/linuxapp/eal/Makefile b/lib/librte_eal/linuxapp/eal/Makefile index fd92c75c2..58455c1a6 100644 --- a/lib/librte_eal/linuxapp/eal/Makefile +++ b/lib/librte_eal/linuxapp/eal/Makefile @@ -70,6 +70,7 @@ SRCS-$(CONFIG_RTE_EXEC_ENV_LINUXAPP) += eal_common_proc.c SRCS-$(CONFIG_RTE_EXEC_ENV_LINUXAPP) += eal_common_fbarray.c SRCS-$(CONFIG_RTE_EXEC_ENV_LINUXAPP) += eal_common_uuid.c SRCS-$(CONFIG_RTE_EXEC_ENV_LINUXAPP) += rte_malloc.c +SRCS-$(CONFIG_RTE_EXEC_ENV_LINUXAPP) += hotplug_mp.c SRCS-$(CONFIG_RTE_EXEC_ENV_LINUXAPP) += malloc_elem.c SRCS-$(CONFIG_RTE_EXEC_ENV_LINUXAPP) += malloc_heap.c SRCS-$(CONFIG_RTE_EXEC_ENV_LINUXAPP) += malloc_mp.c diff --git a/lib/librte_eal/linuxapp/eal/eal.c b/lib/librte_eal/linuxapp/eal/eal.c index e59ac6577..f2c90c528 100644 --- a/lib/librte_eal/linuxapp/eal/eal.c +++ b/lib/librte_eal/linuxapp/eal/eal.c @@ -865,6 +865,12 @@ rte_eal_init(int argc, char **argv) } } + /* register mp action callbacks for hotplug */ + if (rte_dev_hotplug_mp_init() < 0) { + rte_eal_init_alert("failed to register mp callback for hotplug\n"); + return -1; + } + if (rte_bus_scan()) { rte_eal_init_alert("Cannot scan the buses for devices\n"); rte_errno = ENODEV; From patchwork Fri Sep 28 04:23:23 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Qi Zhang X-Patchwork-Id: 45552 X-Patchwork-Delegate: thomas@monjalon.net Return-Path: X-Original-To: patchwork@dpdk.org Delivered-To: patchwork@dpdk.org Received: from [92.243.14.124] (localhost [127.0.0.1]) by dpdk.org (Postfix) with ESMTP id 58E361B150; Fri, 28 Sep 2018 06:23:00 +0200 (CEST) Received: from mga04.intel.com (mga04.intel.com [192.55.52.120]) by dpdk.org (Postfix) with ESMTP id C52C81B11C for ; Fri, 28 Sep 2018 06:22:54 +0200 (CEST) X-Amp-Result: SKIPPED(no attachment in message) X-Amp-File-Uploaded: False Received: from fmsmga005.fm.intel.com ([10.253.24.32]) by fmsmga104.fm.intel.com with ESMTP/TLS/DHE-RSA-AES256-GCM-SHA384; 27 Sep 2018 21:22:52 -0700 X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="5.54,313,1534834800"; d="scan'208";a="266654793" Received: from dpdk51.sh.intel.com ([10.67.110.190]) by fmsmga005.fm.intel.com with ESMTP; 27 Sep 2018 21:22:32 -0700 From: Qi Zhang To: thomas@monjalon.net, gaetan.rivet@6wind.com, anatoly.burakov@intel.com, arybchenko@solarflare.com Cc: konstantin.ananyev@intel.com, dev@dpdk.org, bruce.richardson@intel.com, ferruh.yigit@intel.com, benjamin.h.shelton@intel.com, narender.vangati@intel.com, Qi Zhang Date: Fri, 28 Sep 2018 12:23:23 +0800 Message-Id: <20180928042326.50471-4-qi.z.zhang@intel.com> X-Mailer: git-send-email 2.13.6 In-Reply-To: <20180928042326.50471-1-qi.z.zhang@intel.com> References: <20180607123849.14439-1-qi.z.zhang@intel.com> <20180928042326.50471-1-qi.z.zhang@intel.com> Subject: [dpdk-dev] [PATCH v16 3/6] eal: support attach or detach share device from secondary X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Sender: "dev" This patch cover the multi-process hotplug case when a device attach/detach request be issued from a secondary process device attach on secondary: a) secondary send sync request to the primary. b) primary receive the request and attach the new device if failed goto i). c) primary forward attach sync request to all secondary. d) secondary receive the request and attach the device and send a reply. e) primary check the reply if all success goes to j). f) primary send attach rollback sync request to all secondary. g) secondary receive the request and detach the device and send a reply. h) primary receive the reply and detach device as rollback action. i) send attach fail to secondary as a reply of step a), goto k). j) send attach success to secondary as a reply of step a). k) secondary receive reply and return. device detach on secondary: a) secondary send sync request to the primary. b) primary send detach sync request to all secondary. c) secondary detach the device and send a reply. d) primary check the reply if all success goes to g). e) primary send detach rollback sync request to all secondary. f) secondary receive the request and attach back device. goto h). g) primary detach the device if success goto i), else goto e). h) primary send detach fail to secondary as a reply of step a), goto j). i) primary send detach success to secondary as a reply of step a). j) secondary receive reply and return. Signed-off-by: Qi Zhang Reviewed-by: Anatoly Burakov --- lib/librte_eal/common/hotplug_mp.c | 223 +++++++++++++++++++++++++++++++++++-- 1 file changed, 214 insertions(+), 9 deletions(-) diff --git a/lib/librte_eal/common/hotplug_mp.c b/lib/librte_eal/common/hotplug_mp.c index 1c92e44cb..c31316bf7 100644 --- a/lib/librte_eal/common/hotplug_mp.c +++ b/lib/librte_eal/common/hotplug_mp.c @@ -13,6 +13,11 @@ #define MP_TIMEOUT_S 5 /**< 5 seconds timeouts */ +struct mp_reply_bundle { + struct rte_mp_msg msg; + void *peer; +}; + static int cmp_dev_name(const struct rte_device *dev, const void *_name) { const char *name = _name; @@ -20,17 +25,196 @@ static int cmp_dev_name(const struct rte_device *dev, const void *_name) return strcmp(dev->name, name); } -struct mp_reply_bundle { - struct rte_mp_msg msg; - void *peer; -}; +/** + * Secondary to primary request. + * start from function eal_dev_hotplug_request_to_primary. + * + * device attach on secondary: + * a) secondary send sync request to the primary. + * b) primary receive the request and attach the new device if + * failed goto i). + * c) primary forward attach sync request to all secondary. + * d) secondary receive the request and attach the device and send a reply. + * e) primary check the reply if all success goes to j). + * f) primary send attach rollback sync request to all secondary. + * g) secondary receive the request and detach the device and send a reply. + * h) primary receive the reply and detach device as rollback action. + * i) send attach fail to secondary as a reply of step a), goto k). + * j) send attach success to secondary as a reply of step a). + * k) secondary receive reply and return. + * + * device detach on secondary: + * a) secondary send sync request to the primary. + * b) primary send detach sync request to all secondary. + * c) secondary detach the device and send a reply. + * d) primary check the reply if all success goes to g). + * e) primary send detach rollback sync request to all secondary. + * f) secondary receive the request and attach back device. goto h). + * g) primary detach the device if success goto i), else goto e). + * h) primary send detach fail to secondary as a reply of step a), goto j). + * i) primary send detach success to secondary as a reply of step a). + * j) secondary receive reply and return. + */ + +static int +send_response_to_secondary(const struct eal_dev_mp_req *req, + int result, + const void *peer) +{ + struct rte_mp_msg mp_resp; + struct eal_dev_mp_req *resp = + (struct eal_dev_mp_req *)mp_resp.param; + int ret; + + memset(&mp_resp, 0, sizeof(mp_resp)); + mp_resp.len_param = sizeof(*resp); + strlcpy(mp_resp.name, EAL_DEV_MP_ACTION_REQUEST, sizeof(mp_resp.name)); + memcpy(resp, req, sizeof(*req)); + resp->result = result; + + ret = rte_mp_reply(&mp_resp, peer); + if (ret) + RTE_LOG(ERR, EAL, "failed to send response to secondary\n"); + + return ret; +} + +static void +__handle_secondary_request(void *param) +{ + struct mp_reply_bundle *bundle = param; + const struct rte_mp_msg *msg = &bundle->msg; + const struct eal_dev_mp_req *req = + (const struct eal_dev_mp_req *)msg->param; + struct eal_dev_mp_req tmp_req; + struct rte_devargs *da; + struct rte_device *dev; + struct rte_bus *bus; + int ret = 0; + + tmp_req = *req; + + if (req->t == EAL_DEV_REQ_TYPE_ATTACH) { + ret = local_dev_probe(req->devargs, &dev); + if (ret) { + RTE_LOG(ERR, EAL, "Failed to hotplug add device on primary\n"); + if (ret != -EEXIST) + goto finish; + } + ret = eal_dev_hotplug_request_to_secondary(&tmp_req); + if (ret) { + RTE_LOG(ERR, EAL, "Failed to send hotplug request to secondary\n"); + ret = -ENOMSG; + goto rollback; + } + if (tmp_req.result) { + ret = tmp_req.result; + RTE_LOG(ERR, EAL, "Failed to hotplug add device on secondary\n"); + if (ret != -EEXIST) + goto rollback; + } + } else if (req->t == EAL_DEV_REQ_TYPE_DETACH) { + da = calloc(1, sizeof(*da)); + if (da == NULL) { + ret = -ENOMEM; + goto finish; + } + + ret = rte_devargs_parse(da, req->devargs); + if (ret) + goto finish; + + ret = eal_dev_hotplug_request_to_secondary(&tmp_req); + if (ret) { + RTE_LOG(ERR, EAL, "Failed to send hotplug request to secondary\n"); + ret = -ENOMSG; + goto rollback; + } + + bus = rte_bus_find_by_name(da->bus->name); + if (bus == NULL) { + RTE_LOG(ERR, EAL, "Cannot find bus (%s)\n", da->bus->name); + ret = -ENOENT; + goto finish; + } + + dev = bus->find_device(NULL, cmp_dev_name, da->name); + if (dev == NULL) { + RTE_LOG(ERR, EAL, "Cannot find plugged device (%s)\n", da->name); + ret = -ENOENT; + goto finish; + } + + if (tmp_req.result) { + RTE_LOG(ERR, EAL, "Failed to hotplug remove device on secondary\n"); + ret = tmp_req.result; + if (ret != -ENOENT) + goto rollback; + } + + ret = local_dev_remove(dev); + if (ret) { + RTE_LOG(ERR, EAL, "Failed to hotplug remove device on primary\n"); + if (ret != -ENOENT) + goto rollback; + } + } else { + RTE_LOG(ERR, EAL, "unsupported secondary to primary request\n"); + ret = -ENOTSUP; + } + goto finish; + +rollback: + if (req->t == EAL_DEV_REQ_TYPE_ATTACH) { + tmp_req.t = EAL_DEV_REQ_TYPE_ATTACH_ROLLBACK; + eal_dev_hotplug_request_to_secondary(&tmp_req); + local_dev_remove(dev); + } else { + tmp_req.t = EAL_DEV_REQ_TYPE_DETACH_ROLLBACK; + eal_dev_hotplug_request_to_secondary(&tmp_req); + } + +finish: + ret = send_response_to_secondary(&tmp_req, ret, bundle->peer); + if (ret) + RTE_LOG(ERR, EAL, "failed to send response to secondary\n"); + + free(bundle->peer); + free(bundle); +} static int handle_secondary_request(const struct rte_mp_msg *msg, const void *peer) { - RTE_SET_USED(msg); - RTE_SET_USED(peer); - return -ENOTSUP; + struct mp_reply_bundle *bundle; + const struct eal_dev_mp_req *req = + (const struct eal_dev_mp_req *)msg->param; + int ret = 0; + + bundle = malloc(sizeof(*bundle)); + if (bundle == NULL) { + RTE_LOG(ERR, EAL, "not enough memory\n"); + return send_response_to_secondary(req, -ENOMEM, peer); + } + + bundle->msg = *msg; + /** + * We need to send reply on interrupt thread, but peer can't be + * parsed directly, so this is a temporal hack, need to be fixed + * when it is ready. + */ + bundle->peer = strdup(peer); + + /** + * We are at IPC callback thread, sync IPC is not allowed due to + * dead lock, so we delegate the task to interrupt thread. + */ + ret = rte_eal_alarm_set(1, __handle_secondary_request, bundle); + if (ret) { + RTE_LOG(ERR, EAL, "failed to add mp task\n"); + return send_response_to_secondary(req, ret, peer); + } + return 0; } static void __handle_primary_request(void *param) @@ -149,8 +333,29 @@ handle_primary_request(const struct rte_mp_msg *msg, const void *peer) int eal_dev_hotplug_request_to_primary(struct eal_dev_mp_req *req) { - RTE_SET_USED(req); - return -ENOTSUP; + struct rte_mp_msg mp_req; + struct rte_mp_reply mp_reply; + struct timespec ts = {.tv_sec = MP_TIMEOUT_S, .tv_nsec = 0}; + struct eal_dev_mp_req *resp; + int ret; + + memset(&mp_req, 0, sizeof(mp_req)); + memcpy(mp_req.param, req, sizeof(*req)); + mp_req.len_param = sizeof(*req); + strlcpy(mp_req.name, EAL_DEV_MP_ACTION_REQUEST, sizeof(mp_req.name)); + + ret = rte_mp_request_sync(&mp_req, &mp_reply, &ts); + if (ret || mp_reply.nb_received != 1) { + RTE_LOG(ERR, EAL, "cannot send request to primary"); + if (!ret) + return -1; + return ret; + } + + resp = (struct eal_dev_mp_req *)mp_reply.msgs[0].param; + req->result = resp->result; + + return ret; } int eal_dev_hotplug_request_to_secondary(struct eal_dev_mp_req *req) From patchwork Fri Sep 28 04:23:24 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Qi Zhang X-Patchwork-Id: 45554 X-Patchwork-Delegate: thomas@monjalon.net Return-Path: X-Original-To: patchwork@dpdk.org Delivered-To: patchwork@dpdk.org Received: from [92.243.14.124] (localhost [127.0.0.1]) by dpdk.org (Postfix) with ESMTP id 11F7D1B19D; Fri, 28 Sep 2018 06:23:04 +0200 (CEST) Received: from mga04.intel.com (mga04.intel.com [192.55.52.120]) by dpdk.org (Postfix) with ESMTP id 68A3B1B11C for ; Fri, 28 Sep 2018 06:22:55 +0200 (CEST) X-Amp-Result: SKIPPED(no attachment in message) X-Amp-File-Uploaded: False Received: from fmsmga005.fm.intel.com ([10.253.24.32]) by fmsmga104.fm.intel.com with ESMTP/TLS/DHE-RSA-AES256-GCM-SHA384; 27 Sep 2018 21:22:53 -0700 X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="5.54,313,1534834800"; d="scan'208";a="266654798" Received: from dpdk51.sh.intel.com ([10.67.110.190]) by fmsmga005.fm.intel.com with ESMTP; 27 Sep 2018 21:22:34 -0700 From: Qi Zhang To: thomas@monjalon.net, gaetan.rivet@6wind.com, anatoly.burakov@intel.com, arybchenko@solarflare.com Cc: konstantin.ananyev@intel.com, dev@dpdk.org, bruce.richardson@intel.com, ferruh.yigit@intel.com, benjamin.h.shelton@intel.com, narender.vangati@intel.com, Qi Zhang Date: Fri, 28 Sep 2018 12:23:24 +0800 Message-Id: <20180928042326.50471-5-qi.z.zhang@intel.com> X-Mailer: git-send-email 2.13.6 In-Reply-To: <20180928042326.50471-1-qi.z.zhang@intel.com> References: <20180607123849.14439-1-qi.z.zhang@intel.com> <20180928042326.50471-1-qi.z.zhang@intel.com> Subject: [dpdk-dev] [PATCH v16 4/6] drivers/net: enable hotplug on secondary process X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Sender: "dev" Attach port from secondary should ignore devargs since the private device is not necessary to support. Also previously, detach port on a secondary process will mess primary process and cause the same device can't be attached back again. A secondary process should use rte_eth_dev_release_port_secondary to release a port. Signed-off-by: Qi Zhang --- drivers/net/af_packet/rte_eth_af_packet.c | 6 ++++-- drivers/net/bonding/rte_eth_bond_pmd.c | 6 ++++-- drivers/net/kni/rte_eth_kni.c | 6 ++++-- drivers/net/null/rte_eth_null.c | 6 ++++-- drivers/net/octeontx/octeontx_ethdev.c | 8 ++++++++ drivers/net/pcap/rte_eth_pcap.c | 6 ++++-- drivers/net/tap/rte_eth_tap.c | 8 +++++--- drivers/net/vhost/rte_eth_vhost.c | 6 ++++-- 8 files changed, 37 insertions(+), 15 deletions(-) diff --git a/drivers/net/af_packet/rte_eth_af_packet.c b/drivers/net/af_packet/rte_eth_af_packet.c index bc7daed5e..376d76302 100644 --- a/drivers/net/af_packet/rte_eth_af_packet.c +++ b/drivers/net/af_packet/rte_eth_af_packet.c @@ -926,8 +926,7 @@ rte_pmd_af_packet_probe(struct rte_vdev_device *dev) PMD_LOG(INFO, "Initializing pmd_af_packet for %s", name); - if (rte_eal_process_type() == RTE_PROC_SECONDARY && - strlen(rte_vdev_device_args(dev)) == 0) { + if (rte_eal_process_type() == RTE_PROC_SECONDARY) { eth_dev = rte_eth_dev_attach_secondary(name); if (!eth_dev) { PMD_LOG(ERR, "Failed to probe %s", name); @@ -987,6 +986,9 @@ rte_pmd_af_packet_remove(struct rte_vdev_device *dev) if (eth_dev == NULL) return -1; + if (rte_eal_process_type() != RTE_PROC_PRIMARY) + return rte_eth_dev_release_port_secondary(eth_dev); + internals = eth_dev->data->dev_private; for (q = 0; q < internals->nb_queues; q++) { rte_free(internals->rx_queue[q].rd); diff --git a/drivers/net/bonding/rte_eth_bond_pmd.c b/drivers/net/bonding/rte_eth_bond_pmd.c index 7814258f2..94eed800a 100644 --- a/drivers/net/bonding/rte_eth_bond_pmd.c +++ b/drivers/net/bonding/rte_eth_bond_pmd.c @@ -3116,8 +3116,7 @@ bond_probe(struct rte_vdev_device *dev) name = rte_vdev_device_name(dev); RTE_BOND_LOG(INFO, "Initializing pmd_bond for %s", name); - if (rte_eal_process_type() == RTE_PROC_SECONDARY && - strlen(rte_vdev_device_args(dev)) == 0) { + if (rte_eal_process_type() == RTE_PROC_SECONDARY) { eth_dev = rte_eth_dev_attach_secondary(name); if (!eth_dev) { RTE_BOND_LOG(ERR, "Failed to probe %s", name); @@ -3232,6 +3231,9 @@ bond_remove(struct rte_vdev_device *dev) if (eth_dev == NULL) return -ENODEV; + if (rte_eal_process_type() != RTE_PROC_PRIMARY) + return rte_eth_dev_release_port_secondary(eth_dev); + RTE_ASSERT(eth_dev->device == &dev->device); internals = eth_dev->data->dev_private; diff --git a/drivers/net/kni/rte_eth_kni.c b/drivers/net/kni/rte_eth_kni.c index 8a7015a0b..72f3c16c1 100644 --- a/drivers/net/kni/rte_eth_kni.c +++ b/drivers/net/kni/rte_eth_kni.c @@ -410,8 +410,7 @@ eth_kni_probe(struct rte_vdev_device *vdev) params = rte_vdev_device_args(vdev); PMD_LOG(INFO, "Initializing eth_kni for %s", name); - if (rte_eal_process_type() == RTE_PROC_SECONDARY && - strlen(params) == 0) { + if (rte_eal_process_type() == RTE_PROC_SECONDARY) { eth_dev = rte_eth_dev_attach_secondary(name); if (!eth_dev) { PMD_LOG(ERR, "Failed to probe %s", name); @@ -464,6 +463,9 @@ eth_kni_remove(struct rte_vdev_device *vdev) if (eth_dev == NULL) return -1; + if (rte_eal_process_type() != RTE_PROC_PRIMARY) + return rte_eth_dev_release_port_secondary(eth_dev); + eth_kni_dev_stop(eth_dev); internals = eth_dev->data->dev_private; diff --git a/drivers/net/null/rte_eth_null.c b/drivers/net/null/rte_eth_null.c index de10b5bdf..1e8237a41 100644 --- a/drivers/net/null/rte_eth_null.c +++ b/drivers/net/null/rte_eth_null.c @@ -614,8 +614,7 @@ rte_pmd_null_probe(struct rte_vdev_device *dev) params = rte_vdev_device_args(dev); PMD_LOG(INFO, "Initializing pmd_null for %s", name); - if (rte_eal_process_type() == RTE_PROC_SECONDARY && - strlen(params) == 0) { + if (rte_eal_process_type() == RTE_PROC_SECONDARY) { eth_dev = rte_eth_dev_attach_secondary(name); if (!eth_dev) { PMD_LOG(ERR, "Failed to probe %s", name); @@ -680,6 +679,9 @@ rte_pmd_null_remove(struct rte_vdev_device *dev) if (eth_dev == NULL) return -1; + if (rte_eal_process_type() != RTE_PROC_PRIMARY) + return rte_eth_dev_release_port_secondary(eth_dev); + rte_free(eth_dev->data->dev_private); rte_eth_dev_release_port(eth_dev); diff --git a/drivers/net/octeontx/octeontx_ethdev.c b/drivers/net/octeontx/octeontx_ethdev.c index 71843c63a..5431b44dc 100644 --- a/drivers/net/octeontx/octeontx_ethdev.c +++ b/drivers/net/octeontx/octeontx_ethdev.c @@ -1133,6 +1133,11 @@ octeontx_remove(struct rte_vdev_device *dev) if (eth_dev == NULL) return -ENODEV; + if (rte_eal_process_type() != RTE_PROC_PRIMARY) { + rte_eth_dev_release_port_secondary(eth_dev); + continue; + } + nic = octeontx_pmd_priv(eth_dev); rte_event_dev_stop(nic->evdev); PMD_INIT_LOG(INFO, "Closing octeontx device %s", octtx_name); @@ -1143,6 +1148,9 @@ octeontx_remove(struct rte_vdev_device *dev) rte_event_dev_close(nic->evdev); } + if (rte_eal_process_type() != RTE_PROC_PRIMARY) + return 0; + /* Free FC resource */ octeontx_pko_fc_free(); diff --git a/drivers/net/pcap/rte_eth_pcap.c b/drivers/net/pcap/rte_eth_pcap.c index a015a9d48..8146602a7 100644 --- a/drivers/net/pcap/rte_eth_pcap.c +++ b/drivers/net/pcap/rte_eth_pcap.c @@ -1007,8 +1007,7 @@ pmd_pcap_probe(struct rte_vdev_device *dev) start_cycles = rte_get_timer_cycles(); hz = rte_get_timer_hz(); - if (rte_eal_process_type() == RTE_PROC_SECONDARY && - strlen(rte_vdev_device_args(dev)) == 0) { + if (rte_eal_process_type() == RTE_PROC_SECONDARY) { eth_dev = rte_eth_dev_attach_secondary(name); if (!eth_dev) { PMD_LOG(ERR, "Failed to probe %s", name); @@ -1107,6 +1106,9 @@ pmd_pcap_remove(struct rte_vdev_device *dev) if (eth_dev == NULL) return -1; + if (rte_eal_process_type() != RTE_PROC_PRIMARY) + return rte_eth_dev_release_port_secondary(eth_dev); + rte_free(eth_dev->data->dev_private); rte_eth_dev_release_port(eth_dev); diff --git a/drivers/net/tap/rte_eth_tap.c b/drivers/net/tap/rte_eth_tap.c index ad5ae988b..84aaf2410 100644 --- a/drivers/net/tap/rte_eth_tap.c +++ b/drivers/net/tap/rte_eth_tap.c @@ -1992,8 +1992,7 @@ rte_pmd_tap_probe(struct rte_vdev_device *dev) name = rte_vdev_device_name(dev); params = rte_vdev_device_args(dev); - if (rte_eal_process_type() == RTE_PROC_SECONDARY && - strlen(params) == 0) { + if (rte_eal_process_type() == RTE_PROC_SECONDARY) { eth_dev = rte_eth_dev_attach_secondary(name); if (!eth_dev) { TAP_LOG(ERR, "Failed to probe %s", name); @@ -2075,7 +2074,10 @@ rte_pmd_tap_remove(struct rte_vdev_device *dev) /* find the ethdev entry */ eth_dev = rte_eth_dev_allocated(rte_vdev_device_name(dev)); if (!eth_dev) - return 0; + return -ENODEV; + + if (rte_eal_process_type() != RTE_PROC_PRIMARY) + return rte_eth_dev_release_port_secondary(eth_dev); internals = eth_dev->data->dev_private; diff --git a/drivers/net/vhost/rte_eth_vhost.c b/drivers/net/vhost/rte_eth_vhost.c index aa6052221..a7604ff23 100644 --- a/drivers/net/vhost/rte_eth_vhost.c +++ b/drivers/net/vhost/rte_eth_vhost.c @@ -1344,8 +1344,7 @@ rte_pmd_vhost_probe(struct rte_vdev_device *dev) VHOST_LOG(INFO, "Initializing pmd_vhost for %s\n", name); - if (rte_eal_process_type() == RTE_PROC_SECONDARY && - strlen(rte_vdev_device_args(dev)) == 0) { + if (rte_eal_process_type() == RTE_PROC_SECONDARY) { eth_dev = rte_eth_dev_attach_secondary(name); if (!eth_dev) { VHOST_LOG(ERR, "Failed to probe %s\n", name); @@ -1436,6 +1435,9 @@ rte_pmd_vhost_remove(struct rte_vdev_device *dev) if (eth_dev == NULL) return -ENODEV; + if (rte_eal_process_type() != RTE_PROC_PRIMARY) + return rte_eth_dev_release_port_secondary(eth_dev); + eth_dev_close(eth_dev); rte_free(vring_states[eth_dev->data->port_id]); From patchwork Fri Sep 28 04:23:25 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Qi Zhang X-Patchwork-Id: 45553 X-Patchwork-Delegate: thomas@monjalon.net Return-Path: X-Original-To: patchwork@dpdk.org Delivered-To: patchwork@dpdk.org Received: from [92.243.14.124] (localhost [127.0.0.1]) by dpdk.org (Postfix) with ESMTP id 2C7C01B187; Fri, 28 Sep 2018 06:23:02 +0200 (CEST) Received: from mga04.intel.com (mga04.intel.com [192.55.52.120]) by dpdk.org (Postfix) with ESMTP id 10F6A1B12E for ; Fri, 28 Sep 2018 06:22:54 +0200 (CEST) X-Amp-Result: SKIPPED(no attachment in message) X-Amp-File-Uploaded: False Received: from fmsmga005.fm.intel.com ([10.253.24.32]) by fmsmga104.fm.intel.com with ESMTP/TLS/DHE-RSA-AES256-GCM-SHA384; 27 Sep 2018 21:22:52 -0700 X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="5.54,313,1534834800"; d="scan'208";a="266654829" Received: from dpdk51.sh.intel.com ([10.67.110.190]) by fmsmga005.fm.intel.com with ESMTP; 27 Sep 2018 21:22:36 -0700 From: Qi Zhang To: thomas@monjalon.net, gaetan.rivet@6wind.com, anatoly.burakov@intel.com, arybchenko@solarflare.com Cc: konstantin.ananyev@intel.com, dev@dpdk.org, bruce.richardson@intel.com, ferruh.yigit@intel.com, benjamin.h.shelton@intel.com, narender.vangati@intel.com, Qi Zhang Date: Fri, 28 Sep 2018 12:23:25 +0800 Message-Id: <20180928042326.50471-6-qi.z.zhang@intel.com> X-Mailer: git-send-email 2.13.6 In-Reply-To: <20180928042326.50471-1-qi.z.zhang@intel.com> References: <20180607123849.14439-1-qi.z.zhang@intel.com> <20180928042326.50471-1-qi.z.zhang@intel.com> Subject: [dpdk-dev] [PATCH v16 5/6] drivers/net: enable device detach on secondary X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Sender: "dev" With the enabling for hotplug on multi-process, rte_eth_dev_pci_generic_remove can be used to detach the device from a secondary process also. But we need to take care of the uninit callback parameter to make sure it handles the secondary case correctly. Signed-off-by: Qi Zhang --- drivers/net/bnxt/bnxt_ethdev.c | 6 +++++- drivers/net/ena/ena_ethdev.c | 2 +- drivers/net/liquidio/lio_ethdev.c | 2 +- drivers/net/virtio/virtio_ethdev.c | 2 +- 4 files changed, 8 insertions(+), 4 deletions(-) diff --git a/drivers/net/bnxt/bnxt_ethdev.c b/drivers/net/bnxt/bnxt_ethdev.c index 70c761581..c06b4e828 100644 --- a/drivers/net/bnxt/bnxt_ethdev.c +++ b/drivers/net/bnxt/bnxt_ethdev.c @@ -3514,7 +3514,11 @@ static int bnxt_pci_probe(struct rte_pci_driver *pci_drv __rte_unused, static int bnxt_pci_remove(struct rte_pci_device *pci_dev) { - return rte_eth_dev_pci_generic_remove(pci_dev, bnxt_dev_uninit); + if (rte_eal_process_type() == RTE_PROC_PRIMARY) + return rte_eth_dev_pci_generic_remove(pci_dev, + bnxt_dev_uninit); + else + return rte_eth_dev_pci_generic_remove(pci_dev, NULL); } static struct rte_pci_driver bnxt_rte_pmd = { diff --git a/drivers/net/ena/ena_ethdev.c b/drivers/net/ena/ena_ethdev.c index c255dc6db..c29a581e8 100644 --- a/drivers/net/ena/ena_ethdev.c +++ b/drivers/net/ena/ena_ethdev.c @@ -1703,7 +1703,7 @@ static int eth_ena_dev_uninit(struct rte_eth_dev *eth_dev) (struct ena_adapter *)(eth_dev->data->dev_private); if (rte_eal_process_type() != RTE_PROC_PRIMARY) - return -EPERM; + return 0; if (adapter->state != ENA_ADAPTER_STATE_CLOSED) ena_close(eth_dev); diff --git a/drivers/net/liquidio/lio_ethdev.c b/drivers/net/liquidio/lio_ethdev.c index 93e89007a..0f59e4475 100644 --- a/drivers/net/liquidio/lio_ethdev.c +++ b/drivers/net/liquidio/lio_ethdev.c @@ -2038,7 +2038,7 @@ lio_eth_dev_uninit(struct rte_eth_dev *eth_dev) PMD_INIT_FUNC_TRACE(); if (rte_eal_process_type() != RTE_PROC_PRIMARY) - return -EPERM; + return 0; /* lio_free_sc_buffer_pool */ lio_free_sc_buffer_pool(lio_dev); diff --git a/drivers/net/virtio/virtio_ethdev.c b/drivers/net/virtio/virtio_ethdev.c index b81df0a99..730c41707 100644 --- a/drivers/net/virtio/virtio_ethdev.c +++ b/drivers/net/virtio/virtio_ethdev.c @@ -1697,7 +1697,7 @@ eth_virtio_dev_uninit(struct rte_eth_dev *eth_dev) PMD_INIT_FUNC_TRACE(); if (rte_eal_process_type() == RTE_PROC_SECONDARY) - return -EPERM; + return 0; virtio_dev_stop(eth_dev); virtio_dev_close(eth_dev); From patchwork Fri Sep 28 04:23:26 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Qi Zhang X-Patchwork-Id: 45555 X-Patchwork-Delegate: thomas@monjalon.net Return-Path: X-Original-To: patchwork@dpdk.org Delivered-To: patchwork@dpdk.org Received: from [92.243.14.124] (localhost [127.0.0.1]) by dpdk.org (Postfix) with ESMTP id 0D3E11B173; Fri, 28 Sep 2018 06:23:06 +0200 (CEST) Received: from mga04.intel.com (mga04.intel.com [192.55.52.120]) by dpdk.org (Postfix) with ESMTP id B470F1B13F for ; Fri, 28 Sep 2018 06:22:55 +0200 (CEST) X-Amp-Result: SKIPPED(no attachment in message) X-Amp-File-Uploaded: False Received: from fmsmga005.fm.intel.com ([10.253.24.32]) by fmsmga104.fm.intel.com with ESMTP/TLS/DHE-RSA-AES256-GCM-SHA384; 27 Sep 2018 21:22:53 -0700 X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="5.54,313,1534834800"; d="scan'208";a="266654831" Received: from dpdk51.sh.intel.com ([10.67.110.190]) by fmsmga005.fm.intel.com with ESMTP; 27 Sep 2018 21:22:38 -0700 From: Qi Zhang To: thomas@monjalon.net, gaetan.rivet@6wind.com, anatoly.burakov@intel.com, arybchenko@solarflare.com Cc: konstantin.ananyev@intel.com, dev@dpdk.org, bruce.richardson@intel.com, ferruh.yigit@intel.com, benjamin.h.shelton@intel.com, narender.vangati@intel.com, Qi Zhang Date: Fri, 28 Sep 2018 12:23:26 +0800 Message-Id: <20180928042326.50471-7-qi.z.zhang@intel.com> X-Mailer: git-send-email 2.13.6 In-Reply-To: <20180928042326.50471-1-qi.z.zhang@intel.com> References: <20180607123849.14439-1-qi.z.zhang@intel.com> <20180928042326.50471-1-qi.z.zhang@intel.com> Subject: [dpdk-dev] [PATCH v16 6/6] examples/multi_process: add hotplug sample X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Sender: "dev" The sample code demonstrates device (ethdev only) management at a multi-process environment. The user can attach/detach a device on primary process and see it is synced on secondary process automatically. How to start? ./hotplug_mp --proc-type=auto Command Line Example: >help >list /* attach a pci device */ > attach 0000:81:00.0 /* detach the pci device */ > detach 0000:81:00.0 /* attach a vdev af_packet device */ > attach net_af_packet,iface=eth0 /* detach the vdev af_packet device */ > detach net_af_packet Signed-off-by: Qi Zhang --- examples/multi_process/Makefile | 1 + examples/multi_process/hotplug_mp/Makefile | 23 +++ examples/multi_process/hotplug_mp/commands.c | 214 +++++++++++++++++++++++++++ examples/multi_process/hotplug_mp/commands.h | 10 ++ examples/multi_process/hotplug_mp/main.c | 41 +++++ 5 files changed, 289 insertions(+) create mode 100644 examples/multi_process/hotplug_mp/Makefile create mode 100644 examples/multi_process/hotplug_mp/commands.c create mode 100644 examples/multi_process/hotplug_mp/commands.h create mode 100644 examples/multi_process/hotplug_mp/main.c diff --git a/examples/multi_process/Makefile b/examples/multi_process/Makefile index a6708b7e4..b76b02fcb 100644 --- a/examples/multi_process/Makefile +++ b/examples/multi_process/Makefile @@ -13,5 +13,6 @@ include $(RTE_SDK)/mk/rte.vars.mk DIRS-$(CONFIG_RTE_EXEC_ENV_LINUXAPP) += client_server_mp DIRS-$(CONFIG_RTE_EXEC_ENV_LINUXAPP) += simple_mp DIRS-$(CONFIG_RTE_EXEC_ENV_LINUXAPP) += symmetric_mp +DIRS-$(CONFIG_RTE_EXEC_ENV_LINUXAPP) += hotplug_mp include $(RTE_SDK)/mk/rte.extsubdir.mk diff --git a/examples/multi_process/hotplug_mp/Makefile b/examples/multi_process/hotplug_mp/Makefile new file mode 100644 index 000000000..bc36aeaed --- /dev/null +++ b/examples/multi_process/hotplug_mp/Makefile @@ -0,0 +1,23 @@ +# SPDX-License-Identifier: BSD-3-Clause +# Copyright(c) 2018 Intel Corporation + +ifeq ($(RTE_SDK),) +$(error "Please define RTE_SDK environment variable") +endif + +# Default target, can be overridden by command line or environment +RTE_TARGET ?= x86_64-native-linuxapp-gcc + +include $(RTE_SDK)/mk/rte.vars.mk + +# binary name +APP = hotplug_mp + +# all source are stored in SRCS-y +SRCS-y := main.c commands.c + +CFLAGS += -O3 +CFLAGS += $(WERROR_FLAGS) +CFLAGS += -DALLOW_EXPERIMENTAL_API + +include $(RTE_SDK)/mk/rte.extapp.mk diff --git a/examples/multi_process/hotplug_mp/commands.c b/examples/multi_process/hotplug_mp/commands.c new file mode 100644 index 000000000..b06859393 --- /dev/null +++ b/examples/multi_process/hotplug_mp/commands.c @@ -0,0 +1,214 @@ +/* SPDX-License-Identifier: BSD-3-Clause + * Copyright(c) 2018 Intel Corporation. + */ + +#include +#include +#include +#include +#include +#include +#include + +/**********************************************************/ + +struct cmd_help_result { + cmdline_fixed_string_t help; +}; + +static void cmd_help_parsed(__attribute__((unused)) void *parsed_result, + struct cmdline *cl, + __attribute__((unused)) void *data) +{ + cmdline_printf(cl, + "commands:\n" + "- attach \n" + "- detach \n" + "- list\n\n"); +} + +cmdline_parse_token_string_t cmd_help_help = + TOKEN_STRING_INITIALIZER(struct cmd_help_result, help, "help"); + +cmdline_parse_inst_t cmd_help = { + .f = cmd_help_parsed, /* function to call */ + .data = NULL, /* 2nd arg of func */ + .help_str = "show help", + .tokens = { /* token list, NULL terminated */ + (void *)&cmd_help_help, + NULL, + }, +}; + +/**********************************************************/ + +struct cmd_quit_result { + cmdline_fixed_string_t quit; +}; + +static void cmd_quit_parsed(__attribute__((unused)) void *parsed_result, + struct cmdline *cl, + __attribute__((unused)) void *data) +{ + cmdline_quit(cl); +} + +cmdline_parse_token_string_t cmd_quit_quit = + TOKEN_STRING_INITIALIZER(struct cmd_quit_result, quit, "quit"); + +cmdline_parse_inst_t cmd_quit = { + .f = cmd_quit_parsed, /* function to call */ + .data = NULL, /* 2nd arg of func */ + .help_str = "quit", + .tokens = { /* token list, NULL terminated */ + (void *)&cmd_quit_quit, + NULL, + }, +}; + +/**********************************************************/ + +struct cmd_list_result { + cmdline_fixed_string_t list; +}; + +static void cmd_list_parsed(__attribute__((unused)) void *parsed_result, + struct cmdline *cl, + __attribute__((unused)) void *data) +{ + uint16_t port_id; + char dev_name[RTE_DEV_NAME_MAX_LEN]; + + cmdline_printf(cl, "list all etherdev\n"); + + RTE_ETH_FOREACH_DEV(port_id) { + rte_eth_dev_get_name_by_port(port_id, dev_name); + if (strlen(dev_name) > 0) + cmdline_printf(cl, "%d\t%s\n", port_id, dev_name); + else + printf("empty dev_name is not expected!\n"); + } +} + +cmdline_parse_token_string_t cmd_list_list = + TOKEN_STRING_INITIALIZER(struct cmd_list_result, list, "list"); + +cmdline_parse_inst_t cmd_list = { + .f = cmd_list_parsed, /* function to call */ + .data = NULL, /* 2nd arg of func */ + .help_str = "list all devices", + .tokens = { /* token list, NULL terminated */ + (void *)&cmd_list_list, + NULL, + }, +}; + +/**********************************************************/ + +struct cmd_dev_attach_result { + cmdline_fixed_string_t attach; + cmdline_fixed_string_t devargs; +}; + +static void cmd_dev_attach_parsed(void *parsed_result, + struct cmdline *cl, + __attribute__((unused)) void *data) +{ + struct cmd_dev_attach_result *res = parsed_result; + struct rte_devargs da; + + memset(&da, 0, sizeof(da)); + + if (rte_devargs_parsef(&da, "%s", res->devargs)) { + cmdline_printf(cl, "cannot parse devargs\n"); + if (da.args) + free(da.args); + return; + } + + if (!rte_eal_hotplug_add(da.bus->name, da.name, da.args)) + cmdline_printf(cl, "attached device %s\n", da.name); + else + cmdline_printf(cl, "failed to attached device %s\n", + da.name); +} + +cmdline_parse_token_string_t cmd_dev_attach_attach = + TOKEN_STRING_INITIALIZER(struct cmd_dev_attach_result, attach, + "attach"); +cmdline_parse_token_string_t cmd_dev_attach_devargs = + TOKEN_STRING_INITIALIZER(struct cmd_dev_attach_result, devargs, NULL); + +cmdline_parse_inst_t cmd_attach_device = { + .f = cmd_dev_attach_parsed, /* function to call */ + .data = NULL, /* 2nd arg of func */ + .help_str = "attach a device", + .tokens = { /* token list, NULL terminated */ + (void *)&cmd_dev_attach_attach, + (void *)&cmd_dev_attach_devargs, + NULL, + }, +}; + +/**********************************************************/ + +struct cmd_dev_detach_result { + cmdline_fixed_string_t detach; + cmdline_fixed_string_t devargs; +}; + +static void cmd_dev_detach_parsed(void *parsed_result, + struct cmdline *cl, + __attribute__((unused)) void *data) +{ + struct cmd_dev_detach_result *res = parsed_result; + struct rte_devargs da; + + memset(&da, 0, sizeof(da)); + + if (rte_devargs_parsef(&da, "%s", res->devargs)) { + cmdline_printf(cl, "cannot parse devargs\n"); + if (da.args) + free(da.args); + return; + } + + printf("detaching...\n"); + if (!rte_eal_hotplug_remove(da.bus->name, da.name)) + cmdline_printf(cl, "detached device %s\n", + da.name); + else + cmdline_printf(cl, "failed to dettach device %s\n", + da.name); +} + +cmdline_parse_token_string_t cmd_dev_detach_detach = + TOKEN_STRING_INITIALIZER(struct cmd_dev_detach_result, detach, + "detach"); + +cmdline_parse_token_string_t cmd_dev_detach_devargs = + TOKEN_STRING_INITIALIZER(struct cmd_dev_detach_result, devargs, NULL); + +cmdline_parse_inst_t cmd_detach_device = { + .f = cmd_dev_detach_parsed, /* function to call */ + .data = NULL, /* 2nd arg of func */ + .help_str = "detach a device", + .tokens = { /* token list, NULL terminated */ + (void *)&cmd_dev_detach_detach, + (void *)&cmd_dev_detach_devargs, + NULL, + }, +}; + +/**********************************************************/ +/**********************************************************/ +/****** CONTEXT (list of instruction) */ + +cmdline_parse_ctx_t main_ctx[] = { + (cmdline_parse_inst_t *)&cmd_help, + (cmdline_parse_inst_t *)&cmd_quit, + (cmdline_parse_inst_t *)&cmd_list, + (cmdline_parse_inst_t *)&cmd_attach_device, + (cmdline_parse_inst_t *)&cmd_detach_device, + NULL, +}; diff --git a/examples/multi_process/hotplug_mp/commands.h b/examples/multi_process/hotplug_mp/commands.h new file mode 100644 index 000000000..afcf177db --- /dev/null +++ b/examples/multi_process/hotplug_mp/commands.h @@ -0,0 +1,10 @@ +/* SPDX-License-Identifier: BSD-3-Clause + * Copyright(c) 2018 Intel Corporation + */ + +#ifndef _COMMANDS_H_ +#define _COMMANDS_H_ + +extern cmdline_parse_ctx_t main_ctx[]; + +#endif /* _COMMANDS_H_ */ diff --git a/examples/multi_process/hotplug_mp/main.c b/examples/multi_process/hotplug_mp/main.c new file mode 100644 index 000000000..d66858078 --- /dev/null +++ b/examples/multi_process/hotplug_mp/main.c @@ -0,0 +1,41 @@ +/* SPDX-License-Identifier: BSD-3-Clause + * Copyright(c) 2018 Intel Corporation + */ + +#include +#include +#include +#include +#include +#include + +#include +#include +#include +#include + +#include +#include +#include + +#include "commands.h" + +int main(int argc, char **argv) +{ + int ret; + struct cmdline *cl; + + ret = rte_eal_init(argc, argv); + if (ret < 0) + rte_panic("Cannot init EAL\n"); + + cl = cmdline_stdin_new(main_ctx, "example> "); + if (cl == NULL) + rte_panic("Cannot create cmdline instance\n"); + cmdline_interact(cl); + cmdline_stdin_exit(cl); + + rte_eal_cleanup(); + + return 0; +}