Patch Detail
get:
Show a patch.
patch:
Update a patch.
put:
Update a patch.
GET /api/patches/45551/?format=api
{ "id": 45551, "url": "http://patches.dpdk.org/api/patches/45551/?format=api", "web_url": "http://patches.dpdk.org/project/dpdk/patch/20180928042326.50471-3-qi.z.zhang@intel.com/", "project": { "id": 1, "url": "http://patches.dpdk.org/api/projects/1/?format=api", "name": "DPDK", "link_name": "dpdk", "list_id": "dev.dpdk.org", "list_email": "dev@dpdk.org", "web_url": "http://core.dpdk.org", "scm_url": "git://dpdk.org/dpdk", "webscm_url": "http://git.dpdk.org/dpdk", "list_archive_url": "https://inbox.dpdk.org/dev", "list_archive_url_format": "https://inbox.dpdk.org/dev/{}", "commit_url_format": "" }, "msgid": "<20180928042326.50471-3-qi.z.zhang@intel.com>", "list_archive_url": "https://inbox.dpdk.org/dev/20180928042326.50471-3-qi.z.zhang@intel.com", "date": "2018-09-28T04:23:22", "name": "[v16,2/6] eal: enable hotplug on multi-process", "commit_ref": null, "pull_url": null, "state": "superseded", "archived": true, "hash": "4b38773def56d2d6fe90645b7e8a1912e6f5ac62", "submitter": { "id": 504, "url": "http://patches.dpdk.org/api/people/504/?format=api", "name": "Qi Zhang", "email": "qi.z.zhang@intel.com" }, "delegate": { "id": 1, "url": "http://patches.dpdk.org/api/users/1/?format=api", "username": "tmonjalo", "first_name": "Thomas", "last_name": "Monjalon", "email": "thomas@monjalon.net" }, "mbox": "http://patches.dpdk.org/project/dpdk/patch/20180928042326.50471-3-qi.z.zhang@intel.com/mbox/", "series": [ { "id": 1562, "url": "http://patches.dpdk.org/api/series/1562/?format=api", "web_url": "http://patches.dpdk.org/project/dpdk/list/?series=1562", "date": "2018-09-28T04:23:20", "name": "enable hotplug on multi-process", "version": 16, "mbox": "http://patches.dpdk.org/series/1562/mbox/" } ], "comments": "http://patches.dpdk.org/api/patches/45551/comments/", "check": "success", "checks": "http://patches.dpdk.org/api/patches/45551/checks/", "tags": {}, "related": [], "headers": { "Return-Path": "<dev-bounces@dpdk.org>", "X-Original-To": "patchwork@dpdk.org", "Delivered-To": "patchwork@dpdk.org", "Received": [ "from [92.243.14.124] (localhost [127.0.0.1])\n\tby dpdk.org (Postfix) with ESMTP id 4F0571B145;\n\tFri, 28 Sep 2018 06:22:58 +0200 (CEST)", "from mga04.intel.com (mga04.intel.com [192.55.52.120])\n\tby dpdk.org (Postfix) with ESMTP id D635E1B11C\n\tfor <dev@dpdk.org>; Fri, 28 Sep 2018 06:22:53 +0200 (CEST)", "from fmsmga005.fm.intel.com ([10.253.24.32])\n\tby fmsmga104.fm.intel.com with ESMTP/TLS/DHE-RSA-AES256-GCM-SHA384;\n\t27 Sep 2018 21:22:52 -0700", "from dpdk51.sh.intel.com ([10.67.110.190])\n\tby fmsmga005.fm.intel.com with ESMTP; 27 Sep 2018 21:22:30 -0700" ], "X-Amp-Result": "SKIPPED(no attachment in message)", "X-Amp-File-Uploaded": "False", "X-ExtLoop1": "1", "X-IronPort-AV": "E=Sophos;i=\"5.54,313,1534834800\"; d=\"scan'208\";a=\"266654788\"", "From": "Qi Zhang <qi.z.zhang@intel.com>", "To": "thomas@monjalon.net, gaetan.rivet@6wind.com, anatoly.burakov@intel.com, \n\tarybchenko@solarflare.com", "Cc": "konstantin.ananyev@intel.com, dev@dpdk.org, bruce.richardson@intel.com, \n\tferruh.yigit@intel.com, benjamin.h.shelton@intel.com,\n\tnarender.vangati@intel.com, Qi Zhang <qi.z.zhang@intel.com>", "Date": "Fri, 28 Sep 2018 12:23:22 +0800", "Message-Id": "<20180928042326.50471-3-qi.z.zhang@intel.com>", "X-Mailer": "git-send-email 2.13.6", "In-Reply-To": "<20180928042326.50471-1-qi.z.zhang@intel.com>", "References": "<20180607123849.14439-1-qi.z.zhang@intel.com>\n\t<20180928042326.50471-1-qi.z.zhang@intel.com>", "Subject": "[dpdk-dev] [PATCH v16 2/6] eal: enable hotplug on multi-process", "X-BeenThere": "dev@dpdk.org", "X-Mailman-Version": "2.1.15", "Precedence": "list", "List-Id": "DPDK patches and discussions <dev.dpdk.org>", "List-Unsubscribe": "<https://mails.dpdk.org/options/dev>,\n\t<mailto:dev-request@dpdk.org?subject=unsubscribe>", "List-Archive": "<http://mails.dpdk.org/archives/dev/>", "List-Post": "<mailto:dev@dpdk.org>", "List-Help": "<mailto:dev-request@dpdk.org?subject=help>", "List-Subscribe": "<https://mails.dpdk.org/listinfo/dev>,\n\t<mailto:dev-request@dpdk.org?subject=subscribe>", "Errors-To": "dev-bounces@dpdk.org", "Sender": "\"dev\" <dev-bounces@dpdk.org>" }, "content": "We are going to introduce the solution to handle hotplug in\nmulti-process, it includes the below scenario:\n\n1. Attach a device from the primary\n2. Detach a device from the primary\n3. Attach a device from a secondary\n4. Detach a device from a secondary\n\nIn the primary-secondary process model, we assume devices are shared\nby default. that means attaches or detaches a device on any process\nwill broadcast to all other processes through mp channel then device\ninformation will be synchronized on all processes.\n\nAny failure during attaching/detaching process will cause inconsistent\nstatus between processes, so proper rollback action should be considered.\n\nThis patch covers the implementation of case 1,2.\nCase 3,4 will be implemented on a separate patch.\n\nIPC scenario for Case 1, 2:\n\nattach a device\na) primary attach the new device if failed goto h).\nb) primary send attach sync request to all secondary.\nc) secondary receive request and attach the device and send a reply.\nd) primary check the reply if all success goes to i).\ne) primary send attach rollback sync request to all secondary.\nf) secondary receive the request and detach the device and send a reply.\ng) primary receive the reply and detach device as rollback action.\nh) attach fail\ni) attach success\n\ndetach a device\na) primary send detach sync request to all secondary\nb) secondary detach the device and send reply\nc) primary check the reply if all success goes to f).\nd) primary send detach rollback sync request to all secondary.\ne) secondary receive the request and attach back device. goto g)\nf) primary detach the device if success goto g), else goto d)\ng) detach fail.\nh) detach success.\n\nSigned-off-by: Qi Zhang <qi.z.zhang@intel.com>\nAcked-by: Anatoly Burakov <anatoly.burakov@intel.com>\n---\n doc/guides/rel_notes/release_18_11.rst | 11 ++\n lib/librte_eal/bsdapp/eal/Makefile | 1 +\n lib/librte_eal/common/eal_common_dev.c | 225 ++++++++++++++++++++++++++++++--\n lib/librte_eal/common/eal_private.h | 30 +++++\n lib/librte_eal/common/hotplug_mp.c | 221 +++++++++++++++++++++++++++++++\n lib/librte_eal/common/hotplug_mp.h | 46 +++++++\n lib/librte_eal/common/include/rte_dev.h | 9 ++\n lib/librte_eal/common/meson.build | 1 +\n lib/librte_eal/linuxapp/eal/Makefile | 1 +\n lib/librte_eal/linuxapp/eal/eal.c | 6 +\n 10 files changed, 542 insertions(+), 9 deletions(-)\n create mode 100644 lib/librte_eal/common/hotplug_mp.c\n create mode 100644 lib/librte_eal/common/hotplug_mp.h", "diff": "diff --git a/doc/guides/rel_notes/release_18_11.rst b/doc/guides/rel_notes/release_18_11.rst\nindex bc9b74ec4..f88910c7f 100644\n--- a/doc/guides/rel_notes/release_18_11.rst\n+++ b/doc/guides/rel_notes/release_18_11.rst\n@@ -67,6 +67,12 @@ New Features\n SR-IOV option in Hyper-V and Azure. This is an alternative to the previous\n vdev_netvsc, tap, and failsafe drivers combination.\n \n+* **Support device multi-process hotplug.**\n+\n+ Hotplug and hot-unplug for devices will now be supported in multiprocessing\n+ scenario. Any ethdev devices created in the primary process will be regarded\n+ as shared and will be available for all DPDK processes. Synchronization\n+ between processes will be done using DPDK IPC.\n \n API Changes\n -----------\n@@ -91,6 +97,11 @@ API Changes\n flag the MAC can be properly configured in any case. This is particularly\n important for bonding.\n \n+* eal: scope of rte_eal_hotplug_add and rte_eal_hotplug_remove is extended.\n+\n+ In primary-secondary process model, ``rte_eal_hotplug_add`` will guarantee\n+ that device be attached on all processes, while ``rte_eal_hotplug_remove``\n+ will guarantee device be detached on all processes.\n \n ABI Changes\n -----------\ndiff --git a/lib/librte_eal/bsdapp/eal/Makefile b/lib/librte_eal/bsdapp/eal/Makefile\nindex d27da3d15..4351c6a20 100644\n--- a/lib/librte_eal/bsdapp/eal/Makefile\n+++ b/lib/librte_eal/bsdapp/eal/Makefile\n@@ -62,6 +62,7 @@ SRCS-$(CONFIG_RTE_EXEC_ENV_BSDAPP) += eal_common_proc.c\n SRCS-$(CONFIG_RTE_EXEC_ENV_BSDAPP) += eal_common_fbarray.c\n SRCS-$(CONFIG_RTE_EXEC_ENV_BSDAPP) += eal_common_uuid.c\n SRCS-$(CONFIG_RTE_EXEC_ENV_BSDAPP) += rte_malloc.c\n+SRCS-$(CONFIG_RTE_EXEC_ENV_BSDAPP) += hotplug_mp.c\n SRCS-$(CONFIG_RTE_EXEC_ENV_BSDAPP) += malloc_elem.c\n SRCS-$(CONFIG_RTE_EXEC_ENV_BSDAPP) += malloc_heap.c\n SRCS-$(CONFIG_RTE_EXEC_ENV_BSDAPP) += malloc_mp.c\ndiff --git a/lib/librte_eal/common/eal_common_dev.c b/lib/librte_eal/common/eal_common_dev.c\nindex 85eb1569f..314266041 100644\n--- a/lib/librte_eal/common/eal_common_dev.c\n+++ b/lib/librte_eal/common/eal_common_dev.c\n@@ -19,8 +19,10 @@\n #include <rte_log.h>\n #include <rte_spinlock.h>\n #include <rte_malloc.h>\n+#include <rte_string_fns.h>\n \n #include \"eal_private.h\"\n+#include \"hotplug_mp.h\"\n \n /**\n * The device event callback description.\n@@ -127,9 +129,10 @@ int rte_eal_dev_detach(struct rte_device *dev)\n \treturn ret;\n }\n \n-int\n-rte_eal_hotplug_add(const char *busname, const char *devname,\n- const char *drvargs)\n+/* help funciton to build devargs, caller should free the memory */\n+static char *\n+build_devargs(const char *busname, const char *devname,\n+\t const char *drvargs)\n {\n \tchar *devargs = NULL;\n \tint size, length = -1;\n@@ -140,19 +143,33 @@ rte_eal_hotplug_add(const char *busname, const char *devname,\n \t\tif (length >= size)\n \t\t\tdevargs = malloc(length + 1);\n \t\tif (devargs == NULL)\n-\t\t\treturn -ENOMEM;\n+\t\t\tbreak;\n \t} while (size == 0);\n \n+\treturn devargs;\n+}\n+\n+int\n+rte_eal_hotplug_add(const char *busname, const char *devname,\n+\t\t const char *drvargs)\n+{\n+\tchar *devargs = build_devargs(busname, devname, drvargs);\n+\n+\tif (devargs == NULL)\n+\t\treturn -ENOMEM;\n+\n \treturn rte_dev_probe(devargs);\n }\n \n-int __rte_experimental\n-rte_dev_probe(const char *devargs)\n+/* probe device at local process. */\n+int\n+local_dev_probe(const char *devargs, struct rte_device **new_dev)\n {\n \tstruct rte_device *dev;\n \tstruct rte_devargs *da;\n \tint ret;\n \n+\t*new_dev = NULL;\n \tda = calloc(1, sizeof(*da));\n \tif (da == NULL)\n \t\treturn -ENOMEM;\n@@ -195,6 +212,8 @@ rte_dev_probe(const char *devargs)\n \t\t\tdev->name);\n \t\tgoto err_devarg;\n \t}\n+\n+\t*new_dev = dev;\n \treturn 0;\n \n err_devarg:\n@@ -226,8 +245,9 @@ rte_eal_hotplug_remove(const char *busname, const char *devname)\n \treturn rte_dev_remove(dev);\n }\n \n-int __rte_experimental\n-rte_dev_remove(struct rte_device *dev)\n+/* remove device at local process. */\n+int\n+local_dev_remove(struct rte_device *dev)\n {\n \tstruct rte_bus *bus;\n \tint ret;\n@@ -248,7 +268,194 @@ rte_dev_remove(struct rte_device *dev)\n \tif (ret)\n \t\tRTE_LOG(ERR, EAL, \"Driver cannot detach the device (%s)\\n\",\n \t\t\tdev->name);\n-\trte_devargs_remove(dev->devargs);\n+\telse\n+\t\trte_devargs_remove(dev->devargs);\n+\n+\treturn ret;\n+}\n+\n+int __rte_experimental\n+rte_dev_probe(const char *devargs)\n+{\n+\tstruct eal_dev_mp_req req;\n+\tstruct rte_device *dev;\n+\tint ret;\n+\n+\tmemset(&req, 0, sizeof(req));\n+\treq.t = EAL_DEV_REQ_TYPE_ATTACH;\n+\tstrlcpy(req.devargs, devargs, EAL_DEV_MP_DEV_ARGS_MAX_LEN);\n+\n+\tif (rte_eal_process_type() != RTE_PROC_PRIMARY) {\n+\t\t/**\n+\t\t * If in secondary process, just send IPC request to\n+\t\t * primary process.\n+\t\t */\n+\t\tret = eal_dev_hotplug_request_to_primary(&req);\n+\t\tif (ret) {\n+\t\t\tRTE_LOG(ERR, EAL,\n+\t\t\t\t\"Failed to send hotplug request to primary\\n\");\n+\t\t\treturn -ENOMSG;\n+\t\t}\n+\t\tif (req.result)\n+\t\t\tRTE_LOG(ERR, EAL,\n+\t\t\t\t\"Failed to hotplug add device\\n\");\n+\t\treturn req.result;\n+\t}\n+\n+\t/* attach a shared device from primary start from here: */\n+\n+\t/* primary attach the new device itself. */\n+\tret = local_dev_probe(devargs, &dev);\n+\n+\tif (ret) {\n+\t\tRTE_LOG(ERR, EAL,\n+\t\t\t\"Failed to attach device on primary process\\n\");\n+\n+\t\t/**\n+\t\t * it is possible that secondary process failed to attached a\n+\t\t * device that primary process have during initialization,\n+\t\t * so for -EEXIST case, we still need to sync with secondary\n+\t\t * process.\n+\t\t */\n+\t\tif (ret != -EEXIST)\n+\t\t\treturn ret;\n+\t}\n+\n+\t/* primary send attach sync request to secondary. */\n+\tret = eal_dev_hotplug_request_to_secondary(&req);\n+\n+\t/* if any commnunication error, we need to rollback. */\n+\tif (ret) {\n+\t\tRTE_LOG(ERR, EAL,\n+\t\t\t\"Failed to send hotplug add request to secondary\\n\");\n+\t\tret = -ENOMSG;\n+\t\tgoto rollback;\n+\t}\n+\n+\t/**\n+\t * if any secondary failed to attach, we need to consider if rollback\n+\t * is necessary.\n+\t */\n+\tif (req.result) {\n+\t\tRTE_LOG(ERR, EAL,\n+\t\t\t\"Failed to attach device on secondary process\\n\");\n+\t\tret = req.result;\n+\n+\t\t/* for -EEXIST, we don't need to rollback. */\n+\t\tif (ret == -EEXIST)\n+\t\t\treturn ret;\n+\t\tgoto rollback;\n+\t}\n+\n+\treturn 0;\n+\n+rollback:\n+\treq.t = EAL_DEV_REQ_TYPE_ATTACH_ROLLBACK;\n+\n+\t/* primary send rollback request to secondary. */\n+\tif (eal_dev_hotplug_request_to_secondary(&req))\n+\t\tRTE_LOG(WARNING, EAL,\n+\t\t\t\"Failed to rollback device attach on secondary.\"\n+\t\t\t\"Devices in secondary may not sync with primary\\n\");\n+\n+\t/* primary rollback itself. */\n+\tif (local_dev_remove(dev))\n+\t\tRTE_LOG(WARNING, EAL,\n+\t\t\t\"Failed to rollback device attach on primary.\"\n+\t\t\t\"Devices in secondary may not sync with primary\\n\");\n+\n+\treturn ret;\n+}\n+\n+int __rte_experimental\n+rte_dev_remove(struct rte_device *dev)\n+{\n+\tstruct eal_dev_mp_req req;\n+\tchar *devargs;\n+\tint ret;\n+\n+\tdevargs = build_devargs(dev->devargs->bus->name, dev->name, \"\");\n+\tif (devargs == NULL)\n+\t\treturn -ENOMEM;\n+\n+\tmemset(&req, 0, sizeof(req));\n+\treq.t = EAL_DEV_REQ_TYPE_DETACH;\n+\tstrlcpy(req.devargs, devargs, EAL_DEV_MP_DEV_ARGS_MAX_LEN);\n+\tfree(devargs);\n+\n+\tif (rte_eal_process_type() != RTE_PROC_PRIMARY) {\n+\t\t/**\n+\t\t * If in secondary process, just send IPC request to\n+\t\t * primary process.\n+\t\t */\n+\t\tret = eal_dev_hotplug_request_to_primary(&req);\n+\t\tif (ret) {\n+\t\t\tRTE_LOG(ERR, EAL,\n+\t\t\t\t\"Failed to send hotplug request to primary\\n\");\n+\t\t\treturn -ENOMSG;\n+\t\t}\n+\t\tif (req.result)\n+\t\t\tRTE_LOG(ERR, EAL,\n+\t\t\t\t\"Failed to hotplug remove device\\n\");\n+\t\treturn req.result;\n+\t}\n+\n+\t/* detach a device from primary start from here: */\n+\n+\t/* primary send detach sync request to secondary */\n+\tret = eal_dev_hotplug_request_to_secondary(&req);\n+\n+\t/**\n+\t * if communication error, we need to rollback, because it is possible\n+\t * part of the secondary processes still detached it successfully.\n+\t */\n+\tif (ret) {\n+\t\tRTE_LOG(ERR, EAL,\n+\t\t\t\"Failed to send device detach request to secondary\\n\");\n+\t\tret = -ENOMSG;\n+\t\tgoto rollback;\n+\t}\n+\n+\t/**\n+\t * if any secondary failed to detach, we need to consider if rollback\n+\t * is necessary.\n+\t */\n+\tif (req.result) {\n+\t\tRTE_LOG(ERR, EAL,\n+\t\t\t\"Failed to detach device on secondary process\\n\");\n+\t\tret = req.result;\n+\t\t/**\n+\t\t * if -ENOENT, we don't need to rollback, since devices is\n+\t\t * already detached on secondary process.\n+\t\t */\n+\t\tif (ret != -ENOENT)\n+\t\t\tgoto rollback;\n+\t}\n+\n+\t/* primary detach the device itself. */\n+\tret = local_dev_remove(dev);\n+\n+\t/* if primary failed, still need to consider if rollback is necessary */\n+\tif (ret) {\n+\t\tRTE_LOG(ERR, EAL,\n+\t\t\t\"Failed to detach device on primary process\\n\");\n+\t\t/* if -ENOENT, we don't need to rollback */\n+\t\tif (ret == -ENOENT)\n+\t\t\treturn ret;\n+\t\tgoto rollback;\n+\t}\n+\n+\treturn 0;\n+\n+rollback:\n+\treq.t = EAL_DEV_REQ_TYPE_DETACH_ROLLBACK;\n+\n+\t/* primary send rollback request to secondary. */\n+\tif (eal_dev_hotplug_request_to_secondary(&req))\n+\t\tRTE_LOG(WARNING, EAL,\n+\t\t\t\"Failed to rollback device detach on secondary.\"\n+\t\t\t\"Devices in secondary may not sync with primary\\n\");\n+\n \treturn ret;\n }\n \ndiff --git a/lib/librte_eal/common/eal_private.h b/lib/librte_eal/common/eal_private.h\nindex 4f809a83c..83f10a9f8 100644\n--- a/lib/librte_eal/common/eal_private.h\n+++ b/lib/librte_eal/common/eal_private.h\n@@ -304,4 +304,34 @@ int\n rte_devargs_layers_parse(struct rte_devargs *devargs,\n \t\t\t const char *devstr);\n \n+/*\n+ * probe a device at local process.\n+ *\n+ * @param devargs\n+ * Device arguments including bus, class and driver properties.\n+ * @param new_dev\n+ * new device be probed as output.\n+ * @return\n+ * 0 on success, negative on error.\n+ */\n+int local_dev_probe(const char *devargs, struct rte_device **new_dev);\n+\n+/**\n+ * Hotplug remove a given device from a specific bus at local process.\n+ *\n+ * @param dev\n+ * Data structure of the device to remove.\n+ * @return\n+ * 0 on success, negative on error.\n+ */\n+int local_dev_remove(struct rte_device *dev);\n+\n+/**\n+ * Register all mp action callbacks for hotplug.\n+ *\n+ * @return\n+ * 0 on success, negative on error.\n+ */\n+int rte_dev_hotplug_mp_init(void);\n+\n #endif /* _EAL_PRIVATE_H_ */\ndiff --git a/lib/librte_eal/common/hotplug_mp.c b/lib/librte_eal/common/hotplug_mp.c\nnew file mode 100644\nindex 000000000..1c92e44cb\n--- /dev/null\n+++ b/lib/librte_eal/common/hotplug_mp.c\n@@ -0,0 +1,221 @@\n+/* SPDX-License-Identifier: BSD-3-Clause\n+ * Copyright(c) 2018 Intel Corporation\n+ */\n+#include <string.h>\n+\n+#include <rte_eal.h>\n+#include <rte_alarm.h>\n+#include <rte_string_fns.h>\n+#include <rte_devargs.h>\n+\n+#include \"hotplug_mp.h\"\n+#include \"eal_private.h\"\n+\n+#define MP_TIMEOUT_S 5 /**< 5 seconds timeouts */\n+\n+static int cmp_dev_name(const struct rte_device *dev, const void *_name)\n+{\n+\tconst char *name = _name;\n+\n+\treturn strcmp(dev->name, name);\n+}\n+\n+struct mp_reply_bundle {\n+\tstruct rte_mp_msg msg;\n+\tvoid *peer;\n+};\n+\n+static int\n+handle_secondary_request(const struct rte_mp_msg *msg, const void *peer)\n+{\n+\tRTE_SET_USED(msg);\n+\tRTE_SET_USED(peer);\n+\treturn -ENOTSUP;\n+}\n+\n+static void __handle_primary_request(void *param)\n+{\n+\tstruct mp_reply_bundle *bundle = param;\n+\tstruct rte_mp_msg *msg = &bundle->msg;\n+\tconst struct eal_dev_mp_req *req =\n+\t\t(const struct eal_dev_mp_req *)msg->param;\n+\tstruct rte_mp_msg mp_resp;\n+\tstruct eal_dev_mp_req *resp =\n+\t\t(struct eal_dev_mp_req *)mp_resp.param;\n+\tstruct rte_devargs *da;\n+\tstruct rte_device *dev;\n+\tstruct rte_bus *bus;\n+\tint ret = 0;\n+\n+\tmemset(&mp_resp, 0, sizeof(mp_resp));\n+\n+\tswitch (req->t) {\n+\tcase EAL_DEV_REQ_TYPE_ATTACH:\n+\tcase EAL_DEV_REQ_TYPE_DETACH_ROLLBACK:\n+\t\tret = local_dev_probe(req->devargs, &dev);\n+\t\tbreak;\n+\tcase EAL_DEV_REQ_TYPE_DETACH:\n+\tcase EAL_DEV_REQ_TYPE_ATTACH_ROLLBACK:\n+\t\tda = calloc(1, sizeof(*da));\n+\t\tif (da == NULL) {\n+\t\t\tret = -ENOMEM;\n+\t\t\tgoto quit;\n+\t\t}\n+\n+\t\tret = rte_devargs_parse(da, req->devargs);\n+\t\tif (ret)\n+\t\t\tgoto quit;\n+\n+\t\tbus = rte_bus_find_by_name(da->bus->name);\n+\t\tif (bus == NULL) {\n+\t\t\tRTE_LOG(ERR, EAL, \"Cannot find bus (%s)\\n\", da->bus->name);\n+\t\t\tret = -ENOENT;\n+\t\t\tgoto quit;\n+\t\t}\n+\n+\t\tdev = bus->find_device(NULL, cmp_dev_name, da->name);\n+\t\tif (dev == NULL) {\n+\t\t\tRTE_LOG(ERR, EAL, \"Cannot find plugged device (%s)\\n\", da->name);\n+\t\t\tret = -ENOENT;\n+\t\t\tgoto quit;\n+\t\t}\n+\n+\t\tret = local_dev_remove(dev);\n+quit:\n+\t\tbreak;\n+\tdefault:\n+\t\tret = -EINVAL;\n+\t}\n+\n+\tstrlcpy(mp_resp.name, EAL_DEV_MP_ACTION_REQUEST, sizeof(mp_resp.name));\n+\tmp_resp.len_param = sizeof(*req);\n+\tmemcpy(resp, req, sizeof(*resp));\n+\tresp->result = ret;\n+\tif (rte_mp_reply(&mp_resp, bundle->peer) < 0)\n+\t\tRTE_LOG(ERR, EAL, \"failed to send reply to primary request\\n\");\n+\n+\tfree(bundle->peer);\n+\tfree(bundle);\n+}\n+\n+static int\n+handle_primary_request(const struct rte_mp_msg *msg, const void *peer)\n+{\n+\tstruct rte_mp_msg mp_resp;\n+\tconst struct eal_dev_mp_req *req =\n+\t\t(const struct eal_dev_mp_req *)msg->param;\n+\tstruct eal_dev_mp_req *resp =\n+\t\t(struct eal_dev_mp_req *)mp_resp.param;\n+\tstruct mp_reply_bundle *bundle;\n+\tint ret = 0;\n+\n+\tmemset(&mp_resp, 0, sizeof(mp_resp));\n+\tstrlcpy(mp_resp.name, EAL_DEV_MP_ACTION_REQUEST, sizeof(mp_resp.name));\n+\tmp_resp.len_param = sizeof(*req);\n+\tmemcpy(resp, req, sizeof(*resp));\n+\n+\tbundle = calloc(1, sizeof(*bundle));\n+\tif (bundle == NULL) {\n+\t\tresp->result = -ENOMEM;\n+\t\tret = rte_mp_reply(&mp_resp, peer);\n+\t\tif (ret)\n+\t\t\tRTE_LOG(ERR, EAL, \"failed to send reply to primary request\\n\");\n+\t\treturn ret;\n+\t}\n+\n+\tbundle->msg = *msg;\n+\t/**\n+\t * We need to send reply on interrupt thread, but peer can't be\n+\t * parsed directly, so this is a temporal hack, need to be fixed\n+\t * when it is ready.\n+\t */\n+\tbundle->peer = (void *)strdup(peer);\n+\n+\t/**\n+\t * We are at IPC callback thread, sync IPC is not allowed due to\n+\t * dead lock, so we delegate the task to interrupt thread.\n+\t */\n+\tret = rte_eal_alarm_set(1, __handle_primary_request, bundle);\n+\tif (ret) {\n+\t\tresp->result = ret;\n+\t\tret = rte_mp_reply(&mp_resp, peer);\n+\t\tif (ret) {\n+\t\t\tRTE_LOG(ERR, EAL, \"failed to send reply to primary request\\n\");\n+\t\t\treturn ret;\n+\t\t}\n+\t}\n+\treturn 0;\n+}\n+\n+int eal_dev_hotplug_request_to_primary(struct eal_dev_mp_req *req)\n+{\n+\tRTE_SET_USED(req);\n+\treturn -ENOTSUP;\n+}\n+\n+int eal_dev_hotplug_request_to_secondary(struct eal_dev_mp_req *req)\n+{\n+\tstruct rte_mp_msg mp_req;\n+\tstruct rte_mp_reply mp_reply;\n+\tstruct timespec ts = {.tv_sec = MP_TIMEOUT_S, .tv_nsec = 0};\n+\tint ret;\n+\tint i;\n+\n+\tmemset(&mp_req, 0, sizeof(mp_req));\n+\tmemcpy(mp_req.param, req, sizeof(*req));\n+\tmp_req.len_param = sizeof(*req);\n+\tstrlcpy(mp_req.name, EAL_DEV_MP_ACTION_REQUEST, sizeof(mp_req.name));\n+\n+\tret = rte_mp_request_sync(&mp_req, &mp_reply, &ts);\n+\tif (ret) {\n+\t\tRTE_LOG(ERR, EAL, \"rte_mp_request_sync failed\\n\");\n+\t\treturn ret;\n+\t}\n+\n+\tif (mp_reply.nb_sent != mp_reply.nb_received) {\n+\t\tRTE_LOG(ERR, EAL, \"not all secondary reply\\n\");\n+\t\treturn -1;\n+\t}\n+\n+\treq->result = 0;\n+\tfor (i = 0; i < mp_reply.nb_received; i++) {\n+\t\tstruct eal_dev_mp_req *resp =\n+\t\t\t(struct eal_dev_mp_req *)mp_reply.msgs[i].param;\n+\t\tif (resp->result) {\n+\t\t\treq->result = resp->result;\n+\t\t\tif (req->t == EAL_DEV_REQ_TYPE_ATTACH &&\n+\t\t\t\treq->result != -EEXIST)\n+\t\t\t\tbreak;\n+\t\t\tif (req->t == EAL_DEV_REQ_TYPE_DETACH &&\n+\t\t\t\treq->result != -ENOENT)\n+\t\t\t\tbreak;\n+\t\t}\n+\t}\n+\n+\treturn 0;\n+}\n+\n+int rte_dev_hotplug_mp_init(void)\n+{\n+\tint ret;\n+\n+\tif (rte_eal_process_type() == RTE_PROC_PRIMARY) {\n+\t\tret = rte_mp_action_register(EAL_DEV_MP_ACTION_REQUEST,\n+\t\t\t\t\thandle_secondary_request);\n+\t\tif (ret) {\n+\t\t\tRTE_LOG(ERR, EAL, \"Couldn't register '%s' action\\n\",\n+\t\t\t\tEAL_DEV_MP_ACTION_REQUEST);\n+\t\t\treturn ret;\n+\t\t}\n+\t} else {\n+\t\tret = rte_mp_action_register(EAL_DEV_MP_ACTION_REQUEST,\n+\t\t\t\t\thandle_primary_request);\n+\t\tif (ret) {\n+\t\t\tRTE_LOG(ERR, EAL, \"Couldn't register '%s' action\\n\",\n+\t\t\t\tEAL_DEV_MP_ACTION_REQUEST);\n+\t\t\treturn ret;\n+\t\t}\n+\t}\n+\n+\treturn 0;\n+}\ndiff --git a/lib/librte_eal/common/hotplug_mp.h b/lib/librte_eal/common/hotplug_mp.h\nnew file mode 100644\nindex 000000000..c95c8f1fb\n--- /dev/null\n+++ b/lib/librte_eal/common/hotplug_mp.h\n@@ -0,0 +1,46 @@\n+/* SPDX-License-Identifier: BSD-3-Clause\n+ * Copyright(c) 2018 Intel Corporation\n+ */\n+\n+#ifndef _HOTPLUG_MP_H_\n+#define _HOTPLUG_MP_H_\n+\n+#include <rte_dev.h>\n+#include <rte_bus.h>\n+\n+#define EAL_DEV_MP_ACTION_REQUEST \"eal_dev_mp_request\"\n+#define EAL_DEV_MP_ACTION_RESPONSE \"eal_dev_mp_response\"\n+\n+#define EAL_DEV_MP_DEV_NAME_MAX_LEN RTE_DEV_NAME_MAX_LEN\n+#define EAL_DEV_MP_BUS_NAME_MAX_LEN 32\n+#define EAL_DEV_MP_DEV_ARGS_MAX_LEN 128\n+\n+enum eal_dev_req_type {\n+\tEAL_DEV_REQ_TYPE_ATTACH,\n+\tEAL_DEV_REQ_TYPE_DETACH,\n+\tEAL_DEV_REQ_TYPE_ATTACH_ROLLBACK,\n+\tEAL_DEV_REQ_TYPE_DETACH_ROLLBACK,\n+};\n+\n+struct eal_dev_mp_req {\n+\tenum eal_dev_req_type t;\n+\tchar devargs[EAL_DEV_MP_DEV_ARGS_MAX_LEN];\n+\tint result;\n+};\n+\n+/**\n+ * this is a synchronous wrapper for secondary process send\n+ * request to primary process, this is invoked when an attach\n+ * or detach request issued from primary process.\n+ */\n+int eal_dev_hotplug_request_to_primary(struct eal_dev_mp_req *req);\n+\n+/**\n+ * this is a synchronous wrapper for primary process send\n+ * request to secondary process, this is invoked when an attach\n+ * or detach request issued from secondary process.\n+ */\n+int eal_dev_hotplug_request_to_secondary(struct eal_dev_mp_req *req);\n+\n+\n+#endif /* _HOTPLUG_MP_H_ */\ndiff --git a/lib/librte_eal/common/include/rte_dev.h b/lib/librte_eal/common/include/rte_dev.h\nindex 7a30362c0..266331acd 100644\n--- a/lib/librte_eal/common/include/rte_dev.h\n+++ b/lib/librte_eal/common/include/rte_dev.h\n@@ -190,6 +190,9 @@ int rte_eal_dev_detach(struct rte_device *dev);\n \n /**\n * Hotplug add a given device to a specific bus.\n+ * In multi-process, this function will inform all other processes\n+ * to hotplug add the same device. Any failure on other process rollback\n+ * the action.\n *\n * @param busname\n * The bus name the device is added to.\n@@ -219,6 +222,9 @@ int __rte_experimental rte_dev_probe(const char *devargs);\n \n /**\n * Hotplug remove a given device from a specific bus.\n+ * In multi-process, this function will inform all other processes\n+ * to hotplug remove the same device. Any failure on other process\n+ * will rollback the action.\n *\n * @param busname\n * The bus name the device is removed from.\n@@ -234,6 +240,9 @@ int rte_eal_hotplug_remove(const char *busname, const char *devname);\n * @b EXPERIMENTAL: this API may change without prior notice\n *\n * Remove one device.\n+ * In multi-process, this function will inform all other processes\n+ * to hotplug remove the same device. Any failure on other process\n+ * will rollback the action.\n *\n * @param dev\n * Data structure of the device to remove.\ndiff --git a/lib/librte_eal/common/meson.build b/lib/librte_eal/common/meson.build\nindex b7fc98499..04c414356 100644\n--- a/lib/librte_eal/common/meson.build\n+++ b/lib/librte_eal/common/meson.build\n@@ -28,6 +28,7 @@ common_sources = files(\n \t'eal_common_thread.c',\n \t'eal_common_timer.c',\n \t'eal_common_uuid.c',\n+\t'hotplug_mp.c',\n \t'malloc_elem.c',\n \t'malloc_heap.c',\n \t'malloc_mp.c',\ndiff --git a/lib/librte_eal/linuxapp/eal/Makefile b/lib/librte_eal/linuxapp/eal/Makefile\nindex fd92c75c2..58455c1a6 100644\n--- a/lib/librte_eal/linuxapp/eal/Makefile\n+++ b/lib/librte_eal/linuxapp/eal/Makefile\n@@ -70,6 +70,7 @@ SRCS-$(CONFIG_RTE_EXEC_ENV_LINUXAPP) += eal_common_proc.c\n SRCS-$(CONFIG_RTE_EXEC_ENV_LINUXAPP) += eal_common_fbarray.c\n SRCS-$(CONFIG_RTE_EXEC_ENV_LINUXAPP) += eal_common_uuid.c\n SRCS-$(CONFIG_RTE_EXEC_ENV_LINUXAPP) += rte_malloc.c\n+SRCS-$(CONFIG_RTE_EXEC_ENV_LINUXAPP) += hotplug_mp.c\n SRCS-$(CONFIG_RTE_EXEC_ENV_LINUXAPP) += malloc_elem.c\n SRCS-$(CONFIG_RTE_EXEC_ENV_LINUXAPP) += malloc_heap.c\n SRCS-$(CONFIG_RTE_EXEC_ENV_LINUXAPP) += malloc_mp.c\ndiff --git a/lib/librte_eal/linuxapp/eal/eal.c b/lib/librte_eal/linuxapp/eal/eal.c\nindex e59ac6577..f2c90c528 100644\n--- a/lib/librte_eal/linuxapp/eal/eal.c\n+++ b/lib/librte_eal/linuxapp/eal/eal.c\n@@ -865,6 +865,12 @@ rte_eal_init(int argc, char **argv)\n \t\t}\n \t}\n \n+\t/* register mp action callbacks for hotplug */\n+\tif (rte_dev_hotplug_mp_init() < 0) {\n+\t\trte_eal_init_alert(\"failed to register mp callback for hotplug\\n\");\n+\t\treturn -1;\n+\t}\n+\n \tif (rte_bus_scan()) {\n \t\trte_eal_init_alert(\"Cannot scan the buses for devices\\n\");\n \t\trte_errno = ENODEV;\n", "prefixes": [ "v16", "2/6" ] }