From patchwork Tue Jul 14 14:21:01 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Ophir Munk X-Patchwork-Id: 74011 X-Patchwork-Delegate: rasland@nvidia.com Return-Path: X-Original-To: patchwork@inbox.dpdk.org Delivered-To: patchwork@inbox.dpdk.org Received: from dpdk.org (dpdk.org [92.243.14.124]) by inbox.dpdk.org (Postfix) with ESMTP id 3C3A1A0540; Tue, 14 Jul 2020 16:22:31 +0200 (CEST) Received: from [92.243.14.124] (localhost [127.0.0.1]) by dpdk.org (Postfix) with ESMTP id 93E9E1D5DD; Tue, 14 Jul 2020 16:21:22 +0200 (CEST) Received: from EUR04-DB3-obe.outbound.protection.outlook.com (mail-eopbgr60063.outbound.protection.outlook.com [40.107.6.63]) by dpdk.org (Postfix) with ESMTP id 86C6D1D54C for ; Tue, 14 Jul 2020 16:21:17 +0200 (CEST) ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=ahzCEMUpehXqEnGWQSSn+F4GTBUqOglYtQmNv9xEa5MMqPdHhrec7wD5XayMoZr2DpHDxWQHGEVX69qsY6lHWTkfitsPBUkg4yIitO2PO0n0UMmlKm4rPlHoppA+vaItRGH1TbJA4xgJNnY5r6l9/ionfTi77HumAKWn/FfHU5BQUCELLrPagaPKwXH9HsVXiC0dyBKOSfFw/am/dutOCVHmzyK5tFZrraYXa3/AeVQR+tvS8wQjt6FH/mz9zd08Lo3AFtVOsJRiOc6iRmWGh11eLpesB/9uFKSnhPC/2TdEduXJzlZJZtuyrdy8sv+DHdc2bGE23PR8I+cCm5Caog== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=DUU+wfc2iNyGVsZPMU8Anat+qwxBUiZC1ECTLQqRTpg=; b=MxhXlOHusQaCsx150xu0aek0N6fluk0QCMDJY55KDtU0BB+2rJtESYGa2C4+SxhJ2Q15aYeNL7ldmAXCW/GCl0+A3mZMZllWDJIMSp9GHJwXCioLEx+QXwAJDif2UQ57xYPMbggQ4NrCOKyGFnmxZSyOFHEBCY/Kcl3DaOJFZyqJ9Qbu2tRK4nJxV7i8LJeTcVbfESmGybw2UcjS8N8aHhRUwetyMf7h9hfRF/1ABV58BR3VSUGHVlyPN9qnSks2LepqeFlBmrWM6IjUzsrczfioUpkUQ8FLY7MoLX9INKxFbdbBpFnlHa1jx7e8CMTZgrvwaI5gdsktUSszSQDbig== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=mellanox.com; dmarc=pass action=none header.from=mellanox.com; dkim=pass header.d=mellanox.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=Mellanox.com; s=selector1; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=DUU+wfc2iNyGVsZPMU8Anat+qwxBUiZC1ECTLQqRTpg=; b=QNIK/2uf5eZWDZ6Xu9m4kwI4xW5TxL3G5pp0mJl2fd2e6idKGPeSG54zi2glSo9Kt7Rn0tz10dwgpZ25VIz6JNPr1NwABvmlKAOwyMvc4LDeYSes8JSB3PUe/w+gixIXSxLfs/cbI+wpPX+jM7G7vostmGyqRk2scdhZgqhccYk= Authentication-Results: dpdk.org; dkim=none (message not signed) header.d=none;dpdk.org; dmarc=none action=none header.from=mellanox.com; Received: from AM0PR05MB4209.eurprd05.prod.outlook.com (2603:10a6:208:61::22) by AM0PR05MB4625.eurprd05.prod.outlook.com (2603:10a6:208:b6::17) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.3174.24; Tue, 14 Jul 2020 14:21:16 +0000 Received: from AM0PR05MB4209.eurprd05.prod.outlook.com ([fe80::3949:70c3:126:3972]) by AM0PR05MB4209.eurprd05.prod.outlook.com ([fe80::3949:70c3:126:3972%7]) with mapi id 15.20.3174.026; Tue, 14 Jul 2020 14:21:16 +0000 From: Ophir Munk To: dev@dpdk.org Cc: Raslan Darawsheh , Ophir Munk , Matan Azrad Date: Tue, 14 Jul 2020 14:21:01 +0000 Message-Id: <20200714142102.30606-8-ophirmu@mellanox.com> X-Mailer: git-send-email 2.8.4 In-Reply-To: <20200714142102.30606-1-ophirmu@mellanox.com> References: <20200714142102.30606-1-ophirmu@mellanox.com> X-ClientProxiedBy: AM3PR05CA0100.eurprd05.prod.outlook.com (2603:10a6:207:1::26) To AM0PR05MB4209.eurprd05.prod.outlook.com (2603:10a6:208:61::22) MIME-Version: 1.0 X-MS-Exchange-MessageSentRepresentingType: 1 Received: from mellanox.com (37.142.13.130) by AM3PR05CA0100.eurprd05.prod.outlook.com (2603:10a6:207:1::26) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.3174.21 via Frontend Transport; Tue, 14 Jul 2020 14:21:15 +0000 X-Mailer: git-send-email 2.8.4 X-Originating-IP: [37.142.13.130] X-MS-PublicTrafficType: Email X-MS-Office365-Filtering-HT: Tenant X-MS-Office365-Filtering-Correlation-Id: b9b20f73-7c6f-4592-d867-08d828012b52 X-MS-TrafficTypeDiagnostic: AM0PR05MB4625: X-LD-Processed: a652971c-7d2e-4d9b-a6a4-d149256f461b,ExtFwd X-MS-Exchange-Transport-Forked: True X-Microsoft-Antispam-PRVS: X-MS-Oob-TLC-OOBClassifiers: OLM:216; X-MS-Exchange-SenderADCheck: 1 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: j0s37D24mjUxhaQ8dH6Ltxdbxs1xnOHyNkj04xy3OEljri7SqV96FBhf7ga8yL+JcmDfiW/h0W7Xaqtrmrm0+mQnur61RiV4wefKIn5rgxW7O9FPZfhHpTmnlvld3E1B29iTfhsU14NJtGgS718m7Dc/QWfZoPNSRINBXVKPfXekbyjEmwnNWzl1gkJaMdbWP2urfOJP4S34r7J7TeAbGlV06nDhsAnVOMLVDqFgdhMGzqVYb+N+YIEpGDDDedIfZBRcP9pAbVBSwn/mLxlxo50yKU1SD5pwtjR8lDcIO8JZttSev19FpY1fEnPYORVHE2/LQqp4xzjrMCh0eV5lzwhppM3BX3CLWXzb5ZRthz2Eis2h/TvCNi/ujOTqWvLW X-Forefront-Antispam-Report: CIP:255.255.255.255; CTRY:; LANG:en; SCL:1; SRV:; IPV:NLI; SFV:NSPM; H:AM0PR05MB4209.eurprd05.prod.outlook.com; PTR:; CAT:NONE; SFTY:; SFS:(4636009)(136003)(366004)(376002)(39850400004)(346002)(396003)(36756003)(86362001)(1076003)(30864003)(8886007)(6666004)(66946007)(316002)(66476007)(52116002)(7696005)(5660300002)(26005)(66556008)(54906003)(478600001)(6916009)(83380400001)(16526019)(8676002)(8936002)(4326008)(956004)(2616005)(107886003)(2906002)(55016002)(186003)(21314003); DIR:OUT; SFP:1101; X-MS-Exchange-AntiSpam-MessageData: 4eZQpJuWYIjKJF+Ug0DmL5ELUd90s0XtFC23Orw32jjC/440dXFBPdCwtjk9QaWJ4CaSV4U/qeswlbvdHCJSA9pzk6h6m/W6jC9qedZ3y+j6WlB7lZHad66H8OwX3afU8kLEOhNY80Zb5/rsDIb+/j7fwYfXUTxa+jI5wqKEz6t5imKFhQ/m6ZPtwqikA/rWhqoHzkdLpAEw35NRjIiudQ9xoz3N8BYVZXVB72aPnJ9QOU/cbzVkA59YMEIR4zwnGj/qEpk4a2PrwusAtxXQyDBhxa8qmElHTYsxZUH5PF13tNRLFDO6KuAjpdBom8o9Bp09fD5Y+JRNeAyMu6JEAYdcD6mtz9YhcXWhwOfSyk2UnH9PzmRMShYFnCk0slPHELLtv65qQsp2Gry2sZOsMpmUxra+igRkeIWJXU2kTf5ep1QE0vG8aaU35I3XVhYhA5sDLgv9Y3b3DU6MSa1sahKaW2i5jlRalIZOHguGD9ZtIAGxfgfx+Xk/EbmISkkD X-OriginatorOrg: Mellanox.com X-MS-Exchange-CrossTenant-Network-Message-Id: b9b20f73-7c6f-4592-d867-08d828012b52 X-MS-Exchange-CrossTenant-AuthSource: AM0PR05MB4209.eurprd05.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-OriginalArrivalTime: 14 Jul 2020 14:21:16.4120 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: a652971c-7d2e-4d9b-a6a4-d149256f461b X-MS-Exchange-CrossTenant-MailboxType: HOSTED X-MS-Exchange-CrossTenant-UserPrincipalName: +dflmJFuMO6nPD6h8JeATTvmJ1O3DY1MXLr/f2qLCS2upN7TylBpkDCX8Wpwl3RDVT3o3IJdC59nOGHAazEcxg== X-MS-Exchange-Transport-CrossTenantHeadersStamped: AM0PR05MB4625 Subject: [dpdk-dev] [PATCH v1 7/8] net/mlx5: refactor multi process communication X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Sender: "dev" 1. The shared data communication between the primary and the secondary processes is implemented using Linux API. Move the Linux API code under linux directory (file linux/mlx5_os.c). 2. File net/mlx5/mlx5_mp.c handles requests to the primary and secondary processes (e.g. start_rxtx, stop_rxtx). It is Linux based so it is moved under linux (new file linux/mlx5_mp_os.c). Signed-off-by: Ophir Munk Acked-by: Matan Azrad --- drivers/net/mlx5/Makefile | 2 +- drivers/net/mlx5/linux/meson.build | 1 + drivers/net/mlx5/linux/mlx5_mp_os.c | 211 ++++++++++++++++++++++++++++++++++++ drivers/net/mlx5/linux/mlx5_os.c | 111 +++++++++++++++++++ drivers/net/mlx5/meson.build | 1 - drivers/net/mlx5/mlx5.c | 114 +------------------ drivers/net/mlx5/mlx5.h | 13 ++- drivers/net/mlx5/mlx5_mp.c | 211 ------------------------------------ drivers/net/mlx5/mlx5_trigger.c | 4 +- 9 files changed, 336 insertions(+), 332 deletions(-) create mode 100644 drivers/net/mlx5/linux/mlx5_mp_os.c delete mode 100644 drivers/net/mlx5/mlx5_mp.c diff --git a/drivers/net/mlx5/Makefile b/drivers/net/mlx5/Makefile index a458402..bb45294 100644 --- a/drivers/net/mlx5/Makefile +++ b/drivers/net/mlx5/Makefile @@ -29,12 +29,12 @@ SRCS-$(CONFIG_RTE_LIBRTE_MLX5_PMD) += mlx5_flow.c SRCS-$(CONFIG_RTE_LIBRTE_MLX5_PMD) += mlx5_flow_meter.c SRCS-$(CONFIG_RTE_LIBRTE_MLX5_PMD) += mlx5_flow_dv.c SRCS-$(CONFIG_RTE_LIBRTE_MLX5_PMD) += mlx5_flow_verbs.c -SRCS-$(CONFIG_RTE_LIBRTE_MLX5_PMD) += mlx5_mp.c SRCS-$(CONFIG_RTE_LIBRTE_MLX5_PMD) += mlx5_utils.c SRCS-$(CONFIG_RTE_LIBRTE_MLX5_PMD) += linux/mlx5_socket.c SRCS-$(CONFIG_RTE_LIBRTE_MLX5_PMD) += linux/mlx5_os.c SRCS-$(CONFIG_RTE_LIBRTE_MLX5_PMD) += linux/mlx5_ethdev_os.c SRCS-$(CONFIG_RTE_LIBRTE_MLX5_PMD) += linux/mlx5_verbs.c +SRCS-$(CONFIG_RTE_LIBRTE_MLX5_PMD) += linux/mlx5_mp_os.c # Basic CFLAGS. CFLAGS += -O3 diff --git a/drivers/net/mlx5/linux/meson.build b/drivers/net/mlx5/linux/meson.build index 14eed03..2def8e3 100644 --- a/drivers/net/mlx5/linux/meson.build +++ b/drivers/net/mlx5/linux/meson.build @@ -7,5 +7,6 @@ sources += files( 'mlx5_os.c', 'mlx5_ethdev_os.c', 'mlx5_verbs.c', + 'mlx5_mp_os.c', ) diff --git a/drivers/net/mlx5/linux/mlx5_mp_os.c b/drivers/net/mlx5/linux/mlx5_mp_os.c new file mode 100644 index 0000000..65b0025 --- /dev/null +++ b/drivers/net/mlx5/linux/mlx5_mp_os.c @@ -0,0 +1,211 @@ +/* SPDX-License-Identifier: BSD-3-Clause + * Copyright 2019 6WIND S.A. + * Copyright 2019 Mellanox Technologies, Ltd + */ + +#include +#include + +#include +#include +#include + +#include +#include + +#include "mlx5.h" +#include "mlx5_rxtx.h" +#include "mlx5_utils.h" + +int +mlx5_mp_os_primary_handle(const struct rte_mp_msg *mp_msg, const void *peer) +{ + struct rte_mp_msg mp_res; + struct mlx5_mp_param *res = (struct mlx5_mp_param *)mp_res.param; + const struct mlx5_mp_param *param = + (const struct mlx5_mp_param *)mp_msg->param; + struct rte_eth_dev *dev; + struct mlx5_priv *priv; + struct mr_cache_entry entry; + uint32_t lkey; + int ret; + + MLX5_ASSERT(rte_eal_process_type() == RTE_PROC_PRIMARY); + if (!rte_eth_dev_is_valid_port(param->port_id)) { + rte_errno = ENODEV; + DRV_LOG(ERR, "port %u invalid port ID", param->port_id); + return -rte_errno; + } + dev = &rte_eth_devices[param->port_id]; + priv = dev->data->dev_private; + switch (param->type) { + case MLX5_MP_REQ_CREATE_MR: + mp_init_msg(&priv->mp_id, &mp_res, param->type); + lkey = mlx5_mr_create_primary(priv->sh->pd, + &priv->sh->share_cache, + &entry, param->args.addr, + priv->config.mr_ext_memseg_en); + if (lkey == UINT32_MAX) + res->result = -rte_errno; + ret = rte_mp_reply(&mp_res, peer); + break; + case MLX5_MP_REQ_VERBS_CMD_FD: + mp_init_msg(&priv->mp_id, &mp_res, param->type); + mp_res.num_fds = 1; + mp_res.fds[0] = ((struct ibv_context *)priv->sh->ctx)->cmd_fd; + res->result = 0; + ret = rte_mp_reply(&mp_res, peer); + break; + case MLX5_MP_REQ_QUEUE_STATE_MODIFY: + mp_init_msg(&priv->mp_id, &mp_res, param->type); + res->result = mlx5_queue_state_modify_primary + (dev, ¶m->args.state_modify); + ret = rte_mp_reply(&mp_res, peer); + break; + default: + rte_errno = EINVAL; + DRV_LOG(ERR, "port %u invalid mp request type", + dev->data->port_id); + return -rte_errno; + } + return ret; +} + +/** + * IPC message handler of a secondary process. + * + * @param[in] dev + * Pointer to Ethernet structure. + * @param[in] peer + * Pointer to the peer socket path. + * + * @return + * 0 on success, a negative errno value otherwise and rte_errno is set. + */ +int +mlx5_mp_os_secondary_handle(const struct rte_mp_msg *mp_msg, const void *peer) +{ + struct rte_mp_msg mp_res; + struct mlx5_mp_param *res = (struct mlx5_mp_param *)mp_res.param; + const struct mlx5_mp_param *param = + (const struct mlx5_mp_param *)mp_msg->param; + struct rte_eth_dev *dev; + struct mlx5_priv *priv; + int ret; + + MLX5_ASSERT(rte_eal_process_type() == RTE_PROC_SECONDARY); + if (!rte_eth_dev_is_valid_port(param->port_id)) { + rte_errno = ENODEV; + DRV_LOG(ERR, "port %u invalid port ID", param->port_id); + return -rte_errno; + } + dev = &rte_eth_devices[param->port_id]; + priv = dev->data->dev_private; + switch (param->type) { + case MLX5_MP_REQ_START_RXTX: + DRV_LOG(INFO, "port %u starting datapath", dev->data->port_id); + rte_mb(); + dev->rx_pkt_burst = mlx5_select_rx_function(dev); + dev->tx_pkt_burst = mlx5_select_tx_function(dev); + mp_init_msg(&priv->mp_id, &mp_res, param->type); + res->result = 0; + ret = rte_mp_reply(&mp_res, peer); + break; + case MLX5_MP_REQ_STOP_RXTX: + DRV_LOG(INFO, "port %u stopping datapath", dev->data->port_id); + dev->rx_pkt_burst = removed_rx_burst; + dev->tx_pkt_burst = removed_tx_burst; + rte_mb(); + mp_init_msg(&priv->mp_id, &mp_res, param->type); + res->result = 0; + ret = rte_mp_reply(&mp_res, peer); + break; + default: + rte_errno = EINVAL; + DRV_LOG(ERR, "port %u invalid mp request type", + dev->data->port_id); + return -rte_errno; + } + return ret; +} + +/** + * Broadcast request of stopping/starting data-path to secondary processes. + * + * @param[in] dev + * Pointer to Ethernet structure. + * @param[in] type + * Request type. + */ +static void +mp_req_on_rxtx(struct rte_eth_dev *dev, enum mlx5_mp_req_type type) +{ + struct rte_mp_msg mp_req; + struct rte_mp_msg *mp_res; + struct rte_mp_reply mp_rep; + struct mlx5_mp_param *res; + struct timespec ts = {.tv_sec = MLX5_MP_REQ_TIMEOUT_SEC, .tv_nsec = 0}; + struct mlx5_priv *priv = dev->data->dev_private; + int ret; + int i; + + MLX5_ASSERT(rte_eal_process_type() == RTE_PROC_PRIMARY); + if (!mlx5_shared_data->secondary_cnt) + return; + if (type != MLX5_MP_REQ_START_RXTX && type != MLX5_MP_REQ_STOP_RXTX) { + DRV_LOG(ERR, "port %u unknown request (req_type %d)", + dev->data->port_id, type); + return; + } + mp_init_msg(&priv->mp_id, &mp_req, type); + ret = rte_mp_request_sync(&mp_req, &mp_rep, &ts); + if (ret) { + if (rte_errno != ENOTSUP) + DRV_LOG(ERR, "port %u failed to request stop/start Rx/Tx (%d)", + dev->data->port_id, type); + goto exit; + } + if (mp_rep.nb_sent != mp_rep.nb_received) { + DRV_LOG(ERR, + "port %u not all secondaries responded (req_type %d)", + dev->data->port_id, type); + goto exit; + } + for (i = 0; i < mp_rep.nb_received; i++) { + mp_res = &mp_rep.msgs[i]; + res = (struct mlx5_mp_param *)mp_res->param; + if (res->result) { + DRV_LOG(ERR, "port %u request failed on secondary #%d", + dev->data->port_id, i); + goto exit; + } + } +exit: + free(mp_rep.msgs); +} + +/** + * Broadcast request of starting data-path to secondary processes. The request + * is synchronous. + * + * @param[in] dev + * Pointer to Ethernet structure. + */ +void +mlx5_mp_os_req_start_rxtx(struct rte_eth_dev *dev) +{ + mp_req_on_rxtx(dev, MLX5_MP_REQ_START_RXTX); +} + +/** + * Broadcast request of stopping data-path to secondary processes. The request + * is synchronous. + * + * @param[in] dev + * Pointer to Ethernet structure. + */ +void +mlx5_mp_os_req_stop_rxtx(struct rte_eth_dev *dev) +{ + mp_req_on_rxtx(dev, MLX5_MP_REQ_STOP_RXTX); +} diff --git a/drivers/net/mlx5/linux/mlx5_os.c b/drivers/net/mlx5/linux/mlx5_os.c index a853219..2385d69 100644 --- a/drivers/net/mlx5/linux/mlx5_os.c +++ b/drivers/net/mlx5/linux/mlx5_os.c @@ -66,6 +66,14 @@ #define MLX5DV_CONTEXT_FLAGS_CQE_128B_COMP (1 << 4) #endif +static const char *MZ_MLX5_PMD_SHARED_DATA = "mlx5_pmd_shared_data"; + +/* Spinlock for mlx5_shared_data allocation. */ +static rte_spinlock_t mlx5_shared_data_lock = RTE_SPINLOCK_INITIALIZER; + +/* Process local data for secondary processes. */ +static struct mlx5_local_data mlx5_local_data; + /** * Get mlx5 device attributes. The glue function query_device_ex() is called * with out parameter of type 'struct ibv_device_attr_ex *'. Then fill in mlx5 @@ -356,6 +364,109 @@ mlx5_os_free_shared_dr(struct mlx5_priv *priv) } /** + * Initialize shared data between primary and secondary process. + * + * A memzone is reserved by primary process and secondary processes attach to + * the memzone. + * + * @return + * 0 on success, a negative errno value otherwise and rte_errno is set. + */ +static int +mlx5_init_shared_data(void) +{ + const struct rte_memzone *mz; + int ret = 0; + + rte_spinlock_lock(&mlx5_shared_data_lock); + if (mlx5_shared_data == NULL) { + if (rte_eal_process_type() == RTE_PROC_PRIMARY) { + /* Allocate shared memory. */ + mz = rte_memzone_reserve(MZ_MLX5_PMD_SHARED_DATA, + sizeof(*mlx5_shared_data), + SOCKET_ID_ANY, 0); + if (mz == NULL) { + DRV_LOG(ERR, + "Cannot allocate mlx5 shared data"); + ret = -rte_errno; + goto error; + } + mlx5_shared_data = mz->addr; + memset(mlx5_shared_data, 0, sizeof(*mlx5_shared_data)); + rte_spinlock_init(&mlx5_shared_data->lock); + } else { + /* Lookup allocated shared memory. */ + mz = rte_memzone_lookup(MZ_MLX5_PMD_SHARED_DATA); + if (mz == NULL) { + DRV_LOG(ERR, + "Cannot attach mlx5 shared data"); + ret = -rte_errno; + goto error; + } + mlx5_shared_data = mz->addr; + memset(&mlx5_local_data, 0, sizeof(mlx5_local_data)); + } + } +error: + rte_spinlock_unlock(&mlx5_shared_data_lock); + return ret; +} + +/** + * PMD global initialization. + * + * Independent from individual device, this function initializes global + * per-PMD data structures distinguishing primary and secondary processes. + * Hence, each initialization is called once per a process. + * + * @return + * 0 on success, a negative errno value otherwise and rte_errno is set. + */ +static int +mlx5_init_once(void) +{ + struct mlx5_shared_data *sd; + struct mlx5_local_data *ld = &mlx5_local_data; + int ret = 0; + + if (mlx5_init_shared_data()) + return -rte_errno; + sd = mlx5_shared_data; + MLX5_ASSERT(sd); + rte_spinlock_lock(&sd->lock); + switch (rte_eal_process_type()) { + case RTE_PROC_PRIMARY: + if (sd->init_done) + break; + LIST_INIT(&sd->mem_event_cb_list); + rte_rwlock_init(&sd->mem_event_rwlock); + rte_mem_event_callback_register("MLX5_MEM_EVENT_CB", + mlx5_mr_mem_event_cb, NULL); + ret = mlx5_mp_init_primary(MLX5_MP_NAME, + mlx5_mp_os_primary_handle); + if (ret) + goto out; + sd->init_done = true; + break; + case RTE_PROC_SECONDARY: + if (ld->init_done) + break; + ret = mlx5_mp_init_secondary(MLX5_MP_NAME, + mlx5_mp_os_secondary_handle); + if (ret) + goto out; + ++sd->secondary_cnt; + ld->init_done = true; + break; + default: + break; + } +out: + rte_spinlock_unlock(&sd->lock); + return ret; +} + +/** * Spawn an Ethernet device from Verbs information. * * @param dpdk_dev diff --git a/drivers/net/mlx5/meson.build b/drivers/net/mlx5/meson.build index e95ce02..c8dcb76 100644 --- a/drivers/net/mlx5/meson.build +++ b/drivers/net/mlx5/meson.build @@ -22,7 +22,6 @@ sources = files( 'mlx5_rxmode.c', 'mlx5_rxq.c', 'mlx5_rxtx.c', - 'mlx5_mp.c', 'mlx5_stats.c', 'mlx5_trigger.c', 'mlx5_txq.c', diff --git a/drivers/net/mlx5/mlx5.c b/drivers/net/mlx5/mlx5.c index 0c654ed..10a0306 100644 --- a/drivers/net/mlx5/mlx5.c +++ b/drivers/net/mlx5/mlx5.c @@ -167,16 +167,11 @@ /* Flow memory reclaim mode. */ #define MLX5_RECLAIM_MEM "reclaim_mem_mode" -static const char *MZ_MLX5_PMD_SHARED_DATA = "mlx5_pmd_shared_data"; - /* Shared memory between primary and secondary processes. */ struct mlx5_shared_data *mlx5_shared_data; -/* Spinlock for mlx5_shared_data allocation. */ -static rte_spinlock_t mlx5_shared_data_lock = RTE_SPINLOCK_INITIALIZER; - -/* Process local data for secondary processes. */ -static struct mlx5_local_data mlx5_local_data; +/** Driver-specific log messages type. */ +int mlx5_logtype; static LIST_HEAD(, mlx5_dev_ctx_shared) mlx5_dev_ctx_list = LIST_HEAD_INITIALIZER(); @@ -966,55 +961,6 @@ mlx5_alloc_table_hash_list(struct mlx5_priv *priv) } /** - * Initialize shared data between primary and secondary process. - * - * A memzone is reserved by primary process and secondary processes attach to - * the memzone. - * - * @return - * 0 on success, a negative errno value otherwise and rte_errno is set. - */ -static int -mlx5_init_shared_data(void) -{ - const struct rte_memzone *mz; - int ret = 0; - - rte_spinlock_lock(&mlx5_shared_data_lock); - if (mlx5_shared_data == NULL) { - if (rte_eal_process_type() == RTE_PROC_PRIMARY) { - /* Allocate shared memory. */ - mz = rte_memzone_reserve(MZ_MLX5_PMD_SHARED_DATA, - sizeof(*mlx5_shared_data), - SOCKET_ID_ANY, 0); - if (mz == NULL) { - DRV_LOG(ERR, - "Cannot allocate mlx5 shared data"); - ret = -rte_errno; - goto error; - } - mlx5_shared_data = mz->addr; - memset(mlx5_shared_data, 0, sizeof(*mlx5_shared_data)); - rte_spinlock_init(&mlx5_shared_data->lock); - } else { - /* Lookup allocated shared memory. */ - mz = rte_memzone_lookup(MZ_MLX5_PMD_SHARED_DATA); - if (mz == NULL) { - DRV_LOG(ERR, - "Cannot attach mlx5 shared data"); - ret = -rte_errno; - goto error; - } - mlx5_shared_data = mz->addr; - memset(&mlx5_local_data, 0, sizeof(mlx5_local_data)); - } - } -error: - rte_spinlock_unlock(&mlx5_shared_data_lock); - return ret; -} - -/** * Retrieve integer value from environment variable. * * @param[in] name @@ -1155,7 +1101,7 @@ mlx5_dev_close(struct rte_eth_dev *dev) dev->tx_pkt_burst = removed_tx_burst; rte_wmb(); /* Disable datapath on secondary process. */ - mlx5_mp_req_stop_rxtx(dev); + mlx5_mp_os_req_stop_rxtx(dev); if (priv->rxqs != NULL) { /* XXX race condition if mlx5_rx_burst() is still running. */ usleep(1000); @@ -1461,60 +1407,6 @@ mlx5_args(struct mlx5_dev_config *config, struct rte_devargs *devargs) } /** - * PMD global initialization. - * - * Independent from individual device, this function initializes global - * per-PMD data structures distinguishing primary and secondary processes. - * Hence, each initialization is called once per a process. - * - * @return - * 0 on success, a negative errno value otherwise and rte_errno is set. - */ -int -mlx5_init_once(void) -{ - struct mlx5_shared_data *sd; - struct mlx5_local_data *ld = &mlx5_local_data; - int ret = 0; - - if (mlx5_init_shared_data()) - return -rte_errno; - sd = mlx5_shared_data; - MLX5_ASSERT(sd); - rte_spinlock_lock(&sd->lock); - switch (rte_eal_process_type()) { - case RTE_PROC_PRIMARY: - if (sd->init_done) - break; - LIST_INIT(&sd->mem_event_cb_list); - rte_rwlock_init(&sd->mem_event_rwlock); - rte_mem_event_callback_register("MLX5_MEM_EVENT_CB", - mlx5_mr_mem_event_cb, NULL); - ret = mlx5_mp_init_primary(MLX5_MP_NAME, - mlx5_mp_primary_handle); - if (ret) - goto out; - sd->init_done = true; - break; - case RTE_PROC_SECONDARY: - if (ld->init_done) - break; - ret = mlx5_mp_init_secondary(MLX5_MP_NAME, - mlx5_mp_secondary_handle); - if (ret) - goto out; - ++sd->secondary_cnt; - ld->init_done = true; - break; - default: - break; - } -out: - rte_spinlock_unlock(&sd->lock); - return ret; -} - -/** * Configures the minimal amount of data to inline into WQE * while sending packets. * diff --git a/drivers/net/mlx5/mlx5.h b/drivers/net/mlx5/mlx5.h index 87cfbfe..fa223a1 100644 --- a/drivers/net/mlx5/mlx5.h +++ b/drivers/net/mlx5/mlx5.h @@ -710,7 +710,6 @@ void mlx5_set_min_inline(struct mlx5_dev_spawn_data *spawn, void mlx5_set_metadata_mask(struct rte_eth_dev *dev); int mlx5_dev_check_sibling_config(struct mlx5_priv *priv, struct mlx5_dev_config *config); -int mlx5_init_once(void); int mlx5_dev_configure(struct rte_eth_dev *dev); int mlx5_dev_infos_get(struct rte_eth_dev *dev, struct rte_eth_dev_info *info); int mlx5_fw_version_get(struct rte_eth_dev *dev, char *fw_ver, size_t fw_size); @@ -893,11 +892,13 @@ void mlx5_flow_rxq_dynf_metadata_set(struct rte_eth_dev *dev); int mlx5_flow_get_aged_flows(struct rte_eth_dev *dev, void **contexts, uint32_t nb_contexts, struct rte_flow_error *error); -/* mlx5_mp.c */ -int mlx5_mp_primary_handle(const struct rte_mp_msg *mp_msg, const void *peer); -int mlx5_mp_secondary_handle(const struct rte_mp_msg *mp_msg, const void *peer); -void mlx5_mp_req_start_rxtx(struct rte_eth_dev *dev); -void mlx5_mp_req_stop_rxtx(struct rte_eth_dev *dev); +/* mlx5_mp_os.c */ +int mlx5_mp_os_primary_handle(const struct rte_mp_msg *mp_msg, + const void *peer); +int mlx5_mp_os_secondary_handle(const struct rte_mp_msg *mp_msg, + const void *peer); +void mlx5_mp_os_req_start_rxtx(struct rte_eth_dev *dev); +void mlx5_mp_os_req_stop_rxtx(struct rte_eth_dev *dev); /* mlx5_socket.c */ diff --git a/drivers/net/mlx5/mlx5_mp.c b/drivers/net/mlx5/mlx5_mp.c deleted file mode 100644 index a2b5c40..0000000 --- a/drivers/net/mlx5/mlx5_mp.c +++ /dev/null @@ -1,211 +0,0 @@ -/* SPDX-License-Identifier: BSD-3-Clause - * Copyright 2019 6WIND S.A. - * Copyright 2019 Mellanox Technologies, Ltd - */ - -#include -#include - -#include -#include -#include - -#include -#include - -#include "mlx5.h" -#include "mlx5_rxtx.h" -#include "mlx5_utils.h" - -int -mlx5_mp_primary_handle(const struct rte_mp_msg *mp_msg, const void *peer) -{ - struct rte_mp_msg mp_res; - struct mlx5_mp_param *res = (struct mlx5_mp_param *)mp_res.param; - const struct mlx5_mp_param *param = - (const struct mlx5_mp_param *)mp_msg->param; - struct rte_eth_dev *dev; - struct mlx5_priv *priv; - struct mr_cache_entry entry; - uint32_t lkey; - int ret; - - MLX5_ASSERT(rte_eal_process_type() == RTE_PROC_PRIMARY); - if (!rte_eth_dev_is_valid_port(param->port_id)) { - rte_errno = ENODEV; - DRV_LOG(ERR, "port %u invalid port ID", param->port_id); - return -rte_errno; - } - dev = &rte_eth_devices[param->port_id]; - priv = dev->data->dev_private; - switch (param->type) { - case MLX5_MP_REQ_CREATE_MR: - mp_init_msg(&priv->mp_id, &mp_res, param->type); - lkey = mlx5_mr_create_primary(priv->sh->pd, - &priv->sh->share_cache, - &entry, param->args.addr, - priv->config.mr_ext_memseg_en); - if (lkey == UINT32_MAX) - res->result = -rte_errno; - ret = rte_mp_reply(&mp_res, peer); - break; - case MLX5_MP_REQ_VERBS_CMD_FD: - mp_init_msg(&priv->mp_id, &mp_res, param->type); - mp_res.num_fds = 1; - mp_res.fds[0] = ((struct ibv_context *)priv->sh->ctx)->cmd_fd; - res->result = 0; - ret = rte_mp_reply(&mp_res, peer); - break; - case MLX5_MP_REQ_QUEUE_STATE_MODIFY: - mp_init_msg(&priv->mp_id, &mp_res, param->type); - res->result = mlx5_queue_state_modify_primary - (dev, ¶m->args.state_modify); - ret = rte_mp_reply(&mp_res, peer); - break; - default: - rte_errno = EINVAL; - DRV_LOG(ERR, "port %u invalid mp request type", - dev->data->port_id); - return -rte_errno; - } - return ret; -} - -/** - * IPC message handler of a secondary process. - * - * @param[in] dev - * Pointer to Ethernet structure. - * @param[in] peer - * Pointer to the peer socket path. - * - * @return - * 0 on success, a negative errno value otherwise and rte_errno is set. - */ -int -mlx5_mp_secondary_handle(const struct rte_mp_msg *mp_msg, const void *peer) -{ - struct rte_mp_msg mp_res; - struct mlx5_mp_param *res = (struct mlx5_mp_param *)mp_res.param; - const struct mlx5_mp_param *param = - (const struct mlx5_mp_param *)mp_msg->param; - struct rte_eth_dev *dev; - struct mlx5_priv *priv; - int ret; - - MLX5_ASSERT(rte_eal_process_type() == RTE_PROC_SECONDARY); - if (!rte_eth_dev_is_valid_port(param->port_id)) { - rte_errno = ENODEV; - DRV_LOG(ERR, "port %u invalid port ID", param->port_id); - return -rte_errno; - } - dev = &rte_eth_devices[param->port_id]; - priv = dev->data->dev_private; - switch (param->type) { - case MLX5_MP_REQ_START_RXTX: - DRV_LOG(INFO, "port %u starting datapath", dev->data->port_id); - rte_mb(); - dev->rx_pkt_burst = mlx5_select_rx_function(dev); - dev->tx_pkt_burst = mlx5_select_tx_function(dev); - mp_init_msg(&priv->mp_id, &mp_res, param->type); - res->result = 0; - ret = rte_mp_reply(&mp_res, peer); - break; - case MLX5_MP_REQ_STOP_RXTX: - DRV_LOG(INFO, "port %u stopping datapath", dev->data->port_id); - dev->rx_pkt_burst = removed_rx_burst; - dev->tx_pkt_burst = removed_tx_burst; - rte_mb(); - mp_init_msg(&priv->mp_id, &mp_res, param->type); - res->result = 0; - ret = rte_mp_reply(&mp_res, peer); - break; - default: - rte_errno = EINVAL; - DRV_LOG(ERR, "port %u invalid mp request type", - dev->data->port_id); - return -rte_errno; - } - return ret; -} - -/** - * Broadcast request of stopping/starting data-path to secondary processes. - * - * @param[in] dev - * Pointer to Ethernet structure. - * @param[in] type - * Request type. - */ -static void -mp_req_on_rxtx(struct rte_eth_dev *dev, enum mlx5_mp_req_type type) -{ - struct rte_mp_msg mp_req; - struct rte_mp_msg *mp_res; - struct rte_mp_reply mp_rep; - struct mlx5_mp_param *res; - struct timespec ts = {.tv_sec = MLX5_MP_REQ_TIMEOUT_SEC, .tv_nsec = 0}; - struct mlx5_priv *priv = dev->data->dev_private; - int ret; - int i; - - MLX5_ASSERT(rte_eal_process_type() == RTE_PROC_PRIMARY); - if (!mlx5_shared_data->secondary_cnt) - return; - if (type != MLX5_MP_REQ_START_RXTX && type != MLX5_MP_REQ_STOP_RXTX) { - DRV_LOG(ERR, "port %u unknown request (req_type %d)", - dev->data->port_id, type); - return; - } - mp_init_msg(&priv->mp_id, &mp_req, type); - ret = rte_mp_request_sync(&mp_req, &mp_rep, &ts); - if (ret) { - if (rte_errno != ENOTSUP) - DRV_LOG(ERR, "port %u failed to request stop/start Rx/Tx (%d)", - dev->data->port_id, type); - goto exit; - } - if (mp_rep.nb_sent != mp_rep.nb_received) { - DRV_LOG(ERR, - "port %u not all secondaries responded (req_type %d)", - dev->data->port_id, type); - goto exit; - } - for (i = 0; i < mp_rep.nb_received; i++) { - mp_res = &mp_rep.msgs[i]; - res = (struct mlx5_mp_param *)mp_res->param; - if (res->result) { - DRV_LOG(ERR, "port %u request failed on secondary #%d", - dev->data->port_id, i); - goto exit; - } - } -exit: - free(mp_rep.msgs); -} - -/** - * Broadcast request of starting data-path to secondary processes. The request - * is synchronous. - * - * @param[in] dev - * Pointer to Ethernet structure. - */ -void -mlx5_mp_req_start_rxtx(struct rte_eth_dev *dev) -{ - mp_req_on_rxtx(dev, MLX5_MP_REQ_START_RXTX); -} - -/** - * Broadcast request of stopping data-path to secondary processes. The request - * is synchronous. - * - * @param[in] dev - * Pointer to Ethernet structure. - */ -void -mlx5_mp_req_stop_rxtx(struct rte_eth_dev *dev) -{ - mp_req_on_rxtx(dev, MLX5_MP_REQ_STOP_RXTX); -} diff --git a/drivers/net/mlx5/mlx5_trigger.c b/drivers/net/mlx5/mlx5_trigger.c index ef74609..360fd6f 100644 --- a/drivers/net/mlx5/mlx5_trigger.c +++ b/drivers/net/mlx5/mlx5_trigger.c @@ -340,7 +340,7 @@ mlx5_dev_start(struct rte_eth_dev *dev) dev->tx_pkt_burst = mlx5_select_tx_function(dev); dev->rx_pkt_burst = mlx5_select_rx_function(dev); /* Enable datapath on secondary process. */ - mlx5_mp_req_start_rxtx(dev); + mlx5_mp_os_req_start_rxtx(dev); if (priv->sh->intr_handle.fd >= 0) { priv->sh->port[priv->dev_port - 1].ih_port_id = (uint32_t)dev->data->port_id; @@ -385,7 +385,7 @@ mlx5_dev_stop(struct rte_eth_dev *dev) dev->tx_pkt_burst = removed_tx_burst; rte_wmb(); /* Disable datapath on secondary process. */ - mlx5_mp_req_stop_rxtx(dev); + mlx5_mp_os_req_stop_rxtx(dev); usleep(1000 * priv->rxqs_n); DRV_LOG(DEBUG, "port %u stopping device", dev->data->port_id); mlx5_flow_stop_default(dev);