From patchwork Thu Apr 9 15:02:46 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Tonghao Zhang X-Patchwork-Id: 68055 X-Patchwork-Delegate: david.marchand@redhat.com Return-Path: X-Original-To: patchwork@inbox.dpdk.org Delivered-To: patchwork@inbox.dpdk.org Received: from dpdk.org (dpdk.org [92.243.14.124]) by inbox.dpdk.org (Postfix) with ESMTP id 03459A0597; Thu, 9 Apr 2020 17:03:21 +0200 (CEST) Received: from [92.243.14.124] (localhost [127.0.0.1]) by dpdk.org (Postfix) with ESMTP id 4BCFF1C2F7; Thu, 9 Apr 2020 17:03:21 +0200 (CEST) Received: from mail-pf1-f196.google.com (mail-pf1-f196.google.com [209.85.210.196]) by dpdk.org (Postfix) with ESMTP id 6CFB81C2F6 for ; Thu, 9 Apr 2020 17:03:19 +0200 (CEST) Received: by mail-pf1-f196.google.com with SMTP id v23so4886842pfm.1 for ; Thu, 09 Apr 2020 08:03:19 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=from:to:cc:subject:date:message-id:in-reply-to:references; bh=ztkyWMkbn0j02EPPeLGmN6IX7utPtaS7fadR9ciPvSY=; b=uMczBMp1zEHyq2BO3w4EHopbKJ06Jw8/jMZz5ODKR7E7o4nt0V/gaVWgfIw1c0+AXq gATEuDGjzLQx1VXCaW6FzZwrfZmaBDaOkdZXHtXLyhVVeubsUjypO0Ed6lbKjsB5SLIy +pm9LNl2ZpdOr4I4qXqo+Pw6Z2CcsHKjhTH/7vknGO6bM+5K7CZTXMU0P9ogoHXJtHEA aHuRmUdTe07FujngE8r8yUc5VUWP85RD6WxoGvkuTaN4cPSwrTw51X+IRbl0KFaU8Y7s q2C4ABctruTIG7lgccRptH+b4IGj3X+C98TSBqijhkGnl+hqCZdrLAVa8WDUDKzO8B5f npkg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references; bh=ztkyWMkbn0j02EPPeLGmN6IX7utPtaS7fadR9ciPvSY=; b=hsbg/cGblaXmjoMNSn/7AfDYFdne47EUo1VoPjfGbvomsazSaWYbC89oYmzKXCYJkN 89TGSWfkPGIk1ULRx6c8t+bcY40ggrn00yUjocGJYbd81iKsKHnu6DZcN9s5gWKAzPC9 qDcwlZT7ThtjviT0jmFHnEOpchdumNILPoW1qkcxrEJP+GieXorBhBKxzxZY7JmyLkyV 9IF4ptyi8HxORSOR8itELDG5noe/XOJE9GEbXQ4XTDUhXfhDfSXBIev559d8Qmfwx6wM 4GFWisc58Or05psyV9CjtVwDUJLpmkECUxywtRqMsW72D7XiZTB8qFLkkFR3eD5L/sV8 q8Hw== X-Gm-Message-State: AGi0PualU5fEL3oqNfKITLPVX7UCk66dAhMk7hK4UiNSx2Kr/Ir23IxT ENfSr/4O84d3GEAtUkqq9Ps= X-Google-Smtp-Source: APiQypKdWk7xGnez2oEyZkHkGHSGVoRxNA3xZAqjhSsWYbDt8WYp7a9OTnMcbhM6JTjfqR3frFYwxQ== X-Received: by 2002:a63:c7:: with SMTP id 190mr12117238pga.185.1586444598436; Thu, 09 Apr 2020 08:03:18 -0700 (PDT) Received: from local.opencloud.tech.localdomain ([115.171.63.184]) by smtp.gmail.com with ESMTPSA id e11sm1858819pfh.117.2020.04.09.08.03.10 (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Thu, 09 Apr 2020 08:03:14 -0700 (PDT) From: xiangxia.m.yue@gmail.com To: olivier.matz@6wind.com, arybchenko@solarflare.com, gage.eads@intel.com, artem.andreev@oktetlabs.ru, jerinj@marvell.com, ndabilpuram@marvell.com, vattunuru@marvell.com, hemant.agrawal@nxp.com, david.marchand@redhat.com, anatoly.burakov@intel.com, bruce.richardson@intel.com Cc: dev@dpdk.org, Tonghao Zhang Date: Thu, 9 Apr 2020 23:02:46 +0800 Message-Id: <1586444567-9311-1-git-send-email-xiangxia.m.yue@gmail.com> X-Mailer: git-send-email 1.8.3.1 In-Reply-To: <1583114253-15345-1-git-send-email-xiangxia.m.yue@gmail.com> References: <1583114253-15345-1-git-send-email-xiangxia.m.yue@gmail.com> Subject: [dpdk-dev] [PATCH dpdk-dev v2 1/2] eal: introduce rte-init queue for libraries initialization X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Sender: "dev" From: Tonghao Zhang This patch introduces last-init queue, user can register a callback for theirs initialization. Running rte_last_init_run(), the almost resource of DPDK are available, such as memzone, ring. With this way, user don't introduce additional codes in eal layer. [This patch will be used for next patch.] Signed-off-by: Tonghao Zhang --- v2: * rename rte_last_init_register ->rte_init_register * rename rte_last_init struct ->rte_init * rename rte_init_cb ->rte_init_cb_t * free the rte_init node when not used. * remove rte_init and others to eal_private.h * add comments * fix checkpatch warning --- lib/librte_eal/common/eal_common_init.c | 74 +++++++++++++++++++++++++++++++++ lib/librte_eal/common/eal_private.h | 23 ++++++++++ lib/librte_eal/common/meson.build | 1 + lib/librte_eal/freebsd/Makefile | 1 + lib/librte_eal/freebsd/eal.c | 6 +++ lib/librte_eal/include/rte_init.h | 59 ++++++++++++++++++++++++++ lib/librte_eal/linux/Makefile | 1 + lib/librte_eal/linux/eal.c | 6 +++ 8 files changed, 171 insertions(+) create mode 100644 lib/librte_eal/common/eal_common_init.c create mode 100644 lib/librte_eal/include/rte_init.h diff --git a/lib/librte_eal/common/eal_common_init.c b/lib/librte_eal/common/eal_common_init.c new file mode 100644 index 0000000..1d89db3 --- /dev/null +++ b/lib/librte_eal/common/eal_common_init.c @@ -0,0 +1,74 @@ +/* SPDX-License-Identifier: BSD-3-Clause + * Copyright 2020 DPDK Community + */ + +#include + +#include +#include +#include + +#include "eal_private.h" + +static struct rte_init_list rte_init_list = + TAILQ_HEAD_INITIALIZER(rte_init_list); + +void +rte_init_register(rte_init_cb_t cb, const void *arg, enum rte_init_type type) +{ + struct rte_init *last; + + RTE_VERIFY(cb); + + last = malloc(sizeof(*last)); + if (last == NULL) + rte_panic("Alloc memory for rte_init node failed\n"); + + last->type = type; + last->arg = arg; + last->cb = cb; + + TAILQ_INSERT_TAIL(&rte_init_list, last, next); +} + +static int +eal_rte_init_run_type(enum rte_init_type type) +{ + struct rte_init *last; + int ret; + + TAILQ_FOREACH(last, &rte_init_list, next) { + if (last->type != type) + continue; + + ret = last->cb(last->arg); + if (ret < 0) + return ret; + } + + return 0; +} + +int +eal_rte_init_run(void) +{ + struct rte_init *last; + struct rte_init *tmp; + int ret; + + ret = eal_rte_init_run_type(RTE_INIT_PRE); + if (ret < 0) + return ret; + + ret = eal_rte_init_run_type(RTE_INIT_POST); + if (ret < 0) + return ret; + + /* Free rte_init node, not used anymore. */ + TAILQ_FOREACH_SAFE(last, &rte_init_list, next, tmp) { + TAILQ_REMOVE(&rte_init_list, last, next); + free(last); + } + + return 0; +} diff --git a/lib/librte_eal/common/eal_private.h b/lib/librte_eal/common/eal_private.h index ddcfbe2..f0bcc97 100644 --- a/lib/librte_eal/common/eal_private.h +++ b/lib/librte_eal/common/eal_private.h @@ -11,6 +11,7 @@ #include #include +#include /** * Structure storing internal configuration (per-lcore) @@ -60,6 +61,28 @@ struct rte_config { } __attribute__((__packed__)); /** + * A structure describing a generic initialization. + */ +struct rte_init { + TAILQ_ENTRY(rte_init) next; + enum rte_init_type type; + rte_init_cb_t cb; + const void *arg; +}; + +/** Double linked list of rte_init. */ +TAILQ_HEAD(rte_init_list, rte_init); + +/** + * Run the callback registered in the global double linked list. + * + * @return + * - 0 on success + * - Negative on error + */ +int eal_rte_init_run(void); + +/** * Get the global configuration structure. * * @return diff --git a/lib/librte_eal/common/meson.build b/lib/librte_eal/common/meson.build index 02d9280..ad0ce6a 100644 --- a/lib/librte_eal/common/meson.build +++ b/lib/librte_eal/common/meson.build @@ -43,6 +43,7 @@ sources += files( 'eal_common_thread.c', 'eal_common_timer.c', 'eal_common_uuid.c', + 'eal_common_init.c', 'hotplug_mp.c', 'malloc_elem.c', 'malloc_heap.c', diff --git a/lib/librte_eal/freebsd/Makefile b/lib/librte_eal/freebsd/Makefile index e5d4d8f..89c5649 100644 --- a/lib/librte_eal/freebsd/Makefile +++ b/lib/librte_eal/freebsd/Makefile @@ -60,6 +60,7 @@ SRCS-$(CONFIG_RTE_EXEC_ENV_FREEBSD) += eal_common_thread.c SRCS-$(CONFIG_RTE_EXEC_ENV_FREEBSD) += eal_common_proc.c SRCS-$(CONFIG_RTE_EXEC_ENV_FREEBSD) += eal_common_fbarray.c SRCS-$(CONFIG_RTE_EXEC_ENV_FREEBSD) += eal_common_uuid.c +SRCS-$(CONFIG_RTE_EXEC_ENV_FREEBSD) += eal_common_init.c SRCS-$(CONFIG_RTE_EXEC_ENV_FREEBSD) += rte_malloc.c SRCS-$(CONFIG_RTE_EXEC_ENV_FREEBSD) += hotplug_mp.c SRCS-$(CONFIG_RTE_EXEC_ENV_FREEBSD) += malloc_elem.c diff --git a/lib/librte_eal/freebsd/eal.c b/lib/librte_eal/freebsd/eal.c index 6ae37e7..63af98c 100644 --- a/lib/librte_eal/freebsd/eal.c +++ b/lib/librte_eal/freebsd/eal.c @@ -874,6 +874,12 @@ static void rte_eal_init_alert(const char *msg) eal_check_mem_on_local_socket(); + if (eal_rte_init_run() < 0) { + rte_eal_init_alert("Cannot init objects in rte-init queue"); + rte_errno = EFAULT; + return -1; + } + eal_thread_init_master(rte_config.master_lcore); ret = eal_thread_dump_affinity(cpuset, sizeof(cpuset)); diff --git a/lib/librte_eal/include/rte_init.h b/lib/librte_eal/include/rte_init.h new file mode 100644 index 0000000..636efff --- /dev/null +++ b/lib/librte_eal/include/rte_init.h @@ -0,0 +1,59 @@ +/* SPDX-License-Identifier: BSD-3-Clause + * Copyright 2020 DPDK Community + */ + +#ifndef _RTE_INIT_H_ +#define _RTE_INIT_H_ + +#ifdef __cplusplus +extern "C" { +#endif + +#include +#include + +/** + * Implementation specific callback function which is + * responsible for specificed initialization. + * + * This is called when almost resources are available. + * + * @return + * 0 for successful callback + * Negative for unsuccessful callback with error value + */ +typedef int (*rte_init_cb_t)(const void *arg); + +/** + * rte_init type. + * + * The rte_init of RTE_INIT_PRE are called firstly, + * and then RTE_INIT_POST. + */ +enum rte_init_type { + RTE_INIT_PRE, + RTE_INIT_POST +}; + +/** + * Register a rte_init callback. + * + * @param cb + * A pointer to a rte_init_cb structure, which will be used + * in rte_eal_init(). + * + * @param arg + * The cb will use that as param. + * + * @param type + * The type of rte_init registered. + */ + +void rte_init_register(rte_init_cb_t cb, const void *arg, + enum rte_init_type type); + +#ifdef __cplusplus +} +#endif + +#endif /* _RTE_INIT_H_ */ diff --git a/lib/librte_eal/linux/Makefile b/lib/librte_eal/linux/Makefile index e5f4495..918d94b 100644 --- a/lib/librte_eal/linux/Makefile +++ b/lib/librte_eal/linux/Makefile @@ -67,6 +67,7 @@ SRCS-$(CONFIG_RTE_EXEC_ENV_LINUX) += eal_common_thread.c SRCS-$(CONFIG_RTE_EXEC_ENV_LINUX) += eal_common_proc.c SRCS-$(CONFIG_RTE_EXEC_ENV_LINUX) += eal_common_fbarray.c SRCS-$(CONFIG_RTE_EXEC_ENV_LINUX) += eal_common_uuid.c +SRCS-$(CONFIG_RTE_EXEC_ENV_LINUX) += eal_common_init.c SRCS-$(CONFIG_RTE_EXEC_ENV_LINUX) += rte_malloc.c SRCS-$(CONFIG_RTE_EXEC_ENV_LINUX) += hotplug_mp.c SRCS-$(CONFIG_RTE_EXEC_ENV_LINUX) += malloc_elem.c diff --git a/lib/librte_eal/linux/eal.c b/lib/librte_eal/linux/eal.c index 9530ee5..dd0c258 100644 --- a/lib/librte_eal/linux/eal.c +++ b/lib/librte_eal/linux/eal.c @@ -1203,6 +1203,12 @@ static void rte_eal_init_alert(const char *msg) eal_check_mem_on_local_socket(); + if (eal_rte_init_run() < 0) { + rte_eal_init_alert("Cannot init objects in rte-init queue"); + rte_errno = EFAULT; + return -1; + } + eal_thread_init_master(rte_config.master_lcore); ret = eal_thread_dump_affinity(cpuset, sizeof(cpuset)); From patchwork Thu Apr 9 15:02:47 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Tonghao Zhang X-Patchwork-Id: 68056 X-Patchwork-Delegate: david.marchand@redhat.com Return-Path: X-Original-To: patchwork@inbox.dpdk.org Delivered-To: patchwork@inbox.dpdk.org Received: from dpdk.org (dpdk.org [92.243.14.124]) by inbox.dpdk.org (Postfix) with ESMTP id 169E0A0597; Thu, 9 Apr 2020 17:03:31 +0200 (CEST) Received: from [92.243.14.124] (localhost [127.0.0.1]) by dpdk.org (Postfix) with ESMTP id 883EB1C434; Thu, 9 Apr 2020 17:03:30 +0200 (CEST) Received: from mail-pl1-f193.google.com (mail-pl1-f193.google.com [209.85.214.193]) by dpdk.org (Postfix) with ESMTP id 1D2A81C2F6 for ; Thu, 9 Apr 2020 17:03:29 +0200 (CEST) Received: by mail-pl1-f193.google.com with SMTP id t4so3906312plq.12 for ; Thu, 09 Apr 2020 08:03:29 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=from:to:cc:subject:date:message-id:in-reply-to:references; bh=nPn+HDZt+G4IDcuPjSE/+R5A9+FMOokPoCxxrkJXsdQ=; b=JddKFaTGI7n4Ev32xnu7KAQWEQXUJ+Q7ptlYL3c0l82NSjQis+3rQO/0a3ZieJ47ia i56ciFD1IBkVJCDKR0xXCnYVlRAWQYHeBXb7Ht73TVdcNwcE6QFs9MpsmUlG7LgmExZg S3W8KWgZTnwiMItgadK4wrfNHV1SB4KmGo5Wm/ukk7g0KE7tbbyB6txmcJlsssJJ6esw OGbwCzkvaRSxBC+CFyDTTs2vNV40zWcMwS3Y8LZaEEBfv2Zt5PRkRwHcchYfgaVgUcmL ctX/7oY612DGVdH2kea2DD8OlGt/wwTFMO0MVrUkDlTUWl+aLtj88tSudOT53YY6kqpd 02Vg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references; bh=nPn+HDZt+G4IDcuPjSE/+R5A9+FMOokPoCxxrkJXsdQ=; b=cfU8atUmLX+YLZWc7frZAmT3PodSfR1KpglsOM4+AIMOIuV+HSUcZtYKZ7R4FK7HlJ dKBpAXAspReXgQ7E6v3FlS1doQ62oBJriYt0bOuVxhQ/XNnoiKbJ4+UxPI/YyQyVt0Z9 v7wV/nv6F99AbrXJ+vFy6XDYe045hyQ9i6T+ScXgx29Ao+D2UikkY1hPHMTqHU3CJ6hw Js7rLuqHhJQ/Gyw6NWdfZql3XSyZg9dBaGUjn/uVp85GpfD1OrSO6M/EGfKeg3JamVGj aO5/WDWQkQ1XSrFKlwA54+QnEy94f8ApuEtjcA3ZCRjub8zJ97jrFl8XU49qnh9Joc2G L87g== X-Gm-Message-State: AGi0PubNbvRXT/1PQm5v3gLb/uG2nrPZ3WCDl/q1Wqg84s8SkB/3Dj0X jpYknLgoH3JOLMIn4Nuw4k8= X-Google-Smtp-Source: APiQypJC7qfuWTqpafAVbw6VdL7hD00zw4/g9CbKoWoigN9hStFlkyBAQQEwnSxnt5A/WdGF3znHkw== X-Received: by 2002:a17:902:7244:: with SMTP id c4mr17057pll.25.1586444608153; Thu, 09 Apr 2020 08:03:28 -0700 (PDT) Received: from local.opencloud.tech.localdomain ([115.171.63.184]) by smtp.gmail.com with ESMTPSA id e11sm1858819pfh.117.2020.04.09.08.03.19 (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Thu, 09 Apr 2020 08:03:24 -0700 (PDT) From: xiangxia.m.yue@gmail.com To: olivier.matz@6wind.com, arybchenko@solarflare.com, gage.eads@intel.com, artem.andreev@oktetlabs.ru, jerinj@marvell.com, ndabilpuram@marvell.com, vattunuru@marvell.com, hemant.agrawal@nxp.com, david.marchand@redhat.com, anatoly.burakov@intel.com, bruce.richardson@intel.com Cc: dev@dpdk.org, Tonghao Zhang Date: Thu, 9 Apr 2020 23:02:47 +0800 Message-Id: <1586444567-9311-2-git-send-email-xiangxia.m.yue@gmail.com> X-Mailer: git-send-email 1.8.3.1 In-Reply-To: <1586444567-9311-1-git-send-email-xiangxia.m.yue@gmail.com> References: <1583114253-15345-1-git-send-email-xiangxia.m.yue@gmail.com> <1586444567-9311-1-git-send-email-xiangxia.m.yue@gmail.com> Subject: [dpdk-dev] [PATCH dpdk-dev v2 2/2] mempool: use shared memzone for rte_mempool_ops X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Sender: "dev" From: Tonghao Zhang The order of mempool initiation affects mempool index in the rte_mempool_ops_table. For example, when building APPs with: $ gcc -lrte_mempool_bucket -lrte_mempool_ring ... The "bucket" mempool will be registered firstly, and its index in table is 0 while the index of "ring" mempool is 1. DPDK uses the mk/rte.app.mk to build APPs, and others, for example, Open vSwitch, use the libdpdk.a or libdpdk.so to build it. The mempool lib linked in dpdk and Open vSwitch is different. The mempool can be used between primary and secondary process, such as dpdk-pdump and pdump-pmd/Open vSwitch(pdump enabled). There will be a crash because dpdk-pdump creates the "ring_mp_mc" ring which index in table is 0, but the index of "bucket" ring is 0 in Open vSwitch. If Open vSwitch use the index 0 to get mempool ops and malloc memory from mempool. The crash will occur: bucket_dequeue (access null and crash) rte_mempool_get_ops (should get "ring_mp_mc", but get "bucket" mempool) rte_mempool_ops_dequeue_bulk ... rte_pktmbuf_alloc rte_pktmbuf_copy pdump_copy pdump_rx rte_eth_rx_burst To avoid the crash, there are some solution: * constructor priority: Different mempool uses different priority in RTE_INIT, but it's not easy to maintain. * change mk/rte.app.mk: Change the order in mk/rte.app.mk to be same as libdpdk.a/libdpdk.so, but when adding a new mempool driver in future, we must make sure the order. * register mempool orderly: Sort the mempool when registering, so the lib linked will not affect the index in mempool table. but the number of mempool libraries may be different. * shared memzone: The primary process allocates a struct in shared memory named memzone, When we register a mempool ops, we first get a name and id from the shared struct: with the lock held, lookup for the registered name and return its index, else get the last id and copy the name in the struct. Previous discussion: https://mails.dpdk.org/archives/dev/2020-March/159354.html Suggested-by: Olivier Matz Signed-off-by: Tonghao Zhang --- v2: * fix checkpatch warning --- lib/librte_mempool/rte_mempool.h | 28 +++++++++++- lib/librte_mempool/rte_mempool_ops.c | 89 ++++++++++++++++++++++++++++-------- 2 files changed, 96 insertions(+), 21 deletions(-) diff --git a/lib/librte_mempool/rte_mempool.h b/lib/librte_mempool/rte_mempool.h index c90cf31..2709b9e 100644 --- a/lib/librte_mempool/rte_mempool.h +++ b/lib/librte_mempool/rte_mempool.h @@ -50,6 +50,7 @@ #include #include #include +#include #ifdef __cplusplus extern "C" { @@ -678,7 +679,6 @@ struct rte_mempool_ops { */ struct rte_mempool_ops_table { rte_spinlock_t sl; /**< Spinlock for add/delete. */ - uint32_t num_ops; /**< Number of used ops structs in the table. */ /** * Storage for all possible ops structs. */ @@ -910,6 +910,30 @@ int rte_mempool_ops_get_info(const struct rte_mempool *mp, */ int rte_mempool_register_ops(const struct rte_mempool_ops *ops); +struct rte_mempool_shared_ops { + size_t num_mempool_ops; + struct { + char name[RTE_MEMPOOL_OPS_NAMESIZE]; + } mempool_ops[RTE_MEMPOOL_MAX_OPS_IDX]; + + rte_spinlock_t mempool; +}; + +static inline int +mempool_ops_register_cb(const void *arg) +{ + const struct rte_mempool_ops *h = (const struct rte_mempool_ops *)arg; + + return rte_mempool_register_ops(h); +} + +static inline void +mempool_ops_register(const struct rte_mempool_ops *ops) +{ + rte_init_register(mempool_ops_register_cb, (const void *)ops, + RTE_INIT_PRE); +} + /** * Macro to statically register the ops of a mempool handler. * Note that the rte_mempool_register_ops fails silently here when @@ -918,7 +942,7 @@ int rte_mempool_ops_get_info(const struct rte_mempool *mp, #define MEMPOOL_REGISTER_OPS(ops) \ RTE_INIT(mp_hdlr_init_##ops) \ { \ - rte_mempool_register_ops(&ops); \ + mempool_ops_register(&ops); \ } /** diff --git a/lib/librte_mempool/rte_mempool_ops.c b/lib/librte_mempool/rte_mempool_ops.c index 22c5251..bc01964 100644 --- a/lib/librte_mempool/rte_mempool_ops.c +++ b/lib/librte_mempool/rte_mempool_ops.c @@ -14,43 +14,95 @@ /* indirect jump table to support external memory pools. */ struct rte_mempool_ops_table rte_mempool_ops_table = { .sl = RTE_SPINLOCK_INITIALIZER, - .num_ops = 0 }; -/* add a new ops struct in rte_mempool_ops_table, return its index. */ -int -rte_mempool_register_ops(const struct rte_mempool_ops *h) +static int +rte_mempool_register_shared_ops(const char *name) { - struct rte_mempool_ops *ops; - int16_t ops_index; + static bool mempool_shared_ops_inited = false; + struct rte_mempool_shared_ops *shared_ops; + const struct rte_memzone *mz; + uint32_t ops_index = 0; + + if (rte_eal_process_type() == RTE_PROC_PRIMARY && + !mempool_shared_ops_inited) { + + mz = rte_memzone_reserve("mempool_ops_shared", + sizeof(*shared_ops), 0, 0); + if (mz == NULL) + return -ENOMEM; + + shared_ops = mz->addr; + shared_ops->num_mempool_ops = 0; + rte_spinlock_init(&shared_ops->mempool); + + mempool_shared_ops_inited = true; + } else { + mz = rte_memzone_lookup("mempool_ops_shared"); + if (mz == NULL) + return -ENOMEM; + + shared_ops = mz->addr; + } - rte_spinlock_lock(&rte_mempool_ops_table.sl); + rte_spinlock_lock(&shared_ops->mempool); - if (rte_mempool_ops_table.num_ops >= - RTE_MEMPOOL_MAX_OPS_IDX) { - rte_spinlock_unlock(&rte_mempool_ops_table.sl); + if (shared_ops->num_mempool_ops >= RTE_MEMPOOL_MAX_OPS_IDX) { + rte_spinlock_unlock(&shared_ops->mempool); RTE_LOG(ERR, MEMPOOL, "Maximum number of mempool ops structs exceeded\n"); return -ENOSPC; } + while (shared_ops->mempool_ops[ops_index].name[0]) { + if (!strcmp(name, shared_ops->mempool_ops[ops_index].name)) { + rte_spinlock_unlock(&shared_ops->mempool); + return ops_index; + } + + ops_index++; + } + + strlcpy(shared_ops->mempool_ops[ops_index].name, name, + sizeof(shared_ops->mempool_ops[0].name)); + + shared_ops->num_mempool_ops++; + + rte_spinlock_unlock(&shared_ops->mempool); + return ops_index; +} + +/* add a new ops struct in rte_mempool_ops_table, return its index. */ +int +rte_mempool_register_ops(const struct rte_mempool_ops *h) +{ + struct rte_mempool_ops *ops; + int16_t ops_index; + if (h->alloc == NULL || h->enqueue == NULL || - h->dequeue == NULL || h->get_count == NULL) { - rte_spinlock_unlock(&rte_mempool_ops_table.sl); + h->dequeue == NULL || h->get_count == NULL) { RTE_LOG(ERR, MEMPOOL, "Missing callback while registering mempool ops\n"); + rte_errno = EINVAL; return -EINVAL; } if (strlen(h->name) >= sizeof(ops->name) - 1) { - rte_spinlock_unlock(&rte_mempool_ops_table.sl); - RTE_LOG(DEBUG, EAL, "%s(): mempool_ops <%s>: name too long\n", - __func__, h->name); + RTE_LOG(ERR, MEMPOOL, + "The registering mempool_ops <%s>: name too long\n", + h->name); rte_errno = EEXIST; return -EEXIST; } - ops_index = rte_mempool_ops_table.num_ops++; + ops_index = rte_mempool_register_shared_ops(h->name); + if (ops_index < 0) { + rte_errno = -ops_index; + return ops_index; + } + + rte_spinlock_lock(&rte_mempool_ops_table.sl); + ops = &rte_mempool_ops_table.ops[ops_index]; strlcpy(ops->name, h->name, sizeof(ops->name)); ops->alloc = h->alloc; @@ -165,9 +217,8 @@ struct rte_mempool_ops_table rte_mempool_ops_table = { if (mp->flags & MEMPOOL_F_POOL_CREATED) return -EEXIST; - for (i = 0; i < rte_mempool_ops_table.num_ops; i++) { - if (!strcmp(name, - rte_mempool_ops_table.ops[i].name)) { + for (i = 0; i < RTE_MEMPOOL_MAX_OPS_IDX; i++) { + if (!strcmp(name, rte_mempool_ops_table.ops[i].name)) { ops = &rte_mempool_ops_table.ops[i]; break; }