From patchwork Fri Sep 6 10:26:14 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Ruifeng Wang X-Patchwork-Id: 58822 X-Patchwork-Delegate: david.marchand@redhat.com Return-Path: X-Original-To: patchwork@dpdk.org Delivered-To: patchwork@dpdk.org Received: from [92.243.14.124] (localhost [127.0.0.1]) by dpdk.org (Postfix) with ESMTP id 5A6AD1F2B9; Fri, 6 Sep 2019 12:26:30 +0200 (CEST) Received: from foss.arm.com (foss.arm.com [217.140.110.172]) by dpdk.org (Postfix) with ESMTP id 56D711F29C for ; Fri, 6 Sep 2019 12:26:28 +0200 (CEST) Received: from usa-sjc-imap-foss1.foss.arm.com (unknown [10.121.207.14]) by usa-sjc-mx-foss1.foss.arm.com (Postfix) with ESMTP id 8F3481570; Fri, 6 Sep 2019 03:26:27 -0700 (PDT) Received: from net-arm-c2400-02.shanghai.arm.com (net-arm-c2400-02.shanghai.arm.com [10.169.40.42]) by usa-sjc-imap-foss1.foss.arm.com (Postfix) with ESMTPA id 158E03F59C; Fri, 6 Sep 2019 03:26:25 -0700 (PDT) From: Ruifeng Wang To: tomasz.kantecki@intel.com Cc: dev@dpdk.org, gavin.hu@arm.com, honnappa.nagarahalli@arm.com, nd@arm.com, Ruifeng Wang Date: Fri, 6 Sep 2019 18:26:14 +0800 Message-Id: <20190906102615.36942-2-ruifeng.wang@arm.com> X-Mailer: git-send-email 2.17.1 In-Reply-To: <20190906102615.36942-1-ruifeng.wang@arm.com> References: <20190906102615.36942-1-ruifeng.wang@arm.com> Subject: [dpdk-dev] [PATCH 1/2] examples/l3fwd: add lock-free option for l3fwd X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Sender: "dev" The new option is provided for lock-free read / write concurrency. For EM mode, lock-free APIs will be used when this option is enabled. Signed-off-by: Ruifeng Wang Reviewed-by: Gavin Hu --- doc/guides/sample_app_ug/l3_forward.rst | 3 +++ examples/l3fwd/l3fwd.h | 4 ++-- examples/l3fwd/l3fwd_em.c | 10 +++++++++- examples/l3fwd/l3fwd_lpm.c | 2 +- examples/l3fwd/main.c | 24 +++++++++++++++++++++--- 5 files changed, 36 insertions(+), 7 deletions(-) diff --git a/doc/guides/sample_app_ug/l3_forward.rst b/doc/guides/sample_app_ug/l3_forward.rst index 4cb4b18da..a40beeeb0 100644 --- a/doc/guides/sample_app_ug/l3_forward.rst +++ b/doc/guides/sample_app_ug/l3_forward.rst @@ -56,6 +56,7 @@ The application has a number of command line options:: [--ipv6] [--parse-ptype] [--per-port-pool] + [--lock-free] Where, @@ -86,6 +87,8 @@ Where, * ``--per-port-pool:`` Optional, set to use independent buffer pools per port. Without this option, single buffer pool is used for all ports. +* ``--lock-free:`` Optional, set to enable lock-free table read-write concurrency. + For example, consider a dual processor socket platform with 8 physical cores, where cores 0-7 and 16-23 appear on socket 0, while cores 8-15 and 24-31 appear on socket 1. diff --git a/examples/l3fwd/l3fwd.h b/examples/l3fwd/l3fwd.h index 293fb1fa2..eaf71d429 100644 --- a/examples/l3fwd/l3fwd.h +++ b/examples/l3fwd/l3fwd.h @@ -171,10 +171,10 @@ is_valid_ipv4_pkt(struct rte_ipv4_hdr *pkt, uint32_t link_len) /* Function pointers for LPM or EM functionality. */ void -setup_lpm(const int socketid); +setup_lpm(const int socketid, const unsigned int flags); void -setup_hash(const int socketid); +setup_hash(const int socketid, const unsigned int flags); int em_check_ptype(int portid); diff --git a/examples/l3fwd/l3fwd_em.c b/examples/l3fwd/l3fwd_em.c index 74a7c8fa4..7263ede04 100644 --- a/examples/l3fwd/l3fwd_em.c +++ b/examples/l3fwd/l3fwd_em.c @@ -703,7 +703,7 @@ em_main_loop(__attribute__((unused)) void *dummy) * Initialize exact match (hash) parameters. */ void -setup_hash(const int socketid) +setup_hash(const int socketid, const unsigned int flags) { struct rte_hash_parameters ipv4_l3fwd_hash_params = { .name = NULL, @@ -727,6 +727,10 @@ setup_hash(const int socketid) snprintf(s, sizeof(s), "ipv4_l3fwd_hash_%d", socketid); ipv4_l3fwd_hash_params.name = s; ipv4_l3fwd_hash_params.socket_id = socketid; + /* enable lock free hash algorithm for ipv4 forward*/ + if (flags & RTE_HASH_EXTRA_FLAGS_RW_CONCURRENCY_LF) + ipv4_l3fwd_hash_params.extra_flag |= + RTE_HASH_EXTRA_FLAGS_RW_CONCURRENCY_LF; ipv4_l3fwd_em_lookup_struct[socketid] = rte_hash_create(&ipv4_l3fwd_hash_params); if (ipv4_l3fwd_em_lookup_struct[socketid] == NULL) @@ -738,6 +742,10 @@ setup_hash(const int socketid) snprintf(s, sizeof(s), "ipv6_l3fwd_hash_%d", socketid); ipv6_l3fwd_hash_params.name = s; ipv6_l3fwd_hash_params.socket_id = socketid; + /* enable lock free hash algorithm for ipv6 forward*/ + if (flags & RTE_HASH_EXTRA_FLAGS_RW_CONCURRENCY_LF) + ipv6_l3fwd_hash_params.extra_flag |= + RTE_HASH_EXTRA_FLAGS_RW_CONCURRENCY_LF; ipv6_l3fwd_em_lookup_struct[socketid] = rte_hash_create(&ipv6_l3fwd_hash_params); if (ipv6_l3fwd_em_lookup_struct[socketid] == NULL) diff --git a/examples/l3fwd/l3fwd_lpm.c b/examples/l3fwd/l3fwd_lpm.c index 4143683cb..1435a5711 100644 --- a/examples/l3fwd/l3fwd_lpm.c +++ b/examples/l3fwd/l3fwd_lpm.c @@ -255,7 +255,7 @@ lpm_main_loop(__attribute__((unused)) void *dummy) } void -setup_lpm(const int socketid) +setup_lpm(const int socketid, __rte_unused const unsigned int flags) { struct rte_lpm6_config config; struct rte_lpm_config config_ipv4; diff --git a/examples/l3fwd/main.c b/examples/l3fwd/main.c index 3800bad19..1b435e9eb 100644 --- a/examples/l3fwd/main.c +++ b/examples/l3fwd/main.c @@ -41,6 +41,7 @@ #include #include #include +#include #include #include @@ -76,6 +77,10 @@ static int parse_ptype; /**< Parse packet type using rx callback, and */ /**< disabled by default */ static int per_port_pool; /**< Use separate buffer pools per port; disabled */ /**< by default */ +static int rw_lf; /**< Enable lock-free read-write concurrency, */ + /**< disabled by default */ + +/* Global variables. */ volatile bool force_quit; @@ -139,7 +144,7 @@ static struct rte_mempool *pktmbuf_pool[RTE_MAX_ETHPORTS][NB_SOCKETS]; static uint8_t lkp_per_socket[NB_SOCKETS]; struct l3fwd_lkp_mode { - void (*setup)(int); + void (*setup)(int, unsigned int); int (*check_ptype)(int); rte_rx_callback_fn cb_parse_ptype; int (*main_loop)(void *); @@ -290,6 +295,7 @@ print_usage(const char *prgname) " [--ipv6]" " [--parse-ptype]" " [--per-port-pool]\n\n" + " [--lock-free]\n\n" " -p PORTMASK: Hexadecimal bitmask of ports to configure\n" " -P : Enable promiscuous mode\n" @@ -304,7 +310,8 @@ print_usage(const char *prgname) " --hash-entry-num: Specify the hash entry number in hexadecimal to be setup\n" " --ipv6: Set if running ipv6 packets\n" " --parse-ptype: Set to use software to analyze packet type\n" - " --per-port-pool: Use separate buffer pool per port\n\n", + " --per-port-pool: Use separate buffer pool per port\n\n" + " --lock-free: Set to enable lock-free table read-write concurrency\n\n", prgname); } @@ -458,6 +465,8 @@ static const char short_options[] = #define CMD_LINE_OPT_HASH_ENTRY_NUM "hash-entry-num" #define CMD_LINE_OPT_PARSE_PTYPE "parse-ptype" #define CMD_LINE_OPT_PER_PORT_POOL "per-port-pool" +#define CMD_LINE_OPT_LOCK_FREE "lock-free" + enum { /* long options mapped to a short option */ @@ -472,6 +481,7 @@ enum { CMD_LINE_OPT_HASH_ENTRY_NUM_NUM, CMD_LINE_OPT_PARSE_PTYPE_NUM, CMD_LINE_OPT_PARSE_PER_PORT_POOL, + CMD_LINE_OPT_LOCK_FREE_NUM, }; static const struct option lgopts[] = { @@ -483,6 +493,7 @@ static const struct option lgopts[] = { {CMD_LINE_OPT_HASH_ENTRY_NUM, 1, 0, CMD_LINE_OPT_HASH_ENTRY_NUM_NUM}, {CMD_LINE_OPT_PARSE_PTYPE, 0, 0, CMD_LINE_OPT_PARSE_PTYPE_NUM}, {CMD_LINE_OPT_PER_PORT_POOL, 0, 0, CMD_LINE_OPT_PARSE_PER_PORT_POOL}, + {CMD_LINE_OPT_LOCK_FREE, 0, 0, CMD_LINE_OPT_LOCK_FREE_NUM}, {NULL, 0, 0, 0} }; @@ -607,6 +618,10 @@ parse_args(int argc, char **argv) per_port_pool = 1; break; + case CMD_LINE_OPT_LOCK_FREE_NUM: + rw_lf = 1; + break; + default: print_usage(prgname); return -1; @@ -661,6 +676,7 @@ init_mem(uint16_t portid, unsigned int nb_mbuf) int socketid; unsigned lcore_id; char s[64]; + unsigned int flags = 0; for (lcore_id = 0; lcore_id < RTE_MAX_LCORE; lcore_id++) { if (rte_lcore_is_enabled(lcore_id) == 0) @@ -695,8 +711,10 @@ init_mem(uint16_t portid, unsigned int nb_mbuf) /* Setup either LPM or EM(f.e Hash). But, only once per * available socket. */ + if (rw_lf) + flags |= RTE_HASH_EXTRA_FLAGS_RW_CONCURRENCY_LF; if (!lkp_per_socket[socketid]) { - l3fwd_lkp.setup(socketid); + l3fwd_lkp.setup(socketid, flags); lkp_per_socket[socketid] = 1; } } From patchwork Fri Sep 6 10:26:15 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Ruifeng Wang X-Patchwork-Id: 58823 X-Patchwork-Delegate: david.marchand@redhat.com Return-Path: X-Original-To: patchwork@dpdk.org Delivered-To: patchwork@dpdk.org Received: from [92.243.14.124] (localhost [127.0.0.1]) by dpdk.org (Postfix) with ESMTP id E328C1F2EE; Fri, 6 Sep 2019 12:26:34 +0200 (CEST) Received: from foss.arm.com (foss.arm.com [217.140.110.172]) by dpdk.org (Postfix) with ESMTP id 6A79D1F2DA for ; Fri, 6 Sep 2019 12:26:32 +0200 (CEST) Received: from usa-sjc-imap-foss1.foss.arm.com (unknown [10.121.207.14]) by usa-sjc-mx-foss1.foss.arm.com (Postfix) with ESMTP id F257C1570; Fri, 6 Sep 2019 03:26:31 -0700 (PDT) Received: from net-arm-c2400-02.shanghai.arm.com (net-arm-c2400-02.shanghai.arm.com [10.169.40.42]) by usa-sjc-imap-foss1.foss.arm.com (Postfix) with ESMTPA id 75A543F59C; Fri, 6 Sep 2019 03:26:30 -0700 (PDT) From: Ruifeng Wang To: tomasz.kantecki@intel.com Cc: dev@dpdk.org, gavin.hu@arm.com, honnappa.nagarahalli@arm.com, nd@arm.com, Ruifeng Wang Date: Fri, 6 Sep 2019 18:26:15 +0800 Message-Id: <20190906102615.36942-3-ruifeng.wang@arm.com> X-Mailer: git-send-email 2.17.1 In-Reply-To: <20190906102615.36942-1-ruifeng.wang@arm.com> References: <20190906102615.36942-1-ruifeng.wang@arm.com> Subject: [dpdk-dev] [PATCH 2/2] examples/l3fwd: integrate RCU QSBR for LPM mode X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Sender: "dev" LPM library relies on RCU QSBR for safe tbl8 group reclaim. l3fwd will start LPM RCU QSBR process when lock-free option is enabled. Signed-off-by: Ruifeng Wang Reviewed-by: Gavin Hu --- examples/l3fwd/Makefile | 1 + examples/l3fwd/l3fwd_lpm.c | 70 ++++++++++++++++++++++++++++++++++++-- examples/l3fwd/main.c | 5 +-- examples/l3fwd/meson.build | 1 + 4 files changed, 73 insertions(+), 4 deletions(-) diff --git a/examples/l3fwd/Makefile b/examples/l3fwd/Makefile index c55f5c288..278586fa1 100644 --- a/examples/l3fwd/Makefile +++ b/examples/l3fwd/Makefile @@ -52,6 +52,7 @@ include $(RTE_SDK)/mk/rte.vars.mk CFLAGS += -I$(SRCDIR) CFLAGS += -O3 $(USER_FLAGS) CFLAGS += $(WERROR_FLAGS) +CFLAGS += -DALLOW_EXPERIMENTAL_API include $(RTE_SDK)/mk/rte.extapp.mk endif diff --git a/examples/l3fwd/l3fwd_lpm.c b/examples/l3fwd/l3fwd_lpm.c index 1435a5711..33ee04c0e 100644 --- a/examples/l3fwd/l3fwd_lpm.c +++ b/examples/l3fwd/l3fwd_lpm.c @@ -21,6 +21,7 @@ #include #include #include +#include #include #include #include @@ -77,6 +78,9 @@ static struct ipv6_l3fwd_lpm_route ipv6_l3fwd_lpm_route_array[] = { struct rte_lpm *ipv4_l3fwd_lpm_lookup_struct[NB_SOCKETS]; struct rte_lpm6 *ipv6_l3fwd_lpm_lookup_struct[NB_SOCKETS]; +struct rte_rcu_qsbr *lpm4_qsv[NB_SOCKETS]; /* RCU QSBR variable for LPM4 */ +extern int numa_on; +extern int rw_lf; static inline uint16_t lpm_get_ipv4_dst_port(void *ipv4_hdr, uint16_t portid, void *lookup_struct) @@ -178,7 +182,7 @@ lpm_main_loop(__attribute__((unused)) void *dummy) struct rte_mbuf *pkts_burst[MAX_PKT_BURST]; unsigned lcore_id; uint64_t prev_tsc, diff_tsc, cur_tsc; - int i, nb_rx; + int i, nb_rx, socketid = 0; uint16_t portid; uint8_t queueid; struct lcore_conf *qconf; @@ -206,6 +210,22 @@ lpm_main_loop(__attribute__((unused)) void *dummy) lcore_id, portid, queueid); } + if (rw_lf) { + if (numa_on) + socketid = rte_lcore_to_socket_id(lcore_id); + else + socketid = 0; + + if (rte_rcu_qsbr_thread_register(lpm4_qsv[socketid], + lcore_id) != 0) { + RTE_LOG(ERR, L3FWD, + "lcore %u failed RCU QSBR register\n", + lcore_id); + return -1; + } + rte_rcu_qsbr_thread_online(lpm4_qsv[socketid], lcore_id); + } + while (!force_quit) { cur_tsc = rte_rdtsc(); @@ -237,8 +257,12 @@ lpm_main_loop(__attribute__((unused)) void *dummy) queueid = qconf->rx_queue_list[i].queue_id; nb_rx = rte_eth_rx_burst(portid, queueid, pkts_burst, MAX_PKT_BURST); - if (nb_rx == 0) + if (nb_rx == 0) { + if (rw_lf) + rte_rcu_qsbr_quiescent( + lpm4_qsv[socketid], lcore_id); continue; + } #if defined RTE_ARCH_X86 || defined RTE_MACHINE_CPUFLAG_NEON \ || defined RTE_ARCH_PPC_64 @@ -248,6 +272,20 @@ lpm_main_loop(__attribute__((unused)) void *dummy) l3fwd_lpm_no_opt_send_packets(nb_rx, pkts_burst, portid, qconf); #endif /* X86 */ + if (rw_lf) + rte_rcu_qsbr_quiescent(lpm4_qsv[socketid], + lcore_id); + } + } + + if (rw_lf) { + rte_rcu_qsbr_thread_offline(lpm4_qsv[socketid], lcore_id); + if (rte_rcu_qsbr_thread_unregister(lpm4_qsv[socketid], + lcore_id) != 0) { + RTE_LOG(ERR, L3FWD, + "lcore %u failed RCU QSBR unregister\n", + lcore_id); + return -1; } } @@ -303,6 +341,34 @@ setup_lpm(const int socketid, __rte_unused const unsigned int flags) ipv4_l3fwd_lpm_route_array[i].if_out); } + if (rw_lf) { + size_t sz; + + /* create RCU QSBR variable */ + sz = rte_rcu_qsbr_get_memsize(RTE_MAX_LCORE); + lpm4_qsv[socketid] = (struct rte_rcu_qsbr *)rte_zmalloc_socket( + NULL, sz, RTE_CACHE_LINE_SIZE, + socketid); + if (lpm4_qsv[socketid] == NULL) + rte_exit(EXIT_FAILURE, + "RCU QSBR alloc fails on socket %d\n", + socketid); + else { + if (rte_rcu_qsbr_init(lpm4_qsv[socketid], + RTE_MAX_LCORE) != 0) + rte_exit(EXIT_FAILURE, + "RCU QSBR init fails on socket %d\n", + socketid); + } + + /* attach RCU QSBR to LPM table */ + if (rte_lpm_rcu_qsbr_add(ipv4_l3fwd_lpm_lookup_struct[socketid], + lpm4_qsv[socketid]) != 0) + rte_exit(EXIT_FAILURE, + "RCU QSBR attach to LPM fails on socket %d\n", + socketid); + } + /* create the LPM6 table */ snprintf(s, sizeof(s), "IPV6_L3FWD_LPM_%d", socketid); diff --git a/examples/l3fwd/main.c b/examples/l3fwd/main.c index 1b435e9eb..797b57554 100644 --- a/examples/l3fwd/main.c +++ b/examples/l3fwd/main.c @@ -72,12 +72,12 @@ static int l3fwd_em_on; /* Global variables. */ -static int numa_on = 1; /**< NUMA is enabled by default. */ +int numa_on = 1; /**< NUMA is enabled by default. */ static int parse_ptype; /**< Parse packet type using rx callback, and */ /**< disabled by default */ static int per_port_pool; /**< Use separate buffer pools per port; disabled */ /**< by default */ -static int rw_lf; /**< Enable lock-free read-write concurrency, */ +int rw_lf; /**< Enable lock-free read-write concurrency, */ /**< disabled by default */ /* Global variables. */ @@ -619,6 +619,7 @@ parse_args(int argc, char **argv) break; case CMD_LINE_OPT_LOCK_FREE_NUM: + printf("RCU lock-free is enabled\n"); rw_lf = 1; break; diff --git a/examples/l3fwd/meson.build b/examples/l3fwd/meson.build index 6dd4b9022..d6e462a1f 100644 --- a/examples/l3fwd/meson.build +++ b/examples/l3fwd/meson.build @@ -6,6 +6,7 @@ # To build this example as a standalone application with an already-installed # DPDK instance, use 'make' +allow_experimental_apis = true deps += ['hash', 'lpm'] sources = files( 'l3fwd_em.c', 'l3fwd_lpm.c', 'main.c'