From patchwork Thu Jan 14 10:30:59 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Feifei Wang X-Patchwork-Id: 86583 X-Patchwork-Delegate: jerinj@marvell.com Return-Path: X-Original-To: patchwork@inbox.dpdk.org Delivered-To: patchwork@inbox.dpdk.org Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id B8EB2A0A02; Thu, 14 Jan 2021 11:31:19 +0100 (CET) Received: from [217.70.189.124] (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id 7964D1410EB; Thu, 14 Jan 2021 11:31:18 +0100 (CET) Received: from foss.arm.com (foss.arm.com [217.140.110.172]) by mails.dpdk.org (Postfix) with ESMTP id 40E481410E2; Thu, 14 Jan 2021 11:31:16 +0100 (CET) Received: from usa-sjc-imap-foss1.foss.arm.com (unknown [10.121.207.14]) by usa-sjc-mx-foss1.foss.arm.com (Postfix) with ESMTP id AC1351FB; Thu, 14 Jan 2021 02:31:15 -0800 (PST) Received: from net-x86-dell-8268.shanghai.arm.com (net-x86-dell-8268.shanghai.arm.com [10.169.210.127]) by usa-sjc-imap-foss1.foss.arm.com (Postfix) with ESMTPA id B99D23F70D; Thu, 14 Jan 2021 02:31:12 -0800 (PST) From: Feifei Wang To: Harry van Haaren , David Hunt , Jerin Jacob , John McNamara Cc: dev@dpdk.org, nd@arm.com, Feifei Wang , stable@dpdk.org, Ruifeng Wang Date: Thu, 14 Jan 2021 18:30:59 +0800 Message-Id: <20210114103101.738262-2-feifei.wang2@arm.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20210114103101.738262-1-feifei.wang2@arm.com> References: <20201221053454.47307-1-feifei.wang2@arm.com> <20210114103101.738262-1-feifei.wang2@arm.com> MIME-Version: 1.0 Subject: [dpdk-dev] [PATCH v4 1/3] doc: fix core enabled bug for eventdev pipeline example X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Sender: "dev" In the case that the cores are isolated, if "-l" or "-c" parameter is not added, the cores will not be enabled and can not launch worker function correctly. In the meanwhile, no error information is reported. For example: totally CPUs:16 isolated CPUs:1-8 command: sudo gdb -args ./dpdk-eventdev_pipeline --vdev event_sw0 \ -- -r1 -t1 -e4 -w F00 -s4 -n0 -c32 -W1000 -D cores information: rte_config->lcore_role = {ROLE_RTE, ROLE_OFF, ROLE_OFF, ROLE_OFF, ROLE_OFF, ROLE_OFF, ROLE_OFF, ROLE_OFF, ROLE_OFF, ROLE_RTE, ROLE_RTE, ROLE_RTE, ROLE_RTE, ROLE_RTE, ROLE_RTE, ROLE_RTE} output information: ... [main()] lcore 9 executing worker, using eventdev port 0 [main()] lcore 10 executing worker, using eventdev port 1 [main()] lcore 11 executing worker, using eventdev port 2 This is because "RTE_LCORE_FOREACH_WORKER" chooses the enabled core. In the case that the cores are isolated, "the lcore_role" flag of isolated cores are set as "ROLE_OFF" by default(not enabled). So if we choose these isolated cores as workers, "RTE_LCORE_FOREACH_WORKER" will ignore these cores and not launch worker functions on them. To fix this, add "-l" parameters to doc and add lcore enabled check. Fixes: 1094ca96689c ("doc: add SW eventdev pipeline to sample app guide") Cc: harry.van.haaren@intel.com Cc: stable@dpdk.org Signed-off-by: Feifei Wang Reviewed-by: Ruifeng Wang --- doc/guides/sample_app_ug/eventdev_pipeline.rst | 5 +++-- examples/eventdev_pipeline/main.c | 7 ++++++- 2 files changed, 9 insertions(+), 3 deletions(-) diff --git a/doc/guides/sample_app_ug/eventdev_pipeline.rst b/doc/guides/sample_app_ug/eventdev_pipeline.rst index 4508c3dcc..19ff53803 100644 --- a/doc/guides/sample_app_ug/eventdev_pipeline.rst +++ b/doc/guides/sample_app_ug/eventdev_pipeline.rst @@ -34,6 +34,7 @@ options. An example eventdev pipeline running with the software eventdev PMD using these settings is shown below: + * ``-l 0,2,8-15``: lcore to use * ``-r1``: core mask 0x1 for RX * ``-t1``: core mask 0x1 for TX * ``-e4``: core mask 0x4 for the software scheduler @@ -46,8 +47,8 @@ these settings is shown below: .. code-block:: console - .//examples/dpdk-eventdev_pipeline --vdev event_sw0 -- -r1 -t1 \ - -e4 -w FF00 -s4 -n0 -c32 -W1000 -D + .//examples/dpdk-eventdev_pipeline -l 0,2,8-15 --vdev event_sw0 \ + -- -r1 -t1 -e4 -w FF00 -s4 -n0 -c32 -W1000 -D The application has some sanity checking built-in, so if there is a function (e.g.; the RX core) which doesn't have a cpu core mask assigned, the application diff --git a/examples/eventdev_pipeline/main.c b/examples/eventdev_pipeline/main.c index 823f8b51c..e9a591134 100644 --- a/examples/eventdev_pipeline/main.c +++ b/examples/eventdev_pipeline/main.c @@ -239,8 +239,13 @@ parse_app_args(int argc, char **argv) if (fdata->worker_core[i]) cdata.num_workers++; - if (core_in_use(i)) + if (core_in_use(i)) { + if (!rte_lcore_is_enabled(i)) { + printf("error: lcore %d is not enabled in lcore list\n", i); + rte_exit(EXIT_FAILURE, "check lcore params failed\n"); + } cdata.active_cores++; + } } } From patchwork Thu Jan 14 10:31:00 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Feifei Wang X-Patchwork-Id: 86584 X-Patchwork-Delegate: jerinj@marvell.com Return-Path: X-Original-To: patchwork@inbox.dpdk.org Delivered-To: patchwork@inbox.dpdk.org Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id B18EBA0A02; Thu, 14 Jan 2021 11:31:27 +0100 (CET) Received: from [217.70.189.124] (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id CBDBB1410F4; Thu, 14 Jan 2021 11:31:20 +0100 (CET) Received: from foss.arm.com (foss.arm.com [217.140.110.172]) by mails.dpdk.org (Postfix) with ESMTP id C4D0B1410F0; Thu, 14 Jan 2021 11:31:19 +0100 (CET) Received: from usa-sjc-imap-foss1.foss.arm.com (unknown [10.121.207.14]) by usa-sjc-mx-foss1.foss.arm.com (Postfix) with ESMTP id 448E81FB; Thu, 14 Jan 2021 02:31:19 -0800 (PST) Received: from net-x86-dell-8268.shanghai.arm.com (net-x86-dell-8268.shanghai.arm.com [10.169.210.127]) by usa-sjc-imap-foss1.foss.arm.com (Postfix) with ESMTPA id 516343F70D; Thu, 14 Jan 2021 02:31:16 -0800 (PST) From: Feifei Wang To: Harry van Haaren , John McNamara , Jerin Jacob , David Hunt Cc: dev@dpdk.org, nd@arm.com, Feifei Wang , stable@dpdk.org, Ruifeng Wang Date: Thu, 14 Jan 2021 18:31:00 +0800 Message-Id: <20210114103101.738262-3-feifei.wang2@arm.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20210114103101.738262-1-feifei.wang2@arm.com> References: <20201221053454.47307-1-feifei.wang2@arm.com> <20210114103101.738262-1-feifei.wang2@arm.com> MIME-Version: 1.0 Subject: [dpdk-dev] [PATCH v4 2/3] examples/eventdev: add info output for main core X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Sender: "dev" When the main core is set as tx/rx/sched/worker core, it also needs to print some information to show this. Thus, add info output for the main core, and add a "dump" function to print core information for the sake of code simplicity and easy maintenance. In the meanwhile, fix the count error. For the variable "worker_idx", it should be incremented when the core is set as worker core. However, when the main core is set as rx/tx/sched core, the worker_idx is also incremented. Though this error may not have a substantial impact due to that the main core is the last launched core, but it should be corrected from the perspective of code correctness. Fixes: 1094ca96689c ("doc: add SW eventdev pipeline to sample app guide") Cc: harry.van.haaren@intel.com Cc: stable@dpdk.org Signed-off-by: Feifei Wang Reviewed-by: Ruifeng Wang --- examples/eventdev_pipeline/main.c | 55 +++++++++++++++++++------------ 1 file changed, 34 insertions(+), 21 deletions(-) diff --git a/examples/eventdev_pipeline/main.c b/examples/eventdev_pipeline/main.c index e9a591134..3526d4d3d 100644 --- a/examples/eventdev_pipeline/main.c +++ b/examples/eventdev_pipeline/main.c @@ -22,6 +22,32 @@ struct config_data cdata = { .worker_cq_depth = 16 }; +static void +dump_core_info(unsigned int lcore_id, struct worker_data *data, + unsigned int worker_idx) +{ + if (fdata->rx_core[lcore_id]) + printf( + "[%s()] lcore %d executing NIC Rx\n", + __func__, lcore_id); + + if (fdata->tx_core[lcore_id]) + printf( + "[%s()] lcore %d executing NIC Tx\n", + __func__, lcore_id); + + if (fdata->sched_core[lcore_id]) + printf( + "[%s()] lcore %d executing scheduler\n", + __func__, lcore_id); + + if (fdata->worker_core[lcore_id]) + printf( + "[%s()] lcore %d executing worker, using eventdev port %u\n", + __func__, lcore_id, + data[worker_idx].port_id); +} + static bool core_in_use(unsigned int lcore_id) { return (fdata->rx_core[lcore_id] || fdata->sched_core[lcore_id] || @@ -411,25 +437,7 @@ main(int argc, char **argv) !fdata->sched_core[lcore_id]) continue; - if (fdata->rx_core[lcore_id]) - printf( - "[%s()] lcore %d executing NIC Rx\n", - __func__, lcore_id); - - if (fdata->tx_core[lcore_id]) - printf( - "[%s()] lcore %d executing NIC Tx\n", - __func__, lcore_id); - - if (fdata->sched_core[lcore_id]) - printf("[%s()] lcore %d executing scheduler\n", - __func__, lcore_id); - - if (fdata->worker_core[lcore_id]) - printf( - "[%s()] lcore %d executing worker, using eventdev port %u\n", - __func__, lcore_id, - worker_data[worker_idx].port_id); + dump_core_info(lcore_id, worker_data, worker_idx); err = rte_eal_remote_launch(fdata->cap.worker, &worker_data[worker_idx], lcore_id); @@ -444,8 +452,13 @@ main(int argc, char **argv) lcore_id = rte_lcore_id(); - if (core_in_use(lcore_id)) - fdata->cap.worker(&worker_data[worker_idx++]); + if (core_in_use(lcore_id)) { + dump_core_info(lcore_id, worker_data, worker_idx); + fdata->cap.worker(&worker_data[worker_idx]); + + if (fdata->worker_core[lcore_id]) + worker_idx++; + } rte_eal_mp_wait_lcore(); From patchwork Thu Jan 14 10:31:01 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Feifei Wang X-Patchwork-Id: 86585 X-Patchwork-Delegate: jerinj@marvell.com Return-Path: X-Original-To: patchwork@inbox.dpdk.org Delivered-To: patchwork@inbox.dpdk.org Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id 16246A0A02; Thu, 14 Jan 2021 11:31:38 +0100 (CET) Received: from [217.70.189.124] (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id 87F9B1410FF; Thu, 14 Jan 2021 11:31:25 +0100 (CET) Received: from foss.arm.com (foss.arm.com [217.140.110.172]) by mails.dpdk.org (Postfix) with ESMTP id 92B091410FF; Thu, 14 Jan 2021 11:31:23 +0100 (CET) Received: from usa-sjc-imap-foss1.foss.arm.com (unknown [10.121.207.14]) by usa-sjc-mx-foss1.foss.arm.com (Postfix) with ESMTP id 1CF991FB; Thu, 14 Jan 2021 02:31:23 -0800 (PST) Received: from net-x86-dell-8268.shanghai.arm.com (net-x86-dell-8268.shanghai.arm.com [10.169.210.127]) by usa-sjc-imap-foss1.foss.arm.com (Postfix) with ESMTPA id DCF7A3F70D; Thu, 14 Jan 2021 02:31:19 -0800 (PST) From: Feifei Wang To: Harry van Haaren , Pavan Nikhilesh , Nikhil Rao Cc: dev@dpdk.org, nd@arm.com, Feifei Wang , pbhagavatula@marvell.com, stable@dpdk.org, Ruifeng Wang , Honnappa Nagarahalli Date: Thu, 14 Jan 2021 18:31:01 +0800 Message-Id: <20210114103101.738262-4-feifei.wang2@arm.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20210114103101.738262-1-feifei.wang2@arm.com> References: <20201221053454.47307-1-feifei.wang2@arm.com> <20210114103101.738262-1-feifei.wang2@arm.com> MIME-Version: 1.0 Subject: [dpdk-dev] [PATCH v4 3/3] examples/eventdev: move eth stop to the end X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Sender: "dev" Move eth stop code from "signal_handler" function to the end of "main" function. There are two reasons for this: First, this improves code maintenance and makes code look simple and clear. Based on this change, after receiving the interrupt signal, "fdata->done" is set as 1. Then the main thread will wait all worker lcores to jump out of the loop. Finally, the main thread will stop and then close eth dev port. Second, for older version, the main thread first stops eth dev port and then waits the end of worker lcore. This may cause errors because it may stop the eth dev port which worker lcores are using. This moving change can fix this by waiting all worker threads to exit and then stop the eth dev port. In the meanwhile, remove wmb in signal_handler. This is because when the main lcore receive the stop signal, it stores 1 into fdata->done. And then the worker lcores load "fdata->done" and jump out of the loop to stop running. Nothing should be stored after updating fdata->done, so the wmb is unnecessary. Fixes: 085edac2ca38 ("examples/eventdev_pipeline: support Tx adapter") Cc: pbhagavatula@marvell.com Cc: stable@dpdk.org Suggested-by: Ruifeng Wang Signed-off-by: Feifei Wang Reviewed-by: Ruifeng Wang Reviewed-by: Honnappa Nagarahalli Acked-by: Harry van Haaren Acked-by: Pavan Nikhilesh --- examples/eventdev_pipeline/main.c | 16 ++++------------ 1 file changed, 4 insertions(+), 12 deletions(-) diff --git a/examples/eventdev_pipeline/main.c b/examples/eventdev_pipeline/main.c index 3526d4d3d..4621e8a89 100644 --- a/examples/eventdev_pipeline/main.c +++ b/examples/eventdev_pipeline/main.c @@ -311,7 +311,6 @@ static void signal_handler(int signum) { static uint8_t once; - uint16_t portid; if (fdata->done) rte_exit(1, "Exiting on signal %d\n", signum); @@ -322,17 +321,6 @@ signal_handler(int signum) rte_event_dev_dump(0, stdout); once = 1; fdata->done = 1; - rte_smp_wmb(); - - RTE_ETH_FOREACH_DEV(portid) { - rte_event_eth_rx_adapter_stop(portid); - rte_event_eth_tx_adapter_stop(portid); - if (rte_eth_dev_stop(portid) < 0) - printf("Failed to stop port %u", portid); - } - - rte_eal_mp_wait_lcore(); - } if (signum == SIGTSTP) rte_event_dev_dump(0, stdout); @@ -483,6 +471,10 @@ main(int argc, char **argv) } RTE_ETH_FOREACH_DEV(portid) { + rte_event_eth_rx_adapter_stop(portid); + rte_event_eth_tx_adapter_stop(portid); + if (rte_eth_dev_stop(portid) < 0) + printf("Failed to stop port %u", portid); rte_eth_dev_close(portid); }