From patchwork Fri Oct 13 15:57:23 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Ferruh Yigit X-Patchwork-Id: 132608 X-Patchwork-Delegate: ferruh.yigit@amd.com Return-Path: X-Original-To: patchwork@inbox.dpdk.org Delivered-To: patchwork@inbox.dpdk.org Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id 3BC1643157; Fri, 13 Oct 2023 17:57:48 +0200 (CEST) Received: from mails.dpdk.org (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id 1C914402CC; Fri, 13 Oct 2023 17:57:48 +0200 (CEST) Received: from NAM10-DM6-obe.outbound.protection.outlook.com (mail-dm6nam10on2079.outbound.protection.outlook.com [40.107.93.79]) by mails.dpdk.org (Postfix) with ESMTP id 4AC004027D for ; Fri, 13 Oct 2023 17:57:46 +0200 (CEST) ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=XoC8tVlVOWqd/uHye4T6ASIRgkA9gZx+z63M73lM/4sI5J4yEoT+iYUV7QQdjiC7Hq5J87LBVMsJQkQZReesX8KW+bbXK+4sLG2nSt341TdDdigaV8/zNhk2F8CM1FVI2TUvqSRM8SBgzi7fZS7pVpFjKEtKAxVUZvOOzvj69UUuIQg4QIxN1t6s0SoGDjgA5NrESEQEQXwjXrYhmVS7/Wy6KBlhYxktUfMfK6cqvKHDAIFxg4EcXVGRsLiJ3MVGn56VxIZkqVd1b7faozglny/YZ5pVzSlZSI4xjxmUA1xi76Gvz5rJ4eiyd9Ad78VChav9xWA7hs8/gmJoqGWKFw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=Ysp4p+hfIUsuYNq75E6yJVQbpXAPOaut87KcBQdIdX8=; b=X3G2i7Sy9g0u98WZ7I0p0q6l9kUe8GudYRR9M6/XYGMb9g0uhYUIFustiuo/Drny8PAi2SFtE4gP4Nok3C3X0RPZLJqbbGvjujsgvoNp4fD1bfGu0Ti+Am+yKXrRzHnBWQj/uCfjb84rcqhuylLubElRaW1UdxyYJnwTMGD8+uxByLQalBTe2zk6FfRZPNc2Fj4duzXs9mGmjozwgsO8H6b6GN/8ZR6QqoRR/WTXXJOSfjL1P7PYNKaMb832myP7M8HaaFc02UOudxhbw5gKXJo8UWK8g3EO19h2QauP1HRYtHJGAaKUQWj43V4KbFH1P6lTOJqqwBePCvnwDq4zqg== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass (sender ip is 165.204.84.17) smtp.rcpttodomain=monjalon.net smtp.mailfrom=amd.com; dmarc=pass (p=quarantine sp=quarantine pct=100) action=none header.from=amd.com; dkim=none (message not signed); arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=amd.com; s=selector1; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=Ysp4p+hfIUsuYNq75E6yJVQbpXAPOaut87KcBQdIdX8=; b=sQcWcTGdAsxPkZM6fD7USZ7n+BlUgYEEJOIMOUxmUWgl2p/JY+iYSOT3M007xZdI0Xc3k3J5VGt7Wx+W8O0MFTlOnb7D2ROKHsqC43R8T6p0RZ5V47/lC5PFVHEha9GifNb9pocIQ9vXY3znfmrGSw8skGjrG9duCcPgnpNXM7E= Received: from SA9PR13CA0156.namprd13.prod.outlook.com (2603:10b6:806:28::11) by PH7PR12MB6587.namprd12.prod.outlook.com (2603:10b6:510:211::17) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.6863.46; Fri, 13 Oct 2023 15:57:43 +0000 Received: from SA2PEPF00001509.namprd04.prod.outlook.com (2603:10b6:806:28:cafe::de) by SA9PR13CA0156.outlook.office365.com (2603:10b6:806:28::11) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.6907.16 via Frontend Transport; Fri, 13 Oct 2023 15:57:42 +0000 X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 165.204.84.17) smtp.mailfrom=amd.com; dkim=none (message not signed) header.d=none;dmarc=pass action=none header.from=amd.com; Received-SPF: Pass (protection.outlook.com: domain of amd.com designates 165.204.84.17 as permitted sender) receiver=protection.outlook.com; client-ip=165.204.84.17; helo=SATLEXMB04.amd.com; pr=C Received: from SATLEXMB04.amd.com (165.204.84.17) by SA2PEPF00001509.mail.protection.outlook.com (10.167.242.41) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.20.6838.22 via Frontend Transport; Fri, 13 Oct 2023 15:57:42 +0000 Received: from telcodpdk.amd.com (10.180.168.240) by SATLEXMB04.amd.com (10.181.40.145) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2507.27; Fri, 13 Oct 2023 10:57:41 -0500 From: Ferruh Yigit To: Thomas Monjalon , Andrew Rybchenko CC: , David Marchand , Jie Hai , Song Jiale , Yuan Peng , Raslan Darawsheh , Qiming Yang , Ivan Malov , Huisong Li Subject: [PATCH v2] ethdev: clarify device queue state after start and stop Date: Fri, 13 Oct 2023 16:57:23 +0100 Message-ID: <20231013155723.2631644-1-ferruh.yigit@amd.com> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20230928205930.2619353-1-ferruh.yigit@amd.com> References: <20230928205930.2619353-1-ferruh.yigit@amd.com> MIME-Version: 1.0 X-Originating-IP: [10.180.168.240] X-ClientProxiedBy: SATLEXMB03.amd.com (10.181.40.144) To SATLEXMB04.amd.com (10.181.40.145) X-EOPAttributedMessage: 0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: SA2PEPF00001509:EE_|PH7PR12MB6587:EE_ X-MS-Office365-Filtering-Correlation-Id: 98d011be-3ed4-47ff-6b28-08dbcc052264 X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: CTsLK91dgcd5lCbweQuCcGvpv8DtX0ZGiiiLNpyD0hoYvPhGVxt85SdA3BGfkYjS37BK591RteFK05vUbp6juaaUu+QAjB1zwgSuSJyP6WvWpS1KpMvW46FaT8poSO1FwwUZk7yv6LJRChIpwzjuPrBKb4vaVLD3DKmQAWXSSvc6DPLssFbZ70FlEEpOJ1b6id+qeyCQNzUxNEG5cBkD3PQ9hWTVKvsFoDQKKokCCOfmENvJmCMolJrBHXhJ7fvCTszJShMLJEqybQ9G6gVWDJ/LIqwxdjEZc97Jzg65eFe/zyBnPnLlMmBQRsI1QuJWz6vt/oSeddjySkXyB31jWMQeBycEfot5ABjLtDueE0rR8hTAAQIVrPPNI3MCB0zhLjRvyR6QHKrvtEP5uEsRybKKZZBomeAKRs4SYQt1ZyOvM/MCXggnQD95T9cppB7qVPkKlw97szRC6GW66hFFcEiHE9MrTSwVvaZnEomOTw9bs8Xtkjj9g7bAR9MjXt9i4omRJmB9nsWBikQHyQSfL3M2psf+xVMemPJfLjPl8fYN8capw78M2ZgFfrOtn0O9qmD7OuUTViGEI9gnwR38aaUPFWmiMoYM0/jjh/ux5D0IKunrJY2y+y01b4arD+z9zhz7jhU6s8C/ijXpsbvrvq/dv02XPho4YrkZN6VCQ5hIwLJUXBgsf2R2QILbS1QVvyFLYKL7u6dFb0YJaMznrZ84/CxPcvNLF/7F/PFJh0rm26WWM0+Yt9DnxSIFVF+02UZ0GeMh5+BSNpkxRuYJMg== X-Forefront-Antispam-Report: CIP:165.204.84.17; CTRY:US; LANG:en; SCL:1; SRV:; IPV:CAL; SFV:NSPM; H:SATLEXMB04.amd.com; PTR:InfoDomainNonexistent; CAT:NONE; SFS:(13230031)(4636009)(136003)(346002)(39860400002)(396003)(376002)(230922051799003)(82310400011)(64100799003)(451199024)(1800799009)(186009)(40470700004)(36840700001)(46966006)(40460700003)(40480700001)(7696005)(478600001)(6666004)(36860700001)(47076005)(86362001)(36756003)(81166007)(356005)(82740400003)(7416002)(2906002)(83380400001)(426003)(336012)(2616005)(16526019)(26005)(1076003)(316002)(70206006)(5660300002)(4326008)(54906003)(110136005)(8676002)(8936002)(70586007)(44832011)(41300700001)(36900700001); DIR:OUT; SFP:1101; X-OriginatorOrg: amd.com X-MS-Exchange-CrossTenant-OriginalArrivalTime: 13 Oct 2023 15:57:42.6341 (UTC) X-MS-Exchange-CrossTenant-Network-Message-Id: 98d011be-3ed4-47ff-6b28-08dbcc052264 X-MS-Exchange-CrossTenant-Id: 3dd8961f-e488-4e60-8e11-a82d994e183d X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=3dd8961f-e488-4e60-8e11-a82d994e183d; Ip=[165.204.84.17]; Helo=[SATLEXMB04.amd.com] X-MS-Exchange-CrossTenant-AuthSource: SA2PEPF00001509.namprd04.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Anonymous X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem X-MS-Exchange-Transport-CrossTenantHeadersStamped: PH7PR12MB6587 X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Drivers start/stop device queues on port start/stop, but not all drivers update queue state accordingly. This becomes more visible if a specific queue stopped explicitly and port stopped/started later, in this case although all queues are started, the state of that specific queue is stopped and it is misleading. Misrepresentation of queue state became a defect with commit [1] that does forwarding decision based on queue state and commit [2] that gets up to date queue state from ethdev/device before forwarding. [1] commit 3c4426db54fc ("app/testpmd: do not poll stopped queues") [2] commit 5028f207a4fa ("app/testpmd: fix secondary process packet forwarding") This patch documents that status of all queues of a device should be `RTE_ETH_QUEUE_STATE_STOPPED` after port stop and their status should be`RTE_ETH_QUEUE_STATE_STARTED` after port start. Also an unit test added to verify drivers. Signed-off-by: Ferruh Yigit Acked-by: Thomas Monjalon --- Cc: Jie Hai Cc: Song Jiale Cc: Yuan Peng Cc: Raslan Darawsheh Cc: Qiming Yang Cc: Ivan Malov Cc: Huisong Li v1: * fix memset * remove commented out code * update unit test to skip queue state if rte_eth_[rt]x_queue_info_get() is not supported v2: * return test_skipped when there is no port available * Syntax fixes --- app/test/meson.build | 1 + app/test/test_ethdev_api.c | 185 +++++++++++++++++++++++++++++++++++++ lib/ethdev/rte_ethdev.h | 5 + 3 files changed, 191 insertions(+) create mode 100644 app/test/test_ethdev_api.c diff --git a/app/test/meson.build b/app/test/meson.build index 20a9333c726d..dd0675638b5e 100644 --- a/app/test/meson.build +++ b/app/test/meson.build @@ -68,6 +68,7 @@ source_file_deps = { 'test_efd.c': ['efd', 'net'], 'test_efd_perf.c': ['efd', 'hash'], 'test_errno.c': [], + 'test_ethdev_api.c': ['ethdev'], 'test_ethdev_link.c': ['ethdev'], 'test_event_crypto_adapter.c': ['cryptodev', 'eventdev', 'bus_vdev'], 'test_event_eth_rx_adapter.c': ['ethdev', 'eventdev', 'bus_vdev'], diff --git a/app/test/test_ethdev_api.c b/app/test/test_ethdev_api.c new file mode 100644 index 000000000000..dc72603d000f --- /dev/null +++ b/app/test/test_ethdev_api.c @@ -0,0 +1,185 @@ +/* SPDX-License-Identifier: BSD-3-Clause + * Copyright (C) 2023, Advanced Micro Devices, Inc. + */ + +#include +#include + +#include +#include "test.h" + +#define NUM_RXQ 2 +#define NUM_TXQ 2 +#define NUM_RXD 512 +#define NUM_TXD 512 +#define NUM_MBUF 1024 +#define MBUF_CACHE_SIZE 256 + +static int32_t +ethdev_api_queue_status(void) +{ + struct rte_eth_dev_info dev_info; + struct rte_eth_rxq_info rx_qinfo; + struct rte_eth_txq_info tx_qinfo; + struct rte_mempool *mbuf_pool; + struct rte_eth_conf eth_conf; + uint16_t port_id; + int ret; + + if (rte_eth_dev_count_avail() == 0) + return TEST_SKIPPED; + + mbuf_pool = rte_pktmbuf_pool_create("MBUF_POOL", NUM_MBUF, MBUF_CACHE_SIZE, 0, + RTE_MBUF_DEFAULT_BUF_SIZE, rte_socket_id()); + + RTE_ETH_FOREACH_DEV(port_id) { + memset(ð_conf, 0, sizeof(eth_conf)); + ret = rte_eth_dev_configure(port_id, NUM_RXQ, NUM_TXQ, ð_conf); + TEST_ASSERT(ret == 0, + "Port(%u) failed to configure.\n", port_id); + + /* RxQ setup */ + for (uint16_t queue_id = 0; queue_id < NUM_RXQ; queue_id++) { + ret = rte_eth_rx_queue_setup(port_id, queue_id, NUM_RXD, + rte_socket_id(), NULL, mbuf_pool); + TEST_ASSERT(ret == 0, + "Port(%u), queue(%u) failed to setup RxQ.\n", + port_id, queue_id); + } + + /* TxQ setup */ + for (uint16_t queue_id = 0; queue_id < NUM_TXQ; queue_id++) { + ret = rte_eth_tx_queue_setup(port_id, queue_id, NUM_TXD, + rte_socket_id(), NULL); + TEST_ASSERT(ret == 0, + "Port(%u), queue(%u) failed to setup TxQ.\n", + port_id, queue_id); + } + + ret = rte_eth_dev_info_get(port_id, &dev_info); + TEST_ASSERT(ret == 0, + "Port(%u) failed to get dev info.\n", port_id); + + /* Initial RxQ */ + for (uint16_t queue_id = 0; queue_id < dev_info.nb_rx_queues; queue_id++) { + ret = rte_eth_rx_queue_info_get(port_id, queue_id, &rx_qinfo); + if (ret == -ENOTSUP) + continue; + + TEST_ASSERT(ret == 0, + "Port(%u), queue(%u) failed to get RxQ info.\n", + port_id, queue_id); + + TEST_ASSERT(rx_qinfo.queue_state == RTE_ETH_QUEUE_STATE_STOPPED, + "Wrong initial Rx queue(%u) state(%d)\n", + queue_id, rx_qinfo.queue_state); + } + + /* Initial TxQ */ + for (uint16_t queue_id = 0; queue_id < dev_info.nb_tx_queues; queue_id++) { + ret = rte_eth_tx_queue_info_get(port_id, queue_id, &tx_qinfo); + if (ret == -ENOTSUP) + continue; + + TEST_ASSERT(ret == 0, + "Port(%u), queue(%u) failed to get TxQ info.\n", + port_id, queue_id); + + TEST_ASSERT(tx_qinfo.queue_state == RTE_ETH_QUEUE_STATE_STOPPED, + "Wrong initial Tx queue(%u) state(%d)\n", + queue_id, tx_qinfo.queue_state); + } + + ret = rte_eth_dev_start(port_id); + TEST_ASSERT(ret == 0, + "Port(%u) failed to start.\n", port_id); + + /* Started RxQ */ + for (uint16_t queue_id = 0; queue_id < dev_info.nb_rx_queues; queue_id++) { + ret = rte_eth_rx_queue_info_get(port_id, queue_id, &rx_qinfo); + if (ret == -ENOTSUP) + continue; + + TEST_ASSERT(ret == 0, + "Port(%u), queue(%u) failed to get RxQ info.\n", + port_id, queue_id); + + TEST_ASSERT(rx_qinfo.queue_state == RTE_ETH_QUEUE_STATE_STARTED, + "Wrong started Rx queue(%u) state(%d)\n", + queue_id, rx_qinfo.queue_state); + } + + /* Started TxQ */ + for (uint16_t queue_id = 0; queue_id < dev_info.nb_tx_queues; queue_id++) { + ret = rte_eth_tx_queue_info_get(port_id, queue_id, &tx_qinfo); + if (ret == -ENOTSUP) + continue; + + TEST_ASSERT(ret == 0, + "Port(%u), queue(%u) failed to get TxQ info.\n", + port_id, queue_id); + + TEST_ASSERT(tx_qinfo.queue_state == RTE_ETH_QUEUE_STATE_STARTED, + "Wrong started Tx queue(%u) state(%d)\n", + queue_id, tx_qinfo.queue_state); + } + + ret = rte_eth_dev_stop(port_id); + TEST_ASSERT(ret == 0, + "Port(%u) failed to stop.\n", port_id); + + /* Stopped RxQ */ + for (uint16_t queue_id = 0; queue_id < dev_info.nb_rx_queues; queue_id++) { + ret = rte_eth_rx_queue_info_get(port_id, queue_id, &rx_qinfo); + if (ret == -ENOTSUP) + continue; + + TEST_ASSERT(ret == 0, + "Port(%u), queue(%u) failed to get RxQ info.\n", + port_id, queue_id); + + TEST_ASSERT(rx_qinfo.queue_state == RTE_ETH_QUEUE_STATE_STOPPED, + "Wrong stopped Rx queue(%u) state(%d)\n", + queue_id, rx_qinfo.queue_state); + } + + /* Stopped TxQ */ + for (uint16_t queue_id = 0; queue_id < dev_info.nb_tx_queues; queue_id++) { + ret = rte_eth_tx_queue_info_get(port_id, queue_id, &tx_qinfo); + if (ret == -ENOTSUP) + continue; + + TEST_ASSERT(ret == 0, + "Port(%u), queue(%u) failed to get TxQ info.\n", + port_id, queue_id); + + TEST_ASSERT(tx_qinfo.queue_state == RTE_ETH_QUEUE_STATE_STOPPED, + "Wrong stopped Tx queue(%u) state(%d)\n", + queue_id, tx_qinfo.queue_state); + } + } + + return TEST_SUCCESS; +} + +static struct unit_test_suite ethdev_api_testsuite = { + .suite_name = "ethdev API tests", + .setup = NULL, + .teardown = NULL, + .unit_test_cases = { + TEST_CASE(ethdev_api_queue_status), + /* TODO: Add deferred_start queue status test */ + TEST_CASES_END() /**< NULL terminate unit test array */ + } +}; + +static int +test_ethdev_api(void) +{ + rte_log_set_global_level(RTE_LOG_DEBUG); + rte_log_set_level(RTE_LOGTYPE_EAL, RTE_LOG_DEBUG); + + return unit_test_suite_runner(ðdev_api_testsuite); +} + +REGISTER_TEST_COMMAND(ethdev_api, test_ethdev_api); diff --git a/lib/ethdev/rte_ethdev.h b/lib/ethdev/rte_ethdev.h index f949dfc83dc2..85b9af7a02c0 100644 --- a/lib/ethdev/rte_ethdev.h +++ b/lib/ethdev/rte_ethdev.h @@ -2812,6 +2812,9 @@ int rte_eth_dev_tx_queue_stop(uint16_t port_id, uint16_t tx_queue_id); * Device RTE_ETH_DEV_NOLIVE_MAC_ADDR flag causes MAC address to be set before * PMD port start callback function is invoked. * + * All device queues (except form deferred start queues) status should be + * `RTE_ETH_QUEUE_STATE_STARTED` after start. + * * On success, all basic functions exported by the Ethernet API (link status, * receive/transmit, and so on) can be invoked. * @@ -2828,6 +2831,8 @@ int rte_eth_dev_start(uint16_t port_id); * Stop an Ethernet device. The device can be restarted with a call to * rte_eth_dev_start() * + * All device queues status should be `RTE_ETH_QUEUE_STATE_STOPPED` after stop. + * * @param port_id * The port identifier of the Ethernet device. * @return