From patchwork Mon Nov 8 18:57:57 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Elena Agostini X-Patchwork-Id: 103971 X-Patchwork-Delegate: thomas@monjalon.net Return-Path: X-Original-To: patchwork@inbox.dpdk.org Delivered-To: patchwork@inbox.dpdk.org Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id 03396A0C4D; Mon, 8 Nov 2021 11:47:14 +0100 (CET) Received: from [217.70.189.124] (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id 908404111E; Mon, 8 Nov 2021 11:47:07 +0100 (CET) Received: from NAM12-DM6-obe.outbound.protection.outlook.com (mail-dm6nam12on2056.outbound.protection.outlook.com [40.107.243.56]) by mails.dpdk.org (Postfix) with ESMTP id C0A5040E28 for ; Mon, 8 Nov 2021 11:47:05 +0100 (CET) ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=YXjnGh7m/OBhKJkuO3bLRu1dBHvM35uEkmLMyZ6ip0YAWuVB0opnPGmjD1oWpy0CP15BbClGjEBVO4oU52uE4jK5WsRfjGMHU4DHjeFW3JMHZfsmvof2zNyeZPoj7FKkkX+W8VEx+++pwoEMOBHWU9Ui+psx+F7BZqhBmluj1ra/TSFmx7TYeJu5JcSFBhaY2d76k3TQl6ljVTM56EL9cKAUhP6Tbv7JfMRE5jeiZ0eO4mvWhSzSREcHnaBIIte3LAuipaeuyjop7a/QfX2VKXLbwtsJcy3+rp/y0cEHbA4nA2IrsAz0RP/9B5DCo8MUHfxAYFifiBODP8nkmwjYqg== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=ffG+yRJ4RWSWAMZdD84+MsRT6D4yMjtktQ4/q9wmcFk=; b=GtO0Il+OYSzn+fhyNFjZr2wxNKq6d1jvxzLhTdY5U6zGcHxx5R2rmgGGs6vhRDdzba3ojvRUJWZJuZMsVifrE92PPd3oMU5tmxVNu9C2NF0SMRvoa0PhLYsSX9wKdRmcuecT8IuOKoXZrDBLSu3KA8lYDivpi/BzRdm8OJAN4mf8pVZ6wiiTy2ilEHfJnEq4It8Q8O78Z/HtW8OkiMJ0JfIPQ8KqwDThCc4EMAm3SnxxTS3EbDu6LfY89CDfF2hZ16EGqcQupke7NNLL66QucOTGtf4jH1WJkYSSWjYt21yJ/UtmjvO2oDLq954s0Ar7weciS9tP1JsLoiyg1hiKnw== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass (sender ip is 216.228.112.34) smtp.rcpttodomain=monjalon.net smtp.mailfrom=nvidia.com; dmarc=pass (p=quarantine sp=quarantine pct=100) action=none header.from=nvidia.com; dkim=none (message not signed); arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=Nvidia.com; s=selector2; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=ffG+yRJ4RWSWAMZdD84+MsRT6D4yMjtktQ4/q9wmcFk=; b=osoOUxVPxKbBdrrcL3oA4BpalWcA3ih+cAP9BSAPiNL74mlmZ4X31uYWcHH5GjAn85EomaxLao7oZzWcnBxSmvpuYZMTvalIkidjtZOxr+z97TEx4Pk62wk5LI6Rcg1QFgu7k952BnbS5JYrEDEtGb0DG+uVD+3NoarAr9xG+N1Hy4eFfb52o4GiApKFTBVeuui4//6IYRpvZ5aL2Ys1rPcppV5zpgr2M7T3S+nBfeHt9rTsHqZnilSmi/UU/pi3wWpoNw66a2wXLNa2NLv7Cbx+0A0pkjC3oWwu7g4tPrErmefBcrgsbz8mGeLbGopHCG+Hg9O0uHMDmSOJl1djZQ== Received: from MW2PR16CA0039.namprd16.prod.outlook.com (2603:10b6:907:1::16) by DM6PR12MB2873.namprd12.prod.outlook.com (2603:10b6:5:18a::32) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4669.13; Mon, 8 Nov 2021 10:47:03 +0000 Received: from CO1NAM11FT064.eop-nam11.prod.protection.outlook.com (2603:10b6:907:1:cafe::ce) by MW2PR16CA0039.outlook.office365.com (2603:10b6:907:1::16) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4669.11 via Frontend Transport; Mon, 8 Nov 2021 10:47:03 +0000 X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 216.228.112.34) smtp.mailfrom=nvidia.com; monjalon.net; dkim=none (message not signed) header.d=none;monjalon.net; dmarc=pass action=none header.from=nvidia.com; Received-SPF: Pass (protection.outlook.com: domain of nvidia.com designates 216.228.112.34 as permitted sender) receiver=protection.outlook.com; client-ip=216.228.112.34; helo=mail.nvidia.com; Received: from mail.nvidia.com (216.228.112.34) by CO1NAM11FT064.mail.protection.outlook.com (10.13.175.77) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA384) id 15.20.4669.10 via Frontend Transport; Mon, 8 Nov 2021 10:47:02 +0000 Received: from nvidia.com (172.20.187.6) by HQMAIL107.nvidia.com (172.20.187.13) with Microsoft SMTP Server (TLS) id 15.0.1497.18; Mon, 8 Nov 2021 10:47:00 +0000 From: To: CC: Elena Agostini , Thomas Monjalon Date: Mon, 8 Nov 2021 18:57:57 +0000 Message-ID: <20211108185805.3887-2-eagostini@nvidia.com> X-Mailer: git-send-email 2.17.1 In-Reply-To: <20211108185805.3887-1-eagostini@nvidia.com> References: <20210602203531.2288645-1-thomas@monjalon.net> <20211108185805.3887-1-eagostini@nvidia.com> MIME-Version: 1.0 X-Originating-IP: [172.20.187.6] X-ClientProxiedBy: HQMAIL105.nvidia.com (172.20.187.12) To HQMAIL107.nvidia.com (172.20.187.13) X-EOPAttributedMessage: 0 X-MS-PublicTrafficType: Email X-MS-Office365-Filtering-Correlation-Id: af456ffa-16fe-4510-8a13-08d9a2a51935 X-MS-TrafficTypeDiagnostic: DM6PR12MB2873: X-LD-Processed: 43083d15-7273-40c1-b7db-39efd9ccc17a,ExtAddr X-Microsoft-Antispam-PRVS: X-MS-Oob-TLC-OOBClassifiers: OLM:1824; X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: gA5aFGpV7ufslXHzxYuvAkbx8fAFTMuE3jB2vCXmqWgdfP9xyv/eBfgN0GxijvQVHCgumeYUT1cWIamDk93tGlNRpl1WxnRPgwgYTjj+6ae3Eohicr6ePk7vR99EvwG1dMzkkGgzwGlpvsHVpxzoTUXJR39TIQ/9Xf+0Ai96Ct+ojeYhnboChrmKDJmf4GTVilut8N1qqRmnCLACRYj+M2sb1d1tLwEWkwYle5z7Zk7jsMRWx74syQ+lTJhRxjqXYGPMveD7l9fmQFA1xfRpu+CfRWFNo2xRuxvkI53MbMhT/e0dMdPTOwR0tVnksTKwnKAbYfN+GUQzhKoEeID3U1iC5Z1eireoFbbZdlOuz2ibzny2h54w6DyIe3GMBlc2yxaaXlUtuatFVV2ym1nuHoAVqMBiMAEXkjGFlS47qGkERJTkhPY2pGESqxjblilywRUPAWRUNyTJno9u3I324TVXrQt4/6UoM/Jnoex41YTC2fkt8EHlnPOSYlC40RIPXzNFHTbV9ov7A68CYAV1hDaum2+x0iBlW/mbJHIFuRvfGuM1IU2Po13uVodqqqRz1yBO5wcvxIleVdWsP26GPflEnwBaa2UZ42Rpa/n+v3tcUodzSMVtQikSLH9EX4lk/gjPs/PbDZMfB4WAhsK0UicgKT1Z9qaBYNLskYSTBS3VdWghWhuZiB+FDNIEnK463qzUMQa1XOAqwmKfIrB3AQ== X-Forefront-Antispam-Report: CIP:216.228.112.34; CTRY:US; LANG:en; SCL:1; SRV:; IPV:NLI; SFV:NSPM; H:mail.nvidia.com; PTR:schybrid03.nvidia.com; CAT:NONE; SFS:(4636009)(46966006)(36840700001)(70206006)(70586007)(47076005)(8936002)(508600001)(336012)(426003)(83380400001)(36756003)(54906003)(6916009)(7636003)(30864003)(2616005)(356005)(55016002)(6286002)(16526019)(186003)(82310400003)(2906002)(316002)(5660300002)(2876002)(4326008)(7696005)(6666004)(36860700001)(8676002)(26005)(86362001)(1076003); DIR:OUT; SFP:1101; X-OriginatorOrg: Nvidia.com X-MS-Exchange-CrossTenant-OriginalArrivalTime: 08 Nov 2021 10:47:02.4799 (UTC) X-MS-Exchange-CrossTenant-Network-Message-Id: af456ffa-16fe-4510-8a13-08d9a2a51935 X-MS-Exchange-CrossTenant-Id: 43083d15-7273-40c1-b7db-39efd9ccc17a X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=43083d15-7273-40c1-b7db-39efd9ccc17a; Ip=[216.228.112.34]; Helo=[mail.nvidia.com] X-MS-Exchange-CrossTenant-AuthSource: CO1NAM11FT064.eop-nam11.prod.protection.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Anonymous X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem X-MS-Exchange-Transport-CrossTenantHeadersStamped: DM6PR12MB2873 Subject: [dpdk-dev] [PATCH v5 1/9] gpudev: introduce GPU device class library X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Sender: "dev" From: Elena Agostini In heterogeneous computing system, processing is not only in the CPU. Some tasks can be delegated to devices working in parallel. The new library gpudev is for dealing with GPGPU computing devices from a DPDK application running on the CPU. The infrastructure is prepared to welcome drivers in drivers/gpu/. Signed-off-by: Elena Agostini Signed-off-by: Thomas Monjalon --- .gitignore | 1 + MAINTAINERS | 6 + app/meson.build | 1 + app/test-gpudev/main.c | 107 +++++++++++ app/test-gpudev/meson.build | 5 + doc/api/doxy-api.conf.in | 1 + doc/guides/conf.py | 8 + doc/guides/gpus/features/default.ini | 10 + doc/guides/gpus/index.rst | 11 ++ doc/guides/gpus/overview.rst | 10 + doc/guides/index.rst | 1 + doc/guides/prog_guide/gpudev.rst | 36 ++++ doc/guides/prog_guide/index.rst | 1 + doc/guides/rel_notes/release_21_11.rst | 4 + drivers/gpu/meson.build | 4 + drivers/meson.build | 1 + lib/gpudev/gpudev.c | 249 +++++++++++++++++++++++++ lib/gpudev/gpudev_driver.h | 67 +++++++ lib/gpudev/meson.build | 10 + lib/gpudev/rte_gpudev.h | 168 +++++++++++++++++ lib/gpudev/version.map | 20 ++ lib/meson.build | 1 + 22 files changed, 722 insertions(+) create mode 100644 app/test-gpudev/main.c create mode 100644 app/test-gpudev/meson.build create mode 100644 doc/guides/gpus/features/default.ini create mode 100644 doc/guides/gpus/index.rst create mode 100644 doc/guides/gpus/overview.rst create mode 100644 doc/guides/prog_guide/gpudev.rst create mode 100644 drivers/gpu/meson.build create mode 100644 lib/gpudev/gpudev.c create mode 100644 lib/gpudev/gpudev_driver.h create mode 100644 lib/gpudev/meson.build create mode 100644 lib/gpudev/rte_gpudev.h create mode 100644 lib/gpudev/version.map diff --git a/.gitignore b/.gitignore index 7ec8688342..b98a43a601 100644 --- a/.gitignore +++ b/.gitignore @@ -15,6 +15,7 @@ doc/guides/compressdevs/overview_feature_table.txt doc/guides/regexdevs/overview_feature_table.txt doc/guides/vdpadevs/overview_feature_table.txt doc/guides/bbdevs/overview_feature_table.txt +doc/guides/gpus/overview_feature_table.txt # ignore generated ctags/cscope files cscope.out.po diff --git a/MAINTAINERS b/MAINTAINERS index 3459187e26..a2e67fb1e6 100644 --- a/MAINTAINERS +++ b/MAINTAINERS @@ -467,6 +467,12 @@ M: Bruce Richardson F: examples/dma/ F: doc/guides/sample_app_ug/dma.rst +General-Purpose Graphics Processing Unit (GPU) API - EXPERIMENTAL +M: Elena Agostini +F: lib/gpudev/ +F: doc/guides/prog_guide/gpudev.rst +F: doc/guides/gpus/features/default.ini + Eventdev API M: Jerin Jacob T: git://dpdk.org/next/dpdk-next-eventdev diff --git a/app/meson.build b/app/meson.build index 986c1a4ad4..310e83076f 100644 --- a/app/meson.build +++ b/app/meson.build @@ -13,6 +13,7 @@ apps = [ 'test-eventdev', 'test-fib', 'test-flow-perf', + 'test-gpudev', 'test-pipeline', 'test-pmd', 'test-regex', diff --git a/app/test-gpudev/main.c b/app/test-gpudev/main.c new file mode 100644 index 0000000000..438cfdac54 --- /dev/null +++ b/app/test-gpudev/main.c @@ -0,0 +1,107 @@ +/* SPDX-License-Identifier: BSD-3-Clause + * Copyright (c) 2021 NVIDIA Corporation & Affiliates + */ + +#include +#include +#include +#include +#include +#include +#include +#include + +#include +#include +#include +#include +#include +#include +#include +#include + +#include + +enum app_args { + ARG_HELP, + ARG_MEMPOOL +}; + +static void +usage(const char *prog_name) +{ + printf("%s [EAL options] --\n", + prog_name); +} + +static void +args_parse(int argc, char **argv) +{ + char **argvopt; + int opt; + int opt_idx; + + static struct option lgopts[] = { + { "help", 0, 0, ARG_HELP}, + /* End of options */ + { 0, 0, 0, 0 } + }; + + argvopt = argv; + while ((opt = getopt_long(argc, argvopt, "", + lgopts, &opt_idx)) != EOF) { + switch (opt) { + case ARG_HELP: + usage(argv[0]); + break; + default: + usage(argv[0]); + rte_exit(EXIT_FAILURE, "Invalid option: %s\n", argv[optind]); + break; + } + } +} + +int +main(int argc, char **argv) +{ + int ret; + int nb_gpus = 0; + int16_t gpu_id = 0; + struct rte_gpu_info ginfo; + + /* Init EAL. */ + ret = rte_eal_init(argc, argv); + if (ret < 0) + rte_exit(EXIT_FAILURE, "EAL init failed\n"); + argc -= ret; + argv += ret; + if (argc > 1) + args_parse(argc, argv); + argc -= ret; + argv += ret; + + nb_gpus = rte_gpu_count_avail(); + printf("\n\nDPDK found %d GPUs:\n", nb_gpus); + RTE_GPU_FOREACH(gpu_id) + { + if (rte_gpu_info_get(gpu_id, &ginfo)) + rte_exit(EXIT_FAILURE, "rte_gpu_info_get error - bye\n"); + + printf("\tGPU ID %d\n\t\tparent ID %d GPU Bus ID %s NUMA node %d Tot memory %.02f MB, Tot processors %d\n", + ginfo.dev_id, + ginfo.parent, + ginfo.name, + ginfo.numa_node, + (((float)ginfo.total_memory)/(float)1024)/(float)1024, + ginfo.processor_count + ); + } + printf("\n\n"); + + /* clean up the EAL */ + rte_eal_cleanup(); + printf("Bye...\n"); + + return EXIT_SUCCESS; +} diff --git a/app/test-gpudev/meson.build b/app/test-gpudev/meson.build new file mode 100644 index 0000000000..17bdef3646 --- /dev/null +++ b/app/test-gpudev/meson.build @@ -0,0 +1,5 @@ +# SPDX-License-Identifier: BSD-3-Clause +# Copyright (c) 2021 NVIDIA Corporation & Affiliates + +sources = files('main.c') +deps = ['gpudev', 'ethdev'] diff --git a/doc/api/doxy-api.conf.in b/doc/api/doxy-api.conf.in index 096ebbaf0d..db2ca9b6ed 100644 --- a/doc/api/doxy-api.conf.in +++ b/doc/api/doxy-api.conf.in @@ -41,6 +41,7 @@ INPUT = @TOPDIR@/doc/api/doxy-api-index.md \ @TOPDIR@/lib/eventdev \ @TOPDIR@/lib/fib \ @TOPDIR@/lib/flow_classify \ + @TOPDIR@/lib/gpudev \ @TOPDIR@/lib/graph \ @TOPDIR@/lib/gro \ @TOPDIR@/lib/gso \ diff --git a/doc/guides/conf.py b/doc/guides/conf.py index 0494b0efe7..e6ce929bc8 100644 --- a/doc/guides/conf.py +++ b/doc/guides/conf.py @@ -152,6 +152,9 @@ def generate_overview_table(output_filename, table_id, section, table_name, titl name = ini_filename[:-4] name = name.replace('_vf', 'vf') pmd_names.append(name) + if not pmd_names: + # Add an empty column if table is empty (required by RST syntax) + pmd_names.append(' ') # Pad the table header names. max_header_len = len(max(pmd_names, key=len)) @@ -393,6 +396,11 @@ def setup(app): 'Features', 'Features availability in bbdev drivers', 'Feature') + table_file = dirname(__file__) + '/gpus/overview_feature_table.txt' + generate_overview_table(table_file, 1, + 'Features', + 'Features availability in GPU drivers', + 'Feature') if LooseVersion(sphinx_version) < LooseVersion('1.3.1'): print('Upgrade sphinx to version >= 1.3.1 for ' diff --git a/doc/guides/gpus/features/default.ini b/doc/guides/gpus/features/default.ini new file mode 100644 index 0000000000..ec7a545eb7 --- /dev/null +++ b/doc/guides/gpus/features/default.ini @@ -0,0 +1,10 @@ +; +; Features of GPU drivers. +; +; This file defines the features that are valid for inclusion in +; the other driver files and also the order that they appear in +; the features table in the documentation. The feature description +; string should not exceed feature_str_len defined in conf.py. +; +[Features] +Get device info = diff --git a/doc/guides/gpus/index.rst b/doc/guides/gpus/index.rst new file mode 100644 index 0000000000..1878423239 --- /dev/null +++ b/doc/guides/gpus/index.rst @@ -0,0 +1,11 @@ +.. SPDX-License-Identifier: BSD-3-Clause + Copyright (c) 2021 NVIDIA Corporation & Affiliates + +General-Purpose Graphics Processing Unit Drivers +================================================ + +.. toctree:: + :maxdepth: 2 + :numbered: + + overview diff --git a/doc/guides/gpus/overview.rst b/doc/guides/gpus/overview.rst new file mode 100644 index 0000000000..4830348818 --- /dev/null +++ b/doc/guides/gpus/overview.rst @@ -0,0 +1,10 @@ +.. SPDX-License-Identifier: BSD-3-Clause + Copyright (c) 2021 NVIDIA Corporation & Affiliates + +Overview of GPU Drivers +======================= + +General-Purpose computing on Graphics Processing Unit (GPGPU) +is the use of GPU to perform parallel computation. + +.. include:: overview_feature_table.txt diff --git a/doc/guides/index.rst b/doc/guides/index.rst index 919825992e..5eb5bd9c9a 100644 --- a/doc/guides/index.rst +++ b/doc/guides/index.rst @@ -22,6 +22,7 @@ DPDK documentation vdpadevs/index regexdevs/index dmadevs/index + gpus/index eventdevs/index rawdevs/index mempool/index diff --git a/doc/guides/prog_guide/gpudev.rst b/doc/guides/prog_guide/gpudev.rst new file mode 100644 index 0000000000..6ea7239159 --- /dev/null +++ b/doc/guides/prog_guide/gpudev.rst @@ -0,0 +1,36 @@ +.. SPDX-License-Identifier: BSD-3-Clause + Copyright (c) 2021 NVIDIA Corporation & Affiliates + +General-Purpose Graphics Processing Unit Library +================================================ + +When mixing networking activity with task processing on a GPU device, +there may be the need to put in communication the CPU with the device +in order to manage the memory, synchronize operations, exchange info, etc.. + +By means of the generic GPU interface provided by this library, +it is possible to allocate a chunk of GPU memory and use it +to create a DPDK mempool with external mbufs having the payload +on the GPU memory, enabling any network interface card +(which support this feature like Mellanox NIC) +to directly transmit and receive packets using GPU memory. + +Additionally, this library provides a number of functions +to enhance the dialog between CPU and GPU. + +Out of scope of this library is to provide a wrapper for GPU specific libraries +(e.g. CUDA Toolkit or OpenCL), thus it is not possible to launch workload +on the device or create GPU specific objects +(e.g. CUDA Driver context or CUDA Streams in case of NVIDIA GPUs). + + +Features +-------- + +This library provides a number of features: + +- Interoperability with device-specific library through generic handlers. + + +API Overview +------------ diff --git a/doc/guides/prog_guide/index.rst b/doc/guides/prog_guide/index.rst index 20e5155cf4..7090b5589a 100644 --- a/doc/guides/prog_guide/index.rst +++ b/doc/guides/prog_guide/index.rst @@ -28,6 +28,7 @@ Programmer's Guide compressdev regexdev dmadev + gpudev rte_security rawdev link_bonding_poll_mode_drv_lib diff --git a/doc/guides/rel_notes/release_21_11.rst b/doc/guides/rel_notes/release_21_11.rst index 8da19c613a..9cf59e73bb 100644 --- a/doc/guides/rel_notes/release_21_11.rst +++ b/doc/guides/rel_notes/release_21_11.rst @@ -101,6 +101,10 @@ New Features Added ``rte_eth_macaddrs_get`` to allow user to retrieve all Ethernet addresses assigned to given ethernet port. +* **Introduced GPU device class with first features:** + + * Device information + * **Added new RSS offload types for IPv4/L4 checksum in RSS flow.** Added macros ETH_RSS_IPV4_CHKSUM and ETH_RSS_L4_CHKSUM, now IPv4 and diff --git a/drivers/gpu/meson.build b/drivers/gpu/meson.build new file mode 100644 index 0000000000..e51ad3381b --- /dev/null +++ b/drivers/gpu/meson.build @@ -0,0 +1,4 @@ +# SPDX-License-Identifier: BSD-3-Clause +# Copyright (c) 2021 NVIDIA Corporation & Affiliates + +drivers = [] diff --git a/drivers/meson.build b/drivers/meson.build index 34c0276487..d5f4e1c1f2 100644 --- a/drivers/meson.build +++ b/drivers/meson.build @@ -19,6 +19,7 @@ subdirs = [ 'vdpa', # depends on common, bus and mempool. 'event', # depends on common, bus, mempool and net. 'baseband', # depends on common and bus. + 'gpu', # depends on common and bus. ] if meson.is_cross_build() diff --git a/lib/gpudev/gpudev.c b/lib/gpudev/gpudev.c new file mode 100644 index 0000000000..aeb021f2cc --- /dev/null +++ b/lib/gpudev/gpudev.c @@ -0,0 +1,249 @@ +/* SPDX-License-Identifier: BSD-3-Clause + * Copyright (c) 2021 NVIDIA Corporation & Affiliates + */ + +#include +#include +#include +#include + +#include "rte_gpudev.h" +#include "gpudev_driver.h" + +/* Logging */ +RTE_LOG_REGISTER_DEFAULT(gpu_logtype, NOTICE); +#define GPU_LOG(level, ...) \ + rte_log(RTE_LOG_ ## level, gpu_logtype, RTE_FMT("gpu: " \ + RTE_FMT_HEAD(__VA_ARGS__, ) "\n", RTE_FMT_TAIL(__VA_ARGS__, ))) + +/* Set any driver error as EPERM */ +#define GPU_DRV_RET(function) \ + ((function != 0) ? -(rte_errno = EPERM) : (rte_errno = 0)) + +/* Array of devices */ +static struct rte_gpu *gpus; +/* Number of currently valid devices */ +static int16_t gpu_max; +/* Number of currently valid devices */ +static int16_t gpu_count; + +int +rte_gpu_init(size_t dev_max) +{ + if (dev_max == 0 || dev_max > INT16_MAX) { + GPU_LOG(ERR, "invalid array size"); + rte_errno = EINVAL; + return -rte_errno; + } + + /* No lock, it must be called before or during first probing. */ + if (gpus != NULL) { + GPU_LOG(ERR, "already initialized"); + rte_errno = EBUSY; + return -rte_errno; + } + + gpus = calloc(dev_max, sizeof(struct rte_gpu)); + if (gpus == NULL) { + GPU_LOG(ERR, "cannot initialize library"); + rte_errno = ENOMEM; + return -rte_errno; + } + + gpu_max = dev_max; + return 0; +} + +uint16_t +rte_gpu_count_avail(void) +{ + return gpu_count; +} + +bool +rte_gpu_is_valid(int16_t dev_id) +{ + if (dev_id >= 0 && dev_id < gpu_max && + gpus[dev_id].state == RTE_GPU_STATE_INITIALIZED) + return true; + return false; +} + +int16_t +rte_gpu_find_next(int16_t dev_id) +{ + if (dev_id < 0) + dev_id = 0; + while (dev_id < gpu_max && + gpus[dev_id].state == RTE_GPU_STATE_UNUSED) + dev_id++; + + if (dev_id >= gpu_max) + return RTE_GPU_ID_NONE; + return dev_id; +} + +static int16_t +gpu_find_free_id(void) +{ + int16_t dev_id; + + for (dev_id = 0; dev_id < gpu_max; dev_id++) { + if (gpus[dev_id].state == RTE_GPU_STATE_UNUSED) + return dev_id; + } + return RTE_GPU_ID_NONE; +} + +static struct rte_gpu * +gpu_get_by_id(int16_t dev_id) +{ + if (!rte_gpu_is_valid(dev_id)) + return NULL; + return &gpus[dev_id]; +} + +struct rte_gpu * +rte_gpu_get_by_name(const char *name) +{ + int16_t dev_id; + struct rte_gpu *dev; + + if (name == NULL) { + rte_errno = EINVAL; + return NULL; + } + + RTE_GPU_FOREACH(dev_id) { + dev = &gpus[dev_id]; + if (strncmp(name, dev->name, RTE_DEV_NAME_MAX_LEN) == 0) + return dev; + } + return NULL; +} + +struct rte_gpu * +rte_gpu_allocate(const char *name) +{ + int16_t dev_id; + struct rte_gpu *dev; + + if (rte_eal_process_type() != RTE_PROC_PRIMARY) { + GPU_LOG(ERR, "only primary process can allocate device"); + rte_errno = EPERM; + return NULL; + } + if (name == NULL) { + GPU_LOG(ERR, "allocate device without a name"); + rte_errno = EINVAL; + return NULL; + } + + /* implicit initialization of library before adding first device */ + if (gpus == NULL && rte_gpu_init(RTE_GPU_DEFAULT_MAX) < 0) + return NULL; + + if (rte_gpu_get_by_name(name) != NULL) { + GPU_LOG(ERR, "device with name %s already exists", name); + rte_errno = EEXIST; + return NULL; + } + dev_id = gpu_find_free_id(); + if (dev_id == RTE_GPU_ID_NONE) { + GPU_LOG(ERR, "reached maximum number of devices"); + rte_errno = ENOENT; + return NULL; + } + + dev = &gpus[dev_id]; + memset(dev, 0, sizeof(*dev)); + + if (rte_strscpy(dev->name, name, RTE_DEV_NAME_MAX_LEN) < 0) { + GPU_LOG(ERR, "device name too long: %s", name); + rte_errno = ENAMETOOLONG; + return NULL; + } + dev->info.name = dev->name; + dev->info.dev_id = dev_id; + dev->info.numa_node = -1; + + gpu_count++; + GPU_LOG(DEBUG, "new device %s (id %d) of total %d", + name, dev_id, gpu_count); + return dev; +} + +void +rte_gpu_complete_new(struct rte_gpu *dev) +{ + if (dev == NULL) + return; + + dev->state = RTE_GPU_STATE_INITIALIZED; +} + +int +rte_gpu_release(struct rte_gpu *dev) +{ + if (dev == NULL) { + rte_errno = ENODEV; + return -rte_errno; + } + + GPU_LOG(DEBUG, "free device %s (id %d)", + dev->info.name, dev->info.dev_id); + dev->state = RTE_GPU_STATE_UNUSED; + gpu_count--; + + return 0; +} + +int +rte_gpu_close(int16_t dev_id) +{ + int firsterr, binerr; + int *lasterr = &firsterr; + struct rte_gpu *dev; + + dev = gpu_get_by_id(dev_id); + if (dev == NULL) { + GPU_LOG(ERR, "close invalid device ID %d", dev_id); + rte_errno = ENODEV; + return -rte_errno; + } + + if (dev->ops.dev_close != NULL) { + *lasterr = GPU_DRV_RET(dev->ops.dev_close(dev)); + if (*lasterr != 0) + lasterr = &binerr; + } + + *lasterr = rte_gpu_release(dev); + + rte_errno = -firsterr; + return firsterr; +} + +int +rte_gpu_info_get(int16_t dev_id, struct rte_gpu_info *info) +{ + struct rte_gpu *dev; + + dev = gpu_get_by_id(dev_id); + if (dev == NULL) { + GPU_LOG(ERR, "query invalid device ID %d", dev_id); + rte_errno = ENODEV; + return -rte_errno; + } + if (info == NULL) { + GPU_LOG(ERR, "query without storage"); + rte_errno = EINVAL; + return -rte_errno; + } + + if (dev->ops.dev_info_get == NULL) { + *info = dev->info; + return 0; + } + return GPU_DRV_RET(dev->ops.dev_info_get(dev, info)); +} diff --git a/lib/gpudev/gpudev_driver.h b/lib/gpudev/gpudev_driver.h new file mode 100644 index 0000000000..9e096e3b64 --- /dev/null +++ b/lib/gpudev/gpudev_driver.h @@ -0,0 +1,67 @@ +/* SPDX-License-Identifier: BSD-3-Clause + * Copyright (c) 2021 NVIDIA Corporation & Affiliates + */ + +/* + * This header file must be included only by drivers. + * It is considered internal, i.e. hidden for the application. + * The prefix rte_ is used to avoid namespace clash in drivers. + */ + +#ifndef RTE_GPUDEV_DRIVER_H +#define RTE_GPUDEV_DRIVER_H + +#include + +#include + +#include "rte_gpudev.h" + +/* Flags indicate current state of device. */ +enum rte_gpu_state { + RTE_GPU_STATE_UNUSED, /* not initialized */ + RTE_GPU_STATE_INITIALIZED, /* initialized */ +}; + +struct rte_gpu; +typedef int (rte_gpu_close_t)(struct rte_gpu *dev); +typedef int (rte_gpu_info_get_t)(struct rte_gpu *dev, struct rte_gpu_info *info); + +struct rte_gpu_ops { + /* Get device info. If NULL, info is just copied. */ + rte_gpu_info_get_t *dev_info_get; + /* Close device. */ + rte_gpu_close_t *dev_close; +}; + +struct rte_gpu { + /* Backing device. */ + struct rte_device *device; + /* Unique identifier name. */ + char name[RTE_DEV_NAME_MAX_LEN]; /* Updated by this library. */ + /* Device info structure. */ + struct rte_gpu_info info; + /* Driver functions. */ + struct rte_gpu_ops ops; + /* Current state (used or not) in the running process. */ + enum rte_gpu_state state; /* Updated by this library. */ + /* Driver-specific private data for the running process. */ + void *process_private; +} __rte_cache_aligned; + +__rte_internal +struct rte_gpu *rte_gpu_get_by_name(const char *name); + +/* First step of initialization */ +__rte_internal +struct rte_gpu *rte_gpu_allocate(const char *name); + +/* Last step of initialization. */ +__rte_internal +void rte_gpu_complete_new(struct rte_gpu *dev); + +/* Last step of removal. */ +__rte_internal +int rte_gpu_release(struct rte_gpu *dev); + +#endif /* RTE_GPUDEV_DRIVER_H */ diff --git a/lib/gpudev/meson.build b/lib/gpudev/meson.build new file mode 100644 index 0000000000..608154817b --- /dev/null +++ b/lib/gpudev/meson.build @@ -0,0 +1,10 @@ +# SPDX-License-Identifier: BSD-3-Clause +# Copyright (c) 2021 NVIDIA Corporation & Affiliates + +headers = files( + 'rte_gpudev.h', +) + +sources = files( + 'gpudev.c', +) diff --git a/lib/gpudev/rte_gpudev.h b/lib/gpudev/rte_gpudev.h new file mode 100644 index 0000000000..eb7cfa8c59 --- /dev/null +++ b/lib/gpudev/rte_gpudev.h @@ -0,0 +1,168 @@ +/* SPDX-License-Identifier: BSD-3-Clause + * Copyright (c) 2021 NVIDIA Corporation & Affiliates + */ + +#ifndef RTE_GPUDEV_H +#define RTE_GPUDEV_H + +#include +#include +#include + +#include + +/** + * @file + * Generic library to interact with GPU computing device. + * + * The API is not thread-safe. + * Device management must be done by a single thread. + * + * @warning + * @b EXPERIMENTAL: this API may change without prior notice. + */ + +#ifdef __cplusplus +extern "C" { +#endif + +/** Maximum number of devices if rte_gpu_init() is not called. */ +#define RTE_GPU_DEFAULT_MAX 32 + +/** Empty device ID. */ +#define RTE_GPU_ID_NONE -1 + +/** Store device info. */ +struct rte_gpu_info { + /** Unique identifier name. */ + const char *name; + /** Device ID. */ + int16_t dev_id; + /** Total processors available on device. */ + uint32_t processor_count; + /** Total memory available on device. */ + size_t total_memory; + /* Local NUMA memory ID. -1 if unknown. */ + int16_t numa_node; +}; + +/** + * @warning + * @b EXPERIMENTAL: this API may change without prior notice. + * + * Initialize the device array before probing devices. + * If not called, the maximum of probed devices is RTE_GPU_DEFAULT_MAX. + * + * @param dev_max + * Maximum number of devices. + * + * @return + * 0 on success, -rte_errno otherwise: + * - ENOMEM if out of memory + * - EINVAL if 0 size + * - EBUSY if already initialized + */ +__rte_experimental +int rte_gpu_init(size_t dev_max); + +/** + * @warning + * @b EXPERIMENTAL: this API may change without prior notice. + * + * Return the number of GPU detected and associated to DPDK. + * + * @return + * The number of available computing devices. + */ +__rte_experimental +uint16_t rte_gpu_count_avail(void); + +/** + * @warning + * @b EXPERIMENTAL: this API may change without prior notice. + * + * Check if the device is valid and initialized in DPDK. + * + * @param dev_id + * The input device ID. + * + * @return + * - True if dev_id is a valid and initialized computing device. + * - False otherwise. + */ +__rte_experimental +bool rte_gpu_is_valid(int16_t dev_id); + +/** + * @warning + * @b EXPERIMENTAL: this API may change without prior notice. + * + * Get the ID of the next valid GPU initialized in DPDK. + * + * @param dev_id + * The initial device ID to start the research. + * + * @return + * Next device ID corresponding to a valid and initialized computing device, + * RTE_GPU_ID_NONE if there is none. + */ +__rte_experimental +int16_t rte_gpu_find_next(int16_t dev_id); + +/** + * @warning + * @b EXPERIMENTAL: this API may change without prior notice. + * + * Macro to iterate over all valid GPU devices. + * + * @param dev_id + * The ID of the next possible valid device, usually 0 to iterate all. + */ +#define RTE_GPU_FOREACH(dev_id) \ + for (dev_id = rte_gpu_find_next(0); \ + dev_id > 0; \ + dev_id = rte_gpu_find_next(dev_id + 1)) + +/** + * @warning + * @b EXPERIMENTAL: this API may change without prior notice. + * + * Close device. + * All resources are released. + * + * @param dev_id + * Device ID to close. + * + * @return + * 0 on success, -rte_errno otherwise: + * - ENODEV if invalid dev_id + * - EPERM if driver error + */ +__rte_experimental +int rte_gpu_close(int16_t dev_id); + +/** + * @warning + * @b EXPERIMENTAL: this API may change without prior notice. + * + * Return device specific info. + * + * @param dev_id + * Device ID to get info. + * @param info + * Memory structure to fill with the info. + * + * @return + * 0 on success, -rte_errno otherwise: + * - ENODEV if invalid dev_id + * - EINVAL if NULL info + * - EPERM if driver error + */ +__rte_experimental +int rte_gpu_info_get(int16_t dev_id, struct rte_gpu_info *info); + +#ifdef __cplusplus +} +#endif + +#endif /* RTE_GPUDEV_H */ diff --git a/lib/gpudev/version.map b/lib/gpudev/version.map new file mode 100644 index 0000000000..6ac6b327e2 --- /dev/null +++ b/lib/gpudev/version.map @@ -0,0 +1,20 @@ +EXPERIMENTAL { + global: + + # added in 21.11 + rte_gpu_close; + rte_gpu_count_avail; + rte_gpu_find_next; + rte_gpu_info_get; + rte_gpu_init; + rte_gpu_is_valid; +}; + +INTERNAL { + global: + + rte_gpu_allocate; + rte_gpu_complete_new; + rte_gpu_get_by_name; + rte_gpu_release; +}; diff --git a/lib/meson.build b/lib/meson.build index 499d26060f..8537a5ab80 100644 --- a/lib/meson.build +++ b/lib/meson.build @@ -34,6 +34,7 @@ libraries = [ 'distributor', 'efd', 'eventdev', + 'gpudev', 'gro', 'gso', 'ip_frag', From patchwork Mon Nov 8 18:57:58 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Elena Agostini X-Patchwork-Id: 103972 X-Patchwork-Delegate: thomas@monjalon.net Return-Path: X-Original-To: patchwork@inbox.dpdk.org Delivered-To: patchwork@inbox.dpdk.org Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id 6D540A0C4D; Mon, 8 Nov 2021 11:47:21 +0100 (CET) Received: from [217.70.189.124] (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id 9035841122; Mon, 8 Nov 2021 11:47:08 +0100 (CET) Received: from NAM04-DM6-obe.outbound.protection.outlook.com (mail-dm6nam08on2083.outbound.protection.outlook.com [40.107.102.83]) by mails.dpdk.org (Postfix) with ESMTP id C4066410E4 for ; Mon, 8 Nov 2021 11:47:05 +0100 (CET) ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=JWf7ktbb6+PMl07uiAIEPGDSX0ZfH2P5vepLxNOSBEdwn8zMre6u+i3hDMDeFCJp+kqMqzbHWi4yF6vqZ1bnVgPIsgDpDir9LtxIKonff/LNQz/mEg9T/HTel7NT1Ak6xuDgynFvwAWBERWaGszrBphc4H/3M7X9eqxOH9GWuWlDFUOre+/BuQ/reNjVQvRsFYLCPKdaqUmnUUvClbsEL4EYnGCxevN/lcpPGuYZBNuEFEdXFCxB0kElGc0e/Kdk3lAHzObHqKM6KXqezXOBc/X57GchgS9hyVpwjagNmmo6o7uQaTHKx9lWWWQLNJ61ulwownyQTxv6RK6Kl2N+fg== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=Y6WzzOVaJDEQN6r0+nyZE+iVv62w3kb2RicpotIK7Z0=; b=f5Xds7SyVrODL1c1Qq5Avtx/vAc86Q1qPAXiHNImdjsKLIJnSyPjMxTlKJtOKl/FbWr7agEHuOCiyS9XrzvIiBjD+DPrHddFr3RzRtNao2/ATVaPQfYsRMwsgKG90zc9FsoJ6amC7twtiCsx97/tP1mRNzboMTbewHkMNv/Oft42cJrEq68pkPTWCXR2vqCR+VPiJp5G+7HEb5QWYBe79wotWXZrjVVPFtumarfveuniq3ux+WCGhqm5tDpEUag+4DQooNq30ouxIvZAXjQZ7e0NRC98mlcK436vjx03dr8gWtF5SWzE5l7GyazoWOnYESUtq9uoCJBt3QPPrBQcrw== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass (sender ip is 216.228.112.34) smtp.rcpttodomain=monjalon.net smtp.mailfrom=nvidia.com; dmarc=pass (p=quarantine sp=quarantine pct=100) action=none header.from=nvidia.com; dkim=none (message not signed); arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=Nvidia.com; s=selector2; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=Y6WzzOVaJDEQN6r0+nyZE+iVv62w3kb2RicpotIK7Z0=; b=jN95WoUZibywmZga73cFbPyKSZZMUvA5PlOKxlqIkLvdeLllHvAsBeaUgD/F5dp4lg3bOvkcpnBcoNAi28zYBJiM9Kh5YjTTMk7n0CufM2Kz7vCdjnCQ83OIZAzEUowChfAxBP40wqmw/FI3OcdFz5pnSE5jX2uOax4b6XGiHtoGF5huXsXgWzjBP01M9yQI9o8779Ww3qDrBxHpGouwSrlVtsf+bLE6h+LZHDXukorIawS0PWPda59JKIEl7Gxd2LQtGVlKfgIKuMGx2o7IGHpIlRT15SIAe+udgfGsEYG4rh6TxyMDLcKvnJ9lLfwWet6o8pgKL0z6aUTvBS00aA== Received: from MW2PR16CA0061.namprd16.prod.outlook.com (2603:10b6:907:1::38) by BY5PR12MB4289.namprd12.prod.outlook.com (2603:10b6:a03:204::14) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4669.10; Mon, 8 Nov 2021 10:47:04 +0000 Received: from CO1NAM11FT064.eop-nam11.prod.protection.outlook.com (2603:10b6:907:1:cafe::11) by MW2PR16CA0061.outlook.office365.com (2603:10b6:907:1::38) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4669.10 via Frontend Transport; Mon, 8 Nov 2021 10:47:04 +0000 X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 216.228.112.34) smtp.mailfrom=nvidia.com; monjalon.net; dkim=none (message not signed) header.d=none;monjalon.net; dmarc=pass action=none header.from=nvidia.com; Received-SPF: Pass (protection.outlook.com: domain of nvidia.com designates 216.228.112.34 as permitted sender) receiver=protection.outlook.com; client-ip=216.228.112.34; helo=mail.nvidia.com; Received: from mail.nvidia.com (216.228.112.34) by CO1NAM11FT064.mail.protection.outlook.com (10.13.175.77) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA384) id 15.20.4669.10 via Frontend Transport; Mon, 8 Nov 2021 10:47:03 +0000 Received: from nvidia.com (172.20.187.6) by HQMAIL107.nvidia.com (172.20.187.13) with Microsoft SMTP Server (TLS) id 15.0.1497.18; Mon, 8 Nov 2021 10:47:01 +0000 From: To: CC: Thomas Monjalon Date: Mon, 8 Nov 2021 18:57:58 +0000 Message-ID: <20211108185805.3887-3-eagostini@nvidia.com> X-Mailer: git-send-email 2.17.1 In-Reply-To: <20211108185805.3887-1-eagostini@nvidia.com> References: <20210602203531.2288645-1-thomas@monjalon.net> <20211108185805.3887-1-eagostini@nvidia.com> MIME-Version: 1.0 X-Originating-IP: [172.20.187.6] X-ClientProxiedBy: HQMAIL105.nvidia.com (172.20.187.12) To HQMAIL107.nvidia.com (172.20.187.13) X-EOPAttributedMessage: 0 X-MS-PublicTrafficType: Email X-MS-Office365-Filtering-Correlation-Id: 574e3bec-5bcf-46ed-4168-08d9a2a519e1 X-MS-TrafficTypeDiagnostic: BY5PR12MB4289: X-LD-Processed: 43083d15-7273-40c1-b7db-39efd9ccc17a,ExtAddr X-Microsoft-Antispam-PRVS: X-MS-Oob-TLC-OOBClassifiers: OLM:8273; X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: L8k3PP2XUvxvFKiW5YMOFh1znCxnvlO9gtvdKV0fJ+UWCSP/B6VgO/jjwvuJsQBHZneIOBsyIm2IvOXm2Abl3rjU4ZDnRSCm43LU6FSiTlF/bOuxVXMPLdFZSTXtfwEC93p3tdAGsemInG7solYVnMBivpDA+irtt86l1ZFKzWT0d5nzLpc+tCl2mhVcisimgZKGe/ToTPgAXGdC5A+6imAs/FZRJre2Cv1BcYwhXaMJoWHN3UC7DW+myS8l/uxEXHdouDyIO3Dkma452cj0dUrI2LBQpSWGp9X8WMdE6kmg6Db4QDVk+BL1Nd37x7qk0kiQ6ThR0VMfCnujSvD7aI77P1OA6YsqHFarU7HFpixJ/gBhAXmrXBJywpq0Uw1iYp6MVrmiCgZE8mc8lCzvlbooWiMVv5C6QH6yD94WwZK8KExrj1Lv7Z5cZRUiEw3nx4F8d0s9L3B+QLT3fOI7uczilWUYq4irk2SrJb1CEzY+zRlI41SBPpsR7WoSm2SI9UM4TGfDUFy1xupGpm9XcLT1E+XUWeGG6otta1BlqpsMjiVBGdJnNJND2GXvSEy3VG9Rz/VbJyHggVmyIGyGPzpwMSD/WlGCi/IGGOjz4kPDDIxHLIjard2UpQecaprRrDecrzvoj2co4rk1ZiolTefK8ebLVAoUxDDetUk4AEpUzs0P16qJoF9M2YhnMYZlY5IqD+FN6RVVZy1tOIYfig== X-Forefront-Antispam-Report: CIP:216.228.112.34; CTRY:US; LANG:en; SCL:1; SRV:; IPV:NLI; SFV:NSPM; H:mail.nvidia.com; PTR:schybrid03.nvidia.com; CAT:NONE; SFS:(4636009)(36840700001)(46966006)(8936002)(55016002)(186003)(36756003)(83380400001)(26005)(508600001)(2616005)(2876002)(86362001)(426003)(6286002)(6916009)(6666004)(16526019)(70586007)(336012)(5660300002)(4326008)(82310400003)(7696005)(356005)(8676002)(7636003)(70206006)(36860700001)(15650500001)(316002)(2906002)(47076005)(1076003); DIR:OUT; SFP:1101; X-OriginatorOrg: Nvidia.com X-MS-Exchange-CrossTenant-OriginalArrivalTime: 08 Nov 2021 10:47:03.6092 (UTC) X-MS-Exchange-CrossTenant-Network-Message-Id: 574e3bec-5bcf-46ed-4168-08d9a2a519e1 X-MS-Exchange-CrossTenant-Id: 43083d15-7273-40c1-b7db-39efd9ccc17a X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=43083d15-7273-40c1-b7db-39efd9ccc17a; Ip=[216.228.112.34]; Helo=[mail.nvidia.com] X-MS-Exchange-CrossTenant-AuthSource: CO1NAM11FT064.eop-nam11.prod.protection.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Anonymous X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem X-MS-Exchange-Transport-CrossTenantHeadersStamped: BY5PR12MB4289 Subject: [dpdk-dev] [PATCH v5 2/9] gpudev: add event notification X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Sender: "dev" From: Thomas Monjalon Callback functions may be registered for a device event. Callback management is per-process and not thread-safe. The events RTE_GPU_EVENT_NEW and RTE_GPU_EVENT_DEL are notified respectively after creation and before removal of a device, as part of the library functions. Some future events may be emitted from drivers. Signed-off-by: Thomas Monjalon --- lib/gpudev/gpudev.c | 148 +++++++++++++++++++++++++++++++++++++ lib/gpudev/gpudev_driver.h | 7 ++ lib/gpudev/rte_gpudev.h | 70 ++++++++++++++++++ lib/gpudev/version.map | 3 + 4 files changed, 228 insertions(+) diff --git a/lib/gpudev/gpudev.c b/lib/gpudev/gpudev.c index aeb021f2cc..07572ae040 100644 --- a/lib/gpudev/gpudev.c +++ b/lib/gpudev/gpudev.c @@ -3,6 +3,7 @@ */ #include +#include #include #include #include @@ -27,6 +28,16 @@ static int16_t gpu_max; /* Number of currently valid devices */ static int16_t gpu_count; +/* Event callback object */ +struct rte_gpu_callback { + TAILQ_ENTRY(rte_gpu_callback) next; + rte_gpu_callback_t *function; + void *user_data; + enum rte_gpu_event event; +}; +static rte_rwlock_t gpu_callback_lock = RTE_RWLOCK_INITIALIZER; +static void gpu_free_callbacks(struct rte_gpu *dev); + int rte_gpu_init(size_t dev_max) { @@ -166,6 +177,7 @@ rte_gpu_allocate(const char *name) dev->info.name = dev->name; dev->info.dev_id = dev_id; dev->info.numa_node = -1; + TAILQ_INIT(&dev->callbacks); gpu_count++; GPU_LOG(DEBUG, "new device %s (id %d) of total %d", @@ -180,6 +192,8 @@ rte_gpu_complete_new(struct rte_gpu *dev) return; dev->state = RTE_GPU_STATE_INITIALIZED; + dev->state = RTE_GPU_STATE_INITIALIZED; + rte_gpu_notify(dev, RTE_GPU_EVENT_NEW); } int @@ -192,6 +206,9 @@ rte_gpu_release(struct rte_gpu *dev) GPU_LOG(DEBUG, "free device %s (id %d)", dev->info.name, dev->info.dev_id); + rte_gpu_notify(dev, RTE_GPU_EVENT_DEL); + + gpu_free_callbacks(dev); dev->state = RTE_GPU_STATE_UNUSED; gpu_count--; @@ -224,6 +241,137 @@ rte_gpu_close(int16_t dev_id) return firsterr; } +int +rte_gpu_callback_register(int16_t dev_id, enum rte_gpu_event event, + rte_gpu_callback_t *function, void *user_data) +{ + int16_t next_dev, last_dev; + struct rte_gpu_callback_list *callbacks; + struct rte_gpu_callback *callback; + + if (!rte_gpu_is_valid(dev_id) && dev_id != RTE_GPU_ID_ANY) { + GPU_LOG(ERR, "register callback of invalid ID %d", dev_id); + rte_errno = ENODEV; + return -rte_errno; + } + if (function == NULL) { + GPU_LOG(ERR, "cannot register callback without function"); + rte_errno = EINVAL; + return -rte_errno; + } + + if (dev_id == RTE_GPU_ID_ANY) { + next_dev = 0; + last_dev = gpu_max - 1; + } else { + next_dev = last_dev = dev_id; + } + + rte_rwlock_write_lock(&gpu_callback_lock); + do { + callbacks = &gpus[next_dev].callbacks; + + /* check if not already registered */ + TAILQ_FOREACH(callback, callbacks, next) { + if (callback->event == event && + callback->function == function && + callback->user_data == user_data) { + GPU_LOG(INFO, "callback already registered"); + return 0; + } + } + + callback = malloc(sizeof(*callback)); + if (callback == NULL) { + GPU_LOG(ERR, "cannot allocate callback"); + return -ENOMEM; + } + callback->function = function; + callback->user_data = user_data; + callback->event = event; + TAILQ_INSERT_TAIL(callbacks, callback, next); + + } while (++next_dev <= last_dev); + rte_rwlock_write_unlock(&gpu_callback_lock); + + return 0; +} + +int +rte_gpu_callback_unregister(int16_t dev_id, enum rte_gpu_event event, + rte_gpu_callback_t *function, void *user_data) +{ + int16_t next_dev, last_dev; + struct rte_gpu_callback_list *callbacks; + struct rte_gpu_callback *callback, *nextcb; + + if (!rte_gpu_is_valid(dev_id) && dev_id != RTE_GPU_ID_ANY) { + GPU_LOG(ERR, "unregister callback of invalid ID %d", dev_id); + rte_errno = ENODEV; + return -rte_errno; + } + if (function == NULL) { + GPU_LOG(ERR, "cannot unregister callback without function"); + rte_errno = EINVAL; + return -rte_errno; + } + + if (dev_id == RTE_GPU_ID_ANY) { + next_dev = 0; + last_dev = gpu_max - 1; + } else { + next_dev = last_dev = dev_id; + } + + rte_rwlock_write_lock(&gpu_callback_lock); + do { + callbacks = &gpus[next_dev].callbacks; + RTE_TAILQ_FOREACH_SAFE(callback, callbacks, next, nextcb) { + if (callback->event != event || + callback->function != function || + (callback->user_data != user_data && + user_data != (void *)-1)) + continue; + TAILQ_REMOVE(callbacks, callback, next); + free(callback); + } + } while (++next_dev <= last_dev); + rte_rwlock_write_unlock(&gpu_callback_lock); + + return 0; +} + +static void +gpu_free_callbacks(struct rte_gpu *dev) +{ + struct rte_gpu_callback_list *callbacks; + struct rte_gpu_callback *callback, *nextcb; + + callbacks = &dev->callbacks; + rte_rwlock_write_lock(&gpu_callback_lock); + RTE_TAILQ_FOREACH_SAFE(callback, callbacks, next, nextcb) { + TAILQ_REMOVE(callbacks, callback, next); + free(callback); + } + rte_rwlock_write_unlock(&gpu_callback_lock); +} + +void +rte_gpu_notify(struct rte_gpu *dev, enum rte_gpu_event event) +{ + int16_t dev_id; + struct rte_gpu_callback *callback; + + dev_id = dev->info.dev_id; + rte_rwlock_read_lock(&gpu_callback_lock); + TAILQ_FOREACH(callback, &dev->callbacks, next) { + if (callback->event != event || callback->function == NULL) + continue; + callback->function(dev_id, event, callback->user_data); + } + rte_rwlock_read_unlock(&gpu_callback_lock); +} + int rte_gpu_info_get(int16_t dev_id, struct rte_gpu_info *info) { diff --git a/lib/gpudev/gpudev_driver.h b/lib/gpudev/gpudev_driver.h index 9e096e3b64..2a7089aa52 100644 --- a/lib/gpudev/gpudev_driver.h +++ b/lib/gpudev/gpudev_driver.h @@ -12,6 +12,7 @@ #define RTE_GPUDEV_DRIVER_H #include +#include #include @@ -43,6 +44,8 @@ struct rte_gpu { struct rte_gpu_info info; /* Driver functions. */ struct rte_gpu_ops ops; + /* Event callback list. */ + TAILQ_HEAD(rte_gpu_callback_list, rte_gpu_callback) callbacks; /* Current state (used or not) in the running process. */ enum rte_gpu_state state; /* Updated by this library. */ /* Driver-specific private data for the running process. */ @@ -64,4 +67,8 @@ void rte_gpu_complete_new(struct rte_gpu *dev); __rte_internal int rte_gpu_release(struct rte_gpu *dev); +/* Call registered callbacks. No multi-process event. */ +__rte_internal +void rte_gpu_notify(struct rte_gpu *dev, enum rte_gpu_event); + #endif /* RTE_GPUDEV_DRIVER_H */ diff --git a/lib/gpudev/rte_gpudev.h b/lib/gpudev/rte_gpudev.h index eb7cfa8c59..e1702fbfe4 100644 --- a/lib/gpudev/rte_gpudev.h +++ b/lib/gpudev/rte_gpudev.h @@ -31,6 +31,11 @@ extern "C" { /** Empty device ID. */ #define RTE_GPU_ID_NONE -1 +/** Catch-all device ID. */ +#define RTE_GPU_ID_ANY INT16_MIN + +/** Catch-all callback data. */ +#define RTE_GPU_CALLBACK_ANY_DATA ((void *)-1) /** Store device info. */ struct rte_gpu_info { @@ -46,6 +51,18 @@ struct rte_gpu_info { int16_t numa_node; }; +/** Flags passed in notification callback. */ +enum rte_gpu_event { + /** Device is just initialized. */ + RTE_GPU_EVENT_NEW, + /** Device is going to be released. */ + RTE_GPU_EVENT_DEL, +}; + +/** Prototype of event callback function. */ +typedef void (rte_gpu_callback_t)(int16_t dev_id, + enum rte_gpu_event event, void *user_data); + /** * @warning * @b EXPERIMENTAL: this API may change without prior notice. @@ -141,6 +158,59 @@ int16_t rte_gpu_find_next(int16_t dev_id); __rte_experimental int rte_gpu_close(int16_t dev_id); +/** + * @warning + * @b EXPERIMENTAL: this API may change without prior notice. + * + * Register a function as event callback. + * A function may be registered multiple times for different events. + * + * @param dev_id + * Device ID to get notified about. + * RTE_GPU_ID_ANY means all devices. + * @param event + * Device event to be registered for. + * @param function + * Callback function to be called on event. + * @param user_data + * Optional parameter passed in the callback. + * + * @return + * 0 on success, -rte_errno otherwise: + * - ENODEV if invalid dev_id + * - EINVAL if NULL function + * - ENOMEM if out of memory + */ +__rte_experimental +int rte_gpu_callback_register(int16_t dev_id, enum rte_gpu_event event, + rte_gpu_callback_t *function, void *user_data); + +/** + * @warning + * @b EXPERIMENTAL: this API may change without prior notice. + * + * Unregister for an event. + * + * @param dev_id + * Device ID to be silenced. + * RTE_GPU_ID_ANY means all devices. + * @param event + * Registered event. + * @param function + * Registered function. + * @param user_data + * Optional parameter as registered. + * RTE_GPU_CALLBACK_ANY_DATA is a catch-all. + * + * @return + * 0 on success, -rte_errno otherwise: + * - ENODEV if invalid dev_id + * - EINVAL if NULL function + */ +__rte_experimental +int rte_gpu_callback_unregister(int16_t dev_id, enum rte_gpu_event event, + rte_gpu_callback_t *function, void *user_data); + /** * @warning * @b EXPERIMENTAL: this API may change without prior notice. diff --git a/lib/gpudev/version.map b/lib/gpudev/version.map index 6ac6b327e2..b3b6b76c1c 100644 --- a/lib/gpudev/version.map +++ b/lib/gpudev/version.map @@ -2,6 +2,8 @@ EXPERIMENTAL { global: # added in 21.11 + rte_gpu_callback_register; + rte_gpu_callback_unregister; rte_gpu_close; rte_gpu_count_avail; rte_gpu_find_next; @@ -16,5 +18,6 @@ INTERNAL { rte_gpu_allocate; rte_gpu_complete_new; rte_gpu_get_by_name; + rte_gpu_notify; rte_gpu_release; }; From patchwork Mon Nov 8 18:57:59 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Elena Agostini X-Patchwork-Id: 103973 X-Patchwork-Delegate: thomas@monjalon.net Return-Path: X-Original-To: patchwork@inbox.dpdk.org Delivered-To: patchwork@inbox.dpdk.org Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id 06A3BA0C4D; Mon, 8 Nov 2021 11:47:32 +0100 (CET) Received: from [217.70.189.124] (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id E62784113B; Mon, 8 Nov 2021 11:47:09 +0100 (CET) Received: from NAM10-MW2-obe.outbound.protection.outlook.com (mail-mw2nam10on2073.outbound.protection.outlook.com [40.107.94.73]) by mails.dpdk.org (Postfix) with ESMTP id C82E24111D for ; Mon, 8 Nov 2021 11:47:06 +0100 (CET) ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=nDhpo3Grc96XVELzhveTBXBtniTXEMlNll0IkYk/Fwp612XxBrO+g57wo+yN6ARrTM+yGnBLoCvcRHPOMg5JX9eBORw9m/hjUfG1fJ6EQ0rY2xd8vXUBnjQdDnramH8pmjvnQscwYGqKTTEuSe673eNM54AnVRTa5xEeZqOxGMiE2oLmzzExiLVzLtzbeaiQwUOQGgI+Po+eWke0RKxvQCNKUSnQ2yqOJKhp8klJiKYx2V3g28eWDsF+k9iFGdYjciFlT1ZSr/GxNm7G6flHOe992mKc3v08SXyzWv+njy3hvrrbcHf2SPxckWWf5bAtgB6RQVU/4cw5Atc0AhfL+g== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=TnfqIRyBhFfrJZ+ip7Yeps/B7RWflH5DKzocSlAQKBo=; b=Bg8/c2F7V2JQpoB/OsauGiskreSMkb2EerTJFGoVjsxcIY/9r47myGC0rjGJDjuJXYrDzouKtCOcHFdA6617YpQanc90YBRbcsYeU4sTItlISTFctXINiP6CPBjLgw9TD/cDU44b1Wkd+hBcaH5McvW9G9BU85vESVuuI8/GTCEivMTGMKYU1typzvpgOTo7mWbv39Yo8VJ6Nj98Q/j4IzIlFifnT1OcKsl5reXO3DHl4McYRrxP/RK1af1BR7bQiWhfG519WqUf9oVJlVkBQ5N93oQYvfv+rsfRPYsw2MMOgLeJqw/3kmdI6abZuPYkKJfE0SFiFi5NwwfclwQcfw== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass (sender ip is 216.228.112.34) smtp.rcpttodomain=monjalon.net smtp.mailfrom=nvidia.com; dmarc=pass (p=quarantine sp=quarantine pct=100) action=none header.from=nvidia.com; dkim=none (message not signed); arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=Nvidia.com; s=selector2; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=TnfqIRyBhFfrJZ+ip7Yeps/B7RWflH5DKzocSlAQKBo=; b=AGq16BjD01iiWBbMNeeaWEvAIR+XJB0TnhVzCVyDs33xP8pyb+RHuS5pK7mM5HC8vq28LAyrHQt3YXp5+VwX5CqyTgX5NpOTT7nq/ab9/qy/d6K7lPsaipjklIuhmmWy3+n11wT2k0/tr33rcY8Ekvx8w7t13H8dZcGXC2yQE2bMo1qY5VIRie7yk8dhjygs1z/2Igs2XPiehpfzrMUe0tav3NtYcw+addeP+EcJ1PG0OQhonPFIrwyuHm90FWuoOCVx55qun9C6iYTAOtN4Fm/o6ltQ783wCWPtWYa6vHudFU5klzMwWPFzbSyKMti6nSqDYHtsPdupGPkSSFSUuQ== Received: from MW2PR16CA0057.namprd16.prod.outlook.com (2603:10b6:907:1::34) by BY5PR12MB3874.namprd12.prod.outlook.com (2603:10b6:a03:1ad::15) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4669.11; Mon, 8 Nov 2021 10:47:04 +0000 Received: from CO1NAM11FT064.eop-nam11.prod.protection.outlook.com (2603:10b6:907:1:cafe::50) by MW2PR16CA0057.outlook.office365.com (2603:10b6:907:1::34) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4669.10 via Frontend Transport; Mon, 8 Nov 2021 10:47:04 +0000 X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 216.228.112.34) smtp.mailfrom=nvidia.com; monjalon.net; dkim=none (message not signed) header.d=none;monjalon.net; dmarc=pass action=none header.from=nvidia.com; Received-SPF: Pass (protection.outlook.com: domain of nvidia.com designates 216.228.112.34 as permitted sender) receiver=protection.outlook.com; client-ip=216.228.112.34; helo=mail.nvidia.com; Received: from mail.nvidia.com (216.228.112.34) by CO1NAM11FT064.mail.protection.outlook.com (10.13.175.77) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA384) id 15.20.4669.10 via Frontend Transport; Mon, 8 Nov 2021 10:47:04 +0000 Received: from nvidia.com (172.20.187.6) by HQMAIL107.nvidia.com (172.20.187.13) with Microsoft SMTP Server (TLS) id 15.0.1497.18; Mon, 8 Nov 2021 10:47:01 +0000 From: To: CC: Thomas Monjalon Date: Mon, 8 Nov 2021 18:57:59 +0000 Message-ID: <20211108185805.3887-4-eagostini@nvidia.com> X-Mailer: git-send-email 2.17.1 In-Reply-To: <20211108185805.3887-1-eagostini@nvidia.com> References: <20210602203531.2288645-1-thomas@monjalon.net> <20211108185805.3887-1-eagostini@nvidia.com> MIME-Version: 1.0 X-Originating-IP: [172.20.187.6] X-ClientProxiedBy: HQMAIL105.nvidia.com (172.20.187.12) To HQMAIL107.nvidia.com (172.20.187.13) X-EOPAttributedMessage: 0 X-MS-PublicTrafficType: Email X-MS-Office365-Filtering-Correlation-Id: 50693c8f-c5fb-4652-cc7a-08d9a2a51a5a X-MS-TrafficTypeDiagnostic: BY5PR12MB3874: X-LD-Processed: 43083d15-7273-40c1-b7db-39efd9ccc17a,ExtAddr X-Microsoft-Antispam-PRVS: X-MS-Oob-TLC-OOBClassifiers: OLM:6790; X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: uCX8Il18zHgJ/6LH9zZrzj+gdx2sHbuJbdhTzRaOjW8zO+SQ26ltpsrKCAAUuyjlPSa2koqvntbBPCD+OY82T/xp7vZHVz1sTTIfjc6J0YN3HYmnoIWASzoH/IYCrRstg8JwtCZAHKHlEgUKzdngrXHbGLPO33oqB1Ifugvs1gjE3BXQPTEISgeopj5AMMBeYwTNm/iLla02DJwBIEHrOuP0nlsCYjEUO3mTi6e9UkqO/ucIsMrC78Wfr8xUogCqP0s0WR+QlnrrmVKsborbuBwOAjSGWgM6Y8DG8Rc0iIW6M/NwQX8Z7L4VMrQ4uAjuh/KEWfup3Cs9wOw3Z6G0oTKMZ9PM0QTU2JxYDxpWFQkCn9PVsFJm5p1tfxHYAF8m9aEqgYjqAtv9djNqfGDWnXbjxRMXsHbd8x6iRuzM8Kqs6DUEhaRpkvyQoGw1LT9oc68vaHJ/13MHn7gAyh1uURDbEu8GvS4Uve57S4WR+pmNjouHMc4mYNquk7zdffYwm5U3N1Lmf9dvWsLzWJvhi3JLs7Utg45gTkgsgAhWPI3KM6ye2v8L3Ei0hBqGiQn3lC9C+Xq3dkizj0Dpq1Rtj88vL6YKfG02y9x7cy0IDq6+Hv10MZVk5Io18SIPNOT036MLIhUIXB8pIF1kONWDKC5gdvvPi22SLzoK/MGR1dBNvd7qDp6QtE3jatk+p+2vhjFvVR8m1wIldD0Bgu2nTA== X-Forefront-Antispam-Report: CIP:216.228.112.34; CTRY:US; LANG:en; SCL:1; SRV:; IPV:NLI; SFV:NSPM; H:mail.nvidia.com; PTR:schybrid03.nvidia.com; CAT:NONE; SFS:(4636009)(46966006)(36840700001)(508600001)(8936002)(4326008)(47076005)(82310400003)(336012)(55016002)(6286002)(86362001)(16526019)(7696005)(8676002)(70206006)(186003)(70586007)(316002)(36860700001)(5660300002)(1076003)(36756003)(2876002)(6916009)(7636003)(356005)(83380400001)(426003)(2906002)(2616005)(26005); DIR:OUT; SFP:1101; X-OriginatorOrg: Nvidia.com X-MS-Exchange-CrossTenant-OriginalArrivalTime: 08 Nov 2021 10:47:04.3958 (UTC) X-MS-Exchange-CrossTenant-Network-Message-Id: 50693c8f-c5fb-4652-cc7a-08d9a2a51a5a X-MS-Exchange-CrossTenant-Id: 43083d15-7273-40c1-b7db-39efd9ccc17a X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=43083d15-7273-40c1-b7db-39efd9ccc17a; Ip=[216.228.112.34]; Helo=[mail.nvidia.com] X-MS-Exchange-CrossTenant-AuthSource: CO1NAM11FT064.eop-nam11.prod.protection.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Anonymous X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem X-MS-Exchange-Transport-CrossTenantHeadersStamped: BY5PR12MB3874 Subject: [dpdk-dev] [PATCH v5 3/9] gpudev: add child device representing a device context X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Sender: "dev" From: Thomas Monjalon The computing device may operate in some isolated contexts. Memory and processing are isolated in a silo represented by a child device. The context is provided as an opaque by the caller of rte_gpu_add_child(). Signed-off-by: Thomas Monjalon --- doc/guides/prog_guide/gpudev.rst | 12 ++++++ lib/gpudev/gpudev.c | 45 +++++++++++++++++++- lib/gpudev/gpudev_driver.h | 2 +- lib/gpudev/rte_gpudev.h | 71 +++++++++++++++++++++++++++++--- lib/gpudev/version.map | 1 + 5 files changed, 123 insertions(+), 8 deletions(-) diff --git a/doc/guides/prog_guide/gpudev.rst b/doc/guides/prog_guide/gpudev.rst index 6ea7239159..7694639489 100644 --- a/doc/guides/prog_guide/gpudev.rst +++ b/doc/guides/prog_guide/gpudev.rst @@ -34,3 +34,15 @@ This library provides a number of features: API Overview ------------ + +Child Device +~~~~~~~~~~~~ + +By default, DPDK PCIe module detects and registers physical GPU devices +in the system. +With the gpudev library is also possible to add additional non-physical devices +through an ``uint64_t`` generic handler (e.g. CUDA Driver context) +that will be registered internally by the driver as an additional device (child) +connected to a physical device (parent). +Each device (parent or child) is represented through a ID +required to indicate which device a given operation should be executed on. diff --git a/lib/gpudev/gpudev.c b/lib/gpudev/gpudev.c index 07572ae040..aaf41e6071 100644 --- a/lib/gpudev/gpudev.c +++ b/lib/gpudev/gpudev.c @@ -80,13 +80,22 @@ rte_gpu_is_valid(int16_t dev_id) return false; } +static bool +gpu_match_parent(int16_t dev_id, int16_t parent) +{ + if (parent == RTE_GPU_ID_ANY) + return true; + return gpus[dev_id].info.parent == parent; +} + int16_t -rte_gpu_find_next(int16_t dev_id) +rte_gpu_find_next(int16_t dev_id, int16_t parent) { if (dev_id < 0) dev_id = 0; while (dev_id < gpu_max && - gpus[dev_id].state == RTE_GPU_STATE_UNUSED) + (gpus[dev_id].state == RTE_GPU_STATE_UNUSED || + !gpu_match_parent(dev_id, parent))) dev_id++; if (dev_id >= gpu_max) @@ -177,6 +186,7 @@ rte_gpu_allocate(const char *name) dev->info.name = dev->name; dev->info.dev_id = dev_id; dev->info.numa_node = -1; + dev->info.parent = RTE_GPU_ID_NONE; TAILQ_INIT(&dev->callbacks); gpu_count++; @@ -185,6 +195,28 @@ rte_gpu_allocate(const char *name) return dev; } +int16_t +rte_gpu_add_child(const char *name, int16_t parent, uint64_t child_context) +{ + struct rte_gpu *dev; + + if (!rte_gpu_is_valid(parent)) { + GPU_LOG(ERR, "add child to invalid parent ID %d", parent); + rte_errno = ENODEV; + return -rte_errno; + } + + dev = rte_gpu_allocate(name); + if (dev == NULL) + return -rte_errno; + + dev->info.parent = parent; + dev->info.context = child_context; + + rte_gpu_complete_new(dev); + return dev->info.dev_id; +} + void rte_gpu_complete_new(struct rte_gpu *dev) { @@ -199,10 +231,19 @@ rte_gpu_complete_new(struct rte_gpu *dev) int rte_gpu_release(struct rte_gpu *dev) { + int16_t dev_id, child; + if (dev == NULL) { rte_errno = ENODEV; return -rte_errno; } + dev_id = dev->info.dev_id; + RTE_GPU_FOREACH_CHILD(child, dev_id) { + GPU_LOG(ERR, "cannot release device %d with child %d", + dev_id, child); + rte_errno = EBUSY; + return -rte_errno; + } GPU_LOG(DEBUG, "free device %s (id %d)", dev->info.name, dev->info.dev_id); diff --git a/lib/gpudev/gpudev_driver.h b/lib/gpudev/gpudev_driver.h index 2a7089aa52..4d0077161c 100644 --- a/lib/gpudev/gpudev_driver.h +++ b/lib/gpudev/gpudev_driver.h @@ -31,7 +31,7 @@ typedef int (rte_gpu_info_get_t)(struct rte_gpu *dev, struct rte_gpu_info *info) struct rte_gpu_ops { /* Get device info. If NULL, info is just copied. */ rte_gpu_info_get_t *dev_info_get; - /* Close device. */ + /* Close device or child context. */ rte_gpu_close_t *dev_close; }; diff --git a/lib/gpudev/rte_gpudev.h b/lib/gpudev/rte_gpudev.h index e1702fbfe4..df75dbdbab 100644 --- a/lib/gpudev/rte_gpudev.h +++ b/lib/gpudev/rte_gpudev.h @@ -41,8 +41,12 @@ extern "C" { struct rte_gpu_info { /** Unique identifier name. */ const char *name; + /** Opaque handler of the device context. */ + uint64_t context; /** Device ID. */ int16_t dev_id; + /** ID of the parent device, RTE_GPU_ID_NONE if no parent */ + int16_t parent; /** Total processors available on device. */ uint32_t processor_count; /** Total memory available on device. */ @@ -110,6 +114,33 @@ uint16_t rte_gpu_count_avail(void); __rte_experimental bool rte_gpu_is_valid(int16_t dev_id); +/** + * @warning + * @b EXPERIMENTAL: this API may change without prior notice. + * + * Create a virtual device representing a context in the parent device. + * + * @param name + * Unique string to identify the device. + * @param parent + * Device ID of the parent. + * @param child_context + * Opaque context handler. + * + * @return + * Device ID of the new created child, -rte_errno otherwise: + * - EINVAL if empty name + * - ENAMETOOLONG if long name + * - EEXIST if existing device name + * - ENODEV if invalid parent + * - EPERM if secondary process + * - ENOENT if too many devices + * - ENOMEM if out of space + */ +__rte_experimental +int16_t rte_gpu_add_child(const char *name, + int16_t parent, uint64_t child_context); + /** * @warning * @b EXPERIMENTAL: this API may change without prior notice. @@ -118,13 +149,17 @@ bool rte_gpu_is_valid(int16_t dev_id); * * @param dev_id * The initial device ID to start the research. + * @param parent + * The device ID of the parent. + * RTE_GPU_ID_NONE means no parent. + * RTE_GPU_ID_ANY means no or any parent. * * @return * Next device ID corresponding to a valid and initialized computing device, * RTE_GPU_ID_NONE if there is none. */ __rte_experimental -int16_t rte_gpu_find_next(int16_t dev_id); +int16_t rte_gpu_find_next(int16_t dev_id, int16_t parent); /** * @warning @@ -136,15 +171,41 @@ int16_t rte_gpu_find_next(int16_t dev_id); * The ID of the next possible valid device, usually 0 to iterate all. */ #define RTE_GPU_FOREACH(dev_id) \ - for (dev_id = rte_gpu_find_next(0); \ - dev_id > 0; \ - dev_id = rte_gpu_find_next(dev_id + 1)) + RTE_GPU_FOREACH_CHILD(dev_id, RTE_GPU_ID_ANY) + +/** + * @warning + * @b EXPERIMENTAL: this API may change without prior notice. + * + * Macro to iterate over all valid computing devices having no parent. + * + * @param dev_id + * The ID of the next possible valid device, usually 0 to iterate all. + */ +#define RTE_GPU_FOREACH_PARENT(dev_id) \ + RTE_GPU_FOREACH_CHILD(dev_id, RTE_GPU_ID_NONE) + +/** + * @warning + * @b EXPERIMENTAL: this API may change without prior notice. + * + * Macro to iterate over all valid children of a computing device parent. + * + * @param dev_id + * The ID of the next possible valid device, usually 0 to iterate all. + * @param parent + * The device ID of the parent. + */ +#define RTE_GPU_FOREACH_CHILD(dev_id, parent) \ + for (dev_id = rte_gpu_find_next(0, parent); \ + dev_id >= 0; \ + dev_id = rte_gpu_find_next(dev_id + 1, parent)) /** * @warning * @b EXPERIMENTAL: this API may change without prior notice. * - * Close device. + * Close device or child context. * All resources are released. * * @param dev_id diff --git a/lib/gpudev/version.map b/lib/gpudev/version.map index b3b6b76c1c..4a934ed933 100644 --- a/lib/gpudev/version.map +++ b/lib/gpudev/version.map @@ -2,6 +2,7 @@ EXPERIMENTAL { global: # added in 21.11 + rte_gpu_add_child; rte_gpu_callback_register; rte_gpu_callback_unregister; rte_gpu_close; From patchwork Mon Nov 8 18:58:00 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Elena Agostini X-Patchwork-Id: 103974 X-Patchwork-Delegate: thomas@monjalon.net Return-Path: X-Original-To: patchwork@inbox.dpdk.org Delivered-To: patchwork@inbox.dpdk.org Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id 9E16CA0C4D; Mon, 8 Nov 2021 11:47:41 +0100 (CET) Received: from [217.70.189.124] (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id 4EE1B41134; Mon, 8 Nov 2021 11:47:11 +0100 (CET) Received: from NAM12-MW2-obe.outbound.protection.outlook.com (mail-mw2nam12on2045.outbound.protection.outlook.com [40.107.244.45]) by mails.dpdk.org (Postfix) with ESMTP id 22EDB410FE for ; Mon, 8 Nov 2021 11:47:08 +0100 (CET) ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=CFyJUpGykiKknlJHj3VIoXin5kEuPopVyxHxL5Oz2bXMC14NA0dxwgyqXcX4qFcDh89v3h+hZuq+zZiccBqWyarjLqjw2wYqOH9oX2Dor6QTsu3xje7UDGVlPfKLBPXl5DojI+ywcWf5xLZS2LinD3Qeer40qnq2hUL6+JB9K+TSltDuVaTVL4IaDd55IRqjp9VtuUohbEjTdLu40VUVS0+l89AIgqgs8igJjEyWwBJVJaYz5gZBFU49FjhLAxy5Cr4wYt7HUF0PHJeQ2KZXpoSg9fWhRpnINuksmcH50HQMtZXzfYQ6oQMfTuBKnGEP7Yb0ByJaiDDu5bcfhbZaKA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=1utuGE3qDCyuDfeI8PvdiruktOTYEdv9GIs8dNYtH60=; b=Sj3HH9ijbHYqd+lEqKP2S+lmC+Iipq2/hVkyFDFjZZ+uOG2ok1Jeet+jOOI/Pzm/PPO0O5a9YsLgxj9lYjHsFLeIb8uLQ7FAvRz/1Xy+/73+FdVLyNiRKlCMNqkjANpZ3P3Rt7sFo3NhmYYox/qEVpjLkpGghol0F/3ptadi9NWrlnhIhX//xq+IMEeqX2PFLBSuQVSYhaL4ODjQrRWDtPtLRBgsEomazGia4H2Hipg0nCYutAi5UKbDalJcvO/Xf98yhZv+JHocj6niHsMUTRk7VVjURWuKpg5fusMHuzLSI+b+sBtpgPHRHLdwXN7knixKai7yr0xCPGGlWWiylQ== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass (sender ip is 216.228.112.34) smtp.rcpttodomain=monjalon.net smtp.mailfrom=nvidia.com; dmarc=pass (p=quarantine sp=quarantine pct=100) action=none header.from=nvidia.com; dkim=none (message not signed); arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=Nvidia.com; s=selector2; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=1utuGE3qDCyuDfeI8PvdiruktOTYEdv9GIs8dNYtH60=; b=Uqc/eJ2XNe6lsmbqq1a4TmWCs/SrMIbZPq/H8VMNqnvla/SQx1/1Q7Qsfeo/ostYeDyzzXdB82WbKUp/Gs1+y5oMdrzUB0USNr4WsXxX/d4Qj5bq8DWoXaUULPAY0fHlAvuv6nXUJTFQ3HsUsJAxxywG4G4piRgW7+Z3qonqI/pOoHCF0w+VaZBuvEXvQfIFctuQCJa4ltewaUXovzWfUhvsK40omGhqGmsL4q/A75xaEedY7wwFVv2Us44pwOqvDb7nFxiswNAeukECZ7TMgDcKnnfbq9vL2xti44XEsLRA824NqnzUYwKfopR6g/L9AVdPJS728L/fUPetIE58eg== Received: from MW2PR16CA0042.namprd16.prod.outlook.com (2603:10b6:907:1::19) by CY4PR12MB1848.namprd12.prod.outlook.com (2603:10b6:903:11d::23) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4669.11; Mon, 8 Nov 2021 10:47:05 +0000 Received: from CO1NAM11FT064.eop-nam11.prod.protection.outlook.com (2603:10b6:907:1:cafe::eb) by MW2PR16CA0042.outlook.office365.com (2603:10b6:907:1::19) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4669.10 via Frontend Transport; Mon, 8 Nov 2021 10:47:05 +0000 X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 216.228.112.34) smtp.mailfrom=nvidia.com; monjalon.net; dkim=none (message not signed) header.d=none;monjalon.net; dmarc=pass action=none header.from=nvidia.com; Received-SPF: Pass (protection.outlook.com: domain of nvidia.com designates 216.228.112.34 as permitted sender) receiver=protection.outlook.com; client-ip=216.228.112.34; helo=mail.nvidia.com; Received: from mail.nvidia.com (216.228.112.34) by CO1NAM11FT064.mail.protection.outlook.com (10.13.175.77) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA384) id 15.20.4669.10 via Frontend Transport; Mon, 8 Nov 2021 10:47:05 +0000 Received: from nvidia.com (172.20.187.6) by HQMAIL107.nvidia.com (172.20.187.13) with Microsoft SMTP Server (TLS) id 15.0.1497.18; Mon, 8 Nov 2021 10:47:01 +0000 From: To: CC: Thomas Monjalon Date: Mon, 8 Nov 2021 18:58:00 +0000 Message-ID: <20211108185805.3887-5-eagostini@nvidia.com> X-Mailer: git-send-email 2.17.1 In-Reply-To: <20211108185805.3887-1-eagostini@nvidia.com> References: <20210602203531.2288645-1-thomas@monjalon.net> <20211108185805.3887-1-eagostini@nvidia.com> MIME-Version: 1.0 X-Originating-IP: [172.20.187.6] X-ClientProxiedBy: HQMAIL105.nvidia.com (172.20.187.12) To HQMAIL107.nvidia.com (172.20.187.13) X-EOPAttributedMessage: 0 X-MS-PublicTrafficType: Email X-MS-Office365-Filtering-Correlation-Id: d7f889ca-bb10-4d49-3cad-08d9a2a51ae0 X-MS-TrafficTypeDiagnostic: CY4PR12MB1848: X-LD-Processed: 43083d15-7273-40c1-b7db-39efd9ccc17a,ExtAddr X-Microsoft-Antispam-PRVS: X-MS-Oob-TLC-OOBClassifiers: OLM:2043; X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: ucAkYmN4deypUETUkkPteQU6RasIFnvj0nrZEkpPlletkQ2n53wjhfx6Y5UnD9Y3ttmQqjI7xvKKlB/89vy+e3Ht4T20FjlhZFq3pHRgiS1mF90K0+rn1xnMFgxi451EnFGuexbI2gyLn2vRvn0lvrLf/iVxNucspLZJihUf+954d071rHFkU7v/J+tBtPJo3Mu5hc+EH+3c9xbnlHKdYV9oiwSBBUr3RT5xiyTQWYedPMEgCIXkqO1Q02MVQzYeEKQvT4fG//qZzJ6snzWJXND0vsOz3T0+GyqZfV7E/27mT4LqdKhCOF3lLeGh81eBxR0dQS8X1hnJ5UTwDRjoP4YQRFKJLleVk6Od3yT55/gZk+t6TMwGq8u2pB/S6cPSZIDuB2CYRR/yycbCIRXjRLK3VbLCIKgJjyQS4tWkM6nlPyMtND5q0W1MCk9bQiP5PTooSSwIs+iiTWtRHQJ78GVMZbgkIjQqnuqIyzCEuUtlSMh1T+/96oTDRxPpSCjgWB0Ajf5cAPfusM9KWxUeS1SXsIwdTRLn26GxFqxLN8nQ6EigzWe2SxqFYv9mvrxC5cTRfAaisVZyd5GgSuyKuU4sdGVtptKGOhzBxJqhnigFvEuME7J/960utds3Yz0EmGPfnR3v67MYLkJFt9G2R1azvWTToNv275c4HDzPulhyuYWsq+jp9F8KsuwyHrn1kdNX50G8Oxv8GwXSQGgPBA== X-Forefront-Antispam-Report: CIP:216.228.112.34; CTRY:US; LANG:en; SCL:1; SRV:; IPV:NLI; SFV:NSPM; H:mail.nvidia.com; PTR:schybrid03.nvidia.com; CAT:NONE; SFS:(4636009)(46966006)(36840700001)(7696005)(36860700001)(336012)(55016002)(86362001)(8936002)(16526019)(186003)(2616005)(2876002)(6916009)(36756003)(82310400003)(26005)(316002)(426003)(30864003)(70586007)(70206006)(508600001)(83380400001)(47076005)(8676002)(7636003)(356005)(1076003)(6286002)(2906002)(4326008)(5660300002); DIR:OUT; SFP:1101; X-OriginatorOrg: Nvidia.com X-MS-Exchange-CrossTenant-OriginalArrivalTime: 08 Nov 2021 10:47:05.2373 (UTC) X-MS-Exchange-CrossTenant-Network-Message-Id: d7f889ca-bb10-4d49-3cad-08d9a2a51ae0 X-MS-Exchange-CrossTenant-Id: 43083d15-7273-40c1-b7db-39efd9ccc17a X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=43083d15-7273-40c1-b7db-39efd9ccc17a; Ip=[216.228.112.34]; Helo=[mail.nvidia.com] X-MS-Exchange-CrossTenant-AuthSource: CO1NAM11FT064.eop-nam11.prod.protection.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Anonymous X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem X-MS-Exchange-Transport-CrossTenantHeadersStamped: CY4PR12MB1848 Subject: [dpdk-dev] [PATCH v5 4/9] gpudev: support multi-process X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Sender: "dev" From: Thomas Monjalon The device data shared between processes are moved in a struct allocated in a shared memory (a new memzone for all GPUs). The main struct rte_gpu references the shared memory via the pointer mpshared. The API function rte_gpu_attach() is added to attach a device from the secondary process. The function rte_gpu_allocate() can be used only by primary process. Signed-off-by: Thomas Monjalon --- lib/gpudev/gpudev.c | 127 +++++++++++++++++++++++++++++++------ lib/gpudev/gpudev_driver.h | 25 ++++++-- lib/gpudev/version.map | 1 + 3 files changed, 127 insertions(+), 26 deletions(-) diff --git a/lib/gpudev/gpudev.c b/lib/gpudev/gpudev.c index aaf41e6071..17e371102a 100644 --- a/lib/gpudev/gpudev.c +++ b/lib/gpudev/gpudev.c @@ -5,6 +5,7 @@ #include #include #include +#include #include #include @@ -28,6 +29,12 @@ static int16_t gpu_max; /* Number of currently valid devices */ static int16_t gpu_count; +/* Shared memory between processes. */ +static const char *GPU_MEMZONE = "rte_gpu_shared"; +static struct { + __extension__ struct rte_gpu_mpshared gpus[0]; +} *gpu_shared_mem; + /* Event callback object */ struct rte_gpu_callback { TAILQ_ENTRY(rte_gpu_callback) next; @@ -75,7 +82,7 @@ bool rte_gpu_is_valid(int16_t dev_id) { if (dev_id >= 0 && dev_id < gpu_max && - gpus[dev_id].state == RTE_GPU_STATE_INITIALIZED) + gpus[dev_id].process_state == RTE_GPU_STATE_INITIALIZED) return true; return false; } @@ -85,7 +92,7 @@ gpu_match_parent(int16_t dev_id, int16_t parent) { if (parent == RTE_GPU_ID_ANY) return true; - return gpus[dev_id].info.parent == parent; + return gpus[dev_id].mpshared->info.parent == parent; } int16_t @@ -94,7 +101,7 @@ rte_gpu_find_next(int16_t dev_id, int16_t parent) if (dev_id < 0) dev_id = 0; while (dev_id < gpu_max && - (gpus[dev_id].state == RTE_GPU_STATE_UNUSED || + (gpus[dev_id].process_state == RTE_GPU_STATE_UNUSED || !gpu_match_parent(dev_id, parent))) dev_id++; @@ -109,7 +116,7 @@ gpu_find_free_id(void) int16_t dev_id; for (dev_id = 0; dev_id < gpu_max; dev_id++) { - if (gpus[dev_id].state == RTE_GPU_STATE_UNUSED) + if (gpus[dev_id].process_state == RTE_GPU_STATE_UNUSED) return dev_id; } return RTE_GPU_ID_NONE; @@ -136,12 +143,35 @@ rte_gpu_get_by_name(const char *name) RTE_GPU_FOREACH(dev_id) { dev = &gpus[dev_id]; - if (strncmp(name, dev->name, RTE_DEV_NAME_MAX_LEN) == 0) + if (strncmp(name, dev->mpshared->name, RTE_DEV_NAME_MAX_LEN) == 0) return dev; } return NULL; } +static int +gpu_shared_mem_init(void) +{ + const struct rte_memzone *memzone; + + if (rte_eal_process_type() == RTE_PROC_PRIMARY) { + memzone = rte_memzone_reserve(GPU_MEMZONE, + sizeof(*gpu_shared_mem) + + sizeof(*gpu_shared_mem->gpus) * gpu_max, + SOCKET_ID_ANY, 0); + } else { + memzone = rte_memzone_lookup(GPU_MEMZONE); + } + if (memzone == NULL) { + GPU_LOG(ERR, "cannot initialize shared memory"); + rte_errno = ENOMEM; + return -rte_errno; + } + + gpu_shared_mem = memzone->addr; + return 0; +} + struct rte_gpu * rte_gpu_allocate(const char *name) { @@ -163,6 +193,10 @@ rte_gpu_allocate(const char *name) if (gpus == NULL && rte_gpu_init(RTE_GPU_DEFAULT_MAX) < 0) return NULL; + /* initialize shared memory before adding first device */ + if (gpu_shared_mem == NULL && gpu_shared_mem_init() < 0) + return NULL; + if (rte_gpu_get_by_name(name) != NULL) { GPU_LOG(ERR, "device with name %s already exists", name); rte_errno = EEXIST; @@ -178,16 +212,20 @@ rte_gpu_allocate(const char *name) dev = &gpus[dev_id]; memset(dev, 0, sizeof(*dev)); - if (rte_strscpy(dev->name, name, RTE_DEV_NAME_MAX_LEN) < 0) { + dev->mpshared = &gpu_shared_mem->gpus[dev_id]; + memset(dev->mpshared, 0, sizeof(*dev->mpshared)); + + if (rte_strscpy(dev->mpshared->name, name, RTE_DEV_NAME_MAX_LEN) < 0) { GPU_LOG(ERR, "device name too long: %s", name); rte_errno = ENAMETOOLONG; return NULL; } - dev->info.name = dev->name; - dev->info.dev_id = dev_id; - dev->info.numa_node = -1; - dev->info.parent = RTE_GPU_ID_NONE; + dev->mpshared->info.name = dev->mpshared->name; + dev->mpshared->info.dev_id = dev_id; + dev->mpshared->info.numa_node = -1; + dev->mpshared->info.parent = RTE_GPU_ID_NONE; TAILQ_INIT(&dev->callbacks); + __atomic_fetch_add(&dev->mpshared->process_refcnt, 1, __ATOMIC_RELAXED); gpu_count++; GPU_LOG(DEBUG, "new device %s (id %d) of total %d", @@ -195,6 +233,55 @@ rte_gpu_allocate(const char *name) return dev; } +struct rte_gpu * +rte_gpu_attach(const char *name) +{ + int16_t dev_id; + struct rte_gpu *dev; + struct rte_gpu_mpshared *shared_dev; + + if (rte_eal_process_type() != RTE_PROC_SECONDARY) { + GPU_LOG(ERR, "only secondary process can attach device"); + rte_errno = EPERM; + return NULL; + } + if (name == NULL) { + GPU_LOG(ERR, "attach device without a name"); + rte_errno = EINVAL; + return NULL; + } + + /* implicit initialization of library before adding first device */ + if (gpus == NULL && rte_gpu_init(RTE_GPU_DEFAULT_MAX) < 0) + return NULL; + + /* initialize shared memory before adding first device */ + if (gpu_shared_mem == NULL && gpu_shared_mem_init() < 0) + return NULL; + + for (dev_id = 0; dev_id < gpu_max; dev_id++) { + shared_dev = &gpu_shared_mem->gpus[dev_id]; + if (strncmp(name, shared_dev->name, RTE_DEV_NAME_MAX_LEN) == 0) + break; + } + if (dev_id >= gpu_max) { + GPU_LOG(ERR, "device with name %s not found", name); + rte_errno = ENOENT; + return NULL; + } + dev = &gpus[dev_id]; + memset(dev, 0, sizeof(*dev)); + + TAILQ_INIT(&dev->callbacks); + dev->mpshared = shared_dev; + __atomic_fetch_add(&dev->mpshared->process_refcnt, 1, __ATOMIC_RELAXED); + + gpu_count++; + GPU_LOG(DEBUG, "attached device %s (id %d) of total %d", + name, dev_id, gpu_count); + return dev; +} + int16_t rte_gpu_add_child(const char *name, int16_t parent, uint64_t child_context) { @@ -210,11 +297,11 @@ rte_gpu_add_child(const char *name, int16_t parent, uint64_t child_context) if (dev == NULL) return -rte_errno; - dev->info.parent = parent; - dev->info.context = child_context; + dev->mpshared->info.parent = parent; + dev->mpshared->info.context = child_context; rte_gpu_complete_new(dev); - return dev->info.dev_id; + return dev->mpshared->info.dev_id; } void @@ -223,8 +310,7 @@ rte_gpu_complete_new(struct rte_gpu *dev) if (dev == NULL) return; - dev->state = RTE_GPU_STATE_INITIALIZED; - dev->state = RTE_GPU_STATE_INITIALIZED; + dev->process_state = RTE_GPU_STATE_INITIALIZED; rte_gpu_notify(dev, RTE_GPU_EVENT_NEW); } @@ -237,7 +323,7 @@ rte_gpu_release(struct rte_gpu *dev) rte_errno = ENODEV; return -rte_errno; } - dev_id = dev->info.dev_id; + dev_id = dev->mpshared->info.dev_id; RTE_GPU_FOREACH_CHILD(child, dev_id) { GPU_LOG(ERR, "cannot release device %d with child %d", dev_id, child); @@ -246,11 +332,12 @@ rte_gpu_release(struct rte_gpu *dev) } GPU_LOG(DEBUG, "free device %s (id %d)", - dev->info.name, dev->info.dev_id); + dev->mpshared->info.name, dev->mpshared->info.dev_id); rte_gpu_notify(dev, RTE_GPU_EVENT_DEL); gpu_free_callbacks(dev); - dev->state = RTE_GPU_STATE_UNUSED; + dev->process_state = RTE_GPU_STATE_UNUSED; + __atomic_fetch_sub(&dev->mpshared->process_refcnt, 1, __ATOMIC_RELAXED); gpu_count--; return 0; @@ -403,7 +490,7 @@ rte_gpu_notify(struct rte_gpu *dev, enum rte_gpu_event event) int16_t dev_id; struct rte_gpu_callback *callback; - dev_id = dev->info.dev_id; + dev_id = dev->mpshared->info.dev_id; rte_rwlock_read_lock(&gpu_callback_lock); TAILQ_FOREACH(callback, &dev->callbacks, next) { if (callback->event != event || callback->function == NULL) @@ -431,7 +518,7 @@ rte_gpu_info_get(int16_t dev_id, struct rte_gpu_info *info) } if (dev->ops.dev_info_get == NULL) { - *info = dev->info; + *info = dev->mpshared->info; return 0; } return GPU_DRV_RET(dev->ops.dev_info_get(dev, info)); diff --git a/lib/gpudev/gpudev_driver.h b/lib/gpudev/gpudev_driver.h index 4d0077161c..9459c7e30f 100644 --- a/lib/gpudev/gpudev_driver.h +++ b/lib/gpudev/gpudev_driver.h @@ -35,19 +35,28 @@ struct rte_gpu_ops { rte_gpu_close_t *dev_close; }; -struct rte_gpu { - /* Backing device. */ - struct rte_device *device; +struct rte_gpu_mpshared { /* Unique identifier name. */ char name[RTE_DEV_NAME_MAX_LEN]; /* Updated by this library. */ + /* Driver-specific private data shared in multi-process. */ + void *dev_private; /* Device info structure. */ struct rte_gpu_info info; + /* Counter of processes using the device. */ + uint16_t process_refcnt; /* Updated by this library. */ +}; + +struct rte_gpu { + /* Backing device. */ + struct rte_device *device; + /* Data shared between processes. */ + struct rte_gpu_mpshared *mpshared; /* Driver functions. */ struct rte_gpu_ops ops; /* Event callback list. */ TAILQ_HEAD(rte_gpu_callback_list, rte_gpu_callback) callbacks; /* Current state (used or not) in the running process. */ - enum rte_gpu_state state; /* Updated by this library. */ + enum rte_gpu_state process_state; /* Updated by this library. */ /* Driver-specific private data for the running process. */ void *process_private; } __rte_cache_aligned; @@ -55,15 +64,19 @@ struct rte_gpu { __rte_internal struct rte_gpu *rte_gpu_get_by_name(const char *name); -/* First step of initialization */ +/* First step of initialization in primary process. */ __rte_internal struct rte_gpu *rte_gpu_allocate(const char *name); +/* First step of initialization in secondary process. */ +__rte_internal +struct rte_gpu *rte_gpu_attach(const char *name); + /* Last step of initialization. */ __rte_internal void rte_gpu_complete_new(struct rte_gpu *dev); -/* Last step of removal. */ +/* Last step of removal (primary or secondary process). */ __rte_internal int rte_gpu_release(struct rte_gpu *dev); diff --git a/lib/gpudev/version.map b/lib/gpudev/version.map index 4a934ed933..58dc632393 100644 --- a/lib/gpudev/version.map +++ b/lib/gpudev/version.map @@ -17,6 +17,7 @@ INTERNAL { global: rte_gpu_allocate; + rte_gpu_attach; rte_gpu_complete_new; rte_gpu_get_by_name; rte_gpu_notify; From patchwork Mon Nov 8 18:58:01 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Elena Agostini X-Patchwork-Id: 103975 X-Patchwork-Delegate: thomas@monjalon.net Return-Path: X-Original-To: patchwork@inbox.dpdk.org Delivered-To: patchwork@inbox.dpdk.org Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id EBD02A0C4D; Mon, 8 Nov 2021 11:47:48 +0100 (CET) Received: from [217.70.189.124] (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id 60EE84114D; Mon, 8 Nov 2021 11:47:12 +0100 (CET) Received: from NAM12-DM6-obe.outbound.protection.outlook.com (mail-dm6nam12on2040.outbound.protection.outlook.com [40.107.243.40]) by mails.dpdk.org (Postfix) with ESMTP id 0A78941125 for ; Mon, 8 Nov 2021 11:47:09 +0100 (CET) ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=Lj8qvM+IpktnzMQNKc15NwnvmLzbVKAPF2SN2qLIBc5G4XwLfWy8twF+d4iumY1W2xp7SRJB9dt6qlODcjOY6g27kfSEGWBJJZH/5mZoEYfRlzWv572k4yyNJJ6wQ875xuHLRdDGPxNl/rmqj4Tr8z6yBFbuJiuqlptCPUDr1a58uEBxIkm4oBwG56gP3kNwt4Sns5Rf1t2AhXPziPYkJLKsU/Y4VnVW5VsYeLOFEnA5014VB2yhuhOeQD30sR8f4wHxjkusODH6fZrtWrt9Yr2XbAAn4wJVn6jWD1tmZdJMQoixbCuOFSWCmPQq9br3pMHMWHAgGvoF8til8pVxhQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=o7CZcWUXuFO050dDNOZ4NSFEwiBgM5BtcEzCLkvL52s=; b=V+bUTi/fQYylpvteGcCbe122qfM9UjN1z3jIYhFJvx1Wc9Hz0JrYwVvV11H7jBw6zYg8AgJrX9HuO+QmTFoCxVrIrEnbYQNQ6L/XEQ51DgUQj2kPNMyRl1od5zAopqqomQxQTYMSpv6pMYegkXTSAR0IbzLLT453jDGTsTPliSqaQ9xtWXgqGrW4yUW6xtvcDuSHmhMC6RJNPIv+//EYmvGud/U35v38M04yJs15fb876KSvx/nOoKE3w4gvgdrwlwslqjKXknUgIVu7niT8xTv6WIxR46TRJU73+eeujyJZDCK/W/IxuXfW397WF6UEelLFEpRHDO4XkyV+/ZcsbQ== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass (sender ip is 216.228.112.34) smtp.rcpttodomain=monjalon.net smtp.mailfrom=nvidia.com; dmarc=pass (p=quarantine sp=quarantine pct=100) action=none header.from=nvidia.com; dkim=none (message not signed); arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=Nvidia.com; s=selector2; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=o7CZcWUXuFO050dDNOZ4NSFEwiBgM5BtcEzCLkvL52s=; b=ifmTnIebct1AXu39jkSR9zQd4QahAmMAKkKjwIzV2KiCc7jJYkZbwiP5r7Qq9RAPwdyc7ev9h0fFyvDOv0jw+dMYRZkOi97D9UxRXq+0lC21rLVBQOuobuhQ2SIda9zOcFOW9mABtDlGa1BGQyfbi3OXZqBlWsfg9J6C/9Y5AUUUEPRnVQ5TNHDfLe0g2FZD2fvc7qASk0tNHwLTYJ4F4210a7L8cjtOtN1Q2l0Fr30kw69WJpV+YBXJ7Lxiqv9rU3nwjz5EecZuhIPHVYiCgtp2gq/kDyVPHYvoerrOcgyzvLgft+aqlGJBHalgoumgfUzG8g+r0btcHlBcw/fGcw== Received: from MW2PR16CA0052.namprd16.prod.outlook.com (2603:10b6:907:1::29) by MN2PR12MB4286.namprd12.prod.outlook.com (2603:10b6:208:199::22) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4669.11; Mon, 8 Nov 2021 10:47:07 +0000 Received: from CO1NAM11FT064.eop-nam11.prod.protection.outlook.com (2603:10b6:907:1:cafe::64) by MW2PR16CA0052.outlook.office365.com (2603:10b6:907:1::29) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4669.10 via Frontend Transport; Mon, 8 Nov 2021 10:47:06 +0000 X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 216.228.112.34) smtp.mailfrom=nvidia.com; monjalon.net; dkim=none (message not signed) header.d=none;monjalon.net; dmarc=pass action=none header.from=nvidia.com; Received-SPF: Pass (protection.outlook.com: domain of nvidia.com designates 216.228.112.34 as permitted sender) receiver=protection.outlook.com; client-ip=216.228.112.34; helo=mail.nvidia.com; Received: from mail.nvidia.com (216.228.112.34) by CO1NAM11FT064.mail.protection.outlook.com (10.13.175.77) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA384) id 15.20.4669.10 via Frontend Transport; Mon, 8 Nov 2021 10:47:06 +0000 Received: from nvidia.com (172.20.187.6) by HQMAIL107.nvidia.com (172.20.187.13) with Microsoft SMTP Server (TLS) id 15.0.1497.18; Mon, 8 Nov 2021 10:47:01 +0000 From: To: CC: Elena Agostini , Thomas Monjalon Date: Mon, 8 Nov 2021 18:58:01 +0000 Message-ID: <20211108185805.3887-6-eagostini@nvidia.com> X-Mailer: git-send-email 2.17.1 In-Reply-To: <20211108185805.3887-1-eagostini@nvidia.com> References: <20210602203531.2288645-1-thomas@monjalon.net> <20211108185805.3887-1-eagostini@nvidia.com> MIME-Version: 1.0 X-Originating-IP: [172.20.187.6] X-ClientProxiedBy: HQMAIL105.nvidia.com (172.20.187.12) To HQMAIL107.nvidia.com (172.20.187.13) X-EOPAttributedMessage: 0 X-MS-PublicTrafficType: Email X-MS-Office365-Filtering-Correlation-Id: 60a9f966-050c-4119-6f76-08d9a2a51b6f X-MS-TrafficTypeDiagnostic: MN2PR12MB4286: X-LD-Processed: 43083d15-7273-40c1-b7db-39efd9ccc17a,ExtAddr X-Microsoft-Antispam-PRVS: X-MS-Oob-TLC-OOBClassifiers: OLM:7691; X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: lTPpLI1gzpeNQpu8d3ukFqIm3rUXq2MAuhys0DoxKpXFhQz1K6wz03v1D64V+hjw7FkM2MaUWJGqVLq6qPuKOSQO4OIm+1vUpVXLmZ8CcSA/zx/aLazD+enU3fWer/1vP56kAhNUm/+UhMj3FsWCapKwC2T8BF3PA+fEAyxnyWaay85n034YlvM6YO1emxLBMDvXwp/I6T+Vd8qg4fpbpXibb+u9qDH4oPthjM7CQgMZshiHOrc9k/U5T6wuXKVJQbKHE4x6BGTy2QJGhFCmMA8PiNGxsSCwVF0j6Uu4WsNK3g4fUjEM8sCc2vIk4K/0Y+JbC/jXiEH+GiBXBG43zbk8QHWstZvqjk2yFaq/XsEQcbrsCHLMTlvyFBoxN/ONkRPRQQ4RE99DrJATeDBB+K2+I9hMDZTv29RV+Ah5R4MiI9jiK3Ac/y20a2Nn30eMiCzspGik/AN64JywOtDobW950hI4vDnR9oCZByNe1psPJHIglzGoDOo3ltV8N4hDcpHTZI/0gVAqlDr++N5BZtUbgMYiEe+n2ZGtLfcZ1blvr2vh+J846/M41uKiC2731ZGGQJ+HYmCcJ0yl7aNkovlQAoeT6iWi5iWmHXbUIuWG4qedUfVnUiPbN0QOXB9v+JNqET9DmmeeC/vPO4SqlQis13U1OTcfrDGznNSeTC0AaF1xEwKvvcoNWGvrAWel X-Forefront-Antispam-Report: CIP:216.228.112.34; CTRY:US; LANG:en; SCL:1; SRV:; IPV:NLI; SFV:NSPM; H:mail.nvidia.com; PTR:schybrid03.nvidia.com; CAT:NONE; SFS:(4636009)(46966006)(36840700001)(54906003)(2876002)(336012)(55016002)(86362001)(7696005)(36860700001)(16526019)(186003)(2616005)(8936002)(6916009)(426003)(36756003)(82310400003)(26005)(316002)(8676002)(70586007)(70206006)(30864003)(508600001)(83380400001)(47076005)(356005)(1076003)(7636003)(6286002)(2906002)(4326008)(5660300002); DIR:OUT; SFP:1101; X-OriginatorOrg: Nvidia.com X-MS-Exchange-CrossTenant-OriginalArrivalTime: 08 Nov 2021 10:47:06.2057 (UTC) X-MS-Exchange-CrossTenant-Network-Message-Id: 60a9f966-050c-4119-6f76-08d9a2a51b6f X-MS-Exchange-CrossTenant-Id: 43083d15-7273-40c1-b7db-39efd9ccc17a X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=43083d15-7273-40c1-b7db-39efd9ccc17a; Ip=[216.228.112.34]; Helo=[mail.nvidia.com] X-MS-Exchange-CrossTenant-AuthSource: CO1NAM11FT064.eop-nam11.prod.protection.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Anonymous X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem X-MS-Exchange-Transport-CrossTenantHeadersStamped: MN2PR12MB4286 Subject: [dpdk-dev] [PATCH v5 5/9] gpudev: add memory API X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Sender: "dev" From: Elena Agostini In heterogeneous computing system, processing is not only in the CPU. Some tasks can be delegated to devices working in parallel. Such workload distribution can be achieved by sharing some memory. As a first step, the features are focused on memory management. A function allows to allocate memory inside the device, or in the main (CPU) memory while making it visible for the device. This memory may be used to save packets or for synchronization data. The next step should focus on GPU processing task control. Signed-off-by: Elena Agostini Signed-off-by: Thomas Monjalon --- app/test-gpudev/main.c | 105 +++++++++++++++++++++++++ doc/guides/gpus/features/default.ini | 3 + doc/guides/prog_guide/gpudev.rst | 19 +++++ doc/guides/rel_notes/release_21_11.rst | 1 + lib/gpudev/gpudev.c | 101 ++++++++++++++++++++++++ lib/gpudev/gpudev_driver.h | 12 +++ lib/gpudev/rte_gpudev.h | 95 ++++++++++++++++++++++ lib/gpudev/version.map | 4 + 8 files changed, 340 insertions(+) diff --git a/app/test-gpudev/main.c b/app/test-gpudev/main.c index 438cfdac54..e3aca2225a 100644 --- a/app/test-gpudev/main.c +++ b/app/test-gpudev/main.c @@ -62,6 +62,98 @@ args_parse(int argc, char **argv) } } +static int +alloc_gpu_memory(uint16_t gpu_id) +{ + void *ptr_1 = NULL; + void *ptr_2 = NULL; + size_t buf_bytes = 1024; + int ret; + + printf("\n=======> TEST: Allocate GPU memory\n"); + + /* Alloc memory on GPU 0 */ + ptr_1 = rte_gpu_malloc(gpu_id, buf_bytes); + if (ptr_1 == NULL) { + fprintf(stderr, "rte_gpu_malloc GPU memory returned error\n"); + return -1; + } + printf("GPU memory allocated at 0x%p %zdB\n", ptr_1, buf_bytes); + + ptr_2 = rte_gpu_malloc(gpu_id, buf_bytes); + if (ptr_2 == NULL) { + fprintf(stderr, "rte_gpu_malloc GPU memory returned error\n"); + return -1; + } + printf("GPU memory allocated at 0x%p %zdB\n", ptr_2, buf_bytes); + + ret = rte_gpu_free(gpu_id, (uint8_t *)(ptr_1)+0x700); + if (ret < 0) { + printf("GPU memory 0x%p + 0x700 NOT freed because of memory address not recognized by driver\n", ptr_1); + } else { + fprintf(stderr, "rte_gpu_free erroneusly freed GPU memory 0x%p + 0x700\n", ptr_1); + return -1; + } + + ret = rte_gpu_free(gpu_id, ptr_2); + if (ret < 0) { + fprintf(stderr, "rte_gpu_free returned error %d\n", ret); + return -1; + } + printf("GPU memory 0x%p freed\n", ptr_2); + + ret = rte_gpu_free(gpu_id, ptr_1); + if (ret < 0) { + fprintf(stderr, "rte_gpu_free returned error %d\n", ret); + return -1; + } + printf("GPU memory 0x%p freed\n", ptr_1); + + return 0; +} + +static int +register_cpu_memory(uint16_t gpu_id) +{ + void *ptr = NULL; + size_t buf_bytes = 1024; + int ret; + + printf("\n=======> TEST: Register CPU memory\n"); + + /* Alloc memory on CPU visible from GPU 0 */ + ptr = rte_zmalloc(NULL, buf_bytes, 0); + if (ptr == NULL) { + fprintf(stderr, "Failed to allocate CPU memory.\n"); + return -1; + } + + ret = rte_gpu_register(gpu_id, buf_bytes, ptr); + if (ret < 0) { + fprintf(stderr, "rte_gpu_register CPU memory returned error %d\n", ret); + return -1; + } + printf("CPU memory registered at 0x%p %zdB\n", ptr, buf_bytes); + + ret = rte_gpu_unregister(gpu_id, (uint8_t *)(ptr)+0x700); + if (ret < 0) { + printf("CPU memory 0x%p + 0x700 NOT unregistered because of memory address not recognized by driver\n", ptr); + } else { + fprintf(stderr, "rte_gpu_free erroneusly freed GPU memory 0x%p + 0x700\n", ptr); + return -1; + } + printf("CPU memory 0x%p unregistered\n", ptr); + + ret = rte_gpu_unregister(gpu_id, ptr); + if (ret < 0) { + fprintf(stderr, "rte_gpu_unregister returned error %d\n", ret); + return -1; + } + printf("CPU memory 0x%p unregistered\n", ptr); + + return 0; +} + int main(int argc, char **argv) { @@ -99,6 +191,19 @@ main(int argc, char **argv) } printf("\n\n"); + if (nb_gpus == 0) { + fprintf(stderr, "Need at least one GPU on the system to run the example\n"); + return EXIT_FAILURE; + } + + gpu_id = 0; + + /** + * Memory tests + */ + alloc_gpu_memory(gpu_id); + register_cpu_memory(gpu_id); + /* clean up the EAL */ rte_eal_cleanup(); printf("Bye...\n"); diff --git a/doc/guides/gpus/features/default.ini b/doc/guides/gpus/features/default.ini index ec7a545eb7..87e9966424 100644 --- a/doc/guides/gpus/features/default.ini +++ b/doc/guides/gpus/features/default.ini @@ -8,3 +8,6 @@ ; [Features] Get device info = +Share CPU memory with device = +Allocate device memory = +Free memory = diff --git a/doc/guides/prog_guide/gpudev.rst b/doc/guides/prog_guide/gpudev.rst index 7694639489..9aca69038c 100644 --- a/doc/guides/prog_guide/gpudev.rst +++ b/doc/guides/prog_guide/gpudev.rst @@ -30,6 +30,8 @@ Features This library provides a number of features: - Interoperability with device-specific library through generic handlers. +- Allocate and free memory on the device. +- Register CPU memory to make it visible from the device. API Overview @@ -46,3 +48,20 @@ that will be registered internally by the driver as an additional device (child) connected to a physical device (parent). Each device (parent or child) is represented through a ID required to indicate which device a given operation should be executed on. + +Memory Allocation +~~~~~~~~~~~~~~~~~ + +gpudev can allocate on an input given GPU device a memory area +returning the pointer to that memory. +Later, it's also possible to free that memory with gpudev. +GPU memory allocated outside of the gpudev library +(e.g. with GPU-specific library) cannot be freed by the gpudev library. + +Memory Registration +~~~~~~~~~~~~~~~~~~~ + +gpudev can register a CPU memory area to make it visible from a GPU device. +Later, it's also possible to unregister that memory with gpudev. +CPU memory registered outside of the gpudev library +(e.g. with GPU specific library) cannot be unregistered by the gpudev library. diff --git a/doc/guides/rel_notes/release_21_11.rst b/doc/guides/rel_notes/release_21_11.rst index 9cf59e73bb..a4d07bda9b 100644 --- a/doc/guides/rel_notes/release_21_11.rst +++ b/doc/guides/rel_notes/release_21_11.rst @@ -104,6 +104,7 @@ New Features * **Introduced GPU device class with first features:** * Device information + * Memory management * **Added new RSS offload types for IPv4/L4 checksum in RSS flow.** diff --git a/lib/gpudev/gpudev.c b/lib/gpudev/gpudev.c index 17e371102a..d0826ec881 100644 --- a/lib/gpudev/gpudev.c +++ b/lib/gpudev/gpudev.c @@ -6,6 +6,7 @@ #include #include #include +#include #include #include @@ -523,3 +524,103 @@ rte_gpu_info_get(int16_t dev_id, struct rte_gpu_info *info) } return GPU_DRV_RET(dev->ops.dev_info_get(dev, info)); } + +void * +rte_gpu_malloc(int16_t dev_id, size_t size) +{ + struct rte_gpu *dev; + void *ptr; + int ret; + + dev = gpu_get_by_id(dev_id); + if (dev == NULL) { + GPU_LOG(ERR, "alloc mem for invalid device ID %d", dev_id); + rte_errno = ENODEV; + return NULL; + } + + if (dev->ops.mem_alloc == NULL) { + GPU_LOG(ERR, "mem allocation not supported"); + rte_errno = ENOTSUP; + return NULL; + } + + if (size == 0) /* dry-run */ + return NULL; + + ret = dev->ops.mem_alloc(dev, size, &ptr); + + switch (ret) { + case 0: + return ptr; + case -ENOMEM: + case -E2BIG: + rte_errno = -ret; + return NULL; + default: + rte_errno = -EPERM; + return NULL; + } +} + +int +rte_gpu_register(int16_t dev_id, size_t size, void *ptr) +{ + struct rte_gpu *dev; + + dev = gpu_get_by_id(dev_id); + if (dev == NULL) { + GPU_LOG(ERR, "alloc mem for invalid device ID %d", dev_id); + rte_errno = ENODEV; + return -rte_errno; + } + + if (dev->ops.mem_register == NULL) { + GPU_LOG(ERR, "mem registration not supported"); + rte_errno = ENOTSUP; + return -rte_errno; + } + + if (size == 0 || ptr == NULL) /* dry-run */ + return -EINVAL; + + return GPU_DRV_RET(dev->ops.mem_register(dev, size, ptr)); +} + +int +rte_gpu_unregister(int16_t dev_id, void *ptr) +{ + struct rte_gpu *dev; + + dev = gpu_get_by_id(dev_id); + if (dev == NULL) { + GPU_LOG(ERR, "unregister mem for invalid device ID %d", dev_id); + rte_errno = ENODEV; + return -rte_errno; + } + + if (dev->ops.mem_unregister == NULL) { + rte_errno = ENOTSUP; + return -rte_errno; + } + return GPU_DRV_RET(dev->ops.mem_unregister(dev, ptr)); +} + +int +rte_gpu_free(int16_t dev_id, void *ptr) +{ + struct rte_gpu *dev; + + dev = gpu_get_by_id(dev_id); + if (dev == NULL) { + GPU_LOG(ERR, "free mem for invalid device ID %d", dev_id); + rte_errno = ENODEV; + return -rte_errno; + } + + if (dev->ops.mem_free == NULL) { + rte_errno = ENOTSUP; + return -rte_errno; + } + return GPU_DRV_RET(dev->ops.mem_free(dev, ptr)); +} diff --git a/lib/gpudev/gpudev_driver.h b/lib/gpudev/gpudev_driver.h index 9459c7e30f..11015944a6 100644 --- a/lib/gpudev/gpudev_driver.h +++ b/lib/gpudev/gpudev_driver.h @@ -27,12 +27,24 @@ enum rte_gpu_state { struct rte_gpu; typedef int (rte_gpu_close_t)(struct rte_gpu *dev); typedef int (rte_gpu_info_get_t)(struct rte_gpu *dev, struct rte_gpu_info *info); +typedef int (rte_gpu_mem_alloc_t)(struct rte_gpu *dev, size_t size, void **ptr); +typedef int (rte_gpu_free_t)(struct rte_gpu *dev, void *ptr); +typedef int (rte_gpu_mem_register_t)(struct rte_gpu *dev, size_t size, void *ptr); +typedef int (rte_gpu_mem_unregister_t)(struct rte_gpu *dev, void *ptr); struct rte_gpu_ops { /* Get device info. If NULL, info is just copied. */ rte_gpu_info_get_t *dev_info_get; /* Close device or child context. */ rte_gpu_close_t *dev_close; + /* Allocate memory in device. */ + rte_gpu_mem_alloc_t *mem_alloc; + /* Register CPU memory in device. */ + rte_gpu_mem_register_t *mem_register; + /* Free memory allocated or registered in device. */ + rte_gpu_free_t *mem_free; + /* Unregister CPU memory in device. */ + rte_gpu_mem_unregister_t *mem_unregister; }; struct rte_gpu_mpshared { diff --git a/lib/gpudev/rte_gpudev.h b/lib/gpudev/rte_gpudev.h index df75dbdbab..fee71d60e7 100644 --- a/lib/gpudev/rte_gpudev.h +++ b/lib/gpudev/rte_gpudev.h @@ -9,6 +9,7 @@ #include #include +#include #include /** @@ -292,6 +293,100 @@ int rte_gpu_callback_unregister(int16_t dev_id, enum rte_gpu_event event, __rte_experimental int rte_gpu_info_get(int16_t dev_id, struct rte_gpu_info *info); +/** + * @warning + * @b EXPERIMENTAL: this API may change without prior notice. + * + * Allocate a chunk of memory usable by the device. + * + * @param dev_id + * Device ID requiring allocated memory. + * @param size + * Number of bytes to allocate. + * Requesting 0 will do nothing. + * + * @return + * A pointer to the allocated memory, otherwise NULL and rte_errno is set: + * - ENODEV if invalid dev_id + * - EINVAL if reserved flags + * - ENOTSUP if operation not supported by the driver + * - E2BIG if size is higher than limit + * - ENOMEM if out of space + * - EPERM if driver error + */ +__rte_experimental +void *rte_gpu_malloc(int16_t dev_id, size_t size) +__rte_alloc_size(2); + +/** + * @warning + * @b EXPERIMENTAL: this API may change without prior notice. + * + * Deallocate a chunk of memory allocated with rte_gpu_malloc(). + * + * @param dev_id + * Reference device ID. + * @param ptr + * Pointer to the memory area to be deallocated. + * NULL is a no-op accepted value. + * + * @return + * 0 on success, -rte_errno otherwise: + * - ENODEV if invalid dev_id + * - ENOTSUP if operation not supported by the driver + * - EPERM if driver error + */ +__rte_experimental +int rte_gpu_free(int16_t dev_id, void *ptr); + +/** + * @warning + * @b EXPERIMENTAL: this API may change without prior notice. + * + * Register a chunk of memory on the CPU usable by the device. + * + * @param dev_id + * Device ID requiring allocated memory. + * @param size + * Number of bytes to allocate. + * Requesting 0 will do nothing. + * @param ptr + * Pointer to the memory area to be registered. + * NULL is a no-op accepted value. + + * @return + * A pointer to the allocated memory, otherwise NULL and rte_errno is set: + * - ENODEV if invalid dev_id + * - EINVAL if reserved flags + * - ENOTSUP if operation not supported by the driver + * - E2BIG if size is higher than limit + * - ENOMEM if out of space + * - EPERM if driver error + */ +__rte_experimental +int rte_gpu_register(int16_t dev_id, size_t size, void *ptr); + +/** + * @warning + * @b EXPERIMENTAL: this API may change without prior notice. + * + * Deregister a chunk of memory previusly registered with rte_gpu_mem_register() + * + * @param dev_id + * Reference device ID. + * @param ptr + * Pointer to the memory area to be unregistered. + * NULL is a no-op accepted value. + * + * @return + * 0 on success, -rte_errno otherwise: + * - ENODEV if invalid dev_id + * - ENOTSUP if operation not supported by the driver + * - EPERM if driver error + */ +__rte_experimental +int rte_gpu_unregister(int16_t dev_id, void *ptr); + #ifdef __cplusplus } #endif diff --git a/lib/gpudev/version.map b/lib/gpudev/version.map index 58dc632393..d4a65ebd52 100644 --- a/lib/gpudev/version.map +++ b/lib/gpudev/version.map @@ -8,9 +8,13 @@ EXPERIMENTAL { rte_gpu_close; rte_gpu_count_avail; rte_gpu_find_next; + rte_gpu_free; rte_gpu_info_get; rte_gpu_init; rte_gpu_is_valid; + rte_gpu_malloc; + rte_gpu_register; + rte_gpu_unregister; }; INTERNAL { From patchwork Mon Nov 8 18:58:02 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Elena Agostini X-Patchwork-Id: 103976 X-Patchwork-Delegate: thomas@monjalon.net Return-Path: X-Original-To: patchwork@inbox.dpdk.org Delivered-To: patchwork@inbox.dpdk.org Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id 9D12DA0C4D; Mon, 8 Nov 2021 11:47:55 +0100 (CET) Received: from [217.70.189.124] (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id 5E3DD41151; Mon, 8 Nov 2021 11:47:13 +0100 (CET) Received: from NAM11-DM6-obe.outbound.protection.outlook.com (mail-dm6nam11on2068.outbound.protection.outlook.com [40.107.223.68]) by mails.dpdk.org (Postfix) with ESMTP id 06B2F4113D for ; Mon, 8 Nov 2021 11:47:10 +0100 (CET) ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=TBds01TGv+goJyD1dB72+mYk6wZYifc/UXQcrI40e5qdhOcQYQi7O3JP/nIptMql5nAyarPaGfJMzPiqlsuP6ohFao2j3qm5j9q2EWv2S81kII1w36NztYKZC5bRlJQ4xFT+jikme7LQ3M4UvjTS0Rw7+RX1d/rF2YIDJu6kysIK9zvLgJdjjSroaZ8jbHwwi3DQ71p/qC9AGrorUfroEv1LVs+TIC/NelkHdf/HGaKIt4u05K9pDjErYL75WfEgtP8p1eB/PIydktxqiUy9cglOKVzGt2hMJoZA4clN0A8TmlhsVMp+Bzn6po3540wTcFDqP9DFgv+yoOrzbOWo4w== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=zIGv8TrgVKAELEpyoxP8Y1BNCXyegMlzXpEHofm9OVI=; b=bLRo7zY+3O8beMs/xPFbhwuyWcG9k27MmxS/kIygCMZ+Bsl0f8OJDc1UHbOBqky4JbbwdC5M17fuFeMew2nXGTQx9AsDZxcLP5btPw5Vm1ZAr+Di02lp0Gf5tHySifZWIBwvZ5UqMZ9/hD1o4WLYai8cUfLzxlg6tqdu2N4QioB20EQ3u8DIkgcvU0qjjx1wfkEvb2wwaDSlKgsOay76eQzLGYoeXtcdwc0Py3l3PSvfXORbU1gmpL8aD9prOCbe/7BFzdVqzyVPdtTP3aX0NgqXMdjyl93VwPqswCMtLROzfywFjBpPrOf7sfQ94KyeWAuefVC5rW4a/nxpYEwBlQ== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass (sender ip is 216.228.112.34) smtp.rcpttodomain=dpdk.org smtp.mailfrom=nvidia.com; dmarc=pass (p=quarantine sp=quarantine pct=100) action=none header.from=nvidia.com; dkim=none (message not signed); arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=Nvidia.com; s=selector2; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=zIGv8TrgVKAELEpyoxP8Y1BNCXyegMlzXpEHofm9OVI=; b=BKHoruBlwfREFbAwL0YBmM4TbPHYQsH5oQoH5dTIqZs+D096SCx5lIrfpHwOeTgkuMHyDEHea+lKIhgdMzS8Ukffaj6MKL1J994WNaEn6DlbVvzLYNTLs+ocCRUqW1LXHuOx/OBRLKFjkI2sArgCsMl03JqFFio0AJsmu5JHn7c3bnWRinLW4bQqnao/yEGhzc9zCu0D4HPaWHf3maXK/izxf29wvCDJ5fqGNReuJ3d2GkZefH1+imqTY6ZxoyYvEw1eZdrDA5WE1Zaejvk/0fArkeuM/3fA39lBVxA/UqfMbB4pVjC65gDeLQj1n4tzVVP5BoWNslIY3zSQN61MjQ== Received: from MW2PR16CA0053.namprd16.prod.outlook.com (2603:10b6:907:1::30) by BY5PR12MB4242.namprd12.prod.outlook.com (2603:10b6:a03:203::9) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4669.10; Mon, 8 Nov 2021 10:47:08 +0000 Received: from CO1NAM11FT064.eop-nam11.prod.protection.outlook.com (2603:10b6:907:1:cafe::fd) by MW2PR16CA0053.outlook.office365.com (2603:10b6:907:1::30) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4669.10 via Frontend Transport; Mon, 8 Nov 2021 10:47:08 +0000 X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 216.228.112.34) smtp.mailfrom=nvidia.com; dpdk.org; dkim=none (message not signed) header.d=none;dpdk.org; dmarc=pass action=none header.from=nvidia.com; Received-SPF: Pass (protection.outlook.com: domain of nvidia.com designates 216.228.112.34 as permitted sender) receiver=protection.outlook.com; client-ip=216.228.112.34; helo=mail.nvidia.com; Received: from mail.nvidia.com (216.228.112.34) by CO1NAM11FT064.mail.protection.outlook.com (10.13.175.77) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA384) id 15.20.4669.10 via Frontend Transport; Mon, 8 Nov 2021 10:47:07 +0000 Received: from nvidia.com (172.20.187.6) by HQMAIL107.nvidia.com (172.20.187.13) with Microsoft SMTP Server (TLS) id 15.0.1497.18; Mon, 8 Nov 2021 10:47:01 +0000 From: To: CC: Elena Agostini Date: Mon, 8 Nov 2021 18:58:02 +0000 Message-ID: <20211108185805.3887-7-eagostini@nvidia.com> X-Mailer: git-send-email 2.17.1 In-Reply-To: <20211108185805.3887-1-eagostini@nvidia.com> References: <20210602203531.2288645-1-thomas@monjalon.net> <20211108185805.3887-1-eagostini@nvidia.com> MIME-Version: 1.0 X-Originating-IP: [172.20.187.6] X-ClientProxiedBy: HQMAIL105.nvidia.com (172.20.187.12) To HQMAIL107.nvidia.com (172.20.187.13) X-EOPAttributedMessage: 0 X-MS-PublicTrafficType: Email X-MS-Office365-Filtering-Correlation-Id: f1915d67-6a53-4350-0648-08d9a2a51c78 X-MS-TrafficTypeDiagnostic: BY5PR12MB4242: X-Microsoft-Antispam-PRVS: X-MS-Oob-TLC-OOBClassifiers: OLM:3968; X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: 0dL8hcvA6o+VufsoZ5yZ8Kb3MGm+YtwqIqXwIOYSJk7xtvdlQHtlgT/cGnuadvU5s57gOW4i/If13GW5PJ2VRPVf8al/eaOI35Zzg6p4gD1qHpXVJT26/ZUk+TvStaBvtC7Tmgj8d32xyqKKynnh971hCmyLDEP5aFe5vfyQhvGDIohBRCUJaHy0fZ6PEL3gTlj8VjwcTxv/ZKH+YAcYLl+wzfFB7oblacPkMgzuyq8J6wL4juwYbyuWShgQbIslPpyDdtICiuwERTehEg+UDVR4Z30PUeUhPBjGwFDqLgLNrEiLXKpNyg1kzBFeo3EtOYUFDwV02O2UNBS8Hj7TyDePyiiFkfXvDqiXMdpgj9J4DeVrds1HYGW/WXnwa/SDxp9ckatZalClpiIp4YAbYQCqt5uUc9jU0Sd0a/vMcL0pL7TRgANMbTY2N4wZF12Juup794T7BXA1vYv+4/wHndFjbNhow3KxBv44a/vH0PPa3QvnlMQk79ZNgbD3RDFZZpLuTejUkqgqPk6QzPOKJIkarF5I0bhSxQb+Wr7C0sVeq+KlAxM0tx1eNVAnmuqH3H3k0ggm9ypSi9eklJ0NcQEj3Pdhysoi3ORl8tD95F8NhS8D0r88DRC7jw3Ahh++SvkpNfG9OaA5QjQhROuZ4CHYfQ+moU6QiDvo2m3QZ7am/ASxIjHl3WJOCmiUYx6XbVWJxgEqiHh/J7KlHymzxg== X-Forefront-Antispam-Report: CIP:216.228.112.34; CTRY:US; LANG:en; SCL:1; SRV:; IPV:NLI; SFV:NSPM; H:mail.nvidia.com; PTR:schybrid03.nvidia.com; CAT:NONE; SFS:(4636009)(36840700001)(46966006)(6286002)(5660300002)(86362001)(2906002)(26005)(2876002)(7636003)(186003)(16526019)(7696005)(1076003)(36756003)(36860700001)(508600001)(55016002)(83380400001)(82310400003)(8936002)(4326008)(426003)(47076005)(336012)(6916009)(8676002)(70206006)(316002)(107886003)(356005)(70586007)(2616005); DIR:OUT; SFP:1101; X-OriginatorOrg: Nvidia.com X-MS-Exchange-CrossTenant-OriginalArrivalTime: 08 Nov 2021 10:47:07.9467 (UTC) X-MS-Exchange-CrossTenant-Network-Message-Id: f1915d67-6a53-4350-0648-08d9a2a51c78 X-MS-Exchange-CrossTenant-Id: 43083d15-7273-40c1-b7db-39efd9ccc17a X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=43083d15-7273-40c1-b7db-39efd9ccc17a; Ip=[216.228.112.34]; Helo=[mail.nvidia.com] X-MS-Exchange-CrossTenant-AuthSource: CO1NAM11FT064.eop-nam11.prod.protection.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Anonymous X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem X-MS-Exchange-Transport-CrossTenantHeadersStamped: BY5PR12MB4242 Subject: [dpdk-dev] [PATCH v5 6/9] gpudev: add memory barrier X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Sender: "dev" From: Elena Agostini Add a function for the application to ensure the coherency of the writes executed by another device into the GPU memory. Signed-off-by: Elena Agostini --- doc/guides/prog_guide/gpudev.rst | 8 ++++++++ lib/gpudev/gpudev.c | 19 +++++++++++++++++++ lib/gpudev/gpudev_driver.h | 3 +++ lib/gpudev/rte_gpudev.h | 18 ++++++++++++++++++ lib/gpudev/version.map | 1 + 5 files changed, 49 insertions(+) diff --git a/doc/guides/prog_guide/gpudev.rst b/doc/guides/prog_guide/gpudev.rst index 9aca69038c..eb5f0af817 100644 --- a/doc/guides/prog_guide/gpudev.rst +++ b/doc/guides/prog_guide/gpudev.rst @@ -65,3 +65,11 @@ gpudev can register a CPU memory area to make it visible from a GPU device. Later, it's also possible to unregister that memory with gpudev. CPU memory registered outside of the gpudev library (e.g. with GPU specific library) cannot be unregistered by the gpudev library. + +Memory Barrier +~~~~~~~~~~~~~~ + +Some GPU drivers may need, under certain conditions, +to enforce the coherency of external devices writes (e.g. NIC receiving packets) +into the GPU memory. +gpudev abstracts and exposes this capability. diff --git a/lib/gpudev/gpudev.c b/lib/gpudev/gpudev.c index d0826ec881..49526b335f 100644 --- a/lib/gpudev/gpudev.c +++ b/lib/gpudev/gpudev.c @@ -624,3 +624,22 @@ rte_gpu_free(int16_t dev_id, void *ptr) } return GPU_DRV_RET(dev->ops.mem_free(dev, ptr)); } + +int +rte_gpu_mbw(int16_t dev_id) +{ + struct rte_gpu *dev; + + dev = gpu_get_by_id(dev_id); + if (dev == NULL) { + GPU_LOG(ERR, "memory barrier for invalid device ID %d", dev_id); + rte_errno = ENODEV; + return -rte_errno; + } + + if (dev->ops.mbw == NULL) { + rte_errno = ENOTSUP; + return -rte_errno; + } + return GPU_DRV_RET(dev->ops.mbw(dev)); +} diff --git a/lib/gpudev/gpudev_driver.h b/lib/gpudev/gpudev_driver.h index 11015944a6..ab24de9e28 100644 --- a/lib/gpudev/gpudev_driver.h +++ b/lib/gpudev/gpudev_driver.h @@ -31,6 +31,7 @@ typedef int (rte_gpu_mem_alloc_t)(struct rte_gpu *dev, size_t size, void **ptr); typedef int (rte_gpu_free_t)(struct rte_gpu *dev, void *ptr); typedef int (rte_gpu_mem_register_t)(struct rte_gpu *dev, size_t size, void *ptr); typedef int (rte_gpu_mem_unregister_t)(struct rte_gpu *dev, void *ptr); +typedef int (rte_gpu_mbw_t)(struct rte_gpu *dev); struct rte_gpu_ops { /* Get device info. If NULL, info is just copied. */ @@ -45,6 +46,8 @@ struct rte_gpu_ops { rte_gpu_free_t *mem_free; /* Unregister CPU memory in device. */ rte_gpu_mem_unregister_t *mem_unregister; + /* Enforce GPU memory write barrier. */ + rte_gpu_mbw_t *mbw; }; struct rte_gpu_mpshared { diff --git a/lib/gpudev/rte_gpudev.h b/lib/gpudev/rte_gpudev.h index fee71d60e7..650ebfd700 100644 --- a/lib/gpudev/rte_gpudev.h +++ b/lib/gpudev/rte_gpudev.h @@ -387,6 +387,24 @@ int rte_gpu_register(int16_t dev_id, size_t size, void *ptr); __rte_experimental int rte_gpu_unregister(int16_t dev_id, void *ptr); +/** + * @warning + * @b EXPERIMENTAL: this API may change without prior notice. + * + * Enforce a GPU memory write barrier. + * + * @param dev_id + * Reference device ID. + * + * @return + * 0 on success, -rte_errno otherwise: + * - ENODEV if invalid dev_id + * - ENOTSUP if operation not supported by the driver + * - EPERM if driver error + */ +__rte_experimental +int rte_gpu_mbw(int16_t dev_id); + #ifdef __cplusplus } #endif diff --git a/lib/gpudev/version.map b/lib/gpudev/version.map index d4a65ebd52..d72d470d8e 100644 --- a/lib/gpudev/version.map +++ b/lib/gpudev/version.map @@ -13,6 +13,7 @@ EXPERIMENTAL { rte_gpu_init; rte_gpu_is_valid; rte_gpu_malloc; + rte_gpu_mbw; rte_gpu_register; rte_gpu_unregister; }; From patchwork Mon Nov 8 18:58:03 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Elena Agostini X-Patchwork-Id: 103977 X-Patchwork-Delegate: thomas@monjalon.net Return-Path: X-Original-To: patchwork@inbox.dpdk.org Delivered-To: patchwork@inbox.dpdk.org Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id DE313A0C4D; Mon, 8 Nov 2021 11:48:01 +0100 (CET) Received: from [217.70.189.124] (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id 5D30141157; Mon, 8 Nov 2021 11:47:14 +0100 (CET) Received: from NAM04-MW2-obe.outbound.protection.outlook.com (mail-mw2nam08on2073.outbound.protection.outlook.com [40.107.101.73]) by mails.dpdk.org (Postfix) with ESMTP id 3F42941145 for ; Mon, 8 Nov 2021 11:47:11 +0100 (CET) ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=ExNYGSF1RAI3zzlfXA8bFdU9M7unX4s1p+TGwMEQDHJLg0mI0XrA+ZIdWdDGL3wsN5bZJWsOMwOP3Nfw2NdJZgWRyYRLnr21S4OnxIr+aGRjRdGQ79qOxO2HEt2K/vHgs+ECOwlQcZFNkzQA6EtuE6/lQ1GDwV5QoNpcb0oPSJoJveGFxllzs+31Z3aDTn+7vkxT2tqRxogwOQvEXtzzoYxzHRJqEXLaBBPfwuQZ8jAm/Q8fJxlpn6T1cIH7sjQkCeyE7glAJH90g2pjAMxDxQHtAYaQKSH7DfUTE3VDTq4jjuZLJWSioewna1r0yiFqrrwwvMMMoI8qTP/nbeIxlg== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=qCVTJrRpKsMxoI4JKFRbIYu48qyr6fMfernRBPvHMIU=; b=Exp9LZkQisvHbxD9LtFbXyNQt1BDCeTfPlyqriP0gSkMjKBVVCQzNVxcia3qMQAEYy902B5HLqL+dr5iCJ0pwEGCxLguRfUIWw0WINof+R4LgtB8Y2YFYoeCiuM8WWuy1hLTYXCMz4xdyVprTrJ/gFmfMg5xkh2qitw5wb7wLgfa7bgjG6A2mpgK9qunDmhrR7sN5V9lKcZgHzsg/vd8f+M8YYKgSiV1zacNOJf9gY6ZwtvdzvXHg2Mo3+PGcm2mGvnmNikE2KNvJ/uh39tMiAePnUgmP7kkwHJJlVGrPU4QRMPskfGyeSdJkqvozspt0/zVy+Q5W8sMb5ZZ9e2cpQ== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass (sender ip is 216.228.112.34) smtp.rcpttodomain=dpdk.org smtp.mailfrom=nvidia.com; dmarc=pass (p=quarantine sp=quarantine pct=100) action=none header.from=nvidia.com; dkim=none (message not signed); arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=Nvidia.com; s=selector2; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=qCVTJrRpKsMxoI4JKFRbIYu48qyr6fMfernRBPvHMIU=; b=T7YYsdtuMh117700QOWrr8zniGf08oy6TTN0Z8ex93nwm5SluodH3TZbC0cDRJ0l4+Qj/H6WRci3faJIql4tPuxAXz1jXe+Su2zf7mMLxdDNoP6eKcvYau/iY31fH41CfHpufQqz/A2Ij2i22JJUfZn6+RPq7dUAjm/vk3/CAFYMSnwlQPTrq4YVkEncZX+ICFrGNftG99o8Efy0+UeHDAi9olmStPQUuPpNecrKqQKkUaxrIQI39W4u/OVINstzmypsoXmvjyVPTGKXSiZ8Nqs5ShMOUrPafQMW1PtfFgBcTl759V6dI2K5p+jFH3nB1BJQWXaJCzUzuXycJ8d1uA== Received: from MW2PR16CA0042.namprd16.prod.outlook.com (2603:10b6:907:1::19) by MN2PR12MB4301.namprd12.prod.outlook.com (2603:10b6:208:1d4::22) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4669.11; Mon, 8 Nov 2021 10:47:09 +0000 Received: from CO1NAM11FT064.eop-nam11.prod.protection.outlook.com (2603:10b6:907:1:cafe::eb) by MW2PR16CA0042.outlook.office365.com (2603:10b6:907:1::19) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4669.10 via Frontend Transport; Mon, 8 Nov 2021 10:47:09 +0000 X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 216.228.112.34) smtp.mailfrom=nvidia.com; dpdk.org; dkim=none (message not signed) header.d=none;dpdk.org; dmarc=pass action=none header.from=nvidia.com; Received-SPF: Pass (protection.outlook.com: domain of nvidia.com designates 216.228.112.34 as permitted sender) receiver=protection.outlook.com; client-ip=216.228.112.34; helo=mail.nvidia.com; Received: from mail.nvidia.com (216.228.112.34) by CO1NAM11FT064.mail.protection.outlook.com (10.13.175.77) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA384) id 15.20.4669.10 via Frontend Transport; Mon, 8 Nov 2021 10:47:08 +0000 Received: from nvidia.com (172.20.187.6) by HQMAIL107.nvidia.com (172.20.187.13) with Microsoft SMTP Server (TLS) id 15.0.1497.18; Mon, 8 Nov 2021 10:47:01 +0000 From: To: CC: Elena Agostini Date: Mon, 8 Nov 2021 18:58:03 +0000 Message-ID: <20211108185805.3887-8-eagostini@nvidia.com> X-Mailer: git-send-email 2.17.1 In-Reply-To: <20211108185805.3887-1-eagostini@nvidia.com> References: <20210602203531.2288645-1-thomas@monjalon.net> <20211108185805.3887-1-eagostini@nvidia.com> MIME-Version: 1.0 X-Originating-IP: [172.20.187.6] X-ClientProxiedBy: HQMAIL105.nvidia.com (172.20.187.12) To HQMAIL107.nvidia.com (172.20.187.13) X-EOPAttributedMessage: 0 X-MS-PublicTrafficType: Email X-MS-Office365-Filtering-Correlation-Id: 0c5cf90a-0028-4ef8-20b6-08d9a2a51cef X-MS-TrafficTypeDiagnostic: MN2PR12MB4301: X-Microsoft-Antispam-PRVS: X-MS-Oob-TLC-OOBClassifiers: OLM:117; X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: YYTpV2O/rtsL4R9u8e0B57FzXMOqwGyGrfIe3K+B8OCJu4krxmRHS3cCbM3gWKhkqc3mgcyM4Yw8QPvUYbEIFnKFjI4i+Kjxk4QCsC3926cVFXSEUCk+TN+cvtR4xJgFGRTKg4A5+2ySsHlffx9ntIjjthg6xjk/spz0jdMIq9jX31WVNuuZmEUkhA023g2/8Fsjt7xngV/nSC2FNEstm7HmzczMY3s8MDHG5PsywCo9cGm+BqjwFMxreFvX0g8oJxpXxhY5LSxUnKRQXC76FJUdOuuMC8JuYz62DeayXI9M9mpLh/Oi4MYUoT3fgsdkGjiv0sEam+mqmR7Pwkb0Plp3A5Cvld4FLGNCXZSte+42QJSSizVPfOhC7iQqTDpFS5Miu+fYxAhRFUOHfl2D2YRopXxpwoxV471g3MMvcntosEQTceBrXkV/SZM7brUSHdskm7Ck+HOAuXc4KVYhtwjAzwC8UL+c0A1WYHqZQt4I2qkuBGNZBWliU7uxEM/WFcAUfHhk4s9cA9KUZFcjYkskIjULCgvIzHUsSXqMTR68H2jG93UvREqk5Deqo7yo++wOVC+5jVh5rltOurDJn3VI0NcifgTgSvue1gevJV9R2+HTr7QVsecWxZ7Sk/ij2gZugfAWLSUjXc9RYU360ouoLChww03FEeByUhHxGk8y208/GlF6rqRduBQafgkVEFI2PoqkX1meKNBSFrdBVg== X-Forefront-Antispam-Report: CIP:216.228.112.34; CTRY:US; LANG:en; SCL:1; SRV:; IPV:NLI; SFV:NSPM; H:mail.nvidia.com; PTR:schybrid03.nvidia.com; CAT:NONE; SFS:(4636009)(36840700001)(46966006)(30864003)(6916009)(5660300002)(2876002)(1076003)(36756003)(36860700001)(107886003)(83380400001)(426003)(2906002)(2616005)(26005)(356005)(7636003)(55016002)(47076005)(6286002)(86362001)(336012)(508600001)(8936002)(4326008)(70586007)(186003)(316002)(8676002)(7696005)(16526019)(70206006)(82310400003); DIR:OUT; SFP:1101; X-OriginatorOrg: Nvidia.com X-MS-Exchange-CrossTenant-OriginalArrivalTime: 08 Nov 2021 10:47:08.7353 (UTC) X-MS-Exchange-CrossTenant-Network-Message-Id: 0c5cf90a-0028-4ef8-20b6-08d9a2a51cef X-MS-Exchange-CrossTenant-Id: 43083d15-7273-40c1-b7db-39efd9ccc17a X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=43083d15-7273-40c1-b7db-39efd9ccc17a; Ip=[216.228.112.34]; Helo=[mail.nvidia.com] X-MS-Exchange-CrossTenant-AuthSource: CO1NAM11FT064.eop-nam11.prod.protection.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Anonymous X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem X-MS-Exchange-Transport-CrossTenantHeadersStamped: MN2PR12MB4301 Subject: [dpdk-dev] [PATCH v5 7/9] gpudev: add communication flag X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Sender: "dev" From: Elena Agostini In heterogeneous computing system, processing is not only in the CPU. Some tasks can be delegated to devices working in parallel. When mixing network activity with task processing there may be the need to put in communication the CPU with the device in order to synchronize operations. The purpose of this flag is to allow the CPU and the GPU to exchange ACKs. A possible use-case is described below. CPU: - Trigger some task on the GPU - Prepare some data - Signal to the GPU the data is ready updating the communication flag GPU: - Do some pre-processing - Wait for more data from the CPU polling on the communication flag - Consume the data prepared by the CPU Signed-off-by: Elena Agostini --- app/test-gpudev/main.c | 60 ++++++++++++++ doc/guides/prog_guide/gpudev.rst | 13 +++ doc/guides/rel_notes/release_21_11.rst | 1 + lib/gpudev/gpudev.c | 92 +++++++++++++++++++++ lib/gpudev/rte_gpudev.h | 108 +++++++++++++++++++++++++ lib/gpudev/version.map | 4 + 6 files changed, 278 insertions(+) diff --git a/app/test-gpudev/main.c b/app/test-gpudev/main.c index e3aca2225a..516a01b927 100644 --- a/app/test-gpudev/main.c +++ b/app/test-gpudev/main.c @@ -154,6 +154,61 @@ register_cpu_memory(uint16_t gpu_id) return 0; } +static int +create_update_comm_flag(uint16_t gpu_id) +{ + struct rte_gpu_comm_flag devflag; + int ret = 0; + uint32_t set_val; + uint32_t get_val; + + printf("\n=======> TEST: Communication flag\n"); + + ret = rte_gpu_comm_create_flag(gpu_id, &devflag, RTE_GPU_COMM_FLAG_CPU); + if (ret < 0) { + fprintf(stderr, "rte_gpu_comm_create_flag returned error %d\n", ret); + return -1; + } + + set_val = 25; + ret = rte_gpu_comm_set_flag(&devflag, set_val); + if (ret < 0) { + fprintf(stderr, "rte_gpu_comm_set_flag returned error %d\n", ret); + return -1; + } + + ret = rte_gpu_comm_get_flag_value(&devflag, &get_val); + if (ret < 0) { + fprintf(stderr, "rte_gpu_comm_get_flag_value returned error %d\n", ret); + return -1; + } + + printf("Communication flag value at 0x%p was set to %d and current value is %d\n", devflag.ptr, set_val, get_val); + + set_val = 38; + ret = rte_gpu_comm_set_flag(&devflag, set_val); + if (ret < 0) { + fprintf(stderr, "rte_gpu_comm_set_flag returned error %d\n", ret); + return -1; + } + + ret = rte_gpu_comm_get_flag_value(&devflag, &get_val); + if (ret < 0) { + fprintf(stderr, "rte_gpu_comm_get_flag_value returned error %d\n", ret); + return -1; + } + + printf("Communication flag value at 0x%p was set to %d and current value is %d\n", devflag.ptr, set_val, get_val); + + ret = rte_gpu_comm_destroy_flag(&devflag); + if (ret < 0) { + fprintf(stderr, "rte_gpu_comm_destroy_flags returned error %d\n", ret); + return -1; + } + + return 0; +} + int main(int argc, char **argv) { @@ -204,6 +259,11 @@ main(int argc, char **argv) alloc_gpu_memory(gpu_id); register_cpu_memory(gpu_id); + /** + * Communication items test + */ + create_update_comm_flag(gpu_id); + /* clean up the EAL */ rte_eal_cleanup(); printf("Bye...\n"); diff --git a/doc/guides/prog_guide/gpudev.rst b/doc/guides/prog_guide/gpudev.rst index eb5f0af817..e0db627aed 100644 --- a/doc/guides/prog_guide/gpudev.rst +++ b/doc/guides/prog_guide/gpudev.rst @@ -32,6 +32,10 @@ This library provides a number of features: - Interoperability with device-specific library through generic handlers. - Allocate and free memory on the device. - Register CPU memory to make it visible from the device. +- Communication between the CPU and the device. + +The whole CPU - GPU communication is implemented +using CPU memory visible from the GPU. API Overview @@ -73,3 +77,12 @@ Some GPU drivers may need, under certain conditions, to enforce the coherency of external devices writes (e.g. NIC receiving packets) into the GPU memory. gpudev abstracts and exposes this capability. + +Communication Flag +~~~~~~~~~~~~~~~~~~ + +Considering an application with some GPU task +that's waiting to receive a signal from the CPU +to move forward with the execution. +The communication flag allocates a CPU memory GPU-visible ``uint32_t`` flag +that can be used by the CPU to communicate with a GPU task. diff --git a/doc/guides/rel_notes/release_21_11.rst b/doc/guides/rel_notes/release_21_11.rst index a4d07bda9b..78b29d9a25 100644 --- a/doc/guides/rel_notes/release_21_11.rst +++ b/doc/guides/rel_notes/release_21_11.rst @@ -105,6 +105,7 @@ New Features * Device information * Memory management + * Communication flag * **Added new RSS offload types for IPv4/L4 checksum in RSS flow.** diff --git a/lib/gpudev/gpudev.c b/lib/gpudev/gpudev.c index 49526b335f..f887f3dd93 100644 --- a/lib/gpudev/gpudev.c +++ b/lib/gpudev/gpudev.c @@ -643,3 +643,95 @@ rte_gpu_mbw(int16_t dev_id) } return GPU_DRV_RET(dev->ops.mbw(dev)); } + +int +rte_gpu_comm_create_flag(uint16_t dev_id, struct rte_gpu_comm_flag *devflag, + enum rte_gpu_comm_flag_type mtype) +{ + size_t flag_size; + int ret; + + if (devflag == NULL) { + rte_errno = EINVAL; + return -rte_errno; + } + if (mtype != RTE_GPU_COMM_FLAG_CPU) { + rte_errno = EINVAL; + return -rte_errno; + } + + flag_size = sizeof(uint32_t); + + devflag->ptr = rte_zmalloc(NULL, flag_size, 0); + if (devflag->ptr == NULL) { + rte_errno = ENOMEM; + return -rte_errno; + } + + ret = rte_gpu_register(dev_id, flag_size, devflag->ptr); + if (ret < 0) { + rte_errno = ENOMEM; + return -rte_errno; + } + + devflag->mtype = mtype; + devflag->dev_id = dev_id; + + return 0; +} + +int +rte_gpu_comm_destroy_flag(struct rte_gpu_comm_flag *devflag) +{ + int ret; + + if (devflag == NULL) { + rte_errno = EINVAL; + return -rte_errno; + } + + ret = rte_gpu_unregister(devflag->dev_id, devflag->ptr); + if (ret < 0) { + rte_errno = EINVAL; + return -1; + } + + rte_free(devflag->ptr); + + return 0; +} + +int +rte_gpu_comm_set_flag(struct rte_gpu_comm_flag *devflag, uint32_t val) +{ + if (devflag == NULL) { + rte_errno = EINVAL; + return -rte_errno; + } + + if (devflag->mtype != RTE_GPU_COMM_FLAG_CPU) { + rte_errno = EINVAL; + return -rte_errno; + } + + RTE_GPU_VOLATILE(*devflag->ptr) = val; + + return 0; +} + +int +rte_gpu_comm_get_flag_value(struct rte_gpu_comm_flag *devflag, uint32_t *val) +{ + if (devflag == NULL) { + rte_errno = EINVAL; + return -rte_errno; + } + if (devflag->mtype != RTE_GPU_COMM_FLAG_CPU) { + rte_errno = EINVAL; + return -rte_errno; + } + + *val = RTE_GPU_VOLATILE(*devflag->ptr); + + return 0; +} diff --git a/lib/gpudev/rte_gpudev.h b/lib/gpudev/rte_gpudev.h index 650ebfd700..1466ac164b 100644 --- a/lib/gpudev/rte_gpudev.h +++ b/lib/gpudev/rte_gpudev.h @@ -38,6 +38,9 @@ extern "C" { /** Catch-all callback data. */ #define RTE_GPU_CALLBACK_ANY_DATA ((void *)-1) +/** Access variable as volatile. */ +#define RTE_GPU_VOLATILE(x) (*(volatile typeof(x) *)&(x)) + /** Store device info. */ struct rte_gpu_info { /** Unique identifier name. */ @@ -68,6 +71,22 @@ enum rte_gpu_event { typedef void (rte_gpu_callback_t)(int16_t dev_id, enum rte_gpu_event event, void *user_data); +/** Memory where communication flag is allocated. */ +enum rte_gpu_comm_flag_type { + /** Allocate flag on CPU memory visible from device. */ + RTE_GPU_COMM_FLAG_CPU = 0, +}; + +/** Communication flag to coordinate CPU with the device. */ +struct rte_gpu_comm_flag { + /** Device that will use the device flag. */ + uint16_t dev_id; + /** Pointer to flag memory area. */ + uint32_t *ptr; + /** Type of memory used to allocate the flag. */ + enum rte_gpu_comm_flag_type mtype; +}; + /** * @warning * @b EXPERIMENTAL: this API may change without prior notice. @@ -405,6 +424,95 @@ int rte_gpu_unregister(int16_t dev_id, void *ptr); __rte_experimental int rte_gpu_mbw(int16_t dev_id); +/** + * @warning + * @b EXPERIMENTAL: this API may change without prior notice. + * + * Create a communication flag that can be shared + * between CPU threads and device workload to exchange some status info + * (e.g. work is done, processing can start, etc..). + * + * @param dev_id + * Reference device ID. + * @param devflag + * Pointer to the memory area of the devflag structure. + * @param mtype + * Type of memory to allocate the communication flag. + * + * @return + * 0 on success, -rte_errno otherwise: + * - ENODEV if invalid dev_id + * - EINVAL if invalid inputs + * - ENOTSUP if operation not supported by the driver + * - ENOMEM if out of space + * - EPERM if driver error + */ +__rte_experimental +int rte_gpu_comm_create_flag(uint16_t dev_id, + struct rte_gpu_comm_flag *devflag, + enum rte_gpu_comm_flag_type mtype); + +/** + * @warning + * @b EXPERIMENTAL: this API may change without prior notice. + * + * Deallocate a communication flag. + * + * @param devflag + * Pointer to the memory area of the devflag structure. + * + * @return + * 0 on success, -rte_errno otherwise: + * - ENODEV if invalid dev_id + * - EINVAL if NULL devflag + * - ENOTSUP if operation not supported by the driver + * - EPERM if driver error + */ +__rte_experimental +int rte_gpu_comm_destroy_flag(struct rte_gpu_comm_flag *devflag); + +/** + * @warning + * @b EXPERIMENTAL: this API may change without prior notice. + * + * Set the value of a communication flag as the input value. + * Flag memory area is treated as volatile. + * The flag must have been allocated with RTE_GPU_COMM_FLAG_CPU. + * + * @param devflag + * Pointer to the memory area of the devflag structure. + * @param val + * Value to set in the flag. + * + * @return + * 0 on success, -rte_errno otherwise: + * - EINVAL if invalid input params + */ +__rte_experimental +int rte_gpu_comm_set_flag(struct rte_gpu_comm_flag *devflag, + uint32_t val); + +/** + * @warning + * @b EXPERIMENTAL: this API may change without prior notice. + * + * Get the value of the communication flag. + * Flag memory area is treated as volatile. + * The flag must have been allocated with RTE_GPU_COMM_FLAG_CPU. + * + * @param devflag + * Pointer to the memory area of the devflag structure. + * @param val + * Flag output value. + * + * @return + * 0 on success, -rte_errno otherwise: + * - EINVAL if invalid input params + */ +__rte_experimental +int rte_gpu_comm_get_flag_value(struct rte_gpu_comm_flag *devflag, + uint32_t *val); + #ifdef __cplusplus } #endif diff --git a/lib/gpudev/version.map b/lib/gpudev/version.map index d72d470d8e..2fc039373a 100644 --- a/lib/gpudev/version.map +++ b/lib/gpudev/version.map @@ -6,6 +6,10 @@ EXPERIMENTAL { rte_gpu_callback_register; rte_gpu_callback_unregister; rte_gpu_close; + rte_gpu_comm_create_flag; + rte_gpu_comm_destroy_flag; + rte_gpu_comm_get_flag_value; + rte_gpu_comm_set_flag; rte_gpu_count_avail; rte_gpu_find_next; rte_gpu_free; From patchwork Mon Nov 8 18:58:04 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Elena Agostini X-Patchwork-Id: 103978 X-Patchwork-Delegate: thomas@monjalon.net Return-Path: X-Original-To: patchwork@inbox.dpdk.org Delivered-To: patchwork@inbox.dpdk.org Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id D90F3A0C4D; Mon, 8 Nov 2021 11:48:08 +0100 (CET) Received: from [217.70.189.124] (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id 8838C4115C; Mon, 8 Nov 2021 11:47:15 +0100 (CET) Received: from NAM10-DM6-obe.outbound.protection.outlook.com (mail-dm6nam10on2073.outbound.protection.outlook.com [40.107.93.73]) by mails.dpdk.org (Postfix) with ESMTP id AFEA541135 for ; Mon, 8 Nov 2021 11:47:11 +0100 (CET) ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=cwSnEzZthhwIAjpCJ6d7c3Kw3tjZzoMjLP7ILRvrKp0nNPgGPfw7XFUwCtt/PNXrwVk31amL51XOWPmwhfwl/T6UuQ29Yne4E6ojJaCr2YXlrsW+SOsylbHWtl5vIXt9ZeRjKSy3Ip+IF3ml0M7AW4AjkFQHEb6f8cp/AWvYaoBM2mKiDOSwHrZyYLcy192sY9ioDII2iLJJFdPl9TBV26uq/SYxIDHwHA1/E5VO9qzazD1qQP0OslHYe0Cnbbop4bx8On8vfjgBwftC7WV1M3u8MLD+Eam/n8/m6q6ivtEwHDKbn7NRJzKTFGDPToXzEB0ivHfLkxX6Y5/nBZUtKg== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=VX5zpehkghcafleHeX83G1br/+w3dEjZdifrsXb2y5s=; b=CncCEOXRlBRaDpgCxymV9bK6KCNmRLX55L0LfJJnLVMlW0DFlY9PHSjzJ3Z2IvyDTvHvBxbRT+dCKzX2RKh3k7XB/VLVhjIMkDkaiLpgriDUHrGfacrxjUEuNil+aPfFneN8oZ/8i6LxRSXAf/Dd4n4xyCirZ/mYU6Ld5V101R3ZErfVNofB6HCVWc9eATSH6TdrL8kegMZfiCcN//2JmOahA9X9DdGaxZaxX+ljgI3G3y5eGa7arOzno8S/quZgLGUAgsppew9SnJ9CrDRhdaDVVHPaGvR8i6LYT8Wwv1UGf+BKnuh6jKpPUSKNidFLYf75VY02nk2dbXKtDMRycQ== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass (sender ip is 216.228.112.34) smtp.rcpttodomain=dpdk.org smtp.mailfrom=nvidia.com; dmarc=pass (p=quarantine sp=quarantine pct=100) action=none header.from=nvidia.com; dkim=none (message not signed); arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=Nvidia.com; s=selector2; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=VX5zpehkghcafleHeX83G1br/+w3dEjZdifrsXb2y5s=; b=I3r1mtYYFa+zWGlbrK/teMxsaT8N4KdgQH38VPsO9YOnFWRLoR9/NbGCFgau8XRyC69pj9qJ6QR7f+Q1cWKUnL2Utpi9MBE5ZgHtRJK/KNSWEhlHN3mz++MJAqrzKxtfLlNJ0FXjY+UARhTeZuIL40KNhi2x5dN445NvXAqkYjM9f7tNWe/7XLb6UF60/0XYFRqcXHDE9sXRR1iuJIkB0RPBH8BBZS8etFobTcNkL8RfBV15BRd4sVxNZGJ77nNoaIUInp2kAR0tw2/F1aIk8wGE2mxvF0CxzM7RMAp4+v8/Fp2cjkeNIhDP1uLorN8XseUFF3qe73Pw9bu/8pnblg== Received: from MW2PR16CA0037.namprd16.prod.outlook.com (2603:10b6:907:1::14) by BY5PR12MB4243.namprd12.prod.outlook.com (2603:10b6:a03:20f::15) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4669.10; Mon, 8 Nov 2021 10:47:10 +0000 Received: from CO1NAM11FT064.eop-nam11.prod.protection.outlook.com (2603:10b6:907:1:cafe::4f) by MW2PR16CA0037.outlook.office365.com (2603:10b6:907:1::14) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4669.11 via Frontend Transport; Mon, 8 Nov 2021 10:47:10 +0000 X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 216.228.112.34) smtp.mailfrom=nvidia.com; dpdk.org; dkim=none (message not signed) header.d=none;dpdk.org; dmarc=pass action=none header.from=nvidia.com; Received-SPF: Pass (protection.outlook.com: domain of nvidia.com designates 216.228.112.34 as permitted sender) receiver=protection.outlook.com; client-ip=216.228.112.34; helo=mail.nvidia.com; Received: from mail.nvidia.com (216.228.112.34) by CO1NAM11FT064.mail.protection.outlook.com (10.13.175.77) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA384) id 15.20.4669.10 via Frontend Transport; Mon, 8 Nov 2021 10:47:09 +0000 Received: from nvidia.com (172.20.187.6) by HQMAIL107.nvidia.com (172.20.187.13) with Microsoft SMTP Server (TLS) id 15.0.1497.18; Mon, 8 Nov 2021 10:47:02 +0000 From: To: CC: Elena Agostini Date: Mon, 8 Nov 2021 18:58:04 +0000 Message-ID: <20211108185805.3887-9-eagostini@nvidia.com> X-Mailer: git-send-email 2.17.1 In-Reply-To: <20211108185805.3887-1-eagostini@nvidia.com> References: <20210602203531.2288645-1-thomas@monjalon.net> <20211108185805.3887-1-eagostini@nvidia.com> MIME-Version: 1.0 X-Originating-IP: [172.20.187.6] X-ClientProxiedBy: HQMAIL105.nvidia.com (172.20.187.12) To HQMAIL107.nvidia.com (172.20.187.13) X-EOPAttributedMessage: 0 X-MS-PublicTrafficType: Email X-MS-Office365-Filtering-Correlation-Id: 74bba9df-e19d-4e42-4752-08d9a2a51d72 X-MS-TrafficTypeDiagnostic: BY5PR12MB4243: X-Microsoft-Antispam-PRVS: X-MS-Oob-TLC-OOBClassifiers: OLM:962; X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: nFtp4+7ArNY+I+5f/BjaEUiksqnoPjAqso6FukTpF5n7OJXMlKe+GYShzkXmFzHJRmrTKU2FbThsYAZkIrnI1WPYCqCUwFcKbvlQtKxyrn9FKvWZJ1R7PVYNt9du/DpbgmcH4Cz9Efv+FEBqw15OL91ypZ0IHpbtZhaEETyhICdw3GuTGQuDZlu0VlkCONx9/YjsjWZ3FknU6IAK8p8Vn8FC+TZlg+1CFtJpD1PsDpu8a4ZBJE7pJEgnu5EIyfibSoukqdE8duZpXrz8nWJx2gVnf06ndwC8AXgAezn2dMwiWgWM5HKGPqlRGjsyaHWT9qmHnMoKRkm7yKh/YM1f4vsciJ7Arw5hDcHtuNqM3EwW0t/C0hP6D7vXa44jrf3oI8rhAG1lR5t8eADRQAO9t1homobVWoYOHfWSTaONUBaq6vd54zGQZ8IcukOS/+f2wehyj/qECaD3w9v+Qiebz3AZaP6GdokDG0jhRISRACaELCS9TCJNLfAUysNuHaf78aT9WyBrvh14IEVor984cPrCuhqUdbpHR5Zb4b6ELZArTgFAnkPD6ByACeCh3nlF/KNsnLz+m7iYbK+pcI6vp6ujY0FOn06iMFf6e/6PQUr0DT6qGBPKfHQxtlDtRLCMTVuuyG3TaLxdh2H30JXCBQwF7lCjzViJldcnWrvgKJeYZ5UxvnC3WKuiTev61qcPQ75xrZoROw7K3pBeM9STeQ== X-Forefront-Antispam-Report: CIP:216.228.112.34; CTRY:US; LANG:en; SCL:1; SRV:; IPV:NLI; SFV:NSPM; H:mail.nvidia.com; PTR:schybrid03.nvidia.com; CAT:NONE; SFS:(4636009)(46966006)(36840700001)(7636003)(356005)(82310400003)(7696005)(26005)(426003)(4326008)(83380400001)(16526019)(2616005)(336012)(508600001)(6286002)(186003)(316002)(8936002)(2906002)(86362001)(36860700001)(70586007)(70206006)(36756003)(2876002)(30864003)(1076003)(8676002)(5660300002)(55016002)(107886003)(6916009)(47076005); DIR:OUT; SFP:1101; X-OriginatorOrg: Nvidia.com X-MS-Exchange-CrossTenant-OriginalArrivalTime: 08 Nov 2021 10:47:09.5968 (UTC) X-MS-Exchange-CrossTenant-Network-Message-Id: 74bba9df-e19d-4e42-4752-08d9a2a51d72 X-MS-Exchange-CrossTenant-Id: 43083d15-7273-40c1-b7db-39efd9ccc17a X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=43083d15-7273-40c1-b7db-39efd9ccc17a; Ip=[216.228.112.34]; Helo=[mail.nvidia.com] X-MS-Exchange-CrossTenant-AuthSource: CO1NAM11FT064.eop-nam11.prod.protection.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Anonymous X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem X-MS-Exchange-Transport-CrossTenantHeadersStamped: BY5PR12MB4243 Subject: [dpdk-dev] [PATCH v5 8/9] gpudev: add communication list X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Sender: "dev" From: Elena Agostini In heterogeneous computing system, processing is not only in the CPU. Some tasks can be delegated to devices working in parallel. When mixing network activity with task processing there may be the need to put in communication the CPU with the device in order to synchronize operations. An example could be a receive-and-process application where CPU is responsible for receiving packets in multiple mbufs and the GPU is responsible for processing the content of those packets. The purpose of this list is to provide a buffer in CPU memory visible from the GPU that can be treated as a circular buffer to let the CPU provide fondamental info of received packets to the GPU. A possible use-case is described below. CPU: - Trigger some task on the GPU - in a loop: - receive a number of packets - provide packets info to the GPU GPU: - Do some pre-processing - Wait to receive a new set of packet to be processed Layout of a communication list would be: ------- | 0 | => pkt_list | status | | #pkts | ------- | 1 | => pkt_list | status | | #pkts | ------- | 2 | => pkt_list | status | | #pkts | ------- | .... | => pkt_list ------- Signed-off-by: Elena Agostini --- app/test-gpudev/main.c | 95 ++++++++++++++ doc/guides/prog_guide/gpudev.rst | 16 +++ doc/guides/rel_notes/release_21_11.rst | 2 +- lib/gpudev/gpudev.c | 164 +++++++++++++++++++++++++ lib/gpudev/meson.build | 2 + lib/gpudev/rte_gpudev.h | 129 +++++++++++++++++++ lib/gpudev/version.map | 4 + 7 files changed, 411 insertions(+), 1 deletion(-) diff --git a/app/test-gpudev/main.c b/app/test-gpudev/main.c index 516a01b927..111ed6d415 100644 --- a/app/test-gpudev/main.c +++ b/app/test-gpudev/main.c @@ -209,6 +209,100 @@ create_update_comm_flag(uint16_t gpu_id) return 0; } +static int +simulate_gpu_task(struct rte_gpu_comm_list *comm_list_item, int num_pkts) +{ + int idx; + + if (comm_list_item == NULL) + return -1; + + for (idx = 0; idx < num_pkts; idx++) { + /** + * consume(comm_list_item->pkt_list[idx].addr); + */ + } + comm_list_item->status = RTE_GPU_COMM_LIST_DONE; + + return 0; +} + +static int +create_update_comm_list(uint16_t gpu_id) +{ + int ret = 0; + int i = 0; + struct rte_gpu_comm_list *comm_list; + uint32_t num_comm_items = 1024; + struct rte_mbuf *mbufs[10]; + + printf("\n=======> TEST: Communication list\n"); + + comm_list = rte_gpu_comm_create_list(gpu_id, num_comm_items); + if (comm_list == NULL) { + fprintf(stderr, "rte_gpu_comm_create_list returned error %d\n", ret); + return -1; + } + + /** + * Simulate DPDK receive functions like rte_eth_rx_burst() + */ + for (i = 0; i < 10; i++) { + mbufs[i] = rte_zmalloc(NULL, sizeof(struct rte_mbuf), 0); + if (mbufs[i] == NULL) { + fprintf(stderr, "Failed to allocate fake mbufs in CPU memory.\n"); + return -1; + } + + memset(mbufs[i], 0, sizeof(struct rte_mbuf)); + } + + /** + * Populate just the first item of the list + */ + ret = rte_gpu_comm_populate_list_pkts(&(comm_list[0]), mbufs, 10); + if (ret < 0) { + fprintf(stderr, "rte_gpu_comm_populate_list_pkts returned error %d\n", ret); + return -1; + } + + ret = rte_gpu_comm_cleanup_list(&(comm_list[0])); + if (ret == 0) { + fprintf(stderr, "rte_gpu_comm_cleanup_list erroneusly cleaned the list even if packets have not beeing consumed yet\n"); + return -1; + } else { + fprintf(stderr, "rte_gpu_comm_cleanup_list correctly didn't clean up the packets because they have not beeing consumed yet\n"); + } + + /** + * Simulate a GPU tasks going through the packet list to consume + * mbufs packets and release them + */ + simulate_gpu_task(&(comm_list[0]), 10); + + /** + * Packets have been consumed, now the communication item + * and the related mbufs can be all released + */ + ret = rte_gpu_comm_cleanup_list(&(comm_list[0])); + if (ret < 0) { + fprintf(stderr, "rte_gpu_comm_cleanup_list returned error %d\n", ret); + return -1; + } + + ret = rte_gpu_comm_destroy_list(comm_list, num_comm_items); + if (ret < 0) { + fprintf(stderr, "rte_gpu_comm_destroy_list returned error %d\n", ret); + return -1; + } + + for (i = 0; i < 10; i++) + rte_free(mbufs[i]); + + printf("\nCommunication list test passed!\n"); + return 0; +} + int main(int argc, char **argv) { @@ -263,6 +357,7 @@ main(int argc, char **argv) * Communication items test */ create_update_comm_flag(gpu_id); + create_update_comm_list(gpu_id); /* clean up the EAL */ rte_eal_cleanup(); diff --git a/doc/guides/prog_guide/gpudev.rst b/doc/guides/prog_guide/gpudev.rst index e0db627aed..cbaec5a1e4 100644 --- a/doc/guides/prog_guide/gpudev.rst +++ b/doc/guides/prog_guide/gpudev.rst @@ -86,3 +86,19 @@ that's waiting to receive a signal from the CPU to move forward with the execution. The communication flag allocates a CPU memory GPU-visible ``uint32_t`` flag that can be used by the CPU to communicate with a GPU task. + +Communication list +~~~~~~~~~~~~~~~~~~ + +By default, DPDK pulls free mbufs from a mempool to receive packets. +Best practice, expecially in a multithreaded application, +is to no make any assumption on which mbufs will be used +to receive the next bursts of packets. +Considering an application with a GPU memory mempool +attached to a receive queue having some task waiting on the GPU +to receive a new burst of packets to be processed, +there is the need to communicate from the CPU +the list of mbuf payload addresses where received packet have been stored. +The ``rte_gpu_comm_*()`` functions are responsible to create a list of packets +that can be populated with receive mbuf payload addresses +and communicated to the task running on the GPU. diff --git a/doc/guides/rel_notes/release_21_11.rst b/doc/guides/rel_notes/release_21_11.rst index 78b29d9a25..23d8591f40 100644 --- a/doc/guides/rel_notes/release_21_11.rst +++ b/doc/guides/rel_notes/release_21_11.rst @@ -105,7 +105,7 @@ New Features * Device information * Memory management - * Communication flag + * Communication flag & list * **Added new RSS offload types for IPv4/L4 checksum in RSS flow.** diff --git a/lib/gpudev/gpudev.c b/lib/gpudev/gpudev.c index f887f3dd93..88148eb704 100644 --- a/lib/gpudev/gpudev.c +++ b/lib/gpudev/gpudev.c @@ -735,3 +735,167 @@ rte_gpu_comm_get_flag_value(struct rte_gpu_comm_flag *devflag, uint32_t *val) return 0; } + +struct rte_gpu_comm_list * +rte_gpu_comm_create_list(uint16_t dev_id, + uint32_t num_comm_items) +{ + struct rte_gpu_comm_list *comm_list; + uint32_t idx_l; + int ret; + struct rte_gpu *dev; + + if (num_comm_items == 0) { + rte_errno = EINVAL; + return NULL; + } + + dev = gpu_get_by_id(dev_id); + if (dev == NULL) { + GPU_LOG(ERR, "memory barrier for invalid device ID %d", dev_id); + rte_errno = ENODEV; + return NULL; + } + + comm_list = rte_zmalloc(NULL, sizeof(struct rte_gpu_comm_list) * num_comm_items, 0); + if (comm_list == NULL) { + rte_errno = ENOMEM; + return NULL; + } + + ret = rte_gpu_register(dev_id, sizeof(struct rte_gpu_comm_list) * num_comm_items, comm_list); + if (ret < 0) { + rte_errno = ENOMEM; + return NULL; + } + + for (idx_l = 0; idx_l < num_comm_items; idx_l++) { + comm_list[idx_l].pkt_list = rte_zmalloc(NULL, sizeof(struct rte_gpu_comm_pkt) * RTE_GPU_COMM_LIST_PKTS_MAX, 0); + if (comm_list[idx_l].pkt_list == NULL) { + rte_errno = ENOMEM; + return NULL; + } + + ret = rte_gpu_register(dev_id, sizeof(struct rte_gpu_comm_pkt) * RTE_GPU_COMM_LIST_PKTS_MAX, comm_list[idx_l].pkt_list); + if (ret < 0) { + rte_errno = ENOMEM; + return NULL; + } + + RTE_GPU_VOLATILE(comm_list[idx_l].status) = RTE_GPU_COMM_LIST_FREE; + comm_list[idx_l].num_pkts = 0; + comm_list[idx_l].dev_id = dev_id; + + comm_list[idx_l].mbufs = rte_zmalloc(NULL, sizeof(struct rte_mbuf *) * RTE_GPU_COMM_LIST_PKTS_MAX, 0); + if (comm_list[idx_l].mbufs == NULL) { + rte_errno = ENOMEM; + return NULL; + } + } + + return comm_list; +} + +int +rte_gpu_comm_destroy_list(struct rte_gpu_comm_list *comm_list, + uint32_t num_comm_items) +{ + uint32_t idx_l; + int ret; + uint16_t dev_id; + + if (comm_list == NULL) { + rte_errno = EINVAL; + return -rte_errno; + } + + dev_id = comm_list[0].dev_id; + + for (idx_l = 0; idx_l < num_comm_items; idx_l++) { + ret = rte_gpu_unregister(dev_id, comm_list[idx_l].pkt_list); + if (ret < 0) { + rte_errno = EINVAL; + return -1; + } + + rte_free(comm_list[idx_l].pkt_list); + rte_free(comm_list[idx_l].mbufs); + } + + ret = rte_gpu_unregister(dev_id, comm_list); + if (ret < 0) { + rte_errno = EINVAL; + return -1; + } + + rte_free(comm_list); + + return 0; +} + +int +rte_gpu_comm_populate_list_pkts(struct rte_gpu_comm_list *comm_list_item, + struct rte_mbuf **mbufs, uint32_t num_mbufs) +{ + uint32_t idx; + + if (comm_list_item == NULL || comm_list_item->pkt_list == NULL || + mbufs == NULL || num_mbufs > RTE_GPU_COMM_LIST_PKTS_MAX) { + rte_errno = EINVAL; + return -rte_errno; + } + + for (idx = 0; idx < num_mbufs; idx++) { + /* support only unchained mbufs */ + if (unlikely((mbufs[idx]->nb_segs > 1) || + (mbufs[idx]->next != NULL) || + (mbufs[idx]->data_len != mbufs[idx]->pkt_len))) { + rte_errno = ENOTSUP; + return -rte_errno; + } + comm_list_item->pkt_list[idx].addr = + rte_pktmbuf_mtod_offset(mbufs[idx], uintptr_t, 0); + comm_list_item->pkt_list[idx].size = mbufs[idx]->pkt_len; + comm_list_item->mbufs[idx] = mbufs[idx]; + } + + RTE_GPU_VOLATILE(comm_list_item->num_pkts) = num_mbufs; + rte_gpu_mbw(comm_list_item->dev_id); + RTE_GPU_VOLATILE(comm_list_item->status) = RTE_GPU_COMM_LIST_READY; + rte_gpu_mbw(comm_list_item->dev_id); + + return 0; +} + +int +rte_gpu_comm_cleanup_list(struct rte_gpu_comm_list *comm_list_item) +{ + uint32_t idx = 0; + + if (comm_list_item == NULL) { + rte_errno = EINVAL; + return -rte_errno; + } + + if (RTE_GPU_VOLATILE(comm_list_item->status) == + RTE_GPU_COMM_LIST_READY) { + GPU_LOG(ERR, "packet list is still in progress"); + rte_errno = EINVAL; + return -rte_errno; + } + + for (idx = 0; idx < RTE_GPU_COMM_LIST_PKTS_MAX; idx++) { + if (comm_list_item->pkt_list[idx].addr == 0) + break; + + comm_list_item->pkt_list[idx].addr = 0; + comm_list_item->pkt_list[idx].size = 0; + comm_list_item->mbufs[idx] = NULL; + } + + RTE_GPU_VOLATILE(comm_list_item->status) = RTE_GPU_COMM_LIST_FREE; + RTE_GPU_VOLATILE(comm_list_item->num_pkts) = 0; + rte_mb(); + + return 0; +} diff --git a/lib/gpudev/meson.build b/lib/gpudev/meson.build index 608154817b..89a118f357 100644 --- a/lib/gpudev/meson.build +++ b/lib/gpudev/meson.build @@ -8,3 +8,5 @@ headers = files( sources = files( 'gpudev.c', ) + +deps += ['mbuf'] diff --git a/lib/gpudev/rte_gpudev.h b/lib/gpudev/rte_gpudev.h index 1466ac164b..3023154be8 100644 --- a/lib/gpudev/rte_gpudev.h +++ b/lib/gpudev/rte_gpudev.h @@ -9,6 +9,7 @@ #include #include +#include #include #include @@ -41,6 +42,9 @@ extern "C" { /** Access variable as volatile. */ #define RTE_GPU_VOLATILE(x) (*(volatile typeof(x) *)&(x)) +/** Max number of packets per communication list. */ +#define RTE_GPU_COMM_LIST_PKTS_MAX 1024 + /** Store device info. */ struct rte_gpu_info { /** Unique identifier name. */ @@ -87,6 +91,43 @@ struct rte_gpu_comm_flag { enum rte_gpu_comm_flag_type mtype; }; +/** List of packets shared among CPU and device. */ +struct rte_gpu_comm_pkt { + /** Address of the packet in memory (e.g. mbuf->buf_addr). */ + uintptr_t addr; + /** Size in byte of the packet. */ + size_t size; +}; + +/** Possible status for the list of packets shared among CPU and device. */ +enum rte_gpu_comm_list_status { + /** Packet list can be filled with new mbufs, no one is using it. */ + RTE_GPU_COMM_LIST_FREE = 0, + /** Packet list has been filled with new mbufs and it's ready to be used .*/ + RTE_GPU_COMM_LIST_READY, + /** Packet list has been processed, it's ready to be freed. */ + RTE_GPU_COMM_LIST_DONE, + /** Some error occurred during packet list processing. */ + RTE_GPU_COMM_LIST_ERROR, +}; + +/** + * Communication list holding a number of lists of packets + * each having a status flag. + */ +struct rte_gpu_comm_list { + /** Device that will use the communication list. */ + uint16_t dev_id; + /** List of mbufs populated by the CPU with a set of mbufs. */ + struct rte_mbuf **mbufs; + /** List of packets populated by the CPU with a set of mbufs info. */ + struct rte_gpu_comm_pkt *pkt_list; + /** Number of packets in the list. */ + uint32_t num_pkts; + /** Status of the list. */ + enum rte_gpu_comm_list_status status; +}; + /** * @warning * @b EXPERIMENTAL: this API may change without prior notice. @@ -513,6 +554,94 @@ __rte_experimental int rte_gpu_comm_get_flag_value(struct rte_gpu_comm_flag *devflag, uint32_t *val); +/** + * @warning + * @b EXPERIMENTAL: this API may change without prior notice. + * + * Create a communication list that can be used to share packets + * between CPU and device. + * Each element of the list contains: + * - a packet list of RTE_GPU_COMM_LIST_PKTS_MAX elements + * - number of packets in the list + * - a status flag to communicate if the packet list is FREE, + * READY to be processed, DONE with processing. + * + * The list is allocated in CPU-visible memory. + * At creation time, every list is in FREE state. + * + * @param dev_id + * Reference device ID. + * @param num_comm_items + * Number of items in the communication list. + * + * @return + * A pointer to the allocated list, otherwise NULL and rte_errno is set: + * - EINVAL if invalid input params + */ +__rte_experimental +struct rte_gpu_comm_list *rte_gpu_comm_create_list(uint16_t dev_id, + uint32_t num_comm_items); + +/** + * @warning + * @b EXPERIMENTAL: this API may change without prior notice. + * + * Destroy a communication list. + * + * @param comm_list + * Communication list to be destroyed. + * @param num_comm_items + * Number of items in the communication list. + * + * @return + * 0 on success, -rte_errno otherwise: + * - EINVAL if invalid input params + */ +__rte_experimental +int rte_gpu_comm_destroy_list(struct rte_gpu_comm_list *comm_list, + uint32_t num_comm_items); + +/** + * @warning + * @b EXPERIMENTAL: this API may change without prior notice. + * + * Populate the packets list of the communication item + * with info from a list of mbufs. + * Status flag of that packet list is set to READY. + * + * @param comm_list_item + * Communication list item to fill. + * @param mbufs + * List of mbufs. + * @param num_mbufs + * Number of mbufs. + * + * @return + * 0 on success, -rte_errno otherwise: + * - EINVAL if invalid input params + * - ENOTSUP if mbufs are chained (multiple segments) + */ +__rte_experimental +int rte_gpu_comm_populate_list_pkts(struct rte_gpu_comm_list *comm_list_item, + struct rte_mbuf **mbufs, uint32_t num_mbufs); + +/** + * @warning + * @b EXPERIMENTAL: this API may change without prior notice. + * + * Reset a communication list item to the original state. + * The status flag set to FREE and mbufs are returned to the pool. + * + * @param comm_list_item + * Communication list item to reset. + * + * @return + * 0 on success, -rte_errno otherwise: + * - EINVAL if invalid input params + */ +__rte_experimental +int rte_gpu_comm_cleanup_list(struct rte_gpu_comm_list *comm_list_item); + #ifdef __cplusplus } #endif diff --git a/lib/gpudev/version.map b/lib/gpudev/version.map index 2fc039373a..45a35fa6e4 100644 --- a/lib/gpudev/version.map +++ b/lib/gpudev/version.map @@ -6,9 +6,13 @@ EXPERIMENTAL { rte_gpu_callback_register; rte_gpu_callback_unregister; rte_gpu_close; + rte_gpu_comm_cleanup_list; rte_gpu_comm_create_flag; + rte_gpu_comm_create_list; rte_gpu_comm_destroy_flag; + rte_gpu_comm_destroy_list; rte_gpu_comm_get_flag_value; + rte_gpu_comm_populate_list_pkts; rte_gpu_comm_set_flag; rte_gpu_count_avail; rte_gpu_find_next; From patchwork Mon Nov 8 18:58:05 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Elena Agostini X-Patchwork-Id: 103979 X-Patchwork-Delegate: thomas@monjalon.net Return-Path: X-Original-To: patchwork@inbox.dpdk.org Delivered-To: patchwork@inbox.dpdk.org Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id 4DBD1A0C4D; Mon, 8 Nov 2021 11:48:15 +0100 (CET) Received: from [217.70.189.124] (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id 8FEE241160; Mon, 8 Nov 2021 11:47:16 +0100 (CET) Received: from NAM12-DM6-obe.outbound.protection.outlook.com (mail-dm6nam12on2082.outbound.protection.outlook.com [40.107.243.82]) by mails.dpdk.org (Postfix) with ESMTP id 2206D41135 for ; Mon, 8 Nov 2021 11:47:12 +0100 (CET) ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=YcLK8JYYE23Sf384Hufl3pkOdhu/q7UI5dTd0vzpteNV8IfYScUzfOcDhjbSgK8AZEHZ8ubXjOVDs71neViEh5p6I0uAv3ZrJGWeyAxKbWJCWr2+T36NZ9Z88TSueUDoyR0creJZZHnuuAdGM6sMy0pE4VTnhe/aHEdp7/OYlEnCDZYemRrfycWrnxgesgfMd2sfGr/+vc2Lqxn6QwOmxL/kD9X0BZwrBUntAxvP7lOjkmCHFQSszK9pXUBj1VaOKYwVj6zNuA9DpuJ4n4AAleQ0uXUqd0ZOwIV0otAmGxPJwcqT54+DQ+TdMR+wCMcXTQclqg8ymJ8z8VHAJqapug== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=OxtiHhow+oMMccuwlq9p6l5sIJT9EIjR/wpVQcG82Kw=; b=ab0b/L2cjd7jwKCx2i+8/ZLSTTodqJ3S23PKykTRaxlPULHn2L/0VUM5xjfz4qm/uAacJManN1+edSbYkbFok+XJEoxxeAh9UO/aMJpnSFsv5Wdg72vmrWcf/6Rsm62lcZyWPJsZfFhgwQOpI6aO9sSYTLGXJrkPAvbG5taQ9DaXUojtCDLl49fqW60yUZHSAiXR5WMxZazffEnZF/yJOBAe8I6Y90M7Xk0WDHbn9xXScA3yk6OngiS0u8+8ouZwufQXyRhEnRklaot5EWOaMzvlktgBZGWI4JY4k51e3Sq4JAyaG759Ga3z9s91d562S3YThLiviqxjSc0kd6WH+g== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass (sender ip is 216.228.112.34) smtp.rcpttodomain=dpdk.org smtp.mailfrom=nvidia.com; dmarc=pass (p=quarantine sp=quarantine pct=100) action=none header.from=nvidia.com; dkim=none (message not signed); arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=Nvidia.com; s=selector2; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=OxtiHhow+oMMccuwlq9p6l5sIJT9EIjR/wpVQcG82Kw=; b=mNQ2s0gO3/+0HlGrzjc1TY2N9fVZPOscc/+TkqbfzOdnLvMvdkU3LDLAvU+3VQ4JoAu6/eBxZonKw4FoKB32Iv40kLr2aqhokHzSCczM6R1j+O3SlevIiOtUp7Msn6OYAMC88bB4SSkhdbkvB4o+YFaBu/h1h95qOLLeIYqiqYoqPYOGnOVmEASS/DVdaoEH+15tFZ7Imup37Vnn9ARLg52rEwMiPP1bq2UDFx1CvIFHswY1Dw8mRQLN73P0N7tUFgMAQathP3Z0UZHYOhT2YgBWQQdCMjG7yu0u/WCsAlHwOp1NsSWn4apk6Z28cxjkqLAZV1VBO1e/HZ4/KhcGFQ== Received: from MW2PR16CA0052.namprd16.prod.outlook.com (2603:10b6:907:1::29) by MN2PR12MB4286.namprd12.prod.outlook.com (2603:10b6:208:199::22) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4669.11; Mon, 8 Nov 2021 10:47:11 +0000 Received: from CO1NAM11FT064.eop-nam11.prod.protection.outlook.com (2603:10b6:907:1:cafe::1f) by MW2PR16CA0052.outlook.office365.com (2603:10b6:907:1::29) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4669.10 via Frontend Transport; Mon, 8 Nov 2021 10:47:10 +0000 X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 216.228.112.34) smtp.mailfrom=nvidia.com; dpdk.org; dkim=none (message not signed) header.d=none;dpdk.org; dmarc=pass action=none header.from=nvidia.com; Received-SPF: Pass (protection.outlook.com: domain of nvidia.com designates 216.228.112.34 as permitted sender) receiver=protection.outlook.com; client-ip=216.228.112.34; helo=mail.nvidia.com; Received: from mail.nvidia.com (216.228.112.34) by CO1NAM11FT064.mail.protection.outlook.com (10.13.175.77) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA384) id 15.20.4669.10 via Frontend Transport; Mon, 8 Nov 2021 10:47:10 +0000 Received: from nvidia.com (172.20.187.6) by HQMAIL107.nvidia.com (172.20.187.13) with Microsoft SMTP Server (TLS) id 15.0.1497.18; Mon, 8 Nov 2021 10:47:02 +0000 From: To: CC: Elena Agostini Date: Mon, 8 Nov 2021 18:58:05 +0000 Message-ID: <20211108185805.3887-10-eagostini@nvidia.com> X-Mailer: git-send-email 2.17.1 In-Reply-To: <20211108185805.3887-1-eagostini@nvidia.com> References: <20210602203531.2288645-1-thomas@monjalon.net> <20211108185805.3887-1-eagostini@nvidia.com> MIME-Version: 1.0 X-Originating-IP: [172.20.187.6] X-ClientProxiedBy: HQMAIL105.nvidia.com (172.20.187.12) To HQMAIL107.nvidia.com (172.20.187.13) X-EOPAttributedMessage: 0 X-MS-PublicTrafficType: Email X-MS-Office365-Filtering-Correlation-Id: e9d997b5-059e-4539-3f2f-08d9a2a51de7 X-MS-TrafficTypeDiagnostic: MN2PR12MB4286: X-Microsoft-Antispam-PRVS: X-MS-Oob-TLC-OOBClassifiers: OLM:6430; X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: x5lmRAxraCZ+CvfCzB+rfg3862p9FIyBYqB/2I9c097MhlHqMxr38elTNATXFwrCxlfKHFXWGyUDe7/P+NkmqU3dIwSaDzp1Lc7ntPj0OrDEoJFaop2febJ+z38+qST2THS0077k1MqgrqX8cG2QjHecbfr2SwzDwWNYuf6mIYD46cXKiksF8zOiYsU+G3njTq27VnFhhX2JKh3M0fbFtfXBTNAALJ8jKrGOHx4XXZ4revOdIM3CWVUMEHEmGUxXE6lmfCnOj3rNlBcUfeIJXfq5ck5rZRftsybxCiUP2/lS16rFyiTUmhnNbt77UU8wS0d5/xeSS3RWedTOwAEjeyPUNuvXZ7l+fy3GeJ/FoFKn/JBgXnmzjPExnD7MA5sqWKwWX2rdH1zTOkahZko7M34zdFSJEFwAPa7uPfNlwCPKfsTKFFRvNlzwuY2FW4Dqw2N2ul2UNdH6pg0ZF8t7kGNMHfXcseQbbkEEYG2rFHurm8wVN+Tkw/8vDgU5Wd8XbUEJ8VR7h+dWHKLxPthlRqsZ9A51v6t21DXFfndlMMrEPjx15OXehRFoPSQXdIAMTmwGWmrFrrEQxqmV+YBtNPzCSle+u2x24tdSHB7kJfyTNJVQDBDSnbKrpcxfHsXOALYSQLp8gyaqK7ZvJ5h7kz6TsDeLFpz/2vLjnRHg+KbxwApSumAm4C0/aqgzjP4mkEFybkP8eUNOj8UaY+OE9A== X-Forefront-Antispam-Report: CIP:216.228.112.34; CTRY:US; LANG:en; SCL:1; SRV:; IPV:NLI; SFV:NSPM; H:mail.nvidia.com; PTR:schybrid03.nvidia.com; CAT:NONE; SFS:(4636009)(46966006)(36840700001)(2876002)(336012)(55016002)(86362001)(7696005)(36860700001)(16526019)(186003)(2616005)(8936002)(6916009)(426003)(36756003)(82310400003)(26005)(316002)(8676002)(70586007)(70206006)(508600001)(83380400001)(47076005)(356005)(1076003)(7636003)(6286002)(2906002)(107886003)(4326008)(5660300002); DIR:OUT; SFP:1101; X-OriginatorOrg: Nvidia.com X-MS-Exchange-CrossTenant-OriginalArrivalTime: 08 Nov 2021 10:47:10.3554 (UTC) X-MS-Exchange-CrossTenant-Network-Message-Id: e9d997b5-059e-4539-3f2f-08d9a2a51de7 X-MS-Exchange-CrossTenant-Id: 43083d15-7273-40c1-b7db-39efd9ccc17a X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=43083d15-7273-40c1-b7db-39efd9ccc17a; Ip=[216.228.112.34]; Helo=[mail.nvidia.com] X-MS-Exchange-CrossTenant-AuthSource: CO1NAM11FT064.eop-nam11.prod.protection.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Anonymous X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem X-MS-Exchange-Transport-CrossTenantHeadersStamped: MN2PR12MB4286 Subject: [dpdk-dev] [PATCH v5 9/9] doc: add CUDA example in GPU guide X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Sender: "dev" From: Elena Agostini Signed-off-by: Elena Agostini --- doc/guides/prog_guide/gpudev.rst | 122 +++++++++++++++++++++++++++++++ 1 file changed, 122 insertions(+) diff --git a/doc/guides/prog_guide/gpudev.rst b/doc/guides/prog_guide/gpudev.rst index cbaec5a1e4..1baf0c6772 100644 --- a/doc/guides/prog_guide/gpudev.rst +++ b/doc/guides/prog_guide/gpudev.rst @@ -102,3 +102,125 @@ the list of mbuf payload addresses where received packet have been stored. The ``rte_gpu_comm_*()`` functions are responsible to create a list of packets that can be populated with receive mbuf payload addresses and communicated to the task running on the GPU. + + +CUDA Example +------------ + +In the example below, there is a pseudo-code to give an example +about how to use functions in this library in case of a CUDA application. + +.. code-block:: c + + ////////////////////////////////////////////////////////////////////////// + ///// gpudev library + CUDA functions + ////////////////////////////////////////////////////////////////////////// + #define GPU_PAGE_SHIFT 16 + #define GPU_PAGE_SIZE (1UL << GPU_PAGE_SHIFT) + + int main() { + struct rte_gpu_flag quit_flag; + struct rte_gpu_comm_list *comm_list; + int nb_rx = 0; + int comm_list_entry = 0; + struct rte_mbuf * rx_mbufs[max_rx_mbufs]; + cudaStream_t cstream; + struct rte_mempool *mpool_payload, *mpool_header; + struct rte_pktmbuf_extmem ext_mem; + int16_t dev_id; + int16_t port_id = 0; + + /** Initialize CUDA objects (cstream, context, etc..). */ + /** Use gpudev library to register a new CUDA context if any */ + /** Let's assume the application wants to use the default context of the GPU device 0 */ + + dev_id = 0; + + /** + * Create an external memory mempool using memory allocated on the GPU. + */ + ext_mem.elt_size = mbufs_headroom_size; + ext_mem.buf_len = RTE_ALIGN_CEIL(mbufs_num * ext_mem.elt_size, GPU_PAGE_SIZE); + ext_mem.buf_iova = RTE_BAD_IOVA; + ext_mem.buf_ptr = rte_gpu_malloc(dev_id, ext_mem.buf_len, 0); + rte_extmem_register(ext_mem.buf_ptr, ext_mem.buf_len, NULL, ext_mem.buf_iova, GPU_PAGE_SIZE); + rte_dev_dma_map(rte_eth_devices[port_id].device, ext_mem.buf_ptr, ext_mem.buf_iova, ext_mem.buf_len); + mpool_payload = rte_pktmbuf_pool_create_extbuf("gpu_mempool", mbufs_num, + 0, 0, ext_mem.elt_size, + rte_socket_id(), &ext_mem, 1); + + /** + * Create CPU - device communication flag. With this flag, the CPU can tell to the CUDA kernel + * to exit from the main loop. + */ + rte_gpu_comm_create_flag(dev_id, &quit_flag, RTE_GPU_COMM_FLAG_CPU); + rte_gpu_comm_set_flag(&quit_flag , 0); + + /** + * Create CPU - device communication list. Each entry of this list will be populated by the CPU + * with a new set of received mbufs that the CUDA kernel has to process. + */ + comm_list = rte_gpu_comm_create_list(dev_id, num_entries); + + /** A very simple CUDA kernel with just 1 CUDA block and RTE_GPU_COMM_LIST_PKTS_MAX CUDA threads. */ + cuda_kernel_packet_processing<<<1, RTE_GPU_COMM_LIST_PKTS_MAX, 0, cstream>>>(quit_flag->ptr, comm_list, num_entries, ...); + + /** + * For simplicity, the CPU here receives only 2 bursts of mbufs. + * In a real application, network activity and device processing should overlap. + */ + nb_rx = rte_eth_rx_burst(port_id, queue_id, &(rx_mbufs[0]), max_rx_mbufs); + rte_gpu_comm_populate_list_pkts(comm_list[0], rx_mbufs, nb_rx); + nb_rx = rte_eth_rx_burst(port_id, queue_id, &(rx_mbufs[0]), max_rx_mbufs); + rte_gpu_comm_populate_list_pkts(comm_list[1], rx_mbufs, nb_rx); + + /** + * CPU waits for the completion of the packets' processing on the CUDA kernel + * and then it does a cleanup of the received mbufs. + */ + while(rte_gpu_comm_cleanup_list(comm_list[0])); + while(rte_gpu_comm_cleanup_list(comm_list[1])); + + /** CPU notifies the CUDA kernel that it has to terminate */ + rte_gpu_comm_set_flag(&quit_flag, 1); + + /** gpudev objects cleanup/destruction */ + /** CUDA cleanup */ + + rte_gpu_free(dev_id, ext_mem.buf_len); + + /** DPDK cleanup */ + + return 0; + } + + ////////////////////////////////////////////////////////////////////////// + ///// CUDA kernel + ////////////////////////////////////////////////////////////////////////// + + void cuda_kernel(uint32_t * quit_flag_ptr, struct rte_gpu_comm_list *comm_list, int comm_list_entries) { + int comm_list_index = 0; + struct rte_gpu_comm_pkt *pkt_list = NULL; + + /** Do some pre-processing operations. */ + + /** GPU kernel keeps checking this flag to know if it has to quit or wait for more packets. */ + while(*quit_flag_ptr == 0) + { + if(comm_list[comm_list_index]->status != RTE_GPU_COMM_LIST_READY) + continue; + + if(threadIdx.x < comm_list[comm_list_index]->num_pkts) + { + /** Each CUDA thread processes a different packet. */ + packet_processing(comm_list[comm_list_index]->addr, comm_list[comm_list_index]->size, ..); + } + __threadfence(); + __syncthreads(); + + /** Wait for new packets on the next communication list entry. */ + comm_list_index = (comm_list_index+1) % comm_list_entries; + } + + /** Do some post-processing operations. */ + }