From patchwork Fri Oct 7 19:30:25 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Shijith Thotton X-Patchwork-Id: 117633 X-Patchwork-Delegate: thomas@monjalon.net Return-Path: X-Original-To: patchwork@inbox.dpdk.org Delivered-To: patchwork@inbox.dpdk.org Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id DE860A0542; Fri, 7 Oct 2022 21:31:10 +0200 (CEST) Received: from [217.70.189.124] (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id D135A4280B; Fri, 7 Oct 2022 21:31:10 +0200 (CEST) Received: from mx0b-0016f401.pphosted.com (mx0b-0016f401.pphosted.com [67.231.156.173]) by mails.dpdk.org (Postfix) with ESMTP id F339D42802 for ; Fri, 7 Oct 2022 21:31:08 +0200 (CEST) Received: from pps.filterd (m0045851.ppops.net [127.0.0.1]) by mx0b-0016f401.pphosted.com (8.17.1.5/8.17.1.5) with ESMTP id 297FCib5016019; Fri, 7 Oct 2022 12:31:02 -0700 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=marvell.com; h=from : to : cc : subject : date : message-id : in-reply-to : references : mime-version : content-transfer-encoding : content-type; s=pfpt0220; bh=JIFQ3a1AWCTzY5YSuNss6CNlppLjteVxMp1vKKXjzG4=; b=crRjG4F2AArOJ+qksF02bi6bQj9troCbVeQ2NxXYjP2M3NaBIOjfojdclNKrhWhwrg/V ElqaJN5+7lHJv8F/5qVnQxIAVaLm1IglJik/7g/7xZSZr63xp4GBP7QhA7vsSIkQzNHi fcT3wsfus0rC3Cba9JvYpukXqiDyCkvBnFdRuvJGXaM6HBMEn4ErKH60SQkVVAXq1JbU nLSkNJoLr61UnNlM0Q2doSTxZHZqr+yTTE9o7IiKrpAlj6g1gKxvPaLf5wpNzS26ore6 kUhgWbEuISaexeWree7+el9Fj90lgM7TmL3K8HeV2E363Y9NsD4jYJ8nUfg6NRSwM0iG OQ== Received: from dc5-exch02.marvell.com ([199.233.59.182]) by mx0b-0016f401.pphosted.com (PPS) with ESMTPS id 3k1d7gsjjt-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-SHA384 bits=256 verify=NOT); Fri, 07 Oct 2022 12:31:02 -0700 Received: from DC5-EXCH01.marvell.com (10.69.176.38) by DC5-EXCH02.marvell.com (10.69.176.39) with Microsoft SMTP Server (TLS) id 15.0.1497.18; Fri, 7 Oct 2022 12:30:59 -0700 Received: from maili.marvell.com (10.69.176.80) by DC5-EXCH01.marvell.com (10.69.176.38) with Microsoft SMTP Server id 15.0.1497.2 via Frontend Transport; Fri, 7 Oct 2022 12:30:59 -0700 Received: from localhost.localdomain (unknown [10.28.34.29]) by maili.marvell.com (Postfix) with ESMTP id ADFA43F7051; Fri, 7 Oct 2022 12:30:54 -0700 (PDT) From: Shijith Thotton To: CC: Shijith Thotton , , , , , , , , , , Dongdong Liu , "Yisen Zhuang" Subject: [PATCH v4 3/7] build: add meson option to configure IOVA mode as PA Date: Sat, 8 Oct 2022 01:00:25 +0530 Message-ID: <3ef3f4ac3ce055b45bd6a10a92ffa7e0e594e62e.1665170500.git.sthotton@marvell.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: References: MIME-Version: 1.0 X-Proofpoint-GUID: HrEkVpbYqRxdcGtBodNkm9qdGT1F0mj5 X-Proofpoint-ORIG-GUID: HrEkVpbYqRxdcGtBodNkm9qdGT1F0mj5 X-Proofpoint-Virus-Version: vendor=baseguard engine=ICAP:2.0.205,Aquarius:18.0.895,Hydra:6.0.528,FMLib:17.11.122.1 definitions=2022-10-07_04,2022-10-07_01,2022-06-22_01 X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org IOVA mode in DPDK is either PA or VA. The new build option enable_iova_as_pa configures the mode to PA at compile time. By default, this option is enabled. If the options is disabled, only drivers which supports it are enabled during build. Supported driver can set the flag pmd_supports_disable_iova_as_pa in its build file. mbuf structure holds the physical (PA) and virtual address (VA) of a buffer. if IOVA as PA is disabled at compile time, PA field (buf_iova) of mbuf is redundant as it is the same as VA and is replaced by a dummy field. Signed-off-by: Shijith Thotton --- app/test/test_mbuf.c | 12 +++++++----- config/meson.build | 1 + drivers/meson.build | 6 ++++++ drivers/net/hns3/meson.build | 6 ++++++ lib/eal/linux/eal.c | 6 ++++++ lib/mbuf/rte_mbuf.c | 2 +- lib/mbuf/rte_mbuf.h | 9 +++++++++ lib/mbuf/rte_mbuf_core.h | 6 ++++++ lib/meson.build | 3 +++ meson_options.txt | 2 ++ 10 files changed, 47 insertions(+), 6 deletions(-) diff --git a/app/test/test_mbuf.c b/app/test/test_mbuf.c index 22e45e66c1..2d66786ace 100644 --- a/app/test/test_mbuf.c +++ b/app/test/test_mbuf.c @@ -1232,11 +1232,13 @@ test_failing_mbuf_sanity_check(struct rte_mempool *pktmbuf_pool) return -1; } - badbuf = *buf; - rte_mbuf_iova_set(&badbuf, 0); - if (verify_mbuf_check_panics(&badbuf)) { - printf("Error with bad-physaddr mbuf test\n"); - return -1; + if (RTE_IOVA_AS_PA) { + badbuf = *buf; + rte_mbuf_iova_set(&badbuf, 0); + if (verify_mbuf_check_panics(&badbuf)) { + printf("Error with bad-physaddr mbuf test\n"); + return -1; + } } badbuf = *buf; diff --git a/config/meson.build b/config/meson.build index 7f7b6c92fd..0fc209db01 100644 --- a/config/meson.build +++ b/config/meson.build @@ -309,6 +309,7 @@ endif if get_option('mbuf_refcnt_atomic') dpdk_conf.set('RTE_MBUF_REFCNT_ATOMIC', true) endif +dpdk_conf.set10('RTE_IOVA_AS_PA', get_option('enable_iova_as_pa')) compile_time_cpuflags = [] subdir(arch_subdir) diff --git a/drivers/meson.build b/drivers/meson.build index f6ba5ba4fb..2a29c210b5 100644 --- a/drivers/meson.build +++ b/drivers/meson.build @@ -106,6 +106,7 @@ foreach subpath:subdirs ext_deps = [] pkgconfig_extra_libs = [] testpmd_sources = [] + pmd_supports_disable_iova_as_pa = false if not enable_drivers.contains(drv_path) build = false @@ -123,6 +124,11 @@ foreach subpath:subdirs # pull in driver directory which should update all the local variables subdir(drv_path) + if dpdk_conf.get('RTE_IOVA_AS_PA') == 0 and not pmd_supports_disable_iova_as_pa and not always_enable.contains(drv_path) + build = false + reason = 'driver does not support disabling IOVA as PA mode' + endif + # get dependency objs from strings shared_deps = ext_deps static_deps = ext_deps diff --git a/drivers/net/hns3/meson.build b/drivers/net/hns3/meson.build index f2aede94ed..39d426f232 100644 --- a/drivers/net/hns3/meson.build +++ b/drivers/net/hns3/meson.build @@ -13,6 +13,12 @@ if arch_subdir != 'x86' and arch_subdir != 'arm' or not dpdk_conf.get('RTE_ARCH_ subdir_done() endif +if dpdk_conf.get('RTE_IOVA_AS_PA') == 0 + build = false + reason = 'driver does not support disabling IOVA as PA mode' + subdir_done() +endif + sources = files( 'hns3_cmd.c', 'hns3_dcb.c', diff --git a/lib/eal/linux/eal.c b/lib/eal/linux/eal.c index 46bf52cef0..a6eb45c65a 100644 --- a/lib/eal/linux/eal.c +++ b/lib/eal/linux/eal.c @@ -1128,6 +1128,12 @@ rte_eal_init(int argc, char **argv) return -1; } + if (rte_eal_iova_mode() == RTE_IOVA_PA && !RTE_IOVA_AS_PA) { + rte_eal_init_alert("Cannot use IOVA as 'PA' as it is disabled during build"); + rte_errno = EINVAL; + return -1; + } + RTE_LOG(INFO, EAL, "Selected IOVA mode '%s'\n", rte_eal_iova_mode() == RTE_IOVA_PA ? "PA" : "VA"); diff --git a/lib/mbuf/rte_mbuf.c b/lib/mbuf/rte_mbuf.c index 16f6ed6731..cfd8062f1e 100644 --- a/lib/mbuf/rte_mbuf.c +++ b/lib/mbuf/rte_mbuf.c @@ -388,7 +388,7 @@ int rte_mbuf_check(const struct rte_mbuf *m, int is_header, *reason = "bad mbuf pool"; return -1; } - if (rte_mbuf_iova_get(m) == 0) { + if (RTE_IOVA_AS_PA && rte_mbuf_iova_get(m) == 0) { *reason = "bad IO addr"; return -1; } diff --git a/lib/mbuf/rte_mbuf.h b/lib/mbuf/rte_mbuf.h index c552dfbcac..481e1ec326 100644 --- a/lib/mbuf/rte_mbuf.h +++ b/lib/mbuf/rte_mbuf.h @@ -141,7 +141,11 @@ static inline uint16_t rte_pktmbuf_priv_size(struct rte_mempool *mp); static inline rte_iova_t rte_mbuf_iova_get(const struct rte_mbuf *m) { +#if RTE_IOVA_AS_PA return m->buf_iova; +#else + return (rte_iova_t)m->buf_addr; +#endif } /** @@ -150,7 +154,12 @@ rte_mbuf_iova_get(const struct rte_mbuf *m) static inline void rte_mbuf_iova_set(struct rte_mbuf *m, rte_iova_t iova) { +#if RTE_IOVA_AS_PA m->buf_iova = iova; +#else + RTE_SET_USED(m); + RTE_SET_USED(iova); +#endif } /** diff --git a/lib/mbuf/rte_mbuf_core.h b/lib/mbuf/rte_mbuf_core.h index 51a12a1fb9..91c2211b44 100644 --- a/lib/mbuf/rte_mbuf_core.h +++ b/lib/mbuf/rte_mbuf_core.h @@ -467,13 +467,19 @@ struct rte_mbuf { RTE_MARKER cacheline0; void *buf_addr; /**< Virtual address of segment buffer. */ +#if RTE_IOVA_AS_PA /** * Physical address of segment buffer. + * This field is undefined if the build is configured to use only + * virtual address as IOVA (i.e. RTE_IOVA_AS_PA is 0). * Force alignment to 8-bytes, so as to ensure we have the exact * same mbuf cacheline0 layout for 32-bit and 64-bit. This makes * working on vector drivers easier. */ rte_iova_t buf_iova __rte_aligned(sizeof(rte_iova_t)); +#else + uint64_t dummy; +#endif /* next 8 bytes are initialised on RX descriptor rearm */ RTE_MARKER64 rearm_data; diff --git a/lib/meson.build b/lib/meson.build index c648f7d800..c071a6c8e0 100644 --- a/lib/meson.build +++ b/lib/meson.build @@ -88,6 +88,9 @@ optional_libs = [ disabled_libs = [] opt_disabled_libs = run_command(list_dir_globs, get_option('disable_libs'), check: true).stdout().split() +if dpdk_conf.get('RTE_IOVA_AS_PA') == 0 + opt_disabled_libs += ['kni'] +endif foreach l:opt_disabled_libs if not optional_libs.contains(l) warning('Cannot disable mandatory library "@0@"'.format(l)) diff --git a/meson_options.txt b/meson_options.txt index 7c220ad68d..f6880410e2 100644 --- a/meson_options.txt +++ b/meson_options.txt @@ -44,6 +44,8 @@ option('platform', type: 'string', value: 'native', description: 'Platform to build, either "native", "generic" or a SoC. Please refer to the Linux build guide for more information.') option('enable_trace_fp', type: 'boolean', value: false, description: 'enable fast path trace points.') +option('enable_iova_as_pa', type: 'boolean', value: true, description: + 'Enable or disable support for IOVA as PA mode. Disabling this option removes the buf_iova field of mbuf.') option('tests', type: 'boolean', value: true, description: 'build unit tests') option('use_hpet', type: 'boolean', value: false, description: