From patchwork Wed Oct 3 20:53:12 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: eric zhang X-Patchwork-Id: 45997 X-Patchwork-Delegate: thomas@monjalon.net Return-Path: X-Original-To: patchwork@dpdk.org Delivered-To: patchwork@dpdk.org Received: from [92.243.14.124] (localhost [127.0.0.1]) by dpdk.org (Postfix) with ESMTP id 47D761B45C; Wed, 3 Oct 2018 22:55:39 +0200 (CEST) Received: from mail5.wrs.com (mail5.windriver.com [192.103.53.11]) by dpdk.org (Postfix) with ESMTP id 2A57D1B45B for ; Wed, 3 Oct 2018 22:55:37 +0200 (CEST) Received: from ALA-HCA.corp.ad.wrs.com (ala-hca.corp.ad.wrs.com [147.11.189.40]) by mail5.wrs.com (8.15.2/8.15.2) with ESMTPS id w93KsKge006330 (version=TLSv1 cipher=AES128-SHA bits=128 verify=FAIL); Wed, 3 Oct 2018 13:55:15 -0700 Received: from yow-ezhang-lx.wrs.com (128.224.56.213) by ALA-HCA.corp.ad.wrs.com (147.11.189.50) with Microsoft SMTP Server (TLS) id 14.3.408.0; Wed, 3 Oct 2018 13:53:56 -0700 From: eric zhang To: , CC: , , Date: Wed, 3 Oct 2018 16:53:12 -0400 Message-ID: <1538599994-23331-2-git-send-email-eric.zhang@windriver.com> X-Mailer: git-send-email 2.7.4 In-Reply-To: <1538599994-23331-1-git-send-email-eric.zhang@windriver.com> References: <1538599994-23331-1-git-send-email-eric.zhang@windriver.com> MIME-Version: 1.0 X-Originating-IP: [128.224.56.213] Subject: [dpdk-dev] [PATCH v3 1/3] eal: add eal option to configure iova mode X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Sender: "dev" From: Santosh Shukla In the case of user don't want to use bus iova scheme and want to override. For that, Adding eal option --iova-mode= where valid input string is 'pa' or 'va'. Signed-off-by: Santosh Shukla Signed-off-by: Jerin Jacob Signed-off-by: eric zhang --- lib/librte_eal/common/eal_common_options.c | 30 ++++++++++++++++++++++++++++++ lib/librte_eal/common/eal_internal_cfg.h | 1 + lib/librte_eal/common/eal_options.h | 2 ++ 3 files changed, 33 insertions(+) diff --git a/lib/librte_eal/common/eal_common_options.c b/lib/librte_eal/common/eal_common_options.c index 996a034..70c0fc8 100644 --- a/lib/librte_eal/common/eal_common_options.c +++ b/lib/librte_eal/common/eal_common_options.c @@ -82,6 +82,7 @@ {OPT_HELP, 0, NULL, OPT_HELP_NUM }, {OPT_HUGE_DIR, 1, NULL, OPT_HUGE_DIR_NUM }, {OPT_HUGE_UNLINK, 0, NULL, OPT_HUGE_UNLINK_NUM }, + {OPT_IOVA_MODE, 1, NULL, OPT_IOVA_MODE_NUM }, {OPT_LCORES, 1, NULL, OPT_LCORES_NUM }, {OPT_LOG_LEVEL, 1, NULL, OPT_LOG_LEVEL_NUM }, {OPT_MASTER_LCORE, 1, NULL, OPT_MASTER_LCORE_NUM }, @@ -218,6 +219,7 @@ struct device_option { #endif internal_cfg->vmware_tsc_map = 0; internal_cfg->create_uio_dev = 0; + internal_cfg->iova_mode = RTE_IOVA_DC; internal_cfg->mbuf_pool_ops_name = RTE_MBUF_DEFAULT_MEMPOOL_OPS; } @@ -994,6 +996,25 @@ static int xdigit2val(unsigned char c) return RTE_PROC_INVALID; } +static int +eal_parse_iova_mode(const char *name) +{ + int mode; + + if (name == NULL) + return -1; + + if (!strcmp("pa", name)) + mode = RTE_IOVA_PA; + else if (!strcmp("va", name)) + mode = RTE_IOVA_VA; + else + return -1; + + internal_config.iova_mode = mode; + return 0; +} + int eal_parse_common_option(int opt, const char *optarg, struct internal_config *conf) @@ -1158,6 +1179,13 @@ static int xdigit2val(unsigned char c) } core_parsed = 1; break; + case OPT_IOVA_MODE_NUM: + if (eal_parse_iova_mode(optarg) < 0) { + RTE_LOG(ERR, EAL, "invalid parameters for --" + OPT_IOVA_MODE "\n"); + return -1; + } + break; /* don't know what to do, leave this to caller */ default: @@ -1306,6 +1334,8 @@ static int xdigit2val(unsigned char c) " -h, --help This help\n" "\nEAL options for DEBUG use only:\n" " --"OPT_HUGE_UNLINK" Unlink hugepage files after init\n" + " --"OPT_IOVA_MODE" Set iova mode. 'pa' for IOVA_PA\n" + " 'va' for IOVA_VA\n" " --"OPT_NO_HUGE" Use malloc instead of hugetlbfs\n" " --"OPT_NO_PCI" Disable PCI\n" " --"OPT_NO_HPET" Disable HPET\n" diff --git a/lib/librte_eal/common/eal_internal_cfg.h b/lib/librte_eal/common/eal_internal_cfg.h index fa6ccbe..29bf53f 100644 --- a/lib/librte_eal/common/eal_internal_cfg.h +++ b/lib/librte_eal/common/eal_internal_cfg.h @@ -83,6 +83,7 @@ struct internal_config { const char *hugepage_dir; /**< specific hugetlbfs directory to use */ const char *mbuf_pool_ops_name; /**< mbuf pool ops name */ unsigned num_hugepage_sizes; /**< how many sizes on this system */ + enum rte_iova_mode iova_mode ; /**< Set iova mode on this system */ struct hugepage_info hugepage_info[MAX_HUGEPAGE_SIZES]; }; extern struct internal_config internal_config; /**< Global EAL configuration. */ diff --git a/lib/librte_eal/common/eal_options.h b/lib/librte_eal/common/eal_options.h index 30e6bb4..7786189 100644 --- a/lib/librte_eal/common/eal_options.h +++ b/lib/librte_eal/common/eal_options.h @@ -83,6 +83,8 @@ enum { OPT_VFIO_INTR_NUM, #define OPT_VMWARE_TSC_MAP "vmware-tsc-map" OPT_VMWARE_TSC_MAP_NUM, +#define OPT_IOVA_MODE "iova-mode" + OPT_IOVA_MODE_NUM, OPT_LONG_MAX_NUM }; From patchwork Wed Oct 3 20:53:13 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: eric zhang X-Patchwork-Id: 45998 X-Patchwork-Delegate: thomas@monjalon.net Return-Path: X-Original-To: patchwork@dpdk.org Delivered-To: patchwork@dpdk.org Received: from [92.243.14.124] (localhost [127.0.0.1]) by dpdk.org (Postfix) with ESMTP id E9AC11B46C; Wed, 3 Oct 2018 22:56:18 +0200 (CEST) Received: from mail5.wrs.com (mail5.windriver.com [192.103.53.11]) by dpdk.org (Postfix) with ESMTP id 016491B45B for ; Wed, 3 Oct 2018 22:56:17 +0200 (CEST) Received: from ALA-HCA.corp.ad.wrs.com (ala-hca.corp.ad.wrs.com [147.11.189.40]) by mail5.wrs.com (8.15.2/8.15.2) with ESMTPS id w93KsKgh006330 (version=TLSv1 cipher=AES128-SHA bits=128 verify=FAIL); Wed, 3 Oct 2018 13:55:56 -0700 Received: from yow-ezhang-lx.wrs.com (128.224.56.213) by ALA-HCA.corp.ad.wrs.com (147.11.189.50) with Microsoft SMTP Server (TLS) id 14.3.408.0; Wed, 3 Oct 2018 13:54:32 -0700 From: eric zhang To: , CC: , , Date: Wed, 3 Oct 2018 16:53:13 -0400 Message-ID: <1538599994-23331-3-git-send-email-eric.zhang@windriver.com> X-Mailer: git-send-email 2.7.4 In-Reply-To: <1538599994-23331-1-git-send-email-eric.zhang@windriver.com> References: <1538599994-23331-1-git-send-email-eric.zhang@windriver.com> MIME-Version: 1.0 X-Originating-IP: [128.224.56.213] Subject: [dpdk-dev] [PATCH v3 2/3] eal: force IOVA to a particular mode X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Sender: "dev" This patch uses EAL option "--iova-mode" to force the IOVA mode to a particular value. There exists virtual devices that are not directly attached to the PCI bus, and therefore the auto detectioni of the IOVA mode based on probing the PCI bus and IOMMU configuration may not report the required addressing mode. Using the EAL option permits the mode to be explicitly configured in this scenario. Signed-off-by: eric zhang --- lib/librte_eal/bsdapp/eal/eal.c | 11 +++++++++-- lib/librte_eal/linuxapp/eal/eal.c | 27 +++++++++++++++++---------- 2 files changed, 26 insertions(+), 12 deletions(-) diff --git a/lib/librte_eal/bsdapp/eal/eal.c b/lib/librte_eal/bsdapp/eal/eal.c index 369a682..5392f06 100644 --- a/lib/librte_eal/bsdapp/eal/eal.c +++ b/lib/librte_eal/bsdapp/eal/eal.c @@ -569,8 +569,15 @@ static void rte_eal_init_alert(const char *msg) return -1; } - /* autodetect the iova mapping mode (default is iova_pa) */ - rte_eal_get_configuration()->iova_mode = rte_bus_get_iommu_class(); + /* if no eal option "--iova-mode=", use bus iova scheme */ + if (internal_config.iova_mode == RTE_IOVA_DC) { + /* autodetect the iova mapping mode (default is iova_pa) */ + rte_eal_get_configuration()->iova_mode = + rte_bus_get_iommu_class(); + } else { + rte_eal_get_configuration()->iova_mode = + internal_config.iova_mode; + } if (internal_config.no_hugetlbfs == 0 && internal_config.process_type != RTE_PROC_SECONDARY && diff --git a/lib/librte_eal/linuxapp/eal/eal.c b/lib/librte_eal/linuxapp/eal/eal.c index e0b5ae1..1af9458 100644 --- a/lib/librte_eal/linuxapp/eal/eal.c +++ b/lib/librte_eal/linuxapp/eal/eal.c @@ -805,16 +805,23 @@ static void rte_eal_init_alert(const char *msg) return -1; } - /* autodetect the iova mapping mode (default is iova_pa) */ - rte_eal_get_configuration()->iova_mode = rte_bus_get_iommu_class(); - - /* Workaround for KNI which requires physical address to work */ - if (rte_eal_get_configuration()->iova_mode == RTE_IOVA_VA && - rte_eal_check_module("rte_kni") == 1) { - rte_eal_get_configuration()->iova_mode = RTE_IOVA_PA; - RTE_LOG(WARNING, EAL, - "Some devices want IOVA as VA but PA will be used because.. " - "KNI module inserted\n"); + /* if no eal option "--iova-mode=", use bus iova scheme */ + if (internal_config.iova_mode == RTE_IOVA_DC) { + /* autodetect the iova mapping mode (default is iova_pa) */ + rte_eal_get_configuration()->iova_mode = + rte_bus_get_iommu_class(); + + /* Workaround for KNI which requires physical address to work */ + if (rte_eal_get_configuration()->iova_mode == RTE_IOVA_VA && + rte_eal_check_module("rte_kni") == 1) { + rte_eal_get_configuration()->iova_mode = RTE_IOVA_PA; + RTE_LOG(WARNING, EAL, + "Some devices want IOVA as VA but PA will be used because.. " + "KNI module inserted\n"); + } + } else { + rte_eal_get_configuration()->iova_mode = + internal_config.iova_mode; } if (internal_config.no_hugetlbfs == 0 && From patchwork Wed Oct 3 20:53:14 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: eric zhang X-Patchwork-Id: 45999 X-Patchwork-Delegate: thomas@monjalon.net Return-Path: X-Original-To: patchwork@dpdk.org Delivered-To: patchwork@dpdk.org Received: from [92.243.14.124] (localhost [127.0.0.1]) by dpdk.org (Postfix) with ESMTP id 77C051B473; Wed, 3 Oct 2018 22:56:25 +0200 (CEST) Received: from mail5.wrs.com (mail5.windriver.com [192.103.53.11]) by dpdk.org (Postfix) with ESMTP id 620661B472 for ; Wed, 3 Oct 2018 22:56:23 +0200 (CEST) Received: from ALA-HCA.corp.ad.wrs.com (ala-hca.corp.ad.wrs.com [147.11.189.40]) by mail5.wrs.com (8.15.2/8.15.2) with ESMTPS id w93KsKgj006330 (version=TLSv1 cipher=AES128-SHA bits=128 verify=FAIL); Wed, 3 Oct 2018 13:56:07 -0700 Received: from yow-ezhang-lx.wrs.com (128.224.56.213) by ALA-HCA.corp.ad.wrs.com (147.11.189.50) with Microsoft SMTP Server (TLS) id 14.3.408.0; Wed, 3 Oct 2018 13:55:37 -0700 From: eric zhang To: , CC: , , Date: Wed, 3 Oct 2018 16:53:14 -0400 Message-ID: <1538599994-23331-4-git-send-email-eric.zhang@windriver.com> X-Mailer: git-send-email 2.7.4 In-Reply-To: <1538599994-23331-1-git-send-email-eric.zhang@windriver.com> References: <1538599994-23331-1-git-send-email-eric.zhang@windriver.com> MIME-Version: 1.0 X-Originating-IP: [128.224.56.213] Subject: [dpdk-dev] [PATCH v3 3/3] doc: document --iova-mode EAL option X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Sender: "dev" This patch updates Programmer's Guide and EAL parameter guides to show EAL option "--iova-mode" support. Signed-off-by: eric zhang Reviewed-by: Marko Kovacevic --- doc/guides/prog_guide/env_abstraction_layer.rst | 8 ++++++++ doc/guides/testpmd_app_ug/run_app.rst | 4 ++++ 2 files changed, 12 insertions(+) diff --git a/doc/guides/prog_guide/env_abstraction_layer.rst b/doc/guides/prog_guide/env_abstraction_layer.rst index 34d871c..208c454 100644 --- a/doc/guides/prog_guide/env_abstraction_layer.rst +++ b/doc/guides/prog_guide/env_abstraction_layer.rst @@ -225,6 +225,14 @@ Misc Functions Locks and atomic operations are per-architecture (i686 and x86_64). +IOVA Mode Configuration +~~~~~~~~~~~~~~~~~~~~~~~ + +Auto detection of the IOVA mode, based on probing the bus and IOMMU configuration, may not report +the desired addressing mode when virtual devices that are not directly attached to the bus are present. +To facilitate forcing the IOVA mode to a specific value the EAL command line option ``--iova-mode`` can +be used to select either physical addressing('pa') or virtual addressing('va'). + Memory Segments and Memory Zones (memzone) ------------------------------------------ diff --git a/doc/guides/testpmd_app_ug/run_app.rst b/doc/guides/testpmd_app_ug/run_app.rst index 4c0d2ce..94fd0cc 100644 --- a/doc/guides/testpmd_app_ug/run_app.rst +++ b/doc/guides/testpmd_app_ug/run_app.rst @@ -156,6 +156,10 @@ See the DPDK Getting Started Guides for more information on these options. Use malloc instead of hugetlbfs. +* ``--iova-mode `` + + Force IOVA mode to a specific value. + Testpmd Command-line Options ----------------------------