From patchwork Mon Jul 22 12:56:50 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: David Marchand X-Patchwork-Id: 56875 X-Patchwork-Delegate: thomas@monjalon.net Return-Path: X-Original-To: patchwork@dpdk.org Delivered-To: patchwork@dpdk.org Received: from [92.243.14.124] (localhost [127.0.0.1]) by dpdk.org (Postfix) with ESMTP id 7A45A1BF80; Mon, 22 Jul 2019 14:57:15 +0200 (CEST) Received: from mx1.redhat.com (mx1.redhat.com [209.132.183.28]) by dpdk.org (Postfix) with ESMTP id 903951BF7B for ; Mon, 22 Jul 2019 14:57:11 +0200 (CEST) Received: from smtp.corp.redhat.com (int-mx01.intmail.prod.int.phx2.redhat.com [10.5.11.11]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mx1.redhat.com (Postfix) with ESMTPS id B982A307D914; Mon, 22 Jul 2019 12:57:10 +0000 (UTC) Received: from dmarchan.remote.csb (ovpn-204-177.brq.redhat.com [10.40.204.177]) by smtp.corp.redhat.com (Postfix) with ESMTP id D6E35D1FF; Mon, 22 Jul 2019 12:57:08 +0000 (UTC) From: David Marchand To: dev@dpdk.org Cc: anatoly.burakov@intel.com, jerinj@marvell.com, thomas@monjalon.net Date: Mon, 22 Jul 2019 14:56:50 +0200 Message-Id: <1563800213-29839-2-git-send-email-david.marchand@redhat.com> In-Reply-To: <1563800213-29839-1-git-send-email-david.marchand@redhat.com> References: <1562795329-16652-1-git-send-email-david.marchand@redhat.com> <1563800213-29839-1-git-send-email-david.marchand@redhat.com> X-Scanned-By: MIMEDefang 2.79 on 10.5.11.11 X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-4.5.16 (mx1.redhat.com [10.5.110.48]); Mon, 22 Jul 2019 12:57:10 +0000 (UTC) Subject: [dpdk-dev] [PATCH v4 1/4] Revert "bus/pci: add Mellanox kernel driver type" X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Sender: "dev" This reverts commit 0cb86518db57d35e0abc14d6703fad561a0310e2. The PCI bus now reports DC when faced with a device bound to an unknown driver and, in such a case, the IOVA mode is selected against physical address availability. As a consequence, there is no reason for this special case for Mellanox drivers. Fixes: 703458e19c16 ("bus/pci: consider only usable devices for IOVA mode") Signed-off-by: David Marchand Reviewed-by: Jerin Jacob --- drivers/bus/pci/linux/pci.c | 8 -------- lib/librte_eal/common/include/rte_dev.h | 1 - 2 files changed, 9 deletions(-) diff --git a/drivers/bus/pci/linux/pci.c b/drivers/bus/pci/linux/pci.c index 33c8ea7..b12f10a 100644 --- a/drivers/bus/pci/linux/pci.c +++ b/drivers/bus/pci/linux/pci.c @@ -329,9 +329,6 @@ dev->kdrv = RTE_KDRV_IGB_UIO; else if (!strcmp(driver, "uio_pci_generic")) dev->kdrv = RTE_KDRV_UIO_GENERIC; - else if (!strcmp(driver, "mlx4_core") || - !strcmp(driver, "mlx5_core")) - dev->kdrv = RTE_KDRV_NIC_MLX; else dev->kdrv = RTE_KDRV_UNKNOWN; } else @@ -591,11 +588,6 @@ enum rte_iova_mode break; } - case RTE_KDRV_NIC_MLX: - if ((pdrv->drv_flags & RTE_PCI_DRV_IOVA_AS_VA) == 0) - iova_mode = RTE_IOVA_PA; - break; - case RTE_KDRV_IGB_UIO: case RTE_KDRV_UIO_GENERIC: iova_mode = RTE_IOVA_PA; diff --git a/lib/librte_eal/common/include/rte_dev.h b/lib/librte_eal/common/include/rte_dev.h index 94829f6..c25e09e 100644 --- a/lib/librte_eal/common/include/rte_dev.h +++ b/lib/librte_eal/common/include/rte_dev.h @@ -63,7 +63,6 @@ enum rte_kernel_driver { RTE_KDRV_VFIO, RTE_KDRV_UIO_GENERIC, RTE_KDRV_NIC_UIO, - RTE_KDRV_NIC_MLX, RTE_KDRV_NONE, }; From patchwork Mon Jul 22 12:56:51 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: David Marchand X-Patchwork-Id: 56876 X-Patchwork-Delegate: thomas@monjalon.net Return-Path: X-Original-To: patchwork@dpdk.org Delivered-To: patchwork@dpdk.org Received: from [92.243.14.124] (localhost [127.0.0.1]) by dpdk.org (Postfix) with ESMTP id 58FE11BF87; Mon, 22 Jul 2019 14:57:22 +0200 (CEST) Received: from mx1.redhat.com (mx1.redhat.com [209.132.183.28]) by dpdk.org (Postfix) with ESMTP id 314211BF86 for ; Mon, 22 Jul 2019 14:57:21 +0200 (CEST) Received: from smtp.corp.redhat.com (int-mx01.intmail.prod.int.phx2.redhat.com [10.5.11.11]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mx1.redhat.com (Postfix) with ESMTPS id 484C73086258; Mon, 22 Jul 2019 12:57:20 +0000 (UTC) Received: from dmarchan.remote.csb (ovpn-204-177.brq.redhat.com [10.40.204.177]) by smtp.corp.redhat.com (Postfix) with ESMTP id 9FE5E12A6F; Mon, 22 Jul 2019 12:57:11 +0000 (UTC) From: David Marchand To: dev@dpdk.org Cc: anatoly.burakov@intel.com, jerinj@marvell.com, thomas@monjalon.net, John McNamara , Marko Kovacevic , Igor Russkikh , Pavel Belous , Ajit Khaparde , Somnath Kotur , Wenzhuo Lu , John Daley , Hyong Youb Kim , Qi Zhang , Xiao Wang , Beilei Xing , Jingjing Wu , Qiming Yang , Konstantin Ananyev , Matan Azrad , Shahaf Shuler , Yongseok Koh , Viacheslav Ovsiienko , Alejandro Lucero , Nithin Dabilpuram , Kiran Kumar K , Rasesh Mody , Shahed Shaikh , Bruce Richardson Date: Mon, 22 Jul 2019 14:56:51 +0200 Message-Id: <1563800213-29839-3-git-send-email-david.marchand@redhat.com> In-Reply-To: <1563800213-29839-1-git-send-email-david.marchand@redhat.com> References: <1562795329-16652-1-git-send-email-david.marchand@redhat.com> <1563800213-29839-1-git-send-email-david.marchand@redhat.com> X-Scanned-By: MIMEDefang 2.79 on 10.5.11.11 X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-4.5.16 (mx1.redhat.com [10.5.110.49]); Mon, 22 Jul 2019 12:57:20 +0000 (UTC) Subject: [dpdk-dev] [PATCH v4 2/4] eal: fix IOVA mode selection as VA for PCI drivers X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Sender: "dev" The incriminated commit broke the use of RTE_PCI_DRV_IOVA_AS_VA which was intended to mean "driver only supports VA" but had been understood as "driver supports both PA and VA" by most net drivers and used to let dpdk processes to run as non root (which do not have access to physical addresses on recent kernels). The check on physical addresses actually closed the gap for those drivers. We don't need to mark them with RTE_PCI_DRV_IOVA_AS_VA and this flag can retain its intended meaning. Document explicitly its meaning. We can check that a driver requirement wrt to IOVA mode is fulfilled before trying to probe a device. Finally, document the heuristic used to select the IOVA mode and hope that we won't break it again. Fixes: 703458e19c16 ("bus/pci: consider only usable devices for IOVA mode") Signed-off-by: David Marchand Reviewed-by: Jerin Jacob Tested-by: Jerin Jacob Acked-by: Anatoly Burakov --- Changelog since v3: - fixed typos, --- doc/guides/prog_guide/env_abstraction_layer.rst | 31 +++++++++++++++++++++++++ drivers/bus/pci/linux/pci.c | 16 +++++-------- drivers/bus/pci/pci_common.c | 30 +++++++++++++++++++----- drivers/bus/pci/rte_bus_pci.h | 4 ++-- drivers/net/atlantic/atl_ethdev.c | 3 +-- drivers/net/bnxt/bnxt_ethdev.c | 3 +-- drivers/net/e1000/em_ethdev.c | 3 +-- drivers/net/e1000/igb_ethdev.c | 5 ++-- drivers/net/enic/enic_ethdev.c | 3 +-- drivers/net/fm10k/fm10k_ethdev.c | 3 +-- drivers/net/i40e/i40e_ethdev.c | 3 +-- drivers/net/i40e/i40e_ethdev_vf.c | 2 +- drivers/net/iavf/iavf_ethdev.c | 3 +-- drivers/net/ice/ice_ethdev.c | 3 +-- drivers/net/ixgbe/ixgbe_ethdev.c | 5 ++-- drivers/net/mlx4/mlx4.c | 3 +-- drivers/net/mlx5/mlx5.c | 2 +- drivers/net/nfp/nfp_net.c | 6 ++--- drivers/net/octeontx2/otx2_ethdev.c | 5 ---- drivers/net/qede/qede_ethdev.c | 6 ++--- drivers/raw/ioat/ioat_rawdev.c | 3 +-- lib/librte_eal/common/eal_common_bus.c | 30 +++++++++++++++++++++--- 22 files changed, 110 insertions(+), 62 deletions(-) diff --git a/doc/guides/prog_guide/env_abstraction_layer.rst b/doc/guides/prog_guide/env_abstraction_layer.rst index f15bcd9..1d63675 100644 --- a/doc/guides/prog_guide/env_abstraction_layer.rst +++ b/doc/guides/prog_guide/env_abstraction_layer.rst @@ -419,6 +419,37 @@ Misc Functions Locks and atomic operations are per-architecture (i686 and x86_64). +IOVA Mode Detection +~~~~~~~~~~~~~~~~~~~ + +IOVA Mode is selected by considering what the current usable Devices on the +system require and/or support. + +Below is the 2-step heuristic for this choice. + +For the first step, EAL asks each bus its requirement in terms of IOVA mode +and decides on a preferred IOVA mode. + +- if all buses report RTE_IOVA_PA, then the preferred IOVA mode is RTE_IOVA_PA, +- if all buses report RTE_IOVA_VA, then the preferred IOVA mode is RTE_IOVA_VA, +- if all buses report RTE_IOVA_DC, no bus expressed a preferrence, then the + preferred mode is RTE_IOVA_DC, +- if the buses disagree (at least one wants RTE_IOVA_PA and at least one wants + RTE_IOVA_VA), then the preferred IOVA mode is RTE_IOVA_DC (see below with the + check on Physical Addresses availability), + +The second step checks if the preferred mode complies with the Physical +Addresses availability since those are only available to root user in recent +kernels. + +- if the preferred mode is RTE_IOVA_PA but there is no access to Physical + Addresses, then EAL init fails early, since later probing of the devices + would fail anyway, +- if the preferred mode is RTE_IOVA_DC then based on the Physical Addresses + availability, the preferred mode is adjusted to RTE_IOVA_PA or RTE_IOVA_VA. + In the case when the buses had disagreed on the IOVA Mode at the first step, + part of the buses won't work because of this decision. + IOVA Mode Configuration ~~~~~~~~~~~~~~~~~~~~~~~ diff --git a/drivers/bus/pci/linux/pci.c b/drivers/bus/pci/linux/pci.c index b12f10a..1a2f99b 100644 --- a/drivers/bus/pci/linux/pci.c +++ b/drivers/bus/pci/linux/pci.c @@ -578,12 +578,10 @@ enum rte_iova_mode else is_vfio_noiommu_enabled = 0; } - if ((pdrv->drv_flags & RTE_PCI_DRV_IOVA_AS_VA) == 0) { + if (is_vfio_noiommu_enabled != 0) iova_mode = RTE_IOVA_PA; - } else if (is_vfio_noiommu_enabled != 0) { - RTE_LOG(DEBUG, EAL, "Forcing to 'PA', vfio-noiommu mode configured\n"); - iova_mode = RTE_IOVA_PA; - } + else if ((pdrv->drv_flags & RTE_PCI_DRV_IOVA_AS_VA) != 0) + iova_mode = RTE_IOVA_VA; #endif break; } @@ -594,8 +592,8 @@ enum rte_iova_mode break; default: - RTE_LOG(DEBUG, EAL, "Unsupported kernel driver? Defaulting to IOVA as 'PA'\n"); - iova_mode = RTE_IOVA_PA; + if ((pdrv->drv_flags & RTE_PCI_DRV_IOVA_AS_VA) != 0) + iova_mode = RTE_IOVA_VA; break; } @@ -607,10 +605,8 @@ enum rte_iova_mode if (iommu_no_va == -1) iommu_no_va = pci_one_device_iommu_support_va(pdev) ? 0 : 1; - if (iommu_no_va != 0) { - RTE_LOG(DEBUG, EAL, "Forcing to 'PA', IOMMU does not support IOVA as 'VA'\n"); + if (iommu_no_va != 0) iova_mode = RTE_IOVA_PA; - } } return iova_mode; } diff --git a/drivers/bus/pci/pci_common.c b/drivers/bus/pci/pci_common.c index d2af472..9794552 100644 --- a/drivers/bus/pci/pci_common.c +++ b/drivers/bus/pci/pci_common.c @@ -169,8 +169,22 @@ static struct rte_devargs *pci_devargs_lookup(struct rte_pci_device *dev) * This needs to be before rte_pci_map_device(), as it enables to use * driver flags for adjusting configuration. */ - if (!already_probed) + if (!already_probed) { + enum rte_iova_mode dev_iova_mode; + enum rte_iova_mode iova_mode; + + dev_iova_mode = pci_device_iova_mode(dr, dev); + iova_mode = rte_eal_iova_mode(); + if (dev_iova_mode != RTE_IOVA_DC && + dev_iova_mode != iova_mode) { + RTE_LOG(ERR, EAL, " Expecting '%s' IOVA mode but current mode is '%s', not initializing\n", + dev_iova_mode == RTE_IOVA_PA ? "PA" : "VA", + iova_mode == RTE_IOVA_PA ? "PA" : "VA"); + return -EINVAL; + } + dev->driver = dr; + } if (!already_probed && (dr->drv_flags & RTE_PCI_DRV_NEED_MAPPING)) { /* map resources for devices that use igb_uio */ @@ -629,12 +643,16 @@ enum rte_iova_mode devices_want_va = true; } } - if (devices_want_pa) { - iova_mode = RTE_IOVA_PA; - if (devices_want_va) - RTE_LOG(WARNING, EAL, "Some devices want 'VA' but forcing 'PA' because other devices want it\n"); - } else if (devices_want_va) { + if (devices_want_va && !devices_want_pa) { iova_mode = RTE_IOVA_VA; + } else if (devices_want_pa && !devices_want_va) { + iova_mode = RTE_IOVA_PA; + } else { + iova_mode = RTE_IOVA_DC; + if (devices_want_va) { + RTE_LOG(WARNING, EAL, "Some devices want 'VA' but forcing 'DC' because other devices want 'PA'.\n"); + RTE_LOG(WARNING, EAL, "Depending on the final decision by the EAL, not all devices may be able to initialize.\n"); + } } return iova_mode; } diff --git a/drivers/bus/pci/rte_bus_pci.h b/drivers/bus/pci/rte_bus_pci.h index 06e004c..0f21775 100644 --- a/drivers/bus/pci/rte_bus_pci.h +++ b/drivers/bus/pci/rte_bus_pci.h @@ -187,8 +187,8 @@ struct rte_pci_bus { #define RTE_PCI_DRV_INTR_RMV 0x0010 /** Device driver needs to keep mapped resources if unsupported dev detected */ #define RTE_PCI_DRV_KEEP_MAPPED_RES 0x0020 -/** Device driver supports IOVA as VA */ -#define RTE_PCI_DRV_IOVA_AS_VA 0X0040 +/** Device driver only supports IOVA as VA and cannot work with IOVA as PA */ +#define RTE_PCI_DRV_IOVA_AS_VA 0x0040 /** * Map the PCI device resources in user space virtual memory address diff --git a/drivers/net/atlantic/atl_ethdev.c b/drivers/net/atlantic/atl_ethdev.c index fdc0a7f..fa89ae7 100644 --- a/drivers/net/atlantic/atl_ethdev.c +++ b/drivers/net/atlantic/atl_ethdev.c @@ -157,8 +157,7 @@ static void atl_dev_info_get(struct rte_eth_dev *dev, static struct rte_pci_driver rte_atl_pmd = { .id_table = pci_id_atl_map, - .drv_flags = RTE_PCI_DRV_NEED_MAPPING | RTE_PCI_DRV_INTR_LSC | - RTE_PCI_DRV_IOVA_AS_VA, + .drv_flags = RTE_PCI_DRV_NEED_MAPPING | RTE_PCI_DRV_INTR_LSC, .probe = eth_atl_pci_probe, .remove = eth_atl_pci_remove, }; diff --git a/drivers/net/bnxt/bnxt_ethdev.c b/drivers/net/bnxt/bnxt_ethdev.c index 8fc5103..9306d56 100644 --- a/drivers/net/bnxt/bnxt_ethdev.c +++ b/drivers/net/bnxt/bnxt_ethdev.c @@ -4028,8 +4028,7 @@ static int bnxt_pci_remove(struct rte_pci_device *pci_dev) static struct rte_pci_driver bnxt_rte_pmd = { .id_table = bnxt_pci_id_map, - .drv_flags = RTE_PCI_DRV_NEED_MAPPING | - RTE_PCI_DRV_INTR_LSC | RTE_PCI_DRV_IOVA_AS_VA, + .drv_flags = RTE_PCI_DRV_NEED_MAPPING | RTE_PCI_DRV_INTR_LSC, .probe = bnxt_pci_probe, .remove = bnxt_pci_remove, }; diff --git a/drivers/net/e1000/em_ethdev.c b/drivers/net/e1000/em_ethdev.c index dc88661..0c859e5 100644 --- a/drivers/net/e1000/em_ethdev.c +++ b/drivers/net/e1000/em_ethdev.c @@ -352,8 +352,7 @@ static int eth_em_pci_remove(struct rte_pci_device *pci_dev) static struct rte_pci_driver rte_em_pmd = { .id_table = pci_id_em_map, - .drv_flags = RTE_PCI_DRV_NEED_MAPPING | RTE_PCI_DRV_INTR_LSC | - RTE_PCI_DRV_IOVA_AS_VA, + .drv_flags = RTE_PCI_DRV_NEED_MAPPING | RTE_PCI_DRV_INTR_LSC, .probe = eth_em_pci_probe, .remove = eth_em_pci_remove, }; diff --git a/drivers/net/e1000/igb_ethdev.c b/drivers/net/e1000/igb_ethdev.c index 3ee28cf..e784eeb 100644 --- a/drivers/net/e1000/igb_ethdev.c +++ b/drivers/net/e1000/igb_ethdev.c @@ -1116,8 +1116,7 @@ static int eth_igb_pci_remove(struct rte_pci_device *pci_dev) static struct rte_pci_driver rte_igb_pmd = { .id_table = pci_id_igb_map, - .drv_flags = RTE_PCI_DRV_NEED_MAPPING | RTE_PCI_DRV_INTR_LSC | - RTE_PCI_DRV_IOVA_AS_VA, + .drv_flags = RTE_PCI_DRV_NEED_MAPPING | RTE_PCI_DRV_INTR_LSC, .probe = eth_igb_pci_probe, .remove = eth_igb_pci_remove, }; @@ -1140,7 +1139,7 @@ static int eth_igbvf_pci_remove(struct rte_pci_device *pci_dev) */ static struct rte_pci_driver rte_igbvf_pmd = { .id_table = pci_id_igbvf_map, - .drv_flags = RTE_PCI_DRV_NEED_MAPPING | RTE_PCI_DRV_IOVA_AS_VA, + .drv_flags = RTE_PCI_DRV_NEED_MAPPING, .probe = eth_igbvf_pci_probe, .remove = eth_igbvf_pci_remove, }; diff --git a/drivers/net/enic/enic_ethdev.c b/drivers/net/enic/enic_ethdev.c index 5cfbd31..e9c6f83 100644 --- a/drivers/net/enic/enic_ethdev.c +++ b/drivers/net/enic/enic_ethdev.c @@ -1247,8 +1247,7 @@ static int eth_enic_pci_remove(struct rte_pci_device *pci_dev) static struct rte_pci_driver rte_enic_pmd = { .id_table = pci_id_enic_map, - .drv_flags = RTE_PCI_DRV_NEED_MAPPING | RTE_PCI_DRV_INTR_LSC | - RTE_PCI_DRV_IOVA_AS_VA, + .drv_flags = RTE_PCI_DRV_NEED_MAPPING | RTE_PCI_DRV_INTR_LSC, .probe = eth_enic_pci_probe, .remove = eth_enic_pci_remove, }; diff --git a/drivers/net/fm10k/fm10k_ethdev.c b/drivers/net/fm10k/fm10k_ethdev.c index a1e3836..2d3c477 100644 --- a/drivers/net/fm10k/fm10k_ethdev.c +++ b/drivers/net/fm10k/fm10k_ethdev.c @@ -3268,8 +3268,7 @@ static int eth_fm10k_pci_remove(struct rte_pci_device *pci_dev) static struct rte_pci_driver rte_pmd_fm10k = { .id_table = pci_id_fm10k_map, - .drv_flags = RTE_PCI_DRV_NEED_MAPPING | RTE_PCI_DRV_INTR_LSC | - RTE_PCI_DRV_IOVA_AS_VA, + .drv_flags = RTE_PCI_DRV_NEED_MAPPING | RTE_PCI_DRV_INTR_LSC, .probe = eth_fm10k_pci_probe, .remove = eth_fm10k_pci_remove, }; diff --git a/drivers/net/i40e/i40e_ethdev.c b/drivers/net/i40e/i40e_ethdev.c index 2b9fc45..dd46d4d 100644 --- a/drivers/net/i40e/i40e_ethdev.c +++ b/drivers/net/i40e/i40e_ethdev.c @@ -696,8 +696,7 @@ static int eth_i40e_pci_remove(struct rte_pci_device *pci_dev) static struct rte_pci_driver rte_i40e_pmd = { .id_table = pci_id_i40e_map, - .drv_flags = RTE_PCI_DRV_NEED_MAPPING | RTE_PCI_DRV_INTR_LSC | - RTE_PCI_DRV_IOVA_AS_VA, + .drv_flags = RTE_PCI_DRV_NEED_MAPPING | RTE_PCI_DRV_INTR_LSC, .probe = eth_i40e_pci_probe, .remove = eth_i40e_pci_remove, }; diff --git a/drivers/net/i40e/i40e_ethdev_vf.c b/drivers/net/i40e/i40e_ethdev_vf.c index 5be32b0..3ff2f60 100644 --- a/drivers/net/i40e/i40e_ethdev_vf.c +++ b/drivers/net/i40e/i40e_ethdev_vf.c @@ -1557,7 +1557,7 @@ static int eth_i40evf_pci_remove(struct rte_pci_device *pci_dev) */ static struct rte_pci_driver rte_i40evf_pmd = { .id_table = pci_id_i40evf_map, - .drv_flags = RTE_PCI_DRV_NEED_MAPPING | RTE_PCI_DRV_IOVA_AS_VA, + .drv_flags = RTE_PCI_DRV_NEED_MAPPING, .probe = eth_i40evf_pci_probe, .remove = eth_i40evf_pci_remove, }; diff --git a/drivers/net/iavf/iavf_ethdev.c b/drivers/net/iavf/iavf_ethdev.c index 53dc05c..a97cd76 100644 --- a/drivers/net/iavf/iavf_ethdev.c +++ b/drivers/net/iavf/iavf_ethdev.c @@ -1402,8 +1402,7 @@ static int eth_iavf_pci_remove(struct rte_pci_device *pci_dev) /* Adaptive virtual function driver struct */ static struct rte_pci_driver rte_iavf_pmd = { .id_table = pci_id_iavf_map, - .drv_flags = RTE_PCI_DRV_NEED_MAPPING | RTE_PCI_DRV_INTR_LSC | - RTE_PCI_DRV_IOVA_AS_VA, + .drv_flags = RTE_PCI_DRV_NEED_MAPPING | RTE_PCI_DRV_INTR_LSC, .probe = eth_iavf_pci_probe, .remove = eth_iavf_pci_remove, }; diff --git a/drivers/net/ice/ice_ethdev.c b/drivers/net/ice/ice_ethdev.c index 9ce730c..f05b48c 100644 --- a/drivers/net/ice/ice_ethdev.c +++ b/drivers/net/ice/ice_ethdev.c @@ -3737,8 +3737,7 @@ static int ice_xstats_get_names(__rte_unused struct rte_eth_dev *dev, static struct rte_pci_driver rte_ice_pmd = { .id_table = pci_id_ice_map, - .drv_flags = RTE_PCI_DRV_NEED_MAPPING | RTE_PCI_DRV_INTR_LSC | - RTE_PCI_DRV_IOVA_AS_VA, + .drv_flags = RTE_PCI_DRV_NEED_MAPPING | RTE_PCI_DRV_INTR_LSC, .probe = ice_pci_probe, .remove = ice_pci_remove, }; diff --git a/drivers/net/ixgbe/ixgbe_ethdev.c b/drivers/net/ixgbe/ixgbe_ethdev.c index 22c5b2c..4a6e5c3 100644 --- a/drivers/net/ixgbe/ixgbe_ethdev.c +++ b/drivers/net/ixgbe/ixgbe_ethdev.c @@ -1869,8 +1869,7 @@ static int eth_ixgbe_pci_remove(struct rte_pci_device *pci_dev) static struct rte_pci_driver rte_ixgbe_pmd = { .id_table = pci_id_ixgbe_map, - .drv_flags = RTE_PCI_DRV_NEED_MAPPING | RTE_PCI_DRV_INTR_LSC | - RTE_PCI_DRV_IOVA_AS_VA, + .drv_flags = RTE_PCI_DRV_NEED_MAPPING | RTE_PCI_DRV_INTR_LSC, .probe = eth_ixgbe_pci_probe, .remove = eth_ixgbe_pci_remove, }; @@ -1892,7 +1891,7 @@ static int eth_ixgbevf_pci_remove(struct rte_pci_device *pci_dev) */ static struct rte_pci_driver rte_ixgbevf_pmd = { .id_table = pci_id_ixgbevf_map, - .drv_flags = RTE_PCI_DRV_NEED_MAPPING | RTE_PCI_DRV_IOVA_AS_VA, + .drv_flags = RTE_PCI_DRV_NEED_MAPPING, .probe = eth_ixgbevf_pci_probe, .remove = eth_ixgbevf_pci_remove, }; diff --git a/drivers/net/mlx4/mlx4.c b/drivers/net/mlx4/mlx4.c index 2e169b0..d6e5753 100644 --- a/drivers/net/mlx4/mlx4.c +++ b/drivers/net/mlx4/mlx4.c @@ -1142,8 +1142,7 @@ struct mlx4_conf { }, .id_table = mlx4_pci_id_map, .probe = mlx4_pci_probe, - .drv_flags = RTE_PCI_DRV_INTR_LSC | RTE_PCI_DRV_INTR_RMV | - RTE_PCI_DRV_IOVA_AS_VA, + .drv_flags = RTE_PCI_DRV_INTR_LSC | RTE_PCI_DRV_INTR_RMV, }; #ifdef RTE_IBVERBS_LINK_DLOPEN diff --git a/drivers/net/mlx5/mlx5.c b/drivers/net/mlx5/mlx5.c index d93f92d..0f05853 100644 --- a/drivers/net/mlx5/mlx5.c +++ b/drivers/net/mlx5/mlx5.c @@ -2087,7 +2087,7 @@ struct mlx5_dev_spawn_data { .dma_map = mlx5_dma_map, .dma_unmap = mlx5_dma_unmap, .drv_flags = RTE_PCI_DRV_INTR_LSC | RTE_PCI_DRV_INTR_RMV | - RTE_PCI_DRV_PROBE_AGAIN | RTE_PCI_DRV_IOVA_AS_VA, + RTE_PCI_DRV_PROBE_AGAIN, }; #ifdef RTE_IBVERBS_LINK_DLOPEN diff --git a/drivers/net/nfp/nfp_net.c b/drivers/net/nfp/nfp_net.c index 1a7aa17..f5d33ef 100644 --- a/drivers/net/nfp/nfp_net.c +++ b/drivers/net/nfp/nfp_net.c @@ -3760,16 +3760,14 @@ static int eth_nfp_pci_remove(struct rte_pci_device *pci_dev) static struct rte_pci_driver rte_nfp_net_pf_pmd = { .id_table = pci_id_nfp_pf_net_map, - .drv_flags = RTE_PCI_DRV_NEED_MAPPING | RTE_PCI_DRV_INTR_LSC | - RTE_PCI_DRV_IOVA_AS_VA, + .drv_flags = RTE_PCI_DRV_NEED_MAPPING | RTE_PCI_DRV_INTR_LSC, .probe = nfp_pf_pci_probe, .remove = eth_nfp_pci_remove, }; static struct rte_pci_driver rte_nfp_net_vf_pmd = { .id_table = pci_id_nfp_vf_net_map, - .drv_flags = RTE_PCI_DRV_NEED_MAPPING | RTE_PCI_DRV_INTR_LSC | - RTE_PCI_DRV_IOVA_AS_VA, + .drv_flags = RTE_PCI_DRV_NEED_MAPPING | RTE_PCI_DRV_INTR_LSC, .probe = eth_nfp_pci_probe, .remove = eth_nfp_pci_remove, }; diff --git a/drivers/net/octeontx2/otx2_ethdev.c b/drivers/net/octeontx2/otx2_ethdev.c index fcb1869..5ec5551 100644 --- a/drivers/net/octeontx2/otx2_ethdev.c +++ b/drivers/net/octeontx2/otx2_ethdev.c @@ -1188,11 +1188,6 @@ goto fail; } - if (rte_eal_iova_mode() != RTE_IOVA_VA) { - otx2_err("iova mode should be va"); - goto fail; - } - if (conf->link_speeds & ETH_LINK_SPEED_FIXED) { otx2_err("Setting link speed/duplex not supported"); goto fail; diff --git a/drivers/net/qede/qede_ethdev.c b/drivers/net/qede/qede_ethdev.c index 82363e6..0b3046a 100644 --- a/drivers/net/qede/qede_ethdev.c +++ b/drivers/net/qede/qede_ethdev.c @@ -2737,8 +2737,7 @@ static int qedevf_eth_dev_pci_remove(struct rte_pci_device *pci_dev) static struct rte_pci_driver rte_qedevf_pmd = { .id_table = pci_id_qedevf_map, - .drv_flags = RTE_PCI_DRV_NEED_MAPPING | RTE_PCI_DRV_INTR_LSC | - RTE_PCI_DRV_IOVA_AS_VA, + .drv_flags = RTE_PCI_DRV_NEED_MAPPING | RTE_PCI_DRV_INTR_LSC, .probe = qedevf_eth_dev_pci_probe, .remove = qedevf_eth_dev_pci_remove, }; @@ -2757,8 +2756,7 @@ static int qede_eth_dev_pci_remove(struct rte_pci_device *pci_dev) static struct rte_pci_driver rte_qede_pmd = { .id_table = pci_id_qede_map, - .drv_flags = RTE_PCI_DRV_NEED_MAPPING | RTE_PCI_DRV_INTR_LSC | - RTE_PCI_DRV_IOVA_AS_VA, + .drv_flags = RTE_PCI_DRV_NEED_MAPPING | RTE_PCI_DRV_INTR_LSC, .probe = qede_eth_dev_pci_probe, .remove = qede_eth_dev_pci_remove, }; diff --git a/drivers/raw/ioat/ioat_rawdev.c b/drivers/raw/ioat/ioat_rawdev.c index d509b66..7270ad7 100644 --- a/drivers/raw/ioat/ioat_rawdev.c +++ b/drivers/raw/ioat/ioat_rawdev.c @@ -338,8 +338,7 @@ static struct rte_pci_driver ioat_pmd_drv = { .id_table = pci_id_ioat_map, - .drv_flags = RTE_PCI_DRV_NEED_MAPPING | RTE_PCI_DRV_INTR_LSC | - RTE_PCI_DRV_IOVA_AS_VA, + .drv_flags = RTE_PCI_DRV_NEED_MAPPING | RTE_PCI_DRV_INTR_LSC, .probe = ioat_rawdev_probe, .remove = ioat_rawdev_remove, }; diff --git a/lib/librte_eal/common/eal_common_bus.c b/lib/librte_eal/common/eal_common_bus.c index 77f1be1..0459048 100644 --- a/lib/librte_eal/common/eal_common_bus.c +++ b/lib/librte_eal/common/eal_common_bus.c @@ -228,13 +228,37 @@ struct rte_bus * enum rte_iova_mode rte_bus_get_iommu_class(void) { - int mode = RTE_IOVA_DC; + enum rte_iova_mode mode = RTE_IOVA_DC; + bool buses_want_va = false; + bool buses_want_pa = false; struct rte_bus *bus; TAILQ_FOREACH(bus, &rte_bus_list, next) { + enum rte_iova_mode bus_iova_mode; - if (bus->get_iommu_class) - mode |= bus->get_iommu_class(); + if (bus->get_iommu_class == NULL) + continue; + + bus_iova_mode = bus->get_iommu_class(); + RTE_LOG(DEBUG, EAL, "Bus %s wants IOVA as '%s'\n", + bus->name, + bus_iova_mode == RTE_IOVA_DC ? "DC" : + (bus_iova_mode == RTE_IOVA_PA ? "PA" : "VA")); + if (bus_iova_mode == RTE_IOVA_PA) + buses_want_pa = true; + else if (bus_iova_mode == RTE_IOVA_VA) + buses_want_va = true; + } + if (buses_want_va && !buses_want_pa) { + mode = RTE_IOVA_VA; + } else if (buses_want_pa && !buses_want_va) { + mode = RTE_IOVA_PA; + } else { + mode = RTE_IOVA_DC; + if (buses_want_va) { + RTE_LOG(WARNING, EAL, "Some buses want 'VA' but forcing 'DC' because other buses want 'PA'.\n"); + RTE_LOG(WARNING, EAL, "Depending on the final decision by the EAL, not all buses may be able to initialize.\n"); + } } return mode; From patchwork Mon Jul 22 12:56:52 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: David Marchand X-Patchwork-Id: 56877 X-Patchwork-Delegate: thomas@monjalon.net Return-Path: X-Original-To: patchwork@dpdk.org Delivered-To: patchwork@dpdk.org Received: from [92.243.14.124] (localhost [127.0.0.1]) by dpdk.org (Postfix) with ESMTP id C4ADA1BF90; Mon, 22 Jul 2019 14:57:26 +0200 (CEST) Received: from mx1.redhat.com (mx1.redhat.com [209.132.183.28]) by dpdk.org (Postfix) with ESMTP id 9F34B1BF8E for ; Mon, 22 Jul 2019 14:57:24 +0200 (CEST) Received: from smtp.corp.redhat.com (int-mx01.intmail.prod.int.phx2.redhat.com [10.5.11.11]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mx1.redhat.com (Postfix) with ESMTPS id D21983084248; Mon, 22 Jul 2019 12:57:23 +0000 (UTC) Received: from dmarchan.remote.csb (ovpn-204-177.brq.redhat.com [10.40.204.177]) by smtp.corp.redhat.com (Postfix) with ESMTP id D4D2D60497; Mon, 22 Jul 2019 12:57:20 +0000 (UTC) From: David Marchand To: dev@dpdk.org Cc: anatoly.burakov@intel.com, jerinj@marvell.com, thomas@monjalon.net, Pavan Nikhilesh , Nithin Dabilpuram , Vamsi Attunuru , Kiran Kumar K , Satha Rao Date: Mon, 22 Jul 2019 14:56:52 +0200 Message-Id: <1563800213-29839-4-git-send-email-david.marchand@redhat.com> In-Reply-To: <1563800213-29839-1-git-send-email-david.marchand@redhat.com> References: <1562795329-16652-1-git-send-email-david.marchand@redhat.com> <1563800213-29839-1-git-send-email-david.marchand@redhat.com> X-Scanned-By: MIMEDefang 2.79 on 10.5.11.11 X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-4.5.16 (mx1.redhat.com [10.5.110.40]); Mon, 22 Jul 2019 12:57:23 +0000 (UTC) Subject: [dpdk-dev] [PATCH v4 3/4] drivers: change IOVA as VA PCI flag name X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Sender: "dev" From: Jerin Jacob In order to align name with other PCI driver flag such as RTE_PCI_DRV_NEED_MAPPING and to reflect its purpose, change RTE_PCI_DRV_IOVA_AS_VA flag name as RTE_PCI_DRV_NEED_IOVA_AS_VA. Signed-off-by: Jerin Jacob Signed-off-by: David Marchand --- Changelog since v3: - updated title, --- drivers/bus/pci/linux/pci.c | 4 ++-- drivers/bus/pci/rte_bus_pci.h | 4 ++-- drivers/event/octeontx/timvf_probe.c | 2 +- drivers/event/octeontx2/otx2_evdev.c | 2 +- drivers/mempool/octeontx/octeontx_fpavf.c | 2 +- drivers/mempool/octeontx2/otx2_mempool.c | 2 +- drivers/net/octeontx2/otx2_ethdev.c | 2 +- drivers/raw/octeontx2_dma/otx2_dpi_rawdev.c | 2 +- 8 files changed, 10 insertions(+), 10 deletions(-) diff --git a/drivers/bus/pci/linux/pci.c b/drivers/bus/pci/linux/pci.c index 1a2f99b..1d8d20d 100644 --- a/drivers/bus/pci/linux/pci.c +++ b/drivers/bus/pci/linux/pci.c @@ -580,7 +580,7 @@ enum rte_iova_mode } if (is_vfio_noiommu_enabled != 0) iova_mode = RTE_IOVA_PA; - else if ((pdrv->drv_flags & RTE_PCI_DRV_IOVA_AS_VA) != 0) + else if ((pdrv->drv_flags & RTE_PCI_DRV_NEED_IOVA_AS_VA) != 0) iova_mode = RTE_IOVA_VA; #endif break; @@ -592,7 +592,7 @@ enum rte_iova_mode break; default: - if ((pdrv->drv_flags & RTE_PCI_DRV_IOVA_AS_VA) != 0) + if ((pdrv->drv_flags & RTE_PCI_DRV_NEED_IOVA_AS_VA) != 0) iova_mode = RTE_IOVA_VA; break; } diff --git a/drivers/bus/pci/rte_bus_pci.h b/drivers/bus/pci/rte_bus_pci.h index 0f21775..29bea6d 100644 --- a/drivers/bus/pci/rte_bus_pci.h +++ b/drivers/bus/pci/rte_bus_pci.h @@ -187,8 +187,8 @@ struct rte_pci_bus { #define RTE_PCI_DRV_INTR_RMV 0x0010 /** Device driver needs to keep mapped resources if unsupported dev detected */ #define RTE_PCI_DRV_KEEP_MAPPED_RES 0x0020 -/** Device driver only supports IOVA as VA and cannot work with IOVA as PA */ -#define RTE_PCI_DRV_IOVA_AS_VA 0x0040 +/** Device driver needs IOVA as VA and cannot work with IOVA as PA */ +#define RTE_PCI_DRV_NEED_IOVA_AS_VA 0x0040 /** * Map the PCI device resources in user space virtual memory address diff --git a/drivers/event/octeontx/timvf_probe.c b/drivers/event/octeontx/timvf_probe.c index 08dbd2b..af87625 100644 --- a/drivers/event/octeontx/timvf_probe.c +++ b/drivers/event/octeontx/timvf_probe.c @@ -140,7 +140,7 @@ struct timdev { static struct rte_pci_driver pci_timvf = { .id_table = pci_timvf_map, - .drv_flags = RTE_PCI_DRV_NEED_MAPPING | RTE_PCI_DRV_IOVA_AS_VA, + .drv_flags = RTE_PCI_DRV_NEED_MAPPING | RTE_PCI_DRV_NEED_IOVA_AS_VA, .probe = timvf_probe, .remove = NULL, }; diff --git a/drivers/event/octeontx2/otx2_evdev.c b/drivers/event/octeontx2/otx2_evdev.c index 56716c2..e6379e3 100644 --- a/drivers/event/octeontx2/otx2_evdev.c +++ b/drivers/event/octeontx2/otx2_evdev.c @@ -1630,7 +1630,7 @@ static struct rte_pci_driver pci_sso = { .id_table = pci_sso_map, - .drv_flags = RTE_PCI_DRV_NEED_MAPPING | RTE_PCI_DRV_IOVA_AS_VA, + .drv_flags = RTE_PCI_DRV_NEED_MAPPING | RTE_PCI_DRV_NEED_IOVA_AS_VA, .probe = otx2_sso_probe, .remove = otx2_sso_remove, }; diff --git a/drivers/mempool/octeontx/octeontx_fpavf.c b/drivers/mempool/octeontx/octeontx_fpavf.c index 4cf387e..baabc01 100644 --- a/drivers/mempool/octeontx/octeontx_fpavf.c +++ b/drivers/mempool/octeontx/octeontx_fpavf.c @@ -799,7 +799,7 @@ struct octeontx_fpadev { static struct rte_pci_driver pci_fpavf = { .id_table = pci_fpavf_map, - .drv_flags = RTE_PCI_DRV_NEED_MAPPING | RTE_PCI_DRV_IOVA_AS_VA, + .drv_flags = RTE_PCI_DRV_NEED_MAPPING | RTE_PCI_DRV_NEED_IOVA_AS_VA, .probe = fpavf_probe, }; diff --git a/drivers/mempool/octeontx2/otx2_mempool.c b/drivers/mempool/octeontx2/otx2_mempool.c index 9a5f11c..3a4a942 100644 --- a/drivers/mempool/octeontx2/otx2_mempool.c +++ b/drivers/mempool/octeontx2/otx2_mempool.c @@ -443,7 +443,7 @@ static struct rte_pci_driver pci_npa = { .id_table = pci_npa_map, - .drv_flags = RTE_PCI_DRV_NEED_MAPPING | RTE_PCI_DRV_IOVA_AS_VA, + .drv_flags = RTE_PCI_DRV_NEED_MAPPING | RTE_PCI_DRV_NEED_IOVA_AS_VA, .probe = npa_probe, .remove = npa_remove, }; diff --git a/drivers/net/octeontx2/otx2_ethdev.c b/drivers/net/octeontx2/otx2_ethdev.c index 5ec5551..7b91f6b 100644 --- a/drivers/net/octeontx2/otx2_ethdev.c +++ b/drivers/net/octeontx2/otx2_ethdev.c @@ -2001,7 +2001,7 @@ static struct rte_pci_driver pci_nix = { .id_table = pci_nix_map, - .drv_flags = RTE_PCI_DRV_NEED_MAPPING | RTE_PCI_DRV_IOVA_AS_VA | + .drv_flags = RTE_PCI_DRV_NEED_MAPPING | RTE_PCI_DRV_NEED_IOVA_AS_VA | RTE_PCI_DRV_INTR_LSC, .probe = nix_probe, .remove = nix_remove, diff --git a/drivers/raw/octeontx2_dma/otx2_dpi_rawdev.c b/drivers/raw/octeontx2_dma/otx2_dpi_rawdev.c index 6a1b436..e398abb 100644 --- a/drivers/raw/octeontx2_dma/otx2_dpi_rawdev.c +++ b/drivers/raw/octeontx2_dma/otx2_dpi_rawdev.c @@ -427,7 +427,7 @@ static struct rte_pci_driver rte_dpi_rawdev_pmd = { .id_table = pci_dma_map, - .drv_flags = RTE_PCI_DRV_NEED_MAPPING | RTE_PCI_DRV_IOVA_AS_VA, + .drv_flags = RTE_PCI_DRV_NEED_MAPPING | RTE_PCI_DRV_NEED_IOVA_AS_VA, .probe = otx2_dpi_rawdev_probe, .remove = otx2_dpi_rawdev_remove, }; From patchwork Mon Jul 22 12:56:53 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: David Marchand X-Patchwork-Id: 56878 X-Patchwork-Delegate: thomas@monjalon.net Return-Path: X-Original-To: patchwork@dpdk.org Delivered-To: patchwork@dpdk.org Received: from [92.243.14.124] (localhost [127.0.0.1]) by dpdk.org (Postfix) with ESMTP id 2C67F1BF99; Mon, 22 Jul 2019 14:57:31 +0200 (CEST) Received: from mx1.redhat.com (mx1.redhat.com [209.132.183.28]) by dpdk.org (Postfix) with ESMTP id A5CCC1BF8E for ; Mon, 22 Jul 2019 14:57:26 +0200 (CEST) Received: from smtp.corp.redhat.com (int-mx01.intmail.prod.int.phx2.redhat.com [10.5.11.11]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mx1.redhat.com (Postfix) with ESMTPS id 1D7C73DBC5; Mon, 22 Jul 2019 12:57:26 +0000 (UTC) Received: from dmarchan.remote.csb (ovpn-204-177.brq.redhat.com [10.40.204.177]) by smtp.corp.redhat.com (Postfix) with ESMTP id 5DD2061377; Mon, 22 Jul 2019 12:57:24 +0000 (UTC) From: David Marchand To: dev@dpdk.org Cc: anatoly.burakov@intel.com, jerinj@marvell.com, thomas@monjalon.net, John McNamara , Marko Kovacevic Date: Mon, 22 Jul 2019 14:56:53 +0200 Message-Id: <1563800213-29839-5-git-send-email-david.marchand@redhat.com> In-Reply-To: <1563800213-29839-1-git-send-email-david.marchand@redhat.com> References: <1562795329-16652-1-git-send-email-david.marchand@redhat.com> <1563800213-29839-1-git-send-email-david.marchand@redhat.com> X-Scanned-By: MIMEDefang 2.79 on 10.5.11.11 X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-4.5.16 (mx1.redhat.com [10.5.110.30]); Mon, 22 Jul 2019 12:57:26 +0000 (UTC) Subject: [dpdk-dev] [PATCH v4 4/4] eal: select IOVA as VA mode for default case X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Sender: "dev" From: Jerin Jacob When bus layer reports the preferred mode as RTE_IOVA_DC then select the RTE_IOVA_VA mode: - All drivers work in RTE_IOVA_VA mode, irrespective of physical address availability. - By default, a mempool asks for IOVA-contiguous memory using RTE_MEMZONE_IOVA_CONTIG. This is slow in RTE_IOVA_PA mode and it may affect the application boot time. Signed-off-by: Jerin Jacob Acked-by: Anatoly Burakov Signed-off-by: David Marchand --- Changelog since v3: - moved the explanations on RTE_IOVA_VA choice in the note section, - reworded the comments on the PCI driver flag, --- doc/guides/prog_guide/env_abstraction_layer.rst | 22 ++++++++++++++++++++-- lib/librte_eal/linux/eal/eal.c | 6 ++---- 2 files changed, 22 insertions(+), 6 deletions(-) diff --git a/doc/guides/prog_guide/env_abstraction_layer.rst b/doc/guides/prog_guide/env_abstraction_layer.rst index 1d63675..1487ea5 100644 --- a/doc/guides/prog_guide/env_abstraction_layer.rst +++ b/doc/guides/prog_guide/env_abstraction_layer.rst @@ -445,11 +445,29 @@ kernels. - if the preferred mode is RTE_IOVA_PA but there is no access to Physical Addresses, then EAL init fails early, since later probing of the devices would fail anyway, -- if the preferred mode is RTE_IOVA_DC then based on the Physical Addresses - availability, the preferred mode is adjusted to RTE_IOVA_PA or RTE_IOVA_VA. +- if the preferred mode is RTE_IOVA_DC then EAL selects the RTE_IOVA_VA mode. In the case when the buses had disagreed on the IOVA Mode at the first step, part of the buses won't work because of this decision. +.. note:: + + The RTE_IOVA_VA mode is selected as the default for the following reasons: + + - All drivers are expected to work in RTE_IOVA_VA mode, irrespective of + physical address availability. + - By default, the mempool, first asks for IOVA-contiguous memory using + ``RTE_MEMZONE_IOVA_CONTIG``. This is slow in RTE_IOVA_PA mode and it may + affect the application boot time. + - It is easy to enable large amount of IOVA-contiguous memory use-cases + with IOVA in VA mode. + + It is expected that all PCI drivers work in both RTE_IOVA_PA and + RTE_IOVA_VA modes. + + If a PCI driver does not support RTE_IOVA_PA mode, the + ``RTE_PCI_DRV_NEED_IOVA_AS_VA`` flag is used to dictate that this PCI + driver can only work in RTE_IOVA_VA mode. + IOVA Mode Configuration ~~~~~~~~~~~~~~~~~~~~~~~ diff --git a/lib/librte_eal/linux/eal/eal.c b/lib/librte_eal/linux/eal/eal.c index 2e5499f..34db787 100644 --- a/lib/librte_eal/linux/eal/eal.c +++ b/lib/librte_eal/linux/eal/eal.c @@ -1061,10 +1061,8 @@ static void rte_eal_init_alert(const char *msg) enum rte_iova_mode iova_mode = rte_bus_get_iommu_class(); if (iova_mode == RTE_IOVA_DC) { - iova_mode = phys_addrs ? RTE_IOVA_PA : RTE_IOVA_VA; - RTE_LOG(DEBUG, EAL, - "Buses did not request a specific IOVA mode, using '%s' based on physical addresses availability.\n", - phys_addrs ? "PA" : "VA"); + iova_mode = RTE_IOVA_VA; + RTE_LOG(DEBUG, EAL, "Buses did not request a specific IOVA mode, select IOVA as VA mode.\n"); } #ifdef RTE_LIBRTE_KNI /* Workaround for KNI which requires physical address to work */