From patchwork Tue Sep 15 11:14:59 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Dybkowski, AdamX" X-Patchwork-Id: 77706 X-Patchwork-Delegate: gakhil@marvell.com Return-Path: X-Original-To: patchwork@inbox.dpdk.org Delivered-To: patchwork@inbox.dpdk.org Received: from dpdk.org (dpdk.org [92.243.14.124]) by inbox.dpdk.org (Postfix) with ESMTP id D63E6A04C7; Tue, 15 Sep 2020 13:15:13 +0200 (CEST) Received: from [92.243.14.124] (localhost [127.0.0.1]) by dpdk.org (Postfix) with ESMTP id E27F41C0C2; Tue, 15 Sep 2020 13:15:06 +0200 (CEST) Received: from mga11.intel.com (mga11.intel.com [192.55.52.93]) by dpdk.org (Postfix) with ESMTP id 547EDE07 for ; Tue, 15 Sep 2020 13:15:04 +0200 (CEST) IronPort-SDR: 2uhb2/EUzgzs3/WyuBHxMY1kiLEHwaVZ9ATMcxz11FFoxEOt/xhSicLgUBQorddkrlG2CGNRvX swevt7kz4w2w== X-IronPort-AV: E=McAfee;i="6000,8403,9744"; a="156666205" X-IronPort-AV: E=Sophos;i="5.76,429,1592895600"; d="scan'208";a="156666205" X-Amp-Result: SKIPPED(no attachment in message) X-Amp-File-Uploaded: False Received: from orsmga002.jf.intel.com ([10.7.209.21]) by fmsmga102.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 15 Sep 2020 04:15:03 -0700 IronPort-SDR: NsfvhHMoyfQOHJ6dS/FKk4xTCBVt76NTgsKBB6MFe/YCSQfucUm27eG0KtxMlaNQE1El10yTw2 6u+/S3htJBJg== X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="5.76,429,1592895600"; d="scan'208";a="319390366" Received: from unknown (HELO adamdybx-MOBL.ger.corp.intel.com) ([10.104.113.51]) by orsmga002.jf.intel.com with ESMTP; 15 Sep 2020 04:15:02 -0700 From: Adam Dybkowski To: fiona.trahe@intel.com, akhil.goyal@nxp.com, dev@dpdk.org, anatoly.burakov@intel.com Cc: Adam Dybkowski Date: Tue, 15 Sep 2020 13:14:59 +0200 Message-Id: <20200915111459.456-2-adamx.dybkowski@intel.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20200915111459.456-1-adamx.dybkowski@intel.com> References: <20200908161950.601-1-adamx.dybkowski@intel.com> <20200915111459.456-1-adamx.dybkowski@intel.com> MIME-Version: 1.0 Subject: [dpdk-dev] [PATCH v3 1/1] doc: document vfio-pci usage with QAT PMD X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Sender: "dev" This patch marks the old igb_uio driver as unsecure when used with the QAT PMD and updates all examples to recommend using vfio-pci instead. It also mentions security issues with the QAT CPM and provides information about the new vfio-pci parameter 'disable_denylist' available in Linux kernels 5.9 and later. Signed-off-by: Adam Dybkowski Acked-by: Anatoly Burakov --- doc/guides/cryptodevs/qat.rst | 63 +++++++++++++++++++---------------- 1 file changed, 34 insertions(+), 29 deletions(-) diff --git a/doc/guides/cryptodevs/qat.rst b/doc/guides/cryptodevs/qat.rst index e5d2cf499..dbbdec1c7 100644 --- a/doc/guides/cryptodevs/qat.rst +++ b/doc/guides/cryptodevs/qat.rst @@ -462,7 +462,7 @@ Check that the VFs are available for use. For example ``lspci -d:37c9`` should list 48 VF devices available for a ``C62x`` device. To complete the installation follow the instructions in -`Binding the available VFs to the DPDK UIO driver`_. +`Binding the available VFs to the vfio-pci driver`_. .. Note:: @@ -534,7 +534,8 @@ Confirm the presence of 48 VF devices - 16 per PF:: lspci -d:37c9 -To complete the installation - follow instructions in `Binding the available VFs to the DPDK UIO driver`_. +To complete the installation - follow instructions in +`Binding the available VFs to the vfio-pci driver`_. .. Note:: @@ -584,10 +585,21 @@ To complete the installation - follow instructions in `Binding the available VFs sudo yum install kernel-devel-`uname -r` -Binding the available VFs to the DPDK UIO driver +Binding the available VFs to the vfio-pci driver ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ -Unbind the VFs from the stock driver so they can be bound to the uio driver. +Note: + +* Please note that due to security issues, the usage of older DPDK igb_uio + driver is not recommended. This document shows how to use the more secure + vfio-pci driver. +* If QAT fails to bind to vfio-pci on Linux kernel 5.9+, please see the + QATE-39220 and QATE-7495 issues in + `01.org doc `_ + which details the constraint about trusted guests and add `disable_denylist=1` + to the vfio-pci params to use QAT. See also `this patch description `_. + +Unbind the VFs from the stock driver so they can be bound to the vfio-pci driver. For an Intel(R) QuickAssist Technology DH895xCC device ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ @@ -595,10 +607,10 @@ For an Intel(R) QuickAssist Technology DH895xCC device The unbind command below assumes ``BDFs`` of ``03:01.00-03:04.07``, if your VFs are different adjust the unbind command below:: + cd to the top-level DPDK directory for device in $(seq 1 4); do \ for fn in $(seq 0 7); do \ - echo -n 0000:03:0${device}.${fn} > \ - /sys/bus/pci/devices/0000\:03\:0${device}.${fn}/driver/unbind; \ + usertools/dpdk-devbind.py -u 0000:03:0${device}.${fn}; \ done; \ done @@ -609,16 +621,12 @@ The unbind command below assumes ``BDFs`` of ``1a:01.00-1a:02.07``, ``3d:01.00-3d:02.07`` and ``3f:01.00-3f:02.07``, if your VFs are different adjust the unbind command below:: + cd to the top-level DPDK directory for device in $(seq 1 2); do \ for fn in $(seq 0 7); do \ - echo -n 0000:1a:0${device}.${fn} > \ - /sys/bus/pci/devices/0000\:1a\:0${device}.${fn}/driver/unbind; \ - - echo -n 0000:3d:0${device}.${fn} > \ - /sys/bus/pci/devices/0000\:3d\:0${device}.${fn}/driver/unbind; \ - - echo -n 0000:3f:0${device}.${fn} > \ - /sys/bus/pci/devices/0000\:3f\:0${device}.${fn}/driver/unbind; \ + usertools/dpdk-devbind.py -u 0000:1a:0${device}.${fn}; \ + usertools/dpdk-devbind.py -u 0000:3d:0${device}.${fn}; \ + usertools/dpdk-devbind.py -u 0000:3f:0${device}.${fn}; \ done; \ done @@ -628,32 +636,29 @@ For Intel(R) QuickAssist Technology C3xxx or 200xx or D15xx device The unbind command below assumes ``BDFs`` of ``01:01.00-01:02.07``, if your VFs are different adjust the unbind command below:: + cd to the top-level DPDK directory for device in $(seq 1 2); do \ for fn in $(seq 0 7); do \ - echo -n 0000:01:0${device}.${fn} > \ - /sys/bus/pci/devices/0000\:01\:0${device}.${fn}/driver/unbind; \ + usertools/dpdk-devbind.py -u 0000:01:0${device}.${fn}; \ done; \ done -Bind to the DPDK uio driver +Bind to the vfio-pci driver ^^^^^^^^^^^^^^^^^^^^^^^^^^^ -Install the DPDK igb_uio driver, bind the VF PCI Device id to it and use lspci -to confirm the VF devices are now in use by igb_uio kernel driver, +Load the vfio-pci driver, bind the VF PCI Device id to it using the +``dpdk-devbind.py`` script then use the ``--status`` option +to confirm the VF devices are now in use by vfio-pci kernel driver, e.g. for the C62x device:: cd to the top-level DPDK directory - modprobe uio - insmod ./build/kmod/igb_uio.ko - echo "8086 37c9" > /sys/bus/pci/drivers/igb_uio/new_id - lspci -vvd:37c9 - + modprobe vfio-pci + usertools/dpdk-devbind.py -b vfio-pci 0000:03:01.1 + usertools/dpdk-devbind.py --status -Another way to bind the VFs to the DPDK UIO driver is by using the -``dpdk-devbind.py`` script:: - - cd to the top-level DPDK directory - ./usertools/dpdk-devbind.py -b igb_uio 0000:03:01.1 +Use ``modprobe vfio-pci disable_denylist=1`` from kernel 5.9 onwards. +See note in the section `Binding the available VFs to the vfio-pci driver`_ +above. Testing ~~~~~~~