From patchwork Wed Feb 16 16:06:07 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Bruce Richardson X-Patchwork-Id: 107696 X-Patchwork-Delegate: thomas@monjalon.net Return-Path: X-Original-To: patchwork@inbox.dpdk.org Delivered-To: patchwork@inbox.dpdk.org Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id F3554A00C5; Wed, 16 Feb 2022 17:09:32 +0100 (CET) Received: from [217.70.189.124] (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id E53FC410FF; Wed, 16 Feb 2022 17:09:32 +0100 (CET) Received: from mga01.intel.com (mga01.intel.com [192.55.52.88]) by mails.dpdk.org (Postfix) with ESMTP id E079940150; Wed, 16 Feb 2022 17:09:30 +0100 (CET) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1645027771; x=1676563771; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=uSBsEdJ227f5pPe3zyuctcmrVBvM0pd0xy3gNXYg12Q=; b=KSEtns9xn10mUGn1IsNEbVBcYQJKXimcqovt2os/kbZEnlsV8doUESfD CsIwDrRWR9jPUxoIIhtJN0oCNP0m1hSOXdDB+gpi5Mz2N3dNVQE31JVNX 1ezAHYnmo9BWJMMjDlSuXyDQuECelGl8MZl8ot9tomibuXi1sILgGpr1/ CMEWMvmed9USqmvPChMOIs0O1bUxuxl5D0fNtQwTSiJv6GMU0iuDLWbhO 9uuInpXi7+XNJFZlh0BAqLmgKaicO2CMzYko9rjFl4dRYAo5b0TUcaVtn nvhOqrCKoHj1ha8qas9d+a9RtDYqrQfdMF6NJupYiL1vfz++834/BKnHC A==; X-IronPort-AV: E=McAfee;i="6200,9189,10260"; a="275236466" X-IronPort-AV: E=Sophos;i="5.88,374,1635231600"; d="scan'208";a="275236466" Received: from orsmga005.jf.intel.com ([10.7.209.41]) by fmsmga101.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 16 Feb 2022 08:06:32 -0800 X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="5.88,374,1635231600"; d="scan'208";a="704363183" Received: from silpixa00401026.ir.intel.com ([10.243.22.62]) by orsmga005.jf.intel.com with ESMTP; 16 Feb 2022 08:06:30 -0800 From: Bruce Richardson To: dev@dpdk.org Cc: Bruce Richardson , kevin.laatz@intel.com, stable@dpdk.org, Chengwen Feng , Conor Walsh Subject: [PATCH 1/3] app/test: fix missing checks for DMA device capacity Date: Wed, 16 Feb 2022 16:06:07 +0000 Message-Id: <20220216160610.475242-2-bruce.richardson@intel.com> X-Mailer: git-send-email 2.32.0 In-Reply-To: <20220216160610.475242-1-bruce.richardson@intel.com> References: <20220216160610.475242-1-bruce.richardson@intel.com> MIME-Version: 1.0 X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org For some DMA HW devices, e.g. those using the idxd driver, the maximum burst size is configurable, which can lead to test failures if the value is set too small. Add explicit check for this to give reasonable error messages for devices which need their config adjusted. Fixes: 1b86a66a30c2 ("test/dma: add more comprehensive copy tests") Fixes: 8fa5d2683940 ("test/dma: add burst capacity test") Cc: kevin.laatz@intel.com Cc: stable@dpdk.org Signed-off-by: Bruce Richardson Reviewed-by: Conor Walsh Reviewed-by: Kevin Laatz --- app/test/test_dmadev.c | 8 +++++++- 1 file changed, 7 insertions(+), 1 deletion(-) diff --git a/app/test/test_dmadev.c b/app/test/test_dmadev.c index db5aff701c..2b097e0f47 100644 --- a/app/test/test_dmadev.c +++ b/app/test/test_dmadev.c @@ -775,6 +775,9 @@ test_dmadev_instance(int16_t dev_id) if (rte_dma_stats_get(dev_id, vchan, &stats) != 0) ERR_RETURN("Error with rte_dma_stats_get()\n"); + if (rte_dma_burst_capacity(dev_id, vchan) < 32) + ERR_RETURN("Error: Device does not have sufficient burst capacity to run tests"); + if (stats.completed != 0 || stats.submitted != 0 || stats.errors != 0) ERR_RETURN("Error device stats are not all zero: completed = %"PRIu64", " "submitted = %"PRIu64", errors = %"PRIu64"\n", @@ -796,7 +799,10 @@ test_dmadev_instance(int16_t dev_id) goto err; /* run some burst capacity tests */ - if (runtest("burst capacity", test_burst_capacity, 1, dev_id, vchan, CHECK_ERRS) < 0) + if (rte_dma_burst_capacity(dev_id, vchan) < 64) + printf("DMA Dev %u: insufficient burst capacity (64 required), skipping tests\n", + dev_id); + else if (runtest("burst capacity", test_burst_capacity, 1, dev_id, vchan, CHECK_ERRS) < 0) goto err; /* to test error handling we can provide null pointers for source or dest in copies. This From patchwork Wed Feb 16 16:06:08 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Bruce Richardson X-Patchwork-Id: 107695 X-Patchwork-Delegate: thomas@monjalon.net Return-Path: X-Original-To: patchwork@inbox.dpdk.org Delivered-To: patchwork@inbox.dpdk.org Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id A468FA00C5; Wed, 16 Feb 2022 17:08:29 +0100 (CET) Received: from [217.70.189.124] (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id 9670941169; Wed, 16 Feb 2022 17:08:29 +0100 (CET) Received: from mga14.intel.com (mga14.intel.com [192.55.52.115]) by mails.dpdk.org (Postfix) with ESMTP id F3ADF40150; Wed, 16 Feb 2022 17:08:27 +0100 (CET) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1645027708; x=1676563708; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=3ENkEFvJhg0TDFwKtY5RLQGlBw6dmvox33aQ85eDMf8=; b=YaWYzpfPQ5smhM2tnA8e2UTki1IBgJy4nOnM0TZky7/WVxOwPZC5p8dy 74YZa7edGltDCujEgUD9Gwm8P3NFtCLCeczKumtloIrE0xxfkf+Z3wJAG jrLQ69laGc6PrCu+/82JVGN3PaM41xC9h/ziqLgr0M99EF49+GZ4XlYSV /ZHcK7v0GyOe/BcXqWb+nHxEb9FFAXwaXqY41Pd6d93fOxcKm+pdMmQbp SknFtziZ8uRlR2NpSopEerzFUvmwvlEbeIHgRcQCipTXrLy1X5X9H8LWE vEHvPFsNZPjvFKTrSgRZUsslBleSdMZ9jn9H0qndtB53NqaeggMdjtfvL A==; X-IronPort-AV: E=McAfee;i="6200,9189,10260"; a="250853269" X-IronPort-AV: E=Sophos;i="5.88,374,1635231600"; d="scan'208";a="250853269" Received: from orsmga005.jf.intel.com ([10.7.209.41]) by fmsmga103.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 16 Feb 2022 08:06:39 -0800 X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="5.88,374,1635231600"; d="scan'208";a="704363259" Received: from silpixa00401026.ir.intel.com ([10.243.22.62]) by orsmga005.jf.intel.com with ESMTP; 16 Feb 2022 08:06:37 -0800 From: Bruce Richardson To: dev@dpdk.org Cc: Bruce Richardson , stable@dpdk.org, Kevin Laatz Subject: [PATCH 2/3] dma/idxd: configure max batch size to high value Date: Wed, 16 Feb 2022 16:06:08 +0000 Message-Id: <20220216160610.475242-3-bruce.richardson@intel.com> X-Mailer: git-send-email 2.32.0 In-Reply-To: <20220216160610.475242-1-bruce.richardson@intel.com> References: <20220216160610.475242-1-bruce.richardson@intel.com> MIME-Version: 1.0 X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org When configuring an Intel DSA instance using the utility script dpdk_idxd_cfg.py, explicitly set the max supported batch size value to a high value, to ensure large bursts are supported if so desired. The default in the linux kernel is now just 32 [1], which may not be sufficient for all DPDK apps. [1] https://lore.kernel.org/r/163528473483.3926048.7950067926287180976.stgit@djiang5-desk3.ch.intel.com Cc: stable@dpdk.org Signed-off-by: Bruce Richardson Reviewed-by: Kevin Laatz --- Note: although not fixing a bug in DPDK itself, due to kernel changes this patch should be considered for backport as a fix, so cc'ing stable. --- drivers/dma/idxd/dpdk_idxd_cfg.py | 1 + 1 file changed, 1 insertion(+) diff --git a/drivers/dma/idxd/dpdk_idxd_cfg.py b/drivers/dma/idxd/dpdk_idxd_cfg.py index 34537cb980..3f5d5ee752 100755 --- a/drivers/dma/idxd/dpdk_idxd_cfg.py +++ b/drivers/dma/idxd/dpdk_idxd_cfg.py @@ -89,6 +89,7 @@ def configure_dsa(dsa_id, queues, prefix): "mode": "dedicated", "name": f"{prefix}_wq{dsa_id}.{q}", "priority": 1, + "max_batch_size": 1024, "size": int(max_work_queues_size / nb_queues)}) # enable device and then queues From patchwork Wed Feb 16 16:06:09 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Bruce Richardson X-Patchwork-Id: 107694 X-Patchwork-Delegate: thomas@monjalon.net Return-Path: X-Original-To: patchwork@inbox.dpdk.org Delivered-To: patchwork@inbox.dpdk.org Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id 5FE93A00C5; Wed, 16 Feb 2022 17:06:49 +0100 (CET) Received: from [217.70.189.124] (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id 4DA5D410FF; Wed, 16 Feb 2022 17:06:49 +0100 (CET) Received: from mga17.intel.com (mga17.intel.com [192.55.52.151]) by mails.dpdk.org (Postfix) with ESMTP id 6B8D240150; Wed, 16 Feb 2022 17:06:47 +0100 (CET) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1645027607; x=1676563607; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=aBzsuVz+Ia2E0V8xdESboLKQem4VrwwOF+35mO99IwU=; b=FzGAJwkK+joFYu9fBVO7PUvgLCHi3/8OGrMhoy7lKyhR2SafFLa+5ljG GLwQzFLuj9DAo2LGLNeIKhiHV01lnV2d52/RWQ25Aq2c8r61fzkYQEdWg FKpkiJKJ/SXQIRYadMxR1r0kwEa+gyMRRYFyQ9jSsaumCi84jl3c9u9qd 4c/qA2SrL8bqNGVjR5j8s4sqFrPTHA24LSXZGGjiRbwly85Cpl6Rrhb1G tQbVndV7W58J7iAEiO0heSteLfcLEsRmuGlrI6gEbWu2lavdKjFKwdLRF 4t37WICQ68aa105F0cCMxsMZ8FnSrH8BJn44/1IzYnM3MExnF3HZ/lQCZ Q==; X-IronPort-AV: E=McAfee;i="6200,9189,10260"; a="231279735" X-IronPort-AV: E=Sophos;i="5.88,374,1635231600"; d="scan'208";a="231279735" Received: from orsmga005.jf.intel.com ([10.7.209.41]) by fmsmga107.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 16 Feb 2022 08:06:46 -0800 X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="5.88,374,1635231600"; d="scan'208";a="704363290" Received: from silpixa00401026.ir.intel.com ([10.243.22.62]) by orsmga005.jf.intel.com with ESMTP; 16 Feb 2022 08:06:44 -0800 From: Bruce Richardson To: dev@dpdk.org Cc: Bruce Richardson , stable@dpdk.org, Kevin Laatz Subject: [PATCH 3/3] doc/dmadev/idxd: improve configuration examples Date: Wed, 16 Feb 2022 16:06:09 +0000 Message-Id: <20220216160610.475242-4-bruce.richardson@intel.com> X-Mailer: git-send-email 2.32.0 In-Reply-To: <20220216160610.475242-1-bruce.richardson@intel.com> References: <20220216160610.475242-1-bruce.richardson@intel.com> MIME-Version: 1.0 X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org The documentation on how to configure device instances using accel-config can be improved by a number of changes: * For initial example, when only configuring one queue, omit configuration of a second engine, which is unused later. * Add the "max-batch-size" setting to the options being configured for each queue * Add a final, more complete example, showing configuration of multiple queues on a device. Cc: stable@dpdk.org Signed-off-by: Bruce Richardson Reviewed-by: Kevin Laatz --- Note: although not strictly a bug fix, this doc update is a good candidate for backport to 21.11, so cc'ing stable --- doc/guides/dmadevs/idxd.rst | 29 +++++++++++++++++++++++++++-- 1 file changed, 27 insertions(+), 2 deletions(-) diff --git a/doc/guides/dmadevs/idxd.rst b/doc/guides/dmadevs/idxd.rst index 8d7cea6583..bdfd3e78ad 100644 --- a/doc/guides/dmadevs/idxd.rst +++ b/doc/guides/dmadevs/idxd.rst @@ -55,7 +55,6 @@ such as priority or queue depth, need to be set for each queue. To assign an engine to a group:: $ accel-config config-engine dsa0/engine0.0 --group-id=0 - $ accel-config config-engine dsa0/engine0.1 --group-id=1 To assign work queues to groups for passing descriptors to the engines a similar accel-config command can be used. However, the work queues also need to be configured depending on the use case. @@ -71,7 +70,7 @@ Example configuration for a work queue:: $ accel-config config-wq dsa0/wq0.0 --group-id=0 \ --mode=dedicated --priority=10 --wq-size=8 \ - --type=user --name=dpdk_app1 + --max-batch-size=512 --type=user --name=dpdk_app1 Once the devices have been configured, they need to be enabled:: @@ -82,6 +81,32 @@ Check the device configuration:: $ accel-config list +Every Intel\ |reg| DSA instance supports multiple queues and each should be similarly configured. +As a further example, the following set of commands will configure and enable 4 queues on instance 0, +giving each an equal share of resources:: + + # configure 4 groups, each with one engine + accel-config config-engine dsa0/engine0.0 --group-id=0 + accel-config config-engine dsa0/engine0.1 --group-id=1 + accel-config config-engine dsa0/engine0.2 --group-id=2 + accel-config config-engine dsa0/engine0.3 --group-id=3 + + # configure 4 queues, putting each in a different group, so each + # is backed by a single engine + accel-config config-wq dsa0/wq0.0 --group-id=0 --type=user --wq-size=32 \ + --priority=10 --max-batch-size=1024 --mode=dedicated --name=dpdk_app1 + accel-config config-wq dsa0/wq0.1 --group-id=1 --type=user --wq-size=32 \ + --priority=10 --max-batch-size=1024 --mode=dedicated --name=dpdk_app1 + accel-config config-wq dsa0/wq0.2 --group-id=2 --type=user --wq-size=32 \ + --priority=10 --max-batch-size=1024 --mode=dedicated --name=dpdk_app1 + accel-config config-wq dsa0/wq0.3 --group-id=3 --type=user --wq-size=32 \ + --priority=10 --max-batch-size=1024 --mode=dedicated --name=dpdk_app1 + + # enable device and queues + accel-config enable-device dsa0 + accel-config enable-wq dsa0/wq0.0 dsa0/wq0.1 dsa0/wq0.2 dsa0/wq0.3 + + Devices using VFIO/UIO drivers ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~