Message ID | 1602533402-14795-1-git-send-email-viacheslavo@nvidia.com (mailing list archive) |
---|---|
Headers |
Return-Path: <dev-bounces@dpdk.org> X-Original-To: patchwork@inbox.dpdk.org Delivered-To: patchwork@inbox.dpdk.org Received: from dpdk.org (dpdk.org [92.243.14.124]) by inbox.dpdk.org (Postfix) with ESMTP id B088CA04B6; Mon, 12 Oct 2020 22:10:20 +0200 (CEST) Received: from [92.243.14.124] (localhost [127.0.0.1]) by dpdk.org (Postfix) with ESMTP id F3CF61D9BA; Mon, 12 Oct 2020 22:10:18 +0200 (CEST) Received: from mellanox.co.il (mail-il-dmz.mellanox.com [193.47.165.129]) by dpdk.org (Postfix) with ESMTP id D72311D9B8 for <dev@dpdk.org>; Mon, 12 Oct 2020 22:10:16 +0200 (CEST) Received: from Internal Mail-Server by MTLPINE1 (envelope-from viacheslavo@nvidia.com) with SMTP; 12 Oct 2020 23:10:13 +0300 Received: from nvidia.com (pegasus12.mtr.labs.mlnx [10.210.17.40]) by labmailer.mlnx (8.13.8/8.13.8) with ESMTP id 09CKADf8013670; Mon, 12 Oct 2020 23:10:13 +0300 From: Viacheslav Ovsiienko <viacheslavo@nvidia.com> To: dev@dpdk.org Cc: thomasm@monjalon.net, stephen@networkplumber.org, ferruh.yigit@intel.com, olivier.matz@6wind.com, jerinjacobk@gmail.com, maxime.coquelin@redhat.com, david.marchand@redhat.com, arybchenko@solarflare.com Date: Mon, 12 Oct 2020 20:09:53 +0000 Message-Id: <1602533402-14795-1-git-send-email-viacheslavo@nvidia.com> X-Mailer: git-send-email 1.8.3.1 In-Reply-To: <MWHPR12MB136076E652230CEBD6EE6562DF5F0@MWHPR12MB1360.namprd12.prod.outlook.com> References: <MWHPR12MB136076E652230CEBD6EE6562DF5F0@MWHPR12MB1360.namprd12.prod.outlook.com> Subject: [dpdk-dev] [PATCH v4 0/9] ethdev: introduce Rx buffer split X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: DPDK patches and discussions <dev.dpdk.org> List-Unsubscribe: <https://mails.dpdk.org/options/dev>, <mailto:dev-request@dpdk.org?subject=unsubscribe> List-Archive: <http://mails.dpdk.org/archives/dev/> List-Post: <mailto:dev@dpdk.org> List-Help: <mailto:dev-request@dpdk.org?subject=help> List-Subscribe: <https://mails.dpdk.org/listinfo/dev>, <mailto:dev-request@dpdk.org?subject=subscribe> Errors-To: dev-bounces@dpdk.org Sender: "dev" <dev-bounces@dpdk.org> |
Series |
ethdev: introduce Rx buffer split
|
|
Message
Slava Ovsiienko
Oct. 12, 2020, 8:09 p.m. UTC
The DPDK datapath in the transmit direction is very flexible.
An application can build the multi-segment packet and manages
almost all data aspects - the memory pools where segments
are allocated from, the segment lengths, the memory attributes
like external buffers, registered for DMA, etc.
In the receiving direction, the datapath is much less flexible,
an application can only specify the memory pool to configure the
receiving queue and nothing more. In order to extend receiving
datapath capabilities it is proposed to add the way to provide
extended information how to split the packets being received.
The following structure is introduced to specify the Rx packet
segment:
struct rte_eth_rxseg {
struct rte_mempool *mp; /* memory pools to allocate segment from */
uint16_t length; /* segment maximal data length,
configures "split point" */
uint16_t offset; /* data offset from beginning
of mbuf data buffer */
uint32_t reserved; /* reserved field */
};
The new routine rte_eth_rxseg_queue_setup_ex() is introduced to
setup the given Rx queue using the new extended Rx packet segment
description:
int
rte_eth_rx_queue_setup_ex(uint16_t port_id, uint16_t rx_queue_id,
uint16_t nb_rx_desc, unsigned int socket_id,
const struct rte_eth_rxconf *rx_conf,
const struct rte_eth_rxseg *rx_seg,
uint16_t n_seg)
This routine presents the two new parameters:
rx_seg - pointer the array of segment descriptions, each element
describes the memory pool, maximal data length, initial
data offset from the beginning of data buffer in mbuf.
This array allows to specify the different settings for
each segment in individual fashion.
n_seg - number of elements in the array
The new offload flag RTE_ETH_RX_OFFLOAD_BUFFER_SPLIT in device
capabilities is introduced to present the way for PMD to report to
application about supporting Rx packet split to configurable
segments. Prior invoking the rte_eth_rx_queue_setup_ex() routine
application should check RTE_ETH_RX_OFFLOAD_BUFFER_SPLIT flag.
If the Rx queue is configured with new routine the packets being
received will be split into multiple segments pushed to the mbufs
with specified attributes. The PMD will split the received packets
into multiple segments according to the specification in the
description array:
- the first network buffer will be allocated from the memory pool,
specified in the first segment description element, the second
network buffer - from the pool in the second segment description
element and so on. If there is no enough elements to describe
the buffer for entire packet of maximal length the pool from the
last valid element will be used to allocate the buffers from for the
rest of segments
- the offsets from the segment description elements will provide
the data offset from the buffer beginning except the first mbuf -
for this one the offset is added to the RTE_PKTMBUF_HEADROOM to get
actual offset from the buffer beginning. If there is no enough
elements to describe the buffer for entire packet of maximal length
the offsets for the rest of segment will be supposed to be zero.
- the data length being received to each segment is limited by the
length specified in the segment description element. The data
receiving starts with filling up the first mbuf data buffer, if the
specified maximal segment length is reached and there are data
remaining (packet is longer than buffer in the first mbuf) the
following data will be pushed to the next segment up to its own
maximal length. If the first two segments is not enough to store
all the packet remaining data the next (third) segment will
be engaged and so on. If the length in the segment description
element is zero the actual buffer size will be deduced from
the appropriate memory pool properties. If there is no enough
elements to describe the buffer for entire packet of maximal
length the buffer size will be deduced from the pool of the last
valid element for the remaining segments.
For example, let's suppose we configured the Rx queue with the
following segments:
seg0 - pool0, len0=14B, off0=2
seg1 - pool1, len1=20B, off1=128B
seg2 - pool2, len2=20B, off2=0B
seg3 - pool3, len3=512B, off3=0B
The packet 46 bytes long will look like the following:
seg0 - 14B long @ RTE_PKTMBUF_HEADROOM + 2 in mbuf from pool0
seg1 - 20B long @ 128 in mbuf from pool1
seg2 - 12B long @ 0 in mbuf from pool2
The packet 1500 bytes long will look like the following:
seg0 - 14B @ RTE_PKTMBUF_HEADROOM + 2 in mbuf from pool0
seg1 - 20B @ 128 in mbuf from pool1
seg2 - 20B @ 0 in mbuf from pool2
seg3 - 512B @ 0 in mbuf from pool3
seg4 - 512B @ 0 in mbuf from pool3
seg5 - 422B @ 0 in mbuf from pool3
The offload RTE_ETH_RX_OFFLOAD_SCATTER must be present and
configured to support new buffer split feature (if n_seg
is greater than one).
The new approach would allow splitting the ingress packets into
multiple parts pushed to the memory with different attributes.
For example, the packet headers can be pushed to the embedded
data buffers within mbufs and the application data into
the external buffers attached to mbufs allocated from the
different memory pools. The memory attributes for the split
parts may differ either - for example the application data
may be pushed into the external memory located on the dedicated
physical device, say GPU or NVMe. This would improve the DPDK
receiving datapath flexibility with preserving compatibility
with existing API.
Signed-off-by: Viacheslav Ovsiienko <viacheslavo@nvidia.com>
---
v1: http://patches.dpdk.org/patch/79594/
v2: http://patches.dpdk.org/patch/79893/
- add feature support to mlx5 PMD
v3: http://patches.dpdk.org/patch/80389/
- rte_eth_rx_queue_setup_ex is renamed to rte_eth_rxseg_queue_setup
- DEV_RX_OFFLOAD_BUFFER_SPLIT is renamed to
RTE_ETH_RX_OFFLOAD_BUFFER_SPLIT
- commit message update
- documentaion provided
- release notes update
- minor bug fixes in testpmd related part
v4: - common part of rx_queue_setup/rxseg_queue_setup
Viacheslav Ovsiienko (9):
ethdev: introduce Rx buffer split
app/testpmd: add multiple pools per core creation
app/testpmd: add buffer split offload configuration
app/testpmd: add rxpkts commands and parameters
app/testpmd: add extended Rx queue setup
net/mlx5: add extended Rx queue setup routine
net/mlx5: configure Rx queue to support split
net/mlx5: register multiple pool for Rx queue
net/mlx5: update Rx datapath to support split
app/test-pmd/bpf_cmd.c | 4 +-
app/test-pmd/cmdline.c | 96 +++++++++++---
app/test-pmd/config.c | 63 ++++++++-
app/test-pmd/parameters.c | 39 +++++-
app/test-pmd/testpmd.c | 108 +++++++++++-----
app/test-pmd/testpmd.h | 41 +++++-
doc/guides/nics/features.rst | 15 +++
doc/guides/rel_notes/release_20_11.rst | 6 +
doc/guides/testpmd_app_ug/run_app.rst | 16 ++-
doc/guides/testpmd_app_ug/testpmd_funcs.rst | 21 ++-
drivers/net/mlx5/linux/mlx5_os.c | 2 +
drivers/net/mlx5/mlx5.h | 3 +
drivers/net/mlx5/mlx5_mr.c | 3 +
drivers/net/mlx5/mlx5_rxq.c | 194 +++++++++++++++++++++++-----
drivers/net/mlx5/mlx5_rxtx.c | 3 +-
drivers/net/mlx5/mlx5_rxtx.h | 10 +-
drivers/net/mlx5/mlx5_trigger.c | 20 +--
lib/librte_ethdev/ethdev_trace_points.c | 3 +
lib/librte_ethdev/rte_ethdev.c | 133 +++++++++++++++----
lib/librte_ethdev/rte_ethdev.h | 107 +++++++++++++++
lib/librte_ethdev/rte_ethdev_driver.h | 10 ++
lib/librte_ethdev/rte_ethdev_trace.h | 19 +++
lib/librte_ethdev/rte_ethdev_version.map | 3 +
23 files changed, 780 insertions(+), 139 deletions(-)