Message ID | 1553614522-12151-1-git-send-email-viacheslavo@mellanox.com (mailing list archive) |
---|---|
Headers |
Return-Path: <dev-bounces@dpdk.org> X-Original-To: patchwork@dpdk.org Delivered-To: patchwork@dpdk.org Received: from [92.243.14.124] (localhost [127.0.0.1]) by dpdk.org (Postfix) with ESMTP id 63CFF1B148; Tue, 26 Mar 2019 16:36:12 +0100 (CET) Received: from mellanox.co.il (mail-il-dmz.mellanox.com [193.47.165.129]) by dpdk.org (Postfix) with ESMTP id 60FA21B12A for <dev@dpdk.org>; Tue, 26 Mar 2019 16:36:05 +0100 (CET) Received: from Internal Mail-Server by MTLPINE1 (envelope-from viacheslavo@mellanox.com) with ESMTPS (AES256-SHA encrypted); 26 Mar 2019 17:36:01 +0200 Received: from pegasus12.mtr.labs.mlnx. (pegasus12.mtr.labs.mlnx [10.210.17.40]) by labmailer.mlnx (8.13.8/8.13.8) with ESMTP id x2QFa1wN019637; Tue, 26 Mar 2019 17:36:01 +0200 From: Viacheslav Ovsiienko <viacheslavo@mellanox.com> To: dev@dpdk.org Cc: shahafs@mellanox.com Date: Tue, 26 Mar 2019 15:35:09 +0000 Message-Id: <1553614522-12151-1-git-send-email-viacheslavo@mellanox.com> X-Mailer: git-send-email 1.8.3.1 In-Reply-To: <1553533414-9911-1-git-send-email-viacheslavo@mellanox.com> References: <1553533414-9911-1-git-send-email-viacheslavo@mellanox.com> Subject: [dpdk-dev] [PATCH v3 00/14] net/mlx5: add support for multiport IB devices X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: DPDK patches and discussions <dev.dpdk.org> List-Unsubscribe: <https://mails.dpdk.org/options/dev>, <mailto:dev-request@dpdk.org?subject=unsubscribe> List-Archive: <http://mails.dpdk.org/archives/dev/> List-Post: <mailto:dev@dpdk.org> List-Help: <mailto:dev-request@dpdk.org?subject=help> List-Subscribe: <https://mails.dpdk.org/listinfo/dev>, <mailto:dev-request@dpdk.org?subject=subscribe> Errors-To: dev-bounces@dpdk.org Sender: "dev" <dev-bounces@dpdk.org> |
Series |
net/mlx5: add support for multiport IB devices
|
|
Message
Slava Ovsiienko
March 26, 2019, 3:35 p.m. UTC
The Mellanox NICs support SR-IOV and have E-Switch feature.
When SR-IOV is set up in switchdev mode and E-Switch is enabled
we have so called VF representors in the system. All representors
belonging to the same E-Switch are created on the basis of the
single PCI function and with current implementation each representor
has its own dedicated Infiniband device and operates within its
own Infiniband context. It is proposed to provide representors
as ports of the single Infiniband device and operate on the
shared Infiniband context.
mlx5_pci_probe() routine is planned to be extended to probe
the ports of found Infiniband devices. All active ports,
belonging to the same Infiniband device will use the signle
shared Infiniband context of that device:
- QP are created within shared context, with assigning to
specified port
- Verb flows are also created with specifying port index
- Protection Domain is shared by all representors
- Global Memory Regions Cache is shared by all representors,
local ones are not (not in this patchset)
- Shared Asynchronous Event Handler distributes events
among the device ports (not in this patchset)
- UAR mapping to user space is also shared (not in patchset)
Each representor has its own attached network interface device,
interface name and index can be retrieved via IB device and
port index, so the link control is performed via Netlink
in the same way as in existing implementation.
This patchset also superseedes http://patches.dpdk.org/patch/50486/
Signed-off-by: Viacheslav Ovsiienko <viacheslavo@mellanox.com>
---
v3:
- vport_id field is added, contains associated VF vport
index on E-Switch. representor_id field is reverted
to keep the VF index
- some comment spellcheck
- some error logging is added for master/representor
wrong recognition
- shared IB context alloc/free routines mutex
protection added
v2:
http://patches.dpdk.org/patch/51642/
- clarifying comments added
- secondary process does not probe IB device at all and
does not create shared context
- minor bugs (like ++ in assert) fixed
- representor_id contains corresponding VF vport index,
no corrections +1 is needed anymore. vport index is
calculated in the only place in code and if indexing
schema is changed it would be easy to modify
v1:
http://patches.dpdk.org/cover/51451/
Viacheslav Ovsiienko (13):
net/mlx5: add representor recognition on kernels 5.x
net/mlx5: modify get ifindex routine for multiport IB
net/mlx5: add getting IB ports number for multiport IB
net/mlx5: add multiport IB device support to probing
net/mlx5: add IB shared context alloc/free functions
net/mlx5: switch to the names in the shared IB context
net/mlx5: switch to the shared Protection Domain
net/mlx5: switch to the shared context IB attributes
net/mlx5: switch to the shared IB device context
net/mlx5: provide IB port for the object being created
net/mlx5: update install/uninstall int handler routines
net/mlx5: update event handler for multiport IB devices
net/mlx5: add source vport match to the ingress rules
drivers/net/mlx5/Makefile | 10 +
drivers/net/mlx5/meson.build | 4 +
drivers/net/mlx5/mlx5.c | 651 ++++++++++++++++++++++++++-----------
drivers/net/mlx5/mlx5.h | 45 ++-
drivers/net/mlx5/mlx5_ethdev.c | 248 +++++++++-----
drivers/net/mlx5/mlx5_flow.c | 2 +
drivers/net/mlx5/mlx5_flow_dv.c | 45 ++-
drivers/net/mlx5/mlx5_flow_verbs.c | 7 +-
drivers/net/mlx5/mlx5_mr.c | 4 +-
drivers/net/mlx5/mlx5_nl.c | 108 +++++-
drivers/net/mlx5/mlx5_rxq.c | 41 +--
drivers/net/mlx5/mlx5_socket.c | 4 +-
drivers/net/mlx5/mlx5_stats.c | 22 +-
drivers/net/mlx5/mlx5_trigger.c | 2 +-
drivers/net/mlx5/mlx5_txq.c | 24 +-
15 files changed, 879 insertions(+), 338 deletions(-)
Comments
Tuesday, March 26, 2019 5:35 PM, Viacheslav Ovsiienko: > Subject: [PATCH v3 00/14] net/mlx5: add support for multiport IB devices > > The Mellanox NICs support SR-IOV and have E-Switch feature. > When SR-IOV is set up in switchdev mode and E-Switch is enabled we have > so called VF representors in the system. All representors belonging to the > same E-Switch are created on the basis of the single PCI function and with > current implementation each representor has its own dedicated Infiniband > device and operates within its own Infiniband context. It is proposed to > provide representors as ports of the single Infiniband device and operate on > the shared Infiniband context. > > mlx5_pci_probe() routine is planned to be extended to probe the ports of > found Infiniband devices. All active ports, belonging to the same Infiniband > device will use the signle shared Infiniband context of that device: > - QP are created within shared context, with assigning to > specified port > - Verb flows are also created with specifying port index > - Protection Domain is shared by all representors > - Global Memory Regions Cache is shared by all representors, > local ones are not (not in this patchset) > - Shared Asynchronous Event Handler distributes events > among the device ports (not in this patchset) > - UAR mapping to user space is also shared (not in patchset) > > Each representor has its own attached network interface device, interface > name and index can be retrieved via IB device and port index, so the link > control is performed via Netlink in the same way as in existing > implementation. > > This patchset also superseedes http://patches.dpdk.org/patch/50486/ > > Signed-off-by: Viacheslav Ovsiienko <viacheslavo@mellanox.com> Slava, basically I am OK w/ the series, in order for me to apply: 1. need to rebase on top of latest master. There are some merge conflicts. 2. at least patch[1] seems to be taken from another branch (checkout the HAVE_MLX5DV_DR flag). Please handle. [1] https://patches.dpdk.org/patch/51739/ > > --- > v3: > - vport_id field is added, contains associated VF vport > index on E-Switch. representor_id field is reverted > to keep the VF index > - some comment spellcheck > - some error logging is added for master/representor > wrong recognition > - shared IB context alloc/free routines mutex > protection added > > v2: > http://patches.dpdk.org/patch/51642/ > - clarifying comments added > - secondary process does not probe IB device at all and > does not create shared context > - minor bugs (like ++ in assert) fixed > - representor_id contains corresponding VF vport index, > no corrections +1 is needed anymore. vport index is > calculated in the only place in code and if indexing > schema is changed it would be easy to modify > > v1: > http://patches.dpdk.org/cover/51451/ > > Viacheslav Ovsiienko (13): > net/mlx5: add representor recognition on kernels 5.x > net/mlx5: modify get ifindex routine for multiport IB > net/mlx5: add getting IB ports number for multiport IB > net/mlx5: add multiport IB device support to probing > net/mlx5: add IB shared context alloc/free functions > net/mlx5: switch to the names in the shared IB context > net/mlx5: switch to the shared Protection Domain > net/mlx5: switch to the shared context IB attributes > net/mlx5: switch to the shared IB device context > net/mlx5: provide IB port for the object being created > net/mlx5: update install/uninstall int handler routines > net/mlx5: update event handler for multiport IB devices > net/mlx5: add source vport match to the ingress rules > > drivers/net/mlx5/Makefile | 10 + > drivers/net/mlx5/meson.build | 4 + > drivers/net/mlx5/mlx5.c | 651 ++++++++++++++++++++++++++-------- > --- > drivers/net/mlx5/mlx5.h | 45 ++- > drivers/net/mlx5/mlx5_ethdev.c | 248 +++++++++----- > drivers/net/mlx5/mlx5_flow.c | 2 + > drivers/net/mlx5/mlx5_flow_dv.c | 45 ++- > drivers/net/mlx5/mlx5_flow_verbs.c | 7 +- > drivers/net/mlx5/mlx5_mr.c | 4 +- > drivers/net/mlx5/mlx5_nl.c | 108 +++++- > drivers/net/mlx5/mlx5_rxq.c | 41 +-- > drivers/net/mlx5/mlx5_socket.c | 4 +- > drivers/net/mlx5/mlx5_stats.c | 22 +- > drivers/net/mlx5/mlx5_trigger.c | 2 +- > drivers/net/mlx5/mlx5_txq.c | 24 +- > 15 files changed, 879 insertions(+), 338 deletions(-) > > -- > 1.8.3.1
> -----Original Message----- > From: Shahaf Shuler > Sent: Wednesday, March 27, 2019 8:00 > To: Slava Ovsiienko <viacheslavo@mellanox.com>; dev@dpdk.org > Subject: RE: [PATCH v3 00/14] net/mlx5: add support for multiport IB devices > > Tuesday, March 26, 2019 5:35 PM, Viacheslav Ovsiienko: > > Subject: [PATCH v3 00/14] net/mlx5: add support for multiport IB > > devices > > > > The Mellanox NICs support SR-IOV and have E-Switch feature. [..] > > > > This patchset also superseedes http://patches.dpdk.org/patch/50486/ > > > > Signed-off-by: Viacheslav Ovsiienko <viacheslavo@mellanox.com> > > Slava, basically I am OK w/ the series, in order for me to apply: > 1. need to rebase on top of latest master. There are some merge conflicts. > 2. at least patch[1] seems to be taken from another branch (checkout the > HAVE_MLX5DV_DR flag). Please handle. > > [1] > https://patches.dpdk.org/patch/51739/ Off the list: My patchset is rebased on the top, but there are three Ori's commits not merged I was to told to base on: http://patches.dpdk.org/patch/51403/ > > > > > > --- > > v3: > > - vport_id field is added, contains associated VF vport > > index on E-Switch. representor_id field is reverted > > to keep the VF index > > - some comment spellcheck > > - some error logging is added for master/representor > > wrong recognition > > - shared IB context alloc/free routines mutex > > protection added > > > > v2: > > http://patches.dpdk.org/patch/51642/ > > - clarifying comments added > > - secondary process does not probe IB device at all and > > does not create shared context > > - minor bugs (like ++ in assert) fixed > > - representor_id contains corresponding VF vport index, > > no corrections +1 is needed anymore. vport index is > > calculated in the only place in code and if indexing > > schema is changed it would be easy to modify > > > > v1: > > http://patches.dpdk.org/cover/51451/ > > > > Viacheslav Ovsiienko (13): > > net/mlx5: add representor recognition on kernels 5.x > > net/mlx5: modify get ifindex routine for multiport IB > > net/mlx5: add getting IB ports number for multiport IB > > net/mlx5: add multiport IB device support to probing > > net/mlx5: add IB shared context alloc/free functions > > net/mlx5: switch to the names in the shared IB context > > net/mlx5: switch to the shared Protection Domain > > net/mlx5: switch to the shared context IB attributes > > net/mlx5: switch to the shared IB device context > > net/mlx5: provide IB port for the object being created > > net/mlx5: update install/uninstall int handler routines > > net/mlx5: update event handler for multiport IB devices > > net/mlx5: add source vport match to the ingress rules > > > > drivers/net/mlx5/Makefile | 10 + > > drivers/net/mlx5/meson.build | 4 + > > drivers/net/mlx5/mlx5.c | 651 ++++++++++++++++++++++++++-------- > > --- > > drivers/net/mlx5/mlx5.h | 45 ++- > > drivers/net/mlx5/mlx5_ethdev.c | 248 +++++++++----- > > drivers/net/mlx5/mlx5_flow.c | 2 + > > drivers/net/mlx5/mlx5_flow_dv.c | 45 ++- > > drivers/net/mlx5/mlx5_flow_verbs.c | 7 +- > > drivers/net/mlx5/mlx5_mr.c | 4 +- > > drivers/net/mlx5/mlx5_nl.c | 108 +++++- > > drivers/net/mlx5/mlx5_rxq.c | 41 +-- > > drivers/net/mlx5/mlx5_socket.c | 4 +- > > drivers/net/mlx5/mlx5_stats.c | 22 +- > > drivers/net/mlx5/mlx5_trigger.c | 2 +- > > drivers/net/mlx5/mlx5_txq.c | 24 +- > > 15 files changed, 879 insertions(+), 338 deletions(-) > > > > -- > > 1.8.3.1