Message ID | 1613272907-22563-1-git-send-email-xuemingl@nvidia.com (mailing list archive) |
---|---|
Headers |
Return-Path: <dev-bounces@dpdk.org> X-Original-To: patchwork@inbox.dpdk.org Delivered-To: patchwork@inbox.dpdk.org Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id DEA3BA0546; Sun, 14 Feb 2021 04:22:37 +0100 (CET) Received: from [217.70.189.124] (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id 5D5C3406A3; Sun, 14 Feb 2021 04:22:37 +0100 (CET) Received: from mellanox.co.il (mail-il-dmz.mellanox.com [193.47.165.129]) by mails.dpdk.org (Postfix) with ESMTP id A8611406A2 for <dev@dpdk.org>; Sun, 14 Feb 2021 04:22:35 +0100 (CET) Received: from Internal Mail-Server by MTLPINE1 (envelope-from xuemingl@nvidia.com) with SMTP; 14 Feb 2021 05:22:34 +0200 Received: from nvidia.com (pegasus05.mtr.labs.mlnx [10.210.16.100]) by labmailer.mlnx (8.13.8/8.13.8) with ESMTP id 11E3MYvM008525; Sun, 14 Feb 2021 05:22:34 +0200 From: Xueming Li <xuemingl@nvidia.com> To: Cc: dev@dpdk.org, Viacheslav Ovsiienko <viacheslavo@nvidia.com>, xuemingl@nvidia.com, Asaf Penso <asafp@nvidia.com> Date: Sun, 14 Feb 2021 03:21:30 +0000 Message-Id: <1613272907-22563-1-git-send-email-xuemingl@nvidia.com> X-Mailer: git-send-email 1.8.3.1 In-Reply-To: <1608303356-13089-2-git-send-email-xuemingl@nvidia.com> References: <1608303356-13089-2-git-send-email-xuemingl@nvidia.com> Subject: [dpdk-dev] [PATCH v6 0/9] ethdev: support SubFunction representor X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK patches and discussions <dev.dpdk.org> List-Unsubscribe: <https://mails.dpdk.org/options/dev>, <mailto:dev-request@dpdk.org?subject=unsubscribe> List-Archive: <http://mails.dpdk.org/archives/dev/> List-Post: <mailto:dev@dpdk.org> List-Help: <mailto:dev-request@dpdk.org?subject=help> List-Subscribe: <https://mails.dpdk.org/listinfo/dev>, <mailto:dev-request@dpdk.org?subject=subscribe> Errors-To: dev-bounces@dpdk.org Sender: "dev" <dev-bounces@dpdk.org> |
Series |
ethdev: support SubFunction representor
|
|
Message
Xueming Li
Feb. 14, 2021, 3:21 a.m. UTC
SubFunction [1] is a portion of the PCI device, a SF netdev has its own dedicated queues(txq, rxq). A SF netdev supports E-Switch representation offload similar to existing PF and VF representors. A SF shares PCI level resources with other SFs and/or with its parent PCI function. From SmartNIC perspective, when PCI device is shared for multi-host, representors for host controller and host PF is required. This patch set introduces new representor types in addtion to existing VF representor. Syntax: [[c#]pf#]vf#: VF port representor/s from controller/pf [[c#]pf#]sf#: SF port representor/s from controller/pf #: VF representor - for backwards compatibility "#" is number instance, list or range, valid examples: 1, [1,3,5], [0-3], [0,2-4,6] For backward compatibility, this patch also introduces new netdev capability to indicate the capability of supportting SF representor. Version history: RFC: initial version [2] V2: - separate patch for represnetor infrastructure, controller, pf and sf. - replace representor ID macro with functions: rte_eth_representor_id_encode() rte_eth_representor_id_parse() - new patch to allow devargs with same PCI BDF but different representors. - other minor code updates according to comments, thanks Andrew! - update document V3: - improve probing of allowed devargs with same name. - parse single word of kvargs as key. - update kvargs test cases. V4: - split first representor refactor patch into 1: add representor type 2: refector representor list parsing - push the patch supporting multi-devargs for same device. V5: - add comments for parsing functions - update switch_representation.rst - Thanks Ajit V6: - split representor types into different patches, move to rte_ethdev.h - improvements of rte_eth_devargs_process_list() according to Andrew's suggestion - fixed PF probe failure for Intel i40e - replace ethdev SF capability with rte_eth_representor_info_get() - add new ethdev ops api to get representor info from PMD - replace representor ID encode/decode with conversion from representor info - change ethdev representor iterator to use new ID encoding Xueming Li (9): ethdev: introduce representor type ethdev: support representor port list ethdev: support new VF representor syntax ethdev: support sub function representor ethdev: support PF index in representor ethdev: support multi-host in representor ethdev: new API to get representor info ethdev: representor iterator compare complete info kvargs: update parser to support lists app/test/test_kvargs.c | 46 ++++- config/rte_config.h | 1 + doc/guides/prog_guide/poll_mode_drv.rst | 13 +- .../prog_guide/switch_representation.rst | 35 +++- drivers/net/bnxt/bnxt_ethdev.c | 7 + drivers/net/enic/enic_ethdev.c | 6 + drivers/net/i40e/i40e_ethdev.c | 7 + drivers/net/ixgbe/ixgbe_ethdev.c | 7 + drivers/net/mlx5/linux/mlx5_os.c | 11 ++ lib/librte_ethdev/ethdev_driver.h | 49 ++++- lib/librte_ethdev/ethdev_private.c | 173 ++++++++++++------ lib/librte_ethdev/ethdev_private.h | 3 - lib/librte_ethdev/rte_class_eth.c | 40 ++-- lib/librte_ethdev/rte_ethdev.c | 102 ++++++++++- lib/librte_ethdev/rte_ethdev.h | 53 ++++++ lib/librte_ethdev/version.map | 4 + lib/librte_kvargs/rte_kvargs.c | 101 +++++++--- 17 files changed, 535 insertions(+), 123 deletions(-)
Comments
On Sun, 14 Feb 2021 03:21:30 +0000 Xueming Li <xuemingl@nvidia.com> wrote: > SubFunction [1] is a portion of the PCI device, a SF netdev has its own > dedicated queues(txq, rxq). A SF netdev supports E-Switch representation > offload similar to existing PF and VF representors. A SF shares PCI > level resources with other SFs and/or with its parent PCI function. > > From SmartNIC perspective, when PCI device is shared for multi-host, > representors for host controller and host PF is required. > > This patch set introduces new representor types in addtion to existing > VF representor. Syntax: > > [[c#]pf#]vf#: VF port representor/s from controller/pf > [[c#]pf#]sf#: SF port representor/s from controller/pf > #: VF representor - for backwards compatibility > > "#" is number instance, list or range, valid examples: > 1, [1,3,5], [0-3], [0,2-4,6] > > For backward compatibility, this patch also introduces new netdev > capability to indicate the capability of supportting SF representor. > > Version history: > RFC: > initial version [2] > V2: > - separate patch for represnetor infrastructure, controller, pf and > sf. > - replace representor ID macro with functions: > rte_eth_representor_id_encode() > rte_eth_representor_id_parse() > - new patch to allow devargs with same PCI BDF but different > representors. > - other minor code updates according to comments, thanks Andrew! > - update document > V3: > - improve probing of allowed devargs with same name. > - parse single word of kvargs as key. > - update kvargs test cases. > V4: > - split first representor refactor patch into > 1: add representor type > 2: refector representor list parsing > - push the patch supporting multi-devargs for same device. > V5: > - add comments for parsing functions > - update switch_representation.rst - Thanks Ajit > V6: > - split representor types into different patches, move to > rte_ethdev.h > - improvements of rte_eth_devargs_process_list() according to > Andrew's suggestion > - fixed PF probe failure for Intel i40e > - replace ethdev SF capability with rte_eth_representor_info_get() > - add new ethdev ops api to get representor info from PMD > - replace representor ID encode/decode with conversion from > representor info > - change ethdev representor iterator to use new ID encoding > > > Xueming Li (9): > ethdev: introduce representor type > ethdev: support representor port list > ethdev: support new VF representor syntax > ethdev: support sub function representor > ethdev: support PF index in representor > ethdev: support multi-host in representor > ethdev: new API to get representor info > ethdev: representor iterator compare complete info > kvargs: update parser to support lists > > app/test/test_kvargs.c | 46 ++++- > config/rte_config.h | 1 + > doc/guides/prog_guide/poll_mode_drv.rst | 13 +- > .../prog_guide/switch_representation.rst | 35 +++- > drivers/net/bnxt/bnxt_ethdev.c | 7 + > drivers/net/enic/enic_ethdev.c | 6 + > drivers/net/i40e/i40e_ethdev.c | 7 + > drivers/net/ixgbe/ixgbe_ethdev.c | 7 + > drivers/net/mlx5/linux/mlx5_os.c | 11 ++ > lib/librte_ethdev/ethdev_driver.h | 49 ++++- > lib/librte_ethdev/ethdev_private.c | 173 ++++++++++++------ > lib/librte_ethdev/ethdev_private.h | 3 - > lib/librte_ethdev/rte_class_eth.c | 40 ++-- > lib/librte_ethdev/rte_ethdev.c | 102 ++++++++++- > lib/librte_ethdev/rte_ethdev.h | 53 ++++++ > lib/librte_ethdev/version.map | 4 + > lib/librte_kvargs/rte_kvargs.c | 101 +++++++--- > 17 files changed, 535 insertions(+), 123 deletions(-) > A couple of higher level design questions: 1. How does Linux and other OS handle this in their API? 2. This solution seems quite tied into a specific implementation on your hardware. Is there a PCI standard for this? 3. Maybe a more general solution where these were represented as buses would allow for other connection methods in the future?
> -----Original Message----- > From: dev <dev-bounces@dpdk.org> On Behalf Of Stephen Hemminger > Sent: Tuesday, February 23, 2021 09:55 > To: Xueming Li <xuemingl@nvidia.com> > Cc: dev@dpdk.org; Viacheslav Ovsiienko <viacheslavo@nvidia.com>; Asaf Penso <asafp@nvidia.com> > Subject: Re: [dpdk-dev] [PATCH v6 0/9] ethdev: support SubFunction representor > > On Sun, 14 Feb 2021 03:21:30 +0000 > Xueming Li <xuemingl@nvidia.com> wrote: > > > SubFunction [1] is a portion of the PCI device, a SF netdev has its own > > dedicated queues(txq, rxq). A SF netdev supports E-Switch representation > > offload similar to existing PF and VF representors. A SF shares PCI > > level resources with other SFs and/or with its parent PCI function. > > > > From SmartNIC perspective, when PCI device is shared for multi-host, > > representors for host controller and host PF is required. > > > > This patch set introduces new representor types in addtion to existing > > VF representor. Syntax: > > > > [[c#]pf#]vf#: VF port representor/s from controller/pf > > [[c#]pf#]sf#: SF port representor/s from controller/pf > > #: VF representor - for backwards compatibility > > > > "#" is number instance, list or range, valid examples: > > 1, [1,3,5], [0-3], [0,2-4,6] > > > > For backward compatibility, this patch also introduces new netdev > > capability to indicate the capability of supportting SF representor. > > > > Version history: > > RFC: > > initial version [2] > > V2: > > - separate patch for represnetor infrastructure, controller, pf and > > sf. > > - replace representor ID macro with functions: > > rte_eth_representor_id_encode() > > rte_eth_representor_id_parse() > > - new patch to allow devargs with same PCI BDF but different > > representors. > > - other minor code updates according to comments, thanks Andrew! > > - update document > > V3: > > - improve probing of allowed devargs with same name. > > - parse single word of kvargs as key. > > - update kvargs test cases. > > V4: > > - split first representor refactor patch into > > 1: add representor type > > 2: refector representor list parsing > > - push the patch supporting multi-devargs for same device. > > V5: > > - add comments for parsing functions > > - update switch_representation.rst - Thanks Ajit > > V6: > > - split representor types into different patches, move to > > rte_ethdev.h > > - improvements of rte_eth_devargs_process_list() according to > > Andrew's suggestion > > - fixed PF probe failure for Intel i40e > > - replace ethdev SF capability with rte_eth_representor_info_get() > > - add new ethdev ops api to get representor info from PMD > > - replace representor ID encode/decode with conversion from > > representor info > > - change ethdev representor iterator to use new ID encoding > > > > > > Xueming Li (9): > > ethdev: introduce representor type > > ethdev: support representor port list > > ethdev: support new VF representor syntax > > ethdev: support sub function representor > > ethdev: support PF index in representor > > ethdev: support multi-host in representor > > ethdev: new API to get representor info > > ethdev: representor iterator compare complete info > > kvargs: update parser to support lists > > > > app/test/test_kvargs.c | 46 ++++- > > config/rte_config.h | 1 + > > doc/guides/prog_guide/poll_mode_drv.rst | 13 +- > > .../prog_guide/switch_representation.rst | 35 +++- > > drivers/net/bnxt/bnxt_ethdev.c | 7 + > > drivers/net/enic/enic_ethdev.c | 6 + > > drivers/net/i40e/i40e_ethdev.c | 7 + > > drivers/net/ixgbe/ixgbe_ethdev.c | 7 + > > drivers/net/mlx5/linux/mlx5_os.c | 11 ++ > > lib/librte_ethdev/ethdev_driver.h | 49 ++++- > > lib/librte_ethdev/ethdev_private.c | 173 ++++++++++++------ > > lib/librte_ethdev/ethdev_private.h | 3 - > > lib/librte_ethdev/rte_class_eth.c | 40 ++-- > > lib/librte_ethdev/rte_ethdev.c | 102 ++++++++++- > > lib/librte_ethdev/rte_ethdev.h | 53 ++++++ > > lib/librte_ethdev/version.map | 4 + > > lib/librte_kvargs/rte_kvargs.c | 101 +++++++--- > > 17 files changed, 535 insertions(+), 123 deletions(-) > > > > A couple of higher level design questions: > 1. How does Linux and other OS handle this in their API? > 2. This solution seems quite tied into a specific implementation on your hardware. > Is there a PCI standard for this? > 3. Maybe a more general solution where these were represented as buses would > allow for other connection methods in the future? It should be "auxiliary bus", I think. ;-) https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/tree/drivers/base/auxiliary.c mlx5 subfunction support https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/commit/?id=24a790da0ac4d9bcce2a9becc8799241716672f6
>-----Original Message----- >From: Wang, Haiyue <haiyue.wang@intel.com> >Sent: Tuesday, February 23, 2021 7:46 PM >To: Stephen Hemminger <stephen@networkplumber.org>; Xueming(Steven) Li <xuemingl@nvidia.com> >Cc: dev@dpdk.org; Slava Ovsiienko <viacheslavo@nvidia.com>; Asaf Penso <asafp@nvidia.com> >Subject: RE: [dpdk-dev] [PATCH v6 0/9] ethdev: support SubFunction representor > >> -----Original Message----- >> From: dev <dev-bounces@dpdk.org> On Behalf Of Stephen Hemminger >> Sent: Tuesday, February 23, 2021 09:55 >> To: Xueming Li <xuemingl@nvidia.com> >> Cc: dev@dpdk.org; Viacheslav Ovsiienko <viacheslavo@nvidia.com>; Asaf >> Penso <asafp@nvidia.com> >> Subject: Re: [dpdk-dev] [PATCH v6 0/9] ethdev: support SubFunction >> representor >> >> On Sun, 14 Feb 2021 03:21:30 +0000 >> Xueming Li <xuemingl@nvidia.com> wrote: >> >> > SubFunction [1] is a portion of the PCI device, a SF netdev has its >> > own dedicated queues(txq, rxq). A SF netdev supports E-Switch >> > representation offload similar to existing PF and VF representors. A >> > SF shares PCI level resources with other SFs and/or with its parent PCI function. >> > >> > From SmartNIC perspective, when PCI device is shared for multi-host, >> > representors for host controller and host PF is required. >> > >> > This patch set introduces new representor types in addtion to >> > existing VF representor. Syntax: >> > >> > [[c#]pf#]vf#: VF port representor/s from controller/pf >> > [[c#]pf#]sf#: SF port representor/s from controller/pf >> > #: VF representor - for backwards compatibility >> > >> > "#" is number instance, list or range, valid examples: >> > 1, [1,3,5], [0-3], [0,2-4,6] >> > >> > For backward compatibility, this patch also introduces new netdev >> > capability to indicate the capability of supportting SF representor. >> > >> > Version history: >> > RFC: >> > initial version [2] >> > V2: >> > - separate patch for represnetor infrastructure, controller, pf and >> > sf. >> > - replace representor ID macro with functions: >> > rte_eth_representor_id_encode() >> > rte_eth_representor_id_parse() >> > - new patch to allow devargs with same PCI BDF but different >> > representors. >> > - other minor code updates according to comments, thanks Andrew! >> > - update document >> > V3: >> > - improve probing of allowed devargs with same name. >> > - parse single word of kvargs as key. >> > - update kvargs test cases. >> > V4: >> > - split first representor refactor patch into >> > 1: add representor type >> > 2: refector representor list parsing >> > - push the patch supporting multi-devargs for same device. >> > V5: >> > - add comments for parsing functions >> > - update switch_representation.rst - Thanks Ajit >> > V6: >> > - split representor types into different patches, move to >> > rte_ethdev.h >> > - improvements of rte_eth_devargs_process_list() according to >> > Andrew's suggestion >> > - fixed PF probe failure for Intel i40e >> > - replace ethdev SF capability with rte_eth_representor_info_get() >> > - add new ethdev ops api to get representor info from PMD >> > - replace representor ID encode/decode with conversion from >> > representor info >> > - change ethdev representor iterator to use new ID encoding >> > >> > >> > Xueming Li (9): >> > ethdev: introduce representor type >> > ethdev: support representor port list >> > ethdev: support new VF representor syntax >> > ethdev: support sub function representor >> > ethdev: support PF index in representor >> > ethdev: support multi-host in representor >> > ethdev: new API to get representor info >> > ethdev: representor iterator compare complete info >> > kvargs: update parser to support lists >> > >> > app/test/test_kvargs.c | 46 ++++- >> > config/rte_config.h | 1 + >> > doc/guides/prog_guide/poll_mode_drv.rst | 13 +- >> > .../prog_guide/switch_representation.rst | 35 +++- >> > drivers/net/bnxt/bnxt_ethdev.c | 7 + >> > drivers/net/enic/enic_ethdev.c | 6 + >> > drivers/net/i40e/i40e_ethdev.c | 7 + >> > drivers/net/ixgbe/ixgbe_ethdev.c | 7 + >> > drivers/net/mlx5/linux/mlx5_os.c | 11 ++ >> > lib/librte_ethdev/ethdev_driver.h | 49 ++++- >> > lib/librte_ethdev/ethdev_private.c | 173 ++++++++++++------ >> > lib/librte_ethdev/ethdev_private.h | 3 - >> > lib/librte_ethdev/rte_class_eth.c | 40 ++-- >> > lib/librte_ethdev/rte_ethdev.c | 102 ++++++++++- >> > lib/librte_ethdev/rte_ethdev.h | 53 ++++++ >> > lib/librte_ethdev/version.map | 4 + >> > lib/librte_kvargs/rte_kvargs.c | 101 +++++++--- >> > 17 files changed, 535 insertions(+), 123 deletions(-) >> > >> >> A couple of higher level design questions: >> 1. How does Linux and other OS handle this in their API? Hi Stephen, thanks for looking this! Representor was a type of ethdev, OS independent. SF representor is a new type besides existing VF representor type. >> 2. This solution seems quite tied into a specific implementation on your hardware. >> Is there a PCI standard for this? SF has been carved in kernel, supported by devlink. The devlink document in the link Haiyue posted should be helpful to understand the overall picture: https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/commit/?id=24a790da0ac4d9bcce2a9becc8799241716672f6 > >> 3. Maybe a more general solution where these were represented as buses would >> allow for other connection methods in the future? > >It should be "auxiliary bus", I think. ;-) https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/tree/drivers/base/auxiliary.c Good link, Mellanox has exactly implemented SF based auxiliary bus, I'm going to post another RFC to support auxiliary bus. But this patchset still needed to support representor of SF. > >mlx5 subfunction support >https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/commit/?id=24a790da0ac4d9bcce2a9becc8799241716672f6
>-----Original Message----- >From: Wang, Haiyue <haiyue.wang@intel.com> >Sent: Tuesday, February 23, 2021 7:46 PM >To: Stephen Hemminger <stephen@networkplumber.org>; Xueming(Steven) Li <xuemingl@nvidia.com> >Cc: dev@dpdk.org; Slava Ovsiienko <viacheslavo@nvidia.com>; Asaf Penso <asafp@nvidia.com> >Subject: RE: [dpdk-dev] [PATCH v6 0/9] ethdev: support SubFunction representor > >> -----Original Message----- >> From: dev <dev-bounces@dpdk.org> On Behalf Of Stephen Hemminger >> Sent: Tuesday, February 23, 2021 09:55 >> To: Xueming Li <xuemingl@nvidia.com> >> Cc: dev@dpdk.org; Viacheslav Ovsiienko <viacheslavo@nvidia.com>; Asaf >> Penso <asafp@nvidia.com> >> Subject: Re: [dpdk-dev] [PATCH v6 0/9] ethdev: support SubFunction >> representor >> >> On Sun, 14 Feb 2021 03:21:30 +0000 >> Xueming Li <xuemingl@nvidia.com> wrote: >> >> > SubFunction [1] is a portion of the PCI device, a SF netdev has its >> > own dedicated queues(txq, rxq). A SF netdev supports E-Switch >> > representation offload similar to existing PF and VF representors. A >> > SF shares PCI level resources with other SFs and/or with its parent PCI function. >> > >> > From SmartNIC perspective, when PCI device is shared for multi-host, >> > representors for host controller and host PF is required. >> > >> > This patch set introduces new representor types in addtion to >> > existing VF representor. Syntax: >> > >> > [[c#]pf#]vf#: VF port representor/s from controller/pf >> > [[c#]pf#]sf#: SF port representor/s from controller/pf >> > #: VF representor - for backwards compatibility >> > >> > "#" is number instance, list or range, valid examples: >> > 1, [1,3,5], [0-3], [0,2-4,6] >> > >> > For backward compatibility, this patch also introduces new netdev >> > capability to indicate the capability of supportting SF representor. >> > >> > Version history: >> > RFC: >> > initial version [2] >> > V2: >> > - separate patch for represnetor infrastructure, controller, pf and >> > sf. >> > - replace representor ID macro with functions: >> > rte_eth_representor_id_encode() >> > rte_eth_representor_id_parse() >> > - new patch to allow devargs with same PCI BDF but different >> > representors. >> > - other minor code updates according to comments, thanks Andrew! >> > - update document >> > V3: >> > - improve probing of allowed devargs with same name. >> > - parse single word of kvargs as key. >> > - update kvargs test cases. >> > V4: >> > - split first representor refactor patch into >> > 1: add representor type >> > 2: refector representor list parsing >> > - push the patch supporting multi-devargs for same device. >> > V5: >> > - add comments for parsing functions >> > - update switch_representation.rst - Thanks Ajit >> > V6: >> > - split representor types into different patches, move to >> > rte_ethdev.h >> > - improvements of rte_eth_devargs_process_list() according to >> > Andrew's suggestion >> > - fixed PF probe failure for Intel i40e >> > - replace ethdev SF capability with rte_eth_representor_info_get() >> > - add new ethdev ops api to get representor info from PMD >> > - replace representor ID encode/decode with conversion from >> > representor info >> > - change ethdev representor iterator to use new ID encoding >> > >> > >> > Xueming Li (9): >> > ethdev: introduce representor type >> > ethdev: support representor port list >> > ethdev: support new VF representor syntax >> > ethdev: support sub function representor >> > ethdev: support PF index in representor >> > ethdev: support multi-host in representor >> > ethdev: new API to get representor info >> > ethdev: representor iterator compare complete info >> > kvargs: update parser to support lists >> > >> > app/test/test_kvargs.c | 46 ++++- >> > config/rte_config.h | 1 + >> > doc/guides/prog_guide/poll_mode_drv.rst | 13 +- >> > .../prog_guide/switch_representation.rst | 35 +++- >> > drivers/net/bnxt/bnxt_ethdev.c | 7 + >> > drivers/net/enic/enic_ethdev.c | 6 + >> > drivers/net/i40e/i40e_ethdev.c | 7 + >> > drivers/net/ixgbe/ixgbe_ethdev.c | 7 + >> > drivers/net/mlx5/linux/mlx5_os.c | 11 ++ >> > lib/librte_ethdev/ethdev_driver.h | 49 ++++- >> > lib/librte_ethdev/ethdev_private.c | 173 ++++++++++++------ >> > lib/librte_ethdev/ethdev_private.h | 3 - >> > lib/librte_ethdev/rte_class_eth.c | 40 ++-- >> > lib/librte_ethdev/rte_ethdev.c | 102 ++++++++++- >> > lib/librte_ethdev/rte_ethdev.h | 53 ++++++ >> > lib/librte_ethdev/version.map | 4 + >> > lib/librte_kvargs/rte_kvargs.c | 101 +++++++--- >> > 17 files changed, 535 insertions(+), 123 deletions(-) >> > >> >> A couple of higher level design questions: >> 1. How does Linux and other OS handle this in their API? >> 2. This solution seems quite tied into a specific implementation on your hardware. >> Is there a PCI standard for this? > >> 3. Maybe a more general solution where these were represented as buses would >> allow for other connection methods in the future? > >It should be "auxiliary bus", I think. ;-) https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/tree/drivers/base/auxiliary.c > >mlx5 subfunction support >https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/commit/?id=24a790da0ac4d9bcce2a9becc8799241716672f6 Hi, The Auxiliary driver for DPDK posted, welcome to have a look :) http://patchwork.dpdk.org/project/dpdk/patch/20210311130113.31382-1-xuemingl@nvidia.com/ Thanks
> -----Original Message----- > From: Xueming(Steven) Li <xuemingl@nvidia.com> > Sent: Wednesday, March 31, 2021 13:49 > To: Wang, Haiyue <haiyue.wang@intel.com>; Stephen Hemminger <stephen@networkplumber.org> > Cc: dev@dpdk.org; Slava Ovsiienko <viacheslavo@nvidia.com>; Asaf Penso <asafp@nvidia.com> > Subject: RE: [dpdk-dev] [PATCH v6 0/9] ethdev: support SubFunction representor > > > > >-----Original Message----- > >From: Wang, Haiyue <haiyue.wang@intel.com> > >Sent: Tuesday, February 23, 2021 7:46 PM > >To: Stephen Hemminger <stephen@networkplumber.org>; Xueming(Steven) Li <xuemingl@nvidia.com> > >Cc: dev@dpdk.org; Slava Ovsiienko <viacheslavo@nvidia.com>; Asaf Penso <asafp@nvidia.com> > >Subject: RE: [dpdk-dev] [PATCH v6 0/9] ethdev: support SubFunction representor > > > >> -----Original Message----- > >> From: dev <dev-bounces@dpdk.org> On Behalf Of Stephen Hemminger > >> Sent: Tuesday, February 23, 2021 09:55 > >> To: Xueming Li <xuemingl@nvidia.com> > >> Cc: dev@dpdk.org; Viacheslav Ovsiienko <viacheslavo@nvidia.com>; Asaf > >> Penso <asafp@nvidia.com> > >> Subject: Re: [dpdk-dev] [PATCH v6 0/9] ethdev: support SubFunction > >> representor > >> > >> On Sun, 14 Feb 2021 03:21:30 +0000 > >> Xueming Li <xuemingl@nvidia.com> wrote: > >> > >> > SubFunction [1] is a portion of the PCI device, a SF netdev has its > >> > own dedicated queues(txq, rxq). A SF netdev supports E-Switch > >> > representation offload similar to existing PF and VF representors. A > >> > SF shares PCI level resources with other SFs and/or with its parent PCI function. > >> > > >> > From SmartNIC perspective, when PCI device is shared for multi-host, > >> > representors for host controller and host PF is required. > >> > > >> > This patch set introduces new representor types in addtion to > >> > existing VF representor. Syntax: > >> > > >> > [[c#]pf#]vf#: VF port representor/s from controller/pf > >> > [[c#]pf#]sf#: SF port representor/s from controller/pf > >> > #: VF representor - for backwards compatibility > >> > > >> > "#" is number instance, list or range, valid examples: > >> > 1, [1,3,5], [0-3], [0,2-4,6] > >> > > >> > For backward compatibility, this patch also introduces new netdev > >> > capability to indicate the capability of supportting SF representor. > >> > > >> > Version history: > >> > RFC: > >> > initial version [2] > >> > V2: > >> > - separate patch for represnetor infrastructure, controller, pf and > >> > sf. > >> > - replace representor ID macro with functions: > >> > rte_eth_representor_id_encode() > >> > rte_eth_representor_id_parse() > >> > - new patch to allow devargs with same PCI BDF but different > >> > representors. > >> > - other minor code updates according to comments, thanks Andrew! > >> > - update document > >> > V3: > >> > - improve probing of allowed devargs with same name. > >> > - parse single word of kvargs as key. > >> > - update kvargs test cases. > >> > V4: > >> > - split first representor refactor patch into > >> > 1: add representor type > >> > 2: refector representor list parsing > >> > - push the patch supporting multi-devargs for same device. > >> > V5: > >> > - add comments for parsing functions > >> > - update switch_representation.rst - Thanks Ajit > >> > V6: > >> > - split representor types into different patches, move to > >> > rte_ethdev.h > >> > - improvements of rte_eth_devargs_process_list() according to > >> > Andrew's suggestion > >> > - fixed PF probe failure for Intel i40e > >> > - replace ethdev SF capability with rte_eth_representor_info_get() > >> > - add new ethdev ops api to get representor info from PMD > >> > - replace representor ID encode/decode with conversion from > >> > representor info > >> > - change ethdev representor iterator to use new ID encoding > >> > > >> > > >> > Xueming Li (9): > >> > ethdev: introduce representor type > >> > ethdev: support representor port list > >> > ethdev: support new VF representor syntax > >> > ethdev: support sub function representor > >> > ethdev: support PF index in representor > >> > ethdev: support multi-host in representor > >> > ethdev: new API to get representor info > >> > ethdev: representor iterator compare complete info > >> > kvargs: update parser to support lists > >> > > >> > app/test/test_kvargs.c | 46 ++++- > >> > config/rte_config.h | 1 + > >> > doc/guides/prog_guide/poll_mode_drv.rst | 13 +- > >> > .../prog_guide/switch_representation.rst | 35 +++- > >> > drivers/net/bnxt/bnxt_ethdev.c | 7 + > >> > drivers/net/enic/enic_ethdev.c | 6 + > >> > drivers/net/i40e/i40e_ethdev.c | 7 + > >> > drivers/net/ixgbe/ixgbe_ethdev.c | 7 + > >> > drivers/net/mlx5/linux/mlx5_os.c | 11 ++ > >> > lib/librte_ethdev/ethdev_driver.h | 49 ++++- > >> > lib/librte_ethdev/ethdev_private.c | 173 ++++++++++++------ > >> > lib/librte_ethdev/ethdev_private.h | 3 - > >> > lib/librte_ethdev/rte_class_eth.c | 40 ++-- > >> > lib/librte_ethdev/rte_ethdev.c | 102 ++++++++++- > >> > lib/librte_ethdev/rte_ethdev.h | 53 ++++++ > >> > lib/librte_ethdev/version.map | 4 + > >> > lib/librte_kvargs/rte_kvargs.c | 101 +++++++--- > >> > 17 files changed, 535 insertions(+), 123 deletions(-) > >> > > >> > >> A couple of higher level design questions: > >> 1. How does Linux and other OS handle this in their API? > >> 2. This solution seems quite tied into a specific implementation on your hardware. > >> Is there a PCI standard for this? > > > >> 3. Maybe a more general solution where these were represented as buses would > >> allow for other connection methods in the future? > > > >It should be "auxiliary bus", I think. ;-) > https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/tree/drivers/base/auxiliary.c > > > >mlx5 subfunction support > >https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/commit/?id=24a790da0ac4d9bcce2a9be > cc8799241716672f6 > > Hi, > > The Auxiliary driver for DPDK posted, welcome to have a look :) > http://patchwork.dpdk.org/project/dpdk/patch/20210311130113.31382-1-xuemingl@nvidia.com/ Cool. ;-) > > Thanks >