From patchwork Sat Jan 18 10:48:59 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Anoob Joseph X-Patchwork-Id: 64857 X-Patchwork-Delegate: gakhil@marvell.com Return-Path: X-Original-To: patchwork@inbox.dpdk.org Delivered-To: patchwork@inbox.dpdk.org Received: from dpdk.org (dpdk.org [92.243.14.124]) by inbox.dpdk.org (Postfix) with ESMTP id 21DB6A051C; Sat, 18 Jan 2020 11:49:44 +0100 (CET) Received: from [92.243.14.124] (localhost [127.0.0.1]) by dpdk.org (Postfix) with ESMTP id D590A2F4F; Sat, 18 Jan 2020 11:49:40 +0100 (CET) Received: from mx0b-0016f401.pphosted.com (mx0b-0016f401.pphosted.com [67.231.156.173]) by dpdk.org (Postfix) with ESMTP id 70A7A2C6E for ; Sat, 18 Jan 2020 11:49:39 +0100 (CET) Received: from pps.filterd (m0045851.ppops.net [127.0.0.1]) by mx0b-0016f401.pphosted.com (8.16.0.42/8.16.0.42) with SMTP id 00IAmMHj018888; Sat, 18 Jan 2020 02:49:38 -0800 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=marvell.com; h=from : to : cc : subject : date : message-id : in-reply-to : references : mime-version : content-transfer-encoding : content-type; s=pfpt0818; bh=ZzY3S/QTotqInCHyWcBWdcTD7JEH2pT7NBq75ffP+8Q=; b=VDrxJgXeVS7ou1/edxydaDswrAraGBWjBT4KW4lTxHUYvMKuv0pdSBC6k6u2/2ityGI3 98QEBeT/I8ftJqRTZv+deoduYeAq7Ahnh8Y5fC219duANNBJ5JrUOKZ/l0Nj2tj3sSMq E5DpdFz9k/jTCKwG3J4ZejtrKSnjkjkU4C12uGQP9v2WO8mBP1iTKJyE2K3Y9nDN/UNh QN9ig3Bdau6asdHmYQwxR6b/16FEu0UaPcFAO0kPbYfJgPohN3xuaT6V0m3Yx9LRmt2u 1MAC26RiEcAZlpLFY3c1f5MjNbNmBf6t2wtjD0YzBczoABKvEPqwe1H7Rzun+Sgvsqth ug== Received: from sc-exch02.marvell.com ([199.233.58.182]) by mx0b-0016f401.pphosted.com with ESMTP id 2xk0sn6edv-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-SHA384 bits=256 verify=NOT); Sat, 18 Jan 2020 02:49:38 -0800 Received: from SC-EXCH01.marvell.com (10.93.176.81) by SC-EXCH02.marvell.com (10.93.176.82) with Microsoft SMTP Server (TLS) id 15.0.1497.2; Sat, 18 Jan 2020 02:49:36 -0800 Received: from maili.marvell.com (10.93.176.43) by SC-EXCH01.marvell.com (10.93.176.81) with Microsoft SMTP Server id 15.0.1497.2 via Frontend Transport; Sat, 18 Jan 2020 02:49:36 -0800 Received: from ajoseph83.caveonetworks.com (unknown [10.29.45.60]) by maili.marvell.com (Postfix) with ESMTP id 42ED73F703F; Sat, 18 Jan 2020 02:49:32 -0800 (PST) From: Anoob Joseph To: Akhil Goyal , Declan Doherty , Thomas Monjalon CC: Anoob Joseph , Jerin Jacob , Narayana Prasad , Kiran Kumar K , Nithin Dabilpuram , "Pavan Nikhilesh" , Ankur Dwivedi , Archana Muniganti , Tejasree Kondoj , Vamsi Attunuru , "Lukasz Bartosik" , , Tejasree Kondoj Date: Sat, 18 Jan 2020 16:18:59 +0530 Message-ID: <1579344553-11428-2-git-send-email-anoobj@marvell.com> X-Mailer: git-send-email 2.7.4 In-Reply-To: <1579344553-11428-1-git-send-email-anoobj@marvell.com> References: <1575806094-28391-1-git-send-email-anoobj@marvell.com> <1579344553-11428-1-git-send-email-anoobj@marvell.com> MIME-Version: 1.0 X-Proofpoint-Virus-Version: vendor=fsecure engine=2.50.10434:6.0.138, 18.0.572 definitions=2020-01-18_02:2020-01-16, 2020-01-18 signatures=0 Subject: [dpdk-dev] [PATCH v2 01/15] common/octeontx2: add CPT LF mbox for inline inbound X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Sender: "dev" Adding the new mbox introduced to configure CPT LF to be used for inline inbound. Signed-off-by: Anoob Joseph Signed-off-by: Tejasree Kondoj Acked-by: Jerin Jacob --- drivers/common/octeontx2/otx2_mbox.h | 7 +++++++ 1 file changed, 7 insertions(+) diff --git a/drivers/common/octeontx2/otx2_mbox.h b/drivers/common/octeontx2/otx2_mbox.h index e0e4e2f..70452d1 100644 --- a/drivers/common/octeontx2/otx2_mbox.h +++ b/drivers/common/octeontx2/otx2_mbox.h @@ -193,6 +193,8 @@ M(CPT_SET_CRYPTO_GRP, 0xA03, cpt_set_crypto_grp, \ msg_rsp) \ M(CPT_INLINE_IPSEC_CFG, 0xA04, cpt_inline_ipsec_cfg, \ cpt_inline_ipsec_cfg_msg, msg_rsp) \ +M(CPT_RX_INLINE_LF_CFG, 0xBFE, cpt_rx_inline_lf_cfg, \ + cpt_rx_inline_lf_cfg_msg, msg_rsp) \ /* NPC mbox IDs (range 0x6000 - 0x7FFF) */ \ M(NPC_MCAM_ALLOC_ENTRY, 0x6000, npc_mcam_alloc_entry, \ npc_mcam_alloc_entry_req, \ @@ -1202,6 +1204,11 @@ struct cpt_inline_ipsec_cfg_msg { uint16_t __otx2_io nix_pf_func; /* Outbound path NIX_PF_FUNC */ }; +struct cpt_rx_inline_lf_cfg_msg { + struct mbox_msghdr hdr; + uint16_t __otx2_io sso_pf_func; +}; + /* NPC mbox message structs */ #define NPC_MCAM_ENTRY_INVALID 0xFFFF From patchwork Sat Jan 18 10:49:00 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Anoob Joseph X-Patchwork-Id: 64858 X-Patchwork-Delegate: gakhil@marvell.com Return-Path: X-Original-To: patchwork@inbox.dpdk.org Delivered-To: patchwork@inbox.dpdk.org Received: from dpdk.org (dpdk.org [92.243.14.124]) by inbox.dpdk.org (Postfix) with ESMTP id D8A7BA051C; Sat, 18 Jan 2020 11:49:54 +0100 (CET) Received: from [92.243.14.124] (localhost [127.0.0.1]) by dpdk.org (Postfix) with ESMTP id B44F4316B; Sat, 18 Jan 2020 11:49:54 +0100 (CET) Received: from mx0b-0016f401.pphosted.com (mx0b-0016f401.pphosted.com [67.231.156.173]) by dpdk.org (Postfix) with ESMTP id F2B4229D6 for ; Sat, 18 Jan 2020 11:49:53 +0100 (CET) Received: from pps.filterd (m0045851.ppops.net [127.0.0.1]) by mx0b-0016f401.pphosted.com (8.16.0.42/8.16.0.42) with SMTP id 00IAmMHk018888; Sat, 18 Jan 2020 02:49:53 -0800 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=marvell.com; h=from : to : cc : subject : date : message-id : in-reply-to : references : mime-version : content-transfer-encoding : content-type; s=pfpt0818; bh=ALnP0KttQ2ckBVZnHJpRnkIB053Mqdnv1Hapk6Y2HnY=; b=A+DPsp1zD6rs8JMV0bTJk9RFgVOpwy7feqjjRz6Tamw40RjxBPzK3Ot7RqeFwzgctyWi 9mCQuJ96rVOllYZvnTyqKpsiR+iy3/JCG0YslnLFjU7jV5fcyKddTnSp9KMeksi36l2Y oLUeepLC9VmKtk3lY7H7w31qXbRZF7266FZZZUhB27Q9rQkECH1xFVsDOKnZhduOrZMW KUBJFr/ubgfGmOUfqioKOo7ZiTFVZcahQmDdIlVdXNyIgbZUu8W1d6HKIPNDtUKDqhw3 LmdYqEfxlPzAOXOTepyVjZZ7XTH691vtXUXJwVkrnXfpbBA/fi4gFU68MzUnh4SOSlUr sg== Received: from sc-exch03.marvell.com ([199.233.58.183]) by mx0b-0016f401.pphosted.com with ESMTP id 2xk0sn6een-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-SHA384 bits=256 verify=NOT); Sat, 18 Jan 2020 02:49:53 -0800 Received: from SC-EXCH01.marvell.com (10.93.176.81) by SC-EXCH03.marvell.com (10.93.176.83) with Microsoft SMTP Server (TLS) id 15.0.1497.2; Sat, 18 Jan 2020 02:49:51 -0800 Received: from maili.marvell.com (10.93.176.43) by SC-EXCH01.marvell.com (10.93.176.81) with Microsoft SMTP Server id 15.0.1497.2 via Frontend Transport; Sat, 18 Jan 2020 02:49:51 -0800 Received: from ajoseph83.caveonetworks.com (unknown [10.29.45.60]) by maili.marvell.com (Postfix) with ESMTP id D2C893F703F; Sat, 18 Jan 2020 02:49:46 -0800 (PST) From: Anoob Joseph To: Akhil Goyal , Declan Doherty , Thomas Monjalon CC: Vamsi Attunuru , Jerin Jacob , Narayana Prasad , Kiran Kumar K , Nithin Dabilpuram , "Pavan Nikhilesh" , Ankur Dwivedi , Anoob Joseph , Archana Muniganti , Tejasree Kondoj , Lukasz Bartosik , Date: Sat, 18 Jan 2020 16:19:00 +0530 Message-ID: <1579344553-11428-3-git-send-email-anoobj@marvell.com> X-Mailer: git-send-email 2.7.4 In-Reply-To: <1579344553-11428-1-git-send-email-anoobj@marvell.com> References: <1575806094-28391-1-git-send-email-anoobj@marvell.com> <1579344553-11428-1-git-send-email-anoobj@marvell.com> MIME-Version: 1.0 X-Proofpoint-Virus-Version: vendor=fsecure engine=2.50.10434:6.0.138, 18.0.572 definitions=2020-01-18_02:2020-01-16, 2020-01-18 signatures=0 Subject: [dpdk-dev] [PATCH v2 02/15] common/octeontx2: add routine to check if sec capable otx2 X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Sender: "dev" From: Vamsi Attunuru This routine returns true if given rte_eth_dev is security offload capable and belongs to octeontx2. Signed-off-by: Anoob Joseph Signed-off-by: Tejasree Kondoj Signed-off-by: Vamsi Attunuru Acked-by: Jerin Jacob --- drivers/common/octeontx2/otx2_common.c | 20 ++++++++++++++++++++ drivers/common/octeontx2/otx2_common.h | 2 ++ .../octeontx2/rte_common_octeontx2_version.map | 1 + 3 files changed, 23 insertions(+) diff --git a/drivers/common/octeontx2/otx2_common.c b/drivers/common/octeontx2/otx2_common.c index 7e45366..2f9b167 100644 --- a/drivers/common/octeontx2/otx2_common.c +++ b/drivers/common/octeontx2/otx2_common.c @@ -3,6 +3,7 @@ */ #include +#include #include #include @@ -23,6 +24,25 @@ otx2_npa_set_defaults(struct otx2_idev_cfg *idev) /** * @internal + * Check if rte_eth_dev is security offload capable otx2_eth_dev + */ +uint8_t +otx2_ethdev_is_sec_capable(struct rte_eth_dev *eth_dev) +{ + struct rte_pci_device *pci_dev; + + pci_dev = RTE_ETH_DEV_TO_PCI(eth_dev); + + if (pci_dev->id.device_id == PCI_DEVID_OCTEONTX2_RVU_PF || + pci_dev->id.device_id == PCI_DEVID_OCTEONTX2_RVU_VF || + pci_dev->id.device_id == PCI_DEVID_OCTEONTX2_RVU_AF_VF) + return 1; + + return 0; +} + +/** + * @internal * Get intra device config structure. */ struct otx2_idev_cfg * diff --git a/drivers/common/octeontx2/otx2_common.h b/drivers/common/octeontx2/otx2_common.h index f62c45d..db0cde1 100644 --- a/drivers/common/octeontx2/otx2_common.h +++ b/drivers/common/octeontx2/otx2_common.h @@ -8,6 +8,7 @@ #include #include #include +#include #include #include #include @@ -70,6 +71,7 @@ struct otx2_idev_cfg *otx2_intra_dev_get_cfg(void); void otx2_sso_pf_func_set(uint16_t sso_pf_func); uint16_t otx2_sso_pf_func_get(void); uint16_t otx2_npa_pf_func_get(void); +uint8_t otx2_ethdev_is_sec_capable(struct rte_eth_dev *eth_dev); struct otx2_npa_lf *otx2_npa_lf_obj_get(void); void otx2_npa_set_defaults(struct otx2_idev_cfg *idev); int otx2_npa_lf_active(void *dev); diff --git a/drivers/common/octeontx2/rte_common_octeontx2_version.map b/drivers/common/octeontx2/rte_common_octeontx2_version.map index adad21a..bd9fc41 100644 --- a/drivers/common/octeontx2/rte_common_octeontx2_version.map +++ b/drivers/common/octeontx2/rte_common_octeontx2_version.map @@ -6,6 +6,7 @@ DPDK_20.0 { otx2_dev_priv_init; otx2_disable_irqs; otx2_intra_dev_get_cfg; + otx2_ethdev_is_sec_capable; otx2_logtype_base; otx2_logtype_dpi; otx2_logtype_mbox; From patchwork Sat Jan 18 10:49:01 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Anoob Joseph X-Patchwork-Id: 64859 X-Patchwork-Delegate: gakhil@marvell.com Return-Path: X-Original-To: patchwork@inbox.dpdk.org Delivered-To: patchwork@inbox.dpdk.org Received: from dpdk.org (dpdk.org [92.243.14.124]) by inbox.dpdk.org (Postfix) with ESMTP id 2BC89A051C; Sat, 18 Jan 2020 11:50:01 +0100 (CET) Received: from [92.243.14.124] (localhost [127.0.0.1]) by dpdk.org (Postfix) with ESMTP id 8F1B7378B; Sat, 18 Jan 2020 11:50:00 +0100 (CET) Received: from mx0b-0016f401.pphosted.com (mx0b-0016f401.pphosted.com [67.231.156.173]) by dpdk.org (Postfix) with ESMTP id A951D29D6 for ; Sat, 18 Jan 2020 11:49:59 +0100 (CET) Received: from pps.filterd (m0045851.ppops.net [127.0.0.1]) by mx0b-0016f401.pphosted.com (8.16.0.42/8.16.0.42) with SMTP id 00IAm3Qc018843; Sat, 18 Jan 2020 02:49:59 -0800 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=marvell.com; h=from : to : cc : subject : date : message-id : in-reply-to : references : mime-version : content-transfer-encoding : content-type; s=pfpt0818; bh=+AfIdMVceFe5GhuuUvlSNTj7FUDsAzOnftEzHSmUdvQ=; b=dkPJRZywaPpJNTfrOXKTUencCgAKcgzgeJPt8OC+SjvZxjfr3GHyc842o/F0W1fYp5mJ 8m9rcXhE9LMPsYYEzZFDYuwd9Qoj+el+/X+l7VSL+z4PJzcxrT68Sh69Sl8mOLrsvGGO wNkdic5PfY0yq/JSD0RahBg+bZi2yq01Kqh0bDNIuLdZ+NtAsxd6yYTT5hmp8sw6j772 QTgpscO1EhMfhjH1V6wrMFkIb5Vna4bi+BMcEJ2JPTeRouyqXViKcbwzcUOwNewi4jgt UlpQa/AH/xREyWkYozUHZ+7A5ZWxGSedbzQG68STdTgEaQjktKQIYMX7DLM2VEg2MnOF 8A== Received: from sc-exch02.marvell.com ([199.233.58.182]) by mx0b-0016f401.pphosted.com with ESMTP id 2xk0sn6ef2-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-SHA384 bits=256 verify=NOT); Sat, 18 Jan 2020 02:49:59 -0800 Received: from SC-EXCH03.marvell.com (10.93.176.83) by SC-EXCH02.marvell.com (10.93.176.82) with Microsoft SMTP Server (TLS) id 15.0.1497.2; Sat, 18 Jan 2020 02:49:57 -0800 Received: from maili.marvell.com (10.93.176.43) by SC-EXCH03.marvell.com (10.93.176.83) with Microsoft SMTP Server id 15.0.1497.2 via Frontend Transport; Sat, 18 Jan 2020 02:49:57 -0800 Received: from ajoseph83.caveonetworks.com (unknown [10.29.45.60]) by maili.marvell.com (Postfix) with ESMTP id B92C23F703F; Sat, 18 Jan 2020 02:49:52 -0800 (PST) From: Anoob Joseph To: Akhil Goyal , Declan Doherty , Thomas Monjalon CC: Tejasree Kondoj , Jerin Jacob , Narayana Prasad , Kiran Kumar K , Nithin Dabilpuram , "Pavan Nikhilesh" , Ankur Dwivedi , Anoob Joseph , Archana Muniganti , Vamsi Attunuru , Lukasz Bartosik , Date: Sat, 18 Jan 2020 16:19:01 +0530 Message-ID: <1579344553-11428-4-git-send-email-anoobj@marvell.com> X-Mailer: git-send-email 2.7.4 In-Reply-To: <1579344553-11428-1-git-send-email-anoobj@marvell.com> References: <1575806094-28391-1-git-send-email-anoobj@marvell.com> <1579344553-11428-1-git-send-email-anoobj@marvell.com> MIME-Version: 1.0 X-Proofpoint-Virus-Version: vendor=fsecure engine=2.50.10434:6.0.138, 18.0.572 definitions=2020-01-18_02:2020-01-16, 2020-01-18 signatures=0 Subject: [dpdk-dev] [PATCH v2 03/15] crypto/octeontx2: configure for inline IPsec X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Sender: "dev" From: Tejasree Kondoj For enabling outbound inline IPsec, a CPT queue needs to be tied to a NIX PF_FUNC. Distribute CPT queues fairly among all available otx2 eth ports. For inbound, one CPT LF will be assigned and initialized by kernel. Signed-off-by: Ankur Dwivedi Signed-off-by: Anoob Joseph Signed-off-by: Archana Muniganti Signed-off-by: Tejasree Kondoj Signed-off-by: Vamsi Attunuru --- drivers/crypto/octeontx2/Makefile | 3 +- drivers/crypto/octeontx2/meson.build | 2 + drivers/crypto/octeontx2/otx2_cryptodev_mbox.c | 53 ++++++++++++++++++++++++++ drivers/crypto/octeontx2/otx2_cryptodev_mbox.h | 7 ++++ drivers/crypto/octeontx2/otx2_cryptodev_ops.c | 38 ++++++++++++++++++ 5 files changed, 102 insertions(+), 1 deletion(-) diff --git a/drivers/crypto/octeontx2/Makefile b/drivers/crypto/octeontx2/Makefile index f7d6c37..3ba67ed 100644 --- a/drivers/crypto/octeontx2/Makefile +++ b/drivers/crypto/octeontx2/Makefile @@ -10,7 +10,7 @@ LIB = librte_pmd_octeontx2_crypto.a # build flags CFLAGS += $(WERROR_FLAGS) -LDLIBS += -lrte_eal -lrte_mbuf -lrte_mempool -lrte_ring +LDLIBS += -lrte_eal -lrte_ethdev -lrte_mbuf -lrte_mempool -lrte_ring LDLIBS += -lrte_cryptodev LDLIBS += -lrte_pci -lrte_bus_pci LDLIBS += -lrte_common_cpt -lrte_common_octeontx2 @@ -21,6 +21,7 @@ CFLAGS += -O3 CFLAGS += -I$(RTE_SDK)/drivers/common/cpt CFLAGS += -I$(RTE_SDK)/drivers/common/octeontx2 CFLAGS += -I$(RTE_SDK)/drivers/mempool/octeontx2 +CFLAGS += -I$(RTE_SDK)/drivers/net/octeontx2 CFLAGS += -DALLOW_EXPERIMENTAL_API ifneq ($(CONFIG_RTE_ARCH_64),y) diff --git a/drivers/crypto/octeontx2/meson.build b/drivers/crypto/octeontx2/meson.build index b6e5b73..67deca3 100644 --- a/drivers/crypto/octeontx2/meson.build +++ b/drivers/crypto/octeontx2/meson.build @@ -8,6 +8,7 @@ endif deps += ['bus_pci'] deps += ['common_cpt'] deps += ['common_octeontx2'] +deps += ['ethdev'] name = 'octeontx2_crypto' allow_experimental_apis = true @@ -32,3 +33,4 @@ endforeach includes += include_directories('../../common/cpt') includes += include_directories('../../common/octeontx2') includes += include_directories('../../mempool/octeontx2') +includes += include_directories('../../net/octeontx2') diff --git a/drivers/crypto/octeontx2/otx2_cryptodev_mbox.c b/drivers/crypto/octeontx2/otx2_cryptodev_mbox.c index b54e407..202832d 100644 --- a/drivers/crypto/octeontx2/otx2_cryptodev_mbox.c +++ b/drivers/crypto/octeontx2/otx2_cryptodev_mbox.c @@ -2,10 +2,13 @@ * Copyright (C) 2019 Marvell International Ltd. */ #include +#include #include "otx2_cryptodev.h" +#include "otx2_cryptodev_hw_access.h" #include "otx2_cryptodev_mbox.h" #include "otx2_dev.h" +#include "otx2_ethdev.h" #include "otx2_mbox.h" #include "cpt_pmd_logs.h" @@ -173,3 +176,53 @@ otx2_cpt_af_reg_write(const struct rte_cryptodev *dev, uint64_t reg, return otx2_cpt_send_mbox_msg(vf); } + +int +otx2_cpt_inline_init(const struct rte_cryptodev *dev) +{ + struct otx2_cpt_vf *vf = dev->data->dev_private; + struct otx2_mbox *mbox = vf->otx2_dev.mbox; + struct cpt_rx_inline_lf_cfg_msg *msg; + int ret; + + msg = otx2_mbox_alloc_msg_cpt_rx_inline_lf_cfg(mbox); + msg->sso_pf_func = otx2_sso_pf_func_get(); + + otx2_mbox_msg_send(mbox, 0); + ret = otx2_mbox_process(mbox); + if (ret < 0) + return -EIO; + + return 0; +} + +int +otx2_cpt_qp_ethdev_bind(const struct rte_cryptodev *dev, struct otx2_cpt_qp *qp, + uint16_t port_id) +{ + struct rte_eth_dev *eth_dev = &rte_eth_devices[port_id]; + struct otx2_cpt_vf *vf = dev->data->dev_private; + struct otx2_mbox *mbox = vf->otx2_dev.mbox; + struct cpt_inline_ipsec_cfg_msg *msg; + struct otx2_eth_dev *otx2_eth_dev; + int ret; + + if (!otx2_ethdev_is_sec_capable(&rte_eth_devices[port_id])) + return -EINVAL; + + otx2_eth_dev = otx2_eth_pmd_priv(eth_dev); + + msg = otx2_mbox_alloc_msg_cpt_inline_ipsec_cfg(mbox); + msg->dir = CPT_INLINE_OUTBOUND; + msg->enable = 1; + msg->slot = qp->id; + + msg->nix_pf_func = otx2_eth_dev->pf_func; + + otx2_mbox_msg_send(mbox, 0); + ret = otx2_mbox_process(mbox); + if (ret < 0) + return -EIO; + + return 0; +} diff --git a/drivers/crypto/octeontx2/otx2_cryptodev_mbox.h b/drivers/crypto/octeontx2/otx2_cryptodev_mbox.h index a298718..ae66b08 100644 --- a/drivers/crypto/octeontx2/otx2_cryptodev_mbox.h +++ b/drivers/crypto/octeontx2/otx2_cryptodev_mbox.h @@ -7,6 +7,8 @@ #include +#include "otx2_cryptodev_hw_access.h" + int otx2_cpt_available_queues_get(const struct rte_cryptodev *dev, uint16_t *nb_queues); @@ -22,4 +24,9 @@ int otx2_cpt_af_reg_read(const struct rte_cryptodev *dev, uint64_t reg, int otx2_cpt_af_reg_write(const struct rte_cryptodev *dev, uint64_t reg, uint64_t val); +int otx2_cpt_qp_ethdev_bind(const struct rte_cryptodev *dev, + struct otx2_cpt_qp *qp, uint16_t port_id); + +int otx2_cpt_inline_init(const struct rte_cryptodev *dev); + #endif /* _OTX2_CRYPTODEV_MBOX_H_ */ diff --git a/drivers/crypto/octeontx2/otx2_cryptodev_ops.c b/drivers/crypto/octeontx2/otx2_cryptodev_ops.c index ec0e58d..b45cb82 100644 --- a/drivers/crypto/octeontx2/otx2_cryptodev_ops.c +++ b/drivers/crypto/octeontx2/otx2_cryptodev_ops.c @@ -6,6 +6,7 @@ #include #include +#include #include "otx2_cryptodev.h" #include "otx2_cryptodev_capabilities.h" @@ -127,6 +128,29 @@ otx2_cpt_metabuf_mempool_destroy(struct otx2_cpt_qp *qp) meta_info->sg_mlen = 0; } +static int +otx2_cpt_qp_inline_cfg(const struct rte_cryptodev *dev, struct otx2_cpt_qp *qp) +{ + static rte_atomic16_t port_offset = RTE_ATOMIC16_INIT(-1); + uint16_t port_id, nb_ethport = rte_eth_dev_count_avail(); + int i, ret; + + for (i = 0; i < nb_ethport; i++) { + port_id = rte_atomic16_add_return(&port_offset, 1) % nb_ethport; + if (otx2_ethdev_is_sec_capable(&rte_eth_devices[port_id])) + break; + } + + if (i >= nb_ethport) + return 0; + + ret = otx2_cpt_qp_ethdev_bind(dev, qp, port_id); + if (ret) + return ret; + + return 0; +} + static struct otx2_cpt_qp * otx2_cpt_qp_create(const struct rte_cryptodev *dev, uint16_t qp_id, uint8_t group) @@ -220,6 +244,12 @@ otx2_cpt_qp_create(const struct rte_cryptodev *dev, uint16_t qp_id, otx2_cpt_iq_disable(qp); + ret = otx2_cpt_qp_inline_cfg(dev, qp); + if (ret) { + CPT_LOG_ERR("Could not configure queue for inline IPsec"); + goto mempool_destroy; + } + ret = otx2_cpt_iq_enable(dev, qp, group, OTX2_CPT_QUEUE_HI_PRIO, size_div40); if (ret) { @@ -913,12 +943,20 @@ otx2_cpt_dev_config(struct rte_cryptodev *dev, goto queues_detach; } + ret = otx2_cpt_inline_init(dev); + if (ret) { + CPT_LOG_ERR("Could not enable inline IPsec"); + goto intr_unregister; + } + dev->enqueue_burst = otx2_cpt_enqueue_burst; dev->dequeue_burst = otx2_cpt_dequeue_burst; rte_mb(); return 0; +intr_unregister: + otx2_cpt_err_intr_unregister(dev); queues_detach: otx2_cpt_queues_detach(dev); return ret; From patchwork Sat Jan 18 10:49:02 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Anoob Joseph X-Patchwork-Id: 64860 X-Patchwork-Delegate: gakhil@marvell.com Return-Path: X-Original-To: patchwork@inbox.dpdk.org Delivered-To: patchwork@inbox.dpdk.org Received: from dpdk.org (dpdk.org [92.243.14.124]) by inbox.dpdk.org (Postfix) with ESMTP id 4C5ACA051C; Sat, 18 Jan 2020 11:50:09 +0100 (CET) Received: from [92.243.14.124] (localhost [127.0.0.1]) by dpdk.org (Postfix) with ESMTP id 56D9044C7; Sat, 18 Jan 2020 11:50:07 +0100 (CET) Received: from mx0b-0016f401.pphosted.com (mx0b-0016f401.pphosted.com [67.231.156.173]) by dpdk.org (Postfix) with ESMTP id CC3F144C7 for ; Sat, 18 Jan 2020 11:50:05 +0100 (CET) Received: from pps.filterd (m0045851.ppops.net [127.0.0.1]) by mx0b-0016f401.pphosted.com (8.16.0.42/8.16.0.42) with SMTP id 00IAm3Qe018843; Sat, 18 Jan 2020 02:50:05 -0800 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=marvell.com; h=from : to : cc : subject : date : message-id : in-reply-to : references : mime-version : content-transfer-encoding : content-type; s=pfpt0818; bh=kfYiIi9gYk8rQz3ZiavAnaoYM76MKnZmHfW32oyQLHg=; b=DviqiN1Tsqf8rP5e7znhF4F0AonsaBGkoEiLI02rJqn5jcX65BzhLZje9Vou05TIXBRW G2zR5DI/WMJ3BhXPu/6zMG72bEnDH/V08Mf5u3NVbuR7EEaOFy/veN6dgLqBqZ4GUdGa WpYIkk8gqw6Vgy1lf6JGPMTQJ8iLNtk1K2SE0rz5wZJ0iATWXqQ/3nkrT9RgHTlL5Qle C6/2q2G9kLQ9HvybCuSGdtz5I+enT0yzz/vR8qDIKKfxfkBZwsLYHbqa+FQcbYY9Nwvn 9lGtlnCvEQnVwrZKrJeczkQqcCC0OkbKxe3ab0aspkx2yhkbeKZ4kZIh52EBX1mcVqA6 +Q== Received: from sc-exch01.marvell.com ([199.233.58.181]) by mx0b-0016f401.pphosted.com with ESMTP id 2xk0sn6eff-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-SHA384 bits=256 verify=NOT); Sat, 18 Jan 2020 02:50:05 -0800 Received: from SC-EXCH01.marvell.com (10.93.176.81) by SC-EXCH01.marvell.com (10.93.176.81) with Microsoft SMTP Server (TLS) id 15.0.1497.2; Sat, 18 Jan 2020 02:50:02 -0800 Received: from maili.marvell.com (10.93.176.43) by SC-EXCH01.marvell.com (10.93.176.81) with Microsoft SMTP Server id 15.0.1497.2 via Frontend Transport; Sat, 18 Jan 2020 02:50:02 -0800 Received: from ajoseph83.caveonetworks.com (unknown [10.29.45.60]) by maili.marvell.com (Postfix) with ESMTP id 8024C3F703F; Sat, 18 Jan 2020 02:49:58 -0800 (PST) From: Anoob Joseph To: Akhil Goyal , Declan Doherty , Thomas Monjalon CC: Anoob Joseph , Jerin Jacob , Narayana Prasad , Kiran Kumar K , Nithin Dabilpuram , "Pavan Nikhilesh" , Ankur Dwivedi , Archana Muniganti , Tejasree Kondoj , Vamsi Attunuru , "Lukasz Bartosik" , Date: Sat, 18 Jan 2020 16:19:02 +0530 Message-ID: <1579344553-11428-5-git-send-email-anoobj@marvell.com> X-Mailer: git-send-email 2.7.4 In-Reply-To: <1579344553-11428-1-git-send-email-anoobj@marvell.com> References: <1575806094-28391-1-git-send-email-anoobj@marvell.com> <1579344553-11428-1-git-send-email-anoobj@marvell.com> MIME-Version: 1.0 X-Proofpoint-Virus-Version: vendor=fsecure engine=2.50.10434:6.0.138, 18.0.572 definitions=2020-01-18_02:2020-01-16, 2020-01-18 signatures=0 Subject: [dpdk-dev] [PATCH v2 04/15] crypto/octeontx2: create eth security ctx X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Sender: "dev" Adding security ctx to the eth device. Signed-off-by: Ankur Dwivedi Signed-off-by: Anoob Joseph Signed-off-by: Archana Muniganti Signed-off-by: Tejasree Kondoj Signed-off-by: Vamsi Attunuru --- drivers/common/octeontx2/otx2_common.c | 2 ++ drivers/common/octeontx2/otx2_common.h | 10 +++++++ .../octeontx2/rte_common_octeontx2_version.map | 2 ++ drivers/crypto/octeontx2/Makefile | 3 +- drivers/crypto/octeontx2/meson.build | 4 ++- drivers/crypto/octeontx2/otx2_cryptodev.c | 4 +++ drivers/crypto/octeontx2/otx2_security.c | 35 ++++++++++++++++++++++ drivers/crypto/octeontx2/otx2_security.h | 14 +++++++++ drivers/net/octeontx2/otx2_ethdev.c | 18 ++++++++++- 9 files changed, 89 insertions(+), 3 deletions(-) create mode 100644 drivers/crypto/octeontx2/otx2_security.c create mode 100644 drivers/crypto/octeontx2/otx2_security.h diff --git a/drivers/common/octeontx2/otx2_common.c b/drivers/common/octeontx2/otx2_common.c index 2f9b167..5c41822 100644 --- a/drivers/common/octeontx2/otx2_common.c +++ b/drivers/common/octeontx2/otx2_common.c @@ -11,6 +11,8 @@ #include "otx2_dev.h" #include "otx2_mbox.h" +struct otx2_sec_eth_crypto_idev_ops otx2_sec_idev_ops; + /** * @internal * Set default NPA configuration. diff --git a/drivers/common/octeontx2/otx2_common.h b/drivers/common/octeontx2/otx2_common.h index db0cde1..4e8d0af 100644 --- a/drivers/common/octeontx2/otx2_common.h +++ b/drivers/common/octeontx2/otx2_common.h @@ -77,6 +77,16 @@ void otx2_npa_set_defaults(struct otx2_idev_cfg *idev); int otx2_npa_lf_active(void *dev); int otx2_npa_lf_obj_ref(void); +typedef int (*otx2_sec_eth_ctx_create_t)(struct rte_eth_dev *eth_dev); +typedef void (*otx2_sec_eth_ctx_destroy_t)(struct rte_eth_dev *eth_dev); + +struct otx2_sec_eth_crypto_idev_ops { + otx2_sec_eth_ctx_create_t ctx_create; + otx2_sec_eth_ctx_destroy_t ctx_destroy; +}; + +extern struct otx2_sec_eth_crypto_idev_ops otx2_sec_idev_ops; + /* Log */ extern int otx2_logtype_base; extern int otx2_logtype_mbox; diff --git a/drivers/common/octeontx2/rte_common_octeontx2_version.map b/drivers/common/octeontx2/rte_common_octeontx2_version.map index bd9fc41..1a43bb6 100644 --- a/drivers/common/octeontx2/rte_common_octeontx2_version.map +++ b/drivers/common/octeontx2/rte_common_octeontx2_version.map @@ -32,5 +32,7 @@ DPDK_20.0 { otx2_sso_pf_func_set; otx2_unregister_irq; + otx2_sec_idev_ops; + local: *; }; diff --git a/drivers/crypto/octeontx2/Makefile b/drivers/crypto/octeontx2/Makefile index 3ba67ed..d2e9b9f 100644 --- a/drivers/crypto/octeontx2/Makefile +++ b/drivers/crypto/octeontx2/Makefile @@ -11,7 +11,7 @@ LIB = librte_pmd_octeontx2_crypto.a CFLAGS += $(WERROR_FLAGS) LDLIBS += -lrte_eal -lrte_ethdev -lrte_mbuf -lrte_mempool -lrte_ring -LDLIBS += -lrte_cryptodev +LDLIBS += -lrte_cryptodev -lrte_security LDLIBS += -lrte_pci -lrte_bus_pci LDLIBS += -lrte_common_cpt -lrte_common_octeontx2 @@ -38,6 +38,7 @@ SRCS-$(CONFIG_RTE_LIBRTE_PMD_OCTEONTX2_CRYPTO) += otx2_cryptodev_capabilities.c SRCS-$(CONFIG_RTE_LIBRTE_PMD_OCTEONTX2_CRYPTO) += otx2_cryptodev_hw_access.c SRCS-$(CONFIG_RTE_LIBRTE_PMD_OCTEONTX2_CRYPTO) += otx2_cryptodev_mbox.c SRCS-$(CONFIG_RTE_LIBRTE_PMD_OCTEONTX2_CRYPTO) += otx2_cryptodev_ops.c +SRCS-$(CONFIG_RTE_LIBRTE_PMD_OCTEONTX2_CRYPTO) += otx2_security.c # export include files SYMLINK-y-include += diff --git a/drivers/crypto/octeontx2/meson.build b/drivers/crypto/octeontx2/meson.build index 67deca3..f7b2937 100644 --- a/drivers/crypto/octeontx2/meson.build +++ b/drivers/crypto/octeontx2/meson.build @@ -9,6 +9,7 @@ deps += ['bus_pci'] deps += ['common_cpt'] deps += ['common_octeontx2'] deps += ['ethdev'] +deps += ['security'] name = 'octeontx2_crypto' allow_experimental_apis = true @@ -16,7 +17,8 @@ sources = files('otx2_cryptodev.c', 'otx2_cryptodev_capabilities.c', 'otx2_cryptodev_hw_access.c', 'otx2_cryptodev_mbox.c', - 'otx2_cryptodev_ops.c') + 'otx2_cryptodev_ops.c', + 'otx2_security.c') extra_flags = [] # This integrated controller runs only on a arm64 machine, remove 32bit warnings diff --git a/drivers/crypto/octeontx2/otx2_cryptodev.c b/drivers/crypto/octeontx2/otx2_cryptodev.c index 7fd216b..86c1188 100644 --- a/drivers/crypto/octeontx2/otx2_cryptodev.c +++ b/drivers/crypto/octeontx2/otx2_cryptodev.c @@ -17,6 +17,7 @@ #include "otx2_cryptodev_mbox.h" #include "otx2_cryptodev_ops.h" #include "otx2_dev.h" +#include "otx2_security.h" /* CPT common headers */ #include "cpt_common.h" @@ -154,4 +155,7 @@ RTE_INIT(otx2_cpt_init_log) otx2_cpt_logtype = rte_log_register("pmd.crypto.octeontx2"); if (otx2_cpt_logtype >= 0) rte_log_set_level(otx2_cpt_logtype, RTE_LOG_NOTICE); + + otx2_sec_idev_ops.ctx_create = otx2_sec_eth_ctx_create; + otx2_sec_idev_ops.ctx_destroy = otx2_sec_eth_ctx_destroy; } diff --git a/drivers/crypto/octeontx2/otx2_security.c b/drivers/crypto/octeontx2/otx2_security.c new file mode 100644 index 0000000..531c78b --- /dev/null +++ b/drivers/crypto/octeontx2/otx2_security.c @@ -0,0 +1,35 @@ +/* SPDX-License-Identifier: BSD-3-Clause + * Copyright (C) 2020 Marvell International Ltd. + */ + +#include +#include +#include + +#include "otx2_security.h" + +int +otx2_sec_eth_ctx_create(struct rte_eth_dev *eth_dev) +{ + struct rte_security_ctx *ctx; + + ctx = rte_malloc("otx2_sec_eth_ctx", + sizeof(struct rte_security_ctx), 0); + if (ctx == NULL) + return -ENOMEM; + + /* Populate ctx */ + + ctx->device = eth_dev; + ctx->sess_cnt = 0; + + eth_dev->security_ctx = ctx; + + return 0; +} + +void +otx2_sec_eth_ctx_destroy(struct rte_eth_dev *eth_dev) +{ + rte_free(eth_dev->security_ctx); +} diff --git a/drivers/crypto/octeontx2/otx2_security.h b/drivers/crypto/octeontx2/otx2_security.h new file mode 100644 index 0000000..21b7da4 --- /dev/null +++ b/drivers/crypto/octeontx2/otx2_security.h @@ -0,0 +1,14 @@ +/* SPDX-License-Identifier: BSD-3-Clause + * Copyright (C) 2020 Marvell International Ltd. + */ + +#ifndef __OTX2_SECURITY_H__ +#define __OTX2_SECURITY_H__ + +#include + +int otx2_sec_eth_ctx_create(struct rte_eth_dev *eth_dev); + +void otx2_sec_eth_ctx_destroy(struct rte_eth_dev *eth_dev); + +#endif /* __OTX2_SECURITY_H__ */ diff --git a/drivers/net/octeontx2/otx2_ethdev.c b/drivers/net/octeontx2/otx2_ethdev.c index ed32927..3e19ac2 100644 --- a/drivers/net/octeontx2/otx2_ethdev.c +++ b/drivers/net/octeontx2/otx2_ethdev.c @@ -2236,10 +2236,19 @@ otx2_eth_dev_init(struct rte_eth_dev *eth_dev) dev->hwcap |= OTX2_FIXUP_F_LIMIT_CQ_FULL; } + /* Create security ctx */ + if (otx2_sec_idev_ops.ctx_create != NULL) { + rc = otx2_sec_idev_ops.ctx_create(eth_dev); + if (rc) + goto free_mac_addrs; + dev->tx_offload_capa |= DEV_TX_OFFLOAD_SECURITY; + dev->rx_offload_capa |= DEV_RX_OFFLOAD_SECURITY; + } + /* Initialize rte-flow */ rc = otx2_flow_init(dev); if (rc) - goto free_mac_addrs; + goto sec_ctx_destroy; otx2_nix_mc_filter_init(dev); @@ -2250,6 +2259,9 @@ otx2_eth_dev_init(struct rte_eth_dev *eth_dev) dev->rx_offload_capa, dev->tx_offload_capa); return 0; +sec_ctx_destroy: + if (otx2_sec_idev_ops.ctx_destroy != NULL) + otx2_sec_idev_ops.ctx_destroy(eth_dev); free_mac_addrs: rte_free(eth_dev->data->mac_addrs); unregister_irq: @@ -2333,6 +2345,10 @@ otx2_eth_dev_uninit(struct rte_eth_dev *eth_dev, bool mbox_close) if (rc) otx2_err("Failed to cleanup npa lf, rc=%d", rc); + /* Destroy security ctx */ + if (otx2_sec_idev_ops.ctx_destroy != NULL) + otx2_sec_idev_ops.ctx_destroy(eth_dev); + rte_free(eth_dev->data->mac_addrs); eth_dev->data->mac_addrs = NULL; dev->drv_inited = false; From patchwork Sat Jan 18 10:49:03 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Anoob Joseph X-Patchwork-Id: 64861 X-Patchwork-Delegate: gakhil@marvell.com Return-Path: X-Original-To: patchwork@inbox.dpdk.org Delivered-To: patchwork@inbox.dpdk.org Received: from dpdk.org (dpdk.org [92.243.14.124]) by inbox.dpdk.org (Postfix) with ESMTP id 873BFA051C; Sat, 18 Jan 2020 11:50:19 +0100 (CET) Received: from [92.243.14.124] (localhost [127.0.0.1]) by dpdk.org (Postfix) with ESMTP id B99123253; Sat, 18 Jan 2020 11:50:13 +0100 (CET) Received: from mx0b-0016f401.pphosted.com (mx0a-0016f401.pphosted.com [67.231.148.174]) by dpdk.org (Postfix) with ESMTP id 2B71B29D6 for ; Sat, 18 Jan 2020 11:50:12 +0100 (CET) Received: from pps.filterd (m0045849.ppops.net [127.0.0.1]) by mx0a-0016f401.pphosted.com (8.16.0.42/8.16.0.42) with SMTP id 00IAoB3B019397; Sat, 18 Jan 2020 02:50:11 -0800 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=marvell.com; h=from : to : cc : subject : date : message-id : in-reply-to : references : mime-version : content-transfer-encoding : content-type; s=pfpt0818; bh=mvtLZ1qjHKbREoKJAhYxvENJmFv5f+uuOrA5l1Vz9As=; b=kpXByhbzt6/4hl50pY8VPNlsRLcztgTAhe9mTdyFeMqKCOUXpwyBivP4Z2VQXI0xrogK dY/uZpNqebQnr3KgRhBvrm3zWpYbJjS1OJ3+2j4v8MLxMMHpFnruD9iSGrpTP0Yzn5kX jCE1wk/SWnTU5C6fsRNgAuLd61wJ65Kjxoy5UMrUjHI8iklMeS87q1poMYDPnz/jczV4 Zk4SVXBxpCgbAah5jl8GK1z/N8uyH/myo7dYiDyTCO/c1DB81LFMzqMkIjJqPyHX3OdN yIPh5vcqdZPD8huX6iMlIIeFwv28DHxcq0ihuBzzoZ+m1ZeB5Gt42Jft+IDi12jPk/vh xg== Received: from sc-exch04.marvell.com ([199.233.58.184]) by mx0a-0016f401.pphosted.com with ESMTP id 2xm08v01n7-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-SHA384 bits=256 verify=NOT); Sat, 18 Jan 2020 02:50:11 -0800 Received: from SC-EXCH01.marvell.com (10.93.176.81) by SC-EXCH04.marvell.com (10.93.176.84) with Microsoft SMTP Server (TLS) id 15.0.1497.2; Sat, 18 Jan 2020 02:50:09 -0800 Received: from maili.marvell.com (10.93.176.43) by SC-EXCH01.marvell.com (10.93.176.81) with Microsoft SMTP Server id 15.0.1497.2 via Frontend Transport; Sat, 18 Jan 2020 02:50:09 -0800 Received: from ajoseph83.caveonetworks.com (unknown [10.29.45.60]) by maili.marvell.com (Postfix) with ESMTP id 928D83F7041; Sat, 18 Jan 2020 02:50:04 -0800 (PST) From: Anoob Joseph To: Akhil Goyal , Declan Doherty , Thomas Monjalon CC: Tejasree Kondoj , Jerin Jacob , Narayana Prasad , Kiran Kumar K , Nithin Dabilpuram , "Pavan Nikhilesh" , Ankur Dwivedi , Anoob Joseph , Archana Muniganti , Vamsi Attunuru , Lukasz Bartosik , Date: Sat, 18 Jan 2020 16:19:03 +0530 Message-ID: <1579344553-11428-6-git-send-email-anoobj@marvell.com> X-Mailer: git-send-email 2.7.4 In-Reply-To: <1579344553-11428-1-git-send-email-anoobj@marvell.com> References: <1575806094-28391-1-git-send-email-anoobj@marvell.com> <1579344553-11428-1-git-send-email-anoobj@marvell.com> MIME-Version: 1.0 X-Proofpoint-Virus-Version: vendor=fsecure engine=2.50.10434:6.0.138, 18.0.572 definitions=2020-01-18_02:2020-01-16, 2020-01-18 signatures=0 Subject: [dpdk-dev] [PATCH v2 05/15] crypto/octeontx2: add security in eth dev configure X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Sender: "dev" From: Tejasree Kondoj Adding security in eth device configure. Signed-off-by: Ankur Dwivedi Signed-off-by: Anoob Joseph Signed-off-by: Archana Muniganti Signed-off-by: Tejasree Kondoj Signed-off-by: Vamsi Attunuru --- doc/guides/nics/octeontx2.rst | 20 +++++ doc/guides/rel_notes/release_20_02.rst | 9 ++ drivers/common/octeontx2/otx2_common.h | 4 + drivers/crypto/octeontx2/Makefile | 2 +- drivers/crypto/octeontx2/meson.build | 1 + drivers/crypto/octeontx2/otx2_cryptodev.c | 2 + drivers/crypto/octeontx2/otx2_ipsec_fp.h | 55 +++++++++++++ drivers/crypto/octeontx2/otx2_security.c | 122 ++++++++++++++++++++++++++++ drivers/crypto/octeontx2/otx2_security.h | 4 + drivers/net/octeontx2/otx2_ethdev.c | 22 ++++- drivers/net/octeontx2/otx2_ethdev.h | 2 + drivers/net/octeontx2/otx2_ethdev_devargs.c | 19 +++++ 12 files changed, 260 insertions(+), 2 deletions(-) create mode 100644 drivers/crypto/octeontx2/otx2_ipsec_fp.h diff --git a/doc/guides/nics/octeontx2.rst b/doc/guides/nics/octeontx2.rst index db62a45..fd4e455 100644 --- a/doc/guides/nics/octeontx2.rst +++ b/doc/guides/nics/octeontx2.rst @@ -38,6 +38,7 @@ Features of the OCTEON TX2 Ethdev PMD are: - IEEE1588 timestamping - HW offloaded `ethdev Rx queue` to `eventdev event queue` packet injection - Support Rx interrupt +- Inline IPsec processing support Prerequisites ------------- @@ -178,6 +179,17 @@ Runtime Config Options traffic on this port should be higig2 traffic only. Supported switch header types are "higig2" and "dsa". +- ``Max SPI for inbound inline IPsec`` (default ``1``) + + Max SPI supported for inbound inline IPsec processing can be specified by + ``ipsec_in_max_spi`` ``devargs`` parameter. + + For example:: + -w 0002:02:00.0,ipsec_in_max_spi=128 + + With the above configuration, application can enable inline IPsec processing + on 128 SAs (SPI 0-127). + .. note:: Above devarg parameters are configurable per device, user needs to pass the @@ -211,6 +223,14 @@ SDP interface support ~~~~~~~~~~~~~~~~~~~~~ OCTEON TX2 SDP interface support is limited to PF device, No VF support. +Inline Protocol Processing +~~~~~~~~~~~~~~~~~~~~~~~~~~ +``net_octeontx2`` pmd doesn't support the following features for packets to be +inline protocol processed. +- TSO offload +- VLAN/QinQ offload +- Fragmentation + Debugging Options ----------------- diff --git a/doc/guides/rel_notes/release_20_02.rst b/doc/guides/rel_notes/release_20_02.rst index 6cbe457..8e71fef 100644 --- a/doc/guides/rel_notes/release_20_02.rst +++ b/doc/guides/rel_notes/release_20_02.rst @@ -82,6 +82,15 @@ New Features Added Chacha20-Poly1305 AEAD algorithm. +* **Added inline IPsec support to Marvell OCTEONTX2 PMD.** + + Added inline IPsec support to Marvell OCTEONTX2 PMD. With the feature, + applications would be able to offload entire IPsec offload to the hardware. + For the configured sessions, hardware will do the lookup and perform + decryption and IPsec transformation. For the outbound path, application + can submit a plain packet to the PMD, and it would be sent out on wire + after doing encryption and IPsec transformation of the packet. + Removed Items ------------- diff --git a/drivers/common/octeontx2/otx2_common.h b/drivers/common/octeontx2/otx2_common.h index 4e8d0af..fbe7335 100644 --- a/drivers/common/octeontx2/otx2_common.h +++ b/drivers/common/octeontx2/otx2_common.h @@ -79,10 +79,14 @@ int otx2_npa_lf_obj_ref(void); typedef int (*otx2_sec_eth_ctx_create_t)(struct rte_eth_dev *eth_dev); typedef void (*otx2_sec_eth_ctx_destroy_t)(struct rte_eth_dev *eth_dev); +typedef int (*otx2_sec_eth_init_t)(struct rte_eth_dev *eth_dev); +typedef void (*otx2_sec_eth_fini_t)(struct rte_eth_dev *eth_dev); struct otx2_sec_eth_crypto_idev_ops { otx2_sec_eth_ctx_create_t ctx_create; otx2_sec_eth_ctx_destroy_t ctx_destroy; + otx2_sec_eth_init_t init; + otx2_sec_eth_fini_t fini; }; extern struct otx2_sec_eth_crypto_idev_ops otx2_sec_idev_ops; diff --git a/drivers/crypto/octeontx2/Makefile b/drivers/crypto/octeontx2/Makefile index d2e9b9f..5966ddc 100644 --- a/drivers/crypto/octeontx2/Makefile +++ b/drivers/crypto/octeontx2/Makefile @@ -11,7 +11,7 @@ LIB = librte_pmd_octeontx2_crypto.a CFLAGS += $(WERROR_FLAGS) LDLIBS += -lrte_eal -lrte_ethdev -lrte_mbuf -lrte_mempool -lrte_ring -LDLIBS += -lrte_cryptodev -lrte_security +LDLIBS += -lrte_cryptodev -lrte_security -lrte_eventdev LDLIBS += -lrte_pci -lrte_bus_pci LDLIBS += -lrte_common_cpt -lrte_common_octeontx2 diff --git a/drivers/crypto/octeontx2/meson.build b/drivers/crypto/octeontx2/meson.build index f7b2937..f0f5043 100644 --- a/drivers/crypto/octeontx2/meson.build +++ b/drivers/crypto/octeontx2/meson.build @@ -9,6 +9,7 @@ deps += ['bus_pci'] deps += ['common_cpt'] deps += ['common_octeontx2'] deps += ['ethdev'] +deps += ['eventdev'] deps += ['security'] name = 'octeontx2_crypto' diff --git a/drivers/crypto/octeontx2/otx2_cryptodev.c b/drivers/crypto/octeontx2/otx2_cryptodev.c index 86c1188..34feb82 100644 --- a/drivers/crypto/octeontx2/otx2_cryptodev.c +++ b/drivers/crypto/octeontx2/otx2_cryptodev.c @@ -158,4 +158,6 @@ RTE_INIT(otx2_cpt_init_log) otx2_sec_idev_ops.ctx_create = otx2_sec_eth_ctx_create; otx2_sec_idev_ops.ctx_destroy = otx2_sec_eth_ctx_destroy; + otx2_sec_idev_ops.init = otx2_sec_eth_init; + otx2_sec_idev_ops.fini = otx2_sec_eth_fini; } diff --git a/drivers/crypto/octeontx2/otx2_ipsec_fp.h b/drivers/crypto/octeontx2/otx2_ipsec_fp.h new file mode 100644 index 0000000..bf4181a --- /dev/null +++ b/drivers/crypto/octeontx2/otx2_ipsec_fp.h @@ -0,0 +1,55 @@ +/* SPDX-License-Identifier: BSD-3-Clause + * Copyright(C) 2020 Marvell International Ltd. + */ + +#ifndef __OTX2_IPSEC_FP_H__ +#define __OTX2_IPSEC_FP_H__ + +struct otx2_ipsec_fp_sa_ctl { + rte_be32_t spi : 32; + uint64_t exp_proto_inter_frag : 8; + uint64_t rsvd_42_40 : 3; + uint64_t esn_en : 1; + uint64_t rsvd_45_44 : 2; + uint64_t encap_type : 2; + uint64_t enc_type : 3; + uint64_t rsvd_48 : 1; + uint64_t auth_type : 4; + uint64_t valid : 1; + uint64_t direction : 1; + uint64_t outer_ip_ver : 1; + uint64_t inner_ip_ver : 1; + uint64_t ipsec_mode : 1; + uint64_t ipsec_proto : 1; + uint64_t aes_key_len : 2; +}; + +struct otx2_ipsec_fp_in_sa { + /* w0 */ + struct otx2_ipsec_fp_sa_ctl ctl; + + /* w1 */ + uint8_t nonce[4]; /* Only for AES-GCM */ + uint32_t unused; + + /* w2 */ + uint32_t esn_low; + uint32_t esn_hi; + + /* w3-w6 */ + uint8_t cipher_key[32]; + + /* w7-w12 */ + uint8_t hmac_key[48]; + + RTE_STD_C11 + union { + void *userdata; + uint64_t udata64; + }; + + uint64_t reserved1; + uint64_t reserved2; +}; + +#endif /* __OTX2_IPSEC_FP_H__ */ diff --git a/drivers/crypto/octeontx2/otx2_security.c b/drivers/crypto/octeontx2/otx2_security.c index 531c78b..cdb7950 100644 --- a/drivers/crypto/octeontx2/otx2_security.c +++ b/drivers/crypto/octeontx2/otx2_security.c @@ -3,11 +3,36 @@ */ #include +#include #include +#include #include +#include "otx2_ethdev.h" +#include "otx2_ipsec_fp.h" #include "otx2_security.h" +#define SEC_ETH_MAX_PKT_LEN 1450 + +struct sec_eth_tag_const { + RTE_STD_C11 + union { + struct { + uint32_t rsvd_11_0 : 12; + uint32_t port : 8; + uint32_t event_type : 4; + uint32_t rsvd_31_24 : 8; + }; + uint32_t u32; + }; +}; + +static inline void +in_sa_mz_name_get(char *name, int size, uint16_t port) +{ + snprintf(name, size, "otx2_ipsec_in_sadb_%u", port); +} + int otx2_sec_eth_ctx_create(struct rte_eth_dev *eth_dev) { @@ -33,3 +58,100 @@ otx2_sec_eth_ctx_destroy(struct rte_eth_dev *eth_dev) { rte_free(eth_dev->security_ctx); } + +static int +sec_eth_ipsec_cfg(struct rte_eth_dev *eth_dev, uint8_t tt) +{ + struct otx2_eth_dev *dev = otx2_eth_pmd_priv(eth_dev); + uint16_t port = eth_dev->data->port_id; + struct nix_inline_ipsec_lf_cfg *req; + struct otx2_mbox *mbox = dev->mbox; + struct sec_eth_tag_const tag_const; + char name[RTE_MEMZONE_NAMESIZE]; + const struct rte_memzone *mz; + + in_sa_mz_name_get(name, RTE_MEMZONE_NAMESIZE, port); + mz = rte_memzone_lookup(name); + if (mz == NULL) + return -EINVAL; + + req = otx2_mbox_alloc_msg_nix_inline_ipsec_lf_cfg(mbox); + req->enable = 1; + req->sa_base_addr = mz->iova; + + req->ipsec_cfg0.tt = tt; + + tag_const.u32 = 0; + tag_const.event_type = RTE_EVENT_TYPE_ETHDEV; + tag_const.port = port; + req->ipsec_cfg0.tag_const = tag_const.u32; + + req->ipsec_cfg0.sa_pow2_size = + rte_log2_u32(sizeof(struct otx2_ipsec_fp_in_sa)); + req->ipsec_cfg0.lenm1_max = SEC_ETH_MAX_PKT_LEN - 1; + + req->ipsec_cfg1.sa_idx_w = rte_log2_u32(dev->ipsec_in_max_spi); + req->ipsec_cfg1.sa_idx_max = dev->ipsec_in_max_spi - 1; + + return otx2_mbox_process(mbox); +} + +int +otx2_sec_eth_init(struct rte_eth_dev *eth_dev) +{ + const size_t sa_width = sizeof(struct otx2_ipsec_fp_in_sa); + struct otx2_eth_dev *dev = otx2_eth_pmd_priv(eth_dev); + uint16_t port = eth_dev->data->port_id; + char name[RTE_MEMZONE_NAMESIZE]; + const struct rte_memzone *mz; + int mz_sz, ret; + uint16_t nb_sa; + + RTE_BUILD_BUG_ON(sa_width < 32 || sa_width > 512 || + !RTE_IS_POWER_OF_2(sa_width)); + + if (!(dev->tx_offloads & DEV_TX_OFFLOAD_SECURITY) && + !(dev->rx_offloads & DEV_RX_OFFLOAD_SECURITY)) + return 0; + + nb_sa = dev->ipsec_in_max_spi; + mz_sz = nb_sa * sa_width; + in_sa_mz_name_get(name, RTE_MEMZONE_NAMESIZE, port); + mz = rte_memzone_reserve_aligned(name, mz_sz, rte_socket_id(), + RTE_MEMZONE_IOVA_CONTIG, OTX2_ALIGN); + + if (mz == NULL) { + otx2_err("Could not allocate inbound SA DB"); + return -ENOMEM; + } + + memset(mz->addr, 0, mz_sz); + + ret = sec_eth_ipsec_cfg(eth_dev, SSO_TT_ORDERED); + if (ret < 0) { + otx2_err("Could not configure inline IPsec"); + goto sec_fini; + } + + return 0; + +sec_fini: + otx2_err("Could not configure device for security"); + otx2_sec_eth_fini(eth_dev); + return ret; +} + +void +otx2_sec_eth_fini(struct rte_eth_dev *eth_dev) +{ + struct otx2_eth_dev *dev = otx2_eth_pmd_priv(eth_dev); + uint16_t port = eth_dev->data->port_id; + char name[RTE_MEMZONE_NAMESIZE]; + + if (!(dev->tx_offloads & DEV_TX_OFFLOAD_SECURITY) && + !(dev->rx_offloads & DEV_RX_OFFLOAD_SECURITY)) + return; + + in_sa_mz_name_get(name, RTE_MEMZONE_NAMESIZE, port); + rte_memzone_free(rte_memzone_lookup(name)); +} diff --git a/drivers/crypto/octeontx2/otx2_security.h b/drivers/crypto/octeontx2/otx2_security.h index 21b7da4..023061d 100644 --- a/drivers/crypto/octeontx2/otx2_security.h +++ b/drivers/crypto/octeontx2/otx2_security.h @@ -11,4 +11,8 @@ int otx2_sec_eth_ctx_create(struct rte_eth_dev *eth_dev); void otx2_sec_eth_ctx_destroy(struct rte_eth_dev *eth_dev); +int otx2_sec_eth_init(struct rte_eth_dev *eth_dev); + +void otx2_sec_eth_fini(struct rte_eth_dev *eth_dev); + #endif /* __OTX2_SECURITY_H__ */ diff --git a/drivers/net/octeontx2/otx2_ethdev.c b/drivers/net/octeontx2/otx2_ethdev.c index 3e19ac2..131e883 100644 --- a/drivers/net/octeontx2/otx2_ethdev.c +++ b/drivers/net/octeontx2/otx2_ethdev.c @@ -337,6 +337,10 @@ nix_cq_rq_init(struct rte_eth_dev *eth_dev, struct otx2_eth_dev *dev, aq->op = NIX_AQ_INSTOP_INIT; aq->rq.sso_ena = 0; + + if (rxq->offloads & DEV_RX_OFFLOAD_SECURITY) + aq->rq.ipsech_ena = 1; + aq->rq.cq = qid; /* RQ to CQ 1:1 mapped */ aq->rq.spb_ena = 0; aq->rq.lpb_aura = npa_lf_aura_handle_to_aura(mp->pool_id); @@ -1610,6 +1614,8 @@ otx2_nix_configure(struct rte_eth_dev *eth_dev) /* Free the resources allocated from the previous configure */ if (dev->configured == 1) { + if (otx2_sec_idev_ops.fini != NULL) + otx2_sec_idev_ops.fini(eth_dev); otx2_nix_rxchan_bpid_cfg(eth_dev, false); otx2_nix_vlan_fini(eth_dev); otx2_nix_mc_addr_list_uninstall(eth_dev); @@ -1714,10 +1720,17 @@ otx2_nix_configure(struct rte_eth_dev *eth_dev) goto cq_fini; } + /* Enable security */ + if (otx2_sec_idev_ops.init != NULL) { + rc = otx2_sec_idev_ops.init(eth_dev); + if (rc) + goto cq_fini; + } + rc = otx2_nix_mc_addr_list_install(eth_dev); if (rc < 0) { otx2_err("Failed to install mc address list rc=%d", rc); - goto cq_fini; + goto sec_fini; } /* @@ -1753,6 +1766,9 @@ otx2_nix_configure(struct rte_eth_dev *eth_dev) uninstall_mc_list: otx2_nix_mc_addr_list_uninstall(eth_dev); +sec_fini: + if (otx2_sec_idev_ops.fini != NULL) + otx2_sec_idev_ops.fini(eth_dev); cq_fini: oxt2_nix_unregister_cq_irqs(eth_dev); q_irq_fini: @@ -2345,6 +2361,10 @@ otx2_eth_dev_uninit(struct rte_eth_dev *eth_dev, bool mbox_close) if (rc) otx2_err("Failed to cleanup npa lf, rc=%d", rc); + /* Disable security */ + if (otx2_sec_idev_ops.fini != NULL) + otx2_sec_idev_ops.fini(eth_dev); + /* Destroy security ctx */ if (otx2_sec_idev_ops.ctx_destroy != NULL) otx2_sec_idev_ops.ctx_destroy(eth_dev); diff --git a/drivers/net/octeontx2/otx2_ethdev.h b/drivers/net/octeontx2/otx2_ethdev.h index 987e760..41fef6e 100644 --- a/drivers/net/octeontx2/otx2_ethdev.h +++ b/drivers/net/octeontx2/otx2_ethdev.h @@ -324,6 +324,8 @@ struct otx2_eth_dev { bool mc_tbl_set; struct otx2_nix_mc_filter_tbl mc_fltr_tbl; bool sdp_link; /* SDP flag */ + /* Inline IPsec params */ + uint16_t ipsec_in_max_spi; } __rte_cache_aligned; struct otx2_eth_txq { diff --git a/drivers/net/octeontx2/otx2_ethdev_devargs.c b/drivers/net/octeontx2/otx2_ethdev_devargs.c index 04da1ab..a3f7598 100644 --- a/drivers/net/octeontx2/otx2_ethdev_devargs.c +++ b/drivers/net/octeontx2/otx2_ethdev_devargs.c @@ -64,6 +64,19 @@ parse_reta_size(const char *key, const char *value, void *extra_args) } static int +parse_ipsec_in_max_spi(const char *key, const char *value, void *extra_args) +{ + RTE_SET_USED(key); + uint32_t val; + + val = atoi(value); + + *(uint16_t *)extra_args = val; + + return 0; +} + +static int parse_flag(const char *key, const char *value, void *extra_args) { RTE_SET_USED(key); @@ -104,6 +117,7 @@ parse_switch_header_type(const char *key, const char *value, void *extra_args) } #define OTX2_RSS_RETA_SIZE "reta_size" +#define OTX2_IPSEC_IN_MAX_SPI "ipsec_in_max_spi" #define OTX2_SCL_ENABLE "scalar_enable" #define OTX2_MAX_SQB_COUNT "max_sqb_count" #define OTX2_FLOW_PREALLOC_SIZE "flow_prealloc_size" @@ -118,6 +132,7 @@ otx2_ethdev_parse_devargs(struct rte_devargs *devargs, struct otx2_eth_dev *dev) uint16_t flow_prealloc_size = 8; uint16_t switch_header_type = 0; uint16_t flow_max_priority = 3; + uint16_t ipsec_in_max_spi = 1; uint16_t scalar_enable = 0; struct rte_kvargs *kvlist; @@ -130,6 +145,8 @@ otx2_ethdev_parse_devargs(struct rte_devargs *devargs, struct otx2_eth_dev *dev) rte_kvargs_process(kvlist, OTX2_RSS_RETA_SIZE, &parse_reta_size, &rss_size); + rte_kvargs_process(kvlist, OTX2_IPSEC_IN_MAX_SPI, + &parse_ipsec_in_max_spi, &ipsec_in_max_spi); rte_kvargs_process(kvlist, OTX2_SCL_ENABLE, &parse_flag, &scalar_enable); rte_kvargs_process(kvlist, OTX2_MAX_SQB_COUNT, @@ -143,6 +160,7 @@ otx2_ethdev_parse_devargs(struct rte_devargs *devargs, struct otx2_eth_dev *dev) rte_kvargs_free(kvlist); null_devargs: + dev->ipsec_in_max_spi = ipsec_in_max_spi; dev->scalar_ena = scalar_enable; dev->max_sqb_count = sqb_count; dev->rss_info.rss_size = rss_size; @@ -157,6 +175,7 @@ otx2_ethdev_parse_devargs(struct rte_devargs *devargs, struct otx2_eth_dev *dev) RTE_PMD_REGISTER_PARAM_STRING(net_octeontx2, OTX2_RSS_RETA_SIZE "=<64|128|256>" + OTX2_IPSEC_IN_MAX_SPI "=<1-65535>" OTX2_SCL_ENABLE "=1" OTX2_MAX_SQB_COUNT "=<8-512>" OTX2_FLOW_PREALLOC_SIZE "=<1-32>" From patchwork Sat Jan 18 10:49:04 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Anoob Joseph X-Patchwork-Id: 64862 X-Patchwork-Delegate: gakhil@marvell.com Return-Path: X-Original-To: patchwork@inbox.dpdk.org Delivered-To: patchwork@inbox.dpdk.org Received: from dpdk.org (dpdk.org [92.243.14.124]) by inbox.dpdk.org (Postfix) with ESMTP id DE617A051C; Sat, 18 Jan 2020 11:50:27 +0100 (CET) Received: from [92.243.14.124] (localhost [127.0.0.1]) by dpdk.org (Postfix) with ESMTP id 7413F374E; Sat, 18 Jan 2020 11:50:20 +0100 (CET) Received: from mx0b-0016f401.pphosted.com (mx0b-0016f401.pphosted.com [67.231.156.173]) by dpdk.org (Postfix) with ESMTP id 82BB02C5E for ; Sat, 18 Jan 2020 11:50:18 +0100 (CET) Received: from pps.filterd (m0045851.ppops.net [127.0.0.1]) by mx0b-0016f401.pphosted.com (8.16.0.42/8.16.0.42) with SMTP id 00IAnc9j021837; Sat, 18 Jan 2020 02:50:18 -0800 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=marvell.com; h=from : to : cc : subject : date : message-id : in-reply-to : references : mime-version : content-transfer-encoding : content-type; s=pfpt0818; bh=DPCZttpCZeVXVCReKZ0C/KaZbz2yfEzU0do0wyh0Srk=; b=Df2WCZ55cbJ955K/YD3mQJ3lBIOYDGUB9Q0DY9digKaYfhu/yFEOHJXv+LRpJQ/ZCZtO eF/J/4nqIfSwmHjzaMySKjms2SUazQUaAzjagqBHoqBe6mLkhMy3qqUQ8euDR0QtoB3k EDCGUubj6R5Q1MMiRweqq/Xk6BcB2wT98EiDdJILhgeY15QayX0LhnehoTJC3frOpGKg Cz1oTLlTJaIEg0AHs5pk4E407CG/s8kZnrtxUzUG9deF7m1rk0y9io+++nmrE9oOfhPo z/q+yPNbJZfFspRQ56BFAbLSwVkVcokhgOyn+Kif6hdMAp5ndKI2JdxQNHRA1n6GOTEB Gw== Received: from sc-exch02.marvell.com ([199.233.58.182]) by mx0b-0016f401.pphosted.com with ESMTP id 2xk0sn6eg3-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-SHA384 bits=256 verify=NOT); Sat, 18 Jan 2020 02:50:17 -0800 Received: from SC-EXCH01.marvell.com (10.93.176.81) by SC-EXCH02.marvell.com (10.93.176.82) with Microsoft SMTP Server (TLS) id 15.0.1497.2; Sat, 18 Jan 2020 02:50:15 -0800 Received: from maili.marvell.com (10.93.176.43) by SC-EXCH01.marvell.com (10.93.176.81) with Microsoft SMTP Server id 15.0.1497.2 via Frontend Transport; Sat, 18 Jan 2020 02:50:15 -0800 Received: from ajoseph83.caveonetworks.com (unknown [10.29.45.60]) by maili.marvell.com (Postfix) with ESMTP id 5A5B53F703F; Sat, 18 Jan 2020 02:50:11 -0800 (PST) From: Anoob Joseph To: Akhil Goyal , Declan Doherty , Thomas Monjalon CC: Ankur Dwivedi , Jerin Jacob , Narayana Prasad , Kiran Kumar K , Nithin Dabilpuram , "Pavan Nikhilesh" , Anoob Joseph , Archana Muniganti , Tejasree Kondoj , Vamsi Attunuru , "Lukasz Bartosik" , Date: Sat, 18 Jan 2020 16:19:04 +0530 Message-ID: <1579344553-11428-7-git-send-email-anoobj@marvell.com> X-Mailer: git-send-email 2.7.4 In-Reply-To: <1579344553-11428-1-git-send-email-anoobj@marvell.com> References: <1575806094-28391-1-git-send-email-anoobj@marvell.com> <1579344553-11428-1-git-send-email-anoobj@marvell.com> MIME-Version: 1.0 X-Proofpoint-Virus-Version: vendor=fsecure engine=2.50.10434:6.0.138, 18.0.572 definitions=2020-01-18_02:2020-01-16, 2020-01-18 signatures=0 Subject: [dpdk-dev] [PATCH v2 06/15] crypto/octeontx2: add eth security capabilities X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Sender: "dev" From: Ankur Dwivedi Adding security capabilities supported by the eth PMD. Signed-off-by: Ankur Dwivedi Signed-off-by: Anoob Joseph Signed-off-by: Archana Muniganti Signed-off-by: Tejasree Kondoj Signed-off-by: Vamsi Attunuru --- drivers/crypto/octeontx2/otx2_security.c | 84 ++++++++++++++++++++++++++++++++ drivers/crypto/octeontx2/otx2_security.h | 18 +++++++ 2 files changed, 102 insertions(+) diff --git a/drivers/crypto/octeontx2/otx2_security.c b/drivers/crypto/octeontx2/otx2_security.c index cdb7950..b8c8f91 100644 --- a/drivers/crypto/octeontx2/otx2_security.c +++ b/drivers/crypto/octeontx2/otx2_security.c @@ -2,11 +2,13 @@ * Copyright (C) 2020 Marvell International Ltd. */ +#include #include #include #include #include #include +#include #include "otx2_ethdev.h" #include "otx2_ipsec_fp.h" @@ -27,12 +29,93 @@ struct sec_eth_tag_const { }; }; +static struct rte_cryptodev_capabilities otx2_sec_eth_crypto_caps[] = { + { /* AES GCM */ + .op = RTE_CRYPTO_OP_TYPE_SYMMETRIC, + {.sym = { + .xform_type = RTE_CRYPTO_SYM_XFORM_AEAD, + {.aead = { + .algo = RTE_CRYPTO_AEAD_AES_GCM, + .block_size = 16, + .key_size = { + .min = 16, + .max = 32, + .increment = 8 + }, + .digest_size = { + .min = 16, + .max = 16, + .increment = 0 + }, + .aad_size = { + .min = 8, + .max = 12, + .increment = 4 + }, + .iv_size = { + .min = 12, + .max = 12, + .increment = 0 + } + }, } + }, } + }, + RTE_CRYPTODEV_END_OF_CAPABILITIES_LIST() +}; + +static const struct rte_security_capability otx2_sec_eth_capabilities[] = { + { /* IPsec Inline Protocol ESP Tunnel Ingress */ + .action = RTE_SECURITY_ACTION_TYPE_INLINE_PROTOCOL, + .protocol = RTE_SECURITY_PROTOCOL_IPSEC, + .ipsec = { + .proto = RTE_SECURITY_IPSEC_SA_PROTO_ESP, + .mode = RTE_SECURITY_IPSEC_SA_MODE_TUNNEL, + .direction = RTE_SECURITY_IPSEC_SA_DIR_INGRESS, + .options = { 0 } + }, + .crypto_capabilities = otx2_sec_eth_crypto_caps, + .ol_flags = RTE_SECURITY_TX_OLOAD_NEED_MDATA + }, + { /* IPsec Inline Protocol ESP Tunnel Egress */ + .action = RTE_SECURITY_ACTION_TYPE_INLINE_PROTOCOL, + .protocol = RTE_SECURITY_PROTOCOL_IPSEC, + .ipsec = { + .proto = RTE_SECURITY_IPSEC_SA_PROTO_ESP, + .mode = RTE_SECURITY_IPSEC_SA_MODE_TUNNEL, + .direction = RTE_SECURITY_IPSEC_SA_DIR_EGRESS, + .options = { 0 } + }, + .crypto_capabilities = otx2_sec_eth_crypto_caps, + .ol_flags = RTE_SECURITY_TX_OLOAD_NEED_MDATA + }, + { + .action = RTE_SECURITY_ACTION_TYPE_NONE + } +}; + static inline void in_sa_mz_name_get(char *name, int size, uint16_t port) { snprintf(name, size, "otx2_ipsec_in_sadb_%u", port); } +static unsigned int +otx2_sec_eth_session_get_size(void *device __rte_unused) +{ + return sizeof(struct otx2_sec_session); +} + +static const struct rte_security_capability * +otx2_sec_eth_capabilities_get(void *device __rte_unused) +{ + return otx2_sec_eth_capabilities; +} + +static struct rte_security_ops otx2_sec_eth_ops = { + .session_get_size = otx2_sec_eth_session_get_size, + .capabilities_get = otx2_sec_eth_capabilities_get +}; + int otx2_sec_eth_ctx_create(struct rte_eth_dev *eth_dev) { @@ -46,6 +129,7 @@ otx2_sec_eth_ctx_create(struct rte_eth_dev *eth_dev) /* Populate ctx */ ctx->device = eth_dev; + ctx->ops = &otx2_sec_eth_ops; ctx->sess_cnt = 0; eth_dev->security_ctx = ctx; diff --git a/drivers/crypto/octeontx2/otx2_security.h b/drivers/crypto/octeontx2/otx2_security.h index 023061d..a442f5c 100644 --- a/drivers/crypto/octeontx2/otx2_security.h +++ b/drivers/crypto/octeontx2/otx2_security.h @@ -7,6 +7,24 @@ #include +#include "otx2_ipsec_fp.h" + +/* + * Security session for inline IPsec protocol offload. This is private data of + * inline capable PMD. + */ +struct otx2_sec_session_ipsec_ip { + int dummy; +}; + +struct otx2_sec_session_ipsec { + struct otx2_sec_session_ipsec_ip ip; +}; + +struct otx2_sec_session { + struct otx2_sec_session_ipsec ipsec; +} __rte_cache_aligned; + int otx2_sec_eth_ctx_create(struct rte_eth_dev *eth_dev); void otx2_sec_eth_ctx_destroy(struct rte_eth_dev *eth_dev); From patchwork Sat Jan 18 10:49:05 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Anoob Joseph X-Patchwork-Id: 64863 X-Patchwork-Delegate: gakhil@marvell.com Return-Path: X-Original-To: patchwork@inbox.dpdk.org Delivered-To: patchwork@inbox.dpdk.org Received: from dpdk.org (dpdk.org [92.243.14.124]) by inbox.dpdk.org (Postfix) with ESMTP id 41A88A051C; Sat, 18 Jan 2020 11:50:36 +0100 (CET) Received: from [92.243.14.124] (localhost [127.0.0.1]) by dpdk.org (Postfix) with ESMTP id BE8F02C5E; Sat, 18 Jan 2020 11:50:26 +0100 (CET) Received: from mx0b-0016f401.pphosted.com (mx0b-0016f401.pphosted.com [67.231.156.173]) by dpdk.org (Postfix) with ESMTP id 265F12C5E for ; Sat, 18 Jan 2020 11:50:25 +0100 (CET) Received: from pps.filterd (m0045851.ppops.net [127.0.0.1]) by mx0b-0016f401.pphosted.com (8.16.0.42/8.16.0.42) with SMTP id 00IAkutt017647; Sat, 18 Jan 2020 02:50:24 -0800 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=marvell.com; h=from : to : cc : subject : date : message-id : in-reply-to : references : mime-version : content-transfer-encoding : content-type; s=pfpt0818; bh=t5C4uavmUSFIi0SyCoMXuQi5S82GRkglpVh0MoDhjAs=; b=AzcmB+gBhXtFdrr9hZIX3zQbUrOZD7Z5UA4cAIcrzPNqrAgqmt20mehIBLXAEqsbBNKl LEgutTRN89n5sEhImZnxn1qEtOJG+Pg5NkJL1Fn7iYugwBmBhnuKhdlgis0Mxg2ixijt 6hhorljrlBo0Zi4U6uQFKmOHImgdT3AbU8EaqCqQH3N9be4zSFYCItOPC4QeNPzAuFIE mL1EkqO9cCyiTlwnle91CXvvIpAfDOoZxAphH9ZGDdchuKJj7hjM0fQu26HMTxThOnk6 udi9QzZgNEHvVWWFTlZUHUs8jTQosBAeZ8Nr+oubdWH1i5jmSpo1I96KyTSJjYgpa6QU cg== Received: from sc-exch01.marvell.com ([199.233.58.181]) by mx0b-0016f401.pphosted.com with ESMTP id 2xk0sn6egk-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-SHA384 bits=256 verify=NOT); Sat, 18 Jan 2020 02:50:24 -0800 Received: from SC-EXCH03.marvell.com (10.93.176.83) by SC-EXCH01.marvell.com (10.93.176.81) with Microsoft SMTP Server (TLS) id 15.0.1497.2; Sat, 18 Jan 2020 02:50:22 -0800 Received: from maili.marvell.com (10.93.176.43) by SC-EXCH03.marvell.com (10.93.176.83) with Microsoft SMTP Server id 15.0.1497.2 via Frontend Transport; Sat, 18 Jan 2020 02:50:22 -0800 Received: from ajoseph83.caveonetworks.com (unknown [10.29.45.60]) by maili.marvell.com (Postfix) with ESMTP id DC7253F7048; Sat, 18 Jan 2020 02:50:17 -0800 (PST) From: Anoob Joseph To: Akhil Goyal , Declan Doherty , Thomas Monjalon CC: Anoob Joseph , Jerin Jacob , Narayana Prasad , Kiran Kumar K , Nithin Dabilpuram , "Pavan Nikhilesh" , Ankur Dwivedi , Archana Muniganti , Tejasree Kondoj , Vamsi Attunuru , "Lukasz Bartosik" , Date: Sat, 18 Jan 2020 16:19:05 +0530 Message-ID: <1579344553-11428-8-git-send-email-anoobj@marvell.com> X-Mailer: git-send-email 2.7.4 In-Reply-To: <1579344553-11428-1-git-send-email-anoobj@marvell.com> References: <1575806094-28391-1-git-send-email-anoobj@marvell.com> <1579344553-11428-1-git-send-email-anoobj@marvell.com> MIME-Version: 1.0 X-Proofpoint-Virus-Version: vendor=fsecure engine=2.50.10434:6.0.138, 18.0.572 definitions=2020-01-18_02:2020-01-16, 2020-01-18 signatures=0 Subject: [dpdk-dev] [PATCH v2 07/15] crypto/octeontx2: enable CPT to share QP with ethdev X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Sender: "dev" Adding the infrastructure to save one opaque pointer in idev and implement the consumer-producer in the PMDs which uses it accordingly. Signed-off-by: Ankur Dwivedi Signed-off-by: Anoob Joseph Signed-off-by: Archana Muniganti Signed-off-by: Tejasree Kondoj Signed-off-by: Vamsi Attunuru --- .../crypto/octeontx2/otx2_cryptodev_hw_access.h | 22 +---- drivers/crypto/octeontx2/otx2_cryptodev_ops.c | 18 ++++ drivers/crypto/octeontx2/otx2_cryptodev_qp.h | 35 ++++++++ drivers/crypto/octeontx2/otx2_security.c | 98 ++++++++++++++++++++++ drivers/crypto/octeontx2/otx2_security.h | 20 +++++ 5 files changed, 172 insertions(+), 21 deletions(-) create mode 100644 drivers/crypto/octeontx2/otx2_cryptodev_qp.h diff --git a/drivers/crypto/octeontx2/otx2_cryptodev_hw_access.h b/drivers/crypto/octeontx2/otx2_cryptodev_hw_access.h index 6f78aa4..43db6a6 100644 --- a/drivers/crypto/octeontx2/otx2_cryptodev_hw_access.h +++ b/drivers/crypto/octeontx2/otx2_cryptodev_hw_access.h @@ -15,6 +15,7 @@ #include "cpt_mcode_defines.h" #include "otx2_dev.h" +#include "otx2_cryptodev_qp.h" /* CPT instruction queue length */ #define OTX2_CPT_IQ_LEN 8200 @@ -135,27 +136,6 @@ enum cpt_9x_comp_e { CPT_9X_COMP_E_LAST_ENTRY = 0x06 }; -struct otx2_cpt_qp { - uint32_t id; - /**< Queue pair id */ - uintptr_t base; - /**< Base address where BAR is mapped */ - void *lmtline; - /**< Address of LMTLINE */ - rte_iova_t lf_nq_reg; - /**< LF enqueue register address */ - struct pending_queue pend_q; - /**< Pending queue */ - struct rte_mempool *sess_mp; - /**< Session mempool */ - struct rte_mempool *sess_mp_priv; - /**< Session private data mempool */ - struct cpt_qp_meta_info meta_info; - /**< Metabuf info required to support operations on the queue pair */ - rte_iova_t iq_dma_addr; - /**< Instruction queue address */ -}; - void otx2_cpt_err_intr_unregister(const struct rte_cryptodev *dev); int otx2_cpt_err_intr_register(const struct rte_cryptodev *dev); diff --git a/drivers/crypto/octeontx2/otx2_cryptodev_ops.c b/drivers/crypto/octeontx2/otx2_cryptodev_ops.c index b45cb82..d275478 100644 --- a/drivers/crypto/octeontx2/otx2_cryptodev_ops.c +++ b/drivers/crypto/octeontx2/otx2_cryptodev_ops.c @@ -13,6 +13,7 @@ #include "otx2_cryptodev_hw_access.h" #include "otx2_cryptodev_mbox.h" #include "otx2_cryptodev_ops.h" +#include "otx2_security.h" #include "otx2_mbox.h" #include "cpt_hw_types.h" @@ -148,6 +149,11 @@ otx2_cpt_qp_inline_cfg(const struct rte_cryptodev *dev, struct otx2_cpt_qp *qp) if (ret) return ret; + /* Publish inline Tx QP to eth dev security */ + ret = otx2_sec_tx_cpt_qp_add(port_id, qp); + if (ret) + return ret; + return 0; } @@ -242,6 +248,12 @@ otx2_cpt_qp_create(const struct rte_cryptodev *dev, uint16_t qp_id, qp->lf_nq_reg = qp->base + OTX2_CPT_LF_NQ(0); + ret = otx2_sec_tx_cpt_qp_remove(qp); + if (ret && (ret != -ENOENT)) { + CPT_LOG_ERR("Could not delete inline configuration"); + goto mempool_destroy; + } + otx2_cpt_iq_disable(qp); ret = otx2_cpt_qp_inline_cfg(dev, qp); @@ -275,6 +287,12 @@ otx2_cpt_qp_destroy(const struct rte_cryptodev *dev, struct otx2_cpt_qp *qp) char name[RTE_MEMZONE_NAMESIZE]; int ret; + ret = otx2_sec_tx_cpt_qp_remove(qp); + if (ret && (ret != -ENOENT)) { + CPT_LOG_ERR("Could not delete inline configuration"); + return ret; + } + otx2_cpt_iq_disable(qp); otx2_cpt_metabuf_mempool_destroy(qp); diff --git a/drivers/crypto/octeontx2/otx2_cryptodev_qp.h b/drivers/crypto/octeontx2/otx2_cryptodev_qp.h new file mode 100644 index 0000000..9d48da4 --- /dev/null +++ b/drivers/crypto/octeontx2/otx2_cryptodev_qp.h @@ -0,0 +1,35 @@ +/* SPDX-License-Identifier: BSD-3-Clause + * Copyright (C) 2020 Marvell International Ltd. + */ + +#ifndef _OTX2_CRYPTODEV_QP_H_ +#define _OTX2_CRYPTODEV_QP_H_ + +#include +#include +#include + +#include "cpt_common.h" + +struct otx2_cpt_qp { + uint32_t id; + /**< Queue pair id */ + uintptr_t base; + /**< Base address where BAR is mapped */ + void *lmtline; + /**< Address of LMTLINE */ + rte_iova_t lf_nq_reg; + /**< LF enqueue register address */ + struct pending_queue pend_q; + /**< Pending queue */ + struct rte_mempool *sess_mp; + /**< Session mempool */ + struct rte_mempool *sess_mp_priv; + /**< Session private data mempool */ + struct cpt_qp_meta_info meta_info; + /**< Metabuf info required to support operations on the queue pair */ + rte_iova_t iq_dma_addr; + /**< Instruction queue address */ +}; + +#endif /* _OTX2_CRYPTODEV_QP_H_ */ diff --git a/drivers/crypto/octeontx2/otx2_security.c b/drivers/crypto/octeontx2/otx2_security.c index b8c8f91..0534154 100644 --- a/drivers/crypto/octeontx2/otx2_security.c +++ b/drivers/crypto/octeontx2/otx2_security.c @@ -10,6 +10,7 @@ #include #include +#include "otx2_cryptodev_qp.h" #include "otx2_ethdev.h" #include "otx2_ipsec_fp.h" #include "otx2_security.h" @@ -29,6 +30,8 @@ struct sec_eth_tag_const { }; }; +static struct otx2_sec_eth_cfg sec_cfg[OTX2_MAX_INLINE_PORTS]; + static struct rte_cryptodev_capabilities otx2_sec_eth_crypto_caps[] = { { /* AES GCM */ .op = RTE_CRYPTO_OP_TYPE_SYMMETRIC, @@ -116,16 +119,41 @@ static struct rte_security_ops otx2_sec_eth_ops = { .capabilities_get = otx2_sec_eth_capabilities_get }; +static int +otx2_sec_eth_cfg_init(int port_id) +{ + struct otx2_sec_eth_cfg *cfg; + int i; + + cfg = &sec_cfg[port_id]; + cfg->tx_cpt_idx = 0; + rte_spinlock_init(&cfg->tx_cpt_lock); + + for (i = 0; i < OTX2_MAX_CPT_QP_PER_PORT; i++) { + cfg->tx_cpt[i].qp = NULL; + rte_atomic16_set(&cfg->tx_cpt[i].ref_cnt, 0); + } + + return 0; +} + int otx2_sec_eth_ctx_create(struct rte_eth_dev *eth_dev) { struct rte_security_ctx *ctx; + int ret; ctx = rte_malloc("otx2_sec_eth_ctx", sizeof(struct rte_security_ctx), 0); if (ctx == NULL) return -ENOMEM; + ret = otx2_sec_eth_cfg_init(eth_dev->data->port_id); + if (ret) { + rte_free(ctx); + return ret; + } + /* Populate ctx */ ctx->device = eth_dev; @@ -239,3 +267,73 @@ otx2_sec_eth_fini(struct rte_eth_dev *eth_dev) in_sa_mz_name_get(name, RTE_MEMZONE_NAMESIZE, port); rte_memzone_free(rte_memzone_lookup(name)); } + +int +otx2_sec_tx_cpt_qp_add(uint16_t port_id, struct otx2_cpt_qp *qp) +{ + struct otx2_sec_eth_cfg *cfg; + int i, ret; + + if (qp == NULL || port_id > OTX2_MAX_INLINE_PORTS) + return -EINVAL; + + cfg = &sec_cfg[port_id]; + + /* Find a free slot to save CPT LF */ + + rte_spinlock_lock(&cfg->tx_cpt_lock); + + for (i = 0; i < OTX2_MAX_CPT_QP_PER_PORT; i++) { + if (cfg->tx_cpt[i].qp == NULL) { + cfg->tx_cpt[i].qp = qp; + ret = 0; + goto unlock; + } + } + + ret = -EINVAL; + +unlock: + rte_spinlock_unlock(&cfg->tx_cpt_lock); + return ret; +} + +int +otx2_sec_tx_cpt_qp_remove(struct otx2_cpt_qp *qp) +{ + struct otx2_sec_eth_cfg *cfg; + uint16_t port_id; + int i, ret; + + if (qp == NULL) + return -EINVAL; + + for (port_id = 0; port_id < OTX2_MAX_INLINE_PORTS; port_id++) { + cfg = &sec_cfg[port_id]; + + rte_spinlock_lock(&cfg->tx_cpt_lock); + + for (i = 0; i < OTX2_MAX_CPT_QP_PER_PORT; i++) { + if (cfg->tx_cpt[i].qp != qp) + continue; + + /* Don't free if the QP is in use by any sec session */ + if (rte_atomic16_read(&cfg->tx_cpt[i].ref_cnt)) { + ret = -EBUSY; + } else { + cfg->tx_cpt[i].qp = NULL; + ret = 0; + } + + goto unlock; + } + + rte_spinlock_unlock(&cfg->tx_cpt_lock); + } + + return -ENOENT; + +unlock: + rte_spinlock_unlock(&cfg->tx_cpt_lock); + return ret; +} diff --git a/drivers/crypto/octeontx2/otx2_security.h b/drivers/crypto/octeontx2/otx2_security.h index a442f5c..6086efa 100644 --- a/drivers/crypto/octeontx2/otx2_security.h +++ b/drivers/crypto/octeontx2/otx2_security.h @@ -5,10 +5,27 @@ #ifndef __OTX2_SECURITY_H__ #define __OTX2_SECURITY_H__ +#include #include +#include #include "otx2_ipsec_fp.h" +#define OTX2_MAX_CPT_QP_PER_PORT 64 +#define OTX2_MAX_INLINE_PORTS 64 + +struct otx2_cpt_qp; + +struct otx2_sec_eth_cfg { + struct { + struct otx2_cpt_qp *qp; + rte_atomic16_t ref_cnt; + } tx_cpt[OTX2_MAX_CPT_QP_PER_PORT]; + + uint16_t tx_cpt_idx; + rte_spinlock_t tx_cpt_lock; +}; + /* * Security session for inline IPsec protocol offload. This is private data of * inline capable PMD. @@ -33,4 +50,7 @@ int otx2_sec_eth_init(struct rte_eth_dev *eth_dev); void otx2_sec_eth_fini(struct rte_eth_dev *eth_dev); +int otx2_sec_tx_cpt_qp_add(uint16_t port_id, struct otx2_cpt_qp *qp); + +int otx2_sec_tx_cpt_qp_remove(struct otx2_cpt_qp *qp); #endif /* __OTX2_SECURITY_H__ */ From patchwork Sat Jan 18 10:49:06 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Anoob Joseph X-Patchwork-Id: 64864 X-Patchwork-Delegate: gakhil@marvell.com Return-Path: X-Original-To: patchwork@inbox.dpdk.org Delivered-To: patchwork@inbox.dpdk.org Received: from dpdk.org (dpdk.org [92.243.14.124]) by inbox.dpdk.org (Postfix) with ESMTP id 822D2A051C; Sat, 18 Jan 2020 11:50:47 +0100 (CET) Received: from [92.243.14.124] (localhost [127.0.0.1]) by dpdk.org (Postfix) with ESMTP id E9DAF1BE94; Sat, 18 Jan 2020 11:50:32 +0100 (CET) Received: from mx0b-0016f401.pphosted.com (mx0a-0016f401.pphosted.com [67.231.148.174]) by dpdk.org (Postfix) with ESMTP id C580E29D6 for ; Sat, 18 Jan 2020 11:50:30 +0100 (CET) Received: from pps.filterd (m0045849.ppops.net [127.0.0.1]) by mx0a-0016f401.pphosted.com (8.16.0.42/8.16.0.42) with SMTP id 00IAkGqf009777; Sat, 18 Jan 2020 02:50:30 -0800 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=marvell.com; h=from : to : cc : subject : date : message-id : in-reply-to : references : mime-version : content-transfer-encoding : content-type; s=pfpt0818; bh=4ZFLS5Wj6GlAHWN72csrcekR7nD2sOrVyvAAahW0l9U=; b=cCkEp9MR1uePEZAEXli1ZbLa13ftEDkgYKLB+exBWnsMER2lOulEkcXXDGBD3R46wYgm IkfSmu7qtKqJxGRe4HLv1DcfKXxC+YBT4H/nWMypPahWHqNlAIlqP7lotMnvaAhcPB0r 84txJe/B3eRempIGkFsttHr9J759oLynM3Z7KrD3XL2ll5PdA3pxQ+GIm+0zGv79QFi+ /7LADeIEWnW/KjWpyRHHax87r5LAnsml0Ekgpan3F+EJ+bbrpsH6X1vMdHS4Zpu0zoEH +6U+CjuCbTEqxh16xssb+/4PuA6nDUM6e1gmP6jBnyiB2D9XIXB/jN4G+kyaE6Ebopev 5g== Received: from sc-exch04.marvell.com ([199.233.58.184]) by mx0a-0016f401.pphosted.com with ESMTP id 2xm08v01nw-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-SHA384 bits=256 verify=NOT); Sat, 18 Jan 2020 02:50:29 -0800 Received: from SC-EXCH03.marvell.com (10.93.176.83) by SC-EXCH04.marvell.com (10.93.176.84) with Microsoft SMTP Server (TLS) id 15.0.1497.2; Sat, 18 Jan 2020 02:50:28 -0800 Received: from maili.marvell.com (10.93.176.43) by SC-EXCH03.marvell.com (10.93.176.83) with Microsoft SMTP Server id 15.0.1497.2 via Frontend Transport; Sat, 18 Jan 2020 02:50:28 -0800 Received: from ajoseph83.caveonetworks.com (unknown [10.29.45.60]) by maili.marvell.com (Postfix) with ESMTP id 1530D3F703F; Sat, 18 Jan 2020 02:50:23 -0800 (PST) From: Anoob Joseph To: Akhil Goyal , Declan Doherty , Thomas Monjalon CC: Anoob Joseph , Jerin Jacob , Narayana Prasad , Kiran Kumar K , Nithin Dabilpuram , "Pavan Nikhilesh" , Ankur Dwivedi , Archana Muniganti , Tejasree Kondoj , Vamsi Attunuru , "Lukasz Bartosik" , Date: Sat, 18 Jan 2020 16:19:06 +0530 Message-ID: <1579344553-11428-9-git-send-email-anoobj@marvell.com> X-Mailer: git-send-email 2.7.4 In-Reply-To: <1579344553-11428-1-git-send-email-anoobj@marvell.com> References: <1575806094-28391-1-git-send-email-anoobj@marvell.com> <1579344553-11428-1-git-send-email-anoobj@marvell.com> MIME-Version: 1.0 X-Proofpoint-Virus-Version: vendor=fsecure engine=2.50.10434:6.0.138, 18.0.572 definitions=2020-01-18_02:2020-01-16, 2020-01-18 signatures=0 Subject: [dpdk-dev] [PATCH v2 08/15] crypto/octeontx2: add eth security session operations X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Sender: "dev" Adding security session operations in eth security ctx. Signed-off-by: Ankur Dwivedi Signed-off-by: Anoob Joseph Signed-off-by: Archana Muniganti Signed-off-by: Tejasree Kondoj Signed-off-by: Vamsi Attunuru --- drivers/crypto/octeontx2/otx2_ipsec_fp.h | 293 ++++++++++++++++++++++++++ drivers/crypto/octeontx2/otx2_security.c | 339 +++++++++++++++++++++++++++++++ drivers/crypto/octeontx2/otx2_security.h | 23 ++- 3 files changed, 654 insertions(+), 1 deletion(-) diff --git a/drivers/crypto/octeontx2/otx2_ipsec_fp.h b/drivers/crypto/octeontx2/otx2_ipsec_fp.h index bf4181a..c100dc5 100644 --- a/drivers/crypto/octeontx2/otx2_ipsec_fp.h +++ b/drivers/crypto/octeontx2/otx2_ipsec_fp.h @@ -5,6 +5,67 @@ #ifndef __OTX2_IPSEC_FP_H__ #define __OTX2_IPSEC_FP_H__ +#include +#include + +enum { + OTX2_IPSEC_FP_SA_DIRECTION_INBOUND = 0, + OTX2_IPSEC_FP_SA_DIRECTION_OUTBOUND = 1, +}; + +enum { + OTX2_IPSEC_FP_SA_IP_VERSION_4 = 0, + OTX2_IPSEC_FP_SA_IP_VERSION_6 = 1, +}; + +enum { + OTX2_IPSEC_FP_SA_MODE_TRANSPORT = 0, + OTX2_IPSEC_FP_SA_MODE_TUNNEL = 1, +}; + +enum { + OTX2_IPSEC_FP_SA_PROTOCOL_AH = 0, + OTX2_IPSEC_FP_SA_PROTOCOL_ESP = 1, +}; + +enum { + OTX2_IPSEC_FP_SA_AES_KEY_LEN_128 = 1, + OTX2_IPSEC_FP_SA_AES_KEY_LEN_192 = 2, + OTX2_IPSEC_FP_SA_AES_KEY_LEN_256 = 3, +}; + +enum { + OTX2_IPSEC_FP_SA_ENC_NULL = 0, + OTX2_IPSEC_FP_SA_ENC_DES_CBC = 1, + OTX2_IPSEC_FP_SA_ENC_3DES_CBC = 2, + OTX2_IPSEC_FP_SA_ENC_AES_CBC = 3, + OTX2_IPSEC_FP_SA_ENC_AES_CTR = 4, + OTX2_IPSEC_FP_SA_ENC_AES_GCM = 5, + OTX2_IPSEC_FP_SA_ENC_AES_CCM = 6, +}; + +enum { + OTX2_IPSEC_FP_SA_AUTH_NULL = 0, + OTX2_IPSEC_FP_SA_AUTH_MD5 = 1, + OTX2_IPSEC_FP_SA_AUTH_SHA1 = 2, + OTX2_IPSEC_FP_SA_AUTH_SHA2_224 = 3, + OTX2_IPSEC_FP_SA_AUTH_SHA2_256 = 4, + OTX2_IPSEC_FP_SA_AUTH_SHA2_384 = 5, + OTX2_IPSEC_FP_SA_AUTH_SHA2_512 = 6, + OTX2_IPSEC_FP_SA_AUTH_AES_GMAC = 7, + OTX2_IPSEC_FP_SA_AUTH_AES_XCBC_128 = 8, +}; + +enum { + OTX2_IPSEC_FP_SA_FRAG_POST = 0, + OTX2_IPSEC_FP_SA_FRAG_PRE = 1, +}; + +enum { + OTX2_IPSEC_FP_SA_ENCAP_NONE = 0, + OTX2_IPSEC_FP_SA_ENCAP_UDP = 1, +}; + struct otx2_ipsec_fp_sa_ctl { rte_be32_t spi : 32; uint64_t exp_proto_inter_frag : 8; @@ -24,6 +85,26 @@ struct otx2_ipsec_fp_sa_ctl { uint64_t aes_key_len : 2; }; +struct otx2_ipsec_fp_out_sa { + /* w0 */ + struct otx2_ipsec_fp_sa_ctl ctl; + + /* w1 */ + uint8_t nonce[4]; + uint16_t udp_src; + uint16_t udp_dst; + + /* w2 */ + uint32_t ip_src; + uint32_t ip_dst; + + /* w3-w6 */ + uint8_t cipher_key[32]; + + /* w7-w12 */ + uint8_t hmac_key[48]; +}; + struct otx2_ipsec_fp_in_sa { /* w0 */ struct otx2_ipsec_fp_sa_ctl ctl; @@ -52,4 +133,216 @@ struct otx2_ipsec_fp_in_sa { uint64_t reserved2; }; +static inline int +ipsec_fp_xform_cipher_verify(struct rte_crypto_sym_xform *xform) +{ + if (xform->cipher.algo == RTE_CRYPTO_CIPHER_AES_CBC) { + switch (xform->cipher.key.length) { + case 16: + case 24: + case 32: + break; + default: + return -ENOTSUP; + } + return 0; + } + + return -ENOTSUP; +} + +static inline int +ipsec_fp_xform_auth_verify(struct rte_crypto_sym_xform *xform) +{ + if (xform->auth.algo == RTE_CRYPTO_AUTH_SHA1_HMAC) { + if (xform->auth.key.length == 64) + return 0; + } + + return -ENOTSUP; +} + +static inline int +ipsec_fp_xform_aead_verify(struct rte_security_ipsec_xform *ipsec, + struct rte_crypto_sym_xform *xform) +{ + if (ipsec->direction == RTE_SECURITY_IPSEC_SA_DIR_EGRESS && + xform->aead.op != RTE_CRYPTO_AEAD_OP_ENCRYPT) + return -EINVAL; + + if (ipsec->direction == RTE_SECURITY_IPSEC_SA_DIR_INGRESS && + xform->aead.op != RTE_CRYPTO_AEAD_OP_DECRYPT) + return -EINVAL; + + if (xform->aead.algo == RTE_CRYPTO_AEAD_AES_GCM) { + switch (xform->aead.key.length) { + case 16: + case 24: + case 32: + break; + default: + return -EINVAL; + } + return 0; + } + + return -ENOTSUP; +} + +static inline int +ipsec_fp_xform_verify(struct rte_security_ipsec_xform *ipsec, + struct rte_crypto_sym_xform *xform) +{ + struct rte_crypto_sym_xform *auth_xform, *cipher_xform; + int ret; + + if (xform->type == RTE_CRYPTO_SYM_XFORM_AEAD) + return ipsec_fp_xform_aead_verify(ipsec, xform); + + if (xform->next == NULL) + return -EINVAL; + + if (ipsec->direction == RTE_SECURITY_IPSEC_SA_DIR_INGRESS) { + /* Ingress */ + if (xform->type != RTE_CRYPTO_SYM_XFORM_AUTH || + xform->next->type != RTE_CRYPTO_SYM_XFORM_CIPHER) + return -EINVAL; + auth_xform = xform; + cipher_xform = xform->next; + } else { + /* Egress */ + if (xform->type != RTE_CRYPTO_SYM_XFORM_CIPHER || + xform->next->type != RTE_CRYPTO_SYM_XFORM_AUTH) + return -EINVAL; + cipher_xform = xform; + auth_xform = xform->next; + } + + ret = ipsec_fp_xform_cipher_verify(cipher_xform); + if (ret) + return ret; + + ret = ipsec_fp_xform_auth_verify(auth_xform); + if (ret) + return ret; + + return 0; +} + +static inline int +ipsec_fp_sa_ctl_set(struct rte_security_ipsec_xform *ipsec, + struct rte_crypto_sym_xform *xform, + struct otx2_ipsec_fp_sa_ctl *ctl) +{ + struct rte_crypto_sym_xform *cipher_xform, *auth_xform; + int aes_key_len; + + if (ipsec->direction == RTE_SECURITY_IPSEC_SA_DIR_EGRESS) { + ctl->direction = OTX2_IPSEC_FP_SA_DIRECTION_OUTBOUND; + cipher_xform = xform; + auth_xform = xform->next; + } else if (ipsec->direction == RTE_SECURITY_IPSEC_SA_DIR_INGRESS) { + ctl->direction = OTX2_IPSEC_FP_SA_DIRECTION_INBOUND; + auth_xform = xform; + cipher_xform = xform->next; + } else { + return -EINVAL; + } + + if (ipsec->mode == RTE_SECURITY_IPSEC_SA_MODE_TUNNEL) { + if (ipsec->tunnel.type == RTE_SECURITY_IPSEC_TUNNEL_IPV4) + ctl->outer_ip_ver = OTX2_IPSEC_FP_SA_IP_VERSION_4; + else if (ipsec->tunnel.type == RTE_SECURITY_IPSEC_TUNNEL_IPV6) + ctl->outer_ip_ver = OTX2_IPSEC_FP_SA_IP_VERSION_6; + else + return -EINVAL; + } + + ctl->inner_ip_ver = OTX2_IPSEC_FP_SA_IP_VERSION_4; + + if (ipsec->mode == RTE_SECURITY_IPSEC_SA_MODE_TRANSPORT) + ctl->ipsec_mode = OTX2_IPSEC_FP_SA_MODE_TRANSPORT; + else if (ipsec->mode == RTE_SECURITY_IPSEC_SA_MODE_TUNNEL) + ctl->ipsec_mode = OTX2_IPSEC_FP_SA_MODE_TUNNEL; + else + return -EINVAL; + + if (ipsec->proto == RTE_SECURITY_IPSEC_SA_PROTO_AH) + ctl->ipsec_proto = OTX2_IPSEC_FP_SA_PROTOCOL_AH; + else if (ipsec->proto == RTE_SECURITY_IPSEC_SA_PROTO_ESP) + ctl->ipsec_proto = OTX2_IPSEC_FP_SA_PROTOCOL_ESP; + else + return -EINVAL; + + if (xform->type == RTE_CRYPTO_SYM_XFORM_AEAD) { + if (xform->aead.algo == RTE_CRYPTO_AEAD_AES_GCM) { + ctl->enc_type = OTX2_IPSEC_FP_SA_ENC_AES_GCM; + aes_key_len = xform->aead.key.length; + } else { + return -ENOTSUP; + } + } else if (cipher_xform->cipher.algo == RTE_CRYPTO_CIPHER_AES_CBC) { + ctl->enc_type = OTX2_IPSEC_FP_SA_ENC_AES_CCM; + aes_key_len = xform->cipher.key.length; + } else { + return -ENOTSUP; + } + + switch (aes_key_len) { + case 16: + ctl->aes_key_len = OTX2_IPSEC_FP_SA_AES_KEY_LEN_128; + break; + case 24: + ctl->aes_key_len = OTX2_IPSEC_FP_SA_AES_KEY_LEN_192; + break; + case 32: + ctl->aes_key_len = OTX2_IPSEC_FP_SA_AES_KEY_LEN_256; + break; + default: + return -EINVAL; + } + + if (xform->type != RTE_CRYPTO_SYM_XFORM_AEAD) { + switch (auth_xform->auth.algo) { + case RTE_CRYPTO_AUTH_NULL: + ctl->auth_type = OTX2_IPSEC_FP_SA_AUTH_NULL; + break; + case RTE_CRYPTO_AUTH_MD5_HMAC: + ctl->auth_type = OTX2_IPSEC_FP_SA_AUTH_MD5; + break; + case RTE_CRYPTO_AUTH_SHA1_HMAC: + ctl->auth_type = OTX2_IPSEC_FP_SA_AUTH_SHA1; + break; + case RTE_CRYPTO_AUTH_SHA224_HMAC: + ctl->auth_type = OTX2_IPSEC_FP_SA_AUTH_SHA2_224; + break; + case RTE_CRYPTO_AUTH_SHA256_HMAC: + ctl->auth_type = OTX2_IPSEC_FP_SA_AUTH_SHA2_256; + break; + case RTE_CRYPTO_AUTH_SHA384_HMAC: + ctl->auth_type = OTX2_IPSEC_FP_SA_AUTH_SHA2_384; + break; + case RTE_CRYPTO_AUTH_SHA512_HMAC: + ctl->auth_type = OTX2_IPSEC_FP_SA_AUTH_SHA2_512; + break; + case RTE_CRYPTO_AUTH_AES_GMAC: + ctl->auth_type = OTX2_IPSEC_FP_SA_AUTH_AES_GMAC; + break; + case RTE_CRYPTO_AUTH_AES_XCBC_MAC: + ctl->auth_type = OTX2_IPSEC_FP_SA_AUTH_AES_XCBC_128; + break; + default: + return -ENOTSUP; + } + } + + if (ipsec->options.esn == 1) + ctl->esn_en = 1; + + ctl->spi = rte_cpu_to_be_32(ipsec->spi); + ctl->valid = 1; + + return 0; +} + #endif /* __OTX2_IPSEC_FP_H__ */ diff --git a/drivers/crypto/octeontx2/otx2_security.c b/drivers/crypto/octeontx2/otx2_security.c index 0534154..b4f5c5c 100644 --- a/drivers/crypto/octeontx2/otx2_security.c +++ b/drivers/crypto/octeontx2/otx2_security.c @@ -96,12 +96,349 @@ static const struct rte_security_capability otx2_sec_eth_capabilities[] = { } }; +static int +otx2_sec_eth_tx_cpt_qp_get(uint16_t port_id, struct otx2_cpt_qp **qp) +{ + struct otx2_sec_eth_cfg *cfg; + uint16_t index; + int i, ret; + + if (port_id > OTX2_MAX_INLINE_PORTS || qp == NULL) + return -EINVAL; + + cfg = &sec_cfg[port_id]; + + rte_spinlock_lock(&cfg->tx_cpt_lock); + + index = cfg->tx_cpt_idx; + + /* Get the next index with valid data */ + for (i = 0; i < OTX2_MAX_CPT_QP_PER_PORT; i++) { + if (cfg->tx_cpt[index].qp != NULL) + break; + index = (index + 1) % OTX2_MAX_CPT_QP_PER_PORT; + } + + if (i >= OTX2_MAX_CPT_QP_PER_PORT) { + ret = -EINVAL; + goto unlock; + } + + *qp = cfg->tx_cpt[index].qp; + rte_atomic16_inc(&cfg->tx_cpt[index].ref_cnt); + + cfg->tx_cpt_idx = (index + 1) % OTX2_MAX_CPT_QP_PER_PORT; + + ret = 0; + +unlock: + rte_spinlock_unlock(&cfg->tx_cpt_lock); + return ret; +} + +static int +otx2_sec_eth_tx_cpt_put(struct otx2_cpt_qp *qp) +{ + struct otx2_sec_eth_cfg *cfg; + uint16_t port_id; + int i; + + if (qp == NULL) + return -EINVAL; + + for (port_id = 0; port_id < OTX2_MAX_INLINE_PORTS; port_id++) { + cfg = &sec_cfg[port_id]; + for (i = 0; i < OTX2_MAX_CPT_QP_PER_PORT; i++) { + if (cfg->tx_cpt[i].qp == qp) { + rte_atomic16_dec(&cfg->tx_cpt[i].ref_cnt); + return 0; + } + } + } + + return -EINVAL; +} + static inline void in_sa_mz_name_get(char *name, int size, uint16_t port) { snprintf(name, size, "otx2_ipsec_in_sadb_%u", port); } +static struct otx2_ipsec_fp_in_sa * +in_sa_get(uint16_t port, int sa_index) +{ + char name[RTE_MEMZONE_NAMESIZE]; + struct otx2_ipsec_fp_in_sa *sa; + const struct rte_memzone *mz; + + in_sa_mz_name_get(name, RTE_MEMZONE_NAMESIZE, port); + mz = rte_memzone_lookup(name); + if (mz == NULL) { + otx2_err("Could not get the memzone reserved for IN SA DB"); + return NULL; + } + + sa = mz->addr; + + return sa + sa_index; +} + +static int +sec_eth_ipsec_out_sess_create(struct rte_eth_dev *eth_dev, + struct rte_security_ipsec_xform *ipsec, + struct rte_crypto_sym_xform *crypto_xform, + struct rte_security_session *sec_sess) +{ + struct rte_crypto_sym_xform *auth_xform, *cipher_xform; + struct otx2_sec_session_ipsec_ip *sess; + uint16_t port = eth_dev->data->port_id; + int cipher_key_len, auth_key_len, ret; + const uint8_t *cipher_key, *auth_key; + struct otx2_ipsec_fp_sa_ctl *ctl; + struct otx2_ipsec_fp_out_sa *sa; + struct otx2_sec_session *priv; + struct otx2_cpt_qp *qp; + + priv = get_sec_session_private_data(sec_sess); + sess = &priv->ipsec.ip; + + sa = &sess->out_sa; + ctl = &sa->ctl; + if (ctl->valid) { + otx2_err("SA already registered"); + return -EINVAL; + } + + memset(sess, 0, sizeof(struct otx2_sec_session_ipsec_ip)); + + memcpy(sa->nonce, &ipsec->salt, 4); + + if (ipsec->options.udp_encap == 1) { + sa->udp_src = 4500; + sa->udp_dst = 4500; + } + + if (ipsec->mode == RTE_SECURITY_IPSEC_SA_MODE_TUNNEL) { + if (ipsec->tunnel.type == RTE_SECURITY_IPSEC_TUNNEL_IPV4) { + memcpy(&sa->ip_src, &ipsec->tunnel.ipv4.src_ip, + sizeof(struct in_addr)); + memcpy(&sa->ip_dst, &ipsec->tunnel.ipv4.dst_ip, + sizeof(struct in_addr)); + } else { + return -EINVAL; + } + } else { + return -EINVAL; + } + + cipher_xform = crypto_xform; + auth_xform = crypto_xform->next; + + cipher_key_len = 0; + auth_key_len = 0; + + if (crypto_xform->type == RTE_CRYPTO_SYM_XFORM_AEAD) { + cipher_key = crypto_xform->aead.key.data; + cipher_key_len = crypto_xform->aead.key.length; + } else { + cipher_key = cipher_xform->cipher.key.data; + cipher_key_len = cipher_xform->cipher.key.length; + auth_key = auth_xform->auth.key.data; + auth_key_len = auth_xform->auth.key.length; + } + + if (cipher_key_len != 0) + memcpy(sa->cipher_key, cipher_key, cipher_key_len); + else + return -EINVAL; + + /* Use OPAD & IPAD */ + RTE_SET_USED(auth_key); + RTE_SET_USED(auth_key_len); + + /* Get CPT QP to be used for this SA */ + ret = otx2_sec_eth_tx_cpt_qp_get(port, &qp); + if (ret) + return ret; + + sess->qp = qp; + + sess->cpt_lmtline = qp->lmtline; + sess->cpt_nq_reg = qp->lf_nq_reg; + + /* Populate control word */ + ret = ipsec_fp_sa_ctl_set(ipsec, crypto_xform, ctl); + if (ret) + goto cpt_put; + + return 0; +cpt_put: + otx2_sec_eth_tx_cpt_put(sess->qp); + return ret; +} + +static int +sec_eth_ipsec_in_sess_create(struct rte_eth_dev *eth_dev, + struct rte_security_ipsec_xform *ipsec, + struct rte_crypto_sym_xform *crypto_xform, + struct rte_security_session *sec_sess) +{ + struct rte_crypto_sym_xform *auth_xform, *cipher_xform; + struct otx2_eth_dev *dev = otx2_eth_pmd_priv(eth_dev); + struct otx2_sec_session_ipsec_ip *sess; + uint16_t port = eth_dev->data->port_id; + const uint8_t *cipher_key, *auth_key; + int cipher_key_len, auth_key_len; + struct otx2_ipsec_fp_sa_ctl *ctl; + struct otx2_ipsec_fp_in_sa *sa; + struct otx2_sec_session *priv; + + if (ipsec->spi >= dev->ipsec_in_max_spi) { + otx2_err("SPI exceeds max supported"); + return -EINVAL; + } + + sa = in_sa_get(port, ipsec->spi); + ctl = &sa->ctl; + + priv = get_sec_session_private_data(sec_sess); + sess = &priv->ipsec.ip; + + if (ctl->valid) { + otx2_err("SA already registered"); + return -EINVAL; + } + + memset(sa, 0, sizeof(struct otx2_ipsec_fp_in_sa)); + + auth_xform = crypto_xform; + cipher_xform = crypto_xform->next; + + cipher_key_len = 0; + auth_key_len = 0; + + if (crypto_xform->type == RTE_CRYPTO_SYM_XFORM_AEAD) { + if (crypto_xform->aead.algo == RTE_CRYPTO_AEAD_AES_GCM) + memcpy(sa->nonce, &ipsec->salt, 4); + cipher_key = crypto_xform->aead.key.data; + cipher_key_len = crypto_xform->aead.key.length; + } else { + cipher_key = cipher_xform->cipher.key.data; + cipher_key_len = cipher_xform->cipher.key.length; + auth_key = auth_xform->auth.key.data; + auth_key_len = auth_xform->auth.key.length; + } + + if (cipher_key_len != 0) + memcpy(sa->cipher_key, cipher_key, cipher_key_len); + else + return -EINVAL; + + /* Use OPAD & IPAD */ + RTE_SET_USED(auth_key); + RTE_SET_USED(auth_key_len); + + sess->in_sa = sa; + + sa->userdata = priv->userdata; + + return ipsec_fp_sa_ctl_set(ipsec, crypto_xform, ctl); +} + +static int +sec_eth_ipsec_sess_create(struct rte_eth_dev *eth_dev, + struct rte_security_ipsec_xform *ipsec, + struct rte_crypto_sym_xform *crypto_xform, + struct rte_security_session *sess) +{ + int ret; + + ret = ipsec_fp_xform_verify(ipsec, crypto_xform); + if (ret) + return ret; + + if (ipsec->direction == RTE_SECURITY_IPSEC_SA_DIR_INGRESS) + return sec_eth_ipsec_in_sess_create(eth_dev, ipsec, + crypto_xform, sess); + else + return sec_eth_ipsec_out_sess_create(eth_dev, ipsec, + crypto_xform, sess); +} + +static int +otx2_sec_eth_session_create(void *device, + struct rte_security_session_conf *conf, + struct rte_security_session *sess, + struct rte_mempool *mempool) +{ + struct otx2_sec_session *priv; + int ret; + + if (conf->action_type != RTE_SECURITY_ACTION_TYPE_INLINE_PROTOCOL) + return -ENOTSUP; + + if (rte_mempool_get(mempool, (void **)&priv)) { + otx2_err("Could not allocate security session private data"); + return -ENOMEM; + } + + set_sec_session_private_data(sess, priv); + + /* + * Save userdata provided by the application. For ingress packets, this + * could be used to identify the SA. + */ + priv->userdata = conf->userdata; + + if (conf->protocol == RTE_SECURITY_PROTOCOL_IPSEC) + ret = sec_eth_ipsec_sess_create(device, &conf->ipsec, + conf->crypto_xform, + sess); + else + ret = -ENOTSUP; + + if (ret) + goto mempool_put; + + return 0; + +mempool_put: + rte_mempool_put(mempool, priv); + set_sec_session_private_data(sess, NULL); + return ret; +} + +static int +otx2_sec_eth_session_destroy(void *device __rte_unused, + struct rte_security_session *sess) +{ + struct otx2_sec_session_ipsec_ip *sess_ip; + struct otx2_sec_session *priv; + struct rte_mempool *sess_mp; + int ret; + + priv = get_sec_session_private_data(sess); + if (priv == NULL) + return -EINVAL; + + sess_ip = &priv->ipsec.ip; + + /* Release CPT LF used for this session */ + if (sess_ip->qp != NULL) { + ret = otx2_sec_eth_tx_cpt_put(sess_ip->qp); + if (ret) + return ret; + } + + sess_mp = rte_mempool_from_obj(priv); + + set_sec_session_private_data(sess, NULL); + rte_mempool_put(sess_mp, priv); + + return 0; +} + static unsigned int otx2_sec_eth_session_get_size(void *device __rte_unused) { @@ -115,6 +452,8 @@ otx2_sec_eth_capabilities_get(void *device __rte_unused) } static struct rte_security_ops otx2_sec_eth_ops = { + .session_create = otx2_sec_eth_session_create, + .session_destroy = otx2_sec_eth_session_destroy, .session_get_size = otx2_sec_eth_session_get_size, .capabilities_get = otx2_sec_eth_capabilities_get }; diff --git a/drivers/crypto/octeontx2/otx2_security.h b/drivers/crypto/octeontx2/otx2_security.h index 6086efa..b1a401b 100644 --- a/drivers/crypto/octeontx2/otx2_security.h +++ b/drivers/crypto/octeontx2/otx2_security.h @@ -31,7 +31,26 @@ struct otx2_sec_eth_cfg { * inline capable PMD. */ struct otx2_sec_session_ipsec_ip { - int dummy; + RTE_STD_C11 + union { + /* + * Inbound SA would accessed by crypto block. And so the memory + * is allocated differently and shared with the h/w. Only + * holding a pointer to this memory in the session private + * space. + */ + void *in_sa; + /* Outbound SA */ + struct otx2_ipsec_fp_out_sa out_sa; + }; + + /* Address of CPT LMTLINE */ + void *cpt_lmtline; + /* CPT LF enqueue register address */ + rte_iova_t cpt_nq_reg; + + /* CPT QP used by SA */ + struct otx2_cpt_qp *qp; }; struct otx2_sec_session_ipsec { @@ -40,6 +59,8 @@ struct otx2_sec_session_ipsec { struct otx2_sec_session { struct otx2_sec_session_ipsec ipsec; + void *userdata; + /**< Userdata registered by the application */ } __rte_cache_aligned; int otx2_sec_eth_ctx_create(struct rte_eth_dev *eth_dev); From patchwork Sat Jan 18 10:49:07 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Anoob Joseph X-Patchwork-Id: 64865 X-Patchwork-Delegate: gakhil@marvell.com Return-Path: X-Original-To: patchwork@inbox.dpdk.org Delivered-To: patchwork@inbox.dpdk.org Received: from dpdk.org (dpdk.org [92.243.14.124]) by inbox.dpdk.org (Postfix) with ESMTP id 5F060A051C; Sat, 18 Jan 2020 11:50:56 +0100 (CET) Received: from [92.243.14.124] (localhost [127.0.0.1]) by dpdk.org (Postfix) with ESMTP id 6525B1BEA9; Sat, 18 Jan 2020 11:50:38 +0100 (CET) Received: from mx0b-0016f401.pphosted.com (mx0a-0016f401.pphosted.com [67.231.148.174]) by dpdk.org (Postfix) with ESMTP id 8D64A29D6 for ; Sat, 18 Jan 2020 11:50:36 +0100 (CET) Received: from pps.filterd (m0045849.ppops.net [127.0.0.1]) by mx0a-0016f401.pphosted.com (8.16.0.42/8.16.0.42) with SMTP id 00IAjQOc009563; Sat, 18 Jan 2020 02:50:35 -0800 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=marvell.com; h=from : to : cc : subject : date : message-id : in-reply-to : references : mime-version : content-transfer-encoding : content-type; s=pfpt0818; bh=yiG3K8oW8kj/u7ovZ0IKtrjzDk2mu6+uainyJyhM3es=; b=q5oKA32Ew+6SgH3S7/c4OHzj/y88wibXRwtwfn6oov/rLLWhlcqwjIUQq4iggk44jRf6 ymW70Jk12mH0jjjaxaNs3Ua1Cy7VXLxe3j7FnrvwwaCFjlYyYpuwMsdx80xhkxBrFnxe amESsqWdmYYe9rw1DHs8XsAY9Dw9aIZOtVZcFaxiiuO17bB5ELLw7fIs3cUmLUiib7ff 5X694AcU4o8mRv2CB/dfhM4a+daHvQqcaPyEkugDDZzODAQJ7taNDs8N57zPa/c2dAvN jp/URedr7Ki/1+KddXfK3eDN1hNyxdnVNKujMR043eJDPycWTFdusNAo0dSfYBT0TxM4 tw== Received: from sc-exch03.marvell.com ([199.233.58.183]) by mx0a-0016f401.pphosted.com with ESMTP id 2xm08v01pg-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-SHA384 bits=256 verify=NOT); Sat, 18 Jan 2020 02:50:35 -0800 Received: from SC-EXCH03.marvell.com (10.93.176.83) by SC-EXCH03.marvell.com (10.93.176.83) with Microsoft SMTP Server (TLS) id 15.0.1497.2; Sat, 18 Jan 2020 02:50:34 -0800 Received: from maili.marvell.com (10.93.176.43) by SC-EXCH03.marvell.com (10.93.176.83) with Microsoft SMTP Server id 15.0.1497.2 via Frontend Transport; Sat, 18 Jan 2020 02:50:34 -0800 Received: from ajoseph83.caveonetworks.com (unknown [10.29.45.60]) by maili.marvell.com (Postfix) with ESMTP id E16BA3F703F; Sat, 18 Jan 2020 02:50:29 -0800 (PST) From: Anoob Joseph To: Akhil Goyal , Declan Doherty , Thomas Monjalon CC: Ankur Dwivedi , Jerin Jacob , Narayana Prasad , Kiran Kumar K , Nithin Dabilpuram , "Pavan Nikhilesh" , Anoob Joseph , Archana Muniganti , Tejasree Kondoj , Vamsi Attunuru , "Lukasz Bartosik" , Date: Sat, 18 Jan 2020 16:19:07 +0530 Message-ID: <1579344553-11428-10-git-send-email-anoobj@marvell.com> X-Mailer: git-send-email 2.7.4 In-Reply-To: <1579344553-11428-1-git-send-email-anoobj@marvell.com> References: <1575806094-28391-1-git-send-email-anoobj@marvell.com> <1579344553-11428-1-git-send-email-anoobj@marvell.com> MIME-Version: 1.0 X-Proofpoint-Virus-Version: vendor=fsecure engine=2.50.10434:6.0.138, 18.0.572 definitions=2020-01-18_02:2020-01-16, 2020-01-18 signatures=0 Subject: [dpdk-dev] [PATCH v2 09/15] crypto/octeontx2: add datapath ops in eth security ctx X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Sender: "dev" From: Ankur Dwivedi Adding data path ops in eth security ctx. Signed-off-by: Ankur Dwivedi Signed-off-by: Anoob Joseph Signed-off-by: Archana Muniganti Signed-off-by: Tejasree Kondoj Signed-off-by: Vamsi Attunuru --- drivers/crypto/octeontx2/otx2_security.c | 23 +++++++++++++++++++++++ 1 file changed, 23 insertions(+) diff --git a/drivers/crypto/octeontx2/otx2_security.c b/drivers/crypto/octeontx2/otx2_security.c index b4f5c5c..5606851 100644 --- a/drivers/crypto/octeontx2/otx2_security.c +++ b/drivers/crypto/octeontx2/otx2_security.c @@ -445,6 +445,27 @@ otx2_sec_eth_session_get_size(void *device __rte_unused) return sizeof(struct otx2_sec_session); } +static int +otx2_sec_eth_set_pkt_mdata(void *device __rte_unused, + struct rte_security_session *session, + struct rte_mbuf *m, void *params __rte_unused) +{ + /* Set security session as the pkt metadata */ + m->udata64 = (uint64_t)session; + + return 0; +} + +static int +otx2_sec_eth_get_userdata(void *device __rte_unused, uint64_t md, + void **userdata) +{ + /* Retrieve userdata */ + *userdata = (void *)md; + + return 0; +} + static const struct rte_security_capability * otx2_sec_eth_capabilities_get(void *device __rte_unused) { @@ -455,6 +476,8 @@ static struct rte_security_ops otx2_sec_eth_ops = { .session_create = otx2_sec_eth_session_create, .session_destroy = otx2_sec_eth_session_destroy, .session_get_size = otx2_sec_eth_session_get_size, + .set_pkt_metadata = otx2_sec_eth_set_pkt_mdata, + .get_userdata = otx2_sec_eth_get_userdata, .capabilities_get = otx2_sec_eth_capabilities_get }; From patchwork Sat Jan 18 10:49:08 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Anoob Joseph X-Patchwork-Id: 64866 X-Patchwork-Delegate: gakhil@marvell.com Return-Path: X-Original-To: patchwork@inbox.dpdk.org Delivered-To: patchwork@inbox.dpdk.org Received: from dpdk.org (dpdk.org [92.243.14.124]) by inbox.dpdk.org (Postfix) with ESMTP id E9A4FA051C; Sat, 18 Jan 2020 11:51:05 +0100 (CET) Received: from [92.243.14.124] (localhost [127.0.0.1]) by dpdk.org (Postfix) with ESMTP id CAF79316B; Sat, 18 Jan 2020 11:50:44 +0100 (CET) Received: from mx0b-0016f401.pphosted.com (mx0b-0016f401.pphosted.com [67.231.156.173]) by dpdk.org (Postfix) with ESMTP id 34ADA2F4F for ; Sat, 18 Jan 2020 11:50:43 +0100 (CET) Received: from pps.filterd (m0045851.ppops.net [127.0.0.1]) by mx0b-0016f401.pphosted.com (8.16.0.42/8.16.0.42) with SMTP id 00IAmMHu018888; Sat, 18 Jan 2020 02:50:42 -0800 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=marvell.com; h=from : to : cc : subject : date : message-id : in-reply-to : references : mime-version : content-transfer-encoding : content-type; s=pfpt0818; bh=s3Lh/g7s4atdmse9hg57sTjv28rh9cUHCFSZUhT121w=; b=VXFLPUiilu0CZNzDxFBZiepCfISbPoKIOmUW4ycBg1D8mgl+ifPsvmRP+6H0RWGutRrQ 9EsmLMGdgz6AP4nk5HUol1UpN8kE/EqUzf4GdPyt1yl+1APkHorY133PbTSJiC8YP2HN HngZ5pAWN9H7Jn6bWdnX+VdTeb2BOsI+dU7RxOcHSItYXdQcrxWvkD/X/epfLTAzE+Gv j/QUYuZaJ+0fHX5ajYMmeNxBMeGa/JNUOJ8GrvQaXO8ia9GNzZ7fNvVqAN8DRkot7PGP Q/GFbvyqZBBPT3J1lBfOPIMema0h5M6mTh/HmoJa2BlIIgaNFaR93IUaZ5QsO50cMFVZ 0w== Received: from sc-exch03.marvell.com ([199.233.58.183]) by mx0b-0016f401.pphosted.com with ESMTP id 2xk0sn6ehg-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-SHA384 bits=256 verify=NOT); Sat, 18 Jan 2020 02:50:42 -0800 Received: from SC-EXCH03.marvell.com (10.93.176.83) by SC-EXCH03.marvell.com (10.93.176.83) with Microsoft SMTP Server (TLS) id 15.0.1497.2; Sat, 18 Jan 2020 02:50:40 -0800 Received: from maili.marvell.com (10.93.176.43) by SC-EXCH03.marvell.com (10.93.176.83) with Microsoft SMTP Server id 15.0.1497.2 via Frontend Transport; Sat, 18 Jan 2020 02:50:40 -0800 Received: from ajoseph83.caveonetworks.com (unknown [10.29.45.60]) by maili.marvell.com (Postfix) with ESMTP id E16773F703F; Sat, 18 Jan 2020 02:50:35 -0800 (PST) From: Anoob Joseph To: Akhil Goyal , Declan Doherty , Thomas Monjalon CC: Archana Muniganti , Jerin Jacob , Narayana Prasad , Kiran Kumar K , Nithin Dabilpuram , "Pavan Nikhilesh" , Ankur Dwivedi , Anoob Joseph , Tejasree Kondoj , Vamsi Attunuru , Lukasz Bartosik , Date: Sat, 18 Jan 2020 16:19:08 +0530 Message-ID: <1579344553-11428-11-git-send-email-anoobj@marvell.com> X-Mailer: git-send-email 2.7.4 In-Reply-To: <1579344553-11428-1-git-send-email-anoobj@marvell.com> References: <1575806094-28391-1-git-send-email-anoobj@marvell.com> <1579344553-11428-1-git-send-email-anoobj@marvell.com> MIME-Version: 1.0 X-Proofpoint-Virus-Version: vendor=fsecure engine=2.50.10434:6.0.138, 18.0.572 definitions=2020-01-18_02:2020-01-16, 2020-01-18 signatures=0 Subject: [dpdk-dev] [PATCH v2 10/15] crypto/octeontx2: add lookup mem changes to hold sa indices X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Sender: "dev" From: Archana Muniganti lookup_mem provides fast accessing of data path fields. Storing sa indices in lookup_mem which are required in inline rx data path. Signed-off-by: Ankur Dwivedi Signed-off-by: Anoob Joseph Signed-off-by: Archana Muniganti Signed-off-by: Tejasree Kondoj Signed-off-by: Vamsi Attunuru --- drivers/common/octeontx2/otx2_common.h | 4 +++ drivers/crypto/octeontx2/otx2_security.c | 59 ++++++++++++++++++++++++++++++++ drivers/net/octeontx2/otx2_lookup.c | 11 ++++-- 3 files changed, 72 insertions(+), 2 deletions(-) diff --git a/drivers/common/octeontx2/otx2_common.h b/drivers/common/octeontx2/otx2_common.h index fbe7335..88b4b63 100644 --- a/drivers/common/octeontx2/otx2_common.h +++ b/drivers/common/octeontx2/otx2_common.h @@ -170,4 +170,8 @@ extern int otx2_logtype_dpi; #include "otx2_io_generic.h" #endif +/* Fastpath lookup */ +#define OTX2_NIX_FASTPATH_LOOKUP_MEM "otx2_nix_fastpath_lookup_mem" +#define OTX2_NIX_SA_TBL_START (4096*4 + 69632*2) + #endif /* _OTX2_COMMON_H_ */ diff --git a/drivers/crypto/octeontx2/otx2_security.c b/drivers/crypto/octeontx2/otx2_security.c index 5606851..ab488a0 100644 --- a/drivers/crypto/octeontx2/otx2_security.c +++ b/drivers/crypto/octeontx2/otx2_security.c @@ -10,6 +10,7 @@ #include #include +#include "otx2_common.h" #include "otx2_cryptodev_qp.h" #include "otx2_ethdev.h" #include "otx2_ipsec_fp.h" @@ -96,6 +97,59 @@ static const struct rte_security_capability otx2_sec_eth_capabilities[] = { } }; +static void +lookup_mem_sa_tbl_clear(struct rte_eth_dev *eth_dev) +{ + static const char name[] = OTX2_NIX_FASTPATH_LOOKUP_MEM; + uint16_t port = eth_dev->data->port_id; + const struct rte_memzone *mz; + uint64_t **sa_tbl; + uint8_t *mem; + + mz = rte_memzone_lookup(name); + if (mz == NULL) + return; + + mem = mz->addr; + + sa_tbl = (uint64_t **)RTE_PTR_ADD(mem, OTX2_NIX_SA_TBL_START); + if (sa_tbl[port] == NULL) + return; + + rte_free(sa_tbl[port]); + sa_tbl[port] = NULL; +} + +static int +lookup_mem_sa_index_update(struct rte_eth_dev *eth_dev, int spi, void *sa) +{ + static const char name[] = OTX2_NIX_FASTPATH_LOOKUP_MEM; + struct otx2_eth_dev *dev = otx2_eth_pmd_priv(eth_dev); + uint16_t port = eth_dev->data->port_id; + const struct rte_memzone *mz; + uint64_t **sa_tbl; + uint8_t *mem; + + mz = rte_memzone_lookup(name); + if (mz == NULL) { + otx2_err("Could not find fastpath lookup table"); + return -EINVAL; + } + + mem = mz->addr; + + sa_tbl = (uint64_t **)RTE_PTR_ADD(mem, OTX2_NIX_SA_TBL_START); + + if (sa_tbl[port] == NULL) { + sa_tbl[port] = rte_malloc(NULL, dev->ipsec_in_max_spi * + sizeof(uint64_t), 0); + } + + sa_tbl[port][spi] = (uint64_t)sa; + + return 0; +} + static int otx2_sec_eth_tx_cpt_qp_get(uint16_t port_id, struct otx2_cpt_qp **qp) { @@ -343,6 +397,9 @@ sec_eth_ipsec_in_sess_create(struct rte_eth_dev *eth_dev, sa->userdata = priv->userdata; + if (lookup_mem_sa_index_update(eth_dev, ipsec->spi, sa)) + return -EINVAL; + return ipsec_fp_sa_ctl_set(ipsec, crypto_xform, ctl); } @@ -626,6 +683,8 @@ otx2_sec_eth_fini(struct rte_eth_dev *eth_dev) !(dev->rx_offloads & DEV_RX_OFFLOAD_SECURITY)) return; + lookup_mem_sa_tbl_clear(eth_dev); + in_sa_mz_name_get(name, RTE_MEMZONE_NAMESIZE, port); rte_memzone_free(rte_memzone_lookup(name)); } diff --git a/drivers/net/octeontx2/otx2_lookup.c b/drivers/net/octeontx2/otx2_lookup.c index bcf2ff4..d1cf3c3 100644 --- a/drivers/net/octeontx2/otx2_lookup.c +++ b/drivers/net/octeontx2/otx2_lookup.c @@ -5,6 +5,7 @@ #include #include +#include "otx2_common.h" #include "otx2_ethdev.h" /* NIX_RX_PARSE_S's ERRCODE + ERRLEV (12 bits) */ @@ -12,7 +13,9 @@ #define ERR_ARRAY_SZ ((BIT(ERRCODE_ERRLEN_WIDTH)) *\ sizeof(uint32_t)) -#define LOOKUP_ARRAY_SZ (PTYPE_ARRAY_SZ + ERR_ARRAY_SZ) +#define SA_TBL_SZ (RTE_MAX_ETHPORTS * sizeof(uint64_t)) +#define LOOKUP_ARRAY_SZ (PTYPE_ARRAY_SZ + ERR_ARRAY_SZ +\ + SA_TBL_SZ) const uint32_t * otx2_nix_supported_ptypes_get(struct rte_eth_dev *eth_dev) @@ -314,10 +317,14 @@ nix_create_rx_ol_flags_array(void *mem) void * otx2_nix_fastpath_lookup_mem_get(void) { - const char name[] = "otx2_nix_fastpath_lookup_mem"; + const char name[] = OTX2_NIX_FASTPATH_LOOKUP_MEM; const struct rte_memzone *mz; void *mem; + /* SA_TBL starts after PTYPE_ARRAY & ERR_ARRAY */ + RTE_BUILD_BUG_ON(OTX2_NIX_SA_TBL_START != (PTYPE_ARRAY_SZ + + ERR_ARRAY_SZ)); + mz = rte_memzone_lookup(name); if (mz != NULL) return mz->addr; From patchwork Sat Jan 18 10:49:09 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Anoob Joseph X-Patchwork-Id: 64867 X-Patchwork-Delegate: gakhil@marvell.com Return-Path: X-Original-To: patchwork@inbox.dpdk.org Delivered-To: patchwork@inbox.dpdk.org Received: from dpdk.org (dpdk.org [92.243.14.124]) by inbox.dpdk.org (Postfix) with ESMTP id F201BA051C; Sat, 18 Jan 2020 11:51:16 +0100 (CET) Received: from [92.243.14.124] (localhost [127.0.0.1]) by dpdk.org (Postfix) with ESMTP id 8167A44C7; Sat, 18 Jan 2020 11:50:50 +0100 (CET) Received: from mx0b-0016f401.pphosted.com (mx0a-0016f401.pphosted.com [67.231.148.174]) by dpdk.org (Postfix) with ESMTP id ACA5429D6 for ; Sat, 18 Jan 2020 11:50:48 +0100 (CET) Received: from pps.filterd (m0045849.ppops.net [127.0.0.1]) by mx0a-0016f401.pphosted.com (8.16.0.42/8.16.0.42) with SMTP id 00IAl1TT010466; Sat, 18 Jan 2020 02:50:48 -0800 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=marvell.com; h=from : to : cc : subject : date : message-id : in-reply-to : references : mime-version : content-transfer-encoding : content-type; s=pfpt0818; bh=AFZJSfPSqdCIPe2Om7GKMII+oiRb+0eLI90XnQAefQs=; b=zLTcN3lSUOS8cRFb6yYlaijvDix13Z4hUBPmwjqOry668GYd3iNPwT+FJuK5lKvotaBf fKptvF78qDI9C7h8hOsMYYf8ysLr9F9e8NkpjgfYTBCvVPZnkuMI195G6ouSwQf0RuqK aPyiGwqJYs2nA4pTWfo4OJTZYRiArnooopayeVojGcATH7S7IBY9g4hZwWT62MbnFW0h jajQj0BvvDNJInnBH18SMqhlL6IeHhI+toII6/EjFDpVLQT2aQUf8EcHTkO6B5Yvi64c +aXUYydKVwIL7lyb98+qhL/QZyDoGXF9sAuYlBLY0XMonWxa/9hM8gEEUrngfOEMIe/e NA== Received: from sc-exch03.marvell.com ([199.233.58.183]) by mx0a-0016f401.pphosted.com with ESMTP id 2xm08v01q0-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-SHA384 bits=256 verify=NOT); Sat, 18 Jan 2020 02:50:47 -0800 Received: from SC-EXCH01.marvell.com (10.93.176.81) by SC-EXCH03.marvell.com (10.93.176.83) with Microsoft SMTP Server (TLS) id 15.0.1497.2; Sat, 18 Jan 2020 02:50:46 -0800 Received: from maili.marvell.com (10.93.176.43) by SC-EXCH01.marvell.com (10.93.176.81) with Microsoft SMTP Server id 15.0.1497.2 via Frontend Transport; Sat, 18 Jan 2020 02:50:46 -0800 Received: from ajoseph83.caveonetworks.com (unknown [10.29.45.60]) by maili.marvell.com (Postfix) with ESMTP id D3EBD3F703F; Sat, 18 Jan 2020 02:50:41 -0800 (PST) From: Anoob Joseph To: Akhil Goyal , Declan Doherty , Thomas Monjalon CC: Tejasree Kondoj , Jerin Jacob , Narayana Prasad , Kiran Kumar K , Nithin Dabilpuram , "Pavan Nikhilesh" , Ankur Dwivedi , Anoob Joseph , Archana Muniganti , Vamsi Attunuru , Lukasz Bartosik , Date: Sat, 18 Jan 2020 16:19:09 +0530 Message-ID: <1579344553-11428-12-git-send-email-anoobj@marvell.com> X-Mailer: git-send-email 2.7.4 In-Reply-To: <1579344553-11428-1-git-send-email-anoobj@marvell.com> References: <1575806094-28391-1-git-send-email-anoobj@marvell.com> <1579344553-11428-1-git-send-email-anoobj@marvell.com> MIME-Version: 1.0 X-Proofpoint-Virus-Version: vendor=fsecure engine=2.50.10434:6.0.138, 18.0.572 definitions=2020-01-18_02:2020-01-16, 2020-01-18 signatures=0 Subject: [dpdk-dev] [PATCH v2 11/15] net/octeontx2: add inline ipsec rx path changes X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Sender: "dev" From: Tejasree Kondoj Adding post-processing required for inline IPsec inbound packets. Signed-off-by: Ankur Dwivedi Signed-off-by: Anoob Joseph Signed-off-by: Archana Muniganti Signed-off-by: Tejasree Kondoj Signed-off-by: Vamsi Attunuru --- drivers/crypto/octeontx2/Makefile | 1 + drivers/crypto/octeontx2/otx2_security.h | 19 +++++++++ drivers/event/octeontx2/Makefile | 1 + drivers/event/octeontx2/meson.build | 2 + drivers/net/octeontx2/Makefile | 1 + drivers/net/octeontx2/meson.build | 3 ++ drivers/net/octeontx2/otx2_rx.h | 73 ++++++++++++++++++++++++++++++++ 7 files changed, 100 insertions(+) diff --git a/drivers/crypto/octeontx2/Makefile b/drivers/crypto/octeontx2/Makefile index 5966ddc..62b630e 100644 --- a/drivers/crypto/octeontx2/Makefile +++ b/drivers/crypto/octeontx2/Makefile @@ -20,6 +20,7 @@ VPATH += $(RTE_SDK)/drivers/crypto/octeontx2 CFLAGS += -O3 CFLAGS += -I$(RTE_SDK)/drivers/common/cpt CFLAGS += -I$(RTE_SDK)/drivers/common/octeontx2 +CFLAGS += -I$(RTE_SDK)/drivers/crypto/octeontx2 CFLAGS += -I$(RTE_SDK)/drivers/mempool/octeontx2 CFLAGS += -I$(RTE_SDK)/drivers/net/octeontx2 CFLAGS += -DALLOW_EXPERIMENTAL_API diff --git a/drivers/crypto/octeontx2/otx2_security.h b/drivers/crypto/octeontx2/otx2_security.h index b1a401b..6ec321d 100644 --- a/drivers/crypto/octeontx2/otx2_security.h +++ b/drivers/crypto/octeontx2/otx2_security.h @@ -26,6 +26,25 @@ struct otx2_sec_eth_cfg { rte_spinlock_t tx_cpt_lock; }; +#define OTX2_SEC_CPT_COMP_GOOD 0x1 +#define OTX2_SEC_UC_COMP_GOOD 0x0 +#define OTX2_SEC_COMP_GOOD (OTX2_SEC_UC_COMP_GOOD << 8 | \ + OTX2_SEC_CPT_COMP_GOOD) + +/* CPT Result */ +struct otx2_cpt_res { + union { + struct { + uint64_t compcode:8; + uint64_t uc_compcode:8; + uint64_t doneint:1; + uint64_t reserved_17_63:47; + uint64_t reserved_64_127; + }; + uint16_t u16[8]; + }; +}; + /* * Security session for inline IPsec protocol offload. This is private data of * inline capable PMD. diff --git a/drivers/event/octeontx2/Makefile b/drivers/event/octeontx2/Makefile index 6dab69c..bcd22ee 100644 --- a/drivers/event/octeontx2/Makefile +++ b/drivers/event/octeontx2/Makefile @@ -11,6 +11,7 @@ LIB = librte_pmd_octeontx2_event.a CFLAGS += $(WERROR_FLAGS) CFLAGS += -I$(RTE_SDK)/drivers/common/octeontx2 +CFLAGS += -I$(RTE_SDK)/drivers/crypto/octeontx2 CFLAGS += -I$(RTE_SDK)/drivers/mempool/octeontx2 CFLAGS += -I$(RTE_SDK)/drivers/event/octeontx2 CFLAGS += -I$(RTE_SDK)/drivers/net/octeontx2 diff --git a/drivers/event/octeontx2/meson.build b/drivers/event/octeontx2/meson.build index 807818b..56febb8 100644 --- a/drivers/event/octeontx2/meson.build +++ b/drivers/event/octeontx2/meson.build @@ -32,3 +32,5 @@ foreach flag: extra_flags endforeach deps += ['bus_pci', 'common_octeontx2', 'mempool_octeontx2', 'pmd_octeontx2'] + +includes += include_directories('../../crypto/octeontx2') diff --git a/drivers/net/octeontx2/Makefile b/drivers/net/octeontx2/Makefile index 68f5765..d31ce0a 100644 --- a/drivers/net/octeontx2/Makefile +++ b/drivers/net/octeontx2/Makefile @@ -11,6 +11,7 @@ LIB = librte_pmd_octeontx2.a CFLAGS += $(WERROR_FLAGS) CFLAGS += -I$(RTE_SDK)/drivers/common/octeontx2 +CFLAGS += -I$(RTE_SDK)/drivers/crypto/octeontx2 CFLAGS += -I$(RTE_SDK)/drivers/mempool/octeontx2 CFLAGS += -I$(RTE_SDK)/drivers/net/octeontx2 CFLAGS += -O3 diff --git a/drivers/net/octeontx2/meson.build b/drivers/net/octeontx2/meson.build index fad3076..4a06eb2 100644 --- a/drivers/net/octeontx2/meson.build +++ b/drivers/net/octeontx2/meson.build @@ -25,6 +25,7 @@ sources = files('otx2_rx.c', ) deps += ['bus_pci', 'common_octeontx2', 'mempool_octeontx2'] +deps += ['cryptodev', 'security'] cflags += ['-flax-vector-conversions'] @@ -39,3 +40,5 @@ foreach flag: extra_flags cflags += flag endif endforeach + +includes += include_directories('../../crypto/octeontx2') diff --git a/drivers/net/octeontx2/otx2_rx.h b/drivers/net/octeontx2/otx2_rx.h index 351ad0f..e1715bd 100644 --- a/drivers/net/octeontx2/otx2_rx.h +++ b/drivers/net/octeontx2/otx2_rx.h @@ -5,6 +5,12 @@ #ifndef __OTX2_RX_H__ #define __OTX2_RX_H__ +#include + +#include "otx2_common.h" +#include "otx2_ipsec_fp.h" +#include "otx2_security.h" + /* Default mark value used when none is provided. */ #define OTX2_FLOW_ACTION_FLAG_DEFAULT 0xffff @@ -31,6 +37,12 @@ #define NIX_RX_MULTI_SEG_F BIT(15) #define NIX_TIMESYNC_RX_OFFSET 8 +/* Inline IPsec offsets */ + +#define INLINE_INB_RPTR_HDR 16 +/* nix_cqe_hdr_s + nix_rx_parse_s + nix_rx_sg_s + nix_iova_s */ +#define INLINE_CPT_RESULT_OFFSET 80 + struct otx2_timesync_info { uint64_t rx_tstamp; rte_iova_t tx_tstamp_iova; @@ -190,6 +202,60 @@ nix_cqe_xtract_mseg(const struct nix_rx_parse_s *rx, } } +static __rte_always_inline uint16_t +nix_rx_sec_cptres_get(const void *cq) +{ + volatile const struct otx2_cpt_res *res; + + res = (volatile const struct otx2_cpt_res *)((const char *)cq + + INLINE_CPT_RESULT_OFFSET); + + return res->u16[0]; +} + +static __rte_always_inline void * +nix_rx_sec_sa_get(const void * const lookup_mem, int spi, uint16_t port) +{ + const uint64_t *const *sa_tbl = (const uint64_t * const *) + ((const uint8_t *)lookup_mem + OTX2_NIX_SA_TBL_START); + + return (void *)sa_tbl[port][spi]; +} + +static __rte_always_inline uint64_t +nix_rx_sec_mbuf_update(const struct nix_cqe_hdr_s *cq, struct rte_mbuf *m, + const void * const lookup_mem) +{ + struct otx2_ipsec_fp_in_sa *sa; + struct rte_ipv4_hdr *ipv4; + uint16_t m_len; + uint32_t spi; + char *data; + + if (unlikely(nix_rx_sec_cptres_get(cq) != OTX2_SEC_COMP_GOOD)) + return PKT_RX_SEC_OFFLOAD | PKT_RX_SEC_OFFLOAD_FAILED; + + /* 20 bits of tag would have the SPI */ + spi = cq->tag & 0xFFFFF; + + sa = nix_rx_sec_sa_get(lookup_mem, spi, m->port); + m->udata64 = (uint64_t)sa->userdata; + + data = rte_pktmbuf_mtod(m, char *); + memcpy(data + INLINE_INB_RPTR_HDR, data, RTE_ETHER_HDR_LEN); + + m->data_off += INLINE_INB_RPTR_HDR; + + ipv4 = (struct rte_ipv4_hdr *)(data + INLINE_INB_RPTR_HDR + + RTE_ETHER_HDR_LEN); + + m_len = rte_be_to_cpu_16(ipv4->total_length) + RTE_ETHER_HDR_LEN; + + m->data_len = m_len; + m->pkt_len = m_len; + return PKT_RX_SEC_OFFLOAD; +} + static __rte_always_inline void otx2_nix_cqe_to_mbuf(const struct nix_cqe_hdr_s *cq, const uint32_t tag, struct rte_mbuf *mbuf, const void *lookup_mem, @@ -231,6 +297,13 @@ otx2_nix_cqe_to_mbuf(const struct nix_cqe_hdr_s *cq, const uint32_t tag, if (flag & NIX_RX_OFFLOAD_MARK_UPDATE_F) ol_flags = nix_update_match_id(rx->match_id, ol_flags, mbuf); + if (cq->cqe_type == NIX_XQE_TYPE_RX_IPSECH) { + *(uint64_t *)(&mbuf->rearm_data) = val; + ol_flags |= nix_rx_sec_mbuf_update(cq, mbuf, lookup_mem); + mbuf->ol_flags = ol_flags; + return; + } + mbuf->ol_flags = ol_flags; *(uint64_t *)(&mbuf->rearm_data) = val; mbuf->pkt_len = len; From patchwork Sat Jan 18 10:49:10 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Anoob Joseph X-Patchwork-Id: 64868 X-Patchwork-Delegate: gakhil@marvell.com Return-Path: X-Original-To: patchwork@inbox.dpdk.org Delivered-To: patchwork@inbox.dpdk.org Received: from dpdk.org (dpdk.org [92.243.14.124]) by inbox.dpdk.org (Postfix) with ESMTP id 424EBA051F; Sat, 18 Jan 2020 11:51:25 +0100 (CET) Received: from [92.243.14.124] (localhost [127.0.0.1]) by dpdk.org (Postfix) with ESMTP id ED18E1BC24; Sat, 18 Jan 2020 11:50:59 +0100 (CET) Received: from mx0b-0016f401.pphosted.com (mx0b-0016f401.pphosted.com [67.231.156.173]) by dpdk.org (Postfix) with ESMTP id D8DB91BF73 for ; Sat, 18 Jan 2020 11:50:57 +0100 (CET) Received: from pps.filterd (m0045851.ppops.net [127.0.0.1]) by mx0b-0016f401.pphosted.com (8.16.0.42/8.16.0.42) with SMTP id 00IAnFnN019528; Sat, 18 Jan 2020 02:50:57 -0800 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=marvell.com; h=from : to : cc : subject : date : message-id : in-reply-to : references : mime-version : content-transfer-encoding : content-type; s=pfpt0818; bh=VJar13rUhogKXxDt+8aKtYFffFRMOL7DxdvLQqLq76Q=; b=PC5GPcm/oIu8+HZ73etZUj5237xGyiC2r64pMxedKag69dRjZFrQf6zTIxyAxe7ZWWET VFzbdS4mptGFC6DtdqeIVRGqKqQNfb5Iq+A+L/hVl9h8zEuyXgsJlxIoOfiO4YYs4TPI /wG3QhCtMIgcr6AzMP1/58E2RiFY0kPdjc3KtbYwy+el/1AWcRhIF0eXyPnv6h/48PQp emJ0BevbCXYGMHkBCBRdMM9bXakmX+sRUfDluFyyiRew52r/O949uYuA3nbJsBRqf0MU l3qyrm6MsvSPVfFmEOuR3cgXxEgzTIyMlnwvGlL64MqaiR3zew4B2c7ur+3+OjTr857d hw== Received: from sc-exch03.marvell.com ([199.233.58.183]) by mx0b-0016f401.pphosted.com with ESMTP id 2xk0sn6ehw-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-SHA384 bits=256 verify=NOT); Sat, 18 Jan 2020 02:50:56 -0800 Received: from SC-EXCH01.marvell.com (10.93.176.81) by SC-EXCH03.marvell.com (10.93.176.83) with Microsoft SMTP Server (TLS) id 15.0.1497.2; Sat, 18 Jan 2020 02:50:54 -0800 Received: from maili.marvell.com (10.93.176.43) by SC-EXCH01.marvell.com (10.93.176.81) with Microsoft SMTP Server id 15.0.1497.2 via Frontend Transport; Sat, 18 Jan 2020 02:50:54 -0800 Received: from ajoseph83.caveonetworks.com (unknown [10.29.45.60]) by maili.marvell.com (Postfix) with ESMTP id D5FA03F7043; Sat, 18 Jan 2020 02:50:49 -0800 (PST) From: Anoob Joseph To: Akhil Goyal , Declan Doherty , Thomas Monjalon CC: Archana Muniganti , Jerin Jacob , Narayana Prasad , Kiran Kumar K , Nithin Dabilpuram , "Pavan Nikhilesh" , Ankur Dwivedi , Anoob Joseph , Tejasree Kondoj , Vamsi Attunuru , Lukasz Bartosik , Date: Sat, 18 Jan 2020 16:19:10 +0530 Message-ID: <1579344553-11428-13-git-send-email-anoobj@marvell.com> X-Mailer: git-send-email 2.7.4 In-Reply-To: <1579344553-11428-1-git-send-email-anoobj@marvell.com> References: <1575806094-28391-1-git-send-email-anoobj@marvell.com> <1579344553-11428-1-git-send-email-anoobj@marvell.com> MIME-Version: 1.0 X-Proofpoint-Virus-Version: vendor=fsecure engine=2.50.10434:6.0.138, 18.0.572 definitions=2020-01-18_02:2020-01-16, 2020-01-18 signatures=0 Subject: [dpdk-dev] [PATCH v2 12/15] drivers/octeontx2: add sec in compiler optimized RX fastpath framework X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Sender: "dev" From: Archana Muniganti Added new flag for SECURITY in RX compiler optimized fastpath framework. With this, compiler autogenerates functions which have security enabled. Signed-off-by: Ankur Dwivedi Signed-off-by: Anoob Joseph Signed-off-by: Archana Muniganti Signed-off-by: Tejasree Kondoj Signed-off-by: Vamsi Attunuru --- drivers/event/octeontx2/otx2_evdev.c | 134 ++++++++----- drivers/event/octeontx2/otx2_evdev.h | 2 +- drivers/event/octeontx2/otx2_worker.c | 2 +- drivers/event/octeontx2/otx2_worker_dual.c | 2 +- drivers/net/octeontx2/otx2_ethdev.c | 3 + drivers/net/octeontx2/otx2_rx.c | 27 +-- drivers/net/octeontx2/otx2_rx.h | 306 ++++++++++++++++++++--------- 7 files changed, 320 insertions(+), 156 deletions(-) diff --git a/drivers/event/octeontx2/otx2_evdev.c b/drivers/event/octeontx2/otx2_evdev.c index 2daeba4..f6c641a 100644 --- a/drivers/event/octeontx2/otx2_evdev.c +++ b/drivers/event/octeontx2/otx2_evdev.c @@ -44,61 +44,64 @@ sso_fastpath_fns_set(struct rte_eventdev *event_dev) { struct otx2_sso_evdev *dev = sso_pmd_priv(event_dev); /* Single WS modes */ - const event_dequeue_t ssogws_deq[2][2][2][2][2][2] = { -#define R(name, f5, f4, f3, f2, f1, f0, flags) \ - [f5][f4][f3][f2][f1][f0] = otx2_ssogws_deq_ ##name, + const event_dequeue_t ssogws_deq[2][2][2][2][2][2][2] = { +#define R(name, f6, f5, f4, f3, f2, f1, f0, flags) \ + [f6][f5][f4][f3][f2][f1][f0] = otx2_ssogws_deq_ ##name, SSO_RX_ADPTR_ENQ_FASTPATH_FUNC #undef R }; - const event_dequeue_burst_t ssogws_deq_burst[2][2][2][2][2][2] = { -#define R(name, f5, f4, f3, f2, f1, f0, flags) \ - [f5][f4][f3][f2][f1][f0] = otx2_ssogws_deq_burst_ ##name, + const event_dequeue_burst_t ssogws_deq_burst[2][2][2][2][2][2][2] = { +#define R(name, f6, f5, f4, f3, f2, f1, f0, flags) \ + [f6][f5][f4][f3][f2][f1][f0] = otx2_ssogws_deq_burst_ ##name, SSO_RX_ADPTR_ENQ_FASTPATH_FUNC #undef R }; - const event_dequeue_t ssogws_deq_timeout[2][2][2][2][2][2] = { -#define R(name, f5, f4, f3, f2, f1, f0, flags) \ - [f5][f4][f3][f2][f1][f0] = otx2_ssogws_deq_timeout_ ##name, + const event_dequeue_t ssogws_deq_timeout[2][2][2][2][2][2][2] = { +#define R(name, f6, f5, f4, f3, f2, f1, f0, flags) \ + [f6][f5][f4][f3][f2][f1][f0] = otx2_ssogws_deq_timeout_ ##name, SSO_RX_ADPTR_ENQ_FASTPATH_FUNC #undef R }; const event_dequeue_burst_t - ssogws_deq_timeout_burst[2][2][2][2][2][2] = { -#define R(name, f5, f4, f3, f2, f1, f0, flags) \ - [f5][f4][f3][f2][f1][f0] = \ + ssogws_deq_timeout_burst[2][2][2][2][2][2][2] = { +#define R(name, f6, f5, f4, f3, f2, f1, f0, flags) \ + [f6][f5][f4][f3][f2][f1][f0] = \ otx2_ssogws_deq_timeout_burst_ ##name, SSO_RX_ADPTR_ENQ_FASTPATH_FUNC #undef R }; - const event_dequeue_t ssogws_deq_seg[2][2][2][2][2][2] = { -#define R(name, f5, f4, f3, f2, f1, f0, flags) \ - [f5][f4][f3][f2][f1][f0] = otx2_ssogws_deq_seg_ ##name, + const event_dequeue_t ssogws_deq_seg[2][2][2][2][2][2][2] = { +#define R(name, f6, f5, f4, f3, f2, f1, f0, flags) \ + [f6][f5][f4][f3][f2][f1][f0] = otx2_ssogws_deq_seg_ ##name, SSO_RX_ADPTR_ENQ_FASTPATH_FUNC #undef R }; - const event_dequeue_burst_t ssogws_deq_seg_burst[2][2][2][2][2][2] = { -#define R(name, f5, f4, f3, f2, f1, f0, flags) \ - [f5][f4][f3][f2][f1][f0] = otx2_ssogws_deq_seg_burst_ ##name, + const event_dequeue_burst_t + ssogws_deq_seg_burst[2][2][2][2][2][2][2] = { +#define R(name, f6, f5, f4, f3, f2, f1, f0, flags) \ + [f6][f5][f4][f3][f2][f1][f0] = \ + otx2_ssogws_deq_seg_burst_ ##name, SSO_RX_ADPTR_ENQ_FASTPATH_FUNC #undef R }; - const event_dequeue_t ssogws_deq_seg_timeout[2][2][2][2][2][2] = { -#define R(name, f5, f4, f3, f2, f1, f0, flags) \ - [f5][f4][f3][f2][f1][f0] = otx2_ssogws_deq_seg_timeout_ ##name, + const event_dequeue_t ssogws_deq_seg_timeout[2][2][2][2][2][2][2] = { +#define R(name, f6, f5, f4, f3, f2, f1, f0, flags) \ + [f6][f5][f4][f3][f2][f1][f0] = \ + otx2_ssogws_deq_seg_timeout_ ##name, SSO_RX_ADPTR_ENQ_FASTPATH_FUNC #undef R }; const event_dequeue_burst_t - ssogws_deq_seg_timeout_burst[2][2][2][2][2][2] = { -#define R(name, f5, f4, f3, f2, f1, f0, flags) \ - [f5][f4][f3][f2][f1][f0] = \ + ssogws_deq_seg_timeout_burst[2][2][2][2][2][2][2] = { +#define R(name, f6, f5, f4, f3, f2, f1, f0, flags) \ + [f6][f5][f4][f3][f2][f1][f0] = \ otx2_ssogws_deq_seg_timeout_burst_ ##name, SSO_RX_ADPTR_ENQ_FASTPATH_FUNC #undef R @@ -106,64 +109,69 @@ SSO_RX_ADPTR_ENQ_FASTPATH_FUNC /* Dual WS modes */ - const event_dequeue_t ssogws_dual_deq[2][2][2][2][2][2] = { -#define R(name, f5, f4, f3, f2, f1, f0, flags) \ - [f5][f4][f3][f2][f1][f0] = otx2_ssogws_dual_deq_ ##name, + const event_dequeue_t ssogws_dual_deq[2][2][2][2][2][2][2] = { +#define R(name, f6, f5, f4, f3, f2, f1, f0, flags) \ + [f6][f5][f4][f3][f2][f1][f0] = otx2_ssogws_dual_deq_ ##name, SSO_RX_ADPTR_ENQ_FASTPATH_FUNC #undef R }; - const event_dequeue_burst_t ssogws_dual_deq_burst[2][2][2][2][2][2] = { -#define R(name, f5, f4, f3, f2, f1, f0, flags) \ - [f5][f4][f3][f2][f1][f0] = otx2_ssogws_dual_deq_burst_ ##name, + const event_dequeue_burst_t + ssogws_dual_deq_burst[2][2][2][2][2][2][2] = { +#define R(name, f6, f5, f4, f3, f2, f1, f0, flags) \ + [f6][f5][f4][f3][f2][f1][f0] = \ + otx2_ssogws_dual_deq_burst_ ##name, SSO_RX_ADPTR_ENQ_FASTPATH_FUNC #undef R }; - const event_dequeue_t ssogws_dual_deq_timeout[2][2][2][2][2][2] = { -#define R(name, f5, f4, f3, f2, f1, f0, flags) \ - [f5][f4][f3][f2][f1][f0] = otx2_ssogws_dual_deq_timeout_ ##name, + const event_dequeue_t ssogws_dual_deq_timeout[2][2][2][2][2][2][2] = { +#define R(name, f6, f5, f4, f3, f2, f1, f0, flags) \ + [f6][f5][f4][f3][f2][f1][f0] = \ + otx2_ssogws_dual_deq_timeout_ ##name, SSO_RX_ADPTR_ENQ_FASTPATH_FUNC #undef R }; const event_dequeue_burst_t - ssogws_dual_deq_timeout_burst[2][2][2][2][2][2] = { -#define R(name, f5, f4, f3, f2, f1, f0, flags) \ - [f5][f4][f3][f2][f1][f0] = otx2_ssogws_dual_deq_timeout_burst_ ##name, + ssogws_dual_deq_timeout_burst[2][2][2][2][2][2][2] = { +#define R(name, f6, f5, f4, f3, f2, f1, f0, flags) \ + [f6][f5][f4][f3][f2][f1][f0] = \ + otx2_ssogws_dual_deq_timeout_burst_ ##name, SSO_RX_ADPTR_ENQ_FASTPATH_FUNC #undef R }; - const event_dequeue_t ssogws_dual_deq_seg[2][2][2][2][2][2] = { -#define R(name, f5, f4, f3, f2, f1, f0, flags) \ - [f5][f4][f3][f2][f1][f0] = otx2_ssogws_dual_deq_seg_ ##name, + const event_dequeue_t ssogws_dual_deq_seg[2][2][2][2][2][2][2] = { +#define R(name, f6, f5, f4, f3, f2, f1, f0, flags) \ + [f6][f5][f4][f3][f2][f1][f0] = otx2_ssogws_dual_deq_seg_ ##name, SSO_RX_ADPTR_ENQ_FASTPATH_FUNC #undef R }; const event_dequeue_burst_t - ssogws_dual_deq_seg_burst[2][2][2][2][2][2] = { -#define R(name, f5, f4, f3, f2, f1, f0, flags) \ - [f5][f4][f3][f2][f1][f0] = \ - otx2_ssogws_dual_deq_seg_burst_ ##name, + ssogws_dual_deq_seg_burst[2][2][2][2][2][2][2] = { +#define R(name, f6, f5, f4, f3, f2, f1, f0, flags) \ + [f6][f5][f4][f3][f2][f1][f0] = \ + otx2_ssogws_dual_deq_seg_burst_ ##name, SSO_RX_ADPTR_ENQ_FASTPATH_FUNC #undef R }; - const event_dequeue_t ssogws_dual_deq_seg_timeout[2][2][2][2][2][2] = { -#define R(name, f5, f4, f3, f2, f1, f0, flags) \ - [f5][f4][f3][f2][f1][f0] = \ - otx2_ssogws_dual_deq_seg_timeout_ ##name, + const event_dequeue_t + ssogws_dual_deq_seg_timeout[2][2][2][2][2][2][2] = { +#define R(name, f6, f5, f4, f3, f2, f1, f0, flags) \ + [f6][f5][f4][f3][f2][f1][f0] = \ + otx2_ssogws_dual_deq_seg_timeout_ ##name, SSO_RX_ADPTR_ENQ_FASTPATH_FUNC #undef R }; const event_dequeue_burst_t - ssogws_dual_deq_seg_timeout_burst[2][2][2][2][2][2] = { -#define R(name, f5, f4, f3, f2, f1, f0, flags) \ - [f5][f4][f3][f2][f1][f0] = \ - otx2_ssogws_dual_deq_seg_timeout_burst_ ##name, + ssogws_dual_deq_seg_timeout_burst[2][2][2][2][2][2][2] = { +#define R(name, f6, f5, f4, f3, f2, f1, f0, flags) \ + [f6][f5][f4][f3][f2][f1][f0] = \ + otx2_ssogws_dual_deq_seg_timeout_burst_ ##name, SSO_RX_ADPTR_ENQ_FASTPATH_FUNC #undef R }; @@ -209,6 +217,7 @@ SSO_TX_ADPTR_ENQ_FASTPATH_FUNC event_dev->enqueue_forward_burst = otx2_ssogws_enq_fwd_burst; if (dev->rx_offloads & NIX_RX_MULTI_SEG_F) { event_dev->dequeue = ssogws_deq_seg + [!!(dev->rx_offloads & NIX_RX_OFFLOAD_SECURITY_F)] [!!(dev->rx_offloads & NIX_RX_OFFLOAD_TSTAMP_F)] [!!(dev->rx_offloads & NIX_RX_OFFLOAD_MARK_UPDATE_F)] [!!(dev->rx_offloads & NIX_RX_OFFLOAD_VLAN_STRIP_F)] @@ -216,6 +225,7 @@ SSO_TX_ADPTR_ENQ_FASTPATH_FUNC [!!(dev->rx_offloads & NIX_RX_OFFLOAD_PTYPE_F)] [!!(dev->rx_offloads & NIX_RX_OFFLOAD_RSS_F)]; event_dev->dequeue_burst = ssogws_deq_seg_burst + [!!(dev->rx_offloads & NIX_RX_OFFLOAD_SECURITY_F)] [!!(dev->rx_offloads & NIX_RX_OFFLOAD_TSTAMP_F)] [!!(dev->rx_offloads & NIX_RX_OFFLOAD_MARK_UPDATE_F)] [!!(dev->rx_offloads & NIX_RX_OFFLOAD_VLAN_STRIP_F)] @@ -224,6 +234,7 @@ SSO_TX_ADPTR_ENQ_FASTPATH_FUNC [!!(dev->rx_offloads & NIX_RX_OFFLOAD_RSS_F)]; if (dev->is_timeout_deq) { event_dev->dequeue = ssogws_deq_seg_timeout + [!!(dev->rx_offloads & NIX_RX_OFFLOAD_SECURITY_F)] [!!(dev->rx_offloads & NIX_RX_OFFLOAD_TSTAMP_F)] [!!(dev->rx_offloads & NIX_RX_OFFLOAD_MARK_UPDATE_F)] [!!(dev->rx_offloads & NIX_RX_OFFLOAD_VLAN_STRIP_F)] @@ -232,6 +243,7 @@ SSO_TX_ADPTR_ENQ_FASTPATH_FUNC [!!(dev->rx_offloads & NIX_RX_OFFLOAD_RSS_F)]; event_dev->dequeue_burst = ssogws_deq_seg_timeout_burst + [!!(dev->rx_offloads & NIX_RX_OFFLOAD_SECURITY_F)] [!!(dev->rx_offloads & NIX_RX_OFFLOAD_TSTAMP_F)] [!!(dev->rx_offloads & NIX_RX_OFFLOAD_MARK_UPDATE_F)] [!!(dev->rx_offloads & NIX_RX_OFFLOAD_VLAN_STRIP_F)] @@ -241,6 +253,7 @@ SSO_TX_ADPTR_ENQ_FASTPATH_FUNC } } else { event_dev->dequeue = ssogws_deq + [!!(dev->rx_offloads & NIX_RX_OFFLOAD_SECURITY_F)] [!!(dev->rx_offloads & NIX_RX_OFFLOAD_TSTAMP_F)] [!!(dev->rx_offloads & NIX_RX_OFFLOAD_MARK_UPDATE_F)] [!!(dev->rx_offloads & NIX_RX_OFFLOAD_VLAN_STRIP_F)] @@ -248,6 +261,7 @@ SSO_TX_ADPTR_ENQ_FASTPATH_FUNC [!!(dev->rx_offloads & NIX_RX_OFFLOAD_PTYPE_F)] [!!(dev->rx_offloads & NIX_RX_OFFLOAD_RSS_F)]; event_dev->dequeue_burst = ssogws_deq_burst + [!!(dev->rx_offloads & NIX_RX_OFFLOAD_SECURITY_F)] [!!(dev->rx_offloads & NIX_RX_OFFLOAD_TSTAMP_F)] [!!(dev->rx_offloads & NIX_RX_OFFLOAD_MARK_UPDATE_F)] [!!(dev->rx_offloads & NIX_RX_OFFLOAD_VLAN_STRIP_F)] @@ -256,6 +270,7 @@ SSO_TX_ADPTR_ENQ_FASTPATH_FUNC [!!(dev->rx_offloads & NIX_RX_OFFLOAD_RSS_F)]; if (dev->is_timeout_deq) { event_dev->dequeue = ssogws_deq_timeout + [!!(dev->rx_offloads & NIX_RX_OFFLOAD_SECURITY_F)] [!!(dev->rx_offloads & NIX_RX_OFFLOAD_TSTAMP_F)] [!!(dev->rx_offloads & NIX_RX_OFFLOAD_MARK_UPDATE_F)] [!!(dev->rx_offloads & NIX_RX_OFFLOAD_VLAN_STRIP_F)] @@ -264,6 +279,7 @@ SSO_TX_ADPTR_ENQ_FASTPATH_FUNC [!!(dev->rx_offloads & NIX_RX_OFFLOAD_RSS_F)]; event_dev->dequeue_burst = ssogws_deq_timeout_burst + [!!(dev->rx_offloads & NIX_RX_OFFLOAD_SECURITY_F)] [!!(dev->rx_offloads & NIX_RX_OFFLOAD_TSTAMP_F)] [!!(dev->rx_offloads & NIX_RX_OFFLOAD_MARK_UPDATE_F)] [!!(dev->rx_offloads & NIX_RX_OFFLOAD_VLAN_STRIP_F)] @@ -303,6 +319,8 @@ SSO_TX_ADPTR_ENQ_FASTPATH_FUNC if (dev->rx_offloads & NIX_RX_MULTI_SEG_F) { event_dev->dequeue = ssogws_dual_deq_seg [!!(dev->rx_offloads & + NIX_RX_OFFLOAD_SECURITY_F)] + [!!(dev->rx_offloads & NIX_RX_OFFLOAD_TSTAMP_F)] [!!(dev->rx_offloads & NIX_RX_OFFLOAD_MARK_UPDATE_F)] @@ -313,6 +331,8 @@ SSO_TX_ADPTR_ENQ_FASTPATH_FUNC [!!(dev->rx_offloads & NIX_RX_OFFLOAD_PTYPE_F)] [!!(dev->rx_offloads & NIX_RX_OFFLOAD_RSS_F)]; event_dev->dequeue_burst = ssogws_dual_deq_seg_burst + [!!(dev->rx_offloads & + NIX_RX_OFFLOAD_SECURITY_F)] [!!(dev->rx_offloads & NIX_RX_OFFLOAD_TSTAMP_F)] [!!(dev->rx_offloads & NIX_RX_OFFLOAD_MARK_UPDATE_F)] @@ -326,6 +346,8 @@ SSO_TX_ADPTR_ENQ_FASTPATH_FUNC event_dev->dequeue = ssogws_dual_deq_seg_timeout [!!(dev->rx_offloads & + NIX_RX_OFFLOAD_SECURITY_F)] + [!!(dev->rx_offloads & NIX_RX_OFFLOAD_TSTAMP_F)] [!!(dev->rx_offloads & NIX_RX_OFFLOAD_MARK_UPDATE_F)] @@ -340,6 +362,8 @@ SSO_TX_ADPTR_ENQ_FASTPATH_FUNC event_dev->dequeue_burst = ssogws_dual_deq_seg_timeout_burst [!!(dev->rx_offloads & + NIX_RX_OFFLOAD_SECURITY_F)] + [!!(dev->rx_offloads & NIX_RX_OFFLOAD_TSTAMP_F)] [!!(dev->rx_offloads & NIX_RX_OFFLOAD_MARK_UPDATE_F)] @@ -355,6 +379,8 @@ SSO_TX_ADPTR_ENQ_FASTPATH_FUNC } else { event_dev->dequeue = ssogws_dual_deq [!!(dev->rx_offloads & + NIX_RX_OFFLOAD_SECURITY_F)] + [!!(dev->rx_offloads & NIX_RX_OFFLOAD_TSTAMP_F)] [!!(dev->rx_offloads & NIX_RX_OFFLOAD_MARK_UPDATE_F)] @@ -366,6 +392,8 @@ SSO_TX_ADPTR_ENQ_FASTPATH_FUNC [!!(dev->rx_offloads & NIX_RX_OFFLOAD_RSS_F)]; event_dev->dequeue_burst = ssogws_dual_deq_burst [!!(dev->rx_offloads & + NIX_RX_OFFLOAD_SECURITY_F)] + [!!(dev->rx_offloads & NIX_RX_OFFLOAD_TSTAMP_F)] [!!(dev->rx_offloads & NIX_RX_OFFLOAD_MARK_UPDATE_F)] @@ -379,6 +407,8 @@ SSO_TX_ADPTR_ENQ_FASTPATH_FUNC event_dev->dequeue = ssogws_dual_deq_timeout [!!(dev->rx_offloads & + NIX_RX_OFFLOAD_SECURITY_F)] + [!!(dev->rx_offloads & NIX_RX_OFFLOAD_TSTAMP_F)] [!!(dev->rx_offloads & NIX_RX_OFFLOAD_MARK_UPDATE_F)] @@ -393,6 +423,8 @@ SSO_TX_ADPTR_ENQ_FASTPATH_FUNC event_dev->dequeue_burst = ssogws_dual_deq_timeout_burst [!!(dev->rx_offloads & + NIX_RX_OFFLOAD_SECURITY_F)] + [!!(dev->rx_offloads & NIX_RX_OFFLOAD_TSTAMP_F)] [!!(dev->rx_offloads & NIX_RX_OFFLOAD_MARK_UPDATE_F)] diff --git a/drivers/event/octeontx2/otx2_evdev.h b/drivers/event/octeontx2/otx2_evdev.h index ef523dc..5a44fd3 100644 --- a/drivers/event/octeontx2/otx2_evdev.h +++ b/drivers/event/octeontx2/otx2_evdev.h @@ -278,7 +278,7 @@ uint16_t otx2_ssogws_dual_enq_fwd_burst(void *port, const struct rte_event ev[], uint16_t nb_events); /* Auto generated API's */ -#define R(name, f5, f4, f3, f2, f1, f0, flags) \ +#define R(name, f6, f5, f4, f3, f2, f1, f0, flags) \ uint16_t otx2_ssogws_deq_ ##name(void *port, struct rte_event *ev, \ uint64_t timeout_ticks); \ uint16_t otx2_ssogws_deq_burst_ ##name(void *port, struct rte_event ev[], \ diff --git a/drivers/event/octeontx2/otx2_worker.c b/drivers/event/octeontx2/otx2_worker.c index 34d39f4..12445d9 100644 --- a/drivers/event/octeontx2/otx2_worker.c +++ b/drivers/event/octeontx2/otx2_worker.c @@ -81,7 +81,7 @@ otx2_ssogws_release_event(struct otx2_ssogws *ws) otx2_ssogws_swtag_flush(ws); } -#define R(name, f5, f4, f3, f2, f1, f0, flags) \ +#define R(name, f6, f5, f4, f3, f2, f1, f0, flags) \ uint16_t __hot \ otx2_ssogws_deq_ ##name(void *port, struct rte_event *ev, \ uint64_t timeout_ticks) \ diff --git a/drivers/event/octeontx2/otx2_worker_dual.c b/drivers/event/octeontx2/otx2_worker_dual.c index 7016eee..22a4889 100644 --- a/drivers/event/octeontx2/otx2_worker_dual.c +++ b/drivers/event/octeontx2/otx2_worker_dual.c @@ -140,7 +140,7 @@ otx2_ssogws_dual_enq_fwd_burst(void *port, const struct rte_event ev[], return 1; } -#define R(name, f5, f4, f3, f2, f1, f0, flags) \ +#define R(name, f6, f5, f4, f3, f2, f1, f0, flags) \ uint16_t __hot \ otx2_ssogws_dual_deq_ ##name(void *port, struct rte_event *ev, \ uint64_t timeout_ticks) \ diff --git a/drivers/net/octeontx2/otx2_ethdev.c b/drivers/net/octeontx2/otx2_ethdev.c index 131e883..c215078 100644 --- a/drivers/net/octeontx2/otx2_ethdev.c +++ b/drivers/net/octeontx2/otx2_ethdev.c @@ -626,6 +626,9 @@ nix_rx_offload_flags(struct rte_eth_dev *eth_dev) if ((dev->rx_offloads & DEV_RX_OFFLOAD_TIMESTAMP)) flags |= NIX_RX_OFFLOAD_TSTAMP_F; + if (dev->rx_offloads & DEV_RX_OFFLOAD_SECURITY) + flags |= NIX_RX_OFFLOAD_SECURITY_F; + if (!dev->ptype_disable) flags |= NIX_RX_OFFLOAD_PTYPE_F; diff --git a/drivers/net/octeontx2/otx2_rx.c b/drivers/net/octeontx2/otx2_rx.c index 48565db..ffe6996 100644 --- a/drivers/net/octeontx2/otx2_rx.c +++ b/drivers/net/octeontx2/otx2_rx.c @@ -320,7 +320,7 @@ nix_recv_pkts_vector(void *rx_queue, struct rte_mbuf **rx_pkts, #endif -#define R(name, f5, f4, f3, f2, f1, f0, flags) \ +#define R(name, f6, f5, f4, f3, f2, f1, f0, flags) \ static uint16_t __rte_noinline __hot \ otx2_nix_recv_pkts_ ## name(void *rx_queue, \ struct rte_mbuf **rx_pkts, uint16_t pkts) \ @@ -351,12 +351,13 @@ NIX_RX_FASTPATH_MODES static inline void pick_rx_func(struct rte_eth_dev *eth_dev, - const eth_rx_burst_t rx_burst[2][2][2][2][2][2]) + const eth_rx_burst_t rx_burst[2][2][2][2][2][2][2]) { struct otx2_eth_dev *dev = otx2_eth_pmd_priv(eth_dev); - /* [TSTMP] [MARK] [VLAN] [CKSUM] [PTYPE] [RSS] */ + /* [SEC] [TSTMP] [MARK] [VLAN] [CKSUM] [PTYPE] [RSS] */ eth_dev->rx_pkt_burst = rx_burst + [!!(dev->rx_offload_flags & NIX_RX_OFFLOAD_SECURITY_F)] [!!(dev->rx_offload_flags & NIX_RX_OFFLOAD_TSTAMP_F)] [!!(dev->rx_offload_flags & NIX_RX_OFFLOAD_MARK_UPDATE_F)] [!!(dev->rx_offload_flags & NIX_RX_OFFLOAD_VLAN_STRIP_F)] @@ -370,25 +371,25 @@ otx2_eth_set_rx_function(struct rte_eth_dev *eth_dev) { struct otx2_eth_dev *dev = otx2_eth_pmd_priv(eth_dev); - const eth_rx_burst_t nix_eth_rx_burst[2][2][2][2][2][2] = { -#define R(name, f5, f4, f3, f2, f1, f0, flags) \ - [f5][f4][f3][f2][f1][f0] = otx2_nix_recv_pkts_ ## name, + const eth_rx_burst_t nix_eth_rx_burst[2][2][2][2][2][2][2] = { +#define R(name, f6, f5, f4, f3, f2, f1, f0, flags) \ + [f6][f5][f4][f3][f2][f1][f0] = otx2_nix_recv_pkts_ ## name, NIX_RX_FASTPATH_MODES #undef R }; - const eth_rx_burst_t nix_eth_rx_burst_mseg[2][2][2][2][2][2] = { -#define R(name, f5, f4, f3, f2, f1, f0, flags) \ - [f5][f4][f3][f2][f1][f0] = otx2_nix_recv_pkts_mseg_ ## name, + const eth_rx_burst_t nix_eth_rx_burst_mseg[2][2][2][2][2][2][2] = { +#define R(name, f6, f5, f4, f3, f2, f1, f0, flags) \ + [f6][f5][f4][f3][f2][f1][f0] = otx2_nix_recv_pkts_mseg_ ## name, NIX_RX_FASTPATH_MODES #undef R }; - const eth_rx_burst_t nix_eth_rx_vec_burst[2][2][2][2][2][2] = { -#define R(name, f5, f4, f3, f2, f1, f0, flags) \ - [f5][f4][f3][f2][f1][f0] = otx2_nix_recv_pkts_vec_ ## name, + const eth_rx_burst_t nix_eth_rx_vec_burst[2][2][2][2][2][2][2] = { +#define R(name, f6, f5, f4, f3, f2, f1, f0, flags) \ + [f6][f5][f4][f3][f2][f1][f0] = otx2_nix_recv_pkts_vec_ ## name, NIX_RX_FASTPATH_MODES #undef R @@ -408,6 +409,6 @@ NIX_RX_FASTPATH_MODES /* Copy multi seg version with no offload for tear down sequence */ if (rte_eal_process_type() == RTE_PROC_PRIMARY) dev->rx_pkt_burst_no_offload = - nix_eth_rx_burst_mseg[0][0][0][0][0][0]; + nix_eth_rx_burst_mseg[0][0][0][0][0][0][0]; rte_mb(); } diff --git a/drivers/net/octeontx2/otx2_rx.h b/drivers/net/octeontx2/otx2_rx.h index e1715bd..2262777 100644 --- a/drivers/net/octeontx2/otx2_rx.h +++ b/drivers/net/octeontx2/otx2_rx.h @@ -29,6 +29,7 @@ #define NIX_RX_OFFLOAD_VLAN_STRIP_F BIT(3) #define NIX_RX_OFFLOAD_MARK_UPDATE_F BIT(4) #define NIX_RX_OFFLOAD_TSTAMP_F BIT(5) +#define NIX_RX_OFFLOAD_SECURITY_F BIT(6) /* Flags to control cqe_to_mbuf conversion function. * Defining it from backwards to denote its been @@ -297,7 +298,8 @@ otx2_nix_cqe_to_mbuf(const struct nix_cqe_hdr_s *cq, const uint32_t tag, if (flag & NIX_RX_OFFLOAD_MARK_UPDATE_F) ol_flags = nix_update_match_id(rx->match_id, ol_flags, mbuf); - if (cq->cqe_type == NIX_XQE_TYPE_RX_IPSECH) { + if ((flag & NIX_RX_OFFLOAD_SECURITY_F) && + cq->cqe_type == NIX_XQE_TYPE_RX_IPSECH) { *(uint64_t *)(&mbuf->rearm_data) = val; ol_flags |= nix_rx_sec_mbuf_update(cq, mbuf, lookup_mem); mbuf->ol_flags = ol_flags; @@ -320,94 +322,220 @@ otx2_nix_cqe_to_mbuf(const struct nix_cqe_hdr_s *cq, const uint32_t tag, #define RX_VLAN_F NIX_RX_OFFLOAD_VLAN_STRIP_F #define MARK_F NIX_RX_OFFLOAD_MARK_UPDATE_F #define TS_F NIX_RX_OFFLOAD_TSTAMP_F +#define RX_SEC_F NIX_RX_OFFLOAD_SECURITY_F -/* [TSMP] [MARK] [VLAN] [CKSUM] [PTYPE] [RSS] */ +/* [SEC] [TSMP] [MARK] [VLAN] [CKSUM] [PTYPE] [RSS] */ #define NIX_RX_FASTPATH_MODES \ -R(no_offload, 0, 0, 0, 0, 0, 0, NIX_RX_OFFLOAD_NONE) \ -R(rss, 0, 0, 0, 0, 0, 1, RSS_F) \ -R(ptype, 0, 0, 0, 0, 1, 0, PTYPE_F) \ -R(ptype_rss, 0, 0, 0, 0, 1, 1, PTYPE_F | RSS_F) \ -R(cksum, 0, 0, 0, 1, 0, 0, CKSUM_F) \ -R(cksum_rss, 0, 0, 0, 1, 0, 1, CKSUM_F | RSS_F) \ -R(cksum_ptype, 0, 0, 0, 1, 1, 0, CKSUM_F | PTYPE_F) \ -R(cksum_ptype_rss, 0, 0, 0, 1, 1, 1, CKSUM_F | PTYPE_F | RSS_F)\ -R(vlan, 0, 0, 1, 0, 0, 0, RX_VLAN_F) \ -R(vlan_rss, 0, 0, 1, 0, 0, 1, RX_VLAN_F | RSS_F) \ -R(vlan_ptype, 0, 0, 1, 0, 1, 0, RX_VLAN_F | PTYPE_F) \ -R(vlan_ptype_rss, 0, 0, 1, 0, 1, 1, RX_VLAN_F | PTYPE_F | RSS_F)\ -R(vlan_cksum, 0, 0, 1, 1, 0, 0, RX_VLAN_F | CKSUM_F) \ -R(vlan_cksum_rss, 0, 0, 1, 1, 0, 1, RX_VLAN_F | CKSUM_F | RSS_F)\ -R(vlan_cksum_ptype, 0, 0, 1, 1, 1, 0, \ - RX_VLAN_F | CKSUM_F | PTYPE_F) \ -R(vlan_cksum_ptype_rss, 0, 0, 1, 1, 1, 1, \ - RX_VLAN_F | CKSUM_F | PTYPE_F | RSS_F) \ -R(mark, 0, 1, 0, 0, 0, 0, MARK_F) \ -R(mark_rss, 0, 1, 0, 0, 0, 1, MARK_F | RSS_F) \ -R(mark_ptype, 0, 1, 0, 0, 1, 0, MARK_F | PTYPE_F) \ -R(mark_ptype_rss, 0, 1, 0, 0, 1, 1, MARK_F | PTYPE_F | RSS_F)\ -R(mark_cksum, 0, 1, 0, 1, 0, 0, MARK_F | CKSUM_F) \ -R(mark_cksum_rss, 0, 1, 0, 1, 0, 1, MARK_F | CKSUM_F | RSS_F)\ -R(mark_cksum_ptype, 0, 1, 0, 1, 1, 0, MARK_F | CKSUM_F | PTYPE_F)\ -R(mark_cksum_ptype_rss, 0, 1, 0, 1, 1, 1, \ - MARK_F | CKSUM_F | PTYPE_F | RSS_F) \ -R(mark_vlan, 0, 1, 1, 0, 0, 0, MARK_F | RX_VLAN_F) \ -R(mark_vlan_rss, 0, 1, 1, 0, 0, 1, MARK_F | RX_VLAN_F | RSS_F)\ -R(mark_vlan_ptype, 0, 1, 1, 0, 1, 0, \ - MARK_F | RX_VLAN_F | PTYPE_F) \ -R(mark_vlan_ptype_rss, 0, 1, 1, 0, 1, 1, \ - MARK_F | RX_VLAN_F | PTYPE_F | RSS_F) \ -R(mark_vlan_cksum, 0, 1, 1, 1, 0, 0, \ - MARK_F | RX_VLAN_F | CKSUM_F) \ -R(mark_vlan_cksum_rss, 0, 1, 1, 1, 0, 1, \ - MARK_F | RX_VLAN_F | CKSUM_F | RSS_F) \ -R(mark_vlan_cksum_ptype, 0, 1, 1, 1, 1, 0, \ - MARK_F | RX_VLAN_F | CKSUM_F | PTYPE_F) \ -R(mark_vlan_cksum_ptype_rss, 0, 1, 1, 1, 1, 1, \ - MARK_F | RX_VLAN_F | CKSUM_F | PTYPE_F | RSS_F) \ -R(ts, 1, 0, 0, 0, 0, 0, TS_F) \ -R(ts_rss, 1, 0, 0, 0, 0, 1, TS_F | RSS_F) \ -R(ts_ptype, 1, 0, 0, 0, 1, 0, TS_F | PTYPE_F) \ -R(ts_ptype_rss, 1, 0, 0, 0, 1, 1, TS_F | PTYPE_F | RSS_F)\ -R(ts_cksum, 1, 0, 0, 1, 0, 0, TS_F | CKSUM_F) \ -R(ts_cksum_rss, 1, 0, 0, 1, 0, 1, TS_F | CKSUM_F | RSS_F)\ -R(ts_cksum_ptype, 1, 0, 0, 1, 1, 0, TS_F | CKSUM_F | PTYPE_F)\ -R(ts_cksum_ptype_rss, 1, 0, 0, 1, 1, 1, \ - TS_F | CKSUM_F | PTYPE_F | RSS_F) \ -R(ts_vlan, 1, 0, 1, 0, 0, 0, TS_F | RX_VLAN_F) \ -R(ts_vlan_rss, 1, 0, 1, 0, 0, 1, TS_F | RX_VLAN_F | RSS_F)\ -R(ts_vlan_ptype, 1, 0, 1, 0, 1, 0, TS_F | RX_VLAN_F | PTYPE_F)\ -R(ts_vlan_ptype_rss, 1, 0, 1, 0, 1, 1, \ - TS_F | RX_VLAN_F | PTYPE_F | RSS_F) \ -R(ts_vlan_cksum, 1, 0, 1, 1, 0, 0, \ - TS_F | RX_VLAN_F | CKSUM_F) \ -R(ts_vlan_cksum_rss, 1, 0, 1, 1, 0, 1, \ - MARK_F | RX_VLAN_F | CKSUM_F | RSS_F) \ -R(ts_vlan_cksum_ptype, 1, 0, 1, 1, 1, 0, \ - TS_F | RX_VLAN_F | CKSUM_F | PTYPE_F) \ -R(ts_vlan_cksum_ptype_rss, 1, 0, 1, 1, 1, 1, \ - TS_F | RX_VLAN_F | CKSUM_F | PTYPE_F | RSS_F) \ -R(ts_mark, 1, 1, 0, 0, 0, 0, TS_F | MARK_F) \ -R(ts_mark_rss, 1, 1, 0, 0, 0, 1, TS_F | MARK_F | RSS_F)\ -R(ts_mark_ptype, 1, 1, 0, 0, 1, 0, TS_F | MARK_F | PTYPE_F)\ -R(ts_mark_ptype_rss, 1, 1, 0, 0, 1, 1, \ - TS_F | MARK_F | PTYPE_F | RSS_F) \ -R(ts_mark_cksum, 1, 1, 0, 1, 0, 0, TS_F | MARK_F | CKSUM_F)\ -R(ts_mark_cksum_rss, 1, 1, 0, 1, 0, 1, \ - TS_F | MARK_F | CKSUM_F | RSS_F)\ -R(ts_mark_cksum_ptype, 1, 1, 0, 1, 1, 0, \ - TS_F | MARK_F | CKSUM_F | PTYPE_F) \ -R(ts_mark_cksum_ptype_rss, 1, 1, 0, 1, 1, 1, \ - TS_F | MARK_F | CKSUM_F | PTYPE_F | RSS_F) \ -R(ts_mark_vlan, 1, 1, 1, 0, 0, 0, TS_F | MARK_F | RX_VLAN_F)\ -R(ts_mark_vlan_rss, 1, 1, 1, 0, 0, 1, \ - TS_F | MARK_F | RX_VLAN_F | RSS_F)\ -R(ts_mark_vlan_ptype, 1, 1, 1, 0, 1, 0, \ - TS_F | MARK_F | RX_VLAN_F | PTYPE_F) \ -R(ts_mark_vlan_ptype_rss, 1, 1, 1, 0, 1, 1, \ - TS_F | MARK_F | RX_VLAN_F | PTYPE_F | RSS_F) \ -R(ts_mark_vlan_cksum_ptype, 1, 1, 1, 1, 1, 0, \ - TS_F | MARK_F | RX_VLAN_F | CKSUM_F | PTYPE_F) \ -R(ts_mark_vlan_cksum_ptype_rss, 1, 1, 1, 1, 1, 1, \ - TS_F | MARK_F | RX_VLAN_F | CKSUM_F | PTYPE_F | RSS_F) - +R(no_offload, 0, 0, 0, 0, 0, 0, 0, NIX_RX_OFFLOAD_NONE) \ +R(rss, 0, 0, 0, 0, 0, 0, 1, RSS_F) \ +R(ptype, 0, 0, 0, 0, 0, 1, 0, PTYPE_F) \ +R(ptype_rss, 0, 0, 0, 0, 0, 1, 1, PTYPE_F | RSS_F) \ +R(cksum, 0, 0, 0, 0, 1, 0, 0, CKSUM_F) \ +R(cksum_rss, 0, 0, 0, 0, 1, 0, 1, CKSUM_F | RSS_F) \ +R(cksum_ptype, 0, 0, 0, 0, 1, 1, 0, CKSUM_F | PTYPE_F) \ +R(cksum_ptype_rss, 0, 0, 0, 0, 1, 1, 1, CKSUM_F | PTYPE_F | RSS_F)\ +R(vlan, 0, 0, 0, 1, 0, 0, 0, RX_VLAN_F) \ +R(vlan_rss, 0, 0, 0, 1, 0, 0, 1, RX_VLAN_F | RSS_F) \ +R(vlan_ptype, 0, 0, 0, 1, 0, 1, 0, RX_VLAN_F | PTYPE_F) \ +R(vlan_ptype_rss, 0, 0, 0, 1, 0, 1, 1, \ + RX_VLAN_F | PTYPE_F | RSS_F) \ +R(vlan_cksum, 0, 0, 0, 1, 1, 0, 0, RX_VLAN_F | CKSUM_F) \ +R(vlan_cksum_rss, 0, 0, 0, 1, 1, 0, 1, \ + RX_VLAN_F | CKSUM_F | RSS_F) \ +R(vlan_cksum_ptype, 0, 0, 0, 1, 1, 1, 0, \ + RX_VLAN_F | CKSUM_F | PTYPE_F) \ +R(vlan_cksum_ptype_rss, 0, 0, 0, 1, 1, 1, 1, \ + RX_VLAN_F | CKSUM_F | PTYPE_F | RSS_F) \ +R(mark, 0, 0, 1, 0, 0, 0, 0, MARK_F) \ +R(mark_rss, 0, 0, 1, 0, 0, 0, 1, MARK_F | RSS_F) \ +R(mark_ptype, 0, 0, 1, 0, 0, 1, 0, MARK_F | PTYPE_F) \ +R(mark_ptype_rss, 0, 0, 1, 0, 0, 1, 1, MARK_F | PTYPE_F | RSS_F) \ +R(mark_cksum, 0, 0, 1, 0, 1, 0, 0, MARK_F | CKSUM_F) \ +R(mark_cksum_rss, 0, 0, 1, 0, 1, 0, 1, MARK_F | CKSUM_F | RSS_F) \ +R(mark_cksum_ptype, 0, 0, 1, 0, 1, 1, 0, \ + MARK_F | CKSUM_F | PTYPE_F) \ +R(mark_cksum_ptype_rss, 0, 0, 1, 0, 1, 1, 1, \ + MARK_F | CKSUM_F | PTYPE_F | RSS_F) \ +R(mark_vlan, 0, 0, 1, 1, 0, 0, 0, MARK_F | RX_VLAN_F) \ +R(mark_vlan_rss, 0, 0, 1, 1, 0, 0, 1, \ + MARK_F | RX_VLAN_F | RSS_F) \ +R(mark_vlan_ptype, 0, 0, 1, 1, 0, 1, 0, \ + MARK_F | RX_VLAN_F | PTYPE_F) \ +R(mark_vlan_ptype_rss, 0, 0, 1, 1, 0, 1, 1, \ + MARK_F | RX_VLAN_F | PTYPE_F | RSS_F) \ +R(mark_vlan_cksum, 0, 0, 1, 1, 1, 0, 0, \ + MARK_F | RX_VLAN_F | CKSUM_F) \ +R(mark_vlan_cksum_rss, 0, 0, 1, 1, 1, 0, 1, \ + MARK_F | RX_VLAN_F | CKSUM_F | RSS_F) \ +R(mark_vlan_cksum_ptype, 0, 0, 1, 1, 1, 1, 0, \ + MARK_F | RX_VLAN_F | CKSUM_F | PTYPE_F) \ +R(mark_vlan_cksum_ptype_rss, 0, 0, 1, 1, 1, 1, 1, \ + MARK_F | RX_VLAN_F | CKSUM_F | PTYPE_F | RSS_F) \ +R(ts, 0, 1, 0, 0, 0, 0, 0, TS_F) \ +R(ts_rss, 0, 1, 0, 0, 0, 0, 1, TS_F | RSS_F) \ +R(ts_ptype, 0, 1, 0, 0, 0, 1, 0, TS_F | PTYPE_F) \ +R(ts_ptype_rss, 0, 1, 0, 0, 0, 1, 1, TS_F | PTYPE_F | RSS_F) \ +R(ts_cksum, 0, 1, 0, 0, 1, 0, 0, TS_F | CKSUM_F) \ +R(ts_cksum_rss, 0, 1, 0, 0, 1, 0, 1, TS_F | CKSUM_F | RSS_F) \ +R(ts_cksum_ptype, 0, 1, 0, 0, 1, 1, 0, TS_F | CKSUM_F | PTYPE_F) \ +R(ts_cksum_ptype_rss, 0, 1, 0, 0, 1, 1, 1, \ + TS_F | CKSUM_F | PTYPE_F | RSS_F) \ +R(ts_vlan, 0, 1, 0, 1, 0, 0, 0, TS_F | RX_VLAN_F) \ +R(ts_vlan_rss, 0, 1, 0, 1, 0, 0, 1, TS_F | RX_VLAN_F | RSS_F) \ +R(ts_vlan_ptype, 0, 1, 0, 1, 0, 1, 0, \ + TS_F | RX_VLAN_F | PTYPE_F) \ +R(ts_vlan_ptype_rss, 0, 1, 0, 1, 0, 1, 1, \ + TS_F | RX_VLAN_F | PTYPE_F | RSS_F) \ +R(ts_vlan_cksum, 0, 1, 0, 1, 1, 0, 0, \ + TS_F | RX_VLAN_F | CKSUM_F) \ +R(ts_vlan_cksum_rss, 0, 1, 0, 1, 1, 0, 1, \ + MARK_F | RX_VLAN_F | CKSUM_F | RSS_F) \ +R(ts_vlan_cksum_ptype, 0, 1, 0, 1, 1, 1, 0, \ + TS_F | RX_VLAN_F | CKSUM_F | PTYPE_F) \ +R(ts_vlan_cksum_ptype_rss, 0, 1, 0, 1, 1, 1, 1, \ + TS_F | RX_VLAN_F | CKSUM_F | PTYPE_F | RSS_F) \ +R(ts_mark, 0, 1, 1, 0, 0, 0, 0, TS_F | MARK_F) \ +R(ts_mark_rss, 0, 1, 1, 0, 0, 0, 1, TS_F | MARK_F | RSS_F) \ +R(ts_mark_ptype, 0, 1, 1, 0, 0, 1, 0, TS_F | MARK_F | PTYPE_F) \ +R(ts_mark_ptype_rss, 0, 1, 1, 0, 0, 1, 1, \ + TS_F | MARK_F | PTYPE_F | RSS_F) \ +R(ts_mark_cksum, 0, 1, 1, 0, 1, 0, 0, TS_F | MARK_F | CKSUM_F) \ +R(ts_mark_cksum_rss, 0, 1, 1, 0, 1, 0, 1, \ + TS_F | MARK_F | CKSUM_F | RSS_F) \ +R(ts_mark_cksum_ptype, 0, 1, 1, 0, 1, 1, 0, \ + TS_F | MARK_F | CKSUM_F | PTYPE_F) \ +R(ts_mark_cksum_ptype_rss, 0, 1, 1, 0, 1, 1, 1, \ + TS_F | MARK_F | CKSUM_F | PTYPE_F | RSS_F) \ +R(ts_mark_vlan, 0, 1, 1, 1, 0, 0, 0, TS_F | MARK_F | RX_VLAN_F)\ +R(ts_mark_vlan_rss, 0, 1, 1, 1, 0, 0, 1, \ + TS_F | MARK_F | RX_VLAN_F | RSS_F) \ +R(ts_mark_vlan_ptype, 0, 1, 1, 1, 0, 1, 0, \ + TS_F | MARK_F | RX_VLAN_F | PTYPE_F) \ +R(ts_mark_vlan_ptype_rss, 0, 1, 1, 1, 0, 1, 1, \ + TS_F | MARK_F | RX_VLAN_F | PTYPE_F | RSS_F) \ +R(ts_mark_vlan_cksum_ptype, 0, 1, 1, 1, 1, 1, 0, \ + TS_F | MARK_F | RX_VLAN_F | CKSUM_F | PTYPE_F) \ +R(ts_mark_vlan_cksum_ptype_rss, 0, 1, 1, 1, 1, 1, 1, \ + TS_F | MARK_F | RX_VLAN_F | CKSUM_F | PTYPE_F | RSS_F) \ +R(sec, 1, 0, 0, 0, 0, 0, 0, RX_SEC_F) \ +R(sec_rss, 1, 0, 0, 0, 0, 0, 1, RX_SEC_F | RSS_F) \ +R(sec_ptype, 1, 0, 0, 0, 0, 1, 0, RX_SEC_F | PTYPE_F) \ +R(sec_ptype_rss, 1, 0, 0, 0, 0, 1, 1, \ + RX_SEC_F | PTYPE_F | RSS_F) \ +R(sec_cksum, 1, 0, 0, 0, 1, 0, 0, RX_SEC_F | CKSUM_F) \ +R(sec_cksum_rss, 1, 0, 0, 0, 1, 0, 1, \ + RX_SEC_F | CKSUM_F | RSS_F) \ +R(sec_cksum_ptype, 1, 0, 0, 0, 1, 1, 0, \ + RX_SEC_F | CKSUM_F | PTYPE_F) \ +R(sec_cksum_ptype_rss, 1, 0, 0, 0, 1, 1, 1, \ + RX_SEC_F | CKSUM_F | PTYPE_F | RSS_F) \ +R(sec_vlan, 1, 0, 0, 1, 0, 0, 0, RX_SEC_F | RX_VLAN_F) \ +R(sec_vlan_rss, 1, 0, 0, 1, 0, 0, 1, \ + RX_SEC_F | RX_VLAN_F | RSS_F) \ +R(sec_vlan_ptype, 1, 0, 0, 1, 0, 1, 0, \ + RX_SEC_F | RX_VLAN_F | PTYPE_F) \ +R(sec_vlan_ptype_rss, 1, 0, 0, 1, 0, 1, 1, \ + RX_SEC_F | RX_VLAN_F | PTYPE_F | RSS_F) \ +R(sec_vlan_cksum, 1, 0, 0, 1, 1, 0, 0, \ + RX_SEC_F | RX_VLAN_F | CKSUM_F) \ +R(sec_vlan_cksum_rss, 1, 0, 0, 1, 1, 0, 1, \ + RX_SEC_F | RX_VLAN_F | CKSUM_F | RSS_F) \ +R(sec_vlan_cksum_ptype, 1, 0, 0, 1, 1, 1, 0, \ + RX_SEC_F | RX_VLAN_F | CKSUM_F | PTYPE_F) \ +R(sec_vlan_cksum_ptype_rss, 1, 0, 0, 1, 1, 1, 1, \ + RX_SEC_F | RX_VLAN_F | CKSUM_F | PTYPE_F | RSS_F) \ +R(sec_mark, 1, 0, 1, 0, 0, 0, 0, RX_SEC_F | MARK_F) \ +R(sec_mark_rss, 1, 0, 1, 0, 0, 0, 1, RX_SEC_F | MARK_F | RSS_F)\ +R(sec_mark_ptype, 1, 0, 1, 0, 0, 1, 0, \ + RX_SEC_F | MARK_F | PTYPE_F) \ +R(sec_mark_ptype_rss, 1, 0, 1, 0, 0, 1, 1, \ + RX_SEC_F | MARK_F | PTYPE_F | RSS_F) \ +R(sec_mark_cksum, 1, 0, 1, 0, 1, 0, 0, \ + RX_SEC_F | MARK_F | CKSUM_F) \ +R(sec_mark_cksum_rss, 1, 0, 1, 0, 1, 0, 1, \ + RX_SEC_F | MARK_F | CKSUM_F | RSS_F) \ +R(sec_mark_cksum_ptype, 1, 0, 1, 0, 1, 1, 0, \ + RX_SEC_F | MARK_F | CKSUM_F | PTYPE_F) \ +R(sec_mark_cksum_ptype_rss, 1, 0, 1, 0, 1, 1, 1, \ + RX_SEC_F | MARK_F | CKSUM_F | PTYPE_F | RSS_F) \ +R(sec_mark_vlan, 1, 0, 1, 1, 0, 0, 0, RX_SEC_F | RX_VLAN_F) \ +R(sec_mark_vlan_rss, 1, 0, 1, 1, 0, 0, 1, \ + RX_SEC_F | MARK_F | RX_VLAN_F | RSS_F) \ +R(sec_mark_vlan_ptype, 1, 0, 1, 1, 0, 1, 0, \ + RX_SEC_F | MARK_F | RX_VLAN_F | PTYPE_F) \ +R(sec_mark_vlan_ptype_rss, 1, 0, 1, 1, 0, 1, 1, \ + RX_SEC_F | MARK_F | RX_VLAN_F | PTYPE_F | RSS_F) \ +R(sec_mark_vlan_cksum, 1, 0, 1, 1, 1, 0, 0, \ + RX_SEC_F | MARK_F | RX_VLAN_F | CKSUM_F) \ +R(sec_mark_vlan_cksum_rss, 1, 0, 1, 1, 1, 0, 1, \ + RX_SEC_F | MARK_F | RX_VLAN_F | CKSUM_F | RSS_F) \ +R(sec_mark_vlan_cksum_ptype, 1, 0, 1, 1, 1, 1, 0, \ + RX_SEC_F | MARK_F | RX_VLAN_F | CKSUM_F | PTYPE_F) \ +R(sec_mark_vlan_cksum_ptype_rss, \ + 1, 0, 1, 1, 1, 1, 1, \ + RX_SEC_F | MARK_F | RX_VLAN_F | CKSUM_F | PTYPE_F | \ + RSS_F) \ +R(sec_ts, 1, 1, 0, 0, 0, 0, 0, RX_SEC_F | TS_F) \ +R(sec_ts_rss, 1, 1, 0, 0, 0, 0, 1, RX_SEC_F | TS_F | RSS_F) \ +R(sec_ts_ptype, 1, 1, 0, 0, 0, 1, 0, RX_SEC_F | TS_F | PTYPE_F)\ +R(sec_ts_ptype_rss, 1, 1, 0, 0, 0, 1, 1, \ + RX_SEC_F | TS_F | PTYPE_F | RSS_F) \ +R(sec_ts_cksum, 1, 1, 0, 0, 1, 0, 0, RX_SEC_F | TS_F | CKSUM_F)\ +R(sec_ts_cksum_rss, 1, 1, 0, 0, 1, 0, 1, \ + RX_SEC_F | TS_F | CKSUM_F | RSS_F) \ +R(sec_ts_cksum_ptype, 1, 1, 0, 0, 1, 1, 0, \ + RX_SEC_F | CKSUM_F | PTYPE_F) \ +R(sec_ts_cksum_ptype_rss, 1, 1, 0, 0, 1, 1, 1, \ + RX_SEC_F | TS_F | CKSUM_F | PTYPE_F | RSS_F) \ +R(sec_ts_vlan, 1, 1, 0, 1, 0, 0, 0, \ + RX_SEC_F | TS_F | RX_VLAN_F) \ +R(sec_ts_vlan_rss, 1, 1, 0, 1, 0, 0, 1, \ + RX_SEC_F | TS_F | RX_VLAN_F | RSS_F) \ +R(sec_ts_vlan_ptype, 1, 1, 0, 1, 0, 1, 0, \ + RX_SEC_F | TS_F | RX_VLAN_F | PTYPE_F) \ +R(sec_ts_vlan_ptype_rss, 1, 1, 0, 1, 0, 1, 1, \ + RX_SEC_F | TS_F | RX_VLAN_F | PTYPE_F | RSS_F) \ +R(sec_ts_vlan_cksum, 1, 1, 0, 1, 1, 0, 0, \ + RX_SEC_F | TS_F | RX_VLAN_F | CKSUM_F) \ +R(sec_ts_vlan_cksum_rss, 1, 1, 0, 1, 1, 0, 1, \ + RX_SEC_F | TS_F | RX_VLAN_F | CKSUM_F | RSS_F) \ +R(sec_ts_vlan_cksum_ptype, 1, 1, 0, 1, 1, 1, 0, \ + RX_SEC_F | TS_F | RX_VLAN_F | CKSUM_F | PTYPE_F) \ +R(sec_ts_vlan_cksum_ptype_rss, 1, 1, 0, 1, 1, 1, 1, \ + RX_SEC_F | TS_F | RX_VLAN_F | CKSUM_F | PTYPE_F | \ + RSS_F) \ +R(sec_ts_mark, 1, 1, 1, 0, 0, 0, 0, RX_SEC_F | TS_F | MARK_F) \ +R(sec_ts_mark_rss, 1, 1, 1, 0, 0, 0, 1, \ + RX_SEC_F | TS_F | MARK_F | RSS_F) \ +R(sec_ts_mark_ptype, 1, 1, 1, 0, 0, 1, 0, \ + RX_SEC_F | TS_F | MARK_F | PTYPE_F) \ +R(sec_ts_mark_ptype_rss, 1, 1, 1, 0, 0, 1, 1, \ + RX_SEC_F | TS_F | MARK_F | PTYPE_F | RSS_F) \ +R(sec_ts_mark_cksum, 1, 1, 1, 0, 1, 0, 0, \ + RX_SEC_F | TS_F | MARK_F | CKSUM_F) \ +R(sec_ts_mark_cksum_rss, 1, 1, 1, 0, 1, 0, 1, \ + RX_SEC_F | TS_F | MARK_F | CKSUM_F | RSS_F) \ +R(sec_ts_mark_cksum_ptype, 1, 1, 1, 0, 1, 1, 0, \ + RX_SEC_F | TS_F | MARK_F | CKSUM_F | PTYPE_F) \ +R(sec_ts_mark_cksum_ptype_rss, 1, 1, 1, 0, 1, 1, 1, \ + RX_SEC_F | TS_F | MARK_F | CKSUM_F | PTYPE_F | RSS_F) \ +R(sec_ts_mark_vlan, 1, 1, 1, 1, 0, 0, 0, \ + RX_SEC_F | TS_F | MARK_F | RX_VLAN_F) \ +R(sec_ts_mark_vlan_rss, 1, 1, 1, 1, 0, 0, 1, \ + RX_SEC_F | RX_VLAN_F | RSS_F) \ +R(sec_ts_mark_vlan_ptype, 1, 1, 1, 1, 0, 1, 0, \ + RX_SEC_F | TS_F | MARK_F | RX_VLAN_F | PTYPE_F) \ +R(sec_ts_mark_vlan_ptype_rss, 1, 1, 1, 1, 0, 1, 1, \ + RX_SEC_F | TS_F | MARK_F | RX_VLAN_F | PTYPE_F | RSS_F)\ +R(sec_ts_mark_vlan_cksum, 1, 1, 1, 1, 1, 0, 0, \ + RX_SEC_F | TS_F | MARK_F | RX_VLAN_F | CKSUM_F) \ +R(sec_ts_mark_vlan_cksum_rss, 1, 1, 1, 1, 1, 0, 1, \ + RX_SEC_F | TS_F | MARK_F | RX_VLAN_F | CKSUM_F | RSS_F)\ +R(sec_ts_mark_vlan_cksum_ptype, 1, 1, 1, 1, 1, 1, 0, \ + RX_SEC_F | TS_F | MARK_F | RX_VLAN_F | CKSUM_F | \ + PTYPE_F) \ +R(sec_ts_mark_vlan_cksum_ptype_rss, \ + 1, 1, 1, 1, 1, 1, 1, \ + RX_SEC_F | TS_F | MARK_F | RX_VLAN_F | CKSUM_F | \ + PTYPE_F | RSS_F) #endif /* __OTX2_RX_H__ */ From patchwork Sat Jan 18 10:49:11 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Anoob Joseph X-Patchwork-Id: 64869 X-Patchwork-Delegate: gakhil@marvell.com Return-Path: X-Original-To: patchwork@inbox.dpdk.org Delivered-To: patchwork@inbox.dpdk.org Received: from dpdk.org (dpdk.org [92.243.14.124]) by inbox.dpdk.org (Postfix) with ESMTP id 578FFA051C; Sat, 18 Jan 2020 11:51:36 +0100 (CET) Received: from [92.243.14.124] (localhost [127.0.0.1]) by dpdk.org (Postfix) with ESMTP id C57321BF7C; Sat, 18 Jan 2020 11:51:07 +0100 (CET) Received: from mx0b-0016f401.pphosted.com (mx0a-0016f401.pphosted.com [67.231.148.174]) by dpdk.org (Postfix) with ESMTP id BFCEA1BF85 for ; Sat, 18 Jan 2020 11:51:04 +0100 (CET) Received: from pps.filterd (m0045849.ppops.net [127.0.0.1]) by mx0a-0016f401.pphosted.com (8.16.0.42/8.16.0.42) with SMTP id 00IAkGqk009777; Sat, 18 Jan 2020 02:51:04 -0800 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=marvell.com; h=from : to : cc : subject : date : message-id : in-reply-to : references : mime-version : content-transfer-encoding : content-type; s=pfpt0818; bh=Rn4VF9mdkeATNAYU8R3QbvvXaAV0rI0RmyYArgBItfo=; b=VOianmaB0uOArF/ko7X2Pu58UGsjOACf25ea7DjzLu78c5HCBqEaicV/bz+roeFjAtPu Ln7QUQMp8akVY6Ric7xzOjYHq9x2JcenBCPRwPrDJD7Bx+tyvw7nAvtWRrvcBJx8Iquj eRzvApCo4peIzbE192XIWEPJW2mnvTkVR0Qpd5y3zLxtaaNM4onLOg8d69nIP04vf1Iv Bf4mh825ri4bIlx2G06Rs5IwCobX7Rko8Uu62xqzM1L7DYQdKXKGy4eNYigMDxr57GE9 hZsBLr3UgVDPwUQqKY5lBKuahhwMcgTyoIq7pQpkD79UqZFKPPbvOktTiWemdv8LuJrs 7w== Received: from sc-exch04.marvell.com ([199.233.58.184]) by mx0a-0016f401.pphosted.com with ESMTP id 2xm08v01qs-2 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-SHA384 bits=256 verify=NOT); Sat, 18 Jan 2020 02:51:03 -0800 Received: from SC-EXCH01.marvell.com (10.93.176.81) by SC-EXCH04.marvell.com (10.93.176.84) with Microsoft SMTP Server (TLS) id 15.0.1497.2; Sat, 18 Jan 2020 02:51:02 -0800 Received: from maili.marvell.com (10.93.176.43) by SC-EXCH01.marvell.com (10.93.176.81) with Microsoft SMTP Server id 15.0.1497.2 via Frontend Transport; Sat, 18 Jan 2020 02:51:02 -0800 Received: from ajoseph83.caveonetworks.com (unknown [10.29.45.60]) by maili.marvell.com (Postfix) with ESMTP id 72F883F703F; Sat, 18 Jan 2020 02:50:57 -0800 (PST) From: Anoob Joseph To: Akhil Goyal , Declan Doherty , Thomas Monjalon CC: Archana Muniganti , Jerin Jacob , Narayana Prasad , Kiran Kumar K , Nithin Dabilpuram , "Pavan Nikhilesh" , Ankur Dwivedi , Anoob Joseph , Tejasree Kondoj , Vamsi Attunuru , Lukasz Bartosik , Date: Sat, 18 Jan 2020 16:19:11 +0530 Message-ID: <1579344553-11428-14-git-send-email-anoobj@marvell.com> X-Mailer: git-send-email 2.7.4 In-Reply-To: <1579344553-11428-1-git-send-email-anoobj@marvell.com> References: <1575806094-28391-1-git-send-email-anoobj@marvell.com> <1579344553-11428-1-git-send-email-anoobj@marvell.com> MIME-Version: 1.0 X-Proofpoint-Virus-Version: vendor=fsecure engine=2.50.10434:6.0.138, 18.0.572 definitions=2020-01-18_02:2020-01-16, 2020-01-18 signatures=0 Subject: [dpdk-dev] [PATCH v2 13/15] drivers/octeontx2: add sec in compiler optimized TX fastpath framework X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Sender: "dev" From: Archana Muniganti Added new flag for SECURITY in compiler optimized TX fastpath framework. With this, compiler autogenerates functions which have security enabled. Signed-off-by: Ankur Dwivedi Signed-off-by: Anoob Joseph Signed-off-by: Archana Muniganti Signed-off-by: Tejasree Kondoj Signed-off-by: Vamsi Attunuru --- drivers/event/octeontx2/otx2_evdev.c | 36 ++-- drivers/event/octeontx2/otx2_evdev.h | 2 +- drivers/event/octeontx2/otx2_worker.c | 4 +- drivers/event/octeontx2/otx2_worker_dual.c | 4 +- drivers/net/octeontx2/otx2_ethdev.c | 3 + drivers/net/octeontx2/otx2_tx.c | 29 +-- drivers/net/octeontx2/otx2_tx.h | 271 ++++++++++++++++++++++------- 7 files changed, 250 insertions(+), 99 deletions(-) diff --git a/drivers/event/octeontx2/otx2_evdev.c b/drivers/event/octeontx2/otx2_evdev.c index f6c641a..d20213d 100644 --- a/drivers/event/octeontx2/otx2_evdev.c +++ b/drivers/event/octeontx2/otx2_evdev.c @@ -177,35 +177,37 @@ SSO_RX_ADPTR_ENQ_FASTPATH_FUNC }; /* Tx modes */ - const event_tx_adapter_enqueue ssogws_tx_adptr_enq[2][2][2][2][2][2] = { -#define T(name, f5, f4, f3, f2, f1, f0, sz, flags) \ - [f5][f4][f3][f2][f1][f0] = otx2_ssogws_tx_adptr_enq_ ## name, + const event_tx_adapter_enqueue + ssogws_tx_adptr_enq[2][2][2][2][2][2][2] = { +#define T(name, f6, f5, f4, f3, f2, f1, f0, sz, flags) \ + [f6][f5][f4][f3][f2][f1][f0] = \ + otx2_ssogws_tx_adptr_enq_ ## name, SSO_TX_ADPTR_ENQ_FASTPATH_FUNC #undef T }; const event_tx_adapter_enqueue - ssogws_tx_adptr_enq_seg[2][2][2][2][2][2] = { -#define T(name, f5, f4, f3, f2, f1, f0, sz, flags) \ - [f5][f4][f3][f2][f1][f0] = \ + ssogws_tx_adptr_enq_seg[2][2][2][2][2][2][2] = { +#define T(name, f6, f5, f4, f3, f2, f1, f0, sz, flags) \ + [f6][f5][f4][f3][f2][f1][f0] = \ otx2_ssogws_tx_adptr_enq_seg_ ## name, SSO_TX_ADPTR_ENQ_FASTPATH_FUNC #undef T }; const event_tx_adapter_enqueue - ssogws_dual_tx_adptr_enq[2][2][2][2][2][2] = { -#define T(name, f5, f4, f3, f2, f1, f0, sz, flags) \ - [f5][f4][f3][f2][f1][f0] = \ + ssogws_dual_tx_adptr_enq[2][2][2][2][2][2][2] = { +#define T(name, f6, f5, f4, f3, f2, f1, f0, sz, flags) \ + [f6][f5][f4][f3][f2][f1][f0] = \ otx2_ssogws_dual_tx_adptr_enq_ ## name, SSO_TX_ADPTR_ENQ_FASTPATH_FUNC #undef T }; const event_tx_adapter_enqueue - ssogws_dual_tx_adptr_enq_seg[2][2][2][2][2][2] = { -#define T(name, f5, f4, f3, f2, f1, f0, sz, flags) \ - [f5][f4][f3][f2][f1][f0] = \ + ssogws_dual_tx_adptr_enq_seg[2][2][2][2][2][2][2] = { +#define T(name, f6, f5, f4, f3, f2, f1, f0, sz, flags) \ + [f6][f5][f4][f3][f2][f1][f0] = \ otx2_ssogws_dual_tx_adptr_enq_seg_ ## name, SSO_TX_ADPTR_ENQ_FASTPATH_FUNC #undef T @@ -290,8 +292,9 @@ SSO_TX_ADPTR_ENQ_FASTPATH_FUNC } if (dev->tx_offloads & NIX_TX_MULTI_SEG_F) { - /* [TSMP] [MBUF_NOFF] [VLAN] [OL3_L4_CSUM] [L3_L4_CSUM] */ + /* [SEC] [TSMP] [MBUF_NOFF] [VLAN] [OL3_L4_CSUM] [L3_L4_CSUM] */ event_dev->txa_enqueue = ssogws_tx_adptr_enq_seg + [!!(dev->tx_offloads & NIX_TX_OFFLOAD_SECURITY_F)] [!!(dev->tx_offloads & NIX_TX_OFFLOAD_TSO_F)] [!!(dev->tx_offloads & NIX_TX_OFFLOAD_TSTAMP_F)] [!!(dev->tx_offloads & NIX_TX_OFFLOAD_MBUF_NOFF_F)] @@ -300,6 +303,7 @@ SSO_TX_ADPTR_ENQ_FASTPATH_FUNC [!!(dev->tx_offloads & NIX_TX_OFFLOAD_L3_L4_CSUM_F)]; } else { event_dev->txa_enqueue = ssogws_tx_adptr_enq + [!!(dev->tx_offloads & NIX_TX_OFFLOAD_SECURITY_F)] [!!(dev->tx_offloads & NIX_TX_OFFLOAD_TSO_F)] [!!(dev->tx_offloads & NIX_TX_OFFLOAD_TSTAMP_F)] [!!(dev->tx_offloads & NIX_TX_OFFLOAD_MBUF_NOFF_F)] @@ -440,8 +444,10 @@ SSO_TX_ADPTR_ENQ_FASTPATH_FUNC } if (dev->tx_offloads & NIX_TX_MULTI_SEG_F) { - /* [TSMP] [MBUF_NOFF] [VLAN] [OL3_L4_CSUM] [L3_L4_CSUM] */ + /* [SEC] [TSMP] [MBUF_NOFF] [VLAN] [OL3_L4_CSUM] [L3_L4_CSUM] */ event_dev->txa_enqueue = ssogws_dual_tx_adptr_enq_seg + [!!(dev->tx_offloads & + NIX_TX_OFFLOAD_SECURITY_F)] [!!(dev->tx_offloads & NIX_TX_OFFLOAD_TSO_F)] [!!(dev->tx_offloads & NIX_TX_OFFLOAD_TSTAMP_F)] [!!(dev->tx_offloads & @@ -454,6 +460,8 @@ SSO_TX_ADPTR_ENQ_FASTPATH_FUNC NIX_TX_OFFLOAD_L3_L4_CSUM_F)]; } else { event_dev->txa_enqueue = ssogws_dual_tx_adptr_enq + [!!(dev->tx_offloads & + NIX_TX_OFFLOAD_SECURITY_F)] [!!(dev->tx_offloads & NIX_TX_OFFLOAD_TSO_F)] [!!(dev->tx_offloads & NIX_TX_OFFLOAD_TSTAMP_F)] [!!(dev->tx_offloads & diff --git a/drivers/event/octeontx2/otx2_evdev.h b/drivers/event/octeontx2/otx2_evdev.h index 5a44fd3..3b47782 100644 --- a/drivers/event/octeontx2/otx2_evdev.h +++ b/drivers/event/octeontx2/otx2_evdev.h @@ -335,7 +335,7 @@ uint16_t otx2_ssogws_dual_deq_seg_timeout_burst_ ##name(void *port, \ SSO_RX_ADPTR_ENQ_FASTPATH_FUNC #undef R -#define T(name, f5, f4, f3, f2, f1, f0, sz, flags) \ +#define T(name, f6, f5, f4, f3, f2, f1, f0, sz, flags) \ uint16_t otx2_ssogws_tx_adptr_enq_ ## name(void *port, struct rte_event ev[],\ uint16_t nb_events); \ uint16_t otx2_ssogws_tx_adptr_enq_seg_ ## name(void *port, \ diff --git a/drivers/event/octeontx2/otx2_worker.c b/drivers/event/octeontx2/otx2_worker.c index 12445d9..8bec59e 100644 --- a/drivers/event/octeontx2/otx2_worker.c +++ b/drivers/event/octeontx2/otx2_worker.c @@ -267,7 +267,7 @@ otx2_ssogws_enq_fwd_burst(void *port, const struct rte_event ev[], return 1; } -#define T(name, f5, f4, f3, f2, f1, f0, sz, flags) \ +#define T(name, f6, f5, f4, f3, f2, f1, f0, sz, flags) \ uint16_t __hot \ otx2_ssogws_tx_adptr_enq_ ## name(void *port, struct rte_event ev[], \ uint16_t nb_events) \ @@ -281,7 +281,7 @@ otx2_ssogws_tx_adptr_enq_ ## name(void *port, struct rte_event ev[], \ SSO_TX_ADPTR_ENQ_FASTPATH_FUNC #undef T -#define T(name, f5, f4, f3, f2, f1, f0, sz, flags) \ +#define T(name, f6, f5, f4, f3, f2, f1, f0, sz, flags) \ uint16_t __hot \ otx2_ssogws_tx_adptr_enq_seg_ ## name(void *port, struct rte_event ev[],\ uint16_t nb_events) \ diff --git a/drivers/event/octeontx2/otx2_worker_dual.c b/drivers/event/octeontx2/otx2_worker_dual.c index 22a4889..3cba09c 100644 --- a/drivers/event/octeontx2/otx2_worker_dual.c +++ b/drivers/event/octeontx2/otx2_worker_dual.c @@ -307,7 +307,7 @@ otx2_ssogws_dual_deq_seg_timeout_burst_ ##name(void *port, \ SSO_RX_ADPTR_ENQ_FASTPATH_FUNC #undef R -#define T(name, f5, f4, f3, f2, f1, f0, sz, flags) \ +#define T(name, f6, f5, f4, f3, f2, f1, f0, sz, flags) \ uint16_t __hot \ otx2_ssogws_dual_tx_adptr_enq_ ## name(void *port, \ struct rte_event ev[], \ @@ -324,7 +324,7 @@ otx2_ssogws_dual_tx_adptr_enq_ ## name(void *port, \ SSO_TX_ADPTR_ENQ_FASTPATH_FUNC #undef T -#define T(name, f5, f4, f3, f2, f1, f0, sz, flags) \ +#define T(name, f6, f5, f4, f3, f2, f1, f0, sz, flags) \ uint16_t __hot \ otx2_ssogws_dual_tx_adptr_enq_seg_ ## name(void *port, \ struct rte_event ev[], \ diff --git a/drivers/net/octeontx2/otx2_ethdev.c b/drivers/net/octeontx2/otx2_ethdev.c index c215078..8801173 100644 --- a/drivers/net/octeontx2/otx2_ethdev.c +++ b/drivers/net/octeontx2/otx2_ethdev.c @@ -698,6 +698,9 @@ nix_tx_offload_flags(struct rte_eth_dev *eth_dev) NIX_TX_OFFLOAD_OL3_OL4_CSUM_F | NIX_TX_OFFLOAD_L3_L4_CSUM_F); + if (conf & DEV_TX_OFFLOAD_SECURITY) + flags |= NIX_TX_OFFLOAD_SECURITY_F; + if ((dev->rx_offloads & DEV_RX_OFFLOAD_TIMESTAMP)) flags |= NIX_TX_OFFLOAD_TSTAMP_F; diff --git a/drivers/net/octeontx2/otx2_tx.c b/drivers/net/octeontx2/otx2_tx.c index fa53300..e43590e 100644 --- a/drivers/net/octeontx2/otx2_tx.c +++ b/drivers/net/octeontx2/otx2_tx.c @@ -945,7 +945,7 @@ nix_xmit_pkts_vector(void *tx_queue, struct rte_mbuf **tx_pkts, } #endif -#define T(name, f5, f4, f3, f2, f1, f0, sz, flags) \ +#define T(name, f6, f5, f4, f3, f2, f1, f0, sz, flags) \ static uint16_t __rte_noinline __hot \ otx2_nix_xmit_pkts_ ## name(void *tx_queue, \ struct rte_mbuf **tx_pkts, uint16_t pkts) \ @@ -962,7 +962,7 @@ otx2_nix_xmit_pkts_ ## name(void *tx_queue, \ NIX_TX_FASTPATH_MODES #undef T -#define T(name, f5, f4, f3, f2, f1, f0, sz, flags) \ +#define T(name, f6, f5, f4, f3, f2, f1, f0, sz, flags) \ static uint16_t __rte_noinline __hot \ otx2_nix_xmit_pkts_mseg_ ## name(void *tx_queue, \ struct rte_mbuf **tx_pkts, uint16_t pkts) \ @@ -980,7 +980,7 @@ otx2_nix_xmit_pkts_mseg_ ## name(void *tx_queue, \ NIX_TX_FASTPATH_MODES #undef T -#define T(name, f5, f4, f3, f2, f1, f0, sz, flags) \ +#define T(name, f6, f5, f4, f3, f2, f1, f0, sz, flags) \ static uint16_t __rte_noinline __hot \ otx2_nix_xmit_pkts_vec_ ## name(void *tx_queue, \ struct rte_mbuf **tx_pkts, uint16_t pkts) \ @@ -998,12 +998,13 @@ NIX_TX_FASTPATH_MODES static inline void pick_tx_func(struct rte_eth_dev *eth_dev, - const eth_tx_burst_t tx_burst[2][2][2][2][2][2]) + const eth_tx_burst_t tx_burst[2][2][2][2][2][2][2]) { struct otx2_eth_dev *dev = otx2_eth_pmd_priv(eth_dev); - /* [TSTMP] [NOFF] [VLAN] [OL3_OL4_CSUM] [IL3_IL4_CSUM] */ + /* [SEC] [TSTMP] [NOFF] [VLAN] [OL3_OL4_CSUM] [IL3_IL4_CSUM] */ eth_dev->tx_pkt_burst = tx_burst + [!!(dev->tx_offload_flags & NIX_TX_OFFLOAD_SECURITY_F)] [!!(dev->tx_offload_flags & NIX_TX_OFFLOAD_TSO_F)] [!!(dev->tx_offload_flags & NIX_TX_OFFLOAD_TSTAMP_F)] [!!(dev->tx_offload_flags & NIX_TX_OFFLOAD_MBUF_NOFF_F)] @@ -1017,25 +1018,25 @@ otx2_eth_set_tx_function(struct rte_eth_dev *eth_dev) { struct otx2_eth_dev *dev = otx2_eth_pmd_priv(eth_dev); - const eth_tx_burst_t nix_eth_tx_burst[2][2][2][2][2][2] = { -#define T(name, f5, f4, f3, f2, f1, f0, sz, flags) \ - [f5][f4][f3][f2][f1][f0] = otx2_nix_xmit_pkts_ ## name, + const eth_tx_burst_t nix_eth_tx_burst[2][2][2][2][2][2][2] = { +#define T(name, f6, f5, f4, f3, f2, f1, f0, sz, flags) \ + [f6][f5][f4][f3][f2][f1][f0] = otx2_nix_xmit_pkts_ ## name, NIX_TX_FASTPATH_MODES #undef T }; - const eth_tx_burst_t nix_eth_tx_burst_mseg[2][2][2][2][2][2] = { -#define T(name, f5, f4, f3, f2, f1, f0, sz, flags) \ - [f5][f4][f3][f2][f1][f0] = otx2_nix_xmit_pkts_mseg_ ## name, + const eth_tx_burst_t nix_eth_tx_burst_mseg[2][2][2][2][2][2][2] = { +#define T(name, f6, f5, f4, f3, f2, f1, f0, sz, flags) \ + [f6][f5][f4][f3][f2][f1][f0] = otx2_nix_xmit_pkts_mseg_ ## name, NIX_TX_FASTPATH_MODES #undef T }; - const eth_tx_burst_t nix_eth_tx_vec_burst[2][2][2][2][2][2] = { -#define T(name, f5, f4, f3, f2, f1, f0, sz, flags) \ - [f5][f4][f3][f2][f1][f0] = otx2_nix_xmit_pkts_vec_ ## name, + const eth_tx_burst_t nix_eth_tx_vec_burst[2][2][2][2][2][2][2] = { +#define T(name, f6, f5, f4, f3, f2, f1, f0, sz, flags) \ + [f6][f5][f4][f3][f2][f1][f0] = otx2_nix_xmit_pkts_vec_ ## name, NIX_TX_FASTPATH_MODES #undef T diff --git a/drivers/net/octeontx2/otx2_tx.h b/drivers/net/octeontx2/otx2_tx.h index 04e859b..3c43170 100644 --- a/drivers/net/octeontx2/otx2_tx.h +++ b/drivers/net/octeontx2/otx2_tx.h @@ -12,6 +12,7 @@ #define NIX_TX_OFFLOAD_MBUF_NOFF_F BIT(3) #define NIX_TX_OFFLOAD_TSTAMP_F BIT(4) #define NIX_TX_OFFLOAD_TSO_F BIT(5) +#define NIX_TX_OFFLOAD_SECURITY_F BIT(6) /* Flags to control xmit_prepare function. * Defining it from backwards to denote its been @@ -470,136 +471,274 @@ otx2_nix_xmit_mseg_one(uint64_t *cmd, void *lmt_addr, #define NOFF_F NIX_TX_OFFLOAD_MBUF_NOFF_F #define TSP_F NIX_TX_OFFLOAD_TSTAMP_F #define TSO_F NIX_TX_OFFLOAD_TSO_F +#define TX_SEC_F NIX_TX_OFFLOAD_SECURITY_F -/* [TSO] [TSTMP] [NOFF] [VLAN] [OL3OL4CSUM] [L3L4CSUM] */ +/* [SEC] [TSO] [TSTMP] [NOFF] [VLAN] [OL3OL4CSUM] [L3L4CSUM] */ #define NIX_TX_FASTPATH_MODES \ -T(no_offload, 0, 0, 0, 0, 0, 0, 4, \ +T(no_offload, 0, 0, 0, 0, 0, 0, 0, 4, \ NIX_TX_OFFLOAD_NONE) \ -T(l3l4csum, 0, 0, 0, 0, 0, 1, 4, \ +T(l3l4csum, 0, 0, 0, 0, 0, 0, 1, 4, \ L3L4CSUM_F) \ -T(ol3ol4csum, 0, 0, 0, 0, 1, 0, 4, \ +T(ol3ol4csum, 0, 0, 0, 0, 0, 1, 0, 4, \ OL3OL4CSUM_F) \ -T(ol3ol4csum_l3l4csum, 0, 0, 0, 0, 1, 1, 4, \ +T(ol3ol4csum_l3l4csum, 0, 0, 0, 0, 0, 1, 1, 4, \ OL3OL4CSUM_F | L3L4CSUM_F) \ -T(vlan, 0, 0, 0, 1, 0, 0, 6, \ +T(vlan, 0, 0, 0, 0, 1, 0, 0, 6, \ VLAN_F) \ -T(vlan_l3l4csum, 0, 0, 0, 1, 0, 1, 6, \ +T(vlan_l3l4csum, 0, 0, 0, 0, 1, 0, 1, 6, \ VLAN_F | L3L4CSUM_F) \ -T(vlan_ol3ol4csum, 0, 0, 0, 1, 1, 0, 6, \ +T(vlan_ol3ol4csum, 0, 0, 0, 0, 1, 1, 0, 6, \ VLAN_F | OL3OL4CSUM_F) \ -T(vlan_ol3ol4csum_l3l4csum, 0, 0, 0, 1, 1, 1, 6, \ +T(vlan_ol3ol4csum_l3l4csum, 0, 0, 0, 0, 1, 1, 1, 6, \ VLAN_F | OL3OL4CSUM_F | L3L4CSUM_F) \ -T(noff, 0, 0, 1, 0, 0, 0, 4, \ +T(noff, 0, 0, 0, 1, 0, 0, 0, 4, \ NOFF_F) \ -T(noff_l3l4csum, 0, 0, 1, 0, 0, 1, 4, \ +T(noff_l3l4csum, 0, 0, 0, 1, 0, 0, 1, 4, \ NOFF_F | L3L4CSUM_F) \ -T(noff_ol3ol4csum, 0, 0, 1, 0, 1, 0, 4, \ +T(noff_ol3ol4csum, 0, 0, 0, 1, 0, 1, 0, 4, \ NOFF_F | OL3OL4CSUM_F) \ -T(noff_ol3ol4csum_l3l4csum, 0, 0, 1, 0, 1, 1, 4, \ +T(noff_ol3ol4csum_l3l4csum, 0, 0, 0, 1, 0, 1, 1, 4, \ NOFF_F | OL3OL4CSUM_F | L3L4CSUM_F) \ -T(noff_vlan, 0, 0, 1, 1, 0, 0, 6, \ +T(noff_vlan, 0, 0, 0, 1, 1, 0, 0, 6, \ NOFF_F | VLAN_F) \ -T(noff_vlan_l3l4csum, 0, 0, 1, 1, 0, 1, 6, \ +T(noff_vlan_l3l4csum, 0, 0, 0, 1, 1, 0, 1, 6, \ NOFF_F | VLAN_F | L3L4CSUM_F) \ -T(noff_vlan_ol3ol4csum, 0, 0, 1, 1, 1, 0, 6, \ +T(noff_vlan_ol3ol4csum, 0, 0, 0, 1, 1, 1, 0, 6, \ NOFF_F | VLAN_F | OL3OL4CSUM_F) \ -T(noff_vlan_ol3ol4csum_l3l4csum, 0, 0, 1, 1, 1, 1, 6, \ +T(noff_vlan_ol3ol4csum_l3l4csum, 0, 0, 0, 1, 1, 1, 1, 6, \ NOFF_F | VLAN_F | OL3OL4CSUM_F | L3L4CSUM_F) \ -T(ts, 0, 1, 0, 0, 0, 0, 8, \ +T(ts, 0, 0, 1, 0, 0, 0, 0, 8, \ TSP_F) \ -T(ts_l3l4csum, 0, 1, 0, 0, 0, 1, 8, \ +T(ts_l3l4csum, 0, 0, 1, 0, 0, 0, 1, 8, \ TSP_F | L3L4CSUM_F) \ -T(ts_ol3ol4csum, 0, 1, 0, 0, 1, 0, 8, \ +T(ts_ol3ol4csum, 0, 0, 1, 0, 0, 1, 0, 8, \ TSP_F | OL3OL4CSUM_F) \ -T(ts_ol3ol4csum_l3l4csum, 0, 1, 0, 0, 1, 1, 8, \ +T(ts_ol3ol4csum_l3l4csum, 0, 0, 1, 0, 0, 1, 1, 8, \ TSP_F | OL3OL4CSUM_F | L3L4CSUM_F) \ -T(ts_vlan, 0, 1, 0, 1, 0, 0, 8, \ +T(ts_vlan, 0, 0, 1, 0, 1, 0, 0, 8, \ TSP_F | VLAN_F) \ -T(ts_vlan_l3l4csum, 0, 1, 0, 1, 0, 1, 8, \ +T(ts_vlan_l3l4csum, 0, 0, 1, 0, 1, 0, 1, 8, \ TSP_F | VLAN_F | L3L4CSUM_F) \ -T(ts_vlan_ol3ol4csum, 0, 1, 0, 1, 1, 0, 8, \ +T(ts_vlan_ol3ol4csum, 0, 0, 1, 0, 1, 1, 0, 8, \ TSP_F | VLAN_F | OL3OL4CSUM_F) \ -T(ts_vlan_ol3ol4csum_l3l4csum, 0, 1, 0, 1, 1, 1, 8, \ +T(ts_vlan_ol3ol4csum_l3l4csum, 0, 0, 1, 0, 1, 1, 1, 8, \ TSP_F | VLAN_F | OL3OL4CSUM_F | L3L4CSUM_F) \ -T(ts_noff, 0, 1, 1, 0, 0, 0, 8, \ +T(ts_noff, 0, 0, 1, 1, 0, 0, 0, 8, \ TSP_F | NOFF_F) \ -T(ts_noff_l3l4csum, 0, 1, 1, 0, 0, 1, 8, \ +T(ts_noff_l3l4csum, 0, 0, 1, 1, 0, 0, 1, 8, \ TSP_F | NOFF_F | L3L4CSUM_F) \ -T(ts_noff_ol3ol4csum, 0, 1, 1, 0, 1, 0, 8, \ +T(ts_noff_ol3ol4csum, 0, 0, 1, 1, 0, 1, 0, 8, \ TSP_F | NOFF_F | OL3OL4CSUM_F) \ -T(ts_noff_ol3ol4csum_l3l4csum, 0, 1, 1, 0, 1, 1, 8, \ +T(ts_noff_ol3ol4csum_l3l4csum, 0, 0, 1, 1, 0, 1, 1, 8, \ TSP_F | NOFF_F | OL3OL4CSUM_F | L3L4CSUM_F) \ -T(ts_noff_vlan, 0, 1, 1, 1, 0, 0, 8, \ +T(ts_noff_vlan, 0, 0, 1, 1, 1, 0, 0, 8, \ TSP_F | NOFF_F | VLAN_F) \ -T(ts_noff_vlan_l3l4csum, 0, 1, 1, 1, 0, 1, 8, \ +T(ts_noff_vlan_l3l4csum, 0, 0, 1, 1, 1, 0, 1, 8, \ TSP_F | NOFF_F | VLAN_F | L3L4CSUM_F) \ -T(ts_noff_vlan_ol3ol4csum, 0, 1, 1, 1, 1, 0, 8, \ +T(ts_noff_vlan_ol3ol4csum, 0, 0, 1, 1, 1, 1, 0, 8, \ TSP_F | NOFF_F | VLAN_F | OL3OL4CSUM_F) \ -T(ts_noff_vlan_ol3ol4csum_l3l4csum, 0, 1, 1, 1, 1, 1, 8, \ +T(ts_noff_vlan_ol3ol4csum_l3l4csum, 0, 0, 1, 1, 1, 1, 1, 8, \ TSP_F | NOFF_F | VLAN_F | OL3OL4CSUM_F | L3L4CSUM_F) \ \ -T(tso, 1, 0, 0, 0, 0, 0, 6, \ +T(tso, 0, 1, 0, 0, 0, 0, 0, 6, \ TSO_F) \ -T(tso_l3l4csum, 1, 0, 0, 0, 0, 1, 6, \ +T(tso_l3l4csum, 0, 1, 0, 0, 0, 0, 1, 6, \ TSO_F | L3L4CSUM_F) \ -T(tso_ol3ol4csum, 1, 0, 0, 0, 1, 0, 6, \ +T(tso_ol3ol4csum, 0, 1, 0, 0, 0, 1, 0, 6, \ TSO_F | OL3OL4CSUM_F) \ -T(tso_ol3ol4csum_l3l4csum, 1, 0, 0, 0, 1, 1, 6, \ +T(tso_ol3ol4csum_l3l4csum, 0, 1, 0, 0, 0, 1, 1, 6, \ TSO_F | OL3OL4CSUM_F | L3L4CSUM_F) \ -T(tso_vlan, 1, 0, 0, 1, 0, 0, 6, \ +T(tso_vlan, 0, 1, 0, 0, 1, 0, 0, 6, \ TSO_F | VLAN_F) \ -T(tso_vlan_l3l4csum, 1, 0, 0, 1, 0, 1, 6, \ +T(tso_vlan_l3l4csum, 0, 1, 0, 0, 1, 0, 1, 6, \ TSO_F | VLAN_F | L3L4CSUM_F) \ -T(tso_vlan_ol3ol4csum, 1, 0, 0, 1, 1, 0, 6, \ +T(tso_vlan_ol3ol4csum, 0, 1, 0, 0, 1, 1, 0, 6, \ TSO_F | VLAN_F | OL3OL4CSUM_F) \ -T(tso_vlan_ol3ol4csum_l3l4csum, 1, 0, 0, 1, 1, 1, 6, \ +T(tso_vlan_ol3ol4csum_l3l4csum, 0, 1, 0, 0, 1, 1, 1, 6, \ TSO_F | VLAN_F | OL3OL4CSUM_F | L3L4CSUM_F) \ -T(tso_noff, 1, 0, 1, 0, 0, 0, 6, \ +T(tso_noff, 0, 1, 0, 1, 0, 0, 0, 6, \ TSO_F | NOFF_F) \ -T(tso_noff_l3l4csum, 1, 0, 1, 0, 0, 1, 6, \ +T(tso_noff_l3l4csum, 0, 1, 0, 1, 0, 0, 1, 6, \ TSO_F | NOFF_F | L3L4CSUM_F) \ -T(tso_noff_ol3ol4csum, 1, 0, 1, 0, 1, 0, 6, \ +T(tso_noff_ol3ol4csum, 0, 1, 0, 1, 0, 1, 0, 6, \ TSO_F | NOFF_F | OL3OL4CSUM_F) \ -T(tso_noff_ol3ol4csum_l3l4csum, 1, 0, 1, 0, 1, 1, 6, \ +T(tso_noff_ol3ol4csum_l3l4csum, 0, 1, 0, 1, 0, 1, 1, 6, \ TSO_F | NOFF_F | OL3OL4CSUM_F | L3L4CSUM_F) \ -T(tso_noff_vlan, 1, 0, 1, 1, 0, 0, 6, \ +T(tso_noff_vlan, 0, 1, 0, 1, 1, 0, 0, 6, \ TSO_F | NOFF_F | VLAN_F) \ -T(tso_noff_vlan_l3l4csum, 1, 0, 1, 1, 0, 1, 6, \ +T(tso_noff_vlan_l3l4csum, 0, 1, 0, 1, 1, 0, 1, 6, \ TSO_F | NOFF_F | VLAN_F | L3L4CSUM_F) \ -T(tso_noff_vlan_ol3ol4csum, 1, 0, 1, 1, 1, 0, 6, \ +T(tso_noff_vlan_ol3ol4csum, 0, 1, 0, 1, 1, 1, 0, 6, \ TSO_F | NOFF_F | VLAN_F | OL3OL4CSUM_F) \ -T(tso_noff_vlan_ol3ol4csum_l3l4csum, 1, 0, 1, 1, 1, 1, 6, \ +T(tso_noff_vlan_ol3ol4csum_l3l4csum, 0, 1, 0, 1, 1, 1, 1, 6, \ TSO_F | NOFF_F | VLAN_F | OL3OL4CSUM_F | L3L4CSUM_F) \ -T(tso_ts, 1, 1, 0, 0, 0, 0, 8, \ +T(tso_ts, 0, 1, 1, 0, 0, 0, 0, 8, \ TSO_F | TSP_F) \ -T(tso_ts_l3l4csum, 1, 1, 0, 0, 0, 1, 8, \ +T(tso_ts_l3l4csum, 0, 1, 1, 0, 0, 0, 1, 8, \ TSO_F | TSP_F | L3L4CSUM_F) \ -T(tso_ts_ol3ol4csum, 1, 1, 0, 0, 1, 0, 8, \ +T(tso_ts_ol3ol4csum, 0, 1, 1, 0, 0, 1, 0, 8, \ TSO_F | TSP_F | OL3OL4CSUM_F) \ -T(tso_ts_ol3ol4csum_l3l4csum, 1, 1, 0, 0, 1, 1, 8, \ +T(tso_ts_ol3ol4csum_l3l4csum, 0, 1, 1, 0, 0, 1, 1, 8, \ TSO_F | TSP_F | OL3OL4CSUM_F | L3L4CSUM_F) \ -T(tso_ts_vlan, 1, 1, 0, 1, 0, 0, 8, \ +T(tso_ts_vlan, 0, 1, 1, 0, 1, 0, 0, 8, \ TSO_F | TSP_F | VLAN_F) \ -T(tso_ts_vlan_l3l4csum, 1, 1, 0, 1, 0, 1, 8, \ +T(tso_ts_vlan_l3l4csum, 0, 1, 1, 0, 1, 0, 1, 8, \ TSO_F | TSP_F | VLAN_F | L3L4CSUM_F) \ -T(tso_ts_vlan_ol3ol4csum, 1, 1, 0, 1, 1, 0, 8, \ +T(tso_ts_vlan_ol3ol4csum, 0, 1, 1, 0, 1, 1, 0, 8, \ TSO_F | TSP_F | VLAN_F | OL3OL4CSUM_F) \ -T(tso_ts_vlan_ol3ol4csum_l3l4csum, 1, 1, 0, 1, 1, 1, 8, \ +T(tso_ts_vlan_ol3ol4csum_l3l4csum, 0, 1, 1, 0, 1, 1, 1, 8, \ TSO_F | TSP_F | VLAN_F | OL3OL4CSUM_F | L3L4CSUM_F) \ -T(tso_ts_noff, 1, 1, 1, 0, 0, 0, 8, \ +T(tso_ts_noff, 0, 1, 1, 1, 0, 0, 0, 8, \ TSO_F | TSP_F | NOFF_F) \ -T(tso_ts_noff_l3l4csum, 1, 1, 1, 0, 0, 1, 8, \ +T(tso_ts_noff_l3l4csum, 0, 1, 1, 1, 0, 0, 1, 8, \ TSO_F | TSP_F | NOFF_F | L3L4CSUM_F) \ -T(tso_ts_noff_ol3ol4csum, 1, 1, 1, 0, 1, 0, 8, \ +T(tso_ts_noff_ol3ol4csum, 0, 1, 1, 1, 0, 1, 0, 8, \ TSO_F | TSP_F | NOFF_F | OL3OL4CSUM_F) \ -T(tso_ts_noff_ol3ol4csum_l3l4csum, 1, 1, 1, 0, 1, 1, 8, \ +T(tso_ts_noff_ol3ol4csum_l3l4csum, 0, 1, 1, 1, 0, 1, 1, 8, \ TSO_F | TSP_F | NOFF_F | OL3OL4CSUM_F | L3L4CSUM_F) \ -T(tso_ts_noff_vlan, 1, 1, 1, 1, 0, 0, 8, \ +T(tso_ts_noff_vlan, 0, 1, 1, 1, 1, 0, 0, 8, \ TSO_F | TSP_F | NOFF_F | VLAN_F) \ -T(tso_ts_noff_vlan_l3l4csum, 1, 1, 1, 1, 0, 1, 8, \ +T(tso_ts_noff_vlan_l3l4csum, 0, 1, 1, 1, 1, 0, 1, 8, \ TSO_F | TSP_F | NOFF_F | VLAN_F | L3L4CSUM_F) \ -T(tso_ts_noff_vlan_ol3ol4csum, 1, 1, 1, 1, 1, 0, 8, \ +T(tso_ts_noff_vlan_ol3ol4csum, 0, 1, 1, 1, 1, 1, 0, 8, \ TSO_F | TSP_F | NOFF_F | VLAN_F | OL3OL4CSUM_F) \ -T(tso_ts_noff_vlan_ol3ol4csum_l3l4csum, 1, 1, 1, 1, 1, 1, 8, \ - TSO_F | TSP_F | NOFF_F | VLAN_F | OL3OL4CSUM_F | L3L4CSUM_F) +T(tso_ts_noff_vlan_ol3ol4csum_l3l4csum, 0, 1, 1, 1, 1, 1, 1, 8, \ + TSO_F | TSP_F | NOFF_F | VLAN_F | OL3OL4CSUM_F | \ + L3L4CSUM_F) \ +T(sec, 1, 0, 0, 0, 0, 0, 0, 8, \ + TX_SEC_F) \ +T(sec_l3l4csum, 1, 0, 0, 0, 0, 0, 1, 8, \ + TX_SEC_F | L3L4CSUM_F) \ +T(sec_ol3ol4csum, 1, 0, 0, 0, 0, 1, 0, 8, \ + TX_SEC_F | OL3OL4CSUM_F) \ +T(sec_ol3ol4csum_l3l4csum, 1, 0, 0, 0, 0, 1, 1, 8, \ + TX_SEC_F | OL3OL4CSUM_F | L3L4CSUM_F) \ +T(sec_vlan, 1, 0, 0, 0, 1, 0, 0, 8, \ + TX_SEC_F | VLAN_F) \ +T(sec_vlan_l3l4csum, 1, 0, 0, 0, 1, 0, 1, 8, \ + TX_SEC_F | VLAN_F | L3L4CSUM_F) \ +T(sec_vlan_ol3ol4csum, 1, 0, 0, 0, 1, 1, 0, 8, \ + TX_SEC_F | VLAN_F | OL3OL4CSUM_F) \ +T(sec_vlan_ol3ol4csum_l3l4csum, 1, 0, 0, 0, 1, 1, 1, 8, \ + TX_SEC_F | VLAN_F | OL3OL4CSUM_F | L3L4CSUM_F) \ +T(sec_noff, 1, 0, 0, 1, 0, 0, 0, 8, \ + TX_SEC_F | NOFF_F) \ +T(sec_noff_l3l4csum, 1, 0, 0, 1, 0, 0, 1, 8, \ + TX_SEC_F | NOFF_F | L3L4CSUM_F) \ +T(sec_noff_ol3ol4csum, 1, 0, 0, 1, 0, 1, 0, 8, \ + TX_SEC_F | NOFF_F | OL3OL4CSUM_F) \ +T(sec_noff_ol3ol4csum_l3l4csum, 1, 0, 0, 1, 0, 1, 1, 8, \ + TX_SEC_F | NOFF_F | OL3OL4CSUM_F | L3L4CSUM_F) \ +T(sec_noff_vlan, 1, 0, 0, 1, 1, 0, 0, 8, \ + TX_SEC_F | NOFF_F | VLAN_F) \ +T(sec_noff_vlan_l3l4csum, 1, 0, 0, 1, 1, 0, 1, 8, \ + TX_SEC_F | NOFF_F | VLAN_F | L3L4CSUM_F) \ +T(sec_noff_vlan_ol3ol4csum, 1, 0, 0, 1, 1, 1, 0, 8, \ + TX_SEC_F | NOFF_F | VLAN_F | OL3OL4CSUM_F) \ +T(sec_noff_vlan_ol3ol4csum_l3l4csum, 1, 0, 0, 1, 1, 1, 1, 8, \ + TX_SEC_F | NOFF_F | VLAN_F | OL3OL4CSUM_F | L3L4CSUM_F) \ +T(sec_ts, 1, 0, 1, 0, 0, 0, 0, 8, \ + TX_SEC_F | TSP_F) \ +T(sec_ts_l3l4csum, 1, 0, 1, 0, 0, 0, 1, 8, \ + TX_SEC_F | TSP_F | L3L4CSUM_F) \ +T(sec_ts_ol3ol4csum, 1, 0, 1, 0, 0, 1, 0, 8, \ + TX_SEC_F | TSP_F | OL3OL4CSUM_F) \ +T(sec_ts_ol3ol4csum_l3l4csum, 1, 0, 1, 0, 0, 1, 1, 8, \ + TX_SEC_F | TSP_F | OL3OL4CSUM_F | L3L4CSUM_F) \ +T(sec_ts_vlan, 1, 0, 1, 0, 1, 0, 0, 8, \ + TX_SEC_F | TSP_F | VLAN_F) \ +T(sec_ts_vlan_l3l4csum, 1, 0, 1, 0, 1, 0, 1, 8, \ + TX_SEC_F | TSP_F | VLAN_F | L3L4CSUM_F) \ +T(sec_ts_vlan_ol3ol4csum, 1, 0, 1, 0, 1, 1, 0, 8, \ + TX_SEC_F | TSP_F | VLAN_F | OL3OL4CSUM_F) \ +T(sec_ts_vlan_ol3ol4csum_l3l4csum, 1, 0, 1, 0, 1, 1, 1, 8, \ + TX_SEC_F | TSP_F | VLAN_F | OL3OL4CSUM_F | L3L4CSUM_F) \ +T(sec_ts_noff, 1, 0, 1, 1, 0, 0, 0, 8, \ + TX_SEC_F | TSP_F | NOFF_F) \ +T(sec_ts_noff_l3l4csum, 1, 0, 1, 1, 0, 0, 1, 8, \ + TX_SEC_F | TSP_F | NOFF_F | L3L4CSUM_F) \ +T(sec_ts_noff_ol3ol4csum, 1, 0, 1, 1, 0, 1, 0, 8, \ + TX_SEC_F | TSP_F | NOFF_F | OL3OL4CSUM_F) \ +T(sec_ts_noff_ol3ol4csum_l3l4csum, 1, 0, 1, 1, 0, 1, 1, 8, \ + TX_SEC_F | TSP_F | NOFF_F | OL3OL4CSUM_F | L3L4CSUM_F) \ +T(sec_ts_noff_vlan, 1, 0, 1, 1, 1, 0, 0, 8, \ + TX_SEC_F | TSP_F | NOFF_F | VLAN_F) \ +T(sec_ts_noff_vlan_l3l4csum, 1, 0, 1, 1, 1, 0, 1, 8, \ + TX_SEC_F | TSP_F | NOFF_F | VLAN_F | L3L4CSUM_F) \ +T(sec_ts_noff_vlan_ol3ol4csum, 1, 0, 1, 1, 1, 1, 0, 8, \ + TX_SEC_F | TSP_F | NOFF_F | VLAN_F | OL3OL4CSUM_F) \ +T(sec_ts_noff_vlan_ol3ol4csum_l3l4csum, 1, 0, 1, 1, 1, 1, 1, 8, \ + TX_SEC_F | TSP_F | NOFF_F | VLAN_F | OL3OL4CSUM_F | \ + L3L4CSUM_F) \ +T(sec_tso, 1, 1, 0, 0, 0, 0, 0, 8, \ + TX_SEC_F | TSO_F) \ +T(sec_tso_l3l4csum, 1, 1, 0, 0, 0, 0, 1, 8, \ + TX_SEC_F | TSO_F | L3L4CSUM_F) \ +T(sec_tso_ol3ol4csum, 1, 1, 0, 0, 0, 1, 0, 8, \ + TX_SEC_F | TSO_F | OL3OL4CSUM_F) \ +T(sec_tso_ol3ol4csum_l3l4csum, 1, 1, 0, 0, 0, 1, 1, 8, \ + TX_SEC_F | TSO_F | OL3OL4CSUM_F | L3L4CSUM_F) \ +T(sec_tso_vlan, 1, 1, 0, 0, 1, 0, 0, 8, \ + TX_SEC_F | TSO_F | VLAN_F) \ +T(sec_tso_vlan_l3l4csum, 1, 1, 0, 0, 1, 0, 1, 8, \ + TX_SEC_F | TSO_F | VLAN_F | L3L4CSUM_F) \ +T(sec_tso_vlan_ol3ol4csum, 1, 1, 0, 0, 1, 1, 0, 8, \ + TX_SEC_F | TSO_F | VLAN_F | OL3OL4CSUM_F) \ +T(sec_tso_vlan_ol3ol4csum_l3l4csum, 1, 1, 0, 0, 1, 1, 1, 8, \ + TX_SEC_F | TSO_F | VLAN_F | OL3OL4CSUM_F | L3L4CSUM_F) \ +T(sec_tso_noff, 1, 1, 0, 1, 0, 0, 0, 8, \ + TX_SEC_F | TSO_F | NOFF_F) \ +T(sec_tso_noff_l3l4csum, 1, 1, 0, 1, 0, 0, 1, 8, \ + TX_SEC_F | TSO_F | NOFF_F | L3L4CSUM_F) \ +T(sec_tso_noff_ol3ol4csum, 1, 1, 0, 1, 0, 1, 0, 8, \ + TX_SEC_F | TSO_F | NOFF_F | OL3OL4CSUM_F) \ +T(sec_tso_noff_ol3ol4csum_l3l4csum, 1, 1, 0, 1, 0, 1, 1, 8, \ + TX_SEC_F | TSO_F | NOFF_F | OL3OL4CSUM_F | L3L4CSUM_F) \ +T(sec_tso_noff_vlan, 1, 1, 0, 1, 1, 0, 0, 8, \ + TX_SEC_F | TSO_F | NOFF_F | VLAN_F) \ +T(sec_tso_noff_vlan_l3l4csum, 1, 1, 0, 1, 1, 0, 1, 8, \ + TX_SEC_F | TSO_F | NOFF_F | VLAN_F | L3L4CSUM_F) \ +T(sec_tso_noff_vlan_ol3ol4csum, 1, 1, 0, 1, 1, 1, 0, 8, \ + TX_SEC_F | TSO_F | NOFF_F | VLAN_F | OL3OL4CSUM_F) \ +T(sec_tso_noff_vlan_ol3ol4csum_l3l4csum, \ + 1, 1, 0, 1, 1, 1, 1, 8, \ + TX_SEC_F | TSO_F | NOFF_F | VLAN_F | OL3OL4CSUM_F | \ + L3L4CSUM_F) \ +T(sec_tso_ts, 1, 1, 1, 0, 0, 0, 0, 8, \ + TX_SEC_F | TSO_F | TSP_F) \ +T(sec_tso_ts_l3l4csum, 1, 1, 1, 0, 0, 0, 1, 8, \ + TX_SEC_F | TSO_F | TSP_F | L3L4CSUM_F) \ +T(sec_tso_ts_ol3ol4csum, 1, 1, 1, 0, 0, 1, 0, 8, \ + TX_SEC_F | TSO_F | TSP_F | OL3OL4CSUM_F) \ +T(sec_tso_ts_ol3ol4csum_l3l4csum, 1, 1, 1, 0, 0, 1, 1, 8, \ + TX_SEC_F | TSO_F | TSP_F | OL3OL4CSUM_F | L3L4CSUM_F) \ +T(sec_tso_ts_vlan, 1, 1, 1, 0, 1, 0, 0, 8, \ + TX_SEC_F | TSO_F | TSP_F | VLAN_F) \ +T(sec_tso_ts_vlan_l3l4csum, 1, 1, 1, 0, 1, 0, 1, 8, \ + TX_SEC_F | TSO_F | TSP_F | VLAN_F | L3L4CSUM_F) \ +T(sec_tso_ts_vlan_ol3ol4csum, 1, 1, 1, 0, 1, 1, 0, 8, \ + TX_SEC_F | TSO_F | TSP_F | VLAN_F | OL3OL4CSUM_F) \ +T(sec_tso_ts_vlan_ol3ol4csum_l3l4csum, 1, 1, 1, 0, 1, 1, 1, 8, \ + TX_SEC_F | TSO_F | TSP_F | VLAN_F | OL3OL4CSUM_F | \ + L3L4CSUM_F) \ +T(sec_tso_ts_noff, 1, 1, 1, 1, 0, 0, 0, 8, \ + TX_SEC_F | TSO_F | TSP_F | NOFF_F) \ +T(sec_tso_ts_noff_l3l4csum, 1, 1, 1, 1, 0, 0, 1, 8, \ + TX_SEC_F | TSO_F | TSP_F | NOFF_F | L3L4CSUM_F) \ +T(sec_tso_ts_noff_ol3ol4csum, 1, 1, 1, 1, 0, 1, 0, 8, \ + TX_SEC_F | TSO_F | TSP_F | NOFF_F | OL3OL4CSUM_F) \ +T(sec_tso_ts_noff_ol3ol4csum_l3l4csum, 1, 1, 1, 1, 0, 1, 1, 8, \ + TX_SEC_F | TSO_F | TSP_F | NOFF_F | OL3OL4CSUM_F | \ + L3L4CSUM_F) \ +T(sec_tso_ts_noff_vlan, 1, 1, 1, 1, 1, 0, 0, 8, \ + TX_SEC_F | TSO_F | TSP_F | NOFF_F | VLAN_F) \ +T(sec_tso_ts_noff_vlan_l3l4csum, 1, 1, 1, 1, 1, 0, 1, 8, \ + TX_SEC_F | TSO_F | TSP_F | NOFF_F | VLAN_F | L3L4CSUM_F)\ +T(sec_tso_ts_noff_vlan_ol3ol4csum, 1, 1, 1, 1, 1, 1, 0, 8, \ + TX_SEC_F | TSO_F | TSP_F | NOFF_F | VLAN_F | \ + OL3OL4CSUM_F) \ +T(sec_tso_ts_noff_vlan_ol3ol4csum_l3l4csum, \ + 1, 1, 1, 1, 1, 1, 1, 8, \ + TX_SEC_F | TSO_F | TSP_F | NOFF_F | VLAN_F | \ + OL3OL4CSUM_F | L3L4CSUM_F) #endif /* __OTX2_TX_H__ */ From patchwork Sat Jan 18 10:49:12 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Anoob Joseph X-Patchwork-Id: 64870 X-Patchwork-Delegate: gakhil@marvell.com Return-Path: X-Original-To: patchwork@inbox.dpdk.org Delivered-To: patchwork@inbox.dpdk.org Received: from dpdk.org (dpdk.org [92.243.14.124]) by inbox.dpdk.org (Postfix) with ESMTP id 53726A051C; Sat, 18 Jan 2020 11:51:46 +0100 (CET) Received: from [92.243.14.124] (localhost [127.0.0.1]) by dpdk.org (Postfix) with ESMTP id 85C461BF9B; Sat, 18 Jan 2020 11:51:13 +0100 (CET) Received: from mx0b-0016f401.pphosted.com (mx0b-0016f401.pphosted.com [67.231.156.173]) by dpdk.org (Postfix) with ESMTP id 7A7D81BF8E for ; Sat, 18 Jan 2020 11:51:11 +0100 (CET) Received: from pps.filterd (m0045851.ppops.net [127.0.0.1]) by mx0b-0016f401.pphosted.com (8.16.0.42/8.16.0.42) with SMTP id 00IAnc9q021837; Sat, 18 Jan 2020 02:51:10 -0800 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=marvell.com; h=from : to : cc : subject : date : message-id : in-reply-to : references : mime-version : content-transfer-encoding : content-type; s=pfpt0818; bh=ON+zhd9CkOqzknxkfeDPayCmlbS2Wt/2DshTbi64Wzs=; b=hqmvSS5UNicbHkeizdYKCP8hLCrHJR2kAsRRCdRCpTyh06HCkPXy3oAPEoCpxYBVg+0b HIVHZIVUTEwciqWWyior3+P50R1iV8JHL8RYkMYxrdq3iVAo2MG8F2QGpTQHzJjQtfZ1 SDJPuszcU5G0ZzrUdrUt0xRtgrGdXT2cbBix6jC2ENqn3XF0i1MlNB21Ci0ZrNV252h0 tw1eQTFrwhVKJicUgLQ1au15IGG4edUSvOFC7tB8siaeBeX2Coupc+rSKuP/PexfvKRa BZqqzGkaZZZCwxiJUGk47xLLugLtvG4l6BDYvm4Ei6lDYleKIOkrghrYCL21RVGFQkXr 9g== Received: from sc-exch03.marvell.com ([199.233.58.183]) by mx0b-0016f401.pphosted.com with ESMTP id 2xk0sn6ejr-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-SHA384 bits=256 verify=NOT); Sat, 18 Jan 2020 02:51:10 -0800 Received: from SC-EXCH01.marvell.com (10.93.176.81) by SC-EXCH03.marvell.com (10.93.176.83) with Microsoft SMTP Server (TLS) id 15.0.1497.2; Sat, 18 Jan 2020 02:51:08 -0800 Received: from maili.marvell.com (10.93.176.43) by SC-EXCH01.marvell.com (10.93.176.81) with Microsoft SMTP Server id 15.0.1497.2 via Frontend Transport; Sat, 18 Jan 2020 02:51:08 -0800 Received: from ajoseph83.caveonetworks.com (unknown [10.29.45.60]) by maili.marvell.com (Postfix) with ESMTP id 1A90E3F7044; Sat, 18 Jan 2020 02:51:03 -0800 (PST) From: Anoob Joseph To: Akhil Goyal , Declan Doherty , Thomas Monjalon CC: Ankur Dwivedi , Jerin Jacob , Narayana Prasad , Kiran Kumar K , Nithin Dabilpuram , "Pavan Nikhilesh" , Anoob Joseph , Archana Muniganti , Tejasree Kondoj , Vamsi Attunuru , "Lukasz Bartosik" , Date: Sat, 18 Jan 2020 16:19:12 +0530 Message-ID: <1579344553-11428-15-git-send-email-anoobj@marvell.com> X-Mailer: git-send-email 2.7.4 In-Reply-To: <1579344553-11428-1-git-send-email-anoobj@marvell.com> References: <1575806094-28391-1-git-send-email-anoobj@marvell.com> <1579344553-11428-1-git-send-email-anoobj@marvell.com> MIME-Version: 1.0 X-Proofpoint-Virus-Version: vendor=fsecure engine=2.50.10434:6.0.138, 18.0.572 definitions=2020-01-18_02:2020-01-16, 2020-01-18 signatures=0 Subject: [dpdk-dev] [PATCH v2 14/15] crypto/octeontx2: add inline tx path changes X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Sender: "dev" From: Ankur Dwivedi Adding pre-processing required for inline IPsec outbound packets. Signed-off-by: Ankur Dwivedi Signed-off-by: Anoob Joseph Signed-off-by: Archana Muniganti Signed-off-by: Tejasree Kondoj Signed-off-by: Vamsi Attunuru --- drivers/crypto/octeontx2/otx2_security.c | 82 +++++++++++++ drivers/crypto/octeontx2/otx2_security.h | 60 ++++++++++ drivers/crypto/octeontx2/otx2_security_tx.h | 175 ++++++++++++++++++++++++++++ drivers/event/octeontx2/meson.build | 3 +- drivers/event/octeontx2/otx2_worker.h | 6 + 5 files changed, 325 insertions(+), 1 deletion(-) create mode 100644 drivers/crypto/octeontx2/otx2_security_tx.h diff --git a/drivers/crypto/octeontx2/otx2_security.c b/drivers/crypto/octeontx2/otx2_security.c index ab488a0..9a08849 100644 --- a/drivers/crypto/octeontx2/otx2_security.c +++ b/drivers/crypto/octeontx2/otx2_security.c @@ -3,12 +3,15 @@ */ #include +#include #include #include +#include #include #include #include #include +#include #include "otx2_common.h" #include "otx2_cryptodev_qp.h" @@ -18,6 +21,15 @@ #define SEC_ETH_MAX_PKT_LEN 1450 +#define AH_HDR_LEN 12 +#define AES_GCM_IV_LEN 8 +#define AES_GCM_MAC_LEN 16 +#define AES_CBC_IV_LEN 16 +#define SHA1_HMAC_LEN 12 + +#define AES_GCM_ROUNDUP_BYTE_LEN 4 +#define AES_CBC_ROUNDUP_BYTE_LEN 16 + struct sec_eth_tag_const { RTE_STD_C11 union { @@ -239,6 +251,60 @@ in_sa_get(uint16_t port, int sa_index) } static int +ipsec_sa_const_set(struct rte_security_ipsec_xform *ipsec, + struct rte_crypto_sym_xform *xform, + struct otx2_sec_session_ipsec_ip *sess) +{ + struct rte_crypto_sym_xform *cipher_xform, *auth_xform; + + sess->partial_len = sizeof(struct rte_ipv4_hdr); + + if (ipsec->proto == RTE_SECURITY_IPSEC_SA_PROTO_ESP) { + sess->partial_len += sizeof(struct rte_esp_hdr); + sess->roundup_len = sizeof(struct rte_esp_tail); + } else if (ipsec->proto == RTE_SECURITY_IPSEC_SA_PROTO_AH) { + sess->partial_len += AH_HDR_LEN; + } else { + return -EINVAL; + } + + if (ipsec->options.udp_encap) + sess->partial_len += sizeof(struct rte_udp_hdr); + + if (xform->type == RTE_CRYPTO_SYM_XFORM_AEAD) { + if (xform->aead.algo == RTE_CRYPTO_AEAD_AES_GCM) { + sess->partial_len += AES_GCM_IV_LEN; + sess->partial_len += AES_GCM_MAC_LEN; + sess->roundup_byte = AES_GCM_ROUNDUP_BYTE_LEN; + } + return 0; + } + + if (ipsec->direction == RTE_SECURITY_IPSEC_SA_DIR_EGRESS) { + cipher_xform = xform; + auth_xform = xform->next; + } else if (ipsec->direction == RTE_SECURITY_IPSEC_SA_DIR_INGRESS) { + auth_xform = xform; + cipher_xform = xform->next; + } else { + return -EINVAL; + } + if (cipher_xform->cipher.algo == RTE_CRYPTO_CIPHER_AES_CBC) { + sess->partial_len += AES_CBC_IV_LEN; + sess->roundup_byte = AES_CBC_ROUNDUP_BYTE_LEN; + } else { + return -EINVAL; + } + + if (auth_xform->auth.algo == RTE_CRYPTO_AUTH_SHA1_HMAC) + sess->partial_len += SHA1_HMAC_LEN; + else + return -EINVAL; + + return 0; +} + +static int sec_eth_ipsec_out_sess_create(struct rte_eth_dev *eth_dev, struct rte_security_ipsec_xform *ipsec, struct rte_crypto_sym_xform *crypto_xform, @@ -252,6 +318,7 @@ sec_eth_ipsec_out_sess_create(struct rte_eth_dev *eth_dev, struct otx2_ipsec_fp_sa_ctl *ctl; struct otx2_ipsec_fp_out_sa *sa; struct otx2_sec_session *priv; + struct otx2_cpt_inst_s inst; struct otx2_cpt_qp *qp; priv = get_sec_session_private_data(sec_sess); @@ -266,6 +333,12 @@ sec_eth_ipsec_out_sess_create(struct rte_eth_dev *eth_dev, memset(sess, 0, sizeof(struct otx2_sec_session_ipsec_ip)); + sess->seq = 1; + + ret = ipsec_sa_const_set(ipsec, crypto_xform, sess); + if (ret < 0) + return ret; + memcpy(sa->nonce, &ipsec->salt, 4); if (ipsec->options.udp_encap == 1) { @@ -274,6 +347,9 @@ sec_eth_ipsec_out_sess_create(struct rte_eth_dev *eth_dev, } if (ipsec->mode == RTE_SECURITY_IPSEC_SA_MODE_TUNNEL) { + /* Start ip id from 1 */ + sess->ip_id = 1; + if (ipsec->tunnel.type == RTE_SECURITY_IPSEC_TUNNEL_IPV4) { memcpy(&sa->ip_src, &ipsec->tunnel.ipv4.src_ip, sizeof(struct in_addr)); @@ -307,6 +383,12 @@ sec_eth_ipsec_out_sess_create(struct rte_eth_dev *eth_dev, else return -EINVAL; + /* Determine word 7 of CPT instruction */ + inst.u64[7] = 0; + inst.egrp = OTX2_CPT_EGRP_INLINE_IPSEC; + inst.cptr = rte_mempool_virt2iova(sa); + sess->inst_w7 = inst.u64[7]; + /* Use OPAD & IPAD */ RTE_SET_USED(auth_key); RTE_SET_USED(auth_key_len); diff --git a/drivers/crypto/octeontx2/otx2_security.h b/drivers/crypto/octeontx2/otx2_security.h index 6ec321d..fe7c883 100644 --- a/drivers/crypto/octeontx2/otx2_security.h +++ b/drivers/crypto/octeontx2/otx2_security.h @@ -14,6 +14,15 @@ #define OTX2_MAX_CPT_QP_PER_PORT 64 #define OTX2_MAX_INLINE_PORTS 64 +#define OTX2_CPT_RES_ALIGN 16 +#define OTX2_NIX_SEND_DESC_ALIGN 16 +#define OTX2_CPT_INST_SIZE 64 + +#define OTX2_CPT_EGRP_INLINE_IPSEC 1 + +#define OTX2_CPT_OP_INLINE_IPSEC_OUTB (0x40 | 0x25) +#define OTX2_CPT_OP_INLINE_IPSEC_INB (0x40 | 0x26) + struct otx2_cpt_qp; struct otx2_sec_eth_cfg { @@ -45,6 +54,42 @@ struct otx2_cpt_res { }; }; +struct otx2_cpt_inst_s { + union { + struct { + /* W0 */ + uint64_t nixtxl : 3; + uint64_t doneint : 1; + uint64_t nixtx_addr : 60; + /* W1 */ + uint64_t res_addr : 64; + /* W2 */ + uint64_t tag : 32; + uint64_t tt : 2; + uint64_t grp : 10; + uint64_t rsvd_175_172 : 4; + uint64_t rvu_pf_func : 16; + /* W3 */ + uint64_t qord : 1; + uint64_t rsvd_194_193 : 2; + uint64_t wqe_ptr : 61; + /* W4 */ + uint64_t dlen : 16; + uint64_t param2 : 16; + uint64_t param1 : 16; + uint64_t opcode : 16; + /* W5 */ + uint64_t dptr : 64; + /* W6 */ + uint64_t rptr : 64; + /* W7 */ + uint64_t cptr : 61; + uint64_t egrp : 3; + }; + uint64_t u64[8]; + }; +}; + /* * Security session for inline IPsec protocol offload. This is private data of * inline capable PMD. @@ -68,6 +113,21 @@ struct otx2_sec_session_ipsec_ip { /* CPT LF enqueue register address */ rte_iova_t cpt_nq_reg; + /* Pre calculated lengths and data for a session */ + uint8_t partial_len; + uint8_t roundup_len; + uint8_t roundup_byte; + uint16_t ip_id; + union { + uint64_t esn; + struct { + uint32_t seq; + uint32_t esn_hi; + }; + }; + + uint64_t inst_w7; + /* CPT QP used by SA */ struct otx2_cpt_qp *qp; }; diff --git a/drivers/crypto/octeontx2/otx2_security_tx.h b/drivers/crypto/octeontx2/otx2_security_tx.h new file mode 100644 index 0000000..16b8c66 --- /dev/null +++ b/drivers/crypto/octeontx2/otx2_security_tx.h @@ -0,0 +1,175 @@ +/* SPDX-License-Identifier: BSD-3-Clause + * Copyright(C) 2020 Marvell International Ltd. + */ + +#ifndef __OTX2_SECURITY_TX_H__ +#define __OTX2_SECURITY_TX_H__ + +#include +#include + +#include "otx2_security.h" + +struct otx2_ipsec_fp_out_hdr { + uint32_t ip_id; + uint32_t seq; + uint8_t iv[16]; +}; + +static __rte_always_inline int32_t +otx2_ipsec_fp_out_rlen_get(struct otx2_sec_session_ipsec_ip *sess, + uint32_t plen) +{ + uint32_t enc_payload_len; + + enc_payload_len = RTE_ALIGN_CEIL(plen + sess->roundup_len, + sess->roundup_byte); + + return sess->partial_len + enc_payload_len; +} + +static __rte_always_inline void +otx2_ssogws_head_wait(struct otx2_ssogws *ws); + +static __rte_always_inline int +otx2_sec_event_tx(struct otx2_ssogws *ws, struct rte_event *ev, + struct rte_mbuf *m, const struct otx2_eth_txq *txq, + const uint32_t offload_flags) +{ + uint32_t dlen, rlen, desc_headroom, extend_head, extend_tail; + struct otx2_sec_session_ipsec_ip *sess; + struct otx2_ipsec_fp_out_hdr *hdr; + struct otx2_ipsec_fp_out_sa *sa; + uint64_t data_addr, desc_addr; + struct otx2_sec_session *priv; + struct otx2_cpt_inst_s inst; + uint64_t lmt_status; + char *data; + + struct desc { + struct otx2_cpt_res cpt_res __rte_aligned(OTX2_CPT_RES_ALIGN); + struct nix_send_hdr_s nix_hdr + __rte_aligned(OTX2_NIX_SEND_DESC_ALIGN); + union nix_send_sg_s nix_sg; + struct nix_iova_s nix_iova; + } *sd; + + priv = get_sec_session_private_data((void *)(m->udata64)); + sess = &priv->ipsec.ip; + sa = &sess->out_sa; + + RTE_ASSERT(sess->cpt_lmtline != NULL); + RTE_ASSERT(!(offload_flags & (NIX_TX_OFFLOAD_MBUF_NOFF_F | + NIX_TX_OFFLOAD_VLAN_QINQ))); + + dlen = rte_pktmbuf_pkt_len(m) + sizeof(*hdr) - RTE_ETHER_HDR_LEN; + rlen = otx2_ipsec_fp_out_rlen_get(sess, dlen - sizeof(*hdr)); + + RTE_BUILD_BUG_ON(OTX2_CPT_RES_ALIGN % OTX2_NIX_SEND_DESC_ALIGN); + RTE_BUILD_BUG_ON(sizeof(sd->cpt_res) % OTX2_NIX_SEND_DESC_ALIGN); + + extend_head = sizeof(*hdr); + extend_tail = rlen - dlen; + + desc_headroom = (OTX2_CPT_RES_ALIGN - 1) + sizeof(*sd); + + if (unlikely(!rte_pktmbuf_is_contiguous(m)) || + unlikely(rte_pktmbuf_headroom(m) < extend_head + desc_headroom) || + unlikely(rte_pktmbuf_tailroom(m) < extend_tail)) { + goto drop; + } + + /* + * Extend mbuf data to point to the expected packet buffer for NIX. + * This includes the Ethernet header followed by the encrypted IPsec + * payload + */ + rte_pktmbuf_append(m, extend_tail); + data = rte_pktmbuf_prepend(m, extend_head); + data_addr = rte_pktmbuf_mtophys(m); + + /* + * Move the Ethernet header, to insert otx2_ipsec_fp_out_hdr prior + * to the IP header + */ + memcpy(data, data + sizeof(*hdr), RTE_ETHER_HDR_LEN); + + hdr = (struct otx2_ipsec_fp_out_hdr *)(data + RTE_ETHER_HDR_LEN); + + memcpy(hdr->iv, &sa->nonce, 4); + memset(hdr->iv + 4, 0, 12); //TODO: make it random + + /* Keep CPT result and NIX send descriptors in headroom */ + sd = (void *)RTE_PTR_ALIGN(data - desc_headroom, OTX2_CPT_RES_ALIGN); + desc_addr = data_addr - RTE_PTR_DIFF(data, sd); + + /* Prepare CPT instruction */ + + inst.nixtx_addr = (desc_addr + offsetof(struct desc, nix_hdr)) >> 4; + inst.doneint = 0; + inst.nixtxl = 1; + inst.res_addr = desc_addr + offsetof(struct desc, cpt_res); + inst.u64[2] = 0; + inst.u64[3] = 0; + inst.wqe_ptr = desc_addr >> 3; /* FIXME: Handle errors */ + inst.qord = 1; + inst.opcode = OTX2_CPT_OP_INLINE_IPSEC_OUTB; + inst.dlen = dlen; + inst.dptr = data_addr + RTE_ETHER_HDR_LEN; + inst.u64[7] = sess->inst_w7; + + /* First word contains 8 bit completion code & 8 bit uc comp code */ + sd->cpt_res.u16[0] = 0; + + /* Prepare NIX send descriptors for output expected from CPT */ + + sd->nix_hdr.w0.u = 0; + sd->nix_hdr.w1.u = 0; + sd->nix_hdr.w0.sq = txq->sq; + sd->nix_hdr.w0.sizem1 = 1; + sd->nix_hdr.w0.total = rte_pktmbuf_data_len(m); + sd->nix_hdr.w0.aura = npa_lf_aura_handle_to_aura(m->pool->pool_id); + + sd->nix_sg.u = 0; + sd->nix_sg.subdc = NIX_SUBDC_SG; + sd->nix_sg.ld_type = NIX_SENDLDTYPE_LDD; + sd->nix_sg.segs = 1; + sd->nix_sg.seg1_size = rte_pktmbuf_data_len(m); + + sd->nix_iova.addr = rte_mbuf_data_iova(m); + + /* Mark mempool object as "put" since it is freed by NIX */ + __mempool_check_cookies(m->pool, (void **)&m, 1, 0); + + if (!ev->sched_type) + otx2_ssogws_head_wait(ws); + + inst.param1 = sess->esn_hi >> 16; + inst.param2 = sess->esn_hi & 0xffff; + + hdr->seq = rte_cpu_to_be_32(sess->seq); + hdr->ip_id = rte_cpu_to_be_32(sess->ip_id); + + sess->ip_id++; + sess->esn++; + + rte_cio_wmb(); + + do { + otx2_lmt_mov(sess->cpt_lmtline, &inst, 2); + lmt_status = otx2_lmt_submit(sess->cpt_nq_reg); + } while (lmt_status == 0); + + return 1; + +drop: + if (offload_flags & NIX_TX_OFFLOAD_MBUF_NOFF_F) { + /* Don't free if reference count > 1 */ + if (rte_pktmbuf_prefree_seg(m) == NULL) + return 0; + } + rte_pktmbuf_free(m); + return 0; +} + +#endif /* __OTX2_SECURITY_TX_H__ */ diff --git a/drivers/event/octeontx2/meson.build b/drivers/event/octeontx2/meson.build index 56febb8..be4b47a 100644 --- a/drivers/event/octeontx2/meson.build +++ b/drivers/event/octeontx2/meson.build @@ -31,6 +31,7 @@ foreach flag: extra_flags endif endforeach -deps += ['bus_pci', 'common_octeontx2', 'mempool_octeontx2', 'pmd_octeontx2'] +deps += ['bus_pci', 'common_octeontx2', 'cryptodev', 'mempool_octeontx2', 'pmd_octeontx2', + 'security'] includes += include_directories('../../crypto/octeontx2') diff --git a/drivers/event/octeontx2/otx2_worker.h b/drivers/event/octeontx2/otx2_worker.h index 7d161c8..c5ea4dd 100644 --- a/drivers/event/octeontx2/otx2_worker.h +++ b/drivers/event/octeontx2/otx2_worker.h @@ -10,6 +10,7 @@ #include #include "otx2_evdev.h" +#include "otx2_security_tx.h" /* SSO Operations */ @@ -281,6 +282,11 @@ otx2_ssogws_event_tx(struct otx2_ssogws *ws, struct rte_event ev[], const struct otx2_eth_txq *txq = otx2_ssogws_xtract_meta(m); rte_prefetch_non_temporal(txq); + + if ((flags & NIX_TX_OFFLOAD_SECURITY_F) && + (m->ol_flags & PKT_TX_SEC_OFFLOAD)) + return otx2_sec_event_tx(ws, ev, m, txq, flags); + /* Perform header writes before barrier for TSO */ otx2_nix_xmit_prepare_tso(m, flags); otx2_ssogws_order(ws, !ev->sched_type); From patchwork Sat Jan 18 10:49:13 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Anoob Joseph X-Patchwork-Id: 64871 X-Patchwork-Delegate: gakhil@marvell.com Return-Path: X-Original-To: patchwork@inbox.dpdk.org Delivered-To: patchwork@inbox.dpdk.org Received: from dpdk.org (dpdk.org [92.243.14.124]) by inbox.dpdk.org (Postfix) with ESMTP id 77899A051C; Sat, 18 Jan 2020 11:51:53 +0100 (CET) Received: from [92.243.14.124] (localhost [127.0.0.1]) by dpdk.org (Postfix) with ESMTP id 5920B1BF9D; Sat, 18 Jan 2020 11:51:23 +0100 (CET) Received: from mx0b-0016f401.pphosted.com (mx0a-0016f401.pphosted.com [67.231.148.174]) by dpdk.org (Postfix) with ESMTP id 8C7031BF7B for ; Sat, 18 Jan 2020 11:51:16 +0100 (CET) Received: from pps.filterd (m0045849.ppops.net [127.0.0.1]) by mx0a-0016f401.pphosted.com (8.16.0.42/8.16.0.42) with SMTP id 00IAjQOg009563; Sat, 18 Jan 2020 02:51:15 -0800 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=marvell.com; h=from : to : cc : subject : date : message-id : in-reply-to : references : mime-version : content-transfer-encoding : content-type; s=pfpt0818; bh=Hkz+f31n5KZ/VJ+96Nlmk+QiTj7Hloaot4UZPv4j2mw=; b=Mw1gCazxoy4JOgJmh3dPUmLKimskyexSXQsihdxB8BEynYRnrvscBB0hlu5qze8eX4X9 Px2z4oELksWIPrSszRaY5idzZVNLTs1f7IMpMMKJcZzUjfDQt59TaF8tyzFSVxruVqqG EFolG39Ym6Ex1y4B4IZfllA7NDQMqeLAjdV346+7ufLutoRYgIxr18fnLr8DPa2xKP08 zW9Pt9f3xYNXilp1QBx/0kg0KpVRdGlY+D3FsPcPiem0lsmruNHo/p6D5VQxoEdHI+Pw 5SlDYNbKaOWE0ldBDnw30K/WBm4nMj2Ov7SrCRqsN9CKUH2sBpU2BX6Wrxeqj5mCpAIu Ug== Received: from sc-exch04.marvell.com ([199.233.58.184]) by mx0a-0016f401.pphosted.com with ESMTP id 2xm08v01rf-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-SHA384 bits=256 verify=NOT); Sat, 18 Jan 2020 02:51:15 -0800 Received: from SC-EXCH03.marvell.com (10.93.176.83) by SC-EXCH04.marvell.com (10.93.176.84) with Microsoft SMTP Server (TLS) id 15.0.1497.2; Sat, 18 Jan 2020 02:51:14 -0800 Received: from maili.marvell.com (10.93.176.43) by SC-EXCH03.marvell.com (10.93.176.83) with Microsoft SMTP Server id 15.0.1497.2 via Frontend Transport; Sat, 18 Jan 2020 02:51:14 -0800 Received: from ajoseph83.caveonetworks.com (unknown [10.29.45.60]) by maili.marvell.com (Postfix) with ESMTP id 1A4143F703F; Sat, 18 Jan 2020 02:51:09 -0800 (PST) From: Anoob Joseph To: Akhil Goyal , Declan Doherty , Thomas Monjalon CC: Vamsi Attunuru , Jerin Jacob , Narayana Prasad , Kiran Kumar K , Nithin Dabilpuram , "Pavan Nikhilesh" , Ankur Dwivedi , Anoob Joseph , Archana Muniganti , Tejasree Kondoj , Lukasz Bartosik , Date: Sat, 18 Jan 2020 16:19:13 +0530 Message-ID: <1579344553-11428-16-git-send-email-anoobj@marvell.com> X-Mailer: git-send-email 2.7.4 In-Reply-To: <1579344553-11428-1-git-send-email-anoobj@marvell.com> References: <1575806094-28391-1-git-send-email-anoobj@marvell.com> <1579344553-11428-1-git-send-email-anoobj@marvell.com> MIME-Version: 1.0 X-Proofpoint-Virus-Version: vendor=fsecure engine=2.50.10434:6.0.138, 18.0.572 definitions=2020-01-18_02:2020-01-16, 2020-01-18 signatures=0 Subject: [dpdk-dev] [PATCH v2 15/15] crypto/octeontx2: sync inline tag type cfg with Rx adapter configuration X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Sender: "dev" From: Vamsi Attunuru Tag type configuration for the inline processed packets is set during ethdev configuration, it might conflict with tag type configuration done during Rx adapter configuration which would be setup later. This conflict is fixed as part of flow rule creation by updating tag type config of inline same as Rx adapter configured tag type. Signed-off-by: Ankur Dwivedi Signed-off-by: Anoob Joseph Signed-off-by: Archana Muniganti Signed-off-by: Tejasree Kondoj Signed-off-by: Vamsi Attunuru --- drivers/common/octeontx2/otx2_common.h | 2 ++ drivers/crypto/octeontx2/otx2_cryptodev.c | 2 ++ drivers/crypto/octeontx2/otx2_security.c | 28 ++++++++++++++++++++++++++++ drivers/crypto/octeontx2/otx2_security.h | 2 ++ drivers/net/octeontx2/otx2_flow.c | 26 ++++++++++++++++++++++++++ 5 files changed, 60 insertions(+) diff --git a/drivers/common/octeontx2/otx2_common.h b/drivers/common/octeontx2/otx2_common.h index 88b4b63..01d3a35 100644 --- a/drivers/common/octeontx2/otx2_common.h +++ b/drivers/common/octeontx2/otx2_common.h @@ -81,12 +81,14 @@ typedef int (*otx2_sec_eth_ctx_create_t)(struct rte_eth_dev *eth_dev); typedef void (*otx2_sec_eth_ctx_destroy_t)(struct rte_eth_dev *eth_dev); typedef int (*otx2_sec_eth_init_t)(struct rte_eth_dev *eth_dev); typedef void (*otx2_sec_eth_fini_t)(struct rte_eth_dev *eth_dev); +typedef int (*otx2_sec_eth_update_tag_type_t)(struct rte_eth_dev *eth_dev); struct otx2_sec_eth_crypto_idev_ops { otx2_sec_eth_ctx_create_t ctx_create; otx2_sec_eth_ctx_destroy_t ctx_destroy; otx2_sec_eth_init_t init; otx2_sec_eth_fini_t fini; + otx2_sec_eth_update_tag_type_t update_tag_type; }; extern struct otx2_sec_eth_crypto_idev_ops otx2_sec_idev_ops; diff --git a/drivers/crypto/octeontx2/otx2_cryptodev.c b/drivers/crypto/octeontx2/otx2_cryptodev.c index 34feb82..b944a51 100644 --- a/drivers/crypto/octeontx2/otx2_cryptodev.c +++ b/drivers/crypto/octeontx2/otx2_cryptodev.c @@ -160,4 +160,6 @@ RTE_INIT(otx2_cpt_init_log) otx2_sec_idev_ops.ctx_destroy = otx2_sec_eth_ctx_destroy; otx2_sec_idev_ops.init = otx2_sec_eth_init; otx2_sec_idev_ops.fini = otx2_sec_eth_fini; + otx2_sec_idev_ops.update_tag_type = otx2_sec_eth_update_tag_type; + } diff --git a/drivers/crypto/octeontx2/otx2_security.c b/drivers/crypto/octeontx2/otx2_security.c index 9a08849..37b9e54 100644 --- a/drivers/crypto/octeontx2/otx2_security.c +++ b/drivers/crypto/octeontx2/otx2_security.c @@ -710,6 +710,34 @@ sec_eth_ipsec_cfg(struct rte_eth_dev *eth_dev, uint8_t tt) } int +otx2_sec_eth_update_tag_type(struct rte_eth_dev *eth_dev) +{ + struct otx2_eth_dev *dev = otx2_eth_pmd_priv(eth_dev); + struct otx2_mbox *mbox = dev->mbox; + struct nix_aq_enq_rsp *rsp; + struct nix_aq_enq_req *aq; + int ret; + + aq = otx2_mbox_alloc_msg_nix_aq_enq(mbox); + aq->qidx = 0; /* Read RQ:0 context */ + aq->ctype = NIX_AQ_CTYPE_RQ; + aq->op = NIX_AQ_INSTOP_READ; + + ret = otx2_mbox_process_msg(mbox, (void *)&rsp); + if (ret < 0) { + otx2_err("Could not read RQ context"); + return ret; + } + + /* Update tag type */ + ret = sec_eth_ipsec_cfg(eth_dev, rsp->rq.sso_tt); + if (ret < 0) + otx2_err("Could not update sec eth tag type"); + + return ret; +} + +int otx2_sec_eth_init(struct rte_eth_dev *eth_dev) { const size_t sa_width = sizeof(struct otx2_ipsec_fp_in_sa); diff --git a/drivers/crypto/octeontx2/otx2_security.h b/drivers/crypto/octeontx2/otx2_security.h index fe7c883..3615273 100644 --- a/drivers/crypto/octeontx2/otx2_security.h +++ b/drivers/crypto/octeontx2/otx2_security.h @@ -146,6 +146,8 @@ int otx2_sec_eth_ctx_create(struct rte_eth_dev *eth_dev); void otx2_sec_eth_ctx_destroy(struct rte_eth_dev *eth_dev); +int otx2_sec_eth_update_tag_type(struct rte_eth_dev *eth_dev); + int otx2_sec_eth_init(struct rte_eth_dev *eth_dev); void otx2_sec_eth_fini(struct rte_eth_dev *eth_dev); diff --git a/drivers/net/octeontx2/otx2_flow.c b/drivers/net/octeontx2/otx2_flow.c index f1fb9f9..dea5337 100644 --- a/drivers/net/octeontx2/otx2_flow.c +++ b/drivers/net/octeontx2/otx2_flow.c @@ -299,6 +299,22 @@ flow_free_rss_action(struct rte_eth_dev *eth_dev, return 0; } +static int +flow_update_sec_tt(struct rte_eth_dev *eth_dev, + const struct rte_flow_action actions[]) +{ + int rc = 0; + + for (; actions->type != RTE_FLOW_ACTION_TYPE_END; actions++) { + if (actions->type == RTE_FLOW_ACTION_TYPE_SECURITY) { + if (otx2_sec_idev_ops.update_tag_type != NULL) + rc = otx2_sec_idev_ops.update_tag_type(eth_dev); + break; + } + } + + return rc; +} static int flow_parse_meta_items(__rte_unused struct otx2_parse_state *pst) @@ -491,6 +507,16 @@ otx2_flow_create(struct rte_eth_dev *dev, goto err_exit; } + if (hw->rx_offloads & DEV_RX_OFFLOAD_SECURITY) { + rc = flow_update_sec_tt(dev, actions); + if (rc != 0) { + rte_flow_error_set(error, EIO, + RTE_FLOW_ERROR_TYPE_UNSPECIFIED, + NULL, + "Failed to update tt with sec act"); + goto err_exit; + } + } list = &hw->npc_flow.flow_list[flow->priority]; /* List in ascending order of mcam entries */