From patchwork Wed Jul 15 13:10:20 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Suanming Mou X-Patchwork-Id: 74096 X-Patchwork-Delegate: rasland@nvidia.com Return-Path: X-Original-To: patchwork@inbox.dpdk.org Delivered-To: patchwork@inbox.dpdk.org Received: from dpdk.org (dpdk.org [92.243.14.124]) by inbox.dpdk.org (Postfix) with ESMTP id 897AAA0547; Wed, 15 Jul 2020 15:10:35 +0200 (CEST) Received: from [92.243.14.124] (localhost [127.0.0.1]) by dpdk.org (Postfix) with ESMTP id BB00C1BED7; Wed, 15 Jul 2020 15:10:31 +0200 (CEST) Received: from git-send-mailer.rdmz.labs.mlnx (unknown [37.142.13.130]) by dpdk.org (Postfix) with ESMTP id C02971BEA8 for ; Wed, 15 Jul 2020 15:10:28 +0200 (CEST) From: Suanming Mou To: viacheslavo@mellanox.com, matan@mellanox.com Cc: orika@mellanox.com, rasland@mellanox.com, dev@dpdk.org Date: Wed, 15 Jul 2020 21:10:20 +0800 Message-Id: <1594818621-438919-2-git-send-email-suanmingm@mellanox.com> X-Mailer: git-send-email 1.8.3.1 In-Reply-To: <1594818621-438919-1-git-send-email-suanmingm@mellanox.com> References: <1594818621-438919-1-git-send-email-suanmingm@mellanox.com> Subject: [dpdk-dev] [PATCH 1/2] common/mlx5: add scatter FCS with decap capability query X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Sender: "dev" As scatter FCS might be not supported for decapsulated tunnel packets in some NIC HW, an new capability bit which indicates if scatter FCS works with decap is added. Signed-off-by: Suanming Mou Acked-by: Viacheslav Ovsiienko --- doc/guides/nics/mlx5.rst | 5 +++++ drivers/common/mlx5/mlx5_devx_cmds.c | 2 ++ drivers/common/mlx5/mlx5_devx_cmds.h | 1 + drivers/common/mlx5/mlx5_prm.h | 4 +++- 4 files changed, 11 insertions(+), 1 deletion(-) diff --git a/doc/guides/nics/mlx5.rst b/doc/guides/nics/mlx5.rst index 4b6d8fb..97261d6 100644 --- a/doc/guides/nics/mlx5.rst +++ b/doc/guides/nics/mlx5.rst @@ -279,6 +279,11 @@ Limitations Other TCP packets (e.g. with MPLS label) received on Rx queue with LRO enabled, will be received with bad checksum. +- KEEP_CRC: + + - KEEP_CRC offload cannot be supported with decapsulation for some NICs(such as ConnectX-6DX and BlueField2). + The new capability bit scatter_fcs_w_decap_disable shows the supporting for the NICs. + Statistics ---------- diff --git a/drivers/common/mlx5/mlx5_devx_cmds.c b/drivers/common/mlx5/mlx5_devx_cmds.c index 2179a83..4cd92a9 100644 --- a/drivers/common/mlx5/mlx5_devx_cmds.c +++ b/drivers/common/mlx5/mlx5_devx_cmds.c @@ -467,6 +467,8 @@ struct mlx5_devx_obj * attr->vdpa.queue_counters_valid = !!(MLX5_GET64(cmd_hca_cap, hcattr, general_obj_types) & MLX5_GENERAL_OBJ_TYPES_CAP_VIRTIO_Q_COUNTERS); + attr->scatter_fcs_w_decap_disable = + MLX5_GET(cmd_hca_cap, hcattr, scatter_fcs_w_decap_disable); if (attr->qos.sup) { MLX5_SET(query_hca_cap_in, in, op_mod, MLX5_GET_HCA_CAP_OP_MOD_QOS_CAP | diff --git a/drivers/common/mlx5/mlx5_devx_cmds.h b/drivers/common/mlx5/mlx5_devx_cmds.h index 25704ef..930f78f 100644 --- a/drivers/common/mlx5/mlx5_devx_cmds.h +++ b/drivers/common/mlx5/mlx5_devx_cmds.h @@ -90,6 +90,7 @@ struct mlx5_hca_attr { uint32_t vhca_id:16; uint32_t relaxed_ordering_write:1; uint32_t relaxed_ordering_read:1; + uint32_t scatter_fcs_w_decap_disable:1; struct mlx5_hca_qos_attr qos; struct mlx5_hca_vdpa_attr vdpa; }; diff --git a/drivers/common/mlx5/mlx5_prm.h b/drivers/common/mlx5/mlx5_prm.h index c63795f..d157457 100644 --- a/drivers/common/mlx5/mlx5_prm.h +++ b/drivers/common/mlx5/mlx5_prm.h @@ -996,7 +996,9 @@ struct mlx5_ifc_cmd_hca_cap_bits { u8 log_max_qp[0x5]; u8 reserved_at_a0[0xb]; u8 log_max_srq[0x5]; - u8 reserved_at_b0[0x10]; + u8 reserved_at_b0[0xb]; + u8 scatter_fcs_w_decap_disable[0x1]; + u8 reserved_at_bc[0x4]; u8 reserved_at_c0[0x8]; u8 log_max_cq_sz[0x8]; u8 reserved_at_d0[0xb]; From patchwork Wed Jul 15 13:10:21 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Suanming Mou X-Patchwork-Id: 74097 X-Patchwork-Delegate: rasland@nvidia.com Return-Path: X-Original-To: patchwork@inbox.dpdk.org Delivered-To: patchwork@inbox.dpdk.org Received: from dpdk.org (dpdk.org [92.243.14.124]) by inbox.dpdk.org (Postfix) with ESMTP id 17D88A0547; Wed, 15 Jul 2020 15:10:43 +0200 (CEST) Received: from [92.243.14.124] (localhost [127.0.0.1]) by dpdk.org (Postfix) with ESMTP id 017561BEDF; Wed, 15 Jul 2020 15:10:33 +0200 (CEST) Received: from git-send-mailer.rdmz.labs.mlnx (unknown [37.142.13.130]) by dpdk.org (Postfix) with ESMTP id 86D2C1BED5 for ; Wed, 15 Jul 2020 15:10:30 +0200 (CEST) From: Suanming Mou To: viacheslavo@mellanox.com, matan@mellanox.com Cc: orika@mellanox.com, rasland@mellanox.com, dev@dpdk.org Date: Wed, 15 Jul 2020 21:10:21 +0800 Message-Id: <1594818621-438919-3-git-send-email-suanmingm@mellanox.com> X-Mailer: git-send-email 1.8.3.1 In-Reply-To: <1594818621-438919-1-git-send-email-suanmingm@mellanox.com> References: <1594818621-438919-1-git-send-email-suanmingm@mellanox.com> Subject: [dpdk-dev] [PATCH 2/2] net/mlx5: add decap enable device argument X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Sender: "dev" There are some limitations on the some NICs (at least on ConnectX-6DX and BlueField2) with supporting FCS (frame checksum) scattering for the tunnel decapsulated packets. For the case only one of the features can be supported in the same time, and the new devarg "decap_en" is introduced to provide the choice to the users. If FCS scattering feature is not supposed to be engaged by application, this new devarg should be specified as "decap_en=0", forcing the FCS feature enable and rejecting tunnel decap actions in the rte_flow engine. If FCS scatter is not needed and application supposes to use tunnel decapsulation in rte_flow, the devarg can be omitted or set to non-zero value (this is default settings). Signed-off-by: Suanming Mou Acked-by: Viacheslav Ovsiienko --- doc/guides/nics/mlx5.rst | 10 ++++++++++ drivers/net/mlx5/linux/mlx5_os.c | 12 ++++++++++-- drivers/net/mlx5/mlx5.c | 6 ++++++ drivers/net/mlx5/mlx5.h | 1 + drivers/net/mlx5/mlx5_flow_dv.c | 5 +++++ 5 files changed, 32 insertions(+), 2 deletions(-) diff --git a/doc/guides/nics/mlx5.rst b/doc/guides/nics/mlx5.rst index 97261d6..ccf38b6 100644 --- a/doc/guides/nics/mlx5.rst +++ b/doc/guides/nics/mlx5.rst @@ -884,6 +884,16 @@ Driver options By default, the PMD will set this value to 0. +- ``decap_en`` parameter [int] + + The new devarg ``decap_en`` is introduced to provide the choice to the NICs + which does not support FCS (frame checksum) scattering for the tunnel + decapsulated packets. By set the devarg value 0 to force the FCS feature + enable and rejecting tunnel decap actions in the rte_flow engine for these + special NICs. + + By default, the PMD will set this value to 1. + .. _mlx5_firmware_config: Firmware configuration diff --git a/drivers/net/mlx5/linux/mlx5_os.c b/drivers/net/mlx5/linux/mlx5_os.c index 2dc57b2..37420cb 100644 --- a/drivers/net/mlx5/linux/mlx5_os.c +++ b/drivers/net/mlx5/linux/mlx5_os.c @@ -780,8 +780,6 @@ (config.hw_vlan_strip ? "" : "not ")); config.hw_fcs_strip = !!(sh->device_attr.raw_packet_caps & IBV_RAW_PACKET_CAP_SCATTER_FCS); - DRV_LOG(DEBUG, "FCS stripping configuration is %ssupported", - (config.hw_fcs_strip ? "" : "not ")); #if defined(HAVE_IBV_WQ_FLAG_RX_END_PADDING) hw_padding = !!sh->device_attr.rx_pad_end_addr_align; #elif defined(HAVE_IBV_WQ_FLAGS_PCI_WRITE_END_PADDING) @@ -879,6 +877,15 @@ } #endif } + /* + * If HW has bug working with tunnel packet decapsulation and + * scatter FCS, and decapsulation is needed, clear the hw_fcs_strip + * bit. Then DEV_RX_OFFLOAD_KEEP_CRC bit will not be set anymore. + */ + if (config.hca_attr.scatter_fcs_w_decap_disable && config.decap_en) + config.hw_fcs_strip = 0; + DRV_LOG(DEBUG, "FCS stripping configuration is %ssupported", + (config.hw_fcs_strip ? "" : "not ")); if (config.mprq.enabled && mprq) { if (config.mprq.stride_num_n && (config.mprq.stride_num_n > mprq_max_stride_num_n || @@ -1652,6 +1659,7 @@ }, .dv_esw_en = 1, .dv_flow_en = 1, + .decap_en = 1, .log_hp_size = MLX5_ARG_UNSET, }; /* Device specific configuration. */ diff --git a/drivers/net/mlx5/mlx5.c b/drivers/net/mlx5/mlx5.c index 0c654ed..9daacaf 100644 --- a/drivers/net/mlx5/mlx5.c +++ b/drivers/net/mlx5/mlx5.c @@ -167,6 +167,9 @@ /* Flow memory reclaim mode. */ #define MLX5_RECLAIM_MEM "reclaim_mem_mode" +/* Decap will be used or not. */ +#define MLX5_DECAP_EN "decap_en" + static const char *MZ_MLX5_PMD_SHARED_DATA = "mlx5_pmd_shared_data"; /* Shared memory between primary and secondary processes. */ @@ -1374,6 +1377,8 @@ struct mlx5_dev_ctx_shared * return -rte_errno; } config->reclaim_mode = tmp; + } else if (strcmp(MLX5_DECAP_EN, key) == 0) { + config->decap_en = !!tmp; } else { DRV_LOG(WARNING, "%s: unknown parameter", key); rte_errno = EINVAL; @@ -1430,6 +1435,7 @@ struct mlx5_dev_ctx_shared * MLX5_CLASS_ARG_NAME, MLX5_HP_BUF_SIZE, MLX5_RECLAIM_MEM, + MLX5_DECAP_EN, NULL, }; struct rte_kvargs *kvlist; diff --git a/drivers/net/mlx5/mlx5.h b/drivers/net/mlx5/mlx5.h index 46e66eb..c7b6a84 100644 --- a/drivers/net/mlx5/mlx5.h +++ b/drivers/net/mlx5/mlx5.h @@ -216,6 +216,7 @@ struct mlx5_dev_config { unsigned int devx:1; /* Whether devx interface is available or not. */ unsigned int dest_tir:1; /* Whether advanced DR API is available. */ unsigned int reclaim_mode:2; /* Memory reclaim mode. */ + unsigned int decap_en:1; /* Whether decap will be used or not. */ struct { unsigned int enabled:1; /* Whether MPRQ is enabled. */ unsigned int stride_num_n; /* Number of strides. */ diff --git a/drivers/net/mlx5/mlx5_flow_dv.c b/drivers/net/mlx5/mlx5_flow_dv.c index 8b5b683..37db7c5 100644 --- a/drivers/net/mlx5/mlx5_flow_dv.c +++ b/drivers/net/mlx5/mlx5_flow_dv.c @@ -2428,6 +2428,11 @@ struct field_modify_info modify_tcp[] = { { const struct mlx5_priv *priv = dev->data->dev_private; + if (priv->config.hca_attr.scatter_fcs_w_decap_disable && + !priv->config.decap_en) + return rte_flow_error_set(error, ENOTSUP, + RTE_FLOW_ERROR_TYPE_ACTION, NULL, + "decap is not enabled"); if (action_flags & MLX5_FLOW_XCAP_ACTIONS) return rte_flow_error_set(error, ENOTSUP, RTE_FLOW_ERROR_TYPE_ACTION, NULL,