From patchwork Fri Oct 16 10:41:28 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: Qi Zhang X-Patchwork-Id: 81093 X-Patchwork-Delegate: qi.z.zhang@intel.com Return-Path: X-Original-To: patchwork@inbox.dpdk.org Delivered-To: patchwork@inbox.dpdk.org Received: from dpdk.org (dpdk.org [92.243.14.124]) by inbox.dpdk.org (Postfix) with ESMTP id 12A98A04DB; Fri, 16 Oct 2020 12:38:12 +0200 (CEST) Received: from [92.243.14.124] (localhost [127.0.0.1]) by dpdk.org (Postfix) with ESMTP id 75DB41ED19; Fri, 16 Oct 2020 12:37:43 +0200 (CEST) Received: from mga18.intel.com (mga18.intel.com [134.134.136.126]) by dpdk.org (Postfix) with ESMTP id 6BBB51ECD6 for ; Fri, 16 Oct 2020 12:37:38 +0200 (CEST) IronPort-SDR: zN48ZRt+MB8bHhIsTdLuGEOQioctMZnPp/FoTI2LS2S62+Zy+nT3cTHAwUMKY+NjF+89TIywtX rrO0dRmxC5ZQ== X-IronPort-AV: E=McAfee;i="6000,8403,9775"; a="154378573" X-IronPort-AV: E=Sophos;i="5.77,382,1596524400"; d="scan'208";a="154378573" X-Amp-Result: SKIPPED(no attachment in message) X-Amp-File-Uploaded: False Received: from orsmga001.jf.intel.com ([10.7.209.18]) by orsmga106.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 16 Oct 2020 03:37:37 -0700 IronPort-SDR: QJPNVLxcVvC9BEB8++UXnqM8/DP1e41/jdAU4u2JFJ2RTx7d5gmjgRhoqVj8vMGkw56laNz0Mh WElfqRwtXsXA== X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="5.77,382,1596524400"; d="scan'208";a="391280458" Received: from dpdk51.sh.intel.com ([10.67.111.142]) by orsmga001.jf.intel.com with ESMTP; 16 Oct 2020 03:37:35 -0700 From: Qi Zhang To: ferruh.yigit@intel.com Cc: dev@dpdk.org, beilei.xing@intel.com, Qi Zhang , Ting Xu , Brett Creeley Date: Fri, 16 Oct 2020 18:41:28 +0800 Message-Id: <20201016104128.1049430-3-qi.z.zhang@intel.com> X-Mailer: git-send-email 2.25.4 In-Reply-To: <20201016104128.1049430-1-qi.z.zhang@intel.com> References: <20201016104128.1049430-1-qi.z.zhang@intel.com> MIME-Version: 1.0 Subject: [dpdk-dev] [PATCH v2 2/2] common/iavf: add support for VFs with >16 queue pairs X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Sender: "dev" Currently there are limitations in the virthcnl.h interface that only allow a maximum of 16 queues to be used by a VF driver. Add support in virtchnl.h to allow a VF driver to request >16 queues. Also, the RSS qregion size is currently assumed to be the max number of queues a VF can request and/or is given on initialization. With larger VFs this assumption can no longer be made, so add a new op to support querying the max RSS qregion size. In order to request more queues than the initially given queues the VF driver needs to use the VIRTCHNL_OP_REQUEST_QUEUES opcode. The VF is given more >16 queues it should use the new VIRTCHNL_OP_GET_MAX_RSS_QREGION to determine its max qregion size. This is needed to correctly configure the RSS LUT and/or configure filters based on queue base/offset and queue region size. If the VF is configuring >16 queues it should use the following opcodes to configure the qeueus and interrupts after successfully requesting them. VIRTCHNL_OP_MAP_QUEUE_VECTOR VIRTCHNL_OP_ENABLE_QUEUES_V2 VIRTCHNL_OP_DISABLE_QUEUES_V2 Also, add support in virtchnl_vc_validate_vf_msg() to validate the above messages. As a part of this move the virtchnl_vector_limits enumeration directly above the function it's used. The patch also update base code release version in readme. Signed-off-by: Ting Xu Signed-off-by: Brett Creeley Signed-off-by: Qi Zhang Acked-by: Beilei Xing --- drivers/common/iavf/README | 2 +- drivers/common/iavf/virtchnl.h | 231 +++++++++++++++++++++++++++++---- 2 files changed, 205 insertions(+), 28 deletions(-) diff --git a/drivers/common/iavf/README b/drivers/common/iavf/README index f301ff665a..5a42750465 100644 --- a/drivers/common/iavf/README +++ b/drivers/common/iavf/README @@ -6,7 +6,7 @@ Intel® IAVF driver ================= This directory contains source code of FreeBSD IAVF driver of version -cid-avf.2020.04.09.tar.gz released by the team which develops +cid-avf.2020.10.14.tar.gz released by the team which develops basic drivers for any IAVF NIC. The directory of base/ contains the original source package. diff --git a/drivers/common/iavf/virtchnl.h b/drivers/common/iavf/virtchnl.h index a5a300afe3..b931da61e5 100644 --- a/drivers/common/iavf/virtchnl.h +++ b/drivers/common/iavf/virtchnl.h @@ -140,6 +140,11 @@ enum virtchnl_ops { VIRTCHNL_OP_ADD_FDIR_FILTER = 47, VIRTCHNL_OP_DEL_FDIR_FILTER = 48, VIRTCHNL_OP_QUERY_FDIR_FILTER = 49, + VIRTCHNL_OP_GET_MAX_RSS_QREGION = 50, + VIRTCHNL_OP_ENABLE_QUEUES_V2 = 107, + VIRTCHNL_OP_DISABLE_QUEUES_V2 = 108, + VIRTCHNL_OP_MAP_QUEUE_VECTOR = 111, + VIRTCHNL_OP_MAX, }; /* These macros are used to generate compilation errors if a structure/union @@ -244,6 +249,8 @@ VIRTCHNL_CHECK_STRUCT_LEN(16, virtchnl_vsi_resource); #define VIRTCHNL_VF_OFFLOAD_WB_ON_ITR 0x00000020 #define VIRTCHNL_VF_OFFLOAD_REQ_QUEUES 0x00000040 #define VIRTCHNL_VF_OFFLOAD_CRC 0x00000080 + /* 0X00000100 is reserved */ +#define VIRTCHNL_VF_LARGE_NUM_QPAIRS 0x00000200 #define VIRTCHNL_VF_OFFLOAD_VLAN 0x00010000 #define VIRTCHNL_VF_OFFLOAD_RX_POLLING 0x00020000 #define VIRTCHNL_VF_OFFLOAD_RSS_PCTYPE_V2 0x00040000 @@ -424,6 +431,35 @@ struct virtchnl_queue_select { VIRTCHNL_CHECK_STRUCT_LEN(12, virtchnl_queue_select); +/* VIRTCHNL_OP_GET_MAX_RSS_QREGION + * + * if VIRTCHNL_VF_LARGE_NUM_QPAIRS was negotiated in VIRTCHNL_OP_GET_VF_RESOURCES + * then this op must be supported. + * + * VF sends this message in order to query the max RSS queue region + * size supported by PF, when VIRTCHNL_VF_LARGE_NUM_QPAIRS is enabled. + * This information should be used when configuring the RSS LUT and/or + * configuring queue region based filters. + * + * The maximum RSS queue region is 2^qregion_width. So, a qregion_width + * of 6 would inform the VF that the PF supports a maximum RSS queue region + * of 64. + * + * A queue region represents a range of queues that can be used to configure + * a RSS LUT. For example, if a VF is given 64 queues, but only a max queue + * region size of 16 (i.e. 2^qregion_width = 16) then it will only be able + * to configure the RSS LUT with queue indices from 0 to 15. However, other + * filters can be used to direct packets to queues >15 via specifying a queue + * base/offset and queue region width. + */ +struct virtchnl_max_rss_qregion { + u16 vport_id; + u16 qregion_width; + u8 pad[4]; +}; + +VIRTCHNL_CHECK_STRUCT_LEN(8, virtchnl_max_rss_qregion); + /* VIRTCHNL_OP_ADD_ETH_ADDR * VF sends this message in order to add one or more unicast or multicast * address filters for the specified VSI. @@ -774,33 +810,6 @@ struct virtchnl_pf_event { VIRTCHNL_CHECK_STRUCT_LEN(16, virtchnl_pf_event); -/* Since VF messages are limited by u16 size, precalculate the maximum possible - * values of nested elements in virtchnl structures that virtual channel can - * possibly handle in a single message. - */ -enum virtchnl_vector_limits { - VIRTCHNL_OP_CONFIG_VSI_QUEUES_MAX = - ((u16)(~0) - sizeof(struct virtchnl_vsi_queue_config_info)) / - sizeof(struct virtchnl_queue_pair_info), - - VIRTCHNL_OP_CONFIG_IRQ_MAP_MAX = - ((u16)(~0) - sizeof(struct virtchnl_irq_map_info)) / - sizeof(struct virtchnl_vector_map), - - VIRTCHNL_OP_ADD_DEL_ETH_ADDR_MAX = - ((u16)(~0) - sizeof(struct virtchnl_ether_addr_list)) / - sizeof(struct virtchnl_ether_addr), - - VIRTCHNL_OP_ADD_DEL_VLAN_MAX = - ((u16)(~0) - sizeof(struct virtchnl_vlan_filter_list)) / - sizeof(u16), - - - VIRTCHNL_OP_ENABLE_CHANNELS_MAX = - ((u16)(~0) - sizeof(struct virtchnl_tc_info)) / - sizeof(struct virtchnl_channel_info), -}; - /* VF reset states - these are written into the RSTAT register: * VFGEN_RSTAT on the VF * When the PF initiates a reset, it writes 0 @@ -1142,6 +1151,143 @@ struct virtchnl_fdir_query { }; VIRTCHNL_CHECK_STRUCT_LEN(48, virtchnl_fdir_query); + +/* TX and RX queue types are valid in legacy as well as split queue models. + * With Split Queue model, 2 additional types are introduced - TX_COMPLETION + * and RX_BUFFER. In split queue model, RX corresponds to the queue where HW + * posts completions. + */ +enum virtchnl_queue_type { + VIRTCHNL_QUEUE_TYPE_TX = 0, + VIRTCHNL_QUEUE_TYPE_RX = 1, + VIRTCHNL_QUEUE_TYPE_TX_COMPLETION = 2, + VIRTCHNL_QUEUE_TYPE_RX_BUFFER = 3, + VIRTCHNL_QUEUE_TYPE_CONFIG_TX = 4, + VIRTCHNL_QUEUE_TYPE_CONFIG_RX = 5 +}; + + +/* structure to specify a chunk of contiguous queues */ +struct virtchnl_queue_chunk { + enum virtchnl_queue_type type; + u16 start_queue_id; + u16 num_queues; +}; + +VIRTCHNL_CHECK_STRUCT_LEN(8, virtchnl_queue_chunk); + +/* structure to specify several chunks of contiguous queues */ +struct virtchnl_queue_chunks { + u16 num_chunks; + u16 rsvd; + struct virtchnl_queue_chunk chunks[1]; +}; + +VIRTCHNL_CHECK_STRUCT_LEN(12, virtchnl_queue_chunks); + + +/* VIRTCHNL_OP_ENABLE_QUEUES_V2 + * VIRTCHNL_OP_DISABLE_QUEUES_V2 + * VIRTCHNL_OP_DEL_QUEUES + * + * If VIRTCHNL_CAP_EXT_FEATURES was negotiated in VIRTCHNL_OP_GET_VF_RESOURCES + * then all of these ops are available. + * + * If VIRTCHNL_VF_LARGE_NUM_QPAIRS was negotiated in VIRTCHNL_OP_GET_VF_RESOURCES + * then VIRTCHNL_OP_ENABLE_QUEUES_V2 and VIRTCHNL_OP_DISABLE_QUEUES_V2 are + * available. + * + * PF sends these messages to enable, disable or delete queues specified in + * chunks. PF sends virtchnl_del_ena_dis_queues struct to specify the queues + * to be enabled/disabled/deleted. Also applicable to single queue RX or + * TX. CP performs requested action and returns status. + */ +struct virtchnl_del_ena_dis_queues { + u16 vport_id; + u16 pad; + struct virtchnl_queue_chunks chunks; +}; + +VIRTCHNL_CHECK_STRUCT_LEN(16, virtchnl_del_ena_dis_queues); + +/* Virtchannel interrupt throttling rate index */ +enum virtchnl_itr_idx { + VIRTCHNL_ITR_IDX_0 = 0, + VIRTCHNL_ITR_IDX_1 = 1, + VIRTCHNL_ITR_IDX_NO_ITR = 3, +}; + +/* Queue to vector mapping */ +struct virtchnl_queue_vector { + u16 queue_id; + u16 vector_id; + u8 pad[4]; + enum virtchnl_itr_idx itr_idx; + enum virtchnl_queue_type queue_type; +}; + +VIRTCHNL_CHECK_STRUCT_LEN(16, virtchnl_queue_vector); + +/* VIRTCHNL_OP_MAP_QUEUE_VECTOR + * VIRTCHNL_OP_UNMAP_QUEUE_VECTOR + * + * If VIRTCHNL_CAP_EXT_FEATURES was negotiated in VIRTCHNL_OP_GET_VF_RESOURCES + * then all of these ops are available. + * + * If VIRTCHNL_VF_LARGE_NUM_QPAIRS was negotiated in VIRTCHNL_OP_GET_VF_RESOURCES + * then only VIRTCHNL_OP_MAP_QUEUE_VECTOR is available. + * + * PF sends this message to map or unmap queues to vectors and ITR index + * registers. External data buffer contains virtchnl_queue_vector_maps structure + * that contains num_qv_maps of virtchnl_queue_vector structures. + * CP maps the requested queue vector maps after validating the queue and vector + * ids and returns a status code. + */ +struct virtchnl_queue_vector_maps { + u16 vport_id; + u16 num_qv_maps; + u8 pad[4]; + struct virtchnl_queue_vector qv_maps[1]; +}; + +VIRTCHNL_CHECK_STRUCT_LEN(24, virtchnl_queue_vector_maps); + + +/* Since VF messages are limited by u16 size, precalculate the maximum possible + * values of nested elements in virtchnl structures that virtual channel can + * possibly handle in a single message. + */ +enum virtchnl_vector_limits { + VIRTCHNL_OP_CONFIG_VSI_QUEUES_MAX = + ((u16)(~0) - sizeof(struct virtchnl_vsi_queue_config_info)) / + sizeof(struct virtchnl_queue_pair_info), + + VIRTCHNL_OP_CONFIG_IRQ_MAP_MAX = + ((u16)(~0) - sizeof(struct virtchnl_irq_map_info)) / + sizeof(struct virtchnl_vector_map), + + VIRTCHNL_OP_ADD_DEL_ETH_ADDR_MAX = + ((u16)(~0) - sizeof(struct virtchnl_ether_addr_list)) / + sizeof(struct virtchnl_ether_addr), + + VIRTCHNL_OP_ADD_DEL_VLAN_MAX = + ((u16)(~0) - sizeof(struct virtchnl_vlan_filter_list)) / + sizeof(u16), + + + VIRTCHNL_OP_ENABLE_CHANNELS_MAX = + ((u16)(~0) - sizeof(struct virtchnl_tc_info)) / + sizeof(struct virtchnl_channel_info), + + VIRTCHNL_OP_ENABLE_DISABLE_DEL_QUEUES_V2_MAX = + ((u16)(~0) - sizeof(struct virtchnl_del_ena_dis_queues)) / + sizeof(struct virtchnl_queue_chunk), + + VIRTCHNL_OP_MAP_UNMAP_QUEUE_VECTOR_MAX = + ((u16)(~0) - sizeof(struct virtchnl_queue_vector_maps)) / + sizeof(struct virtchnl_queue_vector), +}; + /** * virtchnl_vc_validate_vf_msg * @ver: Virtchnl version info @@ -1212,6 +1358,8 @@ virtchnl_vc_validate_vf_msg(struct virtchnl_version_info *ver, u32 v_opcode, case VIRTCHNL_OP_DISABLE_QUEUES: valid_len = sizeof(struct virtchnl_queue_select); break; + case VIRTCHNL_OP_GET_MAX_RSS_QREGION: + break; case VIRTCHNL_OP_ADD_ETH_ADDR: case VIRTCHNL_OP_DEL_ETH_ADDR: valid_len = sizeof(struct virtchnl_ether_addr_list); @@ -1338,6 +1486,35 @@ virtchnl_vc_validate_vf_msg(struct virtchnl_version_info *ver, u32 v_opcode, case VIRTCHNL_OP_QUERY_FDIR_FILTER: valid_len = sizeof(struct virtchnl_fdir_query); break; + case VIRTCHNL_OP_ENABLE_QUEUES_V2: + case VIRTCHNL_OP_DISABLE_QUEUES_V2: + valid_len = sizeof(struct virtchnl_del_ena_dis_queues); + if (msglen >= valid_len) { + struct virtchnl_del_ena_dis_queues *qs = + (struct virtchnl_del_ena_dis_queues *)msg; + if (qs->chunks.num_chunks == 0 || + qs->chunks.num_chunks > VIRTCHNL_OP_ENABLE_DISABLE_DEL_QUEUES_V2_MAX) { + err_msg_format = true; + break; + } + valid_len += (qs->chunks.num_chunks - 1) * + sizeof(struct virtchnl_queue_chunk); + } + break; + case VIRTCHNL_OP_MAP_QUEUE_VECTOR: + valid_len = sizeof(struct virtchnl_queue_vector_maps); + if (msglen >= valid_len) { + struct virtchnl_queue_vector_maps *v_qp = + (struct virtchnl_queue_vector_maps *)msg; + if (v_qp->num_qv_maps == 0 || + v_qp->num_qv_maps > VIRTCHNL_OP_MAP_UNMAP_QUEUE_VECTOR_MAX) { + err_msg_format = true; + break; + } + valid_len += (v_qp->num_qv_maps - 1) * + sizeof(struct virtchnl_queue_vector); + } + break; /* These are always errors coming from the VF. */ case VIRTCHNL_OP_EVENT: case VIRTCHNL_OP_UNKNOWN: