Patch Detail
get:
Show a patch.
patch:
Update a patch.
put:
Update a patch.
GET /api/patches/116862/?format=api
{ "id": 116862, "url": "https://patches.dpdk.org/api/patches/116862/?format=api", "web_url": "https://patches.dpdk.org/project/dpdk/patch/20220926084248.1421987-3-junfeng.guo@intel.com/", "project": { "id": 1, "url": "https://patches.dpdk.org/api/projects/1/?format=api", "name": "DPDK", "link_name": "dpdk", "list_id": "dev.dpdk.org", "list_email": "dev@dpdk.org", "web_url": "http://core.dpdk.org", "scm_url": "git://dpdk.org/dpdk", "webscm_url": "http://git.dpdk.org/dpdk", "list_archive_url": "https://inbox.dpdk.org/dev", "list_archive_url_format": "https://inbox.dpdk.org/dev/{}", "commit_url_format": "" }, "msgid": "<20220926084248.1421987-3-junfeng.guo@intel.com>", "list_archive_url": "https://inbox.dpdk.org/dev/20220926084248.1421987-3-junfeng.guo@intel.com", "date": "2022-09-26T08:42:47", "name": "[RFC,v2,2/3] net/idpf: add support for hairpin queue", "commit_ref": null, "pull_url": null, "state": "new", "archived": false, "hash": "79df281f44609beae14f1d6edd6e77cbd2d9cb84", "submitter": { "id": 1785, "url": "https://patches.dpdk.org/api/people/1785/?format=api", "name": "Junfeng Guo", "email": "junfeng.guo@intel.com" }, "delegate": { "id": 1, "url": "https://patches.dpdk.org/api/users/1/?format=api", "username": "tmonjalo", "first_name": "Thomas", "last_name": "Monjalon", "email": "thomas@monjalon.net" }, "mbox": "https://patches.dpdk.org/project/dpdk/patch/20220926084248.1421987-3-junfeng.guo@intel.com/mbox/", "series": [ { "id": 24825, "url": "https://patches.dpdk.org/api/series/24825/?format=api", "web_url": "https://patches.dpdk.org/project/dpdk/list/?series=24825", "date": "2022-09-26T08:42:45", "name": "enable hairpin queue", "version": 2, "mbox": "https://patches.dpdk.org/series/24825/mbox/" } ], "comments": "https://patches.dpdk.org/api/patches/116862/comments/", "check": "success", "checks": "https://patches.dpdk.org/api/patches/116862/checks/", "tags": {}, "related": [], "headers": { "Return-Path": "<dev-bounces@dpdk.org>", "X-Original-To": "patchwork@inbox.dpdk.org", "Delivered-To": "patchwork@inbox.dpdk.org", "Received": [ "from mails.dpdk.org (mails.dpdk.org [217.70.189.124])\n\tby inbox.dpdk.org (Postfix) with ESMTP id 9921EA00C2;\n\tMon, 26 Sep 2022 10:43:33 +0200 (CEST)", "from [217.70.189.124] (localhost [127.0.0.1])\n\tby mails.dpdk.org (Postfix) with ESMTP id 8994A41145;\n\tMon, 26 Sep 2022 10:43:29 +0200 (CEST)", "from mga03.intel.com (mga03.intel.com [134.134.136.65])\n by mails.dpdk.org (Postfix) with ESMTP id 4315C400D7\n for <dev@dpdk.org>; Mon, 26 Sep 2022 10:43:24 +0200 (CEST)", "from fmsmga005.fm.intel.com ([10.253.24.32])\n by orsmga103.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384;\n 26 Sep 2022 01:43:09 -0700", "from dpdk-jf-ntb-v2.sh.intel.com ([10.67.118.246])\n by fmsmga005.fm.intel.com with ESMTP; 26 Sep 2022 01:43:07 -0700" ], "DKIM-Signature": "v=1; a=rsa-sha256; c=relaxed/simple;\n d=intel.com; i=@intel.com; q=dns/txt; s=Intel;\n t=1664181804; x=1695717804;\n h=from:to:cc:subject:date:message-id:in-reply-to:\n references:mime-version:content-transfer-encoding;\n bh=aCKlL+eJ4GhwY2klZfjmtLfetMALN5dzVQtZyMEQ7HE=;\n b=A76wtTxKO6bHpZssPVAz20Sp7m0CaCJBh8Y62AbdOiN4OZ3D4QXqEoyy\n WBNMiD36jvO40b0gSdGLS8Q5oIzgyoYoG5k+nq6Nh9C8wSS1hN/66hdJI\n qRanb8td9wuyXG7jd9gmX4IEkhGJN6aef/BAR1ReFkdWfsktn28RWwKlx\n LqOTcICTSbYPryHHXHpxPcw4QrXxjDhBS/EtIhsgs8tX69tGM4qJaCCXT\n OsfOOaBMFSZNcYspsmXZpY0n2ywE/mu3kXahHgcAWOz55Lv+eM7M/pU5c\n z7J++H5s0ys5+3essEZyMPN3k5TMl5zhvJNSGez2tfP1hkZMd09AjOWAP w==;", "X-IronPort-AV": [ "E=McAfee;i=\"6500,9779,10481\"; a=\"302444382\"", "E=Sophos;i=\"5.93,345,1654585200\"; d=\"scan'208\";a=\"302444382\"", "E=McAfee;i=\"6500,9779,10481\"; a=\"949770637\"", "E=Sophos;i=\"5.93,345,1654585200\"; d=\"scan'208\";a=\"949770637\"" ], "X-ExtLoop1": "1", "From": "Junfeng Guo <junfeng.guo@intel.com>", "To": "qi.z.zhang@intel.com,\n\tjingjing.wu@intel.com,\n\tbeilei.xing@intel.com", "Cc": "dev@dpdk.org,\n\txiao.w.wang@intel.com,\n\tjunfeng.guo@intel.com", "Subject": "[RFC v2 2/3] net/idpf: add support for hairpin queue", "Date": "Mon, 26 Sep 2022 16:42:47 +0800", "Message-Id": "<20220926084248.1421987-3-junfeng.guo@intel.com>", "X-Mailer": "git-send-email 2.25.1", "In-Reply-To": "<20220926084248.1421987-1-junfeng.guo@intel.com>", "References": "<20220905113031.3223290-2-junfeng.guo@intel.com>\n <20220926084248.1421987-1-junfeng.guo@intel.com>", "MIME-Version": "1.0", "Content-Transfer-Encoding": "8bit", "X-BeenThere": "dev@dpdk.org", "X-Mailman-Version": "2.1.29", "Precedence": "list", "List-Id": "DPDK patches and discussions <dev.dpdk.org>", "List-Unsubscribe": "<https://mails.dpdk.org/options/dev>,\n <mailto:dev-request@dpdk.org?subject=unsubscribe>", "List-Archive": "<http://mails.dpdk.org/archives/dev/>", "List-Post": "<mailto:dev@dpdk.org>", "List-Help": "<mailto:dev-request@dpdk.org?subject=help>", "List-Subscribe": "<https://mails.dpdk.org/listinfo/dev>,\n <mailto:dev-request@dpdk.org?subject=subscribe>", "Errors-To": "dev-bounces@dpdk.org" }, "content": "- Implement hairpin queue setup/confige/enable/disable.\n- Cross-vport hairpin queue implemented via hairpin_bind/unbind API.\n\nTest step:\n1. Make sure no bug on CP side.\n2. Add rule on IMC.\n - devmem 0x202920C100 64 0x804\n - opcode=0x1303 prof_id=0x34 sub_prof_id=0x0 cookie=0xa2b87 key=0x18,\\\n 0x0,00,00,00,00,de,0xad,0xbe,0xef,0x20,0x24,0x0,0x0,0x0,0x0,00,00,\\\n 00,00,00,00,0xa,0x2,0x1d,0x64,00,00,00,00,00,00,00,00,00,00,00,00,\\\n 0xa,0x2,0x1d,0x2,00,00,00,00,00,00,00,00,00,00,00,00 act=set_vsi{\\\n act_val=0 val_type=2 dst_pe=0 slot=0x0} act=set_q{\\\n qnum=0x142 no_implicit_vsi=1 prec=5}\n3. Send packets on ixia side\n UDP packets with dmac=de:ad:be:ef:20:24 sip=10.2.29.100\n dip=10.2.29.2\n\nSigned-off-by: Xiao Wang <xiao.w.wang@intel.com>\nSigned-off-by: Beilei Xing <beilei.xing@intel.com>\nSigned-off-by: Junfeng Guo <junfeng.guo@intel.com>\n---\n drivers/net/idpf/base/virtchnl2.h | 6 +\n drivers/net/idpf/idpf_ethdev.c | 181 ++++++++++-\n drivers/net/idpf/idpf_ethdev.h | 5 +\n drivers/net/idpf/idpf_rxtx.c | 521 ++++++++++++++++++++++++++++--\n drivers/net/idpf/idpf_rxtx.h | 27 ++\n drivers/net/idpf/idpf_vchnl.c | 211 +++++++++---\n 6 files changed, 871 insertions(+), 80 deletions(-)", "diff": "diff --git a/drivers/net/idpf/base/virtchnl2.h b/drivers/net/idpf/base/virtchnl2.h\nindex 566afe075f..9fdcccd2ae 100644\n--- a/drivers/net/idpf/base/virtchnl2.h\n+++ b/drivers/net/idpf/base/virtchnl2.h\n@@ -273,6 +273,12 @@\n #define VIRTCHNL2_QUEUE_TYPE_RX_BUFFER\t\t3\n #define VIRTCHNL2_QUEUE_TYPE_CONFIG_TX\t\t4\n #define VIRTCHNL2_QUEUE_TYPE_CONFIG_RX\t\t5\n+#define VIRTCHNL2_QUEUE_TYPE_P2P_TX\t\t6\n+#define VIRTCHNL2_QUEUE_TYPE_P2P_RX\t\t7\n+#define VIRTCHNL2_QUEUE_TYPE_P2P_TX_COMPLETION\t8\n+#define VIRTCHNL2_QUEUE_TYPE_P2P_RX_BUFFER\t9\n+#define VIRTCHNL2_QUEUE_TYPE_MBX_TX\t\t10\n+#define VIRTCHNL2_QUEUE_TYPE_MBX_RX\t\t11\n \n /* VIRTCHNL2_ITR_IDX\n * Virtchannel interrupt throttling rate index\ndiff --git a/drivers/net/idpf/idpf_ethdev.c b/drivers/net/idpf/idpf_ethdev.c\nindex 08ada728b1..41e6391d8b 100644\n--- a/drivers/net/idpf/idpf_ethdev.c\n+++ b/drivers/net/idpf/idpf_ethdev.c\n@@ -38,6 +38,13 @@ static int idpf_dev_mtu_set(struct rte_eth_dev *dev, uint16_t mtu);\n static int idpf_dev_stats_get(struct rte_eth_dev *dev,\n \t\t\tstruct rte_eth_stats *stats);\n static int idpf_dev_stats_reset(struct rte_eth_dev *dev);\n+static int idpf_hairpin_cap_get(struct rte_eth_dev *dev,\n+\t\t\t\tstruct rte_eth_hairpin_cap *cap);\n+static int\n+idpf_hairpin_get_peer_ports(struct rte_eth_dev *dev, uint16_t *peer_ports,\n+\t\tsize_t len, uint32_t tx);\n+static int\n+idpf_hairpin_bind(struct rte_eth_dev *dev, uint16_t rx_port);\n \n int\n idpf_dev_link_update(struct rte_eth_dev *dev,\n@@ -78,8 +85,25 @@ static const struct eth_dev_ops idpf_eth_dev_ops = {\n \t.mtu_set\t\t\t= idpf_dev_mtu_set,\n \t.stats_get\t\t\t= idpf_dev_stats_get,\n \t.stats_reset\t\t\t= idpf_dev_stats_reset,\n+\t.hairpin_cap_get\t\t= idpf_hairpin_cap_get,\n+\t.rx_hairpin_queue_setup\t\t= idpf_rx_hairpin_queue_setup,\n+\t.tx_hairpin_queue_setup\t\t= idpf_tx_hairpin_queue_setup,\n+\t.hairpin_get_peer_ports\t\t= idpf_hairpin_get_peer_ports,\n+\t.hairpin_bind\t\t\t= idpf_hairpin_bind,\n };\n \n+static int\n+idpf_hairpin_cap_get(__rte_unused struct rte_eth_dev *dev,\n+\t\t struct rte_eth_hairpin_cap *cap)\n+{\n+\tcap->max_nb_queues = 1;\n+\tcap->max_rx_2_tx = 1;\n+\tcap->max_tx_2_rx = 1;\n+\tcap->max_nb_desc = 1024;\n+\n+\treturn 0;\n+}\n+\n static int\n idpf_dev_info_get(struct rte_eth_dev *dev, struct rte_eth_dev_info *dev_info)\n {\n@@ -221,7 +245,6 @@ idpf_dev_stats_get(struct rte_eth_dev *dev, struct rte_eth_stats *stats)\n \treturn ret;\n }\n \n-\n static int\n idpf_dev_stats_reset(struct rte_eth_dev *dev)\n {\n@@ -609,14 +632,26 @@ idpf_start_queues(struct rte_eth_dev *dev)\n \tint err = 0;\n \tint i;\n \n+\t/* For normal data queues, configure, init and enale Txq first, then\n+\t * configure, init and enable Qxq.\n+\t * For non-cross vport hairpin queues, configure Txq and Rxq first, then init Rxq.\n+\t */\n \tfor (i = 0; i < dev->data->nb_tx_queues; i++) {\n \t\ttxq = dev->data->tx_queues[i];\n \t\tif (!txq || txq->tx_deferred_start)\n \t\t\tcontinue;\n-\t\terr = idpf_tx_queue_start(dev, i);\n-\t\tif (err) {\n-\t\t\tPMD_DRV_LOG(ERR, \"Fail to start Tx queue %u\", i);\n-\t\t\treturn err;\n+\t\tif (!txq->hairpin_q) {\n+\t\t\terr = idpf_tx_queue_start(dev, i);\n+\t\t\tif (err) {\n+\t\t\t\tPMD_DRV_LOG(ERR, \"Fail to start Tx queue %u\", i);\n+\t\t\t\treturn err;\n+\t\t\t}\n+\t\t} else if (!txq->hairpin_cv) {\n+\t\t\terr = idpf_config_txq(vport, i);\n+\t\t\tif (err) {\n+\t\t\t\tPMD_DRV_LOG(ERR, \"Fail to configure hairpin Tx queue %u\", i);\n+\t\t\t\treturn err;\n+\t\t\t}\n \t\t}\n \t}\n \n@@ -624,10 +659,48 @@ idpf_start_queues(struct rte_eth_dev *dev)\n \t\trxq = dev->data->rx_queues[i];\n \t\tif (!rxq || rxq->rx_deferred_start)\n \t\t\tcontinue;\n-\t\terr = idpf_rx_queue_start(dev, i);\n-\t\tif (err) {\n-\t\t\tPMD_DRV_LOG(ERR, \"Fail to start Rx queue %u\", i);\n-\t\t\treturn err;\n+\t\tif (!rxq->hairpin_q) {\n+\t\t\terr = idpf_rx_queue_start(dev, i);\n+\t\t\tif (err) {\n+\t\t\t\tPMD_DRV_LOG(ERR, \"Fail to start Rx queue %u\", i);\n+\t\t\t\treturn err;\n+\t\t\t}\n+\t\t} else if (!rxq->hairpin_cv) {\n+\t\t\terr = idpf_config_rxq(vport, i);\n+\t\t\tif (err) {\n+\t\t\t\tPMD_DRV_LOG(ERR, \"Fail to configure hairpin Rx queue %u\", i);\n+\t\t\t\treturn err;\n+\t\t\t}\n+\t\t\terr = idpf_rx_queue_init(dev, i);\n+\t\t\tif (err) {\n+\t\t\t\tPMD_DRV_LOG(ERR, \"Fail to init hairpin Rx queue %u\", i);\n+\t\t\t\treturn err;\n+\t\t\t}\n+\t\t}\n+\t}\n+\n+\t/* For non-cross vport hairpin queues, enable Txq and Rxq at last. */\n+\tfor (i = 0; i < dev->data->nb_tx_queues; i++) {\n+\t\ttxq = dev->data->tx_queues[i];\n+\t\tif (txq->hairpin_q && !txq->hairpin_cv) {\n+\t\t\terr = idpf_switch_hairpin_queue(vport, i, false, true);\n+\t\t\tif (err)\n+\t\t\t\tPMD_DRV_LOG(ERR, \"Failed to switch hairpin TX queue %u on\",\n+\t\t\t\t\t i);\n+\t\t\telse\n+\t\t\t\ttxq->q_started = true;\n+\t\t}\n+\t}\n+\n+\tfor (i = 0; i < dev->data->nb_rx_queues; i++) {\n+\t\trxq = dev->data->rx_queues[i];\n+\t\tif (rxq->hairpin_q && !rxq->hairpin_cv) {\n+\t\t\terr = idpf_switch_hairpin_queue(vport, i, true, true);\n+\t\t\tif (err)\n+\t\t\t\tPMD_DRV_LOG(ERR, \"Failed to switch hairpin RX queue %u on\",\n+\t\t\t\t\t i);\n+\t\telse\n+\t\t\trxq->q_started = true;\n \t\t}\n \t}\n \n@@ -696,6 +769,90 @@ idpf_dev_start(struct rte_eth_dev *dev)\n \treturn -1;\n }\n \n+static int\n+idpf_hairpin_get_peer_ports(struct rte_eth_dev *dev, uint16_t *peer_ports,\n+\t\t__rte_unused size_t len, uint32_t tx)\n+{\n+\t/* Assume the last queue is used by app as hairpin */\n+\tint qid = dev->data->nb_tx_queues - 1;\n+\tstruct idpf_tx_queue *txq = dev->data->tx_queues[qid];\n+\tstruct idpf_rx_queue *rxq = dev->data->rx_queues[qid];\n+\n+\tPMD_INIT_FUNC_TRACE();\n+\n+\tif (tx && txq->hairpin_cv) {\n+\t\tpeer_ports[0] = txq->peer_rxp;\n+\t\treturn 1;\n+\t} else if (!tx && rxq->hairpin_cv) {\n+\t\tpeer_ports[0] = rxq->peer_txp;\n+\t\treturn 1;\n+\t}\n+\n+\treturn 0;\n+}\n+\n+static int\n+idpf_hairpin_bind(struct rte_eth_dev *dev, uint16_t rx_port)\n+{\n+\tstruct idpf_vport *vport, *peer_vport;\n+\t/* Assume the last queue is used by app as hairpin */\n+\tint qid = dev->data->nb_tx_queues - 1;\n+\tstruct rte_eth_dev *peer_dev;\n+\tint err;\n+\tstruct idpf_tx_queue *txq = dev->data->tx_queues[qid];\n+\n+\tPMD_INIT_FUNC_TRACE();\n+\tif (rx_port >= RTE_MAX_ETHPORTS)\n+\t\treturn 0;\n+\n+\tif (txq->bound) {\n+\t\tPMD_DRV_LOG(INFO, \"port %u already hairpin bound\",\n+\t\t\t\tdev->data->port_id);\n+\t\treturn 0;\n+\t}\n+\n+\tvport = (struct idpf_vport *)dev->data->dev_private;\n+\terr = idpf_config_txq(vport, qid);\n+\tif (err) {\n+\t\tPMD_DRV_LOG(ERR, \"Fail to configure hairpin Tx queue %u of port %u\",\n+\t\t\t qid, dev->data->port_id);\n+\t\treturn err;\n+\t}\n+\n+\tpeer_dev = &rte_eth_devices[rx_port];\n+\tpeer_vport = (struct idpf_vport *)peer_dev->data->dev_private;\n+\terr = idpf_config_rxq(peer_vport, qid);\n+\tif (err) {\n+\t\tPMD_DRV_LOG(ERR, \"Fail to configure hairpin Rx queue %u of port %u\",\n+\t\t\t qid, peer_dev->data->port_id);\n+\t\treturn err;\n+\t}\n+\terr = idpf_rx_queue_init(peer_dev, qid);\n+\tif (err) {\n+\t\tPMD_DRV_LOG(ERR, \"Fail to init hairpin Rx queue %u of port %u\",\n+\t\t\t qid, peer_dev->data->port_id);\n+\t\treturn err;\n+\t}\n+\n+\terr = idpf_switch_hairpin_queue(vport, qid, false, true);\n+\tif (err) {\n+\t\tPMD_DRV_LOG(ERR, \"Fail to enable hairpin Tx queue %u of port %u\",\n+\t\t\t qid, dev->data->port_id);\n+\t\treturn err;\n+\t}\n+\n+\terr = idpf_switch_hairpin_queue(peer_vport, qid, true, true);\n+\tif (err) {\n+\t\tPMD_DRV_LOG(ERR, \"Fail to enable hairpin Rx queue %u of port %u\",\n+\t\t\t qid, peer_dev->data->port_id);\n+\t\treturn err;\n+\t}\n+\n+\n+\ttxq->bound = true;\n+\treturn 0;\n+}\n+\n static int\n idpf_dev_stop(struct rte_eth_dev *dev)\n {\n@@ -733,6 +890,12 @@ idpf_dev_close(struct rte_eth_dev *dev)\n \t\treturn 0;\n \n \tidpf_dev_stop(dev);\n+\n+\tif (vport->p2p_mp) {\n+\t\trte_mempool_free(vport->p2p_mp);\n+\t\tvport->p2p_mp = NULL;\n+\t}\n+\n \tidpf_destroy_vport(vport);\n \n \trte_free(vport->rss_lut);\ndiff --git a/drivers/net/idpf/idpf_ethdev.h b/drivers/net/idpf/idpf_ethdev.h\nindex 968e0e3cbf..90b931313b 100644\n--- a/drivers/net/idpf/idpf_ethdev.h\n+++ b/drivers/net/idpf/idpf_ethdev.h\n@@ -135,6 +135,9 @@ struct idpf_vport {\n \t/* Chunk info */\n \tstruct idpf_chunks_info chunks_info;\n \n+\t/* p2p mbuf pool */\n+\tstruct rte_mempool *p2p_mp;\n+\n \t/* Event from ipf */\n \tbool link_up;\n \tuint32_t link_speed;\n@@ -256,6 +259,8 @@ int idpf_config_txqs(struct idpf_vport *vport);\n int idpf_config_txq(struct idpf_vport *vport, uint16_t txq_id);\n int idpf_switch_queue(struct idpf_vport *vport, uint16_t qid,\n \t\t bool rx, bool on);\n+int idpf_switch_hairpin_queue(struct idpf_vport *vport, uint16_t qid,\n+\t\t bool rx, bool on);\n int idpf_ena_dis_queues(struct idpf_vport *vport, bool enable);\n int idpf_ena_dis_vport(struct idpf_vport *vport, bool enable);\n int idpf_query_stats(struct idpf_vport *vport,\ndiff --git a/drivers/net/idpf/idpf_rxtx.c b/drivers/net/idpf/idpf_rxtx.c\nindex b0037eca08..13e55d7196 100644\n--- a/drivers/net/idpf/idpf_rxtx.c\n+++ b/drivers/net/idpf/idpf_rxtx.c\n@@ -152,16 +152,25 @@ idpf_rx_queue_release(void *rxq)\n \t\treturn;\n \n \t/* Split queue */\n-\tif (q->bufq1 && q->bufq2) {\n+\tif (q->bufq1) {\n+\t\t/* the mz is shared between Tx/Rx hairpin, let Tx_release\n+\t\t * free the buf.\n+\t\t */\n+\t\tif (!q->hairpin_q) {\n+\t\t\trte_memzone_free(q->bufq1->mz);\n+\t\t\trte_memzone_free(q->bufq2->mz);\n+\t\t\trte_memzone_free(q->mz);\n+\t\t}\n+\n \t\tq->bufq1->ops->release_mbufs(q->bufq1);\n \t\trte_free(q->bufq1->sw_ring);\n-\t\trte_memzone_free(q->bufq1->mz);\n \t\trte_free(q->bufq1);\n-\t\tq->bufq2->ops->release_mbufs(q->bufq2);\n-\t\trte_free(q->bufq2->sw_ring);\n-\t\trte_memzone_free(q->bufq2->mz);\n-\t\trte_free(q->bufq2);\n-\t\trte_memzone_free(q->mz);\n+\n+\t\tif (q->bufq2) {\n+\t\t\tq->bufq2->ops->release_mbufs(q->bufq2);\n+\t\t\trte_free(q->bufq2->sw_ring);\n+\t\t\trte_free(q->bufq2);\n+\t\t}\n \t\trte_free(q);\n \t\treturn;\n \t}\n@@ -244,7 +253,8 @@ reset_split_rx_queue(struct idpf_rx_queue *rxq)\n {\n \treset_split_rx_descq(rxq);\n \treset_split_rx_bufq(rxq->bufq1);\n-\treset_split_rx_bufq(rxq->bufq2);\n+\tif (rxq->bufq2)\n+\t\treset_split_rx_bufq(rxq->bufq2);\n }\n \n static inline void\n@@ -390,6 +400,7 @@ idpf_rx_split_bufq_setup(struct rte_eth_dev *dev, struct idpf_rx_queue *bufq,\n \tbufq->rx_deferred_start = rx_conf->rx_deferred_start;\n \tbufq->rx_hdr_len = 0;\n \tbufq->adapter = adapter;\n+\tbufq->q_type = VIRTCHNL2_QUEUE_TYPE_RX_BUFFER;\n \n \tif (dev->data->dev_conf.rxmode.offloads & RTE_ETH_RX_OFFLOAD_KEEP_CRC)\n \t\tbufq->crc_len = RTE_ETHER_CRC_LEN;\n@@ -503,6 +514,7 @@ idpf_rx_split_queue_setup(struct rte_eth_dev *dev, uint16_t queue_idx,\n \trxq->rx_hdr_len = 0;\n \trxq->adapter = adapter;\n \trxq->offloads = offloads;\n+\trxq->q_type = VIRTCHNL2_QUEUE_TYPE_RX;\n \n \tif (dev->data->dev_conf.rxmode.offloads & RTE_ETH_RX_OFFLOAD_KEEP_CRC)\n \t\trxq->crc_len = RTE_ETHER_CRC_LEN;\n@@ -656,6 +668,7 @@ idpf_rx_single_queue_setup(struct rte_eth_dev *dev, uint16_t queue_idx,\n \trxq->rx_hdr_len = 0;\n \trxq->adapter = adapter;\n \trxq->offloads = offloads;\n+\trxq->q_type = VIRTCHNL2_QUEUE_TYPE_RX;\n \n \tif (dev->data->dev_conf.rxmode.offloads & RTE_ETH_RX_OFFLOAD_KEEP_CRC)\n \t\trxq->crc_len = RTE_ETHER_CRC_LEN;\n@@ -780,6 +793,7 @@ idpf_tx_split_queue_setup(struct rte_eth_dev *dev, uint16_t queue_idx,\n \ttxq->port_id = dev->data->port_id;\n \ttxq->offloads = offloads;\n \ttxq->tx_deferred_start = tx_conf->tx_deferred_start;\n+\ttxq->q_type = VIRTCHNL2_QUEUE_TYPE_TX;\n \n \t/* Allocate software ring */\n \ttxq->sw_nb_desc = 2 * nb_desc;\n@@ -831,6 +845,7 @@ idpf_tx_split_queue_setup(struct rte_eth_dev *dev, uint16_t queue_idx,\n \tcq->port_id = dev->data->port_id;\n \tcq->txqs = dev->data->tx_queues;\n \tcq->tx_start_qid = vport->chunks_info.tx_start_qid;\n+\tcq->q_type = VIRTCHNL2_QUEUE_TYPE_TX_COMPLETION;\n \n \tring_size = sizeof(struct iecm_splitq_tx_compl_desc) * cq->nb_tx_desc;\n \tring_size = RTE_ALIGN(ring_size, IDPF_DMA_MEM_ALIGN);\n@@ -912,6 +927,7 @@ idpf_tx_single_queue_setup(struct rte_eth_dev *dev, uint16_t queue_idx,\n \ttxq->port_id = dev->data->port_id;\n \ttxq->offloads = offloads;\n \ttxq->tx_deferred_start = tx_conf->tx_deferred_start;\n+\ttxq->q_type = VIRTCHNL2_QUEUE_TYPE_TX;\n \n \t/* Allocate software ring */\n \ttxq->sw_ring =\n@@ -985,6 +1001,407 @@ idpf_register_ts_mbuf(struct idpf_rx_queue *rxq)\n \treturn 0;\n }\n \n+static inline void\n+reset_rx_hairpin_descq(struct idpf_rx_queue *rxq)\n+{\n+\tuint16_t len;\n+\tuint32_t i;\n+\n+\tif (!rxq)\n+\t\treturn;\n+\n+\tlen = rxq->nb_rx_desc;\n+\tfor (i = 0; i < len * IDPF_P2P_DESC_LEN; i++)\n+\t\t((volatile char *)rxq->rx_ring)[i] = 0;\n+}\n+\n+static inline void\n+reset_rx_hairpin_bufq(struct idpf_rx_queue *rxbq)\n+{\n+\tuint16_t len;\n+\tuint32_t i;\n+\n+\tif (!rxbq)\n+\t\treturn;\n+\n+\tlen = rxbq->nb_rx_desc;\n+\tfor (i = 0; i < len * IDPF_P2P_DESC_LEN; i++)\n+\t\t((volatile char *)rxbq->rx_ring)[i] = 0;\n+\n+\trxbq->bufq1 = NULL;\n+\trxbq->bufq2 = NULL;\n+}\n+\n+#define IDPF_NB_MBUF\t\t4096\n+#define IDPF_CACHE_SIZE\t\t250\n+#define IDPF_MBUF_SIZE\t\t2048\n+#define IDPF_P2P_RING_BUF\t128\n+\n+static int\n+idpf_rx_hairpin_bufq_setup(struct rte_eth_dev *dev, struct idpf_rx_queue *bufq,\n+\t\t\t uint16_t queue_idx, uint16_t nb_desc,\n+\t\t\t struct idpf_tx_queue *peer_txq)\n+{\n+\tstruct idpf_vport *vport =\n+\t\t(struct idpf_vport *)dev->data->dev_private;\n+\tstruct idpf_adapter *adapter = vport->adapter;\n+\tstruct iecm_hw *hw = &adapter->hw;\n+\tstruct rte_mempool *mp;\n+\tconst struct rte_memzone *mz;\n+\tuint32_t ring_size;\n+\tchar pool_name[RTE_MEMPOOL_NAMESIZE];\n+\n+\tmp = vport->p2p_mp;\n+\tif (!mp) {\n+\t\tsnprintf(pool_name, RTE_MEMPOOL_NAMESIZE, \"p2p_mb_pool_%u\",\n+\t\t\t dev->data->port_id);\n+\t\tmp = rte_pktmbuf_pool_create(pool_name, IDPF_NB_MBUF, IDPF_CACHE_SIZE,\n+\t\t\t\t\t 0, IDPF_MBUF_SIZE, dev->device->numa_node);\n+\t\tif (!mp) {\n+\t\t\tPMD_INIT_LOG(ERR, \"Failed to allocate mbuf pool for p2p\");\n+\t\t\treturn -ENOMEM;\n+\t\t}\n+\t\tvport->p2p_mp = mp;\n+\t}\n+\n+\tbufq->mp = mp;\n+\tbufq->nb_rx_desc = nb_desc;\n+\tbufq->queue_id = vport->chunks_info.rx_buf_start_qid + queue_idx;\n+\tbufq->port_id = dev->data->port_id;\n+\tbufq->adapter = adapter;\n+\tbufq->q_type = VIRTCHNL2_QUEUE_TYPE_P2P_RX_BUFFER;\n+\tbufq->hairpin_q = true;\n+\tbufq->rx_buf_len = 2048;\n+\n+\tbufq->sw_ring = rte_zmalloc(\"sw ring\",\n+\t\t\t\t sizeof(struct rte_mbuf *) * nb_desc,\n+\t\t\t\t RTE_CACHE_LINE_SIZE);\n+\tif (!bufq->sw_ring) {\n+\t\tPMD_INIT_LOG(ERR, \"Failed to allocate memory for SW ring\");\n+\t\treturn -ENOMEM;\n+\t}\n+\n+\tif (peer_txq && peer_txq->complq->mz) {\n+\t\tmz = peer_txq->complq->mz;\n+\t\tbufq->rx_ring_phys_addr = mz->iova;\n+\t\tbufq->rx_ring = mz->addr;\n+\t\tbufq->mz = mz;\n+\t} else {\n+\t\tring_size = RTE_ALIGN(bufq->nb_rx_desc * IDPF_P2P_DESC_LEN,\n+\t\t\t\t IDPF_DMA_MEM_ALIGN);\n+\t\tmz = rte_eth_dma_zone_reserve(dev, \"hairpin_rx_buf_ring\", queue_idx,\n+\t\t\t\t\t ring_size + IDPF_P2P_RING_BUF,\n+\t\t\t\t\t IDPF_RING_BASE_ALIGN,\n+\t\t\t\t\t dev->device->numa_node);\n+\t\tif (!mz) {\n+\t\t\tPMD_INIT_LOG(ERR, \"Failed to reserve DMA memory for hairpin RX buffer queue.\");\n+\t\t\trte_free(bufq->sw_ring);\n+\t\t\treturn -ENOMEM;\n+\t\t}\n+\n+\t\tbufq->rx_ring_phys_addr = mz->iova;\n+\t\tbufq->rx_ring = mz->addr;\n+\t\tbufq->mz = mz;\n+\t}\n+\treset_rx_hairpin_bufq(bufq);\n+\tbufq->q_set = true;\n+\tbufq->qrx_tail = hw->hw_addr + (vport->chunks_info.rx_buf_qtail_start +\n+\t\t\t queue_idx * vport->chunks_info.rx_buf_qtail_spacing);\n+\tbufq->ops = &def_rxq_ops;\n+\n+\treturn 0;\n+}\n+\n+int\n+idpf_rx_hairpin_queue_setup(struct rte_eth_dev *dev, uint16_t queue_idx,\n+\t\t\t uint16_t nb_desc,\n+\t\t\t const struct rte_eth_hairpin_conf *conf)\n+{\n+\tstruct idpf_vport *vport =\n+\t\t(struct idpf_vport *)dev->data->dev_private;\n+\tstruct idpf_adapter *adapter = vport->adapter;\n+\tstruct idpf_vport *peer_vport;\n+\tstruct idpf_rx_queue *rxq;\n+\tstruct idpf_rx_queue *bufq1;\n+\tstruct idpf_tx_queue *peer_txq = NULL;\n+\tconst struct rte_memzone *mz;\n+\tuint32_t ring_size;\n+\tuint16_t qid;\n+\tint ret;\n+\tuint16_t peer_port = conf->peers[0].port;\n+\tuint16_t peer_q = conf->peers[0].queue;\n+\n+\tif (vport->rxq_model == VIRTCHNL2_QUEUE_MODEL_SINGLE) {\n+\t\tPMD_INIT_LOG(ERR, \"Only spilt queue model supports hairpin queue.\");\n+\t\treturn -EINVAL;\n+\t}\n+\n+\tif (nb_desc % IDPF_ALIGN_RING_DESC != 0 ||\n+\t nb_desc > IDPF_MAX_RING_DESC ||\n+\t nb_desc < IDPF_MIN_RING_DESC) {\n+\t\tPMD_INIT_LOG(ERR, \"Number (%u) of receive descriptors is invalid\", nb_desc);\n+\t\treturn -EINVAL;\n+\t}\n+\n+\t/* Free memory if needed */\n+\tif (dev->data->rx_queues[queue_idx]) {\n+\t\tidpf_rx_queue_release(dev->data->rx_queues[queue_idx]);\n+\t\tdev->data->rx_queues[queue_idx] = NULL;\n+\t}\n+\n+\t/* Setup Rx description queue */\n+\trxq = rte_zmalloc(\"idpf_rxq_hairpin\", sizeof(struct idpf_rx_queue),\n+\t\t\t RTE_CACHE_LINE_SIZE);\n+\tif (!rxq) {\n+\t\tPMD_INIT_LOG(ERR, \"Failed to allocate memory for rx queue data structure\");\n+\t\treturn -ENOMEM;\n+\t}\n+\n+\trxq->nb_rx_desc = nb_desc * 2;\n+\trxq->queue_id = vport->chunks_info.rx_start_qid + queue_idx;\n+\trxq->port_id = dev->data->port_id;\n+\trxq->adapter = adapter;\n+\trxq->q_type = VIRTCHNL2_QUEUE_TYPE_P2P_RX;\n+\trxq->hairpin_q = true;\n+\trxq->rx_buf_len = 2048;\n+\n+\tif (peer_port != dev->data->port_id)\n+\t\trxq->hairpin_cv = true;\n+\trxq->peer_txp = peer_port;\n+\tpeer_vport = adapter->vports[peer_port];\n+\tif (peer_q < peer_vport->dev_data->nb_tx_queues)\n+\t\tpeer_txq = peer_vport->dev_data->tx_queues[peer_q];\n+\n+\tif (peer_txq && peer_txq->mz) {\n+\t\tmz = peer_txq->mz;\n+\t\trxq->rx_ring_phys_addr = mz->iova;\n+\t\trxq->rx_ring = mz->addr;\n+\t\trxq->mz = mz;\n+\t} else {\n+\t\tring_size = RTE_ALIGN(rxq->nb_rx_desc * IDPF_P2P_DESC_LEN,\n+\t\t\t\t IDPF_DMA_MEM_ALIGN);\n+\t\tmz = rte_eth_dma_zone_reserve(dev, \"hairpin_rx_ring\", queue_idx,\n+\t\t\t\t\t ring_size + IDPF_P2P_RING_BUF,\n+\t\t\t\t\t IDPF_RING_BASE_ALIGN,\n+\t\t\t\t\t dev->device->numa_node);\n+\t\tif (!mz) {\n+\t\t\tPMD_INIT_LOG(ERR, \"Failed to reserve DMA memory for RX\");\n+\t\t\tret = -ENOMEM;\n+\t\t\tgoto free_rxq;\n+\t\t}\n+\n+\t\trxq->rx_ring_phys_addr = mz->iova;\n+\t\trxq->rx_ring = mz->addr;\n+\t\trxq->mz = mz;\n+\t}\n+\treset_rx_hairpin_descq(rxq);\n+\n+\t/* setup 1 Rx buffer queue for 1 hairpin rxq */\n+\tbufq1 = rte_zmalloc_socket(\"hairpin rx bufq1\",\n+\t\t\t\t sizeof(struct idpf_rx_queue),\n+\t\t\t\t RTE_CACHE_LINE_SIZE,\n+\t\t\t\t SOCKET_ID_ANY);\n+\tif (!bufq1) {\n+\t\tPMD_INIT_LOG(ERR, \"Failed to allocate memory for hairpin Rx buffer queue 1.\");\n+\t\tret = -ENOMEM;\n+\t\tgoto free_mz;\n+\t}\n+\tqid = 2 * queue_idx;\n+\tret = idpf_rx_hairpin_bufq_setup(dev, bufq1, qid, nb_desc, peer_txq);\n+\tif (ret) {\n+\t\tPMD_INIT_LOG(ERR, \"Failed to setup hairpin Rx buffer queue 1\");\n+\t\tret = -EINVAL;\n+\t\tgoto free_bufq1;\n+\t}\n+\trxq->bufq1 = bufq1;\n+\trxq->bufq2 = NULL;\n+\n+\trxq->q_set = true;\n+\tdev->data->rx_queues[queue_idx] = rxq;\n+\n+\treturn 0;\n+\n+free_bufq1:\n+\trte_free(bufq1);\n+free_mz:\n+\trte_memzone_free(mz);\n+free_rxq:\n+\trte_free(rxq);\n+\n+\treturn ret;\n+}\n+\n+static inline void\n+reset_tx_hairpin_descq(struct idpf_tx_queue *txq)\n+{\n+\tuint32_t i, size;\n+\n+\tif (!txq) {\n+\t\tPMD_DRV_LOG(DEBUG, \"Pointer to txq is NULL\");\n+\t\treturn;\n+\t}\n+\n+\tsize = txq->nb_tx_desc * IDPF_P2P_DESC_LEN;\n+\tfor (i = 0; i < size; i++)\n+\t\t((volatile char *)txq->desc_ring)[i] = 0;\n+}\n+\n+static inline void\n+reset_tx_hairpin_complq(struct idpf_tx_queue *cq)\n+{\n+\tuint32_t i, size;\n+\n+\tif (!cq) {\n+\t\tPMD_DRV_LOG(DEBUG, \"Pointer to complq is NULL\");\n+\t\treturn;\n+\t}\n+\n+\tsize = cq->nb_tx_desc * IDPF_P2P_DESC_LEN;\n+\tfor (i = 0; i < size; i++)\n+\t\t((volatile char *)cq->compl_ring)[i] = 0;\n+}\n+\n+int\n+idpf_tx_hairpin_queue_setup(struct rte_eth_dev *dev, uint16_t queue_idx,\n+\t\t\t uint16_t nb_desc,\n+\t\t\t const struct rte_eth_hairpin_conf *conf)\n+{\n+\tstruct idpf_vport *vport =\n+\t\t(struct idpf_vport *)dev->data->dev_private;\n+\tstruct idpf_vport *peer_vport;\n+\tstruct idpf_adapter *adapter = vport->adapter;\n+\tstruct iecm_hw *hw = &adapter->hw;\n+\tstruct idpf_tx_queue *txq, *cq;\n+\tstruct idpf_rx_queue *peer_rxq = NULL;\n+\tconst struct rte_memzone *mz;\n+\tuint32_t ring_size;\n+\tuint16_t peer_port = conf->peers[0].port;\n+\tuint16_t peer_q = conf->peers[0].queue;\n+\n+\tif (vport->txq_model == VIRTCHNL2_QUEUE_MODEL_SINGLE) {\n+\t\tPMD_INIT_LOG(ERR, \"Only spilt queue model supports hairpin queue.\");\n+\t\treturn -EINVAL;\n+\t}\n+\n+\tif (nb_desc % IDPF_ALIGN_RING_DESC != 0 ||\n+\t nb_desc > IDPF_MAX_RING_DESC ||\n+\t nb_desc < IDPF_MIN_RING_DESC) {\n+\t\tPMD_INIT_LOG(ERR, \"Number (%u) of transmit descriptors is invalid\",\n+\t\t\t nb_desc);\n+\t\treturn -EINVAL;\n+\t}\n+\n+\t/* Free memory if needed. */\n+\tif (dev->data->tx_queues[queue_idx]) {\n+\t\tidpf_tx_queue_release(dev->data->tx_queues[queue_idx]);\n+\t\tdev->data->tx_queues[queue_idx] = NULL;\n+\t}\n+\n+\t/* Allocate the TX queue data structure. */\n+\ttxq = rte_zmalloc_socket(\"idpf hairpin txq\",\n+\t\t\t\t sizeof(struct idpf_tx_queue),\n+\t\t\t\t RTE_CACHE_LINE_SIZE,\n+\t\t\t\t SOCKET_ID_ANY);\n+\tif (!txq) {\n+\t\tPMD_INIT_LOG(ERR, \"Failed to allocate memory for tx queue structure\");\n+\t\treturn -ENOMEM;\n+\t}\n+\n+\t/* Txq ring length should be 2 times of Tx completion queue size. */\n+\ttxq->nb_tx_desc = nb_desc * 2;\n+\ttxq->queue_id = vport->chunks_info.tx_start_qid + queue_idx;\n+\ttxq->port_id = dev->data->port_id;\n+\ttxq->q_type = VIRTCHNL2_QUEUE_TYPE_P2P_TX;\n+\ttxq->hairpin_q = true;\n+\n+\tif (peer_port != dev->data->port_id)\n+\t\ttxq->hairpin_cv = true;\n+\ttxq->peer_rxp = peer_port;\n+\tpeer_vport = adapter->vports[peer_port];\n+\ttxq->peer_rxq_id = peer_vport->chunks_info.rx_start_qid + conf->peers[0].queue;\n+\tif (peer_q < peer_vport->dev_data->nb_rx_queues)\n+\t\tpeer_rxq = peer_vport->dev_data->rx_queues[peer_q];\n+\n+\t/* Hairpin Rxq and Txq share the same HW ring */\n+\tif (peer_rxq && peer_rxq->mz) {\n+\t\tmz = peer_rxq->mz;\n+\t\ttxq->tx_ring_phys_addr = mz->iova;\n+\t\ttxq->desc_ring = mz->addr;\n+\t\ttxq->mz = mz;\n+\t} else {\n+\t\tring_size = RTE_ALIGN(txq->nb_tx_desc * IDPF_P2P_DESC_LEN,\n+\t\t\t\t IDPF_DMA_MEM_ALIGN);\n+\t\tmz = rte_eth_dma_zone_reserve(dev, \"hairpin_tx_ring\", queue_idx,\n+\t\t\t\t\t ring_size + IDPF_P2P_RING_BUF,\n+\t\t\t\t\t IDPF_RING_BASE_ALIGN,\n+\t\t\t\t\t dev->device->numa_node);\n+\t\tif (!mz) {\n+\t\t\tPMD_INIT_LOG(ERR, \"Failed to reserve DMA memory for TX\");\n+\t\t\trte_free(txq->sw_ring);\n+\t\t\trte_free(txq);\n+\t\t\treturn -ENOMEM;\n+\t\t}\n+\n+\t\ttxq->tx_ring_phys_addr = mz->iova;\n+\t\ttxq->desc_ring = mz->addr;\n+\t\ttxq->mz = mz;\n+\t}\n+\n+\treset_tx_hairpin_descq(txq);\n+\ttxq->qtx_tail = hw->hw_addr + (vport->chunks_info.tx_qtail_start +\n+\t\t\tqueue_idx * vport->chunks_info.tx_qtail_spacing);\n+\ttxq->ops = &def_txq_ops;\n+\n+\t/* Allocate the TX completion queue data structure. */\n+\ttxq->complq = rte_zmalloc_socket(\"idpf hairpin cq\",\n+\t\t\t\t\t sizeof(struct idpf_tx_queue),\n+\t\t\t\t\t RTE_CACHE_LINE_SIZE,\n+\t\t\t\t\t dev->device->numa_node);\n+\tcq = txq->complq;\n+\tif (!cq) {\n+\t\tPMD_INIT_LOG(ERR, \"Failed to allocate memory for tx queue structure\");\n+\t\treturn -ENOMEM;\n+\t}\n+\n+\tcq->nb_tx_desc = nb_desc;\n+\tcq->queue_id = vport->chunks_info.tx_compl_start_qid + queue_idx;\n+\tcq->port_id = dev->data->port_id;\n+\tcq->q_type = VIRTCHNL2_QUEUE_TYPE_P2P_TX_COMPLETION;\n+\tcq->hairpin_q = true;\n+\tcq->peer_rxq_id = peer_vport->chunks_info.rx_buf_start_qid + conf->peers[0].queue * 2;\n+\n+\t/* Hairpin Rx buffer queue and Tx completion queue share the same HW ring */\n+\tif (peer_rxq && peer_rxq->bufq1->mz) {\n+\t\tmz = peer_rxq->bufq1->mz;\n+\t\tcq->tx_ring_phys_addr = mz->iova;\n+\t\tcq->compl_ring = mz->addr;\n+\t\tcq->mz = mz;\n+\t} else {\n+\t\tring_size = RTE_ALIGN(cq->nb_tx_desc * IDPF_P2P_DESC_LEN,\n+\t\t\t\t IDPF_DMA_MEM_ALIGN);\n+\t\tmz = rte_eth_dma_zone_reserve(dev, \"hairpin_tx_compl_ring\", queue_idx,\n+\t\t\t\t\t ring_size + IDPF_P2P_RING_BUF,\n+\t\t\t\t\t IDPF_RING_BASE_ALIGN,\n+\t\t\t\t\t dev->device->numa_node);\n+\t\tif (!mz) {\n+\t\t\tPMD_INIT_LOG(ERR, \"Failed to reserve DMA memory for TX completion queue\");\n+\t\t\trte_free(txq->sw_ring);\n+\t\t\trte_free(txq);\n+\t\t\treturn -ENOMEM;\n+\t\t}\n+\t\tcq->tx_ring_phys_addr = mz->iova;\n+\t\tcq->compl_ring = mz->addr;\n+\t\tcq->mz = mz;\n+\t}\n+\n+\treset_tx_hairpin_complq(cq);\n+\n+\ttxq->q_set = true;\n+\tdev->data->tx_queues[queue_idx] = txq;\n+\n+\treturn 0;\n+}\n+\n static int\n idpf_alloc_single_rxq_mbufs(struct idpf_rx_queue *rxq)\n {\n@@ -1023,6 +1440,41 @@ idpf_alloc_single_rxq_mbufs(struct idpf_rx_queue *rxq)\n \treturn 0;\n }\n \n+static int\n+idpf_alloc_split_p2p_rxq_mbufs(struct idpf_rx_queue *rxq)\n+{\n+\tvolatile struct virtchnl2_p2p_rx_buf_desc *rxd;\n+\tstruct rte_mbuf *mbuf = NULL;\n+\tuint64_t dma_addr;\n+\tuint16_t i;\n+\n+\tfor (i = 0; i < rxq->nb_rx_desc; i++) {\n+\t\tmbuf = rte_mbuf_raw_alloc(rxq->mp);\n+\t\tif (unlikely(!mbuf)) {\n+\t\t\tPMD_DRV_LOG(ERR, \"Failed to allocate mbuf for RX\");\n+\t\t\treturn -ENOMEM;\n+\t\t}\n+\n+\t\trte_mbuf_refcnt_set(mbuf, 1);\n+\t\tmbuf->next = NULL;\n+\t\tmbuf->data_off = RTE_PKTMBUF_HEADROOM;\n+\t\tmbuf->nb_segs = 1;\n+\t\tmbuf->port = rxq->port_id;\n+\t\tdma_addr = rte_cpu_to_le_64(rte_mbuf_data_iova_default(mbuf));\n+\n+\t\trxd = &((volatile struct virtchnl2_p2p_rx_buf_desc *)(rxq->rx_ring))[i];\n+\t\trxd->reserve0 = 0;\n+\t\trxd->pkt_addr = dma_addr;\n+\n+\t\trxq->sw_ring[i] = mbuf;\n+\t}\n+\n+\trxq->nb_rx_hold = 0;\n+\trxq->rx_tail = rxq->nb_rx_desc - 8;\n+\n+\treturn 0;\n+}\n+\n static int\n idpf_alloc_split_rxq_mbufs(struct idpf_rx_queue *rxq)\n {\n@@ -1102,22 +1554,31 @@ idpf_rx_queue_init(struct rte_eth_dev *dev, uint16_t rx_queue_id)\n \t\tIECM_PCI_REG_WRITE(rxq->qrx_tail, rxq->nb_rx_desc - 1);\n \t} else {\n \t\t/* Split queue */\n-\t\terr = idpf_alloc_split_rxq_mbufs(rxq->bufq1);\n+\t\tif (rxq->hairpin_q)\n+\t\t\terr = idpf_alloc_split_p2p_rxq_mbufs(rxq->bufq1);\n+\t\telse\n+\t\t\terr = idpf_alloc_split_rxq_mbufs(rxq->bufq1);\n \t\tif (err) {\n \t\t\tPMD_DRV_LOG(ERR, \"Failed to allocate RX buffer queue mbuf\");\n \t\t\treturn err;\n \t\t}\n+\n+\t\tif (!rxq->bufq2)\n+\t\t\tgoto doorbell;\n+\n \t\terr = idpf_alloc_split_rxq_mbufs(rxq->bufq2);\n \t\tif (err) {\n \t\t\tPMD_DRV_LOG(ERR, \"Failed to allocate RX buffer queue mbuf\");\n \t\t\treturn err;\n \t\t}\n \n+doorbell:\n \t\trte_wmb();\n \n \t\t/* Init the RX tail register. */\n \t\tIECM_PCI_REG_WRITE(rxq->bufq1->qrx_tail, rxq->bufq1->rx_tail);\n-\t\tIECM_PCI_REG_WRITE(rxq->bufq2->qrx_tail, rxq->bufq2->rx_tail);\n+\t\tif (rxq->bufq2)\n+\t\t\tIECM_PCI_REG_WRITE(rxq->bufq2->qrx_tail, rxq->bufq2->rx_tail);\n \t}\n \n \treturn err;\n@@ -1225,7 +1686,11 @@ idpf_rx_queue_stop(struct rte_eth_dev *dev, uint16_t rx_queue_id)\n \tif (rx_queue_id >= dev->data->nb_rx_queues)\n \t\treturn -EINVAL;\n \n-\terr = idpf_switch_queue(vport, rx_queue_id, true, false);\n+\trxq = dev->data->rx_queues[rx_queue_id];\n+\tif (rxq->hairpin_q)\n+\t\terr = idpf_switch_hairpin_queue(vport, rx_queue_id, true, false);\n+\telse\n+\t\terr = idpf_switch_queue(vport, rx_queue_id, true, false);\n \tif (err) {\n \t\tPMD_DRV_LOG(ERR, \"Failed to switch RX queue %u off\",\n \t\t\t rx_queue_id);\n@@ -1238,10 +1703,18 @@ idpf_rx_queue_stop(struct rte_eth_dev *dev, uint16_t rx_queue_id)\n \t\treset_single_rx_queue(rxq);\n \t} else {\n \t\trxq->bufq1->ops->release_mbufs(rxq->bufq1);\n-\t\trxq->bufq2->ops->release_mbufs(rxq->bufq2);\n-\t\treset_split_rx_queue(rxq);\n+\t\tif (rxq->bufq2)\n+\t\t\trxq->bufq2->ops->release_mbufs(rxq->bufq2);\n+\n+\t\tif (rxq->hairpin_q) {\n+\t\t\treset_rx_hairpin_descq(rxq);\n+\t\t\treset_rx_hairpin_bufq(rxq->bufq1);\n+\t\t} else {\n+\t\t\treset_split_rx_queue(rxq);\n+\t\t}\n \t}\n-\tdev->data->rx_queue_state[rx_queue_id] = RTE_ETH_QUEUE_STATE_STOPPED;\n+\tif (!rxq->hairpin_q)\n+\t\tdev->data->rx_queue_state[rx_queue_id] = RTE_ETH_QUEUE_STATE_STOPPED;\n \n \treturn 0;\n }\n@@ -1258,22 +1731,31 @@ idpf_tx_queue_stop(struct rte_eth_dev *dev, uint16_t tx_queue_id)\n \tif (tx_queue_id >= dev->data->nb_tx_queues)\n \t\treturn -EINVAL;\n \n-\terr = idpf_switch_queue(vport, tx_queue_id, false, false);\n+\ttxq = dev->data->tx_queues[tx_queue_id];\n+\tif (txq->hairpin_q)\n+\t\terr = idpf_switch_hairpin_queue(vport, tx_queue_id, false, false);\n+\telse\n+\t\terr = idpf_switch_queue(vport, tx_queue_id, false, false);\n \tif (err) {\n \t\tPMD_DRV_LOG(ERR, \"Failed to switch TX queue %u off\",\n \t\t\t tx_queue_id);\n \t\treturn err;\n \t}\n \n-\ttxq = dev->data->tx_queues[tx_queue_id];\n \ttxq->ops->release_mbufs(txq);\n \tif (vport->txq_model == VIRTCHNL2_QUEUE_MODEL_SINGLE) {\n \t\treset_single_tx_queue(txq);\n \t} else {\n-\t\treset_split_tx_descq(txq);\n-\t\treset_split_tx_complq(txq->complq);\n+\t\tif (txq->hairpin_q) {\n+\t\t\treset_tx_hairpin_descq(txq);\n+\t\t\treset_tx_hairpin_complq(txq->complq);\n+\t\t} else {\n+\t\t\treset_split_tx_descq(txq);\n+\t\t\treset_split_tx_complq(txq->complq);\n+\t\t}\n \t}\n-\tdev->data->tx_queue_state[tx_queue_id] = RTE_ETH_QUEUE_STATE_STOPPED;\n+\tif (!txq->hairpin_q)\n+\t\tdev->data->tx_queue_state[tx_queue_id] = RTE_ETH_QUEUE_STATE_STOPPED;\n \n \treturn 0;\n }\n@@ -1302,6 +1784,7 @@ idpf_stop_queues(struct rte_eth_dev *dev)\n \t\tif (!rxq)\n \t\t\tcontinue;\n \n+\t\t /* hairpin queue is also stopped here. */\n \t\tif (idpf_rx_queue_stop(dev, i))\n \t\t\tPMD_DRV_LOG(WARNING, \"Fail to stop Rx queue %d\", i);\n \t}\ndiff --git a/drivers/net/idpf/idpf_rxtx.h b/drivers/net/idpf/idpf_rxtx.h\nindex 6fcb441143..285d333a34 100644\n--- a/drivers/net/idpf/idpf_rxtx.h\n+++ b/drivers/net/idpf/idpf_rxtx.h\n@@ -55,6 +55,7 @@\n #define IDPF_MIN_RING_DESC\t32\n #define IDPF_MAX_RING_DESC\t4096\n #define IDPF_DMA_MEM_ALIGN\t4096\n+#define IDPF_P2P_DESC_LEN\t16\n /* Base address of the HW descriptor ring should be 128B aligned. */\n #define IDPF_RING_BASE_ALIGN\t128\n \n@@ -103,6 +104,11 @@\n \n extern uint64_t idpf_timestamp_dynflag;\n \n+struct virtchnl2_p2p_rx_buf_desc {\n+\t__le64 reserve0;\n+\t__le64 pkt_addr; /* Packet buffer address */\n+};\n+\n struct idpf_rx_queue {\n \tstruct idpf_adapter *adapter;\t/* the adapter this queue belongs to */\n \tstruct rte_mempool *mp;\t\t/* mbuf pool to populate Rx ring */\n@@ -138,6 +144,12 @@ struct idpf_rx_queue {\n \tuint16_t max_pkt_len; /* Maximum packet length */\n \tuint8_t crc_len;\t/* 0 if CRC stripped, 4 otherwise */\n \tuint8_t rxdid;\n+\tuint8_t q_type;\n+\n+\tbool hairpin_q;\t\t/* if rx queue is a hairpin queue */\n+\t/* only valid if the hairpin queue pair crosses vport */\n+\tbool hairpin_cv;\n+\tuint16_t peer_txp;\n \n \tbool q_set;\t\t/* if rx queue has been configured */\n \tbool q_started;\t\t/* if rx queue has been started */\n@@ -186,6 +198,7 @@ struct idpf_tx_queue {\n \n \tuint16_t port_id;\n \tuint16_t queue_id;\n+\tuint8_t q_type;\n \tuint64_t offloads;\n \tuint16_t next_dd;\t/* next to set RS, for VPMD */\n \tuint16_t next_rs;\t/* next to check DD, for VPMD */\n@@ -205,6 +218,14 @@ struct idpf_tx_queue {\n \tuint32_t tx_start_qid;\n \tuint8_t expected_gen_id;\n \tstruct idpf_tx_queue *complq;\n+\n+\t/* only valid for hairpin queue */\n+\tbool hairpin_q;\n+\t/* only valid if the hairpin queue pair crosses vport */\n+\tbool hairpin_cv;\n+\tuint16_t peer_rxq_id;\n+\tuint16_t peer_rxp;\n+\tbool bound;\n };\n \n /* Offload features */\n@@ -242,6 +263,12 @@ int idpf_tx_queue_setup(struct rte_eth_dev *dev, uint16_t queue_idx,\n int idpf_tx_queue_init(struct rte_eth_dev *dev, uint16_t tx_queue_id);\n int idpf_tx_queue_start(struct rte_eth_dev *dev, uint16_t tx_queue_id);\n int idpf_tx_queue_stop(struct rte_eth_dev *dev, uint16_t tx_queue_id);\n+int idpf_rx_hairpin_queue_setup(struct rte_eth_dev *dev, uint16_t queue_idx,\n+\t\t\t\tuint16_t nb_desc,\n+\t\t\t\tconst struct rte_eth_hairpin_conf *conf);\n+int idpf_tx_hairpin_queue_setup(struct rte_eth_dev *dev, uint16_t queue_idx,\n+\t\t\t\tuint16_t nb_desc,\n+\t\t\t\tconst struct rte_eth_hairpin_conf *conf);\n void idpf_dev_tx_queue_release(struct rte_eth_dev *dev, uint16_t qid);\n \n uint16_t idpf_singleq_recv_pkts(void *rx_queue, struct rte_mbuf **rx_pkts,\ndiff --git a/drivers/net/idpf/idpf_vchnl.c b/drivers/net/idpf/idpf_vchnl.c\nindex 97fcfb574b..db80293fbd 100644\n--- a/drivers/net/idpf/idpf_vchnl.c\n+++ b/drivers/net/idpf/idpf_vchnl.c\n@@ -585,6 +585,8 @@ idpf_get_caps(struct idpf_adapter *adapter)\n \t\t VIRTCHNL2_CAP_WB_ON_ITR\t\t|\n \t\t VIRTCHNL2_CAP_PROMISC\t\t\t|\n \t\t VIRTCHNL2_CAP_LINK_SPEED\t\t|\n+\t\t VIRTCHNL2_CAP_PTP\t\t\t|\n+\t\t VIRTCHNL2_CAP_RX_FLEX_DESC\t\t|\n \t\t VIRTCHNL2_CAP_VLAN;\n \n \targs.ops = VIRTCHNL2_OP_GET_CAPS;\n@@ -813,7 +815,7 @@ idpf_config_rxqs(struct idpf_vport *vport)\n \t\t\tfor (i = 0; i < num_qs; i++, k++) {\n \t\t\t\trxq_info = &vc_rxqs->qinfo[i];\n \t\t\t\trxq_info->dma_ring_addr = rxq[k]->rx_ring_phys_addr;\n-\t\t\t\trxq_info->type = VIRTCHNL2_QUEUE_TYPE_RX;\n+\t\t\t\trxq_info->type = rxq[k]->q_type;\n \t\t\t\trxq_info->queue_id = rxq[k]->queue_id;\n \t\t\t\trxq_info->model = VIRTCHNL2_QUEUE_MODEL_SINGLE;\n \t\t\t\trxq_info->data_buffer_size = rxq[k]->rx_buf_len;\n@@ -830,7 +832,7 @@ idpf_config_rxqs(struct idpf_vport *vport)\n \t\t\t\trxq_info = &vc_rxqs->qinfo[i * 3];\n \t\t\t\trxq_info->dma_ring_addr =\n \t\t\t\t\trxq[k]->rx_ring_phys_addr;\n-\t\t\t\trxq_info->type = VIRTCHNL2_QUEUE_TYPE_RX;\n+\t\t\t\trxq_info->type = rxq[k]->q_type;\n \t\t\t\trxq_info->queue_id = rxq[k]->queue_id;\n \t\t\t\trxq_info->model = VIRTCHNL2_QUEUE_MODEL_SPLIT;\n \t\t\t\trxq_info->data_buffer_size = rxq[k]->rx_buf_len;\n@@ -851,8 +853,7 @@ idpf_config_rxqs(struct idpf_vport *vport)\n \t\t\t\t\trxq_info = &vc_rxqs->qinfo[i * 3 + j];\n \t\t\t\t\trxq_info->dma_ring_addr =\n \t\t\t\t\t\tbufq->rx_ring_phys_addr;\n-\t\t\t\t\trxq_info->type =\n-\t\t\t\t\t\tVIRTCHNL2_QUEUE_TYPE_RX_BUFFER;\n+\t\t\t\t\trxq_info->type = bufq->q_type;\n \t\t\t\t\trxq_info->queue_id = bufq->queue_id;\n \t\t\t\t\trxq_info->model = VIRTCHNL2_QUEUE_MODEL_SPLIT;\n \t\t\t\t\trxq_info->data_buffer_size = bufq->rx_buf_len;\n@@ -898,6 +899,8 @@ idpf_config_rxq(struct idpf_vport *vport, uint16_t rxq_id)\n \n \tif (vport->rxq_model == VIRTCHNL2_QUEUE_MODEL_SINGLE)\n \t\tnum_qs = IDPF_RXQ_PER_GRP;\n+\telse if (rxq[rxq_id]->hairpin_q)\n+\t\tnum_qs = IDPF_RXQ_PER_GRP + 1;\n \telse\n \t\tnum_qs = IDPF_RXQ_PER_GRP + IDPF_RX_BUFQ_PER_GRP;\n \n@@ -914,7 +917,7 @@ idpf_config_rxq(struct idpf_vport *vport, uint16_t rxq_id)\n \tif (vport->rxq_model == VIRTCHNL2_QUEUE_MODEL_SINGLE) {\n \t\trxq_info = &vc_rxqs->qinfo[0];\n \t\trxq_info->dma_ring_addr = rxq[rxq_id]->rx_ring_phys_addr;\n-\t\trxq_info->type = VIRTCHNL2_QUEUE_TYPE_RX;\n+\t\trxq_info->type = rxq[rxq_id]->q_type;\n \t\trxq_info->queue_id = rxq[rxq_id]->queue_id;\n \t\trxq_info->model = VIRTCHNL2_QUEUE_MODEL_SINGLE;\n \t\trxq_info->data_buffer_size = rxq[rxq_id]->rx_buf_len;\n@@ -925,38 +928,72 @@ idpf_config_rxq(struct idpf_vport *vport, uint16_t rxq_id)\n \n \t\trxq_info->ring_len = rxq[rxq_id]->nb_rx_desc;\n \t} else {\n-\t\t/* Rx queue */\n-\t\trxq_info = &vc_rxqs->qinfo[0];\n-\t\trxq_info->dma_ring_addr = rxq[rxq_id]->rx_ring_phys_addr;\n-\t\trxq_info->type = VIRTCHNL2_QUEUE_TYPE_RX;\n-\t\trxq_info->queue_id = rxq[rxq_id]->queue_id;\n-\t\trxq_info->model = VIRTCHNL2_QUEUE_MODEL_SPLIT;\n-\t\trxq_info->data_buffer_size = rxq[rxq_id]->rx_buf_len;\n-\t\trxq_info->max_pkt_size = vport->max_pkt_len;\n+\t\tif (rxq[rxq_id]->hairpin_q) {\n+\t\t\t/* Rx queue */\n+\t\t\trxq_info = &vc_rxqs->qinfo[0];\n+\t\t\trxq_info->type = rxq[rxq_id]->q_type;\n+\t\t\trxq_info->queue_id = rxq[rxq_id]->queue_id;\n+\t\t\trxq_info->ring_len = rxq[rxq_id]->nb_rx_desc;\n+\t\t\trxq_info->dma_ring_addr = rxq[rxq_id]->rx_ring_phys_addr;\n+\t\t\trxq_info->rx_bufq1_id = rxq[rxq_id]->bufq1->queue_id;\n+\t\t\trxq_info->max_pkt_size = vport->max_pkt_len;\n+\t\t\trxq_info->desc_ids = VIRTCHNL2_RXDID_2_FLEX_SPLITQ_M;\n+\t\t\trxq_info->qflags |= VIRTCHNL2_RX_DESC_SIZE_32BYTE;\n \n-\t\trxq_info->desc_ids = VIRTCHNL2_RXDID_2_FLEX_SPLITQ_M;\n-\t\trxq_info->qflags |= VIRTCHNL2_RX_DESC_SIZE_32BYTE;\n+\t\t\trxq_info->data_buffer_size = rxq[rxq_id]->rx_buf_len;\n+\t\t\trxq_info->model = VIRTCHNL2_QUEUE_MODEL_SPLIT;\n+\t\t\trxq_info->rx_buffer_low_watermark = 64;\n+\t\t\tPMD_DRV_LOG(NOTICE, \"hairpin: vport %u, Rxq id 0x%x\",\n+\t\t\t\tvport->vport_id, rxq_info->queue_id);\n \n-\t\trxq_info->ring_len = rxq[rxq_id]->nb_rx_desc;\n-\t\trxq_info->rx_bufq1_id = rxq[rxq_id]->bufq1->queue_id;\n-\t\trxq_info->rx_bufq2_id = rxq[rxq_id]->bufq2->queue_id;\n-\t\trxq_info->rx_buffer_low_watermark = 64;\n-\n-\t\t/* Buffer queue */\n-\t\tfor (i = 1; i <= IDPF_RX_BUFQ_PER_GRP; i++) {\n-\t\t\tstruct idpf_rx_queue *bufq =\n-\t\t\t\ti == 1 ? rxq[rxq_id]->bufq1 : rxq[rxq_id]->bufq2;\n-\t\t\trxq_info = &vc_rxqs->qinfo[i];\n-\t\t\trxq_info->dma_ring_addr = bufq->rx_ring_phys_addr;\n-\t\t\trxq_info->type = VIRTCHNL2_QUEUE_TYPE_RX_BUFFER;\n+\t\t\t/* Buffer queue */\n+\t\t\trxq_info = &vc_rxqs->qinfo[1];\n+\t\t\tstruct idpf_rx_queue *bufq = rxq[rxq_id]->bufq1;\n+\t\t\trxq_info->type = bufq->q_type;\n \t\t\trxq_info->queue_id = bufq->queue_id;\n+\t\t\trxq_info->ring_len = bufq->nb_rx_desc;\n+\t\t\trxq_info->dma_ring_addr = bufq->rx_ring_phys_addr;\n+\t\t\trxq_info->desc_ids = VIRTCHNL2_RXDID_2_FLEX_SPLITQ_M;\n+\t\t\trxq_info->rx_buffer_low_watermark = 64;\n \t\t\trxq_info->model = VIRTCHNL2_QUEUE_MODEL_SPLIT;\n \t\t\trxq_info->data_buffer_size = bufq->rx_buf_len;\n+\t\t\trxq_info->buffer_notif_stride = IDPF_RX_BUF_STRIDE;\n+\t\t\tPMD_DRV_LOG(NOTICE, \"hairpin: vport %u, Rxbufq id 0x%x\",\n+\t\t\t\tvport->vport_id, rxq_info->queue_id);\n+\t\t} else {\n+\t\t\t/* Rx queue */\n+\t\t\trxq_info = &vc_rxqs->qinfo[0];\n+\t\t\trxq_info->dma_ring_addr = rxq[rxq_id]->rx_ring_phys_addr;\n+\t\t\trxq_info->type = rxq[rxq_id]->q_type;\n+\t\t\trxq_info->queue_id = rxq[rxq_id]->queue_id;\n+\t\t\trxq_info->model = VIRTCHNL2_QUEUE_MODEL_SPLIT;\n+\t\t\trxq_info->data_buffer_size = rxq[rxq_id]->rx_buf_len;\n+\t\t\trxq_info->max_pkt_size = vport->max_pkt_len;\n+\n \t\t\trxq_info->desc_ids = VIRTCHNL2_RXDID_2_FLEX_SPLITQ_M;\n-\t\t\trxq_info->ring_len = bufq->nb_rx_desc;\n+\t\t\trxq_info->qflags |= VIRTCHNL2_RX_DESC_SIZE_32BYTE;\n \n-\t\t\trxq_info->buffer_notif_stride = IDPF_RX_BUF_STRIDE;\n+\t\t\trxq_info->ring_len = rxq[rxq_id]->nb_rx_desc;\n+\t\t\trxq_info->rx_bufq1_id = rxq[rxq_id]->bufq1->queue_id;\n+\t\t\trxq_info->rx_bufq2_id = rxq[rxq_id]->bufq2->queue_id;\n \t\t\trxq_info->rx_buffer_low_watermark = 64;\n+\n+\t\t\t/* Buffer queue */\n+\t\t\tfor (i = 1; i <= IDPF_RX_BUFQ_PER_GRP; i++) {\n+\t\t\t\tstruct idpf_rx_queue *bufq =\n+\t\t\t\t\ti == 1 ? rxq[rxq_id]->bufq1 : rxq[rxq_id]->bufq2;\n+\t\t\t\trxq_info = &vc_rxqs->qinfo[i];\n+\t\t\t\trxq_info->dma_ring_addr = bufq->rx_ring_phys_addr;\n+\t\t\t\trxq_info->type = bufq->q_type;\n+\t\t\t\trxq_info->queue_id = bufq->queue_id;\n+\t\t\t\trxq_info->model = VIRTCHNL2_QUEUE_MODEL_SPLIT;\n+\t\t\t\trxq_info->data_buffer_size = bufq->rx_buf_len;\n+\t\t\t\trxq_info->desc_ids = VIRTCHNL2_RXDID_2_FLEX_SPLITQ_M;\n+\t\t\t\trxq_info->ring_len = bufq->nb_rx_desc;\n+\n+\t\t\t\trxq_info->buffer_notif_stride = IDPF_RX_BUF_STRIDE;\n+\t\t\t\trxq_info->rx_buffer_low_watermark = 64;\n+\t\t\t}\n \t\t}\n \t}\n \n@@ -1011,7 +1048,7 @@ idpf_config_txqs(struct idpf_vport *vport)\n \t\t\tfor (i = 0; i < num_qs; i++, k++) {\n \t\t\t\ttxq_info = &vc_txqs->qinfo[i];\n \t\t\t\ttxq_info->dma_ring_addr = txq[k]->tx_ring_phys_addr;\n-\t\t\t\ttxq_info->type = VIRTCHNL2_QUEUE_TYPE_TX;\n+\t\t\t\ttxq_info->type = txq[k]->q_type;\n \t\t\t\ttxq_info->queue_id = txq[k]->queue_id;\n \t\t\t\ttxq_info->model = VIRTCHNL2_QUEUE_MODEL_SINGLE;\n \t\t\t\ttxq_info->sched_mode = VIRTCHNL2_TXQ_SCHED_MODE_QUEUE;\n@@ -1022,7 +1059,7 @@ idpf_config_txqs(struct idpf_vport *vport)\n \t\t\t\t/* txq info */\n \t\t\t\ttxq_info = &vc_txqs->qinfo[2 * i];\n \t\t\t\ttxq_info->dma_ring_addr = txq[k]->tx_ring_phys_addr;\n-\t\t\t\ttxq_info->type = VIRTCHNL2_QUEUE_TYPE_TX;\n+\t\t\t\ttxq_info->type = txq[k]->q_type;\n \t\t\t\ttxq_info->queue_id = txq[k]->queue_id;\n \t\t\t\ttxq_info->model = VIRTCHNL2_QUEUE_MODEL_SPLIT;\n \t\t\t\ttxq_info->sched_mode = VIRTCHNL2_TXQ_SCHED_MODE_FLOW;\n@@ -1035,7 +1072,7 @@ idpf_config_txqs(struct idpf_vport *vport)\n \t\t\t\ttxq_info = &vc_txqs->qinfo[2 * i + 1];\n \t\t\t\ttxq_info->dma_ring_addr =\n \t\t\t\t\ttxq[k]->complq->tx_ring_phys_addr;\n-\t\t\t\ttxq_info->type = VIRTCHNL2_QUEUE_TYPE_TX_COMPLETION;\n+\t\t\t\ttxq_info->type = txq[k]->complq->q_type;\n \t\t\t\ttxq_info->queue_id = txq[k]->complq->queue_id;\n \t\t\t\ttxq_info->model = VIRTCHNL2_QUEUE_MODEL_SPLIT;\n \t\t\t\ttxq_info->sched_mode = VIRTCHNL2_TXQ_SCHED_MODE_FLOW;\n@@ -1092,31 +1129,61 @@ idpf_config_txq(struct idpf_vport *vport, uint16_t txq_id)\n \tif (vport->txq_model == VIRTCHNL2_QUEUE_MODEL_SINGLE) {\n \t\ttxq_info = &vc_txqs->qinfo[0];\n \t\ttxq_info->dma_ring_addr = txq[txq_id]->tx_ring_phys_addr;\n-\t\ttxq_info->type = VIRTCHNL2_QUEUE_TYPE_TX;\n+\t\ttxq_info->type = txq[txq_id]->q_type;\n \t\ttxq_info->queue_id = txq[txq_id]->queue_id;\n \t\ttxq_info->model = VIRTCHNL2_QUEUE_MODEL_SINGLE;\n \t\ttxq_info->sched_mode = VIRTCHNL2_TXQ_SCHED_MODE_QUEUE;\n \t\ttxq_info->ring_len = txq[txq_id]->nb_tx_desc;\n \t} else {\n-\t\t/* txq info */\n-\t\ttxq_info = &vc_txqs->qinfo[0];\n-\t\ttxq_info->dma_ring_addr = txq[txq_id]->tx_ring_phys_addr;\n-\t\ttxq_info->type = VIRTCHNL2_QUEUE_TYPE_TX;\n-\t\ttxq_info->queue_id = txq[txq_id]->queue_id;\n-\t\ttxq_info->model = VIRTCHNL2_QUEUE_MODEL_SPLIT;\n-\t\ttxq_info->sched_mode = VIRTCHNL2_TXQ_SCHED_MODE_FLOW;\n-\t\ttxq_info->ring_len = txq[txq_id]->nb_tx_desc;\n-\t\ttxq_info->tx_compl_queue_id = txq[txq_id]->complq->queue_id;\n-\t\ttxq_info->relative_queue_id = txq_info->queue_id;\n-\n-\t\t/* tx completion queue info */\n-\t\ttxq_info = &vc_txqs->qinfo[1];\n-\t\ttxq_info->dma_ring_addr = txq[txq_id]->complq->tx_ring_phys_addr;\n-\t\ttxq_info->type = VIRTCHNL2_QUEUE_TYPE_TX_COMPLETION;\n-\t\ttxq_info->queue_id = txq[txq_id]->complq->queue_id;\n-\t\ttxq_info->model = VIRTCHNL2_QUEUE_MODEL_SPLIT;\n-\t\ttxq_info->sched_mode = VIRTCHNL2_TXQ_SCHED_MODE_FLOW;\n-\t\ttxq_info->ring_len = txq[txq_id]->complq->nb_tx_desc;\n+\t\tif (txq[txq_id]->hairpin_q) {\n+\t\t\t/* txq info */\n+\t\t\ttxq_info = &vc_txqs->qinfo[0];\n+\t\t\ttxq_info->dma_ring_addr = txq[txq_id]->tx_ring_phys_addr;\n+\t\t\ttxq_info->type = txq[txq_id]->q_type;\n+\t\t\ttxq_info->queue_id = txq[txq_id]->queue_id;\n+\t\t\ttxq_info->ring_len = txq[txq_id]->nb_tx_desc;\n+\t\t\ttxq_info->tx_compl_queue_id = txq[txq_id]->complq->queue_id;\n+\t\t\ttxq_info->relative_queue_id = txq_info->queue_id;\n+\t\t\ttxq_info->peer_rx_queue_id = txq[txq_id]->peer_rxq_id;\n+\t\t\ttxq_info->model = VIRTCHNL2_QUEUE_MODEL_SPLIT;\n+\t\t\ttxq_info->sched_mode = VIRTCHNL2_TXQ_SCHED_MODE_FLOW;\n+\t\t\tPMD_DRV_LOG(NOTICE, \"hairpin: vport %u, Txq id 0x%x, peer\"\n+\t\t\t\t\" rxq id 0x%x\", vport->vport_id, txq_info->queue_id,\n+\t\t\t\ttxq_info->peer_rx_queue_id);\n+\n+\t\t\t/* tx completion queue info */\n+\t\t\ttxq_info = &vc_txqs->qinfo[1];\n+\t\t\ttxq_info->dma_ring_addr = txq[txq_id]->complq->tx_ring_phys_addr;\n+\t\t\ttxq_info->type = txq[txq_id]->complq->q_type;\n+\t\t\ttxq_info->queue_id = txq[txq_id]->complq->queue_id;\n+\t\t\ttxq_info->ring_len = txq[txq_id]->complq->nb_tx_desc;\n+\t\t\ttxq_info->peer_rx_queue_id = txq[txq_id]->complq->peer_rxq_id;\n+\t\t\ttxq_info->model = VIRTCHNL2_QUEUE_MODEL_SPLIT;\n+\t\t\ttxq_info->sched_mode = VIRTCHNL2_TXQ_SCHED_MODE_FLOW;\n+\t\t\tPMD_DRV_LOG(NOTICE, \"hairpin: vport %u, Txcomplq id 0x%x,\"\n+\t\t\t\t\" peer rxbufq id 0x%x\", vport->vport_id,\n+\t\t\t\ttxq_info->queue_id, txq_info->peer_rx_queue_id);\n+\t\t} else {\n+\t\t\t/* txq info */\n+\t\t\ttxq_info = &vc_txqs->qinfo[0];\n+\t\t\ttxq_info->dma_ring_addr = txq[txq_id]->tx_ring_phys_addr;\n+\t\t\ttxq_info->type = txq[txq_id]->q_type;\n+\t\t\ttxq_info->queue_id = txq[txq_id]->queue_id;\n+\t\t\ttxq_info->model = VIRTCHNL2_QUEUE_MODEL_SPLIT;\n+\t\t\ttxq_info->sched_mode = VIRTCHNL2_TXQ_SCHED_MODE_FLOW;\n+\t\t\ttxq_info->ring_len = txq[txq_id]->nb_tx_desc;\n+\t\t\ttxq_info->tx_compl_queue_id = txq[txq_id]->complq->queue_id;\n+\t\t\ttxq_info->relative_queue_id = txq_info->queue_id;\n+\n+\t\t\t/* tx completion queue info */\n+\t\t\ttxq_info = &vc_txqs->qinfo[1];\n+\t\t\ttxq_info->dma_ring_addr = txq[txq_id]->complq->tx_ring_phys_addr;\n+\t\t\ttxq_info->type = txq[txq_id]->complq->q_type;\n+\t\t\ttxq_info->queue_id = txq[txq_id]->complq->queue_id;\n+\t\t\ttxq_info->model = VIRTCHNL2_QUEUE_MODEL_SPLIT;\n+\t\t\ttxq_info->sched_mode = VIRTCHNL2_TXQ_SCHED_MODE_FLOW;\n+\t\t\ttxq_info->ring_len = txq[txq_id]->complq->nb_tx_desc;\n+\t\t}\n \t}\n \n \tmemset(&args, 0, sizeof(args));\n@@ -1322,6 +1389,46 @@ idpf_switch_queue(struct idpf_vport *vport, uint16_t qid,\n \treturn err;\n }\n \n+int\n+idpf_switch_hairpin_queue(struct idpf_vport *vport, uint16_t qid,\n+\t\t bool rx, bool on)\n+{\n+\tuint32_t type;\n+\tint err, queue_id;\n+\n+\ttype = rx ? VIRTCHNL2_QUEUE_TYPE_P2P_RX : VIRTCHNL2_QUEUE_TYPE_P2P_TX;\n+\n+\t/* switch p2p txq/rxq */\n+\tif (type == VIRTCHNL2_QUEUE_TYPE_P2P_RX)\n+\t\tqueue_id = vport->chunks_info.rx_start_qid + qid;\n+\telse\n+\t\tqueue_id = vport->chunks_info.tx_start_qid + qid;\n+\terr = idpf_ena_dis_one_queue(vport, queue_id, type, on);\n+\tif (err)\n+\t\treturn err;\n+\n+\t/* switch p2p tx completion queue */\n+\tif (!rx && vport->txq_model == VIRTCHNL2_QUEUE_MODEL_SPLIT) {\n+\t\ttype = VIRTCHNL2_QUEUE_TYPE_P2P_TX_COMPLETION;\n+\t\tqueue_id = vport->chunks_info.tx_compl_start_qid + qid;\n+\t\terr = idpf_ena_dis_one_queue(vport, queue_id, type, on);\n+\t\tif (err)\n+\t\t\treturn err;\n+\t}\n+\n+\t/* switch p2p rx buffer queue */\n+\tif (rx && vport->rxq_model == VIRTCHNL2_QUEUE_MODEL_SPLIT) {\n+\t\ttype = VIRTCHNL2_QUEUE_TYPE_P2P_RX_BUFFER;\n+\t\tqueue_id = vport->chunks_info.rx_buf_start_qid + 2 * qid;\n+\t\terr = idpf_ena_dis_one_queue(vport, queue_id, type, on);\n+\t\tif (err)\n+\t\t\treturn err;\n+\t}\n+\n+\n+\treturn err;\n+}\n+\n #define IDPF_RXTX_QUEUE_CHUNKS_NUM\t2\n int\n idpf_ena_dis_queues(struct idpf_vport *vport, bool enable)\n", "prefixes": [ "RFC", "v2", "2/3" ] }