From patchwork Sat Sep 9 12:23:47 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Wei Hu X-Patchwork-Id: 131316 X-Patchwork-Delegate: ferruh.yigit@amd.com Return-Path: X-Original-To: patchwork@inbox.dpdk.org Delivered-To: patchwork@inbox.dpdk.org Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id C266C42554; Sat, 9 Sep 2023 14:24:22 +0200 (CEST) Received: from mails.dpdk.org (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id 405494029E; Sat, 9 Sep 2023 14:24:22 +0200 (CEST) Received: from DM6FTOPR00CU001.outbound.protection.outlook.com (mail-centralusazon11020014.outbound.protection.outlook.com [52.101.61.14]) by mails.dpdk.org (Postfix) with ESMTP id CA7D140295; Sat, 9 Sep 2023 14:24:20 +0200 (CEST) ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=MKswq2nNNS6MEeFPv5ERglrhRG39xKjRYPLdv4ARLoWJ7nLh78STaotk5eo6w25wQt7dcs9gRrxO3O38FC+FMWU9R8k/fIzrPPRBUfhSIBn1kag2vlkOqslohZnFv57WN2pPYxuXsVsKEOdjpSmXjt2nSOVVaV7weBLR1aLXFCc2qxmA1yZT4oKHVWT7+HeRRgDk5pC5i1Ue4EC1qnojPGCFPW/g0UcltquLuGmrmeUSIiE2gR1GAkuCSNLpd4MW6j5cvcAzj4W6N/4r6Ro1HGfbDIcnb9NV/WiJ1eiueTFgtYbK2i5wqGpu+ZhSO9QDYBHao82IOxwA6EfNtFxXmA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=MXvY45+eKed6elwK3UdrAJlUraLR0v+kEIX5bf0j/ik=; b=Yy4nkUaQ1SBiwRRGQjkJEoRDdOzfiwrWY9VR+Rbmqwzw50IhzDKHkvq/hWpSYykkUHWiRLnrsE+o6IpLQm6CDfacByGtas6ON0vmeNLTWsGHgPoIJezTw2V9Vb0MZkKA98UI67pA+M02WkfQnOFp2/F46XYFHjdtfTe0SXsG5ix7cYtpqfCNg2jYgCL999Dcp9fCGh8f7RYhAE4R+SRy38x2na3V3dxy1qBZbRU1SWttY3et1Z1kSwd9tp65QOsF/nRfncWCiPRJeI0ULLRWYK+P2e75MjKrNFZhZ0l/G7kKwFVgyA+ohTic3S2XsR+b1XD6AJnyqzpkTRhyu5zYPg== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=microsoft.com; dmarc=pass action=none header.from=microsoft.com; dkim=pass header.d=microsoft.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=selector2; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=MXvY45+eKed6elwK3UdrAJlUraLR0v+kEIX5bf0j/ik=; b=by5Px4pNKuOiLQd/AEWhTmfr8HbF7KsVtsu5e3STZCXXHjt1fJsyhYsVrFJXOdioX/pWKIgsO+jMZGu4NePTyB+KGfBhOawQOYC8FtgnPbDF0xzb70RMi904oWiRrkqwlUziahZ/9Z7nn4WEQvDW/WNhsDbZrN34h4A0L+FtHFQ= Authentication-Results: dkim=none (message not signed) header.d=none;dmarc=none action=none header.from=microsoft.com; Received: from MN2PR21MB1454.namprd21.prod.outlook.com (2603:10b6:208:208::11) by DM4PR21MB3731.namprd21.prod.outlook.com (2603:10b6:8:a0::5) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.6792.10; Sat, 9 Sep 2023 12:24:17 +0000 Received: from MN2PR21MB1454.namprd21.prod.outlook.com ([fe80::cbd6:26c1:4d57:b9fc]) by MN2PR21MB1454.namprd21.prod.outlook.com ([fe80::cbd6:26c1:4d57:b9fc%5]) with mapi id 15.20.6792.009; Sat, 9 Sep 2023 12:24:17 +0000 From: Wei Hu To: dev@dpdk.org, Long Li Cc: Wei Hu , stable@dpdk.org Subject: [PATCH 1/1] net/mana: add 32 bit short doorbell Date: Sat, 9 Sep 2023 12:23:47 +0000 Message-Id: <20230909122347.2043969-1-weh@microsoft.com> X-Mailer: git-send-email 2.25.1 X-ClientProxiedBy: PH8PR05CA0014.namprd05.prod.outlook.com (2603:10b6:510:2cc::7) To MN2PR21MB1454.namprd21.prod.outlook.com (2603:10b6:208:208::11) MIME-Version: 1.0 X-MS-Exchange-MessageSentRepresentingType: 1 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: MN2PR21MB1454:EE_|DM4PR21MB3731:EE_ X-MS-Office365-Filtering-Correlation-Id: c0997ecb-84e1-4fdb-9e66-08dbb12faf93 X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: H2S3sA35vzXYvFMLNOwgwpCjV5BmwADgxNHppx3G0628QL3Il5T4VuK8f3n0UPyG8qlZWL+y4ivrDQP2g8GjUMQf5CAVngnhUGTk93YbNeaVNqO9Km2NW01skrt7VSI6aelRF6+gXB3wRBTUVlOFIjN5zhg3nB6bnn5IdbmeCGyR4DV7mV3Fvv4MV+Dcc458UwhbVRHNC9c1WQhR9A+ugKqvfJlJKmQ2eptmo0yS+ttfDJKbOpE02WwLjVJSAmW+iU8+8SWSxbcWd9qCqOQJnmABfd0l4MFUD29HKupgjEAnJB8I7/vYxUgJ4BbTbRa16voLYKSzERteV8TakS6z0n/kDsBSDTSD7Wh5K8X02R3d3mrH4iEwCfyanaIVpgutIBAhhUezOtkNa4EwUDg556lHVDwRWRGrpbfIS0/EfHUUSVQ7eBFHK4qcWgt/9jB2uYySgrKJ7jwLrwffh9H68TkdPvoCGs4qnEtH2RlhPD2jcHPfvMxbsNgvcyG7lq6Bhbaurq23cRjGIh29mDOzSRqyLz913IocX/9Jdqqr5Bfg5KxDtirZ3B9MzxVgu1gyQGuIw/F9l13FxtCBlMd0+lCN2V2KuKVWLqZiqYxkLcTJKLwI34Cj4YrrL+xbOdAZsJQg22lTUUH6zBy+a4rfT/B9EhRrZT77Q5vlpLMaeiQ= X-Forefront-Antispam-Report: CIP:255.255.255.255; CTRY:; LANG:en; SCL:1; SRV:; IPV:NLI; SFV:NSPM; H:MN2PR21MB1454.namprd21.prod.outlook.com; PTR:; CAT:NONE; SFS:(13230031)(39860400002)(396003)(376002)(346002)(136003)(366004)(1800799009)(451199024)(186009)(30864003)(2906002)(38100700002)(38350700002)(36756003)(86362001)(82960400001)(82950400001)(41300700001)(8676002)(8936002)(6512007)(450100002)(2616005)(6486002)(66946007)(4326008)(52116002)(66556008)(66476007)(6636002)(37006003)(6506007)(316002)(6862004)(478600001)(10290500003)(26005)(6666004)(5660300002)(1076003)(83380400001); DIR:OUT; SFP:1102; X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1 X-MS-Exchange-AntiSpam-MessageData-0: P1FVVIyZrf4L//L92iVkSRpSeoqa9apGPwGRRB6/lV89nAGLrtwiSI6PgMdg7JKLgEAZZiEAEGaRBlx1p/jg+kZcsaJI9nKpBcO3N0udSjQSsi2gVAY5WDyqVLshk1kkI59DDYhuzI2l/99L+MzSosPS76X+thiNI394w0sX01slPeQFFdS4nSsqE7zZVGz+19EDR7oKT2qWVq5bZjCAmiOdbihYtLHAycWV/Jue5ecgCu+XDIiQ1lF26MFTUNy/+MaT4ZqmzTUSshnDZvd3wuX2+tHbzwRyJEWZ82SkN0N/x+2gHgMUzbClRqtN+aeGhHtGC9QsyuJ7jvPnZg6PgnTJ9ixqbZagvUUdSC9O2lTEhChZqnJ/QrogxEST/GXPCYqKkK5ABREgpo/3RLGywh6IvqmxzCAEE/4e4+SBocFXoHfoEi1egOa2zu+Elk6lmLn2mRENWzayP7SWdhpNlEI/ALZpkQPeGfYaRt18l8os284v6UnIzyVH1Hb5OVBN0UkLZoAIyKUajWIZLhS2OeGw3Lvm082dz2fcyEsyx64yhvvWaG8iJMNiK7xg/Hwp4wrp0VMfbgSFhfw4DGTa05I7dNPn7Wclru/c1NNi563ccPUNPnMKT3Z6046+9pXJG92BASxb6Blcwzjl0cPDzU7EjM1kEaT+BjyT7V1ezw2OV8sYTpCRTYMQ3cIHTYGsyvY++YkI6HrwfR1rZQ7l+J4RgJS8C2kz9lBpqnKYxXn08tzEe21BufwF/6HBMA5SdOy7k4Rhgldiq3mswLCU9hvvXL5Sk/BVHXjtAV/qRBx0WoXT4ubSL/QvEVXhM+XICuQFUf5ywZ9ZEqqwgZUhZaTNcJMuv7cAv0F3SqVeVdru2SY6Tf7kY5qXSZQ3M6NKyoBjV9M4/UiFbhcqljUQKcapMRUB9gB/jCr4aR8mZUQgnqqxWidW+j+dkK2ZiKCzctBYNgMLejq47KMVKtcAUD3PCjmqjnI2tdoyi9XQcERJiNdJSbEGHrXYoMNKFW2eyryNAsP1Us1UJwU+MCnOROGgFkGpDKdk9/p/OJ5F1WxbHmY1BSzqL2fklraZInwqygSfkXyVfwLC9Gc6DVIa45tj1ID+Fv/H1TxuzO1A6JT/TIddsQ5M4WHKkgvBT6CVXVkwdbIY5Nbnscy2S6c6j1e4k8ciSq323y7Qqebe50go1GqS/D6O/FQFcue4SB/qbmUkXnWbtWKYNhIGq18pm7bDK9MSc94McqDwYiX2IxpPJZV0IINCqm2a5Zt9Zs1raBXwQFz7kmo0Eow+j4iqTUP1rZg9LxYp7HCHmfUD9h30HKtrYdOe11iXtu89QPX4Wpx+pPJ8YoHEGmt5hSFTkf76k5ggwYR0C+taQ40OOfLIo5ZLsjL2RKnHEQmokHFAsPyDjtsDC8qg59N3O/dG6yTUWX/Hx6kvBNUkHLkhJYnLmr94t22XhxAqtYwMjM/MIRNcOMHo2dnkqtN5hh+liTseK/VjVYXOdRIbj5zVtVN1YesnZNZijde7MOgccm0RQhhVSuV8m8HKOUQxBESANsfNGWFAbveAfTpuaQgjQz1L0dNKIhfp3DIFBw/FuulB X-OriginatorOrg: microsoft.com X-MS-Exchange-CrossTenant-Network-Message-Id: c0997ecb-84e1-4fdb-9e66-08dbb12faf93 X-MS-Exchange-CrossTenant-AuthSource: MN2PR21MB1454.namprd21.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-OriginalArrivalTime: 09 Sep 2023 12:24:17.4661 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: 72f988bf-86f1-41af-91ab-2d7cd011db47 X-MS-Exchange-CrossTenant-MailboxType: HOSTED X-MS-Exchange-CrossTenant-UserPrincipalName: ONAGhxNuJo6h/BkVy/2PtqU44Tb5f3mZvoAdpM3y0R4xzG8F7D7C3sVtA0N50OIMJyfIlVEXyquVC0DhbPgXIg== X-MS-Exchange-Transport-CrossTenantHeadersStamped: DM4PR21MB3731 X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Add 32 bit short doorbell support. Ring short doorbell when running in 32 bit applicactions. Cc: stable@dpdk.org Signed-off-by: Wei Hu --- drivers/net/mana/gdma.c | 95 +++++++++++++++++++++++++++++++++++++++++ drivers/net/mana/mana.h | 25 +++++++++++ drivers/net/mana/rx.c | 52 ++++++++++++++++++++++ drivers/net/mana/tx.c | 28 ++++++++++++ 4 files changed, 200 insertions(+) diff --git a/drivers/net/mana/gdma.c b/drivers/net/mana/gdma.c index 65685fe236..d1da025d1b 100644 --- a/drivers/net/mana/gdma.c +++ b/drivers/net/mana/gdma.c @@ -166,6 +166,97 @@ gdma_post_work_request(struct mana_gdma_queue *queue, return 0; } +#ifdef RTE_ARCH_32 +union gdma_short_doorbell_entry { + uint32_t as_uint32; + + struct { + uint32_t tail_ptr_incr : 16; /* Number of CQEs */ + uint32_t id : 12; + uint32_t reserved : 3; + uint32_t arm : 1; + } cq; + + struct { + uint32_t tail_ptr_incr : 16; /* In number of bytes */ + uint32_t id : 12; + uint32_t reserved : 4; + } rq; + + struct { + uint32_t tail_ptr_incr : 16; /* In number of bytes */ + uint32_t id : 12; + uint32_t reserved : 4; + } sq; + + struct { + uint32_t tail_ptr_incr : 16; /* Number of EQEs */ + uint32_t id : 12; + uint32_t reserved : 3; + uint32_t arm : 1; + } eq; +}; /* HW DATA */ + +enum { + DOORBELL_SHORT_OFFSET_SQ = 0x10, + DOORBELL_SHORT_OFFSET_RQ = 0x410, + DOORBELL_SHORT_OFFSET_CQ = 0x810, + DOORBELL_SHORT_OFFSET_EQ = 0xFF0, +}; + +/* + * Write to hardware doorbell to notify new activity. + */ +int +mana_ring_short_doorbell(void *db_page, enum gdma_queue_types queue_type, + uint32_t queue_id, uint32_t tail_incr, uint8_t arm) +{ + uint8_t *addr = db_page; + union gdma_short_doorbell_entry e = {}; + + if ((queue_id & ~GDMA_SHORT_DB_QID_MASK) || + (tail_incr & ~GDMA_SHORT_DB_INC_MASK)) { + DP_LOG(ERR, "%s: queue_id %u or " + "tail_incr %u overflowed, queue type %d", + __func__, queue_id, tail_incr, queue_type); + return -EINVAL; + } + + switch (queue_type) { + case GDMA_QUEUE_SEND: + e.sq.id = queue_id; + e.sq.tail_ptr_incr = tail_incr; + addr += DOORBELL_SHORT_OFFSET_SQ; + break; + + case GDMA_QUEUE_RECEIVE: + e.rq.id = queue_id; + e.rq.tail_ptr_incr = tail_incr; + addr += DOORBELL_SHORT_OFFSET_RQ; + break; + + case GDMA_QUEUE_COMPLETION: + e.cq.id = queue_id; + e.cq.tail_ptr_incr = tail_incr; + e.cq.arm = arm; + addr += DOORBELL_SHORT_OFFSET_CQ; + break; + + default: + DP_LOG(ERR, "Unsupported queue type %d", queue_type); + return -1; + } + + /* Ensure all writes are done before ringing doorbell */ + rte_wmb(); + + DP_LOG(DEBUG, "db_page %p addr %p queue_id %u type %u tail %u arm %u", + db_page, addr, queue_id, queue_type, tail_incr, arm); + + rte_write32(e.as_uint32, addr); + return 0; +} +#else union gdma_doorbell_entry { uint64_t as_uint64; @@ -248,6 +339,7 @@ mana_ring_doorbell(void *db_page, enum gdma_queue_types queue_type, rte_write64(e.as_uint64, addr); return 0; } +#endif /* * Poll completion queue for completions. @@ -287,6 +379,9 @@ gdma_poll_completion_queue(struct mana_gdma_queue *cq, num_comp++; cq->head++; +#ifdef RTE_ARCH_32 + cq->head_incr_to_short_db++; +#endif DP_LOG(DEBUG, "comp new 0x%x old 0x%x cqe 0x%x wq %u sq %u head %u", new_owner_bits, old_owner_bits, cqe_owner_bits, diff --git a/drivers/net/mana/mana.h b/drivers/net/mana/mana.h index 5801491d75..848d87c096 100644 --- a/drivers/net/mana/mana.h +++ b/drivers/net/mana/mana.h @@ -50,6 +50,19 @@ struct mana_shared_data { #define MAX_TX_WQE_SIZE 512 #define MAX_RX_WQE_SIZE 256 +/* For 32 bit only */ +#ifdef RTE_ARCH_32 +#define GDMA_SHORT_DB_INC_MASK 0xffff +#define GDMA_SHORT_DB_QID_MASK 0xfff + +#define GDMA_SHORT_DB_MAX_WQE (0x10000 / GDMA_WQE_ALIGNMENT_UNIT_SIZE) + +#define TX_WQE_SHORT_DB_THRESHOLD \ + (GDMA_SHORT_DB_MAX_WQE - (2 * MAX_TX_WQE_SIZE)) +#define RX_WQE_SHORT_DB_THRESHOLD \ + (GDMA_SHORT_DB_MAX_WQE - (2 * MAX_RX_WQE_SIZE)) +#endif + /* Values from the GDMA specification document, WQE format description */ #define INLINE_OOB_SMALL_SIZE_IN_BYTES 8 #define INLINE_OOB_LARGE_SIZE_IN_BYTES 24 @@ -375,6 +388,9 @@ struct mana_gdma_queue { uint32_t id; uint32_t head; uint32_t tail; +#ifdef RTE_ARCH_32 + uint32_t head_incr_to_short_db; +#endif }; #define MANA_MR_BTREE_PER_QUEUE_N 64 @@ -425,6 +441,9 @@ struct mana_rxq { */ uint32_t desc_ring_head, desc_ring_tail; + /* For storing wqe increment count btw each short doorbell ring */ + uint32_t wqe_cnt_to_short_db; + struct mana_gdma_queue gdma_rq; struct mana_gdma_queue gdma_cq; struct gdma_comp *gdma_comp_buf; @@ -455,8 +474,14 @@ extern int mana_logtype_init; #define PMD_INIT_FUNC_TRACE() PMD_INIT_LOG(DEBUG, " >>") +#ifdef RTE_ARCH_32 +int mana_ring_short_doorbell(void *db_page, enum gdma_queue_types queue_type, + uint32_t queue_id, uint32_t tail_incr, + uint8_t arm); +#else int mana_ring_doorbell(void *db_page, enum gdma_queue_types queue_type, uint32_t queue_id, uint32_t tail, uint8_t arm); +#endif int mana_rq_ring_doorbell(struct mana_rxq *rxq); int gdma_post_work_request(struct mana_gdma_queue *queue, diff --git a/drivers/net/mana/rx.c b/drivers/net/mana/rx.c index 14d9085801..303d129e5b 100644 --- a/drivers/net/mana/rx.c +++ b/drivers/net/mana/rx.c @@ -39,10 +39,23 @@ mana_rq_ring_doorbell(struct mana_rxq *rxq) /* Hardware Spec specifies that software client should set 0 for * wqe_cnt for Receive Queues. */ +#ifdef RTE_ARCH_32 + if (rxq->wqe_cnt_to_short_db) { + ret = mana_ring_short_doorbell(db_page, GDMA_QUEUE_RECEIVE, + rxq->gdma_rq.id, + rxq->wqe_cnt_to_short_db * + GDMA_WQE_ALIGNMENT_UNIT_SIZE, + 0); + } else { + /* No need to ring, just return */ + ret = 0; + } +#else ret = mana_ring_doorbell(db_page, GDMA_QUEUE_RECEIVE, rxq->gdma_rq.id, rxq->gdma_rq.head * GDMA_WQE_ALIGNMENT_UNIT_SIZE, 0); +#endif if (ret) DP_LOG(ERR, "failed to ring RX doorbell ret %d", ret); @@ -97,6 +110,7 @@ mana_alloc_and_post_rx_wqe(struct mana_rxq *rxq) /* update queue for tracking pending packets */ desc->pkt = mbuf; desc->wqe_size_in_bu = wqe_size_in_bu; + rxq->wqe_cnt_to_short_db += wqe_size_in_bu; rxq->desc_ring_head = (rxq->desc_ring_head + 1) % rxq->num_desc; } else { DP_LOG(DEBUG, "failed to post recv ret %d", ret); @@ -115,12 +129,22 @@ mana_alloc_and_post_rx_wqes(struct mana_rxq *rxq) int ret; uint32_t i; +#ifdef RTE_ARCH_32 + rxq->wqe_cnt_to_short_db = 0; +#endif for (i = 0; i < rxq->num_desc; i++) { ret = mana_alloc_and_post_rx_wqe(rxq); if (ret) { DP_LOG(ERR, "failed to post RX ret = %d", ret); return ret; } + +#ifdef RTE_ARCH_32 + if (rxq->wqe_cnt_to_short_db > RX_WQE_SHORT_DB_THRESHOLD) { + mana_rq_ring_doorbell(rxq); + rxq->wqe_cnt_to_short_db = 0; + } +#endif } mana_rq_ring_doorbell(rxq); @@ -349,6 +373,9 @@ mana_start_rx_queues(struct rte_eth_dev *dev) /* CQ head starts with count */ rxq->gdma_cq.head = rxq->gdma_cq.count; +#ifdef RTE_ARCH_32 + rxq->gdma_cq.head_incr_to_short_db = 0; +#endif DRV_LOG(INFO, "rxq cq id %u buf %p count %u size %u", rxq->gdma_cq.id, rxq->gdma_cq.buffer, @@ -397,6 +424,10 @@ mana_rx_burst(void *dpdk_rxq, struct rte_mbuf **pkts, uint16_t pkts_n) uint32_t i; int polled = 0; +#ifdef RTE_ARCH_32 + rxq->wqe_cnt_to_short_db = 0; +#endif + repoll: /* Polling on new completions if we have no backlog */ if (rxq->comp_buf_idx == rxq->comp_buf_len) { @@ -505,6 +536,16 @@ mana_rx_burst(void *dpdk_rxq, struct rte_mbuf **pkts, uint16_t pkts_n) wqe_posted++; if (pkt_received == pkts_n) break; + +#ifdef RTE_ARCH_32 + /* Ring short doorbell if approaching the wqe increment + * limit. + */ + if (rxq->wqe_cnt_to_short_db > RX_WQE_SHORT_DB_THRESHOLD) { + mana_rq_ring_doorbell(rxq); + rxq->wqe_cnt_to_short_db = 0; + } +#endif } rxq->backlog_idx = pkt_idx; @@ -529,6 +570,16 @@ static int mana_arm_cq(struct mana_rxq *rxq, uint8_t arm) { struct mana_priv *priv = rxq->priv; +#ifdef RTE_ARCH_32 + uint16_t cqe_incr = (uint16_t)rxq->gdma_cq.head_incr_to_short_db; + + rxq->gdma_cq.head_incr_to_short_db = 0; + DP_LOG(DEBUG, "Ringing completion queue ID %u incr %u arm %d", + rxq->gdma_cq.id, cqe_incr, arm); + + return mana_ring_short_doorbell(priv->db_page, GDMA_QUEUE_COMPLETION, + rxq->gdma_cq.id, cqe_incr, arm); +#else uint32_t head = rxq->gdma_cq.head % (rxq->gdma_cq.count << COMPLETION_QUEUE_ENTRY_OWNER_BITS_SIZE); @@ -537,6 +588,7 @@ mana_arm_cq(struct mana_rxq *rxq, uint8_t arm) return mana_ring_doorbell(priv->db_page, GDMA_QUEUE_COMPLETION, rxq->gdma_cq.id, head, arm); +#endif } int diff --git a/drivers/net/mana/tx.c b/drivers/net/mana/tx.c index 11ba2ee1ac..8ff81bde09 100644 --- a/drivers/net/mana/tx.c +++ b/drivers/net/mana/tx.c @@ -137,6 +137,9 @@ mana_start_tx_queues(struct rte_eth_dev *dev) /* CQ head starts with count (not 0) */ txq->gdma_cq.head = txq->gdma_cq.count; +#ifdef RTE_ARCH_32 + txq->gdma_cq.head_incr_to_short_db = 0; +#endif DRV_LOG(INFO, "txq cq id %u buf %p count %u size %u head %u", txq->gdma_cq.id, txq->gdma_cq.buffer, @@ -176,6 +179,9 @@ mana_tx_burst(void *dpdk_txq, struct rte_mbuf **tx_pkts, uint16_t nb_pkts) void *db_page; uint16_t pkt_sent = 0; uint32_t num_comp, i; +#ifdef RTE_ARCH_32 + uint32_t wqe_count = 0; +#endif /* Process send completions from GDMA */ num_comp = gdma_poll_completion_queue(&txq->gdma_cq, @@ -418,6 +424,20 @@ mana_tx_burst(void *dpdk_txq, struct rte_mbuf **tx_pkts, uint16_t nb_pkts) DP_LOG(DEBUG, "nb_pkts %u pkt[%d] sent", nb_pkts, pkt_idx); +#ifdef RTE_ARCH_32 + wqe_count += wqe_size_in_bu; + if (wqe_count > TX_WQE_SHORT_DB_THRESHOLD) { + /* wqe_count approaching to short doorbell + * increment limit. Stop processing further + * more packets and just ring short + * doorbell. + */ + DP_LOG(DEBUG, "wqe_count %u reaching limit, " + "pkt_sent %d", + wqe_count, pkt_sent); + break; + } +#endif } else { DP_LOG(DEBUG, "pkt[%d] failed to post send ret %d", pkt_idx, ret); @@ -436,11 +456,19 @@ mana_tx_burst(void *dpdk_txq, struct rte_mbuf **tx_pkts, uint16_t nb_pkts) } if (pkt_sent) { +#ifdef RTE_ARCH_32 + ret = mana_ring_short_doorbell(db_page, GDMA_QUEUE_SEND, + txq->gdma_sq.id, + wqe_count * + GDMA_WQE_ALIGNMENT_UNIT_SIZE, + 0); +#else ret = mana_ring_doorbell(db_page, GDMA_QUEUE_SEND, txq->gdma_sq.id, txq->gdma_sq.head * GDMA_WQE_ALIGNMENT_UNIT_SIZE, 0); +#endif if (ret) DP_LOG(ERR, "mana_ring_doorbell failed ret %d", ret); }