From patchwork Mon Mar 16 15:29:11 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Harman Kalra X-Patchwork-Id: 66719 X-Patchwork-Delegate: jerinj@marvell.com Return-Path: X-Original-To: patchwork@inbox.dpdk.org Delivered-To: patchwork@inbox.dpdk.org Received: from dpdk.org (dpdk.org [92.243.14.124]) by inbox.dpdk.org (Postfix) with ESMTP id 0B467A0559; Mon, 16 Mar 2020 16:30:26 +0100 (CET) Received: from [92.243.14.124] (localhost [127.0.0.1]) by dpdk.org (Postfix) with ESMTP id 234AB1C08C; Mon, 16 Mar 2020 16:30:04 +0100 (CET) Received: from mx0b-0016f401.pphosted.com (mx0b-0016f401.pphosted.com [67.231.156.173]) by dpdk.org (Postfix) with ESMTP id 1DCCB1C06C for ; Mon, 16 Mar 2020 16:30:01 +0100 (CET) Received: from pps.filterd (m0045851.ppops.net [127.0.0.1]) by mx0b-0016f401.pphosted.com (8.16.0.42/8.16.0.42) with SMTP id 02GFKT7A029483 for ; Mon, 16 Mar 2020 08:30:01 -0700 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=marvell.com; h=from : to : cc : subject : date : message-id : in-reply-to : references : content-type : mime-version; s=pfpt0818; bh=yGQesIQ0sK0yv4v4DBHhhcutYzM/qmVMGb+7N7KIvYQ=; b=sFGDE5kUfwNMjPCHYwl8AsdmVS9785hc5yJ1qU7kkSXpl9OHsLIbfz3rjnfd7FLU80ow arV++fF+SSQwHE5HH5xoH6cBXHDNNyQtvfau9jyMMMpUrdxRc9OYKZzWJ65Uvdkyu8jW PCvjVy86djMZCiotDaHbbOx+wcR+4YSPwzW7cPSZsaCTLA6Qpr+NlsmlZwaUhGtzUAC6 X2TAkQ0IyeuMy3GiDMWmAelOQXsHYPh4WUZerOk7HXi1AJb2zR8z4S/pFXFaM4ryzcLX SAyzsXepQdFyosfpoJJWP21egGKq13gejeXmuLml/SRrs9fuF5gemTEhmboSSgcXi9+E sQ== Received: from sc-exch02.marvell.com ([199.233.58.182]) by mx0b-0016f401.pphosted.com with ESMTP id 2yrxsmf9e6-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-SHA384 bits=256 verify=NOT) for ; Mon, 16 Mar 2020 08:30:01 -0700 Received: from SC-EXCH04.marvell.com (10.93.176.84) by SC-EXCH02.marvell.com (10.93.176.82) with Microsoft SMTP Server (TLS) id 15.0.1497.2; Mon, 16 Mar 2020 08:29:59 -0700 Received: from NAM04-BN3-obe.outbound.protection.outlook.com (104.47.46.59) by SC-EXCH04.marvell.com (10.93.176.84) with Microsoft SMTP Server (TLS) id 15.0.1497.2 via Frontend Transport; Mon, 16 Mar 2020 08:29:58 -0700 ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=gk5qDQBqrs4aPQrkTwci3Y/i6whgwbbXL2kLCegUzxVHpLq6ZDbIMk8cGHo8f1PxjOd7CLiwmjmoQwtOBWSPilb65ff+RReTm9UmZQ1wTic8otjslWOHw6nkdCp7WW+Esk3M/faTiA2PV2IUAiAqfDBBMjruLoFBHHOCO2S3kWhwUTXTHR6EP0J/uCE3lpmzLQiaffBVg5FQRjAhE/XkKWb8hWH8tqdU7b0pLTWiAIKNYvFShujXBBBB3oCKqQIWoHi452ismhxWkrZkzGkeaEjWZmSGxz5kzFbnoClatXWqq10BGLQ96QAuAUZa4pc0j3uhQDwp+yVJ0E4+LMgmtw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=yGQesIQ0sK0yv4v4DBHhhcutYzM/qmVMGb+7N7KIvYQ=; b=JzawPGwn6PL/O4HSq3HJGoXmGwfU8amszfp2MRLtnodK1xTHBxoZajN+o8K40OlyRbzIZ9s69viAgDnF0W61eBw2O6+do/ChBwiszfcBsGpmTuqr8Nv5jy3mI6e8nFoRpFkP+Z+jKEANJRGl3uIZfZKaPj2JkOUHZ1GMyrsDarXXfxDGoLbhIsmsN7PN5/jJ28xsL+KJzINvcAP4XrokPj7moLaTfctsDKc4hEkPG5PPX11WjIYR8KyBX9x/NYoixiP8A9SQqdkdyP5YMxaQY2NhqKylwqHT+aIU6r0IxHiE5/gK7j/et9kHDZaZKaS4GQOyEmO9Kp3kAekDZzhV9A== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=marvell.com; dmarc=pass action=none header.from=marvell.com; dkim=pass header.d=marvell.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=marvell.onmicrosoft.com; s=selector1-marvell-onmicrosoft-com; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=yGQesIQ0sK0yv4v4DBHhhcutYzM/qmVMGb+7N7KIvYQ=; b=MIzatwAEak/Y0o9YUSBLzgCiGv7EwqJjK0lfqMIeUOHQCvvv8fte08jHEvPTkChuJ4aHq9QZcv3EttBGwO7218o0w2Rw6ngl2YZH/5IYr9N1tKDKsu1df/22krRYDskHlhATWhB5rIiSxPNZ5WHoH1C1YYuKp5FAdnx5n9tnV9M= Received: from MN2PR18MB2848.namprd18.prod.outlook.com (2603:10b6:208:38::19) by MN2PR18MB3592.namprd18.prod.outlook.com (2603:10b6:208:26c::20) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.2814.19; Mon, 16 Mar 2020 15:29:57 +0000 Received: from MN2PR18MB2848.namprd18.prod.outlook.com ([fe80::f829:3e55:94f6:4efb]) by MN2PR18MB2848.namprd18.prod.outlook.com ([fe80::f829:3e55:94f6:4efb%5]) with mapi id 15.20.2814.021; Mon, 16 Mar 2020 15:29:57 +0000 From: Harman Kalra To: CC: , , Harman Kalra Date: Mon, 16 Mar 2020 20:59:11 +0530 Message-ID: <1584372553-28710-3-git-send-email-hkalra@marvell.com> X-Mailer: git-send-email 2.7.4 In-Reply-To: <1584372553-28710-1-git-send-email-hkalra@marvell.com> References: <1584372553-28710-1-git-send-email-hkalra@marvell.com> X-ClientProxiedBy: BM1PR0101CA0036.INDPRD01.PROD.OUTLOOK.COM (2603:1096:b00:1a::22) To MN2PR18MB2848.namprd18.prod.outlook.com (2603:10b6:208:38::19) MIME-Version: 1.0 X-MS-Exchange-MessageSentRepresentingType: 1 Received: from hkarlara-OptiPlex-3046.marvell.com (115.113.156.2) by BM1PR0101CA0036.INDPRD01.PROD.OUTLOOK.COM (2603:1096:b00:1a::22) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_CBC_SHA256) id 15.20.2814.13 via Frontend Transport; Mon, 16 Mar 2020 15:29:56 +0000 X-Mailer: git-send-email 2.7.4 X-Originating-IP: [115.113.156.2] X-MS-PublicTrafficType: Email X-MS-Office365-Filtering-Correlation-Id: 9ea64c91-b911-4a55-80d6-08d7c9bee24e X-MS-TrafficTypeDiagnostic: MN2PR18MB3592: X-MS-Exchange-Transport-Forked: True X-Microsoft-Antispam-PRVS: X-MS-Oob-TLC-OOBClassifiers: OLM:4502; X-Forefront-PRVS: 03449D5DD1 X-Forefront-Antispam-Report: SFV:NSPM; SFS:(10009020)(4636009)(366004)(39850400004)(136003)(376002)(346002)(396003)(199004)(66476007)(30864003)(956004)(478600001)(66556008)(186003)(86362001)(26005)(16526019)(36756003)(66946007)(6486002)(4326008)(6636002)(316002)(55236004)(7696005)(37006003)(52116002)(107886003)(2616005)(8676002)(8936002)(2906002)(81166006)(5660300002)(81156014)(34206002)(6666004); DIR:OUT; SFP:1101; SCL:1; SRVR:MN2PR18MB3592; H:MN2PR18MB2848.namprd18.prod.outlook.com; FPR:; SPF:None; LANG:en; PTR:InfoNoRecords; A:1; Received-SPF: None (protection.outlook.com: marvell.com does not designate permitted sender hosts) X-MS-Exchange-SenderADCheck: 1 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: kLYqCOLc6vGYaQaqZc+qAynPoXxz4TNO8KfIAFGkkisEF+sQYC/ygMPfw2F36NszjPE1GpkUE675LS6URmR+m6EMqY8F0HfbC8LVnUKirnJYhoRE0zDGlpvsnFtqYbnqfnApohMKj6Q2R76SX+DwaEhMD+pr4xWDLVYJD0rHgUC8qKfyUD9sfn/TPFARPBhB4GR+upbMO4rMIxhA7C77LS98IN96svP6xcbG281ESCcuxmTI2F81qVGwmVYjh9kIDAFL4zZih/54MhmEJwQd+Y3lHNbfnDlet5AAgyshFglKmE++lwD7ON3YhA3rB8k7jpLhO4vQTxb2wYXSkMOb8cMpEFC/sfQo0iPhfA90LPHL4G8BhoQ44IP+ZNJFimrIB3fjisycKwsWwoUFDcKJKcaYFcHHmDtYmFqJNz6AJqXEHgmTwmKukAsaRcxdJdat X-MS-Exchange-AntiSpam-MessageData: LJ2x8MfvMC+ytWEtV69uWQDcoud4QUqb9kthj+hAK2bRzEAzgkCCRDcCLhMOhtJKqSA4PbrMnbpRblsXHWXe6AMLcmb0ymASI10GdzmFfaqY6GIXycBAFg62L65aehd5sJJdBzF5jJeUs/pyWG87XQ== X-MS-Exchange-CrossTenant-Network-Message-Id: 9ea64c91-b911-4a55-80d6-08d7c9bee24e X-MS-Exchange-CrossTenant-OriginalArrivalTime: 16 Mar 2020 15:29:57.5318 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: 70e1fb47-1155-421d-87fc-2e58f638b6e0 X-MS-Exchange-CrossTenant-MailboxType: HOSTED X-MS-Exchange-CrossTenant-UserPrincipalName: m2QzFqOP6IpNV3f6P8RRxaeyImucICO0cUNxKH9zrTH7VhUI9V//bBiNMZ5CpvISUd2TMsVqDVKGp7zTp01gGg== X-MS-Exchange-Transport-CrossTenantHeadersStamped: MN2PR18MB3592 X-OriginatorOrg: marvell.com X-Proofpoint-Virus-Version: vendor=fsecure engine=2.50.10434:6.0.138, 18.0.645 definitions=2020-03-16_06:2020-03-12, 2020-03-16 signatures=0 Subject: [dpdk-dev] [PATCH 2/4] event/octeontx: add framework for Rx/Tx offloads X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Sender: "dev" Adding macro based framework to hook dequeue/enqueue function pointers to the appropriate function based on rx/tx offloads. Signed-off-by: Harman Kalra --- drivers/event/octeontx/ssovf_evdev.c | 36 ---- drivers/event/octeontx/ssovf_evdev.h | 24 +-- drivers/event/octeontx/ssovf_worker.c | 259 ++++++++++++++------------ drivers/net/octeontx/octeontx_rxtx.h | 7 + 4 files changed, 150 insertions(+), 176 deletions(-) diff --git a/drivers/event/octeontx/ssovf_evdev.c b/drivers/event/octeontx/ssovf_evdev.c index 1024b7284..5d074bcbc 100644 --- a/drivers/event/octeontx/ssovf_evdev.c +++ b/drivers/event/octeontx/ssovf_evdev.c @@ -137,42 +137,6 @@ ssovf_mbox_timeout_ticks(uint64_t ns, uint64_t *tmo_ticks) return 0; } -static void -ssovf_fastpath_fns_set(struct rte_eventdev *dev) -{ - struct ssovf_evdev *edev = ssovf_pmd_priv(dev); - - dev->enqueue = ssows_enq; - dev->enqueue_burst = ssows_enq_burst; - dev->enqueue_new_burst = ssows_enq_new_burst; - dev->enqueue_forward_burst = ssows_enq_fwd_burst; - - if (!!(edev->rx_offload_flags & OCCTX_RX_MULTI_SEG_F)) { - dev->dequeue = ssows_deq_mseg; - dev->dequeue_burst = ssows_deq_burst_mseg; - - if (edev->is_timeout_deq) { - dev->dequeue = ssows_deq_timeout_mseg; - dev->dequeue_burst = ssows_deq_timeout_burst_mseg; - } - } else { - dev->dequeue = ssows_deq; - dev->dequeue_burst = ssows_deq_burst; - - if (edev->is_timeout_deq) { - dev->dequeue = ssows_deq_timeout; - dev->dequeue_burst = ssows_deq_timeout_burst; - } - } - - if (!!(edev->tx_offload_flags & OCCTX_TX_MULTI_SEG_F)) - dev->txa_enqueue = sso_event_tx_adapter_enqueue_mseg; - else - dev->txa_enqueue = sso_event_tx_adapter_enqueue; - - dev->txa_enqueue_same_dest = dev->txa_enqueue; -} - static void ssovf_info_get(struct rte_eventdev *dev, struct rte_event_dev_info *dev_info) { diff --git a/drivers/event/octeontx/ssovf_evdev.h b/drivers/event/octeontx/ssovf_evdev.h index 1c3ae8556..1f5066c9a 100644 --- a/drivers/event/octeontx/ssovf_evdev.h +++ b/drivers/event/octeontx/ssovf_evdev.h @@ -14,6 +14,9 @@ #include "octeontx_rxtx.h" +#define SSO_RX_ADPTR_ENQ_FASTPATH_FUNC OCCTX_RX_FASTPATH_MODES +#define SSO_TX_ADPTR_ENQ_FASTPATH_FUNC OCCTX_TX_FASTPATH_MODES + #define EVENTDEV_NAME_OCTEONTX_PMD event_octeontx #define SSOVF_LOG(level, fmt, args...) \ @@ -171,32 +174,13 @@ uint16_t ssows_enq_new_burst(void *port, const struct rte_event ev[], uint16_t nb_events); uint16_t ssows_enq_fwd_burst(void *port, const struct rte_event ev[], uint16_t nb_events); -uint16_t ssows_deq(void *port, struct rte_event *ev, uint64_t timeout_ticks); -uint16_t ssows_deq_burst(void *port, struct rte_event ev[], - uint16_t nb_events, uint64_t timeout_ticks); -uint16_t ssows_deq_timeout(void *port, struct rte_event *ev, - uint64_t timeout_ticks); -uint16_t ssows_deq_timeout_burst(void *port, struct rte_event ev[], - uint16_t nb_events, uint64_t timeout_ticks); -uint16_t ssows_deq_mseg(void *port, struct rte_event *ev, - uint64_t timeout_ticks); -uint16_t ssows_deq_burst_mseg(void *port, struct rte_event ev[], - uint16_t nb_events, uint64_t timeout_ticks); -uint16_t ssows_deq_timeout_mseg(void *port, struct rte_event *ev, - uint64_t timeout_ticks); -uint16_t ssows_deq_timeout_burst_mseg(void *port, struct rte_event ev[], - uint16_t nb_events, uint64_t timeout_ticks); - typedef void (*ssows_handle_event_t)(void *arg, struct rte_event ev); void ssows_flush_events(struct ssows *ws, uint8_t queue_id, ssows_handle_event_t fn, void *arg); void ssows_reset(struct ssows *ws); -uint16_t sso_event_tx_adapter_enqueue(void *port, - struct rte_event ev[], uint16_t nb_events); -uint16_t sso_event_tx_adapter_enqueue_mseg(void *port, - struct rte_event ev[], uint16_t nb_events); int ssovf_info(struct ssovf_info *info); void *ssovf_bar(enum ssovf_type, uint8_t id, uint8_t bar); int test_eventdev_octeontx(void); +void ssovf_fastpath_fns_set(struct rte_eventdev *dev); #endif /* __SSOVF_EVDEV_H__ */ diff --git a/drivers/event/octeontx/ssovf_worker.c b/drivers/event/octeontx/ssovf_worker.c index 951f66890..5d8e213ce 100644 --- a/drivers/event/octeontx/ssovf_worker.c +++ b/drivers/event/octeontx/ssovf_worker.c @@ -91,112 +91,62 @@ ssows_release_event(struct ssows *ws) ssows_swtag_untag(ws); } -__rte_always_inline uint16_t __hot -ssows_deq(void *port, struct rte_event *ev, uint64_t timeout_ticks) -{ - struct ssows *ws = port; - - RTE_SET_USED(timeout_ticks); - - if (ws->swtag_req) { - ws->swtag_req = 0; - ssows_swtag_wait(ws); - return 1; - } else { - return ssows_get_work(ws, ev, OCCTX_RX_OFFLOAD_NONE); - } -} - -__rte_always_inline uint16_t __hot -ssows_deq_timeout(void *port, struct rte_event *ev, uint64_t timeout_ticks) -{ - struct ssows *ws = port; - uint64_t iter; - uint16_t ret = 1; - - if (ws->swtag_req) { - ws->swtag_req = 0; - ssows_swtag_wait(ws); - } else { - ret = ssows_get_work(ws, ev, OCCTX_RX_OFFLOAD_NONE); - for (iter = 1; iter < timeout_ticks && (ret == 0); iter++) - ret = ssows_get_work(ws, ev, OCCTX_RX_OFFLOAD_NONE); - } - return ret; +#define R(name, f0, flags) \ +static uint16_t __rte_noinline __hot \ +ssows_deq_ ##name(void *port, struct rte_event *ev, uint64_t timeout_ticks) \ +{ \ + struct ssows *ws = port; \ + \ + RTE_SET_USED(timeout_ticks); \ + \ + if (ws->swtag_req) { \ + ws->swtag_req = 0; \ + ssows_swtag_wait(ws); \ + return 1; \ + } else { \ + return ssows_get_work(ws, ev, flags); \ + } \ +} \ + \ +static uint16_t __hot \ +ssows_deq_burst_ ##name(void *port, struct rte_event ev[], \ + uint16_t nb_events, uint64_t timeout_ticks) \ +{ \ + RTE_SET_USED(nb_events); \ + \ + return ssows_deq_ ##name(port, ev, timeout_ticks); \ +} \ + \ +static uint16_t __hot \ +ssows_deq_timeout_ ##name(void *port, struct rte_event *ev, \ + uint64_t timeout_ticks) \ +{ \ + struct ssows *ws = port; \ + uint64_t iter; \ + uint16_t ret = 1; \ + \ + if (ws->swtag_req) { \ + ws->swtag_req = 0; \ + ssows_swtag_wait(ws); \ + } else { \ + ret = ssows_get_work(ws, ev, flags); \ + for (iter = 1; iter < timeout_ticks && (ret == 0); iter++) \ + ret = ssows_get_work(ws, ev, flags); \ + } \ + return ret; \ +} \ + \ +static uint16_t __hot \ +ssows_deq_timeout_burst_ ##name(void *port, struct rte_event ev[], \ + uint16_t nb_events, uint64_t timeout_ticks) \ +{ \ + RTE_SET_USED(nb_events); \ + \ + return ssows_deq_timeout_ ##name(port, ev, timeout_ticks); \ } -uint16_t __hot -ssows_deq_burst(void *port, struct rte_event ev[], uint16_t nb_events, - uint64_t timeout_ticks) -{ - RTE_SET_USED(nb_events); - - return ssows_deq(port, ev, timeout_ticks); -} - -uint16_t __hot -ssows_deq_timeout_burst(void *port, struct rte_event ev[], uint16_t nb_events, - uint64_t timeout_ticks) -{ - RTE_SET_USED(nb_events); - - return ssows_deq_timeout(port, ev, timeout_ticks); -} - -__rte_always_inline uint16_t __hot -ssows_deq_mseg(void *port, struct rte_event *ev, uint64_t timeout_ticks) -{ - struct ssows *ws = port; - - RTE_SET_USED(timeout_ticks); - - if (ws->swtag_req) { - ws->swtag_req = 0; - ssows_swtag_wait(ws); - return 1; - } else { - return ssows_get_work(ws, ev, OCCTX_RX_OFFLOAD_NONE | - OCCTX_RX_MULTI_SEG_F); - } -} - -__rte_always_inline uint16_t __hot -ssows_deq_timeout_mseg(void *port, struct rte_event *ev, uint64_t timeout_ticks) -{ - struct ssows *ws = port; - uint64_t iter; - uint16_t ret = 1; - - if (ws->swtag_req) { - ws->swtag_req = 0; - ssows_swtag_wait(ws); - } else { - ret = ssows_get_work(ws, ev, OCCTX_RX_OFFLOAD_NONE | - OCCTX_RX_MULTI_SEG_F); - for (iter = 1; iter < timeout_ticks && (ret == 0); iter++) - ret = ssows_get_work(ws, ev, OCCTX_RX_OFFLOAD_NONE | - OCCTX_RX_MULTI_SEG_F); - } - return ret; -} - -uint16_t __hot -ssows_deq_burst_mseg(void *port, struct rte_event ev[], uint16_t nb_events, - uint64_t timeout_ticks) -{ - RTE_SET_USED(nb_events); - - return ssows_deq_mseg(port, ev, timeout_ticks); -} - -uint16_t __hot -ssows_deq_timeout_burst_mseg(void *port, struct rte_event ev[], - uint16_t nb_events, uint64_t timeout_ticks) -{ - RTE_SET_USED(nb_events); - - return ssows_deq_timeout_mseg(port, ev, timeout_ticks); -} +SSO_RX_ADPTR_ENQ_FASTPATH_FUNC +#undef R __rte_always_inline uint16_t __hot ssows_enq(void *port, const struct rte_event *ev) @@ -321,7 +271,8 @@ ssows_reset(struct ssows *ws) static __rte_always_inline uint16_t __sso_event_tx_adapter_enqueue(void *port, struct rte_event ev[], - uint16_t nb_events, const uint16_t flag) + uint16_t nb_events, uint64_t *cmd, + const uint16_t flag) { uint16_t port_id; uint16_t queue_id; @@ -329,9 +280,7 @@ __sso_event_tx_adapter_enqueue(void *port, struct rte_event ev[], struct rte_eth_dev *ethdev; struct ssows *ws = port; struct octeontx_txq *txq; - uint64_t cmd[4]; - RTE_SET_USED(nb_events); switch (ev->sched_type) { case SSO_SYNC_ORDERED: ssows_swtag_norm(ws, ev->event, SSO_SYNC_ATOMIC); @@ -355,22 +304,92 @@ __sso_event_tx_adapter_enqueue(void *port, struct rte_event ev[], ethdev = &rte_eth_devices[port_id]; txq = ethdev->data->tx_queues[queue_id]; - return __octeontx_xmit_pkts(txq, &m, 1, cmd, flag); + return __octeontx_xmit_pkts(txq, &m, nb_events, cmd, flag); } -uint16_t -sso_event_tx_adapter_enqueue(void *port, struct rte_event ev[], - uint16_t nb_events) -{ - return __sso_event_tx_adapter_enqueue(port, ev, nb_events, - OCCTX_TX_OFFLOAD_NONE); +#define T(name, f3, f2, f1, f0, sz, flags) \ +static uint16_t __rte_noinline __hot \ +sso_event_tx_adapter_enqueue_ ## name(void *port, struct rte_event ev[], \ + uint16_t nb_events) \ +{ \ + uint64_t cmd[sz]; \ + return __sso_event_tx_adapter_enqueue(port, ev, nb_events, cmd, \ + flags); \ } -uint16_t -sso_event_tx_adapter_enqueue_mseg(void *port, struct rte_event ev[], - uint16_t nb_events) +SSO_TX_ADPTR_ENQ_FASTPATH_FUNC +#undef T + +void +ssovf_fastpath_fns_set(struct rte_eventdev *dev) { - return __sso_event_tx_adapter_enqueue(port, ev, nb_events, - OCCTX_TX_OFFLOAD_NONE | - OCCTX_TX_MULTI_SEG_F); + struct ssovf_evdev *edev = ssovf_pmd_priv(dev); + + dev->enqueue = ssows_enq; + dev->enqueue_burst = ssows_enq_burst; + dev->enqueue_new_burst = ssows_enq_new_burst; + dev->enqueue_forward_burst = ssows_enq_fwd_burst; + + const event_tx_adapter_enqueue ssow_txa_enqueue[2][2][2][2] = { +#define T(name, f3, f2, f1, f0, sz, flags) \ + [f3][f2][f1][f0] = sso_event_tx_adapter_enqueue_ ##name, + +SSO_TX_ADPTR_ENQ_FASTPATH_FUNC +#undef T + }; + + dev->txa_enqueue = ssow_txa_enqueue + [!!(edev->tx_offload_flags & OCCTX_TX_OFFLOAD_MBUF_NOFF_F)] + [0] + [0] + [!!(edev->tx_offload_flags & OCCTX_TX_MULTI_SEG_F)]; + + dev->txa_enqueue_same_dest = dev->txa_enqueue; + + /* Assigning dequeue func pointers */ + const event_dequeue_t ssow_deq[2] = { +#define R(name, f0, flags) \ + [f0] = ssows_deq_ ##name, + +SSO_RX_ADPTR_ENQ_FASTPATH_FUNC +#undef R + }; + + dev->dequeue = ssow_deq + [!!(edev->rx_offload_flags & OCCTX_RX_MULTI_SEG_F)]; + + const event_dequeue_burst_t ssow_deq_burst[2] = { +#define R(name, f0, flags) \ + [f0] = ssows_deq_burst_ ##name, + +SSO_RX_ADPTR_ENQ_FASTPATH_FUNC +#undef R + }; + + dev->dequeue_burst = ssow_deq_burst + [!!(edev->rx_offload_flags & OCCTX_RX_MULTI_SEG_F)]; + + if (edev->is_timeout_deq) { + const event_dequeue_t ssow_deq_timeout[2] = { +#define R(name, f0, flags) \ + [f0] = ssows_deq_timeout_ ##name, + +SSO_RX_ADPTR_ENQ_FASTPATH_FUNC +#undef R + }; + + dev->dequeue = ssow_deq_timeout + [!!(edev->rx_offload_flags & OCCTX_RX_MULTI_SEG_F)]; + + const event_dequeue_burst_t ssow_deq_timeout_burst[2] = { +#define R(name, f0, flags) \ + [f0] = ssows_deq_timeout_burst_ ##name, + +SSO_RX_ADPTR_ENQ_FASTPATH_FUNC +#undef R + }; + + dev->dequeue_burst = ssow_deq_timeout_burst + [!!(edev->rx_offload_flags & OCCTX_RX_MULTI_SEG_F)]; + } } diff --git a/drivers/net/octeontx/octeontx_rxtx.h b/drivers/net/octeontx/octeontx_rxtx.h index acc1f5cb8..e7656f333 100644 --- a/drivers/net/octeontx/octeontx_rxtx.h +++ b/drivers/net/octeontx/octeontx_rxtx.h @@ -488,4 +488,11 @@ T(noff_ol3ol4csum_l3l4csum_mseg, 1, 1, 1, 1, 14, \ NOFF_F | OL3OL4CSUM_F | L3L4CSUM_F | \ MULT_F) +/* RX offload macros */ +#define MULT_RX_F OCCTX_RX_MULTI_SEG_F +/* [MULTI_SEG] */ +#define OCCTX_RX_FASTPATH_MODES \ +R(no_offload, 0, OCCTX_RX_OFFLOAD_NONE) \ +R(mseg, 1, MULT_RX_F) \ + #endif /* __OCTEONTX_RXTX_H__ */