From patchwork Tue May 4 00:27:01 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Pavan Nikhilesh Bhagavatula X-Patchwork-Id: 92689 X-Patchwork-Delegate: jerinj@marvell.com Return-Path: X-Original-To: patchwork@inbox.dpdk.org Delivered-To: patchwork@inbox.dpdk.org Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id 59030A0562; Tue, 4 May 2021 02:29:11 +0200 (CEST) Received: from [217.70.189.124] (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id 2028641160; Tue, 4 May 2021 02:28:35 +0200 (CEST) Received: from mx0b-0016f401.pphosted.com (mx0b-0016f401.pphosted.com [67.231.156.173]) by mails.dpdk.org (Postfix) with ESMTP id 8000141158 for ; Tue, 4 May 2021 02:28:33 +0200 (CEST) Received: from pps.filterd (m0045851.ppops.net [127.0.0.1]) by mx0b-0016f401.pphosted.com (8.16.0.43/8.16.0.43) with SMTP id 1440AGT2002336 for ; Mon, 3 May 2021 17:28:32 -0700 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=marvell.com; h=from : to : cc : subject : date : message-id : in-reply-to : references : mime-version : content-transfer-encoding : content-type; s=pfpt0220; bh=UhWlThFO/2a0xytd+I/rT+giTY9q/PCIj2t/n8iIcnE=; b=j5hma37/bYrU42Mywm2JOS+gFJGtenY0u47JMaIJWSCGPWtcBVZ6offCJuB52zXyiyXJ c56hHk+QBhod7QDZ5AOUuxeKB9wY57FXvwN5tvTVJwn/FlmhdcEVPYuHF1kvgZfPxZzb dOrLsEMKjFOcJyFE445EOyQ3/Bn+ZSFALzMBq7MAcmzFYwVuOJyNkAzTpn0LkyZqlfc6 C9ZhWBDY4+n4v+km3dqkToD0bRa2mp0rNV09qLfjK9dNSDcuNE7moTfyGoVLetayaIdQ yQMqy4jTjRX3cE/6teKP7dQHW1RLb2ktFVS9X5dpexQANwxGPKlHQXpRuImfqJVWMahP xQ== Received: from dc5-exch01.marvell.com ([199.233.59.181]) by mx0b-0016f401.pphosted.com with ESMTP id 38ad05k6kd-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-SHA384 bits=256 verify=NOT) for ; Mon, 03 May 2021 17:28:32 -0700 Received: from DC5-EXCH01.marvell.com (10.69.176.38) by DC5-EXCH01.marvell.com (10.69.176.38) with Microsoft SMTP Server (TLS) id 15.0.1497.2; Mon, 3 May 2021 17:28:30 -0700 Received: from maili.marvell.com (10.69.176.80) by DC5-EXCH01.marvell.com (10.69.176.38) with Microsoft SMTP Server id 15.0.1497.2 via Frontend Transport; Mon, 3 May 2021 17:28:29 -0700 Received: from BG-LT7430.marvell.com (unknown [10.193.86.144]) by maili.marvell.com (Postfix) with ESMTP id F30C63F703F; Mon, 3 May 2021 17:28:28 -0700 (PDT) From: To: , Pavan Nikhilesh , "Shijith Thotton" CC: Date: Tue, 4 May 2021 05:57:01 +0530 Message-ID: <20210504002726.525-12-pbhagavatula@marvell.com> X-Mailer: git-send-email 2.17.1 In-Reply-To: <20210504002726.525-1-pbhagavatula@marvell.com> References: <20210503152238.2437-1-pbhagavatula@marvell.com> <20210504002726.525-1-pbhagavatula@marvell.com> MIME-Version: 1.0 X-Proofpoint-GUID: eERXKLqqWKIAQzwjDw8byeXfHWdzP5ua X-Proofpoint-ORIG-GUID: eERXKLqqWKIAQzwjDw8byeXfHWdzP5ua X-Proofpoint-Virus-Version: vendor=fsecure engine=2.50.10434:6.0.391, 18.0.761 definitions=2021-05-03_20:2021-05-03, 2021-05-03 signatures=0 Subject: [dpdk-dev] [PATCH v5 11/35] event/cnxk: add devargs to control SSO HWGRP QoS X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Sender: "dev" From: Shijith Thotton SSO HWGRPs i.e. queue uses DRAM & SRAM buffers to hold in-flight events. By default the buffers are assigned to the SSO HWGRPs to satisfy minimum HW requirements. SSO is free to assign the remaining buffers to HWGRPs based on a preconfigured threshold. We can control the QoS of SSO HWGRP by modifying the above mentioned thresholds. HWGRPs that have higher importance can be assigned higher thresholds than the rest. Example: --dev "0002:0e:00.0,qos=[1-50-50-50]" // [Qx-XAQ-TAQ-IAQ] Qx -> Event queue Aka SSO GGRP. XAQ -> DRAM In-flights. TAQ & IAQ -> SRAM In-flights. The values need to be expressed in terms of percentages, 0 represents default. Signed-off-by: Pavan Nikhilesh Signed-off-by: Shijith Thotton --- doc/guides/eventdevs/cnxk.rst | 16 ++++++ drivers/event/cnxk/cn10k_eventdev.c | 3 +- drivers/event/cnxk/cn9k_eventdev.c | 3 +- drivers/event/cnxk/cnxk_eventdev.c | 78 +++++++++++++++++++++++++++++ drivers/event/cnxk/cnxk_eventdev.h | 12 ++++- 5 files changed, 109 insertions(+), 3 deletions(-) diff --git a/doc/guides/eventdevs/cnxk.rst b/doc/guides/eventdevs/cnxk.rst index b556681ff..0583e5fdd 100644 --- a/doc/guides/eventdevs/cnxk.rst +++ b/doc/guides/eventdevs/cnxk.rst @@ -55,6 +55,22 @@ Runtime Config Options -a 0002:0e:00.0,xae_cnt=16384 +- ``Event Group QoS support`` + + SSO GGRPs i.e. queue uses DRAM & SRAM buffers to hold in-flight + events. By default the buffers are assigned to the SSO GGRPs to + satisfy minimum HW requirements. SSO is free to assign the remaining + buffers to GGRPs based on a preconfigured threshold. + We can control the QoS of SSO GGRP by modifying the above mentioned + thresholds. GGRPs that have higher importance can be assigned higher + thresholds than the rest. The dictionary format is as follows + [Qx-XAQ-TAQ-IAQ][Qz-XAQ-TAQ-IAQ] expressed in percentages, 0 represents + default. + + For example:: + + -a 0002:0e:00.0,qos=[1-50-50-50] + Debugging Options ----------------- diff --git a/drivers/event/cnxk/cn10k_eventdev.c b/drivers/event/cnxk/cn10k_eventdev.c index 020905290..0b39c6c09 100644 --- a/drivers/event/cnxk/cn10k_eventdev.c +++ b/drivers/event/cnxk/cn10k_eventdev.c @@ -143,4 +143,5 @@ static struct rte_pci_driver cn10k_pci_sso = { RTE_PMD_REGISTER_PCI(event_cn10k, cn10k_pci_sso); RTE_PMD_REGISTER_PCI_TABLE(event_cn10k, cn10k_pci_sso_map); RTE_PMD_REGISTER_KMOD_DEP(event_cn10k, "vfio-pci"); -RTE_PMD_REGISTER_PARAM_STRING(event_cn10k, CNXK_SSO_XAE_CNT "="); +RTE_PMD_REGISTER_PARAM_STRING(event_cn10k, CNXK_SSO_XAE_CNT "=" + CNXK_SSO_GGRP_QOS "="); diff --git a/drivers/event/cnxk/cn9k_eventdev.c b/drivers/event/cnxk/cn9k_eventdev.c index 50f6fef01..ab165c850 100644 --- a/drivers/event/cnxk/cn9k_eventdev.c +++ b/drivers/event/cnxk/cn9k_eventdev.c @@ -146,4 +146,5 @@ static struct rte_pci_driver cn9k_pci_sso = { RTE_PMD_REGISTER_PCI(event_cn9k, cn9k_pci_sso); RTE_PMD_REGISTER_PCI_TABLE(event_cn9k, cn9k_pci_sso_map); RTE_PMD_REGISTER_KMOD_DEP(event_cn9k, "vfio-pci"); -RTE_PMD_REGISTER_PARAM_STRING(event_cn9k, CNXK_SSO_XAE_CNT "="); +RTE_PMD_REGISTER_PARAM_STRING(event_cn9k, CNXK_SSO_XAE_CNT "=" + CNXK_SSO_GGRP_QOS "="); diff --git a/drivers/event/cnxk/cnxk_eventdev.c b/drivers/event/cnxk/cnxk_eventdev.c index fddd71a8d..e93aaccd8 100644 --- a/drivers/event/cnxk/cnxk_eventdev.c +++ b/drivers/event/cnxk/cnxk_eventdev.c @@ -225,6 +225,82 @@ cnxk_sso_port_def_conf(struct rte_eventdev *event_dev, uint8_t port_id, port_conf->enqueue_depth = 1; } +static void +parse_queue_param(char *value, void *opaque) +{ + struct cnxk_sso_qos queue_qos = {0}; + uint8_t *val = (uint8_t *)&queue_qos; + struct cnxk_sso_evdev *dev = opaque; + char *tok = strtok(value, "-"); + struct cnxk_sso_qos *old_ptr; + + if (!strlen(value)) + return; + + while (tok != NULL) { + *val = atoi(tok); + tok = strtok(NULL, "-"); + val++; + } + + if (val != (&queue_qos.iaq_prcnt + 1)) { + plt_err("Invalid QoS parameter expected [Qx-XAQ-TAQ-IAQ]"); + return; + } + + dev->qos_queue_cnt++; + old_ptr = dev->qos_parse_data; + dev->qos_parse_data = rte_realloc( + dev->qos_parse_data, + sizeof(struct cnxk_sso_qos) * dev->qos_queue_cnt, 0); + if (dev->qos_parse_data == NULL) { + dev->qos_parse_data = old_ptr; + dev->qos_queue_cnt--; + return; + } + dev->qos_parse_data[dev->qos_queue_cnt - 1] = queue_qos; +} + +static void +parse_qos_list(const char *value, void *opaque) +{ + char *s = strdup(value); + char *start = NULL; + char *end = NULL; + char *f = s; + + while (*s) { + if (*s == '[') + start = s; + else if (*s == ']') + end = s; + + if (start && start < end) { + *end = 0; + parse_queue_param(start + 1, opaque); + s = end; + start = end; + } + s++; + } + + free(f); +} + +static int +parse_sso_kvargs_dict(const char *key, const char *value, void *opaque) +{ + RTE_SET_USED(key); + + /* Dict format [Qx-XAQ-TAQ-IAQ][Qz-XAQ-TAQ-IAQ] use '-' cause ',' + * isn't allowed. Everything is expressed in percentages, 0 represents + * default. + */ + parse_qos_list(value, opaque); + + return 0; +} + static void cnxk_sso_parse_devargs(struct cnxk_sso_evdev *dev, struct rte_devargs *devargs) { @@ -238,6 +314,8 @@ cnxk_sso_parse_devargs(struct cnxk_sso_evdev *dev, struct rte_devargs *devargs) rte_kvargs_process(kvlist, CNXK_SSO_XAE_CNT, &parse_kvargs_value, &dev->xae_cnt); + rte_kvargs_process(kvlist, CNXK_SSO_GGRP_QOS, &parse_sso_kvargs_dict, + dev); rte_kvargs_free(kvlist); } diff --git a/drivers/event/cnxk/cnxk_eventdev.h b/drivers/event/cnxk/cnxk_eventdev.h index 202c6e6a7..b96a6a908 100644 --- a/drivers/event/cnxk/cnxk_eventdev.h +++ b/drivers/event/cnxk/cnxk_eventdev.h @@ -14,7 +14,8 @@ #include "roc_api.h" -#define CNXK_SSO_XAE_CNT "xae_cnt" +#define CNXK_SSO_XAE_CNT "xae_cnt" +#define CNXK_SSO_GGRP_QOS "qos" #define USEC2NSEC(__us) ((__us)*1E3) @@ -23,6 +24,13 @@ #define CNXK_SSO_XAQ_CACHE_CNT (0x7) #define CNXK_SSO_XAQ_SLACK (8) +struct cnxk_sso_qos { + uint16_t queue; + uint8_t xaq_prcnt; + uint8_t taq_prcnt; + uint8_t iaq_prcnt; +}; + struct cnxk_sso_evdev { struct roc_sso sso; uint8_t max_event_queues; @@ -41,6 +49,8 @@ struct cnxk_sso_evdev { struct rte_mempool *xaq_pool; /* Dev args */ uint32_t xae_cnt; + uint8_t qos_queue_cnt; + struct cnxk_sso_qos *qos_parse_data; /* CN9K */ uint8_t dual_ws; } __rte_cache_aligned;