From patchwork Wed Jul 1 06:51:52 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Ajit Khaparde X-Patchwork-Id: 72555 X-Patchwork-Delegate: ajit.khaparde@broadcom.com Return-Path: X-Original-To: patchwork@inbox.dpdk.org Delivered-To: patchwork@inbox.dpdk.org Received: from dpdk.org (dpdk.org [92.243.14.124]) by inbox.dpdk.org (Postfix) with ESMTP id D04FCA0350; Wed, 1 Jul 2020 09:00:08 +0200 (CEST) Received: from [92.243.14.124] (localhost [127.0.0.1]) by dpdk.org (Postfix) with ESMTP id 81A7C1D449; Wed, 1 Jul 2020 08:53:04 +0200 (CEST) Received: from rnd-relay.smtp.broadcom.com (rnd-relay.smtp.broadcom.com [192.19.229.170]) by dpdk.org (Postfix) with ESMTP id C55701C195 for ; Wed, 1 Jul 2020 08:52:27 +0200 (CEST) Received: from mail-irv-17.broadcom.com (mail-irv-17.lvn.broadcom.net [10.75.242.48]) by rnd-relay.smtp.broadcom.com (Postfix) with ESMTP id 1C47230C33D; Tue, 30 Jun 2020 23:52:27 -0700 (PDT) DKIM-Filter: OpenDKIM Filter v2.10.3 rnd-relay.smtp.broadcom.com 1C47230C33D DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=broadcom.com; s=dkimrelay; t=1593586347; bh=Xzzf079/79043alzpKjkD4gfYwhLBhVnwnjyRLfCODE=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=obza1cKGSEKwLMuNoE9z4hbCUnsfGIwYV8sTwX21G/Gck6xUpieZxSu0iC+FPXHGM IeGWkaIwo6UrWOUT+aok2hNvrv5NEfpi/vWBuui6aWpx6EtACur03Hnx5VpI5Latb/ uHIkkIU3bmcdWXJACSS+1Z/XPn510PpEruaWJIW0= Received: from localhost.localdomain (unknown [10.230.185.215]) by mail-irv-17.broadcom.com (Postfix) with ESMTP id 9FE7414008E; Tue, 30 Jun 2020 23:52:26 -0700 (PDT) From: Ajit Khaparde To: dev@dpdk.org Cc: Peter Spreadborough , Randy Schacher , Venkat Duvvuru Date: Tue, 30 Jun 2020 23:51:52 -0700 Message-Id: <20200701065212.41391-32-ajit.khaparde@broadcom.com> X-Mailer: git-send-email 2.21.1 (Apple Git-122.3) In-Reply-To: <20200701065212.41391-1-ajit.khaparde@broadcom.com> References: <20200612132934.16488-1-somnath.kotur@broadcom.com> <20200701065212.41391-1-ajit.khaparde@broadcom.com> MIME-Version: 1.0 Subject: [dpdk-dev] [PATCH v2 31/51] net/bnxt: add support for EEM System memory X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Sender: "dev" From: Peter Spreadborough - Select EEM Host or System memory via config parameter - Add EEM system memory support for kernel memory - Dependent on DPDK changes that add support for the HWRM_OEM_CMD. Signed-off-by: Peter Spreadborough Reviewed-by: Randy Schacher Signed-off-by: Venkat Duvvuru --- config/common_base | 1 + drivers/net/bnxt/Makefile | 3 + drivers/net/bnxt/bnxt.h | 8 + drivers/net/bnxt/bnxt_hwrm.c | 27 + drivers/net/bnxt/bnxt_hwrm.h | 1 + drivers/net/bnxt/meson.build | 2 +- drivers/net/bnxt/tf_core/Makefile | 5 +- drivers/net/bnxt/tf_core/tf_core.c | 13 +- drivers/net/bnxt/tf_core/tf_core.h | 4 +- drivers/net/bnxt/tf_core/tf_device.c | 5 +- drivers/net/bnxt/tf_core/tf_device_p4.c | 2 +- drivers/net/bnxt/tf_core/tf_em.h | 113 +--- drivers/net/bnxt/tf_core/tf_em_common.c | 683 ++++++++++++++++++++++- drivers/net/bnxt/tf_core/tf_em_common.h | 30 ++ drivers/net/bnxt/tf_core/tf_em_host.c | 689 +----------------------- drivers/net/bnxt/tf_core/tf_em_system.c | 541 ++++++++++++++++--- drivers/net/bnxt/tf_core/tf_if_tbl.h | 4 +- drivers/net/bnxt/tf_core/tf_msg.c | 24 + drivers/net/bnxt/tf_core/tf_tbl.h | 7 + drivers/net/bnxt/tf_core/tfp.c | 12 + drivers/net/bnxt/tf_core/tfp.h | 15 + 21 files changed, 1319 insertions(+), 870 deletions(-) diff --git a/config/common_base b/config/common_base index fe30c515e..370a48f02 100644 --- a/config/common_base +++ b/config/common_base @@ -220,6 +220,7 @@ CONFIG_RTE_LIBRTE_BNX2X_DEBUG_PERIODIC=n # Compile burst-oriented Broadcom BNXT PMD driver # CONFIG_RTE_LIBRTE_BNXT_PMD=y +CONFIG_RTE_LIBRTE_BNXT_PMD_SYSTEM=n # # Compile burst-oriented Chelsio Terminator (CXGBE) PMD diff --git a/drivers/net/bnxt/Makefile b/drivers/net/bnxt/Makefile index 349b09c36..6b9544b5d 100644 --- a/drivers/net/bnxt/Makefile +++ b/drivers/net/bnxt/Makefile @@ -50,6 +50,9 @@ CFLAGS += -I$(SRCDIR) -I$(SRCDIR)/tf_ulp -I$(SRCDIR)/tf_core -I$(SRCDIR)/hcapi include $(SRCDIR)/tf_ulp/Makefile include $(SRCDIR)/tf_core/Makefile include $(SRCDIR)/hcapi/Makefile +ifeq ($(CONFIG_RTE_LIBRTE_BNXT_PMD_SYSTEM), y) +CFLAGS += -DTF_USE_SYSTEM_MEM +endif endif # diff --git a/drivers/net/bnxt/bnxt.h b/drivers/net/bnxt/bnxt.h index 65862abdc..43e5e7162 100644 --- a/drivers/net/bnxt/bnxt.h +++ b/drivers/net/bnxt/bnxt.h @@ -563,6 +563,13 @@ struct bnxt_rep_info { DEV_RX_OFFLOAD_SCATTER | \ DEV_RX_OFFLOAD_RSS_HASH) +#define MAX_TABLE_SUPPORT 4 +#define MAX_DIR_SUPPORT 2 +struct bnxt_dmabuf_info { + uint32_t entry_num; + int fd[MAX_DIR_SUPPORT][MAX_TABLE_SUPPORT]; +}; + #define BNXT_HWRM_SHORT_REQ_LEN sizeof(struct hwrm_short_input) struct bnxt_flow_stat_info { @@ -780,6 +787,7 @@ struct bnxt { uint16_t port_svif; struct tf tfp; + struct bnxt_dmabuf_info dmabuf; struct bnxt_ulp_context *ulp_ctx; struct bnxt_flow_stat_info *flow_stat; uint8_t flow_xstat; diff --git a/drivers/net/bnxt/bnxt_hwrm.c b/drivers/net/bnxt/bnxt_hwrm.c index e6a28d07c..2605ef039 100644 --- a/drivers/net/bnxt/bnxt_hwrm.c +++ b/drivers/net/bnxt/bnxt_hwrm.c @@ -5506,3 +5506,30 @@ int bnxt_hwrm_cfa_counter_qstats(struct bnxt *bp, return 0; } + +#ifdef RTE_LIBRTE_BNXT_PMD_SYSTEM +int +bnxt_hwrm_oem_cmd(struct bnxt *bp, uint32_t entry_num) +{ + struct hwrm_oem_cmd_input req = {0}; + struct hwrm_oem_cmd_output *resp = bp->hwrm_cmd_resp_addr; + struct bnxt_dmabuf_info oem_data; + int rc = 0; + + HWRM_PREP(&req, HWRM_OEM_CMD, BNXT_USE_CHIMP_MB); + req.IANA = 0x14e4; + + memset(&oem_data, 0, sizeof(struct bnxt_dmabuf_info)); + oem_data.entry_num = (entry_num); + memcpy(&req.oem_data[0], &oem_data, sizeof(struct bnxt_dmabuf_info)); + + rc = bnxt_hwrm_send_message(bp, &req, sizeof(req), BNXT_USE_CHIMP_MB); + HWRM_CHECK_RESULT(); + + bp->dmabuf.entry_num = entry_num; + + HWRM_UNLOCK(); + + return rc; +} +#endif /* RTE_LIBRTE_BNXT_PMD_SYSTEM */ diff --git a/drivers/net/bnxt/bnxt_hwrm.h b/drivers/net/bnxt/bnxt_hwrm.h index 87cd40779..9e0b79904 100644 --- a/drivers/net/bnxt/bnxt_hwrm.h +++ b/drivers/net/bnxt/bnxt_hwrm.h @@ -276,4 +276,5 @@ int bnxt_hwrm_get_dflt_vnic_svif(struct bnxt *bp, uint16_t fid, uint16_t *vnic_id, uint16_t *svif); int bnxt_hwrm_parent_pf_qcfg(struct bnxt *bp); int bnxt_hwrm_port_phy_qcaps(struct bnxt *bp); +int bnxt_hwrm_oem_cmd(struct bnxt *bp, uint32_t entry_num); #endif diff --git a/drivers/net/bnxt/meson.build b/drivers/net/bnxt/meson.build index ace7353be..8f6ed419e 100644 --- a/drivers/net/bnxt/meson.build +++ b/drivers/net/bnxt/meson.build @@ -31,7 +31,6 @@ sources = files('bnxt_cpr.c', 'tf_core/tf_em_common.c', 'tf_core/tf_em_host.c', 'tf_core/tf_em_internal.c', - 'tf_core/tf_em_system.c', 'tf_core/tf_rm.c', 'tf_core/tf_tbl.c', 'tf_core/tfp.c', @@ -46,6 +45,7 @@ sources = files('bnxt_cpr.c', 'tf_core/tf_if_tbl.c', 'tf_core/ll.c', 'tf_core/tf_global_cfg.c', + 'tf_core/tf_em_host.c', 'hcapi/hcapi_cfa_p4.c', diff --git a/drivers/net/bnxt/tf_core/Makefile b/drivers/net/bnxt/tf_core/Makefile index 202db4150..750c25c5e 100644 --- a/drivers/net/bnxt/tf_core/Makefile +++ b/drivers/net/bnxt/tf_core/Makefile @@ -16,8 +16,11 @@ SRCS-$(CONFIG_RTE_LIBRTE_BNXT_PMD) += tf_core/tf_msg.c SRCS-$(CONFIG_RTE_LIBRTE_BNXT_PMD) += tf_core/tf_tbl.c SRCS-$(CONFIG_RTE_LIBRTE_BNXT_PMD) += tf_core/tf_em_common.c SRCS-$(CONFIG_RTE_LIBRTE_BNXT_PMD) += tf_core/tf_em_internal.c +ifeq ($(CONFIG_RTE_LIBRTE_BNXT_PMD_SYSTEM), n) SRCS-$(CONFIG_RTE_LIBRTE_BNXT_PMD) += tf_core/tf_em_host.c -SRCS-$(CONFIG_RTE_LIBRTE_BNXT_PMD) += tf_core/tf_em_system.c +else +SRCS-$(CONFIG_RTE_LIBRTE_BNXT_PMD_SYSTEM) += tf_core/tf_em_system.c +endif SRCS-$(CONFIG_RTE_LIBRTE_BNXT_PMD) += tf_core/tf_session.c SRCS-$(CONFIG_RTE_LIBRTE_BNXT_PMD) += tf_core/tf_device.c SRCS-$(CONFIG_RTE_LIBRTE_BNXT_PMD) += tf_core/tf_device_p4.c diff --git a/drivers/net/bnxt/tf_core/tf_core.c b/drivers/net/bnxt/tf_core/tf_core.c index 0f119b45f..00b2775ed 100644 --- a/drivers/net/bnxt/tf_core/tf_core.c +++ b/drivers/net/bnxt/tf_core/tf_core.c @@ -540,10 +540,12 @@ tf_alloc_tcam_entry(struct tf *tfp, int rc; struct tf_session *tfs; struct tf_dev_info *dev; - struct tf_tcam_alloc_parms aparms = { 0 }; + struct tf_tcam_alloc_parms aparms; TF_CHECK_PARMS2(tfp, parms); + memset(&aparms, 0, sizeof(struct tf_tcam_alloc_parms)); + /* Retrieve the session information */ rc = tf_session_get_session(tfp, &tfs); if (rc) { @@ -598,10 +600,13 @@ tf_set_tcam_entry(struct tf *tfp, int rc; struct tf_session *tfs; struct tf_dev_info *dev; - struct tf_tcam_set_parms sparms = { 0 }; + struct tf_tcam_set_parms sparms; TF_CHECK_PARMS2(tfp, parms); + memset(&sparms, 0, sizeof(struct tf_tcam_set_parms)); + + /* Retrieve the session information */ rc = tf_session_get_session(tfp, &tfs); if (rc) { @@ -667,10 +672,12 @@ tf_free_tcam_entry(struct tf *tfp, int rc; struct tf_session *tfs; struct tf_dev_info *dev; - struct tf_tcam_free_parms fparms = { 0 }; + struct tf_tcam_free_parms fparms; TF_CHECK_PARMS2(tfp, parms); + memset(&fparms, 0, sizeof(struct tf_tcam_free_parms)); + /* Retrieve the session information */ rc = tf_session_get_session(tfp, &tfs); if (rc) { diff --git a/drivers/net/bnxt/tf_core/tf_core.h b/drivers/net/bnxt/tf_core/tf_core.h index 3f54ab16b..9e8042606 100644 --- a/drivers/net/bnxt/tf_core/tf_core.h +++ b/drivers/net/bnxt/tf_core/tf_core.h @@ -1731,7 +1731,7 @@ struct tf_set_if_tbl_entry_parms { /** * [in] Entry data */ - uint32_t *data; + uint8_t *data; /** * [in] Entry size */ @@ -1768,7 +1768,7 @@ struct tf_get_if_tbl_entry_parms { /** * [out] Entry data */ - uint32_t *data; + uint8_t *data; /** * [in] Entry size */ diff --git a/drivers/net/bnxt/tf_core/tf_device.c b/drivers/net/bnxt/tf_core/tf_device.c index ead958418..f08f7eba7 100644 --- a/drivers/net/bnxt/tf_core/tf_device.c +++ b/drivers/net/bnxt/tf_core/tf_device.c @@ -92,8 +92,11 @@ tf_dev_bind_p4(struct tf *tfp, em_cfg.num_elements = TF_EM_TBL_TYPE_MAX; em_cfg.cfg = tf_em_ext_p4; em_cfg.resources = resources; +#ifdef TF_USE_SYSTEM_MEM + em_cfg.mem_type = TF_EEM_MEM_TYPE_SYSTEM; +#else em_cfg.mem_type = TF_EEM_MEM_TYPE_HOST; - +#endif rc = tf_em_ext_common_bind(tfp, &em_cfg); if (rc) { TFP_DRV_LOG(ERR, diff --git a/drivers/net/bnxt/tf_core/tf_device_p4.c b/drivers/net/bnxt/tf_core/tf_device_p4.c index 652608264..dfe626c8a 100644 --- a/drivers/net/bnxt/tf_core/tf_device_p4.c +++ b/drivers/net/bnxt/tf_core/tf_device_p4.c @@ -126,7 +126,7 @@ const struct tf_dev_ops tf_dev_ops_p4 = { .tf_dev_free_ext_tbl = tf_tbl_ext_free, .tf_dev_alloc_search_tbl = NULL, .tf_dev_set_tbl = tf_tbl_set, - .tf_dev_set_ext_tbl = tf_tbl_ext_set, + .tf_dev_set_ext_tbl = tf_tbl_ext_common_set, .tf_dev_get_tbl = tf_tbl_get, .tf_dev_get_bulk_tbl = tf_tbl_bulk_get, .tf_dev_alloc_tcam = tf_tcam_alloc, diff --git a/drivers/net/bnxt/tf_core/tf_em.h b/drivers/net/bnxt/tf_core/tf_em.h index 1c2369c7b..45c0e1168 100644 --- a/drivers/net/bnxt/tf_core/tf_em.h +++ b/drivers/net/bnxt/tf_core/tf_em.h @@ -16,6 +16,9 @@ #include "hcapi/hcapi_cfa_defs.h" +#define TF_EM_MIN_ENTRIES (1 << 15) /* 32K */ +#define TF_EM_MAX_ENTRIES (1 << 27) /* 128M */ + #define TF_HW_EM_KEY_MAX_SIZE 52 #define TF_EM_KEY_RECORD_SIZE 64 @@ -69,8 +72,16 @@ #error "Invalid Page Size specified. Please use a TF_EM_PAGE_SIZE_n define" #endif +/* + * System memory always uses 4K pages + */ +#ifdef TF_USE_SYSTEM_MEM +#define TF_EM_PAGE_SIZE (1 << TF_EM_PAGE_SIZE_4K) +#define TF_EM_PAGE_ALIGNMENT (1 << TF_EM_PAGE_SIZE_4K) +#else #define TF_EM_PAGE_SIZE (1 << TF_EM_PAGE_SHIFT) #define TF_EM_PAGE_ALIGNMENT (1 << TF_EM_PAGE_SHIFT) +#endif /* * Used to build GFID: @@ -168,39 +179,6 @@ struct tf_em_cfg_parms { * @ref tf_em_ext_common_alloc */ -/** - * Allocates EEM Table scope - * - * [in] tfp - * Pointer to TruFlow handle - * - * [in] parms - * Pointer to input parameters - * - * Returns: - * 0 - Success - * -EINVAL - Parameter error - * -ENOMEM - Out of memory - */ -int tf_alloc_eem_tbl_scope(struct tf *tfp, - struct tf_alloc_tbl_scope_parms *parms); - -/** - * Free's EEM Table scope control block - * - * [in] tfp - * Pointer to TruFlow handle - * - * [in] parms - * Pointer to input parameters - * - * Returns: - * 0 - Success - * -EINVAL - Parameter error - */ -int tf_free_eem_tbl_scope_cb(struct tf *tfp, - struct tf_free_tbl_scope_parms *parms); - /** * Insert record in to internal EM table * @@ -374,8 +352,8 @@ int tf_em_ext_common_unbind(struct tf *tfp); * 0 - Success * -EINVAL - Parameter error */ -int tf_em_ext_host_alloc(struct tf *tfp, - struct tf_alloc_tbl_scope_parms *parms); +int tf_em_ext_alloc(struct tf *tfp, + struct tf_alloc_tbl_scope_parms *parms); /** * Free for external EEM using host memory @@ -390,40 +368,8 @@ int tf_em_ext_host_alloc(struct tf *tfp, * 0 - Success * -EINVAL - Parameter error */ -int tf_em_ext_host_free(struct tf *tfp, - struct tf_free_tbl_scope_parms *parms); - -/** - * Alloc for external EEM using system memory - * - * [in] tfp - * Pointer to TruFlow handle - * - * [in] parms - * Pointer to input parameters - * - * Returns: - * 0 - Success - * -EINVAL - Parameter error - */ -int tf_em_ext_system_alloc(struct tf *tfp, - struct tf_alloc_tbl_scope_parms *parms); - -/** - * Free for external EEM using system memory - * - * [in] tfp - * Pointer to TruFlow handle - * - * [in] parms - * Pointer to input parameters - * - * Returns: - * 0 - Success - * -EINVAL - Parameter error - */ -int tf_em_ext_system_free(struct tf *tfp, - struct tf_free_tbl_scope_parms *parms); +int tf_em_ext_free(struct tf *tfp, + struct tf_free_tbl_scope_parms *parms); /** * Common free for external EEM using host or system memory @@ -510,8 +456,8 @@ tf_tbl_ext_free(struct tf *tfp, * - (0) if successful. * - (-EINVAL) on failure. */ -int tf_tbl_ext_set(struct tf *tfp, - struct tf_tbl_set_parms *parms); +int tf_tbl_ext_common_set(struct tf *tfp, + struct tf_tbl_set_parms *parms); /** * Sets the specified external table type element. @@ -529,26 +475,11 @@ int tf_tbl_ext_set(struct tf *tfp, * - (0) if successful. * - (-EINVAL) on failure. */ -int tf_tbl_ext_host_set(struct tf *tfp, - struct tf_tbl_set_parms *parms); +int tf_tbl_ext_set(struct tf *tfp, + struct tf_tbl_set_parms *parms); -/** - * Sets the specified external table type element. - * - * This API sets the specified element data by invoking the - * firmware. - * - * [in] tfp - * Pointer to TF handle - * - * [in] parms - * Pointer to table set parameters - * - * Returns - * - (0) if successful. - * - (-EINVAL) on failure. - */ -int tf_tbl_ext_system_set(struct tf *tfp, - struct tf_tbl_set_parms *parms); +int +tf_em_ext_system_bind(struct tf *tfp, + struct tf_em_cfg_parms *parms); #endif /* _TF_EM_H_ */ diff --git a/drivers/net/bnxt/tf_core/tf_em_common.c b/drivers/net/bnxt/tf_core/tf_em_common.c index 23a7fc9c2..8b02b8ba3 100644 --- a/drivers/net/bnxt/tf_core/tf_em_common.c +++ b/drivers/net/bnxt/tf_core/tf_em_common.c @@ -23,6 +23,8 @@ #include "bnxt.h" +/* Number of pointers per page_size */ +#define MAX_PAGE_PTRS(page_size) ((page_size) / sizeof(void *)) /** * EM DBs. @@ -281,19 +283,602 @@ tf_em_create_key_entry(struct cfa_p4_eem_entry_hdr *result, struct cfa_p4_eem_64b_entry *key_entry) { key_entry->hdr.word1 = result->word1; + key_entry->hdr.pointer = result->pointer; + memcpy(key_entry->key, in_key, TF_HW_EM_KEY_MAX_SIZE + 4); +} - if (result->word1 & CFA_P4_EEM_ENTRY_ACT_REC_INT_MASK) - key_entry->hdr.pointer = result->pointer; - else - key_entry->hdr.pointer = result->pointer; - memcpy(key_entry->key, in_key, TF_HW_EM_KEY_MAX_SIZE + 4); +/** + * Return the number of page table pages needed to + * reference the given number of next level pages. + * + * [in] num_pages + * Number of EM pages + * + * [in] page_size + * Size of each EM page + * + * Returns: + * Number of EM page table pages + */ +static uint32_t +tf_em_page_tbl_pgcnt(uint32_t num_pages, + uint32_t page_size) +{ + return roundup(num_pages, MAX_PAGE_PTRS(page_size)) / + MAX_PAGE_PTRS(page_size); + return 0; +} + +/** + * Given the number of data pages, page_size and the maximum + * number of page table levels (already determined), size + * the number of page table pages required at each level. + * + * [in] max_lvl + * Max number of levels + * + * [in] num_data_pages + * Number of EM data pages + * + * [in] page_size + * Size of an EM page + * + * [out] *page_cnt + * EM page count + */ +static void +tf_em_size_page_tbls(int max_lvl, + uint64_t num_data_pages, + uint32_t page_size, + uint32_t *page_cnt) +{ + if (max_lvl == TF_PT_LVL_0) { + page_cnt[TF_PT_LVL_0] = num_data_pages; + } else if (max_lvl == TF_PT_LVL_1) { + page_cnt[TF_PT_LVL_1] = num_data_pages; + page_cnt[TF_PT_LVL_0] = + tf_em_page_tbl_pgcnt(page_cnt[TF_PT_LVL_1], page_size); + } else if (max_lvl == TF_PT_LVL_2) { + page_cnt[TF_PT_LVL_2] = num_data_pages; + page_cnt[TF_PT_LVL_1] = + tf_em_page_tbl_pgcnt(page_cnt[TF_PT_LVL_2], page_size); + page_cnt[TF_PT_LVL_0] = + tf_em_page_tbl_pgcnt(page_cnt[TF_PT_LVL_1], page_size); + } else { + return; + } +} + +/** + * Given the page size, size of each data item (entry size), + * and the total number of entries needed, determine the number + * of page table levels and the number of data pages required. + * + * [in] page_size + * Page size + * + * [in] entry_size + * Entry size + * + * [in] num_entries + * Number of entries needed + * + * [out] num_data_pages + * Number of pages required + * + * Returns: + * Success - Number of EM page levels required + * -ENOMEM - Out of memory + */ +static int +tf_em_size_page_tbl_lvl(uint32_t page_size, + uint32_t entry_size, + uint32_t num_entries, + uint64_t *num_data_pages) +{ + uint64_t lvl_data_size = page_size; + int lvl = TF_PT_LVL_0; + uint64_t data_size; + + *num_data_pages = 0; + data_size = (uint64_t)num_entries * entry_size; + + while (lvl_data_size < data_size) { + lvl++; + + if (lvl == TF_PT_LVL_1) + lvl_data_size = (uint64_t)MAX_PAGE_PTRS(page_size) * + page_size; + else if (lvl == TF_PT_LVL_2) + lvl_data_size = (uint64_t)MAX_PAGE_PTRS(page_size) * + MAX_PAGE_PTRS(page_size) * page_size; + else + return -ENOMEM; + } + + *num_data_pages = roundup(data_size, page_size) / page_size; + + return lvl; +} + +/** + * Size the EM table based on capabilities + * + * [in] tbl + * EM table to size + * + * Returns: + * 0 - Success + * - EINVAL - Parameter error + * - ENOMEM - Out of memory + */ +int +tf_em_size_table(struct hcapi_cfa_em_table *tbl, + uint32_t page_size) +{ + uint64_t num_data_pages; + uint32_t *page_cnt; + int max_lvl; + uint32_t num_entries; + uint32_t cnt = TF_EM_MIN_ENTRIES; + + /* Ignore entry if both size and number are zero */ + if (!tbl->entry_size && !tbl->num_entries) + return 0; + + /* If only one is set then error */ + if (!tbl->entry_size || !tbl->num_entries) + return -EINVAL; + + /* Determine number of page table levels and the number + * of data pages needed to process the given eem table. + */ + if (tbl->type == TF_RECORD_TABLE) { + /* + * For action records just a memory size is provided. Work + * backwards to resolve to number of entries + */ + num_entries = tbl->num_entries / tbl->entry_size; + if (num_entries < TF_EM_MIN_ENTRIES) { + num_entries = TF_EM_MIN_ENTRIES; + } else { + while (num_entries > cnt && cnt <= TF_EM_MAX_ENTRIES) + cnt *= 2; + num_entries = cnt; + } + } else { + num_entries = tbl->num_entries; + } + + max_lvl = tf_em_size_page_tbl_lvl(page_size, + tbl->entry_size, + tbl->num_entries, + &num_data_pages); + if (max_lvl < 0) { + TFP_DRV_LOG(WARNING, "EEM: Failed to size page table levels\n"); + TFP_DRV_LOG(WARNING, + "table: %d data-sz: %016" PRIu64 " page-sz: %u\n", + tbl->type, (uint64_t)num_entries * tbl->entry_size, + page_size); + return -ENOMEM; + } + + tbl->num_lvl = max_lvl + 1; + tbl->num_data_pages = num_data_pages; + + /* Determine the number of pages needed at each level */ + page_cnt = tbl->page_cnt; + memset(page_cnt, 0, sizeof(tbl->page_cnt)); + tf_em_size_page_tbls(max_lvl, num_data_pages, page_size, + page_cnt); + + TFP_DRV_LOG(INFO, "EEM: Sized page table: %d\n", tbl->type); + TFP_DRV_LOG(INFO, + "EEM: lvls: %d sz: %016" PRIu64 " pgs: %016" PRIu64 \ + " l0: %u l1: %u l2: %u\n", + max_lvl + 1, + (uint64_t)num_data_pages * page_size, + num_data_pages, + page_cnt[TF_PT_LVL_0], + page_cnt[TF_PT_LVL_1], + page_cnt[TF_PT_LVL_2]); + + return 0; +} + +/** + * Validates EM number of entries requested + * + * [in] tbl_scope_cb + * Pointer to table scope control block to be populated + * + * [in] parms + * Pointer to input parameters + * + * Returns: + * 0 - Success + * -EINVAL - Parameter error + */ +int +tf_em_validate_num_entries(struct tf_tbl_scope_cb *tbl_scope_cb, + struct tf_alloc_tbl_scope_parms *parms) +{ + uint32_t cnt; + + if (parms->rx_mem_size_in_mb != 0) { + uint32_t key_b = 2 * ((parms->rx_max_key_sz_in_bits / 8) + 1); + uint32_t action_b = ((parms->rx_max_action_entry_sz_in_bits / 8) + + 1); + uint32_t num_entries = (parms->rx_mem_size_in_mb * + TF_MEGABYTE) / (key_b + action_b); + + if (num_entries < TF_EM_MIN_ENTRIES) { + TFP_DRV_LOG(ERR, "EEM: Insufficient memory requested:" + "%uMB\n", + parms->rx_mem_size_in_mb); + return -EINVAL; + } + + cnt = TF_EM_MIN_ENTRIES; + while (num_entries > cnt && + cnt <= TF_EM_MAX_ENTRIES) + cnt *= 2; + + if (cnt > TF_EM_MAX_ENTRIES) { + TFP_DRV_LOG(ERR, "EEM: Invalid number of Tx requested: " + "%u\n", + (parms->tx_num_flows_in_k * TF_KILOBYTE)); + return -EINVAL; + } + + parms->rx_num_flows_in_k = cnt / TF_KILOBYTE; + } else { + if ((parms->rx_num_flows_in_k * TF_KILOBYTE) < + TF_EM_MIN_ENTRIES || + (parms->rx_num_flows_in_k * TF_KILOBYTE) > + tbl_scope_cb->em_caps[TF_DIR_RX].max_entries_supported) { + TFP_DRV_LOG(ERR, + "EEM: Invalid number of Rx flows " + "requested:%u max:%u\n", + parms->rx_num_flows_in_k * TF_KILOBYTE, + tbl_scope_cb->em_caps[TF_DIR_RX].max_entries_supported); + return -EINVAL; + } + + /* must be a power-of-2 supported value + * in the range 32K - 128M + */ + cnt = TF_EM_MIN_ENTRIES; + while ((parms->rx_num_flows_in_k * TF_KILOBYTE) != cnt && + cnt <= TF_EM_MAX_ENTRIES) + cnt *= 2; + + if (cnt > TF_EM_MAX_ENTRIES) { + TFP_DRV_LOG(ERR, + "EEM: Invalid number of Rx requested: %u\n", + (parms->rx_num_flows_in_k * TF_KILOBYTE)); + return -EINVAL; + } + } + + if (parms->tx_mem_size_in_mb != 0) { + uint32_t key_b = 2 * (parms->tx_max_key_sz_in_bits / 8 + 1); + uint32_t action_b = ((parms->tx_max_action_entry_sz_in_bits / 8) + + 1); + uint32_t num_entries = (parms->tx_mem_size_in_mb * + (TF_KILOBYTE * TF_KILOBYTE)) / + (key_b + action_b); + + if (num_entries < TF_EM_MIN_ENTRIES) { + TFP_DRV_LOG(ERR, + "EEM: Insufficient memory requested:%uMB\n", + parms->rx_mem_size_in_mb); + return -EINVAL; + } + + cnt = TF_EM_MIN_ENTRIES; + while (num_entries > cnt && + cnt <= TF_EM_MAX_ENTRIES) + cnt *= 2; + + if (cnt > TF_EM_MAX_ENTRIES) { + TFP_DRV_LOG(ERR, + "EEM: Invalid number of Tx requested: %u\n", + (parms->tx_num_flows_in_k * TF_KILOBYTE)); + return -EINVAL; + } + + parms->tx_num_flows_in_k = cnt / TF_KILOBYTE; + } else { + if ((parms->tx_num_flows_in_k * TF_KILOBYTE) < + TF_EM_MIN_ENTRIES || + (parms->tx_num_flows_in_k * TF_KILOBYTE) > + tbl_scope_cb->em_caps[TF_DIR_TX].max_entries_supported) { + TFP_DRV_LOG(ERR, + "EEM: Invalid number of Tx flows " + "requested:%u max:%u\n", + (parms->tx_num_flows_in_k * TF_KILOBYTE), + tbl_scope_cb->em_caps[TF_DIR_TX].max_entries_supported); + return -EINVAL; + } + + cnt = TF_EM_MIN_ENTRIES; + while ((parms->tx_num_flows_in_k * TF_KILOBYTE) != cnt && + cnt <= TF_EM_MAX_ENTRIES) + cnt *= 2; + + if (cnt > TF_EM_MAX_ENTRIES) { + TFP_DRV_LOG(ERR, + "EEM: Invalid number of Tx requested: %u\n", + (parms->tx_num_flows_in_k * TF_KILOBYTE)); + return -EINVAL; + } + } + + if (parms->rx_num_flows_in_k != 0 && + parms->rx_max_key_sz_in_bits / 8 == 0) { + TFP_DRV_LOG(ERR, + "EEM: Rx key size required: %u\n", + (parms->rx_max_key_sz_in_bits)); + return -EINVAL; + } + + if (parms->tx_num_flows_in_k != 0 && + parms->tx_max_key_sz_in_bits / 8 == 0) { + TFP_DRV_LOG(ERR, + "EEM: Tx key size required: %u\n", + (parms->tx_max_key_sz_in_bits)); + return -EINVAL; + } + /* Rx */ + tbl_scope_cb->em_ctx_info[TF_DIR_RX].em_tables[TF_KEY0_TABLE].num_entries = + parms->rx_num_flows_in_k * TF_KILOBYTE; + tbl_scope_cb->em_ctx_info[TF_DIR_RX].em_tables[TF_KEY0_TABLE].entry_size = + parms->rx_max_key_sz_in_bits / 8; + + tbl_scope_cb->em_ctx_info[TF_DIR_RX].em_tables[TF_KEY1_TABLE].num_entries = + parms->rx_num_flows_in_k * TF_KILOBYTE; + tbl_scope_cb->em_ctx_info[TF_DIR_RX].em_tables[TF_KEY1_TABLE].entry_size = + parms->rx_max_key_sz_in_bits / 8; + + tbl_scope_cb->em_ctx_info[TF_DIR_RX].em_tables[TF_RECORD_TABLE].num_entries = + parms->rx_num_flows_in_k * TF_KILOBYTE; + tbl_scope_cb->em_ctx_info[TF_DIR_RX].em_tables[TF_RECORD_TABLE].entry_size = + parms->rx_max_action_entry_sz_in_bits / 8; + + tbl_scope_cb->em_ctx_info[TF_DIR_RX].em_tables[TF_EFC_TABLE].num_entries = 0; + + /* Tx */ + tbl_scope_cb->em_ctx_info[TF_DIR_TX].em_tables[TF_KEY0_TABLE].num_entries = + parms->tx_num_flows_in_k * TF_KILOBYTE; + tbl_scope_cb->em_ctx_info[TF_DIR_TX].em_tables[TF_KEY0_TABLE].entry_size = + parms->tx_max_key_sz_in_bits / 8; + + tbl_scope_cb->em_ctx_info[TF_DIR_TX].em_tables[TF_KEY1_TABLE].num_entries = + parms->tx_num_flows_in_k * TF_KILOBYTE; + tbl_scope_cb->em_ctx_info[TF_DIR_TX].em_tables[TF_KEY1_TABLE].entry_size = + parms->tx_max_key_sz_in_bits / 8; + + tbl_scope_cb->em_ctx_info[TF_DIR_TX].em_tables[TF_RECORD_TABLE].num_entries = + parms->tx_num_flows_in_k * TF_KILOBYTE; + tbl_scope_cb->em_ctx_info[TF_DIR_TX].em_tables[TF_RECORD_TABLE].entry_size = + parms->tx_max_action_entry_sz_in_bits / 8; + + tbl_scope_cb->em_ctx_info[TF_DIR_TX].em_tables[TF_EFC_TABLE].num_entries = 0; + + return 0; +} + +/** insert EEM entry API + * + * returns: + * 0 + * TF_ERR - unable to get lock + * + * insert callback returns: + * 0 + * TF_ERR_EM_DUP - key is already in table + */ +static int +tf_insert_eem_entry(struct tf_tbl_scope_cb *tbl_scope_cb, + struct tf_insert_em_entry_parms *parms) +{ + uint32_t mask; + uint32_t key0_hash; + uint32_t key1_hash; + uint32_t key0_index; + uint32_t key1_index; + struct cfa_p4_eem_64b_entry key_entry; + uint32_t index; + enum hcapi_cfa_em_table_type table_type; + uint32_t gfid; + struct hcapi_cfa_hwop op; + struct hcapi_cfa_key_tbl key_tbl; + struct hcapi_cfa_key_data key_obj; + struct hcapi_cfa_key_loc key_loc; + uint64_t big_hash; + int rc; + + /* Get mask to use on hash */ + mask = tf_em_get_key_mask(tbl_scope_cb->em_ctx_info[parms->dir].em_tables[TF_KEY0_TABLE].num_entries); + + if (!mask) + return -EINVAL; + +#ifdef TF_EEM_DEBUG + dump_raw((uint8_t *)parms->key, TF_HW_EM_KEY_MAX_SIZE + 4, "In Key"); +#endif + + big_hash = hcapi_cfa_key_hash((uint64_t *)parms->key, + (TF_HW_EM_KEY_MAX_SIZE + 4) * 8); + key0_hash = (uint32_t)(big_hash >> 32); + key1_hash = (uint32_t)(big_hash & 0xFFFFFFFF); + + key0_index = key0_hash & mask; + key1_index = key1_hash & mask; #ifdef TF_EEM_DEBUG - dump_raw((uint8_t *)key_entry, TF_EM_KEY_RECORD_SIZE, "Create raw:"); + TFP_DRV_LOG(DEBUG, "Key0 hash:0x%08x\n", key0_hash); + TFP_DRV_LOG(DEBUG, "Key1 hash:0x%08x\n", key1_hash); #endif + /* + * Use the "result" arg to populate all of the key entry then + * store the byte swapped "raw" entry in a local copy ready + * for insertion in to the table. + */ + tf_em_create_key_entry((struct cfa_p4_eem_entry_hdr *)parms->em_record, + ((uint8_t *)parms->key), + &key_entry); + + /* + * Try to add to Key0 table, if that does not work then + * try the key1 table. + */ + index = key0_index; + op.opcode = HCAPI_CFA_HWOPS_ADD; + key_tbl.base0 = + (uint8_t *)&tbl_scope_cb->em_ctx_info[parms->dir].em_tables[TF_KEY0_TABLE]; + key_tbl.page_size = TF_EM_PAGE_SIZE; + key_obj.offset = index * TF_EM_KEY_RECORD_SIZE; + key_obj.data = (uint8_t *)&key_entry; + key_obj.size = TF_EM_KEY_RECORD_SIZE; + + rc = hcapi_cfa_key_hw_op(&op, + &key_tbl, + &key_obj, + &key_loc); + + if (rc == 0) { + table_type = TF_KEY0_TABLE; + } else { + index = key1_index; + + key_tbl.base0 = + (uint8_t *)&tbl_scope_cb->em_ctx_info[parms->dir].em_tables[TF_KEY1_TABLE]; + key_obj.offset = index * TF_EM_KEY_RECORD_SIZE; + + rc = hcapi_cfa_key_hw_op(&op, + &key_tbl, + &key_obj, + &key_loc); + if (rc != 0) + return rc; + + table_type = TF_KEY1_TABLE; + } + + TF_SET_GFID(gfid, + index, + table_type); + TF_SET_FLOW_ID(parms->flow_id, + gfid, + TF_GFID_TABLE_EXTERNAL, + parms->dir); + TF_SET_FIELDS_IN_FLOW_HANDLE(parms->flow_handle, + 0, + 0, + 0, + index, + 0, + table_type); + + return 0; +} + +/** delete EEM hash entry API + * + * returns: + * 0 + * -EINVAL - parameter error + * TF_NO_SESSION - bad session ID + * TF_ERR_TBL_SCOPE - invalid table scope + * TF_ERR_TBL_IF - invalid table interface + * + * insert callback returns + * 0 + * TF_NO_EM_MATCH - entry not found + */ +static int +tf_delete_eem_entry(struct tf_tbl_scope_cb *tbl_scope_cb, + struct tf_delete_em_entry_parms *parms) +{ + enum hcapi_cfa_em_table_type hash_type; + uint32_t index; + struct hcapi_cfa_hwop op; + struct hcapi_cfa_key_tbl key_tbl; + struct hcapi_cfa_key_data key_obj; + struct hcapi_cfa_key_loc key_loc; + int rc; + + TF_GET_HASH_TYPE_FROM_FLOW_HANDLE(parms->flow_handle, hash_type); + TF_GET_INDEX_FROM_FLOW_HANDLE(parms->flow_handle, index); + + op.opcode = HCAPI_CFA_HWOPS_DEL; + key_tbl.base0 = + (uint8_t *)&tbl_scope_cb->em_ctx_info[parms->dir].em_tables + [(hash_type == 0 ? TF_KEY0_TABLE : TF_KEY1_TABLE)]; + key_tbl.page_size = TF_EM_PAGE_SIZE; + key_obj.offset = index * TF_EM_KEY_RECORD_SIZE; + key_obj.data = NULL; + key_obj.size = TF_EM_KEY_RECORD_SIZE; + + rc = hcapi_cfa_key_hw_op(&op, + &key_tbl, + &key_obj, + &key_loc); + + if (!rc) + return rc; + + return 0; +} + +/** insert EM hash entry API + * + * returns: + * 0 - Success + * -EINVAL - Error + */ +int +tf_em_insert_ext_entry(struct tf *tfp __rte_unused, + struct tf_insert_em_entry_parms *parms) +{ + struct tf_tbl_scope_cb *tbl_scope_cb; + + tbl_scope_cb = tbl_scope_cb_find(parms->tbl_scope_id); + if (tbl_scope_cb == NULL) { + TFP_DRV_LOG(ERR, "Invalid tbl_scope_cb\n"); + return -EINVAL; + } + + return tf_insert_eem_entry + (tbl_scope_cb, + parms); +} + +/** Delete EM hash entry API + * + * returns: + * 0 - Success + * -EINVAL - Error + */ +int +tf_em_delete_ext_entry(struct tf *tfp __rte_unused, + struct tf_delete_em_entry_parms *parms) +{ + struct tf_tbl_scope_cb *tbl_scope_cb; + + tbl_scope_cb = tbl_scope_cb_find(parms->tbl_scope_id); + if (tbl_scope_cb == NULL) { + TFP_DRV_LOG(ERR, "Invalid tbl_scope_cb\n"); + return -EINVAL; + } + + return tf_delete_eem_entry(tbl_scope_cb, parms); } + int tf_em_ext_common_bind(struct tf *tfp, struct tf_em_cfg_parms *parms) @@ -341,6 +926,7 @@ tf_em_ext_common_bind(struct tf *tfp, init = 1; mem_type = parms->mem_type; + return 0; } @@ -375,31 +961,88 @@ tf_em_ext_common_unbind(struct tf *tfp) return 0; } -int tf_tbl_ext_set(struct tf *tfp, - struct tf_tbl_set_parms *parms) +/** + * Sets the specified external table type element. + * + * This API sets the specified element data + * + * [in] tfp + * Pointer to TF handle + * + * [in] parms + * Pointer to table set parameters + * + * Returns + * - (0) if successful. + * - (-EINVAL) on failure. + */ +int tf_tbl_ext_common_set(struct tf *tfp, + struct tf_tbl_set_parms *parms) { - if (mem_type == TF_EEM_MEM_TYPE_HOST) - return tf_tbl_ext_host_set(tfp, parms); - else - return tf_tbl_ext_system_set(tfp, parms); + int rc = 0; + struct tf_tbl_scope_cb *tbl_scope_cb; + uint32_t tbl_scope_id; + struct hcapi_cfa_hwop op; + struct hcapi_cfa_key_tbl key_tbl; + struct hcapi_cfa_key_data key_obj; + struct hcapi_cfa_key_loc key_loc; + + TF_CHECK_PARMS2(tfp, parms); + + if (parms->data == NULL) { + TFP_DRV_LOG(ERR, + "%s, invalid parms->data\n", + tf_dir_2_str(parms->dir)); + return -EINVAL; + } + + tbl_scope_id = parms->tbl_scope_id; + + if (tbl_scope_id == TF_TBL_SCOPE_INVALID) { + TFP_DRV_LOG(ERR, + "%s, Table scope not allocated\n", + tf_dir_2_str(parms->dir)); + return -EINVAL; + } + + /* Get the table scope control block associated with the + * external pool + */ + tbl_scope_cb = tbl_scope_cb_find(tbl_scope_id); + + if (tbl_scope_cb == NULL) { + TFP_DRV_LOG(ERR, + "%s, table scope error\n", + tf_dir_2_str(parms->dir)); + return -EINVAL; + } + + op.opcode = HCAPI_CFA_HWOPS_PUT; + key_tbl.base0 = + (uint8_t *)&tbl_scope_cb->em_ctx_info[parms->dir].em_tables[TF_RECORD_TABLE]; + key_tbl.page_size = TF_EM_PAGE_SIZE; + key_obj.offset = parms->idx; + key_obj.data = parms->data; + key_obj.size = parms->data_sz_in_bytes; + + rc = hcapi_cfa_key_hw_op(&op, + &key_tbl, + &key_obj, + &key_loc); + + return rc; } int tf_em_ext_common_alloc(struct tf *tfp, struct tf_alloc_tbl_scope_parms *parms) { - if (mem_type == TF_EEM_MEM_TYPE_HOST) - return tf_em_ext_host_alloc(tfp, parms); - else - return tf_em_ext_system_alloc(tfp, parms); + return tf_em_ext_alloc(tfp, parms); } int tf_em_ext_common_free(struct tf *tfp, struct tf_free_tbl_scope_parms *parms) { - if (mem_type == TF_EEM_MEM_TYPE_HOST) - return tf_em_ext_host_free(tfp, parms); - else - return tf_em_ext_system_free(tfp, parms); + return tf_em_ext_free(tfp, parms); } diff --git a/drivers/net/bnxt/tf_core/tf_em_common.h b/drivers/net/bnxt/tf_core/tf_em_common.h index bf01df9b8..fa313c458 100644 --- a/drivers/net/bnxt/tf_core/tf_em_common.h +++ b/drivers/net/bnxt/tf_core/tf_em_common.h @@ -101,4 +101,34 @@ void *tf_em_get_table_page(struct tf_tbl_scope_cb *tbl_scope_cb, uint32_t offset, enum hcapi_cfa_em_table_type table_type); +/** + * Validates EM number of entries requested + * + * [in] tbl_scope_cb + * Pointer to table scope control block to be populated + * + * [in] parms + * Pointer to input parameters + * + * Returns: + * 0 - Success + * -EINVAL - Parameter error + */ +int tf_em_validate_num_entries(struct tf_tbl_scope_cb *tbl_scope_cb, + struct tf_alloc_tbl_scope_parms *parms); + +/** + * Size the EM table based on capabilities + * + * [in] tbl + * EM table to size + * + * Returns: + * 0 - Success + * - EINVAL - Parameter error + * - ENOMEM - Out of memory + */ +int tf_em_size_table(struct hcapi_cfa_em_table *tbl, + uint32_t page_size); + #endif /* _TF_EM_COMMON_H_ */ diff --git a/drivers/net/bnxt/tf_core/tf_em_host.c b/drivers/net/bnxt/tf_core/tf_em_host.c index 2626a59fe..8cc92c438 100644 --- a/drivers/net/bnxt/tf_core/tf_em_host.c +++ b/drivers/net/bnxt/tf_core/tf_em_host.c @@ -22,7 +22,6 @@ #include "bnxt.h" - #define PTU_PTE_VALID 0x1UL #define PTU_PTE_LAST 0x2UL #define PTU_PTE_NEXT_TO_LAST 0x4UL @@ -30,20 +29,6 @@ /* Number of pointers per page_size */ #define MAX_PAGE_PTRS(page_size) ((page_size) / sizeof(void *)) -#define TF_EM_PG_SZ_4K (1 << 12) -#define TF_EM_PG_SZ_8K (1 << 13) -#define TF_EM_PG_SZ_64K (1 << 16) -#define TF_EM_PG_SZ_256K (1 << 18) -#define TF_EM_PG_SZ_1M (1 << 20) -#define TF_EM_PG_SZ_2M (1 << 21) -#define TF_EM_PG_SZ_4M (1 << 22) -#define TF_EM_PG_SZ_1G (1 << 30) - -#define TF_EM_CTX_ID_INVALID 0xFFFF - -#define TF_EM_MIN_ENTRIES (1 << 15) /* 32K */ -#define TF_EM_MAX_ENTRIES (1 << 27) /* 128M */ - /** * EM DBs. */ @@ -294,203 +279,6 @@ tf_em_setup_page_table(struct hcapi_cfa_em_table *tbl) tbl->l0_dma_addr = tbl->pg_tbl[TF_PT_LVL_0].pg_pa_tbl[0]; } -/** - * Given the page size, size of each data item (entry size), - * and the total number of entries needed, determine the number - * of page table levels and the number of data pages required. - * - * [in] page_size - * Page size - * - * [in] entry_size - * Entry size - * - * [in] num_entries - * Number of entries needed - * - * [out] num_data_pages - * Number of pages required - * - * Returns: - * Success - Number of EM page levels required - * -ENOMEM - Out of memory - */ -static int -tf_em_size_page_tbl_lvl(uint32_t page_size, - uint32_t entry_size, - uint32_t num_entries, - uint64_t *num_data_pages) -{ - uint64_t lvl_data_size = page_size; - int lvl = TF_PT_LVL_0; - uint64_t data_size; - - *num_data_pages = 0; - data_size = (uint64_t)num_entries * entry_size; - - while (lvl_data_size < data_size) { - lvl++; - - if (lvl == TF_PT_LVL_1) - lvl_data_size = (uint64_t)MAX_PAGE_PTRS(page_size) * - page_size; - else if (lvl == TF_PT_LVL_2) - lvl_data_size = (uint64_t)MAX_PAGE_PTRS(page_size) * - MAX_PAGE_PTRS(page_size) * page_size; - else - return -ENOMEM; - } - - *num_data_pages = roundup(data_size, page_size) / page_size; - - return lvl; -} - -/** - * Return the number of page table pages needed to - * reference the given number of next level pages. - * - * [in] num_pages - * Number of EM pages - * - * [in] page_size - * Size of each EM page - * - * Returns: - * Number of EM page table pages - */ -static uint32_t -tf_em_page_tbl_pgcnt(uint32_t num_pages, - uint32_t page_size) -{ - return roundup(num_pages, MAX_PAGE_PTRS(page_size)) / - MAX_PAGE_PTRS(page_size); - return 0; -} - -/** - * Given the number of data pages, page_size and the maximum - * number of page table levels (already determined), size - * the number of page table pages required at each level. - * - * [in] max_lvl - * Max number of levels - * - * [in] num_data_pages - * Number of EM data pages - * - * [in] page_size - * Size of an EM page - * - * [out] *page_cnt - * EM page count - */ -static void -tf_em_size_page_tbls(int max_lvl, - uint64_t num_data_pages, - uint32_t page_size, - uint32_t *page_cnt) -{ - if (max_lvl == TF_PT_LVL_0) { - page_cnt[TF_PT_LVL_0] = num_data_pages; - } else if (max_lvl == TF_PT_LVL_1) { - page_cnt[TF_PT_LVL_1] = num_data_pages; - page_cnt[TF_PT_LVL_0] = - tf_em_page_tbl_pgcnt(page_cnt[TF_PT_LVL_1], page_size); - } else if (max_lvl == TF_PT_LVL_2) { - page_cnt[TF_PT_LVL_2] = num_data_pages; - page_cnt[TF_PT_LVL_1] = - tf_em_page_tbl_pgcnt(page_cnt[TF_PT_LVL_2], page_size); - page_cnt[TF_PT_LVL_0] = - tf_em_page_tbl_pgcnt(page_cnt[TF_PT_LVL_1], page_size); - } else { - return; - } -} - -/** - * Size the EM table based on capabilities - * - * [in] tbl - * EM table to size - * - * Returns: - * 0 - Success - * - EINVAL - Parameter error - * - ENOMEM - Out of memory - */ -static int -tf_em_size_table(struct hcapi_cfa_em_table *tbl) -{ - uint64_t num_data_pages; - uint32_t *page_cnt; - int max_lvl; - uint32_t num_entries; - uint32_t cnt = TF_EM_MIN_ENTRIES; - - /* Ignore entry if both size and number are zero */ - if (!tbl->entry_size && !tbl->num_entries) - return 0; - - /* If only one is set then error */ - if (!tbl->entry_size || !tbl->num_entries) - return -EINVAL; - - /* Determine number of page table levels and the number - * of data pages needed to process the given eem table. - */ - if (tbl->type == TF_RECORD_TABLE) { - /* - * For action records just a memory size is provided. Work - * backwards to resolve to number of entries - */ - num_entries = tbl->num_entries / tbl->entry_size; - if (num_entries < TF_EM_MIN_ENTRIES) { - num_entries = TF_EM_MIN_ENTRIES; - } else { - while (num_entries > cnt && cnt <= TF_EM_MAX_ENTRIES) - cnt *= 2; - num_entries = cnt; - } - } else { - num_entries = tbl->num_entries; - } - - max_lvl = tf_em_size_page_tbl_lvl(TF_EM_PAGE_SIZE, - tbl->entry_size, - tbl->num_entries, - &num_data_pages); - if (max_lvl < 0) { - TFP_DRV_LOG(WARNING, "EEM: Failed to size page table levels\n"); - TFP_DRV_LOG(WARNING, - "table: %d data-sz: %016" PRIu64 " page-sz: %u\n", - tbl->type, (uint64_t)num_entries * tbl->entry_size, - TF_EM_PAGE_SIZE); - return -ENOMEM; - } - - tbl->num_lvl = max_lvl + 1; - tbl->num_data_pages = num_data_pages; - - /* Determine the number of pages needed at each level */ - page_cnt = tbl->page_cnt; - memset(page_cnt, 0, sizeof(tbl->page_cnt)); - tf_em_size_page_tbls(max_lvl, num_data_pages, TF_EM_PAGE_SIZE, - page_cnt); - - TFP_DRV_LOG(INFO, "EEM: Sized page table: %d\n", tbl->type); - TFP_DRV_LOG(INFO, - "EEM: lvls: %d sz: %016" PRIu64 " pgs: %016" PRIu64 " l0: %u l1: %u l2: %u\n", - max_lvl + 1, - (uint64_t)num_data_pages * TF_EM_PAGE_SIZE, - num_data_pages, - page_cnt[TF_PT_LVL_0], - page_cnt[TF_PT_LVL_1], - page_cnt[TF_PT_LVL_2]); - - return 0; -} - /** * Unregisters EM Ctx in Firmware * @@ -552,7 +340,7 @@ tf_em_ctx_reg(struct tf *tfp, tbl = &ctxp->em_tables[i]; if (tbl->num_entries && tbl->entry_size) { - rc = tf_em_size_table(tbl); + rc = tf_em_size_table(tbl, TF_EM_PAGE_SIZE); if (rc) goto cleanup; @@ -578,403 +366,8 @@ tf_em_ctx_reg(struct tf *tfp, return rc; } - -/** - * Validates EM number of entries requested - * - * [in] tbl_scope_cb - * Pointer to table scope control block to be populated - * - * [in] parms - * Pointer to input parameters - * - * Returns: - * 0 - Success - * -EINVAL - Parameter error - */ -static int -tf_em_validate_num_entries(struct tf_tbl_scope_cb *tbl_scope_cb, - struct tf_alloc_tbl_scope_parms *parms) -{ - uint32_t cnt; - - if (parms->rx_mem_size_in_mb != 0) { - uint32_t key_b = 2 * ((parms->rx_max_key_sz_in_bits / 8) + 1); - uint32_t action_b = ((parms->rx_max_action_entry_sz_in_bits / 8) - + 1); - uint32_t num_entries = (parms->rx_mem_size_in_mb * - TF_MEGABYTE) / (key_b + action_b); - - if (num_entries < TF_EM_MIN_ENTRIES) { - TFP_DRV_LOG(ERR, "EEM: Insufficient memory requested:" - "%uMB\n", - parms->rx_mem_size_in_mb); - return -EINVAL; - } - - cnt = TF_EM_MIN_ENTRIES; - while (num_entries > cnt && - cnt <= TF_EM_MAX_ENTRIES) - cnt *= 2; - - if (cnt > TF_EM_MAX_ENTRIES) { - TFP_DRV_LOG(ERR, "EEM: Invalid number of Tx requested: " - "%u\n", - (parms->tx_num_flows_in_k * TF_KILOBYTE)); - return -EINVAL; - } - - parms->rx_num_flows_in_k = cnt / TF_KILOBYTE; - } else { - if ((parms->rx_num_flows_in_k * TF_KILOBYTE) < - TF_EM_MIN_ENTRIES || - (parms->rx_num_flows_in_k * TF_KILOBYTE) > - tbl_scope_cb->em_caps[TF_DIR_RX].max_entries_supported) { - TFP_DRV_LOG(ERR, - "EEM: Invalid number of Rx flows " - "requested:%u max:%u\n", - parms->rx_num_flows_in_k * TF_KILOBYTE, - tbl_scope_cb->em_caps[TF_DIR_RX].max_entries_supported); - return -EINVAL; - } - - /* must be a power-of-2 supported value - * in the range 32K - 128M - */ - cnt = TF_EM_MIN_ENTRIES; - while ((parms->rx_num_flows_in_k * TF_KILOBYTE) != cnt && - cnt <= TF_EM_MAX_ENTRIES) - cnt *= 2; - - if (cnt > TF_EM_MAX_ENTRIES) { - TFP_DRV_LOG(ERR, - "EEM: Invalid number of Rx requested: %u\n", - (parms->rx_num_flows_in_k * TF_KILOBYTE)); - return -EINVAL; - } - } - - if (parms->tx_mem_size_in_mb != 0) { - uint32_t key_b = 2 * (parms->tx_max_key_sz_in_bits / 8 + 1); - uint32_t action_b = ((parms->tx_max_action_entry_sz_in_bits / 8) - + 1); - uint32_t num_entries = (parms->tx_mem_size_in_mb * - (TF_KILOBYTE * TF_KILOBYTE)) / - (key_b + action_b); - - if (num_entries < TF_EM_MIN_ENTRIES) { - TFP_DRV_LOG(ERR, - "EEM: Insufficient memory requested:%uMB\n", - parms->rx_mem_size_in_mb); - return -EINVAL; - } - - cnt = TF_EM_MIN_ENTRIES; - while (num_entries > cnt && - cnt <= TF_EM_MAX_ENTRIES) - cnt *= 2; - - if (cnt > TF_EM_MAX_ENTRIES) { - TFP_DRV_LOG(ERR, - "EEM: Invalid number of Tx requested: %u\n", - (parms->tx_num_flows_in_k * TF_KILOBYTE)); - return -EINVAL; - } - - parms->tx_num_flows_in_k = cnt / TF_KILOBYTE; - } else { - if ((parms->tx_num_flows_in_k * TF_KILOBYTE) < - TF_EM_MIN_ENTRIES || - (parms->tx_num_flows_in_k * TF_KILOBYTE) > - tbl_scope_cb->em_caps[TF_DIR_TX].max_entries_supported) { - TFP_DRV_LOG(ERR, - "EEM: Invalid number of Tx flows " - "requested:%u max:%u\n", - (parms->tx_num_flows_in_k * TF_KILOBYTE), - tbl_scope_cb->em_caps[TF_DIR_TX].max_entries_supported); - return -EINVAL; - } - - cnt = TF_EM_MIN_ENTRIES; - while ((parms->tx_num_flows_in_k * TF_KILOBYTE) != cnt && - cnt <= TF_EM_MAX_ENTRIES) - cnt *= 2; - - if (cnt > TF_EM_MAX_ENTRIES) { - TFP_DRV_LOG(ERR, - "EEM: Invalid number of Tx requested: %u\n", - (parms->tx_num_flows_in_k * TF_KILOBYTE)); - return -EINVAL; - } - } - - if (parms->rx_num_flows_in_k != 0 && - (parms->rx_max_key_sz_in_bits / 8 == 0)) { - TFP_DRV_LOG(ERR, - "EEM: Rx key size required: %u\n", - (parms->rx_max_key_sz_in_bits)); - return -EINVAL; - } - - if (parms->tx_num_flows_in_k != 0 && - (parms->tx_max_key_sz_in_bits / 8 == 0)) { - TFP_DRV_LOG(ERR, - "EEM: Tx key size required: %u\n", - (parms->tx_max_key_sz_in_bits)); - return -EINVAL; - } - /* Rx */ - tbl_scope_cb->em_ctx_info[TF_DIR_RX].em_tables[TF_KEY0_TABLE].num_entries = - parms->rx_num_flows_in_k * TF_KILOBYTE; - tbl_scope_cb->em_ctx_info[TF_DIR_RX].em_tables[TF_KEY0_TABLE].entry_size = - parms->rx_max_key_sz_in_bits / 8; - - tbl_scope_cb->em_ctx_info[TF_DIR_RX].em_tables[TF_KEY1_TABLE].num_entries = - parms->rx_num_flows_in_k * TF_KILOBYTE; - tbl_scope_cb->em_ctx_info[TF_DIR_RX].em_tables[TF_KEY1_TABLE].entry_size = - parms->rx_max_key_sz_in_bits / 8; - - tbl_scope_cb->em_ctx_info[TF_DIR_RX].em_tables[TF_RECORD_TABLE].num_entries = - parms->rx_num_flows_in_k * TF_KILOBYTE; - tbl_scope_cb->em_ctx_info[TF_DIR_RX].em_tables[TF_RECORD_TABLE].entry_size = - parms->rx_max_action_entry_sz_in_bits / 8; - - tbl_scope_cb->em_ctx_info[TF_DIR_RX].em_tables[TF_EFC_TABLE].num_entries = 0; - - /* Tx */ - tbl_scope_cb->em_ctx_info[TF_DIR_TX].em_tables[TF_KEY0_TABLE].num_entries = - parms->tx_num_flows_in_k * TF_KILOBYTE; - tbl_scope_cb->em_ctx_info[TF_DIR_TX].em_tables[TF_KEY0_TABLE].entry_size = - parms->tx_max_key_sz_in_bits / 8; - - tbl_scope_cb->em_ctx_info[TF_DIR_TX].em_tables[TF_KEY1_TABLE].num_entries = - parms->tx_num_flows_in_k * TF_KILOBYTE; - tbl_scope_cb->em_ctx_info[TF_DIR_TX].em_tables[TF_KEY1_TABLE].entry_size = - parms->tx_max_key_sz_in_bits / 8; - - tbl_scope_cb->em_ctx_info[TF_DIR_TX].em_tables[TF_RECORD_TABLE].num_entries = - parms->tx_num_flows_in_k * TF_KILOBYTE; - tbl_scope_cb->em_ctx_info[TF_DIR_TX].em_tables[TF_RECORD_TABLE].entry_size = - parms->tx_max_action_entry_sz_in_bits / 8; - - tbl_scope_cb->em_ctx_info[TF_DIR_TX].em_tables[TF_EFC_TABLE].num_entries = 0; - - return 0; -} - -/** insert EEM entry API - * - * returns: - * 0 - * TF_ERR - unable to get lock - * - * insert callback returns: - * 0 - * TF_ERR_EM_DUP - key is already in table - */ -static int -tf_insert_eem_entry(struct tf_tbl_scope_cb *tbl_scope_cb, - struct tf_insert_em_entry_parms *parms) -{ - uint32_t mask; - uint32_t key0_hash; - uint32_t key1_hash; - uint32_t key0_index; - uint32_t key1_index; - struct cfa_p4_eem_64b_entry key_entry; - uint32_t index; - enum hcapi_cfa_em_table_type table_type; - uint32_t gfid; - struct hcapi_cfa_hwop op; - struct hcapi_cfa_key_tbl key_tbl; - struct hcapi_cfa_key_data key_obj; - struct hcapi_cfa_key_loc key_loc; - uint64_t big_hash; - int rc; - - /* Get mask to use on hash */ - mask = tf_em_get_key_mask(tbl_scope_cb->em_ctx_info[parms->dir].em_tables[TF_KEY0_TABLE].num_entries); - - if (!mask) - return -EINVAL; - -#ifdef TF_EEM_DEBUG - dump_raw((uint8_t *)parms->key, TF_HW_EM_KEY_MAX_SIZE + 4, "In Key"); -#endif - - big_hash = hcapi_cfa_key_hash((uint64_t *)parms->key, - (TF_HW_EM_KEY_MAX_SIZE + 4) * 8); - key0_hash = (uint32_t)(big_hash >> 32); - key1_hash = (uint32_t)(big_hash & 0xFFFFFFFF); - - key0_index = key0_hash & mask; - key1_index = key1_hash & mask; - -#ifdef TF_EEM_DEBUG - TFP_DRV_LOG(DEBUG, "Key0 hash:0x%08x\n", key0_hash); - TFP_DRV_LOG(DEBUG, "Key1 hash:0x%08x\n", key1_hash); -#endif - /* - * Use the "result" arg to populate all of the key entry then - * store the byte swapped "raw" entry in a local copy ready - * for insertion in to the table. - */ - tf_em_create_key_entry((struct cfa_p4_eem_entry_hdr *)parms->em_record, - ((uint8_t *)parms->key), - &key_entry); - - /* - * Try to add to Key0 table, if that does not work then - * try the key1 table. - */ - index = key0_index; - op.opcode = HCAPI_CFA_HWOPS_ADD; - key_tbl.base0 = (uint8_t *) - &tbl_scope_cb->em_ctx_info[parms->dir].em_tables[TF_KEY0_TABLE]; - key_tbl.page_size = TF_EM_PAGE_SIZE; - key_obj.offset = index * TF_EM_KEY_RECORD_SIZE; - key_obj.data = (uint8_t *)&key_entry; - key_obj.size = TF_EM_KEY_RECORD_SIZE; - - rc = hcapi_cfa_key_hw_op(&op, - &key_tbl, - &key_obj, - &key_loc); - - if (rc == 0) { - table_type = TF_KEY0_TABLE; - } else { - index = key1_index; - - key_tbl.base0 = (uint8_t *) - &tbl_scope_cb->em_ctx_info[parms->dir].em_tables[TF_KEY1_TABLE]; - key_obj.offset = index * TF_EM_KEY_RECORD_SIZE; - - rc = hcapi_cfa_key_hw_op(&op, - &key_tbl, - &key_obj, - &key_loc); - if (rc != 0) - return rc; - - table_type = TF_KEY1_TABLE; - } - - TF_SET_GFID(gfid, - index, - table_type); - TF_SET_FLOW_ID(parms->flow_id, - gfid, - TF_GFID_TABLE_EXTERNAL, - parms->dir); - TF_SET_FIELDS_IN_FLOW_HANDLE(parms->flow_handle, - 0, - 0, - 0, - index, - 0, - table_type); - - return 0; -} - -/** delete EEM hash entry API - * - * returns: - * 0 - * -EINVAL - parameter error - * TF_NO_SESSION - bad session ID - * TF_ERR_TBL_SCOPE - invalid table scope - * TF_ERR_TBL_IF - invalid table interface - * - * insert callback returns - * 0 - * TF_NO_EM_MATCH - entry not found - */ -static int -tf_delete_eem_entry(struct tf_tbl_scope_cb *tbl_scope_cb, - struct tf_delete_em_entry_parms *parms) -{ - enum hcapi_cfa_em_table_type hash_type; - uint32_t index; - struct hcapi_cfa_hwop op; - struct hcapi_cfa_key_tbl key_tbl; - struct hcapi_cfa_key_data key_obj; - struct hcapi_cfa_key_loc key_loc; - int rc; - - if (parms->flow_handle == 0) - return -EINVAL; - - TF_GET_HASH_TYPE_FROM_FLOW_HANDLE(parms->flow_handle, hash_type); - TF_GET_INDEX_FROM_FLOW_HANDLE(parms->flow_handle, index); - - op.opcode = HCAPI_CFA_HWOPS_DEL; - key_tbl.base0 = (uint8_t *) - &tbl_scope_cb->em_ctx_info[parms->dir].em_tables[(hash_type == 0 ? - TF_KEY0_TABLE : - TF_KEY1_TABLE)]; - key_tbl.page_size = TF_EM_PAGE_SIZE; - key_obj.offset = index * TF_EM_KEY_RECORD_SIZE; - key_obj.data = NULL; - key_obj.size = TF_EM_KEY_RECORD_SIZE; - - rc = hcapi_cfa_key_hw_op(&op, - &key_tbl, - &key_obj, - &key_loc); - - if (!rc) - return rc; - - return 0; -} - -/** insert EM hash entry API - * - * returns: - * 0 - Success - * -EINVAL - Error - */ -int -tf_em_insert_ext_entry(struct tf *tfp __rte_unused, - struct tf_insert_em_entry_parms *parms) -{ - struct tf_tbl_scope_cb *tbl_scope_cb; - - tbl_scope_cb = tbl_scope_cb_find(parms->tbl_scope_id); - if (tbl_scope_cb == NULL) { - TFP_DRV_LOG(ERR, "Invalid tbl_scope_cb\n"); - return -EINVAL; - } - - return tf_insert_eem_entry(tbl_scope_cb, parms); -} - -/** Delete EM hash entry API - * - * returns: - * 0 - Success - * -EINVAL - Error - */ int -tf_em_delete_ext_entry(struct tf *tfp __rte_unused, - struct tf_delete_em_entry_parms *parms) -{ - struct tf_tbl_scope_cb *tbl_scope_cb; - - tbl_scope_cb = tbl_scope_cb_find(parms->tbl_scope_id); - if (tbl_scope_cb == NULL) { - TFP_DRV_LOG(ERR, "Invalid tbl_scope_cb\n"); - return -EINVAL; - } - - return tf_delete_eem_entry(tbl_scope_cb, parms); -} - -int -tf_em_ext_host_alloc(struct tf *tfp, - struct tf_alloc_tbl_scope_parms *parms) +tf_em_ext_alloc(struct tf *tfp, struct tf_alloc_tbl_scope_parms *parms) { int rc; enum tf_dir dir; @@ -1081,7 +474,7 @@ tf_em_ext_host_alloc(struct tf *tfp, cleanup_full: free_parms.tbl_scope_id = parms->tbl_scope_id; - tf_em_ext_host_free(tfp, &free_parms); + tf_em_ext_free(tfp, &free_parms); return -EINVAL; cleanup: @@ -1094,8 +487,8 @@ tf_em_ext_host_alloc(struct tf *tfp, } int -tf_em_ext_host_free(struct tf *tfp, - struct tf_free_tbl_scope_parms *parms) +tf_em_ext_free(struct tf *tfp, + struct tf_free_tbl_scope_parms *parms) { int rc = 0; enum tf_dir dir; @@ -1136,75 +529,3 @@ tf_em_ext_host_free(struct tf *tfp, tbl_scopes[parms->tbl_scope_id].tbl_scope_id = TF_TBL_SCOPE_INVALID; return rc; } - -/** - * Sets the specified external table type element. - * - * This API sets the specified element data - * - * [in] tfp - * Pointer to TF handle - * - * [in] parms - * Pointer to table set parameters - * - * Returns - * - (0) if successful. - * - (-EINVAL) on failure. - */ -int tf_tbl_ext_host_set(struct tf *tfp, - struct tf_tbl_set_parms *parms) -{ - int rc = 0; - struct tf_tbl_scope_cb *tbl_scope_cb; - uint32_t tbl_scope_id; - struct hcapi_cfa_hwop op; - struct hcapi_cfa_key_tbl key_tbl; - struct hcapi_cfa_key_data key_obj; - struct hcapi_cfa_key_loc key_loc; - - TF_CHECK_PARMS2(tfp, parms); - - if (parms->data == NULL) { - TFP_DRV_LOG(ERR, - "%s, invalid parms->data\n", - tf_dir_2_str(parms->dir)); - return -EINVAL; - } - - tbl_scope_id = parms->tbl_scope_id; - - if (tbl_scope_id == TF_TBL_SCOPE_INVALID) { - TFP_DRV_LOG(ERR, - "%s, Table scope not allocated\n", - tf_dir_2_str(parms->dir)); - return -EINVAL; - } - - /* Get the table scope control block associated with the - * external pool - */ - tbl_scope_cb = tbl_scope_cb_find(tbl_scope_id); - - if (tbl_scope_cb == NULL) { - TFP_DRV_LOG(ERR, - "%s, table scope error\n", - tf_dir_2_str(parms->dir)); - return -EINVAL; - } - - op.opcode = HCAPI_CFA_HWOPS_PUT; - key_tbl.base0 = - (uint8_t *)&tbl_scope_cb->em_ctx_info[parms->dir].em_tables[TF_RECORD_TABLE]; - key_tbl.page_size = TF_EM_PAGE_SIZE; - key_obj.offset = parms->idx; - key_obj.data = parms->data; - key_obj.size = parms->data_sz_in_bytes; - - rc = hcapi_cfa_key_hw_op(&op, - &key_tbl, - &key_obj, - &key_loc); - - return rc; -} diff --git a/drivers/net/bnxt/tf_core/tf_em_system.c b/drivers/net/bnxt/tf_core/tf_em_system.c index 10768df03..c47c8b93f 100644 --- a/drivers/net/bnxt/tf_core/tf_em_system.c +++ b/drivers/net/bnxt/tf_core/tf_em_system.c @@ -4,11 +4,24 @@ */ #include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include + #include #include #include #include "tf_core.h" +#include "tf_util.h" +#include "tf_common.h" #include "tf_em.h" #include "tf_em_common.h" #include "tf_msg.h" @@ -18,103 +31,503 @@ #include "bnxt.h" +enum tf_em_req_type { + TF_EM_BNXT_LFC_CFA_EEM_DMABUF_EXPORT_REQ = 5, +}; -/** insert EEM entry API - * - * returns: - * 0 - * TF_ERR - unable to get lock - * - * insert callback returns: - * 0 - * TF_ERR_EM_DUP - key is already in table +struct tf_em_bnxt_lfc_req_hdr { + uint32_t ver; + uint32_t bus; + uint32_t devfn; + enum tf_em_req_type req_type; +}; + +struct tf_em_bnxt_lfc_cfa_eem_std_hdr { + uint16_t version; + uint16_t size; + uint32_t flags; + #define TF_EM_BNXT_LFC_EEM_CFG_PRIMARY_FUNC (1 << 0) +}; + +struct tf_em_bnxt_lfc_dmabuf_fd { + int fd[TF_DIR_MAX][TF_MAX_TABLE]; +}; + +#ifndef __user +#define __user +#endif + +struct tf_em_bnxt_lfc_cfa_eem_dmabuf_export_req { + struct tf_em_bnxt_lfc_cfa_eem_std_hdr std; + uint8_t dir; + uint32_t flags; + void __user *dma_fd; +}; + +struct tf_em_bnxt_lfc_req { + struct tf_em_bnxt_lfc_req_hdr hdr; + union { + struct tf_em_bnxt_lfc_cfa_eem_dmabuf_export_req + eem_dmabuf_export_req; + uint64_t hreq; + } req; +}; + +#define TF_EEM_BNXT_LFC_IOCTL_MAGIC 0x98 +#define BNXT_LFC_REQ \ + _IOW(TF_EEM_BNXT_LFC_IOCTL_MAGIC, 1, struct tf_em_bnxt_lfc_req) + +/** + * EM DBs. */ -static int -tf_insert_eem_entry(struct tf_tbl_scope_cb *tbl_scope_cb __rte_unused, - struct tf_insert_em_entry_parms *parms __rte_unused) +extern void *eem_db[TF_DIR_MAX]; + +extern struct tf_tbl_scope_cb tbl_scopes[TF_NUM_TBL_SCOPE]; + +static void +tf_em_dmabuf_mem_unmap(struct hcapi_cfa_em_table *tbl) { - return 0; + struct hcapi_cfa_em_page_tbl *tp; + int level; + uint32_t page_no, pg_count; + + for (level = (tbl->num_lvl - 1); level < tbl->num_lvl; level++) { + tp = &tbl->pg_tbl[level]; + + pg_count = tbl->page_cnt[level]; + for (page_no = 0; page_no < pg_count; page_no++) { + if (tp->pg_va_tbl != NULL && + tp->pg_va_tbl[page_no] != NULL && + tp->pg_size != 0) { + (void)munmap(tp->pg_va_tbl[page_no], + tp->pg_size); + } + } + + tfp_free((void *)tp->pg_va_tbl); + tfp_free((void *)tp->pg_pa_tbl); + } } -/** delete EEM hash entry API +/** + * Unregisters EM Ctx in Firmware + * + * [in] tfp + * Pointer to a TruFlow handle * - * returns: - * 0 - * -EINVAL - parameter error - * TF_NO_SESSION - bad session ID - * TF_ERR_TBL_SCOPE - invalid table scope - * TF_ERR_TBL_IF - invalid table interface + * [in] tbl_scope_cb + * Pointer to a table scope control block * - * insert callback returns - * 0 - * TF_NO_EM_MATCH - entry not found + * [in] dir + * Receive or transmit direction */ +static void +tf_em_ctx_unreg(struct tf_tbl_scope_cb *tbl_scope_cb, + int dir) +{ + struct hcapi_cfa_em_ctx_mem_info *ctxp = + &tbl_scope_cb->em_ctx_info[dir]; + struct hcapi_cfa_em_table *tbl; + int i; + + for (i = TF_KEY0_TABLE; i < TF_MAX_TABLE; i++) { + tbl = &ctxp->em_tables[i]; + tf_em_dmabuf_mem_unmap(tbl); + } +} + +static int tf_export_tbl_scope(int lfc_fd, + int *fd, + int bus, + int devfn) +{ + struct tf_em_bnxt_lfc_req tf_lfc_req; + struct tf_em_bnxt_lfc_dmabuf_fd *dma_fd; + struct tfp_calloc_parms mparms; + int rc; + + memset(&tf_lfc_req, 0, sizeof(struct tf_em_bnxt_lfc_req)); + tf_lfc_req.hdr.ver = 1; + tf_lfc_req.hdr.bus = bus; + tf_lfc_req.hdr.devfn = devfn; + tf_lfc_req.hdr.req_type = TF_EM_BNXT_LFC_CFA_EEM_DMABUF_EXPORT_REQ; + tf_lfc_req.req.eem_dmabuf_export_req.flags = O_ACCMODE; + tf_lfc_req.req.eem_dmabuf_export_req.std.version = 1; + + mparms.nitems = 1; + mparms.size = sizeof(struct tf_em_bnxt_lfc_dmabuf_fd); + mparms.alignment = 0; + tfp_calloc(&mparms); + dma_fd = (struct tf_em_bnxt_lfc_dmabuf_fd *)mparms.mem_va; + tf_lfc_req.req.eem_dmabuf_export_req.dma_fd = dma_fd; + + rc = ioctl(lfc_fd, BNXT_LFC_REQ, &tf_lfc_req); + if (rc) { + TFP_DRV_LOG(ERR, + "EXT EEM export chanel_fd %d, rc=%d\n", + lfc_fd, + rc); + tfp_free(dma_fd); + return rc; + } + + memcpy(fd, dma_fd->fd, sizeof(dma_fd->fd)); + tfp_free(dma_fd); + + return rc; +} + static int -tf_delete_eem_entry(struct tf_tbl_scope_cb *tbl_scope_cb __rte_unused, - struct tf_delete_em_entry_parms *parms __rte_unused) +tf_em_dmabuf_mem_map(struct hcapi_cfa_em_table *tbl, + int dmabuf_fd) { + struct hcapi_cfa_em_page_tbl *tp; + int level; + uint32_t page_no; + uint32_t pg_count; + uint32_t offset; + struct tfp_calloc_parms parms; + + for (level = (tbl->num_lvl - 1); level < tbl->num_lvl; level++) { + tp = &tbl->pg_tbl[level]; + + pg_count = tbl->page_cnt[level]; + offset = 0; + + parms.nitems = pg_count; + parms.size = sizeof(void *); + parms.alignment = 0; + + if ((tfp_calloc(&parms)) != 0) + return -ENOMEM; + + tp->pg_va_tbl = parms.mem_va; + parms.nitems = pg_count; + parms.size = sizeof(void *); + parms.alignment = 0; + + if ((tfp_calloc(&parms)) != 0) { + tfp_free((void *)tp->pg_va_tbl); + return -ENOMEM; + } + + tp->pg_pa_tbl = parms.mem_va; + tp->pg_count = 0; + tp->pg_size = TF_EM_PAGE_SIZE; + + for (page_no = 0; page_no < pg_count; page_no++) { + tp->pg_va_tbl[page_no] = mmap(NULL, + TF_EM_PAGE_SIZE, + PROT_READ | PROT_WRITE, + MAP_SHARED, + dmabuf_fd, + offset); + if (tp->pg_va_tbl[page_no] == (void *)-1) { + TFP_DRV_LOG(ERR, + "MMap memory error. level:%d page:%d pg_count:%d - %s\n", + level, + page_no, + pg_count, + strerror(errno)); + return -ENOMEM; + } + offset += tp->pg_size; + tp->pg_count++; + } + } + return 0; } -/** insert EM hash entry API - * - * returns: - * 0 - Success - * -EINVAL - Error - */ -int -tf_em_insert_ext_sys_entry(struct tf *tfp __rte_unused, - struct tf_insert_em_entry_parms *parms) +static int tf_mmap_tbl_scope(struct tf_tbl_scope_cb *tbl_scope_cb, + enum tf_dir dir, + int tbl_type, + int dmabuf_fd) { - struct tf_tbl_scope_cb *tbl_scope_cb; + struct hcapi_cfa_em_table *tbl; + + if (tbl_type == TF_EFC_TABLE) + return 0; + + tbl = &tbl_scope_cb->em_ctx_info[dir].em_tables[tbl_type]; + return tf_em_dmabuf_mem_map(tbl, dmabuf_fd); +} + +#define TF_LFC_DEVICE "/dev/bnxt_lfc" + +static int +tf_prepare_dmabuf_bnxt_lfc_device(struct tf_tbl_scope_cb *tbl_scope_cb) +{ + int lfc_fd; - tbl_scope_cb = tbl_scope_cb_find(parms->tbl_scope_id); - if (tbl_scope_cb == NULL) { - TFP_DRV_LOG(ERR, "Invalid tbl_scope_cb\n"); - return -EINVAL; + lfc_fd = open(TF_LFC_DEVICE, O_RDWR); + if (!lfc_fd) { + TFP_DRV_LOG(ERR, + "EEM: open %s device error\n", + TF_LFC_DEVICE); + return -ENOENT; } - return tf_insert_eem_entry - (tbl_scope_cb, parms); + tbl_scope_cb->lfc_fd = lfc_fd; + + return 0; } -/** Delete EM hash entry API - * - * returns: - * 0 - Success - * -EINVAL - Error - */ -int -tf_em_delete_ext_sys_entry(struct tf *tfp __rte_unused, - struct tf_delete_em_entry_parms *parms) +static int +offload_system_mmap(struct tf_tbl_scope_cb *tbl_scope_cb) { - struct tf_tbl_scope_cb *tbl_scope_cb; + int rc; + int dmabuf_fd; + enum tf_dir dir; + enum hcapi_cfa_em_table_type tbl_type; - tbl_scope_cb = tbl_scope_cb_find(parms->tbl_scope_id); - if (tbl_scope_cb == NULL) { - TFP_DRV_LOG(ERR, "Invalid tbl_scope_cb\n"); - return -EINVAL; + rc = tf_prepare_dmabuf_bnxt_lfc_device(tbl_scope_cb); + if (rc) { + TFP_DRV_LOG(ERR, "EEM: Prepare bnxt_lfc channel failed\n"); + return rc; } - return tf_delete_eem_entry(tbl_scope_cb, parms); + rc = tf_export_tbl_scope(tbl_scope_cb->lfc_fd, + (int *)tbl_scope_cb->fd, + tbl_scope_cb->bus, + tbl_scope_cb->devfn); + if (rc) { + TFP_DRV_LOG(ERR, + "export dmabuf fd failed\n"); + return rc; + } + + tbl_scope_cb->valid = true; + + for (dir = 0; dir < TF_DIR_MAX; dir++) { + for (tbl_type = TF_KEY0_TABLE; tbl_type < + TF_MAX_TABLE; tbl_type++) { + if (tbl_type == TF_EFC_TABLE) + continue; + + dmabuf_fd = tbl_scope_cb->fd[(dir ? 0 : 1)][tbl_type]; + rc = tf_mmap_tbl_scope(tbl_scope_cb, + dir, + tbl_type, + dmabuf_fd); + if (rc) { + TFP_DRV_LOG(ERR, + "dir:%d tbl:%d mmap failed rc %d\n", + dir, + tbl_type, + rc); + break; + } + } + } + return 0; } -int -tf_em_ext_system_alloc(struct tf *tfp __rte_unused, - struct tf_alloc_tbl_scope_parms *parms __rte_unused) +static int +tf_destroy_dmabuf_bnxt_lfc_device(struct tf_tbl_scope_cb *tbl_scope_cb) { + close(tbl_scope_cb->lfc_fd); + return 0; } -int -tf_em_ext_system_free(struct tf *tfp __rte_unused, - struct tf_free_tbl_scope_parms *parms __rte_unused) +static int +tf_dmabuf_alloc(struct tf *tfp, struct tf_tbl_scope_cb *tbl_scope_cb) { + int rc; + + rc = tfp_msg_hwrm_oem_cmd(tfp, + tbl_scope_cb->em_ctx_info[TF_DIR_RX].em_tables[TF_KEY0_TABLE].num_entries); + if (rc) + PMD_DRV_LOG(ERR, "EEM: Failed to prepare system memory rc:%d\n", + rc); + return 0; } -int tf_tbl_ext_system_set(struct tf *tfp __rte_unused, - struct tf_tbl_set_parms *parms __rte_unused) +static int +tf_dmabuf_free(struct tf *tfp, struct tf_tbl_scope_cb *tbl_scope_cb) { + int rc; + + rc = tfp_msg_hwrm_oem_cmd(tfp, 0); + if (rc) + TFP_DRV_LOG(ERR, "EEM: Failed to cleanup system memory\n"); + + tf_destroy_dmabuf_bnxt_lfc_device(tbl_scope_cb); + return 0; } + +int +tf_em_ext_alloc(struct tf *tfp, + struct tf_alloc_tbl_scope_parms *parms) +{ + int rc; + struct tf_session *tfs; + struct tf_tbl_scope_cb *tbl_scope_cb; + struct tf_rm_allocate_parms aparms = { 0 }; + struct tf_free_tbl_scope_parms free_parms; + struct tf_rm_free_parms fparms = { 0 }; + int dir; + int i; + struct hcapi_cfa_em_table *em_tables; + + TF_CHECK_PARMS2(tfp, parms); + + /* Retrieve the session information */ + rc = tf_session_get_session(tfp, &tfs); + if (rc) { + TFP_DRV_LOG(ERR, + "Failed to lookup session, rc:%s\n", + strerror(-rc)); + return rc; + } + + aparms.rm_db = eem_db[TF_DIR_RX]; + aparms.db_index = TF_EM_TBL_TYPE_TBL_SCOPE; + aparms.index = (uint32_t *)&parms->tbl_scope_id; + rc = tf_rm_allocate(&aparms); + if (rc) { + TFP_DRV_LOG(ERR, + "Failed to allocate table scope\n"); + return rc; + } + + tbl_scope_cb = &tbl_scopes[parms->tbl_scope_id]; + tbl_scope_cb->index = parms->tbl_scope_id; + tbl_scope_cb->tbl_scope_id = parms->tbl_scope_id; + tbl_scope_cb->bus = tfs->session_id.internal.bus; + tbl_scope_cb->devfn = tfs->session_id.internal.device; + + for (dir = 0; dir < TF_DIR_MAX; dir++) { + rc = tf_msg_em_qcaps(tfp, + dir, + &tbl_scope_cb->em_caps[dir]); + if (rc) { + TFP_DRV_LOG(ERR, + "EEM: Unable to query for EEM capability," + " rc:%s\n", + strerror(-rc)); + goto cleanup; + } + } + + /* + * Validate and setup table sizes + */ + if (tf_em_validate_num_entries(tbl_scope_cb, parms)) + goto cleanup; + + rc = tf_dmabuf_alloc(tfp, tbl_scope_cb); + if (rc) { + TFP_DRV_LOG(ERR, + "System DMA buff alloc failed\n"); + return -EIO; + } + + for (dir = 0; dir < TF_DIR_MAX; dir++) { + for (i = TF_KEY0_TABLE; i < TF_MAX_TABLE; i++) { + if (i == TF_EFC_TABLE) + continue; + + em_tables = + &tbl_scope_cb->em_ctx_info[dir].em_tables[i]; + + rc = tf_em_size_table(em_tables, TF_EM_PAGE_SIZE); + if (rc) { + TFP_DRV_LOG(ERR, "Size table failed\n"); + goto cleanup; + } + } + + em_tables = tbl_scope_cb->em_ctx_info[dir].em_tables; + rc = tf_create_tbl_pool_external(dir, + tbl_scope_cb, + em_tables[TF_RECORD_TABLE].num_entries, + em_tables[TF_RECORD_TABLE].entry_size); + + if (rc) { + TFP_DRV_LOG(ERR, + "%s TBL: Unable to allocate idx pools %s\n", + tf_dir_2_str(dir), + strerror(-rc)); + goto cleanup_full; + } + } + + rc = offload_system_mmap(tbl_scope_cb); + + if (rc) { + TFP_DRV_LOG(ERR, + "System alloc mmap failed\n"); + goto cleanup_full; + } + + return rc; + +cleanup_full: + free_parms.tbl_scope_id = parms->tbl_scope_id; + tf_em_ext_free(tfp, &free_parms); + return -EINVAL; + +cleanup: + /* Free Table control block */ + fparms.rm_db = eem_db[TF_DIR_RX]; + fparms.db_index = TF_EM_TBL_TYPE_TBL_SCOPE; + fparms.index = parms->tbl_scope_id; + tf_rm_free(&fparms); + return -EINVAL; +} + +int +tf_em_ext_free(struct tf *tfp, + struct tf_free_tbl_scope_parms *parms) +{ + int rc; + struct tf_session *tfs; + struct tf_tbl_scope_cb *tbl_scope_cb; + int dir; + struct tf_rm_free_parms aparms = { 0 }; + + TF_CHECK_PARMS2(tfp, parms); + + /* Retrieve the session information */ + rc = tf_session_get_session(tfp, &tfs); + if (rc) { + TFP_DRV_LOG(ERR, + "Failed to lookup session, rc:%s\n", + strerror(-rc)); + return rc; + } + + tbl_scope_cb = &tbl_scopes[parms->tbl_scope_id]; + + /* Free Table control block */ + aparms.rm_db = eem_db[TF_DIR_RX]; + aparms.db_index = TF_EM_TBL_TYPE_TBL_SCOPE; + aparms.index = parms->tbl_scope_id; + rc = tf_rm_free(&aparms); + if (rc) { + TFP_DRV_LOG(ERR, + "Failed to free table scope\n"); + } + + for (dir = 0; dir < TF_DIR_MAX; dir++) { + /* Free associated external pools + */ + tf_destroy_tbl_pool_external(dir, + tbl_scope_cb); + + /* Unmap memory */ + tf_em_ctx_unreg(tbl_scope_cb, dir); + + tf_msg_em_op(tfp, + dir, + HWRM_TF_EXT_EM_OP_INPUT_OP_EXT_EM_DISABLE); + } + + tf_dmabuf_free(tfp, tbl_scope_cb); + + return rc; +} diff --git a/drivers/net/bnxt/tf_core/tf_if_tbl.h b/drivers/net/bnxt/tf_core/tf_if_tbl.h index 54d4c37f5..7eb72bd42 100644 --- a/drivers/net/bnxt/tf_core/tf_if_tbl.h +++ b/drivers/net/bnxt/tf_core/tf_if_tbl.h @@ -113,7 +113,7 @@ struct tf_if_tbl_set_parms { /** * [in] Entry data */ - uint32_t *data; + uint8_t *data; /** * [in] Entry size */ @@ -143,7 +143,7 @@ struct tf_if_tbl_get_parms { /** * [out] Entry data */ - uint32_t *data; + uint8_t *data; /** * [out] Entry size */ diff --git a/drivers/net/bnxt/tf_core/tf_msg.c b/drivers/net/bnxt/tf_core/tf_msg.c index 035c0948d..ed506defa 100644 --- a/drivers/net/bnxt/tf_core/tf_msg.c +++ b/drivers/net/bnxt/tf_core/tf_msg.c @@ -813,7 +813,19 @@ tf_msg_tcam_entry_set(struct tf *tfp, struct tf_msg_dma_buf buf = { 0 }; uint8_t *data = NULL; int data_size = 0; + uint8_t fw_session_id; + rc = tf_session_get_fw_session_id(tfp, &fw_session_id); + if (rc) { + TFP_DRV_LOG(ERR, + "%s: Unable to lookup FW id, rc:%s\n", + tf_dir_2_str(parms->dir), + strerror(-rc)); + return rc; + } + + /* Populate the request */ + req.fw_session_id = tfp_cpu_to_le_32(fw_session_id); req.type = parms->hcapi_type; req.idx = tfp_cpu_to_le_16(parms->idx); if (parms->dir == TF_DIR_TX) @@ -869,7 +881,19 @@ tf_msg_tcam_entry_free(struct tf *tfp, struct hwrm_tf_tcam_free_input req = { 0 }; struct hwrm_tf_tcam_free_output resp = { 0 }; struct tfp_send_msg_parms parms = { 0 }; + uint8_t fw_session_id; + rc = tf_session_get_fw_session_id(tfp, &fw_session_id); + if (rc) { + TFP_DRV_LOG(ERR, + "%s: Unable to lookup FW id, rc:%s\n", + tf_dir_2_str(in_parms->dir), + strerror(-rc)); + return rc; + } + + /* Populate the request */ + req.fw_session_id = tfp_cpu_to_le_32(fw_session_id); req.type = in_parms->hcapi_type; req.count = 1; req.idx_list[0] = tfp_cpu_to_le_16(in_parms->idx); diff --git a/drivers/net/bnxt/tf_core/tf_tbl.h b/drivers/net/bnxt/tf_core/tf_tbl.h index 2a10b47ce..f20e8d729 100644 --- a/drivers/net/bnxt/tf_core/tf_tbl.h +++ b/drivers/net/bnxt/tf_core/tf_tbl.h @@ -38,6 +38,13 @@ struct tf_em_caps { */ struct tf_tbl_scope_cb { uint32_t tbl_scope_id; +#ifdef TF_USE_SYSTEM_MEM + int lfc_fd; + uint32_t bus; + uint32_t devfn; + int fd[TF_DIR_MAX][TF_MAX_TABLE]; + bool valid; +#endif int index; struct hcapi_cfa_em_ctx_mem_info em_ctx_info[TF_DIR_MAX]; struct tf_em_caps em_caps[TF_DIR_MAX]; diff --git a/drivers/net/bnxt/tf_core/tfp.c b/drivers/net/bnxt/tf_core/tfp.c index 426a182a9..3eade3127 100644 --- a/drivers/net/bnxt/tf_core/tfp.c +++ b/drivers/net/bnxt/tf_core/tfp.c @@ -87,6 +87,18 @@ tfp_send_msg_tunneled(struct tf *tfp, return rc; } +#ifdef TF_USE_SYSTEM_MEM +int +tfp_msg_hwrm_oem_cmd(struct tf *tfp, + uint32_t max_flows) +{ + return bnxt_hwrm_oem_cmd(container_of(tfp, + struct bnxt, + tfp), + max_flows); +} +#endif /* TF_USE_SYSTEM_MEM */ + /** * Allocates zero'ed memory from the heap. * diff --git a/drivers/net/bnxt/tf_core/tfp.h b/drivers/net/bnxt/tf_core/tfp.h index 8789eba1f..421a7d9f7 100644 --- a/drivers/net/bnxt/tf_core/tfp.h +++ b/drivers/net/bnxt/tf_core/tfp.h @@ -170,6 +170,21 @@ int tfp_msg_hwrm_oem_cmd(struct tf *tfp, uint32_t max_flows); +/** + * Sends OEM command message to Chimp + * + * [in] session, pointer to session handle + * [in] max_flows, max number of flows requested + * + * Returns: + * 0 - Success + * -1 - Global error like not supported + * -EINVAL - Parameter Error + */ +int +tfp_msg_hwrm_oem_cmd(struct tf *tfp, + uint32_t max_flows); + /** * Allocates zero'ed memory from the heap. *