From patchwork Fri Jun 29 18:12:18 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Rahul Lakkireddy X-Patchwork-Id: 41990 X-Patchwork-Delegate: ferruh.yigit@amd.com Return-Path: X-Original-To: patchwork@dpdk.org Delivered-To: patchwork@dpdk.org Received: from [92.243.14.124] (localhost [127.0.0.1]) by dpdk.org (Postfix) with ESMTP id C4A8B1B91D; Fri, 29 Jun 2018 20:13:42 +0200 (CEST) Received: from stargate.chelsio.com (stargate.chelsio.com [12.32.117.8]) by dpdk.org (Postfix) with ESMTP id B03181B91D for ; Fri, 29 Jun 2018 20:13:40 +0200 (CEST) Received: from localhost (scalar.blr.asicdesigners.com [10.193.185.94]) by stargate.chelsio.com (8.13.8/8.13.8) with ESMTP id w5TIDZ95028620; Fri, 29 Jun 2018 11:13:36 -0700 From: Rahul Lakkireddy To: dev@dpdk.org Cc: shaguna@chelsio.com, indranil@chelsio.com, nirranjan@chelsio.com Date: Fri, 29 Jun 2018 23:42:18 +0530 Message-Id: X-Mailer: git-send-email 2.5.3 In-Reply-To: References: In-Reply-To: References: Subject: [dpdk-dev] [PATCH 3/9] net/cxgbe: add Compressed Local IP region X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Sender: "dev" From: Shagun Agrawal CLIP region holds destination IPv6 addresses to be matched for corresponding flows. Query firmware for CLIP resources and allocate table to manage them. Also update LE-TCAM to use CLIP to reduce number of slots needed to offload IPv6 flows. Signed-off-by: Shagun Agrawal Signed-off-by: Rahul Lakkireddy --- drivers/net/cxgbe/Makefile | 1 + drivers/net/cxgbe/base/adapter.h | 32 ++++++ drivers/net/cxgbe/base/t4fw_interface.h | 19 ++++ drivers/net/cxgbe/clip_tbl.c | 195 ++++++++++++++++++++++++++++++++ drivers/net/cxgbe/clip_tbl.h | 31 +++++ drivers/net/cxgbe/cxgbe_filter.c | 99 ++++++++++++---- drivers/net/cxgbe/cxgbe_filter.h | 1 + drivers/net/cxgbe/cxgbe_main.c | 19 ++++ 8 files changed, 377 insertions(+), 20 deletions(-) create mode 100644 drivers/net/cxgbe/clip_tbl.c create mode 100644 drivers/net/cxgbe/clip_tbl.h diff --git a/drivers/net/cxgbe/Makefile b/drivers/net/cxgbe/Makefile index edc5d8188..5d66c4b3a 100644 --- a/drivers/net/cxgbe/Makefile +++ b/drivers/net/cxgbe/Makefile @@ -52,6 +52,7 @@ SRCS-$(CONFIG_RTE_LIBRTE_CXGBE_PMD) += sge.c SRCS-$(CONFIG_RTE_LIBRTE_CXGBE_PMD) += cxgbe_filter.c SRCS-$(CONFIG_RTE_LIBRTE_CXGBE_PMD) += cxgbe_flow.c SRCS-$(CONFIG_RTE_LIBRTE_CXGBE_PMD) += t4_hw.c +SRCS-$(CONFIG_RTE_LIBRTE_CXGBE_PMD) += clip_tbl.c SRCS-$(CONFIG_RTE_LIBRTE_CXGBE_PMD) += t4vf_hw.c include $(RTE_SDK)/mk/rte.lib.mk diff --git a/drivers/net/cxgbe/base/adapter.h b/drivers/net/cxgbe/base/adapter.h index de46ecfe3..3ed3252e8 100644 --- a/drivers/net/cxgbe/base/adapter.h +++ b/drivers/net/cxgbe/base/adapter.h @@ -11,6 +11,7 @@ #include #include #include +#include #include #include "cxgbe_compat.h" @@ -321,9 +322,40 @@ struct adapter { int use_unpacked_mode; /* unpacked rx mode state */ rte_spinlock_t win0_lock; + unsigned int clipt_start; /* CLIP table start */ + unsigned int clipt_end; /* CLIP table end */ + struct clip_tbl *clipt; /* CLIP table */ + struct tid_info tids; /* Info used to access TID related tables */ }; +/** + * t4_os_rwlock_init - initialize rwlock + * @lock: the rwlock + */ +static inline void t4_os_rwlock_init(rte_rwlock_t *lock) +{ + rte_rwlock_init(lock); +} + +/** + * t4_os_write_lock - get a write lock + * @lock: the rwlock + */ +static inline void t4_os_write_lock(rte_rwlock_t *lock) +{ + rte_rwlock_write_lock(lock); +} + +/** + * t4_os_write_unlock - unlock a write lock + * @lock: the rwlock + */ +static inline void t4_os_write_unlock(rte_rwlock_t *lock) +{ + rte_rwlock_write_unlock(lock); +} + /** * ethdev2pinfo - return the port_info structure associated with a rte_eth_dev * @dev: the rte_eth_dev diff --git a/drivers/net/cxgbe/base/t4fw_interface.h b/drivers/net/cxgbe/base/t4fw_interface.h index 842aa1263..2433bf20c 100644 --- a/drivers/net/cxgbe/base/t4fw_interface.h +++ b/drivers/net/cxgbe/base/t4fw_interface.h @@ -333,6 +333,7 @@ enum fw_cmd_opcodes { FW_RSS_IND_TBL_CMD = 0x20, FW_RSS_GLB_CONFIG_CMD = 0x22, FW_RSS_VI_CONFIG_CMD = 0x23, + FW_CLIP_CMD = 0x28, FW_DEBUG_CMD = 0x81, }; @@ -648,6 +649,8 @@ enum fw_params_param_dev { * physical and virtual function parameters */ enum fw_params_param_pfvf { + FW_PARAMS_PARAM_PFVF_CLIP_START = 0x03, + FW_PARAMS_PARAM_PFVF_CLIP_END = 0x04, FW_PARAMS_PARAM_PFVF_FILTER_START = 0x05, FW_PARAMS_PARAM_PFVF_FILTER_END = 0x06, FW_PARAMS_PARAM_PFVF_CPLFW4MSG_ENCAP = 0x31, @@ -2167,6 +2170,22 @@ struct fw_rss_vi_config_cmd { (((x) >> S_FW_RSS_VI_CONFIG_CMD_UDPEN) & M_FW_RSS_VI_CONFIG_CMD_UDPEN) #define F_FW_RSS_VI_CONFIG_CMD_UDPEN V_FW_RSS_VI_CONFIG_CMD_UDPEN(1U) +struct fw_clip_cmd { + __be32 op_to_write; + __be32 alloc_to_len16; + __be64 ip_hi; + __be64 ip_lo; + __be32 r4[2]; +}; + +#define S_FW_CLIP_CMD_ALLOC 31 +#define V_FW_CLIP_CMD_ALLOC(x) ((x) << S_FW_CLIP_CMD_ALLOC) +#define F_FW_CLIP_CMD_ALLOC V_FW_CLIP_CMD_ALLOC(1U) + +#define S_FW_CLIP_CMD_FREE 30 +#define V_FW_CLIP_CMD_FREE(x) ((x) << S_FW_CLIP_CMD_FREE) +#define F_FW_CLIP_CMD_FREE V_FW_CLIP_CMD_FREE(1U) + /****************************************************************************** * D E B U G C O M M A N D s ******************************************************/ diff --git a/drivers/net/cxgbe/clip_tbl.c b/drivers/net/cxgbe/clip_tbl.c new file mode 100644 index 000000000..fa5281cd4 --- /dev/null +++ b/drivers/net/cxgbe/clip_tbl.c @@ -0,0 +1,195 @@ +/* SPDX-License-Identifier: BSD-3-Clause + * Copyright(c) 2018 Chelsio Communications. + * All rights reserved. + */ + +#include "common.h" +#include "clip_tbl.h" + +/** + * Allocate clip entry in HW with associated IPV4/IPv6 address + */ +static int clip6_get_mbox(const struct rte_eth_dev *dev, const u32 *lip) +{ + struct adapter *adap = ethdev2adap(dev); + struct fw_clip_cmd c; + u64 hi = ((u64)lip[1]) << 32 | lip[0]; + u64 lo = ((u64)lip[3]) << 32 | lip[2]; + + memset(&c, 0, sizeof(c)); + c.op_to_write = cpu_to_be32(V_FW_CMD_OP(FW_CLIP_CMD) | + F_FW_CMD_REQUEST | F_FW_CMD_WRITE); + c.alloc_to_len16 = cpu_to_be32(F_FW_CLIP_CMD_ALLOC | FW_LEN16(c)); + c.ip_hi = hi; + c.ip_lo = lo; + return t4_wr_mbox_meat(adap, adap->mbox, &c, sizeof(c), &c, false); +} + +/** + * Delete clip entry in HW having the associated IPV4/IPV6 address + */ +static int clip6_release_mbox(const struct rte_eth_dev *dev, const u32 *lip) +{ + struct adapter *adap = ethdev2adap(dev); + struct fw_clip_cmd c; + u64 hi = ((u64)lip[1]) << 32 | lip[0]; + u64 lo = ((u64)lip[3]) << 32 | lip[2]; + + memset(&c, 0, sizeof(c)); + c.op_to_write = cpu_to_be32(V_FW_CMD_OP(FW_CLIP_CMD) | + F_FW_CMD_REQUEST | F_FW_CMD_READ); + c.alloc_to_len16 = cpu_to_be32(F_FW_CLIP_CMD_FREE | FW_LEN16(c)); + c.ip_hi = hi; + c.ip_lo = lo; + return t4_wr_mbox_meat(adap, adap->mbox, &c, sizeof(c), &c, false); +} + +/** + * cxgbe_clip_release - Release associated CLIP entry + * @ce: clip entry to release + * + * Releases ref count and frees up a clip entry from CLIP table + */ +void cxgbe_clip_release(struct rte_eth_dev *dev, struct clip_entry *ce) +{ + int ret; + + t4_os_lock(&ce->lock); + if (rte_atomic32_dec_and_test(&ce->refcnt)) { + ret = clip6_release_mbox(dev, ce->addr); + if (ret) + dev_debug(adap, "CLIP FW DEL CMD failed: %d", ret); + } + t4_os_unlock(&ce->lock); +} + +/** + * find_or_alloc_clipe - Find/Allocate a free CLIP entry + * @c: CLIP table + * @lip: IPV4/IPV6 address to compare/add + * Returns pointer to the IPV4/IPV6 entry found/created + * + * Finds/Allocates an CLIP entry to be used for a filter rule. + */ +static struct clip_entry *find_or_alloc_clipe(struct clip_tbl *c, + const u32 *lip) +{ + struct clip_entry *end, *e; + struct clip_entry *first_free = NULL; + unsigned int clipt_size = c->clipt_size; + + for (e = &c->cl_list[0], end = &c->cl_list[clipt_size]; e != end; ++e) { + if (rte_atomic32_read(&e->refcnt) == 0) { + if (!first_free) + first_free = e; + } else { + if (memcmp(lip, e->addr, sizeof(e->addr)) == 0) + goto exists; + } + } + + if (first_free) { + e = first_free; + goto exists; + } + + return NULL; + +exists: + return e; +} + +static struct clip_entry *t4_clip_alloc(struct rte_eth_dev *dev, + u32 *lip, u8 v6) +{ + struct adapter *adap = ethdev2adap(dev); + struct clip_tbl *ctbl = adap->clipt; + struct clip_entry *ce; + int ret; + + if (!ctbl) + return NULL; + + t4_os_write_lock(&ctbl->lock); + ce = find_or_alloc_clipe(ctbl, lip); + if (ce) { + t4_os_lock(&ce->lock); + if (!rte_atomic32_read(&ce->refcnt)) { + rte_memcpy(ce->addr, lip, sizeof(ce->addr)); + if (v6) { + ce->type = FILTER_TYPE_IPV6; + rte_atomic32_set(&ce->refcnt, 1); + ret = clip6_get_mbox(dev, lip); + if (ret) { + dev_debug(adap, + "CLIP FW ADD CMD failed: %d", + ret); + ce = NULL; + } + } else { + ce->type = FILTER_TYPE_IPV4; + } + } else { + rte_atomic32_inc(&ce->refcnt); + } + t4_os_unlock(&ce->lock); + } + t4_os_write_unlock(&ctbl->lock); + + return ce; +} + +/** + * cxgbe_clip_alloc - Allocate a IPV6 CLIP entry + * @dev: rte_eth_dev pointer + * @lip: IPV6 address to add + * Returns pointer to the CLIP entry created + * + * Allocates a IPV6 CLIP entry to be used for a filter rule. + */ +struct clip_entry *cxgbe_clip_alloc(struct rte_eth_dev *dev, u32 *lip) +{ + return t4_clip_alloc(dev, lip, FILTER_TYPE_IPV6); +} + +/** + * Initialize CLIP Table + */ +struct clip_tbl *t4_init_clip_tbl(unsigned int clipt_start, + unsigned int clipt_end) +{ + unsigned int clipt_size; + struct clip_tbl *ctbl; + unsigned int i; + + if (clipt_start >= clipt_end) + return NULL; + + clipt_size = clipt_end - clipt_start + 1; + + ctbl = t4_os_alloc(sizeof(*ctbl) + + clipt_size * sizeof(struct clip_entry)); + if (!ctbl) + return NULL; + + ctbl->clipt_start = clipt_start; + ctbl->clipt_size = clipt_size; + + t4_os_rwlock_init(&ctbl->lock); + + for (i = 0; i < ctbl->clipt_size; i++) { + t4_os_lock_init(&ctbl->cl_list[i].lock); + rte_atomic32_set(&ctbl->cl_list[i].refcnt, 0); + } + + return ctbl; +} + +/** + * Cleanup CLIP Table + */ +void t4_cleanup_clip_tbl(struct adapter *adap) +{ + if (adap->clipt) + t4_os_free(adap->clipt); +} diff --git a/drivers/net/cxgbe/clip_tbl.h b/drivers/net/cxgbe/clip_tbl.h new file mode 100644 index 000000000..737ccc691 --- /dev/null +++ b/drivers/net/cxgbe/clip_tbl.h @@ -0,0 +1,31 @@ +/* SPDX-License-Identifier: BSD-3-Clause + * Copyright(c) 2018 Chelsio Communications. + * All rights reserved. + */ + +#ifndef _CXGBE_CLIP_H_ +#define _CXGBE_CLIP_H_ + +/* + * State for the corresponding entry of the HW CLIP table. + */ +struct clip_entry { + enum filter_type type; /* entry type */ + u32 addr[4]; /* IPV4 or IPV6 address */ + rte_spinlock_t lock; /* entry lock */ + rte_atomic32_t refcnt; /* entry reference count */ +}; + +struct clip_tbl { + unsigned int clipt_start; /* start index of CLIP table */ + unsigned int clipt_size; /* size of CLIP table */ + rte_rwlock_t lock; /* table rw lock */ + struct clip_entry cl_list[0]; /* MUST BE LAST */ +}; + +struct clip_tbl *t4_init_clip_tbl(unsigned int clipt_start, + unsigned int clipt_end); +void t4_cleanup_clip_tbl(struct adapter *adap); +struct clip_entry *cxgbe_clip_alloc(struct rte_eth_dev *dev, u32 *lip); +void cxgbe_clip_release(struct rte_eth_dev *dev, struct clip_entry *ce); +#endif /* _CXGBE_CLIP_H_ */ diff --git a/drivers/net/cxgbe/cxgbe_filter.c b/drivers/net/cxgbe/cxgbe_filter.c index a5d20d164..bb2ebaa62 100644 --- a/drivers/net/cxgbe/cxgbe_filter.c +++ b/drivers/net/cxgbe/cxgbe_filter.c @@ -6,6 +6,7 @@ #include "common.h" #include "t4_regs.h" #include "cxgbe_filter.h" +#include "clip_tbl.h" /** * Initialize Hash Filters @@ -164,6 +165,9 @@ int cxgbe_alloc_ftid(struct adapter *adap, unsigned int family) */ void clear_filter(struct filter_entry *f) { + if (f->clipt) + cxgbe_clip_release(f->dev, f->clipt); + /* * The zeroing of the filter rule below clears the filter valid, * pending, locked flags etc. so it's all we need for @@ -349,11 +353,14 @@ int cxgbe_del_filter(struct rte_eth_dev *dev, unsigned int filter_id, struct port_info *pi = (struct port_info *)(dev->data->dev_private); struct adapter *adapter = pi->adapter; struct filter_entry *f; + unsigned int chip_ver; int ret; if (filter_id >= adapter->tids.nftids) return -ERANGE; + chip_ver = CHELSIO_CHIP_VERSION(adapter->params.chip); + ret = is_filter_set(&adapter->tids, filter_id, fs->type); if (!ret) { dev_warn(adap, "%s: could not find filter entry: %u\n", @@ -361,6 +368,17 @@ int cxgbe_del_filter(struct rte_eth_dev *dev, unsigned int filter_id, return -EINVAL; } + /* + * Ensure filter id is aligned on the 2 slot boundary for T6, + * and 4 slot boundary for cards below T6. + */ + if (fs->type) { + if (chip_ver < CHELSIO_T6) + filter_id &= ~(0x3); + else + filter_id &= ~(0x1); + } + f = &adapter->tids.ftid_tab[filter_id]; ret = writable_filter(f); if (ret) @@ -403,11 +421,15 @@ int cxgbe_set_filter(struct rte_eth_dev *dev, unsigned int filter_id, struct adapter *adapter = pi->adapter; unsigned int fidx, iq, fid_bit = 0; struct filter_entry *f; + unsigned int chip_ver; + uint8_t bitoff[16] = {0}; int ret; if (filter_id >= adapter->tids.nftids) return -ERANGE; + chip_ver = CHELSIO_CHIP_VERSION(adapter->params.chip); + ret = validate_filter(adapter, fs); if (ret) return ret; @@ -426,38 +448,61 @@ int cxgbe_set_filter(struct rte_eth_dev *dev, unsigned int filter_id, iq = get_filter_steerq(dev, fs); /* - * IPv6 filters occupy four slots and must be aligned on - * four-slot boundaries. IPv4 filters only occupy a single - * slot and have no alignment requirements but writing a new - * IPv4 filter into the middle of an existing IPv6 filter - * requires clearing the old IPv6 filter. + * IPv6 filters occupy four slots and must be aligned on four-slot + * boundaries for T5. On T6, IPv6 filters occupy two-slots and + * must be aligned on two-slot boundaries. + * + * IPv4 filters only occupy a single slot and have no alignment + * requirements but writing a new IPv4 filter into the middle + * of an existing IPv6 filter requires clearing the old IPv6 + * filter. */ if (fs->type == FILTER_TYPE_IPV4) { /* IPv4 */ /* - * If our IPv4 filter isn't being written to a - * multiple of four filter index and there's an IPv6 - * filter at the multiple of 4 base slot, then we need + * For T6, If our IPv4 filter isn't being written to a + * multiple of two filter index and there's an IPv6 + * filter at the multiple of 2 base slot, then we need * to delete that IPv6 filter ... + * For adapters below T6, IPv6 filter occupies 4 entries. */ - fidx = filter_id & ~0x3; + if (chip_ver < CHELSIO_T6) + fidx = filter_id & ~0x3; + else + fidx = filter_id & ~0x1; + if (fidx != filter_id && adapter->tids.ftid_tab[fidx].fs.type) { f = &adapter->tids.ftid_tab[fidx]; if (f->valid) return -EBUSY; } } else { /* IPv6 */ - /* - * Ensure that the IPv6 filter is aligned on a - * multiple of 4 boundary. - */ - if (filter_id & 0x3) - return -EINVAL; + unsigned int max_filter_id; + + if (chip_ver < CHELSIO_T6) { + /* + * Ensure that the IPv6 filter is aligned on a + * multiple of 4 boundary. + */ + if (filter_id & 0x3) + return -EINVAL; + + max_filter_id = filter_id + 4; + } else { + /* + * For T6, CLIP being enabled, IPv6 filter would occupy + * 2 entries. + */ + if (filter_id & 0x1) + return -EINVAL; + + max_filter_id = filter_id + 2; + } /* * Check all except the base overlapping IPv4 filter * slots. */ - for (fidx = filter_id + 1; fidx < filter_id + 4; fidx++) { + for (fidx = filter_id + 1; fidx < max_filter_id; fidx++) { f = &adapter->tids.ftid_tab[fidx]; if (f->valid) return -EBUSY; @@ -491,6 +536,16 @@ int cxgbe_set_filter(struct rte_eth_dev *dev, unsigned int filter_id, return ret; } + /* + * Allocate a clip table entry only if we have non-zero IPv6 address + */ + if (chip_ver > CHELSIO_T5 && fs->type && + memcmp(fs->val.lip, bitoff, sizeof(bitoff))) { + f->clipt = cxgbe_clip_alloc(f->dev, (u32 *)&f->fs.val.lip); + if (!f->clipt) + goto free_tid; + } + /* * Convert the filter specification into our internal format. * We copy the PF/VF specification into the Outer VLAN field @@ -510,13 +565,17 @@ int cxgbe_set_filter(struct rte_eth_dev *dev, unsigned int filter_id, ret = set_filter_wr(dev, filter_id); if (ret) { fid_bit = f->tid - adapter->tids.ftid_base; - cxgbe_clear_ftid(&adapter->tids, fid_bit, - fs->type ? FILTER_TYPE_IPV6 : - FILTER_TYPE_IPV4); - clear_filter(f); + goto free_tid; } return ret; + +free_tid: + cxgbe_clear_ftid(&adapter->tids, fid_bit, + fs->type ? FILTER_TYPE_IPV6 : + FILTER_TYPE_IPV4); + clear_filter(f); + return ret; } /** diff --git a/drivers/net/cxgbe/cxgbe_filter.h b/drivers/net/cxgbe/cxgbe_filter.h index 27421a475..ce115f69f 100644 --- a/drivers/net/cxgbe/cxgbe_filter.h +++ b/drivers/net/cxgbe/cxgbe_filter.h @@ -141,6 +141,7 @@ struct filter_entry { u32 locked:1; /* filter is administratively locked */ u32 pending:1; /* filter action is pending FW reply */ struct filter_ctx *ctx; /* caller's completion hook */ + struct clip_entry *clipt; /* CLIP Table entry for IPv6 */ struct rte_eth_dev *dev; /* Port's rte eth device */ void *private; /* For use by apps using filter_entry */ diff --git a/drivers/net/cxgbe/cxgbe_main.c b/drivers/net/cxgbe/cxgbe_main.c index c692939db..2050fe4db 100644 --- a/drivers/net/cxgbe/cxgbe_main.c +++ b/drivers/net/cxgbe/cxgbe_main.c @@ -37,6 +37,7 @@ #include "t4_regs.h" #include "t4_msg.h" #include "cxgbe.h" +#include "clip_tbl.h" /** * Allocate a chunk of memory. The allocated memory is cleared. @@ -995,6 +996,14 @@ static int adap_init0(struct adapter *adap) adap->tids.ftid_base = val[0]; adap->tids.nftids = val[1] - val[0] + 1; + params[0] = FW_PARAM_PFVF(CLIP_START); + params[1] = FW_PARAM_PFVF(CLIP_END); + ret = t4_query_params(adap, adap->mbox, adap->pf, 0, 2, params, val); + if (ret < 0) + goto bye; + adap->clipt_start = val[0]; + adap->clipt_end = val[1]; + /* * Get device capabilities so we can determine what resources we need * to manage. @@ -1509,6 +1518,7 @@ void cxgbe_close(struct adapter *adapter) if (is_pf4(adapter)) t4_intr_disable(adapter); tid_free(&adapter->tids); + t4_cleanup_clip_tbl(adapter); t4_sge_tx_monitor_stop(adapter); t4_free_sge_resources(adapter); for_each_port(adapter, i) { @@ -1672,6 +1682,15 @@ int cxgbe_probe(struct adapter *adapter) print_adapter_info(adapter); print_port_info(adapter); + adapter->clipt = t4_init_clip_tbl(adapter->clipt_start, + adapter->clipt_end); + if (!adapter->clipt) { + /* We tolerate a lack of clip_table, giving up some + * functionality + */ + dev_warn(adapter, "could not allocate CLIP. Continuing\n"); + } + if (tid_init(&adapter->tids) < 0) { /* Disable filtering support */ dev_warn(adapter, "could not allocate TID table, "