From patchwork Thu Oct 11 13:04:33 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Moti Haimovsky X-Patchwork-Id: 46618 X-Patchwork-Delegate: shahafs@mellanox.com Return-Path: X-Original-To: patchwork@dpdk.org Delivered-To: patchwork@dpdk.org Received: from [92.243.14.124] (localhost [127.0.0.1]) by dpdk.org (Postfix) with ESMTP id 9DE391B551; Thu, 11 Oct 2018 15:04:37 +0200 (CEST) Received: from EUR03-AM5-obe.outbound.protection.outlook.com (mail-eopbgr30079.outbound.protection.outlook.com [40.107.3.79]) by dpdk.org (Postfix) with ESMTP id 025671B4C5 for ; Thu, 11 Oct 2018 15:04:35 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=Mellanox.com; s=selector1; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=Ujn4ayvSbx1EzpM634tYpwYJVZgOKPjPoz8c9wITSOE=; b=dGE8JWjIzwwgxwJUysYtj1W3zTU5BLQUG7ta57hw93ZoSpcJGkwlh1kVAITonk6cmIjNboJQr/9WqxBZwznLiD5kkG8D6L/1EHaMnHZd5BoEQ8PZ4abwZ1yMMHjRQLbTPaFq+pbj2ayq1Vu8v0cozpq9Ce2tnNY+c6u4oph1AYI= Received: from AM0PR05MB4435.eurprd05.prod.outlook.com (52.134.92.20) by AM0PR05MB4898.eurprd05.prod.outlook.com (20.177.41.27) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.1207.28; Thu, 11 Oct 2018 13:04:34 +0000 Received: from AM0PR05MB4435.eurprd05.prod.outlook.com ([fe80::2549:a872:4864:b84e]) by AM0PR05MB4435.eurprd05.prod.outlook.com ([fe80::2549:a872:4864:b84e%6]) with mapi id 15.20.1207.029; Thu, 11 Oct 2018 13:04:34 +0000 From: Mordechay Haimovsky To: Shahaf Shuler CC: "dev@dpdk.org" , Mordechay Haimovsky Thread-Topic: [PATCH v2 1/2] net/mlx5: refactor TC-flow infrastructure Thread-Index: AQHUYWL0Utxkj9DMqUCkCsdPOlLAdA== Date: Thu, 11 Oct 2018 13:04:33 +0000 Message-ID: <1539263057-16678-2-git-send-email-motih@mellanox.com> References: <1539263057-16678-1-git-send-email-motih@mellanox.com> In-Reply-To: <1539263057-16678-1-git-send-email-motih@mellanox.com> Accept-Language: he-IL, en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: x-clientproxiedby: HE1PR05CA0223.eurprd05.prod.outlook.com (2603:10a6:3:fa::23) To AM0PR05MB4435.eurprd05.prod.outlook.com (2603:10a6:208:5a::20) authentication-results: spf=none (sender IP is ) smtp.mailfrom=motih@mellanox.com; x-ms-exchange-messagesentrepresentingtype: 1 x-originating-ip: [37.142.13.130] x-ms-publictraffictype: Email x-microsoft-exchange-diagnostics: 1; AM0PR05MB4898; 6:oOE596gI5RlvanS36iyoO2zhV+2JKjG8zOhnDE93Z/jCbEQOWu7hp26B/fLQm86KmEbiHl+j6zzqQ23ffxdz+AzYGyYUZHzOgKCEto1/gQ4KEH7FsTVv0Ag2Vb2GyCwLL2uDYG10sbibasi3+OSy7dJULWU+aSu00BII+0hte/kq+OUWNEX5Ccmtq6apzwlQvSwjbLojYnTgk83g7P7RCHS/Nl5uzRZBC2ZW5755BzXxFw7mD645UESIct+AlG7mADmapVACROEqrzhwu6/akFk7klWZpfhCfV95Uez1vbDg7MYpiVvm2WHUTHZVSmM3fMvLfHvXlnnT29cvUwAekUmofT1JhjlcAeiIt4CEKzOVeIHAgxgIxPYgm1v+K0bPuHlWWSbic9WSEjp9Wg+2mUCBCwgtr0p5RshPUlgiWn2qHenR69zUHCKjBsWRQRR97LOIDlV5e7ANOb5CIc2f4w==; 5:LGjMYb7I5oyc13PuWA/XmRS8w4Y/RDCASu4GCMh9dZ0XhDBzPgkhLLSBHCJz1Ljekuo7CYrzK6cx3JlRlJR1fJVz+YBclJyWSopSPUQV3jSE6XIOfCP8AqzzqLjVUkxaFjMftKQzJmCb42L8jXV0j0yaeBlo+FIXlbC9iePXLBI=; 7:rWYbWoZjC5C3GIDRK4fg17du0Au98OxPshNCBOQeOXODTkB0EE3/vzip1X59iFUMaD3THdIh49sb9bKPJCINRxfyCRABZXlxl447Sfl6EuME83/tF2V5gFz/8PRtVZm32QdQwSBLv43nkBls3OGTlN5HWXG5oEUZxYFK21pulEABh9AAf6u+FYzKaiYcM7e+zidVxqr/a6fYsQ7ffIo28qZlusdbkH8I5VlPpScIXyruMWZ24ruc8fEFmONKh1Ux x-ms-office365-filtering-correlation-id: 1142f05a-7b3e-4182-537f-08d62f7a16e8 x-ms-office365-filtering-ht: Tenant x-microsoft-antispam: BCL:0; PCL:0; RULEID:(7020095)(4652040)(8989299)(4534185)(4627221)(201703031133081)(201702281549075)(8990200)(5600074)(711020)(4618075)(2017052603328)(7153060)(7193020); SRVR:AM0PR05MB4898; x-ms-traffictypediagnostic: AM0PR05MB4898: x-microsoft-antispam-prvs: x-exchange-antispam-report-test: UriScan:(269456686620040); x-ms-exchange-senderadcheck: 1 x-exchange-antispam-report-cfa-test: BCL:0; PCL:0; RULEID:(8211001083)(6040522)(2401047)(5005006)(8121501046)(3002001)(3231355)(944501410)(52105095)(10201501046)(93006095)(93001095)(6055026)(149066)(150057)(6041310)(20161123560045)(201703131423095)(201702281528075)(20161123555045)(201703061421075)(201703061406153)(20161123562045)(20161123558120)(20161123564045)(201708071742011)(7699051); SRVR:AM0PR05MB4898; BCL:0; PCL:0; RULEID:; SRVR:AM0PR05MB4898; x-forefront-prvs: 08220FA8D6 x-forefront-antispam-report: SFV:NSPM; SFS:(10009020)(376002)(136003)(346002)(39860400002)(396003)(366004)(199004)(189003)(71200400001)(71190400001)(105586002)(5250100002)(6862004)(4326008)(54906003)(2900100001)(99286004)(76176011)(52116002)(106356001)(6116002)(97736004)(7736002)(2906002)(25786009)(305945005)(107886003)(3846002)(37006003)(5660300001)(66066001)(6636002)(86362001)(256004)(14444005)(14454004)(8936002)(476003)(2616005)(11346002)(446003)(486006)(36756003)(68736007)(186003)(26005)(102836004)(8676002)(81166006)(81156014)(6436002)(316002)(53936002)(6512007)(386003)(6506007)(6486002)(478600001)(42262002); DIR:OUT; SFP:1101; SCL:1; SRVR:AM0PR05MB4898; H:AM0PR05MB4435.eurprd05.prod.outlook.com; FPR:; SPF:None; LANG:en; PTR:InfoNoRecords; A:1; MX:1; received-spf: None (protection.outlook.com: mellanox.com does not designate permitted sender hosts) x-microsoft-antispam-message-info: dUCV2yP4QeNTgzJZUQnrHd5rTJJ63CWg0A4cHA4HGg7B3LYnfefCOtEDyoAxAMv3Z/+caRroSLRABckLqgNfdV6dqK1CbldSftD4hqefSyrz/xY2izV3PnOheyg1eXNG1FOwvCDZQwo7nKDNPuSq+k+vkpiWiT/up8YpGORBnMOef3941jOQlGbwvxhyyW7h3sucspmzyxf+IAjPcc8zWh6MO93Grouov1WkquwpOahGhjxD7VpFXxKXuuXxZy/RGe5lCnCg0V9/GtPCm5qDPOVSZP4Q5sm3NMzYmMKQu/SVStma0ghZVffENZQHh90kY9AW1RsMeSJeBapmVFs29ZfuCuW+vE3zWwgeZi8sRC4= spamdiagnosticoutput: 1:99 spamdiagnosticmetadata: NSPM MIME-Version: 1.0 X-OriginatorOrg: Mellanox.com X-MS-Exchange-CrossTenant-Network-Message-Id: 1142f05a-7b3e-4182-537f-08d62f7a16e8 X-MS-Exchange-CrossTenant-originalarrivaltime: 11 Oct 2018 13:04:34.0016 (UTC) X-MS-Exchange-CrossTenant-fromentityheader: Hosted X-MS-Exchange-CrossTenant-id: a652971c-7d2e-4d9b-a6a4-d149256f461b X-MS-Exchange-Transport-CrossTenantHeadersStamped: AM0PR05MB4898 Subject: [dpdk-dev] [PATCH v2 1/2] net/mlx5: refactor TC-flow infrastructure X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Sender: "dev" This commit refactors tc_flow as a preparation to coming commits that sends different type of messages and expect differ type of replies while still using the same underlying routines. Signed-off-by: Moti Haimovsky --- drivers/net/mlx5/mlx5.c | 18 +++---- drivers/net/mlx5/mlx5.h | 4 +- drivers/net/mlx5/mlx5_flow.h | 8 +-- drivers/net/mlx5/mlx5_flow_tcf.c | 113 ++++++++++++++++++++++++++++++--------- 4 files changed, 102 insertions(+), 41 deletions(-) diff --git a/drivers/net/mlx5/mlx5.c b/drivers/net/mlx5/mlx5.c index 7425e2f..20adf88 100644 --- a/drivers/net/mlx5/mlx5.c +++ b/drivers/net/mlx5/mlx5.c @@ -286,8 +286,8 @@ close(priv->nl_socket_route); if (priv->nl_socket_rdma >= 0) close(priv->nl_socket_rdma); - if (priv->mnl_socket) - mlx5_flow_tcf_socket_destroy(priv->mnl_socket); + if (priv->tcf_context) + mlx5_flow_tcf_context_destroy(priv->tcf_context); ret = mlx5_hrxq_ibv_verify(dev); if (ret) DRV_LOG(WARNING, "port %u some hash Rx queue still remain", @@ -1137,8 +1137,8 @@ claim_zero(mlx5_mac_addr_add(eth_dev, &mac, 0, 0)); if (vf && config.vf_nl_en) mlx5_nl_mac_addr_sync(eth_dev); - priv->mnl_socket = mlx5_flow_tcf_socket_create(); - if (!priv->mnl_socket) { + priv->tcf_context = mlx5_flow_tcf_context_create(); + if (!priv->tcf_context) { err = -rte_errno; DRV_LOG(WARNING, "flow rules relying on switch offloads will not be" @@ -1153,7 +1153,7 @@ error.message = "cannot retrieve network interface index"; } else { - err = mlx5_flow_tcf_init(priv->mnl_socket, ifindex, + err = mlx5_flow_tcf_init(priv->tcf_context, ifindex, &error); } if (err) { @@ -1161,8 +1161,8 @@ "flow rules relying on switch offloads will" " not be supported: %s: %s", error.message, strerror(rte_errno)); - mlx5_flow_tcf_socket_destroy(priv->mnl_socket); - priv->mnl_socket = NULL; + mlx5_flow_tcf_context_destroy(priv->tcf_context); + priv->tcf_context = NULL; } } TAILQ_INIT(&priv->flows); @@ -1217,8 +1217,8 @@ close(priv->nl_socket_route); if (priv->nl_socket_rdma >= 0) close(priv->nl_socket_rdma); - if (priv->mnl_socket) - mlx5_flow_tcf_socket_destroy(priv->mnl_socket); + if (priv->tcf_context) + mlx5_flow_tcf_context_destroy(priv->tcf_context); if (own_domain_id) claim_zero(rte_eth_switch_domain_free(priv->domain_id)); rte_free(priv); diff --git a/drivers/net/mlx5/mlx5.h b/drivers/net/mlx5/mlx5.h index 2dec88a..d14239c 100644 --- a/drivers/net/mlx5/mlx5.h +++ b/drivers/net/mlx5/mlx5.h @@ -169,7 +169,7 @@ struct mlx5_drop { struct mlx5_rxq_ibv *rxq; /* Verbs Rx queue. */ }; -struct mnl_socket; +struct mlx5_flow_tcf_context; struct priv { LIST_ENTRY(priv) mem_event_cb; /* Called by memory event callback. */ @@ -236,7 +236,7 @@ struct priv { rte_spinlock_t uar_lock[MLX5_UAR_PAGE_NUM_MAX]; /* UAR same-page access control required in 32bit implementations. */ #endif - struct mnl_socket *mnl_socket; /* Libmnl socket. */ + struct mlx5_flow_tcf_context *tcf_context; /* TC flower context. */ }; #define PORT_ID(priv) ((priv)->dev_data->port_id) diff --git a/drivers/net/mlx5/mlx5_flow.h b/drivers/net/mlx5/mlx5_flow.h index fee05f0..41d55e8 100644 --- a/drivers/net/mlx5/mlx5_flow.h +++ b/drivers/net/mlx5/mlx5_flow.h @@ -338,9 +338,9 @@ int mlx5_flow_validate_item_vxlan_gpe(const struct rte_flow_item *item, /* mlx5_flow_tcf.c */ -int mlx5_flow_tcf_init(struct mnl_socket *nl, unsigned int ifindex, - struct rte_flow_error *error); -struct mnl_socket *mlx5_flow_tcf_socket_create(void); -void mlx5_flow_tcf_socket_destroy(struct mnl_socket *nl); +int mlx5_flow_tcf_init(struct mlx5_flow_tcf_context *ctx, + unsigned int ifindex, struct rte_flow_error *error); +struct mlx5_flow_tcf_context *mlx5_flow_tcf_context_create(void); +void mlx5_flow_tcf_context_destroy(struct mlx5_flow_tcf_context *ctx); #endif /* RTE_PMD_MLX5_FLOW_H_ */ diff --git a/drivers/net/mlx5/mlx5_flow_tcf.c b/drivers/net/mlx5/mlx5_flow_tcf.c index 91f6ef6..8535a15 100644 --- a/drivers/net/mlx5/mlx5_flow_tcf.c +++ b/drivers/net/mlx5/mlx5_flow_tcf.c @@ -153,6 +153,19 @@ struct tc_vlan { #define IPV6_ADDR_LEN 16 #endif +/** + * Structure for holding netlink context. + * Note the size of the message buffer which is MNL_SOCKET_BUFFER_SIZE. + * Using this (8KB) buffer size ensures that netlink messages will never be + * truncated. + */ +struct mlx5_flow_tcf_context { + struct mnl_socket *nl; /* NETLINK_ROUTE libmnl socket. */ + uint32_t seq; /* Message sequence number. */ + uint32_t buf_size; /* Message buffer size. */ + uint8_t *buf; /* Message buffer. */ +}; + /** Empty masks for known item types. */ static const union { struct rte_flow_item_port_id port_id; @@ -1429,8 +1442,8 @@ struct flow_tcf_ptoi { /** * Send Netlink message with acknowledgment. * - * @param nl - * Libmnl socket to use. + * @param ctx + * Flow context to use. * @param nlh * Message to send. This function always raises the NLM_F_ACK flag before * sending. @@ -1439,12 +1452,13 @@ struct flow_tcf_ptoi { * 0 on success, a negative errno value otherwise and rte_errno is set. */ static int -flow_tcf_nl_ack(struct mnl_socket *nl, struct nlmsghdr *nlh) +flow_tcf_nl_ack(struct mlx5_flow_tcf_context *ctx, struct nlmsghdr *nlh) { alignas(struct nlmsghdr) uint8_t ans[mnl_nlmsg_size(sizeof(struct nlmsgerr)) + nlh->nlmsg_len - sizeof(*nlh)]; - uint32_t seq = random(); + uint32_t seq = ctx->seq++; + struct mnl_socket *nl = ctx->nl; int ret; nlh->nlmsg_flags |= NLM_F_ACK; @@ -1479,7 +1493,7 @@ struct flow_tcf_ptoi { struct rte_flow_error *error) { struct priv *priv = dev->data->dev_private; - struct mnl_socket *nl = priv->mnl_socket; + struct mlx5_flow_tcf_context *nl = priv->tcf_context; struct mlx5_flow *dev_flow; struct nlmsghdr *nlh; @@ -1508,7 +1522,7 @@ struct flow_tcf_ptoi { flow_tcf_remove(struct rte_eth_dev *dev, struct rte_flow *flow) { struct priv *priv = dev->data->dev_private; - struct mnl_socket *nl = priv->mnl_socket; + struct mlx5_flow_tcf_context *nl = priv->tcf_context; struct mlx5_flow *dev_flow; struct nlmsghdr *nlh; @@ -1560,10 +1574,47 @@ struct flow_tcf_ptoi { }; /** - * Initialize ingress qdisc of a given network interface. + * Create and configure a libmnl socket for Netlink flow rules. + * + * @return + * A valid libmnl socket object pointer on success, NULL otherwise and + * rte_errno is set. + */ +static struct mnl_socket * +mlx5_flow_mnl_socket_create(void) +{ + struct mnl_socket *nl = mnl_socket_open(NETLINK_ROUTE); + + if (nl) { + mnl_socket_setsockopt(nl, NETLINK_CAP_ACK, &(int){ 1 }, + sizeof(int)); + if (!mnl_socket_bind(nl, 0, MNL_SOCKET_AUTOPID)) + return nl; + } + rte_errno = errno; + if (nl) + mnl_socket_close(nl); + return NULL; +} + +/** + * Destroy a libmnl socket. * * @param nl * Libmnl socket of the @p NETLINK_ROUTE kind. + */ +static void +mlx5_flow_mnl_socket_destroy(struct mnl_socket *nl) +{ + if (nl) + mnl_socket_close(nl); +} + +/** + * Initialize ingress qdisc of a given network interface. + * + * @param nl + * Pointer to tc-flower context to use. * @param ifindex * Index of network interface to initialize. * @param[out] error @@ -1573,8 +1624,8 @@ struct flow_tcf_ptoi { * 0 on success, a negative errno value otherwise and rte_errno is set. */ int -mlx5_flow_tcf_init(struct mnl_socket *nl, unsigned int ifindex, - struct rte_flow_error *error) +mlx5_flow_tcf_init(struct mlx5_flow_tcf_context *nl, + unsigned int ifindex, struct rte_flow_error *error) { struct nlmsghdr *nlh; struct tcmsg *tcm; @@ -1616,37 +1667,47 @@ struct flow_tcf_ptoi { } /** - * Create and configure a libmnl socket for Netlink flow rules. + * Create libmnl context for Netlink flow rules. * * @return * A valid libmnl socket object pointer on success, NULL otherwise and * rte_errno is set. */ -struct mnl_socket * -mlx5_flow_tcf_socket_create(void) +struct mlx5_flow_tcf_context * +mlx5_flow_tcf_context_create(void) { - struct mnl_socket *nl = mnl_socket_open(NETLINK_ROUTE); - - if (nl) { - mnl_socket_setsockopt(nl, NETLINK_CAP_ACK, &(int){ 1 }, - sizeof(int)); - if (!mnl_socket_bind(nl, 0, MNL_SOCKET_AUTOPID)) - return nl; - } - rte_errno = errno; - if (nl) - mnl_socket_close(nl); + struct mlx5_flow_tcf_context *ctx = rte_zmalloc(__func__, + sizeof(*ctx), + sizeof(uint32_t)); + if (!ctx) + goto error; + ctx->nl = mlx5_flow_mnl_socket_create(); + if (!ctx->nl) + goto error; + ctx->buf_size = MNL_SOCKET_BUFFER_SIZE; + ctx->buf = rte_zmalloc(__func__, + ctx->buf_size, sizeof(uint32_t)); + if (!ctx->buf) + goto error; + ctx->seq = random(); + return ctx; +error: + mlx5_flow_tcf_context_destroy(ctx); return NULL; } /** - * Destroy a libmnl socket. + * Destroy a libmnl context. * * @param nl * Libmnl socket of the @p NETLINK_ROUTE kind. */ void -mlx5_flow_tcf_socket_destroy(struct mnl_socket *nl) +mlx5_flow_tcf_context_destroy(struct mlx5_flow_tcf_context *ctx) { - mnl_socket_close(nl); + if (!ctx) + return; + mlx5_flow_mnl_socket_destroy(ctx->nl); + rte_free(ctx->buf); + rte_free(ctx); }