From patchwork Wed May 17 12:25:08 2017 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Andrew Rybchenko X-Patchwork-Id: 24339 X-Patchwork-Delegate: ferruh.yigit@amd.com Return-Path: X-Original-To: patchwork@dpdk.org Delivered-To: patchwork@dpdk.org Received: from [92.243.14.124] (localhost [IPv6:::1]) by dpdk.org (Postfix) with ESMTP id B8D9C3251; Wed, 17 May 2017 14:25:36 +0200 (CEST) Received: from dispatch1-us1.ppe-hosted.com (dispatch1-us1.ppe-hosted.com [67.231.154.164]) by dpdk.org (Postfix) with ESMTP id 2B5642986 for ; Wed, 17 May 2017 14:25:32 +0200 (CEST) Received: from pure.maildistiller.com (unknown [10.110.50.29]) by dispatch1-us1.ppe-hosted.com (Proofpoint Essentials ESMTP Server) with ESMTP id 81ECD60079 for ; Wed, 17 May 2017 12:25:31 +0000 (UTC) X-Virus-Scanned: Proofpoint Essentials engine Received: from mx1-us4.ppe-hosted.com (unknown [10.110.49.251]) by pure.maildistiller.com (Proofpoint Essentials ESMTP Server) with ESMTPS id 8800060056 for ; Wed, 17 May 2017 12:25:30 +0000 (UTC) Received: from webmail.solarflare.com (webmail.solarflare.com [12.187.104.26]) (using TLSv1 with cipher ECDHE-RSA-AES256-SHA (256/256 bits)) (No client certificate requested) by mx1-us4.ppe-hosted.com (Proofpoint Essentials ESMTP Server) with ESMTPS id 068BF800BE for ; Wed, 17 May 2017 12:25:30 +0000 (UTC) Received: from ocex03.SolarFlarecom.com (10.20.40.36) by ocex03.SolarFlarecom.com (10.20.40.36) with Microsoft SMTP Server (TLS) id 15.0.1044.25; Wed, 17 May 2017 05:25:26 -0700 Received: from opal.uk.solarflarecom.com (10.17.10.1) by ocex03.SolarFlarecom.com (10.20.40.36) with Microsoft SMTP Server (TLS) id 15.0.1044.25 via Frontend Transport; Wed, 17 May 2017 05:25:25 -0700 Received: from uklogin.uk.solarflarecom.com (uklogin.uk.solarflarecom.com [10.17.10.10]) by opal.uk.solarflarecom.com (8.13.8/8.13.8) with ESMTP id v4HCPO9L012421 for ; Wed, 17 May 2017 13:25:24 +0100 Received: from uklogin.uk.solarflarecom.com (localhost.localdomain [127.0.0.1]) by uklogin.uk.solarflarecom.com (8.13.8/8.13.8) with ESMTP id v4HCPO1E011027 for ; Wed, 17 May 2017 13:25:24 +0100 From: Andrew Rybchenko To: Date: Wed, 17 May 2017 13:25:08 +0100 Message-ID: <1495023908-10977-3-git-send-email-arybchenko@solarflare.com> X-Mailer: git-send-email 1.8.2.3 In-Reply-To: <1495023908-10977-1-git-send-email-arybchenko@solarflare.com> References: <1495023908-10977-1-git-send-email-arybchenko@solarflare.com> MIME-Version: 1.0 X-MDID: 1495023931-Fgx0MBwVBQjb Subject: [dpdk-dev] [PATCH 3/3] net/sfc: support multi-process X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Sender: "dev" Signed-off-by: Andrew Rybchenko Reviewed-by: Andy Moreton --- doc/guides/nics/features/sfc_efx.ini | 1 + drivers/net/sfc/sfc.h | 11 +++ drivers/net/sfc/sfc_dp_rx.h | 1 + drivers/net/sfc/sfc_dp_tx.h | 1 + drivers/net/sfc/sfc_ef10_rx.c | 2 +- drivers/net/sfc/sfc_ef10_tx.c | 5 +- drivers/net/sfc/sfc_ethdev.c | 133 ++++++++++++++++++++++++++++++++++- 7 files changed, 148 insertions(+), 6 deletions(-) diff --git a/doc/guides/nics/features/sfc_efx.ini b/doc/guides/nics/features/sfc_efx.ini index 7957b5e..1db7f67 100644 --- a/doc/guides/nics/features/sfc_efx.ini +++ b/doc/guides/nics/features/sfc_efx.ini @@ -28,6 +28,7 @@ Packet type parsing = Y Basic stats = Y Extended stats = Y FW version = Y +Multiprocess aware = Y BSD nic_uio = Y Linux UIO = Y Linux VFIO = Y diff --git a/drivers/net/sfc/sfc.h b/drivers/net/sfc/sfc.h index 772a713..007ed24 100644 --- a/drivers/net/sfc/sfc.h +++ b/drivers/net/sfc/sfc.h @@ -225,7 +225,18 @@ struct sfc_adapter { uint8_t rss_key[SFC_RSS_KEY_SIZE]; #endif + /* + * Shared memory copy of the Rx datapath name to be used by + * the secondary process to find Rx datapath to be used. + */ + char *dp_rx_name; const struct sfc_dp_rx *dp_rx; + + /* + * Shared memory copy of the Tx datapath name to be used by + * the secondary process to find Rx datapath to be used. + */ + char *dp_tx_name; const struct sfc_dp_tx *dp_tx; }; diff --git a/drivers/net/sfc/sfc_dp_rx.h b/drivers/net/sfc/sfc_dp_rx.h index 9d05a4b..a7b8278 100644 --- a/drivers/net/sfc/sfc_dp_rx.h +++ b/drivers/net/sfc/sfc_dp_rx.h @@ -161,6 +161,7 @@ struct sfc_dp_rx { unsigned int features; #define SFC_DP_RX_FEAT_SCATTER 0x1 +#define SFC_DP_RX_FEAT_MULTI_PROCESS 0x2 sfc_dp_rx_qcreate_t *qcreate; sfc_dp_rx_qdestroy_t *qdestroy; sfc_dp_rx_qstart_t *qstart; diff --git a/drivers/net/sfc/sfc_dp_tx.h b/drivers/net/sfc/sfc_dp_tx.h index 2bb9a2e..c1c3419 100644 --- a/drivers/net/sfc/sfc_dp_tx.h +++ b/drivers/net/sfc/sfc_dp_tx.h @@ -135,6 +135,7 @@ struct sfc_dp_tx { #define SFC_DP_TX_FEAT_VLAN_INSERT 0x1 #define SFC_DP_TX_FEAT_TSO 0x2 #define SFC_DP_TX_FEAT_MULTI_SEG 0x4 +#define SFC_DP_TX_FEAT_MULTI_PROCESS 0x8 sfc_dp_tx_qcreate_t *qcreate; sfc_dp_tx_qdestroy_t *qdestroy; sfc_dp_tx_qstart_t *qstart; diff --git a/drivers/net/sfc/sfc_ef10_rx.c b/drivers/net/sfc/sfc_ef10_rx.c index 1484bab..60812cb 100644 --- a/drivers/net/sfc/sfc_ef10_rx.c +++ b/drivers/net/sfc/sfc_ef10_rx.c @@ -699,7 +699,7 @@ struct sfc_dp_rx sfc_ef10_rx = { .type = SFC_DP_RX, .hw_fw_caps = SFC_DP_HW_FW_CAP_EF10, }, - .features = 0, + .features = SFC_DP_RX_FEAT_MULTI_PROCESS, .qcreate = sfc_ef10_rx_qcreate, .qdestroy = sfc_ef10_rx_qdestroy, .qstart = sfc_ef10_rx_qstart, diff --git a/drivers/net/sfc/sfc_ef10_tx.c b/drivers/net/sfc/sfc_ef10_tx.c index bac9baa..5482db8 100644 --- a/drivers/net/sfc/sfc_ef10_tx.c +++ b/drivers/net/sfc/sfc_ef10_tx.c @@ -534,7 +534,8 @@ struct sfc_dp_tx sfc_ef10_tx = { .type = SFC_DP_TX, .hw_fw_caps = SFC_DP_HW_FW_CAP_EF10, }, - .features = SFC_DP_TX_FEAT_MULTI_SEG, + .features = SFC_DP_TX_FEAT_MULTI_SEG | + SFC_DP_TX_FEAT_MULTI_PROCESS, .qcreate = sfc_ef10_tx_qcreate, .qdestroy = sfc_ef10_tx_qdestroy, .qstart = sfc_ef10_tx_qstart, @@ -549,7 +550,7 @@ struct sfc_dp_tx sfc_ef10_simple_tx = { .name = SFC_KVARG_DATAPATH_EF10_SIMPLE, .type = SFC_DP_TX, }, - .features = 0, + .features = SFC_DP_TX_FEAT_MULTI_PROCESS, .qcreate = sfc_ef10_tx_qcreate, .qdestroy = sfc_ef10_tx_qdestroy, .qstart = sfc_ef10_tx_qstart, diff --git a/drivers/net/sfc/sfc_ethdev.c b/drivers/net/sfc/sfc_ethdev.c index d6bba1d..0bd2de4 100644 --- a/drivers/net/sfc/sfc_ethdev.c +++ b/drivers/net/sfc/sfc_ethdev.c @@ -913,6 +913,10 @@ sfc_set_mc_addr_list(struct rte_eth_dev *dev, struct ether_addr *mc_addr_set, return -rc; } +/* + * The function is used by the secondary process as well. It must not + * use any process-local pointers from the adapter data. + */ static void sfc_rx_queue_info_get(struct rte_eth_dev *dev, uint16_t rx_queue_id, struct rte_eth_rxq_info *qinfo) @@ -939,6 +943,10 @@ sfc_rx_queue_info_get(struct rte_eth_dev *dev, uint16_t rx_queue_id, sfc_adapter_unlock(sa); } +/* + * The function is used by the secondary process as well. It must not + * use any process-local pointers from the adapter data. + */ static void sfc_tx_queue_info_get(struct rte_eth_dev *dev, uint16_t tx_queue_id, struct rte_eth_txq_info *qinfo) @@ -1372,6 +1380,29 @@ static const struct eth_dev_ops sfc_eth_dev_ops = { .fw_version_get = sfc_fw_version_get, }; +/** + * Duplicate a string in potentially shared memory required for + * multi-process support. + * + * strdup() allocates from process-local heap/memory. + */ +static char * +sfc_strdup(const char *str) +{ + size_t size; + char *copy; + + if (str == NULL) + return NULL; + + size = strlen(str) + 1; + copy = rte_malloc(__func__, size, 0); + if (copy != NULL) + rte_memcpy(copy, str, size); + + return copy; +} + static int sfc_eth_dev_set_ops(struct rte_eth_dev *dev) { @@ -1419,7 +1450,13 @@ sfc_eth_dev_set_ops(struct rte_eth_dev *dev) } } - sfc_info(sa, "use %s Rx datapath", sa->dp_rx->dp.name); + sa->dp_rx_name = sfc_strdup(sa->dp_rx->dp.name); + if (sa->dp_rx_name == NULL) { + rc = ENOMEM; + goto fail_dp_rx_name; + } + + sfc_info(sa, "use %s Rx datapath", sa->dp_rx_name); dev->rx_pkt_burst = sa->dp_rx->pkt_burst; @@ -1452,7 +1489,13 @@ sfc_eth_dev_set_ops(struct rte_eth_dev *dev) } } - sfc_info(sa, "use %s Tx datapath", sa->dp_tx->dp.name); + sa->dp_tx_name = sfc_strdup(sa->dp_tx->dp.name); + if (sa->dp_tx_name == NULL) { + rc = ENOMEM; + goto fail_dp_tx_name; + } + + sfc_info(sa, "use %s Tx datapath", sa->dp_tx_name); dev->tx_pkt_burst = sa->dp_tx->pkt_burst; @@ -1460,11 +1503,16 @@ sfc_eth_dev_set_ops(struct rte_eth_dev *dev) return 0; +fail_dp_tx_name: fail_dp_tx_caps: sa->dp_tx = NULL; fail_dp_tx: fail_kvarg_tx_datapath: + rte_free(sa->dp_rx_name); + sa->dp_rx_name = NULL; + +fail_dp_rx_name: fail_dp_rx_caps: sa->dp_rx = NULL; @@ -1482,10 +1530,80 @@ sfc_eth_dev_clear_ops(struct rte_eth_dev *dev) dev->rx_pkt_burst = NULL; dev->tx_pkt_burst = NULL; + rte_free(sa->dp_tx_name); + sa->dp_tx_name = NULL; sa->dp_tx = NULL; + + rte_free(sa->dp_rx_name); + sa->dp_rx_name = NULL; sa->dp_rx = NULL; } +static const struct eth_dev_ops sfc_eth_dev_secondary_ops = { + .rxq_info_get = sfc_rx_queue_info_get, + .txq_info_get = sfc_tx_queue_info_get, +}; + +static int +sfc_eth_dev_secondary_set_ops(struct rte_eth_dev *dev) +{ + /* + * Device private data has really many process-local pointers. + * Below code should be extremely careful to use data located + * in shared memory only. + */ + struct sfc_adapter *sa = dev->data->dev_private; + const struct sfc_dp_rx *dp_rx; + const struct sfc_dp_tx *dp_tx; + int rc; + + dp_rx = sfc_dp_find_rx_by_name(&sfc_dp_head, sa->dp_rx_name); + if (dp_rx == NULL) { + sfc_err(sa, "cannot find %s Rx datapath", sa->dp_tx_name); + rc = ENOENT; + goto fail_dp_rx; + } + if (~dp_rx->features & SFC_DP_RX_FEAT_MULTI_PROCESS) { + sfc_err(sa, "%s Rx datapath does not support multi-process", + sa->dp_tx_name); + rc = EINVAL; + goto fail_dp_rx_multi_process; + } + + dp_tx = sfc_dp_find_tx_by_name(&sfc_dp_head, sa->dp_tx_name); + if (dp_tx == NULL) { + sfc_err(sa, "cannot find %s Tx datapath", sa->dp_tx_name); + rc = ENOENT; + goto fail_dp_tx; + } + if (~dp_tx->features & SFC_DP_TX_FEAT_MULTI_PROCESS) { + sfc_err(sa, "%s Tx datapath does not support multi-process", + sa->dp_tx_name); + rc = EINVAL; + goto fail_dp_tx_multi_process; + } + + dev->rx_pkt_burst = dp_rx->pkt_burst; + dev->tx_pkt_burst = dp_tx->pkt_burst; + dev->dev_ops = &sfc_eth_dev_secondary_ops; + + return 0; + +fail_dp_tx_multi_process: +fail_dp_tx: +fail_dp_rx_multi_process: +fail_dp_rx: + return rc; +} + +static void +sfc_eth_dev_secondary_clear_ops(struct rte_eth_dev *dev) +{ + dev->dev_ops = NULL; + dev->tx_pkt_burst = NULL; + dev->rx_pkt_burst = NULL; +} + static void sfc_register_dp(void) { @@ -1512,6 +1630,9 @@ sfc_eth_dev_init(struct rte_eth_dev *dev) sfc_register_dp(); + if (rte_eal_process_type() != RTE_PROC_PRIMARY) + return -sfc_eth_dev_secondary_set_ops(dev); + /* Required for logging */ sa->pci_addr = pci_dev->addr; sa->port_id = dev->data->port_id; @@ -1595,8 +1716,14 @@ sfc_eth_dev_init(struct rte_eth_dev *dev) static int sfc_eth_dev_uninit(struct rte_eth_dev *dev) { - struct sfc_adapter *sa = dev->data->dev_private; + struct sfc_adapter *sa; + + if (rte_eal_process_type() != RTE_PROC_PRIMARY) { + sfc_eth_dev_secondary_clear_ops(dev); + return 0; + } + sa = dev->data->dev_private; sfc_log_init(sa, "entry"); sfc_adapter_lock(sa);