From patchwork Fri Feb 2 16:46:18 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Adrien Mazarguil X-Patchwork-Id: 34899 X-Patchwork-Delegate: ferruh.yigit@amd.com Return-Path: X-Original-To: patchwork@dpdk.org Delivered-To: patchwork@dpdk.org Received: from [92.243.14.124] (localhost [127.0.0.1]) by dpdk.org (Postfix) with ESMTP id BFAC61B1AA; Fri, 2 Feb 2018 17:46:33 +0100 (CET) Received: from mail-wr0-f194.google.com (mail-wr0-f194.google.com [209.85.128.194]) by dpdk.org (Postfix) with ESMTP id A70331B1A2 for ; Fri, 2 Feb 2018 17:46:31 +0100 (CET) Received: by mail-wr0-f194.google.com with SMTP id g21so23120569wrb.13 for ; Fri, 02 Feb 2018 08:46:31 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=6wind-com.20150623.gappssmtp.com; s=20150623; h=date:from:to:cc:subject:message-id:references:mime-version :content-disposition:in-reply-to; bh=m4Bop6SPV6yyuW9z5uutmdjdLeI2G36dXh4pigaz9fM=; b=XIfepK5VJceRbJIK6VbZqRt7QiohZsCxWc4G5d7UA/BevgDn5HZckP7NwpvhA5hQDz UfoSdqkY4o06YdfAmaNV0R14ttieosY0clXVLYnZ1BgX/e6xk1rfdSXS5o1ZRTx2mmaI dgMbLc2a6EODuqtpFImvWKQZrPS1+lFXWvufwWKdyC1In/rVNlOhTh0t4AI9UCMwmOVM x5Kn0kb36QjITPmBmdQk0NiANi0dBdCOBM64LECfAwvNI+3VE8AG6DvI1ycuel9iy4h0 kT9OOYbK4ePKlt+obk6dKJBFInb7X5VtFkbvDuIx/U//+S+2c69gbxbJ7rHYfSasiRCx +H0g== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:date:from:to:cc:subject:message-id:references :mime-version:content-disposition:in-reply-to; bh=m4Bop6SPV6yyuW9z5uutmdjdLeI2G36dXh4pigaz9fM=; b=ZMf88lMdzmLoI+XmFs+kYx8GOzr4fzOkHS4ZT3RvHxYX4FynC0WXvdrXJK8QclLuWc F6FNyoUk9r3hPN3/H6DWjUk6fu+do78h3Us71TgTG8wYCfn9TOjJ9DfO4mowWLCskEdB KxQZ/a06ckJGcwaotp2Yci55DtsZ9Zsl6dZjGY8+MeGcPZSrcuH60l5ydjYns3roCBFf j9z3CxdHXt5Yz3d2UnPInfcW/jSiOd4B9lhiy7EwcE+ZKy4tXSI5FH675w56TXXpRiL6 rh37q1QO3lqMYu9TKSUe+gcn17YrZzWruDYYPpNXII6MsF8GugJrWMtcnGzWqSByBkCK forw== X-Gm-Message-State: AKwxytcVUC0uGiO5MzrCEjQacTnO6F+xFSUgORqXDXMQpy3yj8wAdMHX fcLSIEaBKMjo3BWfxq92i+4nyw== X-Google-Smtp-Source: AH8x2252tDlSP58lBFJVOThK4HXrZA9SbYzxz0KlUCZrpzkcvwfguSf4UH1ETqvTuiv/PVcBZ9BAvQ== X-Received: by 10.223.192.139 with SMTP id d11mr32008559wrf.260.1517589991052; Fri, 02 Feb 2018 08:46:31 -0800 (PST) Received: from 6wind.com (host.78.145.23.62.rev.coltfrance.com. [62.23.145.78]) by smtp.gmail.com with ESMTPSA id 19sm1638515wrx.73.2018.02.02.08.46.30 (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Fri, 02 Feb 2018 08:46:30 -0800 (PST) Date: Fri, 2 Feb 2018 17:46:18 +0100 From: Adrien Mazarguil To: Shahaf Shuler Cc: Nelio Laranjeiro , dev@dpdk.org, Marcelo Ricardo Leitner Message-ID: <20180202164050.13017-5-adrien.mazarguil@6wind.com> References: <20180202144736.8239-1-adrien.mazarguil@6wind.com> <20180202164050.13017-1-adrien.mazarguil@6wind.com> MIME-Version: 1.0 Content-Disposition: inline In-Reply-To: <20180202164050.13017-1-adrien.mazarguil@6wind.com> X-Mailer: git-send-email 2.11.0 Subject: [dpdk-dev] [PATCH v2 4/4] net/mlx: make rdma-core glue path configurable X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Sender: "dev" Since rdma-core glue libraries are intrinsically tied to their respective PMDs and used as internal plug-ins, their presence in the default search path among other system libraries for the dynamic linker is not necessarily desired. This commit enables their installation and subsequent look-up at run time in RTE_EAL_PMD_PATH if configured to a nonempty string. This path can also be overridden by environment variables MLX[45]_GLUE_PATH. Signed-off-by: Adrien Mazarguil --- doc/guides/nics/mlx4.rst | 17 +++++++++++++++++ doc/guides/nics/mlx5.rst | 14 ++++++++++++++ drivers/net/mlx4/mlx4.c | 43 ++++++++++++++++++++++++++++++++++++++++++- drivers/net/mlx5/mlx5.c | 43 ++++++++++++++++++++++++++++++++++++++++++- 4 files changed, 115 insertions(+), 2 deletions(-) diff --git a/doc/guides/nics/mlx4.rst b/doc/guides/nics/mlx4.rst index 88161781c..9e4fbf692 100644 --- a/doc/guides/nics/mlx4.rst +++ b/doc/guides/nics/mlx4.rst @@ -97,6 +97,11 @@ These options can be modified in the ``.config`` file. ``CONFIG_RTE_BUILD_SHARED_LIB`` disabled) and they won't show up as missing with ``ldd(1)``. + It works by moving these dependencies to a purpose-built rdma-core "glue" + plug-in, which must either be installed in ``CONFIG_RTE_EAL_PMD_PATH`` if + set, or in a standard location for the dynamic linker (e.g. ``/lib``) if + left to the default empty string (``""``). + This option has no performance impact. - ``CONFIG_RTE_LIBRTE_MLX4_DEBUG`` (default **n**) @@ -113,6 +118,18 @@ These options can be modified in the ``.config`` file. This value is always 1 for RX queues since they use a single MP. +Environment variables +~~~~~~~~~~~~~~~~~~~~~ + +- ``MLX4_GLUE_PATH`` + + A list of directories in which to search for the rdma-core "glue" plug-in, + separated by colons or semi-colons. + + Only matters when compiled with ``CONFIG_RTE_LIBRTE_MLX4_DLOPEN_DEPS`` + enabled and most useful when ``CONFIG_RTE_EAL_PMD_PATH`` is also set, + since ``LD_LIBRARY_PATH`` has no effect in this case. + Run-time configuration ~~~~~~~~~~~~~~~~~~~~~~ diff --git a/doc/guides/nics/mlx5.rst b/doc/guides/nics/mlx5.rst index a9e4bf51a..1635dff2b 100644 --- a/doc/guides/nics/mlx5.rst +++ b/doc/guides/nics/mlx5.rst @@ -170,6 +170,11 @@ These options can be modified in the ``.config`` file. ``CONFIG_RTE_BUILD_SHARED_LIB`` disabled) and they won't show up as missing with ``ldd(1)``. + It works by moving these dependencies to a purpose-built rdma-core "glue" + plug-in, which must either be installed in ``CONFIG_RTE_EAL_PMD_PATH`` if + set, or in a standard location for the dynamic linker (e.g. ``/lib``) if + left to the default empty string (``""``). + This option has no performance impact. - ``CONFIG_RTE_LIBRTE_MLX5_DEBUG`` (default **n**) @@ -189,6 +194,15 @@ These options can be modified in the ``.config`` file. Environment variables ~~~~~~~~~~~~~~~~~~~~~ +- ``MLX5_GLUE_PATH`` + + A list of directories in which to search for the rdma-core "glue" plug-in, + separated by colons or semi-colons. + + Only matters when compiled with ``CONFIG_RTE_LIBRTE_MLX5_DLOPEN_DEPS`` + enabled and most useful when ``CONFIG_RTE_EAL_PMD_PATH`` is also set, + since ``LD_LIBRARY_PATH`` has no effect in this case. + - ``MLX5_PMD_ENABLE_PADDING`` Enables HW packet padding in PCI bus transactions. diff --git a/drivers/net/mlx4/mlx4.c b/drivers/net/mlx4/mlx4.c index 61a852fb9..4016ddb7b 100644 --- a/drivers/net/mlx4/mlx4.c +++ b/drivers/net/mlx4/mlx4.c @@ -741,11 +741,52 @@ static struct rte_pci_driver mlx4_driver = { static int mlx4_glue_init(void) { + const char *path[] = { + /* + * A basic security check is necessary before trusting + * MLX4_GLUE_PATH, which may override RTE_EAL_PMD_PATH. + */ + (geteuid() == getuid() && getegid() == getgid() ? + getenv("MLX4_GLUE_PATH") : NULL), + RTE_EAL_PMD_PATH, + }; + unsigned int i = 0; void *handle = NULL; void **sym; const char *dlmsg; - handle = dlopen(MLX4_GLUE, RTLD_LAZY); + while (!handle && i != RTE_DIM(path)) { + const char *end; + size_t len; + int ret; + + if (!path[i]) { + ++i; + continue; + } + end = strpbrk(path[i], ":;"); + if (!end) + end = path[i] + strlen(path[i]); + len = end - path[i]; + ret = 0; + do { + char name[ret + 1]; + + ret = snprintf(name, sizeof(name), "%.*s%s" MLX4_GLUE, + (int)len, path[i], + (!len || *(end - 1) == '/') ? "" : "/"); + if (ret == -1) + break; + if (sizeof(name) != (size_t)ret + 1) + continue; + DEBUG("looking for rdma-core glue as \"%s\"", name); + handle = dlopen(name, RTLD_LAZY); + break; + } while (1); + path[i] = end + 1; + if (!*end) + ++i; + } if (!handle) { rte_errno = EINVAL; dlmsg = dlerror(); diff --git a/drivers/net/mlx5/mlx5.c b/drivers/net/mlx5/mlx5.c index 341230d2b..d1441dcba 100644 --- a/drivers/net/mlx5/mlx5.c +++ b/drivers/net/mlx5/mlx5.c @@ -1085,11 +1085,52 @@ static struct rte_pci_driver mlx5_driver = { static int mlx5_glue_init(void) { + const char *path[] = { + /* + * A basic security check is necessary before trusting + * MLX5_GLUE_PATH, which may override RTE_EAL_PMD_PATH. + */ + (geteuid() == getuid() && getegid() == getgid() ? + getenv("MLX5_GLUE_PATH") : NULL), + RTE_EAL_PMD_PATH, + }; + unsigned int i = 0; void *handle = NULL; void **sym; const char *dlmsg; - handle = dlopen(MLX5_GLUE, RTLD_LAZY); + while (!handle && i != RTE_DIM(path)) { + const char *end; + size_t len; + int ret; + + if (!path[i]) { + ++i; + continue; + } + end = strpbrk(path[i], ":;"); + if (!end) + end = path[i] + strlen(path[i]); + len = end - path[i]; + ret = 0; + do { + char name[ret + 1]; + + ret = snprintf(name, sizeof(name), "%.*s%s" MLX5_GLUE, + (int)len, path[i], + (!len || *(end - 1) == '/') ? "" : "/"); + if (ret == -1) + break; + if (sizeof(name) != (size_t)ret + 1) + continue; + DEBUG("looking for rdma-core glue as \"%s\"", name); + handle = dlopen(name, RTLD_LAZY); + break; + } while (1); + path[i] = end + 1; + if (!*end) + ++i; + } if (!handle) { rte_errno = EINVAL; dlmsg = dlerror();