From patchwork Fri Feb 2 15:16:41 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Adrien Mazarguil X-Patchwork-Id: 34895 Return-Path: X-Original-To: patchwork@dpdk.org Delivered-To: patchwork@dpdk.org Received: from [92.243.14.124] (localhost [127.0.0.1]) by dpdk.org (Postfix) with ESMTP id 7AA521B1B0; Fri, 2 Feb 2018 16:16:55 +0100 (CET) Received: from mail-wm0-f67.google.com (mail-wm0-f67.google.com [74.125.82.67]) by dpdk.org (Postfix) with ESMTP id 9A6EB1B1AC for ; Fri, 2 Feb 2018 16:16:54 +0100 (CET) Received: by mail-wm0-f67.google.com with SMTP id v123so13069638wmd.5 for ; Fri, 02 Feb 2018 07:16:54 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=6wind-com.20150623.gappssmtp.com; s=20150623; h=date:from:to:cc:subject:message-id:references:mime-version :content-disposition:in-reply-to; bh=5slZr5J+PcyvjnUtJdQdOUeGo49HNfpAxQ8IeWtTyjI=; b=URySnAeuyqXBfv46rfl+XrvzGn9Hq773VQmYz+3mV4UpKYIvixrsQDwT2sHvyfKQt4 f+OeIM12ZsclBV7ab4rdNUMAVPakHibgD82LrrcLh0BmbIwmj2qss/0rWm3lV4eTZrJD +OXCFqCXs1cAwYSfyq47tlWMIPkunORsk1HBijtmvNnqvMd8cgAj7fMVpIgaZ6oTD7wP i9OC8/WHNl0NT2/4SLE7HbpbzFES5LmbDhXZIzie1gm42EwNeiRBYqw8jj1jKtpuSxna NUAkWFGzbKx0QYz+tw+0VBY6UxKFyVMrGgeR9hGAQY1DrsbNTfOaJLc6XeIjyty9xb0C 9PNg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:date:from:to:cc:subject:message-id:references :mime-version:content-disposition:in-reply-to; bh=5slZr5J+PcyvjnUtJdQdOUeGo49HNfpAxQ8IeWtTyjI=; b=n1+XctN9Rm56UwbR1SfEcea7Jd5mag+EzMRnZcvBR8D+NKaBOTvo5Dl4K77Yl1K+5H FzmOW534o5mqIzxiWG9GfXz2dpyY5jSFLfWDADD2udhKVevx4YuIXEwj9M5hpG6JnnaH fc7dug5eAom15sIhKxdZisy+S9dwGnqi7bXZ+WtXLHyn8MbctKa86XhO19BHghedZ/7S ptc87wiRLTzKit4FnZPXEuhYCoBh1qNMbeXDTMQeslgdKdTtDqKWkEMIFQxakD+nHQpF 0Y20wADq03+FYRGRDkaN5+RJLe3uAKwSFm+MOtdkYCcGPktNJh3WQhXeEuXIYmLMavIm aMWw== X-Gm-Message-State: AKwxytcfFL7WCGK1pQwyb9PnL/UpCHCBpMxxYgWuBr47MiR/odu1gH6j XabFiHVUL6ODjcb4Di1RvkAh1w== X-Google-Smtp-Source: AH8x225yXoIIqzztFNIwFWK7PabZolYvL1fxKzmohKJRqXheEzg/eIvNi0o/e1oO7y9587GVlOCg6w== X-Received: by 10.80.163.199 with SMTP id t7mr21196885edb.56.1517584614271; Fri, 02 Feb 2018 07:16:54 -0800 (PST) Received: from 6wind.com (host.78.145.23.62.rev.coltfrance.com. [62.23.145.78]) by smtp.gmail.com with ESMTPSA id s26sm1928081eds.49.2018.02.02.07.16.53 (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Fri, 02 Feb 2018 07:16:53 -0800 (PST) Date: Fri, 2 Feb 2018 16:16:41 +0100 From: Adrien Mazarguil To: Shahaf Shuler Cc: Nelio Laranjeiro , dev@dpdk.org, Marcelo Ricardo Leitner Message-ID: <20180202144736.8239-5-adrien.mazarguil@6wind.com> References: <20180202144736.8239-1-adrien.mazarguil@6wind.com> MIME-Version: 1.0 Content-Disposition: inline In-Reply-To: <20180202144736.8239-1-adrien.mazarguil@6wind.com> X-Mailer: git-send-email 2.11.0 Subject: [dpdk-dev] [PATCH v1 4/4] net/mlx: make rdma-core glue path configurable X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Sender: "dev" Since rdma-core glue libraries are intrinsically tied to their respective PMDs and used as internal plug-ins, their presence in the default search path among other system libraries for the dynamic linker is not necessarily desired. This commit enables their installation and subsequent look-up at run time in RTE_EAL_PMD_PATH if configured to a nonempty string. This path can also be overridden by environment variables MLX[45]_GLUE_PATH. Signed-off-by: Adrien Mazarguil --- doc/guides/nics/mlx4.rst | 17 +++++++++++++++++ doc/guides/nics/mlx5.rst | 14 ++++++++++++++ drivers/net/mlx4/mlx4.c | 43 ++++++++++++++++++++++++++++++++++++++++++- drivers/net/mlx5/mlx5.c | 43 ++++++++++++++++++++++++++++++++++++++++++- 4 files changed, 115 insertions(+), 2 deletions(-) diff --git a/doc/guides/nics/mlx4.rst b/doc/guides/nics/mlx4.rst index 88161781c..9e4fbf692 100644 --- a/doc/guides/nics/mlx4.rst +++ b/doc/guides/nics/mlx4.rst @@ -97,6 +97,11 @@ These options can be modified in the ``.config`` file. ``CONFIG_RTE_BUILD_SHARED_LIB`` disabled) and they won't show up as missing with ``ldd(1)``. + It works by moving these dependencies to a purpose-built rdma-core "glue" + plug-in, which must either be installed in ``CONFIG_RTE_EAL_PMD_PATH`` if + set, or in a standard location for the dynamic linker (e.g. ``/lib``) if + left to the default empty string (``""``). + This option has no performance impact. - ``CONFIG_RTE_LIBRTE_MLX4_DEBUG`` (default **n**) @@ -113,6 +118,18 @@ These options can be modified in the ``.config`` file. This value is always 1 for RX queues since they use a single MP. +Environment variables +~~~~~~~~~~~~~~~~~~~~~ + +- ``MLX4_GLUE_PATH`` + + A list of directories in which to search for the rdma-core "glue" plug-in, + separated by colons or semi-colons. + + Only matters when compiled with ``CONFIG_RTE_LIBRTE_MLX4_DLOPEN_DEPS`` + enabled and most useful when ``CONFIG_RTE_EAL_PMD_PATH`` is also set, + since ``LD_LIBRARY_PATH`` has no effect in this case. + Run-time configuration ~~~~~~~~~~~~~~~~~~~~~~ diff --git a/doc/guides/nics/mlx5.rst b/doc/guides/nics/mlx5.rst index a9e4bf51a..1635dff2b 100644 --- a/doc/guides/nics/mlx5.rst +++ b/doc/guides/nics/mlx5.rst @@ -170,6 +170,11 @@ These options can be modified in the ``.config`` file. ``CONFIG_RTE_BUILD_SHARED_LIB`` disabled) and they won't show up as missing with ``ldd(1)``. + It works by moving these dependencies to a purpose-built rdma-core "glue" + plug-in, which must either be installed in ``CONFIG_RTE_EAL_PMD_PATH`` if + set, or in a standard location for the dynamic linker (e.g. ``/lib``) if + left to the default empty string (``""``). + This option has no performance impact. - ``CONFIG_RTE_LIBRTE_MLX5_DEBUG`` (default **n**) @@ -189,6 +194,15 @@ These options can be modified in the ``.config`` file. Environment variables ~~~~~~~~~~~~~~~~~~~~~ +- ``MLX5_GLUE_PATH`` + + A list of directories in which to search for the rdma-core "glue" plug-in, + separated by colons or semi-colons. + + Only matters when compiled with ``CONFIG_RTE_LIBRTE_MLX5_DLOPEN_DEPS`` + enabled and most useful when ``CONFIG_RTE_EAL_PMD_PATH`` is also set, + since ``LD_LIBRARY_PATH`` has no effect in this case. + - ``MLX5_PMD_ENABLE_PADDING`` Enables HW packet padding in PCI bus transactions. diff --git a/drivers/net/mlx4/mlx4.c b/drivers/net/mlx4/mlx4.c index 61a852fb9..4266cb1bb 100644 --- a/drivers/net/mlx4/mlx4.c +++ b/drivers/net/mlx4/mlx4.c @@ -741,11 +741,52 @@ static struct rte_pci_driver mlx4_driver = { static int mlx4_glue_init(void) { + const char *path[] = { + /* + * A basic security check is necessary before trusting + * MLX4_GLUE_PATH, which may override RTE_EAL_PMD_PATH. + */ + (geteuid() == getuid() && getegid() == getgid() ? + getenv("MLX4_GLUE_PATH") : NULL), + RTE_EAL_PMD_PATH, + }; + unsigned int i = 0; void *handle = NULL; void **sym; const char *dlmsg; - handle = dlopen(MLX4_GLUE, RTLD_LAZY); + while (!handle && i != RTE_DIM(path)) { + const char *end; + size_t len; + int ret; + + if (!path[i]) { + ++i; + continue; + } + end = strpbrk(path[i], ":;"); + if (!end) + end = path[i] + strlen(path[i]); + len = end - path[i]; + ret = 0; + do { + char name[ret + 1]; + + ret = snprintf(name, ret, "%.*s%s" MLX4_GLUE "\n", + (int)len, path[i], + (!len || *(end - 1) == '/') ? "" : "/"); + if (ret == -1) + break; + if (sizeof(name) != (size_t)ret + 1) + continue; + DEBUG("looking for rdma-core glue as \"%s\"", name); + handle = dlopen(name, RTLD_LAZY); + break; + } while (1); + path[i] = end + 1; + if (!*end) + ++i; + } if (!handle) { rte_errno = EINVAL; dlmsg = dlerror(); diff --git a/drivers/net/mlx5/mlx5.c b/drivers/net/mlx5/mlx5.c index 341230d2b..caa60339e 100644 --- a/drivers/net/mlx5/mlx5.c +++ b/drivers/net/mlx5/mlx5.c @@ -1085,11 +1085,52 @@ static struct rte_pci_driver mlx5_driver = { static int mlx5_glue_init(void) { + const char *path[] = { + /* + * A basic security check is necessary before trusting + * MLX5_GLUE_PATH, which may override RTE_EAL_PMD_PATH. + */ + (geteuid() == getuid() && getegid() == getgid() ? + getenv("MLX5_GLUE_PATH") : NULL), + RTE_EAL_PMD_PATH, + }; + unsigned int i = 0; void *handle = NULL; void **sym; const char *dlmsg; - handle = dlopen(MLX5_GLUE, RTLD_LAZY); + while (!handle && i != RTE_DIM(path)) { + const char *end; + size_t len; + int ret; + + if (!path[i]) { + ++i; + continue; + } + end = strpbrk(path[i], ":;"); + if (!end) + end = path[i] + strlen(path[i]); + len = end - path[i]; + ret = 0; + do { + char name[ret + 1]; + + ret = snprintf(name, ret, "%.*s%s" MLX5_GLUE "\n", + (int)len, path[i], + (!len || *(end - 1) == '/') ? "" : "/"); + if (ret == -1) + break; + if (sizeof(name) != (size_t)ret + 1) + continue; + DEBUG("looking for rdma-core glue as \"%s\"", name); + handle = dlopen(name, RTLD_LAZY); + break; + } while (1); + path[i] = end + 1; + if (!*end) + ++i; + } if (!handle) { rte_errno = EINVAL; dlmsg = dlerror();