From patchwork Thu Jan 21 11:07:57 2016 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Tetsuya Mukawa X-Patchwork-Id: 10034 Return-Path: X-Original-To: patchwork@dpdk.org Delivered-To: patchwork@dpdk.org Received: from [92.243.14.124] (localhost [IPv6:::1]) by dpdk.org (Postfix) with ESMTP id EC29391AD; Thu, 21 Jan 2016 12:08:21 +0100 (CET) Received: from mail-pf0-f170.google.com (mail-pf0-f170.google.com [209.85.192.170]) by dpdk.org (Postfix) with ESMTP id 0AFA38DB3 for ; Thu, 21 Jan 2016 12:08:19 +0100 (CET) Received: by mail-pf0-f170.google.com with SMTP id e65so22056510pfe.0 for ; Thu, 21 Jan 2016 03:08:18 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=igel-co-jp.20150623.gappssmtp.com; s=20150623; h=from:to:cc:subject:date:message-id:in-reply-to:references :in-reply-to:references; bh=ILSAB/pUzFNnJGnFVMdiOosBmfjj52/1N0Ii4ejOQbY=; b=pF2DOnDo/I2Cnul8NVTo2ApuZQ4+IxZBwopNrLVYuwXZMgn3VTwy03KzetF1LCGboH W/joYqora/Lvcu/lOA8pMo2Fg4ET3bbJopsSKIESqQiFqUGUF1Tiw+gnAiENy0tKy+XC 6UjgXpHosReq3QBEo2oa/694JClUebW7vQWeYiZIxaD4hd7SCtJLTTugr5OKiXRjtL3t JALhoKATHuSa0gyNnf03zTC05OrBAs67/mNJUvepg+heRQNnqN2Nm4aAUzGCb/7R036y xmwSZGhuBIcpWTb1MgbnCLp6xuPAwmpzK37sHEsHGfqnM4F6XnYyr4cMQgyzzJMp+F3h K98Q== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20130820; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:in-reply-to:references; bh=ILSAB/pUzFNnJGnFVMdiOosBmfjj52/1N0Ii4ejOQbY=; b=PQEIP3P6gQg9UnJyEGmoExDccsp96c6HzMFJNMO9+SUDa4bd/xIOFJaEfK05CdbJo8 2cpqV7nLJPHi38YuR1a8IypgZC5YgoTtJ5XTbNDf8fDs7q337bgAeKmFOmMcO0sJ3zPG ETZgK+L4IaKtm3hyklELWhm213eoATrkXKDSWewYkvuBtVmHKXsqphsGtZmbuIcCyTM7 hvx94bYqFBQToWunA99wYfdCb10CFaFnHtvln77gsODc4JfqWUmiRcS691R3FYU1xNQa KZI23jswMRimgxQqbUQOFQpATXjA+0IAN77EFddUZ4vZR7WkSkBYFNzQwa8YUBBgyp1L 08fw== X-Gm-Message-State: ALoCoQlf3saOxhpJAd3qoNOD60iaRmwPh0mfLtEoWv3Lm6d7wXBKQDkbjiWklnhRVEEV7WIDiHCisgHyQSrojZdKiSkVlbVyjA== X-Received: by 10.98.87.204 with SMTP id i73mr59500053pfj.63.1453374498447; Thu, 21 Jan 2016 03:08:18 -0800 (PST) Received: from localhost.localdomain (napt.igel.co.jp. [219.106.231.132]) by smtp.gmail.com with ESMTPSA id wa17sm1792640pac.38.2016.01.21.03.08.16 (version=TLS1_2 cipher=ECDHE-RSA-AES128-SHA bits=128/128); Thu, 21 Jan 2016 03:08:17 -0800 (PST) From: Tetsuya Mukawa To: dev@dpdk.org, yuanhan.liu@linux.intel.com, jianfeng.tan@intel.com Date: Thu, 21 Jan 2016 20:07:57 +0900 Message-Id: <1453374478-30996-5-git-send-email-mukawa@igel.co.jp> X-Mailer: git-send-email 2.1.4 In-Reply-To: <1453374478-30996-1-git-send-email-mukawa@igel.co.jp> References: <1453374478-30996-1-git-send-email-mukawa@igel.co.jp> In-Reply-To: <1453108389-21006-2-git-send-email-mukawa@igel.co.jp> References: <1453108389-21006-2-git-send-email-mukawa@igel.co.jp> Subject: [dpdk-dev] [RFC PATCH 4/5] EAL: Add new EAL "--shm" option. X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: patches and discussions about DPDK List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Sender: "dev" This is a temporary patch to get EAL memory under 16T(1 << 44). The patch adds new EAL "--shm" option. If the option is specified, EAL will allocate one file from hugetlbfs. This memory is for sharing memory between DPDK applicaiton and QEMU ivhsmem device. Signed-off-by: Tetsuya Mukawa --- lib/librte_eal/common/eal_common_options.c | 5 ++ lib/librte_eal/common/eal_internal_cfg.h | 1 + lib/librte_eal/common/eal_options.h | 2 + lib/librte_eal/common/include/rte_memory.h | 5 ++ lib/librte_eal/linuxapp/eal/eal_memory.c | 76 ++++++++++++++++++++++++++++++ 5 files changed, 89 insertions(+) diff --git a/lib/librte_eal/common/eal_common_options.c b/lib/librte_eal/common/eal_common_options.c index 29942ea..a752bf3 100644 --- a/lib/librte_eal/common/eal_common_options.c +++ b/lib/librte_eal/common/eal_common_options.c @@ -86,6 +86,7 @@ eal_long_options[] = { {OPT_NO_HUGE, 0, NULL, OPT_NO_HUGE_NUM }, {OPT_NO_PCI, 0, NULL, OPT_NO_PCI_NUM }, {OPT_NO_SHCONF, 0, NULL, OPT_NO_SHCONF_NUM }, + {OPT_SHM, 0, NULL, OPT_SHM_NUM }, {OPT_PCI_BLACKLIST, 1, NULL, OPT_PCI_BLACKLIST_NUM }, {OPT_PCI_WHITELIST, 1, NULL, OPT_PCI_WHITELIST_NUM }, {OPT_PROC_TYPE, 1, NULL, OPT_PROC_TYPE_NUM }, @@ -834,6 +835,10 @@ eal_parse_common_option(int opt, const char *optarg, conf->no_hugetlbfs = 1; break; + case OPT_SHM_NUM: + conf->shm = 1; + break; + case OPT_NO_PCI_NUM: conf->no_pci = 1; break; diff --git a/lib/librte_eal/common/eal_internal_cfg.h b/lib/librte_eal/common/eal_internal_cfg.h index 5f1367e..362ce12 100644 --- a/lib/librte_eal/common/eal_internal_cfg.h +++ b/lib/librte_eal/common/eal_internal_cfg.h @@ -66,6 +66,7 @@ struct internal_config { volatile unsigned no_hugetlbfs; /**< true to disable hugetlbfs */ unsigned hugepage_unlink; /**< true to unlink backing files */ volatile unsigned xen_dom0_support; /**< support app running on Xen Dom0*/ + volatile unsigned shm; /**< true to create shared memory for ivshmem */ volatile unsigned no_pci; /**< true to disable PCI */ volatile unsigned no_hpet; /**< true to disable HPET */ volatile unsigned vmware_tsc_map; /**< true to use VMware TSC mapping diff --git a/lib/librte_eal/common/eal_options.h b/lib/librte_eal/common/eal_options.h index a881c62..c1e586a 100644 --- a/lib/librte_eal/common/eal_options.h +++ b/lib/librte_eal/common/eal_options.h @@ -55,6 +55,8 @@ enum { OPT_HUGE_DIR_NUM, #define OPT_HUGE_UNLINK "huge-unlink" OPT_HUGE_UNLINK_NUM, +#define OPT_SHM "shm" + OPT_SHM_NUM, #define OPT_LCORES "lcores" OPT_LCORES_NUM, #define OPT_LOG_LEVEL "log-level" diff --git a/lib/librte_eal/common/include/rte_memory.h b/lib/librte_eal/common/include/rte_memory.h index 9c9e40f..3ad155b 100644 --- a/lib/librte_eal/common/include/rte_memory.h +++ b/lib/librte_eal/common/include/rte_memory.h @@ -102,6 +102,7 @@ struct rte_memseg { int32_t socket_id; /**< NUMA socket ID. */ uint32_t nchannel; /**< Number of channels. */ uint32_t nrank; /**< Number of ranks. */ + int fd; /**< fd used for share this memory */ #ifdef RTE_LIBRTE_XEN_DOM0 /**< store segment MFNs */ uint64_t mfn[DOM0_NUM_MEMBLOCK]; @@ -130,6 +131,10 @@ int rte_mem_lock_page(const void *virt); */ phys_addr_t rte_mem_virt2phy(const void *virt); + +int +rte_memseg_info_get(int index, int *pfd, uint64_t *psize, void **paddr); + /** * Get the layout of the available physical memory. * diff --git a/lib/librte_eal/linuxapp/eal/eal_memory.c b/lib/librte_eal/linuxapp/eal/eal_memory.c index 846fd31..7122f16 100644 --- a/lib/librte_eal/linuxapp/eal/eal_memory.c +++ b/lib/librte_eal/linuxapp/eal/eal_memory.c @@ -150,6 +150,21 @@ rte_mem_lock_page(const void *virt) return mlock((void*)aligned, page_size); } +int +rte_memseg_info_get(int index, int *pfd, uint64_t *psize, void **paddr) +{ + struct rte_mem_config *mcfg; + mcfg = rte_eal_get_configuration()->mem_config; + + if (pfd != NULL) + *pfd = mcfg->memseg[index].fd; + if (psize != NULL) + *psize = (uint64_t)mcfg->memseg[index].len; + if (paddr != NULL) + *paddr = (void *)(uint64_t)mcfg->memseg[index].addr; + return 0; +} + /* * Get physical address of any mapped virtual address in the current process. */ @@ -1075,6 +1090,46 @@ calc_num_pages_per_socket(uint64_t * memory, return total_num_pages; } +static void * +rte_eal_shm_create(int *pfd, const char *hugedir) +{ + int ret, fd; + char filepath[256]; + void *vaddr; + uint64_t size = internal_config.memory; + + sprintf(filepath, "%s/%s_cvio", hugedir, + internal_config.hugefile_prefix); + + fd = open(filepath, O_CREAT | O_RDWR, 0600); + if (fd < 0) + rte_panic("open %s failed: %s\n", filepath, strerror(errno)); + + ret = flock(fd, LOCK_EX); + if (ret < 0) { + close(fd); + rte_panic("flock %s failed: %s\n", filepath, strerror(errno)); + } + + ret = ftruncate(fd, size); + if (ret < 0) + rte_panic("ftruncate failed: %s\n", strerror(errno)); + + /* + * Here, we need to map under (1 << 44). + * This is temporary implementation. + */ + vaddr = mmap((void *)(1ULL << 43), size, PROT_READ | PROT_WRITE, + MAP_SHARED | MAP_FIXED, fd, 0); + if (vaddr != MAP_FAILED) { + memset(vaddr, 0, size); + *pfd = fd; + } + memset(vaddr, 0, size); + + return vaddr; +} + /* * Prepare physical memory mapping: fill configuration structure with * these infos, return 0 on success. @@ -1127,6 +1182,27 @@ rte_eal_hugepage_init(void) return 0; } + /* create shared memory consist of only one file */ + if (internal_config.shm) { + int fd; + struct hugepage_info *hpi; + + hpi = &internal_config.hugepage_info[0]; + addr = rte_eal_shm_create(&fd, hpi->hugedir); + if (addr == MAP_FAILED) { + RTE_LOG(ERR, EAL, "%s: mmap() failed: %s\n", __func__, + strerror(errno)); + return -1; + } + mcfg->memseg[0].phys_addr = rte_mem_virt2phy(addr); + mcfg->memseg[0].addr = addr; + mcfg->memseg[0].hugepage_sz = hpi->hugepage_sz; + mcfg->memseg[0].len = internal_config.memory; + mcfg->memseg[0].socket_id = 0; + mcfg->memseg[0].fd = fd; + return 0; + } + /* check if app runs on Xen Dom0 */ if (internal_config.xen_dom0_support) { #ifdef RTE_LIBRTE_XEN_DOM0