From patchwork Fri Nov 14 02:30:40 2014 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Tetsuya Mukawa X-Patchwork-Id: 1268 Return-Path: X-Original-To: patchwork@dpdk.org Delivered-To: patchwork@dpdk.org Received: from [92.243.14.124] (localhost [IPv6:::1]) by dpdk.org (Postfix) with ESMTP id 202AB7F25; Fri, 14 Nov 2014 03:20:45 +0100 (CET) Received: from mail-pa0-f48.google.com (mail-pa0-f48.google.com [209.85.220.48]) by dpdk.org (Postfix) with ESMTP id F3F5B7E75 for ; Fri, 14 Nov 2014 03:20:42 +0100 (CET) Received: by mail-pa0-f48.google.com with SMTP id rd3so2950685pab.21 for ; Thu, 13 Nov 2014 18:30:43 -0800 (PST) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20130820; h=x-gm-message-state:message-id:date:from:user-agent:mime-version:to :cc:subject:references:in-reply-to:content-type :content-transfer-encoding; bh=HM0E8TO7sIlq8VXRsSVLJHDK8+aVsDCLsgiTs2KC3go=; b=IJXzBDlOilB4WB+bklzET7ZlalVkOZhV8anZVefaSxNXdtP4vGFKlrbBbgG8ItgY7S j31ANdM6RM0RAVj9de50JOz+UQzdmhz+22KHjsDdnVD/oHHLmZ92CEKGEHOb0h/bV/iY ngGeOBGQY3/nMBG3UTUxSinP+V+4j1IogsEpMlTGjzJzq+xFB3YLDZog85ubSPeMaZ+6 1KF5wisepUZkNTwMkdX5lk3ntq7uXrxPfWEy7fd6hVXzjHzMZp1pMx6GBkpgUCFZ0uUn VvDl5GH1Q2EfQOqgBcrx5ep6a4Pv/Aq8LIWpfL0iENcDDcOw1dkV6iRiLl3zgmP9aesq N0bg== X-Gm-Message-State: ALoCoQln+pyr/AuTha5Npr52zpN4VDjUanAWuu693ZclfVhPeyEfXCgfGFb8lUfE4y0bx1/GuKoA X-Received: by 10.66.124.130 with SMTP id mi2mr7084815pab.104.1415932243179; Thu, 13 Nov 2014 18:30:43 -0800 (PST) Received: from [10.16.129.101] (napt.igel.co.jp. [219.106.231.132]) by mx.google.com with ESMTPSA id ql6sm18703596pbb.39.2014.11.13.18.30.41 for (version=TLSv1 cipher=ECDHE-RSA-RC4-SHA bits=128/128); Thu, 13 Nov 2014 18:30:42 -0800 (PST) Message-ID: <54656950.1050204@igel.co.jp> Date: Fri, 14 Nov 2014 11:30:40 +0900 From: Tetsuya Mukawa User-Agent: Mozilla/5.0 (Windows NT 6.1; WOW64; rv:24.0) Gecko/20100101 Thunderbird/24.6.0 MIME-Version: 1.0 To: Linhaifeng , "Xie, Huawei" References: <5462DE39.1070006@igel.co.jp> <54645007.3010301@huawei.com> In-Reply-To: <54645007.3010301@huawei.com> Cc: "dev@dpdk.org" Subject: Re: [dpdk-dev] vhost-user technical isssues X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: patches and discussions about DPDK List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Sender: "dev" Hi Lin, (2014/11/13 15:30), Linhaifeng wrote: > On 2014/11/12 12:12, Tetsuya Mukawa wrote: >> Hi Xie, >> >> (2014/11/12 6:37), Xie, Huawei wrote: >>> Hi Tetsuya: >>> There are two major technical issues in my mind for vhost-user implementation. >>> >>> 1) memory region map >>> Vhost-user passes us file fd and offset for each memory region. Unfortunately the mmap offset is "very" wrong. I discovered this issue long time ago, and also found >>> that I couldn't mmap the huge page file even with correct offset(need double check). >>> Just now I find that people reported this issue on Nov 3. >>> [Qemu-devel] [PULL 27/29] vhost-user: fix mmap offset calculation >>> Anyway, I turned to the same idea used in our DPDK vhost-cuse: only use the fd for region(0) to map the whole file. >>> I think we should use this way temporarily to support qemu-2.1 as it has that bug. >> I agree with you. >> Also we may have an issue about un-mapping file on hugetlbfs of linux. >> When I check munmap(), it seems 'size' need to be aligned by hugepage size. >> (I guess it may be a kernel bug. Might be fixed already.) >> Please add return value checking code for munmap(). >> Still munmap() might be failed. >> > are you munmmap the region 0? region 0 is not need to mmap so not need to munmap too. > > I can munmap success with the other regions. Could you please let me know how many size do you specify when you munmap region1? I still fail to munmap region1. Here is a patch to vhost-user test of QEMU. Could you please check it? ---------------------------------- #define QEMU_CMD_CHR " -chardev socket,id=chr0,path=%s" #define QEMU_CMD_NETDEV " -netdev vhost-user,id=net0,chardev=chr0,vhostforce" @@ -221,14 +221,16 @@ static void read_guest_mem(void) /* check for sanity */ g_assert_cmpint(fds_num, >, 0); - g_assert_cmpint(fds_num, ==, memory.nregions); + //g_assert_cmpint(fds_num, ==, memory.nregions); + fprintf(stderr, "%s(%d)\n", __func__, __LINE__); /* iterate all regions */ for (i = 0; i < fds_num; i++) { + int ret = 0; /* We'll check only the region statring at 0x0*/ if (memory.regions[i].guest_phys_addr != 0x0) { - continue; + //continue; } g_assert_cmpint(memory.regions[i].memory_size, >, 1024); @@ -237,6 +239,13 @@ static void read_guest_mem(void) guest_mem = mmap(0, size, PROT_READ | PROT_WRITE, MAP_SHARED, fds[i], 0); + fprintf(stderr, "guest_phys_addr=%lu, memory_size=%lu, " + "userspace_addr=%lu, mmap_offset=%lu\n", + memory.regions[i].guest_phys_addr, + memory.regions[i].memory_size, + memory.regions[i].userspace_addr, + memory.regions[i].mmap_offset); + fprintf(stderr, "mmap=%p, size=%lu\n", guest_mem, size); g_assert(guest_mem != MAP_FAILED); guest_mem += (memory.regions[i].mmap_offset / sizeof(*guest_mem)); @@ -248,7 +257,20 @@ static void read_guest_mem(void) g_assert_cmpint(a, ==, b); } - munmap(guest_mem, memory.regions[i].memory_size); + ret = munmap(guest_mem, memory.regions[i].memory_size); + fprintf(stderr, "munmap=%p, size=%lu, ret=%d\n", + guest_mem, memory.regions[i].memory_size, ret); + { + size_t hugepagesize; + + size = memory.regions[i].memory_size; + /* assume hugepage size is 1GB, try again */ + hugepagesize = 1024 * 1024 * 1024; + size = (size + hugepagesize - 1) / hugepagesize * hugepagesize; + } + ret = munmap(guest_mem, size); + fprintf(stderr, "munmap=%p, size=%lu, ret=%d\n", + guest_mem, size, ret); } g_assert_cmpint(1, ==, 1); ---------------------------------- $ sudo QTEST_HUGETLBFS_PATH=/mnt/huge make check region=0, mmap=0x2aaac0000000, size=6291456000 region=0, munmap=0x2aab80000000, size=3070230528, ret=-1 << failed region=0, munmap=0x2aab80000000, size=3221225472, ret=0 region=1, mmap=0x2aab80000000, size=655360 region=1, munmap=0x2aab80000000, size=655360, ret=-1 << failed region=1, munmap=0x2aab80000000, size=1073741824, ret=0 Thanks, Tetsuya diff --git a/tests/vhost-user-test.c b/tests/vhost-user-test.c index 75fedf0..4e17910 100644 --- a/tests/vhost-user-test.c +++ b/tests/vhost-user-test.c @@ -37,7 +37,7 @@ #endif #define QEMU_CMD_ACCEL " -machine accel=tcg" -#define QEMU_CMD_MEM " -m 512 -object memory-backend-file,id=mem,size=512M,"\ +#define QEMU_CMD_MEM " -m 6000 -object memory-backend-file,id=mem,size=6000M,"\ "mem-path=%s,share=on -numa node,memdev=mem"