From patchwork Fri Sep 16 09:02:02 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: David Marchand X-Patchwork-Id: 116393 X-Patchwork-Delegate: maxime.coquelin@redhat.com Return-Path: X-Original-To: patchwork@inbox.dpdk.org Delivered-To: patchwork@inbox.dpdk.org Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id 4DDC5A0032; Fri, 16 Sep 2022 11:02:24 +0200 (CEST) Received: from [217.70.189.124] (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id 2CA844021D; Fri, 16 Sep 2022 11:02:24 +0200 (CEST) Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.129.124]) by mails.dpdk.org (Postfix) with ESMTP id 8787A40156 for ; Fri, 16 Sep 2022 11:02:22 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1663318942; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding; bh=mvgF2+pCZMfFbR2qEa4ft21K3y4nYW3h+hAP+3LadQY=; b=hwYwhlm5/hZN7JnEYJqDq9TkDntvgCwLL5rLOZ+Pu3PA62Q56b5RS9RrjdHfyWLRKmeUST 9tiAghtyp5MZVwrlmBAD1KwmELFNkMxRBaQal1mvSF/e2kjV8s/aNrhY2hPb6KQUd+6+iT q5AAzALy8rUJW9PQbKMnlRJeG3fFy1I= Received: from mimecast-mx02.redhat.com (mimecast-mx02.redhat.com [66.187.233.88]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id us-mta-39-pJrSrZymPYOZhGhcBEAUcA-1; Fri, 16 Sep 2022 05:02:19 -0400 X-MC-Unique: pJrSrZymPYOZhGhcBEAUcA-1 Received: from smtp.corp.redhat.com (int-mx09.intmail.prod.int.rdu2.redhat.com [10.11.54.9]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx02.redhat.com (Postfix) with ESMTPS id A88DD862FDC; Fri, 16 Sep 2022 09:02:18 +0000 (UTC) Received: from localhost.localdomain (unknown [10.40.194.61]) by smtp.corp.redhat.com (Postfix) with ESMTP id CD680492CA2; Fri, 16 Sep 2022 09:02:17 +0000 (UTC) From: David Marchand To: dev@dpdk.org Cc: Maxime Coquelin , Chenbo Xia Subject: [PATCH] vhost: reduce memory footprint when IOMMU is disabled Date: Fri, 16 Sep 2022 11:02:02 +0200 Message-Id: <20220916090202.1190834-1-david.marchand@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 3.1 on 10.11.54.9 X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org If an application does not request IOMMU support, we can avoid allocating a IOMMU pool. This saves 112kB (IOTLB_CACHE_SIZE * sizeof(struct vhost_iotlb_entry)) per vq. Signed-off-by: David Marchand Reviewed-by: Chenbo Xia --- lib/vhost/iotlb.c | 20 +++++++++++--------- lib/vhost/socket.c | 4 +++- lib/vhost/vhost.c | 7 ++++++- lib/vhost/vhost.h | 5 ++++- 4 files changed, 24 insertions(+), 12 deletions(-) diff --git a/lib/vhost/iotlb.c b/lib/vhost/iotlb.c index 2a78929e78..6a729e8804 100644 --- a/lib/vhost/iotlb.c +++ b/lib/vhost/iotlb.c @@ -341,16 +341,18 @@ vhost_user_iotlb_init(struct virtio_net *dev, struct vhost_virtqueue *vq) TAILQ_INIT(&vq->iotlb_list); TAILQ_INIT(&vq->iotlb_pending_list); - vq->iotlb_pool = rte_calloc_socket("iotlb", IOTLB_CACHE_SIZE, - sizeof(struct vhost_iotlb_entry), 0, socket); - if (!vq->iotlb_pool) { - VHOST_LOG_CONFIG(dev->ifname, ERR, - "Failed to create IOTLB cache pool for vq %"PRIu32"\n", - vq->index); - return -1; + if (dev->flags & VIRTIO_DEV_SUPPORT_IOMMU) { + vq->iotlb_pool = rte_calloc_socket("iotlb", IOTLB_CACHE_SIZE, + sizeof(struct vhost_iotlb_entry), 0, socket); + if (!vq->iotlb_pool) { + VHOST_LOG_CONFIG(dev->ifname, ERR, + "Failed to create IOTLB cache pool for vq %"PRIu32"\n", + vq->index); + return -1; + } + for (i = 0; i < IOTLB_CACHE_SIZE; i++) + vhost_user_iotlb_pool_put(vq, &vq->iotlb_pool[i]); } - for (i = 0; i < IOTLB_CACHE_SIZE; i++) - vhost_user_iotlb_pool_put(vq, &vq->iotlb_pool[i]); vq->iotlb_cache_nr = 0; diff --git a/lib/vhost/socket.c b/lib/vhost/socket.c index a8df2d484a..608ae577fd 100644 --- a/lib/vhost/socket.c +++ b/lib/vhost/socket.c @@ -228,7 +228,8 @@ vhost_user_add_connection(int fd, struct vhost_user_socket *vsocket) vhost_set_ifname(vid, vsocket->path, size); vhost_setup_virtio_net(vid, vsocket->use_builtin_virtio_net, - vsocket->net_compliant_ol_flags, vsocket->stats_enabled); + vsocket->net_compliant_ol_flags, vsocket->stats_enabled, + vsocket->iommu_support); vhost_attach_vdpa_device(vid, vsocket->vdpa_dev); @@ -905,6 +906,7 @@ rte_vhost_driver_register(const char *path, uint64_t flags) vsocket->async_copy = flags & RTE_VHOST_USER_ASYNC_COPY; vsocket->net_compliant_ol_flags = flags & RTE_VHOST_USER_NET_COMPLIANT_OL_FLAGS; vsocket->stats_enabled = flags & RTE_VHOST_USER_NET_STATS_ENABLE; + vsocket->iommu_support = flags & RTE_VHOST_USER_IOMMU_SUPPORT; if (vsocket->async_copy && (flags & (RTE_VHOST_USER_IOMMU_SUPPORT | diff --git a/lib/vhost/vhost.c b/lib/vhost/vhost.c index aa671f47a3..9e5e69f72d 100644 --- a/lib/vhost/vhost.c +++ b/lib/vhost/vhost.c @@ -771,7 +771,8 @@ vhost_set_ifname(int vid, const char *if_name, unsigned int if_len) } void -vhost_setup_virtio_net(int vid, bool enable, bool compliant_ol_flags, bool stats_enabled) +vhost_setup_virtio_net(int vid, bool enable, bool compliant_ol_flags, bool stats_enabled, + bool support_iommu) { struct virtio_net *dev = get_device(vid); @@ -790,6 +791,10 @@ vhost_setup_virtio_net(int vid, bool enable, bool compliant_ol_flags, bool stats dev->flags |= VIRTIO_DEV_STATS_ENABLED; else dev->flags &= ~VIRTIO_DEV_STATS_ENABLED; + if (support_iommu) + dev->flags |= VIRTIO_DEV_SUPPORT_IOMMU; + else + dev->flags &= ~VIRTIO_DEV_SUPPORT_IOMMU; } void diff --git a/lib/vhost/vhost.h b/lib/vhost/vhost.h index 782d916ae0..3da6f217fd 100644 --- a/lib/vhost/vhost.h +++ b/lib/vhost/vhost.h @@ -37,6 +37,8 @@ #define VIRTIO_DEV_LEGACY_OL_FLAGS ((uint32_t)1 << 5) /* Used to indicate the application has requested statistics collection */ #define VIRTIO_DEV_STATS_ENABLED ((uint32_t)1 << 6) +/* Used to indicate the application has requested iommu support */ +#define VIRTIO_DEV_SUPPORT_IOMMU ((uint32_t)1 << 7) /* Backend value set by guest. */ #define VIRTIO_DEV_STOPPED -1 @@ -803,7 +805,8 @@ int alloc_vring_queue(struct virtio_net *dev, uint32_t vring_idx); void vhost_attach_vdpa_device(int vid, struct rte_vdpa_device *dev); void vhost_set_ifname(int, const char *if_name, unsigned int if_len); -void vhost_setup_virtio_net(int vid, bool enable, bool legacy_ol_flags, bool stats_enabled); +void vhost_setup_virtio_net(int vid, bool enable, bool legacy_ol_flags, bool stats_enabled, + bool support_iommu); void vhost_enable_extbuf(int vid); void vhost_enable_linearbuf(int vid); int vhost_enable_guest_notification(struct virtio_net *dev,