From patchwork Tue Aug 11 14:58:07 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Vikas Gupta X-Patchwork-Id: 75626 X-Patchwork-Delegate: gakhil@marvell.com Return-Path: X-Original-To: patchwork@inbox.dpdk.org Delivered-To: patchwork@inbox.dpdk.org Received: from dpdk.org (dpdk.org [92.243.14.124]) by inbox.dpdk.org (Postfix) with ESMTP id 8BFFDA034C; Tue, 18 Aug 2020 15:47:23 +0200 (CEST) Received: from [92.243.14.124] (localhost [127.0.0.1]) by dpdk.org (Postfix) with ESMTP id 28AB51C05C; Tue, 18 Aug 2020 15:47:06 +0200 (CEST) Received: from mail-wm1-f66.google.com (mail-wm1-f66.google.com [209.85.128.66]) by dpdk.org (Postfix) with ESMTP id 249331C025 for ; Tue, 11 Aug 2020 16:58:38 +0200 (CEST) Received: by mail-wm1-f66.google.com with SMTP id t14so3141028wmi.3 for ; Tue, 11 Aug 2020 07:58:38 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=broadcom.com; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references; bh=1sFyVdCh7YYtKQih9jVAKvudJAxKiY4xm7o8j5R+lJw=; b=OxWY9Yex4l8UEifIA4cZhwArQY42lxtiN/2mkpTppffX5HM0j+9046RQmj2AT8qclp fYf/5cT043tYFWsJL2T14en6wCiVxN4ZMkM/zoHND9uBbCPCc85uEYMqDPlih5to0MAf Fn+BRXpLRCz/vqi0U2l2j6nzUL6n7Gga26hcw= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references; bh=1sFyVdCh7YYtKQih9jVAKvudJAxKiY4xm7o8j5R+lJw=; b=LBY7Ds1gRPmGyqR5vf8AvkTd8MMmuvBAhderDwJ+6ZnFJp9M6Dv1u4gzIgZZ+pdZHH XNMdCiAVv2jK2mk/2fW32jTgZUSBSaOlc0v0SsZKCgtujS9Sw8AZFA+rzn8iLrbKZi8B 8TjK1xJpktJxCIyV4yZx0usKOH3djwdyP8rGiQ42NYL22RW9F7ZZgAQ7/Z+zGw/62AgB LirVzd5bTEr8LFfnh7apLNYdsxZOoyehwyqC16BeOudNdUh17aAhz8syCb16fzNewVIg e6vIIPuWwY1aGNIpJzLZExqrIYvjcP5Ldj1KFGk9m8YtoygQ4xvGRvN2Z6VeICML3TIO ding== X-Gm-Message-State: AOAM532tCGvQ+NgncAYIJ9THjqWyYRO2JEYWg4uoTQdsCHuLGIJXlOND OQR1dbVyE93XfGuUTbzYhqA4zz19IdwWVVEQxwUEtzITLpaU2PnUzjbi/XV7xa+XA36jS1aXCT0 0KqGuETO0uvss2ecBW57GSLyszkn5noX63J+JKXuVOdfTQ+XVS9rS1PP6eQ== X-Google-Smtp-Source: ABdhPJzkUanSjLRgAq+UiyOtqoun7C2UAgpGJ2v1X4Q1TdI3vvP1SINHq559UR6nFNBjIUNCi4gGyg== X-Received: by 2002:a7b:c7c6:: with SMTP id z6mr4617976wmk.17.1597157917331; Tue, 11 Aug 2020 07:58:37 -0700 (PDT) Received: from rahul_yocto_ubuntu18.ibn.broadcom.net ([192.19.234.250]) by smtp.gmail.com with ESMTPSA id f15sm5379862wmj.39.2020.08.11.07.58.35 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 11 Aug 2020 07:58:37 -0700 (PDT) From: Vikas Gupta To: dev@dpdk.org, akhil.goyal@nxp.com, ajit.khaparde@broadcom.com Cc: vikram.prakash@broadcom.com, Vikas Gupta Date: Tue, 11 Aug 2020 20:28:07 +0530 Message-Id: <20200811145813.44754-3-vikas.gupta@broadcom.com> X-Mailer: git-send-email 2.17.1 In-Reply-To: <20200811145813.44754-1-vikas.gupta@broadcom.com> References: <20200811145813.44754-1-vikas.gupta@broadcom.com> X-Mailman-Approved-At: Tue, 18 Aug 2020 15:47:01 +0200 Subject: [dpdk-dev] [PATCH 0 2/8] crypto/bcmfs: add vfio support X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Sender: "dev" Add vfio support for device. Signed-off-by: Vikas Gupta --- drivers/crypto/bcmfs/Makefile | 1 + drivers/crypto/bcmfs/bcmfs_device.c | 5 ++ drivers/crypto/bcmfs/bcmfs_device.h | 6 ++ drivers/crypto/bcmfs/bcmfs_vfio.c | 94 +++++++++++++++++++++++++++++ drivers/crypto/bcmfs/bcmfs_vfio.h | 17 ++++++ drivers/crypto/bcmfs/meson.build | 3 +- 6 files changed, 125 insertions(+), 1 deletion(-) create mode 100644 drivers/crypto/bcmfs/bcmfs_vfio.c create mode 100644 drivers/crypto/bcmfs/bcmfs_vfio.h diff --git a/drivers/crypto/bcmfs/Makefile b/drivers/crypto/bcmfs/Makefile index 781ee6efa..5f691f7ba 100644 --- a/drivers/crypto/bcmfs/Makefile +++ b/drivers/crypto/bcmfs/Makefile @@ -19,6 +19,7 @@ CFLAGS += -DALLOW_EXPERIMENTAL_API # SRCS-y += bcmfs_logs.c SRCS-y += bcmfs_device.c +SRCS-y += bcmfs_vfio.c LDLIBS += -lrte_eal -lrte_bus_vdev diff --git a/drivers/crypto/bcmfs/bcmfs_device.c b/drivers/crypto/bcmfs/bcmfs_device.c index 47c776de6..3b5cc9e98 100644 --- a/drivers/crypto/bcmfs/bcmfs_device.c +++ b/drivers/crypto/bcmfs/bcmfs_device.c @@ -11,6 +11,7 @@ #include "bcmfs_device.h" #include "bcmfs_logs.h" +#include "bcmfs_vfio.h" struct bcmfs_device_attr { const char name[BCMFS_MAX_PATH_LEN]; @@ -71,6 +72,10 @@ fsdev_allocate_one_dev(struct rte_vdev_device *vdev, fsdev->vdev = vdev; + /* attach to VFIO */ + if (bcmfs_attach_vfio(fsdev)) + goto cleanup; + TAILQ_INSERT_TAIL(&fsdev_list, fsdev, next); return fsdev; diff --git a/drivers/crypto/bcmfs/bcmfs_device.h b/drivers/crypto/bcmfs/bcmfs_device.h index 4b0c6d3ca..5232bdea5 100644 --- a/drivers/crypto/bcmfs/bcmfs_device.h +++ b/drivers/crypto/bcmfs/bcmfs_device.h @@ -35,6 +35,12 @@ struct bcmfs_device { char name[BCMFS_DEV_NAME_LEN]; /* Parent vdev */ struct rte_vdev_device *vdev; + /* vfio handle */ + int vfio_dev_fd; + /* mapped address */ + uint8_t *mmap_addr; + /* mapped size */ + uint32_t mmap_size; }; #endif /* _BCMFS_DEV_H_ */ diff --git a/drivers/crypto/bcmfs/bcmfs_vfio.c b/drivers/crypto/bcmfs/bcmfs_vfio.c new file mode 100644 index 000000000..9138f96eb --- /dev/null +++ b/drivers/crypto/bcmfs/bcmfs_vfio.c @@ -0,0 +1,94 @@ +/* SPDX-License-Identifier: BSD-3-Clause + * Copyright(C) 2020 Broadcom. + * All rights reserved. + */ + +#include +#include +#include + +#include + +#include "bcmfs_device.h" +#include "bcmfs_logs.h" +#include "bcmfs_vfio.h" + +static int +vfio_map_dev_obj(const char *path, const char *dev_obj, + uint32_t *size, void **addr, int *dev_fd) +{ + int32_t ret; + struct vfio_group_status status = { .argsz = sizeof(status) }; + + struct vfio_device_info d_info = { .argsz = sizeof(d_info) }; + struct vfio_region_info reg_info = { .argsz = sizeof(reg_info) }; + + ret = rte_vfio_setup_device(path, dev_obj, dev_fd, &d_info); + if (ret) { + BCMFS_LOG(ERR, "VFIO Setting for device failed"); + return ret; + } + + /* getting device region info*/ + ret = ioctl(*dev_fd, VFIO_DEVICE_GET_REGION_INFO, ®_info); + if (ret < 0) { + BCMFS_LOG(ERR, "Error in VFIO getting REGION_INFO"); + goto map_failed; + } + + *addr = mmap(NULL, reg_info.size, + PROT_WRITE | PROT_READ, MAP_SHARED, + *dev_fd, reg_info.offset); + if (*addr == MAP_FAILED) { + BCMFS_LOG(ERR, "Error mapping region (errno = %d)", errno); + ret = errno; + goto map_failed; + } + *size = reg_info.size; + + return 0; + +map_failed: + rte_vfio_release_device(path, dev_obj, *dev_fd); + + return ret; +} + +int +bcmfs_attach_vfio(struct bcmfs_device *dev) +{ + int ret; + int vfio_dev_fd; + void *v_addr = NULL; + uint32_t size = 0; + + ret = vfio_map_dev_obj(dev->dirname, dev->name, + &size, &v_addr, &vfio_dev_fd); + if (ret) + return -1; + + dev->mmap_size = size; + dev->mmap_addr = v_addr; + dev->vfio_dev_fd = vfio_dev_fd; + + return 0; +} + +void +bcmfs_release_vfio(struct bcmfs_device *dev) +{ + int ret; + + if (dev == NULL) + return; + + /* unmap the addr */ + munmap(dev->mmap_addr, dev->mmap_size); + /* release the device */ + ret = rte_vfio_release_device(dev->dirname, dev->name, + dev->vfio_dev_fd); + if (ret < 0) { + BCMFS_LOG(ERR, "cannot release device"); + return; + } +} diff --git a/drivers/crypto/bcmfs/bcmfs_vfio.h b/drivers/crypto/bcmfs/bcmfs_vfio.h new file mode 100644 index 000000000..d0fdf6483 --- /dev/null +++ b/drivers/crypto/bcmfs/bcmfs_vfio.h @@ -0,0 +1,17 @@ +/* SPDX-License-Identifier: BSD-3-Clause + * Copyright(c) 2020 Broadcom + * All rights reserved. + */ + +#ifndef _BCMFS_VFIO_H_ +#define _BCMFS_VFIO_H_ + +/* Attach the bcmfs device to vfio */ +int +bcmfs_attach_vfio(struct bcmfs_device *dev); + +/* Release the bcmfs device from vfio */ +void +bcmfs_release_vfio(struct bcmfs_device *dev); + +#endif /* _BCMFS_VFIO_H_ */ diff --git a/drivers/crypto/bcmfs/meson.build b/drivers/crypto/bcmfs/meson.build index a4bdd8ee5..fd39eba20 100644 --- a/drivers/crypto/bcmfs/meson.build +++ b/drivers/crypto/bcmfs/meson.build @@ -6,5 +6,6 @@ deps += ['eal', 'bus_vdev'] sources = files( 'bcmfs_logs.c', - 'bcmfs_device.c' + 'bcmfs_device.c', + 'bcmfs_vfio.c' )