mbox series

[v2,0/6] add support for CDX bus

Message ID 20230413132704.17586-1-nipun.gupta@amd.com (mailing list archive)
Headers
Series add support for CDX bus |

Message

Gupta, Nipun April 13, 2023, 1:26 p.m. UTC 
  Support AMD CDX bus, for FPGA based CDX devices. The CDX 
devices are memory mapped on system bus for embedded CPUs.

It uses sysfs interface and the vfio-cdx driver to discover
and initialize the CDX devices.

The patches are intended for DPDK 23.07 release, and have been sent
as an RFC as patches are yet to be merged in Linux.

The CDX bus and VFIO support is available at Xilinx open source tree:
https://github.com/Xilinx/linux-xlnx (drivers/cdx/ and drivers/vfio/cdx)

Linux CDX bus patches has been added into linux next:
https://git.kernel.org/pub/scm/linux/kernel/git/next/linux-next.git/tree/drivers/cdx

VFIO patches are also submitted in upstream:
https://www.spinics.net/lists/kvm/msg310623.html

CDX is a Hardware Architecture designed for AMD FPGA devices. It
consists of mechanism for interaction between FPGA, Firmware and 
the APUs (Application CPUs).
Firmware resides on RPU (Realtime CPUs) which interacts with
the FPGA program manager and the APUs. The RPU provides memory-mapped
interface (RPU if) which is used to communicate with APUs.

VFIO CDX driver provides the CDX device resources like MMIO and interrupts
to map to user-space. DPDK CDX bus uses sysfs interface and the vfio-cdx
driver to discover and initialize the CDX devices for user-space
applications.

Changes v1->v2:
- Moved file rte_cdx_bus.h to internal bus_cdx_driver.h
  and added this file to deivce_cdx_headers
- Moved cdx.h to private.h
- Removed rte_ prefix from the static symbols in .c files.

Changes RFC->v1:
- Marked few API's as internal which were not required
  to be provided to user.

Nipun Gupta (6):
  bus/cdx: introduce cdx bus
  bus/cdx: add dma map and unmap support
  bus/cdx: add support for MSI
  bus/cdx: support plug unplug and dev iterator
  bus: enable cdx bus
  config/arm: add AMD CDX

 MAINTAINERS                            |   5 +
 config/arm/arm64_cdx_linux_gcc         |  17 +
 config/arm/meson.build                 |  14 +
 drivers/bus/cdx/bus_cdx_driver.h       | 227 ++++++++
 drivers/bus/cdx/cdx.c                  | 694 +++++++++++++++++++++++++
 drivers/bus/cdx/cdx_logs.h             |  37 ++
 drivers/bus/cdx/cdx_vfio.c             | 619 ++++++++++++++++++++++
 drivers/bus/cdx/meson.build            |  12 +
 drivers/bus/cdx/private.h              |  49 ++
 drivers/bus/cdx/version.map            |  13 +
 drivers/bus/meson.build                |   1 +
 lib/eal/common/eal_common_interrupts.c |  21 +
 lib/eal/common/eal_interrupts.h        |   1 +
 lib/eal/include/rte_interrupts.h       |  32 ++
 lib/eal/version.map                    |   2 +
 15 files changed, 1744 insertions(+)
 create mode 100644 config/arm/arm64_cdx_linux_gcc
 create mode 100644 drivers/bus/cdx/bus_cdx_driver.h
 create mode 100644 drivers/bus/cdx/cdx.c
 create mode 100644 drivers/bus/cdx/cdx_logs.h
 create mode 100644 drivers/bus/cdx/cdx_vfio.c
 create mode 100644 drivers/bus/cdx/meson.build
 create mode 100644 drivers/bus/cdx/private.h
 create mode 100644 drivers/bus/cdx/version.map

Comments

Ferruh Yigit April 14, 2023, 4:45 p.m. UTC | #1 
On 4/13/2023 2:26 PM, Nipun Gupta wrote:
> Support AMD CDX bus, for FPGA based CDX devices. The CDX 
> devices are memory mapped on system bus for embedded CPUs.
> 
> It uses sysfs interface and the vfio-cdx driver to discover
> and initialize the CDX devices.
> 
> The patches are intended for DPDK 23.07 release, and have been sent
> as an RFC as patches are yet to be merged in Linux.
> 

not RFC anymore

> The CDX bus and VFIO support is available at Xilinx open source tree:
> https://github.com/Xilinx/linux-xlnx (drivers/cdx/ and drivers/vfio/cdx)
> 
> Linux CDX bus patches has been added into linux next:
> https://git.kernel.org/pub/scm/linux/kernel/git/next/linux-next.git/tree/drivers/cdx
> 
> VFIO patches are also submitted in upstream:
> https://www.spinics.net/lists/kvm/msg310623.html
> 
> CDX is a Hardware Architecture designed for AMD FPGA devices. It
> consists of mechanism for interaction between FPGA, Firmware and 
> the APUs (Application CPUs).
> Firmware resides on RPU (Realtime CPUs) which interacts with
> the FPGA program manager and the APUs. The RPU provides memory-mapped
> interface (RPU if) which is used to communicate with APUs.
> 
> VFIO CDX driver provides the CDX device resources like MMIO and interrupts
> to map to user-space. DPDK CDX bus uses sysfs interface and the vfio-cdx
> driver to discover and initialize the CDX devices for user-space
> applications.
> 

Overall looks good to me,
there are a few warnings by `check-git-log.sh`, can you please check?

Also what do you think to add a release notes update for new bus?

> Changes v1->v2:
> - Moved file rte_cdx_bus.h to internal bus_cdx_driver.h
>   and added this file to deivce_cdx_headers
> - Moved cdx.h to private.h
> - Removed rte_ prefix from the static symbols in .c files.
> 
> Changes RFC->v1:
> - Marked few API's as internal which were not required
>   to be provided to user.
> 
> Nipun Gupta (6):
>   bus/cdx: introduce cdx bus
>   bus/cdx: add dma map and unmap support
>   bus/cdx: add support for MSI
>   bus/cdx: support plug unplug and dev iterator
>   bus: enable cdx bus
>   config/arm: add AMD CDX
> 
>  MAINTAINERS                            |   5 +
>  config/arm/arm64_cdx_linux_gcc         |  17 +
>  config/arm/meson.build                 |  14 +
>  drivers/bus/cdx/bus_cdx_driver.h       | 227 ++++++++
>  drivers/bus/cdx/cdx.c                  | 694 +++++++++++++++++++++++++
>  drivers/bus/cdx/cdx_logs.h             |  37 ++
>  drivers/bus/cdx/cdx_vfio.c             | 619 ++++++++++++++++++++++
>  drivers/bus/cdx/meson.build            |  12 +
>  drivers/bus/cdx/private.h              |  49 ++
>  drivers/bus/cdx/version.map            |  13 +
>  drivers/bus/meson.build                |   1 +
>  lib/eal/common/eal_common_interrupts.c |  21 +
>  lib/eal/common/eal_interrupts.h        |   1 +
>  lib/eal/include/rte_interrupts.h       |  32 ++
>  lib/eal/version.map                    |   2 +
>  15 files changed, 1744 insertions(+)
>  create mode 100644 config/arm/arm64_cdx_linux_gcc
>  create mode 100644 drivers/bus/cdx/bus_cdx_driver.h
>  create mode 100644 drivers/bus/cdx/cdx.c
>  create mode 100644 drivers/bus/cdx/cdx_logs.h
>  create mode 100644 drivers/bus/cdx/cdx_vfio.c
>  create mode 100644 drivers/bus/cdx/meson.build
>  create mode 100644 drivers/bus/cdx/private.h
>  create mode 100644 drivers/bus/cdx/version.map
>