[v23,1/6] dmadev: introduce DMA device library
Checks
Commit Message
The 'dmadevice' is a generic type of DMA device.
This patch introduce the 'dmadevice' device allocation APIs.
The infrastructure is prepared to welcome drivers in drivers/dma/
Signed-off-by: Chengwen Feng <fengchengwen@huawei.com>
Acked-by: Bruce Richardson <bruce.richardson@intel.com>
Acked-by: Morten Brørup <mb@smartsharesystems.com>
Acked-by: Jerin Jacob <jerinjacobk@gmail.com>
Reviewed-by: Kevin Laatz <kevin.laatz@intel.com>
Reviewed-by: Conor Walsh <conor.walsh@intel.com>
---
MAINTAINERS | 5 +
config/rte_config.h | 3 +
doc/api/doxy-api-index.md | 1 +
doc/api/doxy-api.conf.in | 1 +
doc/guides/dmadevs/index.rst | 12 ++
doc/guides/index.rst | 1 +
doc/guides/prog_guide/dmadev.rst | 64 ++++++
doc/guides/prog_guide/img/dmadev.svg | 283 +++++++++++++++++++++++++
doc/guides/prog_guide/index.rst | 1 +
doc/guides/rel_notes/release_21_11.rst | 4 +
drivers/dma/meson.build | 4 +
drivers/meson.build | 1 +
lib/dmadev/meson.build | 7 +
lib/dmadev/rte_dmadev.c | 263 +++++++++++++++++++++++
lib/dmadev/rte_dmadev.h | 134 ++++++++++++
lib/dmadev/rte_dmadev_core.h | 51 +++++
lib/dmadev/rte_dmadev_pmd.h | 60 ++++++
lib/dmadev/version.map | 20 ++
lib/meson.build | 1 +
19 files changed, 916 insertions(+)
create mode 100644 doc/guides/dmadevs/index.rst
create mode 100644 doc/guides/prog_guide/dmadev.rst
create mode 100644 doc/guides/prog_guide/img/dmadev.svg
create mode 100644 drivers/dma/meson.build
create mode 100644 lib/dmadev/meson.build
create mode 100644 lib/dmadev/rte_dmadev.c
create mode 100644 lib/dmadev/rte_dmadev.h
create mode 100644 lib/dmadev/rte_dmadev_core.h
create mode 100644 lib/dmadev/rte_dmadev_pmd.h
create mode 100644 lib/dmadev/version.map
Comments
On Fri, Sep 24, 2021 at 3:58 AM Chengwen Feng <fengchengwen@huawei.com> wrote:
>
> The 'dmadevice' is a generic type of DMA device.
>
> This patch introduce the 'dmadevice' device allocation APIs.
>
> The infrastructure is prepared to welcome drivers in drivers/dma/
>
> Signed-off-by: Chengwen Feng <fengchengwen@huawei.com>
> Acked-by: Bruce Richardson <bruce.richardson@intel.com>
> Acked-by: Morten Brørup <mb@smartsharesystems.com>
> Acked-by: Jerin Jacob <jerinjacobk@gmail.com>
> Reviewed-by: Kevin Laatz <kevin.laatz@intel.com>
> Reviewed-by: Conor Walsh <conor.walsh@intel.com>
> ---
> MAINTAINERS | 5 +
> config/rte_config.h | 3 +
> doc/api/doxy-api-index.md | 1 +
> doc/api/doxy-api.conf.in | 1 +
> doc/guides/dmadevs/index.rst | 12 ++
> doc/guides/index.rst | 1 +
> doc/guides/prog_guide/dmadev.rst | 64 ++++++
> doc/guides/prog_guide/img/dmadev.svg | 283 +++++++++++++++++++++++++
> doc/guides/prog_guide/index.rst | 1 +
> doc/guides/rel_notes/release_21_11.rst | 4 +
> drivers/dma/meson.build | 4 +
> drivers/meson.build | 1 +
> lib/dmadev/meson.build | 7 +
> lib/dmadev/rte_dmadev.c | 263 +++++++++++++++++++++++
> lib/dmadev/rte_dmadev.h | 134 ++++++++++++
> lib/dmadev/rte_dmadev_core.h | 51 +++++
> lib/dmadev/rte_dmadev_pmd.h | 60 ++++++
> lib/dmadev/version.map | 20 ++
> lib/meson.build | 1 +
> 19 files changed, 916 insertions(+)
> create mode 100644 doc/guides/dmadevs/index.rst
> create mode 100644 doc/guides/prog_guide/dmadev.rst
> create mode 100644 doc/guides/prog_guide/img/dmadev.svg
> create mode 100644 drivers/dma/meson.build
> create mode 100644 lib/dmadev/meson.build
> create mode 100644 lib/dmadev/rte_dmadev.c
> create mode 100644 lib/dmadev/rte_dmadev.h
> create mode 100644 lib/dmadev/rte_dmadev_core.h
> create mode 100644 lib/dmadev/rte_dmadev_pmd.h
> create mode 100644 lib/dmadev/version.map
>
<snip>
Hi Chengwen,
I see that the new version removed the "rte_dmadev_get_device_by_name()".
What is the way to get the dmadev from inside the PMD .remove ? I am
looking to get the dev_private as we need to do some cleanup
operations from the remove function.
regards,
Radha Mohan
On 04/10/2021 22:12, Radha Mohan wrote:
> On Fri, Sep 24, 2021 at 3:58 AM Chengwen Feng <fengchengwen@huawei.com> wrote:
>> The 'dmadevice' is a generic type of DMA device.
>>
>> This patch introduce the 'dmadevice' device allocation APIs.
>>
>> The infrastructure is prepared to welcome drivers in drivers/dma/
>>
>> Signed-off-by: Chengwen Feng <fengchengwen@huawei.com>
>> Acked-by: Bruce Richardson <bruce.richardson@intel.com>
>> Acked-by: Morten Brørup <mb@smartsharesystems.com>
>> Acked-by: Jerin Jacob <jerinjacobk@gmail.com>
>> Reviewed-by: Kevin Laatz <kevin.laatz@intel.com>
>> Reviewed-by: Conor Walsh <conor.walsh@intel.com>
>> ---
>> MAINTAINERS | 5 +
>> config/rte_config.h | 3 +
>> doc/api/doxy-api-index.md | 1 +
>> doc/api/doxy-api.conf.in | 1 +
>> doc/guides/dmadevs/index.rst | 12 ++
>> doc/guides/index.rst | 1 +
>> doc/guides/prog_guide/dmadev.rst | 64 ++++++
>> doc/guides/prog_guide/img/dmadev.svg | 283 +++++++++++++++++++++++++
>> doc/guides/prog_guide/index.rst | 1 +
>> doc/guides/rel_notes/release_21_11.rst | 4 +
>> drivers/dma/meson.build | 4 +
>> drivers/meson.build | 1 +
>> lib/dmadev/meson.build | 7 +
>> lib/dmadev/rte_dmadev.c | 263 +++++++++++++++++++++++
>> lib/dmadev/rte_dmadev.h | 134 ++++++++++++
>> lib/dmadev/rte_dmadev_core.h | 51 +++++
>> lib/dmadev/rte_dmadev_pmd.h | 60 ++++++
>> lib/dmadev/version.map | 20 ++
>> lib/meson.build | 1 +
>> 19 files changed, 916 insertions(+)
>> create mode 100644 doc/guides/dmadevs/index.rst
>> create mode 100644 doc/guides/prog_guide/dmadev.rst
>> create mode 100644 doc/guides/prog_guide/img/dmadev.svg
>> create mode 100644 drivers/dma/meson.build
>> create mode 100644 lib/dmadev/meson.build
>> create mode 100644 lib/dmadev/rte_dmadev.c
>> create mode 100644 lib/dmadev/rte_dmadev.h
>> create mode 100644 lib/dmadev/rte_dmadev_core.h
>> create mode 100644 lib/dmadev/rte_dmadev_pmd.h
>> create mode 100644 lib/dmadev/version.map
>>
> <snip>
> Hi Chengwen,
> I see that the new version removed the "rte_dmadev_get_device_by_name()".
> What is the way to get the dmadev from inside the PMD .remove ? I am
> looking to get the dev_private as we need to do some cleanup
> operations from the remove function.
>
> regards,
> Radha Mohan
Hi Radha,
You can use rte_dma_get_dev_id(name) to get the device ID, which can
then be used to get the rte_dma_dev struct (which contains dev_private)
for that device from rte_dma_devices[].
See "idxd_dmadev_destroy()" in
http://patches.dpdk.org/project/dpdk/patch/20210924133916.4042773-6-kevin.laatz@intel.com/
for an example.
Hope that helps,
Kevin
On Tue, Oct 5, 2021 at 1:24 AM Kevin Laatz <kevin.laatz@intel.com> wrote:
>
> On 04/10/2021 22:12, Radha Mohan wrote:
> > On Fri, Sep 24, 2021 at 3:58 AM Chengwen Feng <fengchengwen@huawei.com> wrote:
> >> The 'dmadevice' is a generic type of DMA device.
> >>
> >> This patch introduce the 'dmadevice' device allocation APIs.
> >>
> >> The infrastructure is prepared to welcome drivers in drivers/dma/
> >>
> >> Signed-off-by: Chengwen Feng <fengchengwen@huawei.com>
> >> Acked-by: Bruce Richardson <bruce.richardson@intel.com>
> >> Acked-by: Morten Brørup <mb@smartsharesystems.com>
> >> Acked-by: Jerin Jacob <jerinjacobk@gmail.com>
> >> Reviewed-by: Kevin Laatz <kevin.laatz@intel.com>
> >> Reviewed-by: Conor Walsh <conor.walsh@intel.com>
> >> ---
> >> MAINTAINERS | 5 +
> >> config/rte_config.h | 3 +
> >> doc/api/doxy-api-index.md | 1 +
> >> doc/api/doxy-api.conf.in | 1 +
> >> doc/guides/dmadevs/index.rst | 12 ++
> >> doc/guides/index.rst | 1 +
> >> doc/guides/prog_guide/dmadev.rst | 64 ++++++
> >> doc/guides/prog_guide/img/dmadev.svg | 283 +++++++++++++++++++++++++
> >> doc/guides/prog_guide/index.rst | 1 +
> >> doc/guides/rel_notes/release_21_11.rst | 4 +
> >> drivers/dma/meson.build | 4 +
> >> drivers/meson.build | 1 +
> >> lib/dmadev/meson.build | 7 +
> >> lib/dmadev/rte_dmadev.c | 263 +++++++++++++++++++++++
> >> lib/dmadev/rte_dmadev.h | 134 ++++++++++++
> >> lib/dmadev/rte_dmadev_core.h | 51 +++++
> >> lib/dmadev/rte_dmadev_pmd.h | 60 ++++++
> >> lib/dmadev/version.map | 20 ++
> >> lib/meson.build | 1 +
> >> 19 files changed, 916 insertions(+)
> >> create mode 100644 doc/guides/dmadevs/index.rst
> >> create mode 100644 doc/guides/prog_guide/dmadev.rst
> >> create mode 100644 doc/guides/prog_guide/img/dmadev.svg
> >> create mode 100644 drivers/dma/meson.build
> >> create mode 100644 lib/dmadev/meson.build
> >> create mode 100644 lib/dmadev/rte_dmadev.c
> >> create mode 100644 lib/dmadev/rte_dmadev.h
> >> create mode 100644 lib/dmadev/rte_dmadev_core.h
> >> create mode 100644 lib/dmadev/rte_dmadev_pmd.h
> >> create mode 100644 lib/dmadev/version.map
> >>
> > <snip>
> > Hi Chengwen,
> > I see that the new version removed the "rte_dmadev_get_device_by_name()".
> > What is the way to get the dmadev from inside the PMD .remove ? I am
> > looking to get the dev_private as we need to do some cleanup
> > operations from the remove function.
> >
> > regards,
> > Radha Mohan
>
> Hi Radha,
>
> You can use rte_dma_get_dev_id(name) to get the device ID, which can
> then be used to get the rte_dma_dev struct (which contains dev_private)
> for that device from rte_dma_devices[].
>
> See "idxd_dmadev_destroy()" in
> http://patches.dpdk.org/project/dpdk/patch/20210924133916.4042773-6-kevin.laatz@intel.com/
> for an example.
>
> Hope that helps,
Thanks Kevin. It helped. I wasn't looking at accessing the
rte_dma_devices[] array directly. A library API would've been good.
regards,
Radha
>
> Kevin
>
>
24/09/2021 12:53, Chengwen Feng:
> The 'dmadevice' is a generic type of DMA device.
Do you mean 'dmadev' ?
> This patch introduce the 'dmadevice' device allocation APIs.
>
> The infrastructure is prepared to welcome drivers in drivers/dma/
Good
[...]
> +The DMA library provides a DMA device framework for management and provisioning
> +of hardware and software DMA poll mode drivers, defining generic APIs which
We could consider the whole as *one* API.
> +support a number of different DMA operations.
> +
> +
> +Design Principles
> +-----------------
> +
> +The DMA library follows the same basic principles as those used in DPDK's
> +Ethernet Device framework and the RegEx framework.
Not sure what this sentence means. Better to remove.
> The DMA framework provides
> +a generic DMA device framework which supports both physical (hardware)
> +and virtual (software) DMA devices as well as a generic DMA API which allows
> +DMA devices to be managed and configured and supports DMA operations to be
> +provisioned on DMA poll mode driver.
You could split this long sentence.
[...]
> +Physical DMA controllers are discovered during the PCI probe/enumeration of the
> +EAL function which is executed at DPDK initialization, this is based on their
> +PCI BDF (bus/bridge, device, function). Specific physical DMA controllers, like
> +other physical devices in DPDK can be listed using the EAL command line options.
> +
> +The dmadevs are dynamically allocated by using the API
not API, but function
> +``rte_dma_pmd_allocate`` based on the number of hardware DMA channels. After the
> +dmadev initialized successfully, the driver needs to switch the dmadev state to
> +``RTE_DMA_DEV_READY``.
Are we sure we need these details?
> +Device Identification
> +~~~~~~~~~~~~~~~~~~~~~
> +
> +Each DMA device, whether physical or virtual is uniquely designated by two
> +identifiers:
> +
> +- A unique device index used to designate the DMA device in all functions
> + exported by the DMA API.
> +
> +- A device name used to designate the DMA device in console messages, for
> + administration or debugging purposes.
Good
[...]
> --- a/doc/guides/rel_notes/release_21_11.rst
> +++ b/doc/guides/rel_notes/release_21_11.rst
> @@ -106,6 +106,10 @@ New Features
> Added command-line options to specify total number of processes and
> current process ID. Each process owns subset of Rx and Tx queues.
>
> +* **Introduced dmadev library with:**
> +
> + * Device allocation APIs.
There is no API for that, it is internal.
From an user perspective, you need only to list outstanding features
in the release notes.
[...]
> +++ b/lib/dmadev/rte_dmadev.c
> +RTE_LOG_REGISTER_DEFAULT(rte_dma_logtype, INFO);
> +#define RTE_DMA_LOG(level, fmt, args...) \
> + rte_log(RTE_LOG_ ## level, rte_dma_logtype, "%s(): " fmt "\n", \
> + __func__, ##args)
I don't like having function name in all logs.
I recommend this form of macro:
#define RTE_DMA_LOG(level, ...) \
rte_log(RTE_LOG_ ## level, rte_dma_logtype, RTE_FMT("dma: " \
RTE_FMT_HEAD(__VA_ARGS__,) "\n", RTE_FMT_TAIL(__VA_ARGS__,)))
> +/* Macros to check for valid device id */
> +#define RTE_DMA_VALID_DEV_ID_OR_ERR_RET(dev_id, retval) do { \
> + if (!rte_dma_is_valid(dev_id)) { \
> + RTE_DMA_LOG(ERR, "Invalid dev_id=%d", dev_id); \
> + return retval; \
> + } \
> +} while (0)
I dislike this macro doing a return. It is hiding stuff.
I know we have it in other classes but I think it is a mistake,
we should avoid macro blocks.
> +static int16_t
> +dma_find_free_dev(void)
Actually it is looking for an ID,
so it should be dma_find_free_id.
> +{
> + int16_t i;
> +
> + if (rte_dma_devices == NULL)
> + return -1;
> +
> + for (i = 0; i < dma_devices_max; i++) {
> + if (rte_dma_devices[i].dev_name[0] == '\0')
Instead of checking its name, it looks more logical to check the state.
> + return i;
> + }
> +
> + return -1;
> +}
> +
> +static struct rte_dma_dev*
> +dma_find(const char *name)
dma_find_by_name?
[...]
> +++ b/lib/dmadev/rte_dmadev.h
> + * The dmadev are dynamically allocated by rte_dma_pmd_allocate() during the
> + * PCI/SoC device probing phase performed at EAL initialization time. And could
> + * be released by rte_dma_pmd_release() during the PCI/SoC device removing
> + * phase.
I don't think this text has value,
and we could imagine allocating a device ata later stage.
[...]
> + * Configure the maximum number of dmadevs.
> + * @note This function can be invoked before the primary process rte_eal_init()
> + * to change the maximum number of dmadevs.
You should mention what is the default.
Is the default exported to the app in this file?
> + *
> + * @param dev_max
> + * maximum number of dmadevs.
> + *
> + * @return
> + * 0 on success. Otherwise negative value is returned.
> + */
> +__rte_experimental
> +int rte_dma_dev_max(size_t dev_max);
What about a function able to do more with the name rte_dma_init?
It should allocate the inter-process shared memory,
and do the lookup in case of secondary process.
> +/**
> + * @warning
> + * @b EXPERIMENTAL: this API may change without prior notice.
> + *
> + * Get the device identifier for the named DMA device.
> + *
> + * @param name
> + * DMA device name.
> + *
> + * @return
> + * Returns DMA device identifier on success.
> + * - <0: Failure to find named DMA device.
> + */
> +__rte_experimental
> +int rte_dma_get_dev_id(const char *name);
Should we add _by_name?
We could have a function to retrieve the ID by devargs as well.
> +++ b/lib/dmadev/rte_dmadev_core.h
> +/**
> + * @file
> + *
> + * DMA Device internal header.
> + *
> + * This header contains internal data types, that are used by the DMA devices
> + * in order to expose their ops to the class.
> + *
> + * Applications should not use these API directly.
If it is not part of the API, it should not be exposed at all.
Why not having all these stuff in a file dmadev_driver.h?
Is it used by some inline functions?
[...]
> +++ b/lib/dmadev/rte_dmadev_pmd.h
> +/**
> + * @file
> + *
> + * DMA Device PMD APIs
> + *
> + * Driver facing APIs for a DMA device. These are not to be called directly by
You cannot say API for drivers, because API means application interface.
What you mean is "driver interface".
> + * any application.
> + */
[...]
> + * Allocates a new dmadev slot for an DMA device and returns the pointer
> + * to that slot for the driver to use.
Please in all comments, use the infinitive form. Example:
Allocates -> Allocate
> + *
> + * @param name
> + * DMA device name.
> + * @param numa_node
> + * Driver's private data's numa node.
s/numa/NUMA/
> + * @param private_data_size
> + * Driver's private data size.
> + *
> + * @return
> + * A pointer to the DMA device slot case of success,
> + * NULL otherwise.
> + */
> +__rte_internal
> +struct rte_dma_dev *rte_dma_pmd_allocate(const char *name, int numa_node,
> + size_t private_data_size);
OK, sorry there are a lot of comments.
Overrall, that's a good work.
I know you are in holidays, I hope we can finish during next week.
On 2021/10/5 5:12, Radha Mohan wrote:
> On Fri, Sep 24, 2021 at 3:58 AM Chengwen Feng <fengchengwen@huawei.com> wrote:
>>
>> The 'dmadevice' is a generic type of DMA device.
>>
>> This patch introduce the 'dmadevice' device allocation APIs.
>>
>> The infrastructure is prepared to welcome drivers in drivers/dma/
>>
>> Signed-off-by: Chengwen Feng <fengchengwen@huawei.com>
>> Acked-by: Bruce Richardson <bruce.richardson@intel.com>
>> Acked-by: Morten Brørup <mb@smartsharesystems.com>
>> Acked-by: Jerin Jacob <jerinjacobk@gmail.com>
>> Reviewed-by: Kevin Laatz <kevin.laatz@intel.com>
>> Reviewed-by: Conor Walsh <conor.walsh@intel.com>
>> ---
>> MAINTAINERS | 5 +
>> config/rte_config.h | 3 +
>> doc/api/doxy-api-index.md | 1 +
>> doc/api/doxy-api.conf.in | 1 +
>> doc/guides/dmadevs/index.rst | 12 ++
>> doc/guides/index.rst | 1 +
>> doc/guides/prog_guide/dmadev.rst | 64 ++++++
>> doc/guides/prog_guide/img/dmadev.svg | 283 +++++++++++++++++++++++++
>> doc/guides/prog_guide/index.rst | 1 +
>> doc/guides/rel_notes/release_21_11.rst | 4 +
>> drivers/dma/meson.build | 4 +
>> drivers/meson.build | 1 +
>> lib/dmadev/meson.build | 7 +
>> lib/dmadev/rte_dmadev.c | 263 +++++++++++++++++++++++
>> lib/dmadev/rte_dmadev.h | 134 ++++++++++++
>> lib/dmadev/rte_dmadev_core.h | 51 +++++
>> lib/dmadev/rte_dmadev_pmd.h | 60 ++++++
>> lib/dmadev/version.map | 20 ++
>> lib/meson.build | 1 +
>> 19 files changed, 916 insertions(+)
>> create mode 100644 doc/guides/dmadevs/index.rst
>> create mode 100644 doc/guides/prog_guide/dmadev.rst
>> create mode 100644 doc/guides/prog_guide/img/dmadev.svg
>> create mode 100644 drivers/dma/meson.build
>> create mode 100644 lib/dmadev/meson.build
>> create mode 100644 lib/dmadev/rte_dmadev.c
>> create mode 100644 lib/dmadev/rte_dmadev.h
>> create mode 100644 lib/dmadev/rte_dmadev_core.h
>> create mode 100644 lib/dmadev/rte_dmadev_pmd.h
>> create mode 100644 lib/dmadev/version.map
>>
> <snip>
> Hi Chengwen,
> I see that the new version removed the "rte_dmadev_get_device_by_name()".
> What is the way to get the dmadev from inside the PMD .remove ? I am
> looking to get the dev_private as we need to do some cleanup
> operations from the remove function.
Hi Radha,
PMD should invoke rte_dma_pmd_release when .remove, and the rte_dma_pmd_release will
call dev_close ops, so that PMD could do some cleanup operations in dev_close ops.
Thanks
>
> regards,
> Radha Mohan
>
> .
>
On 2021/10/6 18:26, Thomas Monjalon wrote:
> 24/09/2021 12:53, Chengwen Feng:
>> The 'dmadevice' is a generic type of DMA device.
>
> Do you mean 'dmadev' ?
>
>> This patch introduce the 'dmadevice' device allocation APIs.
>>
>> The infrastructure is prepared to welcome drivers in drivers/dma/
>
> Good
>
> [...]
>> +The DMA library provides a DMA device framework for management and provisioning
>> +of hardware and software DMA poll mode drivers, defining generic APIs which
[snip]
>
> [...]
>> +++ b/lib/dmadev/rte_dmadev.h
>> + * The dmadev are dynamically allocated by rte_dma_pmd_allocate() during the
>> + * PCI/SoC device probing phase performed at EAL initialization time. And could
>> + * be released by rte_dma_pmd_release() during the PCI/SoC device removing
>> + * phase.
>
> I don't think this text has value,
> and we could imagine allocating a device ata later stage.
Yes, we could remove the stage descriptor because it's a well-known knowledge, but I
recommend keeping the rte_dma_pmd_allocate and rte_dma_pmd_release functions, how about:
* The dmadev are dynamically allocated by rte_dma_pmd_allocate(). And could
* be released by rte_dma_pmd_release().
>
> [...]
>> + * Configure the maximum number of dmadevs.
>> + * @note This function can be invoked before the primary process rte_eal_init()
>> + * to change the maximum number of dmadevs.
>
> You should mention what is the default.
> Is the default exported to the app in this file?
The default macro is RTE_DMADEV_DEFAULT_MAX_DEVS, and I place it at rte_config.h.
I think it's better to focus on one place (rte_config.h) than to modify config in multiple places (e.g. rte_dmadev.h/rte_xxx.h).
>
>> + *
>> + * @param dev_max
>> + * maximum number of dmadevs.
>> + *
>> + * @return
>> + * 0 on success. Otherwise negative value is returned.
>> + */
>> +__rte_experimental
>> +int rte_dma_dev_max(size_t dev_max);
>
> What about a function able to do more with the name rte_dma_init?
> It should allocate the inter-process shared memory,
> and do the lookup in case of secondary process.
Yes, we defined dma_data_prepare() which do above thing, it's in 4th patch.
Because we could not invoke some like allocate inter-process shared memory before
rte_eal_init, so I think it's better keep rte_dma_dev_max as it is.
>
>> +/**
>> + * @warning
>> + * @b EXPERIMENTAL: this API may change without prior notice.
>> + *
>> + * Get the device identifier for the named DMA device.
>> + *
>> + * @param name
>> + * DMA device name.
>> + *
>> + * @return
>> + * Returns DMA device identifier on success.
>> + * - <0: Failure to find named DMA device.
>> + */
>> +__rte_experimental
>> +int rte_dma_get_dev_id(const char *name);
>
> Should we add _by_name?
> We could have a function to retrieve the ID by devargs as well.
>
>> +++ b/lib/dmadev/rte_dmadev_core.h
>> +/**
>> + * @file
>> + *
>> + * DMA Device internal header.
>> + *
>> + * This header contains internal data types, that are used by the DMA devices
>> + * in order to expose their ops to the class.
>> + *
>> + * Applications should not use these API directly.
>
> If it is not part of the API, it should not be exposed at all.
> Why not having all these stuff in a file dmadev_driver.h?
> Is it used by some inline functions?
Yes, it's used by dataplane inline functions.
[snip]
> .
>
Thanks
08/10/2021 09:13, fengchengwen:
> On 2021/10/6 18:26, Thomas Monjalon wrote:
> > 24/09/2021 12:53, Chengwen Feng:
> >> +++ b/lib/dmadev/rte_dmadev.h
> >> + * The dmadev are dynamically allocated by rte_dma_pmd_allocate() during the
> >> + * PCI/SoC device probing phase performed at EAL initialization time. And could
> >> + * be released by rte_dma_pmd_release() during the PCI/SoC device removing
> >> + * phase.
> >
> > I don't think this text has value,
> > and we could imagine allocating a device ata later stage.
>
> Yes, we could remove the stage descriptor because it's a well-known knowledge, but I
> recommend keeping the rte_dma_pmd_allocate and rte_dma_pmd_release functions, how about:
>
> * The dmadev are dynamically allocated by rte_dma_pmd_allocate(). And could
> * be released by rte_dma_pmd_release().
These functions are for PMD.
This file is for applications, so it is not appropriate.
> > [...]
> >> + * Configure the maximum number of dmadevs.
> >> + * @note This function can be invoked before the primary process rte_eal_init()
> >> + * to change the maximum number of dmadevs.
> >
> > You should mention what is the default.
> > Is the default exported to the app in this file?
>
> The default macro is RTE_DMADEV_DEFAULT_MAX_DEVS, and I place it at rte_config.h.
No we avoid adding thinds in rte_config.h.
There should a static default which can be changed at runtime only.
> I think it's better to focus on one place (rte_config.h) than to modify config in multiple places (e.g. rte_dmadev.h/rte_xxx.h).
Config is modified only in one place: the function.
> >> + *
> >> + * @param dev_max
> >> + * maximum number of dmadevs.
> >> + *
> >> + * @return
> >> + * 0 on success. Otherwise negative value is returned.
> >> + */
> >> +__rte_experimental
> >> +int rte_dma_dev_max(size_t dev_max);
> >
> > What about a function able to do more with the name rte_dma_init?
> > It should allocate the inter-process shared memory,
> > and do the lookup in case of secondary process.
>
> Yes, we defined dma_data_prepare() which do above thing, it's in 4th patch.
>
> Because we could not invoke some like allocate inter-process shared memory before
> rte_eal_init, so I think it's better keep rte_dma_dev_max as it is.
Good point.
> >> +++ b/lib/dmadev/rte_dmadev_core.h
> >> +/**
> >> + * @file
> >> + *
> >> + * DMA Device internal header.
> >> + *
> >> + * This header contains internal data types, that are used by the DMA devices
> >> + * in order to expose their ops to the class.
> >> + *
> >> + * Applications should not use these API directly.
> >
> > If it is not part of the API, it should not be exposed at all.
> > Why not having all these stuff in a file dmadev_driver.h?
> > Is it used by some inline functions?
>
> Yes, it's used by dataplane inline functions.
OK, please give this reason in the description.
@@ -454,6 +454,11 @@ F: app/test-regex/
F: doc/guides/prog_guide/regexdev.rst
F: doc/guides/regexdevs/features/default.ini
+DMA device API - EXPERIMENTAL
+M: Chengwen Feng <fengchengwen@huawei.com>
+F: lib/dmadev/
+F: doc/guides/prog_guide/dmadev.rst
+
Eventdev API
M: Jerin Jacob <jerinj@marvell.com>
T: git://dpdk.org/next/dpdk-next-eventdev
@@ -70,6 +70,9 @@
/* regexdev defines */
#define RTE_MAX_REGEXDEV_DEVS 32
+/* dmadev defines */
+#define RTE_DMADEV_DEFAULT_MAX_DEVS 64
+
/* eventdev defines */
#define RTE_EVENT_MAX_DEVS 16
#define RTE_EVENT_MAX_QUEUES_PER_DEV 255
@@ -21,6 +21,7 @@ The public API headers are grouped by topics:
[compressdev] (@ref rte_compressdev.h),
[compress] (@ref rte_comp.h),
[regexdev] (@ref rte_regexdev.h),
+ [dmadev] (@ref rte_dmadev.h),
[eventdev] (@ref rte_eventdev.h),
[event_eth_rx_adapter] (@ref rte_event_eth_rx_adapter.h),
[event_eth_tx_adapter] (@ref rte_event_eth_tx_adapter.h),
@@ -35,6 +35,7 @@ INPUT = @TOPDIR@/doc/api/doxy-api-index.md \
@TOPDIR@/lib/compressdev \
@TOPDIR@/lib/cryptodev \
@TOPDIR@/lib/distributor \
+ @TOPDIR@/lib/dmadev \
@TOPDIR@/lib/efd \
@TOPDIR@/lib/ethdev \
@TOPDIR@/lib/eventdev \
new file mode 100644
@@ -0,0 +1,12 @@
+.. SPDX-License-Identifier: BSD-3-Clause
+ Copyright 2021 HiSilicon Limited
+
+DMA Device Drivers
+==================
+
+The following are a list of DMA device drivers, which can be used from
+an application through DMA API.
+
+.. toctree::
+ :maxdepth: 2
+ :numbered:
@@ -21,6 +21,7 @@ DPDK documentation
compressdevs/index
vdpadevs/index
regexdevs/index
+ dmadevs/index
eventdevs/index
rawdevs/index
mempool/index
new file mode 100644
@@ -0,0 +1,64 @@
+.. SPDX-License-Identifier: BSD-3-Clause
+ Copyright 2021 HiSilicon Limited
+
+DMA Device Library
+==================
+
+The DMA library provides a DMA device framework for management and provisioning
+of hardware and software DMA poll mode drivers, defining generic APIs which
+support a number of different DMA operations.
+
+
+Design Principles
+-----------------
+
+The DMA library follows the same basic principles as those used in DPDK's
+Ethernet Device framework and the RegEx framework. The DMA framework provides
+a generic DMA device framework which supports both physical (hardware)
+and virtual (software) DMA devices as well as a generic DMA API which allows
+DMA devices to be managed and configured and supports DMA operations to be
+provisioned on DMA poll mode driver.
+
+.. _figure_dmadev:
+
+.. figure:: img/dmadev.*
+
+The above figure shows the model on which the DMA framework is built on:
+
+ * The DMA controller could have multiple hardware DMA channels (aka. hardware
+ DMA queues), each hardware DMA channel should be represented by a dmadev.
+ * The dmadev could create multiple virtual DMA channels, each virtual DMA
+ channel represents a different transfer context. The DMA operation request
+ must be submitted to the virtual DMA channel. e.g. Application could create
+ virtual DMA channel 0 for memory-to-memory transfer scenario, and create
+ virtual DMA channel 1 for memory-to-device transfer scenario.
+
+
+Device Management
+-----------------
+
+Device Creation
+~~~~~~~~~~~~~~~
+
+Physical DMA controllers are discovered during the PCI probe/enumeration of the
+EAL function which is executed at DPDK initialization, this is based on their
+PCI BDF (bus/bridge, device, function). Specific physical DMA controllers, like
+other physical devices in DPDK can be listed using the EAL command line options.
+
+The dmadevs are dynamically allocated by using the API
+``rte_dma_pmd_allocate`` based on the number of hardware DMA channels. After the
+dmadev initialized successfully, the driver needs to switch the dmadev state to
+``RTE_DMA_DEV_READY``.
+
+
+Device Identification
+~~~~~~~~~~~~~~~~~~~~~
+
+Each DMA device, whether physical or virtual is uniquely designated by two
+identifiers:
+
+- A unique device index used to designate the DMA device in all functions
+ exported by the DMA API.
+
+- A device name used to designate the DMA device in console messages, for
+ administration or debugging purposes.
new file mode 100644
@@ -0,0 +1,283 @@
+<?xml version="1.0" encoding="UTF-8" standalone="no"?>
+<!-- Created with Inkscape (http://www.inkscape.org/) -->
+
+<!-- SPDX-License-Identifier: BSD-3-Clause -->
+<!-- Copyright(c) 2021 HiSilicon Limited -->
+
+<svg
+ width="128.64288mm"
+ height="95.477707mm"
+ viewBox="0 0 192.96433 143.21656"
+ version="1.1"
+ id="svg934"
+ inkscape:version="1.1 (c68e22c387, 2021-05-23)"
+ sodipodi:docname="dmadev.svg"
+ xmlns:inkscape="http://www.inkscape.org/namespaces/inkscape"
+ xmlns:sodipodi="http://sodipodi.sourceforge.net/DTD/sodipodi-0.dtd"
+ xmlns="http://www.w3.org/2000/svg"
+ xmlns:svg="http://www.w3.org/2000/svg">
+ <sodipodi:namedview
+ id="namedview936"
+ pagecolor="#ffffff"
+ bordercolor="#666666"
+ borderopacity="1.0"
+ inkscape:pageshadow="2"
+ inkscape:pageopacity="0.0"
+ inkscape:pagecheckerboard="0"
+ inkscape:document-units="mm"
+ showgrid="false"
+ fit-margin-top="0"
+ fit-margin-left="0"
+ fit-margin-right="0"
+ fit-margin-bottom="0"
+ inkscape:showpageshadow="false"
+ inkscape:zoom="1.332716"
+ inkscape:cx="335.03011"
+ inkscape:cy="143.69152"
+ inkscape:window-width="1920"
+ inkscape:window-height="976"
+ inkscape:window-x="-8"
+ inkscape:window-y="-8"
+ inkscape:window-maximized="1"
+ inkscape:current-layer="layer1"
+ scale-x="1.5"
+ units="mm" />
+ <defs
+ id="defs931">
+ <rect
+ x="342.43954"
+ y="106.56832"
+ width="58.257381"
+ height="137.82834"
+ id="rect17873" />
+ </defs>
+ <g
+ inkscape:label="Layer 1"
+ inkscape:groupmode="layer"
+ id="layer1"
+ transform="translate(-0.13857517,-21.527306)">
+ <rect
+ style="fill:#c9c9ff;fill-opacity:1;stroke-width:0.296755"
+ id="rect31-9"
+ width="50"
+ height="28"
+ x="0.13857517"
+ y="21.527306"
+ ry="0" />
+ <text
+ xml:space="preserve"
+ style="font-style:normal;font-weight:normal;font-size:7.05556px;line-height:1.25;font-family:sans-serif;white-space:pre;inline-size:70.1114;fill:#000000;fill-opacity:1;stroke:none;stroke-width:0.264583"
+ x="54.136707"
+ y="18.045568"
+ id="text803-1"
+ transform="translate(-49.110795,15.205683)"><tspan
+ x="54.136707"
+ y="18.045568"
+ id="tspan1045">virtual DMA </tspan><tspan
+ x="54.136707"
+ y="26.865018"
+ id="tspan1047">channel</tspan></text>
+ <rect
+ style="fill:#c9c9ff;fill-opacity:1;stroke-width:0.296755"
+ id="rect31-9-5"
+ width="50"
+ height="28"
+ x="60.138577"
+ y="21.527306"
+ ry="0" />
+ <text
+ xml:space="preserve"
+ style="font-style:normal;font-weight:normal;font-size:7.05556px;line-height:1.25;font-family:sans-serif;white-space:pre;inline-size:70.1114;fill:#000000;fill-opacity:1;stroke:none;stroke-width:0.264583"
+ x="54.136707"
+ y="18.045568"
+ id="text803-1-4"
+ transform="translate(10.512565,15.373298)"><tspan
+ x="54.136707"
+ y="18.045568"
+ id="tspan1049">virtual DMA </tspan><tspan
+ x="54.136707"
+ y="26.865018"
+ id="tspan1051">channel</tspan></text>
+ <rect
+ style="fill:#c9c9ff;fill-opacity:1;stroke-width:0.296755"
+ id="rect31-9-5-3"
+ width="50"
+ height="28"
+ x="137.43863"
+ y="21.527306"
+ ry="0" />
+ <text
+ xml:space="preserve"
+ style="font-style:normal;font-weight:normal;font-size:7.05556px;line-height:1.25;font-family:sans-serif;white-space:pre;inline-size:70.1114;fill:#000000;fill-opacity:1;stroke:none;stroke-width:0.264583"
+ x="54.136707"
+ y="18.045568"
+ id="text803-1-4-8"
+ transform="translate(88.79231,15.373299)"><tspan
+ x="54.136707"
+ y="18.045568"
+ id="tspan1053">virtual DMA </tspan><tspan
+ x="54.136707"
+ y="26.865018"
+ id="tspan1055">channel</tspan></text>
+ <text
+ xml:space="preserve"
+ transform="matrix(0.26458333,0,0,0.26458333,-0.04940429,21.408845)"
+ id="text17871"
+ style="font-style:normal;font-weight:normal;font-size:40px;line-height:1.25;font-family:sans-serif;white-space:pre;shape-inside:url(#rect17873);fill:#000000;fill-opacity:1;stroke:none" />
+ <rect
+ style="fill:#c9c9ff;fill-opacity:1;stroke-width:0.218145"
+ id="rect31-9-5-8"
+ width="38.34557"
+ height="19.729115"
+ x="36.138577"
+ y="64.827354"
+ ry="0" />
+ <text
+ xml:space="preserve"
+ style="font-style:normal;font-weight:normal;font-size:7.05556px;line-height:1.25;font-family:sans-serif;white-space:pre;inline-size:70.1114;fill:#000000;fill-opacity:1;stroke:none;stroke-width:0.264583"
+ x="54.136707"
+ y="18.045568"
+ id="text803-1-4-3"
+ transform="translate(-13.394978,59.135217)"><tspan
+ x="54.136707"
+ y="18.045568"
+ id="tspan1057">dmadev</tspan></text>
+ <rect
+ style="fill:#c9c9ff;fill-opacity:1;stroke-width:0.307089"
+ id="rect31-9-5-8-0"
+ width="60.902534"
+ height="24.616455"
+ x="25.196909"
+ y="98.47744"
+ ry="0" />
+ <text
+ xml:space="preserve"
+ style="font-style:normal;font-weight:normal;font-size:7.05556px;line-height:1.25;font-family:sans-serif;white-space:pre;inline-size:70.1114;fill:#000000;fill-opacity:1;stroke:none;stroke-width:0.264583"
+ x="54.136707"
+ y="18.045568"
+ id="text803-1-4-3-76"
+ transform="translate(-24.485484,90.97883)"><tspan
+ x="54.136707"
+ y="18.045568"
+ id="tspan1059">hardware DMA </tspan><tspan
+ x="54.136707"
+ y="26.865018"
+ id="tspan1061">channel</tspan></text>
+ <rect
+ style="fill:#c9c9ff;fill-opacity:1;stroke-width:0.307089"
+ id="rect31-9-5-8-0-6"
+ width="60.902534"
+ height="24.616455"
+ x="132.20036"
+ y="98.47744"
+ ry="0" />
+ <text
+ xml:space="preserve"
+ style="font-style:normal;font-weight:normal;font-size:7.05556px;line-height:1.25;font-family:sans-serif;white-space:pre;inline-size:70.1114;fill:#000000;fill-opacity:1;stroke:none;stroke-width:0.264583"
+ x="54.136707"
+ y="18.045568"
+ id="text803-1-4-3-76-7"
+ transform="translate(82.950904,90.79085)"><tspan
+ x="54.136707"
+ y="18.045568"
+ id="tspan1063">hardware DMA </tspan><tspan
+ x="54.136707"
+ y="26.865018"
+ id="tspan1065">channel</tspan></text>
+ <rect
+ style="fill:#c9c9ff;fill-opacity:1;stroke-width:0.307089"
+ id="rect31-9-5-8-0-4"
+ width="60.902534"
+ height="24.616455"
+ x="76.810928"
+ y="140.12741"
+ ry="0" />
+ <text
+ xml:space="preserve"
+ style="font-style:normal;font-weight:normal;font-size:7.05556px;line-height:1.25;font-family:sans-serif;white-space:pre;inline-size:70.1114;fill:#000000;fill-opacity:1;stroke:none;stroke-width:0.264583"
+ x="54.136707"
+ y="18.045568"
+ id="text803-1-4-3-76-4"
+ transform="translate(27.032341,133.10574)"><tspan
+ x="54.136707"
+ y="18.045568"
+ id="tspan1067">hardware DMA </tspan><tspan
+ x="54.136707"
+ y="26.865018"
+ id="tspan1069">controller</tspan></text>
+ <rect
+ style="fill:#c9c9ff;fill-opacity:1;stroke-width:0.218145"
+ id="rect31-9-5-8-5"
+ width="38.34557"
+ height="19.729115"
+ x="143.43863"
+ y="64.827354"
+ ry="0" />
+ <text
+ xml:space="preserve"
+ style="font-style:normal;font-weight:normal;font-size:7.05556px;line-height:1.25;font-family:sans-serif;white-space:pre;inline-size:70.1114;fill:#000000;fill-opacity:1;stroke:none;stroke-width:0.264583"
+ x="54.136707"
+ y="18.045568"
+ id="text803-1-4-3-7"
+ transform="translate(94.92597,59.664385)"><tspan
+ x="54.136707"
+ y="18.045568"
+ id="tspan1071">dmadev</tspan></text>
+ <path
+ style="fill:none;fill-rule:evenodd;stroke:#000000;stroke-width:0.264583px;stroke-linecap:butt;stroke-linejoin:miter;stroke-opacity:1"
+ d="M 74.476373,49.527306 62.82407,64.827354"
+ id="path45308"
+ inkscape:connector-type="polyline"
+ inkscape:connector-curvature="0"
+ inkscape:connection-start="#rect31-9-5"
+ inkscape:connection-end="#rect31-9-5-8" />
+ <path
+ style="fill:none;fill-rule:evenodd;stroke:#000000;stroke-width:0.264583px;stroke-linecap:butt;stroke-linejoin:miter;stroke-opacity:1"
+ d="M 35.924309,49.527306 47.711612,64.827354"
+ id="path45310"
+ inkscape:connector-type="polyline"
+ inkscape:connector-curvature="0"
+ inkscape:connection-start="#rect31-9"
+ inkscape:connection-end="#rect31-9-5-8" />
+ <path
+ style="fill:none;fill-rule:evenodd;stroke:#000000;stroke-width:0.264583px;stroke-linecap:butt;stroke-linejoin:miter;stroke-opacity:1"
+ d="M 55.403414,84.556469 55.53332,98.47744"
+ id="path45312"
+ inkscape:connector-type="polyline"
+ inkscape:connector-curvature="0"
+ inkscape:connection-start="#rect31-9-5-8"
+ inkscape:connection-end="#rect31-9-5-8-0" />
+ <path
+ style="fill:none;fill-rule:evenodd;stroke:#000000;stroke-width:0.264583px;stroke-linecap:butt;stroke-linejoin:miter;stroke-opacity:1"
+ d="m 162.62241,84.556469 0.0155,13.920971"
+ id="path45320"
+ inkscape:connector-type="polyline"
+ inkscape:connector-curvature="0"
+ inkscape:connection-start="#rect31-9-5-8-5"
+ inkscape:connection-end="#rect31-9-5-8-0-6" />
+ <path
+ style="fill:none;fill-rule:evenodd;stroke:#000000;stroke-width:0.264583px;stroke-linecap:butt;stroke-linejoin:miter;stroke-opacity:1"
+ d="m 146.28317,123.09389 -22.65252,17.03352"
+ id="path45586"
+ inkscape:connector-type="polyline"
+ inkscape:connector-curvature="0"
+ inkscape:connection-start="#rect31-9-5-8-0-6"
+ inkscape:connection-end="#rect31-9-5-8-0-4" />
+ <path
+ style="fill:none;fill-rule:evenodd;stroke:#000000;stroke-width:0.264583px;stroke-linecap:butt;stroke-linejoin:miter;stroke-opacity:1"
+ d="m 70.900938,123.09389 21.108496,17.03352"
+ id="path45588"
+ inkscape:connector-type="polyline"
+ inkscape:connector-curvature="0"
+ inkscape:connection-start="#rect31-9-5-8-0"
+ inkscape:connection-end="#rect31-9-5-8-0-4" />
+ <path
+ style="fill:none;fill-rule:evenodd;stroke:#000000;stroke-width:0.264583px;stroke-linecap:butt;stroke-linejoin:miter;stroke-opacity:1"
+ d="m 162.50039,49.527306 0.0675,15.300048"
+ id="path45956"
+ inkscape:connector-type="polyline"
+ inkscape:connector-curvature="0"
+ inkscape:connection-start="#rect31-9-5-3"
+ inkscape:connection-end="#rect31-9-5-8-5" />
+ </g>
+</svg>
@@ -27,6 +27,7 @@ Programmer's Guide
cryptodev_lib
compressdev
regexdev
+ dmadev
rte_security
rawdev
link_bonding_poll_mode_drv_lib
@@ -106,6 +106,10 @@ New Features
Added command-line options to specify total number of processes and
current process ID. Each process owns subset of Rx and Tx queues.
+* **Introduced dmadev library with:**
+
+ * Device allocation APIs.
+
Removed Items
-------------
new file mode 100644
@@ -0,0 +1,4 @@
+# SPDX-License-Identifier: BSD-3-Clause
+# Copyright 2021 HiSilicon Limited
+
+drivers = []
@@ -18,6 +18,7 @@ subdirs = [
'vdpa', # depends on common, bus and mempool.
'event', # depends on common, bus, mempool and net.
'baseband', # depends on common and bus.
+ 'dma', # depends on common and bus.
]
if meson.is_cross_build()
new file mode 100644
@@ -0,0 +1,7 @@
+# SPDX-License-Identifier: BSD-3-Clause
+# Copyright(c) 2021 HiSilicon Limited.
+
+sources = files('rte_dmadev.c')
+headers = files('rte_dmadev.h')
+indirect_headers += files('rte_dmadev_core.h')
+driver_sdk_headers += files('rte_dmadev_pmd.h')
new file mode 100644
@@ -0,0 +1,263 @@
+/* SPDX-License-Identifier: BSD-3-Clause
+ * Copyright(c) 2021 HiSilicon Limited
+ * Copyright(c) 2021 Intel Corporation
+ */
+
+#include <inttypes.h>
+
+#include <rte_eal.h>
+#include <rte_lcore.h>
+#include <rte_log.h>
+#include <rte_malloc.h>
+#include <rte_memzone.h>
+#include <rte_string_fns.h>
+
+#include "rte_dmadev.h"
+#include "rte_dmadev_pmd.h"
+
+struct rte_dma_dev *rte_dma_devices;
+static int16_t dma_devices_max;
+
+RTE_LOG_REGISTER_DEFAULT(rte_dma_logtype, INFO);
+#define RTE_DMA_LOG(level, fmt, args...) \
+ rte_log(RTE_LOG_ ## level, rte_dma_logtype, "%s(): " fmt "\n", \
+ __func__, ##args)
+
+/* Macros to check for valid device id */
+#define RTE_DMA_VALID_DEV_ID_OR_ERR_RET(dev_id, retval) do { \
+ if (!rte_dma_is_valid(dev_id)) { \
+ RTE_DMA_LOG(ERR, "Invalid dev_id=%d", dev_id); \
+ return retval; \
+ } \
+} while (0)
+
+int
+rte_dma_dev_max(size_t dev_max)
+{
+ /* This function may be called before rte_eal_init(), so no rte library
+ * function can be called in this function.
+ */
+ if (dev_max == 0 || dev_max > INT16_MAX)
+ return -EINVAL;
+
+ if (dma_devices_max > 0)
+ return -EINVAL;
+
+ dma_devices_max = dev_max;
+
+ return 0;
+}
+
+static int
+dma_check_name(const char *name)
+{
+ size_t name_len;
+
+ if (name == NULL) {
+ RTE_DMA_LOG(ERR, "Name can't be NULL");
+ return -EINVAL;
+ }
+
+ name_len = strnlen(name, RTE_DEV_NAME_MAX_LEN);
+ if (name_len == 0) {
+ RTE_DMA_LOG(ERR, "Zero length DMA device name");
+ return -EINVAL;
+ }
+ if (name_len >= RTE_DEV_NAME_MAX_LEN) {
+ RTE_DMA_LOG(ERR, "DMA device name is too long");
+ return -EINVAL;
+ }
+
+ return 0;
+}
+
+static int16_t
+dma_find_free_dev(void)
+{
+ int16_t i;
+
+ if (rte_dma_devices == NULL)
+ return -1;
+
+ for (i = 0; i < dma_devices_max; i++) {
+ if (rte_dma_devices[i].dev_name[0] == '\0')
+ return i;
+ }
+
+ return -1;
+}
+
+static struct rte_dma_dev*
+dma_find(const char *name)
+{
+ int16_t i;
+
+ if (rte_dma_devices == NULL)
+ return NULL;
+
+ for (i = 0; i < dma_devices_max; i++) {
+ if ((rte_dma_devices[i].state != RTE_DMA_DEV_UNUSED) &&
+ (!strcmp(name, rte_dma_devices[i].dev_name)))
+ return &rte_dma_devices[i];
+ }
+
+ return NULL;
+}
+
+static int
+dma_process_data_prepare(void)
+{
+ size_t size;
+ void *ptr;
+
+ if (rte_dma_devices != NULL)
+ return 0;
+
+ /* The return value of malloc may not be aligned to the cache line.
+ * Therefore, extra memory is applied for realignment.
+ * note: We do not call posix_memalign/aligned_alloc because it is
+ * version dependent on libc.
+ */
+ size = dma_devices_max * sizeof(struct rte_dma_dev) +
+ RTE_CACHE_LINE_SIZE;
+ ptr = malloc(size);
+ if (ptr == NULL)
+ return -ENOMEM;
+ memset(ptr, 0, size);
+
+ rte_dma_devices = RTE_PTR_ALIGN(ptr, RTE_CACHE_LINE_SIZE);
+
+ return 0;
+}
+
+static int
+dma_data_prepare(void)
+{
+ if (dma_devices_max == 0)
+ dma_devices_max = RTE_DMADEV_DEFAULT_MAX_DEVS;
+ return dma_process_data_prepare();
+}
+
+static struct rte_dma_dev *
+dma_allocate(const char *name, int numa_node, size_t private_data_size)
+{
+ struct rte_dma_dev *dev;
+ void *dev_private;
+ int16_t dev_id;
+ int ret;
+
+ ret = dma_data_prepare();
+ if (ret < 0) {
+ RTE_DMA_LOG(ERR, "Cannot initialize dmadevs data");
+ return NULL;
+ }
+
+ dev = dma_find(name);
+ if (dev != NULL) {
+ RTE_DMA_LOG(ERR, "DMA device already allocated");
+ return NULL;
+ }
+
+ dev_private = rte_zmalloc_socket(name, private_data_size,
+ RTE_CACHE_LINE_SIZE, numa_node);
+ if (dev_private == NULL) {
+ RTE_DMA_LOG(ERR, "Cannot allocate private data");
+ return NULL;
+ }
+
+ dev_id = dma_find_free_dev();
+ if (dev_id < 0) {
+ RTE_DMA_LOG(ERR, "Reached maximum number of DMA devices");
+ rte_free(dev_private);
+ return NULL;
+ }
+
+ dev = &rte_dma_devices[dev_id];
+ dev->dev_private = dev_private;
+ rte_strscpy(dev->dev_name, name, sizeof(dev->dev_name));
+ dev->dev_id = dev_id;
+ dev->numa_node = numa_node;
+ dev->dev_private = dev_private;
+
+ return dev;
+}
+
+static void
+dma_release(struct rte_dma_dev *dev)
+{
+ rte_free(dev->dev_private);
+ memset(dev, 0, sizeof(struct rte_dma_dev));
+}
+
+struct rte_dma_dev *
+rte_dma_pmd_allocate(const char *name, int numa_node, size_t private_data_size)
+{
+ struct rte_dma_dev *dev;
+
+ if (dma_check_name(name) != 0 || private_data_size == 0)
+ return NULL;
+
+ dev = dma_allocate(name, numa_node, private_data_size);
+ if (dev == NULL)
+ return NULL;
+
+ dev->state = RTE_DMA_DEV_REGISTERED;
+
+ return dev;
+}
+
+int
+rte_dma_pmd_release(const char *name)
+{
+ struct rte_dma_dev *dev;
+
+ if (dma_check_name(name) != 0)
+ return -EINVAL;
+
+ dev = dma_find(name);
+ if (dev == NULL)
+ return -EINVAL;
+
+ dma_release(dev);
+ return 0;
+}
+
+int
+rte_dma_get_dev_id(const char *name)
+{
+ struct rte_dma_dev *dev;
+
+ if (dma_check_name(name) != 0)
+ return -EINVAL;
+
+ dev = dma_find(name);
+ if (dev == NULL)
+ return -EINVAL;
+
+ return dev->dev_id;
+}
+
+bool
+rte_dma_is_valid(int16_t dev_id)
+{
+ return (dev_id >= 0) && (dev_id < dma_devices_max) &&
+ rte_dma_devices != NULL &&
+ rte_dma_devices[dev_id].state != RTE_DMA_DEV_UNUSED;
+}
+
+uint16_t
+rte_dma_count_avail(void)
+{
+ uint16_t count = 0;
+ uint16_t i;
+
+ if (rte_dma_devices == NULL)
+ return count;
+
+ for (i = 0; i < dma_devices_max; i++) {
+ if (rte_dma_devices[i].state != RTE_DMA_DEV_UNUSED)
+ count++;
+ }
+
+ return count;
+}
new file mode 100644
@@ -0,0 +1,134 @@
+/* SPDX-License-Identifier: BSD-3-Clause
+ * Copyright(c) 2021 HiSilicon Limited
+ * Copyright(c) 2021 Intel Corporation
+ * Copyright(c) 2021 Marvell International Ltd
+ * Copyright(c) 2021 SmartShare Systems
+ */
+
+#ifndef RTE_DMADEV_H
+#define RTE_DMADEV_H
+
+/**
+ * @file rte_dmadev.h
+ *
+ * DMA (Direct Memory Access) device API.
+ *
+ * The DMA framework is built on the following model:
+ *
+ * --------------- --------------- ---------------
+ * | virtual DMA | | virtual DMA | | virtual DMA |
+ * | channel | | channel | | channel |
+ * --------------- --------------- ---------------
+ * | | |
+ * ------------------ |
+ * | |
+ * ------------ ------------
+ * | dmadev | | dmadev |
+ * ------------ ------------
+ * | |
+ * ------------------ ------------------
+ * | HW DMA channel | | HW DMA channel |
+ * ------------------ ------------------
+ * | |
+ * --------------------------------
+ * |
+ * ---------------------
+ * | HW DMA Controller |
+ * ---------------------
+ *
+ * The DMA controller could have multiple HW-DMA-channels (aka. HW-DMA-queues),
+ * each HW-DMA-channel should be represented by a dmadev.
+ *
+ * The dmadev could create multiple virtual DMA channels, each virtual DMA
+ * channel represents a different transfer context. The DMA operation request
+ * must be submitted to the virtual DMA channel. e.g. Application could create
+ * virtual DMA channel 0 for memory-to-memory transfer scenario, and create
+ * virtual DMA channel 1 for memory-to-device transfer scenario.
+ *
+ * The dmadev are dynamically allocated by rte_dma_pmd_allocate() during the
+ * PCI/SoC device probing phase performed at EAL initialization time. And could
+ * be released by rte_dma_pmd_release() during the PCI/SoC device removing
+ * phase.
+ *
+ * This framework uses 'int16_t dev_id' as the device identifier of a dmadev,
+ * and 'uint16_t vchan' as the virtual DMA channel identifier in one dmadev.
+ *
+ */
+
+#include <stdint.h>
+
+#include <rte_bitops.h>
+#include <rte_common.h>
+#include <rte_compat.h>
+#include <rte_dev.h>
+
+#ifdef __cplusplus
+extern "C" {
+#endif
+
+/**
+ * @warning
+ * @b EXPERIMENTAL: this API may change without prior notice.
+ *
+ * Configure the maximum number of dmadevs.
+ * @note This function can be invoked before the primary process rte_eal_init()
+ * to change the maximum number of dmadevs.
+ *
+ * @param dev_max
+ * maximum number of dmadevs.
+ *
+ * @return
+ * 0 on success. Otherwise negative value is returned.
+ */
+__rte_experimental
+int rte_dma_dev_max(size_t dev_max);
+
+/**
+ * @warning
+ * @b EXPERIMENTAL: this API may change without prior notice.
+ *
+ * Get the device identifier for the named DMA device.
+ *
+ * @param name
+ * DMA device name.
+ *
+ * @return
+ * Returns DMA device identifier on success.
+ * - <0: Failure to find named DMA device.
+ */
+__rte_experimental
+int rte_dma_get_dev_id(const char *name);
+
+/**
+ * @warning
+ * @b EXPERIMENTAL: this API may change without prior notice.
+ *
+ * @param dev_id
+ * DMA device index.
+ *
+ * @return
+ * - If the device index is valid (true) or not (false).
+ */
+__rte_experimental
+bool rte_dma_is_valid(int16_t dev_id);
+
+/**
+ * @warning
+ * @b EXPERIMENTAL: this API may change without prior notice.
+ *
+ * Get the total number of DMA devices that have been successfully
+ * initialised.
+ *
+ * @return
+ * The total number of usable DMA devices.
+ */
+__rte_experimental
+uint16_t rte_dma_count_avail(void);
+
+#include "rte_dmadev_core.h"
+
+#ifdef __cplusplus
+}
+#endif
+
+#endif /* RTE_DMADEV_H */
new file mode 100644
@@ -0,0 +1,51 @@
+/* SPDX-License-Identifier: BSD-3-Clause
+ * Copyright(c) 2021 HiSilicon Limited
+ * Copyright(c) 2021 Intel Corporation
+ */
+
+#ifndef RTE_DMADEV_CORE_H
+#define RTE_DMADEV_CORE_H
+
+/**
+ * @file
+ *
+ * DMA Device internal header.
+ *
+ * This header contains internal data types, that are used by the DMA devices
+ * in order to expose their ops to the class.
+ *
+ * Applications should not use these API directly.
+ *
+ */
+
+/**
+ * Possible states of a DMA device.
+ *
+ * @see struct rte_dmadev::state
+ */
+enum rte_dma_dev_state {
+ RTE_DMA_DEV_UNUSED = 0, /**< Device is unused. */
+ /** Device is registered, but not ready to be used. */
+ RTE_DMA_DEV_REGISTERED,
+ /** Device is ready for use. This is set by the PMD. */
+ RTE_DMA_DEV_READY,
+};
+
+/**
+ * @internal
+ * The generic data structure associated with each DMA device.
+ */
+struct rte_dma_dev {
+ char dev_name[RTE_DEV_NAME_MAX_LEN]; /**< Unique identifier name */
+ int16_t dev_id; /**< Device [external] identifier. */
+ int16_t numa_node; /**< Local NUMA memory ID. -1 if unknown. */
+ void *dev_private; /**< PMD-specific private data. */
+ /** Device info which supplied during device initialization. */
+ struct rte_device *device;
+ enum rte_dma_dev_state state; /**< Flag indicating the device state. */
+ uint64_t reserved[2]; /**< Reserved for future fields. */
+} __rte_cache_aligned;
+
+extern struct rte_dma_dev *rte_dma_devices;
+
+#endif /* RTE_DMADEV_CORE_H */
new file mode 100644
@@ -0,0 +1,60 @@
+/* SPDX-License-Identifier: BSD-3-Clause
+ * Copyright(c) 2021 HiSilicon Limited
+ */
+
+#ifndef RTE_DMADEV_PMD_H
+#define RTE_DMADEV_PMD_H
+
+/**
+ * @file
+ *
+ * DMA Device PMD APIs
+ *
+ * Driver facing APIs for a DMA device. These are not to be called directly by
+ * any application.
+ */
+
+#include "rte_dmadev.h"
+
+#ifdef __cplusplus
+extern "C" {
+#endif
+
+/**
+ * @internal
+ * Allocates a new dmadev slot for an DMA device and returns the pointer
+ * to that slot for the driver to use.
+ *
+ * @param name
+ * DMA device name.
+ * @param numa_node
+ * Driver's private data's numa node.
+ * @param private_data_size
+ * Driver's private data size.
+ *
+ * @return
+ * A pointer to the DMA device slot case of success,
+ * NULL otherwise.
+ */
+__rte_internal
+struct rte_dma_dev *rte_dma_pmd_allocate(const char *name, int numa_node,
+ size_t private_data_size);
+
+/**
+ * @internal
+ * Release the specified dmadev.
+ *
+ * @param name
+ * DMA device name.
+ *
+ * @return
+ * - 0 on success, negative on error.
+ */
+__rte_internal
+int rte_dma_pmd_release(const char *name);
+
+#ifdef __cplusplus
+}
+#endif
+
+#endif /* RTE_DMADEV_PMD_H */
new file mode 100644
@@ -0,0 +1,20 @@
+EXPERIMENTAL {
+ global:
+
+ rte_dma_count_avail;
+ rte_dma_dev_max;
+ rte_dma_get_dev_id;
+ rte_dma_is_valid;
+
+ local: *;
+};
+
+INTERNAL {
+ global:
+
+ rte_dma_devices;
+ rte_dma_pmd_allocate;
+ rte_dma_pmd_release;
+
+ local: *;
+};
@@ -45,6 +45,7 @@ libraries = [
'pdump',
'rawdev',
'regexdev',
+ 'dmadev',
'rib',
'reorder',
'sched',