diff mbox series

[v23,1/6] dmadev: introduce DMA device library

Message ID 20210924105357.15386-2-fengchengwen@huawei.com (mailing list archive)
State Superseded
Headers show
Series support dmadev | expand

Checks

Context Check Description
ci/iol-testing warning apply patch failure
ci/checkpatch warning coding style issues

Commit Message

Chengwen Feng Sept. 24, 2021, 10:53 a.m. UTC
The 'dmadevice' is a generic type of DMA device.

This patch introduce the 'dmadevice' device allocation APIs.

The infrastructure is prepared to welcome drivers in drivers/dma/

Signed-off-by: Chengwen Feng <fengchengwen@huawei.com>
Acked-by: Bruce Richardson <bruce.richardson@intel.com>
Acked-by: Morten Brørup <mb@smartsharesystems.com>
Acked-by: Jerin Jacob <jerinjacobk@gmail.com>
Reviewed-by: Kevin Laatz <kevin.laatz@intel.com>
Reviewed-by: Conor Walsh <conor.walsh@intel.com>
---
 MAINTAINERS                            |   5 +
 config/rte_config.h                    |   3 +
 doc/api/doxy-api-index.md              |   1 +
 doc/api/doxy-api.conf.in               |   1 +
 doc/guides/dmadevs/index.rst           |  12 ++
 doc/guides/index.rst                   |   1 +
 doc/guides/prog_guide/dmadev.rst       |  64 ++++++
 doc/guides/prog_guide/img/dmadev.svg   | 283 +++++++++++++++++++++++++
 doc/guides/prog_guide/index.rst        |   1 +
 doc/guides/rel_notes/release_21_11.rst |   4 +
 drivers/dma/meson.build                |   4 +
 drivers/meson.build                    |   1 +
 lib/dmadev/meson.build                 |   7 +
 lib/dmadev/rte_dmadev.c                | 263 +++++++++++++++++++++++
 lib/dmadev/rte_dmadev.h                | 134 ++++++++++++
 lib/dmadev/rte_dmadev_core.h           |  51 +++++
 lib/dmadev/rte_dmadev_pmd.h            |  60 ++++++
 lib/dmadev/version.map                 |  20 ++
 lib/meson.build                        |   1 +
 19 files changed, 916 insertions(+)
 create mode 100644 doc/guides/dmadevs/index.rst
 create mode 100644 doc/guides/prog_guide/dmadev.rst
 create mode 100644 doc/guides/prog_guide/img/dmadev.svg
 create mode 100644 drivers/dma/meson.build
 create mode 100644 lib/dmadev/meson.build
 create mode 100644 lib/dmadev/rte_dmadev.c
 create mode 100644 lib/dmadev/rte_dmadev.h
 create mode 100644 lib/dmadev/rte_dmadev_core.h
 create mode 100644 lib/dmadev/rte_dmadev_pmd.h
 create mode 100644 lib/dmadev/version.map

Comments

Radha Mohan Oct. 4, 2021, 9:12 p.m. UTC | #1
On Fri, Sep 24, 2021 at 3:58 AM Chengwen Feng <fengchengwen@huawei.com> wrote:
>
> The 'dmadevice' is a generic type of DMA device.
>
> This patch introduce the 'dmadevice' device allocation APIs.
>
> The infrastructure is prepared to welcome drivers in drivers/dma/
>
> Signed-off-by: Chengwen Feng <fengchengwen@huawei.com>
> Acked-by: Bruce Richardson <bruce.richardson@intel.com>
> Acked-by: Morten Brørup <mb@smartsharesystems.com>
> Acked-by: Jerin Jacob <jerinjacobk@gmail.com>
> Reviewed-by: Kevin Laatz <kevin.laatz@intel.com>
> Reviewed-by: Conor Walsh <conor.walsh@intel.com>
> ---
>  MAINTAINERS                            |   5 +
>  config/rte_config.h                    |   3 +
>  doc/api/doxy-api-index.md              |   1 +
>  doc/api/doxy-api.conf.in               |   1 +
>  doc/guides/dmadevs/index.rst           |  12 ++
>  doc/guides/index.rst                   |   1 +
>  doc/guides/prog_guide/dmadev.rst       |  64 ++++++
>  doc/guides/prog_guide/img/dmadev.svg   | 283 +++++++++++++++++++++++++
>  doc/guides/prog_guide/index.rst        |   1 +
>  doc/guides/rel_notes/release_21_11.rst |   4 +
>  drivers/dma/meson.build                |   4 +
>  drivers/meson.build                    |   1 +
>  lib/dmadev/meson.build                 |   7 +
>  lib/dmadev/rte_dmadev.c                | 263 +++++++++++++++++++++++
>  lib/dmadev/rte_dmadev.h                | 134 ++++++++++++
>  lib/dmadev/rte_dmadev_core.h           |  51 +++++
>  lib/dmadev/rte_dmadev_pmd.h            |  60 ++++++
>  lib/dmadev/version.map                 |  20 ++
>  lib/meson.build                        |   1 +
>  19 files changed, 916 insertions(+)
>  create mode 100644 doc/guides/dmadevs/index.rst
>  create mode 100644 doc/guides/prog_guide/dmadev.rst
>  create mode 100644 doc/guides/prog_guide/img/dmadev.svg
>  create mode 100644 drivers/dma/meson.build
>  create mode 100644 lib/dmadev/meson.build
>  create mode 100644 lib/dmadev/rte_dmadev.c
>  create mode 100644 lib/dmadev/rte_dmadev.h
>  create mode 100644 lib/dmadev/rte_dmadev_core.h
>  create mode 100644 lib/dmadev/rte_dmadev_pmd.h
>  create mode 100644 lib/dmadev/version.map
>
<snip>
Hi Chengwen,
I see that the new version removed the "rte_dmadev_get_device_by_name()".
What is the way to get the dmadev from inside the PMD .remove ? I am
looking to get the dev_private as we need to do some cleanup
operations from the remove function.

regards,
Radha Mohan
Kevin Laatz Oct. 5, 2021, 8:24 a.m. UTC | #2
On 04/10/2021 22:12, Radha Mohan wrote:
> On Fri, Sep 24, 2021 at 3:58 AM Chengwen Feng <fengchengwen@huawei.com> wrote:
>> The 'dmadevice' is a generic type of DMA device.
>>
>> This patch introduce the 'dmadevice' device allocation APIs.
>>
>> The infrastructure is prepared to welcome drivers in drivers/dma/
>>
>> Signed-off-by: Chengwen Feng <fengchengwen@huawei.com>
>> Acked-by: Bruce Richardson <bruce.richardson@intel.com>
>> Acked-by: Morten Brørup <mb@smartsharesystems.com>
>> Acked-by: Jerin Jacob <jerinjacobk@gmail.com>
>> Reviewed-by: Kevin Laatz <kevin.laatz@intel.com>
>> Reviewed-by: Conor Walsh <conor.walsh@intel.com>
>> ---
>>   MAINTAINERS                            |   5 +
>>   config/rte_config.h                    |   3 +
>>   doc/api/doxy-api-index.md              |   1 +
>>   doc/api/doxy-api.conf.in               |   1 +
>>   doc/guides/dmadevs/index.rst           |  12 ++
>>   doc/guides/index.rst                   |   1 +
>>   doc/guides/prog_guide/dmadev.rst       |  64 ++++++
>>   doc/guides/prog_guide/img/dmadev.svg   | 283 +++++++++++++++++++++++++
>>   doc/guides/prog_guide/index.rst        |   1 +
>>   doc/guides/rel_notes/release_21_11.rst |   4 +
>>   drivers/dma/meson.build                |   4 +
>>   drivers/meson.build                    |   1 +
>>   lib/dmadev/meson.build                 |   7 +
>>   lib/dmadev/rte_dmadev.c                | 263 +++++++++++++++++++++++
>>   lib/dmadev/rte_dmadev.h                | 134 ++++++++++++
>>   lib/dmadev/rte_dmadev_core.h           |  51 +++++
>>   lib/dmadev/rte_dmadev_pmd.h            |  60 ++++++
>>   lib/dmadev/version.map                 |  20 ++
>>   lib/meson.build                        |   1 +
>>   19 files changed, 916 insertions(+)
>>   create mode 100644 doc/guides/dmadevs/index.rst
>>   create mode 100644 doc/guides/prog_guide/dmadev.rst
>>   create mode 100644 doc/guides/prog_guide/img/dmadev.svg
>>   create mode 100644 drivers/dma/meson.build
>>   create mode 100644 lib/dmadev/meson.build
>>   create mode 100644 lib/dmadev/rte_dmadev.c
>>   create mode 100644 lib/dmadev/rte_dmadev.h
>>   create mode 100644 lib/dmadev/rte_dmadev_core.h
>>   create mode 100644 lib/dmadev/rte_dmadev_pmd.h
>>   create mode 100644 lib/dmadev/version.map
>>
> <snip>
> Hi Chengwen,
> I see that the new version removed the "rte_dmadev_get_device_by_name()".
> What is the way to get the dmadev from inside the PMD .remove ? I am
> looking to get the dev_private as we need to do some cleanup
> operations from the remove function.
>
> regards,
> Radha Mohan

Hi Radha,

You can use rte_dma_get_dev_id(name) to get the device ID, which can 
then be used to get the rte_dma_dev struct (which contains dev_private) 
for that device from rte_dma_devices[].

See "idxd_dmadev_destroy()" in 
http://patches.dpdk.org/project/dpdk/patch/20210924133916.4042773-6-kevin.laatz@intel.com/ 
for an example.

Hope that helps,

Kevin
Radha Mohan Oct. 5, 2021, 4:39 p.m. UTC | #3
On Tue, Oct 5, 2021 at 1:24 AM Kevin Laatz <kevin.laatz@intel.com> wrote:
>
> On 04/10/2021 22:12, Radha Mohan wrote:
> > On Fri, Sep 24, 2021 at 3:58 AM Chengwen Feng <fengchengwen@huawei.com> wrote:
> >> The 'dmadevice' is a generic type of DMA device.
> >>
> >> This patch introduce the 'dmadevice' device allocation APIs.
> >>
> >> The infrastructure is prepared to welcome drivers in drivers/dma/
> >>
> >> Signed-off-by: Chengwen Feng <fengchengwen@huawei.com>
> >> Acked-by: Bruce Richardson <bruce.richardson@intel.com>
> >> Acked-by: Morten Brørup <mb@smartsharesystems.com>
> >> Acked-by: Jerin Jacob <jerinjacobk@gmail.com>
> >> Reviewed-by: Kevin Laatz <kevin.laatz@intel.com>
> >> Reviewed-by: Conor Walsh <conor.walsh@intel.com>
> >> ---
> >>   MAINTAINERS                            |   5 +
> >>   config/rte_config.h                    |   3 +
> >>   doc/api/doxy-api-index.md              |   1 +
> >>   doc/api/doxy-api.conf.in               |   1 +
> >>   doc/guides/dmadevs/index.rst           |  12 ++
> >>   doc/guides/index.rst                   |   1 +
> >>   doc/guides/prog_guide/dmadev.rst       |  64 ++++++
> >>   doc/guides/prog_guide/img/dmadev.svg   | 283 +++++++++++++++++++++++++
> >>   doc/guides/prog_guide/index.rst        |   1 +
> >>   doc/guides/rel_notes/release_21_11.rst |   4 +
> >>   drivers/dma/meson.build                |   4 +
> >>   drivers/meson.build                    |   1 +
> >>   lib/dmadev/meson.build                 |   7 +
> >>   lib/dmadev/rte_dmadev.c                | 263 +++++++++++++++++++++++
> >>   lib/dmadev/rte_dmadev.h                | 134 ++++++++++++
> >>   lib/dmadev/rte_dmadev_core.h           |  51 +++++
> >>   lib/dmadev/rte_dmadev_pmd.h            |  60 ++++++
> >>   lib/dmadev/version.map                 |  20 ++
> >>   lib/meson.build                        |   1 +
> >>   19 files changed, 916 insertions(+)
> >>   create mode 100644 doc/guides/dmadevs/index.rst
> >>   create mode 100644 doc/guides/prog_guide/dmadev.rst
> >>   create mode 100644 doc/guides/prog_guide/img/dmadev.svg
> >>   create mode 100644 drivers/dma/meson.build
> >>   create mode 100644 lib/dmadev/meson.build
> >>   create mode 100644 lib/dmadev/rte_dmadev.c
> >>   create mode 100644 lib/dmadev/rte_dmadev.h
> >>   create mode 100644 lib/dmadev/rte_dmadev_core.h
> >>   create mode 100644 lib/dmadev/rte_dmadev_pmd.h
> >>   create mode 100644 lib/dmadev/version.map
> >>
> > <snip>
> > Hi Chengwen,
> > I see that the new version removed the "rte_dmadev_get_device_by_name()".
> > What is the way to get the dmadev from inside the PMD .remove ? I am
> > looking to get the dev_private as we need to do some cleanup
> > operations from the remove function.
> >
> > regards,
> > Radha Mohan
>
> Hi Radha,
>
> You can use rte_dma_get_dev_id(name) to get the device ID, which can
> then be used to get the rte_dma_dev struct (which contains dev_private)
> for that device from rte_dma_devices[].
>
> See "idxd_dmadev_destroy()" in
> http://patches.dpdk.org/project/dpdk/patch/20210924133916.4042773-6-kevin.laatz@intel.com/
> for an example.
>
> Hope that helps,

Thanks Kevin. It helped. I wasn't looking at accessing the
rte_dma_devices[] array directly. A library API would've been good.

regards,
Radha
>
> Kevin
>
>
Thomas Monjalon Oct. 6, 2021, 10:26 a.m. UTC | #4
24/09/2021 12:53, Chengwen Feng:
> The 'dmadevice' is a generic type of DMA device.

Do you mean 'dmadev' ?

> This patch introduce the 'dmadevice' device allocation APIs.
> 
> The infrastructure is prepared to welcome drivers in drivers/dma/

Good

[...]
> +The DMA library provides a DMA device framework for management and provisioning
> +of hardware and software DMA poll mode drivers, defining generic APIs which

We could consider the whole as *one* API.

> +support a number of different DMA operations.
> +
> +
> +Design Principles
> +-----------------
> +
> +The DMA library follows the same basic principles as those used in DPDK's
> +Ethernet Device framework and the RegEx framework.

Not sure what this sentence means. Better to remove.

> The DMA framework provides
> +a generic DMA device framework which supports both physical (hardware)
> +and virtual (software) DMA devices as well as a generic DMA API which allows
> +DMA devices to be managed and configured and supports DMA operations to be
> +provisioned on DMA poll mode driver.

You could split this long sentence.

[...]
> +Physical DMA controllers are discovered during the PCI probe/enumeration of the
> +EAL function which is executed at DPDK initialization, this is based on their
> +PCI BDF (bus/bridge, device, function). Specific physical DMA controllers, like
> +other physical devices in DPDK can be listed using the EAL command line options.
> +
> +The dmadevs are dynamically allocated by using the API

not API, but function

> +``rte_dma_pmd_allocate`` based on the number of hardware DMA channels. After the
> +dmadev initialized successfully, the driver needs to switch the dmadev state to
> +``RTE_DMA_DEV_READY``.

Are we sure we need these details?

> +Device Identification
> +~~~~~~~~~~~~~~~~~~~~~
> +
> +Each DMA device, whether physical or virtual is uniquely designated by two
> +identifiers:
> +
> +- A unique device index used to designate the DMA device in all functions
> +  exported by the DMA API.
> +
> +- A device name used to designate the DMA device in console messages, for
> +  administration or debugging purposes.

Good

[...]
> --- a/doc/guides/rel_notes/release_21_11.rst
> +++ b/doc/guides/rel_notes/release_21_11.rst
> @@ -106,6 +106,10 @@ New Features
>    Added command-line options to specify total number of processes and
>    current process ID. Each process owns subset of Rx and Tx queues.
>  
> +* **Introduced dmadev library with:**
> +
> +  * Device allocation APIs.

There is no API for that, it is internal.
From an user perspective, you need only to list outstanding features
in the release notes.

[...]
> +++ b/lib/dmadev/rte_dmadev.c
> +RTE_LOG_REGISTER_DEFAULT(rte_dma_logtype, INFO);
> +#define RTE_DMA_LOG(level, fmt, args...) \
> +	rte_log(RTE_LOG_ ## level, rte_dma_logtype, "%s(): " fmt "\n", \
> +		__func__, ##args)

I don't like having function name in all logs.
I recommend this form of macro:
#define RTE_DMA_LOG(level, ...) \
    rte_log(RTE_LOG_ ## level, rte_dma_logtype, RTE_FMT("dma: " \
        RTE_FMT_HEAD(__VA_ARGS__,) "\n", RTE_FMT_TAIL(__VA_ARGS__,)))

> +/* Macros to check for valid device id */
> +#define RTE_DMA_VALID_DEV_ID_OR_ERR_RET(dev_id, retval) do { \
> +	if (!rte_dma_is_valid(dev_id)) { \
> +		RTE_DMA_LOG(ERR, "Invalid dev_id=%d", dev_id); \
> +		return retval; \
> +	} \
> +} while (0)

I dislike this macro doing a return. It is hiding stuff.
I know we have it in other classes but I think it is a mistake,
we should avoid macro blocks.

> +static int16_t
> +dma_find_free_dev(void)

Actually it is looking for an ID,
so it should be dma_find_free_id.

> +{
> +	int16_t i;
> +
> +	if (rte_dma_devices == NULL)
> +		return -1;
> +
> +	for (i = 0; i < dma_devices_max; i++) {
> +		if (rte_dma_devices[i].dev_name[0] == '\0')

Instead of checking its name, it looks more logical to check the state.

> +			return i;
> +	}
> +
> +	return -1;
> +}
> +
> +static struct rte_dma_dev*
> +dma_find(const char *name)

dma_find_by_name?

[...]
> +++ b/lib/dmadev/rte_dmadev.h
> + * The dmadev are dynamically allocated by rte_dma_pmd_allocate() during the
> + * PCI/SoC device probing phase performed at EAL initialization time. And could
> + * be released by rte_dma_pmd_release() during the PCI/SoC device removing
> + * phase.

I don't think this text has value,
and we could imagine allocating a device ata later stage.

[...]
> + * Configure the maximum number of dmadevs.
> + * @note This function can be invoked before the primary process rte_eal_init()
> + * to change the maximum number of dmadevs.

You should mention what is the default.
Is the default exported to the app in this file?

> + *
> + * @param dev_max
> + *   maximum number of dmadevs.
> + *
> + * @return
> + *   0 on success. Otherwise negative value is returned.
> + */
> +__rte_experimental
> +int rte_dma_dev_max(size_t dev_max);

What about a function able to do more with the name rte_dma_init?
It should allocate the inter-process shared memory,
and do the lookup in case of secondary process.

> +/**
> + * @warning
> + * @b EXPERIMENTAL: this API may change without prior notice.
> + *
> + * Get the device identifier for the named DMA device.
> + *
> + * @param name
> + *   DMA device name.
> + *
> + * @return
> + *   Returns DMA device identifier on success.
> + *   - <0: Failure to find named DMA device.
> + */
> +__rte_experimental
> +int rte_dma_get_dev_id(const char *name);

Should we add _by_name?
We could have a function to retrieve the ID by devargs as well.

> +++ b/lib/dmadev/rte_dmadev_core.h
> +/**
> + * @file
> + *
> + * DMA Device internal header.
> + *
> + * This header contains internal data types, that are used by the DMA devices
> + * in order to expose their ops to the class.
> + *
> + * Applications should not use these API directly.

If it is not part of the API, it should not be exposed at all.
Why not having all these stuff in a file dmadev_driver.h?
Is it used by some inline functions?

[...]
> +++ b/lib/dmadev/rte_dmadev_pmd.h
> +/**
> + * @file
> + *
> + * DMA Device PMD APIs
> + *
> + * Driver facing APIs for a DMA device. These are not to be called directly by

You cannot say API for drivers, because API means application interface.
What you mean is "driver interface".

> + * any application.
> + */
[...]
> + * Allocates a new dmadev slot for an DMA device and returns the pointer
> + * to that slot for the driver to use.

Please in all comments, use the infinitive form. Example:
	Allocates -> Allocate

> + *
> + * @param name
> + *   DMA device name.
> + * @param numa_node
> + *   Driver's private data's numa node.

s/numa/NUMA/

> + * @param private_data_size
> + *   Driver's private data size.
> + *
> + * @return
> + *   A pointer to the DMA device slot case of success,
> + *   NULL otherwise.
> + */
> +__rte_internal
> +struct rte_dma_dev *rte_dma_pmd_allocate(const char *name, int numa_node,
> +					 size_t private_data_size);


OK, sorry there are a lot of comments.
Overrall, that's a good work.
I know you are in holidays, I hope we can finish during next week.
Chengwen Feng Oct. 8, 2021, 1:52 a.m. UTC | #5
On 2021/10/5 5:12, Radha Mohan wrote:
> On Fri, Sep 24, 2021 at 3:58 AM Chengwen Feng <fengchengwen@huawei.com> wrote:
>>
>> The 'dmadevice' is a generic type of DMA device.
>>
>> This patch introduce the 'dmadevice' device allocation APIs.
>>
>> The infrastructure is prepared to welcome drivers in drivers/dma/
>>
>> Signed-off-by: Chengwen Feng <fengchengwen@huawei.com>
>> Acked-by: Bruce Richardson <bruce.richardson@intel.com>
>> Acked-by: Morten Brørup <mb@smartsharesystems.com>
>> Acked-by: Jerin Jacob <jerinjacobk@gmail.com>
>> Reviewed-by: Kevin Laatz <kevin.laatz@intel.com>
>> Reviewed-by: Conor Walsh <conor.walsh@intel.com>
>> ---
>>  MAINTAINERS                            |   5 +
>>  config/rte_config.h                    |   3 +
>>  doc/api/doxy-api-index.md              |   1 +
>>  doc/api/doxy-api.conf.in               |   1 +
>>  doc/guides/dmadevs/index.rst           |  12 ++
>>  doc/guides/index.rst                   |   1 +
>>  doc/guides/prog_guide/dmadev.rst       |  64 ++++++
>>  doc/guides/prog_guide/img/dmadev.svg   | 283 +++++++++++++++++++++++++
>>  doc/guides/prog_guide/index.rst        |   1 +
>>  doc/guides/rel_notes/release_21_11.rst |   4 +
>>  drivers/dma/meson.build                |   4 +
>>  drivers/meson.build                    |   1 +
>>  lib/dmadev/meson.build                 |   7 +
>>  lib/dmadev/rte_dmadev.c                | 263 +++++++++++++++++++++++
>>  lib/dmadev/rte_dmadev.h                | 134 ++++++++++++
>>  lib/dmadev/rte_dmadev_core.h           |  51 +++++
>>  lib/dmadev/rte_dmadev_pmd.h            |  60 ++++++
>>  lib/dmadev/version.map                 |  20 ++
>>  lib/meson.build                        |   1 +
>>  19 files changed, 916 insertions(+)
>>  create mode 100644 doc/guides/dmadevs/index.rst
>>  create mode 100644 doc/guides/prog_guide/dmadev.rst
>>  create mode 100644 doc/guides/prog_guide/img/dmadev.svg
>>  create mode 100644 drivers/dma/meson.build
>>  create mode 100644 lib/dmadev/meson.build
>>  create mode 100644 lib/dmadev/rte_dmadev.c
>>  create mode 100644 lib/dmadev/rte_dmadev.h
>>  create mode 100644 lib/dmadev/rte_dmadev_core.h
>>  create mode 100644 lib/dmadev/rte_dmadev_pmd.h
>>  create mode 100644 lib/dmadev/version.map
>>
> <snip>
> Hi Chengwen,
> I see that the new version removed the "rte_dmadev_get_device_by_name()".
> What is the way to get the dmadev from inside the PMD .remove ? I am
> looking to get the dev_private as we need to do some cleanup
> operations from the remove function.

Hi Radha,

PMD should invoke rte_dma_pmd_release when .remove, and the rte_dma_pmd_release will
call dev_close ops, so that PMD could do some cleanup operations in dev_close ops.

Thanks

> 
> regards,
> Radha Mohan
> 
> .
>
Chengwen Feng Oct. 8, 2021, 7:13 a.m. UTC | #6
On 2021/10/6 18:26, Thomas Monjalon wrote:
> 24/09/2021 12:53, Chengwen Feng:
>> The 'dmadevice' is a generic type of DMA device.
> 
> Do you mean 'dmadev' ?
> 
>> This patch introduce the 'dmadevice' device allocation APIs.
>>
>> The infrastructure is prepared to welcome drivers in drivers/dma/
> 
> Good
> 
> [...]
>> +The DMA library provides a DMA device framework for management and provisioning
>> +of hardware and software DMA poll mode drivers, defining generic APIs which

[snip]

> 
> [...]
>> +++ b/lib/dmadev/rte_dmadev.h
>> + * The dmadev are dynamically allocated by rte_dma_pmd_allocate() during the
>> + * PCI/SoC device probing phase performed at EAL initialization time. And could
>> + * be released by rte_dma_pmd_release() during the PCI/SoC device removing
>> + * phase.
> 
> I don't think this text has value,
> and we could imagine allocating a device ata later stage.

Yes, we could remove the stage descriptor because it's a well-known knowledge, but I
recommend keeping the rte_dma_pmd_allocate and rte_dma_pmd_release functions, how about:

 * The dmadev are dynamically allocated by rte_dma_pmd_allocate(). And could
 * be released by rte_dma_pmd_release().

> 
> [...]
>> + * Configure the maximum number of dmadevs.
>> + * @note This function can be invoked before the primary process rte_eal_init()
>> + * to change the maximum number of dmadevs.
> 
> You should mention what is the default.
> Is the default exported to the app in this file?

The default macro is RTE_DMADEV_DEFAULT_MAX_DEVS, and I place it at rte_config.h.

I think it's better to focus on one place (rte_config.h) than to modify config in multiple places (e.g. rte_dmadev.h/rte_xxx.h).

> 
>> + *
>> + * @param dev_max
>> + *   maximum number of dmadevs.
>> + *
>> + * @return
>> + *   0 on success. Otherwise negative value is returned.
>> + */
>> +__rte_experimental
>> +int rte_dma_dev_max(size_t dev_max);
> 
> What about a function able to do more with the name rte_dma_init?
> It should allocate the inter-process shared memory,
> and do the lookup in case of secondary process.

Yes, we defined dma_data_prepare() which do above thing, it's in 4th patch.

Because we could not invoke some like allocate inter-process shared memory before
rte_eal_init, so I think it's better keep rte_dma_dev_max as it is.

> 
>> +/**
>> + * @warning
>> + * @b EXPERIMENTAL: this API may change without prior notice.
>> + *
>> + * Get the device identifier for the named DMA device.
>> + *
>> + * @param name
>> + *   DMA device name.
>> + *
>> + * @return
>> + *   Returns DMA device identifier on success.
>> + *   - <0: Failure to find named DMA device.
>> + */
>> +__rte_experimental
>> +int rte_dma_get_dev_id(const char *name);
> 
> Should we add _by_name?
> We could have a function to retrieve the ID by devargs as well.
> 
>> +++ b/lib/dmadev/rte_dmadev_core.h
>> +/**
>> + * @file
>> + *
>> + * DMA Device internal header.
>> + *
>> + * This header contains internal data types, that are used by the DMA devices
>> + * in order to expose their ops to the class.
>> + *
>> + * Applications should not use these API directly.
> 
> If it is not part of the API, it should not be exposed at all.
> Why not having all these stuff in a file dmadev_driver.h?
> Is it used by some inline functions?

Yes, it's used by dataplane inline functions.

[snip]

> .
> 

Thanks
Thomas Monjalon Oct. 8, 2021, 10:09 a.m. UTC | #7
08/10/2021 09:13, fengchengwen:
> On 2021/10/6 18:26, Thomas Monjalon wrote:
> > 24/09/2021 12:53, Chengwen Feng:
> >> +++ b/lib/dmadev/rte_dmadev.h
> >> + * The dmadev are dynamically allocated by rte_dma_pmd_allocate() during the
> >> + * PCI/SoC device probing phase performed at EAL initialization time. And could
> >> + * be released by rte_dma_pmd_release() during the PCI/SoC device removing
> >> + * phase.
> > 
> > I don't think this text has value,
> > and we could imagine allocating a device ata later stage.
> 
> Yes, we could remove the stage descriptor because it's a well-known knowledge, but I
> recommend keeping the rte_dma_pmd_allocate and rte_dma_pmd_release functions, how about:
> 
>  * The dmadev are dynamically allocated by rte_dma_pmd_allocate(). And could
>  * be released by rte_dma_pmd_release().

These functions are for PMD.
This file is for applications, so it is not appropriate.

> > [...]
> >> + * Configure the maximum number of dmadevs.
> >> + * @note This function can be invoked before the primary process rte_eal_init()
> >> + * to change the maximum number of dmadevs.
> > 
> > You should mention what is the default.
> > Is the default exported to the app in this file?
> 
> The default macro is RTE_DMADEV_DEFAULT_MAX_DEVS, and I place it at rte_config.h.

No we avoid adding thinds in rte_config.h.
There should a static default which can be changed at runtime only.

> I think it's better to focus on one place (rte_config.h) than to modify config in multiple places (e.g. rte_dmadev.h/rte_xxx.h).

Config is modified only in one place: the function.

> >> + *
> >> + * @param dev_max
> >> + *   maximum number of dmadevs.
> >> + *
> >> + * @return
> >> + *   0 on success. Otherwise negative value is returned.
> >> + */
> >> +__rte_experimental
> >> +int rte_dma_dev_max(size_t dev_max);
> > 
> > What about a function able to do more with the name rte_dma_init?
> > It should allocate the inter-process shared memory,
> > and do the lookup in case of secondary process.
> 
> Yes, we defined dma_data_prepare() which do above thing, it's in 4th patch.
> 
> Because we could not invoke some like allocate inter-process shared memory before
> rte_eal_init, so I think it's better keep rte_dma_dev_max as it is.

Good point.

> >> +++ b/lib/dmadev/rte_dmadev_core.h
> >> +/**
> >> + * @file
> >> + *
> >> + * DMA Device internal header.
> >> + *
> >> + * This header contains internal data types, that are used by the DMA devices
> >> + * in order to expose their ops to the class.
> >> + *
> >> + * Applications should not use these API directly.
> > 
> > If it is not part of the API, it should not be exposed at all.
> > Why not having all these stuff in a file dmadev_driver.h?
> > Is it used by some inline functions?
> 
> Yes, it's used by dataplane inline functions.

OK, please give this reason in the description.
diff mbox series

Patch

diff --git a/MAINTAINERS b/MAINTAINERS
index 77a549a5e8..a5b11ac70b 100644
--- a/MAINTAINERS
+++ b/MAINTAINERS
@@ -454,6 +454,11 @@  F: app/test-regex/
 F: doc/guides/prog_guide/regexdev.rst
 F: doc/guides/regexdevs/features/default.ini
 
+DMA device API - EXPERIMENTAL
+M: Chengwen Feng <fengchengwen@huawei.com>
+F: lib/dmadev/
+F: doc/guides/prog_guide/dmadev.rst
+
 Eventdev API
 M: Jerin Jacob <jerinj@marvell.com>
 T: git://dpdk.org/next/dpdk-next-eventdev
diff --git a/config/rte_config.h b/config/rte_config.h
index 590903c07d..6e397a62ab 100644
--- a/config/rte_config.h
+++ b/config/rte_config.h
@@ -70,6 +70,9 @@ 
 /* regexdev defines */
 #define RTE_MAX_REGEXDEV_DEVS 32
 
+/* dmadev defines */
+#define RTE_DMADEV_DEFAULT_MAX_DEVS 64
+
 /* eventdev defines */
 #define RTE_EVENT_MAX_DEVS 16
 #define RTE_EVENT_MAX_QUEUES_PER_DEV 255
diff --git a/doc/api/doxy-api-index.md b/doc/api/doxy-api-index.md
index 1992107a03..2939050431 100644
--- a/doc/api/doxy-api-index.md
+++ b/doc/api/doxy-api-index.md
@@ -21,6 +21,7 @@  The public API headers are grouped by topics:
   [compressdev]        (@ref rte_compressdev.h),
   [compress]           (@ref rte_comp.h),
   [regexdev]           (@ref rte_regexdev.h),
+  [dmadev]             (@ref rte_dmadev.h),
   [eventdev]           (@ref rte_eventdev.h),
   [event_eth_rx_adapter]   (@ref rte_event_eth_rx_adapter.h),
   [event_eth_tx_adapter]   (@ref rte_event_eth_tx_adapter.h),
diff --git a/doc/api/doxy-api.conf.in b/doc/api/doxy-api.conf.in
index 325a0195c6..109ec1f682 100644
--- a/doc/api/doxy-api.conf.in
+++ b/doc/api/doxy-api.conf.in
@@ -35,6 +35,7 @@  INPUT                   = @TOPDIR@/doc/api/doxy-api-index.md \
                           @TOPDIR@/lib/compressdev \
                           @TOPDIR@/lib/cryptodev \
                           @TOPDIR@/lib/distributor \
+                          @TOPDIR@/lib/dmadev \
                           @TOPDIR@/lib/efd \
                           @TOPDIR@/lib/ethdev \
                           @TOPDIR@/lib/eventdev \
diff --git a/doc/guides/dmadevs/index.rst b/doc/guides/dmadevs/index.rst
new file mode 100644
index 0000000000..0bce29d766
--- /dev/null
+++ b/doc/guides/dmadevs/index.rst
@@ -0,0 +1,12 @@ 
+.. SPDX-License-Identifier: BSD-3-Clause
+   Copyright 2021 HiSilicon Limited
+
+DMA Device Drivers
+==================
+
+The following are a list of DMA device drivers, which can be used from
+an application through DMA API.
+
+.. toctree::
+   :maxdepth: 2
+   :numbered:
diff --git a/doc/guides/index.rst b/doc/guides/index.rst
index 857f0363d3..919825992e 100644
--- a/doc/guides/index.rst
+++ b/doc/guides/index.rst
@@ -21,6 +21,7 @@  DPDK documentation
    compressdevs/index
    vdpadevs/index
    regexdevs/index
+   dmadevs/index
    eventdevs/index
    rawdevs/index
    mempool/index
diff --git a/doc/guides/prog_guide/dmadev.rst b/doc/guides/prog_guide/dmadev.rst
new file mode 100644
index 0000000000..822282213c
--- /dev/null
+++ b/doc/guides/prog_guide/dmadev.rst
@@ -0,0 +1,64 @@ 
+.. SPDX-License-Identifier: BSD-3-Clause
+   Copyright 2021 HiSilicon Limited
+
+DMA Device Library
+==================
+
+The DMA library provides a DMA device framework for management and provisioning
+of hardware and software DMA poll mode drivers, defining generic APIs which
+support a number of different DMA operations.
+
+
+Design Principles
+-----------------
+
+The DMA library follows the same basic principles as those used in DPDK's
+Ethernet Device framework and the RegEx framework. The DMA framework provides
+a generic DMA device framework which supports both physical (hardware)
+and virtual (software) DMA devices as well as a generic DMA API which allows
+DMA devices to be managed and configured and supports DMA operations to be
+provisioned on DMA poll mode driver.
+
+.. _figure_dmadev:
+
+.. figure:: img/dmadev.*
+
+The above figure shows the model on which the DMA framework is built on:
+
+ * The DMA controller could have multiple hardware DMA channels (aka. hardware
+   DMA queues), each hardware DMA channel should be represented by a dmadev.
+ * The dmadev could create multiple virtual DMA channels, each virtual DMA
+   channel represents a different transfer context. The DMA operation request
+   must be submitted to the virtual DMA channel. e.g. Application could create
+   virtual DMA channel 0 for memory-to-memory transfer scenario, and create
+   virtual DMA channel 1 for memory-to-device transfer scenario.
+
+
+Device Management
+-----------------
+
+Device Creation
+~~~~~~~~~~~~~~~
+
+Physical DMA controllers are discovered during the PCI probe/enumeration of the
+EAL function which is executed at DPDK initialization, this is based on their
+PCI BDF (bus/bridge, device, function). Specific physical DMA controllers, like
+other physical devices in DPDK can be listed using the EAL command line options.
+
+The dmadevs are dynamically allocated by using the API
+``rte_dma_pmd_allocate`` based on the number of hardware DMA channels. After the
+dmadev initialized successfully, the driver needs to switch the dmadev state to
+``RTE_DMA_DEV_READY``.
+
+
+Device Identification
+~~~~~~~~~~~~~~~~~~~~~
+
+Each DMA device, whether physical or virtual is uniquely designated by two
+identifiers:
+
+- A unique device index used to designate the DMA device in all functions
+  exported by the DMA API.
+
+- A device name used to designate the DMA device in console messages, for
+  administration or debugging purposes.
diff --git a/doc/guides/prog_guide/img/dmadev.svg b/doc/guides/prog_guide/img/dmadev.svg
new file mode 100644
index 0000000000..157d7eb7dc
--- /dev/null
+++ b/doc/guides/prog_guide/img/dmadev.svg
@@ -0,0 +1,283 @@ 
+<?xml version="1.0" encoding="UTF-8" standalone="no"?>
+<!-- Created with Inkscape (http://www.inkscape.org/) -->
+
+<!-- SPDX-License-Identifier: BSD-3-Clause -->
+<!-- Copyright(c) 2021 HiSilicon Limited -->
+
+<svg
+   width="128.64288mm"
+   height="95.477707mm"
+   viewBox="0 0 192.96433 143.21656"
+   version="1.1"
+   id="svg934"
+   inkscape:version="1.1 (c68e22c387, 2021-05-23)"
+   sodipodi:docname="dmadev.svg"
+   xmlns:inkscape="http://www.inkscape.org/namespaces/inkscape"
+   xmlns:sodipodi="http://sodipodi.sourceforge.net/DTD/sodipodi-0.dtd"
+   xmlns="http://www.w3.org/2000/svg"
+   xmlns:svg="http://www.w3.org/2000/svg">
+  <sodipodi:namedview
+     id="namedview936"
+     pagecolor="#ffffff"
+     bordercolor="#666666"
+     borderopacity="1.0"
+     inkscape:pageshadow="2"
+     inkscape:pageopacity="0.0"
+     inkscape:pagecheckerboard="0"
+     inkscape:document-units="mm"
+     showgrid="false"
+     fit-margin-top="0"
+     fit-margin-left="0"
+     fit-margin-right="0"
+     fit-margin-bottom="0"
+     inkscape:showpageshadow="false"
+     inkscape:zoom="1.332716"
+     inkscape:cx="335.03011"
+     inkscape:cy="143.69152"
+     inkscape:window-width="1920"
+     inkscape:window-height="976"
+     inkscape:window-x="-8"
+     inkscape:window-y="-8"
+     inkscape:window-maximized="1"
+     inkscape:current-layer="layer1"
+     scale-x="1.5"
+     units="mm" />
+  <defs
+     id="defs931">
+    <rect
+       x="342.43954"
+       y="106.56832"
+       width="58.257381"
+       height="137.82834"
+       id="rect17873" />
+  </defs>
+  <g
+     inkscape:label="Layer 1"
+     inkscape:groupmode="layer"
+     id="layer1"
+     transform="translate(-0.13857517,-21.527306)">
+    <rect
+       style="fill:#c9c9ff;fill-opacity:1;stroke-width:0.296755"
+       id="rect31-9"
+       width="50"
+       height="28"
+       x="0.13857517"
+       y="21.527306"
+       ry="0" />
+    <text
+       xml:space="preserve"
+       style="font-style:normal;font-weight:normal;font-size:7.05556px;line-height:1.25;font-family:sans-serif;white-space:pre;inline-size:70.1114;fill:#000000;fill-opacity:1;stroke:none;stroke-width:0.264583"
+       x="54.136707"
+       y="18.045568"
+       id="text803-1"
+       transform="translate(-49.110795,15.205683)"><tspan
+         x="54.136707"
+         y="18.045568"
+         id="tspan1045">virtual DMA </tspan><tspan
+         x="54.136707"
+         y="26.865018"
+         id="tspan1047">channel</tspan></text>
+    <rect
+       style="fill:#c9c9ff;fill-opacity:1;stroke-width:0.296755"
+       id="rect31-9-5"
+       width="50"
+       height="28"
+       x="60.138577"
+       y="21.527306"
+       ry="0" />
+    <text
+       xml:space="preserve"
+       style="font-style:normal;font-weight:normal;font-size:7.05556px;line-height:1.25;font-family:sans-serif;white-space:pre;inline-size:70.1114;fill:#000000;fill-opacity:1;stroke:none;stroke-width:0.264583"
+       x="54.136707"
+       y="18.045568"
+       id="text803-1-4"
+       transform="translate(10.512565,15.373298)"><tspan
+         x="54.136707"
+         y="18.045568"
+         id="tspan1049">virtual DMA </tspan><tspan
+         x="54.136707"
+         y="26.865018"
+         id="tspan1051">channel</tspan></text>
+    <rect
+       style="fill:#c9c9ff;fill-opacity:1;stroke-width:0.296755"
+       id="rect31-9-5-3"
+       width="50"
+       height="28"
+       x="137.43863"
+       y="21.527306"
+       ry="0" />
+    <text
+       xml:space="preserve"
+       style="font-style:normal;font-weight:normal;font-size:7.05556px;line-height:1.25;font-family:sans-serif;white-space:pre;inline-size:70.1114;fill:#000000;fill-opacity:1;stroke:none;stroke-width:0.264583"
+       x="54.136707"
+       y="18.045568"
+       id="text803-1-4-8"
+       transform="translate(88.79231,15.373299)"><tspan
+         x="54.136707"
+         y="18.045568"
+         id="tspan1053">virtual DMA </tspan><tspan
+         x="54.136707"
+         y="26.865018"
+         id="tspan1055">channel</tspan></text>
+    <text
+       xml:space="preserve"
+       transform="matrix(0.26458333,0,0,0.26458333,-0.04940429,21.408845)"
+       id="text17871"
+       style="font-style:normal;font-weight:normal;font-size:40px;line-height:1.25;font-family:sans-serif;white-space:pre;shape-inside:url(#rect17873);fill:#000000;fill-opacity:1;stroke:none" />
+    <rect
+       style="fill:#c9c9ff;fill-opacity:1;stroke-width:0.218145"
+       id="rect31-9-5-8"
+       width="38.34557"
+       height="19.729115"
+       x="36.138577"
+       y="64.827354"
+       ry="0" />
+    <text
+       xml:space="preserve"
+       style="font-style:normal;font-weight:normal;font-size:7.05556px;line-height:1.25;font-family:sans-serif;white-space:pre;inline-size:70.1114;fill:#000000;fill-opacity:1;stroke:none;stroke-width:0.264583"
+       x="54.136707"
+       y="18.045568"
+       id="text803-1-4-3"
+       transform="translate(-13.394978,59.135217)"><tspan
+         x="54.136707"
+         y="18.045568"
+         id="tspan1057">dmadev</tspan></text>
+    <rect
+       style="fill:#c9c9ff;fill-opacity:1;stroke-width:0.307089"
+       id="rect31-9-5-8-0"
+       width="60.902534"
+       height="24.616455"
+       x="25.196909"
+       y="98.47744"
+       ry="0" />
+    <text
+       xml:space="preserve"
+       style="font-style:normal;font-weight:normal;font-size:7.05556px;line-height:1.25;font-family:sans-serif;white-space:pre;inline-size:70.1114;fill:#000000;fill-opacity:1;stroke:none;stroke-width:0.264583"
+       x="54.136707"
+       y="18.045568"
+       id="text803-1-4-3-76"
+       transform="translate(-24.485484,90.97883)"><tspan
+         x="54.136707"
+         y="18.045568"
+         id="tspan1059">hardware DMA </tspan><tspan
+         x="54.136707"
+         y="26.865018"
+         id="tspan1061">channel</tspan></text>
+    <rect
+       style="fill:#c9c9ff;fill-opacity:1;stroke-width:0.307089"
+       id="rect31-9-5-8-0-6"
+       width="60.902534"
+       height="24.616455"
+       x="132.20036"
+       y="98.47744"
+       ry="0" />
+    <text
+       xml:space="preserve"
+       style="font-style:normal;font-weight:normal;font-size:7.05556px;line-height:1.25;font-family:sans-serif;white-space:pre;inline-size:70.1114;fill:#000000;fill-opacity:1;stroke:none;stroke-width:0.264583"
+       x="54.136707"
+       y="18.045568"
+       id="text803-1-4-3-76-7"
+       transform="translate(82.950904,90.79085)"><tspan
+         x="54.136707"
+         y="18.045568"
+         id="tspan1063">hardware DMA </tspan><tspan
+         x="54.136707"
+         y="26.865018"
+         id="tspan1065">channel</tspan></text>
+    <rect
+       style="fill:#c9c9ff;fill-opacity:1;stroke-width:0.307089"
+       id="rect31-9-5-8-0-4"
+       width="60.902534"
+       height="24.616455"
+       x="76.810928"
+       y="140.12741"
+       ry="0" />
+    <text
+       xml:space="preserve"
+       style="font-style:normal;font-weight:normal;font-size:7.05556px;line-height:1.25;font-family:sans-serif;white-space:pre;inline-size:70.1114;fill:#000000;fill-opacity:1;stroke:none;stroke-width:0.264583"
+       x="54.136707"
+       y="18.045568"
+       id="text803-1-4-3-76-4"
+       transform="translate(27.032341,133.10574)"><tspan
+         x="54.136707"
+         y="18.045568"
+         id="tspan1067">hardware DMA </tspan><tspan
+         x="54.136707"
+         y="26.865018"
+         id="tspan1069">controller</tspan></text>
+    <rect
+       style="fill:#c9c9ff;fill-opacity:1;stroke-width:0.218145"
+       id="rect31-9-5-8-5"
+       width="38.34557"
+       height="19.729115"
+       x="143.43863"
+       y="64.827354"
+       ry="0" />
+    <text
+       xml:space="preserve"
+       style="font-style:normal;font-weight:normal;font-size:7.05556px;line-height:1.25;font-family:sans-serif;white-space:pre;inline-size:70.1114;fill:#000000;fill-opacity:1;stroke:none;stroke-width:0.264583"
+       x="54.136707"
+       y="18.045568"
+       id="text803-1-4-3-7"
+       transform="translate(94.92597,59.664385)"><tspan
+         x="54.136707"
+         y="18.045568"
+         id="tspan1071">dmadev</tspan></text>
+    <path
+       style="fill:none;fill-rule:evenodd;stroke:#000000;stroke-width:0.264583px;stroke-linecap:butt;stroke-linejoin:miter;stroke-opacity:1"
+       d="M 74.476373,49.527306 62.82407,64.827354"
+       id="path45308"
+       inkscape:connector-type="polyline"
+       inkscape:connector-curvature="0"
+       inkscape:connection-start="#rect31-9-5"
+       inkscape:connection-end="#rect31-9-5-8" />
+    <path
+       style="fill:none;fill-rule:evenodd;stroke:#000000;stroke-width:0.264583px;stroke-linecap:butt;stroke-linejoin:miter;stroke-opacity:1"
+       d="M 35.924309,49.527306 47.711612,64.827354"
+       id="path45310"
+       inkscape:connector-type="polyline"
+       inkscape:connector-curvature="0"
+       inkscape:connection-start="#rect31-9"
+       inkscape:connection-end="#rect31-9-5-8" />
+    <path
+       style="fill:none;fill-rule:evenodd;stroke:#000000;stroke-width:0.264583px;stroke-linecap:butt;stroke-linejoin:miter;stroke-opacity:1"
+       d="M 55.403414,84.556469 55.53332,98.47744"
+       id="path45312"
+       inkscape:connector-type="polyline"
+       inkscape:connector-curvature="0"
+       inkscape:connection-start="#rect31-9-5-8"
+       inkscape:connection-end="#rect31-9-5-8-0" />
+    <path
+       style="fill:none;fill-rule:evenodd;stroke:#000000;stroke-width:0.264583px;stroke-linecap:butt;stroke-linejoin:miter;stroke-opacity:1"
+       d="m 162.62241,84.556469 0.0155,13.920971"
+       id="path45320"
+       inkscape:connector-type="polyline"
+       inkscape:connector-curvature="0"
+       inkscape:connection-start="#rect31-9-5-8-5"
+       inkscape:connection-end="#rect31-9-5-8-0-6" />
+    <path
+       style="fill:none;fill-rule:evenodd;stroke:#000000;stroke-width:0.264583px;stroke-linecap:butt;stroke-linejoin:miter;stroke-opacity:1"
+       d="m 146.28317,123.09389 -22.65252,17.03352"
+       id="path45586"
+       inkscape:connector-type="polyline"
+       inkscape:connector-curvature="0"
+       inkscape:connection-start="#rect31-9-5-8-0-6"
+       inkscape:connection-end="#rect31-9-5-8-0-4" />
+    <path
+       style="fill:none;fill-rule:evenodd;stroke:#000000;stroke-width:0.264583px;stroke-linecap:butt;stroke-linejoin:miter;stroke-opacity:1"
+       d="m 70.900938,123.09389 21.108496,17.03352"
+       id="path45588"
+       inkscape:connector-type="polyline"
+       inkscape:connector-curvature="0"
+       inkscape:connection-start="#rect31-9-5-8-0"
+       inkscape:connection-end="#rect31-9-5-8-0-4" />
+    <path
+       style="fill:none;fill-rule:evenodd;stroke:#000000;stroke-width:0.264583px;stroke-linecap:butt;stroke-linejoin:miter;stroke-opacity:1"
+       d="m 162.50039,49.527306 0.0675,15.300048"
+       id="path45956"
+       inkscape:connector-type="polyline"
+       inkscape:connector-curvature="0"
+       inkscape:connection-start="#rect31-9-5-3"
+       inkscape:connection-end="#rect31-9-5-8-5" />
+  </g>
+</svg>
diff --git a/doc/guides/prog_guide/index.rst b/doc/guides/prog_guide/index.rst
index 2dce507f46..89af28dacb 100644
--- a/doc/guides/prog_guide/index.rst
+++ b/doc/guides/prog_guide/index.rst
@@ -27,6 +27,7 @@  Programmer's Guide
     cryptodev_lib
     compressdev
     regexdev
+    dmadev
     rte_security
     rawdev
     link_bonding_poll_mode_drv_lib
diff --git a/doc/guides/rel_notes/release_21_11.rst b/doc/guides/rel_notes/release_21_11.rst
index 19356ac53c..74639f1e81 100644
--- a/doc/guides/rel_notes/release_21_11.rst
+++ b/doc/guides/rel_notes/release_21_11.rst
@@ -106,6 +106,10 @@  New Features
   Added command-line options to specify total number of processes and
   current process ID. Each process owns subset of Rx and Tx queues.
 
+* **Introduced dmadev library with:**
+
+  * Device allocation APIs.
+
 
 Removed Items
 -------------
diff --git a/drivers/dma/meson.build b/drivers/dma/meson.build
new file mode 100644
index 0000000000..a24c56d8ff
--- /dev/null
+++ b/drivers/dma/meson.build
@@ -0,0 +1,4 @@ 
+# SPDX-License-Identifier: BSD-3-Clause
+# Copyright 2021 HiSilicon Limited
+
+drivers = []
diff --git a/drivers/meson.build b/drivers/meson.build
index 3d08540581..b7d680868a 100644
--- a/drivers/meson.build
+++ b/drivers/meson.build
@@ -18,6 +18,7 @@  subdirs = [
         'vdpa',           # depends on common, bus and mempool.
         'event',          # depends on common, bus, mempool and net.
         'baseband',       # depends on common and bus.
+        'dma',            # depends on common and bus.
 ]
 
 if meson.is_cross_build()
diff --git a/lib/dmadev/meson.build b/lib/dmadev/meson.build
new file mode 100644
index 0000000000..d2fc85e8c7
--- /dev/null
+++ b/lib/dmadev/meson.build
@@ -0,0 +1,7 @@ 
+# SPDX-License-Identifier: BSD-3-Clause
+# Copyright(c) 2021 HiSilicon Limited.
+
+sources = files('rte_dmadev.c')
+headers = files('rte_dmadev.h')
+indirect_headers += files('rte_dmadev_core.h')
+driver_sdk_headers += files('rte_dmadev_pmd.h')
diff --git a/lib/dmadev/rte_dmadev.c b/lib/dmadev/rte_dmadev.c
new file mode 100644
index 0000000000..96af3f0772
--- /dev/null
+++ b/lib/dmadev/rte_dmadev.c
@@ -0,0 +1,263 @@ 
+/* SPDX-License-Identifier: BSD-3-Clause
+ * Copyright(c) 2021 HiSilicon Limited
+ * Copyright(c) 2021 Intel Corporation
+ */
+
+#include <inttypes.h>
+
+#include <rte_eal.h>
+#include <rte_lcore.h>
+#include <rte_log.h>
+#include <rte_malloc.h>
+#include <rte_memzone.h>
+#include <rte_string_fns.h>
+
+#include "rte_dmadev.h"
+#include "rte_dmadev_pmd.h"
+
+struct rte_dma_dev *rte_dma_devices;
+static int16_t dma_devices_max;
+
+RTE_LOG_REGISTER_DEFAULT(rte_dma_logtype, INFO);
+#define RTE_DMA_LOG(level, fmt, args...) \
+	rte_log(RTE_LOG_ ## level, rte_dma_logtype, "%s(): " fmt "\n", \
+		__func__, ##args)
+
+/* Macros to check for valid device id */
+#define RTE_DMA_VALID_DEV_ID_OR_ERR_RET(dev_id, retval) do { \
+	if (!rte_dma_is_valid(dev_id)) { \
+		RTE_DMA_LOG(ERR, "Invalid dev_id=%d", dev_id); \
+		return retval; \
+	} \
+} while (0)
+
+int
+rte_dma_dev_max(size_t dev_max)
+{
+	/* This function may be called before rte_eal_init(), so no rte library
+	 * function can be called in this function.
+	 */
+	if (dev_max == 0 || dev_max > INT16_MAX)
+		return -EINVAL;
+
+	if (dma_devices_max > 0)
+		return -EINVAL;
+
+	dma_devices_max = dev_max;
+
+	return 0;
+}
+
+static int
+dma_check_name(const char *name)
+{
+	size_t name_len;
+
+	if (name == NULL) {
+		RTE_DMA_LOG(ERR, "Name can't be NULL");
+		return -EINVAL;
+	}
+
+	name_len = strnlen(name, RTE_DEV_NAME_MAX_LEN);
+	if (name_len == 0) {
+		RTE_DMA_LOG(ERR, "Zero length DMA device name");
+		return -EINVAL;
+	}
+	if (name_len >= RTE_DEV_NAME_MAX_LEN) {
+		RTE_DMA_LOG(ERR, "DMA device name is too long");
+		return -EINVAL;
+	}
+
+	return 0;
+}
+
+static int16_t
+dma_find_free_dev(void)
+{
+	int16_t i;
+
+	if (rte_dma_devices == NULL)
+		return -1;
+
+	for (i = 0; i < dma_devices_max; i++) {
+		if (rte_dma_devices[i].dev_name[0] == '\0')
+			return i;
+	}
+
+	return -1;
+}
+
+static struct rte_dma_dev*
+dma_find(const char *name)
+{
+	int16_t i;
+
+	if (rte_dma_devices == NULL)
+		return NULL;
+
+	for (i = 0; i < dma_devices_max; i++) {
+		if ((rte_dma_devices[i].state != RTE_DMA_DEV_UNUSED) &&
+		    (!strcmp(name, rte_dma_devices[i].dev_name)))
+			return &rte_dma_devices[i];
+	}
+
+	return NULL;
+}
+
+static int
+dma_process_data_prepare(void)
+{
+	size_t size;
+	void *ptr;
+
+	if (rte_dma_devices != NULL)
+		return 0;
+
+	/* The return value of malloc may not be aligned to the cache line.
+	 * Therefore, extra memory is applied for realignment.
+	 * note: We do not call posix_memalign/aligned_alloc because it is
+	 * version dependent on libc.
+	 */
+	size = dma_devices_max * sizeof(struct rte_dma_dev) +
+		RTE_CACHE_LINE_SIZE;
+	ptr = malloc(size);
+	if (ptr == NULL)
+		return -ENOMEM;
+	memset(ptr, 0, size);
+
+	rte_dma_devices = RTE_PTR_ALIGN(ptr, RTE_CACHE_LINE_SIZE);
+
+	return 0;
+}
+
+static int
+dma_data_prepare(void)
+{
+	if (dma_devices_max == 0)
+		dma_devices_max = RTE_DMADEV_DEFAULT_MAX_DEVS;
+	return dma_process_data_prepare();
+}
+
+static struct rte_dma_dev *
+dma_allocate(const char *name, int numa_node, size_t private_data_size)
+{
+	struct rte_dma_dev *dev;
+	void *dev_private;
+	int16_t dev_id;
+	int ret;
+
+	ret = dma_data_prepare();
+	if (ret < 0) {
+		RTE_DMA_LOG(ERR, "Cannot initialize dmadevs data");
+		return NULL;
+	}
+
+	dev = dma_find(name);
+	if (dev != NULL) {
+		RTE_DMA_LOG(ERR, "DMA device already allocated");
+		return NULL;
+	}
+
+	dev_private = rte_zmalloc_socket(name, private_data_size,
+					 RTE_CACHE_LINE_SIZE, numa_node);
+	if (dev_private == NULL) {
+		RTE_DMA_LOG(ERR, "Cannot allocate private data");
+		return NULL;
+	}
+
+	dev_id = dma_find_free_dev();
+	if (dev_id < 0) {
+		RTE_DMA_LOG(ERR, "Reached maximum number of DMA devices");
+		rte_free(dev_private);
+		return NULL;
+	}
+
+	dev = &rte_dma_devices[dev_id];
+	dev->dev_private = dev_private;
+	rte_strscpy(dev->dev_name, name, sizeof(dev->dev_name));
+	dev->dev_id = dev_id;
+	dev->numa_node = numa_node;
+	dev->dev_private = dev_private;
+
+	return dev;
+}
+
+static void
+dma_release(struct rte_dma_dev *dev)
+{
+	rte_free(dev->dev_private);
+	memset(dev, 0, sizeof(struct rte_dma_dev));
+}
+
+struct rte_dma_dev *
+rte_dma_pmd_allocate(const char *name, int numa_node, size_t private_data_size)
+{
+	struct rte_dma_dev *dev;
+
+	if (dma_check_name(name) != 0 || private_data_size == 0)
+		return NULL;
+
+	dev = dma_allocate(name, numa_node, private_data_size);
+	if (dev == NULL)
+		return NULL;
+
+	dev->state = RTE_DMA_DEV_REGISTERED;
+
+	return dev;
+}
+
+int
+rte_dma_pmd_release(const char *name)
+{
+	struct rte_dma_dev *dev;
+
+	if (dma_check_name(name) != 0)
+		return -EINVAL;
+
+	dev = dma_find(name);
+	if (dev == NULL)
+		return -EINVAL;
+
+	dma_release(dev);
+	return 0;
+}
+
+int
+rte_dma_get_dev_id(const char *name)
+{
+	struct rte_dma_dev *dev;
+
+	if (dma_check_name(name) != 0)
+		return -EINVAL;
+
+	dev = dma_find(name);
+	if (dev == NULL)
+		return -EINVAL;
+
+	return dev->dev_id;
+}
+
+bool
+rte_dma_is_valid(int16_t dev_id)
+{
+	return (dev_id >= 0) && (dev_id < dma_devices_max) &&
+		rte_dma_devices != NULL &&
+		rte_dma_devices[dev_id].state != RTE_DMA_DEV_UNUSED;
+}
+
+uint16_t
+rte_dma_count_avail(void)
+{
+	uint16_t count = 0;
+	uint16_t i;
+
+	if (rte_dma_devices == NULL)
+		return count;
+
+	for (i = 0; i < dma_devices_max; i++) {
+		if (rte_dma_devices[i].state != RTE_DMA_DEV_UNUSED)
+			count++;
+	}
+
+	return count;
+}
diff --git a/lib/dmadev/rte_dmadev.h b/lib/dmadev/rte_dmadev.h
new file mode 100644
index 0000000000..17dc0d1226
--- /dev/null
+++ b/lib/dmadev/rte_dmadev.h
@@ -0,0 +1,134 @@ 
+/* SPDX-License-Identifier: BSD-3-Clause
+ * Copyright(c) 2021 HiSilicon Limited
+ * Copyright(c) 2021 Intel Corporation
+ * Copyright(c) 2021 Marvell International Ltd
+ * Copyright(c) 2021 SmartShare Systems
+ */
+
+#ifndef RTE_DMADEV_H
+#define RTE_DMADEV_H
+
+/**
+ * @file rte_dmadev.h
+ *
+ * DMA (Direct Memory Access) device API.
+ *
+ * The DMA framework is built on the following model:
+ *
+ *     ---------------   ---------------       ---------------
+ *     | virtual DMA |   | virtual DMA |       | virtual DMA |
+ *     | channel     |   | channel     |       | channel     |
+ *     ---------------   ---------------       ---------------
+ *            |                |                      |
+ *            ------------------                      |
+ *                     |                              |
+ *               ------------                    ------------
+ *               |  dmadev  |                    |  dmadev  |
+ *               ------------                    ------------
+ *                     |                              |
+ *            ------------------               ------------------
+ *            | HW DMA channel |               | HW DMA channel |
+ *            ------------------               ------------------
+ *                     |                              |
+ *                     --------------------------------
+ *                                     |
+ *                           ---------------------
+ *                           | HW DMA Controller |
+ *                           ---------------------
+ *
+ * The DMA controller could have multiple HW-DMA-channels (aka. HW-DMA-queues),
+ * each HW-DMA-channel should be represented by a dmadev.
+ *
+ * The dmadev could create multiple virtual DMA channels, each virtual DMA
+ * channel represents a different transfer context. The DMA operation request
+ * must be submitted to the virtual DMA channel. e.g. Application could create
+ * virtual DMA channel 0 for memory-to-memory transfer scenario, and create
+ * virtual DMA channel 1 for memory-to-device transfer scenario.
+ *
+ * The dmadev are dynamically allocated by rte_dma_pmd_allocate() during the
+ * PCI/SoC device probing phase performed at EAL initialization time. And could
+ * be released by rte_dma_pmd_release() during the PCI/SoC device removing
+ * phase.
+ *
+ * This framework uses 'int16_t dev_id' as the device identifier of a dmadev,
+ * and 'uint16_t vchan' as the virtual DMA channel identifier in one dmadev.
+ *
+ */
+
+#include <stdint.h>
+
+#include <rte_bitops.h>
+#include <rte_common.h>
+#include <rte_compat.h>
+#include <rte_dev.h>
+
+#ifdef __cplusplus
+extern "C" {
+#endif
+
+/**
+ * @warning
+ * @b EXPERIMENTAL: this API may change without prior notice.
+ *
+ * Configure the maximum number of dmadevs.
+ * @note This function can be invoked before the primary process rte_eal_init()
+ * to change the maximum number of dmadevs.
+ *
+ * @param dev_max
+ *   maximum number of dmadevs.
+ *
+ * @return
+ *   0 on success. Otherwise negative value is returned.
+ */
+__rte_experimental
+int rte_dma_dev_max(size_t dev_max);
+
+/**
+ * @warning
+ * @b EXPERIMENTAL: this API may change without prior notice.
+ *
+ * Get the device identifier for the named DMA device.
+ *
+ * @param name
+ *   DMA device name.
+ *
+ * @return
+ *   Returns DMA device identifier on success.
+ *   - <0: Failure to find named DMA device.
+ */
+__rte_experimental
+int rte_dma_get_dev_id(const char *name);
+
+/**
+ * @warning
+ * @b EXPERIMENTAL: this API may change without prior notice.
+ *
+ * @param dev_id
+ *   DMA device index.
+ *
+ * @return
+ *   - If the device index is valid (true) or not (false).
+ */
+__rte_experimental
+bool rte_dma_is_valid(int16_t dev_id);
+
+/**
+ * @warning
+ * @b EXPERIMENTAL: this API may change without prior notice.
+ *
+ * Get the total number of DMA devices that have been successfully
+ * initialised.
+ *
+ * @return
+ *   The total number of usable DMA devices.
+ */
+__rte_experimental
+uint16_t rte_dma_count_avail(void);
+
+#include "rte_dmadev_core.h"
+
+#ifdef __cplusplus
+}
+#endif
+
+#endif /* RTE_DMADEV_H */
diff --git a/lib/dmadev/rte_dmadev_core.h b/lib/dmadev/rte_dmadev_core.h
new file mode 100644
index 0000000000..5ed96853b2
--- /dev/null
+++ b/lib/dmadev/rte_dmadev_core.h
@@ -0,0 +1,51 @@ 
+/* SPDX-License-Identifier: BSD-3-Clause
+ * Copyright(c) 2021 HiSilicon Limited
+ * Copyright(c) 2021 Intel Corporation
+ */
+
+#ifndef RTE_DMADEV_CORE_H
+#define RTE_DMADEV_CORE_H
+
+/**
+ * @file
+ *
+ * DMA Device internal header.
+ *
+ * This header contains internal data types, that are used by the DMA devices
+ * in order to expose their ops to the class.
+ *
+ * Applications should not use these API directly.
+ *
+ */
+
+/**
+ * Possible states of a DMA device.
+ *
+ * @see struct rte_dmadev::state
+ */
+enum rte_dma_dev_state {
+	RTE_DMA_DEV_UNUSED = 0, /**< Device is unused. */
+	/** Device is registered, but not ready to be used. */
+	RTE_DMA_DEV_REGISTERED,
+	/** Device is ready for use. This is set by the PMD. */
+	RTE_DMA_DEV_READY,
+};
+
+/**
+ * @internal
+ * The generic data structure associated with each DMA device.
+ */
+struct rte_dma_dev {
+	char dev_name[RTE_DEV_NAME_MAX_LEN]; /**< Unique identifier name */
+	int16_t dev_id; /**< Device [external] identifier. */
+	int16_t numa_node; /**< Local NUMA memory ID. -1 if unknown. */
+	void *dev_private; /**< PMD-specific private data. */
+	/** Device info which supplied during device initialization. */
+	struct rte_device *device;
+	enum rte_dma_dev_state state; /**< Flag indicating the device state. */
+	uint64_t reserved[2]; /**< Reserved for future fields. */
+} __rte_cache_aligned;
+
+extern struct rte_dma_dev *rte_dma_devices;
+
+#endif /* RTE_DMADEV_CORE_H */
diff --git a/lib/dmadev/rte_dmadev_pmd.h b/lib/dmadev/rte_dmadev_pmd.h
new file mode 100644
index 0000000000..02281c74fd
--- /dev/null
+++ b/lib/dmadev/rte_dmadev_pmd.h
@@ -0,0 +1,60 @@ 
+/* SPDX-License-Identifier: BSD-3-Clause
+ * Copyright(c) 2021 HiSilicon Limited
+ */
+
+#ifndef RTE_DMADEV_PMD_H
+#define RTE_DMADEV_PMD_H
+
+/**
+ * @file
+ *
+ * DMA Device PMD APIs
+ *
+ * Driver facing APIs for a DMA device. These are not to be called directly by
+ * any application.
+ */
+
+#include "rte_dmadev.h"
+
+#ifdef __cplusplus
+extern "C" {
+#endif
+
+/**
+ * @internal
+ * Allocates a new dmadev slot for an DMA device and returns the pointer
+ * to that slot for the driver to use.
+ *
+ * @param name
+ *   DMA device name.
+ * @param numa_node
+ *   Driver's private data's numa node.
+ * @param private_data_size
+ *   Driver's private data size.
+ *
+ * @return
+ *   A pointer to the DMA device slot case of success,
+ *   NULL otherwise.
+ */
+__rte_internal
+struct rte_dma_dev *rte_dma_pmd_allocate(const char *name, int numa_node,
+					 size_t private_data_size);
+
+/**
+ * @internal
+ * Release the specified dmadev.
+ *
+ * @param name
+ *   DMA device name.
+ *
+ * @return
+ *   - 0 on success, negative on error.
+ */
+__rte_internal
+int rte_dma_pmd_release(const char *name);
+
+#ifdef __cplusplus
+}
+#endif
+
+#endif /* RTE_DMADEV_PMD_H */
diff --git a/lib/dmadev/version.map b/lib/dmadev/version.map
new file mode 100644
index 0000000000..56ea0332cb
--- /dev/null
+++ b/lib/dmadev/version.map
@@ -0,0 +1,20 @@ 
+EXPERIMENTAL {
+	global:
+
+	rte_dma_count_avail;
+	rte_dma_dev_max;
+	rte_dma_get_dev_id;
+	rte_dma_is_valid;
+
+	local: *;
+};
+
+INTERNAL {
+	global:
+
+	rte_dma_devices;
+	rte_dma_pmd_allocate;
+	rte_dma_pmd_release;
+
+	local: *;
+};
diff --git a/lib/meson.build b/lib/meson.build
index 1673ca4323..3dd920f5c5 100644
--- a/lib/meson.build
+++ b/lib/meson.build
@@ -45,6 +45,7 @@  libraries = [
         'pdump',
         'rawdev',
         'regexdev',
+        'dmadev',
         'rib',
         'reorder',
         'sched',