[dpdk-dev] igb_uio: allow multi-process access

  In some case, one device are accessed by different processes via
different BARs, so one uio device may be opened by more than one
process, for this case we just need to enable interrupt once, and
pci_clear_master only when the last process closed.

Fixes: 5f6ff30dc507 ("igb_uio: fix interrupt enablement after FLR in VM")

Signed-off-by: Xiao Wang <xiao.w.wang@intel.com>
---
 lib/librte_eal/linuxapp/igb_uio/igb_uio.c | 7 +++++++
 1 file changed, 7 insertions(+)
  

On Fri,  8 Dec 2017 17:57:33 -0800
Xiao Wang <xiao.w.wang@intel.com> wrote:

> In some case, one device are accessed by different processes via
> different BARs, so one uio device may be opened by more than one
> process, for this case we just need to enable interrupt once, and
> pci_clear_master only when the last process closed.
> 
> Fixes: 5f6ff30dc507 ("igb_uio: fix interrupt enablement after FLR in VM")


Yes, this makes sense.

> 
> Signed-off-by: Xiao Wang <xiao.w.wang@intel.com>
> ---
>  lib/librte_eal/linuxapp/igb_uio/igb_uio.c | 7 +++++++
>  1 file changed, 7 insertions(+)
> 
> diff --git a/lib/librte_eal/linuxapp/igb_uio/igb_uio.c b/lib/librte_eal/linuxapp/igb_uio/igb_uio.c
> index a3a98c1..c239d98 100644
> --- a/lib/librte_eal/linuxapp/igb_uio/igb_uio.c
> +++ b/lib/librte_eal/linuxapp/igb_uio/igb_uio.c
> @@ -45,6 +45,7 @@ struct rte_uio_pci_dev {
>  	struct uio_info info;
>  	struct pci_dev *pdev;
>  	enum rte_intr_mode mode;
> +	uint32_t ref_cnt;

Simple unsigned reference count is not SMP safe on all architectures.
In kernel it is recommended to use refcount_t and associated API's.
Note: refcount_t was introduced in last 2 years and some DPDK users
still have ancient kernels.

>  };
>  
>  static char *intr_mode;
> @@ -336,6 +337,9 @@ struct rte_uio_pci_dev {
>  	struct pci_dev *dev = udev->pdev;
>  	int err;
>  
> +	if (++(udev->ref_cnt) > 1)
> +		return 0;

Do not use (unnecessary) parenthesis. C precedence order is well defined.


>  	/* set bus master, which was cleared by the reset function */
>  	pci_set_master(dev);
>  
> @@ -354,6 +358,9 @@ struct rte_uio_pci_dev {
>  	struct rte_uio_pci_dev *udev = info->priv;
>  	struct pci_dev *dev = udev->pdev;
>  
> +	if (--(udev->ref_cnt) > 0)
> +		return 0;
> +
>  	/* disable interrupts */
>  	igbuio_pci_disable_interrupts(udev);
>
  

Hi,

> -----Original Message-----
> From: Stephen Hemminger [mailto:stephen@networkplumber.org]
> Sent: Tuesday, December 12, 2017 9:39 AM
> To: Wang, Xiao W <xiao.w.wang@intel.com>
> Cc: Yigit, Ferruh <ferruh.yigit@intel.com>; dev@dpdk.org
> Subject: Re: [dpdk-dev] [PATCH] igb_uio: allow multi-process access
> 
> On Fri,  8 Dec 2017 17:57:33 -0800
> Xiao Wang <xiao.w.wang@intel.com> wrote:
> 
> > In some case, one device are accessed by different processes via
> > different BARs, so one uio device may be opened by more than one
> > process, for this case we just need to enable interrupt once, and
> > pci_clear_master only when the last process closed.
> >
> > Fixes: 5f6ff30dc507 ("igb_uio: fix interrupt enablement after FLR in VM")
> 
> 
> Yes, this makes sense.
> 
> >
> > Signed-off-by: Xiao Wang <xiao.w.wang@intel.com>
> > ---
> >  lib/librte_eal/linuxapp/igb_uio/igb_uio.c | 7 +++++++
> >  1 file changed, 7 insertions(+)
> >
> > diff --git a/lib/librte_eal/linuxapp/igb_uio/igb_uio.c
> b/lib/librte_eal/linuxapp/igb_uio/igb_uio.c
> > index a3a98c1..c239d98 100644
> > --- a/lib/librte_eal/linuxapp/igb_uio/igb_uio.c
> > +++ b/lib/librte_eal/linuxapp/igb_uio/igb_uio.c
> > @@ -45,6 +45,7 @@ struct rte_uio_pci_dev {
> >  	struct uio_info info;
> >  	struct pci_dev *pdev;
> >  	enum rte_intr_mode mode;
> > +	uint32_t ref_cnt;
> 
> Simple unsigned reference count is not SMP safe on all architectures.
> In kernel it is recommended to use refcount_t and associated API's.
> Note: refcount_t was introduced in last 2 years and some DPDK users
> still have ancient kernels.

I think atomic_t associated API will be enough, without worry about kernel version.

> 
> >  };
> >
> >  static char *intr_mode;
> > @@ -336,6 +337,9 @@ struct rte_uio_pci_dev {
> >  	struct pci_dev *dev = udev->pdev;
> >  	int err;
> >
> > +	if (++(udev->ref_cnt) > 1)
> > +		return 0;
> 
> Do not use (unnecessary) parenthesis. C precedence order is well defined.

Agree. Will change it in v2.

Thanks for your comments,
Xiao
  

On Fri,  8 Dec 2017 17:57:33 -0800
Xiao Wang <xiao.w.wang@intel.com> wrote:

> In some case, one device are accessed by different processes via
> different BARs, so one uio device may be opened by more than one
> process, for this case we just need to enable interrupt once, and
> pci_clear_master only when the last process closed.
> 
> Fixes: 5f6ff30dc507 ("igb_uio: fix interrupt enablement after FLR in VM")
> 
> Signed-off-by: Xiao Wang <xiao.w.wang@intel.com>
> ---
>  lib/librte_eal/linuxapp/igb_uio/igb_uio.c | 7 +++++++
>  1 file changed, 7 insertions(+)
> 
> diff --git a/lib/librte_eal/linuxapp/igb_uio/igb_uio.c b/lib/librte_eal/linuxapp/igb_uio/igb_uio.c
> index a3a98c1..c239d98 100644
> --- a/lib/librte_eal/linuxapp/igb_uio/igb_uio.c
> +++ b/lib/librte_eal/linuxapp/igb_uio/igb_uio.c
> @@ -45,6 +45,7 @@ struct rte_uio_pci_dev {
>  	struct uio_info info;
>  	struct pci_dev *pdev;
>  	enum rte_intr_mode mode;
> +	uint32_t ref_cnt;
>  };
>  
>  static char *intr_mode;
> @@ -336,6 +337,9 @@ struct rte_uio_pci_dev {
>  	struct pci_dev *dev = udev->pdev;
>  	int err;
>  
> +	if (++(udev->ref_cnt) > 1)

Minor nit: Parenthesis is unnecessary here.

> +		return 0;
> +
>  	/* set bus master, which was cleared by the reset function */
>  	pci_set_master(dev);
>  
> @@ -354,6 +358,9 @@ struct rte_uio_pci_dev {
>  	struct rte_uio_pci_dev *udev = info->priv;
>  	struct pci_dev *dev = udev->pdev;
>  
> +	if (--(udev->ref_cnt) > 0)
> +		return 0;
> +
>  	/* disable interrupts */
>  	igbuio_pci_disable_interrupts(udev);
>  

You might consider using new reference counting (refcnt_t) in kernel which
protects against accidental under/overflow.