diff mbox series

[v6,1/2] bus/pci: use PCI standard sysfs entry to get PIO address

Message ID 1611890309-99135-2-git-send-email-huawei.xhw@alibaba-inc.com (mailing list archive)
State Superseded
Delegated to: David Marchand
Headers show
Series support both PIO and MMIO BAR for legacy device in virtio PMD | expand

Checks

Context Check Description
ci/checkpatch warning coding style issues

Commit Message

谢华伟(此时此刻) Jan. 29, 2021, 3:18 a.m. UTC
From: "huawei.xhw" <huawei.xhw@alibaba-inc.com>

Currently virtio PMD asssumes legacy device uses PIO bar.
There are three ways to get PIO(PortIO) address for virtio legacy device.
    under igb_uio, get pio address from uio/uio# sysfs attribute
    under uio_pci_generic:
        for X86, get PIO address from /proc/ioport
        for other ARCH, get PIO address from standard PCI sysfs attribute

Actually, igb_uio sysfs attribute exports exactly the same thing as standard PCI sysfs, i.e,
    pci_dev->resource[]

This patch fixes these messy things, and uses standard PCI sysfs attribute.

Signed-off-by: huawei xie <huawei.xhw@alibaba-inc.com>
Reviewed-by: Maxime Coquelin <maxime.coquelin@redhat.com>
---
 drivers/bus/pci/linux/pci.c     | 77 -----------------------------------------
 drivers/bus/pci/linux/pci_uio.c | 64 ++++++++++++++++++++++++----------
 2 files changed, 46 insertions(+), 95 deletions(-)

Comments

Maxime Coquelin Feb. 3, 2021, 9:37 a.m. UTC | #1
On 1/29/21 4:18 AM, 谢华伟(此时此刻) wrote:
> From: "huawei.xhw" <huawei.xhw@alibaba-inc.com>
> 
> Currently virtio PMD asssumes legacy device uses PIO bar.
> There are three ways to get PIO(PortIO) address for virtio legacy device.
>     under igb_uio, get pio address from uio/uio# sysfs attribute
>     under uio_pci_generic:
>         for X86, get PIO address from /proc/ioport
>         for other ARCH, get PIO address from standard PCI sysfs attribute
> 
> Actually, igb_uio sysfs attribute exports exactly the same thing as standard PCI sysfs, i.e,
>     pci_dev->resource[]
> 
> This patch fixes these messy things, and uses standard PCI sysfs attribute.
> 
> Signed-off-by: huawei xie <huawei.xhw@alibaba-inc.com>
> Reviewed-by: Maxime Coquelin <maxime.coquelin@redhat.com>
> ---
>  drivers/bus/pci/linux/pci.c     | 77 -----------------------------------------
>  drivers/bus/pci/linux/pci_uio.c | 64 ++++++++++++++++++++++++----------
>  2 files changed, 46 insertions(+), 95 deletions(-)
> 

Reviewed-by: Maxime Coquelin <maxime.coquelin@redhat.com>
David Marchand Feb. 18, 2021, 9:33 a.m. UTC | #2
On Fri, Jan 29, 2021 at 4:19 AM 谢华伟(此时此刻) <huawei.xhw@alibaba-inc.com> wrote:
>
> From: "huawei.xhw" <huawei.xhw@alibaba-inc.com>
>
> Currently virtio PMD asssumes legacy device uses PIO bar.
> There are three ways to get PIO(PortIO) address for virtio legacy device.
>     under igb_uio, get pio address from uio/uio# sysfs attribute
>     under uio_pci_generic:
>         for X86, get PIO address from /proc/ioport
>         for other ARCH, get PIO address from standard PCI sysfs attribute
>
> Actually, igb_uio sysfs attribute exports exactly the same thing as standard PCI sysfs, i.e,
>     pci_dev->resource[]
>
> This patch fixes these messy things, and uses standard PCI sysfs attribute.

I can not find what is being fixed.
Could you elaborate?
谢华伟(此时此刻) Feb. 21, 2021, 3:58 p.m. UTC | #3
On 2021/2/18 17:33, David Marchand wrote:
> On Fri, Jan 29, 2021 at 4:19 AM 谢华伟(此时此刻) <huawei.xhw@alibaba-inc.com> wrote:
>> From: "huawei.xhw" <huawei.xhw@alibaba-inc.com>
>>
>> Currently virtio PMD asssumes legacy device uses PIO bar.
>> There are three ways to get PIO(PortIO) address for virtio legacy device.
>>      under igb_uio, get pio address from uio/uio# sysfs attribute
>>      under uio_pci_generic:
>>          for X86, get PIO address from /proc/ioport
>>          for other ARCH, get PIO address from standard PCI sysfs attribute
>>
>> Actually, igb_uio sysfs attribute exports exactly the same thing as standard PCI sysfs, i.e,
>>      pci_dev->resource[]
>>
>> This patch fixes these messy things, and uses standard PCI sysfs attribute.
> I can not find what is being fixed.
> Could you elaborate?

Maybe i wrongly use the word fix?  With this patch, different drivers 
use the same way to get IO address for virtio device.

Previously different driver get IO address with different method.

>
David Marchand Feb. 24, 2021, 12:49 p.m. UTC | #4
On Sun, Feb 21, 2021 at 4:58 PM 谢华伟(此时此刻) <huawei.xhw@alibaba-inc.com> wrote:
>
>
> On 2021/2/18 17:33, David Marchand wrote:
> > On Fri, Jan 29, 2021 at 4:19 AM 谢华伟(此时此刻) <huawei.xhw@alibaba-inc.com> wrote:
> >> From: "huawei.xhw" <huawei.xhw@alibaba-inc.com>
> >>
> >> Currently virtio PMD asssumes legacy device uses PIO bar.
> >> There are three ways to get PIO(PortIO) address for virtio legacy device.
> >>      under igb_uio, get pio address from uio/uio# sysfs attribute
> >>      under uio_pci_generic:
> >>          for X86, get PIO address from /proc/ioport
> >>          for other ARCH, get PIO address from standard PCI sysfs attribute
> >>
> >> Actually, igb_uio sysfs attribute exports exactly the same thing as standard PCI sysfs, i.e,
> >>      pci_dev->resource[]
> >>
> >> This patch fixes these messy things, and uses standard PCI sysfs attribute.
> > I can not find what is being fixed.
> > Could you elaborate?
>
> Maybe i wrongly use the word fix?  With this patch, different drivers
> use the same way to get IO address for virtio device.
> Previously different driver get IO address with different method.

The commitlog is confusing, mentioning pci_dev->resource[] (I guess
from the kernel sources?) and talking about a fix while it works fine
so far.
So this is a refactoring.

Did you check that virtio devices bound to uio_pci_generic still works
with legacy mode + PIO?
谢华伟(此时此刻) Feb. 24, 2021, 3:29 p.m. UTC | #5
On 2021/2/24 20:49, David Marchand wrote:
> On Sun, Feb 21, 2021 at 4:58 PM 谢华伟(此时此刻) <huawei.xhw@alibaba-inc.com> wrote:
>>
>> On 2021/2/18 17:33, David Marchand wrote:
>>> On Fri, Jan 29, 2021 at 4:19 AM 谢华伟(此时此刻) <huawei.xhw@alibaba-inc.com> wrote:
>>>> From: "huawei.xhw" <huawei.xhw@alibaba-inc.com>
>>>>
>>>> Currently virtio PMD asssumes legacy device uses PIO bar.
>>>> There are three ways to get PIO(PortIO) address for virtio legacy device.
>>>>       under igb_uio, get pio address from uio/uio# sysfs attribute
>>>>       under uio_pci_generic:
>>>>           for X86, get PIO address from /proc/ioport
>>>>           for other ARCH, get PIO address from standard PCI sysfs attribute
>>>>
>>>> Actually, igb_uio sysfs attribute exports exactly the same thing as standard PCI sysfs, i.e,
>>>>       pci_dev->resource[]
>>>>
>>>> This patch fixes these messy things, and uses standard PCI sysfs attribute.
>>> I can not find what is being fixed.
>>> Could you elaborate?
>> Maybe i wrongly use the word fix?  With this patch, different drivers
>> use the same way to get IO address for virtio device.
>> Previously different driver get IO address with different method.
> The commitlog is confusing, mentioning pci_dev->resource[] (I guess
> from the kernel sources?) and talking about a fix while it works fine
Yes.
> so far.
> So this is a refactoring.
Ok, will change the word fix.
>
> Did you check that virtio devices bound to uio_pci_generic still works
> with legacy mode + PIO?

I had verified PIO, might under igb_uio driver.

Theoretically it works with uio_pci_generic driver. I could do the test 
if needed. Give me some time.  Have to create a legacy VM. Now bars in 
virtio device in the cloud are basically mmio.

>
>
David Marchand Feb. 24, 2021, 5:52 p.m. UTC | #6
On Wed, Feb 24, 2021 at 4:29 PM 谢华伟(此时此刻) <huawei.xhw@alibaba-inc.com> wrote:
> > Did you check that virtio devices bound to uio_pci_generic still works
> > with legacy mode + PIO?
>
> I had verified PIO, might under igb_uio driver.

Well, if you are unsure, please retest both cases, igb_uio and uio_pci_generic.


>
> Theoretically it works with uio_pci_generic driver. I could do the test
> if needed. Give me some time.  Have to create a legacy VM. Now bars in
> virtio device in the cloud are basically mmio.

Sure, take the time to check.
I'll wait for your confirmation, thanks.
谢华伟(此时此刻) March 1, 2021, 3:47 p.m. UTC | #7
On 2021/2/25 1:52, David Marchand wrote:
> On Wed, Feb 24, 2021 at 4:29 PM 谢华伟(此时此刻) <huawei.xhw@alibaba-inc.com> wrote:
>>> Did you check that virtio devices bound to uio_pci_generic still works
>>> with legacy mode + PIO?
>> I had verified PIO, might under igb_uio driver.
> Well, if you are unsure, please retest both cases, igb_uio and uio_pci_generic.
>
I still haven't got a VM with PIO, but retested MMIO with both igb_uio 
and uio_pci_generic. They both works.

@Chenbo

Could you help run  with PIO?

>> Theoretically it works with uio_pci_generic driver. I could do the test
>> if needed. Give me some time.  Have to create a legacy VM. Now bars in
>> virtio device in the cloud are basically mmio.
> Sure, take the time to check.
> I'll wait for your confirmation, thanks.
>
>
谢华伟(此时此刻) March 2, 2021, 12:31 p.m. UTC | #8
On 2021/2/25 1:52, David Marchand wrote:
> On Wed, Feb 24, 2021 at 4:29 PM 谢华伟(此时此刻) <huawei.xhw@alibaba-inc.com> wrote:
>>> Did you check that virtio devices bound to uio_pci_generic still works
>>> with legacy mode + PIO?
>> I had verified PIO, might under igb_uio driver.
> Well, if you are unsure, please retest both cases, igb_uio and uio_pci_generic.
>
>
>> Theoretically it works with uio_pci_generic driver. I could do the test
>> if needed. Give me some time.  Have to create a legacy VM. Now bars in
>> virtio device in the cloud are basically mmio.
> Sure, take the time to check.
> I'll wait for your confirmation, thanks.

Hi David:

Tested PIO under igb_uio and uio_pci_generic driver. Both works.

Also tested MMIO under both drivers.

>
>
diff mbox series

Patch

diff --git a/drivers/bus/pci/linux/pci.c b/drivers/bus/pci/linux/pci.c
index 2e1808b..0f38abf 100644
--- a/drivers/bus/pci/linux/pci.c
+++ b/drivers/bus/pci/linux/pci.c
@@ -677,71 +677,6 @@  int rte_pci_write_config(const struct rte_pci_device *device,
 	}
 }
 
-#if defined(RTE_ARCH_X86)
-static int
-pci_ioport_map(struct rte_pci_device *dev, int bar __rte_unused,
-		struct rte_pci_ioport *p)
-{
-	uint16_t start, end;
-	FILE *fp;
-	char *line = NULL;
-	char pci_id[16];
-	int found = 0;
-	size_t linesz;
-
-	if (rte_eal_iopl_init() != 0) {
-		RTE_LOG(ERR, EAL, "%s(): insufficient ioport permissions for PCI device %s\n",
-			__func__, dev->name);
-		return -1;
-	}
-
-	snprintf(pci_id, sizeof(pci_id), PCI_PRI_FMT,
-		 dev->addr.domain, dev->addr.bus,
-		 dev->addr.devid, dev->addr.function);
-
-	fp = fopen("/proc/ioports", "r");
-	if (fp == NULL) {
-		RTE_LOG(ERR, EAL, "%s(): can't open ioports\n", __func__);
-		return -1;
-	}
-
-	while (getdelim(&line, &linesz, '\n', fp) > 0) {
-		char *ptr = line;
-		char *left;
-		int n;
-
-		n = strcspn(ptr, ":");
-		ptr[n] = 0;
-		left = &ptr[n + 1];
-
-		while (*left && isspace(*left))
-			left++;
-
-		if (!strncmp(left, pci_id, strlen(pci_id))) {
-			found = 1;
-
-			while (*ptr && isspace(*ptr))
-				ptr++;
-
-			sscanf(ptr, "%04hx-%04hx", &start, &end);
-
-			break;
-		}
-	}
-
-	free(line);
-	fclose(fp);
-
-	if (!found)
-		return -1;
-
-	p->base = start;
-	RTE_LOG(DEBUG, EAL, "PCI Port IO found start=0x%x\n", start);
-
-	return 0;
-}
-#endif
-
 int
 rte_pci_ioport_map(struct rte_pci_device *dev, int bar,
 		struct rte_pci_ioport *p)
@@ -756,14 +691,8 @@  int rte_pci_write_config(const struct rte_pci_device *device,
 		break;
 #endif
 	case RTE_PCI_KDRV_IGB_UIO:
-		ret = pci_uio_ioport_map(dev, bar, p);
-		break;
 	case RTE_PCI_KDRV_UIO_GENERIC:
-#if defined(RTE_ARCH_X86)
-		ret = pci_ioport_map(dev, bar, p);
-#else
 		ret = pci_uio_ioport_map(dev, bar, p);
-#endif
 		break;
 	default:
 		break;
@@ -830,14 +759,8 @@  int rte_pci_write_config(const struct rte_pci_device *device,
 		break;
 #endif
 	case RTE_PCI_KDRV_IGB_UIO:
-		ret = pci_uio_ioport_unmap(p);
-		break;
 	case RTE_PCI_KDRV_UIO_GENERIC:
-#if defined(RTE_ARCH_X86)
-		ret = 0;
-#else
 		ret = pci_uio_ioport_unmap(p);
-#endif
 		break;
 	default:
 		break;
diff --git a/drivers/bus/pci/linux/pci_uio.c b/drivers/bus/pci/linux/pci_uio.c
index f3305a2..01f2a40 100644
--- a/drivers/bus/pci/linux/pci_uio.c
+++ b/drivers/bus/pci/linux/pci_uio.c
@@ -373,10 +373,13 @@ 
 pci_uio_ioport_map(struct rte_pci_device *dev, int bar,
 		   struct rte_pci_ioport *p)
 {
+	FILE *f = NULL;
 	char dirname[PATH_MAX];
 	char filename[PATH_MAX];
-	int uio_num;
-	unsigned long start;
+	char buf[BUFSIZ];
+	uint64_t phys_addr, end_addr, flags;
+	unsigned long base;
+	int i;
 
 	if (rte_eal_iopl_init() != 0) {
 		RTE_LOG(ERR, EAL, "%s(): insufficient ioport permissions for PCI device %s\n",
@@ -384,41 +387,66 @@ 
 		return -1;
 	}
 
-	uio_num = pci_get_uio_dev(dev, dirname, sizeof(dirname), 0);
-	if (uio_num < 0)
+	/* open and read addresses of the corresponding resource in sysfs */
+	snprintf(filename, sizeof(filename), "%s/" PCI_PRI_FMT "/resource",
+		rte_pci_get_sysfs_path(), dev->addr.domain, dev->addr.bus,
+		dev->addr.devid, dev->addr.function);
+	f = fopen(filename, "r");
+	if (f == NULL) {
+		RTE_LOG(ERR, EAL, "%s(): Cannot open sysfs resource: %s\n",
+			__func__, strerror(errno));
 		return -1;
+	}
 
-	/* get portio start */
-	snprintf(filename, sizeof(filename),
-		 "%s/portio/port%d/start", dirname, bar);
-	if (eal_parse_sysfs_value(filename, &start) < 0) {
-		RTE_LOG(ERR, EAL, "%s(): cannot parse portio start\n",
-			__func__);
-		return -1;
+	for (i = 0; i < bar + 1; i++) {
+		if (fgets(buf, sizeof(buf), f) == NULL) {
+			RTE_LOG(ERR, EAL, "%s(): Cannot read sysfs resource\n", __func__);
+			goto error;
+		}
 	}
-	/* ensure we don't get anything funny here, read/write will cast to
-	 * uin16_t */
-	if (start > UINT16_MAX)
-		return -1;
+	if (pci_parse_one_sysfs_resource(buf, sizeof(buf), &phys_addr,
+		&end_addr, &flags) < 0)
+		goto error;
+
+	if (!(flags & IORESOURCE_IO)) {
+		RTE_LOG(ERR, EAL, "%s(): bar resource other than IO is not supported\n", __func__);
+		goto error;
+	}
+	base = (unsigned long)phys_addr;
+	RTE_LOG(INFO, EAL, "%s(): PIO BAR %08lx detected\n", __func__, base);
+
+	if (base > UINT16_MAX)
+		goto error;
 
 	/* FIXME only for primary process ? */
 	if (dev->intr_handle.type == RTE_INTR_HANDLE_UNKNOWN) {
+		int uio_num = pci_get_uio_dev(dev, dirname, sizeof(dirname), 0);
+		if (uio_num < 0) {
+			RTE_LOG(ERR, EAL, "cannot open %s: %s\n",
+				dirname, strerror(errno));
+			goto error;
+		}
 
 		snprintf(filename, sizeof(filename), "/dev/uio%u", uio_num);
 		dev->intr_handle.fd = open(filename, O_RDWR);
 		if (dev->intr_handle.fd < 0) {
 			RTE_LOG(ERR, EAL, "Cannot open %s: %s\n",
 				filename, strerror(errno));
-			return -1;
+			goto error;
 		}
 		dev->intr_handle.type = RTE_INTR_HANDLE_UIO;
 	}
 
-	RTE_LOG(DEBUG, EAL, "PCI Port IO found start=0x%lx\n", start);
+	RTE_LOG(DEBUG, EAL, "PCI Port IO found start=0x%lx\n", base);
 
-	p->base = start;
+	p->base = base;
 	p->len = 0;
+	fclose(f);
 	return 0;
+error:
+	if (f)
+		fclose(f);
+	return -1;
 }
 #else
 int