From patchwork Wed Mar 3 18:47:04 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: =?utf-8?b?6LCi5Y2O5LyfKOatpOaXtuatpOWIu++8iQ==?= X-Patchwork-Id: 88432 X-Patchwork-Delegate: david.marchand@redhat.com Return-Path: X-Original-To: patchwork@inbox.dpdk.org Delivered-To: patchwork@inbox.dpdk.org Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id 12D9DA0561; Wed, 3 Mar 2021 19:47:22 +0100 (CET) Received: from [217.70.189.124] (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id 839FD16074D; Wed, 3 Mar 2021 19:47:17 +0100 (CET) Received: from out0-138.mail.aliyun.com (out0-138.mail.aliyun.com [140.205.0.138]) by mails.dpdk.org (Postfix) with ESMTP id 985C040683 for ; Wed, 3 Mar 2021 19:47:14 +0100 (CET) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=alibaba-inc.com; s=default; t=1614797232; h=From:To:Subject:Date:Message-Id; bh=DxBTUWM3vE/6pRd1GwWDgphpatQGN2+Ge71sBRQUHns=; b=afAhQFwy9k4mFRflqVNtaZGdfu9W7SVDjm4KmRrkPM7r+slQTZ+ZfljyzKhF2S6xsOSBDJsSRMgS4ipySxjX/bCYswfGHvBjIl8a1EY4TentgE/IhiQtmZ1yg8MyKu01c2B/GAGKS8UI9STEl+5tewNOOkWqcLo/fpj6/LERlGw= X-Alimail-AntiSpam: AC=PASS; BC=-1|-1; BR=01201311R271e4; CH=green; DM=||false|; DS=||; FP=0|-1|-1|-1|0|-1|-1|-1; HT=ay29a033018047199; MF=huawei.xhw@alibaba-inc.com; NM=1; PH=DS; RN=9; SR=0; TI=SMTPD_---.Jg0HuXE_1614797228; Received: from rs3a10040.et2sqa.z1.et2sqa.tbsite.net(mailfrom:huawei.xhw@alibaba-inc.com fp:SMTPD_---.Jg0HuXE_1614797228) by smtp.aliyun-inc.com(127.0.0.1); Thu, 04 Mar 2021 02:47:12 +0800 From: " =?utf-8?b?6LCi5Y2O5LyfKOatpOaXtuatpOWIu++8iQ==?= " To: ferruh.yigit@intel.com, maxime.coquelin@redhat.com, david.marchand@redhat.com Cc: , , , , , " =?utf-8?b?6LCi5Y2O5LyfKOatpA==?= =?utf-8?b?5pe25q2k5Yi777yJ?= " Date: Thu, 04 Mar 2021 02:47:04 +0800 Message-Id: <1614797225-114594-2-git-send-email-huawei.xhw@alibaba-inc.com> X-Mailer: git-send-email 1.8.3.1 In-Reply-To: <1614797225-114594-1-git-send-email-huawei.xhw@alibaba-inc.com> References: <1614793612-91528-1-git-send-email-huawei.xhw@alibaba-inc.com> <1614797225-114594-1-git-send-email-huawei.xhw@alibaba-inc.com> Subject: [dpdk-dev] [PATCH v10 1/2] bus/pci: use PCI standard sysfs entry to get PIO address X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Sender: "dev" From: "huawei.xhw" Currently virtio PMD asssumes legacy device uses PIO bar. There are three ways to get PIO(PortIO) address for virtio legacy device. under igb_uio, get pio address from uio/uio# sysfs attribute under uio_pci_generic: for X86, get PIO address from /proc/ioport for other ARCH, get PIO address from standard PCI sysfs attribute Actually, igb_uio sysfs attribute exports exactly the same thing as standard PCI sysfs, i.e, pci_dev->resource[] in kernel source code This patch refactors these messy things, and uses standard PCI sysfs attribute. Signed-off-by: huawei xie Reviewed-by: Maxime Coquelin --- drivers/bus/pci/linux/pci.c | 77 ----------------------------------------- drivers/bus/pci/linux/pci_uio.c | 64 ++++++++++++++++++++++++---------- 2 files changed, 46 insertions(+), 95 deletions(-) diff --git a/drivers/bus/pci/linux/pci.c b/drivers/bus/pci/linux/pci.c index 2e1808b..0f38abf 100644 --- a/drivers/bus/pci/linux/pci.c +++ b/drivers/bus/pci/linux/pci.c @@ -677,71 +677,6 @@ int rte_pci_write_config(const struct rte_pci_device *device, } } -#if defined(RTE_ARCH_X86) -static int -pci_ioport_map(struct rte_pci_device *dev, int bar __rte_unused, - struct rte_pci_ioport *p) -{ - uint16_t start, end; - FILE *fp; - char *line = NULL; - char pci_id[16]; - int found = 0; - size_t linesz; - - if (rte_eal_iopl_init() != 0) { - RTE_LOG(ERR, EAL, "%s(): insufficient ioport permissions for PCI device %s\n", - __func__, dev->name); - return -1; - } - - snprintf(pci_id, sizeof(pci_id), PCI_PRI_FMT, - dev->addr.domain, dev->addr.bus, - dev->addr.devid, dev->addr.function); - - fp = fopen("/proc/ioports", "r"); - if (fp == NULL) { - RTE_LOG(ERR, EAL, "%s(): can't open ioports\n", __func__); - return -1; - } - - while (getdelim(&line, &linesz, '\n', fp) > 0) { - char *ptr = line; - char *left; - int n; - - n = strcspn(ptr, ":"); - ptr[n] = 0; - left = &ptr[n + 1]; - - while (*left && isspace(*left)) - left++; - - if (!strncmp(left, pci_id, strlen(pci_id))) { - found = 1; - - while (*ptr && isspace(*ptr)) - ptr++; - - sscanf(ptr, "%04hx-%04hx", &start, &end); - - break; - } - } - - free(line); - fclose(fp); - - if (!found) - return -1; - - p->base = start; - RTE_LOG(DEBUG, EAL, "PCI Port IO found start=0x%x\n", start); - - return 0; -} -#endif - int rte_pci_ioport_map(struct rte_pci_device *dev, int bar, struct rte_pci_ioport *p) @@ -756,14 +691,8 @@ int rte_pci_write_config(const struct rte_pci_device *device, break; #endif case RTE_PCI_KDRV_IGB_UIO: - ret = pci_uio_ioport_map(dev, bar, p); - break; case RTE_PCI_KDRV_UIO_GENERIC: -#if defined(RTE_ARCH_X86) - ret = pci_ioport_map(dev, bar, p); -#else ret = pci_uio_ioport_map(dev, bar, p); -#endif break; default: break; @@ -830,14 +759,8 @@ int rte_pci_write_config(const struct rte_pci_device *device, break; #endif case RTE_PCI_KDRV_IGB_UIO: - ret = pci_uio_ioport_unmap(p); - break; case RTE_PCI_KDRV_UIO_GENERIC: -#if defined(RTE_ARCH_X86) - ret = 0; -#else ret = pci_uio_ioport_unmap(p); -#endif break; default: break; diff --git a/drivers/bus/pci/linux/pci_uio.c b/drivers/bus/pci/linux/pci_uio.c index f3305a2..01f2a40 100644 --- a/drivers/bus/pci/linux/pci_uio.c +++ b/drivers/bus/pci/linux/pci_uio.c @@ -373,10 +373,13 @@ pci_uio_ioport_map(struct rte_pci_device *dev, int bar, struct rte_pci_ioport *p) { + FILE *f = NULL; char dirname[PATH_MAX]; char filename[PATH_MAX]; - int uio_num; - unsigned long start; + char buf[BUFSIZ]; + uint64_t phys_addr, end_addr, flags; + unsigned long base; + int i; if (rte_eal_iopl_init() != 0) { RTE_LOG(ERR, EAL, "%s(): insufficient ioport permissions for PCI device %s\n", @@ -384,41 +387,66 @@ return -1; } - uio_num = pci_get_uio_dev(dev, dirname, sizeof(dirname), 0); - if (uio_num < 0) + /* open and read addresses of the corresponding resource in sysfs */ + snprintf(filename, sizeof(filename), "%s/" PCI_PRI_FMT "/resource", + rte_pci_get_sysfs_path(), dev->addr.domain, dev->addr.bus, + dev->addr.devid, dev->addr.function); + f = fopen(filename, "r"); + if (f == NULL) { + RTE_LOG(ERR, EAL, "%s(): Cannot open sysfs resource: %s\n", + __func__, strerror(errno)); return -1; + } - /* get portio start */ - snprintf(filename, sizeof(filename), - "%s/portio/port%d/start", dirname, bar); - if (eal_parse_sysfs_value(filename, &start) < 0) { - RTE_LOG(ERR, EAL, "%s(): cannot parse portio start\n", - __func__); - return -1; + for (i = 0; i < bar + 1; i++) { + if (fgets(buf, sizeof(buf), f) == NULL) { + RTE_LOG(ERR, EAL, "%s(): Cannot read sysfs resource\n", __func__); + goto error; + } } - /* ensure we don't get anything funny here, read/write will cast to - * uin16_t */ - if (start > UINT16_MAX) - return -1; + if (pci_parse_one_sysfs_resource(buf, sizeof(buf), &phys_addr, + &end_addr, &flags) < 0) + goto error; + + if (!(flags & IORESOURCE_IO)) { + RTE_LOG(ERR, EAL, "%s(): bar resource other than IO is not supported\n", __func__); + goto error; + } + base = (unsigned long)phys_addr; + RTE_LOG(INFO, EAL, "%s(): PIO BAR %08lx detected\n", __func__, base); + + if (base > UINT16_MAX) + goto error; /* FIXME only for primary process ? */ if (dev->intr_handle.type == RTE_INTR_HANDLE_UNKNOWN) { + int uio_num = pci_get_uio_dev(dev, dirname, sizeof(dirname), 0); + if (uio_num < 0) { + RTE_LOG(ERR, EAL, "cannot open %s: %s\n", + dirname, strerror(errno)); + goto error; + } snprintf(filename, sizeof(filename), "/dev/uio%u", uio_num); dev->intr_handle.fd = open(filename, O_RDWR); if (dev->intr_handle.fd < 0) { RTE_LOG(ERR, EAL, "Cannot open %s: %s\n", filename, strerror(errno)); - return -1; + goto error; } dev->intr_handle.type = RTE_INTR_HANDLE_UIO; } - RTE_LOG(DEBUG, EAL, "PCI Port IO found start=0x%lx\n", start); + RTE_LOG(DEBUG, EAL, "PCI Port IO found start=0x%lx\n", base); - p->base = start; + p->base = base; p->len = 0; + fclose(f); return 0; +error: + if (f) + fclose(f); + return -1; } #else int From patchwork Wed Mar 3 18:47:05 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: =?utf-8?b?6LCi5Y2O5LyfKOatpOaXtuatpOWIu++8iQ==?= X-Patchwork-Id: 88433 X-Patchwork-Delegate: david.marchand@redhat.com Return-Path: X-Original-To: patchwork@inbox.dpdk.org Delivered-To: patchwork@inbox.dpdk.org Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id 5051BA0561; Wed, 3 Mar 2021 19:47:28 +0100 (CET) Received: from [217.70.189.124] (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id D2E7D160753; Wed, 3 Mar 2021 19:47:18 +0100 (CET) Received: from out0-136.mail.aliyun.com (out0-136.mail.aliyun.com [140.205.0.136]) by mails.dpdk.org (Postfix) with ESMTP id 119B116073D for ; Wed, 3 Mar 2021 19:47:14 +0100 (CET) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=alibaba-inc.com; s=default; t=1614797233; h=From:To:Subject:Date:Message-Id; bh=sKNXfDTzC6BAFba6eH8isL4hI/CdfRwVdK1nMh2CAks=; b=e0yrpMxvDiTEtK8Ze1S7v90Cdv388LmReGQ9K9UY8yM62OKPwObv0+0cxZhI9IgFyvBq+wW6XeCFrc9F3oDk1f0/lKcVy1ZvdrdbGtyDTTjTjkRGs+U9E/Zavc43Vxmp0qB/dg3I2Yg2E0oBFTwRFkJP2vhbP6U795zNDqebaf0= X-Alimail-AntiSpam: AC=PASS; BC=-1|-1; BR=01201311R601e4; CH=green; DM=||false|; DS=||; FP=0|-1|-1|-1|0|-1|-1|-1; HT=ay29a033018047209; MF=huawei.xhw@alibaba-inc.com; NM=1; PH=DS; RN=9; SR=0; TI=SMTPD_---.Jg0HuXE_1614797228; Received: from rs3a10040.et2sqa.z1.et2sqa.tbsite.net(mailfrom:huawei.xhw@alibaba-inc.com fp:SMTPD_---.Jg0HuXE_1614797228) by smtp.aliyun-inc.com(127.0.0.1); Thu, 04 Mar 2021 02:47:13 +0800 From: " =?utf-8?b?6LCi5Y2O5LyfKOatpOaXtuatpOWIu++8iQ==?= " To: ferruh.yigit@intel.com, maxime.coquelin@redhat.com, david.marchand@redhat.com Cc: , , , , , " =?utf-8?b?6LCi5Y2O5LyfKOatpA==?= =?utf-8?b?5pe25q2k5Yi777yJ?= " Date: Thu, 04 Mar 2021 02:47:05 +0800 Message-Id: <1614797225-114594-3-git-send-email-huawei.xhw@alibaba-inc.com> X-Mailer: git-send-email 1.8.3.1 In-Reply-To: <1614797225-114594-1-git-send-email-huawei.xhw@alibaba-inc.com> References: <1614793612-91528-1-git-send-email-huawei.xhw@alibaba-inc.com> <1614797225-114594-1-git-send-email-huawei.xhw@alibaba-inc.com> Subject: [dpdk-dev] [PATCH v10 2/2] bus/pci: support MMIO in PCI ioport accessors X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Sender: "dev" From: "huawei.xhw" With I/O BAR, we get PIO(port-mapped I/O) address. With MMIO(memory-mapped I/O) BAR, we get mapped virtual address. We distinguish PIO and MMIO by their address range like how kernel does, i.e, address below 64K is PIO.. ioread/write8/16/32 is provided to access PIO/MMIO. By the way, for virtio on arch other than x86, BAR flag indicates PIO but is mapped. Signed-off-by: huawei xie Reviewed-by: Maxime Coquelin --- drivers/bus/pci/linux/pci.c | 4 -- drivers/bus/pci/linux/pci_uio.c | 156 +++++++++++++++++++++++++++++----------- 2 files changed, 113 insertions(+), 47 deletions(-) diff --git a/drivers/bus/pci/linux/pci.c b/drivers/bus/pci/linux/pci.c index 0f38abf..0dc99e9 100644 --- a/drivers/bus/pci/linux/pci.c +++ b/drivers/bus/pci/linux/pci.c @@ -715,8 +715,6 @@ int rte_pci_write_config(const struct rte_pci_device *device, break; #endif case RTE_PCI_KDRV_IGB_UIO: - pci_uio_ioport_read(p, data, len, offset); - break; case RTE_PCI_KDRV_UIO_GENERIC: pci_uio_ioport_read(p, data, len, offset); break; @@ -736,8 +734,6 @@ int rte_pci_write_config(const struct rte_pci_device *device, break; #endif case RTE_PCI_KDRV_IGB_UIO: - pci_uio_ioport_write(p, data, len, offset); - break; case RTE_PCI_KDRV_UIO_GENERIC: pci_uio_ioport_write(p, data, len, offset); break; diff --git a/drivers/bus/pci/linux/pci_uio.c b/drivers/bus/pci/linux/pci_uio.c index 01f2a40..0907051 100644 --- a/drivers/bus/pci/linux/pci_uio.c +++ b/drivers/bus/pci/linux/pci_uio.c @@ -368,6 +368,8 @@ return -1; } +#define PIO_MAX 0x10000 + #if defined(RTE_ARCH_X86) int pci_uio_ioport_map(struct rte_pci_device *dev, int bar, @@ -381,12 +383,6 @@ unsigned long base; int i; - if (rte_eal_iopl_init() != 0) { - RTE_LOG(ERR, EAL, "%s(): insufficient ioport permissions for PCI device %s\n", - __func__, dev->name); - return -1; - } - /* open and read addresses of the corresponding resource in sysfs */ snprintf(filename, sizeof(filename), "%s/" PCI_PRI_FMT "/resource", rte_pci_get_sysfs_path(), dev->addr.domain, dev->addr.bus, @@ -408,15 +404,27 @@ &end_addr, &flags) < 0) goto error; - if (!(flags & IORESOURCE_IO)) { - RTE_LOG(ERR, EAL, "%s(): bar resource other than IO is not supported\n", __func__); - goto error; - } - base = (unsigned long)phys_addr; - RTE_LOG(INFO, EAL, "%s(): PIO BAR %08lx detected\n", __func__, base); + if (flags & IORESOURCE_IO) { + if (rte_eal_iopl_init()) { + RTE_LOG(ERR, EAL, "%s(): insufficient ioport permissions for PCI device %s\n", + __func__, dev->name); + goto error; + } - if (base > UINT16_MAX) + base = (unsigned long)phys_addr; + if (base > PIO_MAX) { + RTE_LOG(ERR, EAL, "%s(): %08lx too large PIO resource\n", __func__, base); + goto error; + } + + RTE_LOG(DEBUG, EAL, "%s(): PIO BAR %08lx detected\n", __func__, base); + } else if (flags & IORESOURCE_MEM) { + base = (unsigned long)dev->mem_resource[bar].addr; + RTE_LOG(DEBUG, EAL, "%s(): MMIO BAR %08lx detected\n", __func__, base); + } else { + RTE_LOG(ERR, EAL, "%s(): unknown BAR type\n", __func__); goto error; + } /* FIXME only for primary process ? */ if (dev->intr_handle.type == RTE_INTR_HANDLE_UNKNOWN) { @@ -517,6 +525,92 @@ } #endif +#if defined(RTE_ARCH_X86) +static inline uint8_t ioread8(void *addr) +{ + uint8_t val; + + val = (uint64_t)(uintptr_t)addr >= PIO_MAX ? + *(volatile uint8_t *)addr : + inb_p((unsigned long)addr); + + return val; +} + +static inline uint16_t ioread16(void *addr) +{ + uint16_t val; + + val = (uint64_t)(uintptr_t)addr >= PIO_MAX ? + *(volatile uint16_t *)addr : + inw_p((unsigned long)addr); + + return val; +} + +static inline uint32_t ioread32(void *addr) +{ + uint32_t val; + + val = (uint64_t)(uintptr_t)addr >= PIO_MAX ? + *(volatile uint32_t *)addr : + inl_p((unsigned long)addr); + + return val; +} + +static inline void iowrite8(uint8_t val, void *addr) +{ + (uint64_t)(uintptr_t)addr >= PIO_MAX ? + *(volatile uint8_t *)addr = val : + outb_p(val, (unsigned long)addr); +} + +static inline void iowrite16(uint16_t val, void *addr) +{ + (uint64_t)(uintptr_t)addr >= PIO_MAX ? + *(volatile uint16_t *)addr = val : + outw_p(val, (unsigned long)addr); +} + +static inline void iowrite32(uint32_t val, void *addr) +{ + (uint64_t)(uintptr_t)addr >= PIO_MAX ? + *(volatile uint32_t *)addr = val : + outl_p(val, (unsigned long)addr); +} +#else +static inline uint8_t ioread8(void *addr) +{ + return *(volatile uint8_t *)addr; +} + +static inline uint16_t ioread16(void *addr) +{ + return *(volatile uint16_t *)addr; +} + +static inline uint32_t ioread32(void *addr) +{ + return *(volatile uint32_t *)addr; +} + +static inline void iowrite8(uint8_t val, void *addr) +{ + *(volatile uint8_t *)addr = val; +} + +static inline void iowrite16(uint16_t val, void *addr) +{ + *(volatile uint16_t *)addr = val; +} + +static inline void iowrite32(uint32_t val, void *addr) +{ + *(volatile uint32_t *)addr = val; +} +#endif + void pci_uio_ioport_read(struct rte_pci_ioport *p, void *data, size_t len, off_t offset) @@ -528,25 +622,13 @@ for (d = data; len > 0; d += size, reg += size, len -= size) { if (len >= 4) { size = 4; -#if defined(RTE_ARCH_X86) - *(uint32_t *)d = inl(reg); -#else - *(uint32_t *)d = *(volatile uint32_t *)reg; -#endif + *(uint32_t *)d = ioread32((void *)reg); } else if (len >= 2) { size = 2; -#if defined(RTE_ARCH_X86) - *(uint16_t *)d = inw(reg); -#else - *(uint16_t *)d = *(volatile uint16_t *)reg; -#endif + *(uint16_t *)d = ioread16((void *)reg); } else { size = 1; -#if defined(RTE_ARCH_X86) - *d = inb(reg); -#else - *d = *(volatile uint8_t *)reg; -#endif + *d = ioread8((void *)reg); } } } @@ -562,25 +644,13 @@ for (s = data; len > 0; s += size, reg += size, len -= size) { if (len >= 4) { size = 4; -#if defined(RTE_ARCH_X86) - outl_p(*(const uint32_t *)s, reg); -#else - *(volatile uint32_t *)reg = *(const uint32_t *)s; -#endif + iowrite32(*(const uint32_t *)s, (void *)reg); } else if (len >= 2) { size = 2; -#if defined(RTE_ARCH_X86) - outw_p(*(const uint16_t *)s, reg); -#else - *(volatile uint16_t *)reg = *(const uint16_t *)s; -#endif + iowrite16(*(const uint16_t *)s, (void *)reg); } else { size = 1; -#if defined(RTE_ARCH_X86) - outb_p(*s, reg); -#else - *(volatile uint8_t *)reg = *s; -#endif + iowrite8(*s, (void *)reg); } } }