[dpdk-dev,11/15] eal/tile: add EAL support for global mPIPE initialization

Message ID 1418029178-25162-12-git-send-email-zlu@ezchip.com (mailing list archive)
State Superseded, archived
Headers

Commit Message

Zhigang Lu Dec. 8, 2014, 8:59 a.m. UTC
  The TileGx mPIPE hardware provides Ethernet connectivity,
packet classification, and packet load balancing services.

Signed-off-by: Zhigang Lu <zlu@ezchip.com>
Signed-off-by: Cyril Chemparathy <cchemparathy@ezchip.com>
---
 .../common/include/arch/tile/rte_mpipe.h           |  67 ++++++++++
 lib/librte_eal/linuxapp/eal/Makefile               |   3 +
 lib/librte_eal/linuxapp/eal/eal.c                  |   9 ++
 lib/librte_eal/linuxapp/eal/eal_mpipe_tile.c       | 147 +++++++++++++++++++++
 mk/rte.app.mk                                      |   4 +
 5 files changed, 230 insertions(+)
 create mode 100644 lib/librte_eal/common/include/arch/tile/rte_mpipe.h
 create mode 100644 lib/librte_eal/linuxapp/eal/eal_mpipe_tile.c
  

Comments

Neil Horman Dec. 8, 2014, 8:03 p.m. UTC | #1
On Mon, Dec 08, 2014 at 04:59:34PM +0800, Zhigang Lu wrote:
> The TileGx mPIPE hardware provides Ethernet connectivity,
> packet classification, and packet load balancing services.
> 
> Signed-off-by: Zhigang Lu <zlu@ezchip.com>
> Signed-off-by: Cyril Chemparathy <cchemparathy@ezchip.com>
> ---
>  .../common/include/arch/tile/rte_mpipe.h           |  67 ++++++++++
>  lib/librte_eal/linuxapp/eal/Makefile               |   3 +
>  lib/librte_eal/linuxapp/eal/eal.c                  |   9 ++
>  lib/librte_eal/linuxapp/eal/eal_mpipe_tile.c       | 147 +++++++++++++++++++++
>  mk/rte.app.mk                                      |   4 +
>  5 files changed, 230 insertions(+)
>  create mode 100644 lib/librte_eal/common/include/arch/tile/rte_mpipe.h
>  create mode 100644 lib/librte_eal/linuxapp/eal/eal_mpipe_tile.c
> 

This seems like the wrong way to implement mpip access.  If you want to use it
for networking access, you should create a pmd to talk to it.  If you just want
raw gxio access, you already have a gxio library that applications can interface
to.  Theres no need to create addtional DPDK api services just to wrap it up,
especially given that those surfaces won't exist outside of the tile arch (i.e.
this allows for the creation of very non-portable applications).

NAK
  
Cyril Chemparathy Dec. 8, 2014, 9:32 p.m. UTC | #2
Hi Neil,


On 12/8/2014 12:03 PM, Neil Horman wrote:
> On Mon, Dec 08, 2014 at 04:59:34PM +0800, Zhigang Lu wrote:
>> The TileGx mPIPE hardware provides Ethernet connectivity,
>> packet classification, and packet load balancing services.
>>
>> Signed-off-by: Zhigang Lu <zlu@ezchip.com>
>> Signed-off-by: Cyril Chemparathy <cchemparathy@ezchip.com>
>> ---
>>   .../common/include/arch/tile/rte_mpipe.h           |  67 ++++++++++
>>   lib/librte_eal/linuxapp/eal/Makefile               |   3 +
>>   lib/librte_eal/linuxapp/eal/eal.c                  |   9 ++
>>   lib/librte_eal/linuxapp/eal/eal_mpipe_tile.c       | 147 +++++++++++++++++++++
>>   mk/rte.app.mk                                      |   4 +
>>   5 files changed, 230 insertions(+)
>>   create mode 100644 lib/librte_eal/common/include/arch/tile/rte_mpipe.h
>>   create mode 100644 lib/librte_eal/linuxapp/eal/eal_mpipe_tile.c
>>
> This seems like the wrong way to implement mpip access.  If you want to use it
> for networking access, you should create a pmd to talk to it.  If you just want
> raw gxio access, you already have a gxio library that applications can interface
> to.  Theres no need to create addtional DPDK api services just to wrap it up,
> especially given that those surfaces won't exist outside of the tile arch (i.e.
> this allows for the creation of very non-portable applications).

Thanks for the taking a look.

The mPIPE hardware block includes hardware managed buffer pools, which 
we've abstracted under the mempool interface in the very next patch in 
the series.  As a result of this mempool dependency, mPIPE needs to be 
globally initialized on TileGX architecture.

The alternative is to not include support for the hardware managed 
buffer pool, but that decision incurs a significant performance hit.

Thanks
-- Cyril.
  
Neil Horman Dec. 9, 2014, 11:36 a.m. UTC | #3
On Mon, Dec 08, 2014 at 01:32:30PM -0800, Cyril Chemparathy wrote:
> Hi Neil,
> 
> 
> On 12/8/2014 12:03 PM, Neil Horman wrote:
> >On Mon, Dec 08, 2014 at 04:59:34PM +0800, Zhigang Lu wrote:
> >>The TileGx mPIPE hardware provides Ethernet connectivity,
> >>packet classification, and packet load balancing services.
> >>
> >>Signed-off-by: Zhigang Lu <zlu@ezchip.com>
> >>Signed-off-by: Cyril Chemparathy <cchemparathy@ezchip.com>
> >>---
> >>  .../common/include/arch/tile/rte_mpipe.h           |  67 ++++++++++
> >>  lib/librte_eal/linuxapp/eal/Makefile               |   3 +
> >>  lib/librte_eal/linuxapp/eal/eal.c                  |   9 ++
> >>  lib/librte_eal/linuxapp/eal/eal_mpipe_tile.c       | 147 +++++++++++++++++++++
> >>  mk/rte.app.mk                                      |   4 +
> >>  5 files changed, 230 insertions(+)
> >>  create mode 100644 lib/librte_eal/common/include/arch/tile/rte_mpipe.h
> >>  create mode 100644 lib/librte_eal/linuxapp/eal/eal_mpipe_tile.c
> >>
> >This seems like the wrong way to implement mpip access.  If you want to use it
> >for networking access, you should create a pmd to talk to it.  If you just want
> >raw gxio access, you already have a gxio library that applications can interface
> >to.  Theres no need to create addtional DPDK api services just to wrap it up,
> >especially given that those surfaces won't exist outside of the tile arch (i.e.
> >this allows for the creation of very non-portable applications).
> 
> Thanks for the taking a look.
> 
> The mPIPE hardware block includes hardware managed buffer pools, which we've
> abstracted under the mempool interface in the very next patch in the series.
> As a result of this mempool dependency, mPIPE needs to be globally
> initialized on TileGX architecture.
> 
Ok, but you already have a mechanism to do that, in that the application can use
the gxio library to do the initialization itself, and you don't need to provide
additional wrapper calls that are arch specific in a common library.

> The alternative is to not include support for the hardware managed buffer
> pool, but that decision incurs a significant performance hit.
> 
No, there are plenty of alternatives to just not doing it.  In fact, you can
create a constructor to do this initialization work, and manage the instance
id's so that the user never has to see it

Neil

> Thanks
> -- Cyril.
>
  

Patch

diff --git a/lib/librte_eal/common/include/arch/tile/rte_mpipe.h b/lib/librte_eal/common/include/arch/tile/rte_mpipe.h
new file mode 100644
index 0000000..11b6485
--- /dev/null
+++ b/lib/librte_eal/common/include/arch/tile/rte_mpipe.h
@@ -0,0 +1,67 @@ 
+/*-
+ *   BSD LICENSE
+ *
+ *   Copyright(c) 2014 Tilera Corporation. All rights reserved.
+ *   All rights reserved.
+ *
+ *   Redistribution and use in source and binary forms, with or without
+ *   modification, are permitted provided that the following conditions
+ *   are met:
+ *
+ *     * Redistributions of source code must retain the above copyright
+ *       notice, this list of conditions and the following disclaimer.
+ *     * Redistributions in binary form must reproduce the above copyright
+ *       notice, this list of conditions and the following disclaimer in
+ *       the documentation and/or other materials provided with the
+ *       distribution.
+ *     * Neither the name of Tilera Corporation nor the names of its
+ *       contributors may be used to endorse or promote products derived
+ *       from this software without specific prior written permission.
+ *
+ *   THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS
+ *   "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT
+ *   LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR
+ *   A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT
+ *   OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL,
+ *   SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT
+ *   LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE,
+ *   DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY
+ *   THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
+ *   (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE
+ *   OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
+ */
+
+#ifndef _RTE_TILE_MPIPE_H_
+#define _RTE_TILE_MPIPE_H_
+
+#define ARCH_ATOMIC_NO_NICKNAMES
+
+#include <gxio/mpipe.h>
+#include <arch/mpipe_xaui_def.h>
+#include <arch/mpipe_gbe_def.h>
+
+#define BSM_ALIGN_SIZE 128
+
+struct rte_eal_mpipe_channel_config {
+	int enable;
+	int first_bucket;
+	int num_buckets;
+	int headroom;
+	gxio_mpipe_rules_stacks_t stacks;
+};
+
+#define __bsm_aligned __attribute__((__aligned__(BSM_ALIGN_SIZE)))
+
+extern int rte_eal_mpipe_instances;
+
+extern int
+rte_eal_mpipe_init(void);
+
+extern gxio_mpipe_context_t *
+rte_eal_mpipe_context(int instance);
+
+extern int
+rte_eal_mpipe_channel_config(int instance, int channel,
+			     struct rte_eal_mpipe_channel_config *config);
+
+#endif /* _RTE_TILE_MPIPE_H_ */
diff --git a/lib/librte_eal/linuxapp/eal/Makefile b/lib/librte_eal/linuxapp/eal/Makefile
index 72ecf3a..99536b6 100644
--- a/lib/librte_eal/linuxapp/eal/Makefile
+++ b/lib/librte_eal/linuxapp/eal/Makefile
@@ -70,6 +70,9 @@  SRCS-$(CONFIG_RTE_LIBRTE_EAL_LINUXAPP) += eal_alarm.c
 ifeq ($(CONFIG_RTE_LIBRTE_IVSHMEM),y)
 SRCS-$(CONFIG_RTE_LIBRTE_EAL_LINUXAPP) += eal_ivshmem.c
 endif
+ifeq ($(CONFIG_RTE_ARCH),"tile")
+SRCS-$(CONFIG_RTE_LIBRTE_EAL_LINUXAPP) += eal_mpipe_tile.c
+endif
 
 # from common dir
 SRCS-$(CONFIG_RTE_LIBRTE_EAL_LINUXAPP) += eal_common_memzone.c
diff --git a/lib/librte_eal/linuxapp/eal/eal.c b/lib/librte_eal/linuxapp/eal/eal.c
index 89f3b5e..c97a090 100644
--- a/lib/librte_eal/linuxapp/eal/eal.c
+++ b/lib/librte_eal/linuxapp/eal/eal.c
@@ -78,6 +78,10 @@ 
 #include <malloc_heap.h>
 #include <rte_eth_ring.h>
 
+#ifdef RTE_ARCH_TILE
+#include <rte_mpipe.h>
+#endif
+
 #include "eal_private.h"
 #include "eal_thread.h"
 #include "eal_internal_cfg.h"
@@ -755,6 +759,11 @@  rte_eal_init(int argc, char **argv)
 	if (rte_eal_pci_init() < 0)
 		rte_panic("Cannot init PCI\n");
 
+#ifdef RTE_ARCH_TILE
+	if (rte_eal_mpipe_init() < 0)
+		rte_panic("Cannot init mPIPE\n");
+#endif
+
 #ifdef RTE_LIBRTE_IVSHMEM
 	if (rte_eal_ivshmem_init() < 0)
 		rte_panic("Cannot init IVSHMEM\n");
diff --git a/lib/librte_eal/linuxapp/eal/eal_mpipe_tile.c b/lib/librte_eal/linuxapp/eal/eal_mpipe_tile.c
new file mode 100644
index 0000000..7b0f94b
--- /dev/null
+++ b/lib/librte_eal/linuxapp/eal/eal_mpipe_tile.c
@@ -0,0 +1,147 @@ 
+/*-
+ *   BSD LICENSE
+ *
+ *   Copyright(c) 2014 Tilera Corporation. All rights reserved.
+ *
+ *   Redistribution and use in source and binary forms, with or without
+ *   modification, are permitted provided that the following conditions
+ *   are met:
+ *
+ *     * Redistributions of source code must retain the above copyright
+ *       notice, this list of conditions and the following disclaimer.
+ *     * Redistributions in binary form must reproduce the above copyright
+ *       notice, this list of conditions and the following disclaimer in
+ *       the documentation and/or other materials provided with the
+ *       distribution.
+ *     * Neither the name of Tilera Corporation nor the names of its
+ *       contributors may be used to endorse or promote products derived
+ *       from this software without specific prior written permission.
+ *
+ *   THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS
+ *   "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT
+ *   LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR
+ *   A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT
+ *   OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL,
+ *   SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT
+ *   LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE,
+ *   DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY
+ *   THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
+ *   (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE
+ *   OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
+ */
+
+#include <string.h>
+#include <stdlib.h>
+#include <stdio.h>
+#include <stdint.h>
+#include <unistd.h>
+#include <fcntl.h>
+#include <inttypes.h>
+#include <sys/mman.h>
+#include <sys/queue.h>
+#include <pthread.h>
+#include <errno.h>
+
+#include <rte_common.h>
+#include <rte_log.h>
+#include <rte_cycles.h>
+#include <rte_tailq.h>
+#include <rte_memory.h>
+#include <rte_memzone.h>
+#include <rte_spinlock.h>
+#include <rte_eal.h>
+#include <rte_debug.h>
+
+#include <rte_mpipe.h>
+
+#include "eal_private.h"
+#include "eal_internal_cfg.h"
+
+#define MPIPE_MAX_CHANNELS	128
+
+struct rte_eal_mpipe_context {
+	rte_spinlock_t        lock;
+	gxio_mpipe_context_t  context;
+	struct rte_eal_mpipe_channel_config channels[MPIPE_MAX_CHANNELS];
+};
+
+struct rte_eal_mpipe_context rte_eal_mpipe_contexts[GXIO_MPIPE_INSTANCE_MAX];
+int rte_eal_mpipe_instances;
+
+int
+rte_eal_mpipe_init(void)
+{
+	struct rte_eal_mpipe_context *context;
+	int rc, instance;
+
+	for (instance = 0; instance < GXIO_MPIPE_INSTANCE_MAX; instance++) {
+		context = &rte_eal_mpipe_contexts[instance];
+
+		rte_spinlock_init(&context->lock);
+		rc = gxio_mpipe_init(&context->context,
+				     instance);
+		if (rc < 0)
+			break;
+	}
+
+	rte_eal_mpipe_instances = instance;
+
+	return instance ? 0 : -ENODEV;
+}
+
+gxio_mpipe_context_t *
+rte_eal_mpipe_context(int instance)
+{
+	if (instance < 0 || instance >= rte_eal_mpipe_instances)
+		return NULL;
+	return &rte_eal_mpipe_contexts[instance].context;
+}
+
+int
+rte_eal_mpipe_channel_config(int instance, int channel,
+			     struct rte_eal_mpipe_channel_config *config)
+{
+	struct rte_eal_mpipe_channel_config *data;
+	struct rte_eal_mpipe_context *context;
+	gxio_mpipe_rules_t rules;
+	int idx, rc = 0;
+
+	if (instance < 0 || instance >= rte_eal_mpipe_instances ||
+	    channel < 0 || channel >= MPIPE_MAX_CHANNELS)
+		return -EINVAL;
+
+	context = &rte_eal_mpipe_contexts[instance];
+
+	rte_spinlock_lock(&context->lock);
+
+	gxio_mpipe_rules_init(&rules, &context->context);
+
+	for (idx = 0; idx < MPIPE_MAX_CHANNELS; idx++) {
+		data = (channel == idx) ? config : &context->channels[idx];
+
+		if (!data->enable)
+			continue;
+
+		rc = gxio_mpipe_rules_begin(&rules, data->first_bucket,
+					    data->num_buckets, &data->stacks);
+		if (rc < 0)
+			goto done;
+
+		rc = gxio_mpipe_rules_add_channel(&rules, idx);
+		if (rc < 0)
+			goto done;
+
+		rc = gxio_mpipe_rules_set_headroom(&rules, data->headroom);
+		if (rc < 0)
+			goto done;
+	}
+
+	rc = gxio_mpipe_rules_commit(&rules);
+	if (rc == 0)
+		memcpy(&context->channels[channel], config, sizeof(*config));
+
+done:
+	rte_spinlock_unlock(&context->lock);
+
+	return rc;
+}
diff --git a/mk/rte.app.mk b/mk/rte.app.mk
index 84ec4df..565aa77 100644
--- a/mk/rte.app.mk
+++ b/mk/rte.app.mk
@@ -170,6 +170,10 @@  LDLIBS += -lrte_pmd_xenvirt
 LDLIBS += -lxenstore
 endif
 
+ifeq ($(CONFIG_RTE_ARCH),"tile")
+LDLIBS += -lgxio
+endif
+
 ifeq ($(CONFIG_RTE_BUILD_SHARED_LIB),n)
 # plugins (link only if static libraries)