List patch comments

GET /api/patches/476/comments/?format=api
HTTP 200 OK
Allow: GET, HEAD, OPTIONS
Content-Type: application/json
Link: 
<https://patches.dpdk.org/api/patches/476/comments/?format=api&page=1>; rel="first",
<https://patches.dpdk.org/api/patches/476/comments/?format=api&page=1>; rel="last"
Vary: Accept
[ { "id": 1138, "web_url": "https://patches.dpdk.org/comment/1138/", "msgid": "<E115CCD9D858EF4F90C690B0DCB4D89722626933@IRSMSX108.ger.corp.intel.com>", "list_archive_url": "https://inbox.dpdk.org/dev/E115CCD9D858EF4F90C690B0DCB4D89722626933@IRSMSX108.ger.corp.intel.com", "date": "2014-09-26T15:11:44", "subject": "Re: [dpdk-dev] [PATCH v2] distributor_app: new sample app", "submitter": { "id": 9, "url": "https://patches.dpdk.org/api/people/9/?format=api", "name": "De Lara Guarch, Pablo", "email": "pablo.de.lara.guarch@intel.com" }, "content": "Hi,\n\n> -----Original Message-----\n> From: dev [mailto:dev-bounces@dpdk.org] On Behalf Of reshmapa\n> Sent: Wednesday, September 24, 2014 3:17 PM\n> To: dev@dpdk.org\n> Subject: [dpdk-dev] [PATCH v2] distributor_app: new sample app\n> \n> From: Reshma Pattan <reshma.pattan@intel.com>\n> \n> A new sample app that shows the usage of the distributor library. This\n> app works as follows:\n> \n> * An RX thread runs which pulls packets from each ethernet port in turn\n> and passes those packets to worker using a distributor component.\n> * The workers take the packets in turn, and determine the output port\n> for those packets using basic l2forwarding doing an xor on the source\n> port id.\n> * The RX thread takes the returned packets from the workers and enqueue\n> those packets into an rte_ring structure.\n> * A TX thread pulls the packets off the rte_ring structure and then\n> sends each packet out the output port specified previously by the worker\n> * Command-line option support provided only for portmask.\n> \n> Signed-off-by: Bruce Richardson <bruce.richardson@intel.com>\n> Signed-off-by: Reshma Pattan <reshma.pattan@intel.com>\n> ---\n> examples/Makefile | 1 +\n> examples/distributor_app/Makefile | 57 ++++\n> examples/distributor_app/main.c | 585\n> ++++++++++++++++++++++++++++++++++++++\n> examples/distributor_app/main.h | 46 +++\n> 4 files changed, 689 insertions(+)\n> create mode 100644 examples/distributor_app/Makefile\n> create mode 100644 examples/distributor_app/main.c\n> create mode 100644 examples/distributor_app/main.h\n> \n> diff --git a/examples/Makefile b/examples/Makefile\n> index 6245f83..2ba82b0 100644\n> --- a/examples/Makefile\n> +++ b/examples/Makefile\n> @@ -66,5 +66,6 @@ DIRS-y += vhost\n> DIRS-$(CONFIG_RTE_LIBRTE_XEN_DOM0) += vhost_xen\n> DIRS-y += vmdq\n> DIRS-y += vmdq_dcb\n> +DIRS-$(CONFIG_RTE_LIBRTE_DISTRIBUTOR) += distributor_app\n> \n> include $(RTE_SDK)/mk/rte.extsubdir.mk\n> diff --git a/examples/distributor_app/Makefile\n> b/examples/distributor_app/Makefile\n> new file mode 100644\n> index 0000000..394785d\n> --- /dev/null\n> +++ b/examples/distributor_app/Makefile\n> @@ -0,0 +1,57 @@\n> +# BSD LICENSE\n> +#\n> +# Copyright(c) 2010-2014 Intel Corporation. All rights reserved.\n> +# All rights reserved.\n> +#\n> +# Redistribution and use in source and binary forms, with or without\n> +# modification, are permitted provided that the following conditions\n> +# are met:\n> +#\n> +# * Redistributions of source code must retain the above copyright\n> +# notice, this list of conditions and the following disclaimer.\n> +# * Redistributions in binary form must reproduce the above copyright\n> +# notice, this list of conditions and the following disclaimer in\n> +# the documentation and/or other materials provided with the\n> +# distribution.\n> +# * Neither the name of Intel Corporation nor the names of its\n> +# contributors may be used to endorse or promote products derived\n> +# from this software without specific prior written permission.\n> +#\n> +# THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND\n> CONTRIBUTORS\n> +# \"AS IS\" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT\n> NOT\n> +# LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND\n> FITNESS FOR\n> +# A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE\n> COPYRIGHT\n> +# OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT,\n> INCIDENTAL,\n> +# SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT\n> NOT\n> +# LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS\n> OF USE,\n> +# DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED\n> AND ON ANY\n> +# THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR\n> TORT\n> +# (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF\n> THE USE\n> +# OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH\n> DAMAGE.\n> +\n> +ifeq ($(RTE_SDK),)\n> +$(error \"Please define RTE_SDK environment variable\")\n> +endif\n> +\n> +# Default target, can be overriden by command line or environment\n> +RTE_TARGET ?= x86_64-default-linuxapp-gcc\n\nThis target is not present anymore. Change it to x86_64-native-linuxapp-gcc.\n\n> +\n> +include $(RTE_SDK)/mk/rte.vars.mk\n> +\n> +# binary name\n> +APP = distributor_app\n> +\n> +# all source are stored in SRCS-y\n> +SRCS-y := main.c\n> +\n> +CFLAGS += $(WERROR_FLAGS)\n> +\n> +# workaround for a gcc bug with noreturn attribute\n> +# http://gcc.gnu.org/bugzilla/show_bug.cgi?id=12603\n> +ifeq ($(CONFIG_RTE_TOOLCHAIN_GCC),y)\n> +CFLAGS_main.o += -Wno-return-type\n> +endif\n> +\n> +EXTRA_CFLAGS += -O3 -Wfatal-errors\n> +\n> +include $(RTE_SDK)/mk/rte.extapp.mk\n> diff --git a/examples/distributor_app/main.c\n> b/examples/distributor_app/main.c\n> new file mode 100644\n> index 0000000..628810a\n> --- /dev/null\n> +++ b/examples/distributor_app/main.c\n> @@ -0,0 +1,585 @@\n> +/*-\n> + * BSD LICENSE\n> + *\n> + * Copyright(c) 2010-2014 Intel Corporation. All rights reserved.\n> + * All rights reserved.\n> + *\n> + * Redistribution and use in source and binary forms, with or without\n> + * modification, are permitted provided that the following conditions\n> + * are met:\n> + *\n> + * * Redistributions of source code must retain the above copyright\n> + * notice, this list of conditions and the following disclaimer.\n> + * * Redistributions in binary form must reproduce the above copyright\n> + * notice, this list of conditions and the following disclaimer in\n> + * the documentation and/or other materials provided with the\n> + * distribution.\n> + * * Neither the name of Intel Corporation nor the names of its\n> + * contributors may be used to endorse or promote products derived\n> + * from this software without specific prior written permission.\n> + *\n> + * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND\n> CONTRIBUTORS\n> + * \"AS IS\" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT\n> NOT\n> + * LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND\n> FITNESS FOR\n> + * A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE\n> COPYRIGHT\n> + * OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT,\n> INCIDENTAL,\n> + * SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT\n> NOT\n> + * LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS\n> OF USE,\n> + * DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED\n> AND ON ANY\n> + * THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR\n> TORT\n> + * (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF\n> THE USE\n> + * OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH\n> DAMAGE.\n> + */\n> +\n> +#include <stdint.h>\n> +#include <inttypes.h>\n> +#include <unistd.h>\n> +#include <signal.h>\n> +#include <getopt.h>\n> +\n> +#include <rte_eal.h>\n> +#include <rte_ethdev.h>\n> +#include <rte_cycles.h>\n> +#include <rte_malloc.h>\n> +#include <rte_debug.h>\n> +#include <rte_distributor.h>\n> +\n> +#include \"main.h\"\n> +\n> +#define RX_RING_SIZE 256\n> +#define RX_FREE_THRESH 32\n> +#define RX_PTHRESH 8\n> +#define RX_HTHRESH 8\n> +#define RX_WTHRESH 0\n> +\n> +#define TX_RING_SIZE 512\n> +#define TX_FREE_THRESH 32\n> +#define TX_PTHRESH 32\n> +#define TX_HTHRESH 0\n> +#define TX_WTHRESH 0\n> +#define TX_RSBIT_THRESH 32\n> +#define TX_Q_FLAGS (ETH_TXQ_FLAGS_NOMULTSEGS |\n> ETH_TXQ_FLAGS_NOVLANOFFL |\\\n> +\tETH_TXQ_FLAGS_NOXSUMSCTP | ETH_TXQ_FLAGS_NOXSUMUDP |\n> \\\n> +\tETH_TXQ_FLAGS_NOXSUMTCP)\n> +\n> +#define NUM_MBUFS ((64*1024)-1)\n> +#define MBUF_SIZE (2048 + sizeof(struct rte_mbuf) +\n> RTE_PKTMBUF_HEADROOM)\n> +#define MBUF_CACHE_SIZE 250\n> +#define BURST_SIZE 32\n> +#define RTE_RING_SZ 1024\n> +\n> +/* mask of enabled ports */\n> +static uint32_t enabled_port_mask = 0;\n> +\n> +static volatile struct app_stats {\n> +\tstruct {\n> +\t\tuint64_t rx_pkts;\n> +\t\tuint64_t returned_pkts;\n> +\t\tuint64_t enqueued_pkts;\n> +\t} rx __rte_cache_aligned;\n> +\n> +\tstruct {\n> +\t\tuint64_t dequeue_pkts;\n> +\t\tuint64_t tx_pkts;\n> +\t} tx __rte_cache_aligned;\n> +} app_stats;\n> +\n> +static const struct rte_eth_conf port_conf_default = {\n> +\t.rxmode = {\n> +\t\t.mq_mode = ETH_MQ_RX_RSS,\n> +\t\t.max_rx_pkt_len = ETHER_MAX_LEN,\n> +\t\t.split_hdr_size = 0,\n> +\t\t.header_split = 0, /**< Header Split disabled */\n> +\t\t.hw_ip_checksum = 0, /**< IP checksum offload enabled */\n> +\t\t.hw_vlan_filter = 0, /**< VLAN filtering disabled */\n> +\t\t.jumbo_frame = 0, /**< Jumbo Frame Support disabled */\n> +\t\t.hw_strip_crc = 0, /**< CRC stripped by hardware */\n> +\t},\n> +\t.txmode = {\n> +\t\t.mq_mode = ETH_MQ_TX_NONE,\n> +\t},\n> +\t.lpbk_mode = 0,\n> +\t.rx_adv_conf = {\n> +\t\t\t.rss_conf = {\n> +\t\t\t\t.rss_hf = ETH_RSS_IPV4 | ETH_RSS_IPV6 |\n> +\t\t\t\t\tETH_RSS_IPV4_TCP |\n> ETH_RSS_IPV4_UDP |\n> +\t\t\t\t\tETH_RSS_IPV6_TCP |\n> ETH_RSS_IPV6_UDP,\n> +\t\t\t}\n> +\t},\n> +};\n> +\n> +static const struct rte_eth_rxconf rx_conf_default = {\n> +\t.rx_thresh = {\n> +\t\t.pthresh = RX_PTHRESH,\n> +\t\t.hthresh = RX_HTHRESH,\n> +\t\t.wthresh = RX_WTHRESH,\n> +\t},\n> +\t.rx_free_thresh = RX_FREE_THRESH,\n> +\t.rx_drop_en = 0,\n> +};\n> +\n> +static const struct rte_eth_txconf tx_conf_default = {\n> +\t.tx_thresh = {\n> +\t\t.pthresh = TX_PTHRESH,\n> +\t\t.hthresh = TX_HTHRESH,\n> +\t\t.wthresh = TX_WTHRESH,\n> +\t},\n> +\t.tx_free_thresh = TX_FREE_THRESH,\n> +\t.tx_rs_thresh = TX_RSBIT_THRESH,\n> +\t.txq_flags = TX_Q_FLAGS\n> +\n> +};\n> +\n> +struct output_buffer {\n> +\tunsigned count;\n> +\tstruct rte_mbuf *mbufs[BURST_SIZE];\n> +};\n> +\n> +/*\n> + * Initialises a given port using global settings and with the rx buffers\n> + * coming from the mbuf_pool passed as parameter\n> + */\n> +static inline int\n> +port_init(uint8_t port, struct rte_mempool *mbuf_pool)\n> +{\n> +\tstruct rte_eth_conf port_conf = port_conf_default;\n> +\tconst uint16_t rxRings = 1, txRings = rte_lcore_count() - 1;\n> +\tint retval;\n> +\tuint16_t q;\n> +\n> +\tif (port >= rte_eth_dev_count())\n> +\t\treturn -1;\n> +\n> +\tretval = rte_eth_dev_configure(port, rxRings, txRings, &port_conf);\n> +\tif (retval != 0)\n> +\t\treturn retval;\n> +\n> +\tfor (q = 0; q < rxRings; q++) {\n> +\t\tretval = rte_eth_rx_queue_setup(port, q, RX_RING_SIZE,\n> +\n> \trte_eth_dev_socket_id(port),\n> +\t\t\t\t\t\t&rx_conf_default,\n> mbuf_pool);\n> +\t\tif (retval < 0)\n> +\t\t\treturn retval;\n> +\t}\n> +\n> +\tfor (q = 0; q < txRings; q++) {\n> +\t\tretval = rte_eth_tx_queue_setup(port, q, TX_RING_SIZE,\n> +\n> \trte_eth_dev_socket_id(port),\n> +\t\t\t\t\t\t&tx_conf_default);\n> +\t\tif (retval < 0)\n> +\t\t\treturn retval;\n> +\t}\n> +\n> +\tretval = rte_eth_dev_start(port);\n> +\tif (retval < 0)\n> +\t\treturn retval;\n> +\n> +\tstruct rte_eth_link link;\n> +\trte_eth_link_get_nowait(port, &link);\n> +\tif (!link.link_status) {\n> +\t\tsleep(1);\n> +\t\trte_eth_link_get_nowait(port, &link);\n> +\t}\n> +\n> +\tif (!link.link_status) {\n> +\t\tprintf(\"Link down on port %\"PRIu8\"\\n\", port);\n> +\t\treturn 0;\n> +\t}\n> +\n> +\tstruct ether_addr addr;\n> +\trte_eth_macaddr_get(port, &addr);\n> +\tprintf(\"Port %u MAC: %02\"PRIx8\" %02\"PRIx8\" %02\"PRIx8\n> +\t\t\t\" %02\"PRIx8\" %02\"PRIx8\" %02\"PRIx8\"\\n\",\n> +\t\t\t(unsigned)port,\n> +\t\t\taddr.addr_bytes[0], addr.addr_bytes[1],\n> +\t\t\taddr.addr_bytes[2], addr.addr_bytes[3],\n> +\t\t\taddr.addr_bytes[4], addr.addr_bytes[5]);\n> +\n> +\trte_eth_promiscuous_enable(port);\n> +\n> +\treturn 0;\n> +}\n> +\n> +struct lcore_params {\n> +\tunsigned worker_id;\n> +\tstruct rte_distributor *d;\n> +\tstruct rte_ring *r;\n> +};\n> +\n> +static __attribute__((noreturn)) void\n> +lcore_rx(struct lcore_params *p)\n> +{\n> +\tstruct rte_distributor *d = p->d;\n> +\tstruct rte_ring *r = p->r;\n> +\tconst uint8_t nb_ports = rte_eth_dev_count();\n> +\tconst int socket_id = rte_socket_id();\n> +\tuint8_t port;\n> +\n> +\tfor (port = 0; port < nb_ports; port++) {\n> +\t\t/* skip ports that are not enabled */\n> +\t\tif ((enabled_port_mask & (1 << port)) == 0)\n> +\t\t\tcontinue;\n> +\n> +\t\tif (rte_eth_dev_socket_id(port) > 0 &&\n> +\t\t\t\trte_eth_dev_socket_id(port) != socket_id)\n> +\t\t\tprintf(\"WARNING, port %u is on remote NUMA node\n> to \"\n> +\t\t\t\t\t\"RX thread.\\n\\tPerformance will not\n> \"\n> +\t\t\t\t\t\"be optimal.\\n\", port);\n> +\t}\n> +\n> +\tprintf(\"\\nCore %u doing packet RX.\\n\", rte_lcore_id());\n> +\tport = 0;\n> +\tfor (;;) {\n> +\t\t/* skip ports that are not enabled */\n> +\t\tif ((enabled_port_mask & (1 << port)) == 0) {\n> +\t\t\tif (++port == nb_ports)\n> +\t\t\t\tport = 0;\n> +\t\t\tcontinue;\n> +\t\t}\n> +\t\tstruct rte_mbuf *bufs[BURST_SIZE*2];\n> +\t\tconst uint16_t nb_rx = rte_eth_rx_burst(port, 0, bufs,\n> +\t\t\t\tBURST_SIZE);\n> +\t\tapp_stats.rx.rx_pkts += nb_rx;\n> +\n> +\t\trte_distributor_process(d, bufs, nb_rx);\n> +\t\tconst uint16_t nb_ret = rte_distributor_returned_pkts(d,\n> +\t\t\t\tbufs, BURST_SIZE*2);\n> +\t\tapp_stats.rx.returned_pkts += nb_ret;\n> +\t\tif (unlikely(nb_ret == 0))\n> +\t\t\tcontinue;\n> +\n> +\t\tuint16_t sent = rte_ring_enqueue_burst(r, (void *)bufs,\n> nb_ret);\n> +\t\tapp_stats.rx.enqueued_pkts += sent;\n> +\t\tif (unlikely(sent < nb_ret)) {\n> +\t\t\tprintf(\"Packet loss due to full ring\\n\");\n\nI would remove this printf. If core is too busy to process packets, this may make performance worse.\nStatistics should reflect this packet loss.\n\n> +\t\t\twhile (sent < nb_ret)\n> +\t\t\t\trte_pktmbuf_free(bufs[sent++]);\n> +\t\t}\n> +\t\tif (++port == nb_ports)\n> +\t\t\tport = 0;\n> +\t}\n> +}\n> +\n> +static inline void\n> +flush_one_port(struct output_buffer *outbuf, uint8_t outp)\n> +{\n> +\tunsigned nb_tx = rte_eth_tx_burst(outp, 0, outbuf->mbufs,\n> +\t\t\toutbuf->count);\n> +\tapp_stats.tx.tx_pkts += nb_tx;\n> +\n> +\tif (unlikely(nb_tx < outbuf->count)) {\n> +\t\tprintf(\"Packet loss with tx_burst\\n\");\n\nSame here. This is fine for debugging, but statistics are updated as well, so we do not need this printf.\n\n> +\t\tdo {\n> +\t\t\trte_pktmbuf_free(outbuf->mbufs[nb_tx]);\n> +\t\t} while (++nb_tx < outbuf->count);\n> +\t}\n> +\toutbuf->count = 0;\n> +}\n> +\n> +static inline void\n> +flush_all_ports(struct output_buffer *tx_buffers, uint8_t nb_ports)\n> +{\n> +\tuint8_t outp;\n> +\tfor (outp = 0; outp < nb_ports; outp++) {\n> +\t\t/* skip ports that are not enabled */\n> +\t\tif ((enabled_port_mask & (1 << outp)) == 0)\n> +\t\t\tcontinue;\n> +\n> +\t\tif (tx_buffers[outp].count == 0)\n> +\t\t\tcontinue;\n> +\n> +\t\tflush_one_port(&tx_buffers[outp], outp);\n> +\t}\n> +}\n> +\n> +static __attribute__((noreturn)) void\n> +lcore_tx(struct rte_ring *in_r)\n> +{\n> +\tstatic struct output_buffer tx_buffers[RTE_MAX_ETHPORTS];\n> +\tconst uint8_t nb_ports = rte_eth_dev_count();\n> +\tconst int socket_id = rte_socket_id();\n> +\tuint8_t port;\n> +\n> +\tfor (port = 0; port < nb_ports; port++) {\n> +\t\t/* skip ports that are not enabled */\n> +\t\tif ((enabled_port_mask & (1 << port)) == 0)\n> +\t\t\tcontinue;\n> +\n> +\t\tif (rte_eth_dev_socket_id(port) > 0 &&\n> +\t\t\t\trte_eth_dev_socket_id(port) != socket_id)\n> +\t\t\tprintf(\"WARNING, port %u is on remote NUMA node\n> to \"\n> +\t\t\t\t\t\"TX thread.\\n\\tPerformance will not \"\n> +\t\t\t\t\t\"be optimal.\\n\", port);\n> +\t}\n> +\n> +\tprintf(\"\\nCore %u doing packet TX.\\n\", rte_lcore_id());\n> +\tfor (;;) {\n> +\t\tfor (port = 0; port < nb_ports; port++) {\n> +\t\t\t/* skip ports that are not enabled */\n> +\t\t\tif ((enabled_port_mask & (1 << port)) == 0)\n> +\t\t\t\tcontinue;\n> +\n> +\t\t\tstruct rte_mbuf *bufs[BURST_SIZE];\n> +\t\t\tconst uint16_t nb_rx = rte_ring_dequeue_burst(in_r,\n> +\t\t\t\t\t(void *)bufs, BURST_SIZE);\n> +\t\t\tapp_stats.tx.dequeue_pkts += nb_rx;\n> +\n> +\t\t\t/* if we get no traffic, flush anything we have */\n> +\t\t\tif (unlikely(nb_rx == 0)) {\n> +\t\t\t\tflush_all_ports(tx_buffers, nb_ports);\n> +\t\t\t\tcontinue;\n> +\t\t\t}\n> +\n> +\t\t\t/* for traffic we receive, queue it up for transmit */\n> +\t\t\tuint16_t i;\n> +\t\t\t_mm_prefetch(bufs[0], 0);\n> +\t\t\t_mm_prefetch(bufs[1], 0);\n> +\t\t\t_mm_prefetch(bufs[2], 0);\n> +\t\t\tfor (i = 0; i < nb_rx; i++) {\n> +\t\t\t\tstruct output_buffer *outbuf;\n> +\t\t\t\tuint8_t outp;\n> +\t\t\t\t_mm_prefetch(bufs[i + 3], 0);\n> +\t\t\t\t/* workers should update in_port to hold the\n> +\t\t\t\t * output port value */\n> +\t\t\t\toutp = bufs[i]->port;\n> +\t\t\t\t/* skip ports that are not enabled */\n> +\t\t\t\tif ((enabled_port_mask & (1 << outp)) == 0)\n> +\t\t\t\t\tcontinue;\n> +\n> +\t\t\t\toutbuf = &tx_buffers[outp];\n> +\t\t\t\toutbuf->mbufs[outbuf->count++] = bufs[i];\n> +\t\t\t\tif (outbuf->count == BURST_SIZE)\n> +\t\t\t\t\tflush_one_port(outbuf, outp);\n> +\t\t\t}\n> +\t\t}\n> +\t}\n> +}\n> +\n> +\n> +static __attribute__((noreturn)) void\n> +lcore_worker(struct lcore_params *p)\n> +{\n> +\tstruct rte_distributor *d = p->d;\n> +\tconst unsigned id = p->worker_id;\n> +\t/* for single port, xor_val will be zero so we won't modify the output\n> +\t * port, otherwise we send traffic from 0 to 1, 2 to 3, and vice versa\n> +\t */\n> +\tconst unsigned xor_val = (rte_eth_dev_count() > 1);\n> +\tstruct rte_mbuf *buf = NULL;\n> +\n> +\tprintf(\"\\nCore %u acting as worker core.\\n\", rte_lcore_id());\n> +\tfor (;;) {\n> +\t\tbuf = rte_distributor_get_pkt(d, id, buf);\n> +\t\tbuf->port ^= xor_val;\n> +\t}\n> +}\n> +\n> +static void\n> +int_handler(int sig_num)\n> +{\n> +\tstruct rte_eth_stats eth_stats;\n> +\tunsigned i;\n> +\n> +\tprintf(\"Exiting on signal %d\\n\", sig_num);\n> +\n> +\tprintf(\"\\nRX thread stats:\\n\");\n> +\tprintf(\" - Received: %\"PRIu64\"\\n\", app_stats.rx.rx_pkts);\n> +\tprintf(\" - Processed: %\"PRIu64\"\\n\", app_stats.rx.returned_pkts);\n> +\tprintf(\" - Enqueued: %\"PRIu64\"\\n\", app_stats.rx.enqueued_pkts);\n> +\n> +\tprintf(\"\\nTX thread stats:\\n\");\n> +\tprintf(\" - Dequeued: %\"PRIu64\"\\n\", app_stats.tx.dequeue_pkts);\n> +\tprintf(\" - Transmitted: %\"PRIu64\"\\n\", app_stats.tx.tx_pkts);\n> +\n> +\tfor (i = 0; i < rte_eth_dev_count(); i++) {\n> +\t\trte_eth_stats_get(i, &eth_stats);\n> +\t\tprintf(\"\\nPort %u stats:\\n\", i);\n> +\t\tprintf(\" - Pkts in: %\"PRIu64\"\\n\", eth_stats.ipackets);\n> +\t\tprintf(\" - Pkts out: %\"PRIu64\"\\n\", eth_stats.opackets);\n> +\t\tprintf(\" - In Errs: %\"PRIu64\"\\n\", eth_stats.ierrors);\n> +\t\tprintf(\" - Out Errs: %\"PRIu64\"\\n\", eth_stats.oerrors);\n> +\t\tprintf(\" - Mbuf Errs: %\"PRIu64\"\\n\", eth_stats.rx_nombuf);\n> +\t}\n> +\texit(0);\n> +}\n> +\n> +/* display usage */\n> +static void\n> +print_usage(const char *prgname)\n> +{\n> +\tprintf(\"%s [EAL options] -- -p PORTMASK\\n\"\n> +\t\t\t\" -p PORTMASK: hexadecimal bitmask of ports to\n> configure\\n\",\n> +\t\t\tprgname);\n> +}\n> +\n> +static int\n> +parse_portmask(const char *portmask)\n> +{\n> +\tchar *end = NULL;\n> +\tunsigned long pm;\n> +\n> +\t/* parse hexadecimal string */\n> +\tpm = strtoul(portmask, &end, 16);\n> +\tif ((portmask[0] == '\\0') || (end == NULL) || (*end != '\\0'))\n> +\t\treturn -1;\n> +\n> +\tif (pm == 0)\n> +\t\treturn -1;\n> +\n> +\treturn pm;\n> +}\n> +\n> +/* Parse the argument given in the command line of the application */\n> +static int\n> +parse_args(int argc, char **argv)\n> +{\n> +\tint opt;\n> +\tchar **argvopt;\n> +\tint option_index;\n> +\tchar *prgname = argv[0];\n> +\tstatic struct option lgopts[] = {\n> +\t\t{NULL, 0, 0, 0}\n> +\t};\n> +\n> +\targvopt = argv;\n> +\n> +\twhile ((opt = getopt_long(argc, argvopt, \"p:\",\n> +\t\t\tlgopts, &option_index)) != EOF) {\n> +\n> +\t\tswitch (opt) {\n> +\t\t\t/* portmask */\n> +\t\t\tcase 'p':\n> +\t\t\t\tenabled_port_mask =\n> parse_portmask(optarg);\n> +\t\t\t\tif (enabled_port_mask == 0) {\n> +\t\t\t\t\tprintf(\"invalid portmask\\n\");\n> +\t\t\t\t\tprint_usage(prgname);\n> +\t\t\t\t\treturn -1;\n> +\t\t\t\t}\n> +\t\t\t\tbreak;\n> +\n> +\t\t\tdefault:\n> +\t\t\t\tprint_usage(prgname);\n> +\t\t\t\treturn -1;\n> +\t\t}\n> +\t}\n> +\n> +if (optind <= 1) {\n> +\tprint_usage(prgname);\n> +\treturn -1;\n> +}\n> +\n> +\targv[optind-1] = prgname;\n> +\n> +\toptind = 0; /* reset getopt lib */\n> +\treturn 0;\n> +}\n> +\n> +/* Main function, does initialization and calls the per-lcore functions */\n> +int\n> +MAIN(int argc, char *argv[])\n> +{\n> +\tstruct rte_mempool *mbuf_pool;\n> +\tstruct rte_distributor *d;\n> +\tstruct rte_ring *output_ring;\n> +\tunsigned lcore_id, worker_id = 0;\n> +\tunsigned nb_ports;\n> +\tuint8_t portid;\n> +\tuint8_t nb_ports_available;\n> +\n> +\t/* catch ctrl-c so we can print on exit */\n> +\tsignal(SIGINT, int_handler);\n> +\n> +\t/* init EAL */\n> +\tint ret = rte_eal_init(argc, argv);\n> +\tif (ret < 0)\n> +\t\trte_exit(EXIT_FAILURE, \"Error with EAL initialization\\n\");\n> +\targc -= ret;\n> +\targv += ret;\n> +\n> +\t/* parse application arguments (after the EAL ones) */\n> +\tret = parse_args(argc, argv);\n> +\tif (ret < 0)\n> +\t\trte_exit(EXIT_FAILURE, \"Invalid distributor parameters\\n\");\n> +\n> +\tif (rte_lcore_count() < 3)\n> +\t\trte_exit(EXIT_FAILURE, \"Error, This application needs at \"\n> +\t\t\t\t\"least 3 logical cores to run:\\n\"\n> +\t\t\t\t\"1 lcore for packet RX and distribution\\n\"\n> +\t\t\t\t\"1 lcore for packet TX\\n\"\n> +\t\t\t\t\"and at least 1 lcore for worker threads\\n\");\n> +\n> +\tif (rte_eal_pci_probe() != 0)\n> +\t\trte_exit(EXIT_FAILURE, \"Error with PCI probing\\n\");\n> +\n> +\tnb_ports = rte_eth_dev_count();\n> +\tif (nb_ports == 0)\n> +\t\trte_exit(EXIT_FAILURE, \"Error: no ethernet ports\n> detected\\n\");\n> +\tif (nb_ports != 1 && (nb_ports & 1))\n> +\t\trte_exit(EXIT_FAILURE, \"Error: number of ports must be\n> even, except \"\n> +\t\t\t\t\"when using a single port\\n\");\n> +\n> +\tmbuf_pool = rte_mempool_create(\"MBUF_POOL\", NUM_MBUFS *\n> nb_ports,\n> +\t\t\tMBUF_SIZE, MBUF_CACHE_SIZE,\n> +\t\t\tsizeof(struct rte_pktmbuf_pool_private),\n> +\t\t\trte_pktmbuf_pool_init, NULL,\n> +\t\t\trte_pktmbuf_init, NULL,\n> +\t\t\trte_socket_id(), 0);\n> +\tif (mbuf_pool == NULL)\n> +\t\trte_exit(EXIT_FAILURE, \"Cannot create mbuf pool\\n\");\n> +\n> +\tnb_ports_available = nb_ports;\n> +\n> +\t/* initialize all ports */\n> +\tfor (portid = 0; portid < nb_ports; portid++) {\n> +\t\t/* skip ports that are not enabled */\n> +\t\tif ((enabled_port_mask & (1 << portid)) == 0) {\n> +\t\t\tprintf(\"\\nSkipping disabled port %d\\n\", portid);\n> +\t\t\tnb_ports_available--;\n> +\t\t\tcontinue;\n> +\t\t}\n> +\t\t/* init port */\n> +\t\tprintf(\"Initializing port %u... done\\n\", (unsigned) portid);\n> +\n> +\t\tif (port_init(portid, mbuf_pool) != 0)\n> +\t\t\trte_exit(EXIT_FAILURE, \"Cannot initialize port\n> %\"PRIu8\"\\n\",\n> +\t\t\t\t\tportid);\n> +\t}\n> +\n> +\tif (!nb_ports_available) {\n> +\t\trte_exit(EXIT_FAILURE,\n> +\t\t\t\t\"All available ports are disabled. Please set\n> portmask.\\n\");\n> +\t}\n> +\n> +\td = rte_distributor_create(\"PKT_DIST\", rte_socket_id(),\n> +\t\t\trte_lcore_count() - 2);\n> +\tif (d == NULL)\n> +\t\trte_exit(EXIT_FAILURE, \"Cannot create distributor\\n\");\n> +\n> +\t/* scheduler ring is read only by the transmitter core, but written to\n> +\t * by multiple threads\n> +\t */\n> +\toutput_ring = rte_ring_create(\"Output_ring\", RTE_RING_SZ,\n> +\t\t\trte_socket_id(), RING_F_SC_DEQ);\n> +\tif (output_ring == NULL)\n> +\t\trte_exit(EXIT_FAILURE, \"Cannot create output ring\\n\");\n> +\n> +\tRTE_LCORE_FOREACH_SLAVE(lcore_id) {\n> +\t\tif (worker_id == rte_lcore_count() - 2)\n> +\t\t\trte_eal_remote_launch((lcore_function_t\n> *)lcore_tx,\n> +\t\t\t\t\toutput_ring, lcore_id);\n> +\t\telse {\n> +\t\t\tstruct lcore_params *p =\n> +\t\t\t\t\trte_malloc(NULL, sizeof(*p), 0);\n> +\t\t\tif (!p)\n> +\t\t\t\trte_panic(\"malloc failure\\n\");\n> +\t\t\t*p = (struct lcore_params){worker_id, d,\n> output_ring};\n> +\t\t\trte_eal_remote_launch((lcore_function_t\n> *)lcore_worker,\n> +\t\t\t\t\tp, lcore_id);\n> +\t\t}\n> +\t\tworker_id++;\n> +\t}\n> +\t/* call lcore_main on master core only */\n> +\tstruct lcore_params p = { 0, d, output_ring };\n> +\tlcore_rx(&p);\n> +\treturn 0;\n> +}\n> diff --git a/examples/distributor_app/main.h\n> b/examples/distributor_app/main.h\n> new file mode 100644\n> index 0000000..2682d15\n> --- /dev/null\n> +++ b/examples/distributor_app/main.h\n> @@ -0,0 +1,46 @@\n> +/*-\n> + * BSD LICENSE\n> + *\n> + * Copyright(c) 2010-2014 Intel Corporation. All rights reserved.\n> + * All rights reserved.\n> + *\n> + * Redistribution and use in source and binary forms, with or without\n> + * modification, are permitted provided that the following conditions\n> + * are met:\n> + *\n> + * * Redistributions of source code must retain the above copyright\n> + * notice, this list of conditions and the following disclaimer.\n> + * * Redistributions in binary form must reproduce the above copyright\n> + * notice, this list of conditions and the following disclaimer in\n> + * the documentation and/or other materials provided with the\n> + * distribution.\n> + * * Neither the name of Intel Corporation nor the names of its\n> + * contributors may be used to endorse or promote products derived\n> + * from this software without specific prior written permission.\n> + *\n> + * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND\n> CONTRIBUTORS\n> + * \"AS IS\" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT\n> NOT\n> + * LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND\n> FITNESS FOR\n> + * A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE\n> COPYRIGHT\n> + * OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT,\n> INCIDENTAL,\n> + * SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT\n> NOT\n> + * LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS\n> OF USE,\n> + * DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED\n> AND ON ANY\n> + * THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR\n> TORT\n> + * (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF\n> THE USE\n> + * OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH\n> DAMAGE.\n> + */\n> +\n> +#ifndef _MAIN_H_\n> +#define _MAIN_H_\n> +\n> +\n> +#ifdef RTE_EXEC_ENV_BAREMETAL\n> +#define MAIN _main\n> +#else\n> +#define MAIN main\n> +#endif\n> +\n> +int MAIN(int argc, char *argv[]);\n> +\n> +#endif /* ifndef _MAIN_H_ */\n> --\n> 1.9.3", "headers": { "Return-Path": "<dev-bounces@dpdk.org>", "X-Original-To": "patchwork@dpdk.org", "Delivered-To": "patchwork@dpdk.org", "Received": [ "from [92.243.14.124] (localhost [IPv6:::1])\n\tby dpdk.org (Postfix) with ESMTP id 3F8477E43;\n\tFri, 26 Sep 2014 17:08:01 +0200 (CEST)", "from mga02.intel.com (mga02.intel.com [134.134.136.20])\n\tby dpdk.org (Postfix) with ESMTP id B860E7E07\n\tfor <dev@dpdk.org>; Fri, 26 Sep 2014 17:07:58 +0200 (CEST)", "from orsmga002.jf.intel.com ([10.7.209.21])\n\tby orsmga101.jf.intel.com with ESMTP; 26 Sep 2014 08:13:09 -0700", "from irsmsx101.ger.corp.intel.com ([163.33.3.153])\n\tby orsmga002.jf.intel.com with ESMTP; 26 Sep 2014 08:11:59 -0700", "from irsmsx105.ger.corp.intel.com (163.33.3.28) by\n\tIRSMSX101.ger.corp.intel.com (163.33.3.153) with Microsoft SMTP\n\tServer (TLS) id 14.3.195.1; Fri, 26 Sep 2014 16:11:45 +0100", "from irsmsx108.ger.corp.intel.com ([169.254.11.21]) by\n\tIRSMSX105.ger.corp.intel.com ([169.254.7.57]) with mapi id\n\t14.03.0195.001; Fri, 26 Sep 2014 16:11:45 +0100" ], "X-ExtLoop1": "1", "X-IronPort-AV": "E=Sophos;i=\"5.04,604,1406617200\"; d=\"scan'208\";a=\"609145910\"", "From": "\"De Lara Guarch, Pablo\" <pablo.de.lara.guarch@intel.com>", "To": "\"Pattan, Reshma\" <reshma.pattan@intel.com>, \"dev@dpdk.org\" <dev@dpdk.org>", "Thread-Topic": "[dpdk-dev] [PATCH v2] distributor_app: new sample app", "Thread-Index": "AQHP2AJi1zXXgpMcD0+yx1ATRe0wtpwThu2Q", "Date": "Fri, 26 Sep 2014 15:11:44 +0000", "Message-ID": "<E115CCD9D858EF4F90C690B0DCB4D89722626933@IRSMSX108.ger.corp.intel.com>", "References": "<1410869607-16842-1-git-send-email-reshma.pattan@intel.com>\n\t<1411568210-2555-1-git-send-email-reshma.pattan@intel.com>", "In-Reply-To": "<1411568210-2555-1-git-send-email-reshma.pattan@intel.com>", "Accept-Language": "en-US", "Content-Language": "en-US", "X-MS-Has-Attach": "", "X-MS-TNEF-Correlator": "", "x-originating-ip": "[163.33.239.182]", "Content-Type": "text/plain; charset=\"us-ascii\"", "Content-Transfer-Encoding": "quoted-printable", "MIME-Version": "1.0", "Subject": "Re: [dpdk-dev] [PATCH v2] distributor_app: new sample app", "X-BeenThere": "dev@dpdk.org", "X-Mailman-Version": "2.1.15", "Precedence": "list", "List-Id": "patches and discussions about DPDK <dev.dpdk.org>", "List-Unsubscribe": "<http://dpdk.org/ml/options/dev>,\n\t<mailto:dev-request@dpdk.org?subject=unsubscribe>", "List-Archive": "<http://dpdk.org/ml/archives/dev/>", "List-Post": "<mailto:dev@dpdk.org>", "List-Help": "<mailto:dev-request@dpdk.org?subject=help>", "List-Subscribe": "<http://dpdk.org/ml/listinfo/dev>,\n\t<mailto:dev-request@dpdk.org?subject=subscribe>", "Errors-To": "dev-bounces@dpdk.org", "Sender": "\"dev\" <dev-bounces@dpdk.org>" }, "addressed": null }, { "id": 1144, "web_url": "https://patches.dpdk.org/comment/1144/", "msgid": "<2601191342CEEE43887BDE71AB9772582137D8B1@IRSMSX104.ger.corp.intel.com>", "list_archive_url": "https://inbox.dpdk.org/dev/2601191342CEEE43887BDE71AB9772582137D8B1@IRSMSX104.ger.corp.intel.com", "date": "2014-09-26T15:51:38", "subject": "Re: [dpdk-dev] [PATCH v2] distributor_app: new sample app", "submitter": { "id": 33, "url": "https://patches.dpdk.org/api/people/33/?format=api", "name": "Ananyev, Konstantin", "email": "konstantin.ananyev@intel.com" }, "content": "> -----Original Message-----\n> From: dev [mailto:dev-bounces@dpdk.org] On Behalf Of De Lara Guarch, Pablo\n> Sent: Friday, September 26, 2014 4:12 PM\n> To: Pattan, Reshma; dev@dpdk.org\n> Subject: Re: [dpdk-dev] [PATCH v2] distributor_app: new sample app\n> \n> Hi,\n> \n> > -----Original Message-----\n> > From: dev [mailto:dev-bounces@dpdk.org] On Behalf Of reshmapa\n> > Sent: Wednesday, September 24, 2014 3:17 PM\n> > To: dev@dpdk.org\n> > Subject: [dpdk-dev] [PATCH v2] distributor_app: new sample app\n> >\n> > From: Reshma Pattan <reshma.pattan@intel.com>\n> >\n> > A new sample app that shows the usage of the distributor library. This\n> > app works as follows:\n> >\n> > * An RX thread runs which pulls packets from each ethernet port in turn\n> > and passes those packets to worker using a distributor component.\n> > * The workers take the packets in turn, and determine the output port\n> > for those packets using basic l2forwarding doing an xor on the source\n> > port id.\n> > * The RX thread takes the returned packets from the workers and enqueue\n> > those packets into an rte_ring structure.\n> > * A TX thread pulls the packets off the rte_ring structure and then\n> > sends each packet out the output port specified previously by the worker\n> > * Command-line option support provided only for portmask.\n> >\n> > Signed-off-by: Bruce Richardson <bruce.richardson@intel.com>\n> > Signed-off-by: Reshma Pattan <reshma.pattan@intel.com>\n> > ---\n> > examples/Makefile | 1 +\n> > examples/distributor_app/Makefile | 57 ++++\n> > examples/distributor_app/main.c | 585\n> > ++++++++++++++++++++++++++++++++++++++\n> > examples/distributor_app/main.h | 46 +++\n> > 4 files changed, 689 insertions(+)\n> > create mode 100644 examples/distributor_app/Makefile\n> > create mode 100644 examples/distributor_app/main.c\n> > create mode 100644 examples/distributor_app/main.h\n> >\n> > diff --git a/examples/Makefile b/examples/Makefile\n> > index 6245f83..2ba82b0 100644\n> > --- a/examples/Makefile\n> > +++ b/examples/Makefile\n> > @@ -66,5 +66,6 @@ DIRS-y += vhost\n> > DIRS-$(CONFIG_RTE_LIBRTE_XEN_DOM0) += vhost_xen\n> > DIRS-y += vmdq\n> > DIRS-y += vmdq_dcb\n> > +DIRS-$(CONFIG_RTE_LIBRTE_DISTRIBUTOR) += distributor_app\n> >\n> > include $(RTE_SDK)/mk/rte.extsubdir.mk\n> > diff --git a/examples/distributor_app/Makefile\n> > b/examples/distributor_app/Makefile\n> > new file mode 100644\n> > index 0000000..394785d\n> > --- /dev/null\n> > +++ b/examples/distributor_app/Makefile\n> > @@ -0,0 +1,57 @@\n> > +# BSD LICENSE\n> > +#\n> > +# Copyright(c) 2010-2014 Intel Corporation. All rights reserved.\n> > +# All rights reserved.\n> > +#\n> > +# Redistribution and use in source and binary forms, with or without\n> > +# modification, are permitted provided that the following conditions\n> > +# are met:\n> > +#\n> > +# * Redistributions of source code must retain the above copyright\n> > +# notice, this list of conditions and the following disclaimer.\n> > +# * Redistributions in binary form must reproduce the above copyright\n> > +# notice, this list of conditions and the following disclaimer in\n> > +# the documentation and/or other materials provided with the\n> > +# distribution.\n> > +# * Neither the name of Intel Corporation nor the names of its\n> > +# contributors may be used to endorse or promote products derived\n> > +# from this software without specific prior written permission.\n> > +#\n> > +# THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND\n> > CONTRIBUTORS\n> > +# \"AS IS\" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT\n> > NOT\n> > +# LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND\n> > FITNESS FOR\n> > +# A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE\n> > COPYRIGHT\n> > +# OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT,\n> > INCIDENTAL,\n> > +# SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT\n> > NOT\n> > +# LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS\n> > OF USE,\n> > +# DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED\n> > AND ON ANY\n> > +# THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR\n> > TORT\n> > +# (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF\n> > THE USE\n> > +# OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH\n> > DAMAGE.\n> > +\n> > +ifeq ($(RTE_SDK),)\n> > +$(error \"Please define RTE_SDK environment variable\")\n> > +endif\n> > +\n> > +# Default target, can be overriden by command line or environment\n> > +RTE_TARGET ?= x86_64-default-linuxapp-gcc\n> \n> This target is not present anymore. Change it to x86_64-native-linuxapp-gcc.\n> \n> > +\n> > +include $(RTE_SDK)/mk/rte.vars.mk\n> > +\n> > +# binary name\n> > +APP = distributor_app\n> > +\n> > +# all source are stored in SRCS-y\n> > +SRCS-y := main.c\n> > +\n> > +CFLAGS += $(WERROR_FLAGS)\n> > +\n> > +# workaround for a gcc bug with noreturn attribute\n> > +# http://gcc.gnu.org/bugzilla/show_bug.cgi?id=12603\n> > +ifeq ($(CONFIG_RTE_TOOLCHAIN_GCC),y)\n> > +CFLAGS_main.o += -Wno-return-type\n> > +endif\n> > +\n> > +EXTRA_CFLAGS += -O3 -Wfatal-errors\n> > +\n> > +include $(RTE_SDK)/mk/rte.extapp.mk\n> > diff --git a/examples/distributor_app/main.c\n> > b/examples/distributor_app/main.c\n> > new file mode 100644\n> > index 0000000..628810a\n> > --- /dev/null\n> > +++ b/examples/distributor_app/main.c\n> > @@ -0,0 +1,585 @@\n> > +/*-\n> > + * BSD LICENSE\n> > + *\n> > + * Copyright(c) 2010-2014 Intel Corporation. All rights reserved.\n> > + * All rights reserved.\n> > + *\n> > + * Redistribution and use in source and binary forms, with or without\n> > + * modification, are permitted provided that the following conditions\n> > + * are met:\n> > + *\n> > + * * Redistributions of source code must retain the above copyright\n> > + * notice, this list of conditions and the following disclaimer.\n> > + * * Redistributions in binary form must reproduce the above copyright\n> > + * notice, this list of conditions and the following disclaimer in\n> > + * the documentation and/or other materials provided with the\n> > + * distribution.\n> > + * * Neither the name of Intel Corporation nor the names of its\n> > + * contributors may be used to endorse or promote products derived\n> > + * from this software without specific prior written permission.\n> > + *\n> > + * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND\n> > CONTRIBUTORS\n> > + * \"AS IS\" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT\n> > NOT\n> > + * LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND\n> > FITNESS FOR\n> > + * A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE\n> > COPYRIGHT\n> > + * OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT,\n> > INCIDENTAL,\n> > + * SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT\n> > NOT\n> > + * LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS\n> > OF USE,\n> > + * DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED\n> > AND ON ANY\n> > + * THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR\n> > TORT\n> > + * (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF\n> > THE USE\n> > + * OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH\n> > DAMAGE.\n> > + */\n> > +\n> > +#include <stdint.h>\n> > +#include <inttypes.h>\n> > +#include <unistd.h>\n> > +#include <signal.h>\n> > +#include <getopt.h>\n> > +\n> > +#include <rte_eal.h>\n> > +#include <rte_ethdev.h>\n> > +#include <rte_cycles.h>\n> > +#include <rte_malloc.h>\n> > +#include <rte_debug.h>\n> > +#include <rte_distributor.h>\n> > +\n> > +#include \"main.h\"\n> > +\n> > +#define RX_RING_SIZE 256\n> > +#define RX_FREE_THRESH 32\n> > +#define RX_PTHRESH 8\n> > +#define RX_HTHRESH 8\n> > +#define RX_WTHRESH 0\n> > +\n> > +#define TX_RING_SIZE 512\n> > +#define TX_FREE_THRESH 32\n> > +#define TX_PTHRESH 32\n> > +#define TX_HTHRESH 0\n> > +#define TX_WTHRESH 0\n> > +#define TX_RSBIT_THRESH 32\n> > +#define TX_Q_FLAGS (ETH_TXQ_FLAGS_NOMULTSEGS |\n> > ETH_TXQ_FLAGS_NOVLANOFFL |\\\n> > +\tETH_TXQ_FLAGS_NOXSUMSCTP | ETH_TXQ_FLAGS_NOXSUMUDP |\n> > \\\n> > +\tETH_TXQ_FLAGS_NOXSUMTCP)\n> > +\n> > +#define NUM_MBUFS ((64*1024)-1)\n> > +#define MBUF_SIZE (2048 + sizeof(struct rte_mbuf) +\n> > RTE_PKTMBUF_HEADROOM)\n> > +#define MBUF_CACHE_SIZE 250\n> > +#define BURST_SIZE 32\n> > +#define RTE_RING_SZ 1024\n> > +\n> > +/* mask of enabled ports */\n> > +static uint32_t enabled_port_mask = 0;\n> > +\n> > +static volatile struct app_stats {\n> > +\tstruct {\n> > +\t\tuint64_t rx_pkts;\n> > +\t\tuint64_t returned_pkts;\n> > +\t\tuint64_t enqueued_pkts;\n> > +\t} rx __rte_cache_aligned;\n> > +\n> > +\tstruct {\n> > +\t\tuint64_t dequeue_pkts;\n> > +\t\tuint64_t tx_pkts;\n> > +\t} tx __rte_cache_aligned;\n> > +} app_stats;\n> > +\n> > +static const struct rte_eth_conf port_conf_default = {\n> > +\t.rxmode = {\n> > +\t\t.mq_mode = ETH_MQ_RX_RSS,\n> > +\t\t.max_rx_pkt_len = ETHER_MAX_LEN,\n> > +\t\t.split_hdr_size = 0,\n> > +\t\t.header_split = 0, /**< Header Split disabled */\n> > +\t\t.hw_ip_checksum = 0, /**< IP checksum offload enabled */\n> > +\t\t.hw_vlan_filter = 0, /**< VLAN filtering disabled */\n> > +\t\t.jumbo_frame = 0, /**< Jumbo Frame Support disabled */\n> > +\t\t.hw_strip_crc = 0, /**< CRC stripped by hardware */\n> > +\t},\n> > +\t.txmode = {\n> > +\t\t.mq_mode = ETH_MQ_TX_NONE,\n> > +\t},\n> > +\t.lpbk_mode = 0,\n> > +\t.rx_adv_conf = {\n> > +\t\t\t.rss_conf = {\n> > +\t\t\t\t.rss_hf = ETH_RSS_IPV4 | ETH_RSS_IPV6 |\n> > +\t\t\t\t\tETH_RSS_IPV4_TCP |\n> > ETH_RSS_IPV4_UDP |\n> > +\t\t\t\t\tETH_RSS_IPV6_TCP |\n> > ETH_RSS_IPV6_UDP,\n> > +\t\t\t}\n> > +\t},\n> > +};\n> > +\n> > +static const struct rte_eth_rxconf rx_conf_default = {\n> > +\t.rx_thresh = {\n> > +\t\t.pthresh = RX_PTHRESH,\n> > +\t\t.hthresh = RX_HTHRESH,\n> > +\t\t.wthresh = RX_WTHRESH,\n> > +\t},\n> > +\t.rx_free_thresh = RX_FREE_THRESH,\n> > +\t.rx_drop_en = 0,\n> > +};\n> > +\n> > +static const struct rte_eth_txconf tx_conf_default = {\n> > +\t.tx_thresh = {\n> > +\t\t.pthresh = TX_PTHRESH,\n> > +\t\t.hthresh = TX_HTHRESH,\n> > +\t\t.wthresh = TX_WTHRESH,\n> > +\t},\n> > +\t.tx_free_thresh = TX_FREE_THRESH,\n> > +\t.tx_rs_thresh = TX_RSBIT_THRESH,\n> > +\t.txq_flags = TX_Q_FLAGS\n> > +\n> > +};\n> > +\n> > +struct output_buffer {\n> > +\tunsigned count;\n> > +\tstruct rte_mbuf *mbufs[BURST_SIZE];\n> > +};\n> > +\n> > +/*\n> > + * Initialises a given port using global settings and with the rx buffers\n> > + * coming from the mbuf_pool passed as parameter\n> > + */\n> > +static inline int\n> > +port_init(uint8_t port, struct rte_mempool *mbuf_pool)\n> > +{\n> > +\tstruct rte_eth_conf port_conf = port_conf_default;\n> > +\tconst uint16_t rxRings = 1, txRings = rte_lcore_count() - 1;\n> > +\tint retval;\n> > +\tuint16_t q;\n> > +\n> > +\tif (port >= rte_eth_dev_count())\n> > +\t\treturn -1;\n> > +\n> > +\tretval = rte_eth_dev_configure(port, rxRings, txRings, &port_conf);\n> > +\tif (retval != 0)\n> > +\t\treturn retval;\n> > +\n> > +\tfor (q = 0; q < rxRings; q++) {\n> > +\t\tretval = rte_eth_rx_queue_setup(port, q, RX_RING_SIZE,\n> > +\n> > \trte_eth_dev_socket_id(port),\n> > +\t\t\t\t\t\t&rx_conf_default,\n> > mbuf_pool);\n> > +\t\tif (retval < 0)\n> > +\t\t\treturn retval;\n> > +\t}\n> > +\n> > +\tfor (q = 0; q < txRings; q++) {\n> > +\t\tretval = rte_eth_tx_queue_setup(port, q, TX_RING_SIZE,\n> > +\n> > \trte_eth_dev_socket_id(port),\n> > +\t\t\t\t\t\t&tx_conf_default);\n> > +\t\tif (retval < 0)\n> > +\t\t\treturn retval;\n> > +\t}\n> > +\n> > +\tretval = rte_eth_dev_start(port);\n> > +\tif (retval < 0)\n> > +\t\treturn retval;\n> > +\n> > +\tstruct rte_eth_link link;\n> > +\trte_eth_link_get_nowait(port, &link);\n> > +\tif (!link.link_status) {\n> > +\t\tsleep(1);\n> > +\t\trte_eth_link_get_nowait(port, &link);\n> > +\t}\n> > +\n> > +\tif (!link.link_status) {\n> > +\t\tprintf(\"Link down on port %\"PRIu8\"\\n\", port);\n> > +\t\treturn 0;\n> > +\t}\n> > +\n> > +\tstruct ether_addr addr;\n> > +\trte_eth_macaddr_get(port, &addr);\n> > +\tprintf(\"Port %u MAC: %02\"PRIx8\" %02\"PRIx8\" %02\"PRIx8\n> > +\t\t\t\" %02\"PRIx8\" %02\"PRIx8\" %02\"PRIx8\"\\n\",\n> > +\t\t\t(unsigned)port,\n> > +\t\t\taddr.addr_bytes[0], addr.addr_bytes[1],\n> > +\t\t\taddr.addr_bytes[2], addr.addr_bytes[3],\n> > +\t\t\taddr.addr_bytes[4], addr.addr_bytes[5]);\n> > +\n> > +\trte_eth_promiscuous_enable(port);\n> > +\n> > +\treturn 0;\n> > +}\n> > +\n> > +struct lcore_params {\n> > +\tunsigned worker_id;\n> > +\tstruct rte_distributor *d;\n> > +\tstruct rte_ring *r;\n> > +};\n> > +\n> > +static __attribute__((noreturn)) void\n> > +lcore_rx(struct lcore_params *p)\n> > +{\n> > +\tstruct rte_distributor *d = p->d;\n> > +\tstruct rte_ring *r = p->r;\n> > +\tconst uint8_t nb_ports = rte_eth_dev_count();\n> > +\tconst int socket_id = rte_socket_id();\n> > +\tuint8_t port;\n> > +\n> > +\tfor (port = 0; port < nb_ports; port++) {\n> > +\t\t/* skip ports that are not enabled */\n> > +\t\tif ((enabled_port_mask & (1 << port)) == 0)\n> > +\t\t\tcontinue;\n> > +\n> > +\t\tif (rte_eth_dev_socket_id(port) > 0 &&\n> > +\t\t\t\trte_eth_dev_socket_id(port) != socket_id)\n> > +\t\t\tprintf(\"WARNING, port %u is on remote NUMA node\n> > to \"\n> > +\t\t\t\t\t\"RX thread.\\n\\tPerformance will not\n> > \"\n> > +\t\t\t\t\t\"be optimal.\\n\", port);\n> > +\t}\n> > +\n> > +\tprintf(\"\\nCore %u doing packet RX.\\n\", rte_lcore_id());\n> > +\tport = 0;\n> > +\tfor (;;) {\n> > +\t\t/* skip ports that are not enabled */\n> > +\t\tif ((enabled_port_mask & (1 << port)) == 0) {\n> > +\t\t\tif (++port == nb_ports)\n> > +\t\t\t\tport = 0;\n> > +\t\t\tcontinue;\n> > +\t\t}\n> > +\t\tstruct rte_mbuf *bufs[BURST_SIZE*2];\n> > +\t\tconst uint16_t nb_rx = rte_eth_rx_burst(port, 0, bufs,\n> > +\t\t\t\tBURST_SIZE);\n> > +\t\tapp_stats.rx.rx_pkts += nb_rx;\n> > +\n> > +\t\trte_distributor_process(d, bufs, nb_rx);\n> > +\t\tconst uint16_t nb_ret = rte_distributor_returned_pkts(d,\n> > +\t\t\t\tbufs, BURST_SIZE*2);\n> > +\t\tapp_stats.rx.returned_pkts += nb_ret;\n> > +\t\tif (unlikely(nb_ret == 0))\n> > +\t\t\tcontinue;\n> > +\n> > +\t\tuint16_t sent = rte_ring_enqueue_burst(r, (void *)bufs,\n> > nb_ret);\n> > +\t\tapp_stats.rx.enqueued_pkts += sent;\n> > +\t\tif (unlikely(sent < nb_ret)) {\n> > +\t\t\tprintf(\"Packet loss due to full ring\\n\");\n> \n> I would remove this printf. If core is too busy to process packets, this may make performance worse.\n> Statistics should reflect this packet loss.\n\nProbably use RTE_LOG(DEBUG, ...).\nThat's way you can switch it on/off at compile time.\n\n> \n> > +\t\t\twhile (sent < nb_ret)\n> > +\t\t\t\trte_pktmbuf_free(bufs[sent++]);\n> > +\t\t}\n> > +\t\tif (++port == nb_ports)\n> > +\t\t\tport = 0;\n> > +\t}\n> > +}\n> > +\n> > +static inline void\n> > +flush_one_port(struct output_buffer *outbuf, uint8_t outp)\n> > +{\n> > +\tunsigned nb_tx = rte_eth_tx_burst(outp, 0, outbuf->mbufs,\n> > +\t\t\toutbuf->count);\n> > +\tapp_stats.tx.tx_pkts += nb_tx;\n> > +\n> > +\tif (unlikely(nb_tx < outbuf->count)) {\n> > +\t\tprintf(\"Packet loss with tx_burst\\n\");\n> \n> Same here. This is fine for debugging, but statistics are updated as well, so we do not need this printf.\n> \n> > +\t\tdo {\n> > +\t\t\trte_pktmbuf_free(outbuf->mbufs[nb_tx]);\n> > +\t\t} while (++nb_tx < outbuf->count);\n> > +\t}\n> > +\toutbuf->count = 0;\n> > +}\n> > +\n> > +static inline void\n> > +flush_all_ports(struct output_buffer *tx_buffers, uint8_t nb_ports)\n> > +{\n> > +\tuint8_t outp;\n> > +\tfor (outp = 0; outp < nb_ports; outp++) {\n> > +\t\t/* skip ports that are not enabled */\n> > +\t\tif ((enabled_port_mask & (1 << outp)) == 0)\n> > +\t\t\tcontinue;\n> > +\n> > +\t\tif (tx_buffers[outp].count == 0)\n> > +\t\t\tcontinue;\n> > +\n> > +\t\tflush_one_port(&tx_buffers[outp], outp);\n> > +\t}\n> > +}\n> > +\n> > +static __attribute__((noreturn)) void\n> > +lcore_tx(struct rte_ring *in_r)\n> > +{\n> > +\tstatic struct output_buffer tx_buffers[RTE_MAX_ETHPORTS];\n> > +\tconst uint8_t nb_ports = rte_eth_dev_count();\n> > +\tconst int socket_id = rte_socket_id();\n> > +\tuint8_t port;\n> > +\n> > +\tfor (port = 0; port < nb_ports; port++) {\n> > +\t\t/* skip ports that are not enabled */\n> > +\t\tif ((enabled_port_mask & (1 << port)) == 0)\n> > +\t\t\tcontinue;\n> > +\n> > +\t\tif (rte_eth_dev_socket_id(port) > 0 &&\n> > +\t\t\t\trte_eth_dev_socket_id(port) != socket_id)\n> > +\t\t\tprintf(\"WARNING, port %u is on remote NUMA node\n> > to \"\n> > +\t\t\t\t\t\"TX thread.\\n\\tPerformance will not \"\n> > +\t\t\t\t\t\"be optimal.\\n\", port);\n> > +\t}\n> > +\n> > +\tprintf(\"\\nCore %u doing packet TX.\\n\", rte_lcore_id());\n> > +\tfor (;;) {\n> > +\t\tfor (port = 0; port < nb_ports; port++) {\n> > +\t\t\t/* skip ports that are not enabled */\n> > +\t\t\tif ((enabled_port_mask & (1 << port)) == 0)\n> > +\t\t\t\tcontinue;\n> > +\n> > +\t\t\tstruct rte_mbuf *bufs[BURST_SIZE];\n> > +\t\t\tconst uint16_t nb_rx = rte_ring_dequeue_burst(in_r,\n> > +\t\t\t\t\t(void *)bufs, BURST_SIZE);\n> > +\t\t\tapp_stats.tx.dequeue_pkts += nb_rx;\n> > +\n> > +\t\t\t/* if we get no traffic, flush anything we have */\n> > +\t\t\tif (unlikely(nb_rx == 0)) {\n> > +\t\t\t\tflush_all_ports(tx_buffers, nb_ports);\n> > +\t\t\t\tcontinue;\n> > +\t\t\t}\n> > +\n> > +\t\t\t/* for traffic we receive, queue it up for transmit */\n> > +\t\t\tuint16_t i;\n> > +\t\t\t_mm_prefetch(bufs[0], 0);\n> > +\t\t\t_mm_prefetch(bufs[1], 0);\n> > +\t\t\t_mm_prefetch(bufs[2], 0);\n> > +\t\t\tfor (i = 0; i < nb_rx; i++) {\n> > +\t\t\t\tstruct output_buffer *outbuf;\n> > +\t\t\t\tuint8_t outp;\n> > +\t\t\t\t_mm_prefetch(bufs[i + 3], 0);\n> > +\t\t\t\t/* workers should update in_port to hold the\n> > +\t\t\t\t * output port value */\n> > +\t\t\t\toutp = bufs[i]->port;\n> > +\t\t\t\t/* skip ports that are not enabled */\n> > +\t\t\t\tif ((enabled_port_mask & (1 << outp)) == 0)\n> > +\t\t\t\t\tcontinue;\n> > +\n> > +\t\t\t\toutbuf = &tx_buffers[outp];\n> > +\t\t\t\toutbuf->mbufs[outbuf->count++] = bufs[i];\n> > +\t\t\t\tif (outbuf->count == BURST_SIZE)\n> > +\t\t\t\t\tflush_one_port(outbuf, outp);\n> > +\t\t\t}\n> > +\t\t}\n> > +\t}\n> > +}\n> > +\n> > +\n> > +static __attribute__((noreturn)) void\n> > +lcore_worker(struct lcore_params *p)\n> > +{\n> > +\tstruct rte_distributor *d = p->d;\n> > +\tconst unsigned id = p->worker_id;\n> > +\t/* for single port, xor_val will be zero so we won't modify the output\n> > +\t * port, otherwise we send traffic from 0 to 1, 2 to 3, and vice versa\n> > +\t */\n> > +\tconst unsigned xor_val = (rte_eth_dev_count() > 1);\n> > +\tstruct rte_mbuf *buf = NULL;\n> > +\n> > +\tprintf(\"\\nCore %u acting as worker core.\\n\", rte_lcore_id());\n> > +\tfor (;;) {\n> > +\t\tbuf = rte_distributor_get_pkt(d, id, buf);\n> > +\t\tbuf->port ^= xor_val;\n> > +\t}\n> > +}\n> > +\n> > +static void\n> > +int_handler(int sig_num)\n> > +{\n> > +\tstruct rte_eth_stats eth_stats;\n> > +\tunsigned i;\n> > +\n> > +\tprintf(\"Exiting on signal %d\\n\", sig_num);\n> > +\n> > +\tprintf(\"\\nRX thread stats:\\n\");\n> > +\tprintf(\" - Received: %\"PRIu64\"\\n\", app_stats.rx.rx_pkts);\n> > +\tprintf(\" - Processed: %\"PRIu64\"\\n\", app_stats.rx.returned_pkts);\n> > +\tprintf(\" - Enqueued: %\"PRIu64\"\\n\", app_stats.rx.enqueued_pkts);\n> > +\n> > +\tprintf(\"\\nTX thread stats:\\n\");\n> > +\tprintf(\" - Dequeued: %\"PRIu64\"\\n\", app_stats.tx.dequeue_pkts);\n> > +\tprintf(\" - Transmitted: %\"PRIu64\"\\n\", app_stats.tx.tx_pkts);\n> > +\n> > +\tfor (i = 0; i < rte_eth_dev_count(); i++) {\n> > +\t\trte_eth_stats_get(i, &eth_stats);\n> > +\t\tprintf(\"\\nPort %u stats:\\n\", i);\n> > +\t\tprintf(\" - Pkts in: %\"PRIu64\"\\n\", eth_stats.ipackets);\n> > +\t\tprintf(\" - Pkts out: %\"PRIu64\"\\n\", eth_stats.opackets);\n> > +\t\tprintf(\" - In Errs: %\"PRIu64\"\\n\", eth_stats.ierrors);\n> > +\t\tprintf(\" - Out Errs: %\"PRIu64\"\\n\", eth_stats.oerrors);\n> > +\t\tprintf(\" - Mbuf Errs: %\"PRIu64\"\\n\", eth_stats.rx_nombuf);\n> > +\t}\n> > +\texit(0);\n> > +}\n> > +\n> > +/* display usage */\n> > +static void\n> > +print_usage(const char *prgname)\n> > +{\n> > +\tprintf(\"%s [EAL options] -- -p PORTMASK\\n\"\n> > +\t\t\t\" -p PORTMASK: hexadecimal bitmask of ports to\n> > configure\\n\",\n> > +\t\t\tprgname);\n> > +}\n> > +\n> > +static int\n> > +parse_portmask(const char *portmask)\n> > +{\n> > +\tchar *end = NULL;\n> > +\tunsigned long pm;\n> > +\n> > +\t/* parse hexadecimal string */\n> > +\tpm = strtoul(portmask, &end, 16);\n> > +\tif ((portmask[0] == '\\0') || (end == NULL) || (*end != '\\0'))\n> > +\t\treturn -1;\n> > +\n> > +\tif (pm == 0)\n> > +\t\treturn -1;\n> > +\n> > +\treturn pm;\n> > +}\n> > +\n> > +/* Parse the argument given in the command line of the application */\n> > +static int\n> > +parse_args(int argc, char **argv)\n> > +{\n> > +\tint opt;\n> > +\tchar **argvopt;\n> > +\tint option_index;\n> > +\tchar *prgname = argv[0];\n> > +\tstatic struct option lgopts[] = {\n> > +\t\t{NULL, 0, 0, 0}\n> > +\t};\n> > +\n> > +\targvopt = argv;\n> > +\n> > +\twhile ((opt = getopt_long(argc, argvopt, \"p:\",\n> > +\t\t\tlgopts, &option_index)) != EOF) {\n> > +\n> > +\t\tswitch (opt) {\n> > +\t\t\t/* portmask */\n> > +\t\t\tcase 'p':\n> > +\t\t\t\tenabled_port_mask =\n> > parse_portmask(optarg);\n> > +\t\t\t\tif (enabled_port_mask == 0) {\n> > +\t\t\t\t\tprintf(\"invalid portmask\\n\");\n> > +\t\t\t\t\tprint_usage(prgname);\n> > +\t\t\t\t\treturn -1;\n> > +\t\t\t\t}\n> > +\t\t\t\tbreak;\n> > +\n> > +\t\t\tdefault:\n> > +\t\t\t\tprint_usage(prgname);\n> > +\t\t\t\treturn -1;\n> > +\t\t}\n> > +\t}\n> > +\n> > +if (optind <= 1) {\n> > +\tprint_usage(prgname);\n> > +\treturn -1;\n> > +}\n> > +\n> > +\targv[optind-1] = prgname;\n> > +\n> > +\toptind = 0; /* reset getopt lib */\n> > +\treturn 0;\n> > +}\n> > +\n> > +/* Main function, does initialization and calls the per-lcore functions */\n> > +int\n> > +MAIN(int argc, char *argv[])\n> > +{\n> > +\tstruct rte_mempool *mbuf_pool;\n> > +\tstruct rte_distributor *d;\n> > +\tstruct rte_ring *output_ring;\n> > +\tunsigned lcore_id, worker_id = 0;\n> > +\tunsigned nb_ports;\n> > +\tuint8_t portid;\n> > +\tuint8_t nb_ports_available;\n> > +\n> > +\t/* catch ctrl-c so we can print on exit */\n> > +\tsignal(SIGINT, int_handler);\n> > +\n> > +\t/* init EAL */\n> > +\tint ret = rte_eal_init(argc, argv);\n> > +\tif (ret < 0)\n> > +\t\trte_exit(EXIT_FAILURE, \"Error with EAL initialization\\n\");\n> > +\targc -= ret;\n> > +\targv += ret;\n> > +\n> > +\t/* parse application arguments (after the EAL ones) */\n> > +\tret = parse_args(argc, argv);\n> > +\tif (ret < 0)\n> > +\t\trte_exit(EXIT_FAILURE, \"Invalid distributor parameters\\n\");\n> > +\n> > +\tif (rte_lcore_count() < 3)\n> > +\t\trte_exit(EXIT_FAILURE, \"Error, This application needs at \"\n> > +\t\t\t\t\"least 3 logical cores to run:\\n\"\n> > +\t\t\t\t\"1 lcore for packet RX and distribution\\n\"\n> > +\t\t\t\t\"1 lcore for packet TX\\n\"\n> > +\t\t\t\t\"and at least 1 lcore for worker threads\\n\");\n> > +\n> > +\tif (rte_eal_pci_probe() != 0)\n> > +\t\trte_exit(EXIT_FAILURE, \"Error with PCI probing\\n\");\n> > +\n> > +\tnb_ports = rte_eth_dev_count();\n> > +\tif (nb_ports == 0)\n> > +\t\trte_exit(EXIT_FAILURE, \"Error: no ethernet ports\n> > detected\\n\");\n> > +\tif (nb_ports != 1 && (nb_ports & 1))\n> > +\t\trte_exit(EXIT_FAILURE, \"Error: number of ports must be\n> > even, except \"\n> > +\t\t\t\t\"when using a single port\\n\");\n> > +\n> > +\tmbuf_pool = rte_mempool_create(\"MBUF_POOL\", NUM_MBUFS *\n> > nb_ports,\n> > +\t\t\tMBUF_SIZE, MBUF_CACHE_SIZE,\n> > +\t\t\tsizeof(struct rte_pktmbuf_pool_private),\n> > +\t\t\trte_pktmbuf_pool_init, NULL,\n> > +\t\t\trte_pktmbuf_init, NULL,\n> > +\t\t\trte_socket_id(), 0);\n> > +\tif (mbuf_pool == NULL)\n> > +\t\trte_exit(EXIT_FAILURE, \"Cannot create mbuf pool\\n\");\n> > +\n> > +\tnb_ports_available = nb_ports;\n> > +\n> > +\t/* initialize all ports */\n> > +\tfor (portid = 0; portid < nb_ports; portid++) {\n> > +\t\t/* skip ports that are not enabled */\n> > +\t\tif ((enabled_port_mask & (1 << portid)) == 0) {\n> > +\t\t\tprintf(\"\\nSkipping disabled port %d\\n\", portid);\n> > +\t\t\tnb_ports_available--;\n> > +\t\t\tcontinue;\n> > +\t\t}\n> > +\t\t/* init port */\n> > +\t\tprintf(\"Initializing port %u... done\\n\", (unsigned) portid);\n> > +\n> > +\t\tif (port_init(portid, mbuf_pool) != 0)\n> > +\t\t\trte_exit(EXIT_FAILURE, \"Cannot initialize port\n> > %\"PRIu8\"\\n\",\n> > +\t\t\t\t\tportid);\n> > +\t}\n> > +\n> > +\tif (!nb_ports_available) {\n> > +\t\trte_exit(EXIT_FAILURE,\n> > +\t\t\t\t\"All available ports are disabled. Please set\n> > portmask.\\n\");\n> > +\t}\n> > +\n> > +\td = rte_distributor_create(\"PKT_DIST\", rte_socket_id(),\n> > +\t\t\trte_lcore_count() - 2);\n> > +\tif (d == NULL)\n> > +\t\trte_exit(EXIT_FAILURE, \"Cannot create distributor\\n\");\n> > +\n> > +\t/* scheduler ring is read only by the transmitter core, but written to\n> > +\t * by multiple threads\n> > +\t */\n> > +\toutput_ring = rte_ring_create(\"Output_ring\", RTE_RING_SZ,\n> > +\t\t\trte_socket_id(), RING_F_SC_DEQ);\n> > +\tif (output_ring == NULL)\n> > +\t\trte_exit(EXIT_FAILURE, \"Cannot create output ring\\n\");\n> > +\n> > +\tRTE_LCORE_FOREACH_SLAVE(lcore_id) {\n> > +\t\tif (worker_id == rte_lcore_count() - 2)\n> > +\t\t\trte_eal_remote_launch((lcore_function_t\n> > *)lcore_tx,\n> > +\t\t\t\t\toutput_ring, lcore_id);\n> > +\t\telse {\n> > +\t\t\tstruct lcore_params *p =\n> > +\t\t\t\t\trte_malloc(NULL, sizeof(*p), 0);\n> > +\t\t\tif (!p)\n> > +\t\t\t\trte_panic(\"malloc failure\\n\");\n> > +\t\t\t*p = (struct lcore_params){worker_id, d,\n> > output_ring};\n> > +\t\t\trte_eal_remote_launch((lcore_function_t\n> > *)lcore_worker,\n> > +\t\t\t\t\tp, lcore_id);\n> > +\t\t}\n> > +\t\tworker_id++;\n> > +\t}\n> > +\t/* call lcore_main on master core only */\n> > +\tstruct lcore_params p = { 0, d, output_ring };\n> > +\tlcore_rx(&p);\n> > +\treturn 0;\n> > +}\n> > diff --git a/examples/distributor_app/main.h\n> > b/examples/distributor_app/main.h\n> > new file mode 100644\n> > index 0000000..2682d15\n> > --- /dev/null\n> > +++ b/examples/distributor_app/main.h\n> > @@ -0,0 +1,46 @@\n> > +/*-\n> > + * BSD LICENSE\n> > + *\n> > + * Copyright(c) 2010-2014 Intel Corporation. All rights reserved.\n> > + * All rights reserved.\n> > + *\n> > + * Redistribution and use in source and binary forms, with or without\n> > + * modification, are permitted provided that the following conditions\n> > + * are met:\n> > + *\n> > + * * Redistributions of source code must retain the above copyright\n> > + * notice, this list of conditions and the following disclaimer.\n> > + * * Redistributions in binary form must reproduce the above copyright\n> > + * notice, this list of conditions and the following disclaimer in\n> > + * the documentation and/or other materials provided with the\n> > + * distribution.\n> > + * * Neither the name of Intel Corporation nor the names of its\n> > + * contributors may be used to endorse or promote products derived\n> > + * from this software without specific prior written permission.\n> > + *\n> > + * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND\n> > CONTRIBUTORS\n> > + * \"AS IS\" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT\n> > NOT\n> > + * LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND\n> > FITNESS FOR\n> > + * A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE\n> > COPYRIGHT\n> > + * OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT,\n> > INCIDENTAL,\n> > + * SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT\n> > NOT\n> > + * LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS\n> > OF USE,\n> > + * DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED\n> > AND ON ANY\n> > + * THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR\n> > TORT\n> > + * (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF\n> > THE USE\n> > + * OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH\n> > DAMAGE.\n> > + */\n> > +\n> > +#ifndef _MAIN_H_\n> > +#define _MAIN_H_\n> > +\n> > +\n> > +#ifdef RTE_EXEC_ENV_BAREMETAL\n> > +#define MAIN _main\n> > +#else\n> > +#define MAIN main\n> > +#endif\n> > +\n> > +int MAIN(int argc, char *argv[]);\n> > +\n> > +#endif /* ifndef _MAIN_H_ */\n> > --\n> > 1.9.3", "headers": { "Return-Path": "<dev-bounces@dpdk.org>", "X-Original-To": "patchwork@dpdk.org", "Delivered-To": "patchwork@dpdk.org", "Received": [ "from [92.243.14.124] (localhost [IPv6:::1])\n\tby dpdk.org (Postfix) with ESMTP id 541667E24;\n\tFri, 26 Sep 2014 17:45:35 +0200 (CEST)", "from mga14.intel.com (mga14.intel.com [192.55.52.115])\n\tby dpdk.org (Postfix) with ESMTP id 501877E17\n\tfor <dev@dpdk.org>; Fri, 26 Sep 2014 17:45:22 +0200 (CEST)", "from fmsmga003.fm.intel.com ([10.253.24.29])\n\tby fmsmga103.fm.intel.com with ESMTP; 26 Sep 2014 08:42:27 -0700", "from irsmsx102.ger.corp.intel.com ([163.33.3.155])\n\tby FMSMGA003.fm.intel.com with ESMTP; 26 Sep 2014 08:45:36 -0700", "from irsmsx154.ger.corp.intel.com (163.33.192.96) by\n\tIRSMSX102.ger.corp.intel.com (163.33.3.155) with Microsoft SMTP\n\tServer (TLS) id 14.3.195.1; Fri, 26 Sep 2014 16:51:38 +0100", "from irsmsx104.ger.corp.intel.com ([169.254.5.248]) by\n\tIRSMSX154.ger.corp.intel.com ([169.254.12.124]) with mapi id\n\t14.03.0195.001; Fri, 26 Sep 2014 16:51:38 +0100" ], "X-ExtLoop1": "1", "X-IronPort-AV": "E=Sophos;i=\"4.97,862,1389772800\"; d=\"scan'208\";a=\"392071977\"", "From": "\"Ananyev, Konstantin\" <konstantin.ananyev@intel.com>", "To": "\"De Lara Guarch, Pablo\" <pablo.de.lara.guarch@intel.com>, \"Pattan,\n\tReshma\" <reshma.pattan@intel.com>, \"dev@dpdk.org\" <dev@dpdk.org>", "Thread-Topic": "[dpdk-dev] [PATCH v2] distributor_app: new sample app", "Thread-Index": "AQHP2AJemxe7phgTsUqwEI/Y1qWz+ZwTd5UAgAAbAeA=", "Date": "Fri, 26 Sep 2014 15:51:38 +0000", "Message-ID": "<2601191342CEEE43887BDE71AB9772582137D8B1@IRSMSX104.ger.corp.intel.com>", "References": "<1410869607-16842-1-git-send-email-reshma.pattan@intel.com>\n\t<1411568210-2555-1-git-send-email-reshma.pattan@intel.com>\n\t<E115CCD9D858EF4F90C690B0DCB4D89722626933@IRSMSX108.ger.corp.intel.com>", "In-Reply-To": "<E115CCD9D858EF4F90C690B0DCB4D89722626933@IRSMSX108.ger.corp.intel.com>", "Accept-Language": "en-IE, en-US", "Content-Language": "en-US", "X-MS-Has-Attach": "", "X-MS-TNEF-Correlator": "", "x-originating-ip": "[163.33.239.180]", "Content-Type": "text/plain; charset=\"us-ascii\"", "Content-Transfer-Encoding": "quoted-printable", "MIME-Version": "1.0", "Subject": "Re: [dpdk-dev] [PATCH v2] distributor_app: new sample app", "X-BeenThere": "dev@dpdk.org", "X-Mailman-Version": "2.1.15", "Precedence": "list", "List-Id": "patches and discussions about DPDK <dev.dpdk.org>", "List-Unsubscribe": "<http://dpdk.org/ml/options/dev>,\n\t<mailto:dev-request@dpdk.org?subject=unsubscribe>", "List-Archive": "<http://dpdk.org/ml/archives/dev/>", "List-Post": "<mailto:dev@dpdk.org>", "List-Help": "<mailto:dev-request@dpdk.org?subject=help>", "List-Subscribe": "<http://dpdk.org/ml/listinfo/dev>,\n\t<mailto:dev-request@dpdk.org?subject=subscribe>", "Errors-To": "dev-bounces@dpdk.org", "Sender": "\"dev\" <dev-bounces@dpdk.org>" }, "addressed": null }, { "id": 1226, "web_url": "https://patches.dpdk.org/comment/1226/", "msgid": "<3AEA2BF9852C6F48A459DA490692831FE20598@IRSMSX109.ger.corp.intel.com>", "list_archive_url": "https://inbox.dpdk.org/dev/3AEA2BF9852C6F48A459DA490692831FE20598@IRSMSX109.ger.corp.intel.com", "date": "2014-09-29T12:39:33", "subject": "Re: [dpdk-dev] [PATCH v2] distributor_app: new sample app", "submitter": { "id": 70, "url": "https://patches.dpdk.org/api/people/70/?format=api", "name": "Pattan, Reshma", "email": "reshma.pattan@intel.com" }, "content": "-----Original Message-----\nFrom: Ananyev, Konstantin \nSent: Friday, September 26, 2014 4:52 PM\nTo: De Lara Guarch, Pablo; Pattan, Reshma; dev@dpdk.org\nSubject: RE: [dpdk-dev] [PATCH v2] distributor_app: new sample app\n\n\n\n> -----Original Message-----\n> From: dev [mailto:dev-bounces@dpdk.org] On Behalf Of De Lara Guarch, \n> Pablo\n> Sent: Friday, September 26, 2014 4:12 PM\n> To: Pattan, Reshma; dev@dpdk.org\n> Subject: Re: [dpdk-dev] [PATCH v2] distributor_app: new sample app\n> \n> Hi,\n> \n> > -----Original Message-----\n> > From: dev [mailto:dev-bounces@dpdk.org] On Behalf Of reshmapa\n> > Sent: Wednesday, September 24, 2014 3:17 PM\n> > To: dev@dpdk.org\n> > Subject: [dpdk-dev] [PATCH v2] distributor_app: new sample app\n> >\n> > From: Reshma Pattan <reshma.pattan@intel.com>\n> >\n> > A new sample app that shows the usage of the distributor library. \n> > This app works as follows:\n> >\n> > * An RX thread runs which pulls packets from each ethernet port in turn\n> > and passes those packets to worker using a distributor component.\n> > * The workers take the packets in turn, and determine the output port\n> > for those packets using basic l2forwarding doing an xor on the source\n> > port id.\n> > * The RX thread takes the returned packets from the workers and enqueue\n> > those packets into an rte_ring structure.\n> > * A TX thread pulls the packets off the rte_ring structure and then\n> > sends each packet out the output port specified previously by the \n> > worker\n> > * Command-line option support provided only for portmask.\n> >\n> > Signed-off-by: Bruce Richardson <bruce.richardson@intel.com>\n> > Signed-off-by: Reshma Pattan <reshma.pattan@intel.com>\n> > ---\n> > examples/Makefile | 1 +\n> > examples/distributor_app/Makefile | 57 ++++\n> > examples/distributor_app/main.c | 585\n> > ++++++++++++++++++++++++++++++++++++++\n> > examples/distributor_app/main.h | 46 +++\n> > 4 files changed, 689 insertions(+)\n> > create mode 100644 examples/distributor_app/Makefile create mode \n> > 100644 examples/distributor_app/main.c create mode 100644 \n> > examples/distributor_app/main.h\n> >\n> > diff --git a/examples/Makefile b/examples/Makefile index \n> > 6245f83..2ba82b0 100644\n> > --- a/examples/Makefile\n> > +++ b/examples/Makefile\n> > @@ -66,5 +66,6 @@ DIRS-y += vhost\n> > DIRS-$(CONFIG_RTE_LIBRTE_XEN_DOM0) += vhost_xen DIRS-y += vmdq \n> > DIRS-y += vmdq_dcb\n> > +DIRS-$(CONFIG_RTE_LIBRTE_DISTRIBUTOR) += distributor_app\n> >\n> > include $(RTE_SDK)/mk/rte.extsubdir.mk diff --git \n> > a/examples/distributor_app/Makefile\n> > b/examples/distributor_app/Makefile\n> > new file mode 100644\n> > index 0000000..394785d\n> > --- /dev/null\n> > +++ b/examples/distributor_app/Makefile\n> > @@ -0,0 +1,57 @@\n> > +# BSD LICENSE\n> > +#\n> > +# Copyright(c) 2010-2014 Intel Corporation. All rights reserved.\n> > +# All rights reserved.\n> > +#\n> > +# Redistribution and use in source and binary forms, with or without\n> > +# modification, are permitted provided that the following conditions\n> > +# are met:\n> > +#\n> > +# * Redistributions of source code must retain the above copyright\n> > +# notice, this list of conditions and the following disclaimer.\n> > +# * Redistributions in binary form must reproduce the above copyright\n> > +# notice, this list of conditions and the following disclaimer in\n> > +# the documentation and/or other materials provided with the\n> > +# distribution.\n> > +# * Neither the name of Intel Corporation nor the names of its\n> > +# contributors may be used to endorse or promote products derived\n> > +# from this software without specific prior written permission.\n> > +#\n> > +# THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND\n> > CONTRIBUTORS\n> > +# \"AS IS\" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT\n> > NOT\n> > +# LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND\n> > FITNESS FOR\n> > +# A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE\n> > COPYRIGHT\n> > +# OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT,\n> > INCIDENTAL,\n> > +# SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT\n> > NOT\n> > +# LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS\n> > OF USE,\n> > +# DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED\n> > AND ON ANY\n> > +# THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR\n> > TORT\n> > +# (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF\n> > THE USE\n> > +# OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH\n> > DAMAGE.\n> > +\n> > +ifeq ($(RTE_SDK),)\n> > +$(error \"Please define RTE_SDK environment variable\") endif\n> > +\n> > +# Default target, can be overriden by command line or environment \n> > +RTE_TARGET ?= x86_64-default-linuxapp-gcc\n> \n> This target is not present anymore. Change it to x86_64-native-linuxapp-gcc.\n> \n> > +\n> > +include $(RTE_SDK)/mk/rte.vars.mk\n> > +\n> > +# binary name\n> > +APP = distributor_app\n> > +\n> > +# all source are stored in SRCS-y\n> > +SRCS-y := main.c\n> > +\n> > +CFLAGS += $(WERROR_FLAGS)\n> > +\n> > +# workaround for a gcc bug with noreturn attribute # \n> > +http://gcc.gnu.org/bugzilla/show_bug.cgi?id=12603\n> > +ifeq ($(CONFIG_RTE_TOOLCHAIN_GCC),y) CFLAGS_main.o += \n> > +-Wno-return-type endif\n> > +\n> > +EXTRA_CFLAGS += -O3 -Wfatal-errors\n> > +\n> > +include $(RTE_SDK)/mk/rte.extapp.mk\n> > diff --git a/examples/distributor_app/main.c \n> > b/examples/distributor_app/main.c new file mode 100644 index \n> > 0000000..628810a\n> > --- /dev/null\n> > +++ b/examples/distributor_app/main.c\n> > @@ -0,0 +1,585 @@\n> > +/*-\n> > + * BSD LICENSE\n> > + *\n> > + * Copyright(c) 2010-2014 Intel Corporation. All rights reserved.\n> > + * All rights reserved.\n> > + *\n> > + * Redistribution and use in source and binary forms, with or without\n> > + * modification, are permitted provided that the following conditions\n> > + * are met:\n> > + *\n> > + * * Redistributions of source code must retain the above copyright\n> > + * notice, this list of conditions and the following disclaimer.\n> > + * * Redistributions in binary form must reproduce the above copyright\n> > + * notice, this list of conditions and the following disclaimer in\n> > + * the documentation and/or other materials provided with the\n> > + * distribution.\n> > + * * Neither the name of Intel Corporation nor the names of its\n> > + * contributors may be used to endorse or promote products derived\n> > + * from this software without specific prior written permission.\n> > + *\n> > + * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND\n> > CONTRIBUTORS\n> > + * \"AS IS\" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT\n> > NOT\n> > + * LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND\n> > FITNESS FOR\n> > + * A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE\n> > COPYRIGHT\n> > + * OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT,\n> > INCIDENTAL,\n> > + * SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT\n> > NOT\n> > + * LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS\n> > OF USE,\n> > + * DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED\n> > AND ON ANY\n> > + * THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR\n> > TORT\n> > + * (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF\n> > THE USE\n> > + * OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH\n> > DAMAGE.\n> > + */\n> > +\n> > +#include <stdint.h>\n> > +#include <inttypes.h>\n> > +#include <unistd.h>\n> > +#include <signal.h>\n> > +#include <getopt.h>\n> > +\n> > +#include <rte_eal.h>\n> > +#include <rte_ethdev.h>\n> > +#include <rte_cycles.h>\n> > +#include <rte_malloc.h>\n> > +#include <rte_debug.h>\n> > +#include <rte_distributor.h>\n> > +\n> > +#include \"main.h\"\n> > +\n> > +#define RX_RING_SIZE 256\n> > +#define RX_FREE_THRESH 32\n> > +#define RX_PTHRESH 8\n> > +#define RX_HTHRESH 8\n> > +#define RX_WTHRESH 0\n> > +\n> > +#define TX_RING_SIZE 512\n> > +#define TX_FREE_THRESH 32\n> > +#define TX_PTHRESH 32\n> > +#define TX_HTHRESH 0\n> > +#define TX_WTHRESH 0\n> > +#define TX_RSBIT_THRESH 32\n> > +#define TX_Q_FLAGS (ETH_TXQ_FLAGS_NOMULTSEGS |\n> > ETH_TXQ_FLAGS_NOVLANOFFL |\\\n> > +\tETH_TXQ_FLAGS_NOXSUMSCTP | ETH_TXQ_FLAGS_NOXSUMUDP |\n> > \\\n> > +\tETH_TXQ_FLAGS_NOXSUMTCP)\n> > +\n> > +#define NUM_MBUFS ((64*1024)-1)\n> > +#define MBUF_SIZE (2048 + sizeof(struct rte_mbuf) +\n> > RTE_PKTMBUF_HEADROOM)\n> > +#define MBUF_CACHE_SIZE 250\n> > +#define BURST_SIZE 32\n> > +#define RTE_RING_SZ 1024\n> > +\n> > +/* mask of enabled ports */\n> > +static uint32_t enabled_port_mask = 0;\n> > +\n> > +static volatile struct app_stats {\n> > +\tstruct {\n> > +\t\tuint64_t rx_pkts;\n> > +\t\tuint64_t returned_pkts;\n> > +\t\tuint64_t enqueued_pkts;\n> > +\t} rx __rte_cache_aligned;\n> > +\n> > +\tstruct {\n> > +\t\tuint64_t dequeue_pkts;\n> > +\t\tuint64_t tx_pkts;\n> > +\t} tx __rte_cache_aligned;\n> > +} app_stats;\n> > +\n> > +static const struct rte_eth_conf port_conf_default = {\n> > +\t.rxmode = {\n> > +\t\t.mq_mode = ETH_MQ_RX_RSS,\n> > +\t\t.max_rx_pkt_len = ETHER_MAX_LEN,\n> > +\t\t.split_hdr_size = 0,\n> > +\t\t.header_split = 0, /**< Header Split disabled */\n> > +\t\t.hw_ip_checksum = 0, /**< IP checksum offload enabled */\n> > +\t\t.hw_vlan_filter = 0, /**< VLAN filtering disabled */\n> > +\t\t.jumbo_frame = 0, /**< Jumbo Frame Support disabled */\n> > +\t\t.hw_strip_crc = 0, /**< CRC stripped by hardware */\n> > +\t},\n> > +\t.txmode = {\n> > +\t\t.mq_mode = ETH_MQ_TX_NONE,\n> > +\t},\n> > +\t.lpbk_mode = 0,\n> > +\t.rx_adv_conf = {\n> > +\t\t\t.rss_conf = {\n> > +\t\t\t\t.rss_hf = ETH_RSS_IPV4 | ETH_RSS_IPV6 |\n> > +\t\t\t\t\tETH_RSS_IPV4_TCP |\n> > ETH_RSS_IPV4_UDP |\n> > +\t\t\t\t\tETH_RSS_IPV6_TCP |\n> > ETH_RSS_IPV6_UDP,\n> > +\t\t\t}\n> > +\t},\n> > +};\n> > +\n> > +static const struct rte_eth_rxconf rx_conf_default = {\n> > +\t.rx_thresh = {\n> > +\t\t.pthresh = RX_PTHRESH,\n> > +\t\t.hthresh = RX_HTHRESH,\n> > +\t\t.wthresh = RX_WTHRESH,\n> > +\t},\n> > +\t.rx_free_thresh = RX_FREE_THRESH,\n> > +\t.rx_drop_en = 0,\n> > +};\n> > +\n> > +static const struct rte_eth_txconf tx_conf_default = {\n> > +\t.tx_thresh = {\n> > +\t\t.pthresh = TX_PTHRESH,\n> > +\t\t.hthresh = TX_HTHRESH,\n> > +\t\t.wthresh = TX_WTHRESH,\n> > +\t},\n> > +\t.tx_free_thresh = TX_FREE_THRESH,\n> > +\t.tx_rs_thresh = TX_RSBIT_THRESH,\n> > +\t.txq_flags = TX_Q_FLAGS\n> > +\n> > +};\n> > +\n> > +struct output_buffer {\n> > +\tunsigned count;\n> > +\tstruct rte_mbuf *mbufs[BURST_SIZE]; };\n> > +\n> > +/*\n> > + * Initialises a given port using global settings and with the rx \n> > +buffers\n> > + * coming from the mbuf_pool passed as parameter */ static inline \n> > +int port_init(uint8_t port, struct rte_mempool *mbuf_pool) {\n> > +\tstruct rte_eth_conf port_conf = port_conf_default;\n> > +\tconst uint16_t rxRings = 1, txRings = rte_lcore_count() - 1;\n> > +\tint retval;\n> > +\tuint16_t q;\n> > +\n> > +\tif (port >= rte_eth_dev_count())\n> > +\t\treturn -1;\n> > +\n> > +\tretval = rte_eth_dev_configure(port, rxRings, txRings, &port_conf);\n> > +\tif (retval != 0)\n> > +\t\treturn retval;\n> > +\n> > +\tfor (q = 0; q < rxRings; q++) {\n> > +\t\tretval = rte_eth_rx_queue_setup(port, q, RX_RING_SIZE,\n> > +\n> > \trte_eth_dev_socket_id(port),\n> > +\t\t\t\t\t\t&rx_conf_default,\n> > mbuf_pool);\n> > +\t\tif (retval < 0)\n> > +\t\t\treturn retval;\n> > +\t}\n> > +\n> > +\tfor (q = 0; q < txRings; q++) {\n> > +\t\tretval = rte_eth_tx_queue_setup(port, q, TX_RING_SIZE,\n> > +\n> > \trte_eth_dev_socket_id(port),\n> > +\t\t\t\t\t\t&tx_conf_default);\n> > +\t\tif (retval < 0)\n> > +\t\t\treturn retval;\n> > +\t}\n> > +\n> > +\tretval = rte_eth_dev_start(port);\n> > +\tif (retval < 0)\n> > +\t\treturn retval;\n> > +\n> > +\tstruct rte_eth_link link;\n> > +\trte_eth_link_get_nowait(port, &link);\n> > +\tif (!link.link_status) {\n> > +\t\tsleep(1);\n> > +\t\trte_eth_link_get_nowait(port, &link);\n> > +\t}\n> > +\n> > +\tif (!link.link_status) {\n> > +\t\tprintf(\"Link down on port %\"PRIu8\"\\n\", port);\n> > +\t\treturn 0;\n> > +\t}\n> > +\n> > +\tstruct ether_addr addr;\n> > +\trte_eth_macaddr_get(port, &addr);\n> > +\tprintf(\"Port %u MAC: %02\"PRIx8\" %02\"PRIx8\" %02\"PRIx8\n> > +\t\t\t\" %02\"PRIx8\" %02\"PRIx8\" %02\"PRIx8\"\\n\",\n> > +\t\t\t(unsigned)port,\n> > +\t\t\taddr.addr_bytes[0], addr.addr_bytes[1],\n> > +\t\t\taddr.addr_bytes[2], addr.addr_bytes[3],\n> > +\t\t\taddr.addr_bytes[4], addr.addr_bytes[5]);\n> > +\n> > +\trte_eth_promiscuous_enable(port);\n> > +\n> > +\treturn 0;\n> > +}\n> > +\n> > +struct lcore_params {\n> > +\tunsigned worker_id;\n> > +\tstruct rte_distributor *d;\n> > +\tstruct rte_ring *r;\n> > +};\n> > +\n> > +static __attribute__((noreturn)) void lcore_rx(struct lcore_params \n> > +*p) {\n> > +\tstruct rte_distributor *d = p->d;\n> > +\tstruct rte_ring *r = p->r;\n> > +\tconst uint8_t nb_ports = rte_eth_dev_count();\n> > +\tconst int socket_id = rte_socket_id();\n> > +\tuint8_t port;\n> > +\n> > +\tfor (port = 0; port < nb_ports; port++) {\n> > +\t\t/* skip ports that are not enabled */\n> > +\t\tif ((enabled_port_mask & (1 << port)) == 0)\n> > +\t\t\tcontinue;\n> > +\n> > +\t\tif (rte_eth_dev_socket_id(port) > 0 &&\n> > +\t\t\t\trte_eth_dev_socket_id(port) != socket_id)\n> > +\t\t\tprintf(\"WARNING, port %u is on remote NUMA node\n> > to \"\n> > +\t\t\t\t\t\"RX thread.\\n\\tPerformance will not\n> > \"\n> > +\t\t\t\t\t\"be optimal.\\n\", port);\n> > +\t}\n> > +\n> > +\tprintf(\"\\nCore %u doing packet RX.\\n\", rte_lcore_id());\n> > +\tport = 0;\n> > +\tfor (;;) {\n> > +\t\t/* skip ports that are not enabled */\n> > +\t\tif ((enabled_port_mask & (1 << port)) == 0) {\n> > +\t\t\tif (++port == nb_ports)\n> > +\t\t\t\tport = 0;\n> > +\t\t\tcontinue;\n> > +\t\t}\n> > +\t\tstruct rte_mbuf *bufs[BURST_SIZE*2];\n> > +\t\tconst uint16_t nb_rx = rte_eth_rx_burst(port, 0, bufs,\n> > +\t\t\t\tBURST_SIZE);\n> > +\t\tapp_stats.rx.rx_pkts += nb_rx;\n> > +\n> > +\t\trte_distributor_process(d, bufs, nb_rx);\n> > +\t\tconst uint16_t nb_ret = rte_distributor_returned_pkts(d,\n> > +\t\t\t\tbufs, BURST_SIZE*2);\n> > +\t\tapp_stats.rx.returned_pkts += nb_ret;\n> > +\t\tif (unlikely(nb_ret == 0))\n> > +\t\t\tcontinue;\n> > +\n> > +\t\tuint16_t sent = rte_ring_enqueue_burst(r, (void *)bufs,\n> > nb_ret);\n> > +\t\tapp_stats.rx.enqueued_pkts += sent;\n> > +\t\tif (unlikely(sent < nb_ret)) {\n> > +\t\t\tprintf(\"Packet loss due to full ring\\n\");\n> \n> I would remove this printf. If core is too busy to process packets, this may make performance worse.\n> Statistics should reflect this packet loss.\n\n>Probably use RTE_LOG(DEBUG, ...).\n>That's way you can switch it on/off at compile time.\n\nBy default LOG_LEVEL is 8 in config/common_linuxapp, hence I see RTE_LOG(DEBUG,...) always works. \nSo until unless change the LOG-LEVEL to lesser value, RTE_LOG(DEBUG,..) will be logged ..\nPlease let me know if I can go ahead and remove the printf as of now until the LOG_LEVEL is fixed. (or) I have to add some logic to log RTE_LOG(DEBUG,..) only once.\n\n> \n> > +\t\t\twhile (sent < nb_ret)\n> > +\t\t\t\trte_pktmbuf_free(bufs[sent++]);\n> > +\t\t}\n> > +\t\tif (++port == nb_ports)\n> > +\t\t\tport = 0;\n> > +\t}\n> > +}\n> > +\n> > +static inline void\n> > +flush_one_port(struct output_buffer *outbuf, uint8_t outp) {\n> > +\tunsigned nb_tx = rte_eth_tx_burst(outp, 0, outbuf->mbufs,\n> > +\t\t\toutbuf->count);\n> > +\tapp_stats.tx.tx_pkts += nb_tx;\n> > +\n> > +\tif (unlikely(nb_tx < outbuf->count)) {\n> > +\t\tprintf(\"Packet loss with tx_burst\\n\");\n> \n> Same here. This is fine for debugging, but statistics are updated as well, so we do not need this printf.\n> \n> > +\t\tdo {\n> > +\t\t\trte_pktmbuf_free(outbuf->mbufs[nb_tx]);\n> > +\t\t} while (++nb_tx < outbuf->count);\n> > +\t}\n> > +\toutbuf->count = 0;\n> > +}\n> > +\n> > +static inline void\n> > +flush_all_ports(struct output_buffer *tx_buffers, uint8_t nb_ports) \n> > +{\n> > +\tuint8_t outp;\n> > +\tfor (outp = 0; outp < nb_ports; outp++) {\n> > +\t\t/* skip ports that are not enabled */\n> > +\t\tif ((enabled_port_mask & (1 << outp)) == 0)\n> > +\t\t\tcontinue;\n> > +\n> > +\t\tif (tx_buffers[outp].count == 0)\n> > +\t\t\tcontinue;\n> > +\n> > +\t\tflush_one_port(&tx_buffers[outp], outp);\n> > +\t}\n> > +}\n> > +\n> > +static __attribute__((noreturn)) void lcore_tx(struct rte_ring \n> > +*in_r) {\n> > +\tstatic struct output_buffer tx_buffers[RTE_MAX_ETHPORTS];\n> > +\tconst uint8_t nb_ports = rte_eth_dev_count();\n> > +\tconst int socket_id = rte_socket_id();\n> > +\tuint8_t port;\n> > +\n> > +\tfor (port = 0; port < nb_ports; port++) {\n> > +\t\t/* skip ports that are not enabled */\n> > +\t\tif ((enabled_port_mask & (1 << port)) == 0)\n> > +\t\t\tcontinue;\n> > +\n> > +\t\tif (rte_eth_dev_socket_id(port) > 0 &&\n> > +\t\t\t\trte_eth_dev_socket_id(port) != socket_id)\n> > +\t\t\tprintf(\"WARNING, port %u is on remote NUMA node\n> > to \"\n> > +\t\t\t\t\t\"TX thread.\\n\\tPerformance will not \"\n> > +\t\t\t\t\t\"be optimal.\\n\", port);\n> > +\t}\n> > +\n> > +\tprintf(\"\\nCore %u doing packet TX.\\n\", rte_lcore_id());\n> > +\tfor (;;) {\n> > +\t\tfor (port = 0; port < nb_ports; port++) {\n> > +\t\t\t/* skip ports that are not enabled */\n> > +\t\t\tif ((enabled_port_mask & (1 << port)) == 0)\n> > +\t\t\t\tcontinue;\n> > +\n> > +\t\t\tstruct rte_mbuf *bufs[BURST_SIZE];\n> > +\t\t\tconst uint16_t nb_rx = rte_ring_dequeue_burst(in_r,\n> > +\t\t\t\t\t(void *)bufs, BURST_SIZE);\n> > +\t\t\tapp_stats.tx.dequeue_pkts += nb_rx;\n> > +\n> > +\t\t\t/* if we get no traffic, flush anything we have */\n> > +\t\t\tif (unlikely(nb_rx == 0)) {\n> > +\t\t\t\tflush_all_ports(tx_buffers, nb_ports);\n> > +\t\t\t\tcontinue;\n> > +\t\t\t}\n> > +\n> > +\t\t\t/* for traffic we receive, queue it up for transmit */\n> > +\t\t\tuint16_t i;\n> > +\t\t\t_mm_prefetch(bufs[0], 0);\n> > +\t\t\t_mm_prefetch(bufs[1], 0);\n> > +\t\t\t_mm_prefetch(bufs[2], 0);\n> > +\t\t\tfor (i = 0; i < nb_rx; i++) {\n> > +\t\t\t\tstruct output_buffer *outbuf;\n> > +\t\t\t\tuint8_t outp;\n> > +\t\t\t\t_mm_prefetch(bufs[i + 3], 0);\n> > +\t\t\t\t/* workers should update in_port to hold the\n> > +\t\t\t\t * output port value */\n> > +\t\t\t\toutp = bufs[i]->port;\n> > +\t\t\t\t/* skip ports that are not enabled */\n> > +\t\t\t\tif ((enabled_port_mask & (1 << outp)) == 0)\n> > +\t\t\t\t\tcontinue;\n> > +\n> > +\t\t\t\toutbuf = &tx_buffers[outp];\n> > +\t\t\t\toutbuf->mbufs[outbuf->count++] = bufs[i];\n> > +\t\t\t\tif (outbuf->count == BURST_SIZE)\n> > +\t\t\t\t\tflush_one_port(outbuf, outp);\n> > +\t\t\t}\n> > +\t\t}\n> > +\t}\n> > +}\n> > +\n> > +\n> > +static __attribute__((noreturn)) void lcore_worker(struct \n> > +lcore_params *p) {\n> > +\tstruct rte_distributor *d = p->d;\n> > +\tconst unsigned id = p->worker_id;\n> > +\t/* for single port, xor_val will be zero so we won't modify the output\n> > +\t * port, otherwise we send traffic from 0 to 1, 2 to 3, and vice versa\n> > +\t */\n> > +\tconst unsigned xor_val = (rte_eth_dev_count() > 1);\n> > +\tstruct rte_mbuf *buf = NULL;\n> > +\n> > +\tprintf(\"\\nCore %u acting as worker core.\\n\", rte_lcore_id());\n> > +\tfor (;;) {\n> > +\t\tbuf = rte_distributor_get_pkt(d, id, buf);\n> > +\t\tbuf->port ^= xor_val;\n> > +\t}\n> > +}\n> > +\n> > +static void\n> > +int_handler(int sig_num)\n> > +{\n> > +\tstruct rte_eth_stats eth_stats;\n> > +\tunsigned i;\n> > +\n> > +\tprintf(\"Exiting on signal %d\\n\", sig_num);\n> > +\n> > +\tprintf(\"\\nRX thread stats:\\n\");\n> > +\tprintf(\" - Received: %\"PRIu64\"\\n\", app_stats.rx.rx_pkts);\n> > +\tprintf(\" - Processed: %\"PRIu64\"\\n\", app_stats.rx.returned_pkts);\n> > +\tprintf(\" - Enqueued: %\"PRIu64\"\\n\", app_stats.rx.enqueued_pkts);\n> > +\n> > +\tprintf(\"\\nTX thread stats:\\n\");\n> > +\tprintf(\" - Dequeued: %\"PRIu64\"\\n\", app_stats.tx.dequeue_pkts);\n> > +\tprintf(\" - Transmitted: %\"PRIu64\"\\n\", app_stats.tx.tx_pkts);\n> > +\n> > +\tfor (i = 0; i < rte_eth_dev_count(); i++) {\n> > +\t\trte_eth_stats_get(i, &eth_stats);\n> > +\t\tprintf(\"\\nPort %u stats:\\n\", i);\n> > +\t\tprintf(\" - Pkts in: %\"PRIu64\"\\n\", eth_stats.ipackets);\n> > +\t\tprintf(\" - Pkts out: %\"PRIu64\"\\n\", eth_stats.opackets);\n> > +\t\tprintf(\" - In Errs: %\"PRIu64\"\\n\", eth_stats.ierrors);\n> > +\t\tprintf(\" - Out Errs: %\"PRIu64\"\\n\", eth_stats.oerrors);\n> > +\t\tprintf(\" - Mbuf Errs: %\"PRIu64\"\\n\", eth_stats.rx_nombuf);\n> > +\t}\n> > +\texit(0);\n> > +}\n> > +\n> > +/* display usage */\n> > +static void\n> > +print_usage(const char *prgname)\n> > +{\n> > +\tprintf(\"%s [EAL options] -- -p PORTMASK\\n\"\n> > +\t\t\t\" -p PORTMASK: hexadecimal bitmask of ports to\n> > configure\\n\",\n> > +\t\t\tprgname);\n> > +}\n> > +\n> > +static int\n> > +parse_portmask(const char *portmask) {\n> > +\tchar *end = NULL;\n> > +\tunsigned long pm;\n> > +\n> > +\t/* parse hexadecimal string */\n> > +\tpm = strtoul(portmask, &end, 16);\n> > +\tif ((portmask[0] == '\\0') || (end == NULL) || (*end != '\\0'))\n> > +\t\treturn -1;\n> > +\n> > +\tif (pm == 0)\n> > +\t\treturn -1;\n> > +\n> > +\treturn pm;\n> > +}\n> > +\n> > +/* Parse the argument given in the command line of the application \n> > +*/ static int parse_args(int argc, char **argv) {\n> > +\tint opt;\n> > +\tchar **argvopt;\n> > +\tint option_index;\n> > +\tchar *prgname = argv[0];\n> > +\tstatic struct option lgopts[] = {\n> > +\t\t{NULL, 0, 0, 0}\n> > +\t};\n> > +\n> > +\targvopt = argv;\n> > +\n> > +\twhile ((opt = getopt_long(argc, argvopt, \"p:\",\n> > +\t\t\tlgopts, &option_index)) != EOF) {\n> > +\n> > +\t\tswitch (opt) {\n> > +\t\t\t/* portmask */\n> > +\t\t\tcase 'p':\n> > +\t\t\t\tenabled_port_mask =\n> > parse_portmask(optarg);\n> > +\t\t\t\tif (enabled_port_mask == 0) {\n> > +\t\t\t\t\tprintf(\"invalid portmask\\n\");\n> > +\t\t\t\t\tprint_usage(prgname);\n> > +\t\t\t\t\treturn -1;\n> > +\t\t\t\t}\n> > +\t\t\t\tbreak;\n> > +\n> > +\t\t\tdefault:\n> > +\t\t\t\tprint_usage(prgname);\n> > +\t\t\t\treturn -1;\n> > +\t\t}\n> > +\t}\n> > +\n> > +if (optind <= 1) {\n> > +\tprint_usage(prgname);\n> > +\treturn -1;\n> > +}\n> > +\n> > +\targv[optind-1] = prgname;\n> > +\n> > +\toptind = 0; /* reset getopt lib */\n> > +\treturn 0;\n> > +}\n> > +\n> > +/* Main function, does initialization and calls the per-lcore \n> > +functions */ int MAIN(int argc, char *argv[]) {\n> > +\tstruct rte_mempool *mbuf_pool;\n> > +\tstruct rte_distributor *d;\n> > +\tstruct rte_ring *output_ring;\n> > +\tunsigned lcore_id, worker_id = 0;\n> > +\tunsigned nb_ports;\n> > +\tuint8_t portid;\n> > +\tuint8_t nb_ports_available;\n> > +\n> > +\t/* catch ctrl-c so we can print on exit */\n> > +\tsignal(SIGINT, int_handler);\n> > +\n> > +\t/* init EAL */\n> > +\tint ret = rte_eal_init(argc, argv);\n> > +\tif (ret < 0)\n> > +\t\trte_exit(EXIT_FAILURE, \"Error with EAL initialization\\n\");\n> > +\targc -= ret;\n> > +\targv += ret;\n> > +\n> > +\t/* parse application arguments (after the EAL ones) */\n> > +\tret = parse_args(argc, argv);\n> > +\tif (ret < 0)\n> > +\t\trte_exit(EXIT_FAILURE, \"Invalid distributor parameters\\n\");\n> > +\n> > +\tif (rte_lcore_count() < 3)\n> > +\t\trte_exit(EXIT_FAILURE, \"Error, This application needs at \"\n> > +\t\t\t\t\"least 3 logical cores to run:\\n\"\n> > +\t\t\t\t\"1 lcore for packet RX and distribution\\n\"\n> > +\t\t\t\t\"1 lcore for packet TX\\n\"\n> > +\t\t\t\t\"and at least 1 lcore for worker threads\\n\");\n> > +\n> > +\tif (rte_eal_pci_probe() != 0)\n> > +\t\trte_exit(EXIT_FAILURE, \"Error with PCI probing\\n\");\n> > +\n> > +\tnb_ports = rte_eth_dev_count();\n> > +\tif (nb_ports == 0)\n> > +\t\trte_exit(EXIT_FAILURE, \"Error: no ethernet ports\n> > detected\\n\");\n> > +\tif (nb_ports != 1 && (nb_ports & 1))\n> > +\t\trte_exit(EXIT_FAILURE, \"Error: number of ports must be\n> > even, except \"\n> > +\t\t\t\t\"when using a single port\\n\");\n> > +\n> > +\tmbuf_pool = rte_mempool_create(\"MBUF_POOL\", NUM_MBUFS *\n> > nb_ports,\n> > +\t\t\tMBUF_SIZE, MBUF_CACHE_SIZE,\n> > +\t\t\tsizeof(struct rte_pktmbuf_pool_private),\n> > +\t\t\trte_pktmbuf_pool_init, NULL,\n> > +\t\t\trte_pktmbuf_init, NULL,\n> > +\t\t\trte_socket_id(), 0);\n> > +\tif (mbuf_pool == NULL)\n> > +\t\trte_exit(EXIT_FAILURE, \"Cannot create mbuf pool\\n\");\n> > +\n> > +\tnb_ports_available = nb_ports;\n> > +\n> > +\t/* initialize all ports */\n> > +\tfor (portid = 0; portid < nb_ports; portid++) {\n> > +\t\t/* skip ports that are not enabled */\n> > +\t\tif ((enabled_port_mask & (1 << portid)) == 0) {\n> > +\t\t\tprintf(\"\\nSkipping disabled port %d\\n\", portid);\n> > +\t\t\tnb_ports_available--;\n> > +\t\t\tcontinue;\n> > +\t\t}\n> > +\t\t/* init port */\n> > +\t\tprintf(\"Initializing port %u... done\\n\", (unsigned) portid);\n> > +\n> > +\t\tif (port_init(portid, mbuf_pool) != 0)\n> > +\t\t\trte_exit(EXIT_FAILURE, \"Cannot initialize port\n> > %\"PRIu8\"\\n\",\n> > +\t\t\t\t\tportid);\n> > +\t}\n> > +\n> > +\tif (!nb_ports_available) {\n> > +\t\trte_exit(EXIT_FAILURE,\n> > +\t\t\t\t\"All available ports are disabled. Please set\n> > portmask.\\n\");\n> > +\t}\n> > +\n> > +\td = rte_distributor_create(\"PKT_DIST\", rte_socket_id(),\n> > +\t\t\trte_lcore_count() - 2);\n> > +\tif (d == NULL)\n> > +\t\trte_exit(EXIT_FAILURE, \"Cannot create distributor\\n\");\n> > +\n> > +\t/* scheduler ring is read only by the transmitter core, but written to\n> > +\t * by multiple threads\n> > +\t */\n> > +\toutput_ring = rte_ring_create(\"Output_ring\", RTE_RING_SZ,\n> > +\t\t\trte_socket_id(), RING_F_SC_DEQ);\n> > +\tif (output_ring == NULL)\n> > +\t\trte_exit(EXIT_FAILURE, \"Cannot create output ring\\n\");\n> > +\n> > +\tRTE_LCORE_FOREACH_SLAVE(lcore_id) {\n> > +\t\tif (worker_id == rte_lcore_count() - 2)\n> > +\t\t\trte_eal_remote_launch((lcore_function_t\n> > *)lcore_tx,\n> > +\t\t\t\t\toutput_ring, lcore_id);\n> > +\t\telse {\n> > +\t\t\tstruct lcore_params *p =\n> > +\t\t\t\t\trte_malloc(NULL, sizeof(*p), 0);\n> > +\t\t\tif (!p)\n> > +\t\t\t\trte_panic(\"malloc failure\\n\");\n> > +\t\t\t*p = (struct lcore_params){worker_id, d,\n> > output_ring};\n> > +\t\t\trte_eal_remote_launch((lcore_function_t\n> > *)lcore_worker,\n> > +\t\t\t\t\tp, lcore_id);\n> > +\t\t}\n> > +\t\tworker_id++;\n> > +\t}\n> > +\t/* call lcore_main on master core only */\n> > +\tstruct lcore_params p = { 0, d, output_ring };\n> > +\tlcore_rx(&p);\n> > +\treturn 0;\n> > +}\n> > diff --git a/examples/distributor_app/main.h \n> > b/examples/distributor_app/main.h new file mode 100644 index \n> > 0000000..2682d15\n> > --- /dev/null\n> > +++ b/examples/distributor_app/main.h\n> > @@ -0,0 +1,46 @@\n> > +/*-\n> > + * BSD LICENSE\n> > + *\n> > + * Copyright(c) 2010-2014 Intel Corporation. All rights reserved.\n> > + * All rights reserved.\n> > + *\n> > + * Redistribution and use in source and binary forms, with or without\n> > + * modification, are permitted provided that the following conditions\n> > + * are met:\n> > + *\n> > + * * Redistributions of source code must retain the above copyright\n> > + * notice, this list of conditions and the following disclaimer.\n> > + * * Redistributions in binary form must reproduce the above copyright\n> > + * notice, this list of conditions and the following disclaimer in\n> > + * the documentation and/or other materials provided with the\n> > + * distribution.\n> > + * * Neither the name of Intel Corporation nor the names of its\n> > + * contributors may be used to endorse or promote products derived\n> > + * from this software without specific prior written permission.\n> > + *\n> > + * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND\n> > CONTRIBUTORS\n> > + * \"AS IS\" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT\n> > NOT\n> > + * LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND\n> > FITNESS FOR\n> > + * A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE\n> > COPYRIGHT\n> > + * OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT,\n> > INCIDENTAL,\n> > + * SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT\n> > NOT\n> > + * LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS\n> > OF USE,\n> > + * DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED\n> > AND ON ANY\n> > + * THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR\n> > TORT\n> > + * (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF\n> > THE USE\n> > + * OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH\n> > DAMAGE.\n> > + */\n> > +\n> > +#ifndef _MAIN_H_\n> > +#define _MAIN_H_\n> > +\n> > +\n> > +#ifdef RTE_EXEC_ENV_BAREMETAL\n> > +#define MAIN _main\n> > +#else\n> > +#define MAIN main\n> > +#endif\n> > +\n> > +int MAIN(int argc, char *argv[]);\n> > +\n> > +#endif /* ifndef _MAIN_H_ */\n> > --\n> > 1.9.3\n\n--------------------------------------------------------------\nIntel Shannon Limited\nRegistered in Ireland\nRegistered Office: Collinstown Industrial Park, Leixlip, County Kildare\nRegistered Number: 308263\nBusiness address: Dromore House, East Park, Shannon, Co. Clare\n\nThis e-mail and any attachments may contain confidential material for the sole use of the intended recipient(s). Any review or distribution by others is strictly prohibited. If you are not the intended recipient, please contact the sender and delete all copies.", "headers": { "Return-Path": "<dev-bounces@dpdk.org>", "X-Original-To": "patchwork@dpdk.org", "Delivered-To": "patchwork@dpdk.org", "Received": [ "from [92.243.14.124] (localhost [IPv6:::1])\n\tby dpdk.org (Postfix) with ESMTP id 4650F6AA9;\n\tMon, 29 Sep 2014 14:33:21 +0200 (CEST)", "from mga01.intel.com (mga01.intel.com [192.55.52.88])\n\tby dpdk.org (Postfix) with ESMTP id C9DD66A87\n\tfor <dev@dpdk.org>; Mon, 29 Sep 2014 14:33:17 +0200 (CEST)", "from fmsmga002.fm.intel.com ([10.253.24.26])\n\tby fmsmga101.fm.intel.com with ESMTP; 29 Sep 2014 05:39:52 -0700", "from irsmsx103.ger.corp.intel.com ([163.33.3.157])\n\tby fmsmga002.fm.intel.com with ESMTP; 29 Sep 2014 05:39:50 -0700", "from irsmsx152.ger.corp.intel.com (163.33.192.66) by\n\tIRSMSX103.ger.corp.intel.com (163.33.3.157) with Microsoft SMTP\n\tServer (TLS) id 14.3.195.1; Mon, 29 Sep 2014 13:39:34 +0100", "from irsmsx109.ger.corp.intel.com ([169.254.13.253]) by\n\tIRSMSX152.ger.corp.intel.com ([169.254.6.118]) with mapi id\n\t14.03.0195.001; Mon, 29 Sep 2014 13:39:34 +0100" ], "X-ExtLoop1": "1", "X-IronPort-AV": "E=Sophos;i=\"5.04,620,1406617200\"; d=\"scan'208\";a=\"606961858\"", "From": "\"Pattan, Reshma\" <reshma.pattan@intel.com>", "To": "\"Ananyev, Konstantin\" <konstantin.ananyev@intel.com>, \"De Lara Guarch,\n\tPablo\" <pablo.de.lara.guarch@intel.com>, \"dev@dpdk.org\" <dev@dpdk.org>", "Thread-Topic": "[dpdk-dev] [PATCH v2] distributor_app: new sample app", "Thread-Index": "AQHP2AJLRlkqtOmcpECqEROZe22ZQpwTd5UAgAALJgCABI2YkA==", "Date": "Mon, 29 Sep 2014 12:39:33 +0000", "Message-ID": "<3AEA2BF9852C6F48A459DA490692831FE20598@IRSMSX109.ger.corp.intel.com>", "References": "<1410869607-16842-1-git-send-email-reshma.pattan@intel.com>\n\t<1411568210-2555-1-git-send-email-reshma.pattan@intel.com>\n\t<E115CCD9D858EF4F90C690B0DCB4D89722626933@IRSMSX108.ger.corp.intel.com>\n\t<2601191342CEEE43887BDE71AB9772582137D8B1@IRSMSX104.ger.corp.intel.com>", "In-Reply-To": "<2601191342CEEE43887BDE71AB9772582137D8B1@IRSMSX104.ger.corp.intel.com>", "Accept-Language": "en-US", "Content-Language": "en-US", "X-MS-Has-Attach": "", "X-MS-TNEF-Correlator": "", "x-originating-ip": "[163.33.239.181]", "Content-Type": "text/plain; charset=\"us-ascii\"", "MIME-Version": "1.0", "Content-Transfer-Encoding": "quoted-printable", "Subject": "Re: [dpdk-dev] [PATCH v2] distributor_app: new sample app", "X-BeenThere": "dev@dpdk.org", "X-Mailman-Version": "2.1.15", "Precedence": "list", "List-Id": "patches and discussions about DPDK <dev.dpdk.org>", "List-Unsubscribe": "<http://dpdk.org/ml/options/dev>,\n\t<mailto:dev-request@dpdk.org?subject=unsubscribe>", "List-Archive": "<http://dpdk.org/ml/archives/dev/>", "List-Post": "<mailto:dev@dpdk.org>", "List-Help": "<mailto:dev-request@dpdk.org?subject=help>", "List-Subscribe": "<http://dpdk.org/ml/listinfo/dev>,\n\t<mailto:dev-request@dpdk.org?subject=subscribe>", "Errors-To": "dev-bounces@dpdk.org", "Sender": "\"dev\" <dev-bounces@dpdk.org>" }, "addressed": null }, { "id": 1228, "web_url": "https://patches.dpdk.org/comment/1228/", "msgid": "<2601191342CEEE43887BDE71AB97725821387654@IRSMSX104.ger.corp.intel.com>", "list_archive_url": "https://inbox.dpdk.org/dev/2601191342CEEE43887BDE71AB97725821387654@IRSMSX104.ger.corp.intel.com", "date": "2014-09-29T13:06:57", "subject": "Re: [dpdk-dev] [PATCH v2] distributor_app: new sample app", "submitter": { "id": 33, "url": "https://patches.dpdk.org/api/people/33/?format=api", "name": "Ananyev, Konstantin", "email": "konstantin.ananyev@intel.com" }, "content": "> -----Original Message-----\n> From: Pattan, Reshma\n> Sent: Monday, September 29, 2014 1:40 PM\n> To: Ananyev, Konstantin; De Lara Guarch, Pablo; dev@dpdk.org\n> Subject: RE: [dpdk-dev] [PATCH v2] distributor_app: new sample app\n> \n> \n> \n> -----Original Message-----\n> From: Ananyev, Konstantin\n> Sent: Friday, September 26, 2014 4:52 PM\n> To: De Lara Guarch, Pablo; Pattan, Reshma; dev@dpdk.org\n> Subject: RE: [dpdk-dev] [PATCH v2] distributor_app: new sample app\n> \n> \n> \n> > -----Original Message-----\n> > From: dev [mailto:dev-bounces@dpdk.org] On Behalf Of De Lara Guarch,\n> > Pablo\n> > Sent: Friday, September 26, 2014 4:12 PM\n> > To: Pattan, Reshma; dev@dpdk.org\n> > Subject: Re: [dpdk-dev] [PATCH v2] distributor_app: new sample app\n> >\n> > Hi,\n> >\n> > > -----Original Message-----\n> > > From: dev [mailto:dev-bounces@dpdk.org] On Behalf Of reshmapa\n> > > Sent: Wednesday, September 24, 2014 3:17 PM\n> > > To: dev@dpdk.org\n> > > Subject: [dpdk-dev] [PATCH v2] distributor_app: new sample app\n> > >\n> > > From: Reshma Pattan <reshma.pattan@intel.com>\n> > >\n> > > A new sample app that shows the usage of the distributor library.\n> > > This app works as follows:\n> > >\n> > > * An RX thread runs which pulls packets from each ethernet port in turn\n> > > and passes those packets to worker using a distributor component.\n> > > * The workers take the packets in turn, and determine the output port\n> > > for those packets using basic l2forwarding doing an xor on the source\n> > > port id.\n> > > * The RX thread takes the returned packets from the workers and enqueue\n> > > those packets into an rte_ring structure.\n> > > * A TX thread pulls the packets off the rte_ring structure and then\n> > > sends each packet out the output port specified previously by the\n> > > worker\n> > > * Command-line option support provided only for portmask.\n> > >\n> > > Signed-off-by: Bruce Richardson <bruce.richardson@intel.com>\n> > > Signed-off-by: Reshma Pattan <reshma.pattan@intel.com>\n> > > ---\n> > > examples/Makefile | 1 +\n> > > examples/distributor_app/Makefile | 57 ++++\n> > > examples/distributor_app/main.c | 585\n> > > ++++++++++++++++++++++++++++++++++++++\n> > > examples/distributor_app/main.h | 46 +++\n> > > 4 files changed, 689 insertions(+)\n> > > create mode 100644 examples/distributor_app/Makefile create mode\n> > > 100644 examples/distributor_app/main.c create mode 100644\n> > > examples/distributor_app/main.h\n> > >\n> > > diff --git a/examples/Makefile b/examples/Makefile index\n> > > 6245f83..2ba82b0 100644\n> > > --- a/examples/Makefile\n> > > +++ b/examples/Makefile\n> > > @@ -66,5 +66,6 @@ DIRS-y += vhost\n> > > DIRS-$(CONFIG_RTE_LIBRTE_XEN_DOM0) += vhost_xen DIRS-y += vmdq\n> > > DIRS-y += vmdq_dcb\n> > > +DIRS-$(CONFIG_RTE_LIBRTE_DISTRIBUTOR) += distributor_app\n> > >\n> > > include $(RTE_SDK)/mk/rte.extsubdir.mk diff --git\n> > > a/examples/distributor_app/Makefile\n> > > b/examples/distributor_app/Makefile\n> > > new file mode 100644\n> > > index 0000000..394785d\n> > > --- /dev/null\n> > > +++ b/examples/distributor_app/Makefile\n> > > @@ -0,0 +1,57 @@\n> > > +# BSD LICENSE\n> > > +#\n> > > +# Copyright(c) 2010-2014 Intel Corporation. All rights reserved.\n> > > +# All rights reserved.\n> > > +#\n> > > +# Redistribution and use in source and binary forms, with or without\n> > > +# modification, are permitted provided that the following conditions\n> > > +# are met:\n> > > +#\n> > > +# * Redistributions of source code must retain the above copyright\n> > > +# notice, this list of conditions and the following disclaimer.\n> > > +# * Redistributions in binary form must reproduce the above copyright\n> > > +# notice, this list of conditions and the following disclaimer in\n> > > +# the documentation and/or other materials provided with the\n> > > +# distribution.\n> > > +# * Neither the name of Intel Corporation nor the names of its\n> > > +# contributors may be used to endorse or promote products derived\n> > > +# from this software without specific prior written permission.\n> > > +#\n> > > +# THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND\n> > > CONTRIBUTORS\n> > > +# \"AS IS\" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT\n> > > NOT\n> > > +# LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND\n> > > FITNESS FOR\n> > > +# A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE\n> > > COPYRIGHT\n> > > +# OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT,\n> > > INCIDENTAL,\n> > > +# SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT\n> > > NOT\n> > > +# LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS\n> > > OF USE,\n> > > +# DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED\n> > > AND ON ANY\n> > > +# THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR\n> > > TORT\n> > > +# (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF\n> > > THE USE\n> > > +# OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH\n> > > DAMAGE.\n> > > +\n> > > +ifeq ($(RTE_SDK),)\n> > > +$(error \"Please define RTE_SDK environment variable\") endif\n> > > +\n> > > +# Default target, can be overriden by command line or environment\n> > > +RTE_TARGET ?= x86_64-default-linuxapp-gcc\n> >\n> > This target is not present anymore. Change it to x86_64-native-linuxapp-gcc.\n> >\n> > > +\n> > > +include $(RTE_SDK)/mk/rte.vars.mk\n> > > +\n> > > +# binary name\n> > > +APP = distributor_app\n> > > +\n> > > +# all source are stored in SRCS-y\n> > > +SRCS-y := main.c\n> > > +\n> > > +CFLAGS += $(WERROR_FLAGS)\n> > > +\n> > > +# workaround for a gcc bug with noreturn attribute #\n> > > +http://gcc.gnu.org/bugzilla/show_bug.cgi?id=12603\n> > > +ifeq ($(CONFIG_RTE_TOOLCHAIN_GCC),y) CFLAGS_main.o +=\n> > > +-Wno-return-type endif\n> > > +\n> > > +EXTRA_CFLAGS += -O3 -Wfatal-errors\n> > > +\n> > > +include $(RTE_SDK)/mk/rte.extapp.mk\n> > > diff --git a/examples/distributor_app/main.c\n> > > b/examples/distributor_app/main.c new file mode 100644 index\n> > > 0000000..628810a\n> > > --- /dev/null\n> > > +++ b/examples/distributor_app/main.c\n> > > @@ -0,0 +1,585 @@\n> > > +/*-\n> > > + * BSD LICENSE\n> > > + *\n> > > + * Copyright(c) 2010-2014 Intel Corporation. All rights reserved.\n> > > + * All rights reserved.\n> > > + *\n> > > + * Redistribution and use in source and binary forms, with or without\n> > > + * modification, are permitted provided that the following conditions\n> > > + * are met:\n> > > + *\n> > > + * * Redistributions of source code must retain the above copyright\n> > > + * notice, this list of conditions and the following disclaimer.\n> > > + * * Redistributions in binary form must reproduce the above copyright\n> > > + * notice, this list of conditions and the following disclaimer in\n> > > + * the documentation and/or other materials provided with the\n> > > + * distribution.\n> > > + * * Neither the name of Intel Corporation nor the names of its\n> > > + * contributors may be used to endorse or promote products derived\n> > > + * from this software without specific prior written permission.\n> > > + *\n> > > + * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND\n> > > CONTRIBUTORS\n> > > + * \"AS IS\" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT\n> > > NOT\n> > > + * LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND\n> > > FITNESS FOR\n> > > + * A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE\n> > > COPYRIGHT\n> > > + * OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT,\n> > > INCIDENTAL,\n> > > + * SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT\n> > > NOT\n> > > + * LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS\n> > > OF USE,\n> > > + * DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED\n> > > AND ON ANY\n> > > + * THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR\n> > > TORT\n> > > + * (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF\n> > > THE USE\n> > > + * OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH\n> > > DAMAGE.\n> > > + */\n> > > +\n> > > +#include <stdint.h>\n> > > +#include <inttypes.h>\n> > > +#include <unistd.h>\n> > > +#include <signal.h>\n> > > +#include <getopt.h>\n> > > +\n> > > +#include <rte_eal.h>\n> > > +#include <rte_ethdev.h>\n> > > +#include <rte_cycles.h>\n> > > +#include <rte_malloc.h>\n> > > +#include <rte_debug.h>\n> > > +#include <rte_distributor.h>\n> > > +\n> > > +#include \"main.h\"\n> > > +\n> > > +#define RX_RING_SIZE 256\n> > > +#define RX_FREE_THRESH 32\n> > > +#define RX_PTHRESH 8\n> > > +#define RX_HTHRESH 8\n> > > +#define RX_WTHRESH 0\n> > > +\n> > > +#define TX_RING_SIZE 512\n> > > +#define TX_FREE_THRESH 32\n> > > +#define TX_PTHRESH 32\n> > > +#define TX_HTHRESH 0\n> > > +#define TX_WTHRESH 0\n> > > +#define TX_RSBIT_THRESH 32\n> > > +#define TX_Q_FLAGS (ETH_TXQ_FLAGS_NOMULTSEGS |\n> > > ETH_TXQ_FLAGS_NOVLANOFFL |\\\n> > > +\tETH_TXQ_FLAGS_NOXSUMSCTP | ETH_TXQ_FLAGS_NOXSUMUDP |\n> > > \\\n> > > +\tETH_TXQ_FLAGS_NOXSUMTCP)\n> > > +\n> > > +#define NUM_MBUFS ((64*1024)-1)\n> > > +#define MBUF_SIZE (2048 + sizeof(struct rte_mbuf) +\n> > > RTE_PKTMBUF_HEADROOM)\n> > > +#define MBUF_CACHE_SIZE 250\n> > > +#define BURST_SIZE 32\n> > > +#define RTE_RING_SZ 1024\n> > > +\n> > > +/* mask of enabled ports */\n> > > +static uint32_t enabled_port_mask = 0;\n> > > +\n> > > +static volatile struct app_stats {\n> > > +\tstruct {\n> > > +\t\tuint64_t rx_pkts;\n> > > +\t\tuint64_t returned_pkts;\n> > > +\t\tuint64_t enqueued_pkts;\n> > > +\t} rx __rte_cache_aligned;\n> > > +\n> > > +\tstruct {\n> > > +\t\tuint64_t dequeue_pkts;\n> > > +\t\tuint64_t tx_pkts;\n> > > +\t} tx __rte_cache_aligned;\n> > > +} app_stats;\n> > > +\n> > > +static const struct rte_eth_conf port_conf_default = {\n> > > +\t.rxmode = {\n> > > +\t\t.mq_mode = ETH_MQ_RX_RSS,\n> > > +\t\t.max_rx_pkt_len = ETHER_MAX_LEN,\n> > > +\t\t.split_hdr_size = 0,\n> > > +\t\t.header_split = 0, /**< Header Split disabled */\n> > > +\t\t.hw_ip_checksum = 0, /**< IP checksum offload enabled */\n> > > +\t\t.hw_vlan_filter = 0, /**< VLAN filtering disabled */\n> > > +\t\t.jumbo_frame = 0, /**< Jumbo Frame Support disabled */\n> > > +\t\t.hw_strip_crc = 0, /**< CRC stripped by hardware */\n> > > +\t},\n> > > +\t.txmode = {\n> > > +\t\t.mq_mode = ETH_MQ_TX_NONE,\n> > > +\t},\n> > > +\t.lpbk_mode = 0,\n> > > +\t.rx_adv_conf = {\n> > > +\t\t\t.rss_conf = {\n> > > +\t\t\t\t.rss_hf = ETH_RSS_IPV4 | ETH_RSS_IPV6 |\n> > > +\t\t\t\t\tETH_RSS_IPV4_TCP |\n> > > ETH_RSS_IPV4_UDP |\n> > > +\t\t\t\t\tETH_RSS_IPV6_TCP |\n> > > ETH_RSS_IPV6_UDP,\n> > > +\t\t\t}\n> > > +\t},\n> > > +};\n> > > +\n> > > +static const struct rte_eth_rxconf rx_conf_default = {\n> > > +\t.rx_thresh = {\n> > > +\t\t.pthresh = RX_PTHRESH,\n> > > +\t\t.hthresh = RX_HTHRESH,\n> > > +\t\t.wthresh = RX_WTHRESH,\n> > > +\t},\n> > > +\t.rx_free_thresh = RX_FREE_THRESH,\n> > > +\t.rx_drop_en = 0,\n> > > +};\n> > > +\n> > > +static const struct rte_eth_txconf tx_conf_default = {\n> > > +\t.tx_thresh = {\n> > > +\t\t.pthresh = TX_PTHRESH,\n> > > +\t\t.hthresh = TX_HTHRESH,\n> > > +\t\t.wthresh = TX_WTHRESH,\n> > > +\t},\n> > > +\t.tx_free_thresh = TX_FREE_THRESH,\n> > > +\t.tx_rs_thresh = TX_RSBIT_THRESH,\n> > > +\t.txq_flags = TX_Q_FLAGS\n> > > +\n> > > +};\n> > > +\n> > > +struct output_buffer {\n> > > +\tunsigned count;\n> > > +\tstruct rte_mbuf *mbufs[BURST_SIZE]; };\n> > > +\n> > > +/*\n> > > + * Initialises a given port using global settings and with the rx\n> > > +buffers\n> > > + * coming from the mbuf_pool passed as parameter */ static inline\n> > > +int port_init(uint8_t port, struct rte_mempool *mbuf_pool) {\n> > > +\tstruct rte_eth_conf port_conf = port_conf_default;\n> > > +\tconst uint16_t rxRings = 1, txRings = rte_lcore_count() - 1;\n> > > +\tint retval;\n> > > +\tuint16_t q;\n> > > +\n> > > +\tif (port >= rte_eth_dev_count())\n> > > +\t\treturn -1;\n> > > +\n> > > +\tretval = rte_eth_dev_configure(port, rxRings, txRings, &port_conf);\n> > > +\tif (retval != 0)\n> > > +\t\treturn retval;\n> > > +\n> > > +\tfor (q = 0; q < rxRings; q++) {\n> > > +\t\tretval = rte_eth_rx_queue_setup(port, q, RX_RING_SIZE,\n> > > +\n> > > \trte_eth_dev_socket_id(port),\n> > > +\t\t\t\t\t\t&rx_conf_default,\n> > > mbuf_pool);\n> > > +\t\tif (retval < 0)\n> > > +\t\t\treturn retval;\n> > > +\t}\n> > > +\n> > > +\tfor (q = 0; q < txRings; q++) {\n> > > +\t\tretval = rte_eth_tx_queue_setup(port, q, TX_RING_SIZE,\n> > > +\n> > > \trte_eth_dev_socket_id(port),\n> > > +\t\t\t\t\t\t&tx_conf_default);\n> > > +\t\tif (retval < 0)\n> > > +\t\t\treturn retval;\n> > > +\t}\n> > > +\n> > > +\tretval = rte_eth_dev_start(port);\n> > > +\tif (retval < 0)\n> > > +\t\treturn retval;\n> > > +\n> > > +\tstruct rte_eth_link link;\n> > > +\trte_eth_link_get_nowait(port, &link);\n> > > +\tif (!link.link_status) {\n> > > +\t\tsleep(1);\n> > > +\t\trte_eth_link_get_nowait(port, &link);\n> > > +\t}\n> > > +\n> > > +\tif (!link.link_status) {\n> > > +\t\tprintf(\"Link down on port %\"PRIu8\"\\n\", port);\n> > > +\t\treturn 0;\n> > > +\t}\n> > > +\n> > > +\tstruct ether_addr addr;\n> > > +\trte_eth_macaddr_get(port, &addr);\n> > > +\tprintf(\"Port %u MAC: %02\"PRIx8\" %02\"PRIx8\" %02\"PRIx8\n> > > +\t\t\t\" %02\"PRIx8\" %02\"PRIx8\" %02\"PRIx8\"\\n\",\n> > > +\t\t\t(unsigned)port,\n> > > +\t\t\taddr.addr_bytes[0], addr.addr_bytes[1],\n> > > +\t\t\taddr.addr_bytes[2], addr.addr_bytes[3],\n> > > +\t\t\taddr.addr_bytes[4], addr.addr_bytes[5]);\n> > > +\n> > > +\trte_eth_promiscuous_enable(port);\n> > > +\n> > > +\treturn 0;\n> > > +}\n> > > +\n> > > +struct lcore_params {\n> > > +\tunsigned worker_id;\n> > > +\tstruct rte_distributor *d;\n> > > +\tstruct rte_ring *r;\n> > > +};\n> > > +\n> > > +static __attribute__((noreturn)) void lcore_rx(struct lcore_params\n> > > +*p) {\n> > > +\tstruct rte_distributor *d = p->d;\n> > > +\tstruct rte_ring *r = p->r;\n> > > +\tconst uint8_t nb_ports = rte_eth_dev_count();\n> > > +\tconst int socket_id = rte_socket_id();\n> > > +\tuint8_t port;\n> > > +\n> > > +\tfor (port = 0; port < nb_ports; port++) {\n> > > +\t\t/* skip ports that are not enabled */\n> > > +\t\tif ((enabled_port_mask & (1 << port)) == 0)\n> > > +\t\t\tcontinue;\n> > > +\n> > > +\t\tif (rte_eth_dev_socket_id(port) > 0 &&\n> > > +\t\t\t\trte_eth_dev_socket_id(port) != socket_id)\n> > > +\t\t\tprintf(\"WARNING, port %u is on remote NUMA node\n> > > to \"\n> > > +\t\t\t\t\t\"RX thread.\\n\\tPerformance will not\n> > > \"\n> > > +\t\t\t\t\t\"be optimal.\\n\", port);\n> > > +\t}\n> > > +\n> > > +\tprintf(\"\\nCore %u doing packet RX.\\n\", rte_lcore_id());\n> > > +\tport = 0;\n> > > +\tfor (;;) {\n> > > +\t\t/* skip ports that are not enabled */\n> > > +\t\tif ((enabled_port_mask & (1 << port)) == 0) {\n> > > +\t\t\tif (++port == nb_ports)\n> > > +\t\t\t\tport = 0;\n> > > +\t\t\tcontinue;\n> > > +\t\t}\n> > > +\t\tstruct rte_mbuf *bufs[BURST_SIZE*2];\n> > > +\t\tconst uint16_t nb_rx = rte_eth_rx_burst(port, 0, bufs,\n> > > +\t\t\t\tBURST_SIZE);\n> > > +\t\tapp_stats.rx.rx_pkts += nb_rx;\n> > > +\n> > > +\t\trte_distributor_process(d, bufs, nb_rx);\n> > > +\t\tconst uint16_t nb_ret = rte_distributor_returned_pkts(d,\n> > > +\t\t\t\tbufs, BURST_SIZE*2);\n> > > +\t\tapp_stats.rx.returned_pkts += nb_ret;\n> > > +\t\tif (unlikely(nb_ret == 0))\n> > > +\t\t\tcontinue;\n> > > +\n> > > +\t\tuint16_t sent = rte_ring_enqueue_burst(r, (void *)bufs,\n> > > nb_ret);\n> > > +\t\tapp_stats.rx.enqueued_pkts += sent;\n> > > +\t\tif (unlikely(sent < nb_ret)) {\n> > > +\t\t\tprintf(\"Packet loss due to full ring\\n\");\n> >\n> > I would remove this printf. If core is too busy to process packets, this may make performance worse.\n> > Statistics should reflect this packet loss.\n> \n> >Probably use RTE_LOG(DEBUG, ...).\n> >That's way you can switch it on/off at compile time.\n> \n> By default LOG_LEVEL is 8 in config/common_linuxapp, hence I see RTE_LOG(DEBUG,...) always works.\n> So until unless change the LOG-LEVEL to lesser value, RTE_LOG(DEBUG,..) will be logged ..\n> Please let me know if I can go ahead and remove the printf as of now until the LOG_LEVEL is fixed. (or) I have to add some logic to log\n> RTE_LOG(DEBUG,..) only once.\n\nSorry, I probably wasn't clear enough.\nWhat I am saying - create something like that inside your app code:\n\n#ifdef XXX_DEBUG\n#define XXX_LOG(level, fmt, args...) \\\n RTE_LOG(level, PMD, \"%s(): \" fmt \"\\n\", __func__, ## args)\n#else\n#define XXX_LOG(level, fmt, args...) do { } while(0)\n#endif\n\nAnd replace your pritnf()s with it.\n\n> \n> >\n> > > +\t\t\twhile (sent < nb_ret)\n> > > +\t\t\t\trte_pktmbuf_free(bufs[sent++]);\n> > > +\t\t}\n> > > +\t\tif (++port == nb_ports)\n> > > +\t\t\tport = 0;\n> > > +\t}\n> > > +}\n> > > +\n> > > +static inline void\n> > > +flush_one_port(struct output_buffer *outbuf, uint8_t outp) {\n> > > +\tunsigned nb_tx = rte_eth_tx_burst(outp, 0, outbuf->mbufs,\n> > > +\t\t\toutbuf->count);\n> > > +\tapp_stats.tx.tx_pkts += nb_tx;\n> > > +\n> > > +\tif (unlikely(nb_tx < outbuf->count)) {\n> > > +\t\tprintf(\"Packet loss with tx_burst\\n\");\n> >\n> > Same here. This is fine for debugging, but statistics are updated as well, so we do not need this printf.\n> >\n> > > +\t\tdo {\n> > > +\t\t\trte_pktmbuf_free(outbuf->mbufs[nb_tx]);\n> > > +\t\t} while (++nb_tx < outbuf->count);\n> > > +\t}\n> > > +\toutbuf->count = 0;\n> > > +}\n> > > +\n> > > +static inline void\n> > > +flush_all_ports(struct output_buffer *tx_buffers, uint8_t nb_ports)\n> > > +{\n> > > +\tuint8_t outp;\n> > > +\tfor (outp = 0; outp < nb_ports; outp++) {\n> > > +\t\t/* skip ports that are not enabled */\n> > > +\t\tif ((enabled_port_mask & (1 << outp)) == 0)\n> > > +\t\t\tcontinue;\n> > > +\n> > > +\t\tif (tx_buffers[outp].count == 0)\n> > > +\t\t\tcontinue;\n> > > +\n> > > +\t\tflush_one_port(&tx_buffers[outp], outp);\n> > > +\t}\n> > > +}\n> > > +\n> > > +static __attribute__((noreturn)) void lcore_tx(struct rte_ring\n> > > +*in_r) {\n> > > +\tstatic struct output_buffer tx_buffers[RTE_MAX_ETHPORTS];\n> > > +\tconst uint8_t nb_ports = rte_eth_dev_count();\n> > > +\tconst int socket_id = rte_socket_id();\n> > > +\tuint8_t port;\n> > > +\n> > > +\tfor (port = 0; port < nb_ports; port++) {\n> > > +\t\t/* skip ports that are not enabled */\n> > > +\t\tif ((enabled_port_mask & (1 << port)) == 0)\n> > > +\t\t\tcontinue;\n> > > +\n> > > +\t\tif (rte_eth_dev_socket_id(port) > 0 &&\n> > > +\t\t\t\trte_eth_dev_socket_id(port) != socket_id)\n> > > +\t\t\tprintf(\"WARNING, port %u is on remote NUMA node\n> > > to \"\n> > > +\t\t\t\t\t\"TX thread.\\n\\tPerformance will not \"\n> > > +\t\t\t\t\t\"be optimal.\\n\", port);\n> > > +\t}\n> > > +\n> > > +\tprintf(\"\\nCore %u doing packet TX.\\n\", rte_lcore_id());\n> > > +\tfor (;;) {\n> > > +\t\tfor (port = 0; port < nb_ports; port++) {\n> > > +\t\t\t/* skip ports that are not enabled */\n> > > +\t\t\tif ((enabled_port_mask & (1 << port)) == 0)\n> > > +\t\t\t\tcontinue;\n> > > +\n> > > +\t\t\tstruct rte_mbuf *bufs[BURST_SIZE];\n> > > +\t\t\tconst uint16_t nb_rx = rte_ring_dequeue_burst(in_r,\n> > > +\t\t\t\t\t(void *)bufs, BURST_SIZE);\n> > > +\t\t\tapp_stats.tx.dequeue_pkts += nb_rx;\n> > > +\n> > > +\t\t\t/* if we get no traffic, flush anything we have */\n> > > +\t\t\tif (unlikely(nb_rx == 0)) {\n> > > +\t\t\t\tflush_all_ports(tx_buffers, nb_ports);\n> > > +\t\t\t\tcontinue;\n> > > +\t\t\t}\n> > > +\n> > > +\t\t\t/* for traffic we receive, queue it up for transmit */\n> > > +\t\t\tuint16_t i;\n> > > +\t\t\t_mm_prefetch(bufs[0], 0);\n> > > +\t\t\t_mm_prefetch(bufs[1], 0);\n> > > +\t\t\t_mm_prefetch(bufs[2], 0);\n> > > +\t\t\tfor (i = 0; i < nb_rx; i++) {\n> > > +\t\t\t\tstruct output_buffer *outbuf;\n> > > +\t\t\t\tuint8_t outp;\n> > > +\t\t\t\t_mm_prefetch(bufs[i + 3], 0);\n> > > +\t\t\t\t/* workers should update in_port to hold the\n> > > +\t\t\t\t * output port value */\n> > > +\t\t\t\toutp = bufs[i]->port;\n> > > +\t\t\t\t/* skip ports that are not enabled */\n> > > +\t\t\t\tif ((enabled_port_mask & (1 << outp)) == 0)\n> > > +\t\t\t\t\tcontinue;\n> > > +\n> > > +\t\t\t\toutbuf = &tx_buffers[outp];\n> > > +\t\t\t\toutbuf->mbufs[outbuf->count++] = bufs[i];\n> > > +\t\t\t\tif (outbuf->count == BURST_SIZE)\n> > > +\t\t\t\t\tflush_one_port(outbuf, outp);\n> > > +\t\t\t}\n> > > +\t\t}\n> > > +\t}\n> > > +}\n> > > +\n> > > +\n> > > +static __attribute__((noreturn)) void lcore_worker(struct\n> > > +lcore_params *p) {\n> > > +\tstruct rte_distributor *d = p->d;\n> > > +\tconst unsigned id = p->worker_id;\n> > > +\t/* for single port, xor_val will be zero so we won't modify the output\n> > > +\t * port, otherwise we send traffic from 0 to 1, 2 to 3, and vice versa\n> > > +\t */\n> > > +\tconst unsigned xor_val = (rte_eth_dev_count() > 1);\n> > > +\tstruct rte_mbuf *buf = NULL;\n> > > +\n> > > +\tprintf(\"\\nCore %u acting as worker core.\\n\", rte_lcore_id());\n> > > +\tfor (;;) {\n> > > +\t\tbuf = rte_distributor_get_pkt(d, id, buf);\n> > > +\t\tbuf->port ^= xor_val;\n> > > +\t}\n> > > +}\n> > > +\n> > > +static void\n> > > +int_handler(int sig_num)\n> > > +{\n> > > +\tstruct rte_eth_stats eth_stats;\n> > > +\tunsigned i;\n> > > +\n> > > +\tprintf(\"Exiting on signal %d\\n\", sig_num);\n> > > +\n> > > +\tprintf(\"\\nRX thread stats:\\n\");\n> > > +\tprintf(\" - Received: %\"PRIu64\"\\n\", app_stats.rx.rx_pkts);\n> > > +\tprintf(\" - Processed: %\"PRIu64\"\\n\", app_stats.rx.returned_pkts);\n> > > +\tprintf(\" - Enqueued: %\"PRIu64\"\\n\", app_stats.rx.enqueued_pkts);\n> > > +\n> > > +\tprintf(\"\\nTX thread stats:\\n\");\n> > > +\tprintf(\" - Dequeued: %\"PRIu64\"\\n\", app_stats.tx.dequeue_pkts);\n> > > +\tprintf(\" - Transmitted: %\"PRIu64\"\\n\", app_stats.tx.tx_pkts);\n> > > +\n> > > +\tfor (i = 0; i < rte_eth_dev_count(); i++) {\n> > > +\t\trte_eth_stats_get(i, &eth_stats);\n> > > +\t\tprintf(\"\\nPort %u stats:\\n\", i);\n> > > +\t\tprintf(\" - Pkts in: %\"PRIu64\"\\n\", eth_stats.ipackets);\n> > > +\t\tprintf(\" - Pkts out: %\"PRIu64\"\\n\", eth_stats.opackets);\n> > > +\t\tprintf(\" - In Errs: %\"PRIu64\"\\n\", eth_stats.ierrors);\n> > > +\t\tprintf(\" - Out Errs: %\"PRIu64\"\\n\", eth_stats.oerrors);\n> > > +\t\tprintf(\" - Mbuf Errs: %\"PRIu64\"\\n\", eth_stats.rx_nombuf);\n> > > +\t}\n> > > +\texit(0);\n> > > +}\n> > > +\n> > > +/* display usage */\n> > > +static void\n> > > +print_usage(const char *prgname)\n> > > +{\n> > > +\tprintf(\"%s [EAL options] -- -p PORTMASK\\n\"\n> > > +\t\t\t\" -p PORTMASK: hexadecimal bitmask of ports to\n> > > configure\\n\",\n> > > +\t\t\tprgname);\n> > > +}\n> > > +\n> > > +static int\n> > > +parse_portmask(const char *portmask) {\n> > > +\tchar *end = NULL;\n> > > +\tunsigned long pm;\n> > > +\n> > > +\t/* parse hexadecimal string */\n> > > +\tpm = strtoul(portmask, &end, 16);\n> > > +\tif ((portmask[0] == '\\0') || (end == NULL) || (*end != '\\0'))\n> > > +\t\treturn -1;\n> > > +\n> > > +\tif (pm == 0)\n> > > +\t\treturn -1;\n> > > +\n> > > +\treturn pm;\n> > > +}\n> > > +\n> > > +/* Parse the argument given in the command line of the application\n> > > +*/ static int parse_args(int argc, char **argv) {\n> > > +\tint opt;\n> > > +\tchar **argvopt;\n> > > +\tint option_index;\n> > > +\tchar *prgname = argv[0];\n> > > +\tstatic struct option lgopts[] = {\n> > > +\t\t{NULL, 0, 0, 0}\n> > > +\t};\n> > > +\n> > > +\targvopt = argv;\n> > > +\n> > > +\twhile ((opt = getopt_long(argc, argvopt, \"p:\",\n> > > +\t\t\tlgopts, &option_index)) != EOF) {\n> > > +\n> > > +\t\tswitch (opt) {\n> > > +\t\t\t/* portmask */\n> > > +\t\t\tcase 'p':\n> > > +\t\t\t\tenabled_port_mask =\n> > > parse_portmask(optarg);\n> > > +\t\t\t\tif (enabled_port_mask == 0) {\n> > > +\t\t\t\t\tprintf(\"invalid portmask\\n\");\n> > > +\t\t\t\t\tprint_usage(prgname);\n> > > +\t\t\t\t\treturn -1;\n> > > +\t\t\t\t}\n> > > +\t\t\t\tbreak;\n> > > +\n> > > +\t\t\tdefault:\n> > > +\t\t\t\tprint_usage(prgname);\n> > > +\t\t\t\treturn -1;\n> > > +\t\t}\n> > > +\t}\n> > > +\n> > > +if (optind <= 1) {\n> > > +\tprint_usage(prgname);\n> > > +\treturn -1;\n> > > +}\n> > > +\n> > > +\targv[optind-1] = prgname;\n> > > +\n> > > +\toptind = 0; /* reset getopt lib */\n> > > +\treturn 0;\n> > > +}\n> > > +\n> > > +/* Main function, does initialization and calls the per-lcore\n> > > +functions */ int MAIN(int argc, char *argv[]) {\n> > > +\tstruct rte_mempool *mbuf_pool;\n> > > +\tstruct rte_distributor *d;\n> > > +\tstruct rte_ring *output_ring;\n> > > +\tunsigned lcore_id, worker_id = 0;\n> > > +\tunsigned nb_ports;\n> > > +\tuint8_t portid;\n> > > +\tuint8_t nb_ports_available;\n> > > +\n> > > +\t/* catch ctrl-c so we can print on exit */\n> > > +\tsignal(SIGINT, int_handler);\n> > > +\n> > > +\t/* init EAL */\n> > > +\tint ret = rte_eal_init(argc, argv);\n> > > +\tif (ret < 0)\n> > > +\t\trte_exit(EXIT_FAILURE, \"Error with EAL initialization\\n\");\n> > > +\targc -= ret;\n> > > +\targv += ret;\n> > > +\n> > > +\t/* parse application arguments (after the EAL ones) */\n> > > +\tret = parse_args(argc, argv);\n> > > +\tif (ret < 0)\n> > > +\t\trte_exit(EXIT_FAILURE, \"Invalid distributor parameters\\n\");\n> > > +\n> > > +\tif (rte_lcore_count() < 3)\n> > > +\t\trte_exit(EXIT_FAILURE, \"Error, This application needs at \"\n> > > +\t\t\t\t\"least 3 logical cores to run:\\n\"\n> > > +\t\t\t\t\"1 lcore for packet RX and distribution\\n\"\n> > > +\t\t\t\t\"1 lcore for packet TX\\n\"\n> > > +\t\t\t\t\"and at least 1 lcore for worker threads\\n\");\n> > > +\n> > > +\tif (rte_eal_pci_probe() != 0)\n> > > +\t\trte_exit(EXIT_FAILURE, \"Error with PCI probing\\n\");\n> > > +\n> > > +\tnb_ports = rte_eth_dev_count();\n> > > +\tif (nb_ports == 0)\n> > > +\t\trte_exit(EXIT_FAILURE, \"Error: no ethernet ports\n> > > detected\\n\");\n> > > +\tif (nb_ports != 1 && (nb_ports & 1))\n> > > +\t\trte_exit(EXIT_FAILURE, \"Error: number of ports must be\n> > > even, except \"\n> > > +\t\t\t\t\"when using a single port\\n\");\n> > > +\n> > > +\tmbuf_pool = rte_mempool_create(\"MBUF_POOL\", NUM_MBUFS *\n> > > nb_ports,\n> > > +\t\t\tMBUF_SIZE, MBUF_CACHE_SIZE,\n> > > +\t\t\tsizeof(struct rte_pktmbuf_pool_private),\n> > > +\t\t\trte_pktmbuf_pool_init, NULL,\n> > > +\t\t\trte_pktmbuf_init, NULL,\n> > > +\t\t\trte_socket_id(), 0);\n> > > +\tif (mbuf_pool == NULL)\n> > > +\t\trte_exit(EXIT_FAILURE, \"Cannot create mbuf pool\\n\");\n> > > +\n> > > +\tnb_ports_available = nb_ports;\n> > > +\n> > > +\t/* initialize all ports */\n> > > +\tfor (portid = 0; portid < nb_ports; portid++) {\n> > > +\t\t/* skip ports that are not enabled */\n> > > +\t\tif ((enabled_port_mask & (1 << portid)) == 0) {\n> > > +\t\t\tprintf(\"\\nSkipping disabled port %d\\n\", portid);\n> > > +\t\t\tnb_ports_available--;\n> > > +\t\t\tcontinue;\n> > > +\t\t}\n> > > +\t\t/* init port */\n> > > +\t\tprintf(\"Initializing port %u... done\\n\", (unsigned) portid);\n> > > +\n> > > +\t\tif (port_init(portid, mbuf_pool) != 0)\n> > > +\t\t\trte_exit(EXIT_FAILURE, \"Cannot initialize port\n> > > %\"PRIu8\"\\n\",\n> > > +\t\t\t\t\tportid);\n> > > +\t}\n> > > +\n> > > +\tif (!nb_ports_available) {\n> > > +\t\trte_exit(EXIT_FAILURE,\n> > > +\t\t\t\t\"All available ports are disabled. Please set\n> > > portmask.\\n\");\n> > > +\t}\n> > > +\n> > > +\td = rte_distributor_create(\"PKT_DIST\", rte_socket_id(),\n> > > +\t\t\trte_lcore_count() - 2);\n> > > +\tif (d == NULL)\n> > > +\t\trte_exit(EXIT_FAILURE, \"Cannot create distributor\\n\");\n> > > +\n> > > +\t/* scheduler ring is read only by the transmitter core, but written to\n> > > +\t * by multiple threads\n> > > +\t */\n> > > +\toutput_ring = rte_ring_create(\"Output_ring\", RTE_RING_SZ,\n> > > +\t\t\trte_socket_id(), RING_F_SC_DEQ);\n> > > +\tif (output_ring == NULL)\n> > > +\t\trte_exit(EXIT_FAILURE, \"Cannot create output ring\\n\");\n> > > +\n> > > +\tRTE_LCORE_FOREACH_SLAVE(lcore_id) {\n> > > +\t\tif (worker_id == rte_lcore_count() - 2)\n> > > +\t\t\trte_eal_remote_launch((lcore_function_t\n> > > *)lcore_tx,\n> > > +\t\t\t\t\toutput_ring, lcore_id);\n> > > +\t\telse {\n> > > +\t\t\tstruct lcore_params *p =\n> > > +\t\t\t\t\trte_malloc(NULL, sizeof(*p), 0);\n> > > +\t\t\tif (!p)\n> > > +\t\t\t\trte_panic(\"malloc failure\\n\");\n> > > +\t\t\t*p = (struct lcore_params){worker_id, d,\n> > > output_ring};\n> > > +\t\t\trte_eal_remote_launch((lcore_function_t\n> > > *)lcore_worker,\n> > > +\t\t\t\t\tp, lcore_id);\n> > > +\t\t}\n> > > +\t\tworker_id++;\n> > > +\t}\n> > > +\t/* call lcore_main on master core only */\n> > > +\tstruct lcore_params p = { 0, d, output_ring };\n> > > +\tlcore_rx(&p);\n> > > +\treturn 0;\n> > > +}\n> > > diff --git a/examples/distributor_app/main.h\n> > > b/examples/distributor_app/main.h new file mode 100644 index\n> > > 0000000..2682d15\n> > > --- /dev/null\n> > > +++ b/examples/distributor_app/main.h\n> > > @@ -0,0 +1,46 @@\n> > > +/*-\n> > > + * BSD LICENSE\n> > > + *\n> > > + * Copyright(c) 2010-2014 Intel Corporation. All rights reserved.\n> > > + * All rights reserved.\n> > > + *\n> > > + * Redistribution and use in source and binary forms, with or without\n> > > + * modification, are permitted provided that the following conditions\n> > > + * are met:\n> > > + *\n> > > + * * Redistributions of source code must retain the above copyright\n> > > + * notice, this list of conditions and the following disclaimer.\n> > > + * * Redistributions in binary form must reproduce the above copyright\n> > > + * notice, this list of conditions and the following disclaimer in\n> > > + * the documentation and/or other materials provided with the\n> > > + * distribution.\n> > > + * * Neither the name of Intel Corporation nor the names of its\n> > > + * contributors may be used to endorse or promote products derived\n> > > + * from this software without specific prior written permission.\n> > > + *\n> > > + * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND\n> > > CONTRIBUTORS\n> > > + * \"AS IS\" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT\n> > > NOT\n> > > + * LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND\n> > > FITNESS FOR\n> > > + * A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE\n> > > COPYRIGHT\n> > > + * OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT,\n> > > INCIDENTAL,\n> > > + * SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT\n> > > NOT\n> > > + * LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS\n> > > OF USE,\n> > > + * DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED\n> > > AND ON ANY\n> > > + * THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR\n> > > TORT\n> > > + * (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF\n> > > THE USE\n> > > + * OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH\n> > > DAMAGE.\n> > > + */\n> > > +\n> > > +#ifndef _MAIN_H_\n> > > +#define _MAIN_H_\n> > > +\n> > > +\n> > > +#ifdef RTE_EXEC_ENV_BAREMETAL\n> > > +#define MAIN _main\n> > > +#else\n> > > +#define MAIN main\n> > > +#endif\n> > > +\n> > > +int MAIN(int argc, char *argv[]);\n> > > +\n> > > +#endif /* ifndef _MAIN_H_ */\n> > > --\n> > > 1.9.3", "headers": { "Return-Path": "<dev-bounces@dpdk.org>", "X-Original-To": "patchwork@dpdk.org", "Delivered-To": "patchwork@dpdk.org", "Received": [ "from [92.243.14.124] (localhost [IPv6:::1])\n\tby dpdk.org (Postfix) with ESMTP id 1D0EB7E18;\n\tMon, 29 Sep 2014 15:00:33 +0200 (CEST)", "from mga09.intel.com (mga09.intel.com [134.134.136.24])\n\tby dpdk.org (Postfix) with ESMTP id 1C7396A95\n\tfor <dev@dpdk.org>; Mon, 29 Sep 2014 15:00:29 +0200 (CEST)", "from orsmga002.jf.intel.com ([10.7.209.21])\n\tby orsmga102.jf.intel.com with ESMTP; 29 Sep 2014 06:00:44 -0700", "from irsmsx101.ger.corp.intel.com ([163.33.3.153])\n\tby orsmga002.jf.intel.com with ESMTP; 29 Sep 2014 06:06:59 -0700", "from irsmsx104.ger.corp.intel.com ([169.254.5.248]) by\n\tIRSMSX101.ger.corp.intel.com ([169.254.1.201]) with mapi id\n\t14.03.0195.001; Mon, 29 Sep 2014 14:06:58 +0100" ], "X-ExtLoop1": "1", "X-IronPort-AV": "E=Sophos;i=\"5.04,620,1406617200\"; d=\"scan'208\";a=\"610247844\"", "From": "\"Ananyev, Konstantin\" <konstantin.ananyev@intel.com>", "To": "\"Pattan, Reshma\" <reshma.pattan@intel.com>, \"De Lara Guarch, Pablo\"\n\t<pablo.de.lara.guarch@intel.com>, \"dev@dpdk.org\" <dev@dpdk.org>", "Thread-Topic": "[dpdk-dev] [PATCH v2] distributor_app: new sample app", "Thread-Index": "AQHP2AJemxe7phgTsUqwEI/Y1qWz+ZwTd5UAgAAbAeCABHF4gIAAFzyw", "Date": "Mon, 29 Sep 2014 13:06:57 +0000", "Message-ID": "<2601191342CEEE43887BDE71AB97725821387654@IRSMSX104.ger.corp.intel.com>", "References": "<1410869607-16842-1-git-send-email-reshma.pattan@intel.com>\n\t<1411568210-2555-1-git-send-email-reshma.pattan@intel.com>\n\t<E115CCD9D858EF4F90C690B0DCB4D89722626933@IRSMSX108.ger.corp.intel.com>\n\t<2601191342CEEE43887BDE71AB9772582137D8B1@IRSMSX104.ger.corp.intel.com>\n\t<3AEA2BF9852C6F48A459DA490692831FE20598@IRSMSX109.ger.corp.intel.com>", "In-Reply-To": "<3AEA2BF9852C6F48A459DA490692831FE20598@IRSMSX109.ger.corp.intel.com>", "Accept-Language": "en-IE, en-US", "Content-Language": "en-US", "X-MS-Has-Attach": "", "X-MS-TNEF-Correlator": "", "x-originating-ip": "[163.33.239.181]", "Content-Type": "text/plain; charset=\"us-ascii\"", "Content-Transfer-Encoding": "quoted-printable", "MIME-Version": "1.0", "Subject": "Re: [dpdk-dev] [PATCH v2] distributor_app: new sample app", "X-BeenThere": "dev@dpdk.org", "X-Mailman-Version": "2.1.15", "Precedence": "list", "List-Id": "patches and discussions about DPDK <dev.dpdk.org>", "List-Unsubscribe": "<http://dpdk.org/ml/options/dev>,\n\t<mailto:dev-request@dpdk.org?subject=unsubscribe>", "List-Archive": "<http://dpdk.org/ml/archives/dev/>", "List-Post": "<mailto:dev@dpdk.org>", "List-Help": "<mailto:dev-request@dpdk.org?subject=help>", "List-Subscribe": "<http://dpdk.org/ml/listinfo/dev>,\n\t<mailto:dev-request@dpdk.org?subject=subscribe>", "Errors-To": "dev-bounces@dpdk.org", "Sender": "\"dev\" <dev-bounces@dpdk.org>" }, "addressed": null }, { "id": 1232, "web_url": "https://patches.dpdk.org/comment/1232/", "msgid": "<E115CCD9D858EF4F90C690B0DCB4D8972262823A@IRSMSX108.ger.corp.intel.com>", "list_archive_url": "https://inbox.dpdk.org/dev/E115CCD9D858EF4F90C690B0DCB4D8972262823A@IRSMSX108.ger.corp.intel.com", "date": "2014-09-29T13:35:21", "subject": "Re: [dpdk-dev] [PATCH v2] distributor_app: new sample app", "submitter": { "id": 9, "url": "https://patches.dpdk.org/api/people/9/?format=api", "name": "De Lara Guarch, Pablo", "email": "pablo.de.lara.guarch@intel.com" }, "content": "> -----Original Message-----\n> From: Ananyev, Konstantin\n> Sent: Monday, September 29, 2014 2:07 PM\n> To: Pattan, Reshma; De Lara Guarch, Pablo; dev@dpdk.org\n> Subject: RE: [dpdk-dev] [PATCH v2] distributor_app: new sample app\n> \n> \n> \n> > -----Original Message-----\n> > From: Pattan, Reshma\n> > Sent: Monday, September 29, 2014 1:40 PM\n> > To: Ananyev, Konstantin; De Lara Guarch, Pablo; dev@dpdk.org\n> > Subject: RE: [dpdk-dev] [PATCH v2] distributor_app: new sample app\n> >\n> >\n> >\n> > -----Original Message-----\n> > From: Ananyev, Konstantin\n> > Sent: Friday, September 26, 2014 4:52 PM\n> > To: De Lara Guarch, Pablo; Pattan, Reshma; dev@dpdk.org\n> > Subject: RE: [dpdk-dev] [PATCH v2] distributor_app: new sample app\n> >\n> >\n> >\n> > > -----Original Message-----\n> > > From: dev [mailto:dev-bounces@dpdk.org] On Behalf Of De Lara Guarch,\n> > > Pablo\n> > > Sent: Friday, September 26, 2014 4:12 PM\n> > > To: Pattan, Reshma; dev@dpdk.org\n> > > Subject: Re: [dpdk-dev] [PATCH v2] distributor_app: new sample app\n> > >\n> > > Hi,\n> > >\n> > > > -----Original Message-----\n> > > > From: dev [mailto:dev-bounces@dpdk.org] On Behalf Of reshmapa\n> > > > Sent: Wednesday, September 24, 2014 3:17 PM\n> > > > To: dev@dpdk.org\n> > > > Subject: [dpdk-dev] [PATCH v2] distributor_app: new sample app\n> > > >\n> > > > From: Reshma Pattan <reshma.pattan@intel.com>\n> > > >\n> > > > A new sample app that shows the usage of the distributor library.\n> > > > This app works as follows:\n> > > >\n> > > > * An RX thread runs which pulls packets from each ethernet port in turn\n> > > > and passes those packets to worker using a distributor component.\n> > > > * The workers take the packets in turn, and determine the output port\n> > > > for those packets using basic l2forwarding doing an xor on the source\n> > > > port id.\n> > > > * The RX thread takes the returned packets from the workers and\n> enqueue\n> > > > those packets into an rte_ring structure.\n> > > > * A TX thread pulls the packets off the rte_ring structure and then\n> > > > sends each packet out the output port specified previously by the\n> > > > worker\n> > > > * Command-line option support provided only for portmask.\n> > > >\n> > > > Signed-off-by: Bruce Richardson <bruce.richardson@intel.com>\n> > > > Signed-off-by: Reshma Pattan <reshma.pattan@intel.com>\n> > > > ---\n> > > > examples/Makefile | 1 +\n> > > > examples/distributor_app/Makefile | 57 ++++\n> > > > examples/distributor_app/main.c | 585\n> > > > ++++++++++++++++++++++++++++++++++++++\n> > > > examples/distributor_app/main.h | 46 +++\n> > > > 4 files changed, 689 insertions(+)\n> > > > create mode 100644 examples/distributor_app/Makefile create mode\n> > > > 100644 examples/distributor_app/main.c create mode 100644\n> > > > examples/distributor_app/main.h\n> > > >\n> > > > diff --git a/examples/Makefile b/examples/Makefile index\n> > > > 6245f83..2ba82b0 100644\n> > > > --- a/examples/Makefile\n> > > > +++ b/examples/Makefile\n> > > > @@ -66,5 +66,6 @@ DIRS-y += vhost\n> > > > DIRS-$(CONFIG_RTE_LIBRTE_XEN_DOM0) += vhost_xen DIRS-y +=\n> vmdq\n> > > > DIRS-y += vmdq_dcb\n> > > > +DIRS-$(CONFIG_RTE_LIBRTE_DISTRIBUTOR) += distributor_app\n> > > >\n> > > > include $(RTE_SDK)/mk/rte.extsubdir.mk diff --git\n> > > > a/examples/distributor_app/Makefile\n> > > > b/examples/distributor_app/Makefile\n> > > > new file mode 100644\n> > > > index 0000000..394785d\n> > > > --- /dev/null\n> > > > +++ b/examples/distributor_app/Makefile\n> > > > @@ -0,0 +1,57 @@\n> > > > +# BSD LICENSE\n> > > > +#\n> > > > +# Copyright(c) 2010-2014 Intel Corporation. All rights reserved.\n> > > > +# All rights reserved.\n> > > > +#\n> > > > +# Redistribution and use in source and binary forms, with or without\n> > > > +# modification, are permitted provided that the following conditions\n> > > > +# are met:\n> > > > +#\n> > > > +# * Redistributions of source code must retain the above copyright\n> > > > +# notice, this list of conditions and the following disclaimer.\n> > > > +# * Redistributions in binary form must reproduce the above\n> copyright\n> > > > +# notice, this list of conditions and the following disclaimer in\n> > > > +# the documentation and/or other materials provided with the\n> > > > +# distribution.\n> > > > +# * Neither the name of Intel Corporation nor the names of its\n> > > > +# contributors may be used to endorse or promote products\n> derived\n> > > > +# from this software without specific prior written permission.\n> > > > +#\n> > > > +# THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND\n> > > > CONTRIBUTORS\n> > > > +# \"AS IS\" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING,\n> BUT\n> > > > NOT\n> > > > +# LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND\n> > > > FITNESS FOR\n> > > > +# A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE\n> > > > COPYRIGHT\n> > > > +# OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT,\n> > > > INCIDENTAL,\n> > > > +# SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING,\n> BUT\n> > > > NOT\n> > > > +# LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES;\n> LOSS\n> > > > OF USE,\n> > > > +# DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER\n> CAUSED\n> > > > AND ON ANY\n> > > > +# THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR\n> > > > TORT\n> > > > +# (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY\n> OUT OF\n> > > > THE USE\n> > > > +# OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH\n> > > > DAMAGE.\n> > > > +\n> > > > +ifeq ($(RTE_SDK),)\n> > > > +$(error \"Please define RTE_SDK environment variable\") endif\n> > > > +\n> > > > +# Default target, can be overriden by command line or environment\n> > > > +RTE_TARGET ?= x86_64-default-linuxapp-gcc\n> > >\n> > > This target is not present anymore. Change it to x86_64-native-linuxapp-\n> gcc.\n> > >\n> > > > +\n> > > > +include $(RTE_SDK)/mk/rte.vars.mk\n> > > > +\n> > > > +# binary name\n> > > > +APP = distributor_app\n> > > > +\n> > > > +# all source are stored in SRCS-y\n> > > > +SRCS-y := main.c\n> > > > +\n> > > > +CFLAGS += $(WERROR_FLAGS)\n> > > > +\n> > > > +# workaround for a gcc bug with noreturn attribute #\n> > > > +http://gcc.gnu.org/bugzilla/show_bug.cgi?id=12603\n> > > > +ifeq ($(CONFIG_RTE_TOOLCHAIN_GCC),y) CFLAGS_main.o +=\n> > > > +-Wno-return-type endif\n> > > > +\n> > > > +EXTRA_CFLAGS += -O3 -Wfatal-errors\n> > > > +\n> > > > +include $(RTE_SDK)/mk/rte.extapp.mk\n> > > > diff --git a/examples/distributor_app/main.c\n> > > > b/examples/distributor_app/main.c new file mode 100644 index\n> > > > 0000000..628810a\n> > > > --- /dev/null\n> > > > +++ b/examples/distributor_app/main.c\n> > > > @@ -0,0 +1,585 @@\n> > > > +/*-\n> > > > + * BSD LICENSE\n> > > > + *\n> > > > + * Copyright(c) 2010-2014 Intel Corporation. All rights reserved.\n> > > > + * All rights reserved.\n> > > > + *\n> > > > + * Redistribution and use in source and binary forms, with or without\n> > > > + * modification, are permitted provided that the following conditions\n> > > > + * are met:\n> > > > + *\n> > > > + * * Redistributions of source code must retain the above copyright\n> > > > + * notice, this list of conditions and the following disclaimer.\n> > > > + * * Redistributions in binary form must reproduce the above\n> copyright\n> > > > + * notice, this list of conditions and the following disclaimer in\n> > > > + * the documentation and/or other materials provided with the\n> > > > + * distribution.\n> > > > + * * Neither the name of Intel Corporation nor the names of its\n> > > > + * contributors may be used to endorse or promote products\n> derived\n> > > > + * from this software without specific prior written permission.\n> > > > + *\n> > > > + * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND\n> > > > CONTRIBUTORS\n> > > > + * \"AS IS\" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING,\n> BUT\n> > > > NOT\n> > > > + * LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND\n> > > > FITNESS FOR\n> > > > + * A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE\n> > > > COPYRIGHT\n> > > > + * OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT,\n> > > > INCIDENTAL,\n> > > > + * SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES\n> (INCLUDING, BUT\n> > > > NOT\n> > > > + * LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES;\n> LOSS\n> > > > OF USE,\n> > > > + * DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER\n> CAUSED\n> > > > AND ON ANY\n> > > > + * THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY,\n> OR\n> > > > TORT\n> > > > + * (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY\n> OUT OF\n> > > > THE USE\n> > > > + * OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH\n> > > > DAMAGE.\n> > > > + */\n> > > > +\n> > > > +#include <stdint.h>\n> > > > +#include <inttypes.h>\n> > > > +#include <unistd.h>\n> > > > +#include <signal.h>\n> > > > +#include <getopt.h>\n> > > > +\n> > > > +#include <rte_eal.h>\n> > > > +#include <rte_ethdev.h>\n> > > > +#include <rte_cycles.h>\n> > > > +#include <rte_malloc.h>\n> > > > +#include <rte_debug.h>\n> > > > +#include <rte_distributor.h>\n> > > > +\n> > > > +#include \"main.h\"\n> > > > +\n> > > > +#define RX_RING_SIZE 256\n> > > > +#define RX_FREE_THRESH 32\n> > > > +#define RX_PTHRESH 8\n> > > > +#define RX_HTHRESH 8\n> > > > +#define RX_WTHRESH 0\n> > > > +\n> > > > +#define TX_RING_SIZE 512\n> > > > +#define TX_FREE_THRESH 32\n> > > > +#define TX_PTHRESH 32\n> > > > +#define TX_HTHRESH 0\n> > > > +#define TX_WTHRESH 0\n> > > > +#define TX_RSBIT_THRESH 32\n> > > > +#define TX_Q_FLAGS (ETH_TXQ_FLAGS_NOMULTSEGS |\n> > > > ETH_TXQ_FLAGS_NOVLANOFFL |\\\n> > > > +\tETH_TXQ_FLAGS_NOXSUMSCTP | ETH_TXQ_FLAGS_NOXSUMUDP |\n> > > > \\\n> > > > +\tETH_TXQ_FLAGS_NOXSUMTCP)\n> > > > +\n> > > > +#define NUM_MBUFS ((64*1024)-1)\n> > > > +#define MBUF_SIZE (2048 + sizeof(struct rte_mbuf) +\n> > > > RTE_PKTMBUF_HEADROOM)\n> > > > +#define MBUF_CACHE_SIZE 250\n> > > > +#define BURST_SIZE 32\n> > > > +#define RTE_RING_SZ 1024\n> > > > +\n> > > > +/* mask of enabled ports */\n> > > > +static uint32_t enabled_port_mask = 0;\n> > > > +\n> > > > +static volatile struct app_stats {\n> > > > +\tstruct {\n> > > > +\t\tuint64_t rx_pkts;\n> > > > +\t\tuint64_t returned_pkts;\n> > > > +\t\tuint64_t enqueued_pkts;\n> > > > +\t} rx __rte_cache_aligned;\n> > > > +\n> > > > +\tstruct {\n> > > > +\t\tuint64_t dequeue_pkts;\n> > > > +\t\tuint64_t tx_pkts;\n> > > > +\t} tx __rte_cache_aligned;\n> > > > +} app_stats;\n> > > > +\n> > > > +static const struct rte_eth_conf port_conf_default = {\n> > > > +\t.rxmode = {\n> > > > +\t\t.mq_mode = ETH_MQ_RX_RSS,\n> > > > +\t\t.max_rx_pkt_len = ETHER_MAX_LEN,\n> > > > +\t\t.split_hdr_size = 0,\n> > > > +\t\t.header_split = 0, /**< Header Split disabled */\n> > > > +\t\t.hw_ip_checksum = 0, /**< IP checksum offload enabled */\n> > > > +\t\t.hw_vlan_filter = 0, /**< VLAN filtering disabled */\n> > > > +\t\t.jumbo_frame = 0, /**< Jumbo Frame Support disabled */\n> > > > +\t\t.hw_strip_crc = 0, /**< CRC stripped by hardware */\n> > > > +\t},\n> > > > +\t.txmode = {\n> > > > +\t\t.mq_mode = ETH_MQ_TX_NONE,\n> > > > +\t},\n> > > > +\t.lpbk_mode = 0,\n> > > > +\t.rx_adv_conf = {\n> > > > +\t\t\t.rss_conf = {\n> > > > +\t\t\t\t.rss_hf = ETH_RSS_IPV4 | ETH_RSS_IPV6 |\n> > > > +\t\t\t\t\tETH_RSS_IPV4_TCP |\n> > > > ETH_RSS_IPV4_UDP |\n> > > > +\t\t\t\t\tETH_RSS_IPV6_TCP |\n> > > > ETH_RSS_IPV6_UDP,\n> > > > +\t\t\t}\n> > > > +\t},\n> > > > +};\n> > > > +\n> > > > +static const struct rte_eth_rxconf rx_conf_default = {\n> > > > +\t.rx_thresh = {\n> > > > +\t\t.pthresh = RX_PTHRESH,\n> > > > +\t\t.hthresh = RX_HTHRESH,\n> > > > +\t\t.wthresh = RX_WTHRESH,\n> > > > +\t},\n> > > > +\t.rx_free_thresh = RX_FREE_THRESH,\n> > > > +\t.rx_drop_en = 0,\n> > > > +};\n> > > > +\n> > > > +static const struct rte_eth_txconf tx_conf_default = {\n> > > > +\t.tx_thresh = {\n> > > > +\t\t.pthresh = TX_PTHRESH,\n> > > > +\t\t.hthresh = TX_HTHRESH,\n> > > > +\t\t.wthresh = TX_WTHRESH,\n> > > > +\t},\n> > > > +\t.tx_free_thresh = TX_FREE_THRESH,\n> > > > +\t.tx_rs_thresh = TX_RSBIT_THRESH,\n> > > > +\t.txq_flags = TX_Q_FLAGS\n> > > > +\n> > > > +};\n> > > > +\n> > > > +struct output_buffer {\n> > > > +\tunsigned count;\n> > > > +\tstruct rte_mbuf *mbufs[BURST_SIZE]; };\n> > > > +\n> > > > +/*\n> > > > + * Initialises a given port using global settings and with the rx\n> > > > +buffers\n> > > > + * coming from the mbuf_pool passed as parameter */ static inline\n> > > > +int port_init(uint8_t port, struct rte_mempool *mbuf_pool) {\n> > > > +\tstruct rte_eth_conf port_conf = port_conf_default;\n> > > > +\tconst uint16_t rxRings = 1, txRings = rte_lcore_count() - 1;\n> > > > +\tint retval;\n> > > > +\tuint16_t q;\n> > > > +\n> > > > +\tif (port >= rte_eth_dev_count())\n> > > > +\t\treturn -1;\n> > > > +\n> > > > +\tretval = rte_eth_dev_configure(port, rxRings, txRings, &port_conf);\n> > > > +\tif (retval != 0)\n> > > > +\t\treturn retval;\n> > > > +\n> > > > +\tfor (q = 0; q < rxRings; q++) {\n> > > > +\t\tretval = rte_eth_rx_queue_setup(port, q, RX_RING_SIZE,\n> > > > +\n> > > > \trte_eth_dev_socket_id(port),\n> > > > +\t\t\t\t\t\t&rx_conf_default,\n> > > > mbuf_pool);\n> > > > +\t\tif (retval < 0)\n> > > > +\t\t\treturn retval;\n> > > > +\t}\n> > > > +\n> > > > +\tfor (q = 0; q < txRings; q++) {\n> > > > +\t\tretval = rte_eth_tx_queue_setup(port, q, TX_RING_SIZE,\n> > > > +\n> > > > \trte_eth_dev_socket_id(port),\n> > > > +\t\t\t\t\t\t&tx_conf_default);\n> > > > +\t\tif (retval < 0)\n> > > > +\t\t\treturn retval;\n> > > > +\t}\n> > > > +\n> > > > +\tretval = rte_eth_dev_start(port);\n> > > > +\tif (retval < 0)\n> > > > +\t\treturn retval;\n> > > > +\n> > > > +\tstruct rte_eth_link link;\n> > > > +\trte_eth_link_get_nowait(port, &link);\n> > > > +\tif (!link.link_status) {\n> > > > +\t\tsleep(1);\n> > > > +\t\trte_eth_link_get_nowait(port, &link);\n> > > > +\t}\n> > > > +\n> > > > +\tif (!link.link_status) {\n> > > > +\t\tprintf(\"Link down on port %\"PRIu8\"\\n\", port);\n> > > > +\t\treturn 0;\n> > > > +\t}\n> > > > +\n> > > > +\tstruct ether_addr addr;\n> > > > +\trte_eth_macaddr_get(port, &addr);\n> > > > +\tprintf(\"Port %u MAC: %02\"PRIx8\" %02\"PRIx8\" %02\"PRIx8\n> > > > +\t\t\t\" %02\"PRIx8\" %02\"PRIx8\" %02\"PRIx8\"\\n\",\n> > > > +\t\t\t(unsigned)port,\n> > > > +\t\t\taddr.addr_bytes[0], addr.addr_bytes[1],\n> > > > +\t\t\taddr.addr_bytes[2], addr.addr_bytes[3],\n> > > > +\t\t\taddr.addr_bytes[4], addr.addr_bytes[5]);\n> > > > +\n> > > > +\trte_eth_promiscuous_enable(port);\n> > > > +\n> > > > +\treturn 0;\n> > > > +}\n> > > > +\n> > > > +struct lcore_params {\n> > > > +\tunsigned worker_id;\n> > > > +\tstruct rte_distributor *d;\n> > > > +\tstruct rte_ring *r;\n> > > > +};\n> > > > +\n> > > > +static __attribute__((noreturn)) void lcore_rx(struct lcore_params\n> > > > +*p) {\n> > > > +\tstruct rte_distributor *d = p->d;\n> > > > +\tstruct rte_ring *r = p->r;\n> > > > +\tconst uint8_t nb_ports = rte_eth_dev_count();\n> > > > +\tconst int socket_id = rte_socket_id();\n> > > > +\tuint8_t port;\n> > > > +\n> > > > +\tfor (port = 0; port < nb_ports; port++) {\n> > > > +\t\t/* skip ports that are not enabled */\n> > > > +\t\tif ((enabled_port_mask & (1 << port)) == 0)\n> > > > +\t\t\tcontinue;\n> > > > +\n> > > > +\t\tif (rte_eth_dev_socket_id(port) > 0 &&\n> > > > +\t\t\t\trte_eth_dev_socket_id(port) != socket_id)\n> > > > +\t\t\tprintf(\"WARNING, port %u is on remote NUMA node\n> > > > to \"\n> > > > +\t\t\t\t\t\"RX thread.\\n\\tPerformance will not\n> > > > \"\n> > > > +\t\t\t\t\t\"be optimal.\\n\", port);\n> > > > +\t}\n> > > > +\n> > > > +\tprintf(\"\\nCore %u doing packet RX.\\n\", rte_lcore_id());\n> > > > +\tport = 0;\n> > > > +\tfor (;;) {\n> > > > +\t\t/* skip ports that are not enabled */\n> > > > +\t\tif ((enabled_port_mask & (1 << port)) == 0) {\n> > > > +\t\t\tif (++port == nb_ports)\n> > > > +\t\t\t\tport = 0;\n> > > > +\t\t\tcontinue;\n> > > > +\t\t}\n> > > > +\t\tstruct rte_mbuf *bufs[BURST_SIZE*2];\n> > > > +\t\tconst uint16_t nb_rx = rte_eth_rx_burst(port, 0, bufs,\n> > > > +\t\t\t\tBURST_SIZE);\n> > > > +\t\tapp_stats.rx.rx_pkts += nb_rx;\n> > > > +\n> > > > +\t\trte_distributor_process(d, bufs, nb_rx);\n> > > > +\t\tconst uint16_t nb_ret = rte_distributor_returned_pkts(d,\n> > > > +\t\t\t\tbufs, BURST_SIZE*2);\n> > > > +\t\tapp_stats.rx.returned_pkts += nb_ret;\n> > > > +\t\tif (unlikely(nb_ret == 0))\n> > > > +\t\t\tcontinue;\n> > > > +\n> > > > +\t\tuint16_t sent = rte_ring_enqueue_burst(r, (void *)bufs,\n> > > > nb_ret);\n> > > > +\t\tapp_stats.rx.enqueued_pkts += sent;\n> > > > +\t\tif (unlikely(sent < nb_ret)) {\n> > > > +\t\t\tprintf(\"Packet loss due to full ring\\n\");\n> > >\n> > > I would remove this printf. If core is too busy to process packets, this may\n> make performance worse.\n> > > Statistics should reflect this packet loss.\n> >\n> > >Probably use RTE_LOG(DEBUG, ...).\n> > >That's way you can switch it on/off at compile time.\n> >\n> > By default LOG_LEVEL is 8 in config/common_linuxapp, hence I see\n> RTE_LOG(DEBUG,...) always works.\n> > So until unless change the LOG-LEVEL to lesser value, RTE_LOG(DEBUG,..)\n> will be logged ..\n> > Please let me know if I can go ahead and remove the printf as of now until\n> the LOG_LEVEL is fixed. (or) I have to add some logic to log\n> > RTE_LOG(DEBUG,..) only once.\n> \n> Sorry, I probably wasn't clear enough.\n> What I am saying - create something like that inside your app code:\n> \n> #ifdef XXX_DEBUG\n> #define XXX_LOG(level, fmt, args...) \\\n> RTE_LOG(level, PMD, \"%s(): \" fmt \"\\n\", __func__, ## args)\n> #else\n> #define XXX_LOG(level, fmt, args...) do { } while(0)\n> #endif\n> \n> And replace your pritnf()s with it.\n\nI think we should make some changes in some logs in DPDK, and change the default log level to be INFO or less, so we could use the DEBUG level for things like this, but anyway, that would be another patch that I am thinking to submit some other day.\nAnyway, for this one, this option could work, but still that message may be shown hundreds of thousands of times per second, considering that CPU is overloaded. So, should we include something like a flag here, so message is shown just once?\nI know that this will only be used when DEBUG is enabled, but still that could flood the screen, as it is in an infinite loop.\n\n> \n> >\n> > >\n> > > > +\t\t\twhile (sent < nb_ret)\n> > > > +\t\t\t\trte_pktmbuf_free(bufs[sent++]);\n> > > > +\t\t}\n> > > > +\t\tif (++port == nb_ports)\n> > > > +\t\t\tport = 0;\n> > > > +\t}\n> > > > +}\n> > > > +\n> > > > +static inline void\n> > > > +flush_one_port(struct output_buffer *outbuf, uint8_t outp) {\n> > > > +\tunsigned nb_tx = rte_eth_tx_burst(outp, 0, outbuf->mbufs,\n> > > > +\t\t\toutbuf->count);\n> > > > +\tapp_stats.tx.tx_pkts += nb_tx;\n> > > > +\n> > > > +\tif (unlikely(nb_tx < outbuf->count)) {\n> > > > +\t\tprintf(\"Packet loss with tx_burst\\n\");\n> > >\n> > > Same here. This is fine for debugging, but statistics are updated as well,\n> so we do not need this printf.\n> > >\n> > > > +\t\tdo {\n> > > > +\t\t\trte_pktmbuf_free(outbuf->mbufs[nb_tx]);\n> > > > +\t\t} while (++nb_tx < outbuf->count);\n> > > > +\t}\n> > > > +\toutbuf->count = 0;\n> > > > +}\n> > > > +", "headers": { "Return-Path": "<dev-bounces@dpdk.org>", "X-Original-To": "patchwork@dpdk.org", "Delivered-To": "patchwork@dpdk.org", "Received": [ "from [92.243.14.124] (localhost [IPv6:::1])\n\tby dpdk.org (Postfix) with ESMTP id 438266893;\n\tMon, 29 Sep 2014 15:30:13 +0200 (CEST)", "from mga01.intel.com (mga01.intel.com [192.55.52.88])\n\tby dpdk.org (Postfix) with ESMTP id 4657D1F7\n\tfor <dev@dpdk.org>; Mon, 29 Sep 2014 15:30:10 +0200 (CEST)", "from azsmga001.ch.intel.com ([10.2.17.19])\n\tby fmsmga101.fm.intel.com with ESMTP; 29 Sep 2014 06:36:37 -0700", "from irsmsx102.ger.corp.intel.com ([163.33.3.155])\n\tby azsmga001.ch.intel.com with ESMTP; 29 Sep 2014 06:36:28 -0700", "from irsmsx108.ger.corp.intel.com ([169.254.11.21]) by\n\tIRSMSX102.ger.corp.intel.com ([169.254.2.200]) with mapi id\n\t14.03.0195.001; Mon, 29 Sep 2014 14:35:22 +0100" ], "X-ExtLoop1": "1", "X-IronPort-AV": "E=Sophos;i=\"5.04,620,1406617200\"; d=\"scan'208\";a=\"480737309\"", "From": "\"De Lara Guarch, Pablo\" <pablo.de.lara.guarch@intel.com>", "To": "\"Ananyev, Konstantin\" <konstantin.ananyev@intel.com>, \"Pattan, Reshma\"\n\t<reshma.pattan@intel.com>, \"dev@dpdk.org\" <dev@dpdk.org>", "Thread-Topic": "[dpdk-dev] [PATCH v2] distributor_app: new sample app", "Thread-Index": "AQHP2AJi1zXXgpMcD0+yx1ATRe0wtpwThu2Q///7zgCABIFTgIAAB6iAgAAVjlA=", "Date": "Mon, 29 Sep 2014 13:35:21 +0000", "Message-ID": "<E115CCD9D858EF4F90C690B0DCB4D8972262823A@IRSMSX108.ger.corp.intel.com>", "References": "<1410869607-16842-1-git-send-email-reshma.pattan@intel.com>\n\t<1411568210-2555-1-git-send-email-reshma.pattan@intel.com>\n\t<E115CCD9D858EF4F90C690B0DCB4D89722626933@IRSMSX108.ger.corp.intel.com>\n\t<2601191342CEEE43887BDE71AB9772582137D8B1@IRSMSX104.ger.corp.intel.com>\n\t<3AEA2BF9852C6F48A459DA490692831FE20598@IRSMSX109.ger.corp.intel.com>\n\t<2601191342CEEE43887BDE71AB97725821387654@IRSMSX104.ger.corp.intel.com>", "In-Reply-To": "<2601191342CEEE43887BDE71AB97725821387654@IRSMSX104.ger.corp.intel.com>", "Accept-Language": "en-US", "Content-Language": "en-US", "X-MS-Has-Attach": "", "X-MS-TNEF-Correlator": "", "x-originating-ip": "[163.33.239.180]", "Content-Type": "text/plain; charset=\"us-ascii\"", "Content-Transfer-Encoding": "quoted-printable", "MIME-Version": "1.0", "Subject": "Re: [dpdk-dev] [PATCH v2] distributor_app: new sample app", "X-BeenThere": "dev@dpdk.org", "X-Mailman-Version": "2.1.15", "Precedence": "list", "List-Id": "patches and discussions about DPDK <dev.dpdk.org>", "List-Unsubscribe": "<http://dpdk.org/ml/options/dev>,\n\t<mailto:dev-request@dpdk.org?subject=unsubscribe>", "List-Archive": "<http://dpdk.org/ml/archives/dev/>", "List-Post": "<mailto:dev@dpdk.org>", "List-Help": "<mailto:dev-request@dpdk.org?subject=help>", "List-Subscribe": "<http://dpdk.org/ml/listinfo/dev>,\n\t<mailto:dev-request@dpdk.org?subject=subscribe>", "Errors-To": "dev-bounces@dpdk.org", "Sender": "\"dev\" <dev-bounces@dpdk.org>" }, "addressed": null }, { "id": 1235, "web_url": "https://patches.dpdk.org/comment/1235/", "msgid": "<20140929143510.GD26483@hmsreliant.think-freely.org>", "list_archive_url": "https://inbox.dpdk.org/dev/20140929143510.GD26483@hmsreliant.think-freely.org", "date": "2014-09-29T14:35:10", "subject": "Re: [dpdk-dev] [PATCH v2] distributor_app: new sample app", "submitter": { "id": 32, "url": "https://patches.dpdk.org/api/people/32/?format=api", "name": "Neil Horman", "email": "nhorman@tuxdriver.com" }, "content": "On Mon, Sep 29, 2014 at 01:35:21PM +0000, De Lara Guarch, Pablo wrote:\n> \n> \n> > -----Original Message-----\n> > From: Ananyev, Konstantin\n> > Sent: Monday, September 29, 2014 2:07 PM\n> > To: Pattan, Reshma; De Lara Guarch, Pablo; dev@dpdk.org\n> > Subject: RE: [dpdk-dev] [PATCH v2] distributor_app: new sample app\n> > \n> > \n> > \n> > > -----Original Message-----\n> > > From: Pattan, Reshma\n> > > Sent: Monday, September 29, 2014 1:40 PM\n> > > To: Ananyev, Konstantin; De Lara Guarch, Pablo; dev@dpdk.org\n> > > Subject: RE: [dpdk-dev] [PATCH v2] distributor_app: new sample app\n> > >\n> > >\n> > >\n> > > -----Original Message-----\n> > > From: Ananyev, Konstantin\n> > > Sent: Friday, September 26, 2014 4:52 PM\n> > > To: De Lara Guarch, Pablo; Pattan, Reshma; dev@dpdk.org\n> > > Subject: RE: [dpdk-dev] [PATCH v2] distributor_app: new sample app\n> > >\n> > >\n> > >\n> > > > -----Original Message-----\n> > > > From: dev [mailto:dev-bounces@dpdk.org] On Behalf Of De Lara Guarch,\n> > > > Pablo\n> > > > Sent: Friday, September 26, 2014 4:12 PM\n> > > > To: Pattan, Reshma; dev@dpdk.org\n> > > > Subject: Re: [dpdk-dev] [PATCH v2] distributor_app: new sample app\n> > > >\n> > > > Hi,\n> > > >\n> > > > > -----Original Message-----\n> > > > > From: dev [mailto:dev-bounces@dpdk.org] On Behalf Of reshmapa\n> > > > > Sent: Wednesday, September 24, 2014 3:17 PM\n> > > > > To: dev@dpdk.org\n> > > > > Subject: [dpdk-dev] [PATCH v2] distributor_app: new sample app\n> > > > >\n> > > > > From: Reshma Pattan <reshma.pattan@intel.com>\n> > > > >\n> > > > > A new sample app that shows the usage of the distributor library.\n> > > > > This app works as follows:\n> > > > >\n> > > > > * An RX thread runs which pulls packets from each ethernet port in turn\n> > > > > and passes those packets to worker using a distributor component.\n> > > > > * The workers take the packets in turn, and determine the output port\n> > > > > for those packets using basic l2forwarding doing an xor on the source\n> > > > > port id.\n> > > > > * The RX thread takes the returned packets from the workers and\n> > enqueue\n> > > > > those packets into an rte_ring structure.\n> > > > > * A TX thread pulls the packets off the rte_ring structure and then\n> > > > > sends each packet out the output port specified previously by the\n> > > > > worker\n> > > > > * Command-line option support provided only for portmask.\n> > > > >\n> > > > > Signed-off-by: Bruce Richardson <bruce.richardson@intel.com>\n> > > > > Signed-off-by: Reshma Pattan <reshma.pattan@intel.com>\n> > > > > ---\n> > > > > examples/Makefile | 1 +\n> > > > > examples/distributor_app/Makefile | 57 ++++\n> > > > > examples/distributor_app/main.c | 585\n> > > > > ++++++++++++++++++++++++++++++++++++++\n> > > > > examples/distributor_app/main.h | 46 +++\n> > > > > 4 files changed, 689 insertions(+)\n> > > > > create mode 100644 examples/distributor_app/Makefile create mode\n> > > > > 100644 examples/distributor_app/main.c create mode 100644\n> > > > > examples/distributor_app/main.h\n> > > > >\n> > > > > diff --git a/examples/Makefile b/examples/Makefile index\n> > > > > 6245f83..2ba82b0 100644\n> > > > > --- a/examples/Makefile\n> > > > > +++ b/examples/Makefile\n> > > > > @@ -66,5 +66,6 @@ DIRS-y += vhost\n> > > > > DIRS-$(CONFIG_RTE_LIBRTE_XEN_DOM0) += vhost_xen DIRS-y +=\n> > vmdq\n> > > > > DIRS-y += vmdq_dcb\n> > > > > +DIRS-$(CONFIG_RTE_LIBRTE_DISTRIBUTOR) += distributor_app\n> > > > >\n> > > > > include $(RTE_SDK)/mk/rte.extsubdir.mk diff --git\n> > > > > a/examples/distributor_app/Makefile\n> > > > > b/examples/distributor_app/Makefile\n> > > > > new file mode 100644\n> > > > > index 0000000..394785d\n> > > > > --- /dev/null\n> > > > > +++ b/examples/distributor_app/Makefile\n> > > > > @@ -0,0 +1,57 @@\n> > > > > +# BSD LICENSE\n> > > > > +#\n> > > > > +# Copyright(c) 2010-2014 Intel Corporation. All rights reserved.\n> > > > > +# All rights reserved.\n> > > > > +#\n> > > > > +# Redistribution and use in source and binary forms, with or without\n> > > > > +# modification, are permitted provided that the following conditions\n> > > > > +# are met:\n> > > > > +#\n> > > > > +# * Redistributions of source code must retain the above copyright\n> > > > > +# notice, this list of conditions and the following disclaimer.\n> > > > > +# * Redistributions in binary form must reproduce the above\n> > copyright\n> > > > > +# notice, this list of conditions and the following disclaimer in\n> > > > > +# the documentation and/or other materials provided with the\n> > > > > +# distribution.\n> > > > > +# * Neither the name of Intel Corporation nor the names of its\n> > > > > +# contributors may be used to endorse or promote products\n> > derived\n> > > > > +# from this software without specific prior written permission.\n> > > > > +#\n> > > > > +# THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND\n> > > > > CONTRIBUTORS\n> > > > > +# \"AS IS\" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING,\n> > BUT\n> > > > > NOT\n> > > > > +# LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND\n> > > > > FITNESS FOR\n> > > > > +# A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE\n> > > > > COPYRIGHT\n> > > > > +# OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT,\n> > > > > INCIDENTAL,\n> > > > > +# SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING,\n> > BUT\n> > > > > NOT\n> > > > > +# LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES;\n> > LOSS\n> > > > > OF USE,\n> > > > > +# DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER\n> > CAUSED\n> > > > > AND ON ANY\n> > > > > +# THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR\n> > > > > TORT\n> > > > > +# (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY\n> > OUT OF\n> > > > > THE USE\n> > > > > +# OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH\n> > > > > DAMAGE.\n> > > > > +\n> > > > > +ifeq ($(RTE_SDK),)\n> > > > > +$(error \"Please define RTE_SDK environment variable\") endif\n> > > > > +\n> > > > > +# Default target, can be overriden by command line or environment\n> > > > > +RTE_TARGET ?= x86_64-default-linuxapp-gcc\n> > > >\n> > > > This target is not present anymore. Change it to x86_64-native-linuxapp-\n> > gcc.\n> > > >\n> > > > > +\n> > > > > +include $(RTE_SDK)/mk/rte.vars.mk\n> > > > > +\n> > > > > +# binary name\n> > > > > +APP = distributor_app\n> > > > > +\n> > > > > +# all source are stored in SRCS-y\n> > > > > +SRCS-y := main.c\n> > > > > +\n> > > > > +CFLAGS += $(WERROR_FLAGS)\n> > > > > +\n> > > > > +# workaround for a gcc bug with noreturn attribute #\n> > > > > +http://gcc.gnu.org/bugzilla/show_bug.cgi?id=12603\n> > > > > +ifeq ($(CONFIG_RTE_TOOLCHAIN_GCC),y) CFLAGS_main.o +=\n> > > > > +-Wno-return-type endif\n> > > > > +\n> > > > > +EXTRA_CFLAGS += -O3 -Wfatal-errors\n> > > > > +\n> > > > > +include $(RTE_SDK)/mk/rte.extapp.mk\n> > > > > diff --git a/examples/distributor_app/main.c\n> > > > > b/examples/distributor_app/main.c new file mode 100644 index\n> > > > > 0000000..628810a\n> > > > > --- /dev/null\n> > > > > +++ b/examples/distributor_app/main.c\n> > > > > @@ -0,0 +1,585 @@\n> > > > > +/*-\n> > > > > + * BSD LICENSE\n> > > > > + *\n> > > > > + * Copyright(c) 2010-2014 Intel Corporation. All rights reserved.\n> > > > > + * All rights reserved.\n> > > > > + *\n> > > > > + * Redistribution and use in source and binary forms, with or without\n> > > > > + * modification, are permitted provided that the following conditions\n> > > > > + * are met:\n> > > > > + *\n> > > > > + * * Redistributions of source code must retain the above copyright\n> > > > > + * notice, this list of conditions and the following disclaimer.\n> > > > > + * * Redistributions in binary form must reproduce the above\n> > copyright\n> > > > > + * notice, this list of conditions and the following disclaimer in\n> > > > > + * the documentation and/or other materials provided with the\n> > > > > + * distribution.\n> > > > > + * * Neither the name of Intel Corporation nor the names of its\n> > > > > + * contributors may be used to endorse or promote products\n> > derived\n> > > > > + * from this software without specific prior written permission.\n> > > > > + *\n> > > > > + * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND\n> > > > > CONTRIBUTORS\n> > > > > + * \"AS IS\" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING,\n> > BUT\n> > > > > NOT\n> > > > > + * LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND\n> > > > > FITNESS FOR\n> > > > > + * A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE\n> > > > > COPYRIGHT\n> > > > > + * OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT,\n> > > > > INCIDENTAL,\n> > > > > + * SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES\n> > (INCLUDING, BUT\n> > > > > NOT\n> > > > > + * LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES;\n> > LOSS\n> > > > > OF USE,\n> > > > > + * DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER\n> > CAUSED\n> > > > > AND ON ANY\n> > > > > + * THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY,\n> > OR\n> > > > > TORT\n> > > > > + * (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY\n> > OUT OF\n> > > > > THE USE\n> > > > > + * OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH\n> > > > > DAMAGE.\n> > > > > + */\n> > > > > +\n> > > > > +#include <stdint.h>\n> > > > > +#include <inttypes.h>\n> > > > > +#include <unistd.h>\n> > > > > +#include <signal.h>\n> > > > > +#include <getopt.h>\n> > > > > +\n> > > > > +#include <rte_eal.h>\n> > > > > +#include <rte_ethdev.h>\n> > > > > +#include <rte_cycles.h>\n> > > > > +#include <rte_malloc.h>\n> > > > > +#include <rte_debug.h>\n> > > > > +#include <rte_distributor.h>\n> > > > > +\n> > > > > +#include \"main.h\"\n> > > > > +\n> > > > > +#define RX_RING_SIZE 256\n> > > > > +#define RX_FREE_THRESH 32\n> > > > > +#define RX_PTHRESH 8\n> > > > > +#define RX_HTHRESH 8\n> > > > > +#define RX_WTHRESH 0\n> > > > > +\n> > > > > +#define TX_RING_SIZE 512\n> > > > > +#define TX_FREE_THRESH 32\n> > > > > +#define TX_PTHRESH 32\n> > > > > +#define TX_HTHRESH 0\n> > > > > +#define TX_WTHRESH 0\n> > > > > +#define TX_RSBIT_THRESH 32\n> > > > > +#define TX_Q_FLAGS (ETH_TXQ_FLAGS_NOMULTSEGS |\n> > > > > ETH_TXQ_FLAGS_NOVLANOFFL |\\\n> > > > > +\tETH_TXQ_FLAGS_NOXSUMSCTP | ETH_TXQ_FLAGS_NOXSUMUDP |\n> > > > > \\\n> > > > > +\tETH_TXQ_FLAGS_NOXSUMTCP)\n> > > > > +\n> > > > > +#define NUM_MBUFS ((64*1024)-1)\n> > > > > +#define MBUF_SIZE (2048 + sizeof(struct rte_mbuf) +\n> > > > > RTE_PKTMBUF_HEADROOM)\n> > > > > +#define MBUF_CACHE_SIZE 250\n> > > > > +#define BURST_SIZE 32\n> > > > > +#define RTE_RING_SZ 1024\n> > > > > +\n> > > > > +/* mask of enabled ports */\n> > > > > +static uint32_t enabled_port_mask = 0;\n> > > > > +\n> > > > > +static volatile struct app_stats {\n> > > > > +\tstruct {\n> > > > > +\t\tuint64_t rx_pkts;\n> > > > > +\t\tuint64_t returned_pkts;\n> > > > > +\t\tuint64_t enqueued_pkts;\n> > > > > +\t} rx __rte_cache_aligned;\n> > > > > +\n> > > > > +\tstruct {\n> > > > > +\t\tuint64_t dequeue_pkts;\n> > > > > +\t\tuint64_t tx_pkts;\n> > > > > +\t} tx __rte_cache_aligned;\n> > > > > +} app_stats;\n> > > > > +\n> > > > > +static const struct rte_eth_conf port_conf_default = {\n> > > > > +\t.rxmode = {\n> > > > > +\t\t.mq_mode = ETH_MQ_RX_RSS,\n> > > > > +\t\t.max_rx_pkt_len = ETHER_MAX_LEN,\n> > > > > +\t\t.split_hdr_size = 0,\n> > > > > +\t\t.header_split = 0, /**< Header Split disabled */\n> > > > > +\t\t.hw_ip_checksum = 0, /**< IP checksum offload enabled */\n> > > > > +\t\t.hw_vlan_filter = 0, /**< VLAN filtering disabled */\n> > > > > +\t\t.jumbo_frame = 0, /**< Jumbo Frame Support disabled */\n> > > > > +\t\t.hw_strip_crc = 0, /**< CRC stripped by hardware */\n> > > > > +\t},\n> > > > > +\t.txmode = {\n> > > > > +\t\t.mq_mode = ETH_MQ_TX_NONE,\n> > > > > +\t},\n> > > > > +\t.lpbk_mode = 0,\n> > > > > +\t.rx_adv_conf = {\n> > > > > +\t\t\t.rss_conf = {\n> > > > > +\t\t\t\t.rss_hf = ETH_RSS_IPV4 | ETH_RSS_IPV6 |\n> > > > > +\t\t\t\t\tETH_RSS_IPV4_TCP |\n> > > > > ETH_RSS_IPV4_UDP |\n> > > > > +\t\t\t\t\tETH_RSS_IPV6_TCP |\n> > > > > ETH_RSS_IPV6_UDP,\n> > > > > +\t\t\t}\n> > > > > +\t},\n> > > > > +};\n> > > > > +\n> > > > > +static const struct rte_eth_rxconf rx_conf_default = {\n> > > > > +\t.rx_thresh = {\n> > > > > +\t\t.pthresh = RX_PTHRESH,\n> > > > > +\t\t.hthresh = RX_HTHRESH,\n> > > > > +\t\t.wthresh = RX_WTHRESH,\n> > > > > +\t},\n> > > > > +\t.rx_free_thresh = RX_FREE_THRESH,\n> > > > > +\t.rx_drop_en = 0,\n> > > > > +};\n> > > > > +\n> > > > > +static const struct rte_eth_txconf tx_conf_default = {\n> > > > > +\t.tx_thresh = {\n> > > > > +\t\t.pthresh = TX_PTHRESH,\n> > > > > +\t\t.hthresh = TX_HTHRESH,\n> > > > > +\t\t.wthresh = TX_WTHRESH,\n> > > > > +\t},\n> > > > > +\t.tx_free_thresh = TX_FREE_THRESH,\n> > > > > +\t.tx_rs_thresh = TX_RSBIT_THRESH,\n> > > > > +\t.txq_flags = TX_Q_FLAGS\n> > > > > +\n> > > > > +};\n> > > > > +\n> > > > > +struct output_buffer {\n> > > > > +\tunsigned count;\n> > > > > +\tstruct rte_mbuf *mbufs[BURST_SIZE]; };\n> > > > > +\n> > > > > +/*\n> > > > > + * Initialises a given port using global settings and with the rx\n> > > > > +buffers\n> > > > > + * coming from the mbuf_pool passed as parameter */ static inline\n> > > > > +int port_init(uint8_t port, struct rte_mempool *mbuf_pool) {\n> > > > > +\tstruct rte_eth_conf port_conf = port_conf_default;\n> > > > > +\tconst uint16_t rxRings = 1, txRings = rte_lcore_count() - 1;\n> > > > > +\tint retval;\n> > > > > +\tuint16_t q;\n> > > > > +\n> > > > > +\tif (port >= rte_eth_dev_count())\n> > > > > +\t\treturn -1;\n> > > > > +\n> > > > > +\tretval = rte_eth_dev_configure(port, rxRings, txRings, &port_conf);\n> > > > > +\tif (retval != 0)\n> > > > > +\t\treturn retval;\n> > > > > +\n> > > > > +\tfor (q = 0; q < rxRings; q++) {\n> > > > > +\t\tretval = rte_eth_rx_queue_setup(port, q, RX_RING_SIZE,\n> > > > > +\n> > > > > \trte_eth_dev_socket_id(port),\n> > > > > +\t\t\t\t\t\t&rx_conf_default,\n> > > > > mbuf_pool);\n> > > > > +\t\tif (retval < 0)\n> > > > > +\t\t\treturn retval;\n> > > > > +\t}\n> > > > > +\n> > > > > +\tfor (q = 0; q < txRings; q++) {\n> > > > > +\t\tretval = rte_eth_tx_queue_setup(port, q, TX_RING_SIZE,\n> > > > > +\n> > > > > \trte_eth_dev_socket_id(port),\n> > > > > +\t\t\t\t\t\t&tx_conf_default);\n> > > > > +\t\tif (retval < 0)\n> > > > > +\t\t\treturn retval;\n> > > > > +\t}\n> > > > > +\n> > > > > +\tretval = rte_eth_dev_start(port);\n> > > > > +\tif (retval < 0)\n> > > > > +\t\treturn retval;\n> > > > > +\n> > > > > +\tstruct rte_eth_link link;\n> > > > > +\trte_eth_link_get_nowait(port, &link);\n> > > > > +\tif (!link.link_status) {\n> > > > > +\t\tsleep(1);\n> > > > > +\t\trte_eth_link_get_nowait(port, &link);\n> > > > > +\t}\n> > > > > +\n> > > > > +\tif (!link.link_status) {\n> > > > > +\t\tprintf(\"Link down on port %\"PRIu8\"\\n\", port);\n> > > > > +\t\treturn 0;\n> > > > > +\t}\n> > > > > +\n> > > > > +\tstruct ether_addr addr;\n> > > > > +\trte_eth_macaddr_get(port, &addr);\n> > > > > +\tprintf(\"Port %u MAC: %02\"PRIx8\" %02\"PRIx8\" %02\"PRIx8\n> > > > > +\t\t\t\" %02\"PRIx8\" %02\"PRIx8\" %02\"PRIx8\"\\n\",\n> > > > > +\t\t\t(unsigned)port,\n> > > > > +\t\t\taddr.addr_bytes[0], addr.addr_bytes[1],\n> > > > > +\t\t\taddr.addr_bytes[2], addr.addr_bytes[3],\n> > > > > +\t\t\taddr.addr_bytes[4], addr.addr_bytes[5]);\n> > > > > +\n> > > > > +\trte_eth_promiscuous_enable(port);\n> > > > > +\n> > > > > +\treturn 0;\n> > > > > +}\n> > > > > +\n> > > > > +struct lcore_params {\n> > > > > +\tunsigned worker_id;\n> > > > > +\tstruct rte_distributor *d;\n> > > > > +\tstruct rte_ring *r;\n> > > > > +};\n> > > > > +\n> > > > > +static __attribute__((noreturn)) void lcore_rx(struct lcore_params\n> > > > > +*p) {\n> > > > > +\tstruct rte_distributor *d = p->d;\n> > > > > +\tstruct rte_ring *r = p->r;\n> > > > > +\tconst uint8_t nb_ports = rte_eth_dev_count();\n> > > > > +\tconst int socket_id = rte_socket_id();\n> > > > > +\tuint8_t port;\n> > > > > +\n> > > > > +\tfor (port = 0; port < nb_ports; port++) {\n> > > > > +\t\t/* skip ports that are not enabled */\n> > > > > +\t\tif ((enabled_port_mask & (1 << port)) == 0)\n> > > > > +\t\t\tcontinue;\n> > > > > +\n> > > > > +\t\tif (rte_eth_dev_socket_id(port) > 0 &&\n> > > > > +\t\t\t\trte_eth_dev_socket_id(port) != socket_id)\n> > > > > +\t\t\tprintf(\"WARNING, port %u is on remote NUMA node\n> > > > > to \"\n> > > > > +\t\t\t\t\t\"RX thread.\\n\\tPerformance will not\n> > > > > \"\n> > > > > +\t\t\t\t\t\"be optimal.\\n\", port);\n> > > > > +\t}\n> > > > > +\n> > > > > +\tprintf(\"\\nCore %u doing packet RX.\\n\", rte_lcore_id());\n> > > > > +\tport = 0;\n> > > > > +\tfor (;;) {\n> > > > > +\t\t/* skip ports that are not enabled */\n> > > > > +\t\tif ((enabled_port_mask & (1 << port)) == 0) {\n> > > > > +\t\t\tif (++port == nb_ports)\n> > > > > +\t\t\t\tport = 0;\n> > > > > +\t\t\tcontinue;\n> > > > > +\t\t}\n> > > > > +\t\tstruct rte_mbuf *bufs[BURST_SIZE*2];\n> > > > > +\t\tconst uint16_t nb_rx = rte_eth_rx_burst(port, 0, bufs,\n> > > > > +\t\t\t\tBURST_SIZE);\n> > > > > +\t\tapp_stats.rx.rx_pkts += nb_rx;\n> > > > > +\n> > > > > +\t\trte_distributor_process(d, bufs, nb_rx);\n> > > > > +\t\tconst uint16_t nb_ret = rte_distributor_returned_pkts(d,\n> > > > > +\t\t\t\tbufs, BURST_SIZE*2);\n> > > > > +\t\tapp_stats.rx.returned_pkts += nb_ret;\n> > > > > +\t\tif (unlikely(nb_ret == 0))\n> > > > > +\t\t\tcontinue;\n> > > > > +\n> > > > > +\t\tuint16_t sent = rte_ring_enqueue_burst(r, (void *)bufs,\n> > > > > nb_ret);\n> > > > > +\t\tapp_stats.rx.enqueued_pkts += sent;\n> > > > > +\t\tif (unlikely(sent < nb_ret)) {\n> > > > > +\t\t\tprintf(\"Packet loss due to full ring\\n\");\n> > > >\n> > > > I would remove this printf. If core is too busy to process packets, this may\n> > make performance worse.\n> > > > Statistics should reflect this packet loss.\n> > >\n> > > >Probably use RTE_LOG(DEBUG, ...).\n> > > >That's way you can switch it on/off at compile time.\n> > >\n> > > By default LOG_LEVEL is 8 in config/common_linuxapp, hence I see\n> > RTE_LOG(DEBUG,...) always works.\n> > > So until unless change the LOG-LEVEL to lesser value, RTE_LOG(DEBUG,..)\n> > will be logged ..\n> > > Please let me know if I can go ahead and remove the printf as of now until\n> > the LOG_LEVEL is fixed. (or) I have to add some logic to log\n> > > RTE_LOG(DEBUG,..) only once.\n> > \n> > Sorry, I probably wasn't clear enough.\n> > What I am saying - create something like that inside your app code:\n> > \n> > #ifdef XXX_DEBUG\n> > #define XXX_LOG(level, fmt, args...) \\\n> > RTE_LOG(level, PMD, \"%s(): \" fmt \"\\n\", __func__, ## args)\n> > #else\n> > #define XXX_LOG(level, fmt, args...) do { } while(0)\n> > #endif\n> > \n> > And replace your pritnf()s with it.\n> \n> I think we should make some changes in some logs in DPDK, and change the default log level to be INFO or less, so we could use the DEBUG level for things like this, but anyway, that would be another patch that I am thinking to submit some other day.\n> Anyway, for this one, this option could work, but still that message may be shown hundreds of thousands of times per second, considering that CPU is overloaded. So, should we include something like a flag here, so message is shown just once?\n> I know that this will only be used when DEBUG is enabled, but still that could flood the screen, as it is in an infinite loop.\nMay you need a ratelimiting function?\nNeil\n\n> \n> > \n> > >\n> > > >\n> > > > > +\t\t\twhile (sent < nb_ret)\n> > > > > +\t\t\t\trte_pktmbuf_free(bufs[sent++]);\n> > > > > +\t\t}\n> > > > > +\t\tif (++port == nb_ports)\n> > > > > +\t\t\tport = 0;\n> > > > > +\t}\n> > > > > +}\n> > > > > +\n> > > > > +static inline void\n> > > > > +flush_one_port(struct output_buffer *outbuf, uint8_t outp) {\n> > > > > +\tunsigned nb_tx = rte_eth_tx_burst(outp, 0, outbuf->mbufs,\n> > > > > +\t\t\toutbuf->count);\n> > > > > +\tapp_stats.tx.tx_pkts += nb_tx;\n> > > > > +\n> > > > > +\tif (unlikely(nb_tx < outbuf->count)) {\n> > > > > +\t\tprintf(\"Packet loss with tx_burst\\n\");\n> > > >\n> > > > Same here. This is fine for debugging, but statistics are updated as well,\n> > so we do not need this printf.\n> > > >\n> > > > > +\t\tdo {\n> > > > > +\t\t\trte_pktmbuf_free(outbuf->mbufs[nb_tx]);\n> > > > > +\t\t} while (++nb_tx < outbuf->count);\n> > > > > +\t}\n> > > > > +\toutbuf->count = 0;\n> > > > > +}\n> > > > > +\n> \n>", "headers": { "Return-Path": "<dev-bounces@dpdk.org>", "X-Original-To": "patchwork@dpdk.org", "Delivered-To": "patchwork@dpdk.org", "Received": [ "from [92.243.14.124] (localhost [IPv6:::1])\n\tby dpdk.org (Postfix) with ESMTP id B89006AA9;\n\tMon, 29 Sep 2014 16:28:48 +0200 (CEST)", "from smtp.tuxdriver.com (charlotte.tuxdriver.com [70.61.120.58])\n\tby dpdk.org (Postfix) with ESMTP id 37E176A99\n\tfor <dev@dpdk.org>; Mon, 29 Sep 2014 16:28:44 +0200 (CEST)", "from hmsreliant.think-freely.org\n\t([2001:470:8:a08:7aac:c0ff:fec2:933b] helo=localhost)\n\tby smtp.tuxdriver.com with esmtpsa (TLSv1:AES128-SHA:128) (Exim 4.63)\n\t(envelope-from <nhorman@tuxdriver.com>)\n\tid 1XYc2h-0004Bn-26; Mon, 29 Sep 2014 10:35:19 -0400" ], "Date": "Mon, 29 Sep 2014 10:35:10 -0400", "From": "Neil Horman <nhorman@tuxdriver.com>", "To": "\"De Lara Guarch, Pablo\" <pablo.de.lara.guarch@intel.com>", "Message-ID": "<20140929143510.GD26483@hmsreliant.think-freely.org>", "References": "<1410869607-16842-1-git-send-email-reshma.pattan@intel.com>\n\t<1411568210-2555-1-git-send-email-reshma.pattan@intel.com>\n\t<E115CCD9D858EF4F90C690B0DCB4D89722626933@IRSMSX108.ger.corp.intel.com>\n\t<2601191342CEEE43887BDE71AB9772582137D8B1@IRSMSX104.ger.corp.intel.com>\n\t<3AEA2BF9852C6F48A459DA490692831FE20598@IRSMSX109.ger.corp.intel.com>\n\t<2601191342CEEE43887BDE71AB97725821387654@IRSMSX104.ger.corp.intel.com>\n\t<E115CCD9D858EF4F90C690B0DCB4D8972262823A@IRSMSX108.ger.corp.intel.com>", "MIME-Version": "1.0", "Content-Type": "text/plain; charset=us-ascii", "Content-Disposition": "inline", "In-Reply-To": "<E115CCD9D858EF4F90C690B0DCB4D8972262823A@IRSMSX108.ger.corp.intel.com>", "User-Agent": "Mutt/1.5.23 (2014-03-12)", "X-Spam-Score": "-2.9 (--)", "X-Spam-Status": "No", "Cc": "\"dev@dpdk.org\" <dev@dpdk.org>", "Subject": "Re: [dpdk-dev] [PATCH v2] distributor_app: new sample app", "X-BeenThere": "dev@dpdk.org", "X-Mailman-Version": "2.1.15", "Precedence": "list", "List-Id": "patches and discussions about DPDK <dev.dpdk.org>", "List-Unsubscribe": "<http://dpdk.org/ml/options/dev>,\n\t<mailto:dev-request@dpdk.org?subject=unsubscribe>", "List-Archive": "<http://dpdk.org/ml/archives/dev/>", "List-Post": "<mailto:dev@dpdk.org>", "List-Help": "<mailto:dev-request@dpdk.org?subject=help>", "List-Subscribe": "<http://dpdk.org/ml/listinfo/dev>,\n\t<mailto:dev-request@dpdk.org?subject=subscribe>", "Errors-To": "dev-bounces@dpdk.org", "Sender": "\"dev\" <dev-bounces@dpdk.org>" }, "addressed": null }, { "id": 1278, "web_url": "https://patches.dpdk.org/comment/1278/", "msgid": "<3AEA2BF9852C6F48A459DA490692831FE208B3@IRSMSX109.ger.corp.intel.com>", "list_archive_url": "https://inbox.dpdk.org/dev/3AEA2BF9852C6F48A459DA490692831FE208B3@IRSMSX109.ger.corp.intel.com", "date": "2014-09-30T08:02:58", "subject": "Re: [dpdk-dev] [PATCH v2] distributor_app: new sample app", "submitter": { "id": 70, "url": "https://patches.dpdk.org/api/people/70/?format=api", "name": "Pattan, Reshma", "email": "reshma.pattan@intel.com" }, "content": "Hi Konstantin,\n\nAny comments on below Pablos comment? If so please provide.\n\nThanks,\nReshma\n\n\n\n-----Original Message-----\nFrom: De Lara Guarch, Pablo \nSent: Monday, September 29, 2014 2:35 PM\nTo: Ananyev, Konstantin; Pattan, Reshma; dev@dpdk.org\nSubject: RE: [dpdk-dev] [PATCH v2] distributor_app: new sample app\n\n\n\n> -----Original Message-----\n> From: Ananyev, Konstantin\n> Sent: Monday, September 29, 2014 2:07 PM\n> To: Pattan, Reshma; De Lara Guarch, Pablo; dev@dpdk.org\n> Subject: RE: [dpdk-dev] [PATCH v2] distributor_app: new sample app\n> \n> \n> \n> > -----Original Message-----\n> > From: Pattan, Reshma\n> > Sent: Monday, September 29, 2014 1:40 PM\n> > To: Ananyev, Konstantin; De Lara Guarch, Pablo; dev@dpdk.org\n> > Subject: RE: [dpdk-dev] [PATCH v2] distributor_app: new sample app\n> >\n> >\n> >\n> > -----Original Message-----\n> > From: Ananyev, Konstantin\n> > Sent: Friday, September 26, 2014 4:52 PM\n> > To: De Lara Guarch, Pablo; Pattan, Reshma; dev@dpdk.org\n> > Subject: RE: [dpdk-dev] [PATCH v2] distributor_app: new sample app\n> >\n> >\n> >\n> > > -----Original Message-----\n> > > From: dev [mailto:dev-bounces@dpdk.org] On Behalf Of De Lara \n> > > Guarch, Pablo\n> > > Sent: Friday, September 26, 2014 4:12 PM\n> > > To: Pattan, Reshma; dev@dpdk.org\n> > > Subject: Re: [dpdk-dev] [PATCH v2] distributor_app: new sample app\n> > >\n> > > Hi,\n> > >\n> > > > -----Original Message-----\n> > > > From: dev [mailto:dev-bounces@dpdk.org] On Behalf Of reshmapa\n> > > > Sent: Wednesday, September 24, 2014 3:17 PM\n> > > > To: dev@dpdk.org\n> > > > Subject: [dpdk-dev] [PATCH v2] distributor_app: new sample app\n> > > >\n> > > > From: Reshma Pattan <reshma.pattan@intel.com>\n> > > >\n> > > > A new sample app that shows the usage of the distributor library.\n> > > > This app works as follows:\n> > > >\n> > > > * An RX thread runs which pulls packets from each ethernet port in turn\n> > > > and passes those packets to worker using a distributor component.\n> > > > * The workers take the packets in turn, and determine the output port\n> > > > for those packets using basic l2forwarding doing an xor on the source\n> > > > port id.\n> > > > * The RX thread takes the returned packets from the workers and\n> enqueue\n> > > > those packets into an rte_ring structure.\n> > > > * A TX thread pulls the packets off the rte_ring structure and then\n> > > > sends each packet out the output port specified previously by \n> > > > the worker\n> > > > * Command-line option support provided only for portmask.\n> > > >\n> > > > Signed-off-by: Bruce Richardson <bruce.richardson@intel.com>\n> > > > Signed-off-by: Reshma Pattan <reshma.pattan@intel.com>\n> > > > ---\n> > > > examples/Makefile | 1 +\n> > > > examples/distributor_app/Makefile | 57 ++++\n> > > > examples/distributor_app/main.c | 585\n> > > > ++++++++++++++++++++++++++++++++++++++\n> > > > examples/distributor_app/main.h | 46 +++\n> > > > 4 files changed, 689 insertions(+) create mode 100644 \n> > > > examples/distributor_app/Makefile create mode\n> > > > 100644 examples/distributor_app/main.c create mode 100644 \n> > > > examples/distributor_app/main.h\n> > > >\n> > > > diff --git a/examples/Makefile b/examples/Makefile index\n> > > > 6245f83..2ba82b0 100644\n> > > > --- a/examples/Makefile\n> > > > +++ b/examples/Makefile\n> > > > @@ -66,5 +66,6 @@ DIRS-y += vhost\n> > > > DIRS-$(CONFIG_RTE_LIBRTE_XEN_DOM0) += vhost_xen DIRS-y +=\n> vmdq\n> > > > DIRS-y += vmdq_dcb\n> > > > +DIRS-$(CONFIG_RTE_LIBRTE_DISTRIBUTOR) += distributor_app\n> > > >\n> > > > include $(RTE_SDK)/mk/rte.extsubdir.mk diff --git \n> > > > a/examples/distributor_app/Makefile\n> > > > b/examples/distributor_app/Makefile\n> > > > new file mode 100644\n> > > > index 0000000..394785d\n> > > > --- /dev/null\n> > > > +++ b/examples/distributor_app/Makefile\n> > > > @@ -0,0 +1,57 @@\n> > > > +# BSD LICENSE\n> > > > +#\n> > > > +# Copyright(c) 2010-2014 Intel Corporation. All rights reserved.\n> > > > +# All rights reserved.\n> > > > +#\n> > > > +# Redistribution and use in source and binary forms, with or without\n> > > > +# modification, are permitted provided that the following conditions\n> > > > +# are met:\n> > > > +#\n> > > > +# * Redistributions of source code must retain the above copyright\n> > > > +# notice, this list of conditions and the following disclaimer.\n> > > > +# * Redistributions in binary form must reproduce the above\n> copyright\n> > > > +# notice, this list of conditions and the following disclaimer in\n> > > > +# the documentation and/or other materials provided with the\n> > > > +# distribution.\n> > > > +# * Neither the name of Intel Corporation nor the names of its\n> > > > +# contributors may be used to endorse or promote products\n> derived\n> > > > +# from this software without specific prior written permission.\n> > > > +#\n> > > > +# THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND\n> > > > CONTRIBUTORS\n> > > > +# \"AS IS\" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING,\n> BUT\n> > > > NOT\n> > > > +# LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND\n> > > > FITNESS FOR\n> > > > +# A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE\n> > > > COPYRIGHT\n> > > > +# OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT,\n> > > > INCIDENTAL,\n> > > > +# SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING,\n> BUT\n> > > > NOT\n> > > > +# LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES;\n> LOSS\n> > > > OF USE,\n> > > > +# DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER\n> CAUSED\n> > > > AND ON ANY\n> > > > +# THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR\n> > > > TORT\n> > > > +# (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY\n> OUT OF\n> > > > THE USE\n> > > > +# OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH\n> > > > DAMAGE.\n> > > > +\n> > > > +ifeq ($(RTE_SDK),)\n> > > > +$(error \"Please define RTE_SDK environment variable\") endif\n> > > > +\n> > > > +# Default target, can be overriden by command line or \n> > > > +environment RTE_TARGET ?= x86_64-default-linuxapp-gcc\n> > >\n> > > This target is not present anymore. Change it to \n> > > x86_64-native-linuxapp-\n> gcc.\n> > >\n> > > > +\n> > > > +include $(RTE_SDK)/mk/rte.vars.mk\n> > > > +\n> > > > +# binary name\n> > > > +APP = distributor_app\n> > > > +\n> > > > +# all source are stored in SRCS-y SRCS-y := main.c\n> > > > +\n> > > > +CFLAGS += $(WERROR_FLAGS)\n> > > > +\n> > > > +# workaround for a gcc bug with noreturn attribute #\n> > > > +http://gcc.gnu.org/bugzilla/show_bug.cgi?id=12603\n> > > > +ifeq ($(CONFIG_RTE_TOOLCHAIN_GCC),y) CFLAGS_main.o += \n> > > > +-Wno-return-type endif\n> > > > +\n> > > > +EXTRA_CFLAGS += -O3 -Wfatal-errors\n> > > > +\n> > > > +include $(RTE_SDK)/mk/rte.extapp.mk\n> > > > diff --git a/examples/distributor_app/main.c \n> > > > b/examples/distributor_app/main.c new file mode 100644 index \n> > > > 0000000..628810a\n> > > > --- /dev/null\n> > > > +++ b/examples/distributor_app/main.c\n> > > > @@ -0,0 +1,585 @@\n> > > > +/*-\n> > > > + * BSD LICENSE\n> > > > + *\n> > > > + * Copyright(c) 2010-2014 Intel Corporation. All rights reserved.\n> > > > + * All rights reserved.\n> > > > + *\n> > > > + * Redistribution and use in source and binary forms, with or without\n> > > > + * modification, are permitted provided that the following conditions\n> > > > + * are met:\n> > > > + *\n> > > > + * * Redistributions of source code must retain the above copyright\n> > > > + * notice, this list of conditions and the following disclaimer.\n> > > > + * * Redistributions in binary form must reproduce the above\n> copyright\n> > > > + * notice, this list of conditions and the following disclaimer in\n> > > > + * the documentation and/or other materials provided with the\n> > > > + * distribution.\n> > > > + * * Neither the name of Intel Corporation nor the names of its\n> > > > + * contributors may be used to endorse or promote products\n> derived\n> > > > + * from this software without specific prior written permission.\n> > > > + *\n> > > > + * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND\n> > > > CONTRIBUTORS\n> > > > + * \"AS IS\" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING,\n> BUT\n> > > > NOT\n> > > > + * LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND\n> > > > FITNESS FOR\n> > > > + * A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE\n> > > > COPYRIGHT\n> > > > + * OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT,\n> > > > INCIDENTAL,\n> > > > + * SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES\n> (INCLUDING, BUT\n> > > > NOT\n> > > > + * LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES;\n> LOSS\n> > > > OF USE,\n> > > > + * DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER\n> CAUSED\n> > > > AND ON ANY\n> > > > + * THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY,\n> OR\n> > > > TORT\n> > > > + * (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY\n> OUT OF\n> > > > THE USE\n> > > > + * OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH\n> > > > DAMAGE.\n> > > > + */\n> > > > +\n> > > > +#include <stdint.h>\n> > > > +#include <inttypes.h>\n> > > > +#include <unistd.h>\n> > > > +#include <signal.h>\n> > > > +#include <getopt.h>\n> > > > +\n> > > > +#include <rte_eal.h>\n> > > > +#include <rte_ethdev.h>\n> > > > +#include <rte_cycles.h>\n> > > > +#include <rte_malloc.h>\n> > > > +#include <rte_debug.h>\n> > > > +#include <rte_distributor.h>\n> > > > +\n> > > > +#include \"main.h\"\n> > > > +\n> > > > +#define RX_RING_SIZE 256\n> > > > +#define RX_FREE_THRESH 32\n> > > > +#define RX_PTHRESH 8\n> > > > +#define RX_HTHRESH 8\n> > > > +#define RX_WTHRESH 0\n> > > > +\n> > > > +#define TX_RING_SIZE 512\n> > > > +#define TX_FREE_THRESH 32\n> > > > +#define TX_PTHRESH 32\n> > > > +#define TX_HTHRESH 0\n> > > > +#define TX_WTHRESH 0\n> > > > +#define TX_RSBIT_THRESH 32\n> > > > +#define TX_Q_FLAGS (ETH_TXQ_FLAGS_NOMULTSEGS |\n> > > > ETH_TXQ_FLAGS_NOVLANOFFL |\\\n> > > > +\tETH_TXQ_FLAGS_NOXSUMSCTP | ETH_TXQ_FLAGS_NOXSUMUDP |\n> > > > \\\n> > > > +\tETH_TXQ_FLAGS_NOXSUMTCP)\n> > > > +\n> > > > +#define NUM_MBUFS ((64*1024)-1) #define MBUF_SIZE (2048 + \n> > > > +sizeof(struct rte_mbuf) +\n> > > > RTE_PKTMBUF_HEADROOM)\n> > > > +#define MBUF_CACHE_SIZE 250\n> > > > +#define BURST_SIZE 32\n> > > > +#define RTE_RING_SZ 1024\n> > > > +\n> > > > +/* mask of enabled ports */\n> > > > +static uint32_t enabled_port_mask = 0;\n> > > > +\n> > > > +static volatile struct app_stats {\n> > > > +\tstruct {\n> > > > +\t\tuint64_t rx_pkts;\n> > > > +\t\tuint64_t returned_pkts;\n> > > > +\t\tuint64_t enqueued_pkts;\n> > > > +\t} rx __rte_cache_aligned;\n> > > > +\n> > > > +\tstruct {\n> > > > +\t\tuint64_t dequeue_pkts;\n> > > > +\t\tuint64_t tx_pkts;\n> > > > +\t} tx __rte_cache_aligned;\n> > > > +} app_stats;\n> > > > +\n> > > > +static const struct rte_eth_conf port_conf_default = {\n> > > > +\t.rxmode = {\n> > > > +\t\t.mq_mode = ETH_MQ_RX_RSS,\n> > > > +\t\t.max_rx_pkt_len = ETHER_MAX_LEN,\n> > > > +\t\t.split_hdr_size = 0,\n> > > > +\t\t.header_split = 0, /**< Header Split disabled */\n> > > > +\t\t.hw_ip_checksum = 0, /**< IP checksum offload enabled */\n> > > > +\t\t.hw_vlan_filter = 0, /**< VLAN filtering disabled */\n> > > > +\t\t.jumbo_frame = 0, /**< Jumbo Frame Support disabled */\n> > > > +\t\t.hw_strip_crc = 0, /**< CRC stripped by hardware */\n> > > > +\t},\n> > > > +\t.txmode = {\n> > > > +\t\t.mq_mode = ETH_MQ_TX_NONE,\n> > > > +\t},\n> > > > +\t.lpbk_mode = 0,\n> > > > +\t.rx_adv_conf = {\n> > > > +\t\t\t.rss_conf = {\n> > > > +\t\t\t\t.rss_hf = ETH_RSS_IPV4 | ETH_RSS_IPV6 |\n> > > > +\t\t\t\t\tETH_RSS_IPV4_TCP |\n> > > > ETH_RSS_IPV4_UDP |\n> > > > +\t\t\t\t\tETH_RSS_IPV6_TCP |\n> > > > ETH_RSS_IPV6_UDP,\n> > > > +\t\t\t}\n> > > > +\t},\n> > > > +};\n> > > > +\n> > > > +static const struct rte_eth_rxconf rx_conf_default = {\n> > > > +\t.rx_thresh = {\n> > > > +\t\t.pthresh = RX_PTHRESH,\n> > > > +\t\t.hthresh = RX_HTHRESH,\n> > > > +\t\t.wthresh = RX_WTHRESH,\n> > > > +\t},\n> > > > +\t.rx_free_thresh = RX_FREE_THRESH,\n> > > > +\t.rx_drop_en = 0,\n> > > > +};\n> > > > +\n> > > > +static const struct rte_eth_txconf tx_conf_default = {\n> > > > +\t.tx_thresh = {\n> > > > +\t\t.pthresh = TX_PTHRESH,\n> > > > +\t\t.hthresh = TX_HTHRESH,\n> > > > +\t\t.wthresh = TX_WTHRESH,\n> > > > +\t},\n> > > > +\t.tx_free_thresh = TX_FREE_THRESH,\n> > > > +\t.tx_rs_thresh = TX_RSBIT_THRESH,\n> > > > +\t.txq_flags = TX_Q_FLAGS\n> > > > +\n> > > > +};\n> > > > +\n> > > > +struct output_buffer {\n> > > > +\tunsigned count;\n> > > > +\tstruct rte_mbuf *mbufs[BURST_SIZE]; };\n> > > > +\n> > > > +/*\n> > > > + * Initialises a given port using global settings and with the \n> > > > +rx buffers\n> > > > + * coming from the mbuf_pool passed as parameter */ static \n> > > > +inline int port_init(uint8_t port, struct rte_mempool *mbuf_pool) {\n> > > > +\tstruct rte_eth_conf port_conf = port_conf_default;\n> > > > +\tconst uint16_t rxRings = 1, txRings = rte_lcore_count() - 1;\n> > > > +\tint retval;\n> > > > +\tuint16_t q;\n> > > > +\n> > > > +\tif (port >= rte_eth_dev_count())\n> > > > +\t\treturn -1;\n> > > > +\n> > > > +\tretval = rte_eth_dev_configure(port, rxRings, txRings, &port_conf);\n> > > > +\tif (retval != 0)\n> > > > +\t\treturn retval;\n> > > > +\n> > > > +\tfor (q = 0; q < rxRings; q++) {\n> > > > +\t\tretval = rte_eth_rx_queue_setup(port, q, RX_RING_SIZE,\n> > > > +\n> > > > \trte_eth_dev_socket_id(port),\n> > > > +\t\t\t\t\t\t&rx_conf_default,\n> > > > mbuf_pool);\n> > > > +\t\tif (retval < 0)\n> > > > +\t\t\treturn retval;\n> > > > +\t}\n> > > > +\n> > > > +\tfor (q = 0; q < txRings; q++) {\n> > > > +\t\tretval = rte_eth_tx_queue_setup(port, q, TX_RING_SIZE,\n> > > > +\n> > > > \trte_eth_dev_socket_id(port),\n> > > > +\t\t\t\t\t\t&tx_conf_default);\n> > > > +\t\tif (retval < 0)\n> > > > +\t\t\treturn retval;\n> > > > +\t}\n> > > > +\n> > > > +\tretval = rte_eth_dev_start(port);\n> > > > +\tif (retval < 0)\n> > > > +\t\treturn retval;\n> > > > +\n> > > > +\tstruct rte_eth_link link;\n> > > > +\trte_eth_link_get_nowait(port, &link);\n> > > > +\tif (!link.link_status) {\n> > > > +\t\tsleep(1);\n> > > > +\t\trte_eth_link_get_nowait(port, &link);\n> > > > +\t}\n> > > > +\n> > > > +\tif (!link.link_status) {\n> > > > +\t\tprintf(\"Link down on port %\"PRIu8\"\\n\", port);\n> > > > +\t\treturn 0;\n> > > > +\t}\n> > > > +\n> > > > +\tstruct ether_addr addr;\n> > > > +\trte_eth_macaddr_get(port, &addr);\n> > > > +\tprintf(\"Port %u MAC: %02\"PRIx8\" %02\"PRIx8\" %02\"PRIx8\n> > > > +\t\t\t\" %02\"PRIx8\" %02\"PRIx8\" %02\"PRIx8\"\\n\",\n> > > > +\t\t\t(unsigned)port,\n> > > > +\t\t\taddr.addr_bytes[0], addr.addr_bytes[1],\n> > > > +\t\t\taddr.addr_bytes[2], addr.addr_bytes[3],\n> > > > +\t\t\taddr.addr_bytes[4], addr.addr_bytes[5]);\n> > > > +\n> > > > +\trte_eth_promiscuous_enable(port);\n> > > > +\n> > > > +\treturn 0;\n> > > > +}\n> > > > +\n> > > > +struct lcore_params {\n> > > > +\tunsigned worker_id;\n> > > > +\tstruct rte_distributor *d;\n> > > > +\tstruct rte_ring *r;\n> > > > +};\n> > > > +\n> > > > +static __attribute__((noreturn)) void lcore_rx(struct \n> > > > +lcore_params\n> > > > +*p) {\n> > > > +\tstruct rte_distributor *d = p->d;\n> > > > +\tstruct rte_ring *r = p->r;\n> > > > +\tconst uint8_t nb_ports = rte_eth_dev_count();\n> > > > +\tconst int socket_id = rte_socket_id();\n> > > > +\tuint8_t port;\n> > > > +\n> > > > +\tfor (port = 0; port < nb_ports; port++) {\n> > > > +\t\t/* skip ports that are not enabled */\n> > > > +\t\tif ((enabled_port_mask & (1 << port)) == 0)\n> > > > +\t\t\tcontinue;\n> > > > +\n> > > > +\t\tif (rte_eth_dev_socket_id(port) > 0 &&\n> > > > +\t\t\t\trte_eth_dev_socket_id(port) != socket_id)\n> > > > +\t\t\tprintf(\"WARNING, port %u is on remote NUMA node\n> > > > to \"\n> > > > +\t\t\t\t\t\"RX thread.\\n\\tPerformance will not\n> > > > \"\n> > > > +\t\t\t\t\t\"be optimal.\\n\", port);\n> > > > +\t}\n> > > > +\n> > > > +\tprintf(\"\\nCore %u doing packet RX.\\n\", rte_lcore_id());\n> > > > +\tport = 0;\n> > > > +\tfor (;;) {\n> > > > +\t\t/* skip ports that are not enabled */\n> > > > +\t\tif ((enabled_port_mask & (1 << port)) == 0) {\n> > > > +\t\t\tif (++port == nb_ports)\n> > > > +\t\t\t\tport = 0;\n> > > > +\t\t\tcontinue;\n> > > > +\t\t}\n> > > > +\t\tstruct rte_mbuf *bufs[BURST_SIZE*2];\n> > > > +\t\tconst uint16_t nb_rx = rte_eth_rx_burst(port, 0, bufs,\n> > > > +\t\t\t\tBURST_SIZE);\n> > > > +\t\tapp_stats.rx.rx_pkts += nb_rx;\n> > > > +\n> > > > +\t\trte_distributor_process(d, bufs, nb_rx);\n> > > > +\t\tconst uint16_t nb_ret = rte_distributor_returned_pkts(d,\n> > > > +\t\t\t\tbufs, BURST_SIZE*2);\n> > > > +\t\tapp_stats.rx.returned_pkts += nb_ret;\n> > > > +\t\tif (unlikely(nb_ret == 0))\n> > > > +\t\t\tcontinue;\n> > > > +\n> > > > +\t\tuint16_t sent = rte_ring_enqueue_burst(r, (void *)bufs,\n> > > > nb_ret);\n> > > > +\t\tapp_stats.rx.enqueued_pkts += sent;\n> > > > +\t\tif (unlikely(sent < nb_ret)) {\n> > > > +\t\t\tprintf(\"Packet loss due to full ring\\n\");\n> > >\n> > > I would remove this printf. If core is too busy to process \n> > > packets, this may\n> make performance worse.\n> > > Statistics should reflect this packet loss.\n> >\n> > >Probably use RTE_LOG(DEBUG, ...).\n> > >That's way you can switch it on/off at compile time.\n> >\n> > By default LOG_LEVEL is 8 in config/common_linuxapp, hence I see\n> RTE_LOG(DEBUG,...) always works.\n> > So until unless change the LOG-LEVEL to lesser value, \n> > RTE_LOG(DEBUG,..)\n> will be logged ..\n> > Please let me know if I can go ahead and remove the printf as of now \n> > until\n> the LOG_LEVEL is fixed. (or) I have to add some logic to log\n> > RTE_LOG(DEBUG,..) only once.\n> \n> Sorry, I probably wasn't clear enough.\n> What I am saying - create something like that inside your app code:\n> \n> #ifdef XXX_DEBUG\n> #define XXX_LOG(level, fmt, args...) \\\n> RTE_LOG(level, PMD, \"%s(): \" fmt \"\\n\", __func__, ## args) \n> #else #define XXX_LOG(level, fmt, args...) do { } while(0) #endif\n> \n> And replace your pritnf()s with it.\n\nI think we should make some changes in some logs in DPDK, and change the default log level to be INFO or less, so we could use the DEBUG level for things like this, but anyway, that would be another patch that I am thinking to submit some other day.\nAnyway, for this one, this option could work, but still that message may be shown hundreds of thousands of times per second, considering that CPU is overloaded. So, should we include something like a flag here, so message is shown just once?\nI know that this will only be used when DEBUG is enabled, but still that could flood the screen, as it is in an infinite loop.\n\n> \n> >\n> > >\n> > > > +\t\t\twhile (sent < nb_ret)\n> > > > +\t\t\t\trte_pktmbuf_free(bufs[sent++]);\n> > > > +\t\t}\n> > > > +\t\tif (++port == nb_ports)\n> > > > +\t\t\tport = 0;\n> > > > +\t}\n> > > > +}\n> > > > +\n> > > > +static inline void\n> > > > +flush_one_port(struct output_buffer *outbuf, uint8_t outp) {\n> > > > +\tunsigned nb_tx = rte_eth_tx_burst(outp, 0, outbuf->mbufs,\n> > > > +\t\t\toutbuf->count);\n> > > > +\tapp_stats.tx.tx_pkts += nb_tx;\n> > > > +\n> > > > +\tif (unlikely(nb_tx < outbuf->count)) {\n> > > > +\t\tprintf(\"Packet loss with tx_burst\\n\");\n> > >\n> > > Same here. This is fine for debugging, but statistics are updated \n> > > as well,\n> so we do not need this printf.\n> > >\n> > > > +\t\tdo {\n> > > > +\t\t\trte_pktmbuf_free(outbuf->mbufs[nb_tx]);\n> > > > +\t\t} while (++nb_tx < outbuf->count);\n> > > > +\t}\n> > > > +\toutbuf->count = 0;\n> > > > +}\n> > > > +\n\n--------------------------------------------------------------\nIntel Shannon Limited\nRegistered in Ireland\nRegistered Office: Collinstown Industrial Park, Leixlip, County Kildare\nRegistered Number: 308263\nBusiness address: Dromore House, East Park, Shannon, Co. Clare\n\nThis e-mail and any attachments may contain confidential material for the sole use of the intended recipient(s). Any review or distribution by others is strictly prohibited. If you are not the intended recipient, please contact the sender and delete all copies.", "headers": { "Return-Path": "<dev-bounces@dpdk.org>", "X-Original-To": "patchwork@dpdk.org", "Delivered-To": "patchwork@dpdk.org", "Received": [ "from [92.243.14.124] (localhost [IPv6:::1])\n\tby dpdk.org (Postfix) with ESMTP id 67A366A87;\n\tTue, 30 Sep 2014 09:56:55 +0200 (CEST)", "from mga01.intel.com (mga01.intel.com [192.55.52.88])\n\tby dpdk.org (Postfix) with ESMTP id C5B6C6A7D\n\tfor <dev@dpdk.org>; Tue, 30 Sep 2014 09:56:52 +0200 (CEST)", "from fmsmga002.fm.intel.com ([10.253.24.26])\n\tby fmsmga101.fm.intel.com with ESMTP; 30 Sep 2014 01:03:07 -0700", "from irsmsx104.ger.corp.intel.com ([163.33.3.159])\n\tby fmsmga002.fm.intel.com with ESMTP; 30 Sep 2014 01:03:01 -0700", "from irsmsx105.ger.corp.intel.com (163.33.3.28) by\n\tIRSMSX104.ger.corp.intel.com (163.33.3.159) with Microsoft SMTP\n\tServer (TLS) id 14.3.195.1; Tue, 30 Sep 2014 09:02:59 +0100", "from irsmsx109.ger.corp.intel.com ([169.254.13.253]) by\n\tIRSMSX105.ger.corp.intel.com ([169.254.7.174]) with mapi id\n\t14.03.0195.001; Tue, 30 Sep 2014 09:02:58 +0100" ], "X-ExtLoop1": "1", "X-IronPort-AV": "E=Sophos;i=\"5.04,625,1406617200\"; d=\"scan'208\";a=\"607423980\"", "From": "\"Pattan, Reshma\" <reshma.pattan@intel.com>", "To": "\"Ananyev, Konstantin\" <konstantin.ananyev@intel.com>, \"De Lara Guarch,\n\tPablo\" <pablo.de.lara.guarch@intel.com>, \"dev@dpdk.org\" <dev@dpdk.org>", "Thread-Topic": "[dpdk-dev] [PATCH v2] distributor_app: new sample app", "Thread-Index": "AQHP2AJLRlkqtOmcpECqEROZe22ZQpwTd5UAgAALJgCABI2YkP//+2OAgAAH8ICAAUXScA==", "Date": "Tue, 30 Sep 2014 08:02:58 +0000", "Message-ID": "<3AEA2BF9852C6F48A459DA490692831FE208B3@IRSMSX109.ger.corp.intel.com>", "References": "<1410869607-16842-1-git-send-email-reshma.pattan@intel.com>\n\t<1411568210-2555-1-git-send-email-reshma.pattan@intel.com>\n\t<E115CCD9D858EF4F90C690B0DCB4D89722626933@IRSMSX108.ger.corp.intel.com>\n\t<2601191342CEEE43887BDE71AB9772582137D8B1@IRSMSX104.ger.corp.intel.com>\n\t<3AEA2BF9852C6F48A459DA490692831FE20598@IRSMSX109.ger.corp.intel.com>\n\t<2601191342CEEE43887BDE71AB97725821387654@IRSMSX104.ger.corp.intel.com>\n\t<E115CCD9D858EF4F90C690B0DCB4D8972262823A@IRSMSX108.ger.corp.intel.com>", "In-Reply-To": "<E115CCD9D858EF4F90C690B0DCB4D8972262823A@IRSMSX108.ger.corp.intel.com>", "Accept-Language": "en-US", "Content-Language": "en-US", "X-MS-Has-Attach": "", "X-MS-TNEF-Correlator": "", "x-originating-ip": "[163.33.239.180]", "Content-Type": "text/plain; charset=\"us-ascii\"", "MIME-Version": "1.0", "Content-Transfer-Encoding": "quoted-printable", "Subject": "Re: [dpdk-dev] [PATCH v2] distributor_app: new sample app", "X-BeenThere": "dev@dpdk.org", "X-Mailman-Version": "2.1.15", "Precedence": "list", "List-Id": "patches and discussions about DPDK <dev.dpdk.org>", "List-Unsubscribe": "<http://dpdk.org/ml/options/dev>,\n\t<mailto:dev-request@dpdk.org?subject=unsubscribe>", "List-Archive": "<http://dpdk.org/ml/archives/dev/>", "List-Post": "<mailto:dev@dpdk.org>", "List-Help": "<mailto:dev-request@dpdk.org?subject=help>", "List-Subscribe": "<http://dpdk.org/ml/listinfo/dev>,\n\t<mailto:dev-request@dpdk.org?subject=subscribe>", "Errors-To": "dev-bounces@dpdk.org", "Sender": "\"dev\" <dev-bounces@dpdk.org>" }, "addressed": null }, { "id": 1281, "web_url": "https://patches.dpdk.org/comment/1281/", "msgid": "<2601191342CEEE43887BDE71AB9772582138CCA0@IRSMSX105.ger.corp.intel.com>", "list_archive_url": "https://inbox.dpdk.org/dev/2601191342CEEE43887BDE71AB9772582138CCA0@IRSMSX105.ger.corp.intel.com", "date": "2014-09-30T09:21:19", "subject": "Re: [dpdk-dev] [PATCH v2] distributor_app: new sample app", "submitter": { "id": 33, "url": "https://patches.dpdk.org/api/people/33/?format=api", "name": "Ananyev, Konstantin", "email": "konstantin.ananyev@intel.com" }, "content": "> -----Original Message-----\n> From: Pattan, Reshma\n> Sent: Tuesday, September 30, 2014 9:03 AM\n> To: Ananyev, Konstantin; De Lara Guarch, Pablo; dev@dpdk.org\n> Subject: RE: [dpdk-dev] [PATCH v2] distributor_app: new sample app\n> \n> Hi Konstantin,\n> \n> Any comments on below Pablos comment? If so please provide.\n> \n> Thanks,\n> Reshma\n> \n\nHi Reshma,\n\nNo really.\nI would just change printf to what I suggested.\nFor your app, I believe that is more than enough.\nBut if you'd like to introduce some sort of rate limiting for logging -\nsure go ahead would be interesting to see that patch.\nKonstantin\n\n> \n> \n> -----Original Message-----\n> From: De Lara Guarch, Pablo\n> Sent: Monday, September 29, 2014 2:35 PM\n> To: Ananyev, Konstantin; Pattan, Reshma; dev@dpdk.org\n> Subject: RE: [dpdk-dev] [PATCH v2] distributor_app: new sample app\n> \n> \n> \n> > -----Original Message-----\n> > From: Ananyev, Konstantin\n> > Sent: Monday, September 29, 2014 2:07 PM\n> > To: Pattan, Reshma; De Lara Guarch, Pablo; dev@dpdk.org\n> > Subject: RE: [dpdk-dev] [PATCH v2] distributor_app: new sample app\n> >\n> >\n> >\n> > > -----Original Message-----\n> > > From: Pattan, Reshma\n> > > Sent: Monday, September 29, 2014 1:40 PM\n> > > To: Ananyev, Konstantin; De Lara Guarch, Pablo; dev@dpdk.org\n> > > Subject: RE: [dpdk-dev] [PATCH v2] distributor_app: new sample app\n> > >\n> > >\n> > >\n> > > -----Original Message-----\n> > > From: Ananyev, Konstantin\n> > > Sent: Friday, September 26, 2014 4:52 PM\n> > > To: De Lara Guarch, Pablo; Pattan, Reshma; dev@dpdk.org\n> > > Subject: RE: [dpdk-dev] [PATCH v2] distributor_app: new sample app\n> > >\n> > >\n> > >\n> > > > -----Original Message-----\n> > > > From: dev [mailto:dev-bounces@dpdk.org] On Behalf Of De Lara\n> > > > Guarch, Pablo\n> > > > Sent: Friday, September 26, 2014 4:12 PM\n> > > > To: Pattan, Reshma; dev@dpdk.org\n> > > > Subject: Re: [dpdk-dev] [PATCH v2] distributor_app: new sample app\n> > > >\n> > > > Hi,\n> > > >\n> > > > > -----Original Message-----\n> > > > > From: dev [mailto:dev-bounces@dpdk.org] On Behalf Of reshmapa\n> > > > > Sent: Wednesday, September 24, 2014 3:17 PM\n> > > > > To: dev@dpdk.org\n> > > > > Subject: [dpdk-dev] [PATCH v2] distributor_app: new sample app\n> > > > >\n> > > > > From: Reshma Pattan <reshma.pattan@intel.com>\n> > > > >\n> > > > > A new sample app that shows the usage of the distributor library.\n> > > > > This app works as follows:\n> > > > >\n> > > > > * An RX thread runs which pulls packets from each ethernet port in turn\n> > > > > and passes those packets to worker using a distributor component.\n> > > > > * The workers take the packets in turn, and determine the output port\n> > > > > for those packets using basic l2forwarding doing an xor on the source\n> > > > > port id.\n> > > > > * The RX thread takes the returned packets from the workers and\n> > enqueue\n> > > > > those packets into an rte_ring structure.\n> > > > > * A TX thread pulls the packets off the rte_ring structure and then\n> > > > > sends each packet out the output port specified previously by\n> > > > > the worker\n> > > > > * Command-line option support provided only for portmask.\n> > > > >\n> > > > > Signed-off-by: Bruce Richardson <bruce.richardson@intel.com>\n> > > > > Signed-off-by: Reshma Pattan <reshma.pattan@intel.com>\n> > > > > ---\n> > > > > examples/Makefile | 1 +\n> > > > > examples/distributor_app/Makefile | 57 ++++\n> > > > > examples/distributor_app/main.c | 585\n> > > > > ++++++++++++++++++++++++++++++++++++++\n> > > > > examples/distributor_app/main.h | 46 +++\n> > > > > 4 files changed, 689 insertions(+) create mode 100644\n> > > > > examples/distributor_app/Makefile create mode\n> > > > > 100644 examples/distributor_app/main.c create mode 100644\n> > > > > examples/distributor_app/main.h\n> > > > >\n> > > > > diff --git a/examples/Makefile b/examples/Makefile index\n> > > > > 6245f83..2ba82b0 100644\n> > > > > --- a/examples/Makefile\n> > > > > +++ b/examples/Makefile\n> > > > > @@ -66,5 +66,6 @@ DIRS-y += vhost\n> > > > > DIRS-$(CONFIG_RTE_LIBRTE_XEN_DOM0) += vhost_xen DIRS-y +=\n> > vmdq\n> > > > > DIRS-y += vmdq_dcb\n> > > > > +DIRS-$(CONFIG_RTE_LIBRTE_DISTRIBUTOR) += distributor_app\n> > > > >\n> > > > > include $(RTE_SDK)/mk/rte.extsubdir.mk diff --git\n> > > > > a/examples/distributor_app/Makefile\n> > > > > b/examples/distributor_app/Makefile\n> > > > > new file mode 100644\n> > > > > index 0000000..394785d\n> > > > > --- /dev/null\n> > > > > +++ b/examples/distributor_app/Makefile\n> > > > > @@ -0,0 +1,57 @@\n> > > > > +# BSD LICENSE\n> > > > > +#\n> > > > > +# Copyright(c) 2010-2014 Intel Corporation. All rights reserved.\n> > > > > +# All rights reserved.\n> > > > > +#\n> > > > > +# Redistribution and use in source and binary forms, with or without\n> > > > > +# modification, are permitted provided that the following conditions\n> > > > > +# are met:\n> > > > > +#\n> > > > > +# * Redistributions of source code must retain the above copyright\n> > > > > +# notice, this list of conditions and the following disclaimer.\n> > > > > +# * Redistributions in binary form must reproduce the above\n> > copyright\n> > > > > +# notice, this list of conditions and the following disclaimer in\n> > > > > +# the documentation and/or other materials provided with the\n> > > > > +# distribution.\n> > > > > +# * Neither the name of Intel Corporation nor the names of its\n> > > > > +# contributors may be used to endorse or promote products\n> > derived\n> > > > > +# from this software without specific prior written permission.\n> > > > > +#\n> > > > > +# THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND\n> > > > > CONTRIBUTORS\n> > > > > +# \"AS IS\" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING,\n> > BUT\n> > > > > NOT\n> > > > > +# LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND\n> > > > > FITNESS FOR\n> > > > > +# A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE\n> > > > > COPYRIGHT\n> > > > > +# OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT,\n> > > > > INCIDENTAL,\n> > > > > +# SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING,\n> > BUT\n> > > > > NOT\n> > > > > +# LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES;\n> > LOSS\n> > > > > OF USE,\n> > > > > +# DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER\n> > CAUSED\n> > > > > AND ON ANY\n> > > > > +# THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR\n> > > > > TORT\n> > > > > +# (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY\n> > OUT OF\n> > > > > THE USE\n> > > > > +# OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH\n> > > > > DAMAGE.\n> > > > > +\n> > > > > +ifeq ($(RTE_SDK),)\n> > > > > +$(error \"Please define RTE_SDK environment variable\") endif\n> > > > > +\n> > > > > +# Default target, can be overriden by command line or\n> > > > > +environment RTE_TARGET ?= x86_64-default-linuxapp-gcc\n> > > >\n> > > > This target is not present anymore. Change it to\n> > > > x86_64-native-linuxapp-\n> > gcc.\n> > > >\n> > > > > +\n> > > > > +include $(RTE_SDK)/mk/rte.vars.mk\n> > > > > +\n> > > > > +# binary name\n> > > > > +APP = distributor_app\n> > > > > +\n> > > > > +# all source are stored in SRCS-y SRCS-y := main.c\n> > > > > +\n> > > > > +CFLAGS += $(WERROR_FLAGS)\n> > > > > +\n> > > > > +# workaround for a gcc bug with noreturn attribute #\n> > > > > +http://gcc.gnu.org/bugzilla/show_bug.cgi?id=12603\n> > > > > +ifeq ($(CONFIG_RTE_TOOLCHAIN_GCC),y) CFLAGS_main.o +=\n> > > > > +-Wno-return-type endif\n> > > > > +\n> > > > > +EXTRA_CFLAGS += -O3 -Wfatal-errors\n> > > > > +\n> > > > > +include $(RTE_SDK)/mk/rte.extapp.mk\n> > > > > diff --git a/examples/distributor_app/main.c\n> > > > > b/examples/distributor_app/main.c new file mode 100644 index\n> > > > > 0000000..628810a\n> > > > > --- /dev/null\n> > > > > +++ b/examples/distributor_app/main.c\n> > > > > @@ -0,0 +1,585 @@\n> > > > > +/*-\n> > > > > + * BSD LICENSE\n> > > > > + *\n> > > > > + * Copyright(c) 2010-2014 Intel Corporation. All rights reserved.\n> > > > > + * All rights reserved.\n> > > > > + *\n> > > > > + * Redistribution and use in source and binary forms, with or without\n> > > > > + * modification, are permitted provided that the following conditions\n> > > > > + * are met:\n> > > > > + *\n> > > > > + * * Redistributions of source code must retain the above copyright\n> > > > > + * notice, this list of conditions and the following disclaimer.\n> > > > > + * * Redistributions in binary form must reproduce the above\n> > copyright\n> > > > > + * notice, this list of conditions and the following disclaimer in\n> > > > > + * the documentation and/or other materials provided with the\n> > > > > + * distribution.\n> > > > > + * * Neither the name of Intel Corporation nor the names of its\n> > > > > + * contributors may be used to endorse or promote products\n> > derived\n> > > > > + * from this software without specific prior written permission.\n> > > > > + *\n> > > > > + * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND\n> > > > > CONTRIBUTORS\n> > > > > + * \"AS IS\" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING,\n> > BUT\n> > > > > NOT\n> > > > > + * LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND\n> > > > > FITNESS FOR\n> > > > > + * A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE\n> > > > > COPYRIGHT\n> > > > > + * OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT,\n> > > > > INCIDENTAL,\n> > > > > + * SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES\n> > (INCLUDING, BUT\n> > > > > NOT\n> > > > > + * LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES;\n> > LOSS\n> > > > > OF USE,\n> > > > > + * DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER\n> > CAUSED\n> > > > > AND ON ANY\n> > > > > + * THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY,\n> > OR\n> > > > > TORT\n> > > > > + * (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY\n> > OUT OF\n> > > > > THE USE\n> > > > > + * OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH\n> > > > > DAMAGE.\n> > > > > + */\n> > > > > +\n> > > > > +#include <stdint.h>\n> > > > > +#include <inttypes.h>\n> > > > > +#include <unistd.h>\n> > > > > +#include <signal.h>\n> > > > > +#include <getopt.h>\n> > > > > +\n> > > > > +#include <rte_eal.h>\n> > > > > +#include <rte_ethdev.h>\n> > > > > +#include <rte_cycles.h>\n> > > > > +#include <rte_malloc.h>\n> > > > > +#include <rte_debug.h>\n> > > > > +#include <rte_distributor.h>\n> > > > > +\n> > > > > +#include \"main.h\"\n> > > > > +\n> > > > > +#define RX_RING_SIZE 256\n> > > > > +#define RX_FREE_THRESH 32\n> > > > > +#define RX_PTHRESH 8\n> > > > > +#define RX_HTHRESH 8\n> > > > > +#define RX_WTHRESH 0\n> > > > > +\n> > > > > +#define TX_RING_SIZE 512\n> > > > > +#define TX_FREE_THRESH 32\n> > > > > +#define TX_PTHRESH 32\n> > > > > +#define TX_HTHRESH 0\n> > > > > +#define TX_WTHRESH 0\n> > > > > +#define TX_RSBIT_THRESH 32\n> > > > > +#define TX_Q_FLAGS (ETH_TXQ_FLAGS_NOMULTSEGS |\n> > > > > ETH_TXQ_FLAGS_NOVLANOFFL |\\\n> > > > > +\tETH_TXQ_FLAGS_NOXSUMSCTP | ETH_TXQ_FLAGS_NOXSUMUDP |\n> > > > > \\\n> > > > > +\tETH_TXQ_FLAGS_NOXSUMTCP)\n> > > > > +\n> > > > > +#define NUM_MBUFS ((64*1024)-1) #define MBUF_SIZE (2048 +\n> > > > > +sizeof(struct rte_mbuf) +\n> > > > > RTE_PKTMBUF_HEADROOM)\n> > > > > +#define MBUF_CACHE_SIZE 250\n> > > > > +#define BURST_SIZE 32\n> > > > > +#define RTE_RING_SZ 1024\n> > > > > +\n> > > > > +/* mask of enabled ports */\n> > > > > +static uint32_t enabled_port_mask = 0;\n> > > > > +\n> > > > > +static volatile struct app_stats {\n> > > > > +\tstruct {\n> > > > > +\t\tuint64_t rx_pkts;\n> > > > > +\t\tuint64_t returned_pkts;\n> > > > > +\t\tuint64_t enqueued_pkts;\n> > > > > +\t} rx __rte_cache_aligned;\n> > > > > +\n> > > > > +\tstruct {\n> > > > > +\t\tuint64_t dequeue_pkts;\n> > > > > +\t\tuint64_t tx_pkts;\n> > > > > +\t} tx __rte_cache_aligned;\n> > > > > +} app_stats;\n> > > > > +\n> > > > > +static const struct rte_eth_conf port_conf_default = {\n> > > > > +\t.rxmode = {\n> > > > > +\t\t.mq_mode = ETH_MQ_RX_RSS,\n> > > > > +\t\t.max_rx_pkt_len = ETHER_MAX_LEN,\n> > > > > +\t\t.split_hdr_size = 0,\n> > > > > +\t\t.header_split = 0, /**< Header Split disabled */\n> > > > > +\t\t.hw_ip_checksum = 0, /**< IP checksum offload enabled */\n> > > > > +\t\t.hw_vlan_filter = 0, /**< VLAN filtering disabled */\n> > > > > +\t\t.jumbo_frame = 0, /**< Jumbo Frame Support disabled */\n> > > > > +\t\t.hw_strip_crc = 0, /**< CRC stripped by hardware */\n> > > > > +\t},\n> > > > > +\t.txmode = {\n> > > > > +\t\t.mq_mode = ETH_MQ_TX_NONE,\n> > > > > +\t},\n> > > > > +\t.lpbk_mode = 0,\n> > > > > +\t.rx_adv_conf = {\n> > > > > +\t\t\t.rss_conf = {\n> > > > > +\t\t\t\t.rss_hf = ETH_RSS_IPV4 | ETH_RSS_IPV6 |\n> > > > > +\t\t\t\t\tETH_RSS_IPV4_TCP |\n> > > > > ETH_RSS_IPV4_UDP |\n> > > > > +\t\t\t\t\tETH_RSS_IPV6_TCP |\n> > > > > ETH_RSS_IPV6_UDP,\n> > > > > +\t\t\t}\n> > > > > +\t},\n> > > > > +};\n> > > > > +\n> > > > > +static const struct rte_eth_rxconf rx_conf_default = {\n> > > > > +\t.rx_thresh = {\n> > > > > +\t\t.pthresh = RX_PTHRESH,\n> > > > > +\t\t.hthresh = RX_HTHRESH,\n> > > > > +\t\t.wthresh = RX_WTHRESH,\n> > > > > +\t},\n> > > > > +\t.rx_free_thresh = RX_FREE_THRESH,\n> > > > > +\t.rx_drop_en = 0,\n> > > > > +};\n> > > > > +\n> > > > > +static const struct rte_eth_txconf tx_conf_default = {\n> > > > > +\t.tx_thresh = {\n> > > > > +\t\t.pthresh = TX_PTHRESH,\n> > > > > +\t\t.hthresh = TX_HTHRESH,\n> > > > > +\t\t.wthresh = TX_WTHRESH,\n> > > > > +\t},\n> > > > > +\t.tx_free_thresh = TX_FREE_THRESH,\n> > > > > +\t.tx_rs_thresh = TX_RSBIT_THRESH,\n> > > > > +\t.txq_flags = TX_Q_FLAGS\n> > > > > +\n> > > > > +};\n> > > > > +\n> > > > > +struct output_buffer {\n> > > > > +\tunsigned count;\n> > > > > +\tstruct rte_mbuf *mbufs[BURST_SIZE]; };\n> > > > > +\n> > > > > +/*\n> > > > > + * Initialises a given port using global settings and with the\n> > > > > +rx buffers\n> > > > > + * coming from the mbuf_pool passed as parameter */ static\n> > > > > +inline int port_init(uint8_t port, struct rte_mempool *mbuf_pool) {\n> > > > > +\tstruct rte_eth_conf port_conf = port_conf_default;\n> > > > > +\tconst uint16_t rxRings = 1, txRings = rte_lcore_count() - 1;\n> > > > > +\tint retval;\n> > > > > +\tuint16_t q;\n> > > > > +\n> > > > > +\tif (port >= rte_eth_dev_count())\n> > > > > +\t\treturn -1;\n> > > > > +\n> > > > > +\tretval = rte_eth_dev_configure(port, rxRings, txRings, &port_conf);\n> > > > > +\tif (retval != 0)\n> > > > > +\t\treturn retval;\n> > > > > +\n> > > > > +\tfor (q = 0; q < rxRings; q++) {\n> > > > > +\t\tretval = rte_eth_rx_queue_setup(port, q, RX_RING_SIZE,\n> > > > > +\n> > > > > \trte_eth_dev_socket_id(port),\n> > > > > +\t\t\t\t\t\t&rx_conf_default,\n> > > > > mbuf_pool);\n> > > > > +\t\tif (retval < 0)\n> > > > > +\t\t\treturn retval;\n> > > > > +\t}\n> > > > > +\n> > > > > +\tfor (q = 0; q < txRings; q++) {\n> > > > > +\t\tretval = rte_eth_tx_queue_setup(port, q, TX_RING_SIZE,\n> > > > > +\n> > > > > \trte_eth_dev_socket_id(port),\n> > > > > +\t\t\t\t\t\t&tx_conf_default);\n> > > > > +\t\tif (retval < 0)\n> > > > > +\t\t\treturn retval;\n> > > > > +\t}\n> > > > > +\n> > > > > +\tretval = rte_eth_dev_start(port);\n> > > > > +\tif (retval < 0)\n> > > > > +\t\treturn retval;\n> > > > > +\n> > > > > +\tstruct rte_eth_link link;\n> > > > > +\trte_eth_link_get_nowait(port, &link);\n> > > > > +\tif (!link.link_status) {\n> > > > > +\t\tsleep(1);\n> > > > > +\t\trte_eth_link_get_nowait(port, &link);\n> > > > > +\t}\n> > > > > +\n> > > > > +\tif (!link.link_status) {\n> > > > > +\t\tprintf(\"Link down on port %\"PRIu8\"\\n\", port);\n> > > > > +\t\treturn 0;\n> > > > > +\t}\n> > > > > +\n> > > > > +\tstruct ether_addr addr;\n> > > > > +\trte_eth_macaddr_get(port, &addr);\n> > > > > +\tprintf(\"Port %u MAC: %02\"PRIx8\" %02\"PRIx8\" %02\"PRIx8\n> > > > > +\t\t\t\" %02\"PRIx8\" %02\"PRIx8\" %02\"PRIx8\"\\n\",\n> > > > > +\t\t\t(unsigned)port,\n> > > > > +\t\t\taddr.addr_bytes[0], addr.addr_bytes[1],\n> > > > > +\t\t\taddr.addr_bytes[2], addr.addr_bytes[3],\n> > > > > +\t\t\taddr.addr_bytes[4], addr.addr_bytes[5]);\n> > > > > +\n> > > > > +\trte_eth_promiscuous_enable(port);\n> > > > > +\n> > > > > +\treturn 0;\n> > > > > +}\n> > > > > +\n> > > > > +struct lcore_params {\n> > > > > +\tunsigned worker_id;\n> > > > > +\tstruct rte_distributor *d;\n> > > > > +\tstruct rte_ring *r;\n> > > > > +};\n> > > > > +\n> > > > > +static __attribute__((noreturn)) void lcore_rx(struct\n> > > > > +lcore_params\n> > > > > +*p) {\n> > > > > +\tstruct rte_distributor *d = p->d;\n> > > > > +\tstruct rte_ring *r = p->r;\n> > > > > +\tconst uint8_t nb_ports = rte_eth_dev_count();\n> > > > > +\tconst int socket_id = rte_socket_id();\n> > > > > +\tuint8_t port;\n> > > > > +\n> > > > > +\tfor (port = 0; port < nb_ports; port++) {\n> > > > > +\t\t/* skip ports that are not enabled */\n> > > > > +\t\tif ((enabled_port_mask & (1 << port)) == 0)\n> > > > > +\t\t\tcontinue;\n> > > > > +\n> > > > > +\t\tif (rte_eth_dev_socket_id(port) > 0 &&\n> > > > > +\t\t\t\trte_eth_dev_socket_id(port) != socket_id)\n> > > > > +\t\t\tprintf(\"WARNING, port %u is on remote NUMA node\n> > > > > to \"\n> > > > > +\t\t\t\t\t\"RX thread.\\n\\tPerformance will not\n> > > > > \"\n> > > > > +\t\t\t\t\t\"be optimal.\\n\", port);\n> > > > > +\t}\n> > > > > +\n> > > > > +\tprintf(\"\\nCore %u doing packet RX.\\n\", rte_lcore_id());\n> > > > > +\tport = 0;\n> > > > > +\tfor (;;) {\n> > > > > +\t\t/* skip ports that are not enabled */\n> > > > > +\t\tif ((enabled_port_mask & (1 << port)) == 0) {\n> > > > > +\t\t\tif (++port == nb_ports)\n> > > > > +\t\t\t\tport = 0;\n> > > > > +\t\t\tcontinue;\n> > > > > +\t\t}\n> > > > > +\t\tstruct rte_mbuf *bufs[BURST_SIZE*2];\n> > > > > +\t\tconst uint16_t nb_rx = rte_eth_rx_burst(port, 0, bufs,\n> > > > > +\t\t\t\tBURST_SIZE);\n> > > > > +\t\tapp_stats.rx.rx_pkts += nb_rx;\n> > > > > +\n> > > > > +\t\trte_distributor_process(d, bufs, nb_rx);\n> > > > > +\t\tconst uint16_t nb_ret = rte_distributor_returned_pkts(d,\n> > > > > +\t\t\t\tbufs, BURST_SIZE*2);\n> > > > > +\t\tapp_stats.rx.returned_pkts += nb_ret;\n> > > > > +\t\tif (unlikely(nb_ret == 0))\n> > > > > +\t\t\tcontinue;\n> > > > > +\n> > > > > +\t\tuint16_t sent = rte_ring_enqueue_burst(r, (void *)bufs,\n> > > > > nb_ret);\n> > > > > +\t\tapp_stats.rx.enqueued_pkts += sent;\n> > > > > +\t\tif (unlikely(sent < nb_ret)) {\n> > > > > +\t\t\tprintf(\"Packet loss due to full ring\\n\");\n> > > >\n> > > > I would remove this printf. If core is too busy to process\n> > > > packets, this may\n> > make performance worse.\n> > > > Statistics should reflect this packet loss.\n> > >\n> > > >Probably use RTE_LOG(DEBUG, ...).\n> > > >That's way you can switch it on/off at compile time.\n> > >\n> > > By default LOG_LEVEL is 8 in config/common_linuxapp, hence I see\n> > RTE_LOG(DEBUG,...) always works.\n> > > So until unless change the LOG-LEVEL to lesser value,\n> > > RTE_LOG(DEBUG,..)\n> > will be logged ..\n> > > Please let me know if I can go ahead and remove the printf as of now\n> > > until\n> > the LOG_LEVEL is fixed. (or) I have to add some logic to log\n> > > RTE_LOG(DEBUG,..) only once.\n> >\n> > Sorry, I probably wasn't clear enough.\n> > What I am saying - create something like that inside your app code:\n> >\n> > #ifdef XXX_DEBUG\n> > #define XXX_LOG(level, fmt, args...) \\\n> > RTE_LOG(level, PMD, \"%s(): \" fmt \"\\n\", __func__, ## args)\n> > #else #define XXX_LOG(level, fmt, args...) do { } while(0) #endif\n> >\n> > And replace your pritnf()s with it.\n> \n> I think we should make some changes in some logs in DPDK, and change the default log level to be INFO or less, so we could use the\n> DEBUG level for things like this, but anyway, that would be another patch that I am thinking to submit some other day.\n> Anyway, for this one, this option could work, but still that message may be shown hundreds of thousands of times per second,\n> considering that CPU is overloaded. So, should we include something like a flag here, so message is shown just once?\n> I know that this will only be used when DEBUG is enabled, but still that could flood the screen, as it is in an infinite loop.\n> \n> >\n> > >\n> > > >\n> > > > > +\t\t\twhile (sent < nb_ret)\n> > > > > +\t\t\t\trte_pktmbuf_free(bufs[sent++]);\n> > > > > +\t\t}\n> > > > > +\t\tif (++port == nb_ports)\n> > > > > +\t\t\tport = 0;\n> > > > > +\t}\n> > > > > +}\n> > > > > +\n> > > > > +static inline void\n> > > > > +flush_one_port(struct output_buffer *outbuf, uint8_t outp) {\n> > > > > +\tunsigned nb_tx = rte_eth_tx_burst(outp, 0, outbuf->mbufs,\n> > > > > +\t\t\toutbuf->count);\n> > > > > +\tapp_stats.tx.tx_pkts += nb_tx;\n> > > > > +\n> > > > > +\tif (unlikely(nb_tx < outbuf->count)) {\n> > > > > +\t\tprintf(\"Packet loss with tx_burst\\n\");\n> > > >\n> > > > Same here. This is fine for debugging, but statistics are updated\n> > > > as well,\n> > so we do not need this printf.\n> > > >\n> > > > > +\t\tdo {\n> > > > > +\t\t\trte_pktmbuf_free(outbuf->mbufs[nb_tx]);\n> > > > > +\t\t} while (++nb_tx < outbuf->count);\n> > > > > +\t}\n> > > > > +\toutbuf->count = 0;\n> > > > > +}\n> > > > > +", "headers": { "Return-Path": "<dev-bounces@dpdk.org>", "X-Original-To": "patchwork@dpdk.org", "Delivered-To": "patchwork@dpdk.org", "Received": [ "from [92.243.14.124] (localhost [IPv6:::1])\n\tby dpdk.org (Postfix) with ESMTP id 38DD96A90;\n\tTue, 30 Sep 2014 11:14:48 +0200 (CEST)", "from mga14.intel.com (mga14.intel.com [192.55.52.115])\n\tby dpdk.org (Postfix) with ESMTP id D103D6A89\n\tfor <dev@dpdk.org>; Tue, 30 Sep 2014 11:14:44 +0200 (CEST)", "from fmsmga003.fm.intel.com ([10.253.24.29])\n\tby fmsmga103.fm.intel.com with ESMTP; 30 Sep 2014 02:12:00 -0700", "from irsmsx101.ger.corp.intel.com ([163.33.3.153])\n\tby FMSMGA003.fm.intel.com with ESMTP; 30 Sep 2014 02:15:04 -0700", "from irsmsx107.ger.corp.intel.com (163.33.3.99) by\n\tIRSMSX101.ger.corp.intel.com (163.33.3.153) with Microsoft SMTP\n\tServer (TLS) id 14.3.195.1; Tue, 30 Sep 2014 10:21:20 +0100", "from irsmsx105.ger.corp.intel.com ([169.254.7.174]) by\n\tIRSMSX107.ger.corp.intel.com ([169.254.10.68]) with mapi id\n\t14.03.0195.001; Tue, 30 Sep 2014 10:21:19 +0100" ], "X-ExtLoop1": "1", "X-IronPort-AV": "E=Sophos;i=\"4.97,862,1389772800\"; d=\"scan'208\";a=\"393463225\"", "From": "\"Ananyev, Konstantin\" <konstantin.ananyev@intel.com>", "To": "\"Pattan, Reshma\" <reshma.pattan@intel.com>, \"De Lara Guarch, Pablo\"\n\t<pablo.de.lara.guarch@intel.com>, \"dev@dpdk.org\" <dev@dpdk.org>", "Thread-Topic": "[dpdk-dev] [PATCH v2] distributor_app: new sample app", "Thread-Index": "AQHP2AJemxe7phgTsUqwEI/Y1qWz+ZwTd5UAgAAbAeCABHF4gIAAFzyw///4W4CAATV3AIAAJaHw", "Date": "Tue, 30 Sep 2014 09:21:19 +0000", "Message-ID": "<2601191342CEEE43887BDE71AB9772582138CCA0@IRSMSX105.ger.corp.intel.com>", "References": "<1410869607-16842-1-git-send-email-reshma.pattan@intel.com>\n\t<1411568210-2555-1-git-send-email-reshma.pattan@intel.com>\n\t<E115CCD9D858EF4F90C690B0DCB4D89722626933@IRSMSX108.ger.corp.intel.com>\n\t<2601191342CEEE43887BDE71AB9772582137D8B1@IRSMSX104.ger.corp.intel.com>\n\t<3AEA2BF9852C6F48A459DA490692831FE20598@IRSMSX109.ger.corp.intel.com>\n\t<2601191342CEEE43887BDE71AB97725821387654@IRSMSX104.ger.corp.intel.com>\n\t<E115CCD9D858EF4F90C690B0DCB4D8972262823A@IRSMSX108.ger.corp.intel.com>\n\t<3AEA2BF9852C6F48A459DA490692831FE208B3@IRSMSX109.ger.corp.intel.com>", "In-Reply-To": "<3AEA2BF9852C6F48A459DA490692831FE208B3@IRSMSX109.ger.corp.intel.com>", "Accept-Language": "en-IE, en-US", "Content-Language": "en-US", "X-MS-Has-Attach": "", "X-MS-TNEF-Correlator": "", "x-originating-ip": "[163.33.239.181]", "Content-Type": "text/plain; charset=\"us-ascii\"", "Content-Transfer-Encoding": "quoted-printable", "MIME-Version": "1.0", "Subject": "Re: [dpdk-dev] [PATCH v2] distributor_app: new sample app", "X-BeenThere": "dev@dpdk.org", "X-Mailman-Version": "2.1.15", "Precedence": "list", "List-Id": "patches and discussions about DPDK <dev.dpdk.org>", "List-Unsubscribe": "<http://dpdk.org/ml/options/dev>,\n\t<mailto:dev-request@dpdk.org?subject=unsubscribe>", "List-Archive": "<http://dpdk.org/ml/archives/dev/>", "List-Post": "<mailto:dev@dpdk.org>", "List-Help": "<mailto:dev-request@dpdk.org?subject=help>", "List-Subscribe": "<http://dpdk.org/ml/listinfo/dev>,\n\t<mailto:dev-request@dpdk.org?subject=subscribe>", "Errors-To": "dev-bounces@dpdk.org", "Sender": "\"dev\" <dev-bounces@dpdk.org>" }, "addressed": null } ]