[v11,01/12] app/graph: support application CLI framework

  From: Sunil Kumar Kori <skori@marvell.com>

Adds base framework to read a given .cli file as a command line
parameter "-s".

Example:
 # ./dpdk-graph -c 0xff -- -s ./app/graph/examples/dummy.cli

Each .cli file will contain commands to configure different module like
mempool, ethdev, lookup tables, graph etc. Command parsing is backed by
commandline library.

Each module needs to expose its supported commands & corresponding
callback functions to commandline library to get them parsed.

Signed-off-by: Sunil Kumar Kori <skori@marvell.com>
Signed-off-by: Rakesh Kudurumalla <rkudurumalla@marvell.com>
---
v10..v11
 - Add information to kill telnet session in user guide.
 - Merge l3fwd related information in a single section in user guide.
 - Fix spellings.

v9..v10
 - Add l3fwd_pcap.cli for pcap devices.
 - Update table generation mechanism user guide document

v8..v9:
 - Replace strcpy() to rte_strscpy()
 - Update release note.
 - Update user guide

v7..v8:
 - Fix klocwork issues.

v6..v7:
 - Fix FreeBSD build error.
 - Make route and neigh runtime configuration too.

v5..v6:
 - Fix build errors.
 - Fix checkpatch errors.
 - Fix individual patch build errors.

v4..v5:
 - Fix application exit issue.
 - Enable graph packet capture feature.
 - Fix graph coremask synchronization with eal coremask.
 - Update user guide.

https://patches.dpdk.org/project/dpdk/patch/20230919160455.1678716-1-skori@marvell.com/

v3..v4:
 - Use commandline library to parse command tokens.
 - Split to multiple smaller patches.
 - Make neigh and route as dynamic database.
 - add ethdev and graph stats command via telnet.
 - Update user guide.

https://patches.dpdk.org/project/dpdk/patch/20230908104907.4060511-1-skori@marvell.com/

 MAINTAINERS                            |   7 ++
 app/graph/cli.c                        | 115 ++++++++++++++++++++++
 app/graph/cli.h                        |  32 +++++++
 app/graph/main.c                       | 128 +++++++++++++++++++++++++
 app/graph/meson.build                  |  15 +++
 app/graph/module_api.h                 |  16 ++++
 app/meson.build                        |   1 +
 doc/guides/rel_notes/release_23_11.rst |   7 ++
 doc/guides/tools/graph.rst             |  75 +++++++++++++++
 doc/guides/tools/index.rst             |   1 +
 10 files changed, 397 insertions(+)
 create mode 100644 app/graph/cli.c
 create mode 100644 app/graph/cli.h
 create mode 100644 app/graph/main.c
 create mode 100644 app/graph/meson.build
 create mode 100644 app/graph/module_api.h
 create mode 100644 doc/guides/tools/graph.rst
  

Acked-By: Nithin Dabilpuram <ndabilpuram@marvell.com>

On Fri, Oct 20, 2023 at 9:37 AM <skori@marvell.com> wrote:
>
> From: Sunil Kumar Kori <skori@marvell.com>
>
> Adds base framework to read a given .cli file as a command line
> parameter "-s".
>
> Example:
>  # ./dpdk-graph -c 0xff -- -s ./app/graph/examples/dummy.cli
>
> Each .cli file will contain commands to configure different module like
> mempool, ethdev, lookup tables, graph etc. Command parsing is backed by
> commandline library.
>
> Each module needs to expose its supported commands & corresponding
> callback functions to commandline library to get them parsed.
>
> Signed-off-by: Sunil Kumar Kori <skori@marvell.com>
> Signed-off-by: Rakesh Kudurumalla <rkudurumalla@marvell.com>
> ---
> v10..v11
>  - Add information to kill telnet session in user guide.
>  - Merge l3fwd related information in a single section in user guide.
>  - Fix spellings.
>
> v9..v10
>  - Add l3fwd_pcap.cli for pcap devices.
>  - Update table generation mechanism user guide document
>
> v8..v9:
>  - Replace strcpy() to rte_strscpy()
>  - Update release note.
>  - Update user guide
>
> v7..v8:
>  - Fix klocwork issues.
>
> v6..v7:
>  - Fix FreeBSD build error.
>  - Make route and neigh runtime configuration too.
>
> v5..v6:
>  - Fix build errors.
>  - Fix checkpatch errors.
>  - Fix individual patch build errors.
>
> v4..v5:
>  - Fix application exit issue.
>  - Enable graph packet capture feature.
>  - Fix graph coremask synchronization with eal coremask.
>  - Update user guide.
>
> https://patches.dpdk.org/project/dpdk/patch/20230919160455.1678716-1-skori@marvell.com/
>
> v3..v4:
>  - Use commandline library to parse command tokens.
>  - Split to multiple smaller patches.
>  - Make neigh and route as dynamic database.
>  - add ethdev and graph stats command via telnet.
>  - Update user guide.
>
> https://patches.dpdk.org/project/dpdk/patch/20230908104907.4060511-1-skori@marvell.com/
>
>  MAINTAINERS                            |   7 ++
>  app/graph/cli.c                        | 115 ++++++++++++++++++++++
>  app/graph/cli.h                        |  32 +++++++
>  app/graph/main.c                       | 128 +++++++++++++++++++++++++
>  app/graph/meson.build                  |  15 +++
>  app/graph/module_api.h                 |  16 ++++
>  app/meson.build                        |   1 +
>  doc/guides/rel_notes/release_23_11.rst |   7 ++
>  doc/guides/tools/graph.rst             |  75 +++++++++++++++
>  doc/guides/tools/index.rst             |   1 +
>  10 files changed, 397 insertions(+)
>  create mode 100644 app/graph/cli.c
>  create mode 100644 app/graph/cli.h
>  create mode 100644 app/graph/main.c
>  create mode 100644 app/graph/meson.build
>  create mode 100644 app/graph/module_api.h
>  create mode 100644 doc/guides/tools/graph.rst
>
> diff --git a/MAINTAINERS b/MAINTAINERS
> index 4083658697..88a71d5455 100644
> --- a/MAINTAINERS
> +++ b/MAINTAINERS
> @@ -1818,6 +1818,13 @@ F: dts/
>  F: devtools/dts-check-format.sh
>  F: doc/guides/tools/dts.rst
>
> +Graph application
> +M: Sunil Kumar Kori <skori@marvell.com>
> +M: Rakesh Kudurumalla <rkudurumalla@marvell.com>
> +F: app/graph/
> +F: doc/guides/tools/graph.rst
> +F: doc/guides/tools/img/graph-usecase-l3fwd.svg
> +
>
>  Other Example Applications
>  --------------------------
> diff --git a/app/graph/cli.c b/app/graph/cli.c
> new file mode 100644
> index 0000000000..df4f8fcbb8
> --- /dev/null
> +++ b/app/graph/cli.c
> @@ -0,0 +1,115 @@
> +/* SPDX-License-Identifier: BSD-3-Clause
> + * Copyright(c) 2023 Marvell.
> + */
> +
> +#include <errno.h>
> +#include <stdio.h>
> +#include <stdint.h>
> +#include <stdlib.h>
> +#include <string.h>
> +
> +#include <cmdline_parse.h>
> +#include <cmdline_parse_num.h>
> +#include <cmdline_parse_string.h>
> +#include <cmdline_socket.h>
> +#include <rte_common.h>
> +
> +#include "module_api.h"
> +
> +#define CMD_MAX_TOKENS 256
> +#define MAX_LINE_SIZE 2048
> +
> +cmdline_parse_ctx_t modules_ctx[] = {
> +       NULL,
> +};
> +
> +static struct cmdline *cl;
> +
> +static int
> +is_comment(char *in)
> +{
> +       if ((strlen(in) && index("!#%;", in[0])) ||
> +               (strncmp(in, "//", 2) == 0) ||
> +               (strncmp(in, "--", 2) == 0))
> +               return 1;
> +
> +       return 0;
> +}
> +
> +void
> +cli_init(void)
> +{
> +       cl = cmdline_stdin_new(modules_ctx, "");
> +}
> +
> +void
> +cli_exit(void)
> +{
> +       cmdline_stdin_exit(cl);
> +}
> +
> +void
> +cli_process(char *in, char *out, size_t out_size, __rte_unused void *obj)
> +{
> +       int rc;
> +
> +       if (is_comment(in))
> +               return;
> +
> +       rc = cmdline_parse(cl, in);
> +       if (rc == CMDLINE_PARSE_AMBIGUOUS)
> +               snprintf(out, out_size, MSG_CMD_FAIL, "Ambiguous command");
> +       else if (rc == CMDLINE_PARSE_NOMATCH)
> +               snprintf(out, out_size, MSG_CMD_FAIL, "Command mismatch");
> +       else if (rc == CMDLINE_PARSE_BAD_ARGS)
> +               snprintf(out, out_size, MSG_CMD_FAIL, "Bad arguments");
> +
> +       return;
> +
> +}
> +
> +int
> +cli_script_process(const char *file_name, size_t msg_in_len_max, size_t msg_out_len_max, void *obj)
> +{
> +       char *msg_in = NULL, *msg_out = NULL;
> +       int rc = -EINVAL;
> +       FILE *f = NULL;
> +
> +       /* Check input arguments */
> +       if ((file_name == NULL) || (strlen(file_name) == 0) || (msg_in_len_max == 0) ||
> +           (msg_out_len_max == 0))
> +               return rc;
> +
> +       msg_in = malloc(msg_in_len_max + 1);
> +       msg_out = malloc(msg_out_len_max + 1);
> +       if ((msg_in == NULL) || (msg_out == NULL)) {
> +               rc = -ENOMEM;
> +               goto exit;
> +       }
> +
> +       /* Open input file */
> +       f = fopen(file_name, "r");
> +       if (f == NULL) {
> +               rc = -EIO;
> +               goto exit;
> +       }
> +
> +       /* Read file */
> +       while (fgets(msg_in, msg_in_len_max, f) != NULL) {
> +               msg_out[0] = 0;
> +
> +               cli_process(msg_in, msg_out, msg_out_len_max, obj);
> +
> +               if (strlen(msg_out))
> +                       printf("%s", msg_out);
> +       }
> +
> +       /* Close file */
> +       fclose(f);
> +       rc = 0;
> +
> +exit:
> +       free(msg_out);
> +       free(msg_in);
> +       return rc;
> +}
> diff --git a/app/graph/cli.h b/app/graph/cli.h
> new file mode 100644
> index 0000000000..652f948352
> --- /dev/null
> +++ b/app/graph/cli.h
> @@ -0,0 +1,32 @@
> +/* SPDX-License-Identifier: BSD-3-Clause
> + * Copyright(c) 2023 Marvell.
> + */
> +
> +#ifndef APP_GRAPH_CLI_H
> +#define APP_GRAPH_CLI_H
> +
> +/* Macros */
> +#define MSG_OUT_OF_MEMORY   "Not enough memory.\n"
> +#define MSG_CMD_UNKNOWN     "Unknown command \"%s\".\n"
> +#define MSG_CMD_UNIMPLEM    "Command \"%s\" not implemented.\n"
> +#define MSG_ARG_NOT_ENOUGH  "Not enough arguments for command \"%s\".\n"
> +#define MSG_ARG_TOO_MANY    "Too many arguments for command \"%s\".\n"
> +#define MSG_ARG_MISMATCH    "Wrong number of arguments for command \"%s\".\n"
> +#define MSG_ARG_NOT_FOUND   "Argument \"%s\" not found.\n"
> +#define MSG_ARG_INVALID     "Invalid value for argument \"%s\".\n"
> +#define MSG_FILE_ERR        "Error in file \"%s\" at line %u.\n"
> +#define MSG_FILE_NOT_ENOUGH "Not enough rules in file \"%s\".\n"
> +#define MSG_CMD_FAIL        "Command \"%s\" failed.\n"
> +
> +#define APP_CLI_CMD_NAME_SIZE  64
> +
> +void cli_init(void);
> +
> +void cli_exit(void);
> +
> +void cli_process(char *in, char *out, size_t out_size, void *arg);
> +
> +int cli_script_process(const char *file_name, size_t msg_in_len_max, size_t msg_out_len_max,
> +                      void *arg);
> +
> +#endif
> diff --git a/app/graph/main.c b/app/graph/main.c
> new file mode 100644
> index 0000000000..734a94444e
> --- /dev/null
> +++ b/app/graph/main.c
> @@ -0,0 +1,128 @@
> +/* SPDX-License-Identifier: BSD-3-Clause
> + * Copyright(c) 2023 Marvell.
> + */
> +
> +#include <fcntl.h>
> +#include <getopt.h>
> +#include <signal.h>
> +#include <stdio.h>
> +#include <stdlib.h>
> +#include <string.h>
> +#include <sys/select.h>
> +#include <unistd.h>
> +
> +#include <rte_eal.h>
> +#include <rte_launch.h>
> +
> +#include "module_api.h"
> +
> +volatile bool force_quit;
> +
> +static const char usage[] = "%s EAL_ARGS -- -s SCRIPT "
> +                           "[--help]\n";
> +
> +static struct app_params {
> +       char *script_name;
> +} app = {
> +       .script_name = NULL,
> +};
> +
> +static void
> +signal_handler(int signum)
> +{
> +       if (signum == SIGINT || signum == SIGTERM) {
> +               printf("\n\nSignal %d received, preparing to exit...\n", signum);
> +               force_quit = true;
> +       }
> +}
> +
> +static int
> +app_args_parse(int argc, char **argv)
> +{
> +       struct option lgopts[] = {
> +               {"help", 0, 0, 'H'},
> +       };
> +       int s_present, n_args, i;
> +       char *app_name = argv[0];
> +       int opt, option_index;
> +
> +       /* Skip EAL input args */
> +       n_args = argc;
> +       for (i = 0; i < n_args; i++)
> +               if (strcmp(argv[i], "--") == 0) {
> +                       argc -= i;
> +                       argv += i;
> +                       break;
> +               }
> +
> +       if (i == n_args)
> +               return 0;
> +
> +       /* Parse args */
> +       s_present = 0;
> +
> +       while ((opt = getopt_long(argc, argv, "s:", lgopts, &option_index)) != EOF) {
> +               switch (opt) {
> +               case 's':
> +                       if (s_present) {
> +                               printf("Error: Multiple -s arguments\n");
> +                               return -1;
> +                       }
> +                       s_present = 1;
> +
> +                       if (!strlen(optarg)) {
> +                               printf("Error: Argument for -s not provided\n");
> +                               return -1;
> +                       }
> +
> +                       app.script_name = strdup(optarg);
> +                       if (app.script_name == NULL) {
> +                               printf("Error: Not enough memory\n");
> +                               return -1;
> +                       }
> +                       break;
> +
> +               case 'H':
> +               default:
> +                       printf(usage, app_name);
> +                       return -1;
> +               }
> +       }
> +       optind = 1; /* reset getopt lib */
> +
> +       return 0;
> +}
> +
> +int
> +main(int argc, char **argv)
> +{
> +       int rc;
> +
> +       /* Parse application arguments */
> +       rc = app_args_parse(argc, argv);
> +       if (rc < 0)
> +               return rc;
> +
> +       /* EAL */
> +       rc = rte_eal_init(argc, argv);
> +       if (rc < 0) {
> +               printf("Error: EAL initialization failed (%d)\n", rc);
> +               return rc;
> +       };
> +
> +       force_quit = false;
> +       signal(SIGINT, signal_handler);
> +       signal(SIGTERM, signal_handler);
> +
> +       cli_init();
> +
> +       /* Script */
> +       if (app.script_name) {
> +               cli_script_process(app.script_name, 0,
> +                       0, NULL);
> +       }
> +
> +       cli_exit();
> +       rte_eal_cleanup();
> +       return 0;
> +}
> diff --git a/app/graph/meson.build b/app/graph/meson.build
> new file mode 100644
> index 0000000000..ed33a04476
> --- /dev/null
> +++ b/app/graph/meson.build
> @@ -0,0 +1,15 @@
> +# SPDX-License-Identifier: BSD-3-Clause
> +# Copyright(c) 2023 Marvell.
> +
> +# override default name to drop the hyphen
> +name = 'graph'
> +build = cc.has_header('sys/epoll.h')
> +if not build
> +    subdir_done()
> +endif
> +
> +deps += ['graph', 'eal', 'lpm', 'ethdev', 'node', 'cmdline']
> +sources = files(
> +        'cli.c',
> +        'main.c',
> +)
> diff --git a/app/graph/module_api.h b/app/graph/module_api.h
> new file mode 100644
> index 0000000000..372aeae7e3
> --- /dev/null
> +++ b/app/graph/module_api.h
> @@ -0,0 +1,16 @@
> +/* SPDX-License-Identifier: BSD-3-Clause
> + * Copyright(c) 2023 Marvell.
> + */
> +
> +#ifndef APP_GRAPH_MODULE_API_H
> +#define APP_GRAPH_MODULE_API_H
> +
> +#include <stdint.h>
> +#include <stdbool.h>
> +#include "cli.h"
> +/*
> + * Externs
> + */
> +extern volatile bool force_quit;
> +
> +#endif
> diff --git a/app/meson.build b/app/meson.build
> index e4bf5c531c..728c936383 100644
> --- a/app/meson.build
> +++ b/app/meson.build
> @@ -17,6 +17,7 @@ endif
>  apps = [
>          'dumpcap',
>          'pdump',
> +        'graph',
>          'proc-info',
>          'test-acl',
>          'test-bbdev',
> diff --git a/doc/guides/rel_notes/release_23_11.rst b/doc/guides/rel_notes/release_23_11.rst
> index 0a6fc76a9d..f43b416565 100644
> --- a/doc/guides/rel_notes/release_23_11.rst
> +++ b/doc/guides/rel_notes/release_23_11.rst
> @@ -243,6 +243,13 @@ New Features
>    Added dispatcher library which purpose is to help decouple different
>    parts (modules) of an eventdev-based application.
>
> +* **Added CLI based graph application.**
> +
> +  Added CLI based graph application which exercises on different usecases.
> +  Application provides a framework so that each usecase can be added via CLI
> +  file. Each CLI will further be translated into a graph representing user's
> +  required usecase.
> +
>
>  Removed Items
>  -------------
> diff --git a/doc/guides/tools/graph.rst b/doc/guides/tools/graph.rst
> new file mode 100644
> index 0000000000..cb005e7856
> --- /dev/null
> +++ b/doc/guides/tools/graph.rst
> @@ -0,0 +1,75 @@
> +..  SPDX-License-Identifier: BSD-3-Clause
> +    Copyright(c) 2023 Marvell.
> +
> +dpdk-graph Application
> +======================
> +
> +The ``dpdk-graph`` tool is a Data Plane Development Kit (DPDK)
> +application that allows exercising various graph use cases.
> +This application has a generic framework to add new graph based use cases to
> +verify functionality. Each use case is defined as a ``<usecase>.cli`` file.
> +Based on the input file, application creates a graph to cater the use case.
> +
> +Also this application framework can be used by other graph based applications.
> +
> +Running the Application
> +-----------------------
> +
> +The application has a number of command line options which can be provided in
> +following syntax
> +
> +.. code-block:: console
> +
> +   dpdk-graph [EAL Options] -- [application options]
> +
> +EAL Options
> +~~~~~~~~~~~
> +
> +Following are the EAL command-line options that can be used in conjunction
> +with the ``dpdk-graph`` application.
> +See the DPDK Getting Started Guides for more information on these options.
> +
> +*   ``-c <COREMASK>`` or ``-l <CORELIST>``
> +
> +        Set the hexadecimal bit mask of the cores to run on. The CORELIST is a
> +        list of cores to be used.
> +
> +Application Options
> +~~~~~~~~~~~~~~~~~~~
> +
> +Following are the application command-line options:
> +
> +* ``-s``
> +
> +        Script name with absolute path which specifies the use case. It is
> +        a mandatory parameter which will be used to create desired graph
> +        for a given use case.
> +
> +* ``--help``
> +
> +       Dumps application usage
> +
> +Supported CLI commands
> +----------------------
> +
> +This section provides details on commands which can be used in ``<usecase>.cli``
> +file to express the requested use case configuration.
> +
> +.. table:: Exposed CLIs
> +   :widths: auto
> +
> +   +--------------------------------------+-----------------------------------+---------+----------+
> +   |               Command                |             Description           | Dynamic | Optional |
> +   +======================================+===================================+=========+==========+
> +   |                                      |                                   |         |          |
> +   +--------------------------------------+-----------------------------------+---------+----------+
> +
> +Runtime configuration
> +---------------------
> +
> +
> +Created graph for use case
> +--------------------------
> +
> +On the successful execution of ``<usecase>.cli`` file, corresponding graph will be created.
> +This section mentions the created graph for each use case.
> diff --git a/doc/guides/tools/index.rst b/doc/guides/tools/index.rst
> index f2afb1fcc5..4f4dc8b518 100644
> --- a/doc/guides/tools/index.rst
> +++ b/doc/guides/tools/index.rst
> @@ -23,4 +23,5 @@ DPDK Tools User Guides
>      testeventdev
>      testregex
>      testmldev
> +    graph
>      dts
> --
> 2.25.1
>