From patchwork Thu Jan 18 23:41:20 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Adam Hassick X-Patchwork-Id: 135971 Return-Path: X-Original-To: patchwork@inbox.dpdk.org Delivered-To: patchwork@inbox.dpdk.org Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id 95935438F8; Fri, 19 Jan 2024 00:42:12 +0100 (CET) Received: from mails.dpdk.org (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id 8BA07402D7; Fri, 19 Jan 2024 00:42:12 +0100 (CET) Received: from mail-qk1-f182.google.com (mail-qk1-f182.google.com [209.85.222.182]) by mails.dpdk.org (Postfix) with ESMTP id E3D5D4026F for ; Fri, 19 Jan 2024 00:42:10 +0100 (CET) Received: by mail-qk1-f182.google.com with SMTP id af79cd13be357-783182d4a09so17302385a.2 for ; Thu, 18 Jan 2024 15:42:10 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=iol.unh.edu; s=unh-iol; t=1705621330; x=1706226130; darn=dpdk.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=LNtvfBP+uwvRgZuk/5lZhzA9GDxE9g+8am4ZY9TWBHY=; b=ISOPdtfmgWzXviecMXqIBdyJ/aNDCfZ7nQ3L6GVpHuHjq/cGoU2KKmCGR5yD++CLFL IqWDgnjpg4wJHi2p7YwJcZL/bD5FVZwRc9TojEIEN04eDu1QyA1cQgrbuNdgHqU+e/KF FVyC/5oQeYk2RQOsUYpScKpE54x0QyrImUCqM= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1705621330; x=1706226130; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=LNtvfBP+uwvRgZuk/5lZhzA9GDxE9g+8am4ZY9TWBHY=; b=dNJeVSIkmciCjXoXNXC0tAHEau4OlheMBqXbG0j7+z3YwTg38xzIbkpg+mMrMb881b s6CidYjJs5U3LAD3sFzuc/+xKTmW0fVC3sjWu4pWc7CT03M6ThMjprteFKS4dpAY4hW3 StrI4wCWIgdmRdsr4bwXN6iPrexizOrtIuC2Wec8YEBfhQvkQXmXkQlCtyhA0BGs25lg hkw+D87Sa1WpbibnHj3/R4ViyXgiWbnP60UgeGjyQdtqZhg97IunRSzZosYyemM0aEo/ QXJLtsoIypelUn3AqUC/EVfgsiu+mDP9X2a29uCFzRPh/JbPj8lxtvh59QGqDJuPlEL0 eMPQ== X-Gm-Message-State: AOJu0YxWXeX09WfiheknuLkRuflItkgaIvUYrHvHdhYe6/WzW6wz0o2d bB8fbr4SJMbBLSiFtd4gS1d8zZQMqme8EQWY8TzBAzbvzMUXRpyXTi66rlIZnOd3iz8RWFmxBub eTuYQcvCyS6KZ5DPZR0UaC5Ns6/r78qirlfBtF8F4+nY6YgXural6OrdoOAcxBY+nAKUcMGE3Cr dtL8V5/rgGeohibs47IhI9zfm2lg== X-Google-Smtp-Source: AGHT+IEKNNS7uk4ecJaOBLKEAH1rzDQfQnam+79mBsyI4nJA74Qy4mGobDYUGwgVhr7p67RWUuEbHQ== X-Received: by 2002:a05:620a:28c5:b0:783:6720:b969 with SMTP id l5-20020a05620a28c500b007836720b969mr662447qkp.143.1705621330230; Thu, 18 Jan 2024 15:42:10 -0800 (PST) Received: from pogmachine2.loudonlune.net ([216.212.51.182]) by smtp.gmail.com with ESMTPSA id oq6-20020a05620a610600b0078334ada139sm5625641qkn.7.2024.01.18.15.42.09 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 18 Jan 2024 15:42:10 -0800 (PST) From: Adam Hassick To: ci@dpdk.org Cc: Adam Hassick Subject: [PATCH v3 3/3] doc: Add documentation Date: Thu, 18 Jan 2024 18:41:20 -0500 Message-ID: <20240118234120.29256-4-ahassick@iol.unh.edu> X-Mailer: git-send-email 2.43.0 In-Reply-To: <20240118234120.29256-1-ahassick@iol.unh.edu> References: <20240118234120.29256-1-ahassick@iol.unh.edu> MIME-Version: 1.0 X-BeenThere: ci@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK CI discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: ci-bounces@dpdk.org Adds a new directory for storing documentation related to scripts and other utilities in the dpdk-ci repository. Adds a new document describing the create series artifact script. Signed-off-by: Adam Hassick --- doc/create_series_artifact.rst | 103 +++++++++++++++++++++++++++++++++ 1 file changed, 103 insertions(+) create mode 100644 doc/create_series_artifact.rst diff --git a/doc/create_series_artifact.rst b/doc/create_series_artifact.rst new file mode 100644 index 0000000..d3a60f9 --- /dev/null +++ b/doc/create_series_artifact.rst @@ -0,0 +1,103 @@ +.. SPDX-License-Identifier: BSD-3-Clause + Copyright(c) 2024 University of New Hampshire + +Create Series Artifact Script +============================= + +The create series artifact script is a small Python script for automatically applying patches and +storing the resulting source tree in an archive. +It gathers some information about the patches into a Java properties file for later use. +This script is meant to be used in the context of a CI pipeline. + + +Purpose and Context +------------------- + +A CI pipeline needs to make branching decisions when running on a patch or series depending on the +contents of the series or what tree it's being applied onto. +Tests running in a pipeline should run in parallel in order to shorten the time to report results. +This script helps to meet these needs by creating a portable archive containing the DPDK source +tree with the patch or series applied, and some metadata about that patch or series. +A CI pipeline and tests can use this information to make branching decisions based on the scope of +the patch. +Tests running in a CI pipeline can easily copy this archive and unpack it to obtain a clean +workspace, and this can be done independently of other tests. +At the Community Lab, the output archive from this script is stored as a Jenkins artifact. +This is the origin of the term "artifact" in the context of this script. +Jobs triggered downstream can then copy the artifact into their workspaces and extract them. +This script may also serve as a reference process for applying patches that new testing labs can +pull down and get started with. + + +Dependencies +------------ + +The script requires the following packages to be installed in the environment: + +* pygit2 +* pyyaml +* git-pw + +Additionally, the script depends on pw_maintainers_cli and patch-parser, which are included. +More information about configuring the script to find these dependencies can be found in the +configuration section. + + +Configuration +------------- + +A YAML file is used to store configuration options for this script. +The path to one of these configuration files is a required argument of the script. + +It is responsible for storing the following information: + +* Patchwork server, project, and user token +* URL to use when cloning the DPDK repository +* User information for Git +* Paths to dependent scripts and their configuration files + +When providing a server URL, make sure to include the API version in the url. +For example: ``https://patches.dpdk.org/api/1.3/`` + +An example configuration file is given in the config directory of this repository at +``config/artifacts.yml``. + +If you intend to execute this script from a different working directory, other than the tools +directory inside this repository, then you will need to change the paths in the example +configuration. + +Note that if you do not provide a patchwork token in the configuration file, then the optional +argument ``--pw-token`` on the command line becomes a required argument. +More detail on the command line interface can be found in the next section. + + +Running the Script +------------------ + +Once you have a configuration file set up, running the script is simple. + +The script expects two required arguments: + +* A path to a configuration file +* An API URL to a Series object in Patchwork + +The following block is an example of how to run the script: + +.. code-block:: console + + ./create_series_artifact.py ../config/artifacts.yml \ + https://patches.dpdk.org/api/1.3/series/12345 + +If you choose not to provide a patchwork token in the configuration file, then you must provide +it on the command line as follows: + +.. code-block:: console + + ./create_series_artifact.py --pw-token \ + ../config/artifacts.yml https://patches.dpdk.org/api/1.3/series/12345`` + +It is not required to delete the ``dpdk`` directory that the script creates, or any of the other +output files created by the script. +The properties and log file will be overwritten in the next execution. +If the ``dpdk`` directory is still present, then the script will retrieve and check out to the +remote head rather than clone the repository again.