| Message ID | 20251003110201.1541183-1-bruce.richardson@intel.com (mailing list archive) |
|---|---|
| Headers |
Return-Path: <dev-bounces@dpdk.org> X-Original-To: patchwork@inbox.dpdk.org Delivered-To: patchwork@inbox.dpdk.org Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id DFEC64889E; Fri, 3 Oct 2025 13:02:32 +0200 (CEST) Received: from mails.dpdk.org (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id A6FD74028B; Fri, 3 Oct 2025 13:02:32 +0200 (CEST) Received: from mgamail.intel.com (mgamail.intel.com [198.175.65.13]) by mails.dpdk.org (Postfix) with ESMTP id 210424025F for <dev@dpdk.org>; Fri, 3 Oct 2025 13:02:29 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1759489350; x=1791025350; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=oiHHUhtMuyM7XHX+edBM6eUJhDfeZSIYt4HRLecjCIg=; b=bmPZofwGJQ8nVNIenC5uNwqKfygoomUUIJShzjnr2R2paShWdCJ+vc9M uYIqt1eyf7QohaA+bBwqjq/JrBswepqYtfUZjpTHvegPWJ/HZ0xeW751Q bzdQaZpM2VNmPBP8LY/Alf84bvDLT/OYXG/ln0XQeNw8f5FPPtXDCPBp+ fZoNFEFjN/LeBoK2du1to7HucsV6jWmnQ9uFXIetcYhoftXdKT8cS/+T9 OwERwjU39d1K2SIPKQPR7cMUQ2snfm/ycAlX+bACZSoW/16/mGoIpv3hm 0TBbLqbC5KfUQKDpM8k46MPxVZmmgl7ugTm0jPPIAXX5EjCiKQVqsxm/g w==; X-CSE-ConnectionGUID: iSb6L4bpQoeaowQ+lgrm9Q== X-CSE-MsgGUID: 5a+Zrer/QGyit2pyRLII3g== X-IronPort-AV: E=McAfee;i="6800,10657,11570"; a="72873471" X-IronPort-AV: E=Sophos;i="6.18,312,1751266800"; d="scan'208";a="72873471" Received: from orviesa007.jf.intel.com ([10.64.159.147]) by orvoesa105.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 03 Oct 2025 04:02:16 -0700 X-CSE-ConnectionGUID: 7I3VdHdjR0iY6K2LEcUxJg== X-CSE-MsgGUID: fNAIlHXqR8SwxfBHWrqUzQ== X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="6.18,312,1751266800"; d="scan'208";a="179077627" Received: from silpixa00401385.ir.intel.com ([10.20.224.226]) by orviesa007.jf.intel.com with ESMTP; 03 Oct 2025 04:02:14 -0700 From: Bruce Richardson <bruce.richardson@intel.com> To: dev@dpdk.org Cc: stephen@networkplumber.org, thomas@monjalon.net, Bruce Richardson <bruce.richardson@intel.com> Subject: [PATCH v3 0/7] remove deprecated queue stats Date: Fri, 3 Oct 2025 12:01:54 +0100 Message-ID: <20251003110201.1541183-1-bruce.richardson@intel.com> X-Mailer: git-send-email 2.48.1 In-Reply-To: <20250923141207.10403-1-bruce.richardson@intel.com> References: <20250923141207.10403-1-bruce.richardson@intel.com> MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK patches and discussions <dev.dpdk.org> List-Unsubscribe: <https://mails.dpdk.org/options/dev>, <mailto:dev-request@dpdk.org?subject=unsubscribe> List-Archive: <http://mails.dpdk.org/archives/dev/> List-Post: <mailto:dev@dpdk.org> List-Help: <mailto:dev-request@dpdk.org?subject=help> List-Subscribe: <https://mails.dpdk.org/listinfo/dev>, <mailto:dev-request@dpdk.org?subject=subscribe> Errors-To: dev-bounces@dpdk.org |
| Series |
remove deprecated queue stats
|
|
Message
Bruce Richardson
Oct. 3, 2025, 11:01 a.m. UTC
NOTE: as called out below, on apply patches 3-6 should be squashed. They
are separated out here for easier review!
Since DPDK 20.11 release, the use of queue stats inside the rte_eth_stats
structure has been deprecated with the intention to remove them. Sadly,
despite 5 years passing that has still not been done. This patchset
finally attempts to fix that situation and remove the queue stats fields.
The biggest complication here is the fact that, as part of the deprecation,
a new driver flag was added which caused ethdev to automatically add the
queue stats into xstats. While this was good, in that it allowed quick use
of xstats for getting queue statistics, it now causes lots of problems
because we have 35 drivers (by a rough count using grep) which rely on this
functionality to export their queue stats via xstats. This means that if we
drop the queue stats fields from the regular stats structure, then 35
drivers will lose *all* queue stats reporting functionality! This is not an
acceptable situation IMHO. [And in one patchset trying to change all 35
drivers to directly export via xstats is not feasible either, since adding
extra xstats can be very complicated at times.]
Therefore, we need to make changes that a) removes the queue fields from
the regular stats structure, while also b) continuing to allow the
individual driver stats_get functions return those stats to ethdev for
filling in. I tried a number of approaches here to find one that was most
feasible to implement for large numbers of drivers, so that either scripts
or AI assistants could automate a lot of the changes. The most feasible
approach I found was to define a DPDK-internal queue-stats-only structure,
which would be passed as a 3rd parameter to the drivers' stats_get
functions. When calling stats_get from ethdev, this third parameter would
be NULL, but for gathering xstats, it would be non-NULL for drivers which
have the RTE_ETH_DEV_AUTOFILL_QUEUE_XSTATS flag set.
For this set, I've separated out the driver changes from the ethdev changes
and the app changes, since the driver changes are so numerous, making the
rest hard to review. In order to ensure clean compilation and git history,
patches 3 through 6 should be squashed into a single one on apply.
Beyond these patches, more cleanup should/could be done:
* ensure that any drivers which don't set the AUTOFILL flag have the qstats
parameter set as __rte_unused.
* For those that do use the AUTOFILL flag, we should ensure that they
actually do fill in the stats, and not just set them to zero. (First two
patches here fix that for a couple of Intel drivers)
* then work to reduce the number of drivers which use the flag to set the
xstats, with a view to completely removing the queue stats from regular
stats functions.
---
V3:
* added deprecation of the queue stats mapping functions at as final
patch at the end.
Bruce Richardson (7):
net/ice: don't report empty queue xstats
net/ipn3ke: drop unsupported per-queue xstats
ethdev: remove queue stats from ethdev stats structure
drivers/net: update to remove queue stats from eth stats
app: remove queue stats from eth stats
doc: update docs for ethdev changes
ethdev: deprecate queue stats mapping functions
app/proc-info/main.c | 16 ---
app/test-pmd/cmdline.c | 57 --------
app/test-pmd/config.c | 36 -----
app/test/virtual_pmd.c | 3 +-
config/rte_config.h | 1 -
doc/guides/rel_notes/deprecation.rst | 13 +-
doc/guides/rel_notes/release_25_11.rst | 6 +
drivers/net/af_packet/rte_eth_af_packet.c | 13 +-
drivers/net/af_xdp/rte_eth_af_xdp.c | 35 +++--
drivers/net/ark/ark_ethdev.c | 36 +++--
drivers/net/ark/ark_ethdev_rx.c | 14 +-
drivers/net/ark/ark_ethdev_rx.h | 3 +-
drivers/net/ark/ark_ethdev_tx.c | 12 +-
drivers/net/ark/ark_ethdev_tx.h | 3 +-
drivers/net/atlantic/atl_ethdev.c | 19 +--
drivers/net/atlantic/atl_types.h | 1 +
drivers/net/avp/avp_ethdev.c | 20 ++-
drivers/net/axgbe/axgbe_ethdev.c | 22 +--
drivers/net/axgbe/axgbe_ethdev.h | 1 +
drivers/net/bnx2x/bnx2x_ethdev.c | 3 +-
drivers/net/bnxt/bnxt_reps.c | 14 +-
drivers/net/bnxt/bnxt_reps.h | 2 +-
drivers/net/bnxt/bnxt_stats.c | 133 ++++++++++--------
drivers/net/bnxt/bnxt_stats.h | 2 +-
drivers/net/bonding/rte_eth_bond_pmd.c | 14 +-
drivers/net/cnxk/cnxk_ethdev.h | 3 +-
drivers/net/cnxk/cnxk_rep.h | 3 +-
drivers/net/cnxk/cnxk_rep_ops.c | 15 +-
drivers/net/cnxk/cnxk_stats.c | 49 ++++---
drivers/net/cxgbe/cxgbe_ethdev.c | 3 +-
drivers/net/cxgbe/cxgbevf_ethdev.c | 3 +-
drivers/net/dpaa/dpaa_ethdev.c | 3 +-
drivers/net/dpaa2/dpaa2_ethdev.c | 28 ++--
drivers/net/ena/ena_ethdev.c | 46 +++---
drivers/net/enetc/enetc_ethdev.c | 4 +-
drivers/net/enetfec/enet_ethdev.c | 3 +-
drivers/net/enic/enic.h | 3 +-
drivers/net/enic/enic_ethdev.c | 4 +-
drivers/net/enic/enic_main.c | 3 +-
drivers/net/enic/enic_vf_representor.c | 3 +-
drivers/net/failsafe/failsafe_ether.c | 9 --
drivers/net/failsafe/failsafe_ops.c | 3 +-
drivers/net/gve/gve_ethdev.c | 4 +-
drivers/net/hinic/hinic_pmd_ethdev.c | 64 ++++++---
drivers/net/hns3/hns3_stats.c | 4 +-
drivers/net/hns3/hns3_stats.h | 3 +-
drivers/net/intel/cpfl/cpfl_ethdev.c | 3 +-
drivers/net/intel/e1000/em_ethdev.c | 7 +-
drivers/net/intel/e1000/igb_ethdev.c | 14 +-
drivers/net/intel/e1000/igc_ethdev.c | 33 +++--
drivers/net/intel/fm10k/fm10k_ethdev.c | 27 ++--
drivers/net/intel/i40e/i40e_ethdev.c | 5 +-
drivers/net/intel/i40e/i40e_vf_representor.c | 2 +-
drivers/net/intel/iavf/iavf_ethdev.c | 5 +-
drivers/net/intel/ice/ice_dcf_ethdev.c | 3 +-
drivers/net/intel/ice/ice_ethdev.c | 7 +-
drivers/net/intel/idpf/idpf_ethdev.c | 3 +-
drivers/net/intel/ipn3ke/ipn3ke_representor.c | 14 +-
drivers/net/intel/ixgbe/ixgbe_ethdev.c | 29 ++--
drivers/net/ionic/ionic_ethdev.c | 6 +-
drivers/net/ionic/ionic_lif.c | 35 +++--
drivers/net/ionic/ionic_lif.h | 3 +-
drivers/net/mana/mana.c | 15 +-
drivers/net/memif/rte_eth_memif.c | 15 +-
drivers/net/mlx4/mlx4.h | 3 +-
drivers/net/mlx4/mlx4_ethdev.c | 17 +--
drivers/net/mlx5/mlx5.h | 3 +-
drivers/net/mlx5/mlx5_stats.c | 17 +--
drivers/net/mvneta/mvneta_ethdev.c | 5 +-
drivers/net/mvpp2/mrvl_ethdev.c | 23 +--
drivers/net/netvsc/hn_ethdev.c | 17 +--
drivers/net/netvsc/hn_var.h | 3 +-
drivers/net/netvsc/hn_vf.c | 3 +-
drivers/net/nfp/flower/nfp_flower.c | 8 +-
.../net/nfp/flower/nfp_flower_representor.c | 19 +--
.../net/nfp/flower/nfp_flower_representor.h | 3 +
drivers/net/nfp/nfp_net_common.c | 46 +++---
drivers/net/nfp/nfp_net_common.h | 4 +-
drivers/net/ngbe/ngbe_ethdev.c | 53 +++----
drivers/net/ngbe/ngbe_ethdev_vf.c | 5 +-
drivers/net/ntnic/ntnic_ethdev.c | 22 +--
drivers/net/null/rte_eth_null.c | 15 +-
drivers/net/octeon_ep/otx_ep_ethdev.c | 17 ++-
drivers/net/octeontx/octeontx_ethdev.c | 3 +-
drivers/net/pcap/pcap_ethdev.c | 23 +--
drivers/net/pfe/pfe_ethdev.c | 3 +-
drivers/net/qede/qede_ethdev.c | 23 ++-
drivers/net/r8169/r8169_ethdev.c | 6 +-
drivers/net/ring/rte_eth_ring.c | 13 +-
drivers/net/rnp/rnp_ethdev.c | 15 +-
drivers/net/sfc/sfc_ethdev.c | 3 +-
drivers/net/sfc/sfc_repr.c | 3 +-
drivers/net/tap/rte_eth_tap.c | 23 +--
drivers/net/thunderx/nicvf_ethdev.c | 27 ++--
drivers/net/txgbe/txgbe_ethdev.c | 53 +++----
drivers/net/txgbe/txgbe_ethdev_vf.c | 5 +-
drivers/net/vhost/rte_eth_vhost.c | 26 ++--
drivers/net/virtio/virtio_ethdev.c | 23 +--
drivers/net/vmxnet3/vmxnet3_ethdev.c | 20 +--
drivers/net/xsc/xsc_ethdev.c | 19 +--
drivers/net/zxdh/zxdh_ethdev_ops.c | 39 ++---
drivers/net/zxdh/zxdh_ethdev_ops.h | 3 +-
lib/ethdev/ethdev_driver.h | 35 ++++-
lib/ethdev/ethdev_private.c | 27 ++++
lib/ethdev/ethdev_private.h | 5 +
lib/ethdev/rte_ethdev.c | 37 ++---
lib/ethdev/rte_ethdev.h | 13 +-
lib/ethdev/rte_ethdev_telemetry.c | 20 +--
108 files changed, 921 insertions(+), 794 deletions(-)
--
2.48.1
Comments
On Fri, 3 Oct 2025 12:01:54 +0100 Bruce Richardson <bruce.richardson@intel.com> wrote: > NOTE: as called out below, on apply patches 3-6 should be squashed. They > are separated out here for easier review! > > Since DPDK 20.11 release, the use of queue stats inside the rte_eth_stats > structure has been deprecated with the intention to remove them. Sadly, > despite 5 years passing that has still not been done. This patchset > finally attempts to fix that situation and remove the queue stats fields. > > The biggest complication here is the fact that, as part of the deprecation, > a new driver flag was added which caused ethdev to automatically add the > queue stats into xstats. While this was good, in that it allowed quick use > of xstats for getting queue statistics, it now causes lots of problems > because we have 35 drivers (by a rough count using grep) which rely on this > functionality to export their queue stats via xstats. This means that if we > drop the queue stats fields from the regular stats structure, then 35 > drivers will lose *all* queue stats reporting functionality! This is not an > acceptable situation IMHO. [And in one patchset trying to change all 35 > drivers to directly export via xstats is not feasible either, since adding > extra xstats can be very complicated at times.] > > Therefore, we need to make changes that a) removes the queue fields from > the regular stats structure, while also b) continuing to allow the > individual driver stats_get functions return those stats to ethdev for > filling in. I tried a number of approaches here to find one that was most > feasible to implement for large numbers of drivers, so that either scripts > or AI assistants could automate a lot of the changes. The most feasible > approach I found was to define a DPDK-internal queue-stats-only structure, > which would be passed as a 3rd parameter to the drivers' stats_get > functions. When calling stats_get from ethdev, this third parameter would > be NULL, but for gathering xstats, it would be non-NULL for drivers which > have the RTE_ETH_DEV_AUTOFILL_QUEUE_XSTATS flag set. > > For this set, I've separated out the driver changes from the ethdev changes > and the app changes, since the driver changes are so numerous, making the > rest hard to review. In order to ensure clean compilation and git history, > patches 3 through 6 should be squashed into a single one on apply. > > Beyond these patches, more cleanup should/could be done: > * ensure that any drivers which don't set the AUTOFILL flag have the qstats > parameter set as __rte_unused. > * For those that do use the AUTOFILL flag, we should ensure that they > actually do fill in the stats, and not just set them to zero. (First two > patches here fix that for a couple of Intel drivers) > * then work to reduce the number of drivers which use the flag to set the > xstats, with a view to completely removing the queue stats from regular > stats functions. > > --- > > V3: > * added deprecation of the queue stats mapping functions at as final > patch at the end. > > Bruce Richardson (7): > net/ice: don't report empty queue xstats > net/ipn3ke: drop unsupported per-queue xstats > ethdev: remove queue stats from ethdev stats structure > drivers/net: update to remove queue stats from eth stats > app: remove queue stats from eth stats > doc: update docs for ethdev changes > ethdev: deprecate queue stats mapping functions > > app/proc-info/main.c | 16 --- > app/test-pmd/cmdline.c | 57 -------- > app/test-pmd/config.c | 36 ----- > app/test/virtual_pmd.c | 3 +- > config/rte_config.h | 1 - > doc/guides/rel_notes/deprecation.rst | 13 +- > doc/guides/rel_notes/release_25_11.rst | 6 + > drivers/net/af_packet/rte_eth_af_packet.c | 13 +- > drivers/net/af_xdp/rte_eth_af_xdp.c | 35 +++-- > drivers/net/ark/ark_ethdev.c | 36 +++-- > drivers/net/ark/ark_ethdev_rx.c | 14 +- > drivers/net/ark/ark_ethdev_rx.h | 3 +- > drivers/net/ark/ark_ethdev_tx.c | 12 +- > drivers/net/ark/ark_ethdev_tx.h | 3 +- > drivers/net/atlantic/atl_ethdev.c | 19 +-- > drivers/net/atlantic/atl_types.h | 1 + > drivers/net/avp/avp_ethdev.c | 20 ++- > drivers/net/axgbe/axgbe_ethdev.c | 22 +-- > drivers/net/axgbe/axgbe_ethdev.h | 1 + > drivers/net/bnx2x/bnx2x_ethdev.c | 3 +- > drivers/net/bnxt/bnxt_reps.c | 14 +- > drivers/net/bnxt/bnxt_reps.h | 2 +- > drivers/net/bnxt/bnxt_stats.c | 133 ++++++++++-------- > drivers/net/bnxt/bnxt_stats.h | 2 +- > drivers/net/bonding/rte_eth_bond_pmd.c | 14 +- > drivers/net/cnxk/cnxk_ethdev.h | 3 +- > drivers/net/cnxk/cnxk_rep.h | 3 +- > drivers/net/cnxk/cnxk_rep_ops.c | 15 +- > drivers/net/cnxk/cnxk_stats.c | 49 ++++--- > drivers/net/cxgbe/cxgbe_ethdev.c | 3 +- > drivers/net/cxgbe/cxgbevf_ethdev.c | 3 +- > drivers/net/dpaa/dpaa_ethdev.c | 3 +- > drivers/net/dpaa2/dpaa2_ethdev.c | 28 ++-- > drivers/net/ena/ena_ethdev.c | 46 +++--- > drivers/net/enetc/enetc_ethdev.c | 4 +- > drivers/net/enetfec/enet_ethdev.c | 3 +- > drivers/net/enic/enic.h | 3 +- > drivers/net/enic/enic_ethdev.c | 4 +- > drivers/net/enic/enic_main.c | 3 +- > drivers/net/enic/enic_vf_representor.c | 3 +- > drivers/net/failsafe/failsafe_ether.c | 9 -- > drivers/net/failsafe/failsafe_ops.c | 3 +- > drivers/net/gve/gve_ethdev.c | 4 +- > drivers/net/hinic/hinic_pmd_ethdev.c | 64 ++++++--- > drivers/net/hns3/hns3_stats.c | 4 +- > drivers/net/hns3/hns3_stats.h | 3 +- > drivers/net/intel/cpfl/cpfl_ethdev.c | 3 +- > drivers/net/intel/e1000/em_ethdev.c | 7 +- > drivers/net/intel/e1000/igb_ethdev.c | 14 +- > drivers/net/intel/e1000/igc_ethdev.c | 33 +++-- > drivers/net/intel/fm10k/fm10k_ethdev.c | 27 ++-- > drivers/net/intel/i40e/i40e_ethdev.c | 5 +- > drivers/net/intel/i40e/i40e_vf_representor.c | 2 +- > drivers/net/intel/iavf/iavf_ethdev.c | 5 +- > drivers/net/intel/ice/ice_dcf_ethdev.c | 3 +- > drivers/net/intel/ice/ice_ethdev.c | 7 +- > drivers/net/intel/idpf/idpf_ethdev.c | 3 +- > drivers/net/intel/ipn3ke/ipn3ke_representor.c | 14 +- > drivers/net/intel/ixgbe/ixgbe_ethdev.c | 29 ++-- > drivers/net/ionic/ionic_ethdev.c | 6 +- > drivers/net/ionic/ionic_lif.c | 35 +++-- > drivers/net/ionic/ionic_lif.h | 3 +- > drivers/net/mana/mana.c | 15 +- > drivers/net/memif/rte_eth_memif.c | 15 +- > drivers/net/mlx4/mlx4.h | 3 +- > drivers/net/mlx4/mlx4_ethdev.c | 17 +-- > drivers/net/mlx5/mlx5.h | 3 +- > drivers/net/mlx5/mlx5_stats.c | 17 +-- > drivers/net/mvneta/mvneta_ethdev.c | 5 +- > drivers/net/mvpp2/mrvl_ethdev.c | 23 +-- > drivers/net/netvsc/hn_ethdev.c | 17 +-- > drivers/net/netvsc/hn_var.h | 3 +- > drivers/net/netvsc/hn_vf.c | 3 +- > drivers/net/nfp/flower/nfp_flower.c | 8 +- > .../net/nfp/flower/nfp_flower_representor.c | 19 +-- > .../net/nfp/flower/nfp_flower_representor.h | 3 + > drivers/net/nfp/nfp_net_common.c | 46 +++--- > drivers/net/nfp/nfp_net_common.h | 4 +- > drivers/net/ngbe/ngbe_ethdev.c | 53 +++---- > drivers/net/ngbe/ngbe_ethdev_vf.c | 5 +- > drivers/net/ntnic/ntnic_ethdev.c | 22 +-- > drivers/net/null/rte_eth_null.c | 15 +- > drivers/net/octeon_ep/otx_ep_ethdev.c | 17 ++- > drivers/net/octeontx/octeontx_ethdev.c | 3 +- > drivers/net/pcap/pcap_ethdev.c | 23 +-- > drivers/net/pfe/pfe_ethdev.c | 3 +- > drivers/net/qede/qede_ethdev.c | 23 ++- > drivers/net/r8169/r8169_ethdev.c | 6 +- > drivers/net/ring/rte_eth_ring.c | 13 +- > drivers/net/rnp/rnp_ethdev.c | 15 +- > drivers/net/sfc/sfc_ethdev.c | 3 +- > drivers/net/sfc/sfc_repr.c | 3 +- > drivers/net/tap/rte_eth_tap.c | 23 +-- > drivers/net/thunderx/nicvf_ethdev.c | 27 ++-- > drivers/net/txgbe/txgbe_ethdev.c | 53 +++---- > drivers/net/txgbe/txgbe_ethdev_vf.c | 5 +- > drivers/net/vhost/rte_eth_vhost.c | 26 ++-- > drivers/net/virtio/virtio_ethdev.c | 23 +-- > drivers/net/vmxnet3/vmxnet3_ethdev.c | 20 +-- > drivers/net/xsc/xsc_ethdev.c | 19 +-- > drivers/net/zxdh/zxdh_ethdev_ops.c | 39 ++--- > drivers/net/zxdh/zxdh_ethdev_ops.h | 3 +- > lib/ethdev/ethdev_driver.h | 35 ++++- > lib/ethdev/ethdev_private.c | 27 ++++ > lib/ethdev/ethdev_private.h | 5 + > lib/ethdev/rte_ethdev.c | 37 ++--- > lib/ethdev/rte_ethdev.h | 13 +- > lib/ethdev/rte_ethdev_telemetry.c | 20 +-- > 108 files changed, 921 insertions(+), 794 deletions(-) > > -- > 2.48.1 > LGTM Do you want me to fix the checkpatch stuff or do you want to send a new version.
On Wed, Oct 15, 2025 at 03:17:00PM -0700, Stephen Hemminger wrote: > On Fri, 3 Oct 2025 12:01:54 +0100 > Bruce Richardson <bruce.richardson@intel.com> wrote: > > > NOTE: as called out below, on apply patches 3-6 should be squashed. They > > are separated out here for easier review! > > > > Since DPDK 20.11 release, the use of queue stats inside the rte_eth_stats > > structure has been deprecated with the intention to remove them. Sadly, > > despite 5 years passing that has still not been done. This patchset > > finally attempts to fix that situation and remove the queue stats fields. > > > > The biggest complication here is the fact that, as part of the deprecation, > > a new driver flag was added which caused ethdev to automatically add the > > queue stats into xstats. While this was good, in that it allowed quick use > > of xstats for getting queue statistics, it now causes lots of problems > > because we have 35 drivers (by a rough count using grep) which rely on this > > functionality to export their queue stats via xstats. This means that if we > > drop the queue stats fields from the regular stats structure, then 35 > > drivers will lose *all* queue stats reporting functionality! This is not an > > acceptable situation IMHO. [And in one patchset trying to change all 35 > > drivers to directly export via xstats is not feasible either, since adding > > extra xstats can be very complicated at times.] > > > > Therefore, we need to make changes that a) removes the queue fields from > > the regular stats structure, while also b) continuing to allow the > > individual driver stats_get functions return those stats to ethdev for > > filling in. I tried a number of approaches here to find one that was most > > feasible to implement for large numbers of drivers, so that either scripts > > or AI assistants could automate a lot of the changes. The most feasible > > approach I found was to define a DPDK-internal queue-stats-only structure, > > which would be passed as a 3rd parameter to the drivers' stats_get > > functions. When calling stats_get from ethdev, this third parameter would > > be NULL, but for gathering xstats, it would be non-NULL for drivers which > > have the RTE_ETH_DEV_AUTOFILL_QUEUE_XSTATS flag set. > > > > For this set, I've separated out the driver changes from the ethdev changes > > and the app changes, since the driver changes are so numerous, making the > > rest hard to review. In order to ensure clean compilation and git history, > > patches 3 through 6 should be squashed into a single one on apply. > > > > Beyond these patches, more cleanup should/could be done: > > * ensure that any drivers which don't set the AUTOFILL flag have the qstats > > parameter set as __rte_unused. > > * For those that do use the AUTOFILL flag, we should ensure that they > > actually do fill in the stats, and not just set them to zero. (First two > > patches here fix that for a couple of Intel drivers) > > * then work to reduce the number of drivers which use the flag to set the > > xstats, with a view to completely removing the queue stats from regular > > stats functions. > > > > --- > > > > V3: > > * added deprecation of the queue stats mapping functions at as final > > patch at the end. > > > > Bruce Richardson (7): > > net/ice: don't report empty queue xstats > > net/ipn3ke: drop unsupported per-queue xstats > > ethdev: remove queue stats from ethdev stats structure > > drivers/net: update to remove queue stats from eth stats > > app: remove queue stats from eth stats > > doc: update docs for ethdev changes > > ethdev: deprecate queue stats mapping functions > > > > app/proc-info/main.c | 16 --- > > app/test-pmd/cmdline.c | 57 -------- > > app/test-pmd/config.c | 36 ----- > > app/test/virtual_pmd.c | 3 +- > > config/rte_config.h | 1 - > > doc/guides/rel_notes/deprecation.rst | 13 +- > > doc/guides/rel_notes/release_25_11.rst | 6 + > > drivers/net/af_packet/rte_eth_af_packet.c | 13 +- > > drivers/net/af_xdp/rte_eth_af_xdp.c | 35 +++-- > > drivers/net/ark/ark_ethdev.c | 36 +++-- > > drivers/net/ark/ark_ethdev_rx.c | 14 +- > > drivers/net/ark/ark_ethdev_rx.h | 3 +- > > drivers/net/ark/ark_ethdev_tx.c | 12 +- > > drivers/net/ark/ark_ethdev_tx.h | 3 +- > > drivers/net/atlantic/atl_ethdev.c | 19 +-- > > drivers/net/atlantic/atl_types.h | 1 + > > drivers/net/avp/avp_ethdev.c | 20 ++- > > drivers/net/axgbe/axgbe_ethdev.c | 22 +-- > > drivers/net/axgbe/axgbe_ethdev.h | 1 + > > drivers/net/bnx2x/bnx2x_ethdev.c | 3 +- > > drivers/net/bnxt/bnxt_reps.c | 14 +- > > drivers/net/bnxt/bnxt_reps.h | 2 +- > > drivers/net/bnxt/bnxt_stats.c | 133 ++++++++++-------- > > drivers/net/bnxt/bnxt_stats.h | 2 +- > > drivers/net/bonding/rte_eth_bond_pmd.c | 14 +- > > drivers/net/cnxk/cnxk_ethdev.h | 3 +- > > drivers/net/cnxk/cnxk_rep.h | 3 +- > > drivers/net/cnxk/cnxk_rep_ops.c | 15 +- > > drivers/net/cnxk/cnxk_stats.c | 49 ++++--- > > drivers/net/cxgbe/cxgbe_ethdev.c | 3 +- > > drivers/net/cxgbe/cxgbevf_ethdev.c | 3 +- > > drivers/net/dpaa/dpaa_ethdev.c | 3 +- > > drivers/net/dpaa2/dpaa2_ethdev.c | 28 ++-- > > drivers/net/ena/ena_ethdev.c | 46 +++--- > > drivers/net/enetc/enetc_ethdev.c | 4 +- > > drivers/net/enetfec/enet_ethdev.c | 3 +- > > drivers/net/enic/enic.h | 3 +- > > drivers/net/enic/enic_ethdev.c | 4 +- > > drivers/net/enic/enic_main.c | 3 +- > > drivers/net/enic/enic_vf_representor.c | 3 +- > > drivers/net/failsafe/failsafe_ether.c | 9 -- > > drivers/net/failsafe/failsafe_ops.c | 3 +- > > drivers/net/gve/gve_ethdev.c | 4 +- > > drivers/net/hinic/hinic_pmd_ethdev.c | 64 ++++++--- > > drivers/net/hns3/hns3_stats.c | 4 +- > > drivers/net/hns3/hns3_stats.h | 3 +- > > drivers/net/intel/cpfl/cpfl_ethdev.c | 3 +- > > drivers/net/intel/e1000/em_ethdev.c | 7 +- > > drivers/net/intel/e1000/igb_ethdev.c | 14 +- > > drivers/net/intel/e1000/igc_ethdev.c | 33 +++-- > > drivers/net/intel/fm10k/fm10k_ethdev.c | 27 ++-- > > drivers/net/intel/i40e/i40e_ethdev.c | 5 +- > > drivers/net/intel/i40e/i40e_vf_representor.c | 2 +- > > drivers/net/intel/iavf/iavf_ethdev.c | 5 +- > > drivers/net/intel/ice/ice_dcf_ethdev.c | 3 +- > > drivers/net/intel/ice/ice_ethdev.c | 7 +- > > drivers/net/intel/idpf/idpf_ethdev.c | 3 +- > > drivers/net/intel/ipn3ke/ipn3ke_representor.c | 14 +- > > drivers/net/intel/ixgbe/ixgbe_ethdev.c | 29 ++-- > > drivers/net/ionic/ionic_ethdev.c | 6 +- > > drivers/net/ionic/ionic_lif.c | 35 +++-- > > drivers/net/ionic/ionic_lif.h | 3 +- > > drivers/net/mana/mana.c | 15 +- > > drivers/net/memif/rte_eth_memif.c | 15 +- > > drivers/net/mlx4/mlx4.h | 3 +- > > drivers/net/mlx4/mlx4_ethdev.c | 17 +-- > > drivers/net/mlx5/mlx5.h | 3 +- > > drivers/net/mlx5/mlx5_stats.c | 17 +-- > > drivers/net/mvneta/mvneta_ethdev.c | 5 +- > > drivers/net/mvpp2/mrvl_ethdev.c | 23 +-- > > drivers/net/netvsc/hn_ethdev.c | 17 +-- > > drivers/net/netvsc/hn_var.h | 3 +- > > drivers/net/netvsc/hn_vf.c | 3 +- > > drivers/net/nfp/flower/nfp_flower.c | 8 +- > > .../net/nfp/flower/nfp_flower_representor.c | 19 +-- > > .../net/nfp/flower/nfp_flower_representor.h | 3 + > > drivers/net/nfp/nfp_net_common.c | 46 +++--- > > drivers/net/nfp/nfp_net_common.h | 4 +- > > drivers/net/ngbe/ngbe_ethdev.c | 53 +++---- > > drivers/net/ngbe/ngbe_ethdev_vf.c | 5 +- > > drivers/net/ntnic/ntnic_ethdev.c | 22 +-- > > drivers/net/null/rte_eth_null.c | 15 +- > > drivers/net/octeon_ep/otx_ep_ethdev.c | 17 ++- > > drivers/net/octeontx/octeontx_ethdev.c | 3 +- > > drivers/net/pcap/pcap_ethdev.c | 23 +-- > > drivers/net/pfe/pfe_ethdev.c | 3 +- > > drivers/net/qede/qede_ethdev.c | 23 ++- > > drivers/net/r8169/r8169_ethdev.c | 6 +- > > drivers/net/ring/rte_eth_ring.c | 13 +- > > drivers/net/rnp/rnp_ethdev.c | 15 +- > > drivers/net/sfc/sfc_ethdev.c | 3 +- > > drivers/net/sfc/sfc_repr.c | 3 +- > > drivers/net/tap/rte_eth_tap.c | 23 +-- > > drivers/net/thunderx/nicvf_ethdev.c | 27 ++-- > > drivers/net/txgbe/txgbe_ethdev.c | 53 +++---- > > drivers/net/txgbe/txgbe_ethdev_vf.c | 5 +- > > drivers/net/vhost/rte_eth_vhost.c | 26 ++-- > > drivers/net/virtio/virtio_ethdev.c | 23 +-- > > drivers/net/vmxnet3/vmxnet3_ethdev.c | 20 +-- > > drivers/net/xsc/xsc_ethdev.c | 19 +-- > > drivers/net/zxdh/zxdh_ethdev_ops.c | 39 ++--- > > drivers/net/zxdh/zxdh_ethdev_ops.h | 3 +- > > lib/ethdev/ethdev_driver.h | 35 ++++- > > lib/ethdev/ethdev_private.c | 27 ++++ > > lib/ethdev/ethdev_private.h | 5 + > > lib/ethdev/rte_ethdev.c | 37 ++--- > > lib/ethdev/rte_ethdev.h | 13 +- > > lib/ethdev/rte_ethdev_telemetry.c | 20 +-- > > 108 files changed, 921 insertions(+), 794 deletions(-) > > > > -- > > 2.48.1 > > > > > LGTM > Do you want me to fix the checkpatch stuff or do you want to send > a new version. If you can just fix on apply it's probably easiest, please.
On Fri, 3 Oct 2025 12:01:54 +0100 Bruce Richardson <bruce.richardson@intel.com> wrote: > NOTE: as called out below, on apply patches 3-6 should be squashed. They > are separated out here for easier review! > > Since DPDK 20.11 release, the use of queue stats inside the rte_eth_stats > structure has been deprecated with the intention to remove them. Sadly, > despite 5 years passing that has still not been done. This patchset > finally attempts to fix that situation and remove the queue stats fields. > > The biggest complication here is the fact that, as part of the deprecation, > a new driver flag was added which caused ethdev to automatically add the > queue stats into xstats. While this was good, in that it allowed quick use > of xstats for getting queue statistics, it now causes lots of problems > because we have 35 drivers (by a rough count using grep) which rely on this > functionality to export their queue stats via xstats. This means that if we > drop the queue stats fields from the regular stats structure, then 35 > drivers will lose *all* queue stats reporting functionality! This is not an > acceptable situation IMHO. [And in one patchset trying to change all 35 > drivers to directly export via xstats is not feasible either, since adding > extra xstats can be very complicated at times.] > > Therefore, we need to make changes that a) removes the queue fields from > the regular stats structure, while also b) continuing to allow the > individual driver stats_get functions return those stats to ethdev for > filling in. I tried a number of approaches here to find one that was most > feasible to implement for large numbers of drivers, so that either scripts > or AI assistants could automate a lot of the changes. The most feasible > approach I found was to define a DPDK-internal queue-stats-only structure, > which would be passed as a 3rd parameter to the drivers' stats_get > functions. When calling stats_get from ethdev, this third parameter would > be NULL, but for gathering xstats, it would be non-NULL for drivers which > have the RTE_ETH_DEV_AUTOFILL_QUEUE_XSTATS flag set. > > For this set, I've separated out the driver changes from the ethdev changes > and the app changes, since the driver changes are so numerous, making the > rest hard to review. In order to ensure clean compilation and git history, > patches 3 through 6 should be squashed into a single one on apply. > > Beyond these patches, more cleanup should/could be done: > * ensure that any drivers which don't set the AUTOFILL flag have the qstats > parameter set as __rte_unused. > * For those that do use the AUTOFILL flag, we should ensure that they > actually do fill in the stats, and not just set them to zero. (First two > patches here fix that for a couple of Intel drivers) > * then work to reduce the number of drivers which use the flag to set the > xstats, with a view to completely removing the queue stats from regular > stats functions. > > --- > > V3: > * added deprecation of the queue stats mapping functions at as final > patch at the end. > > Bruce Richardson (7): > net/ice: don't report empty queue xstats > net/ipn3ke: drop unsupported per-queue xstats > ethdev: remove queue stats from ethdev stats structure > drivers/net: update to remove queue stats from eth stats > app: remove queue stats from eth stats > doc: update docs for ethdev changes > ethdev: deprecate queue stats mapping functions > > app/proc-info/main.c | 16 --- > app/test-pmd/cmdline.c | 57 -------- > app/test-pmd/config.c | 36 ----- > app/test/virtual_pmd.c | 3 +- > config/rte_config.h | 1 - > doc/guides/rel_notes/deprecation.rst | 13 +- > doc/guides/rel_notes/release_25_11.rst | 6 + > drivers/net/af_packet/rte_eth_af_packet.c | 13 +- > drivers/net/af_xdp/rte_eth_af_xdp.c | 35 +++-- > drivers/net/ark/ark_ethdev.c | 36 +++-- > drivers/net/ark/ark_ethdev_rx.c | 14 +- > drivers/net/ark/ark_ethdev_rx.h | 3 +- > drivers/net/ark/ark_ethdev_tx.c | 12 +- > drivers/net/ark/ark_ethdev_tx.h | 3 +- > drivers/net/atlantic/atl_ethdev.c | 19 +-- > drivers/net/atlantic/atl_types.h | 1 + > drivers/net/avp/avp_ethdev.c | 20 ++- > drivers/net/axgbe/axgbe_ethdev.c | 22 +-- > drivers/net/axgbe/axgbe_ethdev.h | 1 + > drivers/net/bnx2x/bnx2x_ethdev.c | 3 +- > drivers/net/bnxt/bnxt_reps.c | 14 +- > drivers/net/bnxt/bnxt_reps.h | 2 +- > drivers/net/bnxt/bnxt_stats.c | 133 ++++++++++-------- > drivers/net/bnxt/bnxt_stats.h | 2 +- > drivers/net/bonding/rte_eth_bond_pmd.c | 14 +- > drivers/net/cnxk/cnxk_ethdev.h | 3 +- > drivers/net/cnxk/cnxk_rep.h | 3 +- > drivers/net/cnxk/cnxk_rep_ops.c | 15 +- > drivers/net/cnxk/cnxk_stats.c | 49 ++++--- > drivers/net/cxgbe/cxgbe_ethdev.c | 3 +- > drivers/net/cxgbe/cxgbevf_ethdev.c | 3 +- > drivers/net/dpaa/dpaa_ethdev.c | 3 +- > drivers/net/dpaa2/dpaa2_ethdev.c | 28 ++-- > drivers/net/ena/ena_ethdev.c | 46 +++--- > drivers/net/enetc/enetc_ethdev.c | 4 +- > drivers/net/enetfec/enet_ethdev.c | 3 +- > drivers/net/enic/enic.h | 3 +- > drivers/net/enic/enic_ethdev.c | 4 +- > drivers/net/enic/enic_main.c | 3 +- > drivers/net/enic/enic_vf_representor.c | 3 +- > drivers/net/failsafe/failsafe_ether.c | 9 -- > drivers/net/failsafe/failsafe_ops.c | 3 +- > drivers/net/gve/gve_ethdev.c | 4 +- > drivers/net/hinic/hinic_pmd_ethdev.c | 64 ++++++--- > drivers/net/hns3/hns3_stats.c | 4 +- > drivers/net/hns3/hns3_stats.h | 3 +- > drivers/net/intel/cpfl/cpfl_ethdev.c | 3 +- > drivers/net/intel/e1000/em_ethdev.c | 7 +- > drivers/net/intel/e1000/igb_ethdev.c | 14 +- > drivers/net/intel/e1000/igc_ethdev.c | 33 +++-- > drivers/net/intel/fm10k/fm10k_ethdev.c | 27 ++-- > drivers/net/intel/i40e/i40e_ethdev.c | 5 +- > drivers/net/intel/i40e/i40e_vf_representor.c | 2 +- > drivers/net/intel/iavf/iavf_ethdev.c | 5 +- > drivers/net/intel/ice/ice_dcf_ethdev.c | 3 +- > drivers/net/intel/ice/ice_ethdev.c | 7 +- > drivers/net/intel/idpf/idpf_ethdev.c | 3 +- > drivers/net/intel/ipn3ke/ipn3ke_representor.c | 14 +- > drivers/net/intel/ixgbe/ixgbe_ethdev.c | 29 ++-- > drivers/net/ionic/ionic_ethdev.c | 6 +- > drivers/net/ionic/ionic_lif.c | 35 +++-- > drivers/net/ionic/ionic_lif.h | 3 +- > drivers/net/mana/mana.c | 15 +- > drivers/net/memif/rte_eth_memif.c | 15 +- > drivers/net/mlx4/mlx4.h | 3 +- > drivers/net/mlx4/mlx4_ethdev.c | 17 +-- > drivers/net/mlx5/mlx5.h | 3 +- > drivers/net/mlx5/mlx5_stats.c | 17 +-- > drivers/net/mvneta/mvneta_ethdev.c | 5 +- > drivers/net/mvpp2/mrvl_ethdev.c | 23 +-- > drivers/net/netvsc/hn_ethdev.c | 17 +-- > drivers/net/netvsc/hn_var.h | 3 +- > drivers/net/netvsc/hn_vf.c | 3 +- > drivers/net/nfp/flower/nfp_flower.c | 8 +- > .../net/nfp/flower/nfp_flower_representor.c | 19 +-- > .../net/nfp/flower/nfp_flower_representor.h | 3 + > drivers/net/nfp/nfp_net_common.c | 46 +++--- > drivers/net/nfp/nfp_net_common.h | 4 +- > drivers/net/ngbe/ngbe_ethdev.c | 53 +++---- > drivers/net/ngbe/ngbe_ethdev_vf.c | 5 +- > drivers/net/ntnic/ntnic_ethdev.c | 22 +-- > drivers/net/null/rte_eth_null.c | 15 +- > drivers/net/octeon_ep/otx_ep_ethdev.c | 17 ++- > drivers/net/octeontx/octeontx_ethdev.c | 3 +- > drivers/net/pcap/pcap_ethdev.c | 23 +-- > drivers/net/pfe/pfe_ethdev.c | 3 +- > drivers/net/qede/qede_ethdev.c | 23 ++- > drivers/net/r8169/r8169_ethdev.c | 6 +- > drivers/net/ring/rte_eth_ring.c | 13 +- > drivers/net/rnp/rnp_ethdev.c | 15 +- > drivers/net/sfc/sfc_ethdev.c | 3 +- > drivers/net/sfc/sfc_repr.c | 3 +- > drivers/net/tap/rte_eth_tap.c | 23 +-- > drivers/net/thunderx/nicvf_ethdev.c | 27 ++-- > drivers/net/txgbe/txgbe_ethdev.c | 53 +++---- > drivers/net/txgbe/txgbe_ethdev_vf.c | 5 +- > drivers/net/vhost/rte_eth_vhost.c | 26 ++-- > drivers/net/virtio/virtio_ethdev.c | 23 +-- > drivers/net/vmxnet3/vmxnet3_ethdev.c | 20 +-- > drivers/net/xsc/xsc_ethdev.c | 19 +-- > drivers/net/zxdh/zxdh_ethdev_ops.c | 39 ++--- > drivers/net/zxdh/zxdh_ethdev_ops.h | 3 +- > lib/ethdev/ethdev_driver.h | 35 ++++- > lib/ethdev/ethdev_private.c | 27 ++++ > lib/ethdev/ethdev_private.h | 5 + > lib/ethdev/rte_ethdev.c | 37 ++--- > lib/ethdev/rte_ethdev.h | 13 +- > lib/ethdev/rte_ethdev_telemetry.c | 20 +-- > 108 files changed, 921 insertions(+), 794 deletions(-) > Added to next-net. Fixed the newer drivers and squashed a couple commits so that each commit builds.