doc: update Linux core isolation guide

Message ID 20220502072414.4643-1-pbhagavatula@marvell.com (mailing list archive)
State Superseded, archived
Delegated to: Thomas Monjalon
Headers
Series doc: update Linux core isolation guide |

Checks

Context Check Description
ci/checkpatch success coding style OK
ci/Intel-compilation success Compilation OK
ci/intel-Testing success Testing PASS
ci/github-robot: build success github build: passed
ci/iol-mellanox-Performance success Performance Testing PASS
ci/iol-intel-Performance success Performance Testing PASS
ci/iol-intel-Functional success Functional Testing PASS
ci/iol-aarch64-unit-testing success Testing PASS
ci/iol-aarch64-compile-testing success Testing PASS
ci/iol-x86_64-unit-testing success Testing PASS
ci/iol-abi-testing success Testing PASS
ci/iol-x86_64-compile-testing success Testing PASS

Commit Message

Pavan Nikhilesh Bhagavatula May 2, 2022, 7:24 a.m. UTC
  Update Linux core isolation guide to include isolation from
timers, rcu processing and IRQs.

Signed-off-by: Pavan Nikhilesh <pbhagavatula@marvell.com>
---
 doc/guides/linux_gsg/enable_func.rst | 10 +++++-----
 1 file changed, 5 insertions(+), 5 deletions(-)
  

Comments

Stephen Hemminger May 2, 2022, 8:17 p.m. UTC | #1
On Mon, 2 May 2022 12:54:14 +0530
Pavan Nikhilesh <pbhagavatula@marvell.com> wrote:

> Update Linux core isolation guide to include isolation from
> timers, rcu processing and IRQs.
> 
> Signed-off-by: Pavan Nikhilesh <pbhagavatula@marvell.com>
> ---
>  doc/guides/linux_gsg/enable_func.rst | 10 +++++-----
>  1 file changed, 5 insertions(+), 5 deletions(-)
> 
> diff --git a/doc/guides/linux_gsg/enable_func.rst
> b/doc/guides/linux_gsg/enable_func.rst index 1df3ab0255..f567c713e6
> 100644 --- a/doc/guides/linux_gsg/enable_func.rst
> +++ b/doc/guides/linux_gsg/enable_func.rst
> @@ -90,16 +90,16 @@ Using Linux Core Isolation to Reduce Context
> Switches -----------------------------------------------------
>  
>  While the threads used by a DPDK application are pinned to logical
> cores on the system, -it is possible for the Linux scheduler to run
> other tasks on those cores also. -To help prevent additional
> workloads from running on those cores, -it is possible to use the
> ``isolcpus`` Linux kernel parameter to isolate them from the general
> Linux scheduler. +it is possible for the Linux scheduler to run other
> tasks on those cores. +To help prevent additional workloads, timers,
> rcu processing and IRQs from running on those cores, it is possible
> to use +the Linux kernel parameters ``isolcpus``, ``nohz_full``,
> ``irqaffinity`` to isolate them from the general Linux scheduler
> tasks. -For example, if DPDK applications are to run on logical cores
> 2, 4 and 6, +For example, if a given CPU has 0-7 cores and DPDK
> applications are to run on logical cores 2, 4 and 6, the following
> should be added to the kernel parameter list: .. code-block:: console 
> -    isolcpus=2,4,6
> +    isolcpus=2,4,6 nohz_full=2,4,6 irqaffinity=0,1,3,5,7
>  
>  .. _High_Precision_Event_Timer:
>  

Also cgroups is often a better solution than the kernel command line.
There are also caveats around some of these settings so worth reading
the
kernel documentation.
  
Pavan Nikhilesh Bhagavatula May 10, 2022, 3:50 p.m. UTC | #2
> On Mon, 2 May 2022 12:54:14 +0530
> Pavan Nikhilesh <pbhagavatula@marvell.com> wrote:
> 
> > Update Linux core isolation guide to include isolation from
> > timers, rcu processing and IRQs.
> >
> > Signed-off-by: Pavan Nikhilesh <pbhagavatula@marvell.com>
> > ---
> >  doc/guides/linux_gsg/enable_func.rst | 10 +++++-----
> >  1 file changed, 5 insertions(+), 5 deletions(-)
> >
> > diff --git a/doc/guides/linux_gsg/enable_func.rst
> > b/doc/guides/linux_gsg/enable_func.rst index 1df3ab0255..f567c713e6
> > 100644 --- a/doc/guides/linux_gsg/enable_func.rst
> > +++ b/doc/guides/linux_gsg/enable_func.rst
> > @@ -90,16 +90,16 @@ Using Linux Core Isolation to Reduce Context
> > Switches -----------------------------------------------------
> >
> >  While the threads used by a DPDK application are pinned to logical
> > cores on the system, -it is possible for the Linux scheduler to run
> > other tasks on those cores also. -To help prevent additional
> > workloads from running on those cores, -it is possible to use the
> > ``isolcpus`` Linux kernel parameter to isolate them from the general
> > Linux scheduler. +it is possible for the Linux scheduler to run other
> > tasks on those cores. +To help prevent additional workloads, timers,
> > rcu processing and IRQs from running on those cores, it is possible
> > to use +the Linux kernel parameters ``isolcpus``, ``nohz_full``,
> > ``irqaffinity`` to isolate them from the general Linux scheduler
> > tasks. -For example, if DPDK applications are to run on logical cores
> > 2, 4 and 6, +For example, if a given CPU has 0-7 cores and DPDK
> > applications are to run on logical cores 2, 4 and 6, the following
> > should be added to the kernel parameter list: .. code-block:: console
> > -    isolcpus=2,4,6
> > +    isolcpus=2,4,6 nohz_full=2,4,6 irqaffinity=0,1,3,5,7
> >
> >  .. _High_Precision_Event_Timer:
> >
> 
> Also cgroups is often a better solution than the kernel command line.
> There are also caveats around some of these settings so worth reading
> the
> kernel documentation.

Most of the performance tests that we advertise include these parameters and a note 
In docs would helpful if users are trying to replicating those numbers.
I will add a note about cgroups and links to kernel documentation in the next version.

Thanks,
Pavan.
  

Patch

diff --git a/doc/guides/linux_gsg/enable_func.rst b/doc/guides/linux_gsg/enable_func.rst
index 1df3ab0255..f567c713e6 100644
--- a/doc/guides/linux_gsg/enable_func.rst
+++ b/doc/guides/linux_gsg/enable_func.rst
@@ -90,16 +90,16 @@  Using Linux Core Isolation to Reduce Context Switches
 -----------------------------------------------------
 
 While the threads used by a DPDK application are pinned to logical cores on the system,
-it is possible for the Linux scheduler to run other tasks on those cores also.
-To help prevent additional workloads from running on those cores,
-it is possible to use the ``isolcpus`` Linux kernel parameter to isolate them from the general Linux scheduler.
+it is possible for the Linux scheduler to run other tasks on those cores.
+To help prevent additional workloads, timers, rcu processing and IRQs from running on those cores, it is possible to use
+the Linux kernel parameters ``isolcpus``, ``nohz_full``, ``irqaffinity`` to isolate them from the general Linux scheduler tasks.
 
-For example, if DPDK applications are to run on logical cores 2, 4 and 6,
+For example, if a given CPU has 0-7 cores and DPDK applications are to run on logical cores 2, 4 and 6,
 the following should be added to the kernel parameter list:
 
 .. code-block:: console
 
-    isolcpus=2,4,6
+    isolcpus=2,4,6 nohz_full=2,4,6 irqaffinity=0,1,3,5,7
 
 .. _High_Precision_Event_Timer: