[1/4] examples/l3fwd: tune prefetch for better performance
Checks
Commit Message
Packet header is prefetched before packet processing for better
memory access performance. As L2 header will be updated by l3fwd,
using of prefetch for store hint will set cache line to proper
status and reduce cache maintenance overhead.
With this change, 12.9% performance uplift was measured on N1SDP
platform with MLX5 NIC.
Suggested-by: Honnappa Nagarahalli <honnappa.nagarahalli@arm.com>
Signed-off-by: Ruifeng Wang <ruifeng.wang@arm.com>
Reviewed-by: Honnappa Nagarahalli <honnappa.nagarahalli@arm.com>
---
examples/l3fwd/l3fwd_lpm_neon.h | 10 +++++-----
1 file changed, 5 insertions(+), 5 deletions(-)
Comments
On Thu, Mar 18, 2021 at 3:56 PM Ruifeng Wang <ruifeng.wang@arm.com> wrote:
>
> Packet header is prefetched before packet processing for better
> memory access performance. As L2 header will be updated by l3fwd,
> using of prefetch for store hint will set cache line to proper
> status and reduce cache maintenance overhead.
The code does read the cache line too. Right?
>
> With this change, 12.9% performance uplift was measured on N1SDP
> platform with MLX5 NIC.
>
> Suggested-by: Honnappa Nagarahalli <honnappa.nagarahalli@arm.com>
>
> Signed-off-by: Ruifeng Wang <ruifeng.wang@arm.com>
> Reviewed-by: Honnappa Nagarahalli <honnappa.nagarahalli@arm.com>
On the octeontx2 platform, It is 2% regression.
Looks like micro architecture-specific item of handing write hint on the memory
the area that does read and write.
I am testing the LPM lookup miss case.
My test command:
./build/examples/dpdk-l3fwd -c 0x0100 -- -p 0x1 --config="(0,0,8)" -P
> ---
> examples/l3fwd/l3fwd_lpm_neon.h | 10 +++++-----
> 1 file changed, 5 insertions(+), 5 deletions(-)
>
> diff --git a/examples/l3fwd/l3fwd_lpm_neon.h b/examples/l3fwd/l3fwd_lpm_neon.h
> index d6c0ba64a..ae8840694 100644
> --- a/examples/l3fwd/l3fwd_lpm_neon.h
> +++ b/examples/l3fwd/l3fwd_lpm_neon.h
> @@ -97,13 +97,13 @@ l3fwd_lpm_send_packets(int nb_rx, struct rte_mbuf **pkts_burst,
>
> if (k) {
> for (i = 0; i < FWDSTEP; i++) {
> - rte_prefetch0(rte_pktmbuf_mtod(pkts_burst[i],
> + rte_prefetch0_write(rte_pktmbuf_mtod(pkts_burst[i],
> struct rte_ether_hdr *) + 1);
> }
>
> for (j = 0; j != k - FWDSTEP; j += FWDSTEP) {
> for (i = 0; i < FWDSTEP; i++) {
> - rte_prefetch0(rte_pktmbuf_mtod(
> + rte_prefetch0_write(rte_pktmbuf_mtod(
> pkts_burst[j + i + FWDSTEP],
> struct rte_ether_hdr *) + 1);
> }
> @@ -124,17 +124,17 @@ l3fwd_lpm_send_packets(int nb_rx, struct rte_mbuf **pkts_burst,
> /* Prefetch last up to 3 packets one by one */
> switch (m) {
> case 3:
> - rte_prefetch0(rte_pktmbuf_mtod(pkts_burst[j],
> + rte_prefetch0_write(rte_pktmbuf_mtod(pkts_burst[j],
> struct rte_ether_hdr *) + 1);
> j++;
> /* fallthrough */
> case 2:
> - rte_prefetch0(rte_pktmbuf_mtod(pkts_burst[j],
> + rte_prefetch0_write(rte_pktmbuf_mtod(pkts_burst[j],
> struct rte_ether_hdr *) + 1);
> j++;
> /* fallthrough */
> case 1:
> - rte_prefetch0(rte_pktmbuf_mtod(pkts_burst[j],
> + rte_prefetch0_write(rte_pktmbuf_mtod(pkts_burst[j],
> struct rte_ether_hdr *) + 1);
> j++;
> }
> --
> 2.25.1
>
<snip>
> On Thu, Mar 18, 2021 at 3:56 PM Ruifeng Wang <ruifeng.wang@arm.com>
> wrote:
> >
> > Packet header is prefetched before packet processing for better memory
> > access performance. As L2 header will be updated by l3fwd, using of
> > prefetch for store hint will set cache line to proper status and
> > reduce cache maintenance overhead.
>
> The code does read the cache line too. Right?
>
> >
> > With this change, 12.9% performance uplift was measured on N1SDP
> > platform with MLX5 NIC.
> >
> > Suggested-by: Honnappa Nagarahalli <honnappa.nagarahalli@arm.com>
> >
> > Signed-off-by: Ruifeng Wang <ruifeng.wang@arm.com>
> > Reviewed-by: Honnappa Nagarahalli <honnappa.nagarahalli@arm.com>
>
>
> On the octeontx2 platform, It is 2% regression.
Thanks Jerin for testing.
It would be good to know the results from others with A72 platforms.
>
> Looks like micro architecture-specific item of handing write hint on the memory
> the area that does read and write.
>
>
> I am testing the LPM lookup miss case.
>
> My test command:
> ./build/examples/dpdk-l3fwd -c 0x0100 -- -p 0x1 --config="(0,0,8)" -P
>
>
>
> > ---
> > examples/l3fwd/l3fwd_lpm_neon.h | 10 +++++-----
> > 1 file changed, 5 insertions(+), 5 deletions(-)
> >
> > diff --git a/examples/l3fwd/l3fwd_lpm_neon.h
> > b/examples/l3fwd/l3fwd_lpm_neon.h index d6c0ba64a..ae8840694 100644
> > --- a/examples/l3fwd/l3fwd_lpm_neon.h
> > +++ b/examples/l3fwd/l3fwd_lpm_neon.h
> > @@ -97,13 +97,13 @@ l3fwd_lpm_send_packets(int nb_rx, struct rte_mbuf
> > **pkts_burst,
> >
> > if (k) {
> > for (i = 0; i < FWDSTEP; i++) {
> > - rte_prefetch0(rte_pktmbuf_mtod(pkts_burst[i],
> > +
> > + rte_prefetch0_write(rte_pktmbuf_mtod(pkts_burst[i],
> > struct rte_ether_hdr *) + 1);
> > }
> >
> > for (j = 0; j != k - FWDSTEP; j += FWDSTEP) {
> > for (i = 0; i < FWDSTEP; i++) {
> > - rte_prefetch0(rte_pktmbuf_mtod(
> > + rte_prefetch0_write(rte_pktmbuf_mtod(
> > pkts_burst[j + i + FWDSTEP],
> > struct rte_ether_hdr *) + 1);
> > }
> > @@ -124,17 +124,17 @@ l3fwd_lpm_send_packets(int nb_rx, struct rte_mbuf
> **pkts_burst,
> > /* Prefetch last up to 3 packets one by one */
> > switch (m) {
> > case 3:
> > - rte_prefetch0(rte_pktmbuf_mtod(pkts_burst[j],
> > +
> > + rte_prefetch0_write(rte_pktmbuf_mtod(pkts_burst[j],
> > struct rte_ether_hdr *) + 1);
> > j++;
> > /* fallthrough */
> > case 2:
> > - rte_prefetch0(rte_pktmbuf_mtod(pkts_burst[j],
> > +
> > + rte_prefetch0_write(rte_pktmbuf_mtod(pkts_burst[j],
> > struct rte_ether_hdr *) + 1);
> > j++;
> > /* fallthrough */
> > case 1:
> > - rte_prefetch0(rte_pktmbuf_mtod(pkts_burst[j],
> > +
> > + rte_prefetch0_write(rte_pktmbuf_mtod(pkts_burst[j],
> > struct rte_ether_hdr *) + 1);
> > j++;
> > }
> > --
> > 2.25.1
> >
> -----Original Message-----
> From: Jerin Jacob <jerinjacobk@gmail.com>
> Sent: Wednesday, April 14, 2021 2:51 AM
> To: Ruifeng Wang <Ruifeng.Wang@arm.com>
> Cc: jerinj@marvell.com; hemant.agrawal@nxp.com; Ferruh Yigit
> <ferruh.yigit@intel.com>; thomas@monjalon.net; David Marchand
> <david.marchand@redhat.com>; dpdk-dev <dev@dpdk.org>; nd
> <nd@arm.com>; Honnappa Nagarahalli <Honnappa.Nagarahalli@arm.com>
> Subject: Re: [dpdk-dev] [PATCH 1/4] examples/l3fwd: tune prefetch for
> better performance
>
> On Thu, Mar 18, 2021 at 3:56 PM Ruifeng Wang <ruifeng.wang@arm.com>
> wrote:
> >
> > Packet header is prefetched before packet processing for better memory
> > access performance. As L2 header will be updated by l3fwd, using of
> > prefetch for store hint will set cache line to proper status and
> > reduce cache maintenance overhead.
>
> The code does read the cache line too. Right?
>
Yes, the code also read the cache line.
And prefetch to write helps writes. It saves snooping cost.
> >
> > With this change, 12.9% performance uplift was measured on N1SDP
> > platform with MLX5 NIC.
> >
> > Suggested-by: Honnappa Nagarahalli <honnappa.nagarahalli@arm.com>
> >
> > Signed-off-by: Ruifeng Wang <ruifeng.wang@arm.com>
> > Reviewed-by: Honnappa Nagarahalli <honnappa.nagarahalli@arm.com>
>
>
> On the octeontx2 platform, It is 2% regression.
>
> Looks like micro architecture-specific item of handing write hint on the
> memory the area that does read and write.
>
OK. Performance impact of the write hint may be different on various micro architecture implementations.
The 12% measured on N1 is not a small enough number to ignore. How about using a flag to distinguish prefetches invoked on different SoCs?
A compile time flag like RTE_USE_PREFETCH_WRITE is introduced. And SoCs enable it based on need.
>
> I am testing the LPM lookup miss case.
>
> My test command:
> ./build/examples/dpdk-l3fwd -c 0x0100 -- -p 0x1 --config="(0,0,8)" -P
>
>
>
> > ---
> > examples/l3fwd/l3fwd_lpm_neon.h | 10 +++++-----
> > 1 file changed, 5 insertions(+), 5 deletions(-)
> >
> > diff --git a/examples/l3fwd/l3fwd_lpm_neon.h
> > b/examples/l3fwd/l3fwd_lpm_neon.h index d6c0ba64a..ae8840694 100644
> > --- a/examples/l3fwd/l3fwd_lpm_neon.h
> > +++ b/examples/l3fwd/l3fwd_lpm_neon.h
> > @@ -97,13 +97,13 @@ l3fwd_lpm_send_packets(int nb_rx, struct
> rte_mbuf
> > **pkts_burst,
> >
> > if (k) {
> > for (i = 0; i < FWDSTEP; i++) {
> > - rte_prefetch0(rte_pktmbuf_mtod(pkts_burst[i],
> > +
> > + rte_prefetch0_write(rte_pktmbuf_mtod(pkts_burst[i],
> > struct rte_ether_hdr *) + 1);
> > }
> >
> > for (j = 0; j != k - FWDSTEP; j += FWDSTEP) {
> > for (i = 0; i < FWDSTEP; i++) {
> > - rte_prefetch0(rte_pktmbuf_mtod(
> > + rte_prefetch0_write(rte_pktmbuf_mtod(
> > pkts_burst[j + i + FWDSTEP],
> > struct rte_ether_hdr *) + 1);
> > }
> > @@ -124,17 +124,17 @@ l3fwd_lpm_send_packets(int nb_rx, struct
> rte_mbuf **pkts_burst,
> > /* Prefetch last up to 3 packets one by one */
> > switch (m) {
> > case 3:
> > - rte_prefetch0(rte_pktmbuf_mtod(pkts_burst[j],
> > +
> > + rte_prefetch0_write(rte_pktmbuf_mtod(pkts_burst[j],
> > struct rte_ether_hdr *) + 1);
> > j++;
> > /* fallthrough */
> > case 2:
> > - rte_prefetch0(rte_pktmbuf_mtod(pkts_burst[j],
> > +
> > + rte_prefetch0_write(rte_pktmbuf_mtod(pkts_burst[j],
> > struct rte_ether_hdr *) + 1);
> > j++;
> > /* fallthrough */
> > case 1:
> > - rte_prefetch0(rte_pktmbuf_mtod(pkts_burst[j],
> > +
> > + rte_prefetch0_write(rte_pktmbuf_mtod(pkts_burst[j],
> > struct rte_ether_hdr *) + 1);
> > j++;
> > }
> > --
> > 2.25.1
> >
@@ -97,13 +97,13 @@ l3fwd_lpm_send_packets(int nb_rx, struct rte_mbuf **pkts_burst,
if (k) {
for (i = 0; i < FWDSTEP; i++) {
- rte_prefetch0(rte_pktmbuf_mtod(pkts_burst[i],
+ rte_prefetch0_write(rte_pktmbuf_mtod(pkts_burst[i],
struct rte_ether_hdr *) + 1);
}
for (j = 0; j != k - FWDSTEP; j += FWDSTEP) {
for (i = 0; i < FWDSTEP; i++) {
- rte_prefetch0(rte_pktmbuf_mtod(
+ rte_prefetch0_write(rte_pktmbuf_mtod(
pkts_burst[j + i + FWDSTEP],
struct rte_ether_hdr *) + 1);
}
@@ -124,17 +124,17 @@ l3fwd_lpm_send_packets(int nb_rx, struct rte_mbuf **pkts_burst,
/* Prefetch last up to 3 packets one by one */
switch (m) {
case 3:
- rte_prefetch0(rte_pktmbuf_mtod(pkts_burst[j],
+ rte_prefetch0_write(rte_pktmbuf_mtod(pkts_burst[j],
struct rte_ether_hdr *) + 1);
j++;
/* fallthrough */
case 2:
- rte_prefetch0(rte_pktmbuf_mtod(pkts_burst[j],
+ rte_prefetch0_write(rte_pktmbuf_mtod(pkts_burst[j],
struct rte_ether_hdr *) + 1);
j++;
/* fallthrough */
case 1:
- rte_prefetch0(rte_pktmbuf_mtod(pkts_burst[j],
+ rte_prefetch0_write(rte_pktmbuf_mtod(pkts_burst[j],
struct rte_ether_hdr *) + 1);
j++;
}