[1/2] net/netvsc: allow setting rx and tx copy break
Checks
Commit Message
From: Stephen Hemminger <stephen@networkplumber.org>
The values for Rx and Tx copy break should be tunable rather
than hard coded constants.
The rx_copybreak sets the threshold where the driver uses an
external mbuf to avoid having to copy data. Setting 0 for copybreak
will cause driver to always create an external mbuf. Setting
a value greater than the MTU would prevent it from ever making
an external mbuf and always copy. The default value is 256 (bytes).
Likewise the tx_copybreak sets the threshold where the driver
aggregates multiple small packets into one request. If tx_copybreak
is 0 then each packet goes as a VMBus request (no copying).
If tx_copybreak is set larger than the MTU, then all packets smaller
than the chunk size of the VMBus send buffer will be copied; larger
packets always have to go as a single direct request. The default
value is 512 (bytes).
Signed-off-by: Stephen Hemminger <stephen@networkplumber.org>
Signed-off-by: Long Li <longli@microsoft.com>
---
drivers/net/netvsc/hn_ethdev.c | 63 +++++++++++++++++++---------------
drivers/net/netvsc/hn_rxtx.c | 7 ++--
drivers/net/netvsc/hn_var.h | 5 +++
3 files changed, 43 insertions(+), 32 deletions(-)
Comments
On 10/22/2020 8:46 PM, Long Li wrote:
> From: Stephen Hemminger <stephen@networkplumber.org>
>
> The values for Rx and Tx copy break should be tunable rather
> than hard coded constants.
>
> The rx_copybreak sets the threshold where the driver uses an
> external mbuf to avoid having to copy data. Setting 0 for copybreak
> will cause driver to always create an external mbuf. Setting
> a value greater than the MTU would prevent it from ever making
> an external mbuf and always copy. The default value is 256 (bytes).
>
> Likewise the tx_copybreak sets the threshold where the driver
> aggregates multiple small packets into one request. If tx_copybreak
> is 0 then each packet goes as a VMBus request (no copying).
> If tx_copybreak is set larger than the MTU, then all packets smaller
> than the chunk size of the VMBus send buffer will be copied; larger
> packets always have to go as a single direct request. The default
> value is 512 (bytes).
>
> Signed-off-by: Stephen Hemminger <stephen@networkplumber.org>
> Signed-off-by: Long Li <longli@microsoft.com>
<...>
> @@ -45,6 +45,10 @@
> DEV_RX_OFFLOAD_VLAN_STRIP | \
> DEV_RX_OFFLOAD_RSS_HASH)
>
> +#define NETVSC_ARG_LATENCY "latency"
> +#define NETVSC_ARG_RXBREAK "rx_copybreak"
> +#define NETVSC_ARG_TXBREAK "tx_copybreak"
> +
Can you please document new devargs in the driver documentation?
<...>
> @@ -181,12 +167,32 @@ static int hn_parse_args(const struct rte_eth_dev *dev)
> return -EINVAL;
> }
>
> - ret = rte_kvargs_process(kvlist, "latency", hn_set_latency, hv);
> - if (ret)
> - PMD_DRV_LOG(ERR, "Unable to process latency arg\n");
> + for (i = 0; i != kvlist->count; ++i) {
> + const struct rte_kvargs_pair *pair = &kvlist->pairs[i];
> +
> + if (!strcmp(pair->key, NETVSC_ARG_LATENCY))
> + latency = atoi(pair->value);
> + else if (!strcmp(pair->key, NETVSC_ARG_RXBREAK))
> + rx_break = atoi(pair->value);
> + else if (!strcmp(pair->key, NETVSC_ARG_TXBREAK))
> + tx_break = atoi(pair->value);
> + }
> +
Instead of accessing to the kvlist internals, I think better to use
'rte_kvargs_process()' as done previously.
If the reason to remove callback is to not create a callback for each argument,
a generic one can be used for all.
> + if (latency >= 0) {
> + PMD_DRV_LOG(DEBUG, "set latency %d usec", latency);
> + hv->latency = latency * 1000; /* usec to nsec */
> + }
> + if (rx_break >= 0) {
> + PMD_DRV_LOG(DEBUG, "rx copy break set to %d", rx_break);
> + hv->rx_copybreak = rx_break;
> + }
> + if (tx_break >= 0) {
> + PMD_DRV_LOG(DEBUG, "tx copy break set to %d", tx_break);
> + hv->tx_copybreak = tx_break;
> + }
>
When 'rte_kvargs_process()' used, the valued can be assigned directly to
'hv->tx_copybreak', if the argument is not available, it won't be updated, so
above check can be dropped.
> Subject: Re: [dpdk-dev] [PATCH 1/2] net/netvsc: allow setting rx and tx copy
> break
>
> On 10/22/2020 8:46 PM, Long Li wrote:
> > From: Stephen Hemminger <stephen@networkplumber.org>
> >
> > The values for Rx and Tx copy break should be tunable rather than hard
> > coded constants.
> >
> > The rx_copybreak sets the threshold where the driver uses an external
> > mbuf to avoid having to copy data. Setting 0 for copybreak will cause
> > driver to always create an external mbuf. Setting a value greater than
> > the MTU would prevent it from ever making an external mbuf and always
> > copy. The default value is 256 (bytes).
> >
> > Likewise the tx_copybreak sets the threshold where the driver
> > aggregates multiple small packets into one request. If tx_copybreak is
> > 0 then each packet goes as a VMBus request (no copying).
> > If tx_copybreak is set larger than the MTU, then all packets smaller
> > than the chunk size of the VMBus send buffer will be copied; larger
> > packets always have to go as a single direct request. The default
> > value is 512 (bytes).
> >
> > Signed-off-by: Stephen Hemminger <stephen@networkplumber.org>
> > Signed-off-by: Long Li <longli@microsoft.com>
>
> <...>
>
> > @@ -45,6 +45,10 @@
> > DEV_RX_OFFLOAD_VLAN_STRIP | \
> > DEV_RX_OFFLOAD_RSS_HASH)
> >
> > +#define NETVSC_ARG_LATENCY "latency"
> > +#define NETVSC_ARG_RXBREAK "rx_copybreak"
> > +#define NETVSC_ARG_TXBREAK "tx_copybreak"
> > +
>
> Can you please document new devargs in the driver documentation?
>
> <...>
>
> > @@ -181,12 +167,32 @@ static int hn_parse_args(const struct rte_eth_dev
> *dev)
> > return -EINVAL;
> > }
> >
> > - ret = rte_kvargs_process(kvlist, "latency", hn_set_latency, hv);
> > - if (ret)
> > - PMD_DRV_LOG(ERR, "Unable to process latency arg\n");
> > + for (i = 0; i != kvlist->count; ++i) {
> > + const struct rte_kvargs_pair *pair = &kvlist->pairs[i];
> > +
> > + if (!strcmp(pair->key, NETVSC_ARG_LATENCY))
> > + latency = atoi(pair->value);
> > + else if (!strcmp(pair->key, NETVSC_ARG_RXBREAK))
> > + rx_break = atoi(pair->value);
> > + else if (!strcmp(pair->key, NETVSC_ARG_TXBREAK))
> > + tx_break = atoi(pair->value);
> > + }
> > +
>
> Instead of accessing to the kvlist internals, I think better to use
> 'rte_kvargs_process()' as done previously.
> If the reason to remove callback is to not create a callback for each argument, a
> generic one can be used for all.
>
> > + if (latency >= 0) {
> > + PMD_DRV_LOG(DEBUG, "set latency %d usec", latency);
> > + hv->latency = latency * 1000; /* usec to nsec */
> > + }
> > + if (rx_break >= 0) {
> > + PMD_DRV_LOG(DEBUG, "rx copy break set to %d", rx_break);
> > + hv->rx_copybreak = rx_break;
> > + }
> > + if (tx_break >= 0) {
> > + PMD_DRV_LOG(DEBUG, "tx copy break set to %d", tx_break);
> > + hv->tx_copybreak = tx_break;
> > + }
> >
>
> When 'rte_kvargs_process()' used, the valued can be assigned directly to 'hv-
> >tx_copybreak', if the argument is not available, it won't be updated, so above
> check can be dropped.
Thanks Ferruh, I will send V2 to address comments.
Long
@@ -45,6 +45,10 @@
DEV_RX_OFFLOAD_VLAN_STRIP | \
DEV_RX_OFFLOAD_RSS_HASH)
+#define NETVSC_ARG_LATENCY "latency"
+#define NETVSC_ARG_RXBREAK "rx_copybreak"
+#define NETVSC_ARG_TXBREAK "tx_copybreak"
+
struct hn_xstats_name_off {
char name[RTE_ETH_XSTATS_NAME_SIZE];
unsigned int offset;
@@ -136,38 +140,20 @@ eth_dev_vmbus_release(struct rte_eth_dev *eth_dev)
eth_dev->intr_handle = NULL;
}
-/* handle "latency=X" from devargs */
-static int hn_set_latency(const char *key, const char *value, void *opaque)
-{
- struct hn_data *hv = opaque;
- char *endp = NULL;
- unsigned long lat;
-
- errno = 0;
- lat = strtoul(value, &endp, 0);
-
- if (*value == '\0' || *endp != '\0') {
- PMD_DRV_LOG(ERR, "invalid parameter %s=%s", key, value);
- return -EINVAL;
- }
-
- PMD_DRV_LOG(DEBUG, "set latency %lu usec", lat);
-
- hv->latency = lat * 1000; /* usec to nsec */
- return 0;
-}
-
/* Parse device arguments */
static int hn_parse_args(const struct rte_eth_dev *dev)
{
struct hn_data *hv = dev->data->dev_private;
struct rte_devargs *devargs = dev->device->devargs;
static const char * const valid_keys[] = {
- "latency",
+ NETVSC_ARG_LATENCY,
+ NETVSC_ARG_RXBREAK,
+ NETVSC_ARG_TXBREAK,
NULL
};
+ int latency = -1, rx_break = -1, tx_break = -1;
struct rte_kvargs *kvlist;
- int ret;
+ unsigned int i;
if (!devargs)
return 0;
@@ -181,12 +167,32 @@ static int hn_parse_args(const struct rte_eth_dev *dev)
return -EINVAL;
}
- ret = rte_kvargs_process(kvlist, "latency", hn_set_latency, hv);
- if (ret)
- PMD_DRV_LOG(ERR, "Unable to process latency arg\n");
+ for (i = 0; i != kvlist->count; ++i) {
+ const struct rte_kvargs_pair *pair = &kvlist->pairs[i];
+
+ if (!strcmp(pair->key, NETVSC_ARG_LATENCY))
+ latency = atoi(pair->value);
+ else if (!strcmp(pair->key, NETVSC_ARG_RXBREAK))
+ rx_break = atoi(pair->value);
+ else if (!strcmp(pair->key, NETVSC_ARG_TXBREAK))
+ tx_break = atoi(pair->value);
+ }
+
+ if (latency >= 0) {
+ PMD_DRV_LOG(DEBUG, "set latency %d usec", latency);
+ hv->latency = latency * 1000; /* usec to nsec */
+ }
+ if (rx_break >= 0) {
+ PMD_DRV_LOG(DEBUG, "rx copy break set to %d", rx_break);
+ hv->rx_copybreak = rx_break;
+ }
+ if (tx_break >= 0) {
+ PMD_DRV_LOG(DEBUG, "tx copy break set to %d", tx_break);
+ hv->tx_copybreak = tx_break;
+ }
rte_kvargs_free(kvlist);
- return ret;
+ return 0;
}
/* Update link status.
@@ -966,7 +972,10 @@ eth_hn_dev_init(struct rte_eth_dev *eth_dev)
hv->chim_res = &vmbus->resource[HV_SEND_BUF_MAP];
hv->port_id = eth_dev->data->port_id;
hv->latency = HN_CHAN_LATENCY_NS;
+ hv->rx_copybreak = HN_RXCOPY_THRESHOLD;
+ hv->tx_copybreak = HN_TXCOPY_THRESHOLD;
hv->max_queues = 1;
+
rte_rwlock_init(&hv->vf_lock);
hv->vf_port = HN_INVALID_PORT;
@@ -40,9 +40,6 @@
(sizeof(struct vmbus_chanpkt_hdr) + sizeof(struct hn_nvs_rndis))
#define HN_TXD_CACHE_SIZE 32 /* per cpu tx_descriptor pool cache */
-#define HN_TXCOPY_THRESHOLD 512
-
-#define HN_RXCOPY_THRESHOLD 256
#define HN_RXQ_EVENT_DEFAULT 2048
struct hn_rxinfo {
@@ -568,7 +565,7 @@ static void hn_rxpkt(struct hn_rx_queue *rxq, struct hn_rx_bufinfo *rxb,
* For large packets, avoid copy if possible but need to keep
* some space available in receive area for later packets.
*/
- if (dlen >= HN_RXCOPY_THRESHOLD &&
+ if (dlen > hv->rx_copybreak &&
(uint32_t)rte_atomic32_read(&rxq->rxbuf_outstanding) <
hv->rxbuf_section_cnt / 2) {
struct rte_mbuf_ext_shared_info *shinfo;
@@ -1516,7 +1513,7 @@ hn_xmit_pkts(void *ptxq, struct rte_mbuf **tx_pkts, uint16_t nb_pkts)
break;
/* For small packets aggregate them in chimney buffer */
- if (m->pkt_len < HN_TXCOPY_THRESHOLD && pkt_size <= txq->agg_szmax) {
+ if (m->pkt_len <= hv->tx_copybreak && pkt_size <= txq->agg_szmax) {
/* If this packet will not fit, then flush */
if (txq->agg_pktleft == 0 ||
RTE_ALIGN(pkt_size, txq->agg_align) > txq->agg_szleft) {
@@ -23,6 +23,9 @@
/* Host monitor interval */
#define HN_CHAN_LATENCY_NS 50000
+#define HN_TXCOPY_THRESHOLD 512
+#define HN_RXCOPY_THRESHOLD 256
+
/* Buffers need to be aligned */
#ifndef PAGE_SIZE
#define PAGE_SIZE 4096
@@ -114,6 +117,7 @@ struct hn_data {
struct rte_mem_resource *rxbuf_res; /* UIO resource for Rx */
uint32_t rxbuf_section_cnt; /* # of Rx sections */
+ uint32_t rx_copybreak;
uint16_t max_queues; /* Max available queues */
uint16_t num_queues;
uint64_t rss_offloads;
@@ -122,6 +126,7 @@ struct hn_data {
struct rte_mem_resource *chim_res; /* UIO resource for Tx */
struct rte_bitmap *chim_bmap; /* Send buffer map */
void *chim_bmem;
+ uint32_t tx_copybreak;
uint32_t chim_szmax; /* Max size per buffer */
uint32_t chim_cnt; /* Max packets per buffer */