[dpdk-dev,v2,01/15] mbuf: add definitions of unified packet types
Commit Message
As there are only 6 bit flags in ol_flags for indicating packet types,
which is not enough to describe all the possible packet types hardware
can recognize. For example, i40e hardware can recognize more than 150
packet types. Unified packet type is composed of tunnel type, L3 type,
L4 type and inner L3 type fields, and can be stored in mbuf field of
'packet_type' which is modified from 16 bits to 32 bits in mbuf structure.
Accordingly, the structure of 'rte_kni_mbuf' needs to be modifed as well.
Signed-off-by: Helin Zhang <helin.zhang@intel.com>
Signed-off-by: Cunming Liang <cunming.liang@intel.com>
Signed-off-by: Jijiang Liu <jijiang.liu@intel.com>
---
.../linuxapp/eal/include/exec-env/rte_kni_common.h | 4 +-
lib/librte_mbuf/rte_mbuf.h | 113 +++++++++++++++++++--
2 files changed, 108 insertions(+), 9 deletions(-)
v2 changes:
* Enlarged the packet_type field from 16 bits to 32 bits.
* Redefined the packet type sub-fields.
* Updated the 'struct rte_kni_mbuf' for KNI according to the mbuf changes.
Comments
On Mon, Feb 09, 2015 at 02:40:35PM +0800, Helin Zhang wrote:
> As there are only 6 bit flags in ol_flags for indicating packet types,
> which is not enough to describe all the possible packet types hardware
> can recognize. For example, i40e hardware can recognize more than 150
> packet types. Unified packet type is composed of tunnel type, L3 type,
> L4 type and inner L3 type fields, and can be stored in mbuf field of
> 'packet_type' which is modified from 16 bits to 32 bits in mbuf structure.
> Accordingly, the structure of 'rte_kni_mbuf' needs to be modifed as well.
>
> Signed-off-by: Helin Zhang <helin.zhang@intel.com>
> Signed-off-by: Cunming Liang <cunming.liang@intel.com>
> Signed-off-by: Jijiang Liu <jijiang.liu@intel.com>
> ---
> .../linuxapp/eal/include/exec-env/rte_kni_common.h | 4 +-
> lib/librte_mbuf/rte_mbuf.h | 113 +++++++++++++++++++--
> 2 files changed, 108 insertions(+), 9 deletions(-)
>
> v2 changes:
> * Enlarged the packet_type field from 16 bits to 32 bits.
> * Redefined the packet type sub-fields.
> * Updated the 'struct rte_kni_mbuf' for KNI according to the mbuf changes.
>
Since these changes to the mbuf will break the operation of the vector driver,
that vector driver needs to be taken into account here.
Some suggestions/options:
1. Temporarily disable the VPMD at compile time or at run time as part of this
patch, and put the vector changes as the next patch (re-enabling the driver too)
2. Put in the minimum changes for the new mbuf layout into this patch. It will
make this patch a little longer, but may still be doable as it's only a couple
of fields changing, not the whole structure.
/Bruce
Hi Bruce
Fortunately I have Steve as the author of a sub-patch for vector PMD in this
patch set. That means we have already taken into account the VPMD in it.
All is workable with vPMD, and with performance result mentioned.
Everything is done for this mbuf changes.
Regards,
Helin
> -----Original Message-----
> From: Richardson, Bruce
> Sent: Monday, February 9, 2015 6:27 PM
> To: Zhang, Helin
> Cc: dev@dpdk.org; Cao, Waterman; Liang, Cunming; Liu, Jijiang; Ananyev,
> Konstantin
> Subject: Re: [PATCH v2 01/15] mbuf: add definitions of unified packet types
>
> On Mon, Feb 09, 2015 at 02:40:35PM +0800, Helin Zhang wrote:
> > As there are only 6 bit flags in ol_flags for indicating packet types,
> > which is not enough to describe all the possible packet types hardware
> > can recognize. For example, i40e hardware can recognize more than 150
> > packet types. Unified packet type is composed of tunnel type, L3 type,
> > L4 type and inner L3 type fields, and can be stored in mbuf field of
> > 'packet_type' which is modified from 16 bits to 32 bits in mbuf structure.
> > Accordingly, the structure of 'rte_kni_mbuf' needs to be modifed as well.
> >
> > Signed-off-by: Helin Zhang <helin.zhang@intel.com>
> > Signed-off-by: Cunming Liang <cunming.liang@intel.com>
> > Signed-off-by: Jijiang Liu <jijiang.liu@intel.com>
> > ---
> > .../linuxapp/eal/include/exec-env/rte_kni_common.h | 4 +-
> > lib/librte_mbuf/rte_mbuf.h | 113
> +++++++++++++++++++--
> > 2 files changed, 108 insertions(+), 9 deletions(-)
> >
> > v2 changes:
> > * Enlarged the packet_type field from 16 bits to 32 bits.
> > * Redefined the packet type sub-fields.
> > * Updated the 'struct rte_kni_mbuf' for KNI according to the mbuf changes.
> >
>
> Since these changes to the mbuf will break the operation of the vector driver,
> that vector driver needs to be taken into account here.
>
> Some suggestions/options:
> 1. Temporarily disable the VPMD at compile time or at run time as part of this
> patch, and put the vector changes as the next patch (re-enabling the driver too)
> 2. Put in the minimum changes for the new mbuf layout into this patch. It will
> make this patch a little longer, but may still be doable as it's only a couple of
> fields changing, not the whole structure.
>
> /Bruce
On Tue, Feb 10, 2015 at 12:53:52AM +0000, Zhang, Helin wrote:
> Hi Bruce
>
> Fortunately I have Steve as the author of a sub-patch for vector PMD in this
> patch set. That means we have already taken into account the VPMD in it.
> All is workable with vPMD, and with performance result mentioned.
> Everything is done for this mbuf changes.
>
> Regards,
> Helin
>
I see that helin, but between applying this patch and applying the subsequent
patch for the vector PMD, the DPDK vector PMD code is broken, which would cause
problems for anyone doing a git bisect. Hence my suggestion that changes to take
account of the vpmd need to go in this patch (not just in the patch set) to
avoid having broken code following this commit.
/Bruce
> > -----Original Message-----
> > From: Richardson, Bruce
> > Sent: Monday, February 9, 2015 6:27 PM
> > To: Zhang, Helin
> > Cc: dev@dpdk.org; Cao, Waterman; Liang, Cunming; Liu, Jijiang; Ananyev,
> > Konstantin
> > Subject: Re: [PATCH v2 01/15] mbuf: add definitions of unified packet types
> >
> > On Mon, Feb 09, 2015 at 02:40:35PM +0800, Helin Zhang wrote:
> > > As there are only 6 bit flags in ol_flags for indicating packet types,
> > > which is not enough to describe all the possible packet types hardware
> > > can recognize. For example, i40e hardware can recognize more than 150
> > > packet types. Unified packet type is composed of tunnel type, L3 type,
> > > L4 type and inner L3 type fields, and can be stored in mbuf field of
> > > 'packet_type' which is modified from 16 bits to 32 bits in mbuf structure.
> > > Accordingly, the structure of 'rte_kni_mbuf' needs to be modifed as well.
> > >
> > > Signed-off-by: Helin Zhang <helin.zhang@intel.com>
> > > Signed-off-by: Cunming Liang <cunming.liang@intel.com>
> > > Signed-off-by: Jijiang Liu <jijiang.liu@intel.com>
> > > ---
> > > .../linuxapp/eal/include/exec-env/rte_kni_common.h | 4 +-
> > > lib/librte_mbuf/rte_mbuf.h | 113
> > +++++++++++++++++++--
> > > 2 files changed, 108 insertions(+), 9 deletions(-)
> > >
> > > v2 changes:
> > > * Enlarged the packet_type field from 16 bits to 32 bits.
> > > * Redefined the packet type sub-fields.
> > > * Updated the 'struct rte_kni_mbuf' for KNI according to the mbuf changes.
> > >
> >
> > Since these changes to the mbuf will break the operation of the vector driver,
> > that vector driver needs to be taken into account here.
> >
> > Some suggestions/options:
> > 1. Temporarily disable the VPMD at compile time or at run time as part of this
> > patch, and put the vector changes as the next patch (re-enabling the driver too)
> > 2. Put in the minimum changes for the new mbuf layout into this patch. It will
> > make this patch a little longer, but may still be doable as it's only a couple of
> > fields changing, not the whole structure.
> >
> > /Bruce
@@ -117,9 +117,9 @@ struct rte_kni_mbuf {
uint16_t data_off; /**< Start address of data in segment buffer. */
char pad1[4];
uint64_t ol_flags; /**< Offload features. */
- char pad2[2];
- uint16_t data_len; /**< Amount of data in segment buffer. */
+ char pad2[4];
uint32_t pkt_len; /**< Total pkt len: sum of all segment data_len. */
+ uint16_t data_len; /**< Amount of data in segment buffer. */
/* fields on second cache line */
char pad3[8] __attribute__((__aligned__(RTE_CACHE_LINE_SIZE)));
@@ -165,6 +165,96 @@ extern "C" {
/* Use final bit of flags to indicate a control mbuf */
#define CTRL_MBUF_FLAG (1ULL << 63) /**< Mbuf contains control data */
+/*
+ * 32 bits are divided into several fields to mark packet types. Note that
+ * each field is indexical.
+ * - Bit 3:0 is for L2 types.
+ * - Bit 7:4 is for L3 or outer L3 (for tunneling case) types.
+ * - Bit 11:8 is for L4 or outer L4 (for tunneling case) types.
+ * - Bit 15:12 is for tunnel types.
+ * - Bit 19:16 is for inner L2 types.
+ * - Bit 23:20 is for inner L3 types.
+ * - Bit 27:24 is for inner L4 types.
+ * - Bit 31:28 is reserved.
+ *
+ * To be compatible with Vector PMD, RTE_PTYPE_L3_IPV4, RTE_PTYPE_L3_IPV4_EXT,
+ * RTE_PTYPE_L3_IPV6, RTE_PTYPE_L3_IPV6_EXT, RTE_PTYPE_L4_TCP, RTE_PTYPE_L4_UDP
+ * and RTE_PTYPE_L4_SCTP should be kept as below in a contiguous 7 bits.
+ *
+ * Note that L3 types values are selected for checking IPV4/IPV6 header from
+ * performance point of view. Reading annotations of RTE_ETH_IS_IPV4_HDR and
+ * RTE_ETH_IS_IPV6_HDR is needed for any future changes of L3 type values.
+ */
+#define RTE_PTYPE_UNKNOWN 0x00000000
+/* bit 3:0 for L2 types */
+#define RTE_PTYPE_L2_MAC 0x00000001
+#define RTE_PTYPE_L2_MAC_TIMESYNC 0x00000002
+#define RTE_PTYPE_L2_ARP 0x00000003
+#define RTE_PTYPE_L2_LLDP 0x00000004
+#define RTE_PTYPE_L2_MASK 0x0000000f
+/* bit 7:4 for L3 types */
+#define RTE_PTYPE_L3_IPV4 0x00000010
+#define RTE_PTYPE_L3_IPV4_EXT 0x00000030
+#define RTE_PTYPE_L3_IPV6 0x00000040
+#define RTE_PTYPE_L3_IPV4_EXT_UNKNOWN 0x00000090
+#define RTE_PTYPE_L3_IPV6_EXT 0x000000c0
+#define RTE_PTYPE_L3_IPV6_EXT_UNKNOWN 0x000000e0
+#define RTE_PTYPE_L3_MASK 0x000000f0
+/* bit 11:8 for L4 types */
+#define RTE_PTYPE_L4_TCP 0x00000100
+#define RTE_PTYPE_L4_UDP 0x00000200
+#define RTE_PTYPE_L4_FRAG 0x00000300
+#define RTE_PTYPE_L4_SCTP 0x00000400
+#define RTE_PTYPE_L4_ICMP 0x00000500
+#define RTE_PTYPE_L4_NONFRAG 0x00000600
+#define RTE_PTYPE_L4_MASK 0x00000f00
+/* bit 15:12 for tunnel types */
+#define RTE_PTYPE_TUNNEL_IP 0x00001000
+#define RTE_PTYPE_TUNNEL_GRE 0x00002000
+#define RTE_PTYPE_TUNNEL_VXLAN 0x00003000
+#define RTE_PTYPE_TUNNEL_NVGRE 0x00004000
+#define RTE_PTYPE_TUNNEL_GENEVE 0x00005000
+#define RTE_PTYPE_TUNNEL_GRENAT 0x00006000
+#define RTE_PTYPE_TUNNEL_MASK 0x0000f000
+/* bit 19:16 for inner L2 types */
+#define RTE_PTYPE_INNER_L2_MAC 0x00010000
+#define RTE_PTYPE_INNER_L2_MAC_VLAN 0x00020000
+#define RTE_PTYPE_INNER_L2_MASK 0x000f0000
+/* bit 23:20 for inner L3 types */
+#define RTE_PTYPE_INNER_L3_IPV4 0x00100000
+#define RTE_PTYPE_INNER_L3_IPV4_EXT 0x00200000
+#define RTE_PTYPE_INNER_L3_IPV6 0x00300000
+#define RTE_PTYPE_INNER_L3_IPV4_EXT_UNKNOWN 0x00400000
+#define RTE_PTYPE_INNER_L3_IPV6_EXT 0x00500000
+#define RTE_PTYPE_INNER_L3_IPV6_EXT_UNKNOWN 0x00600000
+#define RTE_PTYPE_INNER_INNER_L3_MASK 0x00f00000
+/* bit 27:24 for inner L4 types */
+#define RTE_PTYPE_INNER_L4_TCP 0x01000000
+#define RTE_PTYPE_INNER_L4_UDP 0x02000000
+#define RTE_PTYPE_INNER_L4_FRAG 0x03000000
+#define RTE_PTYPE_INNER_L4_SCTP 0x04000000
+#define RTE_PTYPE_INNER_L4_ICMP 0x05000000
+#define RTE_PTYPE_INNER_L4_NONFRAG 0x06000000
+#define RTE_PTYPE_INNER_L4_MASK 0x0f000000
+/* bit 31:28 reserved */
+
+/**
+ * Check if the (outer) L3 header is IPv4. To avoid comparing IPv4 types one by
+ * one, bit 4 is selected to be used for IPv4 only. Then checking bit 4 can
+ * determin if it is an IPV4 packet.
+ */
+#define RTE_ETH_IS_IPV4_HDR(ptype) ((ptype) & RTE_PTYPE_L3_IPV4)
+
+/**
+ * Check if the (outer) L3 header is IPv4. To avoid comparing IPv4 types one by
+ * one, bit 6 is selected to be used for IPv4 only. Then checking bit 6 can
+ * determin if it is an IPV4 packet.
+ */
+#define RTE_ETH_IS_IPV6_HDR(ptype) ((ptype) & RTE_PTYPE_L3_IPV6)
+
+/* Check if it is a tunneling packet */
+#define RTE_ETH_IS_TUNNEL_PKT(ptype) ((ptype) & RTE_PTYPE_TUNNEL_MASK)
+
/**
* Get the name of a RX offload flag
*
@@ -232,17 +322,26 @@ struct rte_mbuf {
/* remaining bytes are set on RX when pulling packet from descriptor */
MARKER rx_descriptor_fields1;
- /**
- * The packet type, which is used to indicate ordinary packet and also
- * tunneled packet format, i.e. each number is represented a type of
- * packet.
+ /*
+ * The packet type, which is the combination of outer/inner L2, L3, L4
+ * and tunnel types.
*/
- uint16_t packet_type;
+ union {
+ uint32_t packet_type; /**< L2/L3/L4 and tunnel information. */
+ struct {
+ uint32_t l2_type:4; /**< (Outer) L2 type. */
+ uint32_t l3_type:4; /**< (Outer) L3 type. */
+ uint32_t l4_type:4; /**< (Outer) L4 type. */
+ uint32_t tun_type:4; /**< Tunnel type. */
+ uint32_t inner_l2_type:4; /**< Inner L2 type. */
+ uint32_t inner_l3_type:4; /**< Inner L3 type. */
+ uint32_t inner_l4_type:4; /**< Inner L4 type. */
+ };
+ };
- uint16_t data_len; /**< Amount of data in segment buffer. */
uint32_t pkt_len; /**< Total pkt len: sum of all segments. */
+ uint16_t data_len; /**< Amount of data in segment buffer. */
uint16_t vlan_tci; /**< VLAN Tag Control Identifier (CPU order) */
- uint16_t reserved;
union {
uint32_t rss; /**< RSS hash result if RSS enabled */
struct {