[dpdk-dev] mk: enable next abi in static libs

  When a change makes really hard to keep ABI compatibility,
instead of waiting next release to break the ABI, it is smoother
to introduce the new code and enable it only for static libraries.
The flag RTE_NEXT_ABI may be used to "ifdef" the new code.
When the release is out, a dynamically linked application can use
the new shared libraries without rebuild while developpers can prepare
their application for the next ABI by reading the deprecation notice
and easily testing the new code.
When starting the next release cycle, the "ifdefs" will be removed
and the ABI break will be marked by incrementing LIBABIVER.

The new option CONFIG_RTE_NEXT_ABI is not defined in the configuration
templates because it is deduced from CONFIG_RTE_BUILD_SHARED_LIB.
It is automatically enabled for static libraries and disabled for
shared libraries.
It can be forced to another value by editing the generated .config file.
It shouldn't be enabled for shared libraries because it would break the
ABI without changing the version number LIBABIVER. That's why a warning
is printed in this case.

The guideline is also updated to integrate this new possibility.

Signed-off-by: Thomas Monjalon <thomas.monjalon@6wind.com>
---
 doc/guides/guidelines/versioning.rst | 2 ++
 lib/Makefile                         | 4 ++++
 mk/rte.sdkconfig.mk                  | 3 +++
 pkg/dpdk.spec                        | 1 +
 scripts/validate-abi.sh              | 2 ++
 5 files changed, 12 insertions(+)
  

Any comment or ack?

2015-07-03 00:05, Thomas Monjalon:
> When a change makes really hard to keep ABI compatibility,
> instead of waiting next release to break the ABI, it is smoother
> to introduce the new code and enable it only for static libraries.
> The flag RTE_NEXT_ABI may be used to "ifdef" the new code.
> When the release is out, a dynamically linked application can use
> the new shared libraries without rebuild while developpers can prepare
> their application for the next ABI by reading the deprecation notice
> and easily testing the new code.
> When starting the next release cycle, the "ifdefs" will be removed
> and the ABI break will be marked by incrementing LIBABIVER.
> 
> The new option CONFIG_RTE_NEXT_ABI is not defined in the configuration
> templates because it is deduced from CONFIG_RTE_BUILD_SHARED_LIB.
> It is automatically enabled for static libraries and disabled for
> shared libraries.
> It can be forced to another value by editing the generated .config file.
> It shouldn't be enabled for shared libraries because it would break the
> ABI without changing the version number LIBABIVER. That's why a warning
> is printed in this case.
> 
> The guideline is also updated to integrate this new possibility.
> 
> Signed-off-by: Thomas Monjalon <thomas.monjalon@6wind.com>
  

On Mon, Jul 06, 2015 at 03:18:51PM +0200, Thomas Monjalon wrote:
> Any comment or ack?
> 
> 2015-07-03 00:05, Thomas Monjalon:
> > When a change makes really hard to keep ABI compatibility,
> > instead of waiting next release to break the ABI, it is smoother
> > to introduce the new code and enable it only for static libraries.
> > The flag RTE_NEXT_ABI may be used to "ifdef" the new code.
> > When the release is out, a dynamically linked application can use
> > the new shared libraries without rebuild while developpers can prepare
> > their application for the next ABI by reading the deprecation notice
> > and easily testing the new code.
> > When starting the next release cycle, the "ifdefs" will be removed
> > and the ABI break will be marked by incrementing LIBABIVER.
> > 
> > The new option CONFIG_RTE_NEXT_ABI is not defined in the configuration
> > templates because it is deduced from CONFIG_RTE_BUILD_SHARED_LIB.
> > It is automatically enabled for static libraries and disabled for
> > shared libraries.
> > It can be forced to another value by editing the generated .config file.
> > It shouldn't be enabled for shared libraries because it would break the
> > ABI without changing the version number LIBABIVER. That's why a warning
> > is printed in this case.
> > 
> > The guideline is also updated to integrate this new possibility.
> > 
> > Signed-off-by: Thomas Monjalon <thomas.monjalon@6wind.com>
> 
> 
Yeah, I'm not sure why this is necessecary.  That is to say, if you want to
introduce a new ABI operation prior to the old one being removed, that is precisely what
the versioning macros are for, and can be used to map the static api to the new
version. e.g, given function X that you want to enhance in an ABI breaking way:

1) Separate function X to X_v1 and X_v2
2) Map X_v2 to X@DPDK_v2, map X_v1 to X@DPDK_v1
3) Map the static symbol X to X_v2
4) Post the deprecation notice of X for release 3 immediately

Splitting the static ABI from the shared ABI just means that applications will
have the opportunity to isolate themselves to one kind of build, which is a bad
idea.

Neil
  

2015-07-06 09:35, Neil Horman:
> On Mon, Jul 06, 2015 at 03:18:51PM +0200, Thomas Monjalon wrote:
> > Any comment or ack?
> > 
> > 2015-07-03 00:05, Thomas Monjalon:
> > > When a change makes really hard to keep ABI compatibility,
> > > instead of waiting next release to break the ABI, it is smoother
> > > to introduce the new code and enable it only for static libraries.
> > > The flag RTE_NEXT_ABI may be used to "ifdef" the new code.
> > > When the release is out, a dynamically linked application can use
> > > the new shared libraries without rebuild while developpers can prepare
> > > their application for the next ABI by reading the deprecation notice
> > > and easily testing the new code.
> > > When starting the next release cycle, the "ifdefs" will be removed
> > > and the ABI break will be marked by incrementing LIBABIVER.
> > > 
> > > The new option CONFIG_RTE_NEXT_ABI is not defined in the configuration
> > > templates because it is deduced from CONFIG_RTE_BUILD_SHARED_LIB.
> > > It is automatically enabled for static libraries and disabled for
> > > shared libraries.
> > > It can be forced to another value by editing the generated .config file.
> > > It shouldn't be enabled for shared libraries because it would break the
> > > ABI without changing the version number LIBABIVER. That's why a warning
> > > is printed in this case.
> > > 
> > > The guideline is also updated to integrate this new possibility.
> > > 
> > > Signed-off-by: Thomas Monjalon <thomas.monjalon@6wind.com>
> > 
> > 
> Yeah, I'm not sure why this is necessecary.  That is to say, if you want to

It's explained at the beginning:
"When a change makes really hard to keep ABI compatibility", e.g. mbuf change.

> introduce a new ABI operation prior to the old one being removed, that is precisely what
> the versioning macros are for, and can be used to map the static api to the new
> version. e.g, given function X that you want to enhance in an ABI breaking way:
> 
> 1) Separate function X to X_v1 and X_v2
> 2) Map X_v2 to X@DPDK_v2, map X_v1 to X@DPDK_v1
> 3) Map the static symbol X to X_v2
> 4) Post the deprecation notice of X for release 3 immediately

We cannot do that for mbuf change.

> Splitting the static ABI from the shared ABI just means that applications will
> have the opportunity to isolate themselves to one kind of build, which is a bad
> idea.

It helps to be prepared for the next release by testing the app with static libraries.
We agreed to allow API breaking for important changes like mbuf rework.
This option NEXT_ABI is a step between announcement and effective ABI breaking.

I think it's a reasonnable approach. But if nobody ack it, I'm perfectly OK to
drop it and related features (unified packet type and interrupt mode).
  

On Mon, Jul 06, 2015 at 03:49:50PM +0200, Thomas Monjalon wrote:
> 2015-07-06 09:35, Neil Horman:
> > On Mon, Jul 06, 2015 at 03:18:51PM +0200, Thomas Monjalon wrote:
> > > Any comment or ack?
> > > 
> > > 2015-07-03 00:05, Thomas Monjalon:
> > > > When a change makes really hard to keep ABI compatibility,
> > > > instead of waiting next release to break the ABI, it is smoother
> > > > to introduce the new code and enable it only for static libraries.
> > > > The flag RTE_NEXT_ABI may be used to "ifdef" the new code.
> > > > When the release is out, a dynamically linked application can use
> > > > the new shared libraries without rebuild while developpers can prepare
> > > > their application for the next ABI by reading the deprecation notice
> > > > and easily testing the new code.
> > > > When starting the next release cycle, the "ifdefs" will be removed
> > > > and the ABI break will be marked by incrementing LIBABIVER.
> > > > 
> > > > The new option CONFIG_RTE_NEXT_ABI is not defined in the configuration
> > > > templates because it is deduced from CONFIG_RTE_BUILD_SHARED_LIB.
> > > > It is automatically enabled for static libraries and disabled for
> > > > shared libraries.
> > > > It can be forced to another value by editing the generated .config file.
> > > > It shouldn't be enabled for shared libraries because it would break the
> > > > ABI without changing the version number LIBABIVER. That's why a warning
> > > > is printed in this case.
> > > > 
> > > > The guideline is also updated to integrate this new possibility.
> > > > 
> > > > Signed-off-by: Thomas Monjalon <thomas.monjalon@6wind.com>
> > > 
> > > 
> > Yeah, I'm not sure why this is necessecary.  That is to say, if you want to
> 
> It's explained at the beginning:
> "When a change makes really hard to keep ABI compatibility", e.g. mbuf change.
> 

Thats not what I was referring to.  I was referring to the need to split out
ABI's based on a separate config item only for static libraries.  I understand
that sometimes you want a 'preview' of the next abi.

> > introduce a new ABI operation prior to the old one being removed, that is precisely what
> > the versioning macros are for, and can be used to map the static api to the new
> > version. e.g, given function X that you want to enhance in an ABI breaking way:
> > 
> > 1) Separate function X to X_v1 and X_v2
> > 2) Map X_v2 to X@DPDK_v2, map X_v1 to X@DPDK_v1
> > 3) Map the static symbol X to X_v2
> > 4) Post the deprecation notice of X for release 3 immediately
> 
> We cannot do that for mbuf change.
> 
You can actually, its just alot of work.  Also, I know this doesn't relate very
closely to the subject, and I apologize, I was really just reacting to the
immediately preceding sentence in the origional post.

> > Splitting the static ABI from the shared ABI just means that applications will
> > have the opportunity to isolate themselves to one kind of build, which is a bad
> > idea.
> 
> It helps to be prepared for the next release by testing the app with static libraries.
> We agreed to allow API breaking for important changes like mbuf rework.
> This option NEXT_ABI is a step between announcement and effective ABI breaking.
> 
> I think it's a reasonnable approach. But if nobody ack it, I'm perfectly OK to
> drop it and related features (unified packet type and interrupt mode).
> 

I'd be ok with it iff:

1) It applies to static and shared ABI's together.  That is to say that setting
the NEXT_ABI config flag creates the same ABI changes regardless of other build
configuration.  It needs to be used in such a way that a consistent ABI is
presented when set, otherwise it won't be useful.

2) It only applies to the next ABI.  That is to say, it can't be a hodgepodge of
the next ABI and the one after that, and the one after that, or it won't provide
an appropriate preview for anyone.


If we can meet those two standards, it would likely be a useful feature to have.
Neil

>
  

2015-07-06 14:22, Neil Horman:
> On Mon, Jul 06, 2015 at 03:49:50PM +0200, Thomas Monjalon wrote:
> > 2015-07-06 09:35, Neil Horman:
> > > On Mon, Jul 06, 2015 at 03:18:51PM +0200, Thomas Monjalon wrote:
> > > > Any comment or ack?
> > > > 
> > > > 2015-07-03 00:05, Thomas Monjalon:
> > > > > When a change makes really hard to keep ABI compatibility,
> > > > > instead of waiting next release to break the ABI, it is smoother
> > > > > to introduce the new code and enable it only for static libraries.
> > > > > The flag RTE_NEXT_ABI may be used to "ifdef" the new code.
> > > > > When the release is out, a dynamically linked application can use
> > > > > the new shared libraries without rebuild while developpers can prepare
> > > > > their application for the next ABI by reading the deprecation notice
> > > > > and easily testing the new code.
> > > > > When starting the next release cycle, the "ifdefs" will be removed
> > > > > and the ABI break will be marked by incrementing LIBABIVER.
> > > > > 
> > > > > The new option CONFIG_RTE_NEXT_ABI is not defined in the configuration
> > > > > templates because it is deduced from CONFIG_RTE_BUILD_SHARED_LIB.
> > > > > It is automatically enabled for static libraries and disabled for
> > > > > shared libraries.
> > > > > It can be forced to another value by editing the generated .config file.
> > > > > It shouldn't be enabled for shared libraries because it would break the
> > > > > ABI without changing the version number LIBABIVER. That's why a warning
> > > > > is printed in this case.
> > > > > 
> > > > > The guideline is also updated to integrate this new possibility.
[...]
> I'd be ok with it iff:
> 
> 1) It applies to static and shared ABI's together.  That is to say that setting
> the NEXT_ABI config flag creates the same ABI changes regardless of other build
> configuration.  It needs to be used in such a way that a consistent ABI is
> presented when set, otherwise it won't be useful.

Yes the option trigger exactly the same ABI for static and shared libraries.
But it's too complicated (at least for 2.1) to make LIBABIVER and version map
dependant of this build-time option.
That's why, it should not be enabled to deploy shared libraries, though it can
be used for tests and development.
As static libraries are almost never packaged, they will be built and linked
at the same time. That's why users of static libraries tend to prefer the
newest ABI, which is the default in this case.

> 2) It only applies to the next ABI.  That is to say, it can't be a hodgepodge of
> the next ABI and the one after that, and the one after that, or it won't provide
> an appropriate preview for anyone.

If you mean the next ABI must be promoted as the standard ABI in the next release,
yes: ifdefs will be cleaned when starting a new release.
Thanks, I learnt the english word hodgepodge :)
  

On Mon, Jul 06, 2015 at 11:44:59PM +0200, Thomas Monjalon wrote:
> 2015-07-06 14:22, Neil Horman:
> > On Mon, Jul 06, 2015 at 03:49:50PM +0200, Thomas Monjalon wrote:
> > > 2015-07-06 09:35, Neil Horman:
> > > > On Mon, Jul 06, 2015 at 03:18:51PM +0200, Thomas Monjalon wrote:
> > > > > Any comment or ack?
> > > > > 
> > > > > 2015-07-03 00:05, Thomas Monjalon:
> > > > > > When a change makes really hard to keep ABI compatibility,
> > > > > > instead of waiting next release to break the ABI, it is smoother
> > > > > > to introduce the new code and enable it only for static libraries.
> > > > > > The flag RTE_NEXT_ABI may be used to "ifdef" the new code.
> > > > > > When the release is out, a dynamically linked application can use
> > > > > > the new shared libraries without rebuild while developpers can prepare
> > > > > > their application for the next ABI by reading the deprecation notice
> > > > > > and easily testing the new code.
> > > > > > When starting the next release cycle, the "ifdefs" will be removed
> > > > > > and the ABI break will be marked by incrementing LIBABIVER.
> > > > > > 
> > > > > > The new option CONFIG_RTE_NEXT_ABI is not defined in the configuration
> > > > > > templates because it is deduced from CONFIG_RTE_BUILD_SHARED_LIB.
> > > > > > It is automatically enabled for static libraries and disabled for
> > > > > > shared libraries.
> > > > > > It can be forced to another value by editing the generated .config file.
> > > > > > It shouldn't be enabled for shared libraries because it would break the
> > > > > > ABI without changing the version number LIBABIVER. That's why a warning
> > > > > > is printed in this case.
> > > > > > 
> > > > > > The guideline is also updated to integrate this new possibility.
> [...]
> > I'd be ok with it iff:
> > 
> > 1) It applies to static and shared ABI's together.  That is to say that setting
> > the NEXT_ABI config flag creates the same ABI changes regardless of other build
> > configuration.  It needs to be used in such a way that a consistent ABI is
> > presented when set, otherwise it won't be useful.
> 
> Yes the option trigger exactly the same ABI for static and shared libraries.
> But it's too complicated (at least for 2.1) to make LIBABIVER and version map
> dependant of this build-time option.

No, I think thats a bridge too far.  I'm not sure whats difficult about
overriding LIBABIVER in lib.rte.mk and bump all numbers 1 higher (or better
still just add a .1 to the end of it), by checking CONFIG_NEXT_ABI

As for maintaining the version map, I don't see any problem with duplicating the
map files, to a -next variant, and changing the CPU_LDFLAGS in rte.lib.mk based
on the NEXT_ABI config option again.

In fact, if this is a thing that people want, that might be beneficial, as
something else occurs to me.  I think you're going to want this to be a mandated
piece of the update process.  That is to say, if someone wants to deprecate an
aspect of the ABI, or change it, I think you'll want to mandate that they submit
the change at the same time they submit the deprecation notice, and simply
protect it with this NEXT_ABI config option.  That provides several advantages:

1) It ensures that the notice is submitted at the same time as the actual change
2) It ensures that the NEXT_ABI provides a complete view of what the next ABI
version looks like, not just a partial view of it

Adding a *-version-next.map file for each library makes adhering to the above
easier, and allows for an easy converstion, in that when its time to officially
update the ABI, fixing the version map is a matter of copying
<library>-version-next.map to <library>-version.map.

The use case that I'm thinking of here is as such:

Consider two ABI modifying updates, A and B.

The author of A writes his changes, submits them with appropriate ifdefs for
CONFIG_NEXT_ABI, along with a deprecation notice.

The author of B writes his changes, but doesn't submit them, instead submitting
only a deprecation notice, with plans to post the actuall patches after the
deprecation notice is shipped in a release

After the release CONFIG_NEXT_ABI exposes the ABI changes made by A, but not by
B (because they don't yet exist in the code).  I think to give users a complete
view of the NEXT_ABI, changes to the ABI, should be done by submitting the patch
(gated on the NEXT_ABI config option), along with the deprecation notice, at the
same time.  That way the ABI view is complete and consistent.  And if you do the
above bits with the cloned version map and LIBABIVER bump, its also consistent
between shared and static libraries.

> That's why, it should not be enabled to deploy shared libraries, though it can
> be used for tests and development.
> As static libraries are almost never packaged, they will be built and linked
> at the same time. That's why users of static libraries tend to prefer the
> newest ABI, which is the default in this case.
> 
> > 2) It only applies to the next ABI.  That is to say, it can't be a hodgepodge of
> > the next ABI and the one after that, and the one after that, or it won't provide
> > an appropriate preview for anyone.
> 
> If you mean the next ABI must be promoted as the standard ABI in the next release,
> yes: ifdefs will be cleaned when starting a new release.
> Thanks, I learnt the english word hodgepodge :)
> 
Je-mexcuse, une meli-melo? :)

I mean't what you indicate yes, and in addition to that, I just wanted to
clarify that this option could strictly _only_ apply to the very next ABI.  That
is to say, someone can't use this without also posting an ABI deprecation
notice, or we would find ourselves in a situation where something would only be
available in NEXT_ABI for more than one release, which would be unacceptable.
But I think we're saying the same thing.

>
  

Thanks Neil, we are making good progress.

2015-07-07 07:14, Neil Horman:
> On Mon, Jul 06, 2015 at 11:44:59PM +0200, Thomas Monjalon wrote:
> > 2015-07-06 14:22, Neil Horman:
> > > On Mon, Jul 06, 2015 at 03:49:50PM +0200, Thomas Monjalon wrote:
> > > > 2015-07-06 09:35, Neil Horman:
> > > > > On Mon, Jul 06, 2015 at 03:18:51PM +0200, Thomas Monjalon wrote:
> > > > > > Any comment or ack?
> > > > > > 
> > > > > > 2015-07-03 00:05, Thomas Monjalon:
> > > > > > > When a change makes really hard to keep ABI compatibility,
> > > > > > > instead of waiting next release to break the ABI, it is smoother
> > > > > > > to introduce the new code and enable it only for static libraries.
> > > > > > > The flag RTE_NEXT_ABI may be used to "ifdef" the new code.
> > > > > > > When the release is out, a dynamically linked application can use
> > > > > > > the new shared libraries without rebuild while developpers can prepare
> > > > > > > their application for the next ABI by reading the deprecation notice
> > > > > > > and easily testing the new code.
> > > > > > > When starting the next release cycle, the "ifdefs" will be removed
> > > > > > > and the ABI break will be marked by incrementing LIBABIVER.
> > > > > > > 
> > > > > > > The new option CONFIG_RTE_NEXT_ABI is not defined in the configuration
> > > > > > > templates because it is deduced from CONFIG_RTE_BUILD_SHARED_LIB.
> > > > > > > It is automatically enabled for static libraries and disabled for
> > > > > > > shared libraries.
> > > > > > > It can be forced to another value by editing the generated .config file.
> > > > > > > It shouldn't be enabled for shared libraries because it would break the
> > > > > > > ABI without changing the version number LIBABIVER. That's why a warning
> > > > > > > is printed in this case.
> > > > > > > 
> > > > > > > The guideline is also updated to integrate this new possibility.
> > [...]
> > > I'd be ok with it iff:
> > > 
> > > 1) It applies to static and shared ABI's together.  That is to say that setting
> > > the NEXT_ABI config flag creates the same ABI changes regardless of other build
> > > configuration.  It needs to be used in such a way that a consistent ABI is
> > > presented when set, otherwise it won't be useful.
> > 
> > Yes the option trigger exactly the same ABI for static and shared libraries.
> > But it's too complicated (at least for 2.1) to make LIBABIVER and version map
> > dependant of this build-time option.
> 
> No, I think thats a bridge too far.  I'm not sure whats difficult about
> overriding LIBABIVER in lib.rte.mk and bump all numbers 1 higher (or better
> still just add a .1 to the end of it), by checking CONFIG_NEXT_ABI

Good idea, I will submit a v2 which adds .1.

> As for maintaining the version map, I don't see any problem with duplicating the
> map files, to a -next variant, and changing the CPU_LDFLAGS in rte.lib.mk based
> on the NEXT_ABI config option again.

OK

> In fact, if this is a thing that people want, that might be beneficial, as
> something else occurs to me.  I think you're going to want this to be a mandated
> piece of the update process.  That is to say, if someone wants to deprecate an
> aspect of the ABI, or change it, I think you'll want to mandate that they submit
> the change at the same time they submit the deprecation notice, and simply
> protect it with this NEXT_ABI config option.  That provides several advantages:

For the release 2.1, we have some deprecation notices without code. It was
the policy agreed in 2.0 release.
Maybe we can force code to be submitted with deprecation notices, starting
with release 2.2.
It needs to be amended in v2 of this patch.

> 1) It ensures that the notice is submitted at the same time as the actual change
> 2) It ensures that the NEXT_ABI provides a complete view of what the next ABI
> version looks like, not just a partial view of it

Yes it would be probably useful.

> Adding a *-version-next.map file for each library makes adhering to the above
> easier, and allows for an easy converstion, in that when its time to officially
> update the ABI, fixing the version map is a matter of copying
> <library>-version-next.map to <library>-version.map.

OK

[...]
> > That's why, it should not be enabled to deploy shared libraries, though it can
> > be used for tests and development.
> > As static libraries are almost never packaged, they will be built and linked
> > at the same time. That's why users of static libraries tend to prefer the
> > newest ABI, which is the default in this case.
> > 
> > > 2) It only applies to the next ABI.  That is to say, it can't be a hodgepodge of
> > > the next ABI and the one after that, and the one after that, or it won't provide
> > > an appropriate preview for anyone.
> > 
> > If you mean the next ABI must be promoted as the standard ABI in the next release,
> > yes: ifdefs will be cleaned when starting a new release.
> > Thanks, I learnt the english word hodgepodge :)
> > 
> Je-mexcuse, une meli-melo? :)

Oui un meli-melo, un ramassis. un beau bordel en somme.

> I mean't what you indicate yes, and in addition to that, I just wanted to
> clarify that this option could strictly _only_ apply to the very next ABI.  That
> is to say, someone can't use this without also posting an ABI deprecation
> notice, or we would find ourselves in a situation where something would only be
> available in NEXT_ABI for more than one release, which would be unacceptable.
> But I think we're saying the same thing.

Yes. I'll try to make it clear in v2.

Neil, in the meantime, could you please help to check ABI breakage in the HEAD?
  

On Tue, Jul 07, 2015 at 05:46:08AM -0700, Thomas Monjalon wrote:
> Thanks Neil, we are making good progress.
> 
> 2015-07-07 07:14, Neil Horman:
> > On Mon, Jul 06, 2015 at 11:44:59PM +0200, Thomas Monjalon wrote:
> > > 2015-07-06 14:22, Neil Horman:
> > > > On Mon, Jul 06, 2015 at 03:49:50PM +0200, Thomas Monjalon wrote:
> > > > > 2015-07-06 09:35, Neil Horman:
> > > > > > On Mon, Jul 06, 2015 at 03:18:51PM +0200, Thomas Monjalon wrote:
> > > > > > > Any comment or ack?
> > > > > > > 
> > > > > > > 2015-07-03 00:05, Thomas Monjalon:
> > > > > > > > When a change makes really hard to keep ABI compatibility,
> > > > > > > > instead of waiting next release to break the ABI, it is smoother
> > > > > > > > to introduce the new code and enable it only for static libraries.
> > > > > > > > The flag RTE_NEXT_ABI may be used to "ifdef" the new code.
> > > > > > > > When the release is out, a dynamically linked application can use
> > > > > > > > the new shared libraries without rebuild while developpers can prepare
> > > > > > > > their application for the next ABI by reading the deprecation notice
> > > > > > > > and easily testing the new code.
> > > > > > > > When starting the next release cycle, the "ifdefs" will be removed
> > > > > > > > and the ABI break will be marked by incrementing LIBABIVER.
> > > > > > > > 
> > > > > > > > The new option CONFIG_RTE_NEXT_ABI is not defined in the configuration
> > > > > > > > templates because it is deduced from CONFIG_RTE_BUILD_SHARED_LIB.
> > > > > > > > It is automatically enabled for static libraries and disabled for
> > > > > > > > shared libraries.
> > > > > > > > It can be forced to another value by editing the generated .config file.
> > > > > > > > It shouldn't be enabled for shared libraries because it would break the
> > > > > > > > ABI without changing the version number LIBABIVER. That's why a warning
> > > > > > > > is printed in this case.
> > > > > > > > 
> > > > > > > > The guideline is also updated to integrate this new possibility.
> > > [...]
> > > > I'd be ok with it iff:
> > > > 
> > > > 1) It applies to static and shared ABI's together.  That is to say that setting
> > > > the NEXT_ABI config flag creates the same ABI changes regardless of other build
> > > > configuration.  It needs to be used in such a way that a consistent ABI is
> > > > presented when set, otherwise it won't be useful.
> > > 
> > > Yes the option trigger exactly the same ABI for static and shared libraries.
> > > But it's too complicated (at least for 2.1) to make LIBABIVER and version map
> > > dependant of this build-time option.
> > 
> > No, I think thats a bridge too far.  I'm not sure whats difficult about
> > overriding LIBABIVER in lib.rte.mk and bump all numbers 1 higher (or better
> > still just add a .1 to the end of it), by checking CONFIG_NEXT_ABI
> 
> Good idea, I will submit a v2 which adds .1.
> 
> > As for maintaining the version map, I don't see any problem with duplicating the
> > map files, to a -next variant, and changing the CPU_LDFLAGS in rte.lib.mk based
> > on the NEXT_ABI config option again.
> 
> OK
> 
> > In fact, if this is a thing that people want, that might be beneficial, as
> > something else occurs to me.  I think you're going to want this to be a mandated
> > piece of the update process.  That is to say, if someone wants to deprecate an
> > aspect of the ABI, or change it, I think you'll want to mandate that they submit
> > the change at the same time they submit the deprecation notice, and simply
> > protect it with this NEXT_ABI config option.  That provides several advantages:
> 
> For the release 2.1, we have some deprecation notices without code. It was
> the policy agreed in 2.0 release.
Right, we're stuck with that for the next release, but this idea of yours I
think will help make sure we get both together in the future.

> Maybe we can force code to be submitted with deprecation notices, starting
> with release 2.2.
> It needs to be amended in v2 of this patch.
Yes, I think that makes sense

> 
> > 1) It ensures that the notice is submitted at the same time as the actual change
> > 2) It ensures that the NEXT_ABI provides a complete view of what the next ABI
> > version looks like, not just a partial view of it
> 
> Yes it would be probably useful.
> 
> > Adding a *-version-next.map file for each library makes adhering to the above
> > easier, and allows for an easy converstion, in that when its time to officially
> > update the ABI, fixing the version map is a matter of copying
> > <library>-version-next.map to <library>-version.map.
> 
> OK
> 
> [...]
> > > That's why, it should not be enabled to deploy shared libraries, though it can
> > > be used for tests and development.
> > > As static libraries are almost never packaged, they will be built and linked
> > > at the same time. That's why users of static libraries tend to prefer the
> > > newest ABI, which is the default in this case.
> > > 
> > > > 2) It only applies to the next ABI.  That is to say, it can't be a hodgepodge of
> > > > the next ABI and the one after that, and the one after that, or it won't provide
> > > > an appropriate preview for anyone.
> > > 
> > > If you mean the next ABI must be promoted as the standard ABI in the next release,
> > > yes: ifdefs will be cleaned when starting a new release.
> > > Thanks, I learnt the english word hodgepodge :)
> > > 
> > Je-mexcuse, une meli-melo? :)
> 
> Oui un meli-melo, un ramassis. un beau bordel en somme.
> 
exactement! :)

> > I mean't what you indicate yes, and in addition to that, I just wanted to
> > clarify that this option could strictly _only_ apply to the very next ABI.  That
> > is to say, someone can't use this without also posting an ABI deprecation
> > notice, or we would find ourselves in a situation where something would only be
> > available in NEXT_ABI for more than one release, which would be unacceptable.
> > But I think we're saying the same thing.
> 
> Yes. I'll try to make it clear in v2.
> 
> Neil, in the meantime, could you please help to check ABI breakage in the HEAD?
I'll try to take a look today
Neil

>
  

On Tue, Jul 07, 2015 at 05:46:08AM -0700, Thomas Monjalon wrote:
> Thanks Neil, we are making good progress.
> 
> 2015-07-07 07:14, Neil Horman:
> > On Mon, Jul 06, 2015 at 11:44:59PM +0200, Thomas Monjalon wrote:
> > > 2015-07-06 14:22, Neil Horman:
> > > > On Mon, Jul 06, 2015 at 03:49:50PM +0200, Thomas Monjalon wrote:
> > > > > 2015-07-06 09:35, Neil Horman:
> > > > > > On Mon, Jul 06, 2015 at 03:18:51PM +0200, Thomas Monjalon wrote:
> > > > > > > Any comment or ack?
> > > > > > > 
> > > > > > > 2015-07-03 00:05, Thomas Monjalon:
> > > > > > > > When a change makes really hard to keep ABI compatibility,
> > > > > > > > instead of waiting next release to break the ABI, it is smoother
> > > > > > > > to introduce the new code and enable it only for static libraries.
> > > > > > > > The flag RTE_NEXT_ABI may be used to "ifdef" the new code.
> > > > > > > > When the release is out, a dynamically linked application can use
> > > > > > > > the new shared libraries without rebuild while developpers can prepare
> > > > > > > > their application for the next ABI by reading the deprecation notice
> > > > > > > > and easily testing the new code.
> > > > > > > > When starting the next release cycle, the "ifdefs" will be removed
> > > > > > > > and the ABI break will be marked by incrementing LIBABIVER.
> > > > > > > > 
> > > > > > > > The new option CONFIG_RTE_NEXT_ABI is not defined in the configuration
> > > > > > > > templates because it is deduced from CONFIG_RTE_BUILD_SHARED_LIB.
> > > > > > > > It is automatically enabled for static libraries and disabled for
> > > > > > > > shared libraries.
> > > > > > > > It can be forced to another value by editing the generated .config file.
> > > > > > > > It shouldn't be enabled for shared libraries because it would break the
> > > > > > > > ABI without changing the version number LIBABIVER. That's why a warning
> > > > > > > > is printed in this case.
> > > > > > > > 
> > > > > > > > The guideline is also updated to integrate this new possibility.
> > > [...]
> > > > I'd be ok with it iff:
> > > > 
> > > > 1) It applies to static and shared ABI's together.  That is to say that setting
> > > > the NEXT_ABI config flag creates the same ABI changes regardless of other build
> > > > configuration.  It needs to be used in such a way that a consistent ABI is
> > > > presented when set, otherwise it won't be useful.
> > > 
> > > Yes the option trigger exactly the same ABI for static and shared libraries.
> > > But it's too complicated (at least for 2.1) to make LIBABIVER and version map
> > > dependant of this build-time option.
> > 
> > No, I think thats a bridge too far.  I'm not sure whats difficult about
> > overriding LIBABIVER in lib.rte.mk and bump all numbers 1 higher (or better
> > still just add a .1 to the end of it), by checking CONFIG_NEXT_ABI
> 
> Good idea, I will submit a v2 which adds .1.
> 
> > As for maintaining the version map, I don't see any problem with duplicating the
> > map files, to a -next variant, and changing the CPU_LDFLAGS in rte.lib.mk based
> > on the NEXT_ABI config option again.
> 
> OK
> 
> > In fact, if this is a thing that people want, that might be beneficial, as
> > something else occurs to me.  I think you're going to want this to be a mandated
> > piece of the update process.  That is to say, if someone wants to deprecate an
> > aspect of the ABI, or change it, I think you'll want to mandate that they submit
> > the change at the same time they submit the deprecation notice, and simply
> > protect it with this NEXT_ABI config option.  That provides several advantages:
> 
> For the release 2.1, we have some deprecation notices without code. It was
> the policy agreed in 2.0 release.
> Maybe we can force code to be submitted with deprecation notices, starting
> with release 2.2.
> It needs to be amended in v2 of this patch.
> 
> > 1) It ensures that the notice is submitted at the same time as the actual change
> > 2) It ensures that the NEXT_ABI provides a complete view of what the next ABI
> > version looks like, not just a partial view of it
> 
> Yes it would be probably useful.
> 
> > Adding a *-version-next.map file for each library makes adhering to the above
> > easier, and allows for an easy converstion, in that when its time to officially
> > update the ABI, fixing the version map is a matter of copying
> > <library>-version-next.map to <library>-version.map.
> 
> OK
> 
> [...]
> > > That's why, it should not be enabled to deploy shared libraries, though it can
> > > be used for tests and development.
> > > As static libraries are almost never packaged, they will be built and linked
> > > at the same time. That's why users of static libraries tend to prefer the
> > > newest ABI, which is the default in this case.
> > > 
> > > > 2) It only applies to the next ABI.  That is to say, it can't be a hodgepodge of
> > > > the next ABI and the one after that, and the one after that, or it won't provide
> > > > an appropriate preview for anyone.
> > > 
> > > If you mean the next ABI must be promoted as the standard ABI in the next release,
> > > yes: ifdefs will be cleaned when starting a new release.
> > > Thanks, I learnt the english word hodgepodge :)
> > > 
> > Je-mexcuse, une meli-melo? :)
> 
> Oui un meli-melo, un ramassis. un beau bordel en somme.
> 
> > I mean't what you indicate yes, and in addition to that, I just wanted to
> > clarify that this option could strictly _only_ apply to the very next ABI.  That
> > is to say, someone can't use this without also posting an ABI deprecation
> > notice, or we would find ourselves in a situation where something would only be
> > available in NEXT_ABI for more than one release, which would be unacceptable.
> > But I think we're saying the same thing.
> 
> Yes. I'll try to make it clear in v2.
> 
> Neil, in the meantime, could you please help to check ABI breakage in the HEAD?
> 
Took a look, the only ABI break I see that we need to worry about is the one
introduced in commit 8eecb3295aed0a979def52245564d03be172a83c. It adds a
bitfield called lro into the existing uint8_t there, but does so in the middle
of the set, which pushes the other bits around, breaking ABI.  It should have
been added to the end.

Neil
  

This is the second version of the NEXT_ABI policy.
It can now be used for shared and static libraries.

While updating rte.lib.mk, it appeared that some useless
(and not consistent) variables were used for some config
options. The first patch clean them.

Thomas Monjalon (2):
  mk: remove variables identical to config ones
  mk: enable next abi preview

 config/common_bsdapp                 |  5 +++++
 config/common_linuxapp               |  5 +++++
 doc/guides/guidelines/versioning.rst | 12 +++++++++---
 mk/exec-env/bsdapp/rte.vars.mk       |  4 ++--
 mk/exec-env/linuxapp/rte.vars.mk     |  4 ++--
 mk/rte.lib.mk                        | 16 +++++++++-------
 mk/rte.sdkbuild.mk                   |  2 +-
 mk/rte.sharelib.mk                   |  8 ++++----
 mk/rte.vars.mk                       |  8 --------
 pkg/dpdk.spec                        |  1 +
 scripts/validate-abi.sh              |  2 ++
 11 files changed, 40 insertions(+), 27 deletions(-)
  

On Wed, Jul 08, 2015 at 04:55:21PM +0200, Thomas Monjalon wrote:
> This is the second version of the NEXT_ABI policy.
> It can now be used for shared and static libraries.
> 
> While updating rte.lib.mk, it appeared that some useless
> (and not consistent) variables were used for some config
> options. The first patch clean them.
> 
> Thomas Monjalon (2):
>   mk: remove variables identical to config ones
>   mk: enable next abi preview
> 
>  config/common_bsdapp                 |  5 +++++
>  config/common_linuxapp               |  5 +++++
>  doc/guides/guidelines/versioning.rst | 12 +++++++++---
>  mk/exec-env/bsdapp/rte.vars.mk       |  4 ++--
>  mk/exec-env/linuxapp/rte.vars.mk     |  4 ++--
>  mk/rte.lib.mk                        | 16 +++++++++-------
>  mk/rte.sdkbuild.mk                   |  2 +-
>  mk/rte.sharelib.mk                   |  8 ++++----
>  mk/rte.vars.mk                       |  8 --------
>  pkg/dpdk.spec                        |  1 +
>  scripts/validate-abi.sh              |  2 ++
>  11 files changed, 40 insertions(+), 27 deletions(-)
> 
> -- 
> 2.4.2
> 
> 
For the series
Acked-by: Neil Horman <nhorman@tuxdriver.com>
  

2015-07-08 12:50, Neil Horman:
> On Wed, Jul 08, 2015 at 04:55:21PM +0200, Thomas Monjalon wrote:
> > This is the second version of the NEXT_ABI policy.
> > It can now be used for shared and static libraries.
> > 
> > While updating rte.lib.mk, it appeared that some useless
> > (and not consistent) variables were used for some config
> > options. The first patch clean them.
> > 
> > Thomas Monjalon (2):
> >   mk: remove variables identical to config ones
> >   mk: enable next abi preview
> 
> For the series
> Acked-by: Neil Horman <nhorman@tuxdriver.com>

Applied, thanks Neil for your help
  

> -----Original Message-----
> From: dev [mailto:dev-bounces@dpdk.org] On Behalf Of Neil Horman
> Sent: Tuesday, July 7, 2015 2:44 PM
> To: Thomas Monjalon
> Cc: dev@dpdk.org
> Subject: Re: [dpdk-dev] [PATCH] mk: enable next abi in static libs
> 
> On Tue, Jul 07, 2015 at 05:46:08AM -0700, Thomas Monjalon wrote:
> > Neil, in the meantime, could you please help to check ABI breakage in
> the HEAD?
> >
> Took a look, the only ABI break I see that we need to worry about is the
> one introduced in commit 8eecb3295aed0a979def52245564d03be172a83c. It adds
> a bitfield called lro into the existing uint8_t there, but does so in the
> middle of the set, which pushes the other bits around, breaking ABI.  It
> should have been added to the end.

Hi,

Is it okay to submit a patch to move it to the end?

John.
--
  

On Fri, Jul 10, 2015 at 04:07:53PM +0000, Mcnamara, John wrote:
> > -----Original Message-----
> > From: dev [mailto:dev-bounces@dpdk.org] On Behalf Of Neil Horman
> > Sent: Tuesday, July 7, 2015 2:44 PM
> > To: Thomas Monjalon
> > Cc: dev@dpdk.org
> > Subject: Re: [dpdk-dev] [PATCH] mk: enable next abi in static libs
> > 
> > On Tue, Jul 07, 2015 at 05:46:08AM -0700, Thomas Monjalon wrote:
> > > Neil, in the meantime, could you please help to check ABI breakage in
> > the HEAD?
> > >
> > Took a look, the only ABI break I see that we need to worry about is the
> > one introduced in commit 8eecb3295aed0a979def52245564d03be172a83c. It adds
> > a bitfield called lro into the existing uint8_t there, but does so in the
> > middle of the set, which pushes the other bits around, breaking ABI.  It
> > should have been added to the end.
> 
> Hi,
> 
> Is it okay to submit a patch to move it to the end?
> 
Assuming that fixes the problem, I think thats the only thing you can do right
now.  I expect that will work, but I would run it through the ABI checker to be
certain
Neil


> John.
> -- 
> 
> 
> 
>
  

> -----Original Message-----
> From: Neil Horman [mailto:nhorman@tuxdriver.com]
> Sent: Saturday, July 11, 2015 3:20 PM
> To: Mcnamara, John
> Cc: Thomas Monjalon; dev@dpdk.org
> Subject: Re: [dpdk-dev] [PATCH] mk: enable next abi in static libs
> 
> On Fri, Jul 10, 2015 at 04:07:53PM +0000, Mcnamara, John wrote:
> > > -----Original Message-----
> > > From: dev [mailto:dev-bounces@dpdk.org] On Behalf Of Neil Horman
> > > Sent: Tuesday, July 7, 2015 2:44 PM
> > > To: Thomas Monjalon
> > > Cc: dev@dpdk.org
> > > Subject: Re: [dpdk-dev] [PATCH] mk: enable next abi in static libs
> > >
> ...
>
> > Is it okay to submit a patch to move it to the end?
> >
> Assuming that fixes the problem, I think thats the only thing you can do
> right now.  I expect that will work, but I would run it through the ABI
> checker to be certain.

Hi Neil,

I ran the change through the abi checker before and after and the lro field no longer generates a Medium Severity ABI warning. I'll submit a patch to move the field.

John.
--