[v1] doc: policy on promotion of experimental APIs

  Clarifying the ABI policy on the promotion of experimental APIS to stable.
We have a fair number of APIs that have been experimental for more than
2 years. This policy ammendment indicates that these APIs should be
promoted or removed, or should at least form a conservation between the
maintainer and original contributor.

Signed-off-by: Ray Kinsella <mdr@ashroe.eu>
---
 doc/guides/contributing/abi_policy.rst | 20 +++++++++++++++++---
 1 file changed, 17 insertions(+), 3 deletions(-)
  

On Tue, Jun 29, 2021 at 05:00:31PM +0100, Ray Kinsella wrote:
> Clarifying the ABI policy on the promotion of experimental APIS to stable.
> We have a fair number of APIs that have been experimental for more than
> 2 years. This policy ammendment indicates that these APIs should be
> promoted or removed, or should at least form a conservation between the
> maintainer and original contributor.
> 
> Signed-off-by: Ray Kinsella <mdr@ashroe.eu>
> ---
>  doc/guides/contributing/abi_policy.rst | 20 +++++++++++++++++---
>  1 file changed, 17 insertions(+), 3 deletions(-)
> 
> diff --git a/doc/guides/contributing/abi_policy.rst b/doc/guides/contributing/abi_policy.rst
> index 4ad87dbfed..58bc45b8a5 100644
> --- a/doc/guides/contributing/abi_policy.rst
> +++ b/doc/guides/contributing/abi_policy.rst
> @@ -26,9 +26,10 @@ General Guidelines
>     symbols is managed with :ref:`ABI Versioning <abi_versioning>`.
>  #. The removal of symbols is considered an :ref:`ABI breakage <abi_breakages>`,
>     once approved these will form part of the next ABI version.
> -#. Libraries or APIs marked as :ref:`experimental <experimental_apis>` may
> -   be changed or removed without prior notice, as they are not considered part
> -   of an ABI version.
> +#. Libraries or APIs marked as :ref:`experimental <experimental_apis>` may be
> +   changed or removed without prior notice, as they are not considered part of
> +   an ABI version. The :ref:`experimental <experimental_apis>` status of an API
> +   is not an indefinite state.
>  #. Updates to the :ref:`minimum hardware requirements <hw_rqmts>`, which drop
>     support for hardware which was previously supported, should be treated as an
>     ABI change.
> @@ -358,3 +359,16 @@ Libraries
>  Libraries marked as ``experimental`` are entirely not considered part of an ABI
>  version.
>  All functions in such libraries may be changed or removed without prior notice.
> +
> +Promotion to stable
> +~~~~~~~~~~~~~~~~~~~
> +
> +Ordinarily APIs marked as ``experimental`` will be promoted to the stable API
> +once a maintainer and/or the original contributor is satisfied that the API is
> +reasonably mature. In exceptional circumstances, should an API still be

this seems vague and arbitrary. is there a way we can have a more
quantitative metric for what "reasonably mature" means.

> +classified as ``experimental`` after two years and is without any prospect of
> +becoming part of the stable API. The API will then become a candidate for
> +removal, to avoid the acculumation of abandoned symbols.

i think with the above comment the basis for removal then depends on
whatever metric is used to determine maturity. if it is still changing
then it seems like it is useful and still evolving so perhaps should not
be removed but hasn't changed but doesn't meet the metric for being made
stable then perhaps it becomes a candidate for removal.

> +
> +The promotion or removal of symbols will typically form part of a conversation
> +between the maintainer and the original contributor.

this should extend beyond just symbols. there are other changes that
impact the abi where exported symbols don't change. e.g. additions to
return values sets.

thanks for working on this.
  

On 29/06/2021 17:28, Tyler Retzlaff wrote:
> On Tue, Jun 29, 2021 at 05:00:31PM +0100, Ray Kinsella wrote:
>> Clarifying the ABI policy on the promotion of experimental APIS to stable.
>> We have a fair number of APIs that have been experimental for more than
>> 2 years. This policy ammendment indicates that these APIs should be
>> promoted or removed, or should at least form a conservation between the
>> maintainer and original contributor.
>>
>> Signed-off-by: Ray Kinsella <mdr@ashroe.eu>
>> ---
>>  doc/guides/contributing/abi_policy.rst | 20 +++++++++++++++++---
>>  1 file changed, 17 insertions(+), 3 deletions(-)
>>
>> diff --git a/doc/guides/contributing/abi_policy.rst b/doc/guides/contributing/abi_policy.rst
>> index 4ad87dbfed..58bc45b8a5 100644
>> --- a/doc/guides/contributing/abi_policy.rst
>> +++ b/doc/guides/contributing/abi_policy.rst
>> @@ -26,9 +26,10 @@ General Guidelines
>>     symbols is managed with :ref:`ABI Versioning <abi_versioning>`.
>>  #. The removal of symbols is considered an :ref:`ABI breakage <abi_breakages>`,
>>     once approved these will form part of the next ABI version.
>> -#. Libraries or APIs marked as :ref:`experimental <experimental_apis>` may
>> -   be changed or removed without prior notice, as they are not considered part
>> -   of an ABI version.
>> +#. Libraries or APIs marked as :ref:`experimental <experimental_apis>` may be
>> +   changed or removed without prior notice, as they are not considered part of
>> +   an ABI version. The :ref:`experimental <experimental_apis>` status of an API
>> +   is not an indefinite state.
>>  #. Updates to the :ref:`minimum hardware requirements <hw_rqmts>`, which drop
>>     support for hardware which was previously supported, should be treated as an
>>     ABI change.
>> @@ -358,3 +359,16 @@ Libraries
>>  Libraries marked as ``experimental`` are entirely not considered part of an ABI
>>  version.
>>  All functions in such libraries may be changed or removed without prior notice.
>> +
>> +Promotion to stable
>> +~~~~~~~~~~~~~~~~~~~
>> +
>> +Ordinarily APIs marked as ``experimental`` will be promoted to the stable API
>> +once a maintainer and/or the original contributor is satisfied that the API is
>> +reasonably mature. In exceptional circumstances, should an API still be
> 
> this seems vague and arbitrary. is there a way we can have a more
> quantitative metric for what "reasonably mature" means.
> 
>> +classified as ``experimental`` after two years and is without any prospect of
>> +becoming part of the stable API. The API will then become a candidate for
>> +removal, to avoid the acculumation of abandoned symbols.
> 
> i think with the above comment the basis for removal then depends on
> whatever metric is used to determine maturity. 
> if it is still changing
> then it seems like it is useful and still evolving so perhaps should not
> be removed but hasn't changed but doesn't meet the metric for being made
> stable then perhaps it becomes a candidate for removal.

Good idea. 

I think it is reasonable to add a clause that indicates that any change 
to the "API signature" would reset the clock.

However equally any changes to the implementation do not reset the clock.

Would that work?

> 
>> +
>> +The promotion or removal of symbols will typically form part of a conversation
>> +between the maintainer and the original contributor.
> 
> this should extend beyond just symbols. there are other changes that
> impact the abi where exported symbols don't change. e.g. additions to
> return values sets.> 
> thanks for working on this.
>
  

On Tue, Jun 29, 2021 at 07:38:05PM +0100, Kinsella, Ray wrote:
> 
> 
> >> +Promotion to stable
> >> +~~~~~~~~~~~~~~~~~~~
> >> +
> >> +Ordinarily APIs marked as ``experimental`` will be promoted to the stable API
> >> +once a maintainer and/or the original contributor is satisfied that the API is
> >> +reasonably mature. In exceptional circumstances, should an API still be
> > 
> > this seems vague and arbitrary. is there a way we can have a more
> > quantitative metric for what "reasonably mature" means.
> > 
> >> +classified as ``experimental`` after two years and is without any prospect of
> >> +becoming part of the stable API. The API will then become a candidate for
> >> +removal, to avoid the acculumation of abandoned symbols.
> > 
> > i think with the above comment the basis for removal then depends on
> > whatever metric is used to determine maturity. 
> > if it is still changing
> > then it seems like it is useful and still evolving so perhaps should not
> > be removed but hasn't changed but doesn't meet the metric for being made
> > stable then perhaps it becomes a candidate for removal.
> 
> Good idea. 
> 
> I think it is reasonable to add a clause that indicates that any change 
> to the "API signature" would reset the clock.

a time based strategy works but i guess the follow-on to that is how is
the clock tracked and how does it get updated? i don't think trying to
troll through git history will be effective.

one nit, i think "api signature" doesn't cover all cases of what i would
regard as change. i would prefer to define it as "no change where api/abi
compatibility or semantic change occurred"? which is a lot more strict
but in practice is necessary to support binaries when abi/api is stable.

i.e. if a recompile is necessary with or without code change then it's a
change.

> 
> However equally any changes to the implementation do not reset the clock.
> 
> Would that work?

that works for me.

> 
> > 
> >> +
> >> +The promotion or removal of symbols will typically form part of a conversation
> >> +between the maintainer and the original contributor.
> > 
> > this should extend beyond just symbols. there are other changes that
> > impact the abi where exported symbols don't change. e.g. additions to
> > return values sets.> 
> > thanks for working on this.
> >
  

On 6/30/2021 8:56 PM, Tyler Retzlaff wrote:
> On Tue, Jun 29, 2021 at 07:38:05PM +0100, Kinsella, Ray wrote:
>>
>>
>>>> +Promotion to stable
>>>> +~~~~~~~~~~~~~~~~~~~
>>>> +
>>>> +Ordinarily APIs marked as ``experimental`` will be promoted to the stable API
>>>> +once a maintainer and/or the original contributor is satisfied that the API is
>>>> +reasonably mature. In exceptional circumstances, should an API still be
>>>
>>> this seems vague and arbitrary. is there a way we can have a more
>>> quantitative metric for what "reasonably mature" means.
>>>
>>>> +classified as ``experimental`` after two years and is without any prospect of
>>>> +becoming part of the stable API. The API will then become a candidate for
>>>> +removal, to avoid the acculumation of abandoned symbols.
>>>
>>> i think with the above comment the basis for removal then depends on
>>> whatever metric is used to determine maturity. 
>>> if it is still changing
>>> then it seems like it is useful and still evolving so perhaps should not
>>> be removed but hasn't changed but doesn't meet the metric for being made
>>> stable then perhaps it becomes a candidate for removal.
>>
>> Good idea. 
>>
>> I think it is reasonable to add a clause that indicates that any change 
>> to the "API signature" would reset the clock.
> 
> a time based strategy works but i guess the follow-on to that is how is
> the clock tracked and how does it get updated? i don't think trying to
> troll through git history will be effective.
> 

We are grouping the new experimental APIs in the version file based on the
release they are added with a comment, thanks to Dave. Like:

        # added in 19.02
        rte_extmem_attach;
        rte_extmem_detach;
        rte_extmem_register;
        rte_extmem_unregister;

        # added in 19.05
        rte_dev_dma_map;
        rte_dev_dma_unmap;
        ....

Please check 'lib/eal/version.map' as sample.

This enables us easily see the release experimental APIs are added.

> one nit, i think "api signature" doesn't cover all cases of what i would
> regard as change. i would prefer to define it as "no change where api/abi
> compatibility or semantic change occurred"? which is a lot more strict
> but in practice is necessary to support binaries when abi/api is stable.
> 
> i.e. if a recompile is necessary with or without code change then it's a
> change.
> 
>>
>> However equally any changes to the implementation do not reset the clock.
>>
>> Would that work?
> 
> that works for me.
> 
>>
>>>
>>>> +
>>>> +The promotion or removal of symbols will typically form part of a conversation
>>>> +between the maintainer and the original contributor.
>>>
>>> this should extend beyond just symbols. there are other changes that
>>> impact the abi where exported symbols don't change. e.g. additions to
>>> return values sets.> 
>>> thanks for working on this.
>>>
  

On 30/06/2021 20:56, Tyler Retzlaff wrote:
> On Tue, Jun 29, 2021 at 07:38:05PM +0100, Kinsella, Ray wrote:
>>
>>
>>>> +Promotion to stable
>>>> +~~~~~~~~~~~~~~~~~~~
>>>> +
>>>> +Ordinarily APIs marked as ``experimental`` will be promoted to the stable API
>>>> +once a maintainer and/or the original contributor is satisfied that the API is
>>>> +reasonably mature. In exceptional circumstances, should an API still be
>>>
>>> this seems vague and arbitrary. is there a way we can have a more
>>> quantitative metric for what "reasonably mature" means.
>>>
>>>> +classified as ``experimental`` after two years and is without any prospect of
>>>> +becoming part of the stable API. The API will then become a candidate for
>>>> +removal, to avoid the acculumation of abandoned symbols.
>>>
>>> i think with the above comment the basis for removal then depends on
>>> whatever metric is used to determine maturity. 
>>> if it is still changing
>>> then it seems like it is useful and still evolving so perhaps should not
>>> be removed but hasn't changed but doesn't meet the metric for being made
>>> stable then perhaps it becomes a candidate for removal.
>>
>> Good idea. 
>>
>> I think it is reasonable to add a clause that indicates that any change 
>> to the "API signature" would reset the clock.
> 
> a time based strategy works but i guess the follow-on to that is how is
> the clock tracked and how does it get updated? i don't think trying to
> troll through git history will be effective.
> 
> one nit, i think "api signature" doesn't cover all cases of what i would
> regard as change. i would prefer to define it as "no change where api/abi
> compatibility or semantic change occurred"? which is a lot more strict
> but in practice is necessary to support binaries when abi/api is stable.
> 
> i.e. if a recompile is necessary with or without code change then it's a
> change.

Having thought a bit ... this becomes a bit problematic.

Many data-structures in DPDK are nested, 
these can have a ripple effect when changed - a change to mbuf is a good example.

What I saying is ...
I don't think changes in ABI due to in-direct reasons should count.
If there is a change due to a deliberate change in the ABI signature 
that is fine, reset the clock.

If there is a change due to some nested data-structure, 
3-levels down changing in my book that doesn't count. 
As that may or may not have been deliberate, and is almost impossible to police. 

Checking anything but a deliberate change to the ABI signature,
would be practically impossible IMHO. 

> 
>>
>> However equally any changes to the implementation do not reset the clock.
>>
>> Would that work?
> 
> that works for me.

v2 on the way.

> 
>>
>>>
>>>> +
>>>> +The promotion or removal of symbols will typically form part of a conversation
>>>> +between the maintainer and the original contributor.
>>>
>>> this should extend beyond just symbols. there are other changes that
>>> impact the abi where exported symbols don't change. e.g. additions to
>>> return values sets.> 
>>> thanks for working on this.
>>>
  

On Thu, Jul 01, 2021 at 08:56:22AM +0100, Ferruh Yigit wrote:
> On 6/30/2021 8:56 PM, Tyler Retzlaff wrote:
> > On Tue, Jun 29, 2021 at 07:38:05PM +0100, Kinsella, Ray wrote:
> >>
> >>
> >>>> +Promotion to stable
> >>>> +~~~~~~~~~~~~~~~~~~~
> >>>> +
> >>>> +Ordinarily APIs marked as ``experimental`` will be promoted to the stable API
> >>>> +once a maintainer and/or the original contributor is satisfied that the API is
> >>>> +reasonably mature. In exceptional circumstances, should an API still be
> >>>
> >>> this seems vague and arbitrary. is there a way we can have a more
> >>> quantitative metric for what "reasonably mature" means.
> >>>
> >>>> +classified as ``experimental`` after two years and is without any prospect of
> >>>> +becoming part of the stable API. The API will then become a candidate for
> >>>> +removal, to avoid the acculumation of abandoned symbols.
> >>>
> >>> i think with the above comment the basis for removal then depends on
> >>> whatever metric is used to determine maturity. 
> >>> if it is still changing
> >>> then it seems like it is useful and still evolving so perhaps should not
> >>> be removed but hasn't changed but doesn't meet the metric for being made
> >>> stable then perhaps it becomes a candidate for removal.
> >>
> >> Good idea. 
> >>
> >> I think it is reasonable to add a clause that indicates that any change 
> >> to the "API signature" would reset the clock.
> > 
> > a time based strategy works but i guess the follow-on to that is how is
> > the clock tracked and how does it get updated? i don't think trying to
> > troll through git history will be effective.
> > 
> 
> We are grouping the new experimental APIs in the version file based on the
> release they are added with a comment, thanks to Dave. Like:
> 
>         # added in 19.02
>         rte_extmem_attach;
>         rte_extmem_detach;
>         rte_extmem_register;
>         rte_extmem_unregister;
> 
>         # added in 19.05
>         rte_dev_dma_map;
>         rte_dev_dma_unmap;
>         ....
> 
> Please check 'lib/eal/version.map' as sample.
> 
> This enables us easily see the release experimental APIs are added.

this is fine but the subject being discussed is oriented around how long
an api/abi has been unchanged to identify it as a candidate for qualifying
it as stable (not experimental). are you suggesting that if api/abi changes
then it is moved to the -current version to "restart the clock" as it were?
  

On Thu, Jul 01, 2021 at 11:19:27AM +0100, Kinsella, Ray wrote:
> 
> 
> On 30/06/2021 20:56, Tyler Retzlaff wrote:
> > On Tue, Jun 29, 2021 at 07:38:05PM +0100, Kinsella, Ray wrote:
> >>
> >>
> >>>> +Promotion to stable
> >>>> +~~~~~~~~~~~~~~~~~~~
> >>>> +
> >>>> +Ordinarily APIs marked as ``experimental`` will be promoted to the stable API
> >>>> +once a maintainer and/or the original contributor is satisfied that the API is
> >>>> +reasonably mature. In exceptional circumstances, should an API still be
> >>>
> >>> this seems vague and arbitrary. is there a way we can have a more
> >>> quantitative metric for what "reasonably mature" means.
> >>>
> >>>> +classified as ``experimental`` after two years and is without any prospect of
> >>>> +becoming part of the stable API. The API will then become a candidate for
> >>>> +removal, to avoid the acculumation of abandoned symbols.
> >>>
> >>> i think with the above comment the basis for removal then depends on
> >>> whatever metric is used to determine maturity. 
> >>> if it is still changing
> >>> then it seems like it is useful and still evolving so perhaps should not
> >>> be removed but hasn't changed but doesn't meet the metric for being made
> >>> stable then perhaps it becomes a candidate for removal.
> >>
> >> Good idea. 
> >>
> >> I think it is reasonable to add a clause that indicates that any change 
> >> to the "API signature" would reset the clock.
> > 
> > a time based strategy works but i guess the follow-on to that is how is
> > the clock tracked and how does it get updated? i don't think trying to
> > troll through git history will be effective.
> > 
> > one nit, i think "api signature" doesn't cover all cases of what i would
> > regard as change. i would prefer to define it as "no change where api/abi
> > compatibility or semantic change occurred"? which is a lot more strict
> > but in practice is necessary to support binaries when abi/api is stable.
> > 
> > i.e. if a recompile is necessary with or without code change then it's a
> > change.
> 
> Having thought a bit ... this becomes a bit problematic.
> 
> Many data-structures in DPDK are nested, 
> these can have a ripple effect when changed - a change to mbuf is a good example.
> 
> What I saying is ...
> I don't think changes in ABI due to in-direct reasons should count.
> If there is a change due to a deliberate change in the ABI signature 
> that is fine, reset the clock.
>
> 
> If there is a change due to some nested data-structure, 
> 3-levels down changing in my book that doesn't count. 

it has to count otherwise dpdk's abi stability promise for major version
releases is meaningless. or are you suggesting it doesn't count for the
purpose of determining whether or not an experimental api/abi has
changed?

> As that may or may not have been deliberate, and is almost impossible to police. 
> 
> Checking anything but a deliberate change to the ABI signature,
> would be practically impossible IMHO. 

well, it isn't impossible but it does take knowledge, mechanism and
process maintain the abi for a major version.
  

On 01/07/2021 16:09, Tyler Retzlaff wrote:
> On Thu, Jul 01, 2021 at 11:19:27AM +0100, Kinsella, Ray wrote:
>>
>>
>> On 30/06/2021 20:56, Tyler Retzlaff wrote:
>>> On Tue, Jun 29, 2021 at 07:38:05PM +0100, Kinsella, Ray wrote:
>>>>
>>>>
>>>>>> +Promotion to stable
>>>>>> +~~~~~~~~~~~~~~~~~~~
>>>>>> +
>>>>>> +Ordinarily APIs marked as ``experimental`` will be promoted to the stable API
>>>>>> +once a maintainer and/or the original contributor is satisfied that the API is
>>>>>> +reasonably mature. In exceptional circumstances, should an API still be
>>>>>
>>>>> this seems vague and arbitrary. is there a way we can have a more
>>>>> quantitative metric for what "reasonably mature" means.
>>>>>
>>>>>> +classified as ``experimental`` after two years and is without any prospect of
>>>>>> +becoming part of the stable API. The API will then become a candidate for
>>>>>> +removal, to avoid the acculumation of abandoned symbols.
>>>>>
>>>>> i think with the above comment the basis for removal then depends on
>>>>> whatever metric is used to determine maturity. 
>>>>> if it is still changing
>>>>> then it seems like it is useful and still evolving so perhaps should not
>>>>> be removed but hasn't changed but doesn't meet the metric for being made
>>>>> stable then perhaps it becomes a candidate for removal.
>>>>
>>>> Good idea. 
>>>>
>>>> I think it is reasonable to add a clause that indicates that any change 
>>>> to the "API signature" would reset the clock.
>>>
>>> a time based strategy works but i guess the follow-on to that is how is
>>> the clock tracked and how does it get updated? i don't think trying to
>>> troll through git history will be effective.
>>>
>>> one nit, i think "api signature" doesn't cover all cases of what i would
>>> regard as change. i would prefer to define it as "no change where api/abi
>>> compatibility or semantic change occurred"? which is a lot more strict
>>> but in practice is necessary to support binaries when abi/api is stable.
>>>
>>> i.e. if a recompile is necessary with or without code change then it's a
>>> change.
>>
>> Having thought a bit ... this becomes a bit problematic.
>>
>> Many data-structures in DPDK are nested, 
>> these can have a ripple effect when changed - a change to mbuf is a good example.
>>
>> What I saying is ...
>> I don't think changes in ABI due to in-direct reasons should count.
>> If there is a change due to a deliberate change in the ABI signature 
>> that is fine, reset the clock.
>>
>>
>> If there is a change due to some nested data-structure, 
>> 3-levels down changing in my book that doesn't count. 
> 
> it has to count otherwise dpdk's abi stability promise for major version
> releases is meaningless. or are you suggesting it doesn't count for the
> purpose of determining whether or not an experimental api/abi has
> changed?
"it doesn't count for the purpose of determining whether or not an experimental api/abi has changed?".

Exactly - that is what I meant - apologies if I was unclear. 
In this case the change is not a deliberate act, 
in that it is not really happening because of any maturing of the ABI.

> 
>> As that may or may not have been deliberate, and is almost impossible to police. 
>>
>> Checking anything but a deliberate change to the ABI signature,
>> would be practically impossible IMHO. 
> 
> well, it isn't impossible but it does take knowledge, mechanism and
> process maintain the abi for a major version.

100% agree with this statement.

What do you think of the v3?