[v4,1/2] drivers: fix symbol exports when map is omitted
Checks
Commit Message
ld exports any global symbol by default if no version script is passed.
As a consequence, the incriminated change let any public symbol leak
out of the driver shared libraries.
Hide again those symbols by providing a default map file which
unexports any global symbol using a local: * catch-all statement.
The checks are skipped for this default map file as it is intentionnally
an empty map (see commit b67bdda86cd4 ("devtools: catch empty symbol
maps")) and there is nothing else to check in this map.
This change impacts the exported symbols, hence, bump the version in the
ABI check to the v22.11.1 from the 22.11 LTS branch.
Fixes: 7dde9c844a37 ("drivers: omit symbol map when unneeded")
Cc: stable@dpdk.org
Reported-by: Luca Boccassi <luca.boccassi@microsoft.com>
Signed-off-by: David Marchand <david.marchand@redhat.com>
Tested-by: Ferruh Yigit <ferruh.yigit@amd.com>
Reviewed-by: Bruce Richardson <bruce.richardson@intel.com>
---
Changes since v3:
- updated ABI reference now that 22.11.1 is released,
Changes since v2:
- separated the Windows cleanup in next patch,
Changes since v1:
- excluded drivers/version.map from maps checked by default in
check-symbol-maps.sh,
---
.github/workflows/build.yml | 3 +-
.travis.yml | 3 +-
devtools/check-symbol-maps.sh | 2 +-
drivers/meson.build | 68 +++++++++++++++++++----------------
drivers/version.map | 3 ++
5 files changed, 45 insertions(+), 34 deletions(-)
create mode 100644 drivers/version.map
Comments
David Marchand <david.marchand@redhat.com> writes:
> ld exports any global symbol by default if no version script is passed.
> As a consequence, the incriminated change let any public symbol leak
> out of the driver shared libraries.
>
> Hide again those symbols by providing a default map file which
> unexports any global symbol using a local: * catch-all statement.
>
> The checks are skipped for this default map file as it is intentionnally
> an empty map (see commit b67bdda86cd4 ("devtools: catch empty symbol
> maps")) and there is nothing else to check in this map.
>
> This change impacts the exported symbols, hence, bump the version in the
> ABI check to the v22.11.1 from the 22.11 LTS branch.
>
> Fixes: 7dde9c844a37 ("drivers: omit symbol map when unneeded")
> Cc: stable@dpdk.org
>
> Reported-by: Luca Boccassi <luca.boccassi@microsoft.com>
> Signed-off-by: David Marchand <david.marchand@redhat.com>
> Tested-by: Ferruh Yigit <ferruh.yigit@amd.com>
> Reviewed-by: Bruce Richardson <bruce.richardson@intel.com>
> ---
Acked-by: Aaron Conole <aconole@redhat.com>
On Fri, Dec 2, 2022 at 2:39 PM Aaron Conole <aconole@redhat.com> wrote:
>
> David Marchand <david.marchand@redhat.com> writes:
>
> > ld exports any global symbol by default if no version script is passed.
> > As a consequence, the incriminated change let any public symbol leak
> > out of the driver shared libraries.
> >
> > Hide again those symbols by providing a default map file which
> > unexports any global symbol using a local: * catch-all statement.
> >
> > The checks are skipped for this default map file as it is intentionnally
> > an empty map (see commit b67bdda86cd4 ("devtools: catch empty symbol
> > maps")) and there is nothing else to check in this map.
> >
> > This change impacts the exported symbols, hence, bump the version in the
> > ABI check to the v22.11.1 from the 22.11 LTS branch.
> >
> > Fixes: 7dde9c844a37 ("drivers: omit symbol map when unneeded")
> > Cc: stable@dpdk.org
> >
> > Reported-by: Luca Boccassi <luca.boccassi@microsoft.com>
> > Signed-off-by: David Marchand <david.marchand@redhat.com>
> > Tested-by: Ferruh Yigit <ferruh.yigit@amd.com>
> > Reviewed-by: Bruce Richardson <bruce.richardson@intel.com>
> Acked-by: Aaron Conole <aconole@redhat.com>
Series applied.
Please, maintainers and CI teams, when you enable ABI checks in the
main branch, or in the 22.11 LTS branch, use the dpdk-stable 22.11.1
tag as a reference.
Thanks.
> On Fri, Dec 2, 2022 at 2:39 PM Aaron Conole <aconole@redhat.com> wrote:
> >
> > David Marchand <david.marchand@redhat.com> writes:
> >
> > > ld exports any global symbol by default if no version script is passed.
> > > As a consequence, the incriminated change let any public symbol leak
> > > out of the driver shared libraries.
> > >
> > > Hide again those symbols by providing a default map file which
> > > unexports any global symbol using a local: * catch-all statement.
> > >
> > > The checks are skipped for this default map file as it is intentionnally
> > > an empty map (see commit b67bdda86cd4 ("devtools: catch empty symbol
> > > maps")) and there is nothing else to check in this map.
> > >
> > > This change impacts the exported symbols, hence, bump the version in the
> > > ABI check to the v22.11.1 from the 22.11 LTS branch.
> > >
> > > Fixes: 7dde9c844a37 ("drivers: omit symbol map when unneeded")
> > > Cc: stable@dpdk.org
> > >
> > > Reported-by: Luca Boccassi <luca.boccassi@microsoft.com>
> > > Signed-off-by: David Marchand <david.marchand@redhat.com>
> > > Tested-by: Ferruh Yigit <ferruh.yigit@amd.com>
> > > Reviewed-by: Bruce Richardson <bruce.richardson@intel.com>
> > Acked-by: Aaron Conole <aconole@redhat.com>
>
> Series applied.
>
> Please, maintainers and CI teams, when you enable ABI checks in the
> main branch, or in the 22.11 LTS branch, use the dpdk-stable 22.11.1
> tag as a reference.
> Thanks.
Should we also add a tag on main repo, as new development does not happen
on stable tree?
Regards,
Akhil
On Mon, Dec 5, 2022 at 11:44 AM Akhil Goyal <gakhil@marvell.com> wrote:
> > Please, maintainers and CI teams, when you enable ABI checks in the
> > main branch, or in the 22.11 LTS branch, use the dpdk-stable 22.11.1
> > tag as a reference.
> > Thanks.
>
> Should we also add a tag on main repo, as new development does not happen
> on stable tree?
You can fetch the v22.11.1 tag from the stable tree.
> On Mon, Dec 5, 2022 at 11:44 AM Akhil Goyal <gakhil@marvell.com> wrote:
> > > Please, maintainers and CI teams, when you enable ABI checks in the
> > > main branch, or in the 22.11 LTS branch, use the dpdk-stable 22.11.1
> > > tag as a reference.
> > > Thanks.
> >
> > Should we also add a tag on main repo, as new development does not happen
> > on stable tree?
>
> You can fetch the v22.11.1 tag from the stable tree.
Yes, that is an obvious option.
But adding a tag on same repo is more convenient from developer point of view.
However, it is my personal view, others may differ.
-Akhil
05/12/2022 14:47, Akhil Goyal:
> > On Mon, Dec 5, 2022 at 11:44 AM Akhil Goyal <gakhil@marvell.com> wrote:
> > > > Please, maintainers and CI teams, when you enable ABI checks in the
> > > > main branch, or in the 22.11 LTS branch, use the dpdk-stable 22.11.1
> > > > tag as a reference.
> > > > Thanks.
> > >
> > > Should we also add a tag on main repo, as new development does not happen
> > > on stable tree?
> >
> > You can fetch the v22.11.1 tag from the stable tree.
> Yes, that is an obvious option.
> But adding a tag on same repo is more convenient from developer point of view.
> However, it is my personal view, others may differ.
From developer point of view, you should use devtools/test-meson-builds.sh
which does the "git clone" for you.
This is what I have in ~/.config/dpdk/devel.config
export DPDK_ABI_REF_DIR=$root/dpdk-build/abiref
export DPDK_ABI_REF_VERSION=v22.11.1
> 05/12/2022 14:47, Akhil Goyal:
> > > On Mon, Dec 5, 2022 at 11:44 AM Akhil Goyal <gakhil@marvell.com> wrote:
> > > > > Please, maintainers and CI teams, when you enable ABI checks in the
> > > > > main branch, or in the 22.11 LTS branch, use the dpdk-stable 22.11.1
> > > > > tag as a reference.
> > > > > Thanks.
> > > >
> > > > Should we also add a tag on main repo, as new development does not
> happen
> > > > on stable tree?
> > >
> > > You can fetch the v22.11.1 tag from the stable tree.
> > Yes, that is an obvious option.
> > But adding a tag on same repo is more convenient from developer point of
> view.
> > However, it is my personal view, others may differ.
>
> From developer point of view, you should use devtools/test-meson-builds.sh
> which does the "git clone" for you.
>
> This is what I have in ~/.config/dpdk/devel.config
> export DPDK_ABI_REF_DIR=$root/dpdk-build/abiref
> export DPDK_ABI_REF_VERSION=v22.11.1
Ok got it, thanks.
On 12/5/2022 3:37 PM, Thomas Monjalon wrote:
> 05/12/2022 14:47, Akhil Goyal:
>>> On Mon, Dec 5, 2022 at 11:44 AM Akhil Goyal <gakhil@marvell.com> wrote:
>>>>> Please, maintainers and CI teams, when you enable ABI checks in the
>>>>> main branch, or in the 22.11 LTS branch, use the dpdk-stable 22.11.1
>>>>> tag as a reference.
>>>>> Thanks.
>>>>
>>>> Should we also add a tag on main repo, as new development does not happen
>>>> on stable tree?
>>>
>>> You can fetch the v22.11.1 tag from the stable tree.
>> Yes, that is an obvious option.
>> But adding a tag on same repo is more convenient from developer point of view.
>> However, it is my personal view, others may differ.
>
> From developer point of view, you should use devtools/test-meson-builds.sh
> which does the "git clone" for you.
>
> This is what I have in ~/.config/dpdk/devel.config
> export DPDK_ABI_REF_DIR=$root/dpdk-build/abiref
> export DPDK_ABI_REF_VERSION=v22.11.1
>
Does it help to update 'test-meson-builds.sh' to use an environment
variable to select which repo to clone?
If so, I can send a patch for it.
On Tue, Dec 6, 2022 at 11:13 AM Ferruh Yigit <ferruh.yigit@amd.com> wrote:
> On 12/5/2022 3:37 PM, Thomas Monjalon wrote:
> > 05/12/2022 14:47, Akhil Goyal:
> >> But adding a tag on same repo is more convenient from developer point of view.
> >> However, it is my personal view, others may differ.
> >
> > From developer point of view, you should use devtools/test-meson-builds.sh
> > which does the "git clone" for you.
> >
> > This is what I have in ~/.config/dpdk/devel.config
> > export DPDK_ABI_REF_DIR=$root/dpdk-build/abiref
> > export DPDK_ABI_REF_VERSION=v22.11.1
> >
>
> Does it help to update 'test-meson-builds.sh' to use an environment
> variable to select which repo to clone?
> If so, I can send a patch for it.
I was wondering too...
I would expect most maintainers have the stable repo in their config
but it would not hurt to handle the case when they don't for others.
+1
On 12/6/2022 10:18 AM, David Marchand wrote:
> On Tue, Dec 6, 2022 at 11:13 AM Ferruh Yigit <ferruh.yigit@amd.com> wrote:
>> On 12/5/2022 3:37 PM, Thomas Monjalon wrote:
>>> 05/12/2022 14:47, Akhil Goyal:
>>>> But adding a tag on same repo is more convenient from developer point of view.
>>>> However, it is my personal view, others may differ.
>>>
>>> From developer point of view, you should use devtools/test-meson-builds.sh
>>> which does the "git clone" for you.
>>>
>>> This is what I have in ~/.config/dpdk/devel.config
>>> export DPDK_ABI_REF_DIR=$root/dpdk-build/abiref
>>> export DPDK_ABI_REF_VERSION=v22.11.1
>>>
>>
>> Does it help to update 'test-meson-builds.sh' to use an environment
>> variable to select which repo to clone?
>> If so, I can send a patch for it.
>
> I was wondering too...
> I would expect most maintainers have the stable repo in their config
> but it would not hurt to handle the case when they don't for others.
>
> +1
>
>
Sent following if it helps:
https://patches.dpdk.org/project/dpdk/list/?series=26015
Hello,
Community Lab team members are wondering whether it is possible to bump
v22.11 to include at least this patch. We have an existing codebase which
relies on a vXX.XX scheme for producing ABI references. We parse that out
at different places in our code, so fixing this to handle vXX.XX.X will
require some changes on our end. We can do that, but it puts the timeline
on turning on ABI testing at the community lab back some. A v22.11 tagged
release with this patch would allow for us to turn on ABI testing
immediately. There was also a suggestion that having the "base" tag (like
the simple v22.11) point to the latest revision is a more standard version
naming approach and may be more intuitive than what is being used currently.
If that is not possible, we will update our code to be able to refer to a
vXX.XX.X tag for producing the ABI reference. If we adopt this approach, we
would like to request that with future releases, a "vXX.XX.0" tag could
always be made available for us to produce ABI references from. That way,
we can prepare for turning on ABI testing knowing beforehand the tag name
we will be using.
On Tue, Dec 6, 2022 at 7:25 AM Ferruh Yigit <ferruh.yigit@amd.com> wrote:
> On 12/6/2022 10:18 AM, David Marchand wrote:
> > On Tue, Dec 6, 2022 at 11:13 AM Ferruh Yigit <ferruh.yigit@amd.com>
> wrote:
> >> On 12/5/2022 3:37 PM, Thomas Monjalon wrote:
> >>> 05/12/2022 14:47, Akhil Goyal:
> >>>> But adding a tag on same repo is more convenient from developer point
> of view.
> >>>> However, it is my personal view, others may differ.
> >>>
> >>> From developer point of view, you should use
> devtools/test-meson-builds.sh
> >>> which does the "git clone" for you.
> >>>
> >>> This is what I have in ~/.config/dpdk/devel.config
> >>> export DPDK_ABI_REF_DIR=$root/dpdk-build/abiref
> >>> export DPDK_ABI_REF_VERSION=v22.11.1
> >>>
> >>
> >> Does it help to update 'test-meson-builds.sh' to use an environment
> >> variable to select which repo to clone?
> >> If so, I can send a patch for it.
> >
> > I was wondering too...
> > I would expect most maintainers have the stable repo in their config
> > but it would not hurt to handle the case when they don't for others.
> >
> > +1
> >
> >
>
> Sent following if it helps:
> https://patches.dpdk.org/project/dpdk/list/?series=26015
>
I'm sorry Patrick that it gives you more work.
Your proposals below don't look possible because a tag is something fixed forever.
We had an ABI issue in the initial tag so we cannot use the tag v22.11 as planned.
I don't see how we can better plan except having more tests on release candidates.
07/12/2022 19:00, Patrick Robb:
> Hello,
>
> Community Lab team members are wondering whether it is possible to bump
> v22.11 to include at least this patch. We have an existing codebase which
> relies on a vXX.XX scheme for producing ABI references. We parse that out
> at different places in our code, so fixing this to handle vXX.XX.X will
> require some changes on our end. We can do that, but it puts the timeline
> on turning on ABI testing at the community lab back some. A v22.11 tagged
> release with this patch would allow for us to turn on ABI testing
> immediately. There was also a suggestion that having the "base" tag (like
> the simple v22.11) point to the latest revision is a more standard version
> naming approach and may be more intuitive than what is being used currently.
>
> If that is not possible, we will update our code to be able to refer to a
> vXX.XX.X tag for producing the ABI reference. If we adopt this approach, we
> would like to request that with future releases, a "vXX.XX.0" tag could
> always be made available for us to produce ABI references from. That way,
> we can prepare for turning on ABI testing knowing beforehand the tag name
> we will be using.
>
> On Tue, Dec 6, 2022 at 7:25 AM Ferruh Yigit <ferruh.yigit@amd.com> wrote:
>
> > On 12/6/2022 10:18 AM, David Marchand wrote:
> > > On Tue, Dec 6, 2022 at 11:13 AM Ferruh Yigit <ferruh.yigit@amd.com>
> > wrote:
> > >> On 12/5/2022 3:37 PM, Thomas Monjalon wrote:
> > >>> 05/12/2022 14:47, Akhil Goyal:
> > >>>> But adding a tag on same repo is more convenient from developer point
> > of view.
> > >>>> However, it is my personal view, others may differ.
> > >>>
> > >>> From developer point of view, you should use
> > devtools/test-meson-builds.sh
> > >>> which does the "git clone" for you.
> > >>>
> > >>> This is what I have in ~/.config/dpdk/devel.config
> > >>> export DPDK_ABI_REF_DIR=$root/dpdk-build/abiref
> > >>> export DPDK_ABI_REF_VERSION=v22.11.1
> > >>>
> > >>
> > >> Does it help to update 'test-meson-builds.sh' to use an environment
> > >> variable to select which repo to clone?
> > >> If so, I can send a patch for it.
> > >
> > > I was wondering too...
> > > I would expect most maintainers have the stable repo in their config
> > > but it would not hurt to handle the case when they don't for others.
> > >
> > > +1
> >
> > Sent following if it helps:
> > https://patches.dpdk.org/project/dpdk/list/?series=26015
Thomas - thanks for the response. We will proceed with making the necessary
changes for using v22.11.1.
On Thu, Dec 8, 2022 at 8:22 AM Thomas Monjalon <thomas@monjalon.net> wrote:
> I'm sorry Patrick that it gives you more work.
> Your proposals below don't look possible because a tag is something fixed
> forever.
> We had an ABI issue in the initial tag so we cannot use the tag v22.11 as
> planned.
> I don't see how we can better plan except having more tests on release
> candidates.
>
>
> 07/12/2022 19:00, Patrick Robb:
> > Hello,
> >
> > Community Lab team members are wondering whether it is possible to bump
> > v22.11 to include at least this patch. We have an existing codebase which
> > relies on a vXX.XX scheme for producing ABI references. We parse that out
> > at different places in our code, so fixing this to handle vXX.XX.X will
> > require some changes on our end. We can do that, but it puts the timeline
> > on turning on ABI testing at the community lab back some. A v22.11 tagged
> > release with this patch would allow for us to turn on ABI testing
> > immediately. There was also a suggestion that having the "base" tag (like
> > the simple v22.11) point to the latest revision is a more standard
> version
> > naming approach and may be more intuitive than what is being used
> currently.
> >
> > If that is not possible, we will update our code to be able to refer to a
> > vXX.XX.X tag for producing the ABI reference. If we adopt this approach,
> we
> > would like to request that with future releases, a "vXX.XX.0" tag could
> > always be made available for us to produce ABI references from. That way,
> > we can prepare for turning on ABI testing knowing beforehand the tag name
> > we will be using.
> >
> > On Tue, Dec 6, 2022 at 7:25 AM Ferruh Yigit <ferruh.yigit@amd.com>
> wrote:
> >
> > > On 12/6/2022 10:18 AM, David Marchand wrote:
> > > > On Tue, Dec 6, 2022 at 11:13 AM Ferruh Yigit <ferruh.yigit@amd.com>
> > > wrote:
> > > >> On 12/5/2022 3:37 PM, Thomas Monjalon wrote:
> > > >>> 05/12/2022 14:47, Akhil Goyal:
> > > >>>> But adding a tag on same repo is more convenient from developer
> point
> > > of view.
> > > >>>> However, it is my personal view, others may differ.
> > > >>>
> > > >>> From developer point of view, you should use
> > > devtools/test-meson-builds.sh
> > > >>> which does the "git clone" for you.
> > > >>>
> > > >>> This is what I have in ~/.config/dpdk/devel.config
> > > >>> export DPDK_ABI_REF_DIR=$root/dpdk-build/abiref
> > > >>> export DPDK_ABI_REF_VERSION=v22.11.1
> > > >>>
> > > >>
> > > >> Does it help to update 'test-meson-builds.sh' to use an environment
> > > >> variable to select which repo to clone?
> > > >> If so, I can send a patch for it.
> > > >
> > > > I was wondering too...
> > > > I would expect most maintainers have the stable repo in their config
> > > > but it would not hurt to handle the case when they don't for others.
> > > >
> > > > +1
> > >
> > > Sent following if it helps:
> > > https://patches.dpdk.org/project/dpdk/list/?series=26015
>
>
>
>
>
>
@@ -25,7 +25,8 @@ jobs:
MINGW: ${{ matrix.config.cross == 'mingw' }}
MINI: ${{ matrix.config.mini != '' }}
PPC64LE: ${{ matrix.config.cross == 'ppc64le' }}
- REF_GIT_TAG: v22.11
+ REF_GIT_REPO: https://dpdk.org/git/dpdk-stable
+ REF_GIT_TAG: v22.11.1
RISCV64: ${{ matrix.config.cross == 'riscv64' }}
RUN_TESTS: ${{ contains(matrix.config.checks, 'tests') }}
@@ -42,7 +42,8 @@ script: ./.ci/${TRAVIS_OS_NAME}-build.sh
env:
global:
- LIBABIGAIL_VERSION=libabigail-2.1
- - REF_GIT_TAG=v22.11
+ - REF_GIT_REPO=https://dpdk.org/git/dpdk-stable
+ - REF_GIT_TAG=v22.11.1
jobs:
include:
@@ -8,7 +8,7 @@ cd $(dirname $0)/..
export LC_ALL=C
if [ $# = 0 ] ; then
- set -- $(find lib drivers -name '*.map')
+ set -- $(find lib drivers -name '*.map' -a ! -path drivers/version.map)
fi
ret=0
@@ -210,40 +210,46 @@ foreach subpath:subdirs
lk_deps = []
lk_args = []
- if fs.is_file(version_map)
- def_file = custom_target(lib_name + '_def',
- command: [map_to_win_cmd, '@INPUT@', '@OUTPUT@'],
- input: version_map,
- output: '@0@_exports.def'.format(lib_name))
-
- mingw_map = custom_target(lib_name + '_mingw',
- command: [map_to_win_cmd, '@INPUT@', '@OUTPUT@'],
- input: version_map,
- output: '@0@_mingw.map'.format(lib_name))
-
- lk_deps = [version_map, def_file, mingw_map]
- if is_windows
- if is_ms_linker
- lk_args = ['-Wl,/def:' + def_file.full_path()]
- if meson.version().version_compare('<0.54.0')
- lk_args += ['-Wl,/implib:drivers\\' + implib]
- endif
- else
- lk_args = ['-Wl,--version-script=' + mingw_map.full_path()]
+ if not fs.is_file(version_map)
+ version_map = '@0@/version.map'.format(meson.current_source_dir())
+ lk_deps += [version_map]
+ else
+ lk_deps += [version_map]
+ if not is_windows and developer_mode
+ # on unix systems check the output of the
+ # check-symbols.sh script, using it as a
+ # dependency of the .so build
+ lk_deps += custom_target(lib_name + '.sym_chk',
+ command: [check_symbols, version_map, '@INPUT@'],
+ capture: true,
+ input: static_lib,
+ output: lib_name + '.sym_chk')
+ endif
+ endif
+
+ def_file = custom_target(lib_name + '_def',
+ command: [map_to_win_cmd, '@INPUT@', '@OUTPUT@'],
+ input: version_map,
+ output: '@0@_exports.def'.format(lib_name))
+
+ mingw_map = custom_target(lib_name + '_mingw',
+ command: [map_to_win_cmd, '@INPUT@', '@OUTPUT@'],
+ input: version_map,
+ output: '@0@_mingw.map'.format(lib_name))
+
+ lk_deps += [def_file, mingw_map]
+
+ if is_windows
+ if is_ms_linker
+ lk_args = ['-Wl,/def:' + def_file.full_path()]
+ if meson.version().version_compare('<0.54.0')
+ lk_args += ['-Wl,/implib:drivers\\' + implib]
endif
else
- lk_args = ['-Wl,--version-script=' + version_map]
- if developer_mode
- # on unix systems check the output of the
- # check-symbols.sh script, using it as a
- # dependency of the .so build
- lk_deps += custom_target(lib_name + '.sym_chk',
- command: [check_symbols, version_map, '@INPUT@'],
- capture: true,
- input: static_lib,
- output: lib_name + '.sym_chk')
- endif
+ lk_args = ['-Wl,--version-script=' + mingw_map.full_path()]
endif
+ else
+ lk_args = ['-Wl,--version-script=' + version_map]
endif
shared_lib = shared_library(lib_name, sources,
new file mode 100644
@@ -0,0 +1,3 @@
+DPDK_23 {
+ local: *;
+};