[v2,00/10] experimental tags fixes
mbox series

Message ID 1561809533-6545-1-git-send-email-david.marchand@redhat.com
Headers show
Series
  • experimental tags fixes
Related show

Message

David Marchand June 29, 2019, 11:58 a.m. UTC
Here is a new series on __rte_experimental tags.

Following the build error reported by Aaron [1], I noticed that some
experimental functions could go unnoticed because of a gcc peculiarity.

To catch those, I went and added a new check on the object files to
ensure that any experimental api flagged in the map files is really
exported as such.

Then went with my previous idea of only adding the tags on the functions
prototypes and enforcing it (a new check in checkpatches.sh).
And finally enforcing that the __rte_experimental tag is always the first
part of a function prototype which seems to work with both gcc and clang.

Comments and reviews highly welcome :-).

Changelog since v1:
- rebased on master, caught newly introduced issues in eal
- added acks
- fixed telemetry issue
- squashed Adrien proposition in the last patch

[1]: http://mails.dpdk.org/archives/dev/2019-June/135365.html

Comments

Thomas Monjalon June 29, 2019, 5:06 p.m. UTC | #1
29/06/2019 13:58, David Marchand:
> Following the build error reported by Aaron [1], I noticed that some
> experimental functions could go unnoticed because of a gcc peculiarity.
> 
> To catch those, I went and added a new check on the object files to
> ensure that any experimental api flagged in the map files is really
> exported as such.
> 
> Then went with my previous idea of only adding the tags on the functions
> prototypes and enforcing it (a new check in checkpatches.sh).
> And finally enforcing that the __rte_experimental tag is always the first
> part of a function prototype which seems to work with both gcc and clang.

Applied, thanks
Ferruh Yigit July 1, 2019, 2:15 p.m. UTC | #2
On 6/29/2019 6:06 PM, Thomas Monjalon wrote:
> 29/06/2019 13:58, David Marchand:
>> Following the build error reported by Aaron [1], I noticed that some
>> experimental functions could go unnoticed because of a gcc peculiarity.
>>
>> To catch those, I went and added a new check on the object files to
>> ensure that any experimental api flagged in the map files is really
>> exported as such.
>>
>> Then went with my previous idea of only adding the tags on the functions
>> prototypes and enforcing it (a new check in checkpatches.sh).
>> And finally enforcing that the __rte_experimental tag is always the first
>> part of a function prototype which seems to work with both gcc and clang.
> 
> Applied, thanks
> 


Getting an odd build error with "i686-native-linuxapp-icc" [1].
Beware of the "." at the end: "rte_flow_conv."

Objdump shows two symbols with one "." at the end and one without it [2].

And this seems not the problem of only experimental APIs [3]. But this is only
happening with "i686-native-linuxapp-icc".

Do you have any idea what is going on here?



[1]
Building i686-native-linuxapp-icc ...
rte_flow_conv. is flagged as experimental
but is not listed in version map
Please add rte_flow_conv. to the version map

rte_eth_dev_is_removed. is flagged as experimental
but is not listed in version map
Please add rte_eth_dev_is_removed. to the version map




[2]
$ objdump -x -j '.text.experimental' ./build/build/lib/librte_ethdev/rte_ethdev.o

./build/build/lib/librte_ethdev/rte_ethdev.o:     file format elf32-i386
./build/build/lib/librte_ethdev/rte_ethdev.o
architecture: i386, flags 0x00000011:
HAS_RELOC, HAS_SYMS
start address 0x00000000

Sections:
Idx Name          Size      VMA       LMA       File off  Algn
  4 .text.experimental 00001b70  00000000  00000000  0000e17d  2**4
                  CONTENTS, ALLOC, LOAD, RELOC, READONLY, CODE
SYMBOL TABLE:
00000000 l    d  .text.experimental     00000000 .text.experimental
00000000 g     F .text.experimental     00000090 rte_eth_find_next_of
00000090 g     F .text.experimental     000000d0 rte_eth_find_next_sibling
00000160 g     F .text.experimental     00000110 rte_eth_dev_owner_new
00000270 g     F .text.experimental     00000240 rte_eth_dev_owner_set
000004b0 g     F .text.experimental     000002c0 rte_eth_dev_owner_unset
00000770 g     F .text.experimental     000001b0 rte_eth_dev_owner_delete
00000920 g     F .text.experimental     00000190 rte_eth_dev_owner_get
00000ab0 g     F .text.experimental     000000e0 rte_eth_dev_rx_intr_ctl_q_get_fd
00000b90 g     F .text.experimental     000007d0 rte_eth_dev_create
00001360 g     F .text.experimental     000003a0 rte_eth_dev_destroy
00001700 g     F .text.experimental     000000e0 rte_eth_read_clock
000017e0 g     F .text.experimental     00000070 rte_eth_dev_get_module_info
00001850 g     F .text.experimental     00000070 rte_eth_dev_get_module_eeprom
000018c0 g     F .text.experimental     00000040 rte_eth_switch_domain_alloc
00001900 g     F .text.experimental     00000040 rte_eth_switch_domain_free
00001940 g     F .text.experimental     000001a0 rte_eth_devargs_parse
00001ae0 g     F .text.experimental     00000005 rte_eth_dev_is_removed
00001ae5 g     F .text.experimental     0000008b rte_eth_dev_is_removed.




[3]
objdump -x  ./build/build/lib/librte_ethdev/rte_ethdev.o | grep '\.$'
00002075 g     F .text  0000006b rte_eth_promiscuous_enable.
000020e5 g     F .text  0000005b rte_eth_promiscuous_get.
00002145 g     F .text  0000006b rte_eth_promiscuous_disable.
000021b5 g     F .text  0000006b rte_eth_allmulticast_enable.
00002225 g     F .text  0000005b rte_eth_allmulticast_get.
00002285 g     F .text  0000006b rte_eth_allmulticast_disable.
0000458d g     F .text  00000bc3 rte_eth_xstats_get_names_by_id.
00008109 g     F .text  00000147 rte_eth_dev_info_get.
00001ae5 g     F .text.experimental     0000008b rte_eth_dev_is_removed.
000043cf R_386_PC32        rte_eth_xstats_get_names_by_id.
00004406 R_386_PC32        rte_eth_xstats_get_names_by_id.
David Marchand July 1, 2019, 2:36 p.m. UTC | #3
On Mon, Jul 1, 2019 at 4:15 PM Ferruh Yigit <ferruh.yigit@intel.com> wrote:

> On 6/29/2019 6:06 PM, Thomas Monjalon wrote:
> > 29/06/2019 13:58, David Marchand:
> >> Following the build error reported by Aaron [1], I noticed that some
> >> experimental functions could go unnoticed because of a gcc peculiarity.
> >>
> >> To catch those, I went and added a new check on the object files to
> >> ensure that any experimental api flagged in the map files is really
> >> exported as such.
> >>
> >> Then went with my previous idea of only adding the tags on the functions
> >> prototypes and enforcing it (a new check in checkpatches.sh).
> >> And finally enforcing that the __rte_experimental tag is always the
> first
> >> part of a function prototype which seems to work with both gcc and
> clang.
> >
> > Applied, thanks
> >
>
>
> Getting an odd build error with "i686-native-linuxapp-icc" [1].
> Beware of the "." at the end: "rte_flow_conv."
>
> Objdump shows two symbols with one "." at the end and one without it [2].
>
> And this seems not the problem of only experimental APIs [3]. But this is
> only
> happening with "i686-native-linuxapp-icc".
>
> Do you have any idea what is going on here?
>
>
Looked at rte_flow_conv, and I can not see anything special about it.

There might be a subtility in the way symbol names are chosen by ICC.
Can ICC guys look at this and give us some enlightment?
Ferruh Yigit July 1, 2019, 3:30 p.m. UTC | #4
On 7/1/2019 3:36 PM, David Marchand wrote:
> 
> 
> On Mon, Jul 1, 2019 at 4:15 PM Ferruh Yigit <ferruh.yigit@intel.com
> <mailto:ferruh.yigit@intel.com>> wrote:
> 
>     On 6/29/2019 6:06 PM, Thomas Monjalon wrote:
>     > 29/06/2019 13:58, David Marchand:
>     >> Following the build error reported by Aaron [1], I noticed that some
>     >> experimental functions could go unnoticed because of a gcc peculiarity.
>     >>
>     >> To catch those, I went and added a new check on the object files to
>     >> ensure that any experimental api flagged in the map files is really
>     >> exported as such.
>     >>
>     >> Then went with my previous idea of only adding the tags on the functions
>     >> prototypes and enforcing it (a new check in checkpatches.sh).
>     >> And finally enforcing that the __rte_experimental tag is always the first
>     >> part of a function prototype which seems to work with both gcc and clang.
>     >
>     > Applied, thanks
>     >
> 
> 
>     Getting an odd build error with "i686-native-linuxapp-icc" [1].
>     Beware of the "." at the end: "rte_flow_conv."
> 
>     Objdump shows two symbols with one "." at the end and one without it [2].
> 
>     And this seems not the problem of only experimental APIs [3]. But this is only
>     happening with "i686-native-linuxapp-icc".
> 
>     Do you have any idea what is going on here?
> 
> 
> Looked at rte_flow_conv, and I can not see anything special about it.
> 
> There might be a subtility in the way symbol names are chosen by ICC.
> Can ICC guys look at this and give us some enlightment?

This is the sample disassembler of one of the "." functions [1], it looks like
this notation is used by compiler to prepend some code at the very begging of
the function, Harry (cc'ed) let me know this is may be security feature, not a
defect of compiler :)

So briefly, it looks like compiler can add this "." version of the symbols to
the ".text.experimental" section, I believe the solution is detect this notation
and handle it. What do you think?



[1]
00002070 <rte_eth_promiscuous_enable>:
    2070:       0f b7 44 24 04          movzwl 0x4(%esp),%eax

00002075 <rte_eth_promiscuous_enable.>:
    2075:       56                      push   %esi
    2076:       57                      push   %edi
    2077:       83 ec 14                sub    $0x14,%esp
    207a:       0f b7 c0                movzwl %ax,%eax
    207d:       83 f8 20                cmp    $0x20,%eax
    2080:       7d 14                   jge    2096
<rte_eth_promiscuous_enable.+0x21>
    2082:       8b f0                   mov    %eax,%esi
    2084:       8b f8                   mov    %eax,%edi
    2086:       c1 e6 06                shl    $0x6,%esi
    2089:       c1 e7 0d                shl    $0xd,%edi
    208c:       83 bc 3e 28 20 00 00    cmpl   $0x0,0x2028(%esi,%edi,1)
    2093:       00
    2094:       75 1c                   jne    20b2
<rte_eth_promiscuous_enable.+0x3d>
    2096:       50                      push   %eax
    2097:       68 00 00 00 00          push   $0x0
    209c:       ff 35 00 00 00 00       pushl  0x0
    20a2:       6a 04                   push   $0x4
    20a4:       e8 fc ff ff ff          call   20a5
<rte_eth_promiscuous_enable.+0x30>
    20a9:       83 c4 10                add    $0x10,%esp
....
David Marchand July 1, 2019, 7:27 p.m. UTC | #5
On Mon, Jul 1, 2019 at 5:30 PM Ferruh Yigit <ferruh.yigit@intel.com> wrote:

> On 7/1/2019 3:36 PM, David Marchand wrote:
> >
> >
> > On Mon, Jul 1, 2019 at 4:15 PM Ferruh Yigit <ferruh.yigit@intel.com
> > <mailto:ferruh.yigit@intel.com>> wrote:
> >
> >     On 6/29/2019 6:06 PM, Thomas Monjalon wrote:
> >     > 29/06/2019 13:58, David Marchand:
> >     >> Following the build error reported by Aaron [1], I noticed that
> some
> >     >> experimental functions could go unnoticed because of a gcc
> peculiarity.
> >     >>
> >     >> To catch those, I went and added a new check on the object files
> to
> >     >> ensure that any experimental api flagged in the map files is
> really
> >     >> exported as such.
> >     >>
> >     >> Then went with my previous idea of only adding the tags on the
> functions
> >     >> prototypes and enforcing it (a new check in checkpatches.sh).
> >     >> And finally enforcing that the __rte_experimental tag is always
> the first
> >     >> part of a function prototype which seems to work with both gcc
> and clang.
> >     >
> >     > Applied, thanks
> >     >
> >
> >
> >     Getting an odd build error with "i686-native-linuxapp-icc" [1].
> >     Beware of the "." at the end: "rte_flow_conv."
> >
> >     Objdump shows two symbols with one "." at the end and one without it
> [2].
> >
> >     And this seems not the problem of only experimental APIs [3]. But
> this is only
> >     happening with "i686-native-linuxapp-icc".
> >
> >     Do you have any idea what is going on here?
> >
> >
> > Looked at rte_flow_conv, and I can not see anything special about it.
> >
> > There might be a subtility in the way symbol names are chosen by ICC.
> > Can ICC guys look at this and give us some enlightment?
>
> This is the sample disassembler of one of the "." functions [1], it looks
> like
> this notation is used by compiler to prepend some code at the very begging
> of
> the function, Harry (cc'ed) let me know this is may be security feature,
> not a
> defect of compiler :)
>
> So briefly, it looks like compiler can add this "." version of the symbols
> to
> the ".text.experimental" section, I believe the solution is detect this
> notation
> and handle it. What do you think?
>

Iiuc, we would skip the symbols finishing with a '.', is this all?
Ferruh Yigit July 1, 2019, 9:12 p.m. UTC | #6
On 7/1/2019 8:27 PM, David Marchand wrote:
> 
> 
> On Mon, Jul 1, 2019 at 5:30 PM Ferruh Yigit <ferruh.yigit@intel.com
> <mailto:ferruh.yigit@intel.com>> wrote:
> 
>     On 7/1/2019 3:36 PM, David Marchand wrote:
>     >
>     >
>     > On Mon, Jul 1, 2019 at 4:15 PM Ferruh Yigit <ferruh.yigit@intel.com
>     <mailto:ferruh.yigit@intel.com>
>     > <mailto:ferruh.yigit@intel.com <mailto:ferruh.yigit@intel.com>>> wrote:
>     >
>     >     On 6/29/2019 6:06 PM, Thomas Monjalon wrote:
>     >     > 29/06/2019 13:58, David Marchand:
>     >     >> Following the build error reported by Aaron [1], I noticed that some
>     >     >> experimental functions could go unnoticed because of a gcc peculiarity.
>     >     >>
>     >     >> To catch those, I went and added a new check on the object files to
>     >     >> ensure that any experimental api flagged in the map files is really
>     >     >> exported as such.
>     >     >>
>     >     >> Then went with my previous idea of only adding the tags on the
>     functions
>     >     >> prototypes and enforcing it (a new check in checkpatches.sh).
>     >     >> And finally enforcing that the __rte_experimental tag is always the
>     first
>     >     >> part of a function prototype which seems to work with both gcc and
>     clang.
>     >     >
>     >     > Applied, thanks
>     >     >
>     >
>     >
>     >     Getting an odd build error with "i686-native-linuxapp-icc" [1].
>     >     Beware of the "." at the end: "rte_flow_conv."
>     >
>     >     Objdump shows two symbols with one "." at the end and one without it [2].
>     >
>     >     And this seems not the problem of only experimental APIs [3]. But this
>     is only
>     >     happening with "i686-native-linuxapp-icc".
>     >
>     >     Do you have any idea what is going on here?
>     >
>     >
>     > Looked at rte_flow_conv, and I can not see anything special about it.
>     >
>     > There might be a subtility in the way symbol names are chosen by ICC.
>     > Can ICC guys look at this and give us some enlightment?
> 
>     This is the sample disassembler of one of the "." functions [1], it looks like
>     this notation is used by compiler to prepend some code at the very begging of
>     the function, Harry (cc'ed) let me know this is may be security feature, not a
>     defect of compiler :)
> 
>     So briefly, it looks like compiler can add this "." version of the symbols to
>     the ".text.experimental" section, I believe the solution is detect this notation
>     and handle it. What do you think?
> 
> 
> Iiuc, we would skip the symbols finishing with a '.', is this all?
> 

yes