[dpdk-dev] l3fwd error, port 0 is not present on the board

  On Wed, Dec 31, 2014 at 10:37:45PM -0500, Neil Horman wrote:
> On Wed, Dec 31, 2014 at 10:50:10AM -0600, Lyn M wrote:
> > Neil,
> > When I build DPDK 1.8.0 with the default value CONFIG_RTE_BUILD_COMBINE_LIBS=n,
> > I am able to run the testpmd successfully.  So, that does appear to be the
> > culprit.
> > The objdump -t output for testpmd (with COMBINE_LIBS=n and COMBINE_LIBS=y)
> > files are here:
> > https://www.dropbox.com/sh/6os9yfn22jjcnmy/AAAjEV3u06XQ1t3UXdWcgru0a?dl=0
> > 
> > Thanks,
> > Lyn
> > 
> I see the problem.  none of the constructors are included in the l3fwd app.  I
> think you might need 174572477028b7f04700389e38f87d2ba01a0adc.  Its also
> possible something else has gone wrong withthe COMBINE_LIBS option.  Try
> building with the head of the tree, and if that doesn't fix it, we can start
> debugging in earnest.
> Neil

Found the problem.  Please try this patch
  

Neil, your patch works.  I am now able to run testpmd, as well as l3fwd
with DPDK 1.8.0 and CONFIG_RTE_BUILD_COMBINE_LIBS=y.  Thank you much!

In my original post, I thought my choice of hex portmask -p 0x3 was causing
this issue -- now I know that was not the case.  But, I am still curious
about how the hex portmask is determined.  Since I only have two ports
bound to igb_uio, my hex postmask will always be 0x3?  What if I choose to
bind other ports to igb_uio?  Is there a Linux command I can run to
determine what mask to use with the -p option?

Happy New Year --
Lyn

On Wed, Dec 31, 2014 at 10:00 PM, Neil Horman <nhorman@tuxdriver.com> wrote:

> On Wed, Dec 31, 2014 at 10:37:45PM -0500, Neil Horman wrote:
> > On Wed, Dec 31, 2014 at 10:50:10AM -0600, Lyn M wrote:
> > > Neil,
> > > When I build DPDK 1.8.0 with the default value
> CONFIG_RTE_BUILD_COMBINE_LIBS=n,
> > > I am able to run the testpmd successfully.  So, that does appear to be
> the
> > > culprit.
> > > The objdump -t output for testpmd (with COMBINE_LIBS=n and
> COMBINE_LIBS=y)
> > > files are here:
> > >
> https://www.dropbox.com/sh/6os9yfn22jjcnmy/AAAjEV3u06XQ1t3UXdWcgru0a?dl=0
> > >
> > > Thanks,
> > > Lyn
> > >
> > I see the problem.  none of the constructors are included in the l3fwd
> app.  I
> > think you might need 174572477028b7f04700389e38f87d2ba01a0adc.  Its also
> > possible something else has gone wrong withthe COMBINE_LIBS option.  Try
> > building with the head of the tree, and if that doesn't fix it, we can
> start
> > debugging in earnest.
> > Neil
>
> Found the problem.  Please try this patch
>
>
> diff --git a/mk/rte.app.mk b/mk/rte.app.mk
> index e1a0dbf..40afb2c 100644
> --- a/mk/rte.app.mk
> +++ b/mk/rte.app.mk
> @@ -61,6 +61,10 @@ ifeq ($(NO_AUTOLIBS),)
>
>  LDLIBS += --whole-archive
>
> +ifeq ($(CONFIG_RTE_BUILD_COMBINE_LIBS),y)
> +LDLIBS += -l$(RTE_LIBNAME)
> +endif
> +
>  ifeq ($(CONFIG_RTE_BUILD_COMBINE_LIBS),n)
>
>  ifeq ($(CONFIG_RTE_LIBRTE_DISTRIBUTOR),y)
> @@ -251,10 +255,6 @@ build: _postbuild
>
>  exe2cmd = $(strip $(call dotfile,$(patsubst %,%.cmd,$(1))))
>
> -ifeq ($(CONFIG_RTE_BUILD_COMBINE_LIBS),y)
> -LDLIBS += -l$(RTE_LIBNAME)
> -endif
> -
>  ifeq ($(LINK_USING_CC),1)
>  override EXTRA_LDFLAGS := $(call linkerprefix,$(EXTRA_LDFLAGS))
>  O_TO_EXE = $(CC) $(CFLAGS) $(LDFLAGS_$(@)) \
>
  

On Fri, Jan 02, 2015 at 09:07:40AM -0600, Lyn M wrote:
> Neil, your patch works.  I am now able to run testpmd, as well as l3fwd
> with DPDK 1.8.0 and CONFIG_RTE_BUILD_COMBINE_LIBS=y.  Thank you much!
> 
I'll post it for official review shortly.

> In my original post, I thought my choice of hex portmask -p 0x3 was causing
> this issue -- now I know that was not the case.  But, I am still curious
> about how the hex portmask is determined.  Since I only have two ports
> bound to igb_uio, my hex postmask will always be 0x3?  What if I choose to
> bind other ports to igb_uio?  Is there a Linux command I can run to
> determine what mask to use with the -p option?
> 
portmask IIRC is determined by the cpus you decide you want to service various
hardware queues.  you can select whatever cpus you want, so in a sense, the
portmask is entirely your decision.  If you want it to be optimal, you should
select a portmask where the cpus servicing your hardware are on the same numa
node as the NIC itself.  You should be able to determine the cpu numa node
locality via /sys/class/net/<iface>/device/local_cpus

Neil

> Happy New Year --
> Lyn
> 
> On Wed, Dec 31, 2014 at 10:00 PM, Neil Horman <nhorman@tuxdriver.com> wrote:
> 
> > On Wed, Dec 31, 2014 at 10:37:45PM -0500, Neil Horman wrote:
> > > On Wed, Dec 31, 2014 at 10:50:10AM -0600, Lyn M wrote:
> > > > Neil,
> > > > When I build DPDK 1.8.0 with the default value
> > CONFIG_RTE_BUILD_COMBINE_LIBS=n,
> > > > I am able to run the testpmd successfully.  So, that does appear to be
> > the
> > > > culprit.
> > > > The objdump -t output for testpmd (with COMBINE_LIBS=n and
> > COMBINE_LIBS=y)
> > > > files are here:
> > > >
> > https://www.dropbox.com/sh/6os9yfn22jjcnmy/AAAjEV3u06XQ1t3UXdWcgru0a?dl=0
> > > >
> > > > Thanks,
> > > > Lyn
> > > >
> > > I see the problem.  none of the constructors are included in the l3fwd
> > app.  I
> > > think you might need 174572477028b7f04700389e38f87d2ba01a0adc.  Its also
> > > possible something else has gone wrong withthe COMBINE_LIBS option.  Try
> > > building with the head of the tree, and if that doesn't fix it, we can
> > start
> > > debugging in earnest.
> > > Neil
> >
> > Found the problem.  Please try this patch
> >
> >
> > diff --git a/mk/rte.app.mk b/mk/rte.app.mk
> > index e1a0dbf..40afb2c 100644
> > --- a/mk/rte.app.mk
> > +++ b/mk/rte.app.mk
> > @@ -61,6 +61,10 @@ ifeq ($(NO_AUTOLIBS),)
> >
> >  LDLIBS += --whole-archive
> >
> > +ifeq ($(CONFIG_RTE_BUILD_COMBINE_LIBS),y)
> > +LDLIBS += -l$(RTE_LIBNAME)
> > +endif
> > +
> >  ifeq ($(CONFIG_RTE_BUILD_COMBINE_LIBS),n)
> >
> >  ifeq ($(CONFIG_RTE_LIBRTE_DISTRIBUTOR),y)
> > @@ -251,10 +255,6 @@ build: _postbuild
> >
> >  exe2cmd = $(strip $(call dotfile,$(patsubst %,%.cmd,$(1))))
> >
> > -ifeq ($(CONFIG_RTE_BUILD_COMBINE_LIBS),y)
> > -LDLIBS += -l$(RTE_LIBNAME)
> > -endif
> > -
> >  ifeq ($(LINK_USING_CC),1)
> >  override EXTRA_LDFLAGS := $(call linkerprefix,$(EXTRA_LDFLAGS))
> >  O_TO_EXE = $(CC) $(CFLAGS) $(LDFLAGS_$(@)) \
> >
  

On 1/2/2015 11:08 PM, Lyn M wrote:
> Neil, your patch works.  I am now able to run testpmd, as well as l3fwd
> with DPDK 1.8.0 and CONFIG_RTE_BUILD_COMBINE_LIBS=y.  Thank you much!
>
> In my original post, I thought my choice of hex portmask -p 0x3 was causing
> this issue -- now I know that was not the case.  But, I am still curious
> about how the hex portmask is determined.  Since I only have two ports
> bound to igb_uio, my hex postmask will always be 0x3?  What if I choose to
> bind other ports to igb_uio?  Is there a Linux command I can run to
> determine what mask to use with the -p option?

It is simply a mask used to enable or disable ports in DPDK.
For example:
If DPDK owns two ports, -p 0x03 means all two ports are enabled.
If DPDK owns four ports, -p 0x03 means only port 0 and 1 are enabled,
port 2 and 3 are unable to use. If you want to enable all ports, you
should use -p 0x0f

0x03 is 0b11, 0x0f is 0b1111

Thanks,
Michael
> Happy New Year --
> Lyn
>
> On Wed, Dec 31, 2014 at 10:00 PM, Neil Horman <nhorman@tuxdriver.com> wrote:
>
>> On Wed, Dec 31, 2014 at 10:37:45PM -0500, Neil Horman wrote:
>>> On Wed, Dec 31, 2014 at 10:50:10AM -0600, Lyn M wrote:
>>>> Neil,
>>>> When I build DPDK 1.8.0 with the default value
>> CONFIG_RTE_BUILD_COMBINE_LIBS=n,
>>>> I am able to run the testpmd successfully.  So, that does appear to be
>> the
>>>> culprit.
>>>> The objdump -t output for testpmd (with COMBINE_LIBS=n and
>> COMBINE_LIBS=y)
>>>> files are here:
>>>>
>> https://www.dropbox.com/sh/6os9yfn22jjcnmy/AAAjEV3u06XQ1t3UXdWcgru0a?dl=0
>>>> Thanks,
>>>> Lyn
>>>>
>>> I see the problem.  none of the constructors are included in the l3fwd
>> app.  I
>>> think you might need 174572477028b7f04700389e38f87d2ba01a0adc.  Its also
>>> possible something else has gone wrong withthe COMBINE_LIBS option.  Try
>>> building with the head of the tree, and if that doesn't fix it, we can
>> start
>>> debugging in earnest.
>>> Neil
>> Found the problem.  Please try this patch
>>
>>
>> diff --git a/mk/rte.app.mk b/mk/rte.app.mk
>> index e1a0dbf..40afb2c 100644
>> --- a/mk/rte.app.mk
>> +++ b/mk/rte.app.mk
>> @@ -61,6 +61,10 @@ ifeq ($(NO_AUTOLIBS),)
>>
>>  LDLIBS += --whole-archive
>>
>> +ifeq ($(CONFIG_RTE_BUILD_COMBINE_LIBS),y)
>> +LDLIBS += -l$(RTE_LIBNAME)
>> +endif
>> +
>>  ifeq ($(CONFIG_RTE_BUILD_COMBINE_LIBS),n)
>>
>>  ifeq ($(CONFIG_RTE_LIBRTE_DISTRIBUTOR),y)
>> @@ -251,10 +255,6 @@ build: _postbuild
>>
>>  exe2cmd = $(strip $(call dotfile,$(patsubst %,%.cmd,$(1))))
>>
>> -ifeq ($(CONFIG_RTE_BUILD_COMBINE_LIBS),y)
>> -LDLIBS += -l$(RTE_LIBNAME)
>> -endif
>> -
>>  ifeq ($(LINK_USING_CC),1)
>>  override EXTRA_LDFLAGS := $(call linkerprefix,$(EXTRA_LDFLAGS))
>>  O_TO_EXE = $(CC) $(CFLAGS) $(LDFLAGS_$(@)) \
>>
  

2015-01-02 09:07, Lyn M:
> In my original post, I thought my choice of hex portmask -p 0x3 was causing
> this issue -- now I know that was not the case.  But, I am still curious
> about how the hex portmask is determined.  Since I only have two ports
> bound to igb_uio, my hex postmask will always be 0x3?  What if I choose to
> bind other ports to igb_uio?  Is there a Linux command I can run to
> determine what mask to use with the -p option?

That's something which could be done.
Port numbering is defined by probing order. Does a tool showing the (dry-run)
probed devices, their number and mask, would help you?
By the way, you are welcome to try developing it.
  

On Tue, Jan 06, 2015 at 10:02:25AM +0100, Thomas Monjalon wrote:
> 2015-01-02 09:07, Lyn M:
> > In my original post, I thought my choice of hex portmask -p 0x3 was causing
> > this issue -- now I know that was not the case.  But, I am still curious
> > about how the hex portmask is determined.  Since I only have two ports
> > bound to igb_uio, my hex postmask will always be 0x3?  What if I choose to
> > bind other ports to igb_uio?  Is there a Linux command I can run to
> > determine what mask to use with the -p option?
> 
> That's something which could be done.
> Port numbering is defined by probing order. Does a tool showing the (dry-run)
> probed devices, their number and mask, would help you?
> By the way, you are welcome to try developing it.
> 
> -- 
> Thomas

The port numbering should generally be in the same order as the ports are listed
in dpdk_nic_bind script, so perhaps we could just add a documentation note to 
that effect.

/Bruce