diff mbox series

eal/freebsd: lock memory device to prevent conflicts

Message ID 20210913110635.130387-1-bruce.richardson@intel.com (mailing list archive)
State Superseded, archived
Headers
Series eal/freebsd: lock memory device to prevent conflicts |

Checks

Context Check Description
ci/checkpatch success coding style OK
ci/github-robot: build success github build: passed

Commit Message

Bruce Richardson Sept. 13, 2021, 11:06 a.m. UTC 
  Only a single DPDK process on the system can be using the /dev/contigmem
mappings at a time, but this was never explicitly enforced, e.g. when
using --in-memory flag on two processes. To prevent possible conflict
issues, we lock the dev node when it's in use, preventing other DPDK
processes from starting up and causing problems for us.

Fixes: 764bf26873b9 ("add FreeBSD support")
Cc: stable@dpdk.org

Signed-off-by: Bruce Richardson <bruce.richardson@intel.com>
---
 lib/eal/freebsd/eal_hugepage_info.c | 4 ++++
 1 file changed, 4 insertions(+)

Comments

Anatoly Burakov Sept. 13, 2021, 1:14 p.m. UTC | #1 
On 13-Sep-21 12:06 PM, Bruce Richardson wrote:
> Only a single DPDK process on the system can be using the /dev/contigmem
> mappings at a time, but this was never explicitly enforced, e.g. when
> using --in-memory flag on two processes. To prevent possible conflict
> issues, we lock the dev node when it's in use, preventing other DPDK
> processes from starting up and causing problems for us.
> 
> Fixes: 764bf26873b9 ("add FreeBSD support")
> Cc: stable@dpdk.org
> 
> Signed-off-by: Bruce Richardson <bruce.richardson@intel.com>
> ---
>   lib/eal/freebsd/eal_hugepage_info.c | 4 ++++
>   1 file changed, 4 insertions(+)
> 
> diff --git a/lib/eal/freebsd/eal_hugepage_info.c b/lib/eal/freebsd/eal_hugepage_info.c
> index 408f054f7a..4a8d87c23e 100644
> --- a/lib/eal/freebsd/eal_hugepage_info.c
> +++ b/lib/eal/freebsd/eal_hugepage_info.c
> @@ -90,6 +90,10 @@ eal_hugepage_info_init(void)
>   		RTE_LOG(ERR, EAL, "could not open "CONTIGMEM_DEV"\n");
>   		return -1;
>   	}
> +	if (flock(fd, LOCK_EX) < 0) {
> +		RTE_LOG(ERR, EAL, "could not lock memory. Is another DPDK process running?\n");
> +		return -1;
> +	}
>   
>   	if (buffer_size >= 1<<30)
>   		RTE_LOG(INFO, EAL, "Contigmem driver has %d buffers, each of size %dGB\n",
> 

This only gets triggered when regular init path is chosen, i.e. 
--no-huge still works. I'm a bit uneasy with --in-memory mode pretending 
to work on FreeBSD and Windows, but that's a separate problem :) As far 
as the patch goes, the problem it addresses does get fixed.

Reviewed-by: Anatoly Burakov <anatoly.burakov@intel.com>
Bruce Richardson Sept. 13, 2021, 1:36 p.m. UTC | #2 
On Mon, Sep 13, 2021 at 02:14:55PM +0100, Burakov, Anatoly wrote:
> On 13-Sep-21 12:06 PM, Bruce Richardson wrote:
> > Only a single DPDK process on the system can be using the /dev/contigmem
> > mappings at a time, but this was never explicitly enforced, e.g. when
> > using --in-memory flag on two processes. To prevent possible conflict
> > issues, we lock the dev node when it's in use, preventing other DPDK
> > processes from starting up and causing problems for us.
> > 
> > Fixes: 764bf26873b9 ("add FreeBSD support")
> > Cc: stable@dpdk.org
> > 
> > Signed-off-by: Bruce Richardson <bruce.richardson@intel.com>
> > ---
> >   lib/eal/freebsd/eal_hugepage_info.c | 4 ++++
> >   1 file changed, 4 insertions(+)
> > 
> > diff --git a/lib/eal/freebsd/eal_hugepage_info.c b/lib/eal/freebsd/eal_hugepage_info.c
> > index 408f054f7a..4a8d87c23e 100644
> > --- a/lib/eal/freebsd/eal_hugepage_info.c
> > +++ b/lib/eal/freebsd/eal_hugepage_info.c
> > @@ -90,6 +90,10 @@ eal_hugepage_info_init(void)
> >   		RTE_LOG(ERR, EAL, "could not open "CONTIGMEM_DEV"\n");
> >   		return -1;
> >   	}
> > +	if (flock(fd, LOCK_EX) < 0) {
> > +		RTE_LOG(ERR, EAL, "could not lock memory. Is another DPDK process running?\n");
> > +		return -1;
> > +	}
> >   	if (buffer_size >= 1<<30)
> >   		RTE_LOG(INFO, EAL, "Contigmem driver has %d buffers, each of size %dGB\n",
> > 
> 
> This only gets triggered when regular init path is chosen, i.e. --no-huge
> still works.

Yes, but that is ok, I think, since no-huge doesn't use these resources or
suffer from this problem. On the other hand, except for running unit tests,
no-huge mode is pretty useless on FreeBSD as we don't have any
vfio-equivalent support, so all HW access has to use physical addresses
which can only be got using contigmem.

> I'm a bit uneasy with --in-memory mode pretending to work on
> FreeBSD and Windows, but that's a separate problem :)

Yes, it is, though one that does belong is the same area as this one. The
"fix" is probably to just print a warning when --in-memory is used,
informing the user that the flag is ignored and then continue.
Alternatively we can error out, but I think the warn+continue is better,
myself.

> As far as the patch
> goes, the problem it addresses does get fixed.
> 
> Reviewed-by: Anatoly Burakov <anatoly.burakov@intel.com>
> 
Thanks.

/Bruce
Dmitry Kozlyuk Sept. 13, 2021, 1:59 p.m. UTC | #3 
2021-09-13 14:14 (UTC+0100), Burakov, Anatoly:
> [...]
> I'm a bit uneasy with --in-memory mode pretending 
> to work on FreeBSD and Windows, but that's a separate problem :)

On Windows, --in-memory does not pretend to work, just the opposite,
it is enabled implicitly as it's the only working mode.
Anatoly Burakov Sept. 13, 2021, 2:40 p.m. UTC | #4 
On 13-Sep-21 2:36 PM, Bruce Richardson wrote:
> On Mon, Sep 13, 2021 at 02:14:55PM +0100, Burakov, Anatoly wrote:
>> On 13-Sep-21 12:06 PM, Bruce Richardson wrote:
>>> Only a single DPDK process on the system can be using the /dev/contigmem
>>> mappings at a time, but this was never explicitly enforced, e.g. when
>>> using --in-memory flag on two processes. To prevent possible conflict
>>> issues, we lock the dev node when it's in use, preventing other DPDK
>>> processes from starting up and causing problems for us.
>>>
>>> Fixes: 764bf26873b9 ("add FreeBSD support")
>>> Cc: stable@dpdk.org
>>>
>>> Signed-off-by: Bruce Richardson <bruce.richardson@intel.com>
>>> ---
>>>    lib/eal/freebsd/eal_hugepage_info.c | 4 ++++
>>>    1 file changed, 4 insertions(+)
>>>
>>> diff --git a/lib/eal/freebsd/eal_hugepage_info.c b/lib/eal/freebsd/eal_hugepage_info.c
>>> index 408f054f7a..4a8d87c23e 100644
>>> --- a/lib/eal/freebsd/eal_hugepage_info.c
>>> +++ b/lib/eal/freebsd/eal_hugepage_info.c
>>> @@ -90,6 +90,10 @@ eal_hugepage_info_init(void)
>>>    		RTE_LOG(ERR, EAL, "could not open "CONTIGMEM_DEV"\n");
>>>    		return -1;
>>>    	}
>>> +	if (flock(fd, LOCK_EX) < 0) {
>>> +		RTE_LOG(ERR, EAL, "could not lock memory. Is another DPDK process running?\n");
>>> +		return -1;
>>> +	}
>>>    	if (buffer_size >= 1<<30)
>>>    		RTE_LOG(INFO, EAL, "Contigmem driver has %d buffers, each of size %dGB\n",
>>>
>>
>> This only gets triggered when regular init path is chosen, i.e. --no-huge
>> still works.
> 
> Yes, but that is ok, I think, since no-huge doesn't use these resources or
> suffer from this problem. On the other hand, except for running unit tests,
> no-huge mode is pretty useless on FreeBSD as we don't have any
> vfio-equivalent support, so all HW access has to use physical addresses
> which can only be got using contigmem.

What i meant to say was, i've checked this against '--no-huge' which 
*should* still work with this patch, and it does :) So, the phrasing was 
unfortunate, but we agree!

> 
>> I'm a bit uneasy with --in-memory mode pretending to work on
>> FreeBSD and Windows, but that's a separate problem :)
> 
> Yes, it is, though one that does belong is the same area as this one. The
> "fix" is probably to just print a warning when --in-memory is used,
> informing the user that the flag is ignored and then continue.
> Alternatively we can error out, but I think the warn+continue is better,
> myself.

I think erroring out is better. The feature is intended to work a 
certain way, so if we can't guarantee that it does, we can't pretend it 
is "supported" or "is working". But again, irrelevant to this patch :)

> 
>> As far as the patch
>> goes, the problem it addresses does get fixed.
>>
>> Reviewed-by: Anatoly Burakov <anatoly.burakov@intel.com>
>>
> Thanks.
> 
> /Bruce
>
David Marchand Oct. 2, 2021, 2:42 p.m. UTC | #5 
On Mon, Sep 13, 2021 at 3:15 PM Burakov, Anatoly
<anatoly.burakov@intel.com> wrote:
> On 13-Sep-21 12:06 PM, Bruce Richardson wrote:
> > Only a single DPDK process on the system can be using the /dev/contigmem
> > mappings at a time, but this was never explicitly enforced, e.g. when
> > using --in-memory flag on two processes. To prevent possible conflict
> > issues, we lock the dev node when it's in use, preventing other DPDK
> > processes from starting up and causing problems for us.
> >
> > Fixes: 764bf26873b9 ("add FreeBSD support")
> > Cc: stable@dpdk.org
> >
> > Signed-off-by: Bruce Richardson <bruce.richardson@intel.com>
> Reviewed-by: Anatoly Burakov <anatoly.burakov@intel.com>

Applied, thanks.
David Marchand Oct. 2, 2021, 2:43 p.m. UTC | #6 
On Mon, Sep 13, 2021 at 4:40 PM Burakov, Anatoly
<anatoly.burakov@intel.com> wrote:
> >> I'm a bit uneasy with --in-memory mode pretending to work on
> >> FreeBSD and Windows, but that's a separate problem :)
> >
> > Yes, it is, though one that does belong is the same area as this one. The
> > "fix" is probably to just print a warning when --in-memory is used,
> > informing the user that the flag is ignored and then continue.
> > Alternatively we can error out, but I think the warn+continue is better,
> > myself.
>
> I think erroring out is better. The feature is intended to work a
> certain way, so if we can't guarantee that it does, we can't pretend it
> is "supported" or "is working". But again, irrelevant to this patch :)
>

Please, can you review
https://patchwork.dpdk.org/project/dpdk/patch/20210913140848.184928-1-bruce.richardson@intel.com/
?
Thanks.
diff mbox series

Patch

diff --git a/lib/eal/freebsd/eal_hugepage_info.c b/lib/eal/freebsd/eal_hugepage_info.c
index 408f054f7a..4a8d87c23e 100644
--- a/lib/eal/freebsd/eal_hugepage_info.c
+++ b/lib/eal/freebsd/eal_hugepage_info.c
@@ -90,6 +90,10 @@  eal_hugepage_info_init(void)
 		RTE_LOG(ERR, EAL, "could not open "CONTIGMEM_DEV"\n");
 		return -1;
 	}
+	if (flock(fd, LOCK_EX) < 0) {
+		RTE_LOG(ERR, EAL, "could not lock memory. Is another DPDK process running?\n");
+		return -1;
+	}
 
 	if (buffer_size >= 1<<30)
 		RTE_LOG(INFO, EAL, "Contigmem driver has %d buffers, each of size %dGB\n",