[dpdk-dev,v2] log: respect rte_openlog_stream calls before rte_eal_init
Commit Message
Before this patch, application-specific loggers could not be
installed before rte_eal_init completed (the initialization process
called rte_openlog_stream, overwriting any previously installed
logger). This made it impossible for an application to capture the
initial log messages generated during rte_eal_init. This patch changes
initialization so that information from a previous call to
rte_openlog_stream is not lost. Specifically:
* The default log stream is now maintained separately from an
application-specific log stream installed with rte_openlog_stream.
* rte_eal_common_log_init has been renamed to rte_eal_log_set_default,
since this is all it does. It no longer invokes rte_openlog_stream; it
just updates the default stream. Also, this method now returns void,
rather than int, since there are no errors.
* The "early log" mechanism (e.g. rte_eal_log_early_init) has been
removed; all of the desired functionality can be achieved by calling
rte_eal_log_set_default.
Signed-off-by: John Ousterhout <ouster@cs.stanford.edu>
----
v2:
* Removed the early log mechanism, renamed rte_eal_common_log_init.
Note: I see from the code that Linux and BSD set different default streams:
Linux uses stdout, while BSD uses stderr. This patch retains the distinction,
though I'm not sure why it is there.
---
lib/librte_eal/bsdapp/eal/eal.c | 3 +--
lib/librte_eal/bsdapp/eal/eal_log.c | 11 +--------
lib/librte_eal/common/eal_common_log.c | 19 +++++++---------
lib/librte_eal/common/eal_private.h | 16 ++++---------
lib/librte_eal/common/include/rte_log.h | 2 +-
lib/librte_eal/linuxapp/eal/eal.c | 3 +--
lib/librte_eal/linuxapp/eal/eal_log.c | 40 +--------------------------------
7 files changed, 17 insertions(+), 77 deletions(-)
Comments
2016-10-10 15:39, John Ousterhout:
> Before this patch, application-specific loggers could not be
> installed before rte_eal_init completed (the initialization process
> called rte_openlog_stream, overwriting any previously installed
> logger). This made it impossible for an application to capture the
> initial log messages generated during rte_eal_init. This patch changes
> initialization so that information from a previous call to
> rte_openlog_stream is not lost. Specifically:
> * The default log stream is now maintained separately from an
> application-specific log stream installed with rte_openlog_stream.
> * rte_eal_common_log_init has been renamed to rte_eal_log_set_default,
> since this is all it does. It no longer invokes rte_openlog_stream; it
> just updates the default stream. Also, this method now returns void,
> rather than int, since there are no errors.
> * The "early log" mechanism (e.g. rte_eal_log_early_init) has been
> removed; all of the desired functionality can be achieved by calling
> rte_eal_log_set_default.
>
> Signed-off-by: John Ousterhout <ouster@cs.stanford.edu>
> ----
> v2:
> * Removed the early log mechanism, renamed rte_eal_common_log_init.
>
> Note: I see from the code that Linux and BSD set different default streams:
> Linux uses stdout, while BSD uses stderr. This patch retains the distinction,
> though I'm not sure why it is there.
I don't know either.
What is best between stdout and stderr for logs?
[...]
> -int
> -rte_eal_log_early_init(void)
> -{
> - rte_openlog_stream(stderr);
> - return 0;
> + rte_eal_set_default(stderr);
It should be rte_eal_log_set_default.
[...]
> /*
> - * called by environment-specific log init function
> + * Called by environment-specific initialization functions.
> */
> -int
> -rte_eal_common_log_init(FILE *default_log)
> +void
> +rte_eal_log_set_default(FILE *default_log)
> {
> default_log_stream = default_log;
> - rte_openlog_stream(default_log);
>
> #if RTE_LOG_LEVEL >= RTE_LOG_DEBUG
> RTE_LOG(NOTICE, EAL, "Debug logs available - lower performance\n");
> #endif
> -
> - return 0;
> }
Do we really need a function for that?
Why not just initialize default_log_stream statically?
[...]
> /**
> - * Common log initialization function (private to eal).
> + * Common log initialization function (private to eal). Determines
> + * where log data is written when no call to eal_openlog_stream is
> + * in effect.
It should be rte_openlog_stream.
On Tue, Oct 11, 2016 at 1:08 AM, Thomas Monjalon <thomas.monjalon@6wind.com>
wrote:
>
> 2016-10-10 15:39, John Ousterhout:
> > ...
> >
> > Note: I see from the code that Linux and BSD set different default
streams:
> > Linux uses stdout, while BSD uses stderr. This patch retains the
distinction,
> > though I'm not sure why it is there.
>
> I don't know either.
> What is best between stdout and stderr for logs?
I would guess that stdout makes more sense, since most log entries describe
normal operation, not errors. I'm happy to make these consistent, but this
would introduce a behavior change for BSD (which currently uses stderr);
would that be considered antisocial?
> [...]
> > -int
> > -rte_eal_log_early_init(void)
> > -{
> > - rte_openlog_stream(stderr);
> > - return 0;
> > + rte_eal_set_default(stderr);
>
> It should be rte_eal_log_set_default.
Oops, right; will fix.
>
> [...]
> > /*
> > - * called by environment-specific log init function
> > + * Called by environment-specific initialization functions.
> > */
> > -int
> > -rte_eal_common_log_init(FILE *default_log)
> > +void
> > +rte_eal_log_set_default(FILE *default_log)
> > {
> > default_log_stream = default_log;
> > - rte_openlog_stream(default_log);
> >
> > #if RTE_LOG_LEVEL >= RTE_LOG_DEBUG
> > RTE_LOG(NOTICE, EAL, "Debug logs available - lower
performance\n");
> > #endif
> > -
> > - return 0;
> > }
>
> Do we really need a function for that?
> Why not just initialize default_log_stream statically?
Right now, different platforms have different defaults. BSD uses stderr
always. Linux starts out with stdout as the default, but later during
initialization it switches to a different default that logs messages both
to standard output and also to syslog. I don't have enough experience with
DPDK to know whether a single approach is really right for all systems (or
which approach it should be), and since I'm a DPDK newbie I thought it best
to take a more conservative approach and avoid behavioral changes. My
personal preference would be to minimize mission creep with this patch and
leave that behavior in place for someone with more expertise to deal with
later (and this issue is orthogonal to the main goal of the patch). But, if
unifying the log defaults is considered essential to the patch (and is
noncontroversial), I'm willing to implement it.
> [...]
> > /**
> > - * Common log initialization function (private to eal).
> > + * Common log initialization function (private to eal). Determines
> > + * where log data is written when no call to eal_openlog_stream is
> > + * in effect.
>
> It should be rte_openlog_stream.
Oops; fixed.
Thanks for the comments.
-John-
2016-10-11 09:30, John Ousterhout:
> On Tue, Oct 11, 2016 at 1:08 AM, Thomas Monjalon <thomas.monjalon@6wind.com>
> wrote:
> >
> > 2016-10-10 15:39, John Ousterhout:
> > > ...
> > >
> > > Note: I see from the code that Linux and BSD set different default
> streams:
> > > Linux uses stdout, while BSD uses stderr. This patch retains the
> distinction,
> > > though I'm not sure why it is there.
> >
> > I don't know either.
> > What is best between stdout and stderr for logs?
>
> I would guess that stdout makes more sense, since most log entries describe
> normal operation, not errors. I'm happy to make these consistent, but this
> would introduce a behavior change for BSD (which currently uses stderr);
> would that be considered antisocial?
No, that's OK to use stdout on BSD.
> > > -int
> > > -rte_eal_common_log_init(FILE *default_log)
> > > +void
> > > +rte_eal_log_set_default(FILE *default_log)
> > > {
> > > default_log_stream = default_log;
> > > - rte_openlog_stream(default_log);
> > >
> > > #if RTE_LOG_LEVEL >= RTE_LOG_DEBUG
> > > RTE_LOG(NOTICE, EAL, "Debug logs available - lower
> performance\n");
> > > #endif
> > > -
> > > - return 0;
> > > }
> >
> > Do we really need a function for that?
> > Why not just initialize default_log_stream statically?
>
> Right now, different platforms have different defaults. BSD uses stderr
> always. Linux starts out with stdout as the default, but later during
> initialization it switches to a different default that logs messages both
> to standard output and also to syslog. I don't have enough experience with
> DPDK to know whether a single approach is really right for all systems (or
> which approach it should be), and since I'm a DPDK newbie I thought it best
> to take a more conservative approach and avoid behavioral changes. My
> personal preference would be to minimize mission creep with this patch and
> leave that behavior in place for someone with more expertise to deal with
> later (and this issue is orthogonal to the main goal of the patch). But, if
> unifying the log defaults is considered essential to the patch (and is
> noncontroversial), I'm willing to implement it.
OK sorry, I'm mixing things.
1/ When removing early log functions, you are replacing early init with
a default set to stderr/stdout via rte_eal_log_set_default.
I think you can just set statically to stdout:
static FILE *default_log_stream = stdout;
2/ Yes, on Linux, a more complex stream with stdout + syslog is set.
It is OK to use rte_eal_log_set_default for that usage.
Note that there is a stream which is not used and can be removed in eal_private.h:
extern FILE *eal_default_log_stream;
Other note: rte_eal_log_set_default is not a public function so should be
named eal_log_set_default.
3/ When calling rte_eal_log_set_default on BSD from rte_eal_log_init,
you can keep stderr but an empty function would be better because
it is not called and already set (to stderr or stdout if 1/).
4/ rte_eal_log_init can be called earlier to replace early init.
5/ It would be simpler to understand by splitting in two patches
(remove early log + use non default log)
I understand that you prefer to focus on your fix and I'm more or less
suggesting a cleanup of logging. That's why I can do the first cleanup
patch if you are really not confortable with it. (I feel you could do it)
Just let me know.
All of your suggestions look reasonable and fairly straightforward; I'll
work on a new patch that includes them.
Given that rte_eal_log_init is a no-op (and won't even be invoked), would
it be better to remove that function completely, and even delete the file
containing it (eal_log.c), or is it better to retain the empty function in
order to maintain a parallel structure with Linux? Personally I'd lean
towards deleting the file. As it stands, the interface to that function
doesn't even make sense for BSD; the arguments were chosen for Linux and
are ignored in BSD.
Let me know your preference.
-John-
On Tue, Oct 11, 2016 at 1:30 PM, Thomas Monjalon <thomas.monjalon@6wind.com>
wrote:
> 2016-10-11 09:30, John Ousterhout:
> > On Tue, Oct 11, 2016 at 1:08 AM, Thomas Monjalon <
> thomas.monjalon@6wind.com>
> > wrote:
> > >
> > > 2016-10-10 15:39, John Ousterhout:
> > > > ...
> > > >
> > > > Note: I see from the code that Linux and BSD set different default
> > streams:
> > > > Linux uses stdout, while BSD uses stderr. This patch retains the
> > distinction,
> > > > though I'm not sure why it is there.
> > >
> > > I don't know either.
> > > What is best between stdout and stderr for logs?
> >
> > I would guess that stdout makes more sense, since most log entries
> describe
> > normal operation, not errors. I'm happy to make these consistent, but
> this
> > would introduce a behavior change for BSD (which currently uses stderr);
> > would that be considered antisocial?
>
> No, that's OK to use stdout on BSD.
>
> > > > -int
> > > > -rte_eal_common_log_init(FILE *default_log)
> > > > +void
> > > > +rte_eal_log_set_default(FILE *default_log)
> > > > {
> > > > default_log_stream = default_log;
> > > > - rte_openlog_stream(default_log);
> > > >
> > > > #if RTE_LOG_LEVEL >= RTE_LOG_DEBUG
> > > > RTE_LOG(NOTICE, EAL, "Debug logs available - lower
> > performance\n");
> > > > #endif
> > > > -
> > > > - return 0;
> > > > }
> > >
> > > Do we really need a function for that?
> > > Why not just initialize default_log_stream statically?
> >
> > Right now, different platforms have different defaults. BSD uses stderr
> > always. Linux starts out with stdout as the default, but later during
> > initialization it switches to a different default that logs messages both
> > to standard output and also to syslog. I don't have enough experience
> with
> > DPDK to know whether a single approach is really right for all systems
> (or
> > which approach it should be), and since I'm a DPDK newbie I thought it
> best
> > to take a more conservative approach and avoid behavioral changes. My
> > personal preference would be to minimize mission creep with this patch
> and
> > leave that behavior in place for someone with more expertise to deal with
> > later (and this issue is orthogonal to the main goal of the patch). But,
> if
> > unifying the log defaults is considered essential to the patch (and is
> > noncontroversial), I'm willing to implement it.
>
> OK sorry, I'm mixing things.
>
> 1/ When removing early log functions, you are replacing early init with
> a default set to stderr/stdout via rte_eal_log_set_default.
> I think you can just set statically to stdout:
> static FILE *default_log_stream = stdout;
>
> 2/ Yes, on Linux, a more complex stream with stdout + syslog is set.
> It is OK to use rte_eal_log_set_default for that usage.
> Note that there is a stream which is not used and can be removed in
> eal_private.h:
> extern FILE *eal_default_log_stream;
> Other note: rte_eal_log_set_default is not a public function so should be
> named eal_log_set_default.
>
> 3/ When calling rte_eal_log_set_default on BSD from rte_eal_log_init,
> you can keep stderr but an empty function would be better because
> it is not called and already set (to stderr or stdout if 1/).
>
> 4/ rte_eal_log_init can be called earlier to replace early init.
>
> 5/ It would be simpler to understand by splitting in two patches
> (remove early log + use non default log)
>
> I understand that you prefer to focus on your fix and I'm more or less
> suggesting a cleanup of logging. That's why I can do the first cleanup
> patch if you are really not confortable with it. (I feel you could do it)
> Just let me know.
>
> -----Original Message-----
> From: John Ousterhout [mailto:ouster@cs.stanford.edu]
> Sent: Tuesday, October 11, 2016 9:30 AM
> To: Thomas Monjalon <thomas.monjalon@6wind.com>
> Cc: dev@dpdk.org
> Subject: Re: [dpdk-dev] [PATCH v2] log: respect rte_openlog_stream calls
> before rte_eal_init
>
> On Tue, Oct 11, 2016 at 1:08 AM, Thomas Monjalon
> <thomas.monjalon@6wind.com>
> wrote:
> > I don't know either.
> > What is best between stdout and stderr for logs?
>
> I would guess that stdout makes more sense, since most log entries describe
> normal operation, not errors. I'm happy to make these consistent, but this
> would introduce a behavior change for BSD (which currently uses stderr);
> would that be considered antisocial?
I've never seen a pronouncement or anything, but as a linux programmer,
my attitude is that stdout should be the output the application is producing
when carrying out its function. Debugging output isn't part of what the
application is trying to accomplish, so it should be sent to stderr where it
can be segregated from the functional output when needed.
-don
dprovan@bivio.net
Don's argument for stderr over stdout makes sense to me. Does anyone else
disagree?
-John-
On Tue, Oct 11, 2016 at 3:16 PM, Don Provan <dprovan@bivio.net> wrote:
> > -----Original Message-----
> > From: John Ousterhout [mailto:ouster@cs.stanford.edu]
> > Sent: Tuesday, October 11, 2016 9:30 AM
> > To: Thomas Monjalon <thomas.monjalon@6wind.com>
> > Cc: dev@dpdk.org
> > Subject: Re: [dpdk-dev] [PATCH v2] log: respect rte_openlog_stream calls
> > before rte_eal_init
> >
> > On Tue, Oct 11, 2016 at 1:08 AM, Thomas Monjalon
> > <thomas.monjalon@6wind.com>
> > wrote:
> > > I don't know either.
> > > What is best between stdout and stderr for logs?
> >
> > I would guess that stdout makes more sense, since most log entries
> describe
> > normal operation, not errors. I'm happy to make these consistent, but
> this
> > would introduce a behavior change for BSD (which currently uses stderr);
> > would that be considered antisocial?
>
> I've never seen a pronouncement or anything, but as a linux programmer,
> my attitude is that stdout should be the output the application is
> producing
> when carrying out its function. Debugging output isn't part of what the
> application is trying to accomplish, so it should be sent to stderr where
> it
> can be segregated from the functional output when needed.
> -don
> dprovan@bivio.net
>
>
2016-10-11 14:46, John Ousterhout:
> All of your suggestions look reasonable and fairly straightforward; I'll
> work on a new patch that includes them.
>
> Given that rte_eal_log_init is a no-op (and won't even be invoked), would
> it be better to remove that function completely, and even delete the file
> containing it (eal_log.c), or is it better to retain the empty function in
> order to maintain a parallel structure with Linux? Personally I'd lean
> towards deleting the file. As it stands, the interface to that function
> doesn't even make sense for BSD; the arguments were chosen for Linux and
> are ignored in BSD.
>
> Let me know your preference.
Yes you can remove the file.
@@ -501,8 +501,7 @@ rte_eal_init(int argc, char **argv)
thread_id = pthread_self();
- if (rte_eal_log_early_init() < 0)
- rte_panic("Cannot init early logs\n");
+ rte_eal_log_set_default(stderr);
eal_log_level_parse(argc, argv);
@@ -44,14 +44,5 @@
int
rte_eal_log_init(const char *id __rte_unused, int facility __rte_unused)
{
- if (rte_eal_common_log_init(stderr) < 0)
- return -1;
- return 0;
-}
-
-int
-rte_eal_log_early_init(void)
-{
- rte_openlog_stream(stderr);
- return 0;
+ rte_eal_set_default(stderr);
}
@@ -48,11 +48,12 @@ struct rte_logs rte_logs = {
.file = NULL,
};
+/* Stream to use for logging if rte_logs.file is NULL */
static FILE *default_log_stream;
/**
* This global structure stores some informations about the message
- * that is currently beeing processed by one lcore
+ * that is currently being processed by one lcore
*/
struct log_cur_msg {
uint32_t loglevel; /**< log level - see rte_log.h */
@@ -68,10 +69,7 @@ static RTE_DEFINE_PER_LCORE(struct log_cur_msg, log_cur_msg);
int
rte_openlog_stream(FILE *f)
{
- if (f == NULL)
- rte_logs.file = default_log_stream;
- else
- rte_logs.file = f;
+ rte_logs.file = f;
return 0;
}
@@ -127,6 +125,8 @@ rte_vlog(uint32_t level, uint32_t logtype, const char *format, va_list ap)
{
int ret;
FILE *f = rte_logs.file;
+ if (f == NULL)
+ f = default_log_stream;
if ((level > rte_logs.level) || !(logtype & rte_logs.type))
return 0;
@@ -158,17 +158,14 @@ rte_log(uint32_t level, uint32_t logtype, const char *format, ...)
}
/*
- * called by environment-specific log init function
+ * Called by environment-specific initialization functions.
*/
-int
-rte_eal_common_log_init(FILE *default_log)
+void
+rte_eal_log_set_default(FILE *default_log)
{
default_log_stream = default_log;
- rte_openlog_stream(default_log);
#if RTE_LOG_LEVEL >= RTE_LOG_DEBUG
RTE_LOG(NOTICE, EAL, "Debug logs available - lower performance\n");
#endif
-
- return 0;
}
@@ -47,7 +47,9 @@
int rte_eal_memzone_init(void);
/**
- * Common log initialization function (private to eal).
+ * Common log initialization function (private to eal). Determines
+ * where log data is written when no call to eal_openlog_stream is
+ * in effect.
*
* @param default_log
* The default log stream to be used.
@@ -55,7 +57,7 @@ int rte_eal_memzone_init(void);
* - 0 on success
* - Negative on error
*/
-int rte_eal_common_log_init(FILE *default_log);
+void rte_eal_log_set_default(FILE *default_log);
/**
* Fill configuration with number of physical and logical processors
@@ -97,16 +99,6 @@ int rte_eal_memory_init(void);
int rte_eal_timer_init(void);
/**
- * Init early logs
- *
- * This function is private to EAL.
- *
- * @return
- * 0 on success, negative on error
- */
-int rte_eal_log_early_init(void);
-
-/**
* Init the default log stream
*
* This function is private to EAL.
@@ -54,7 +54,7 @@ extern "C" {
struct rte_logs {
uint32_t type; /**< Bitfield with enabled logs. */
uint32_t level; /**< Log level. */
- FILE *file; /**< Pointer to current FILE* for logs. */
+ FILE *file; /**< Output file set by rte_openlog_stream, or NULL. */
};
/** Global log informations */
@@ -748,8 +748,7 @@ rte_eal_init(int argc, char **argv)
thread_id = pthread_self();
- if (rte_eal_log_early_init() < 0)
- rte_panic("Cannot init early logs\n");
+ rte_eal_log_set_default(stdout);
eal_log_level_parse(argc, argv);
@@ -97,45 +97,7 @@ rte_eal_log_init(const char *id, int facility)
openlog(id, LOG_NDELAY | LOG_PID, facility);
- if (rte_eal_common_log_init(log_stream) < 0)
- return -1;
-
- return 0;
-}
-
-/* early logs */
-
-/*
- * early log function, used before rte_eal_log_init
- */
-static ssize_t
-early_log_write(__attribute__((unused)) void *c, const char *buf, size_t size)
-{
- ssize_t ret;
- ret = fwrite(buf, size, 1, stdout);
- fflush(stdout);
- if (ret == 0)
- return -1;
- return ret;
-}
-
-static cookie_io_functions_t early_log_func = {
- .write = early_log_write,
-};
-static FILE *early_log_stream;
+ rte_eal_log_set_default(log_stream);
-/*
- * init the log library, called by rte_eal_init() to enable early
- * logs
- */
-int
-rte_eal_log_early_init(void)
-{
- early_log_stream = fopencookie(NULL, "w+", early_log_func);
- if (early_log_stream == NULL) {
- printf("Cannot configure early_log_stream\n");
- return -1;
- }
- rte_openlog_stream(early_log_stream);
return 0;
}