On Tuesday, 15 October 2013 at 15:16:44 UTC, Andrei Alexandrescu wrote:
> Eric, could you please enumerate a short list of features of log4j that you think would be really missed if absent?

Certainly.  Here's my top 3, with some background to explain why I think they'd be missed.

- Hierarchical logging with a centralized (singleton) logging facility.  The strength of this is that it would allow us to freely integrate D libraries that use std.logger, yet filter their log output from *outside* the library through the std.logger API. This can be accomplished by tagging each log event with a category name.  Log events are then filtered by prefix matching of the category name, as well as by log level.  Without this feature, library authors would have to provide explicit API calls to manipulate their library's logging, or require API users to pass logging contexts forward to the library.

- Configurable output strategies for each log category.  In log4cpp, these are known as "appenders."  Appenders need not be registered explicitly for each category, but can be registered by category name prefix match, just like the filtering for the hierarchical system (above). The idea is to allow for different formatting strategies and output targets, including the logrotate issue I mentioned earlier.  This provides a nice integration point to tackle basic capabilities today, like file logging and syslog support, and more advanced features by 3rd party authors.

- Nested Diagnostic Context support ("Mapped Diagnostic Context" in log4j).  The NDC facility in log4cpp/log4cxx is incredibly handy for cutting down on the amount of data one usually puts into a given log event.  The gist of an NDC is just a way to bracket a series of log calls with a prefix that is emitted with the rest of the log line.  These contexts are maintained on a thread-specific stack, such that each log event is prefixed with all the information in the entire stack, at the time of the event. Without this, one winds up re-inventing the concept (usually poorly) to forward the same information to each call to emit a log message.  It also eliminates the need for a stack trace in a log message in most cases, which is something that people who use SIEM software (e.g. Splunk) would appreciate.

There are other things that would be nice - configuration file support, lazy evaluation of log event arguments, custom output formats - but I think the above is really core of what's needed.

For what it's worth: my opinions mostly come from my experience in integrating with log4j and log4cpp.  Log4j is a very heavyweight library - I don't think we need anything anywhere close to that nuanced.  In contrast, log4cpp is very small library and worth a look as something that may be a good model for what std.logger could accomplish.

- Eric

On Thursday, 17 October 2013 at 02:13:12 UTC, Eric Anderton wrote:
> The strength of this is that it would allow us to freely integrate D libraries that use std.logger, yet filter their log output from *outside* the library through the std.logger API.

This is one of the most important aspects in my opinion.
Std.logger should be easy to use, so library writers are
encouraged to use it. Compare this with the "unittest" keyword,
which makes it easy to write some simple tests. Of course,
flexibility to use complex machinery for using the messages/tests
is necessary. Just like we can hook up more advanced unit testing
frameworks, we should be able to hook up more advanced logging
machinery. The advanced stuff is not for Phobos though. Advanced
stuff for unittests is for example, parallel execution and
graphical reports. Advanced stuff for logging is for example log
rotation and networking.

On Thursday, 17 October 2013 at 02:13:12 UTC, Eric Anderton wrote:
> On Tuesday, 15 October 2013 at 15:16:44 UTC, Andrei Alexandrescu wrote:
>> Eric, could you please enumerate a short list of features of log4j that you think would be really missed if absent?
>
> Certainly.  Here's my top 3, with some background to explain why I think they'd be missed.
>
> - Hierarchical logging with a centralized (singleton) logging facility.  The strength of this is that it would allow us to freely integrate D libraries that use std.logger, yet filter their log output from *outside* the library through the std.logger API. This can be accomplished by tagging each log event with a category name.  Log events are then filtered by prefix matching of the category name, as well as by log level.  Without this feature, library authors would have to provide explicit API calls to manipulate their library's logging, or require API users to pass logging contexts forward to the library.
>
> - Configurable output strategies for each log category.  In log4cpp, these are known as "appenders."  Appenders need not be registered explicitly for each category, but can be registered by category name prefix match, just like the filtering for the hierarchical system (above). The idea is to allow for different formatting strategies and output targets, including the logrotate issue I mentioned earlier.  This provides a nice integration point to tackle basic capabilities today, like file logging and syslog support, and more advanced features by 3rd party authors.
>
> - Nested Diagnostic Context support ("Mapped Diagnostic Context" in log4j).  The NDC facility in log4cpp/log4cxx is incredibly handy for cutting down on the amount of data one usually puts into a given log event.  The gist of an NDC is just a way to bracket a series of log calls with a prefix that is emitted with the rest of the log line.  These contexts are maintained on a thread-specific stack, such that each log event is prefixed with all the information in the entire stack, at the time of the event. Without this, one winds up re-inventing the concept (usually poorly) to forward the same information to each call to emit a log message.  It also eliminates the need for a stack trace in a log message in most cases, which is something that people who use SIEM software (e.g. Splunk) would appreciate.
>
> There are other things that would be nice - configuration file support, lazy evaluation of log event arguments, custom output formats - but I think the above is really core of what's needed.
>
> For what it's worth: my opinions mostly come from my experience in integrating with log4j and log4cpp.  Log4j is a very heavyweight library - I don't think we need anything anywhere close to that nuanced.  In contrast, log4cpp is very small library and worth a look as something that may be a good model for what std.logger could accomplish.
>
> - Eric

+1

I have used log4net some years ago, really liked the the 'context logging' feature. (Nested Diagnostic Context support).

I've also used www.panteios.org logging api, liked that approach too.

On 10/17/2013 09:34 AM, qznc wrote:
> On Thursday, 17 October 2013 at 02:13:12 UTC, Eric Anderton wrote:
>> The strength of this is that it would allow us to freely integrate D libraries that use std.logger, yet filter their log output from *outside* the library through the std.logger API.
>
> This is one of the most important aspects in my opinion. Std.logger should be easy to use, so library writers are encouraged to use it. Compare this with the "unittest" keyword, which makes it easy to write some simple tests. Of course, flexibility to use complex machinery for using the messages/tests is necessary. Just like we can hook up more advanced unit testing frameworks, we should be able to hook up more advanced logging machinery. The advanced stuff is not for Phobos though. Advanced stuff for unittests is for example, parallel execution and graphical reports. Advanced stuff for logging is for example log rotation and networking.
+1

On 10/17/2013 04:13 AM, Eric Anderton wrote:
> On Tuesday, 15 October 2013 at 15:16:44 UTC, Andrei Alexandrescu wrote:
>> Eric, could you please enumerate a short list of features of log4j that you think would be really missed if absent?
>
> Certainly.  Here's my top 3, with some background to explain why I think they'd be missed.
>
> - Hierarchical logging with a centralized (singleton) logging facility.  The strength of this is that it would allow us to freely integrate D libraries that use std.logger, yet filter their log output from *outside* the library through the std.logger API. This can be accomplished by tagging each log event with a category name.  Log events are then filtered by prefix matching of the category name, as well as by log level.  Without this feature, library authors would have to provide explicit API calls to manipulate their library's logging, or require API users to pass logging contexts forward to the library.
With the new MultiLogger and names for Logger, I consider this done.
>
> - Configurable output strategies for each log category.  In log4cpp, these are known as "appenders."  Appenders need not be registered explicitly for each category, but can be registered by category name prefix match, just like the filtering for the hierarchical system (above). The idea is to allow for different formatting strategies and output targets, including the logrotate issue I mentioned earlier. This provides a nice integration point to tackle basic capabilities today, like file logging and syslog support, and more advanced features by 3rd party authors.
>
> - Nested Diagnostic Context support ("Mapped Diagnostic Context" in log4j).  The NDC facility in log4cpp/log4cxx is incredibly handy for cutting down on the amount of data one usually puts into a given log event.  The gist of an NDC is just a way to bracket a series of log calls with a prefix that is emitted with the rest of the log line. These contexts are maintained on a thread-specific stack, such that each log event is prefixed with all the information in the entire stack, at the time of the event. Without this, one winds up re-inventing the concept (usually poorly) to forward the same information to each call to emit a log message.  It also eliminates the need for a stack trace in a log message in most cases, which is something that people who use SIEM software (e.g. Splunk) would appreciate.
>
> There are other things that would be nice - configuration file support, lazy evaluation of log event arguments, custom output formats - but I think the above is really core of what's needed.
IMO, this is to heavyweight and would water the current design to much. That being said, I think it is easy to subclass Logger or MultiLogger to achieve this. But I don't think that this should be part of the default feature set.

Can you please re-generate the documentation after all recent updates?

On 10/18/2013 02:55 PM, Dicebot wrote:
> Can you please re-generate the documentation after all recent updates?
I usually do that. The only documentation missing is for MultiLogger, as I'm not sure if the current implementation is done.

On Friday, 18 October 2013 at 13:35:19 UTC, Robert Schadek wrote:
> On 10/18/2013 02:55 PM, Dicebot wrote:
>> Can you please re-generate the documentation after all recent updates?
> I usually do that. The only documentation missing is for MultiLogger, as
> I'm not sure if the current implementation is done.

Hm. I don't see anything related to Logger names there. It is supposed to be part of MultiLogger?

As discussion was slowly fading I wanted to make some sort of summary for this thread judging by personal observations - having quick overview of current state in form of docs will be helpful.

On 10/18/2013 03:49 PM, Dicebot wrote:
> On Friday, 18 October 2013 at 13:35:19 UTC, Robert Schadek wrote:
>> On 10/18/2013 02:55 PM, Dicebot wrote:
>>> Can you please re-generate the documentation after all recent updates?
>> I usually do that. The only documentation missing is for MultiLogger, as I'm not sure if the current implementation is done.
>
> Hm. I don't see anything related to Logger names there. It is supposed to be part of MultiLogger?
>
> As discussion was slowly fading I wanted to make some sort of summary for this thread judging by personal observations - having quick overview of current state in form of docs will be helpful.
I rebuild the docs and pushed them. MultiLogger docu is just a stub.

On Tuesday, 15 October 2013 at 15:21:52 UTC, Johannes Pfau wrote:
> I think one increasingly important point for std.log is 'structured
> logging'.
>
> Structured logging is basically not simply logging textual messages, but
> also logging additional KEY/VALUE pairs of data. The idea is that logs
> should not only be readable by humans but also easy to parse and
> analyze. Structured logging often also includes UUIDs to simplify
> finding similar errors/problems in a log file.
>
> For example: Instead of logging
> logf("Couldn't login as %s on server %s : Reason: %s", user, server,
>       errorCode);
>
> we'd do:
> log(LOGIN_FAILED_UUID, "Couldn't log in", ["User": user, "Server":
>     server, "ErrorCode": errorCode]);
>
> The log can then be queried for all events with 'LOGIN_FAILED_UUID'
> uuid, Server="..." ErrorCode=42, etc.

+1
I like the idea of structured logging.
I really need to log some complex data, not only string. I usually use `to!string(value)` to convert complex data to string.

About syntax - we can use something like this:
log!(user, server, errorCode)(LOGIN_FAILED_UUID, "Couldn't log in");

user, server and errorCode is variables passed to template. We can get variable value as usually and variable name via `__traits(identifier, variable)`, it should work after DMD 2.064.

For example, Vide.d use this idea, see `render` function:
http://vibed.org/docs

We can store complex data as string via `to!string(value)` cast until we can use real backend implementation like SystemDs.