>
> Is it too late for an integration in std.expirimental with dmd 2.66?
>>
> ask Andrew, but I'm 99.99999% sure it is to late
>

Thanks to dub, the need for std.experimental is significantly reduced.  You can just pull in the package and use it right now:

http://code.dlang.org/packages/logger

On Friday, 11 July 2014 at 21:34:37 UTC, Robert burner Schadek wrote:
>> Is it too late for an integration in std.expirimental with dmd 2.66?
> ask Andrew, but I'm 99.99999% sure it is to late

Despite how reasonable it may seem I believe that being stupid formal about release process is the Right Thing -> it is too late.

On Friday, 11 July 2014 at 14:39:09 UTC, David Nadlinger wrote:
> On Friday, 11 July 2014 at 14:36:34 UTC, Dicebot wrote:
>> Round of a formal review before proceeding to voting. Subject for Phobos inclusion : http://wiki.dlang.org/Review/std.logger authored by Robert Schadek.
>
> Is this for std.* or std.experimental.*?
>
> David

I'm of the opinion we should review for std.* and vote for std.experimental.*

Prior to or during beta we vote to move into std.*

Am Fri, 11 Jul 2014 21:14:36 +0000
schrieb "Robert burner Schadek" <rburners@gmail.com>:

> On Friday, 11 July 2014 at 15:20:51 UTC, Johannes Pfau wrote:
> > An logger can still receive the complete string by appending
> > the chunks
> > received in put but we might need some 'finish' function to
> > signal the
> > end of the message. I guess not all loggers can fit into this
> > interface, so we should try to make this optional. But simple
> > loggers
> > (file, console) which write the header first and the message
> > last could
> > work without any dynamic memory allocation. (formattedWrite
> > probably
> > uses an fixed-size buffer on the stack, but that's fine)
> 
> The api for none printf like logging has changed into something
> like
> write. So put properly needs to become a template. Any I'm not
> sure if
> there is a nice way around the template/inheritance problematic.
> Other
> than options one and two.

Last time we discussed this I also thought we'd need to make put a template. But we overlooked the obvious solution:

Type->string formatting stays part of the 'frontend' functions. We only pass strings to the backend. But instead of insisting that message is one string, we allow to pass the msg as many strings. This means that formatting can go in a fixed size stack buffer and still support arbitrary length strings.

Here's a pull request to implement this idea: https://github.com/burner/logger/pull/9

Am Fri, 11 Jul 2014 21:26:25 +0000
schrieb "Robert burner Schadek" <rburners@gmail.com>:

> On Friday, 11 July 2014 at 18:02:58 UTC, Marc Schütz wrote:
> > Some logging backends (e.g. systemd journal) support structured logging. Should support for this be included (as a subclass, presumably)?
> 
> Well, the idea behind std.logger is that I can not and should not even try to give you implementations for all your backend needs. It will just fall short and you will tell me that is it bad and should not go into phobos. And even if I manage it is going to be curl all over again.
> 
> So subclass Logger, implement writeLogMsg to your needs and be happy.

Yes, but for structured loggers the frontend log* functions do not provide enough information to the backend. Of course you can use the systemd journal as a normal backend, but you lose features.


I think we can provide structured logging support as a non-breaking API extension, so we should not make this part of this review. But here's how I'd imagine such an API to work:

Frontend:

* log* get new overloads which accept (T...) as the last parameter (or
  if T... is already the last parameter that's fine).

* Add a new struct to logger.core: struct MsgID which is just a
  strong typedef for UUID
* Add a templated type, KeyValue, which can be used like this:
  KeyValue("user", "nobody") //string key / string value
  KeyValue("age", 42); //string key / T value
  KeyValue("msg", "Hello %s! %s", "World", 42); //string key/fmt val
* KeyValue stores it's parameters, no string processing yet
* Multivalue parameters handled by many KeyValue with same key? Might
  complicate backend. Or don't support multivalue at all? Or
  KeyValue("key", MultiValue(a, ,b, c)) (MultiValue == Tuple?)
* Structured loggers do not use msg, instead they use a KeyValue with
  "msg" key. This is cause you usually want different messages with
  structured loggers. We still keep everything in one function, so the
  user doesn't have to do "if(structuredlogger) logstruct() else log()"
  for every log message.
* MsgID marks the end of normal format parameters. See example below.
  This is also the reason why we can't use UUID directly

Usage:
string error;
logf("Something bad happened: %s", error,
     MsgID("abcd-valid-uuid"),       //MsgID--> end of fmt params
     KeyValue("msg", "Something bad happend"),
     KeyValue("error-code", error));

output:
normal backend: test.d:42 Something bad happened: out of memory
structured backend: (only example, exact format backend specific)
{
  "msg": "Something bad happened",
  "error": "out of memory",
  "file": "test.d",
  "line": 42
}




The next part is an efficient Backend Layer:

class StructuredLogger : Logger
{
    logHeader;
    writeLogMsg; //Not used
    finishLogMsg;

    void logKey(string key);
    void valuePart(const(char)[] part);
    void finishValue(bool last); //Last only if we support multivalue
}

Usage:

auto slog = new StructuredLogger();
slog.logHeader(...);
foreach(KeyValue kv; T...)
{
    slog.logKey(kv.key);
    //Need slog -> outputrange adapter: map put<>valuePart
    //see https://github.com/burner/logger/pull/9
    formattedWrite(wrap(slog), kv.formatstring, kv.args);
    slog.finishValue(true);
}
finishLogMsg();

On Saturday, 12 July 2014 at 03:22:10 UTC, Jesse Phillips wrote:
> On Friday, 11 July 2014 at 14:39:09 UTC, David Nadlinger wrote:
>> On Friday, 11 July 2014 at 14:36:34 UTC, Dicebot wrote:
>>> Round of a formal review before proceeding to voting. Subject for Phobos inclusion : http://wiki.dlang.org/Review/std.logger authored by Robert Schadek.
>>
>> Is this for std.* or std.experimental.*?
>>
>> David
>
> I'm of the opinion we should review for std.* and vote for std.experimental.*
>
> Prior to or during beta we vote to move into std.*

Agreed.

Am Fri, 11 Jul 2014 14:36:30 +0000
schrieb "Dicebot" <public@dicebot.lv>:

> Round of a formal review before proceeding to voting. Subject for Phobos inclusion : http://wiki.dlang.org/Review/std.logger authored by Robert Schadek.
> 
> Code:
>      https://github.com/D-Programming-Language/phobos/pull/1500
> Documentation:

Overall design & API looks good.

Some detail comments and / or questions:

logger.core:
* For some people the EBNF might be useful, but it looks kinda scary to
  me if I open the documentation ;-)
* The docs do not mention thread safety at all. Can I change the
  default logger while other threads are logging?
* The docs should clearly mention the default logger (this is kinda
  mentioned, but I'd make it more explicit) and the default loglevel.
* Can the logger be influenced by command line flags / environment
  variables? I guess not, but if it can it should be documented.
* I think an example how to change the default logger should be at the
  top of the module. (I think it's quite common people would want to
  log to files as well)
* Why does LogManager.defaultLogger return by ref instead of just using
  a setter?
* The logger.core documentation should have an overview of available
  logger types and link to them. Probably a table with
  Logger      |     module             | Description
  ----------------------------------------------------
  StdIOLogger | std.logger.stdiologger | log to standard output(console)


logger.multilogger:
* Should mention thread safety. If other threads log to the multilogger
  can I simply add/remove loggers?

On Saturday, 12 July 2014 at 09:19:47 UTC, Johannes Pfau wrote:

> I think we can provide structured logging support as a non-breaking API
> extension, so we should not make this part of this review. But here's
> how I'd imagine such an API to work:


I look at the journald spec and LoggerPayload already contains most of the key value pairs. MSG_ID seams to be the last important one missing inside LoggerPayload.

Thank you for taking the time to create a PR put, IMO that approach is to complicated. Anyway, if there are performance problems down the road this will be the first approach to try.

On Saturday, 12 July 2014 at 12:19:03 UTC, Johannes Pfau wrote:
> Am Fri, 11 Jul 2014 14:36:30 +0000
> schrieb "Dicebot" <public@dicebot.lv>:
>
>> Round of a formal review before proceeding to voting. Subject for Phobos inclusion : http://wiki.dlang.org/Review/std.logger authored by Robert Schadek.
>> 
>> Code:
>>      https://github.com/D-Programming-Language/phobos/pull/1500
>> Documentation:
>
> Overall design & API looks good.
>
> Some detail comments and / or questions:
>
> logger.core:
> * For some people the EBNF might be useful, but it looks kinda scary to
>   me if I open the documentation ;-)
I know, I will make it less scary.

> * The docs do not mention thread safety at all. Can I change the
>   default logger while other threads are logging?
That is already on my todo list.
> * The docs should clearly mention the default logger (this is kinda
>   mentioned, but I'd make it more explicit) and the default loglevel.
ok
> * Can the logger be influenced by command line flags / environment
>   variables? I guess not, but if it can it should be documented.
I thought not writing about it makes it clear that you can not.

> * I think an example how to change the default logger should be at the
>   top of the module. (I think it's quite common people would want to
>   log to files as well)
Will do
> * Why does LogManager.defaultLogger return by ref instead of just using
>   a setter?
Because I use the defaultLogger at quite a lot of places and treating it as an property just felt right.
> * The logger.core documentation should have an overview of available
>   logger types and link to them. Probably a table with
>   Logger      |     module             | Description
>   ----------------------------------------------------
>   StdIOLogger | std.logger.stdiologger | log to standard output(console)
>
>
Will do
> logger.multilogger:
> * Should mention thread safety. If other threads log to the multilogger
I know, threads
>   can I simply add/remove loggers?
yes: new WhatEverLoggerYouWant("some_good_name", LogLevel.xxxx);

Am Sat, 12 Jul 2014 13:09:37 +0000
schrieb "Robert burner Schadek" <rburners@gmail.com>:

> On Saturday, 12 July 2014 at 09:19:47 UTC, Johannes Pfau wrote:
> 
> > I think we can provide structured logging support as a
> > non-breaking API
> > extension, so we should not make this part of this review. But
> > here's
> > how I'd imagine such an API to work:
> 
> 
> I look at the journald spec and LoggerPayload already contains most of the key value pairs. MSG_ID seams to be the last important one missing inside LoggerPayload.
> 
> Thank you for taking the time to create a PR put, IMO that approach is to complicated. Anyway, if there are performance problems down the road this will be the first approach to try.

The whole point of journald is that the user can define arbitrary key/value pairs. you're not limited to the special key/value pairs. http://0pointer.de/blog/projects/journal-submit.html

Also giving log messages uuids is an important property of structured logging.