On Tuesday, 28 October 2014 at 12:02:16 UTC, Robert burner Schadek wrote:
> It is a design goal to disable certain LogLevel at CT of a compile unit (CU).
> e.g. make all logs to trace function template do nothing

One idea to make this working is to use prefixed version identifiers.
Obviously this is all boilerplate so it could be done in a mixin template in std.log.

module mylib.log;

version (MyLibLogInfo)
    enum logLevel = LogLevel.info;
version (MyLibLogTrace)
    enum logLevel = LogLevel.trace;
version (MyLibLogWarning)
    enum logLevel = LogLevel.warning;
version (MyLibLogError)
    enum logLevel = LogLevel.error;

static if (is(typeof(logLevel)))
{
    // wrapper around Logger interface/class for CT logLevel
    static struct FixedLogLevel
    {
        enum logLevel = logLevel;
        Logger impl;
        alias impl this;
    }

    void setLogger(Logger logger)
    {
        logger.impl = logger;
    }

    private __gshared FixedLogLevel logger;
}

module mylib.foo;

void foo()
{
    import mylib.log;
    logger.info("bla");
    logger.error("wat");
}

module client.bar;

void bar()
{
    import mylib.log, mylib.foo;
    auto logger = new FileLogger("log.txt", LogLevel.warning);
    setLogger(logger);
    foo(); // CT check for LogLevel
    logger.info("runtime check whether info logging is on");
}

On 10/28/2014 07:22 PM, Martin Nowak wrote:
> On Tuesday, 28 October 2014 at 12:02:16 UTC, Robert burner Schadek wrote:
>> It is a design goal to disable certain LogLevel at CT of a compile
>> unit (CU).
>> e.g. make all logs to trace function template do nothing
>
> One idea to make this working is to use prefixed version identifiers.
> Obviously this is all boilerplate so it could be done in a mixin
> template in std.log.

2nd iteration of that idea.

cat > main.d << CODE
import std_logger;

LoggerCT!"MyApp" appLogger = new StdoutLogger(LogLevel.info);
LoggerCT!"MyLib" libLogger = new StdoutLogger(LogLevel.trace);

void main()
{
    appLogger.log!(LogLevel.info)("app");
    appLogger.log!(LogLevel.warning)("app");
    libLogger.log!(LogLevel.info)("lib");
    libLogger.log!(LogLevel.warning)("lib");

    Logger rtLogger = appLogger;
    rtLogger.log!(LogLevel.info)("rt");
    rtLogger.log!(LogLevel.warning)("rt");
}
CODE

cat > std_logger.d << CODE
enum LogLevel { all, info, trace, warning, error, none }

template minLogLevel(string prefix)
{
    mixin("
    version ("~prefix~"LogAll)
        enum minLogLevel = LogLevel.all;
    else version ("~prefix~"LogInfo)
        enum minLogLevel = LogLevel.info;
    else version ("~prefix~"LogTrace)
        enum minLogLevel = LogLevel.trace;
    else version ("~prefix~"LogWarning)
        enum minLogLevel = LogLevel.warning;
    else version ("~prefix~"LogError)
        enum minLogLevel = LogLevel.error;
    else version ("~prefix~"LogNone)
        enum minLogLevel = LogLevel.none;
    else
        enum minLogLevel = LogLevel.all;
    ");
}

interface Logger
{
    @property LogLevel logLevel();
    void write(LogLevel ll, string msg);
}

class StdoutLogger : Logger
{
    this(LogLevel l) { _logLevel = l; }
    LogLevel _logLevel;
    @property LogLevel logLevel() { return _logLevel; }
    void write(LogLevel ll, string msg) { import std.stdio; writeln(ll, ": ", msg); }
}

/// used for library/app specific version prefixes
struct LoggerCT(string prefix)
{
    this(Logger logger) { impl = logger; }
    enum versionPrefix = prefix;
    Logger impl;
    alias impl this;
}

void log(LogLevel ll)(Logger logger, string msg)
{
    if (ll >= logger.logLevel) // runtime check
        logger.write(ll, msg);
}

/// when using a logger with prefix
void log(LogLevel ll, L:LoggerCT!pfx, string pfx)(auto ref L logger, string msg)
    if (ll < minLogLevel!pfx)
{
}

void log(LogLevel ll, L:LoggerCT!pfx, string pfx)(auto ref L logger, string msg)
    if (ll >= minLogLevel!pfx)
{
    if (ll >= logger.logLevel) // additional runtime check, because logger might require a higher log level
        logger.write(ll, msg);
}
CODE

dmd std_logger -run main

dmd -version=MyAppLogWarning std_logger.d -run main
dmd -version=MyAppLogError -version=MyLibLogNone std_logger.d -run main

On 10/28/14 2:46 AM, Robert burner Schadek wrote:
> On Tuesday, 28 October 2014 at 05:44:48 UTC, Andrei Alexandrescu wrote:
>> Being able to select maximum logging level statically at client
>> application level is a deal maker/breaker for me. The mechanics aren't
>> important but it's likely they will affect the API. So I think that
>> needs to be resolved now, not in a future pull request.
>>
>> Andrei
>
> please elaborate. It is a must to be able to disable LogLevel when
> calling the compiler? Xor the opposite?
>
> passing -version=StdLoggerDisableTrace when compiling the user code will
> yield empty functions (thank you static if) for every call to any
> function or method having the word "trace" in its name, like for example
> trace("My log call");
>
> If you call something like log(computeMyLogLevelAtRuntime(), "my log data)
> and that function returns LogLevel.trace and you haved passed
> -version=StdLoggerDisableTrace at CT of the user code you will have to
> pay for one if.

isFloatingPoint is an exact check, which is fine. We'd need isLikeXxx templates that support Liskov. -- Andrei

On Tuesday, 28 October 2014 at 11:11:09 UTC, Martin Nowak wrote:
> I think part of the misunderstanding is that I'm thinking of an app as user code plus a number of libraries all on top of phobos. Say I have an app using vibe.d and I want to enable logging in my app, but disable it in phobos.
> Was it even a design goal that logging can be enabled on a per library level?

Whether it was an original design goal or not, I don't think it can be done with today's std.logger, and I would argue that it should not be a function of std.logger at all.

Selective logging of a mix of libraries requires a namespace to identify each library, along with a method for libraries to obtain their Loggers based on that namespace (e.g. giving Logger a name, which it does not have now, and using a factory method like Logger.getLogger()).  We could assume that module names will always be available to do that job (and by default I do provide that as a reasonable default in log4d), but I can easily imagine other namespace choices for unknown future users.  I also think that most end users would prefer their selective enabling/disabling of libraries' logging to happen at runtime, not compile time, and the HOW of turning things on/off will also be very user-dependent (regex, config file, environment var, ...).

We are bleeding between "mechanism" and "policy" here.  I feel that std.logger is mechanism: given that a client has a Logger "thing" (struct, class, whatever), which methods will the client call on it to emit output to somewhere?  Frameworks like log4d are policy: how are the Loggers that are obtained by lots of clients organized into a coherent whole, and the outputs filtered/directed/controlled by the final end-user?  Policy should always be deferred to third party libraries.  I obviously like the log4d/log4j approach, but other people will like vibe.d logger, some will roll-your-own, some will just send everything to syslog, etc.

If we really want to insist that std.logger provides selective control of libraries, then I think we will also need to establish policy at this time.  I see two reasonable choices.

Option 1:  minimal change

1. The policy is that we control library logging by letting the client replace Logger references at runtime.
2. We say that libraries either use stdlog, i.e. info(...), error(...), etc., OR that they expose a Logger reference that clients can replace.
3. Clients replace stdlog or the libraries' Loggers with Loggers from their chosen framework.

Option 2:  invasive change

1. We make policy that there is a namespace of all loggers and it is __MODULE__.
2. stdlog is removed.
3. Loggers are obtained only via a getLogger() type call.  Any function that wants to log must call getLogger() first.
4. log(...), info(...), etc. are removed.  All logging goes through a Logger: Logger.info/error/etc(...)
5. Clients replace the getLogger() factory method with a function/delegate provided by their framework.

I think that option 1 would scale better long-term.  Phobos can choose to use stdlog for its own logging, and other libraries can make their own (vibelog, foolog, ...).  std.logger would need no changes.



Final thought: filtering.  We have several filtering methods now, and I am confused as to what is actually missing:

* ANY AND ALL logging can be disabled at compile time via versions.  This only includes client code and libraries the client is compiling too; I would assume that the default shipped phobos would not version-remove log calls.

* ALL logging can be disabled at runtime via globalLogLevel and a single if check at the call site.

* Per-Logger logging can be disabled at runtime via Logger.logLevel and a single if check at the call site.

* All other kinds of filtering (on msg, timestamp, method, etc.) are available at runtime to Logger subclasses, but happens after the logImpl call and setup of LogEntry.

What other capabilities are needed for filtering?

Am Tue, 28 Oct 2014 16:32:34 +0000
schrieb "Dicebot" <public@dicebot.lv>:

> On Tuesday, 28 October 2014 at 16:05:19 UTC, Andrei Alexandrescu wrote:
> > Agreed. Just to restate my position: so long as we don't have a way to statically control maximum logging level in the client, we don't have a logging library. There is no negotiation. -- Andrei
> 
> We have way to statically control logging level of the client from the client. Argument is mostly about precompiled 3d party libraries.

Could somebody summarize that discussion?
You can't 'statically' disable logging in a _precompiled_ library. So
is this about statically disabling logging in libraries when source
code is available or disabling logging at runtime for precompiled
libraries?

On Saturday, 25 October 2014 at 16:43:10 UTC, Dicebot wrote:
> However, I don't think any of those are truly critical and this proposal has been hanging there for just too long. In my opinion it will be more efficient to resolve any remainining issues in follow-up pull requests on case by case basis then forcing Robert to do it himself - there will be some time left before next release to break a thing or two before public statement is made.

(Am I a voter?  No idea.  But here is my feedback, based on my recent experience implementing a fuller backend for std.logger.)

I agree, I think std.logger is ready to go forward and needs to get into end-user hands.

I see four areas that are incomplete but SHOULD be deferred to future bug reports / enhancement requests:

1. Additional LogEntry fields: Thread, Fiber, Throwable, maybe others.  These should only be adopted if the end users can provide a compelling case for why their Logger subclasses (or backend framework) can't give them what they want.

2. Additional Logger functions: caught(), throwing(), maybe others.  These should be driven by the needs of phobos' use of std.logger.  (And phobos SHOULD use std.logger and eat its own dog food.)

3. Modification of the order or method in which the Logger functions obtain their data in constructing the LogEntry, i.e. call Clock.currTime() differently or not at all.  These should be controlled by future additional fields/flags in Logger, with no changes to the functions one calls on a Logger.

4. Control of logging across multiple libraries/applications.  This should absolutely be deferred until phobos itself starts using std.logger.

Of these issues, I think that #4 is a huge can of worms that is impossible to make everyone happy, and we keep poking into it as a result of not accepting that the only people who can decide on it are the application writers, not the language and phobos designers.  The two solutions for it at the extremes are:

A) Deciding if a Logger is enabled when it is obtained by the client (via a Logger.getLogger() factory function), which requires imposing a namespace of some kind that ALL future libraries will have to adhere to.  Note that log4j -- currently the fastest and "biggest" logging system out there -- started with a fixed className namespace, but moved away from that later on to more general user-defined "categories".  Even in a straightjacketed language like Java one namespace isn't good enough.

B) Deciding if a Logger should emit (based on end-user decision via config file, environment variable, ...) at the Logger.writeLogEntry() call via filtering on the LogEntry fields.

Option A is cumbersome in client code but very fast in practice because getLogger() can set the Logger's logLevel to filter messages before the logImpl() call.  Option B is easier in client code (and also lets many functions/modules/libraries write to a single Logger ala stdlog) but very slow at runtime because the namespace filtering can't occur until after the LogEntry is fully formed (formatted (including allocation) + Clock.currTime + ...).

The application writers are the only people in a position to choose which end of this tradeoff makes the most sense for them.  Phobos will have to choose for itself someday, and I think that various subsystems may very well choose differently based on their most common usage.  std.net.curl might choose option B (just logging to stdlog) because it's got this nice clearly-defined border between the client and libcurl; whereas std.range might go for option A (letting the users set a getLogger() delegate) because it is very cross-cutting and users will only want to see log entries when doing range stuff in their code and not all over phobos or other libraries.

On Tuesday, 28 October 2014 at 22:03:18 UTC, Martin Nowak wrote:
> On 10/28/2014 07:22 PM, Martin Nowak wrote:
>> On Tuesday, 28 October 2014 at 12:02:16 UTC, Robert burner Schadek wrote:
>>> It is a design goal to disable certain LogLevel at CT of a compile
>>> unit (CU).
>>> e.g. make all logs to trace function template do nothing
>>
>> One idea to make this working is to use prefixed version identifiers.
>> Obviously this is all boilerplate so it could be done in a mixin
>> template in std.log.
>
> 2nd iteration of that idea.
>
> cat > main.d << CODE
> import std_logger;
>
> LoggerCT!"MyApp" appLogger = new StdoutLogger(LogLevel.info);
> LoggerCT!"MyLib" libLogger = new StdoutLogger(LogLevel.trace);
>
> void main()
> {
>     appLogger.log!(LogLevel.info)("app");
>     appLogger.log!(LogLevel.warning)("app");
>     libLogger.log!(LogLevel.info)("lib");
>     libLogger.log!(LogLevel.warning)("lib");
>
>     Logger rtLogger = appLogger;
>     rtLogger.log!(LogLevel.info)("rt");
>     rtLogger.log!(LogLevel.warning)("rt");
> }
> CODE
>
> cat > std_logger.d << CODE
> enum LogLevel { all, info, trace, warning, error, none }
>
> template minLogLevel(string prefix)
> {
>     mixin("
>     version ("~prefix~"LogAll)
>         enum minLogLevel = LogLevel.all;
>     else version ("~prefix~"LogInfo)
>         enum minLogLevel = LogLevel.info;
>     else version ("~prefix~"LogTrace)
>         enum minLogLevel = LogLevel.trace;
>     else version ("~prefix~"LogWarning)
>         enum minLogLevel = LogLevel.warning;
>     else version ("~prefix~"LogError)
>         enum minLogLevel = LogLevel.error;
>     else version ("~prefix~"LogNone)
>         enum minLogLevel = LogLevel.none;
>     else
>         enum minLogLevel = LogLevel.all;
>     ");
> }
>
> interface Logger
> {
>     @property LogLevel logLevel();
>     void write(LogLevel ll, string msg);
> }
>
> class StdoutLogger : Logger
> {
>     this(LogLevel l) { _logLevel = l; }
>     LogLevel _logLevel;
>     @property LogLevel logLevel() { return _logLevel; }
>     void write(LogLevel ll, string msg) { import std.stdio; writeln(ll, ": ", msg); }
> }
>
> /// used for library/app specific version prefixes
> struct LoggerCT(string prefix)
> {
>     this(Logger logger) { impl = logger; }
>     enum versionPrefix = prefix;
>     Logger impl;
>     alias impl this;
> }
>
> void log(LogLevel ll)(Logger logger, string msg)
> {
>     if (ll >= logger.logLevel) // runtime check
>         logger.write(ll, msg);
> }
>
> /// when using a logger with prefix
> void log(LogLevel ll, L:LoggerCT!pfx, string pfx)(auto ref L logger, string msg)
>     if (ll < minLogLevel!pfx)
> {
> }
>
> void log(LogLevel ll, L:LoggerCT!pfx, string pfx)(auto ref L logger, string msg)
>     if (ll >= minLogLevel!pfx)
> {
>     if (ll >= logger.logLevel) // additional runtime check, because logger might require a higher log level
>         logger.write(ll, msg);
> }
> CODE
>
> dmd std_logger -run main
>
> dmd -version=MyAppLogWarning std_logger.d -run main
> dmd -version=MyAppLogError -version=MyLibLogNone std_logger.d -run main

That can actually be added to the current state of std.logger without breaking any api. The string mixin, version string matching isn't really pretty, but it gets the job done. Anyway IMO your approach presented here and my approach can go hand in hang. Yours should be propagated as the idiomatic way and if you really need the crowbar, which you need sometimes, use StdLoggerDisableXXXXX.

The reason for the crowbar sometimes you need to disable all calls to the Logger or any calls to a specific LogLevel in the compile unit, even for Logger not wrapped in LoggerCT.

On Tuesday, 28 October 2014 at 22:03:18 UTC, Martin Nowak wrote:
> 2nd iteration of that idea.

For me it looks like the cure is worse than the disease. Simply not distributing precompiled libraries with log level force-reduced (or at least not exposing it in provided .di files) feels like a more practical approach than proposed API complication.

Bringing this back to the first page