This DIP provides a way to handle unittest blocks inside of templates which works with ddoc without compiling the unittest blocks into each instantiation.

http://wiki.dlang.org/DIP82


At present, we really can't put unittest blocks - including ddoc-ed unittest blocks - inside of templated types, which has been a problem for Phobos and came up again just the other day when someone was helpful and went and tried to turn the examples in std.datetime's *Interval types in ddoc-ed unit tests, but we had to reject the changes, because it would have resulted in those tests being compiled into every instantiation of those templated types, polluting the code of anyone who uses them as well as making the Phobos unit tests take longer to run for no extra benefit whatsoever.

So, I wrote up this DIP in the hopes of solving the problem in a clean and simple manner which fits in well with the existing language features.

- Jonathan M Davis

On 2015-09-27 07:01, Jonathan M Davis wrote:
> This DIP provides a way to handle unittest blocks inside of templates
> which works with ddoc without compiling the unittest blocks into each
> instantiation.
>
> http://wiki.dlang.org/DIP82

How would this work in the compiler? Currently it only lexes and parses uninstantiated templates. I think that Andrei also wants to make the lexing and parsing lazy.

-- 
/Jacob Carlborg

2015-09-27 14:01 GMT+09:00 Jonathan M Davis via Digitalmars-d < digitalmars-d@puremagic.com>:

> This DIP provides a way to handle unittest blocks inside of templates which works with ddoc without compiling the unittest blocks into each instantiation.
>
> http://wiki.dlang.org/DIP82
>
>
> At present, we really can't put unittest blocks - including ddoc-ed unittest blocks - inside of templated types, which has been a problem for Phobos and came up again just the other day when someone was helpful and went and tried to turn the examples in std.datetime's *Interval types in ddoc-ed unit tests, but we had to reject the changes, because it would have resulted in those tests being compiled into every instantiation of those templated types, polluting the code of anyone who uses them as well as making the Phobos unit tests take longer to run for no extra benefit whatsoever.
>
> So, I wrote up this DIP in the hopes of solving the problem in a clean and simple manner which fits in well with the existing language features.


Interesting proposal.

Quick questions:

1. I think the token `static` should be placed at the immediate front of `unittest`

    static unittest { ... }
    static nothrow unittest { ... }   // NG

It's consistent with the behavior of `static this()` family.

2. Currently the members of template won't be semantically analyzed until it's instantiated. So, when the `static unittest` is enclosed in other blocks, how it works?

    template Foo(T) {
        // inside conditional compilation blocks:
        static if (is(T == int)) {
            static unittest { ... }
        }
        version (UserDefinedVersion) {
            static unittest { ... }
        }
        debug (UserDefinedDebugId) {
            static unittest { ... }
        }

        // inside block/labeled attributes
        nothrow {
            static unittest { ... }
        }
        @safe:
            static unittest { ... }

        mixin("static unittest { ... }");
        mixin(makeStaticUnittest());
    }

    string makeStaticUnittest() { return "static unittest { ... }"; }

Kenji Hara

On Sunday, 27 September 2015 at 05:01:48 UTC, Jonathan M Davis wrote:
> This DIP provides a way to handle unittest blocks inside of templates which works with ddoc without compiling the unittest blocks into each instantiation.

I understand that this DIP solves a genuine problem, so understand that this bike-sheding comment is not a dismissal of the idea.

I think it's a bad idea to further confuse what static means, as it currently has no less than five different distinct uses.

* static ifs
* static array construction
* static assert
* static/shared static this
* static functions/data

Maybe a new keyword or some @attribute would make this clearer.

On Sunday, 27 September 2015 at 09:24:29 UTC, Jacob Carlborg wrote:
> On 2015-09-27 07:01, Jonathan M Davis wrote:
>> This DIP provides a way to handle unittest blocks inside of templates
>> which works with ddoc without compiling the unittest blocks into each
>> instantiation.
>>
>> http://wiki.dlang.org/DIP82
>
> How would this work in the compiler? Currently it only lexes and parses uninstantiated templates. I think that Andrei also wants to make the lexing and parsing lazy.

I don't know the details of how this would be implemented in the compiler, but clearly what that the compiler currently does is enough to generate documentation from the ddoc comments inside of templates, and I wouldn't think that detecting a static unittest would be much more complicated than that. It can then do full analysis of the static unittest if it's the module with the template that's being compiled, and it's being compiled with -unittest, and otherwise, it can pretty much ignore it (or at least, ignore it as much as any other unittest block gets ignored when -unittest isn't used).

But we have a real need here if we want to be able to actually have unit tests inside of templated types next to what they're unit testing. So, I would certainly hope that it's reasonable to implement something like this without complicating the compiler much. Certainly, the fact that ddoc works inside of templates makes it seem like this should be feasible.

- Jonathan M Davis

On Sunday, 27 September 2015 at 14:10:06 UTC, Jack Stouffer wrote:
> On Sunday, 27 September 2015 at 05:01:48 UTC, Jonathan M Davis wrote:
>> This DIP provides a way to handle unittest blocks inside of templates which works with ddoc without compiling the unittest blocks into each instantiation.
>
> I understand that this DIP solves a genuine problem, so understand that this bike-sheding comment is not a dismissal of the idea.
>
> I think it's a bad idea to further confuse what static means, as it currently has no less than five different distinct uses.
>
> * static ifs
> * static array construction
> * static assert
> * static/shared static this
> * static functions/data
>
> Maybe a new keyword or some @attribute would make this clearer.

static in this case is consistent with how static is used on constructors or any symbol at the class or struct level where it makes it so that the symbol is for the class or struct as a whole rather than per instance. For static unittest blocks, it makes it so that the unittest is for the template as a whole and not for a particular instantiation. So, this definitely in line with how static works already, and I think that it would be much worse to add a new keyword or attribute for this. static is the obvious choice IMHO.

- Jonathan M Davis

On Sunday, 27 September 2015 at 10:32:00 UTC, Kenji Hara wrote:
> 2015-09-27 14:01 GMT+09:00 Jonathan M Davis via Digitalmars-d < digitalmars-d@puremagic.com>:
>
>> This DIP provides a way to handle unittest blocks inside of templates which works with ddoc without compiling the unittest blocks into each instantiation.
>>
>> http://wiki.dlang.org/DIP82
>>
>>
>> At present, we really can't put unittest blocks - including ddoc-ed unittest blocks - inside of templated types, which has been a problem for Phobos and came up again just the other day when someone was helpful and went and tried to turn the examples in std.datetime's *Interval types in ddoc-ed unit tests, but we had to reject the changes, because it would have resulted in those tests being compiled into every instantiation of those templated types, polluting the code of anyone who uses them as well as making the Phobos unit tests take longer to run for no extra benefit whatsoever.
>>
>> So, I wrote up this DIP in the hopes of solving the problem in a clean and simple manner which fits in well with the existing language features.
>
>
> Interesting proposal.
>
> Quick questions:
>
> 1. I think the token `static` should be placed at the immediate front of `unittest`
>
>     static unittest { ... }
>     static nothrow unittest { ... }   // NG
>
> It's consistent with the behavior of `static this()` family.

I confess that I've always found it to be inconsistent that static constructors required that the static be immediately before rather than being allowed before or after and among other attributes like would occur with any other function. But if we're going to retain that inconsistency with static constructors, it doesn't really make things much worse to do the same with unnittest blocks.

> 2. Currently the members of template won't be semantically analyzed until it's instantiated. So, when the `static unittest` is enclosed in other blocks, how it works?
>
>     template Foo(T) {
>         // inside conditional compilation blocks:
>         static if (is(T == int)) {
>             static unittest { ... }
>         }
>         version (UserDefinedVersion) {
>             static unittest { ... }
>         }
>         debug (UserDefinedDebugId) {
>             static unittest { ... }
>         }
>
>         // inside block/labeled attributes
>         nothrow {
>             static unittest { ... }
>         }
>         @safe:
>             static unittest { ... }
>
>         mixin("static unittest { ... }");
>         mixin(makeStaticUnittest());
>     }
>
>     string makeStaticUnittest() { return "static unittest { ... }"; }

Well, my first question would be what happens with a ddoc comment on a symbol within a template? It works to put ddoc comments on symbols inside of templates, so clearly we have to have rules of some kind which deal with each of the cases that you have here, though I'm not as familiar as I probably should be with which of those results in a symbol having its ddoc comment in the documentation and which doesn't.

Regardles, it does indeed get a bit interesting. Ideally, I would think that all of them save for the static if and mixins would act exactly like they would outside of a template, since nothing about this is specific to a template instantiation, but if no semantic analysis is currently done on a template until it's instantiated, then supporting a static unittest inside of a version block, debug block, a block for an attribute, or after an attribute used like a label would mean that at least some amount of semantic analysis would have to occur that doesn't necessarily happen now (though again, I wonder how that works with ddoc comments). Maybe no semantic analysis is done if no static unittest is found inside of them, and if one or more static unittest is found inside of them, the minimal amount of semantic analysis required to determine whether they get compiled in is done?

As for the static if, I don't see how it can work if it depends on any template arguments. I would think that it would work easily enough like the version blocks and debug blocks and whatnot if the condition did not depend on the template arguments, but I think that it's clear that if the condition depends on the template arguments at all, then any static unittest blocks in there aren't going to work and probably should be considered errors (even if it's just when the template is instantiated and the static if's condition is true).

Now, as to the mixins... It would be nice if they worked, but I have a hard time believing that it's reasonable to make them work. We would like to be able to have mixins work with ddoc comments, since without that, we can't put documentation on code that's mixed in. So, if we ever implement that, then it might be reasonable to do the same with static unittest blocks. But other than that, the result would be that every mixin inside of a template would have to be generated just to see whether it had a static unittest in it, which is kind of ugly. If the argument to mixin required a template argument to generate it, then it could clearly be skipped, but if not, then we'd have to generate it just to check, which would increase the compilation times for any template which did that, which wouldn't exactly be pretty. It probably wouldn't be a big deal in general, since in most cases, if you're going to mix in code like that, you're probably going to want to do it based on one or more template arguments, in which case, the compiler would be able to see that it didn't need to generate the mixin. Still, it would probably be better to just not support mixins like this (at least initially) and possibly to make it an error if a static unittest gets mixed in, since it would only be mixed in when the template was instantiated.

But even if we did something simple and made it so that static unittest blocks couldn't legally go inside of a version block, debug block, etc. and that all attributes on it had to be applied to it directly in order to have any effect, it would still be an improvement over what we have now. I would hope though that we could do something similar to what we do with ddoc comments inside of a template and that the rules that that uses would work for this too. But I don't know exactly what those rules are, so I don't know how well that will work.

- Jonathan M Davis

On Sunday, 27 September 2015 at 15:06:28 UTC, Jonathan M Davis wrote:
> On Sunday, 27 September 2015 at 10:32:00 UTC, Kenji Hara wrote:

>> 2. Currently the members of template won't be semantically analyzed until it's instantiated. So, when the `static unittest` is enclosed in other blocks, how it works?
[snip]

Another possibility is that because a static unittest block only matters when the module that it's in is compiled - and only when compiled with -unittest - we could make it so that a template has full semantic analysis done on it (or at least whatever is necessary to compile static unittest blocks) when it's the module that it's in which is being compiled (and -unittest is used), whereas when the module is merely imported, or -unittest is not used, only what's done currently gets done, and static unittest blocks can be ignored when the template is instantiated or -unittest is not used. And doing that would even work with mixins.

It still wouldn't work to have static unittest blocks inside of static if blocks or mixins that required any template arguments, but the rest should work as long as the semantic analysis is done when the module that the template is in is being compiled with -unittest, and the static unittest blocks inside of code that can't be semantically analyzed without instantiating the template can be errors when the template gets instantiated (since that would be better than having them silently do nothing).

That would increase compilation times for templates when compiling the module that they're in with -unittest, but it would only be when compiling that module with -untitest and not for anyone importing the module, so I think that the additional cost would be worth it - particularly in light of how it harms maintenance to have to put all of the unittest blocks outside of the template like we do now.

- Jonathan M Davis

On 2015-09-27 07:01, Jonathan M Davis wrote:
> This DIP provides a way to handle unittest blocks inside of templates
> which works with ddoc without compiling the unittest blocks into each
> instantiation.
>
> http://wiki.dlang.org/DIP82

I assume you won't have access to the template parameters?

-- 
/Jacob Carlborg

On Monday, 28 September 2015 at 09:06:52 UTC, Jacob Carlborg wrote:
> On 2015-09-27 07:01, Jonathan M Davis wrote:
>> This DIP provides a way to handle unittest blocks inside of templates
>> which works with ddoc without compiling the unittest blocks into each
>> instantiation.
>>
>> http://wiki.dlang.org/DIP82
>
> I assume you won't have access to the template parameters?

No. It wouldn't make sense to, since it's not an instantiation of the template and thus there are no template arguments. Basically, aside from ddoc, this should be the same as if the unittest block were outside of the template, but by having a unittest block inside of the template, it can be immediately after the function that it's testing, allowing it to be used for generating the examples for ddoc as well as making it easier to find and maintain the tests that go with each function.

It would be kind of like how with a static function in a class/struct, you don't have access to the this pointer/reference, since you're not dealing with an actual instance. Only in this case, it's the template arguments which you wouldn't have access to, not the this pointer/reference, because you're not dealing with an actual instantiation of the template.

- Jonathan M Davis