On 9/10/2014 1:01 AM, Kagamin wrote:
> It's questionable that testing every template instantiation is intended:
> you can't do the same for templated functions.

Regardless of the (de)merits of deliberately putting unittests in a template, the important thing is this:

Whether a person intentionally *or inadvertently* winds up with a unittest inside a template, the whole "causes some of my unittests run when my lib's *users* run *their* project's unittests" is a highly unintuitive non-localized consequence.

On Wednesday, 10 September 2014 at 02:14:39 UTC, Nick Sabalausky wrote:
> This is getting to be (or rather, *continuing* to be) a royal PITA:
>
> https://github.com/rejectedsoftware/vibe.d/issues/673
>
> I don't mean to pick on Vibe.d in particular, but can we have a solution (that doesn't involve obscure corners of druntime, or demanding everyone use the same build system) for running our unittests *without* triggering a cascading avalanche of unittests from ALL third party libs that don't cooperate with the [frankly] quite clumsy version(mylib_unittests) hack?!

My opinion stays the same - it is good reliable default and should stay that way. And yes, I absolutely hate when libraries do version(mylib_unittest) hacks forcing me to bother with millions of versions to enable them back.

Simple solutions is to provide alternative unit test runner module as part of Phobos which does the module/package filtering.

On Tue, 09 Sep 2014 22:37:23 -0400
Nick Sabalausky via Digitalmars-d <digitalmars-d@puremagic.com> wrote:

> If you think that's reasonable when it's the COMPILER that inserts all those calls (or druntime, whatever), then go ahead with it yourself.
sure it's a compiler. that's why we have "-unittest" arg. and separate compilation rocks (why, it's cheap! and windows... ah, i don't care).

yes, templates still sux (considering built-in unittests). this is where compiler should be fixed: it should compile unittest invocations only for templates that comes from the module it compiles, i think.

On Tue, 09 Sep 2014 23:00:06 -0400
Nick Sabalausky via Digitalmars-d <digitalmars-d@puremagic.com> wrote:

> Well, ok, I guess it could be done with separate invocations of DMD, but that has non-trivial implications for build performance
we can cache analyzed ASTs somewhere. this can speedup
separate compilations a little. but i don't see any problems with
separate compilation: it's fast enough even with alot of modules.

On 9/10/2014 3:57 AM, Dicebot wrote:
>
> My opinion stays the same - it is good reliable default and should stay
> that way. And yes, I absolutely hate when libraries do
> version(mylib_unittest) hacks forcing me to bother with millions of
> versions to enable them back.
>

Being "default" is one thing, I really don't care what the default is - but for now it's not just "the default", it's the only realistic option without getting into PITA workarounds.

Some people like including 3rd party library unittests with their own (apparently), some people like *not* including them. So we have some projects using version(foo_unittest) and other projects not using it. The current result is: *Nobody* can easily get what they want: Not people who like it my way, and not people who like it your way. Not easily enough anyway.

So it's a global ecosystem problem. It needs fixed. (And to start with, general agreeance that it should be fixed.)

> Simple solutions is to provide alternative unit test runner module as
> part of Phobos which does the module/package filtering.

It needs to be *crystal* clear and trivially simple:

A. Knowing that it exists.
B. Sussing out how to use it.
C. Actually using it.

Otherwise, version(foo_unittest) will continue to linger out of an individual's convenience, and so once again, not everyone gets to have things their way.

A related concern I have about a Phobos-only solution: When/where do you configure it? Obviously it can't happen anywhere in the scope of main. Are static ctors sufficient? I would *think* so, but I'm not certain.

On Wednesday, 10 September 2014 at 08:21:34 UTC, Nick Sabalausky wrote:
>> Simple solutions is to provide alternative unit test runner module as
>> part of Phobos which does the module/package filtering.
>
> It needs to be *crystal* clear and trivially simple:
>
> A. Knowing that it exists.
> B. Sussing out how to use it.
> C. Actually using it.
>
> Otherwise, version(foo_unittest) will continue to linger out of an individual's convenience, and so once again, not everyone gets to have things their way.
>
> A related concern I have about a Phobos-only solution: When/where do you configure it? Obviously it can't happen anywhere in the scope of main. Are static ctors sufficient? I would *think* so, but I'm not certain.

Leandro Lucarella has written similar solution for our internal testing purposes. It provides a test runner that is configured by CLI arguments and allows filtering of modules and non-fatal test failures. It is for D1/tango but I am pretty sure it can be ported to D2 runtime with close to 0 changes. I have not managed to convince him it is worth making public though :(

On Wednesday, 10 September 2014 at 08:05:28 UTC, ketmar via Digitalmars-d wrote:
> yes, templates still sux (considering built-in unittests). this is
> where compiler should be fixed: it should compile unittest invocations
> only for templates that comes from the module it compiles, i think.

That way it will be clear.

By the way, will this link?
---
template A(T)
{
  int a;
  unittest{ f(); }
}

version(unittest) void f(){}
---
If this module is compiled without unittests, `f` function will not be compiled, then if another module is compiled with unittests and instantiates the template, the template's unittest is compiled and it will link with `f` function, which wasn't compiled, so linking will fail?

On 09/10/2014 07:01 AM, Kagamin wrote:
> It's questionable that testing every template instantiation is intended:
> you can't do the same for templated functions.

Yes you can.

On Tue, 09 Sep 2014 22:13:57 -0400, Nick Sabalausky <SeeWebsiteToContactMe@semitwist.com> wrote:

> This is getting to be (or rather, *continuing* to be) a royal PITA:
>
> https://github.com/rejectedsoftware/vibe.d/issues/673
>
> I don't mean to pick on Vibe.d in particular, but can we have a solution (that doesn't involve obscure corners of druntime, or demanding everyone use the same build system) for running our unittests *without* triggering a cascading avalanche of unittests from ALL third party libs that don't cooperate with the [frankly] quite clumsy version(mylib_unittests) hack?!

Hm... would it be acceptable if unit tests were compiled, but not run? Because the unit tester runtime just foreach's over every module and runs the tests contained in that module.

Now, I don't know how it works if you compile a template unit test -- does it get deposited into the module that instantiated or into the module that defined it? Worth looking into I think.

-Steve

A fairly simple solution to this would be to compile unittests only for root modules (the ones that are part of the compilation), not for imported modules.
Then everyone can decide for which modules to use -unittest.