Thread overview
Helpers for writing unittests
Jul 31, 2015
Idan Arye
Jul 31, 2015
Jonathan M Davis
Jul 31, 2015
Idan Arye
Jul 31, 2015
Jonathan M Davis
Jul 31, 2015
jmh530
July 31, 2015
The "Rant after trying Rust a bit" thread(http://forum.dlang.org/thread/ckjukjfkgrguhfhkdhhj@forum.dlang.org) talks mostly about traits, and how people want Rust's traits(or C++'s Concepts) in D. As a general rule, I believe that taking a feature from one language and sticking it in another works just as nicely as taking off a leg from one chair and using it to replace a broken leg from a different firm's chair. You'd be lucky if they have similar length, and even if you sand the legs to have exact same length the balance will be off. You might be able to sit on it, but it won't be as comfortable as a chair that all it's parts fit together.

Instead, it's better to look at the problem the feature solves, and how the language without that feature approaches the problem. In our case, the problem is that compile-time errors in the templates only pop up once the template is instantiated. Rust's solution is to use Traits, and Walter made it pretty clear that the D way to solve this problem is to instantiate the template in a unittest.

As seen in that thread, many people find this solution lacking. Writing 100% coverage tests for instantiating templates is a long and tiring process. In response, the anti-traits camp accuses these people of being lazy.

Well, I say - programmers should be lazy! That's why we are programming - because we are lazy and want the computers to do our work for us. So the current D solution is not enough - but that doesn't mean a feature transplant from Rust is a good solution - we need to find a D style solution, that'll fit with the rest of the language.


What I'm thinking about is a unittest helper that'll help in checking different instantiations of the template. A quick proof of concept - http://dpaste.dzfl.pl/8907c3a7d54c - shows how the unittest found that foo doesn't work with long and ulong, and printed easy-to-understand errors.

Just like IntegerTypes we can have many more lists of types for different categories of types we want to test - string types, range types etc. With this in Phobos, writing unittests with full coverage(compile-time only) will be much easier.

Note that we just want to see that it compiles - we don't want to actually run it, because than we'd have to supply templated test data and test results to compare with, which is a much harder problem. These are compile-time mocks - the problem they solve is limited to compilation so they will be able to solve it well and elegantly. This does not come instead of tests that actually run - a unittest can test compilation on all the relavant types and actually run and check the results only for a subset of these types.

Thoughts?
July 31, 2015
On Friday, 31 July 2015 at 00:07:43 UTC, Idan Arye wrote:
> Thoughts?

Some unit test helpers for this sort of thing might be nice, but I don't think that it really buys us much with this particular case. You could just as easily do

unittest
{
    foreach(T; TypeTuple!(ubyte, byte, ushort, short, uint, int, ulong, long))
        static assert(is(typeof(foo(T.init)));
}

and the code is basically as long as is with assertCompilesWith - shorter even. The above example is longer due to not using an alias for the integral types like you did, but if that same alias were used, then it becomes

unittest
{
    foreach(T; IntegerTypes)
        static assert(is(typeof(foo(T.init)));
}

which isn't all that different than

unittest
{
    assertCompilesWith!(IntegerTypes, (x) {
        foo(x);
    });
}

but it doesn't require a lambda or creating any helpers, and it cuts down on the number of unit testing functions that are templated (it can cost a lot in terms of compilation time and memory, if you use a lot of templates as helpers for unit tests). In addition, in most cases, I don't see much point in simply testing that the templated function or type compiles with various types. It's generally far better to do something like

foreach(T; IntegerTypes)
{
    // Tests which actually test foo with T rather than just that it compiles
}

And in that case, explicitly testing compilation is unnecessary. You're already testing far more than that.

So, I applaud the attitude and motivation behind your suggestion, but this particular helper doesn't really help IMHO.

- Jonathan M Davis
July 31, 2015
On Friday, 31 July 2015 at 00:30:23 UTC, Jonathan M Davis wrote:
> On Friday, 31 July 2015 at 00:07:43 UTC, Idan Arye wrote:
>> Thoughts?
>
> Some unit test helpers for this sort of thing might be nice, but I don't think that it really buys us much with this particular case. You could just as easily do
>
> unittest
> {
>     foreach(T; TypeTuple!(ubyte, byte, ushort, short, uint, int, ulong, long))
>         static assert(is(typeof(foo(T.init)));
> }
>
> and the code is basically as long as is with assertCompilesWith - shorter even. The above example is longer due to not using an alias for the integral types like you did, but if that same alias were used, then it becomes
>
> unittest
> {
>     foreach(T; IntegerTypes)
>         static assert(is(typeof(foo(T.init)));
> }
>
> which isn't all that different than
>
> unittest
> {
>     assertCompilesWith!(IntegerTypes, (x) {
>         foo(x);
>     });
> }

The resulting compilation errors are extremely different. With your method we get:

/d391/f994.d(31): Error: static assert (is(typeof(__error))) is false

which doesn't tell us what the problem is. With my method we get:

/d433/f500.d(25): Error: cannot implicitly convert expression (x) of type ulong to int /d433/f500.d(31): Error: template instance f500.foo!ulong error instantiating

And yes, a lot of "static stack trace" after that, but these too lines tells us what the problem is and which template parameters caused it.

Of course, if you just ran the function inside the foreach loop you'd get nice error messages as well - but then you'd have to write tests that actually run. Which is easy for numbers, because they are all the same type of data with different sizes, but can get tricky when you have more complex types, that differ more in their behavior.
July 31, 2015
On Friday, 31 July 2015 at 00:07:43 UTC, Idan Arye wrote:
> people want Rust's traits(or C++'s Concepts) in D.

Just to quibble, that wasn't my take-a-way from the thread. My take-a-way is that Jonathan Davis is really good at explaining things.



July 31, 2015
On Friday, 31 July 2015 at 00:54:30 UTC, Idan Arye wrote:
> The resulting compilation errors are extremely different. With your method we get:
>
> /d391/f994.d(31): Error: static assert (is(typeof(__error))) is false
>
> which doesn't tell us what the problem is. With my method we get:
>
> /d433/f500.d(25): Error: cannot implicitly convert expression (x) of type ulong to int /d433/f500.d(31): Error: template instance f500.foo!ulong error instantiating
>
> And yes, a lot of "static stack trace" after that, but these too lines tells us what the problem is and which template parameters caused it.

True enough, but you can simply add T.stringof to the static assertion, and you'll know which instantiation is failing. And maybe what you're suggesting is worth having, but you can get pretty much the same thing easily without creating a template to do the test for you.

> Of course, if you just ran the function inside the foreach loop you'd get nice error messages as well - but then you'd have to write tests that actually run. Which is easy for numbers, because they are all the same type of data with different sizes, but can get tricky when you have more complex types, that differ more in their behavior.

Yes. It's generally hard to write tests which work with a variety of template instantiations, but the point is that knowing that the template instantiates with a particular type really doesn't tell you much. If you actually want to be testing the template with those types, you should be writing tests with those types whether you can templatize the tests or whether each instantiation needs to be tested separately. That being the case, I don't think that unit tests should normally be testing explicitly that a template instantiates with a type. Doing that is so insufficient that there isn't much point.

- Jonathan M Davis