On Thursday, March 14, 2013 13:07:16 Andrei Alexandrescu wrote:
> Very simple. Traditionally there's two crucial epochs known as compilation time and run time. (There's some minor distinctions like link time etc.) The whole notion of concepts and other type systems for templates is predicated on three crucial epochs: library compilation time, library user compilation time, and run time. The logic goes, someone writes a generic library and wants to distribute it to users. Users shouldn't ever see bugs caused by e.g. typos in the library.
> 
> So the crowd that use meta-type systems is formed of library writers who want to distribute libraries without ever instantiating them. I don't think that's a good crowd to cater for.
> 
> I've been surprised to figure how many people don't get this flow, or only have a vague image of it. Although meta-types are arguably "the right thing" to do, they're a lot less attractive once it's clear what scenarios they help.

The main problem is not people who don't unit test their templates but the fact that it's pretty much impossible to cover every possible instantiation of a template, and so it can be pretty easy to miss stuff. Better unit testing will obviously cover more, and on the whole, I think that that's a fine solution, but it would also be nice if the compiler caught more. However, given that it's pretty much a given that the compiler won't catch the stuff that's actually going to be a problem (e.g. a type with weird characteristics due to alias this or whatnot ends up not working correctly with a template), I'm not sure that it the compiler really can be improved to help out in any meaningful way. In general, the more the compiler can catch for you at compile time, the better, but there are definite limits to that even with a very smart compiler (and the smarter the compiler, the more likely it is to be buggy).

- Jonathan M Davis

On Thu, Mar 14, 2013 at 08:49:11PM +0100, Timon Gehr wrote:
> On 03/14/2013 06:48 PM, H. S. Teoh wrote:
> >...
> >
> >We could, of course, automatically generate a very large number of template argument combinations and check to see if they're instantiable.  In fact, we could write unittests that loop over every combination of a given list of types and instantiate the template to be tested with them.  This is well within D's metaprogramming capabilities.
> >
> >...
> 
> But way behind state-of-the-art research in program validation.

Yes, which was my point.


T

-- 
MACINTOSH: Most Applications Crash, If Not, The Operating System Hangs

On 3/14/2013 11:04 AM, H. S. Teoh wrote:
> But that was my point, if we had concepts, then the compiler could
> statically check the syntax of the template without needing to iterate
> over all combinations of types.

A syntax check is not a semantic check.

> Right now, we *have* to essentially check all combinations in unittests
> if we want to be sure that the template is instantiable in all cases.

You have to anyway.

Consider functions. Just because the function passes syntactical and semantic checks does not at all mean it passes with all combinations of parameter values. We still need unittests, and while unittests also do not guarantee the function is correct in all cases, it is considered unprofessional to ship a function that was never called at runtime in any tests.

On 3/14/2013 12:51 PM, Timon Gehr wrote:
> On 03/14/2013 08:46 PM, Walter Bright wrote:
>> On 3/14/2013 10:20 AM, Dmitry Olshansky wrote:
>>> Maybe we then should help people that routinely instantiate their
>>> templates to
>>> see if they compile.
>>
>> We already have a tool to do this:
>>
>>     dmd -cov test.d
>>
>
> But it does not work.
>
> See this recent D.learn thread:
> http://forum.dlang.org/thread/wwjaeexnyaeqnqsqydte@forum.dlang.org

-cov works fine for templates containing executable code.

On 3/14/2013 12:53 PM, Dmitry Olshansky wrote:
> No it's not.
> It's I'm out of words to even begin to describe how it doesn't do what's needed
> in this case. To start 'dmd -cov test.d' doesn't instantiate all of templates
> (or generate code to do so) with the right sets of types auto-magically.


It helps by showing which paths were run by the testing.

On 3/14/2013 11:38 AM, Jonathan M Davis wrote:
> Though this is exactly a case where 100% unit test coverage doesn't mean much.
> All that means is that each path has been instantiated and run. It doesn't
> mean that it's covered enough possible instantiations to properly test the
> template.

I understand this. But my experience in the matter has been that if the tests cover 100% of the code paths, the incidence of undetected bugs in the code goes very, very low.

So, I disagree with "doesn't mean much". Pedantically, it means little, but practically, it means a great deal.

(Of course, for multithreading you'd be right - runtime testing is notoriously unproductive in finding concurrency mistakes.)

15-Mar-2013 00:12, Walter Bright пишет:
> On 3/14/2013 12:53 PM, Dmitry Olshansky wrote:
>> No it's not.
>> It's I'm out of words to even begin to describe how it doesn't do
>> what's needed
>> in this case. To start 'dmd -cov test.d' doesn't instantiate all of
>> templates
>> (or generate code to do so) with the right sets of types auto-magically.
>
>
> It helps by showing which paths were run by the testing.

It doesn't show which types ended up in each of template instantiation.
This is a critical piece of information.

The way around that would be having a separate file per combination of type and run each with cov separately then use external tool to analyze and aggregate the logs.

Far, far from 'having a tool to do this'. The aspect of running separate tests is even more frustrating as  the fact compiler knows all of the relevant info already.

-- 
Dmitry Olshansky

On 3/14/2013 12:53 PM, Dmitry Olshansky wrote:
> No it's not.
> It's I'm out of words to even begin to describe how it doesn't do what's needed
> in this case. To start 'dmd -cov test.d' doesn't instantiate all of templates
> (or generate code to do so) with the right sets of types auto-magically.

For the following program:
---------test.d-------------
import core.stdc.stdio;
public import std.whatever;

int main()
{
    printf("Success!\n");
    return 0;
}
----------------------------

Try compiling:

    dmd -cov test.d std\whatever -unittest

and examine std-whatever.lst.

We can do a lot, lot better before we need something better than -cov.

15-Mar-2013 00:27, Walter Bright пишет:
> On 3/14/2013 12:53 PM, Dmitry Olshansky wrote:
>> No it's not.
>> It's I'm out of words to even begin to describe how it doesn't do
>> what's needed
>> in this case. To start 'dmd -cov test.d' doesn't instantiate all of
>> templates
>> (or generate code to do so) with the right sets of types auto-magically.
>
> For the following program:
> ---------test.d-------------
> import core.stdc.stdio;
> public import std.whatever;
>
> int main()
> {
>      printf("Success!\n");
>      return 0;
> }
> ----------------------------
>
> Try compiling:
>
>      dmd -cov test.d std\whatever -unittest
>
> and examine std-whatever.lst.
>
> We can do a lot, lot better before we need something better than -cov.
>
>

Point taken. That doesn't detract us from:
a) fixing issues with -cov

It counts time a LOC is executed. Would nice to add instantations counter (determined at compile time) as well so as to see if declarations are all covered. That  + CTFE-only counted as 0.

b) unifying template fuzzy testing in Phobos

We have lots of these wheels reinvented across Phobos alone.

c) prototyping other higher-level tools to aid debugging generic code

-- 
Dmitry Olshansky

On Fri, Mar 15, 2013 at 12:37:00AM +0400, Dmitry Olshansky wrote:
> 15-Mar-2013 00:27, Walter Bright пишет:
[...]
> >For the following program:
> >---------test.d-------------
> >import core.stdc.stdio;
> >public import std.whatever;
> >
> >int main()
> >{
> >     printf("Success!\n");
> >     return 0;
> >}
> >----------------------------
> >
> >Try compiling:
> >
> >     dmd -cov test.d std\whatever -unittest
> >
> >and examine std-whatever.lst.
> >
> >We can do a lot, lot better before we need something better than -cov.
> >
> >
> 
> Point taken. That doesn't detract us from:
> a) fixing issues with -cov
> 
> It counts time a LOC is executed. Would nice to add instantations counter (determined at compile time) as well so as to see if declarations are all covered. That  + CTFE-only counted as 0.
> 
> b) unifying template fuzzy testing in Phobos
> 
> We have lots of these wheels reinvented across Phobos alone.
> 
> c) prototyping other higher-level tools to aid debugging generic code
[...]

d) factor out useful test data/objects (e.g. ranges of particular types, with particular sets of data, etc.) into a common place so that they can be reused across unittests.


T

-- 
"Maybe" is a strange word.  When mom or dad says it it means "yes", but when my big brothers say it it means "no"! -- PJ jr.