On 03/14/2013 06:51 PM, Andrei Alexandrescu wrote:
> On 3/14/13 1:48 PM, H. S. Teoh wrote:
>> I don't agree. Phobos is a prime example. Does Phobos have unittests?
>> Yes, and lots of them. Does it still have non-compilable template
>> instantiations? Yes, because unittests can't cover all possibilities --
>> there are too many possible combinations of template arguments. There
>> are bound to be untested combinations which don't work but we're unaware
>> of.
>
> If you found a few, that would be great. I don't think you'll have an
> easy time.
>
> Andrei
>

Challenge accepted. Clearly the Phobos developers do not instantiate their templates before shipping them. :o)

The following breaks most of std.range, and most of std.algorithm could likely be broken too, but I am too lazy to investigate.

import std.range, std.algorithm;

struct TrollFace{
	@property string front()const{ return "troll"; }
	@property string back()const{ return "face"; }
	@property bool empty()const{ return true; }
	void popFront()const{ }
	void popBack()const{ }
	@property inout(TrollFace) save()inout{ return this; }
	auto opIndex(size_t index){ return front; }
	@property size_t length()inout{ return 0; }
	int* x;
}
struct TrollierFace{
	TrollFace face;
	alias face this;
	@disable this(this);
	@property inout(TrollierFace) save()inout{ return inout(TrollierFace)(face); }
}

void main(){
	immutable TrollFace a,b,c;
	a.retro();
	a.stride(2);
	chain(a,b,c);
	roundRobin(a,b,c);
	a.radial();
	a.radial(0);
	a.take(2);
	(immutable(TrollierFace)()).takeExactly(2);
	(immutable(TrollierFace)()).takeOne();
	(immutable(TrollierFace)()).takeNone();
	(immutable(TrollierFace)()).drop(2);
	(immutable(TrollierFace)()).dropExactly(0);
	(immutable(TrollierFace)()).dropOne();
	(immutable(TrollierFace)()).repeat();
	a.cycle();
	a.zip(b);
	lockstep((immutable(TrollierFace)()),b);
	a.frontTransversal();
	a.transversal(0);
	a.indexed([0]);
	a.chunks(5);
	a.filter!(a=>true);
	a.map!(a=>a);
	// ... (and so on)
}

On 03/14/2013 09:11 PM, Walter Bright wrote:
> On 3/14/2013 12:51 PM, Timon Gehr wrote:
>> On 03/14/2013 08:46 PM, Walter Bright wrote:
>>> On 3/14/2013 10:20 AM, Dmitry Olshansky wrote:
>>>> Maybe we then should help people that routinely instantiate their
>>>> templates to
>>>> see if they compile.
>>>
>>> We already have a tool to do this:
>>>
>>>     dmd -cov test.d
>>>
>>
>> But it does not work.
>>
>> See this recent D.learn thread:
>> http://forum.dlang.org/thread/wwjaeexnyaeqnqsqydte@forum.dlang.org
>
> -cov works fine for templates containing executable code.

Then it is not what Dmitry was asking for. Anyway -cov is near-useless for me because it does not support string mixins (execution attributed to wrong code line) and executable code coming from template mixins (sometimes counted, sometimes not).

Walter Bright:

> I understand this. But my experience in the matter has been that if the tests cover 100% of the code paths, the incidence of undetected bugs in the code goes very, very low.

This is not so much true for template-heavy code.

And 100% code coverage is not enough even for regular code, that's why they have invented this for Java:

http://dev.theladders.com/2013/02/mutation-testing-with-pit-a-step-beyond-normal-code-coverage/

Bye,
bearophile

You are awesome.

On 3/14/2013 2:01 PM, Timon Gehr wrote:
> Anyway -cov is near-useless for me
> because it does not support string mixins (execution attributed to wrong code
> line) and executable code coming from template mixins (sometimes counted,
> sometimes not).

Please file bugzilla entries for these.

On 3/14/2013 1:57 PM, Timon Gehr wrote:
> The following breaks most of std.range, and most of std.algorithm could likely
> be broken too, but I am too lazy to investigate.

Please file this with bugzilla.

On 3/14/2013 2:26 PM, bearophile wrote:
> Walter Bright:
>
>> I understand this. But my experience in the matter has been that if the tests
>> cover 100% of the code paths, the incidence of undetected bugs in the code
>> goes very, very low.
>
> This is not so much true for template-heavy code.

I don't believe that without further evidence.


> And 100% code coverage is not enough even for regular code,

I didn't say it was perfect. I said it is very effective, and doesn't leave much left. I have very positive experience with code I took the time to do coverage testing on, and so have others.

On 3/14/13 3:53 PM, Dmitry Olshansky wrote:
> It's I'm out of words to even begin to describe how it doesn't do what's
> needed in this case. To start 'dmd -cov test.d' doesn't instantiate all
> of templates (or generate code to do so) with the right sets of types
> auto-magically.

You are well understood. What a coverage test does help with is show you there is code that has never been instantiated.

Andrei

On Thursday, 14 March 2013 at 20:46:11 UTC, H. S. Teoh wrote:
> b) unifying template fuzzy testing in Phobos
> 
> We have lots of these wheels reinvented across Phobos alone.

By the way, is there any theoretical way to separate compilation errors due to failed constraints from errors due to errors in template body? I can't imagine one now, but if it is possible, some really cool unit test helper module can be written (In fact I almost started writing it when remembered about this requirement)

On Thu, Mar 14, 2013 at 02:37:29PM -0700, Walter Bright wrote:
> On 3/14/2013 1:57 PM, Timon Gehr wrote:
> >The following breaks most of std.range, and most of std.algorithm could likely be broken too, but I am too lazy to investigate.
> 
> Please file this with bugzilla.

I think it deserves several different bug reports, as they appear to be failing in different places.


T

-- 
Always remember that you are unique. Just like everybody else. -- despair.com