I have started some unit tests:

	unittest
	{
		// Check row column indexing is correct
		Matrix	m;
		m[2,0] = 59;
		assert(m.matrix_flat[2]==59, "opIndexAssign failed");
		assert(m.matrix_flat[8]!=59, "opIndexAssign failed");
		
	}

If an assert fails the other assertions in a program do not fire. This is inconvient since the "real bug" introduced may be caused by some other assertion that has not been evaluated yet.

What I am looking for, therefore is more of a writefln of each failure, together with a single assertion to abort the program if any test has failed.

If there isn't such a feature then I suppose I need a "Global" class which holds a bit flag to be set when an assertion is registered, and then in main this class could be made to assert if the flag is set.

Spacen Jasset wrote:
>     I have started some unit tests:
> 
>     unittest
>     {
>         // Check row column indexing is correct
>         Matrix    m;
>         m[2,0] = 59;
>         assert(m.matrix_flat[2]==59, "opIndexAssign failed");
>         assert(m.matrix_flat[8]!=59, "opIndexAssign failed");
>            }
> 
> If an assert fails the other assertions in a program do not fire. This is inconvient since the "real bug" introduced may be caused by some other assertion that has not been evaluated yet.
> 
> What I am looking for, therefore is more of a writefln of each failure, together with a single assertion to abort the program if any test has failed.
> 
> If there isn't such a feature then I suppose I need a "Global" class which holds a bit flag to be set when an assertion is registered, and then in main this class could be made to assert if the flag is set.

You're looking for DUnit, which hasn't been written yet. I'd like to write such a thing, but a natural, xUnit-like syntax is difficult in D.

If you want something that targets D2, that's doable; you could write something like:

import dunit.api;
class FooTests : TestFixture
{
   mixin(DunitTest);
   override void setup () {}
   override void teardown () {}
   void testSomething()
   {
      // ...
   }
}

But in D1, there's no way (as far as I know) to iterate over the methods for a type.

Well, no, there is the vtbl. But that is a REALLY ugly solution.

So you're left with some syntax like:

class FooTests : TestFixture
{
   mixin(DunitTest);
   override void setup () {}
   override void teardown () {}
   this ()
   {
      tests["test for something"] =
      {
         // blah
      }
      tests["test for something else"] =
      {
         // blah
      }
   }
}

I guess that isn't so ugly. I can hack something together tonight.

Christopher Wright wrote:
> I guess that isn't so ugly. I can hack something together tonight.

That was actually quite easy. I'll add it to Felt and post a link in a few minutes.

Spacen Jasset wrote:
>     I have started some unit tests:
> 
>     unittest
>     {
>         // Check row column indexing is correct
>         Matrix    m;
>         m[2,0] = 59;
>         assert(m.matrix_flat[2]==59, "opIndexAssign failed");
>         assert(m.matrix_flat[8]!=59, "opIndexAssign failed");
>            }
> 
> If an assert fails the other assertions in a program do not fire. This is inconvient since the "real bug" introduced may be caused by some other assertion that has not been evaluated yet.
> 
> What I am looking for, therefore is more of a writefln of each failure, together with a single assertion to abort the program if any test has failed.
> 
> If there isn't such a feature then I suppose I need a "Global" class which holds a bit flag to be set when an assertion is registered, and then in main this class could be made to assert if the flag is set.

Not to knock on Mr. Wright's solution, which is probably excellent and more directly addressable to the problem at hand, but I'm working on a solution (designed to use IPC to be used from other applications, etc.) that does just this and a lot more. Check:
http://www.dsource.org/projects/descent/browser/trunk/descent.unittest/flute/src/org/dsource/descent/unittests/flute.d

And the docs at:
http://svn.dsource.org/projects/descent/trunk/descent.unittest/flute/doc/flute.html

Robert Fraser wrote:
> Spacen Jasset wrote:
>>     I have started some unit tests:
>>
>>     unittest
>>     {
>>         // Check row column indexing is correct
>>         Matrix    m;
>>         m[2,0] = 59;
>>         assert(m.matrix_flat[2]==59, "opIndexAssign failed");
>>         assert(m.matrix_flat[8]!=59, "opIndexAssign failed");
>>            }
>>
>> If an assert fails the other assertions in a program do not fire. This is inconvient since the "real bug" introduced may be caused by some other assertion that has not been evaluated yet.
>>
>> What I am looking for, therefore is more of a writefln of each failure, together with a single assertion to abort the program if any test has failed.
>>
>> If there isn't such a feature then I suppose I need a "Global" class which holds a bit flag to be set when an assertion is registered, and then in main this class could be made to assert if the flag is set.
> 
> Not to knock on Mr. Wright's solution, which is probably excellent and more directly addressable to the problem at hand, but I'm working on a solution (designed to use IPC to be used from other applications, etc.) that does just this and a lot more. Check:
> http://www.dsource.org/projects/descent/browser/trunk/descent.unittest/flute/src/org/dsource/descent/unittests/flute.d 
> 
> 
> And the docs at:
> http://svn.dsource.org/projects/descent/trunk/descent.unittest/flute/doc/flute.html 
> 

I should mention flute is in-progress, so unless _really_ don't want to change your unittest { } blocks into DUnit tests, use DUnit instead.

Robert Fraser wrote:
> Not to knock on Mr. Wright's solution, which is probably excellent and more directly addressable to the problem at hand, but I'm working on a solution (designed to use IPC to be used from other applications, etc.) that does just this and a lot more. Check:
> http://www.dsource.org/projects/descent/browser/trunk/descent.unittest/flute/src/org/dsource/descent/unittests/flute.d 
> 
> 
> And the docs at:
> http://svn.dsource.org/projects/descent/trunk/descent.unittest/flute/doc/flute.html 
> 

Flute seems like a general solution that can easily be adapted to DUnit, and that support would increase the popularity of DUnit. But can I use Flute today?

Flute plus standard D unittests will be a smaller commitment than DUnit, and will require somewhat less from programmers. But there's value in requiring organization and separation for your tests, I think. For one, it's easier to abstract out shared setup code, and be certain that that won't be included in your release builds.

Your trick for naming unittests is also scary. It works, though, I'll give it that. Against all odds, and without having to run any of the tests.

Christopher Wright wrote:
> Robert Fraser wrote:
>> Not to knock on Mr. Wright's solution, which is probably excellent and more directly addressable to the problem at hand, but I'm working on a solution (designed to use IPC to be used from other applications, etc.) that does just this and a lot more. Check:
>> http://www.dsource.org/projects/descent/browser/trunk/descent.unittest/flute/src/org/dsource/descent/unittests/flute.d 
>>
>>
>> And the docs at:
>> http://svn.dsource.org/projects/descent/trunk/descent.unittest/flute/doc/flute.html 
>>
> 
> Flute seems like a general solution that can easily be adapted to DUnit, and that support would increase the popularity of DUnit. But can I use Flute today?

Not unless you're on Windows/Phobos/D1 (I haven't gotten around to updating it for supporting other configs, since it's not officially being released until I get it integrated into Descent, which is a much larger project than Flute itself).

> Flute plus standard D unittests will be a smaller commitment than DUnit, and will require somewhat less from programmers. But there's value in requiring organization and separation for your tests, I think. For one, it's easier to abstract out shared setup code, and be certain that that won't be included in your release builds.

Strongly agreed. xUnit is a much better solution for large-scale development, but sticking mini unit tests in your code is hlpful during development phases.

> Your trick for naming unittests is also scary. It works, though, I'll give it that. Against all odds, and without having to run any of the tests.

Indeed, indeed. It would be very helpful if D allowed unittest(name) { } syntax.

Flute is mainly designed to be integrated into frontends, in particular Descent and CruiseControl. It would be brilliant if I could get DUnit tests in there, too, sicne as D moves towards large enterprise-class solutions, stronger unit testing support is certainly needed. Would it be okay if I added DUnit support alongside Flute support to Descent and to my Ant/CruiseControl plugin when I get around to doing that?

== Quote from Robert Fraser (fraserofthenight@gmail.com)'s article
> Flute is mainly designed to be integrated into frontends, in particular Descent and CruiseControl. It would be brilliant if I could get DUnit tests in there, too, sicne as D moves towards large enterprise-class solutions, stronger unit testing support is certainly needed. Would it be okay if I added DUnit support alongside Flute support to Descent and to my Ant/CruiseControl plugin when I get around to doing that?

Have you started such a plugin? If so, I'd love to help out. If not, any suggestions on where to get started?

I'd be quite thrilled to have DUnit support in Descent, as well. Let me know if there's anything I can do to make this simpler.

Christopher Wright wrote:
> Have you started such a plugin? If so, I'd love to help out. If not, any
> suggestions on where to get started?

Nope, it's on my *long* TODO list. I'm not even sure how to go about making it a CC plugin, I'd have to check how the JUnit plugin works. It might end up being an Ant task with some sort of listener/presentation component in CC.

Robert Fraser wrote:
> Christopher Wright wrote:
>> Have you started such a plugin? If so, I'd love to help out. If not, any
>> suggestions on where to get started?
> 
> Nope, it's on my *long* TODO list. I'm not even sure how to go about making it a CC plugin, I'd have to check how the JUnit plugin works. It might end up being an Ant task with some sort of listener/presentation component in CC.

I took a look into this & it appears that the best solution is to create a tool that outputs unit test result XML in the same format as JUnit. CC can just pick that up & display it as JUnit tests. For full CC integration, adding another builder (such as a DSSS builder) would be a good idea, though.

That said, it would be a cool idea to have a unified, configurable D "testing framework" that could run both DUnit tests and built-in unittest { } blocks with a plugable reporter interface for output (i.e. a JUnit-compatible XML for CC plugability, a pretty Web 2.0-ish page for Zen staring sessions, etc.)