https://github.com/linkrope/dunit/tree/v0.7.0

The xUnit testing framework for D is used in production for one year now.

The latest changes are:
- added XML test reporting in JUnitReport format
- dub support
- changed @Ignore to @Ignore("reason to skip the test")
- added assertOp together with corresponding aliases
- added assertEmpty and assertNotEmpty
- extended assertArrayEquals to associative arrays

The XML reporting is currently used for a good integration with the CI tool Jenkins.

While the dunit/framework is best suited for "testing in the large", dunit/assertion can be used on its own in simple unittest blocks for improved failure messages:

unittest
{
    import dunit.assertion;

    assertArrayEquals([1: "foo", 2: "bar"], [1: "foo", 2: "baz"]);
}

You will get the following message:
array mismatch at key 2; expected: <ba<r>> but was: <ba<z>>

Even the difference between the string values "bar" and "baz" is highlighted within <...>.
(Still, I would wish for something like Python's ndiff for multi-line string values.)

On Sunday, 29 September 2013 at 21:06:16 UTC, linkrope wrote:
> https://github.com/linkrope/dunit/tree/v0.7.0
>
> The xUnit testing framework for D is used in production for one year now.

Guys, we have at least 5 (!) different unit test projects!
Can you cooperate your efforts to create one, but wonderful?

On Monday, 21 October 2013 at 11:09:25 UTC, ilya-stromberg wrote:
> Guys, we have at least 5 (!) different unit test projects!
> Can you cooperate your efforts to create one, but wonderful?

...and add it to Phobos review queue ;)

On Monday, October 21, 2013 13:48:16 Dicebot wrote:
> On Monday, 21 October 2013 at 11:09:25 UTC, ilya-stromberg wrote:
> > Guys, we have at least 5 (!) different unit test projects! Can you cooperate your efforts to create one, but wonderful?
> 
> ...and add it to Phobos review queue ;)

I confess that I don't understand why anyone is creating any unit test projects for D, and I'd likely vote against any attempt to add such a thing to Phobos. D has built in unit testing functionality, and it works great. Maybe some additional assert-like functions could be useful (similar to assertThrown or assertNotThrown), but we really don't need much beyond what the language provides.

- Jonathan m Davis

On Monday, 21 October 2013 at 11:58:14 UTC, Jonathan M Davis wrote:
> I confess that I don't understand why anyone is creating any unit test
> projects for D, and I'd likely vote against any attempt to add such a thing to
> Phobos. D has built in unit testing functionality, and it works great. Maybe
> some additional assert-like functions could be useful (similar to assertThrown
> or assertNotThrown), but we really don't need much beyond what the language
> provides.

Sorry but it feels like you didn't really investigate the topic. D built-in unit-test facilities are a basic and extendable framework. Many notable unittest libraries work on top of it (using custom assert helpers and/or __traits(getUnittests)) being at the same time perfectly compatible with raw `dmd -unittest -main -run file.d`.

At the very same time built-in reporting facilities are very limited and almost any bigger project will want some extension. Assuming full compatibility with built-in ones and zero external dependencies I don't see why this can't go to Phobos.

On Monday, 21 October 2013 at 11:58:14 UTC, Jonathan M Davis wrote:
> On Monday, October 21, 2013 13:48:16 Dicebot wrote:
>> On Monday, 21 October 2013 at 11:09:25 UTC, ilya-stromberg wrote:
>> > Guys, we have at least 5 (!) different unit test projects!
>> > Can you cooperate your efforts to create one, but wonderful?
>> 
>> ...and add it to Phobos review queue ;)
>
> I confess that I don't understand why anyone is creating any unit test
> projects for D, and I'd likely vote against any attempt to add such a thing to
> Phobos. D has built in unit testing functionality, and it works great. Maybe
> some additional assert-like functions could be useful (similar to assertThrown
> or assertNotThrown), but we really don't need much beyond what the language
> provides.
>
> - Jonathan m Davis

Yes, you are right if we talk about SIMPLE unit tests.
For more complex cases like integration tests you have A LOT of problems.
It isn't only my opinion - for example, look at the "Higgs, an Experimental JIT Compiler written in D" Slides from last dconf (page 57):
http://dconf.org/2013/talks/chevalier_boisvert.html

On Monday, October 21, 2013 14:10:13 Dicebot wrote:
> On Monday, 21 October 2013 at 11:58:14 UTC, Jonathan M Davis
> 
> wrote:
> > I confess that I don't understand why anyone is creating any
> > unit test
> > projects for D, and I'd likely vote against any attempt to add
> > such a thing to
> > Phobos. D has built in unit testing functionality, and it works
> > great. Maybe
> > some additional assert-like functions could be useful (similar
> > to assertThrown
> > or assertNotThrown), but we really don't need much beyond what
> > the language
> > provides.
> 
> Sorry but it feels like you didn't really investigate the topic. D built-in unit-test facilities are a basic and extendable framework. Many notable unittest libraries work on top of it (using custom assert helpers and/or __traits(getUnittests)) being at the same time perfectly compatible with raw `dmd -unittest -main -run file.d`.
> 
> At the very same time built-in reporting facilities are very limited and almost any bigger project will want some extension. Assuming full compatibility with built-in ones and zero external dependencies I don't see why this can't go to Phobos.

I know that you can extend the built-in facilities by overriding how assert works and the like. I also see no reason to do so. IIRC, such facilities were even removed at one point, because no one was using them. They were readded after someone complained about wanting them, but every time I see anyone talking about doing anything with those, it seems like overkill to me. And since it's frequently for nonsense like making it so that the tests continue after an assertion fails (which is outright bad practice IMHO), I have a very low opinion of attempts to override the built-in assert for unit tests.

If someone wants to present something to the review queue that's related to unit tests, that's fine. But I really don't see any problem with the built-in unit tests facilities and would expect to be against any such submission, particularly since every time I've delved into any discussions on them, what folks are doing with them has seemed like a really bad idea to me. But I guess that we'll just have to wait and see what gets submitted for review, if anything.

- Jonathan M Davis

On Mon, 2013-10-21 at 04:58 -0700, Jonathan M Davis wrote: […]
> I confess that I don't understand why anyone is creating any unit test projects for D, and I'd likely vote against any attempt to add such a thing to Phobos. D has built in unit testing functionality, and it works great. Maybe some additional assert-like functions could be useful (similar to assertThrown or assertNotThrown), but we really don't need much beyond what the language provides.

The focus on unit testing is a problem for me, unit testing is but ⅓ of the testing needed. There is integration testing and system testing to consider as well. Not to mention the different features needed of BDD rather than TDD.

I appreciate that many people on this list think of anything to do with
the JVM beneath contempt ;-) but the JUnit → TestNG → Spock and Cucumber
journey is worth considering as a lesson in why not to get too focused
on programmer generation of units. Python has the same lessons with the
way unittest (aka PyUnit) and py.test are used. It is all about a
testing framework supporting unit test, integration test, and system
test usage.

See Catch for the current C++ front-runner. https://github.com/philsquared/Catch

-- 
Russel. ============================================================================= Dr Russel Winder      t: +44 20 7585 2200   voip: sip:russel.winder@ekiga.net 41 Buckmaster Road    m: +44 7770 465 077   xmpp: russel@winder.org.uk London SW11 1EN, UK   w: www.russel.org.uk  skype: russel_winder

On Monday, 21 October 2013 at 13:45:06 UTC, Russel Winder wrote:
> It is all about a
> testing framework supporting unit test, integration test, and system
> test usage.

Somewhat off topic, but out of curiosity: How do you distinguish between integration and system testing?

The descriptions I found usually sound like system testing is a special case of integration testing, where you simply integrate all components.

On Monday, 21 October 2013 at 12:24:18 UTC, Jonathan M Davis wrote:
> I know that you can extend the built-in facilities by overriding how assert
> works and the like.

Overriding assert is dangerous because changes behavior of program itself and lacks context data. Own test runner implemented using __traits(getUnittests) is perfectly safe.

> And
> since it's frequently for nonsense like making it so that the tests continue
> after an assertion fails (which is outright bad practice IMHO),

It is absolutely necessary feature in any big project of you want to keep reasonable edit-build cycle times. Tests are always hierarchical naturally, there is no reason to stop the world if completely unrelated ones fail.

> I have a very
> low opinion of attempts to override the built-in assert for unit tests.

You are right here but it is not needed anymore.

> But I really don't see any problem with the built-in
> unit tests facilities and would expect to be against any such submission,

There is nothing wrong with built-in ones, just some higher-level tools on top of them are lacking.

> But I guess
> that we'll just have to wait and see what gets submitted for review, if
> anything.

Exactly, lets argue in time ;)