Hi all,

I've released a new version of dunit, r247. (I have finally decided on
subversion revision numbers as a simple, unambiguous versioning scheme).
It's recommended for immediate use and is certified alpha-quality software.

Documentation and Downloads
---------------------------
Documentation is included in the .zip file and also on the wiki.
Docs: http://dsource.org/projects/dmocks/wiki/DUnit
Download: http://dsource.org/projects/dmocks/browser/downloads/dunit.r247.zip?format=raw

Assertions
----------
The major production-ready element of dunit is a greatly-tweaked
assertion system. There are minor API changes; in general, you can convert:
	expect(value).constraint;
to:
	expect(value, constraint);

Additionally, you can combine constraints using | or &:
expect(6, equals(3) | greaterThan(5));

Additionally, it is much more reasonable to add additional messages to
an assertion:
expect(foo(), lessThan(5), "foo is being mean again!");

The assertions system is now extensible; deriving from
dunit.assertions.model.Constraint will allow you to use your constraint
as if it were a regular, builtin constraint.

Parameterized tests
-------------------
A parameterized test is one that uses data generated externally. Dunit
now supports parameterized testing. The builtin parameterization systems
are combinatorial and sequential, with test data generators for random
and sequential integers and floating point values.

Christopher Wright:

Having other testing frameworks/tools for D is good. There are many kinds of testing, and the built-in one isn't supposed to implement them all.

Regarding the issues of unit testing with unittest{}, I think the built-in unittesting has to be improved, to removed some of such issues. I am not looking for an universal and perfect built-in unittesting, and I think the built-in unittesting has to be kept simple, but the following things have to be fixed, maybe Walter will eventually understand why they are important:
- Unittests are not labeled.
- There is no output that specifically indicates that the tests were run.
- A failing test will prevent any other tests from running.
- There is no indication of which test failed, if any.
Such things are bare-bone functionality for any unit testing system.
And I'd like to add a way to unittest at compile time too, to test types, templates, etc. (Until few weeks ago I didn't know any way at all to do this, then someone has given me a hint).


What's the advantage of:
expect(foo(5), equals(3) | greaterThan(5));
Compared to:
expect(foo(5) == 3 | foo(5) > 5);
Or:
auto aux5 = foo(5);
expect(aux5 == 3 | aux5 > 5);
?

>If you write Dunit tests in separate modules, while your production code doesn't include dunit, you cannot test private methods. Dunit encourages the practice of separating tests and modules.<

For me it's often better to keep tests very close to the things they test. It helps me spot and fix bugs faster, to avoid jumping across files, and when I quickly move a block of code (function, class, template, etc) when I reorganize the code it is less likely for me to lose its tests along the way. I think tests are a part of a function/class/template, just like its ddocs.

Bye,
bearophile

"bearophile" <bearophileHUGS@lycos.com> wrote in message news:gptoms$bmm$1@digitalmars.com...
> Christopher Wright:
>
> Having other testing frameworks/tools for D is good. There are many kinds of testing, and the built-in one isn't supposed to implement them all.
>
> Regarding the issues of unit testing with unittest{}, I think the built-in
> unittesting has to be improved, to removed some of such issues. I am not
> looking for an universal and perfect built-in unittesting, and I think the
> built-in unittesting has to be kept simple, but the following things have
> to be fixed, maybe Walter will eventually understand why they are
> important:
> - Unittests are not labeled.
> - There is no output that specifically indicates that the tests were run.
> - A failing test will prevent any other tests from running.
> - There is no indication of which test failed, if any.
> Such things are bare-bone functionality for any unit testing system.
> And I'd like to add a way to unittest at compile time too, to test types,
> templates, etc. (Until few weeks ago I didn't know any way at all to do
> this, then someone has given me a hint).
>
>
> What's the advantage of:
> expect(foo(5), equals(3) | greaterThan(5));
> Compared to:
> expect(foo(5) == 3 | foo(5) > 5);
> Or:
> auto aux5 = foo(5);
> expect(aux5 == 3 | aux5 > 5);
> ?
>
>>If you write Dunit tests in separate modules, while your production code doesn't include dunit, you cannot test private methods. Dunit encourages the practice of separating tests and modules.<
>
> For me it's often better to keep tests very close to the things they test. It helps me spot and fix bugs faster, to avoid jumping across files, and when I quickly move a block of code (function, class, template, etc) when I reorganize the code it is less likely for me to lose its tests along the way. I think tests are a part of a function/class/template, just like its ddocs.
>

Seconded, with the additional reason that I've found unittests to be an incredibly useful form of documentation-by-example. There have been numerous occasions where I've read documentation that still left me with questions, so I opened up the actual source to see what was going on, saw the unittests, and thought "A-ha! So that's how it's used! And I *know* it's correct because it's part of the unittest."

On Thu, 19 Mar 2009 11:34:52 -0400, bearophile <bearophileHUGS@lycos.com> wrote:

>Christopher Wright:
>
>Having other testing frameworks/tools for D is good. There are many kinds of testing, and the built-in one isn't supposed to implement them all.
>
>Regarding the issues of unit testing with unittest{}, I think the built-in unittesting has to be improved, to removed some of such issues. I am not looking for an universal and perfect built-in unittesting, and I think the built-in unittesting has to be kept simple, but the following things have to be fixed, maybe Walter will eventually understand why they are important:
>- Unittests are not labeled.
>- There is no output that specifically indicates that the tests were run.
>- A failing test will prevent any other tests from running.
>- There is no indication of which test failed, if any.
>Such things are bare-bone functionality for any unit testing system.
>And I'd like to add a way to unittest at compile time too, to test types, templates, etc. (Until few weeks ago I didn't know any way at all to do this, then someone has given me a hint).
>
I think that nestable named unittest would be nice. I'll raise it as an enhancement request.

unittest ("XML") {
	unittest("elements") {
	    assert(isValidXml("<aaa />"));
	    assert(isValidXml("<aaa/>"));
	    assert(isValidXml("<aaa></aaa>"));
            ...
	}
	unittest("attributes") {
	    assert(isValidXml("<aaa abc="\x\"/>"));
	    assert(isValidXml("<aaa abc=\"x\" def=\"y\"/>"));
            ...
	}
	unittest("encoding") {
	    assert(encode("hello") is "hello");
	    assert(encode("a > b") == "a &gt; b");
	    ...
	}
}

Gide

On Thu, 19 Mar 2009 18:26:32 +0000, Gide Nwawudu <gide@btinternet.com> wrote:

>On Thu, 19 Mar 2009 11:34:52 -0400, bearophile <bearophileHUGS@lycos.com> wrote:
>
>>Christopher Wright:
>>
>>Having other testing frameworks/tools for D is good. There are many kinds of testing, and the built-in one isn't supposed to implement them all.
>>
>>Regarding the issues of unit testing with unittest{}, I think the built-in unittesting has to be improved, to removed some of such issues. I am not looking for an universal and perfect built-in unittesting, and I think the built-in unittesting has to be kept simple, but the following things have to be fixed, maybe Walter will eventually understand why they are important:
>>- Unittests are not labeled.
>>- There is no output that specifically indicates that the tests were run.
>>- A failing test will prevent any other tests from running.
>>- There is no indication of which test failed, if any.
>>Such things are bare-bone functionality for any unit testing system.
>>And I'd like to add a way to unittest at compile time too, to test types, templates, etc. (Until few weeks ago I didn't know any way at all to do this, then someone has given me a hint).
>>
>I think that nestable named unittest would be nice. I'll raise it as an enhancement request.
>

Added http://d.puremagic.com/issues/show_bug.cgi?id=2749

Gide

bearophile wrote:
> Christopher Wright:
> 
> Having other testing frameworks/tools for D is good. There are many kinds of testing, and the built-in one isn't supposed to implement them all.
> 
> Regarding the issues of unit testing with unittest{}, I think the built-in unittesting has to be improved, to removed some of such issues. I am not looking for an universal and perfect built-in unittesting, and I think the built-in unittesting has to be kept simple, but the following things have to be fixed, maybe Walter will eventually understand why they are important:
> - Unittests are not labeled.
> - There is no output that specifically indicates that the tests were run.
> - A failing test will prevent any other tests from running. - There is no indication of which test failed, if any.
> Such things are bare-bone functionality for any unit testing system.
> And I'd like to add a way to unittest at compile time too, to test types, templates, etc. (Until few weeks ago I didn't know any way at all to do this, then someone has given me a hint).

What are you using for this? __traits(compiles) works for d2, to an extent, and for d1, is (typeof(expression)). But for templates that have to be mixed into some context, that's more tricky.

> What's the advantage of:
> expect(foo(5), equals(3) | greaterThan(5));
> Compared to:
> expect(foo(5) == 3 | foo(5) > 5);

What error message should that give?

The former gives:
Expected: equal to 3 or greater than 5
But was: <whatever value foo(5) returned>

The latter gives:
Assertion error

>> If you write Dunit tests in separate modules, while your production code doesn't include dunit, you cannot test private methods. Dunit encourages the practice of separating tests and modules.<
> 
> For me it's often better to keep tests very close to the things they test. It helps me spot and fix bugs faster, to avoid jumping across files, and when I quickly move a block of code (function, class, template, etc) when I reorganize the code it is less likely for me to lose its tests along the way. I think tests are a part of a function/class/template, just like its ddocs.

I believe that there is a benefit to keeping the tests close to the tested code. I have not noticed a significant lack, however, when using junit or nunit.

> Bye,
> bearophile

bearophile wrote:
> For me it's often better to keep tests very close to the things they test. It helps me spot and fix bugs faster, to avoid jumping across files, and when I quickly move a block of code (function, class, template, etc) when I reorganize the code it is less likely for me to lose its tests along the way. I think tests are a part of a function/class/template, just like its ddocs.

Well, you can always put the Dunit tests in the same module; this was just a suggestion.

"Christopher Wright" <dhasenan@gmail.com> wrote in message news:gpuibc$1nrv$1@digitalmars.com...
> bearophile wrote:
>> What's the advantage of:
>> expect(foo(5), equals(3) | greaterThan(5));
>> Compared to:
>> expect(foo(5) == 3 | foo(5) > 5);
>
> What error message should that give?
>
> The former gives:
> Expected: equal to 3 or greater than 5
> But was: <whatever value foo(5) returned>
>
> The latter gives:
> Assertion error
>

The nice thing about D is how often it lets you, as I prefer to word it, "Eat your cake and then still have it."

Attached is the assert-alternative that I use (D1/Tango). It's used like this:

---------------------
module nonFatalAssertTest.main;

import semitwist.util.nonFatalAssert;

void main()
{
    FatalizeAsserts();
    // Main program code here
}

unittest
{
    int foo = 2;

    // *REALLY* need a way for a template to automatically get
    // the file/line of instantiation.
    // Improvement to mixin syntax would also be nice
    // Also, my editor doesn't know that backticks indicate a string,
    // so it's still properly highlighted as code :)
    mixin(NonFatalAssert!(__LINE__, __FILE__, `foo == 3 || foo > 5`, "foo is
bad"));
    mixin(NonFatalAssert!(__LINE__, __FILE__, `2 + 2 == 4`, "Basic
arithmetic"));
    mixin(NonFatalAssert!(__LINE__, __FILE__, `false`));
}
---------------------

Output of that is:

---------------------
nonFatalAssertTest\main.d(16): Assert Failure (foo == 3 || foo > 5): foo is
bad
nonFatalAssertTest\main.d(18): Assert Failure (false)
tango.core.Exception.AssertException@semitwist\util\nonFatalAssert.d(62): 2
Assert Failures
---------------------

Granted, this doesn't currently output foo's actual value, but it probably wouldn't be too hard to modify it to do so via std.algorithm-style trickery.

Christopher Wright wrote:
> bearophile wrote:
>> What's the advantage of:
>> expect(foo(5), equals(3) | greaterThan(5));
>> Compared to:
>> expect(foo(5) == 3 | foo(5) > 5);
> 
> What error message should that give?
> 
> The former gives:
> Expected: equal to 3 or greater than 5
> But was: <whatever value foo(5) returned>
> 
> The latter gives:
> Assertion error

What about this syntax?

check!("% == 3 || % == 5")(foo(5));

This could display:

Expected: (% == 3 || % == 5) for <value of foo(5)>.

Then you don't have to dick about writing classes when you want to test something.  Rule one of a good API: get out of my way. :D

  -- Daniel

"Nick Sabalausky" <a@a.a> wrote in message news:gpunvp$20u8$1@digitalmars.com...
>
> Granted, this doesn't currently output foo's actual value, but it probably wouldn't be too hard to modify it to do so via std.algorithm-style trickery.
>

Done <g>. New alternate-assert module is attached.

Sample usage:

--------------
module nonFatalAssertTest.main;

import semitwist.util.nonFatalAssert;

void main()
{
    FatalizeAsserts();
    // Main program code here
}

unittest
{
    int foo = 2;
    char[] bar = "hello";

    // *REALLY* need a way for a template to automatically get
    // the file/line of instantiation.
    // Improvement to mixin syntax would also be nice
    // Also, my editor doesn't know that backticks indicate a string,
    // so it's still properly highlighted as code :)
    mixin(NonFatalAssert!(__LINE__, __FILE__, `foo == 3 || foo > 5`, "foo is
bad"));
    mixin(NonFatalAssert!(__LINE__, __FILE__, `2 + 2 == 4`, "Basic
arithmetic"));
    mixin(NonFatalAssert!(__LINE__, __FILE__, `false`));

    mixin(NonFatalEnsure!(__LINE__, __FILE__, foo, `_ == 3 || _ > 5`,
"ensure foo failed"));
    mixin(NonFatalEnsure!(__LINE__, __FILE__, foo, `_ > 0`));
    mixin(NonFatalEnsure!(__LINE__, __FILE__, bar, `_ == "hola"`));
}
--------------

Output:

--------------
nonFatalAssertTest\main.d(21): Assert Failed (foo == 3 || foo > 5): foo is
bad
nonFatalAssertTest\main.d(23): Assert Failed (false)
nonFatalAssertTest\main.d(25): Ensure Failed: ensure foo failed
Value 'foo':
Expected: _ == 3 || _ > 5
Actual: 2
nonFatalAssertTest\main.d(27): Ensure Failed
Value 'bar':
Expected: _ == "hola"
Actual: hello
tango.core.Exception.AssertException@semitwist\util\nonFatalAssert.d(86): 4
Assert Failures
--------------

Sorry if this comes across as thread hijacking, that wasn't my intent. My main point is that in D, we don't have to choose between having detailed assert messages and having natural syntax for assert's conditional. We can have both.