On 01/24/2011 10:03 PM, Andrei Alexandrescu wrote:
>> One other thing, using writefln is considered bad form in unit tests
>> (you want *no* output if the unit test works). But many examples might
>> want to demonstrate how e.g. an object interacts with writefln. Any
>> suggestions? The assert line above is not very pretty for example...
>
> Yah, that is an issue. For examples that do non-unittesty stuff (e.g. writeln,
> use sockets etc.) we can still use the old-style documentation. By the way, for
> all examples that don't explicitly describe writeln, we shouldn't use writeln
> anyway. Instead, we should use assert to clearly describe what happens:
>
> // BAD: example of concatenation
> string s1 = "Hello, ";
> string s2 = "world!";
> writeln(s1 ~ s2);  // writes Hello, world!
>
>
> // GOOD: example of concatenation
> string s1 = "Hello, ";
> string s2 = "world!";
> assert(s1 ~ s2 == "Hello, world!"); // no need for comment

This is a very good example.
It seems that in the D style, and in Phobos use, unittests are only considered for maintenance. (What is sometimes called regression test suites: http://en.wikipedia.org/wiki/Regression_testing).
Many programmers instead use test funcs to provide feedback on how pieces of code work during development. Even when it does not bug. I for one constantly run tests. Using the example above, my initial test func would be eg:

string s1 = "Hello, ";
string s2 = "world!";
writefln("%s ~ %s --> %s", s1,s2, s1 ~ s2);

This goes well with "exploratory programming". To setup things correctly, I need much information on all aspects of the part of code I'm currently dealing with, including object internal state, for instance (toString is vital, often defined before anything else). Data on what works fine also helps in fixing what doesn't.
Then, when all work fine, I would comment out writelfn and replace it with an assert. But I keep it so that in case of later bug introduced by changes, code is still there to provide all necessary information to diagnose. (Hope I'm clear.)
Such verbose test suite, when done right, is also a 'show' of all kinds of functionality a module is able to provide, at best with self-explaining output.

Now, for sure assert and write* are somewhat redondant for testing. What I dream of is a testing func similar to assert() but that writes info out even on success; then a silent mode would switch that off and write only on failure. Would be easier if this func takes 2 params, eg:
	check(expression, expectation);
writes out on success:
	expression --> to!string(expectation)
writes out on failure:
****** test error **************************
expression	: (expression)
expectation	: to!string(expectation)
outcome		: to!string(outcome)
********************************************
This would also avoid having a separate testing func for cases where expectation is an exception.


Denis
-- 
_________________
vita es estrany
spir.wikidot.com

On Mon, 24 Jan 2011 17:32:12 -0500, Andrew Wiley <debio264@gmail.com> wrote:
>
> Here's another approach:
> When you think about it, what we're after is tagging unit tests as code
> examples, so why not do exactly that with annotations?
>
> @example(Foo.bar)
> unittest {
> //...
> }
>
> It means there are no new keywords to worry about, and this feels more like
> a job for metadata than a language feature to me. We'd probably have to
> formalize adding parameters to annotations, but that's pretty much
> inevitable if they're ever going to reach a useful state.

This changes the compiled language.  A comment is safe to ignore, so it's not part of the compiled language, similar to how ddoc comments are not part of the compiled code.  It fits perfectly with a ddoc comment.

Using a ddoc comment to trigger an example in ddoc feels perfectly correct.  What I'd like to do is make it less repetitive.

-Steve

Jonathan M Davis wrote:
> In case you didn't know, I have a set of unit test helper functions which have been being reviewed for possible inclusion in phobos. Here's an update.
> 
> Most recent code: http://is.gd/F1OHat
> 
> Okay. I took the previous suggestions into consideration and adjusted the code a bit more. However, most of the changes are to the documentation (though there are some changes to the code). Some of the code duplication was removed, and the way that some of the assertPred functions' errors are formatted has been altered so that values line up vertically, making them easier to compare. The big change is the docs though. There's now a fake version of assertPred at the top with an overall description for assertPred followed by the individual versions with as little documentation as seemed appropriate while still getting all of the necessary information across. A couple of the functions still have irritatingly long example sections, but anything less wouldn't get the functionality across.
> 
> In any case. Here's the updated code. Review away. Andrei set the vote deadline for February 7th, at which point, if it passes majority vote, then it will go into Phobos. The number of functions is small enough now (thanks to having consolidated most of them into the fantastically versatile assertPred) that it looks like it will likely go in std.exception if the vote passes rather than becoming a new module. So, the std.unittests title has now become a bit of a misnomer, but that's what I've been calling it, so it seemed appropriate to continue to label it that way in the thread's title.

What do you think about renaming
collectExceptionMsg
to
collectThrowableMsg?
Since it handles both errors and exceptions.
One could make it handle only exceptions and then it would be a perfect
fit for std.exception. Does one need really need to collect error
messages? If so then collectException in std.exception should handle
errors as well and should be called collectThrowable.

Jens