Jason House Wrote:

> Sean Kelly wrote:
> 
> > == Quote from Jason House (jason.james.house@gmail.com)'s article
> >> I'd like to stay with the D style of unit tests.  Maybe I should check
> >> what
> >> enhancement requests are in there.  It'd be nice to be able to hook in a
> >> unit test handler...
> > 
> > Tango has one.  Look at "moduleUnitTester" here:
> > 
> > http://www.dsource.org/projects/tango/docs/current/tango.core.Runtime.html
> > 
> > 
> > Sean
> 
> See http://d.puremagic.com/issues/show_bug.cgi?id=1952 for the enhancement request.  Per unit test customization (such as a unit test name) is very useful to have.  I also don't see a way to tell the program to run all unit tests (and just report the individual passes/failures)

Here is a simple program that uses the above hook  - note that you need to link one of the stacktrace libs (jive on linux) to actually get stacktraces printed on exceptions, not just assert line numbers. Also, I think it will be unable to continue after a segfault.:

module myunittestrunner;

import tango.io.Stdout;
import tango.core.Runtime;

bool tangoUnitTester()
{
    uint count = 0;
    Stdout ("NOTE: This is still fairly rudimentary, and will only report the").newline;
    Stdout ("    first error per module.").newline;
    foreach ( m; ModuleInfo ) // moduleinfo array
    {
        if ( m.unitTest) {
            Stdout.format ("{}. Executing unittests in '{}' ", count, m.name);
            try {
               m.unitTest();
            }
            catch (Exception e) {
                Stdout(" - Unittest failed.").newline;
                Stdout.format("   File '{}', line '{}'.", e.file, e.line).newline;
                Stdout.format("     Message is : '{}'", e.msg).newline;
                if (e.info)
                    Stdout.format("     TraceInfo: {}", e.info.toString).newline;
                continue;
            }
            Stdout(" - Success.").newline;
            count++;
        }
    }
    return true;
}

static this() {
    Runtime.moduleUnitTester( &tangoUnitTester );
}

void main() {}

Lars Ivar Igesund Wrote:

> Jason House Wrote:
> 
> > Sean Kelly wrote:
> > 
> > > == Quote from Jason House (jason.james.house@gmail.com)'s article
> > >> I'd like to stay with the D style of unit tests.  Maybe I should check
> > >> what
> > >> enhancement requests are in there.  It'd be nice to be able to hook in a
> > >> unit test handler...
> > > 
> > > Tango has one.  Look at "moduleUnitTester" here:
> > > 
> > > http://www.dsource.org/projects/tango/docs/current/tango.core.Runtime.html
> > > 
> > > 
> > > Sean
> > 
> > See http://d.puremagic.com/issues/show_bug.cgi?id=1952 for the enhancement request.  Per unit test customization (such as a unit test name) is very useful to have.  I also don't see a way to tell the program to run all unit tests (and just report the individual passes/failures)
> 
> Here is a simple program that uses the above hook  - note that you need to link one of the stacktrace libs (jive on linux) to actually get stacktraces printed on exceptions, not just assert line numbers. Also, I think it will be unable to continue after a segfault.:

Hmmm...  There's some black magic in your sample.  What's "ModuleInfo"?  Where is it documented?  The code looks like it does not run unit tests one at a time, but I'll wait for real analysis of the code until I know what's going on.

== Quote from Jason House (jason.james.house@gmail.com)'s article
> Sean Kelly wrote:
> > == Quote from Jason House (jason.james.house@gmail.com)'s article
> >> I'd like to stay with the D style of unit tests.  Maybe I should check
> >> what
> >> enhancement requests are in there.  It'd be nice to be able to hook in a
> >> unit test handler...
> >
> > Tango has one.  Look at "moduleUnitTester" here:
> >
> > http://www.dsource.org/projects/tango/docs/current/tango.core.Runtime.html
> >
> See http://d.puremagic.com/issues/show_bug.cgi?id=1952 for the enhancement request.  Per unit test customization (such as a unit test name) is very useful to have.  I also don't see a way to tell the program to run all unit tests (and just report the individual passes/failures)

Yes, there is no way to name the built-in unit tests.  That would be a nice feature to have, so long as the name were optional.  I've been asking for unittest blocks to get a more function-like syntax anyway, to match invariant in D 2.0, so this provides a good reason for the syntax.  As for running all unit tests, simply have your unit tester perform each unit test in a try block/catch block.  If an exception is thrown then the test failed, otherwise it succeeded.


Sean

== Quote from Jason House (jason.james.house@gmail.com)'s article
> Lars Ivar Igesund Wrote:
> > Jason House Wrote:
> >
> > > Sean Kelly wrote:
> > >
> > > > == Quote from Jason House (jason.james.house@gmail.com)'s article
> > > >> I'd like to stay with the D style of unit tests.  Maybe I should check
> > > >> what
> > > >> enhancement requests are in there.  It'd be nice to be able to hook in a
> > > >> unit test handler...
> > > >
> > > > Tango has one.  Look at "moduleUnitTester" here:
> > > >
> > > > http://www.dsource.org/projects/tango/docs/current/tango.core.Runtime.html
> > > >
> > > >
> > > > Sean
> > >
> > > See http://d.puremagic.com/issues/show_bug.cgi?id=1952 for the enhancement request.  Per unit test customization (such as a unit test name) is very useful to have.  I also don't see a way to tell the program to run all unit tests (and just report the individual passes/failures)
> >
> > Here is a simple program that uses the above hook  - note that you need to link one of the stacktrace libs (jive on linux) to actually
get stacktraces printed on exceptions, not just assert line numbers. Also, I think it will be unable to continue after a segfault.:
> Hmmm...  There's some black magic in your sample.  What's "ModuleInfo"?  Where is it documented?

ModuleInfo is declared in the global object.di in Tango, and it isn't documented anywhere.  Sorry about that.  I need to look into
documenting what's in object.di.

> The code looks like it does not run unit tests one at a time, but I'll wait for real analysis of the code until I know what's going on.

Unfortunately, the compiler lumps all the unit tests in a particular module together.  I would prefer if they were split out into a per-
module list however, and this would have to happen for your enhancement request to go through.


Sean

Sean Kelly wrote:

> == Quote from Jason House (jason.james.house@gmail.com)'s article
>> Lars Ivar Igesund Wrote:
>> > Jason House Wrote:
>> >
>> > > Sean Kelly wrote:
>> > >
>> > > > == Quote from Jason House (jason.james.house@gmail.com)'s article
>> > > >> I'd like to stay with the D style of unit tests.  Maybe I should
>> > > >> check what
>> > > >> enhancement requests are in there.  It'd be nice to be able to
>> > > >> hook in a unit test handler...
>> > > >
>> > > > Tango has one.  Look at "moduleUnitTester" here:
>> > > >
>> > > >
http://www.dsource.org/projects/tango/docs/current/tango.core.Runtime.html
>> > > >
>> > > >
>> > > > Sean
>> > >
>> > > See http://d.puremagic.com/issues/show_bug.cgi?id=1952 for the
>> > > enhancement
>> > > request.  Per unit test customization (such as a unit test name) is
>> > > very
>> > > useful to have.  I also don't see a way to tell the program to run
>> > > all unit tests (and just report the individual passes/failures)
>> >
>> > Here is a simple program that uses the above hook  - note that you need to link one of the stacktrace libs (jive on linux) to actually
> get stacktraces printed on exceptions, not just assert line numbers. Also, I think it will be unable to continue after a segfault.:
>> Hmmm...  There's some black magic in your sample.  What's "ModuleInfo"? Where is it documented?
> 
> ModuleInfo is declared in the global object.di in Tango, and it isn't documented anywhere.  Sorry about that.  I need to look into documenting what's in object.di.
> 
>> The code looks like it does not run unit tests one at a time, but I'll wait for real analysis of the code until I know what's going on.
> 
> Unfortunately, the compiler lumps all the unit tests in a particular module together.  I would prefer if they were split out into a per- module list however, and this would have to happen for your enhancement request to go through.
> 
> 
> Sean

Please, please add the extra detail to the request!  How should the compiler do it?

Christopher Wright wrote:

> Jason House wrote:
>> Christopher Wright wrote:
>> 
>>> Jason House wrote:
>>>> I'm sensitive to per-unittest code-writing overhead, but already have some in my code. If DUnit offers comparable overhead, and enhanced functionality, I'll switch.
>>> DUnit uses test fixture classes, so it probably won't suit you at present. I'll look into a method with less overhead.
>> 
>> Hmmm... maybe :(
>> 
>> Would this be possible?
>> 
>> unittest{
>>   mixin DUnitTest!("My test name",
>>   {
>>     ...// my code
>>   });
>> }
> 
> Yes.


Cool :)


> I initially had some issues with alias parameters and delegates,
> but those seem to have disappeared. You wouldn't want to put that in a
> unittest block, though, since it's redundant and would give me, and by
> extension you, less control.

Huh?  Can you explain?  Would I have less control than DUnit currently provides or less control thant just a hand made unittest{...}?  In either case, how severe of a gap are you envisioning?  (So far, D unittest blocks are my only real unit testing experience.  Work will eventually have me using NUnit, but that hasn't happened yet)


> You'd also need to either version out your main function or call dunit_main from yours when running tests.

I'm totally ok with that!  One-time overhead at a program level isn't too bad.  Most programs that really use unit tests are moderate or large in size anyway.  It's the per-test overhead that would really irk me as I code away.

== Quote from Jason House (jason.james.house@gmail.com)'s article
> Sean Kelly wrote:
> > == Quote from Jason House (jason.james.house@gmail.com)'s article
> >> Lars Ivar Igesund Wrote:
> >> > Jason House Wrote:
> >> >
> >> > > Sean Kelly wrote:
> >> > >
> >> > > > == Quote from Jason House (jason.james.house@gmail.com)'s article
> >> > > >> I'd like to stay with the D style of unit tests.  Maybe I should
> >> > > >> check what
> >> > > >> enhancement requests are in there.  It'd be nice to be able to
> >> > > >> hook in a unit test handler...
> >> > > >
> >> > > > Tango has one.  Look at "moduleUnitTester" here:
> >> > > >
> >> > > >
> http://www.dsource.org/projects/tango/docs/current/tango.core.Runtime.html
> >> > > >
> >> > > >
> >> > > > Sean
> >> > >
> >> > > See http://d.puremagic.com/issues/show_bug.cgi?id=1952 for the
> >> > > enhancement
> >> > > request.  Per unit test customization (such as a unit test name) is
> >> > > very
> >> > > useful to have.  I also don't see a way to tell the program to run
> >> > > all unit tests (and just report the individual passes/failures)
> >> >
> >> > Here is a simple program that uses the above hook  - note that you need to link one of the stacktrace libs (jive on linux) to actually
> > get stacktraces printed on exceptions, not just assert line numbers. Also, I think it will be unable to continue after a segfault.:
> >> Hmmm...  There's some black magic in your sample.  What's "ModuleInfo"? Where is it documented?
> >
> > ModuleInfo is declared in the global object.di in Tango, and it isn't documented anywhere.  Sorry about that.  I need to look into documenting what's in object.di.
> >
> >> The code looks like it does not run unit tests one at a time, but I'll wait for real analysis of the code until I know what's going on.
> >
> > Unfortunately, the compiler lumps all the unit tests in a particular module together.  I would prefer if they were split out into a per- module list however, and this would have to happen for your enhancement request to go through.
> >
> >
> > Sean
> Please, please add the extra detail to the request!  How should the compiler do it?

I'd like ModuleInfo to get something vaguely like this:

alias void function() UnitTest;
struct NamedPair { char[] name; UnitTest test; }
NamedPair[] unitTests;

Then executing the unit tests for a module could be done on a per-test basis, with the names you suggested visible as well.  Of course, this means even more meta-data per module, but it would make unittest more flexible so...


Sean

Sean Kelly wrote:

> == Quote from Jason House (jason.james.house@gmail.com)'s article
>> Sean Kelly wrote:
>> > == Quote from Jason House (jason.james.house@gmail.com)'s article
>> >> Lars Ivar Igesund Wrote:
>> >> > Jason House Wrote:
>> >> >
>> >> > > Sean Kelly wrote:
>> >> > >
>> >> > > > == Quote from Jason House (jason.james.house@gmail.com)'s
>> >> > > > article
>> >> > > >> I'd like to stay with the D style of unit tests.  Maybe I
>> >> > > >> should check what
>> >> > > >> enhancement requests are in there.  It'd be nice to be able to
>> >> > > >> hook in a unit test handler...
>> >> > > >
>> >> > > > Tango has one.  Look at "moduleUnitTester" here:
>> >> > > >
>> >> > > >
>>
http://www.dsource.org/projects/tango/docs/current/tango.core.Runtime.html
>> >> > > >
>> >> > > >
>> >> > > > Sean
>> >> > >
>> >> > > See http://d.puremagic.com/issues/show_bug.cgi?id=1952 for the
>> >> > > enhancement
>> >> > > request.  Per unit test customization (such as a unit test name)
>> >> > > is very
>> >> > > useful to have.  I also don't see a way to tell the program to run
>> >> > > all unit tests (and just report the individual passes/failures)
>> >> >
>> >> > Here is a simple program that uses the above hook  - note that you need to link one of the stacktrace libs (jive on linux) to actually
>> > get stacktraces printed on exceptions, not just assert line numbers. Also, I think it will be unable to continue after a segfault.:
>> >> Hmmm...  There's some black magic in your sample.  What's "ModuleInfo"? Where is it documented?
>> >
>> > ModuleInfo is declared in the global object.di in Tango, and it isn't documented anywhere.  Sorry about that.  I need to look into documenting what's in object.di.
>> >
>> >> The code looks like it does not run unit tests one at a time, but I'll wait for real analysis of the code until I know what's going on.
>> >
>> > Unfortunately, the compiler lumps all the unit tests in a particular module together.  I would prefer if they were split out into a per- module list however, and this would have to happen for your enhancement request to go through.
>> >
>> >
>> > Sean
>> Please, please add the extra detail to the request!  How should the compiler do it?
> 
> I'd like ModuleInfo to get something vaguely like this:
> 
> alias void function() UnitTest;
> struct NamedPair { char[] name; UnitTest test; }
> NamedPair[] unitTests;
> 
> Then executing the unit tests for a module could be done on a per-test
> basis,
> with the names you suggested visible as well.  Of course, this means even
> more meta-data per module, but it would make unittest more flexible so...
> 
> 
> Sean

I was hoping those kind of thoughts would go into the feature request ;)

Jason House wrote:
> Christopher Wright wrote:
> 
>> Jason House wrote:
>>> Christopher Wright wrote:
>>>
>>>> Jason House wrote:
>>>>> I'm sensitive to per-unittest code-writing overhead, but already have
>>>>> some in my code. If DUnit offers comparable overhead, and enhanced
>>>>> functionality, I'll switch.
>>>> DUnit uses test fixture classes, so it probably won't suit you at
>>>> present. I'll look into a method with less overhead.
>>> Hmmm... maybe :(
>>>
>>> Would this be possible?
>>>
>>> unittest{
>>>   mixin DUnitTest!("My test name",
>>>   {
>>>     ...// my code
>>>   });
>>> }
>> Yes. 
> 
> 
> Cool :)
> 
> 
>> I initially had some issues with alias parameters and delegates, but those seem to have disappeared. You wouldn't want to put that in a
>> unittest block, though, since it's redundant and would give me, and by
>> extension you, less control.
> 
> Huh?  Can you explain?  Would I have less control than DUnit currently
> provides or less control thant just a hand made unittest{...}?  In either
> case, how severe of a gap are you envisioning?  (So far, D unittest blocks
> are my only real unit testing experience.  Work will eventually have me
> using NUnit, but that hasn't happened yet)

Currently DUnit does a lot of work with static constructors, and I'm not sure how they interact with unittest blocks.

If DUnit used unittest blocks instead of delegates and static constructors, that would be less control, since you'd only get two benefits: named tests and continuing testing after one failure. Well, three, actually; I could also do some aggregate reports and such, if you used a main() that I supplied.

If I add some feature later that lets you specify which tests to run via commandline options, those wouldn't be available with a unittest{} wrapper. Anything that affects the actual running of the tests and not the accounting.

>> You'd also need to either version out your main function or call
>> dunit_main from yours when running tests.
> 
> I'm totally ok with that!  One-time overhead at a program level isn't too
> bad.  Most programs that really use unit tests are moderate or large in
> size anyway.  It's the per-test overhead that would really irk me as I code
> away.

Of course, you can make editor macros to take most of the pain away.