Hi,

I've been a lurker on the D newsgroup for a while now and I'm a definite  D devotee - after having done several months of C, I appreciate the features of D greatly. I _finally_ have something to say... It's a feature suggestion to throw to the wolves:

Would it be possible/desirable/moronic to add a command line option to get DMD to generate unit tests as a seperate executable to the main code?

There are situations where it is desirable to run unit tests seperately and not with the main program. I'm not very good at explaining things, so I'll give an example from my experience:

I've done a little work in a large-ish C project (30ish developers, similar number of testers) where code is built on a machine that is not capable of running the program produced (due to software setup/hardware present) even though both the build and target machine use Linux and x86 hardware.

In this case, unit tests were built and run on the build machine after the main program had successfully built since none of the unit tests relied on the specific hardware of the target machines.

The build was not 'released' to the testers unless the unit tests had completed successfully. Having the unit tests seperate from the program let the unit tests be run on the build machine without tying up test machines.

Also in this case once the unit tests (possibly time-consuming and complex) have been successful once for each build, there is no additional benefit from running them on program start - in fact it could be harmful as the program ran as part of a cluster and made guarantees about time taken to start up/restart the program.

As far as I can see generating a unit test only executable would be as simple as generating a main function which only ran 	
_moduleCtor();
_moduleUnitTests();

and _not_ run the user-supplied main

Hopefully someone can get my meaning from these ramblings - I think seperating unit test code from program could be a useful aid to large scale projects or programs which rely on specific hardware.

Alex Stevenson

Alex Stevenson wrote:
> Hi,
> 
> I've been a lurker on the D newsgroup for a while now and I'm a definite  D devotee - after having done several months of C, I appreciate the features of D greatly. I _finally_ have something to say... It's a feature suggestion to throw to the wolves:
> 
> Would it be possible/desirable/moronic to add a command line option to get DMD to generate unit tests as a seperate executable to the main code?
> 
> There are situations where it is desirable to run unit tests seperately and not with the main program.
> 
> As far as I can see generating a unit test only executable would be as simple as generating a main function which only ran    _moduleCtor();
> _moduleUnitTests();
> 
> and _not_ run the user-supplied main

This would be a huge timesaver for just about any project using test-driven development.

Two other things that would be exceptionally useful, especially in combination with this, would be the ability to name unit tests, and having the resulting executable run every single test instead of stopping at the first failure.

 -- andy

In article <cu88hv$2uq6$1@digitaldaemon.com>, Alex Stevenson says...
>
>Would it be possible/desirable/moronic to add a command line option to get DMD to generate unit tests as a seperate executable to the main code?

It's not a bad suggestion.  In fact, similar things have been brought up in the past on this group.  However, it has also been noted that DMD is really designed to be at the bottom of a tool-chain, hence why a behavior like this may not be justifiable for the compiler itself.

my $0.02:

In your particular build environment, it looks like all you need are some custom bits added to your makefile and/or use a shell script.  Just don't use the 'unittest{}' block and compile your application against a unit-test oriented 'int main()' as a separate build step.  With a little clever coding, you can use this to catch exceptions so you can run a report of which tests failed, and which passed; its much more flexible than a basic 'unittest{}' this way.

The shell-script option would also allow you to branch based on the result of 'unittest.exe' (or whatever you want to call it), so you can skip the call to compile the target application.  If the test passes, then the script can then build your main executable and run it (if need be).

If shell scripting's not your thing, you could just write your builder in D.

> // builder program (untested, w/o useful error messages)
> import std.process;
> static const int SUCCESS = 0; //errors are usually non-zero
> void main(){
>     int result;
>     if(std.process.system("make unittest") != SUCCESS) return;
>     if(std.process.system("unittest.exe") != SUCCESS) return;
>     if(std.process.system("make application") != SUCCESS) return;
>     std.process.system("application.exe");
> }

- EricAnderton at yahoo

> Hi,
>
> I've been a lurker on the D newsgroup for a while now and I'm a definite D devotee - after having done several months of C, I appreciate the features of D greatly. I _finally_ have something to say... It's a feature suggestion to throw to the wolves:
>
> Would it be possible/desirable/moronic to add a command line option to get DMD to generate unit tests as a seperate executable to the main code?
>
> There are situations where it is desirable to run unit tests seperately and not with the main program. I'm not very good at explaining things, so I'll give an example from my experience:
>
> I've done a little work in a large-ish C project (30ish developers, similar number of testers) where code is built on a machine that is not capable of running the program produced (due to software setup/hardware present) even though both the build and target machine use Linux and x86 hardware.
>
> In this case, unit tests were built and run on the build machine after the main program had successfully built since none of the unit tests relied on the specific hardware of the target machines.
>
> The build was not 'released' to the testers unless the unit tests had completed successfully. Having the unit tests seperate from the program let the unit tests be run on the build machine without tying up test machines.
>
> Also in this case once the unit tests (possibly time-consuming and complex) have been successful once for each build, there is no additional benefit from running them on program start - in fact it could be harmful as the program ran as part of a cluster and made guarantees about time taken to start up/restart the program.
>
> As far as I can see generating a unit test only executable would be as
> simple as generating a main function which only ran _moduleCtor();
> _moduleUnitTests();
>
> and _not_ run the user-supplied main
>
> Hopefully someone can get my meaning from these ramblings - I think seperating unit test code from program could be a useful aid to large scale projects or programs which rely on specific hardware.

Seems quite sensible to me. STLSoft has unittests in all (well, that's the theory) components, which are enabled by definition of the STLSOFT_UNITTEST preprocessor symbol. One needs to include any/all STLSoft headers in a main program and compile/link to a (ridiculously) simple component that iterates through them and executes them.

There's no automatic provision for unit-tests to be run as part of programs using STLSoft code, and while people _could_ do that if they wished, it seems a very odd thing to want to do, since, as you say, once they've run successfully, they will always do so. (Notwithstanding cosmic rays, overclocking and quantum fluctuations, of course. <g>)

IMO, unit-tests are quite different (at least in terms of their purpose) from contracts. Your post is very useful, since I don't think anyone's pointed this out before, yet it's one of those things that seem both obvious and important once raised.

It would be pretty simple to modify internal\dmain2.d for your application to just run the unit tests and not call main().

On Tue, 8 Feb 2005 12:55:42 -0800, Walter <newshound@digitalmars.com> wrote:

> It would be pretty simple to modify internal\dmain2.d for your application
> to just run the unit tests and not call main().
>
>

Yes, looking at the code it seems like a fairly trivial matter, but is there any advantage to running unit tests on every program start? Once they've passed once the binary will always pass them unless you're using them for purposes of a non-unit test nature (like testing for the original Pentium FP problems). Unless there's something I'm overlooking, I can't think of a case where you'd want to run integrated unit tests and program.


-- 
Using Opera's revolutionary e-mail client: http://www.opera.com/m2/

"Alex Stevenson" <ans104@cs.york.ac.uk> wrote in message news:opslwfn3ir08qma6@mjolnir.spamnet.local...
> On Tue, 8 Feb 2005 12:55:42 -0800, Walter <newshound@digitalmars.com> wrote:
>
> > It would be pretty simple to modify internal\dmain2.d for your
> > application
> > to just run the unit tests and not call main().
> >
> >
>
> Yes, looking at the code it seems like a fairly trivial matter, but is there any advantage to running unit tests on every program start? Once they've passed once the binary will always pass them unless you're using them for purposes of a non-unit test nature (like testing for the original Pentium FP problems). Unless there's something I'm overlooking, I can't think of a case where you'd want to run integrated unit tests and program.

They're not really meant for including in a released binary. Just an easy way to get them run.

On Thu, 10 Feb 2005 00:13:24 -0800, "Walter" <newshound@digitalmars.com> wrote:

>They're not really meant for including in a released binary. Just an easy way to get them run.

I'm not too familiar with the D unit tests, so please ignore my suggestion if it's already possible.

But when I do my java development, I always put my unit tests into a different source tree than my application code. The test_src directory has a package structure that mirrors the package hierarchy of the application code's src directory. Here's an example...

/src
   /net
      /benjismith
         /myproject
            /MyClass.java
/test_src
   /net
      /benjismith
         /myproject
            /MyTestClass.java

When I build my project, it's easy to either include or exclude the classes in my unit-testing hierarchy. I don't know whether such a separation is possible in D, but this is how I prefer to structure my projects.

--Benji Smith

Benji Smith wrote:

> I'm not too familiar with the D unit tests, so please ignore my
> suggestion if it's already possible.
> 
> But when I do my java development, I always put my unit tests into a
> different source tree than my application code. The test_src directory
> has a package structure that mirrors the package hierarchy of the
> application code's src directory. Here's an example...

> When I build my project, it's easy to either include or exclude the
> classes in my unit-testing hierarchy. I don't know whether such a
> separation is possible in D, but this is how I prefer to structure my
> projects.

D makes quite a deal of including unittest { } into the core language.
Thus the tests go in the usual code, and not in a separate hierarchy.

They are enabled with the use of -unittest, and disabled with -release
(actually just by not adding the -unittest option in the first place)

See the various Phobos std modules for examples on how this looks...


Building the "main" program for the unit test is much more of a manual
process with D, since you have to make sure to reference all modules
that you want to test, and since there is no reflection like in Java.

Again, see "phobos/unittest.d" that comes with DMD for an example.

--anders

PS. In case you ask about "JavaDoc", that's usually done with Doxygen...
    Ant is working on some replacement: http://leds.sourceforge.net/

On Thu, 10 Feb 2005 00:13:24 -0800, Walter wrote:


> They're not really meant for including in a released binary. Just an easy way to get them run.

Hi

So, far I seen that people would like to:

1) Split the test program from the main program.(Alex)
2) Make sure that the tests get executed. (Walter)
3) Be able to run single named tests (Andy)
4) Be able to run unit test on -release programs. (Anders)

A way to make all this happen would be
for the compiler to generate a separate unit test executable (1)
and make the compiler run the test executable as the last step (2).
If the test executable could be called with parameters we got (3)
and if dynamic linked to the main code would give us (4).

Knud

ps.)
Some time ago I posted some unit test suggestions on the dlanguage wiki at
http://dlanguage.netunify.com/59

But the wiki seams to have gone away.