I would be nice that unittests were automatically launched by the compiler as
soon as it has finished the code-generation phase and that unittests weren't
really included in the object file at last.
This way you'll know if a unittest failed as soon as you compile the code, and
the -unittest option would be active by default when -release is not enabled.
What do you think?
-Dario

"Dario" <Dario_member@pathlink.com> wrote in message news:bepjq1$1q68$1@digitaldaemon.com...
> I would be nice that unittests were automatically launched by the compiler
as
> soon as it has finished the code-generation phase and that unittests
weren't
> really included in the object file at last.
> This way you'll know if a unittest failed as soon as you compile the code,
and
> the -unittest option would be active by default when -release is not
enabled.
> What do you think?

It's an intriguing idea, but the compiler has no way of executing code in a .obj file without doing a full link. Doing a full link gets you where you are today anyway with running unittests <g>.

Walter wrote:
> 
> "Dario" <Dario_member@pathlink.com> wrote in message news:bepjq1$1q68$1@digitaldaemon.com...
> > I would be nice that unittests were automatically launched by the compiler
> as
> > soon as it has finished the code-generation phase and that unittests
> weren't
> > really included in the object file at last.
> > This way you'll know if a unittest failed as soon as you compile the code,
> and
> > the -unittest option would be active by default when -release is not
> enabled.
> > What do you think?
> 
> It's an intriguing idea, but the compiler has no way of executing code in a .obj file without doing a full link. Doing a full link gets you where you are today anyway with running unittests <g>.

I think unittests should:

   - always tell which test(s) exactly have failed.

   - never stop on an error but always run through all tests.
     This is important if you break code while refactoring. You must have
     the complete information on the total effects.
     Typically this is accomplished by forking the process and reporting
     the state of the process back to the main unittest code.

   - should have a visual display to show how many tests have already been
     run and how many have failed (this is important for refactoring) because
     unittests may run (on more complex) systems up to hours.

   - run only once from an completed exe (an app.exe could create an marker
     file app.uok whose date-time-stamp is tested against app.exe)

--
Helmut Leitner    leitner@hls.via.at Graz, Austria   www.hls-software.com

"Helmut Leitner" <helmut.leitner@chello.at> wrote in message news:3F11B6D6.EFEB8CFD@chello.at...
> Walter wrote:
> > "Dario" <Dario_member@pathlink.com> wrote in message news:bepjq1$1q68$1@digitaldaemon.com...
> > > I would be nice that unittests were automatically launched by the
compiler
> > as
> > > soon as it has finished the code-generation phase and that unittests
> > weren't
> > > really included in the object file at last.
> > > This way you'll know if a unittest failed as soon as you compile the
code,
> > and
> > > the -unittest option would be active by default when -release is not
> > enabled.
> > > What do you think?
> > It's an intriguing idea, but the compiler has no way of executing code
in a
> > .obj file without doing a full link. Doing a full link gets you where
you
> > are today anyway with running unittests <g>.
>
> I think unittests should:
>
>    - always tell which test(s) exactly have failed.

It does already, with the file/line of the failed contract. Of course, you can put any code you want in the unittest blocks, not just assert()'s.


>    - never stop on an error but always run through all tests.
>      This is important if you break code while refactoring. You must have
>      the complete information on the total effects.
>      Typically this is accomplished by forking the process and reporting
>      the state of the process back to the main unittest code.

My experience usually is that if one test fails, subsequent tests fail as well because of cascading errors from the first fault.

>    - should have a visual display to show how many tests have already been
>      run and how many have failed (this is important for refactoring)
because
>      unittests may run (on more complex) systems up to hours.

You can do this by writing such code into the unittest blocks.


>    - run only once from an completed exe (an app.exe could create an
marker
>      file app.uok whose date-time-stamp is tested against app.exe)

That's an interesting approach. I've used self-patching exe's in the past to do initializations; I had to stop the practice because anti-virus programs had fits about it.

Walter wrote:
> > I think unittests should:
> >
> >    - always tell which test(s) exactly have failed.
> 
> It does already, with the file/line of the failed contract. Of course, you can put any code you want in the unittest blocks, not just assert()'s.

We had an fault during ICFP-contest where it didn't.

> >    - never stop on an error but always run through all tests.
> >      This is important if you break code while refactoring. You must have
> >      the complete information on the total effects.
> >      Typically this is accomplished by forking the process and reporting
> >      the state of the process back to the main unittest code.
> 
> My experience usually is that if one test fails, subsequent tests fail as well because of cascading errors from the first fault.

This may or may not be the case. But that's not important.

Important is that the people that invented unittests and write
books about unittests and advocate test driven development
expect it - I think with good reasons - to work that way.

> >    - should have a visual display to show how many tests have already been
> >      run and how many have failed (this is important for refactoring)
> because
> >      unittests may run (on more complex) systems up to hours.
> 
> You can do this by writing such code into the unittest blocks.

It should soon become a phobos module.

--
Helmut Leitner    leitner@hls.via.at Graz, Austria   www.hls-software.com

"Helmut Leitner" <helmut.leitner@chello.at> wrote in message news:3F1247B1.2788A5B@chello.at...
> Walter wrote:
> > > I think unittests should:
> > >    - always tell which test(s) exactly have failed.
> > It does already, with the file/line of the failed contract. Of course,
you
> > can put any code you want in the unittest blocks, not just assert()'s.
> We had an fault during ICFP-contest where it didn't.

What happened? It sounds like it could have been a GP fault, not an assert fail.

Walter wrote:
> 
> "Helmut Leitner" <helmut.leitner@chello.at> wrote in message news:3F1247B1.2788A5B@chello.at...
> > Walter wrote:
> > > > I think unittests should:
> > > >    - always tell which test(s) exactly have failed.
> > > It does already, with the file/line of the failed contract. Of course,
> you
> > > can put any code you want in the unittest blocks, not just assert()'s.
> > We had an fault during ICFP-contest where it didn't.
> 
> What happened? It sounds like it could have been a GP fault, not an assert fail.

Yes, something like that, or a DIV/0 fault. I think it was with a sin/cos
function that faulted in my local sandbox. I got rid of it by updating DMD to 0.67.
I think that Burton Radons knows better, what the real reason was, he seemed
to recognize the problem.

A unittest has to report an error position (and continue) no matter what caused it.

--
Helmut Leitner    leitner@hls.via.at Graz, Austria   www.hls-software.com

"Helmut Leitner" <helmut.leitner@chello.at> wrote in message news:3F13934D.C12084C8@chello.at...
> Walter wrote:
> > What happened? It sounds like it could have been a GP fault, not an
assert
> > fail.
> Yes, something like that, or a DIV/0 fault. I think it was with a sin/cos function that faulted in my local sandbox. I got rid of it by updating DMD
to 0.67.
> I think that Burton Radons knows better, what the real reason was, he
seemed
> to recognize the problem.
>
> A unittest has to report an error position (and continue) no matter what
caused it.

The position of a GPF will be revealed by running it under a debugger. You can write contracts that continue by just using something other than the built-in assert:

Print message and stop:
    assert(succeeded);

Print message and continue:
    if (!succeeded) printf("test xyz failed\n");

Walter wrote:
> 
> "Helmut Leitner" <helmut.leitner@chello.at> wrote in message news:3F13934D.C12084C8@chello.at...
> > Walter wrote:
> > > What happened? It sounds like it could have been a GP fault, not an
> assert
> > > fail.
> > Yes, something like that, or a DIV/0 fault. I think it was with a sin/cos function that faulted in my local sandbox. I got rid of it by updating DMD
> to 0.67.
> > I think that Burton Radons knows better, what the real reason was, he
> seemed
> > to recognize the problem.
> >
> > A unittest has to report an error position (and continue) no matter what
> caused it.
> 
> The position of a GPF will be revealed by running it under a debugger. You can write contracts that continue by just using something other than the built-in assert:
> 
> Print message and stop:
>     assert(succeeded);
> 
> Print message and continue:
>     if (!succeeded) printf("test xyz failed\n");

If you can't locate a GPF by a unittest, then it's not a real unittest. Real unittests fork the program and and GPF won't kill the main process.

--
Helmut Leitner    leitner@hls.via.at Graz, Austria   www.hls-software.com

"Helmut Leitner" <helmut.leitner@chello.at> wrote in message news:3F1472BF.99DE612@chello.at...
> If you can't locate a GPF by a unittest, then it's not a real unittest. Real unittests fork the program and and GPF won't kill the main process.

I see we have different ideas on what unit tests should be!