Walter wrote:
> 
> "Helmut Leitner" <helmut.leitner@chello.at> wrote in message news:3F1472BF.99DE612@chello.at...
> > If you can't locate a GPF by a unittest, then it's not a real unittest. Real unittests fork the program and and GPF won't kill the main process.
> 
> I see we have different ideas on what unit tests should be!

You have just implemented - and I can understand that - what's easy to implement. At some point you will have to live up to the expectations of test-oriented people used to unittest frameworks.

Even if you use a free open source C testing framework under
windows, you
   - catch all errors (including GPFs)
   - run through all tests no matter how many errors
     and what errors occur

I think it would not even be a problem to supply sample code how to accomplish that.

But obviously when you say "have different ideas on what unit tests should be" then the master has spoken his final words...

--
Helmut Leitner    leitner@hls.via.at Graz, Austria   www.hls-software.com

Surely if a test fails, then other subsequent tests cannot be guaranteed their correct pre-conditions (not to mention a stable host) so their results may be meaningless? A little like relying on later compiler errors when you should be fixing the first one.



"Helmut Leitner" <helmut.leitner@chello.at> wrote in message news:3F163019.BDDD344E@chello.at...
>
>
> Walter wrote:
> >
> > "Helmut Leitner" <helmut.leitner@chello.at> wrote in message news:3F1472BF.99DE612@chello.at...
> > > If you can't locate a GPF by a unittest, then it's not a real
unittest.
> > > Real unittests fork the program and and GPF won't kill the main
process.
> >
> > I see we have different ideas on what unit tests should be!
>
> You have just implemented - and I can understand that - what's easy to implement. At some point you will have to live up to the expectations of test-oriented people used to unittest frameworks.
>
> Even if you use a free open source C testing framework under
> windows, you
>    - catch all errors (including GPFs)
>    - run through all tests no matter how many errors
>      and what errors occur
>
> I think it would not even be a problem to supply sample code how to accomplish that.
>
> But obviously when you say "have different ideas on what unit tests should be" then the master has spoken his final words...
>
> --
> Helmut Leitner    leitner@hls.via.at
> Graz, Austria   www.hls-software.com

"Helmut Leitner" <helmut.leitner@chello.at> wrote in message news:3F163019.BDDD344E@chello.at...
> You have just implemented - and I can understand that - what's easy to implement. At some point you will have to live up to the expectations of test-oriented people used to unittest frameworks.

I agree that what D provides is basic. However, it is way, *way* ahead of what any other language provides, which is nothing. In my own use of D unit tests, it has proven its value time and again.

> Even if you use a free open source C testing framework under
> windows, you
>    - catch all errors (including GPFs)
>    - run through all tests no matter how many errors
>      and what errors occur
>
> I think it would not even be a problem to supply sample code how to accomplish that.
>
> But obviously when you say "have different ideas on what unit tests should be" then the master has spoken his final words...

D doesn't prevent unittests from operating the way you want them too - but you'd need to write some additional code to implement that.

Walter wrote:
> 
> "Helmut Leitner" <helmut.leitner@chello.at> wrote in message news:3F163019.BDDD344E@chello.at...
> > You have just implemented - and I can understand that - what's easy to implement. At some point you will have to live up to the expectations of test-oriented people used to unittest frameworks.
> 
> I agree that what D provides is basic. However, it is way, *way* ahead of what any other language provides, which is nothing. In my own use of D unit tests, it has proven its value time and again.
> 
> > Even if you use a free open source C testing framework under
> > windows, you
> >    - catch all errors (including GPFs)
> >    - run through all tests no matter how many errors
> >      and what errors occur
> >
> > I think it would not even be a problem to supply sample code how to accomplish that.
> >
> > But obviously when you say "have different ideas on what unit tests should be" then the master has spoken his final words...
> 
> D doesn't prevent unittests from operating the way you want them too - but you'd need to write some additional code to implement that.

Only if you provide an interface to get at the unittest function units.

Lets say any unittest is like a
  void unittest_xx();
then there should be an array of pointers in any module compiled for unittests
  array_unitest_modulexyz (pointer, codeline)
and some datastructure (like a linked list or similar)
  list_unittest_arrays (arraypointer, modulename)

There should be a way to compile the unittests without actually being called at the program start.

It should be guaranteed that different unittest blocks within one module are not combined into one functional unit (I don't know if you do).

-- 
Helmut Leitner    leitner@hls.via.at
Graz, Austria   www.hls-software.com

Matthew Wilson wrote:
> 
> Surely if a test fails, then other subsequent tests cannot be guaranteed their correct pre-conditions (not to mention a stable host) so their results may be meaningless? A little like relying on later compiler errors when you should be fixing the first one.

Just look at the way unittests are written. They typically don't build on each other. They are not like the logical flow of a program.

-- 
Helmut Leitner    leitner@hls.via.at
Graz, Austria   www.hls-software.com

To the "stop or not to stop" debate: would it be
OK adding a compiler (err... linker??) switch like
"do not abort on failed asserts"? (I can imagine
that it may not be as trivial as it sounds, but
I don't know the D implementation of assert().)

Sz.

"Helmut Leitner" <leitner@hls.via.at> wrote in message news:3F165C52.F2624CD1@hls.via.at...

> > D doesn't prevent unittests from operating the way you want them too -
but
> > you'd need to write some additional code to implement that.
> Only if you provide an interface to get at the unittest function units.
>
> Lets say any unittest is like a
>   void unittest_xx();
> then there should be an array of pointers in any module compiled for
unittests
>   array_unitest_modulexyz (pointer, codeline)
> and some datastructure (like a linked list or similar)
>   list_unittest_arrays (arraypointer, modulename)

You can see how they are called in phobos/moduleinit.d. It's equivalent to as you suggest.

> There should be a way to compile the unittests without actually being called at the program start.

You can do this by modifying phobos/moduleinit.d.

> It should be guaranteed that different unittest blocks within one module are not combined into one functional unit (I don't know if you do).

Check out this code:

unittest
{
 printf("1\n");
}

unittest
{
 printf("2\n");
}

void main()
{
}

which compiles to:

_DATA   segment
        db      031h,00ah,000h,032h,00ah,000h,074h,065h
        db      073h,074h,000h,000h
__ModuleInfo_test:
        db      000h,000h,000h,000h
        db      000h,000h,000h,000h,004h,000h,000h,000h
        dd      offset FLAT:_DATA[6]
        db      000h,000h,000h,000h
        db      000h,000h,000h,000h,000h,000h,000h,000h
        db      000h,000h,000h,000h,000h,000h,000h,000h
        db      000h,000h,000h,000h,000h,000h,000h,000h
        dd      offset FLAT:__modtest_test
_DATA   ends
CONST   segment
CONST   ends
_BSS    segment
_BSS    ends
FMB     segment
FMB     ends
FM      segment
        dd      offset FLAT:__ModuleInfo_test
FM      ends
FME     segment
FME     ends
_Dtest_unittest0_FZv    comdat
        assume  CS:_Dtest_unittest0_FZv
L0:             push    offset FLAT:_DATA
                call    near ptr _printf
                add     ESP,4
                ret
_Dtest_unittest0_FZv    ends
_Dtest_unittest1_FZv    comdat
        assume  CS:_Dtest_unittest1_FZv
L0:             push    offset FLAT:_DATA[3]
                call    near ptr _printf
                add     ESP,4
                ret
_Dtest_unittest1_FZv    ends
__Dmain comdat
        assume  CS:__Dmain
                ret
__Dmain ends
__modtest_test  comdat
        assume  CS:__modtest_test
L0:             call    near ptr _Dtest_unittest0_FZv
                call    near ptr _Dtest_unittest1_FZv
                ret
__modtest_test  ends
        end

"Luna Kid" <lunakid@neuropolis.org> wrote in message news:bf8ae1$18bu$1@digitaldaemon.com...
> To the "stop or not to stop" debate: would it be
> OK adding a compiler (err... linker??) switch like
> "do not abort on failed asserts"? (I can imagine
> that it may not be as trivial as it sounds, but
> I don't know the D implementation of assert().)

You can add an exception handler to catch them and continue. In fact, that's what the startup code does in order to catch & print any out, phobos\dmain2.d:

    try
    {
        _moduleCtor();
        _moduleUnitTests();

        for (i = 0; i < argc; i++)
        {
            int len = strlen(argv[i]);
            am[i] = argv[i][0 .. len];
        }

        args = am[0 .. argc];

        result = main(args);
    }

    catch (Object o)
    {
        printf("Error: ");
        o.print();
        result = EXIT_FAILURE;
    }

Hi,

    Comments embedded.

"Walter" <walter@digitalmars.com> escreveu na mensagem news:besod7$1qb8$1@digitaldaemon.com...
> "Helmut Leitner" <helmut.leitner@chello.at> wrote in message news:3F11B6D6.EFEB8CFD@chello.at...
> > Walter wrote:
> > > "Dario" <Dario_member@pathlink.com> wrote in message news:bepjq1$1q68$1@digitaldaemon.com...
> > > > I would be nice that unittests were automatically launched by the
> compiler
> > > as
> > > > soon as it has finished the code-generation phase and that unittests
> > > weren't
> > > > really included in the object file at last.
> > > > This way you'll know if a unittest failed as soon as you compile the
> code,
> > > and
> > > > the -unittest option would be active by default when -release is not
> > > enabled.
> > > > What do you think?
> > > It's an intriguing idea, but the compiler has no way of executing code
> in a
> > > .obj file without doing a full link. Doing a full link gets you where
> you
> > > are today anyway with running unittests <g>.

    I think a make-like utility is essential in D projects, I usually have a
dummy "main" for unittests that just return 0, so I just type "make tests"
and everything is fine. You shouldn't compile and link with your real "main"
using -unittest.

> > I think unittests should:
> >
> >    - always tell which test(s) exactly have failed.
>
> It does already, with the file/line of the failed contract. Of course, you can put any code you want in the unittest blocks, not just assert()'s.

    It isn't enough for a couple of reasons: assert just give me the
file/line of the assert failed, not the call stack. If I have a contract
failing inside a method after twenty calls from my unittest the error
message don't give me any context to work with. A very good solution would
be making the contract failure info Eiffel-like, dumping the call stack and
the variables in stack at the time (so we can figure out the context). The
current state is really crude, I usually resort to insert several lines of
printf to pinpoint the error. Of course I could debug instead of this, but I
use TDD and in any other language the tests are enough.

> >    - never stop on an error but always run through all tests.
> >      This is important if you break code while refactoring. You must
have
> >      the complete information on the total effects.
> >      Typically this is accomplished by forking the process and reporting
> >      the state of the process back to the main unittest code.
>
> My experience usually is that if one test fails, subsequent tests fail as well because of cascading errors from the first fault.

    Today we have units of unittests so I think each unit would be
independent. If I have three "unittest" blocks they are independent (they
are test fixtures, and each must be independent). When I change something
it's important to know that the "insert" operation failed but everything
else is still fine. Today it's all or nothing.

> >    - should have a visual display to show how many tests have already
been
> >      run and how many have failed (this is important for refactoring)
> because
> >      unittests may run (on more complex) systems up to hours.
>
> You can do this by writing such code into the unittest blocks.

    If unittests had names (like "unittest testFindString { ... }") using
reflective libraries we could automate this process and create different
test reporters for different environments (e.g. text, GUI, XML, etc.) like
any of the xUnit tools make. Walter, how much do you know about the xUnit
tools (e.g. jUnit, sUnit)? There's a lot of info in www.junit.org about how
they could develop a powerful, extensible, tool using Java's reflexive
mechanisms. I'll make a summary of jUnit basic features:

- It has classes representing test cases, test suites, test results and test
runners.
- Test cases are just classes extending TestCase containing methods named
testXXXXXXXX and two special methods: setUp and tearDown. When the test case
is executed the test runner uses reflection to find all methods named
testXXXXXX, and for each one executes the TestCase setUp method, the current
test method and finally calls the TestCase tearDown method. If an assertion
fails inside a TestCase test method, it raises an exception that is used to
report the failure.
- TestSuites are a bunch of TestCases bundled together.
- TestResults encapsulate the details of the executed tests, including which
failed, which succeded and which raised uncatched exceptions.
- TestRunners run the tests and report the failures in a given medium (e.g.
text, GUI, XML, e-mail to the developers, etc.).

   This is just a quick look of how it works, but almost every programmer
coming to D will look after these features. I use jUnit all the time in
Java, and D unittest support is pale in comparison. If we need some kind of
house-keeping operations (i.e. setUp and tearDown) we have to remember
calling them explicitly, if we need some kind of alternative medium (GUI is
much better than text) we must explicit code each every time. I'm tempted to
rewrite jUnit in D, actually I'm doing that right now, just to have these
features.

> >    - run only once from an completed exe (an app.exe could create an
> marker
> >      file app.uok whose date-time-stamp is tested against app.exe)
>
> That's an interesting approach. I've used self-patching exe's in the past
to
> do initializations; I had to stop the practice because anti-virus programs had fits about it.

    Having unittests as first class objects is nice, but if the unittesting
support is incomplete regarding other languages, it won't be necessary and
probably discarded in larger projects. jUnit is an example of how it should
be, for other frameworks you can check
http://www.artima.com/suiterunner/index.html , http://www.fitnesse.org/ ,
http://fit.c2.com and http://c2.com/cgi/wiki?FrameworkForIntegratedTest

    Best regards,
    Daniel Yokomiso.

A: Top-posters.
Q: What is the most annoying thing on Usenet?


---
Outgoing mail is certified Virus Free.
Checked by AVG anti-virus system (http://www.grisoft.com).
Version: 6.0.502 / Virus Database: 300 - Release Date: 18/7/2003

Thank you for these hints. They may or may not be sufficient.

The problem is that unittests are implemented on top of almost any programming language. Most of the time they are separate testing frameworks.

The D claim to support unittests would make a lot of sense if - and only if - the support is out of the box. Otherwise it means nothing.

I'll try to build on the hooks you gave me, but it will only make sense when reintegrated into D.

Walter wrote:
> 
> "Helmut Leitner" <leitner@hls.via.at> wrote in message news:3F165C52.F2624CD1@hls.via.at...
> 
> > > D doesn't prevent unittests from operating the way you want them too -
> but
> > > you'd need to write some additional code to implement that.
> > Only if you provide an interface to get at the unittest function units.
> >
> > Lets say any unittest is like a
> >   void unittest_xx();
> > then there should be an array of pointers in any module compiled for
> unittests
> >   array_unitest_modulexyz (pointer, codeline)
> > and some datastructure (like a linked list or similar)
> >   list_unittest_arrays (arraypointer, modulename)
> 
> You can see how they are called in phobos/moduleinit.d. It's equivalent to as you suggest.
> 
> > There should be a way to compile the unittests without actually being called at the program start.
> 
> You can do this by modifying phobos/moduleinit.d.
> 
> > It should be guaranteed that different unittest blocks within one module are not combined into one functional unit (I don't know if you do).
> 
> Check out this code:
> 
> unittest
> {
>  printf("1\n");
> }
> 
> unittest
> {
>  printf("2\n");
> }
> 
> void main()
> {
> }
> 
> which compiles to:
> 
> _DATA   segment
>         db      031h,00ah,000h,032h,00ah,000h,074h,065h
>         db      073h,074h,000h,000h
> __ModuleInfo_test:
>         db      000h,000h,000h,000h
>         db      000h,000h,000h,000h,004h,000h,000h,000h
>         dd      offset FLAT:_DATA[6]
>         db      000h,000h,000h,000h
>         db      000h,000h,000h,000h,000h,000h,000h,000h
>         db      000h,000h,000h,000h,000h,000h,000h,000h
>         db      000h,000h,000h,000h,000h,000h,000h,000h
>         dd      offset FLAT:__modtest_test
> _DATA   ends
> CONST   segment
> CONST   ends
> _BSS    segment
> _BSS    ends
> FMB     segment
> FMB     ends
> FM      segment
>         dd      offset FLAT:__ModuleInfo_test
> FM      ends
> FME     segment
> FME     ends
> _Dtest_unittest0_FZv    comdat
>         assume  CS:_Dtest_unittest0_FZv
> L0:             push    offset FLAT:_DATA
>                 call    near ptr _printf
>                 add     ESP,4
>                 ret
> _Dtest_unittest0_FZv    ends
> _Dtest_unittest1_FZv    comdat
>         assume  CS:_Dtest_unittest1_FZv
> L0:             push    offset FLAT:_DATA[3]
>                 call    near ptr _printf
>                 add     ESP,4
>                 ret
> _Dtest_unittest1_FZv    ends
> __Dmain comdat
>         assume  CS:__Dmain
>                 ret
> __Dmain ends
> __modtest_test  comdat
>         assume  CS:__modtest_test
> L0:             call    near ptr _Dtest_unittest0_FZv
>                 call    near ptr _Dtest_unittest1_FZv
>                 ret
> __modtest_test  ends
>         end

--
Helmut Leitner    leitner@hls.via.at Graz, Austria   www.hls-software.com