http://d.puremagic.com/issues/show_bug.cgi?id=5091

           Summary: main runs after unittests
           Product: D
           Version: D1 & D2
          Platform: All
        OS/Version: All
            Status: NEW
          Severity: enhancement
          Priority: P2
         Component: DMD
        AssignedTo: nobody@puremagic.com
        ReportedBy: simen.kjaras@gmail.com


--- Comment #0 from Simen Kjaeraas <simen.kjaras@gmail.com> 2010-10-20 17:22:25 PDT ---
When compiling a program with dmd -unittest, after the unittests are run, main() is called. This is rarely wanted behavior, so should not be the default.

Currently, it is possible to customize main to sidestep the problem:

void main( ) {
    version( unittest ) {
    } else {
        // Your program
    }
}

However, this is cluttering and mixes your application code with versioning code.

-- 
Configure issuemail: http://d.puremagic.com/issues/userprefs.cgi?tab=email
------- You are receiving this mail because: -------

http://d.puremagic.com/issues/show_bug.cgi?id=5091


Peter Alexander <peter.alexander.au@gmail.com> changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
                 CC|                            |peter.alexander.au@gmail.co
                   |                            |m


--- Comment #1 from Peter Alexander <peter.alexander.au@gmail.com> 2010-10-21 01:07:14 PDT ---
(In reply to comment #0)
> When compiling a program with dmd -unittest, after the unittests are run, main() is called. This is rarely wanted behavior, so should not be the default.

Why is this rarely wanted?

If your way was the default then your work flow would be:

- Make change
- Run with unit test
- Run without unit test
- Make change
- Run with unit test
- Run without unit test
- etc.

The whole point of unit tests is that they are run regularly, preferably after ever change. If you only run them after lots of changes then it makes it much more difficult to find out what broke the build.

If you have a lot of time-consuming unit tests in your code then these should be put inside a slowtest version block so that you can leave those for your nightly testing (or whatever you want to do). In general however, unittest should always be on.

-- 
Configure issuemail: http://d.puremagic.com/issues/userprefs.cgi?tab=email
------- You are receiving this mail because: -------

http://d.puremagic.com/issues/show_bug.cgi?id=5091



--- Comment #2 from Peter Alexander <peter.alexander.au@gmail.com> 2010-10-21 01:11:22 PDT ---
Even if you don't buy that argument, consider this: with the current unittest, you only have to add a few lines of code to turn it off (as you described). This is something that you'll only ever have to do once, and really isn't that much of a burden.

On the other hand, if running main was off by default then there would be no way to turn it back on! You would have to resort to scripting your build to get it working again.

Therefore, the current unittest is unarguably more flexible to people's needs, and arguably more desirable anyway.

-- 
Configure issuemail: http://d.puremagic.com/issues/userprefs.cgi?tab=email
------- You are receiving this mail because: -------

http://d.puremagic.com/issues/show_bug.cgi?id=5091


bearophile_hugs@eml.cc changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
                 CC|                            |bearophile_hugs@eml.cc


--- Comment #3 from bearophile_hugs@eml.cc 2010-10-21 03:19:14 PDT ---
(In reply to comment #1)
> Why is this rarely wanted?

D unittesting needs a global redesign, not small changes.

-- 
Configure issuemail: http://d.puremagic.com/issues/userprefs.cgi?tab=email
------- You are receiving this mail because: -------

http://d.puremagic.com/issues/show_bug.cgi?id=5091



--- Comment #4 from Simen Kjaeraas <simen.kjaras@gmail.com> 2010-10-21 10:32:36 PDT ---
(In reply to comment #1)
> (In reply to comment #0)
> > When compiling a program with dmd -unittest, after the unittests are run, main() is called. This is rarely wanted behavior, so should not be the default.
> 
> Why is this rarely wanted?

Because I never want my program to run after I run my unittests? If the unittests pass, my program should be working, apart from things that are impossible/hard to unittest.

> If your way was the default then your work flow would be:
> 
> - Make change
> - Run with unit test
> - Run without unit test
> - Make change
> - Run with unit test
> - Run without unit test
> - etc.

That's one possibility. When I program, I aim for such coverage with my unittests that running the program to see if things work is not necessary. Hence, a more likely scenario:

1 Make change
2 Run unittests (quick)
3 If unittests fail, goto 1
4 Run normally (slow)
5 Goto 1

I have not once wanted to run both the full application and the unittests at the same time. when I want to run the unittests, running the application would be a waste of time. When I want to run the program, I don't care about the unittests.

-- 
Configure issuemail: http://d.puremagic.com/issues/userprefs.cgi?tab=email
------- You are receiving this mail because: -------

http://d.puremagic.com/issues/show_bug.cgi?id=5091



--- Comment #5 from Peter Alexander <peter.alexander.au@gmail.com> 2010-10-22 00:43:56 PDT ---
(In reply to comment #4)
> I have not once wanted to run both the full application and the unittests at the same time. when I want to run the unittests, running the application would be a waste of time. When I want to run the program, I don't care about the unittests.

Well I have to say that this is totally alien to me.

Putting our disagreement on exactly when unit tests should be run, we can at least agree on 3 things:

1. Some people want to run the program and unit tests at the same time, and
others don't.
2. Leaving things as they are, you have the choice between the two (although
one choice requires a few extra lines of code).
3. Changing things makes only one option possible.

Unless I've missed anything, I think this is a clear argument for leaving things as they are, regardless of which choice we may be biased towards.

-- 
Configure issuemail: http://d.puremagic.com/issues/userprefs.cgi?tab=email
------- You are receiving this mail because: -------

http://d.puremagic.com/issues/show_bug.cgi?id=5091


Andrej Mitrovic <andrej.mitrovich@gmail.com> changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
                 CC|                            |andrej.mitrovich@gmail.com


--- Comment #6 from Andrej Mitrovic <andrej.mitrovich@gmail.com> 2013-01-07 17:14:36 PST ---
You can override this behavior by defining your own unittest function:

import core.runtime;
import std.stdio;
import std.string;
import std.path;

bool testRunner()
{
    foreach (m; ModuleInfo)
    {
        if (m is null)
            continue;

        if (auto fp = m.unitTest)
        {
            try
            {
                fp();
            }
            catch (Throwable e)
            {
                writeln(e);
            }
        }
    }

    // stop execution after tests are done
    return false;
}

shared static this()
{
    Runtime.moduleUnitTester(&testRunner);
}

unittest
{
    writeln("in unittest");
    assert(1);
}

void main()
{
    writeln("not in main");
}

--------------

I've written a similar test system where I can also set these at runtime via getopt switches:

- specify which modules to run the tests on (--testMods=foo.bar)
- ditto but set modules which should *not* have their tests ran on
(--ignoreMods=foo.bar.doo)
- specify whether to run the unittests, and whether to run main() if the tests
were successful

I might write a DWiki entry later on how to do this if there's any interest.

-- 
Configure issuemail: http://d.puremagic.com/issues/userprefs.cgi?tab=email
------- You are receiving this mail because: -------

http://d.puremagic.com/issues/show_bug.cgi?id=5091


Martin Nowak <code@dawg.eu> changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
                 CC|                            |code@dawg.eu


--- Comment #7 from Martin Nowak <code@dawg.eu> 2013-03-13 07:10:31 PDT ---
It's a little hacky though, because returning false is interpreted as failed tests.

-- 
Configure issuemail: http://d.puremagic.com/issues/userprefs.cgi?tab=email
------- You are receiving this mail because: -------

http://d.puremagic.com/issues/show_bug.cgi?id=5091



--- Comment #8 from Andrej Mitrovic <andrej.mitrovich@gmail.com> 2013-03-13 12:17:24 PDT ---
(In reply to comment #7)
> It's a little hacky though, because returning false is interpreted as failed tests.

Yeah, I've realized that after posting. Currently I use getOpt and a --runMain switch in my own code.

-- 
Configure issuemail: http://d.puremagic.com/issues/userprefs.cgi?tab=email
------- You are receiving this mail because: -------

http://d.puremagic.com/issues/show_bug.cgi?id=5091



--- Comment #9 from Andrej Mitrovic <andrej.mitrovich@gmail.com> 2013-03-13 16:27:36 PDT ---
(In reply to comment #7)
> It's a little hacky though, because returning false is interpreted as failed tests.

Perhaps we could change ModuleUnitTester to return an int instead of a bool. Then 0 would mean tests failed, 1 would mean continue execution to main, and a new return (say 2) would mean don't execute main().

runModuleUnitTests would also have to return this status code, and then in dmain2.d we'd change:

if (runModuleUnitTests())

to:

if (runModuleUnitTests() == 1)

Of course it might be best to use an enumeration for this:

TestStatus
{
    Fail, Ok, SkipMain
}

-- 
Configure issuemail: http://d.puremagic.com/issues/userprefs.cgi?tab=email
------- You are receiving this mail because: -------