On 1/25/14 7:52 PM, Jesse Phillips wrote:
> On Saturday, 25 January 2014 at 22:55:33 UTC, Andrei Alexandrescu wrote:
>> There's this simple realization that unittests could (should?) be
>> considered an intrinsic part of the build process. In order for an
>> executable to be worth running, it should pass the regular semantic
>> checks and also the unittests, which in a sense are extended semantic
>> checks that fall outside the traditional charter of the compiler.
>>
>> In that view, asserts inside unittests should fail with the same
>> message format as regular compilation errors, i.e.
>>
>> ./modulename.d(103): Unittest failed: user-defined message
>
> I already attempt to get unittests to fail at compile time by using
> static assert. I think running it during compilation would be a great
> change.

That's great but I wasn't thinking of CTFEing unittests as much as just running them right after building.

Andrei

On 1/25/14 8:29 PM, Rikki Cattermole wrote:
> On Saturday, 25 January 2014 at 22:55:33 UTC, Andrei Alexandrescu wrote:
>> ./modulename.d(103): Unittest failed: user-defined message
> My idea for pragma(error,) and pragma(warning,) would definately be
> invaluable to doing this without adding this specific behaviour. (Really
> should deal with that.)

What about static assert(false, "message")?

Andrei

On 1/26/14 3:20 AM, anonymous wrote:
> On Sunday, 26 January 2014 at 05:17:15 UTC, H. S. Teoh wrote:
>> On Sun, Jan 26, 2014 at 04:29:10AM +0000, Rikki Cattermole wrote:
>>> Having the ability to run unittests at both compile time and runtime
>>> would be useful. Because what happens when you need to test e.g. an
>>> OS feature with it? Or have a dependency that simply cannot run at
>>> compile time?
>>>
>>> I'm all for being able to selectively run unittests and having the
>>> ability to have some run at compile time.
>>
>> +1. Some of my unittests can only be run at runtime: like testing file
>> I/O.  Obviously, other unittests can also be run at compile-time, so
>> it's useful to have both.
>
> I don't think the tests are supposed to be CTFE'd. They'd be
> compiled (to a temporary executable?) and then run normally.

Yah, that was my idea.

Andrei

On 1/26/14 3:25 AM, simendsjo wrote:
> On Saturday, 25 January 2014 at 22:55:33 UTC, Andrei Alexandrescu wrote:
>> There's this simple realization that unittests could (should?) be
>> considered an intrinsic part of the build process.
> (...)
>
>> In particular, this view of unittests declares our current stance on
>> running unittests ("run unittests just before main()") as meaningless.
> (...)
>
> I wouldn't mind having unittests be a part of the compilation process,
> but I really don't think "running before main" is useless.
> This lets me compile a version on my desktop and push the binary to a
> small VPS staging for running runtime tests. Compiling any non-trivial
> code on 512MB with the DMD frontend is impossible. I can also send a
> fully working application with unittests to a client as a beta before
> giving a production version.

Does it help that the client runs them every time? I.e. does the n+1th run of the unittests bring more information than the first?

Andrei

On 1/26/14 4:58 AM, Idan Arye wrote:
> I don't think unit tests should be run on *each* build - if I'm
> fidgeting with a specific piece of my code, and building&running it
> after each change, I don't want to waste time running the unittests of
> the entire project.

Hah, interesting. So unittests of one module should run again iff there were changes in that module or other modules that that depends on.

> How about keeping the old behavior of `-unittest` for backward
> compatibility, and having a new syntax - `-unittest=filename`, where
> "filename" is the name of the unittests executable.

Sounds interesting.

> That way both the
> unittest and the regular executables will be created at the same time -
> which will speed things up, as most of the compilation work can be
> shared between those two builds - and have different file names, so the
> build system - which will make configuring the build system more
> straightforward.

Yeppers. One other thought I had was to define a special flag e.g. --4c5ad7908c2aa1b3de32ea25968cdf49 that says "just run unittests".

> If we make the unittest a separate executable, there won't be a `main`
> function waiting for command line arguments, so we can use the command
> line arguments for the unittests. We can have flags for only running
> tests of specific packages, or for running the tests under certain
> restrictions etc.

There we go :o).


Andrei

On 1/26/14 12:43 PM, Steven Schveighoffer wrote:
> On Sat, 25 Jan 2014 17:55:30 -0500, Andrei Alexandrescu
> <SeeWebsiteForEmail@erdani.org> wrote:
>
>> What do you think? Logistically it shouldn't be too hard to arrange
>> things to cater to this approach.
>
> After reading other opinions, this is what I think:
>
> 1. unit tests should be built as a separate binary. So when you build
> foo with -unittest, you get foo_unittest in addition to foo.
> 2. If you want to run unit tests, run foo_unittest. If you want simply
> the program, run foo.

Makes sense, though we could massage the unittests into the executable.

> I don't see a huge need for having unit tests run before the normal
> program. But maybe we should add a switch to make that work.

Guess we'd need to keep things running as they do for backwards compatibility.

> Building and running it as part of compilation seems like an incorrect
> function of the compiler. This is best left to an IDE/build script.
> Also, building unit tests still needs to be opt-in. Some projects can
> take a long time to build unit tests (dcollections takes about 20x
> longer to compile unit tests, last time I checked), and a quick
> compile-test-debug cycle is a key feature of D.

Yah it should be all nicely controllable by the build system.


Andrei

On Monday, 27 January 2014 at 03:45:00 UTC, Andrei Alexandrescu wrote:
> On 1/25/14 7:52 PM, Jesse Phillips wrote:
>> I already attempt to get unittests to fail at compile time by using
>> static assert. I think running it during compilation would be a great
>> change.
>
> That's great but I wasn't thinking of CTFEing unittests as much as just running them right after building.
>
> Andrei

I'm aware of that, my point being that I'm already trying to get them to run as frequently as possible.

On 1/26/14 1:04 PM, Gary Willoughby wrote:
> I don't fully understand this. Are you proposing to disconnect running
> unittest and evoking main? So unittests can be compiled and run without
> actually running the program?

Right.

> If this is the case then yes it would be nice to be able to do this but
> i would also recommend keeping the current behaviour of invoking main
> after the unittests have run, purely because developers (myself
> included) like to run the application often in debug/unittest mode
> during development.

We'd need to preserve backward compatibility.


Andrei

On 1/26/14 5:36 PM, Ary Borenszweig wrote:
> On 1/25/14, 7:55 PM, Andrei Alexandrescu wrote:
>> There's this simple realization that unittests could (should?) be
>> considered an intrinsic part of the build process. In order for an
>> executable to be worth running, it should pass the regular semantic
>> checks and also the unittests, which in a sense are extended semantic
>> checks that fall outside the traditional charter of the compiler.
>
> I can imagine someone who discovered a bug late at night, has a fix and
> needs to upload the new executable as soon as possible: he quickly
> comments all failing unit tests to make them pass. The next morning he
> uncomments them and fixes them with tranquility.

The point being?

Andrei

On Monday, 27 January 2014 at 03:58:54 UTC, Andrei Alexandrescu wrote:
> Yeppers. One other thought I had was to define a special flag e.g. --4c5ad7908c2aa1b3de32ea25968cdf49 that says "just run unittests".

I really think it would be better to use --4c5ad7908c2aa1b3de42ea25968cdf49 instead, it just makes the intent clearer.