Thread overview
Re: How can one reliably run unittests
Sep 03, 2021
H. S. Teoh
Sep 03, 2021
deadalnix
Sep 05, 2021
H. S. Teoh
Sep 05, 2021
jfondren
Sep 05, 2021
jfondren
Sep 05, 2021
H. S. Teoh
Sep 05, 2021
jfondren
Sep 05, 2021
jfondren
Sep 05, 2021
deadalnix
September 02, 2021
On Fri, Sep 03, 2021 at 01:12:13AM +0000, deadalnix via Digitalmars-d wrote:
> On Friday, 3 September 2021 at 00:09:37 UTC, H. S. Teoh wrote:
> > Would this satisfy everybody?
> > 
> 
> 1/ make module the default. This is really the sensible behavior. Changing imports really change the list of test that are run, especially not recursively.

Making module the default would break existing usage. But whatever, the important thing is to implement -unittest=xxx first, then changing the default is a 1-line change later.


> 2/ the `-main` problem remains.

Even though it's kinda related, -main is really an orthogonal issue. I've also run into the problem myself where -main is needed when your modules don't contain main() but must be suppressed or worked around when it does declare main().  This should be fixed so that -main *optionally* inserts an empty main(), e.g., as a weak symbol, or whatever, so that the existence of a real main() doesn't cause a linker error.


> 3/ it still isn't possible to run the unit tests of several modules in one go by passing a list of modules to the compiler (in the same way you can compile several modules in one go and generate an object file for all of them at once).

Of course you can.  Under my proposal:

	dmd -unittest=module -i -run mod1.d mod2.d mod3.d main.d


T

-- 
Democracy: The triumph of popularity over principle. -- C.Bond
September 03, 2021
On Friday, 3 September 2021 at 03:06:27 UTC, H. S. Teoh wrote:
>> 2/ the `-main` problem remains.
>
> Even though it's kinda related, -main is really an orthogonal issue. I've also run into the problem myself where -main is needed when your modules don't contain main() but must be suppressed or worked around when it does declare main().  This should be fixed so that -main *optionally* inserts an empty main(), e.g., as a weak symbol, or whatever, so that the existence of a real main() doesn't cause a linker error.
>
>

Well, yes and no. It can be solved in an orthogonal fashion, but this really is the same problem: the flag you can pass to the compiler are doing nonsensical stuff and are not amendable to common use cases.

>> 3/ it still isn't possible to run the unit tests of several modules in one go by passing a list of modules to the compiler (in the same way you can compile several modules in one go and generate an object file for all of them at once).
>
> Of course you can.  Under my proposal:
>
> 	dmd -unittest=module -i -run mod1.d mod2.d mod3.d main.d
>

No, this will run the code in mod1.d , passing ["mod2.d" "mod3.d" "main.d"] as command line arguments.

September 05, 2021
On Fri, Sep 03, 2021 at 08:39:35AM +0000, deadalnix via Digitalmars-d wrote: [...]
> > > 3/ it still isn't possible to run the unit tests of several modules in one go by passing a list of modules to the compiler (in the same way you can compile several modules in one go and generate an object file for all of them at once).
> > 
> > Of course you can.  Under my proposal:
> > 
> > 	dmd -unittest=module -i -run mod1.d mod2.d mod3.d main.d
> > 
> 
> No, this will run the code in mod1.d , passing ["mod2.d" "mod3.d" "main.d"] as command line arguments.

Oh right, this is the other bogonity of -run: it expects the following argument to be the module that gets run.  I really don't understand the logic of this: surely the compiler ought to be able to know which module main() is declared in, so why does it need the user to select one specific module to be run?  This should be automated within the compiler itself.


T

-- 
Never criticize a man until you've walked a mile in his shoes. Then when you do criticize him, you'll be a mile away and he won't have his shoes.
September 05, 2021

On Sunday, 5 September 2021 at 15:04:58 UTC, H. S. Teoh wrote:

>

On Fri, Sep 03, 2021 at 08:39:35AM +0000, deadalnix via Digitalmars-d wrote: [...]

> > >

3/ it still isn't possible to run the unit tests of several modules in one go by passing a list of modules to the compiler (in the same way you can compile several modules in one go and generate an object file for all of them at once).

Of course you can. Under my proposal:

dmd -unittest=module -i -run mod1.d mod2.d mod3.d main.d

No, this will run the code in mod1.d , passing ["mod2.d" "mod3.d" "main.d"] as command line arguments.

I really don't understand the logic of this: surely the compiler ought to be able to know which module main() is declared in, so why does it need the user to select one specific module to be run?

$ grep -H . mod?.d
mod1.d:void main(string[] args) {
mod1.d:    import std.stdio : writeln;
mod1.d:    writeln(args[1 .. $]);
mod1.d:}
$ dmd -run mod1.d mod2.d mod3.d
["mod2.d", "mod3.d"]

mod1.d is a program that prints its args that I'd like to run with some arguments. mod2.d and mod3.d are not modules that even exist; I just want them printed out like that. Distinguishing between dmd and program arguments isn't a matter of compiler smarts.

September 05, 2021

On Sunday, 5 September 2021 at 15:52:51 UTC, jfondren wrote:

>

On Sunday, 5 September 2021 at 15:04:58 UTC, H. S. Teoh wrote:

>

I really don't understand the logic of this: surely the compiler ought to be able to know which module main() is declared in, so why does it need the user to select one specific module to be run?

$ grep -H . mod?.d
mod1.d:void main(string[] args) {
mod1.d:    import std.stdio : writeln;
mod1.d:    writeln(args[1 .. $]);
mod1.d:}
$ dmd -run mod1.d mod2.d mod3.d
["mod2.d", "mod3.d"]

mod1.d is a program that prints its args that I'd like to run with some arguments. mod2.d and mod3.d are not modules that even exist; I just want them printed out like that. Distinguishing between dmd and program arguments isn't a matter of compiler smarts.

More:

$ grep -H . mod?.d
mod1.d:void main(string[] args) {
mod1.d:    import std.stdio : writeln;
mod1.d:    writeln(args[1 .. $]);
mod1.d:}
mod2.d:shared static this() {
mod2.d:    import std.stdio : writeln;
mod2.d:    writeln("module init: mod2.d");
mod2.d:}
mod3.d:shared static this() {
mod3.d:    import std.stdio : writeln;
mod3.d:    writeln("module init: mod3.d");
mod3.d:}
$ dmd -run mod1.d mod2.d mod3.d
["mod2.d", "mod3.d"]
$ dmd mod2.d mod3.d -run mod1.d mod2.d mod3.d
module init: mod2.d
module init: mod3.d
["mod2.d", "mod3.d"]
$ dmd mod2.d mod1.d -run mod3.d mod2.d mod1.d
module init: mod2.d
module init: mod3.d
["mod2.d", "mod1.d"]
$ echo 'import mod2, mod3;' >> mod1.d
$ dmd -i -run mod1.d
module init: mod2.d
module init: mod3.d
[]

In particular:

$ dmd mod2.d mod3.d -run mod1.d mod2.d mod3.d
$ dmd mod2.d mod1.d -run mod3.d mod2.d mod1.d

dmd links all the objects together to make an executable and runs it, and the behavior's the same with main()-having mod1.d and main()-lacking mod3.d as the argument after -run.

September 05, 2021
On Sun, Sep 05, 2021 at 03:52:51PM +0000, jfondren via Digitalmars-d wrote:
> On Sunday, 5 September 2021 at 15:04:58 UTC, H. S. Teoh wrote:
> > On Fri, Sep 03, 2021 at 08:39:35AM +0000, deadalnix via Digitalmars-d wrote: [...]
> > > > > 	dmd -unittest=module -i -run mod1.d mod2.d mod3.d main.d
> > > >
> > > 
> > > No, this will run the code in mod1.d , passing ["mod2.d" "mod3.d" "main.d"] as command line arguments.
> > 
> > I really don't understand the logic of this: surely the compiler ought to be able to know which module main() is declared in, so why does it need the user to select one specific module to be run?
> 
> ```
> $ grep -H . mod?.d
> mod1.d:void main(string[] args) {
> mod1.d:    import std.stdio : writeln;
> mod1.d:    writeln(args[1 .. $]);
> mod1.d:}
> $ dmd -run mod1.d mod2.d mod3.d
> ["mod2.d", "mod3.d"]
> ```
> 
> mod1.d is a program that prints its args that I'd like to run with some arguments. mod2.d and mod3.d are not modules that even exist; I just want them printed out like that. Distinguishing between dmd and program arguments isn't a matter of compiler smarts.

This is seriously a wrong conflation of the program arguments with a program module.  If the whole point is to support specifying the command-line to the program, then the argument list should appear immediately after -run; the first argument after -run should not be conflated with argv[0]. (For example, what if I'm writing an analogue of busybox, and need to specify a different argv[0] from the name of the executable?)

The current behaviour is counterintuitive, doesn't cover all use cases, and has inconsistencies that lead to confusion.  It's poor design.


T

-- 
I am a consultant. My job is to make your job redundant. -- Mr Tom
September 05, 2021

On Sunday, 5 September 2021 at 16:12:26 UTC, H. S. Teoh wrote:

>

This is seriously a wrong conflation of the program arguments with a program module. If the whole point is to support specifying the command-line to the program, then the argument list should appear immediately after -run; the first argument after -run should not be conflated with argv[0]. (For example, what if I'm writing an analogue of busybox, and need to specify a different argv[0] from the name of the executable?)

The current behaviour is counterintuitive, doesn't cover all use cases, and has inconsistencies that lead to confusion. It's poor design.

The current behavior is entirely intuitive, it covers all cases, it's consistent, it's fine. The only difference from it and a lazy getopt solution is lacking a -- before the arguments. The only difference between it and your proposed solution is that -run would no longer require an argument.

Listen to yourself:

$ dmd -run mod1.d  # UNCLEAN, UNCLEAN, UNCLEAN!

$ dmd mod1.d -run  # this is completely fine <3

It is absolutely not the case that one of these is wretched bad design and the other is pure and great design. These are completely identical designs. From the perspective of someone who's never seen -run used, it's a coinflip whether they'll guess it works one way or another, and it's no longer a coinflip after they read the manpage or have seen it used--it's just memory at that point, and either design is justifiable and easy to remember.

If the current design were counterintuitive, then people would occasionally screw it up even after learning it. I use -run all the time, almost more often than rdmd --eval, and I've never once had a problem. "I want to -run this" even follows SVO grammar.

I don't think I'd even be bothered--or notice--if a PR went through that made -run nonpositional, but you obviously don't use this feature all the time, what's with the ludicrously over-the-top vitriol about it?

>

the first argument after -run should not be conflated with argv[0]. (For example, what if I'm writing an analogue of busybox, and need to specify a different argv[0] from the name of the executable?)

argv[0] is provided by the kernel, not userspace. man execve and man posix_spawn and such will have an argv parameter, but that's for subsequent arguments. The most busybox can do is tell the kernel what to run with one path or another, in the extreme by creating a temporary symlink.

September 05, 2021

On Sunday, 5 September 2021 at 16:38:01 UTC, jfondren wrote:

>

I don't think I'd even be bothered--or notice--if a PR went through that made -run nonpositional, but you obviously don't use this feature all the time, what's with the ludicrously over-the-top vitriol about it?

Regrettably, avoiding criticizing something after 5 seconds of thought isn't an adequate replacement for thinking it through, either. Because -run still needs to know where the program's parameters begin, it can't be completely non-positional.

September 05, 2021
On Sunday, 5 September 2021 at 16:12:26 UTC, H. S. Teoh wrote:
> The current behaviour is counterintuitive, doesn't cover all use cases, and has inconsistencies that lead to confusion.  It's poor design.
>

I'm glad that, one by one, people are waking up to this fact. It took 80+ messages, but we're getting there.

This is especially bonkers in the unittest case, because we are not even planning to run main.