I'm staring a new thread here on the topic of Name unittests because the existing one is getting too long [1].

The best way to add support for named unit tests, without adding any new syntax to the language, is to leverage UDAs. This has already been discussed in the previous post [1]. The problem with UDAs and the current unit test runner is by the time the unit test runner runs the unit tests, the UDAs are long gone. They need to be fetched at compile-time. It's also not possible to continue running unit tests in the same module after a failed unit test with the current runner. This is because the compiler generates one function that calls all the unit tests in the whole module.

To solve these problems `__traits(getUnitTests)` can be used to access the unit tests at compile-time. This leads to the next problem. To access the unit tests one needs to import the modules where the unit tests are defined. There are currently no way to get a list of all modules. Existing third party unit test runners usually use a pre-build script that collects all files with unit tests and generates a new file with all the imports. This is also noted in the previous thread [2].

The solution of running a pre-build script is not workable to include in druntime. I purpose we add two new traits: `__traits(getRootModules)` and `__traits(getImportedModules)`. `getRootModules` will return all root modules, i.e. the files which were passed to the compiler on the command line. `getImportedModules` will return all imported modules, i.e. the non-root modules. Adding these two together will give access to all modules the compiler has processed. Note that these traits are useful for other things than building a unit test runner.

With these two traits (only getRootModules might be necessary) and using `static foreach` to generate the imports, `getUnitTests` can later be used to retrieve the unit tests at compile-time which also gives access to the UDAs. Using this technique to run the unit tests allows to add additional features, like continue running after a failed test, before and after callbacks and other features.

There's already a unit test runner in the DMD test suite that uses this technique (but with the pre-build script). With these new traits this unit test runner could be added pretty easily to druntime.

Thoughts?

[1] https://forum.dlang.org/thread/mfcgj3$12a0$1@digitalmars.com
[2] https://forum.dlang.org/post/qbr7dd$16fd$1@digitalmars.com

-- 
/Jacob Carlborg

On Sunday, 19 May 2019 at 18:56:33 UTC, Jacob Carlborg wrote:
> I'm staring a new thread here on the topic of Name unittests because the existing one is getting too long [1].
>
> [...]

As far as i remember there was another suggestion of Andrei (in another context). By importing a module B in module A, the module B can specify coding which is executed and gets the module A as info.

This might could also solve this issue.

Kind regards
Andre

On 5/19/19 8:13 PM, Andre Pany wrote:
> On Sunday, 19 May 2019 at 18:56:33 UTC, Jacob Carlborg wrote:
>> I'm staring a new thread here on the topic of Name unittests because the existing one is getting too long [1].
>>
>> [...]
> 
> As far as i remember there was another suggestion of Andrei (in another context). By importing a module B in module A, the module B can specify coding which is executed and gets the module A as info.
> 
> This might could also solve this issue.

Yah, it's quite a universal pattern. It can be used as "import core.rtti; to add RTTI support for this module" etc.

On 5/19/19 11:41 PM, Andrei Alexandrescu wrote:
> On 5/19/19 8:13 PM, Andre Pany wrote:
>> On Sunday, 19 May 2019 at 18:56:33 UTC, Jacob Carlborg wrote:
>>> I'm staring a new thread here on the topic of Name unittests because the existing one is getting too long [1].
>>>
>>> [...]
>>
>> As far as i remember there was another suggestion of Andrei (in another context). By importing a module B in module A, the module B can specify coding which is executed and gets the module A as info.
>>
>> This might could also solve this issue.
> 
> Yah, it's quite a universal pattern. It can be used as "import core.rtti; to add RTTI support for this module" etc.

Related: https://github.com/dlang/dmd/pull/9814. There, the idea is to predicate an import for each language feature or groups of features. Many, however, can be done as templates and realized naturally by means of template instantiation (e.g. a[] = b[] triggers the generation of the array assign support function). RTTI is different because it instructs the compiler to generate code a specific way without a detectable action in the source code. That kind of feature could be done with import + hook "execute this on the importing module".

On Monday, 20 May 2019 at 00:14:01 UTC, Andrei Alexandrescu wrote:

> RTTI is different because it instructs the compiler to generate code a specific way without a detectable action in the source code. That kind of feature could be done with import + hook "execute this on the importing module".

I'm not sure I fully understand this, so forgive me if I'm just making noise, but does the pattern in this PR give you what you need? https://github.com/dlang/dmd/pull/7799

The idea is to put a detectable action in the source code.  In the case of the PR referenced above, the detectable action is the existence of the `class TypeInfo` declaration.  The compiler has been programmed to look for that declaration, and generate code based on whether or not it exists.  Because the declaration exists in druntime's source code, users can also check for its existence in their source code.

Basically, the compiler is doing design-by-introspection just like the user.  The runtime's source code informs the compiler what to do rather than the other way around.

So the compiler does have a detectable action in the source code (i.e. the existence of `class TypeInfo`) and that same detectable action is available to the user as well.

Mike

On 2019-05-19 21:13, Andre Pany wrote:

> As far as i remember there was another suggestion of Andrei (in another context). By importing a module B in module A, the module B can specify coding which is executed and gets the module A as info.

Ok, that sounds useful as well. But in this case I think my solution is easier to implement, works less like magic and is better suited to implement a unit test runner.

With Andrei's proposal, as far as I understand, would require to add an import. The implementation would also iterate the unit tests locally for each module. I'm not sure how you would be able to combine all of them into a single list. Also how to know when all unit tests have been collected and the runner can actually start running the tests.

-- 
/Jacob Carlborg

On 5/19/19 8:55 PM, Mike Franklin wrote:
> On Monday, 20 May 2019 at 00:14:01 UTC, Andrei Alexandrescu wrote:
> 
>> RTTI is different because it instructs the compiler to generate code a specific way without a detectable action in the source code. That kind of feature could be done with import + hook "execute this on the importing module".
> 
> I'm not sure I fully understand this, so forgive me if I'm just making noise, but does the pattern in this PR give you what you need? https://github.com/dlang/dmd/pull/7799
> 
> The idea is to put a detectable action in the source code.  In the case of the PR referenced above, the detectable action is the existence of the `class TypeInfo` declaration.  The compiler has been programmed to look for that declaration, and generate code based on whether or not it exists.  Because the declaration exists in druntime's source code, users can also check for its existence in their source code.
> 
> Basically, the compiler is doing design-by-introspection just like the user.  The runtime's source code informs the compiler what to do rather than the other way around.
> 
> So the compiler does have a detectable action in the source code (i.e. the existence of `class TypeInfo`) and that same detectable action is available to the user as well.

Having the compiler detect constructs in code would work, as would built-in attributes such as:

@noRTTI module mymodule;

and/or

@noRTTI class BareBones { ... }

The disadvantage is that the compiler would need to be modified for each of these, whereas if we define and avail ourselves of powerful introspection, we can achieve this kind of stuff in library code.

On 5/19/19 2:56 PM, Jacob Carlborg wrote:
> I'm staring a new thread here on the topic of Name unittests because the existing one is getting too long [1].
> 
> [1] https://forum.dlang.org/thread/mfcgj3$12a0$1@digitalmars.com

Keep in mind, most of that thread is from 4 years ago.

But, umm, seriously dudes...we have unit-threaded now. Takes care of all that and then some. Just make it official, bake to into Phobos or whatever, and be done with it. Don't we have more important things to do than re-implement stuff we already have that's *already* working quite well?

I mean seriously, we've got people highly averse to *small, but actual, improvements*, but completely re-implementing something that's already working very well is worthwhile??? WAT???

(Or am I misunderstanding the gist of the small, non-outdated branch of that thread?)

On Monday, 20 May 2019 at 16:38:39 UTC, Nick Sabalausky (Abscissa) wrote:
> On 5/19/19 2:56 PM, Jacob Carlborg wrote:
>> I'm staring a new thread here on the topic of Name unittests because the existing one is getting too long [1].
>> 
>> [1] https://forum.dlang.org/thread/mfcgj3$12a0$1@digitalmars.com
>
> Keep in mind, most of that thread is from 4 years ago.
>
> But, umm, seriously dudes...we have unit-threaded now. Takes care of all that and then some. Just make it official, bake to into Phobos or whatever, and be done with it. Don't we have more important things to do than re-implement stuff we already have that's *already* working quite well?
>
> I mean seriously, we've got people highly averse to *small, but actual, improvements*, but completely re-implementing something that's already working very well is worthwhile??? WAT???
>
> (Or am I misunderstanding the gist of the small, non-outdated branch of that thread?)

unit-threaded is great but it requires a prebuild command to gather all the modules. Can’t really make that official...

This is also not currently fixable on a library level because there is no way to reflect on what modules are being compiled. Adding a way to do this is basically what Jacob was suggesting.

On 5/20/19 4:07 PM, Johannes Loher wrote:
> 
> unit-threaded is great but it requires a prebuild command to gather all the modules. Can’t really make that official...
> 
> This is also not currently fixable on a library level because there is no way to reflect on what modules are being compiled. Adding a way to do this is basically what Jacob was suggesting.

Ahh, right, right, I gotcha. And TBH, that's been LOOOOONG standing giant gaping hole in D's reflection. Would be fantastic to finally get that fixed even regardless of the unittest stuff. IIRC, that was also the one main technical issue in the way of being able to build a Java-style runtime reflection system on top of D's compiletime reflection (well, that and the odd choice of reflection not being able to see through private, which has luckily been fixed now AIUI).