I'd like to print out the name given to a unit test via a UDA, as the tests are being ran by the Runtime.moduleUnitTester. I'm working on writing a custom unit test runner. It would seem that the ModuleInfo.unitTest property is an aggregated function of all the individual unit test blocks in a module, and therefore the UDA value is lost.

I've been attempting to use the __traits(getUnitTests, m) function, but that doesn't seem to work with ModuleInfo because it is a variable and not a symbol. Additionally, if you run __traits(getUnitTests, m) against a module, that doesn't retrieve any unit tests that are nested inside classes or structs. So even if it did work with ModuleInfo, there may still be problems.

What I would like is to have a module like:
___
module app;

int square(int x) { return x*x; }

@name("square test 1")
unittest {
    assert(square(10) == 100);
}

@name("square test 2")
unittest {
    assert(square(5) == 25);
}

class Foo {
    void someFunc() { return; }

    @name("Foo test")
    unittest {
        assert(someFunc() == something);
    }
}
___

So as mentioned above, the first problem is that using ModuleInfo.unitTest returns an aggregated function of all 3 unit tests for that module, instead of each one individually, so the UDA information is lost. The second problem is that running __traits(getUnitTests, app) only returns the 2 tests at module scope, and not the one nested inside Foo. Even then, I haven't been able to figure out how to dynamically get the module symbol to pass to __traits inside of the Runtime.moduleUnitTester function.

Its possible that I've got some of this backwards and there is an easy solution. Its also possible that what i'm trying to do isn't currently possible.

Thanks in advance for any help.
--Matt

On Wednesday, 26 July 2017 at 03:27:30 UTC, Matthew Remmel wrote:
> I'd like to print out the name given to a unit test via a UDA, as the tests are being ran by the Runtime.moduleUnitTester. I'm working on writing a custom unit test runner. It would seem that the ModuleInfo.unitTest property is an aggregated function of all the individual unit test blocks in a module, and therefore the UDA value is lost.
>
> [...]


Do you know https://github.com/atilaneves/unit-threaded?

On 2017-07-26 05:27, Matthew Remmel wrote:

> So as mentioned above, the first problem is that using ModuleInfo.unitTest returns an aggregated function of all 3 unit tests for that module, instead of each one individually, so the UDA information is lost. The second problem is that running __traits(getUnitTests, app) only returns the 2 tests at module scope, and not the one nested inside Foo.

Unless you want to go with the approach Seb suggested, using unit-threaded, you need to recursively iterate the module to get all aggregates using __traits(allMembers) then use __traits(getUnitTests) for each aggregate to get all unit tests.

> Even then, I haven't been able to figure out how to dynamically get the module symbol to pass to __traits inside of the Runtime.moduleUnitTester function.

That's not possible, the UDAs are lost after compile time. Also, all the unit tests block are combined into one function per module, which is what Runtime.moduleUnitTester is running. The separate unit test blocks are gone at runtime so there's nothing to connect the UDAs to.

-- 
/Jacob Carlborg

On Wednesday, 26 July 2017 at 06:47:20 UTC, Jacob Carlborg wrote:
> On 2017-07-26 05:27, Matthew Remmel wrote:
>
>> [...]
>
> Unless you want to go with the approach Seb suggested, using unit-threaded, you need to recursively iterate the module to get all aggregates using __traits(allMembers) then use __traits(getUnitTests) for each aggregate to get all unit tests.
>
>> [...]
>
> That's not possible, the UDAs are lost after compile time. Also, all the unit tests block are combined into one function per module, which is what Runtime.moduleUnitTester is running. The separate unit test blocks are gone at runtime so there's nothing to connect the UDAs to.

Thanks for the info, I'll look into the threaded unit test library and see what they are doing, and how.

-Matt