On 12/18/2016 06:53 PM, Andrei Alexandrescu wrote:
> Compulsive ModuleInfo generation seems to be a drag. I'm not very
> familiar with the particulars but my understanding is ModuleInfo is
> needed for (a) Object.factory and (b) finding and running unittests.
> 
> Walter and I think Object.factory was a mistake and we'd like to make it opt-in long term (either by means of a build flag or a class attribute).
> 
> Until then, a very nice step forward is to NOT generate ModuleInfo if a module introduces no class.
> 
> For unittests, I figure things like static introspection should make it easier to enumerate and run unittests without a need for ModuleInfo.
> 
> What other issues/opportunities do you see related to ModuleInfo?
> 
> 
> Thanks,
> 
> Andrei

We rely on ModuleInfo for custom test runner in ocean
(https://github.com/sociomantic-tsunami/ocean/blob/v2.x.x/src/ocean/core/UnitTestRunner.d).
Static introspection can't provide same functionality in general case
(even if all bugs will be fixed) because it requires either maintaining
list of all modules or ensuring everything is imported from some root
module. Former is inconvenience, latter is simply not true for libraries
(though can be worked around by build system).

On Monday, 19 December 2016 at 18:29:13 UTC, Andrei Alexandrescu wrote:
> On 12/19/16 12:25 PM, Atila Neves wrote:
>> On Monday, 19 December 2016 at 17:11:38 UTC, Adam D. Ruppe wrote:
>>> [...]
>>
>> I'd forgotten about separate compilation. The issue there is the
>> algorithm for naming unittest blocks gives different results depending
>> on how the code is compiled, so __traits(getUnitTests) unfortunately
>> currently only works with "compile all the things!". I'd forgotten I
>> wanted to change the compiler code responsible for the naming.
>
> Interesting, could you please add a detailed issue to bugzilla? Thanks! -- Andrei

https://issues.dlang.org/show_bug.cgi?id=16995

Atila

On 12/20/2016 12:01 AM, Dicebot wrote:
> We rely on ModuleInfo for custom test runner in ocean
> (https://github.com/sociomantic-tsunami/ocean/blob/v2.x.x/src/ocean/core/UnitTestRunner.d).
> Static introspection can't provide same functionality in general case
> (even if all bugs will be fixed) because it requires either maintaining
> list of all modules or ensuring everything is imported from some root
> module. Former is inconvenience, latter is simply not true for libraries
> (though can be worked around by build system).

Does that simply need to find all the unit tests? (There are ways to do that without ModuleInfo, it's just that it's convenient and portable with ModuleInfo.)

On 12/20/2016 01:15 PM, Atila Neves wrote:
> On Monday, 19 December 2016 at 18:29:13 UTC, Andrei Alexandrescu wrote:
>> On 12/19/16 12:25 PM, Atila Neves wrote:
>>> On Monday, 19 December 2016 at 17:11:38 UTC, Adam D. Ruppe wrote:
>>>> [...]
>>>
>>> I'd forgotten about separate compilation. The issue there is the
>>> algorithm for naming unittest blocks gives different results depending
>>> on how the code is compiled, so __traits(getUnitTests) unfortunately
>>> currently only works with "compile all the things!". I'd forgotten I
>>> wanted to change the compiler code responsible for the naming.
>>
>> Interesting, could you please add a detailed issue to bugzilla?
>> Thanks! -- Andrei
>
> https://issues.dlang.org/show_bug.cgi?id=16995

Gracias! -- Andrei

On 12/20/2016 08:33 PM, Walter Bright wrote:
> On 12/20/2016 12:01 AM, Dicebot wrote:
>> We rely on ModuleInfo for custom test runner in ocean (https://github.com/sociomantic-tsunami/ocean/blob/v2.x.x/src/ocean/core/UnitTestRunner.d).
>>
>> Static introspection can't provide same functionality in general case (even if all bugs will be fixed) because it requires either maintaining list of all modules or ensuring everything is imported from some root module. Former is inconvenience, latter is simply not true for libraries (though can be worked around by build system).
> 
> Does that simply need to find all the unit tests? (There are ways to do that without ModuleInfo, it's just that it's convenient and portable with ModuleInfo.)

Yes, pretty much. What ways do you have in mind? I am only aware of two:

1) ModuleInfo
2) https://dlang.org/spec/traits.html#getUnitTests

On 12/20/2016 11:05 AM, Dicebot wrote:
> Yes, pretty much. What ways do you have in mind? I am only aware of two:
>
> 1) ModuleInfo
> 2) https://dlang.org/spec/traits.html#getUnitTests


Put pointers to them in a special segment.

Am Tue, 20 Dec 2016 12:36:53 -0800
schrieb Walter Bright <newshound2@digitalmars.com>:

> On 12/20/2016 11:05 AM, Dicebot wrote:
> > Yes, pretty much. What ways do you have in mind? I am only aware of two:
> >
> > 1) ModuleInfo
> > 2) https://dlang.org/spec/traits.html#getUnitTests
> 
> 
> Put pointers to them in a special segment.

You need some kind of linker support to do this to provide the start/end symbols. Binutils has got a nice feature to do that automatically for you though: https://sourceware.org/binutils/docs/ld/Orphan-Sections.html

" If an orphaned section's name is representable as a C identifier then the linker will automatically see PROVIDE two symbols: __start_SECNAME and __stop_SECNAME, where SECNAME is the name of the section."


However, things get a little more complicated when shared libraries are involved. You'll have to make the __start/__stop symbols private to the library (that's easy) and somehow provide a function in every library to access the library private __start/__stop symbols. That's basically how we assemble the list of moduleinfos in rt.sections.



Back to topic: I'd really love to see a generalization of RTInfo/mixin templates in D:
----------------------------------------------------------------------
@auto mixin template scanUnittests(Module)
{
    static if (hasUnittest!Module)
    {
        static this()
        {
            foreach(test, getUnittests!Module)
                test();
        }
    }
}
----------------------------------------------------------------------

Whenever you import a module containing an @auto mixin the compiler
would mixin the declaration into the module. This should be
incredibly powerful for serialization, std.benchmark and all kind of
introspection tasks. (You then still need some way to pass this
information to the 'runtime' world. Either static this, C-style ctors
or custom data sections are possibilities. OTOH a mixin can also define
members that are accessible from the outside if the module name is
known)

On 12/20/2016 10:36 PM, Walter Bright wrote:
> On 12/20/2016 11:05 AM, Dicebot wrote:
>> Yes, pretty much. What ways do you have in mind? I am only aware of two:
>>
>> 1) ModuleInfo
>> 2) https://dlang.org/spec/traits.html#getUnitTests
> 
> 
> Put pointers to them in a special segment.

Oh, so you have meant "other ways can be implemented", not "other ways exist"? Sure. It does need to include qualified module names in that info though to preserve existing test runner functionality.

On 2016-12-21 18:43, Johannes Pfau wrote:

> Back to topic: I'd really love to see a generalization of RTInfo/mixin
> templates in D:

I implemented RTInfo for modules a couple of years ago [1]. Unfortunately it was rejected because it had the same problem as RTInfo, it only works inside object.d. I had some ideas for that as well but it was not liked either.

[1] https://github.com/dlang/dmd/pull/2271

-- 
/Jacob Carlborg

On 12/21/2016 9:43 AM, Johannes Pfau wrote:
> You need some kind of linker support to do this to provide the
> start/end symbols.

That's partially correct. I've done this for decades by having the compiler itself emit those symbols.

There are other tricks one can do, such as putting the pointers into the exception handler tables sections.