I am currently investigating http://d.puremagic.com/issues/show_bug.cgi?id=9571 and after brief exploration of symbols emitted to object file as well as code that emits them I can't rid of an idea that I am missing something about module system.

Why does dmd emits horde of symbols from imported modules into object files, most of whose are not actually used in this one? No "-inline" used and I kind of expected to see there only directly called function and symbols from their signatures.

What does actually happen here?

On Thursday, 4 April 2013 at 19:34:06 UTC, Dicebot wrote:
> I am currently investigating http://d.puremagic.com/issues/show_bug.cgi?id=9571 and after brief exploration of symbols emitted to object file as well as code that emits them I can't rid of an idea that I am missing something about module system.
>
> Why does dmd emits horde of symbols from imported modules into object files, most of whose are not actually used in this one? No "-inline" used and I kind of expected to see there only directly called function and symbols from their signatures.
>
> What does actually happen here?

Okay, after some digging on the topic I am quite sure that DMD does add instantiated template symbols from imported modules to root module symbol list (template.c:5023). It does not care if root module was actually one to start template instantiation chain (and thus uses this symbol) and just bloats the symbol table.

Repeating initial question - was there rationale for this other than being reluctant to store additional context information about initial template chain instantiation scope?

I'd like to make a pull request for this and make one step closer to making separate compilation reality but don't want to waste time on dmd source code (argh) without real need, thus preliminary question.

I am really gonna rise that up until I get some answer.
"GTFO" is acceptable answer and still better than no answer.

.

Sorry I have no answer for the issue. That is still unknown area to me...

Kenji Hara


2013/4/5 Dicebot <m.strashun@gmail.com>

> I am currently investigating http://d.puremagic.com/issues/**
> show_bug.cgi?id=9571 <http://d.puremagic.com/issues/show_bug.cgi?id=9571>and after brief exploration of symbols emitted to object file as well as
> code that emits them I can't rid of an idea that I am missing something
> about module system.
>
> Why does dmd emits horde of symbols from imported modules into object files, most of whose are not actually used in this one? No "-inline" used and I kind of expected to see there only directly called function and symbols from their signatures.
>
> What does actually happen here?
>

On Wednesday, 10 April 2013 at 08:41:35 UTC, kenji hara wrote:
> Sorry I have no answer for the issue. That is still unknown area to me...
>
> Kenji Hara

Thanks for replying! Really appreciated.

May be you can give me a small insight on dmd internal data representation then? I think I have got the issue and know what needs to be done, but can't find reasonably short solution within existing code structure.

2013/4/10 Dicebot <m.strashun@gmail.com>

> On Wednesday, 10 April 2013 at 08:41:35 UTC, kenji hara wrote:
>
>> Sorry I have no answer for the issue. That is still unknown area to me...
>>
>> Kenji Hara
>>
>
> Thanks for replying! Really appreciated.
>
> May be you can give me a small insight on dmd internal data representation then? I think I have got the issue and know what needs to be done, but can't find reasonably short solution within existing code structure.
>

OK. I don't understand the issue enough, but I'd like to help you.

- TemplateInstance::tinst may represent the enclosing template instance that 'this' instance is instantiated. It is set in TemplateInstance::semantic(|2|3) for its member's semantic process.

- TemplateInstance::enclosing may represent the parent of nested template instance. If a template is instantiated in function scope with implicit context, it points the function. It is set in TemplateInstance::hasNestedArgs.

- Module::importedFrom may represents the "root module" of import chain. it points one of the module that listed in command line.

Good luck.

Kenji Hara

On Wednesday, 10 April 2013 at 09:37:44 UTC, kenji hara wrote:
> OK. I don't understand the issue enough, but I'd like to help you.
>
> - TemplateInstance::tinst may represent the enclosing template instance
> that 'this' instance is instantiated. It is set in
> TemplateInstance::semantic(|2|3) for its member's semantic process.
>
> - TemplateInstance::enclosing may represent the parent of nested template
> instance. If a template is instantiated in function scope with implicit
> context, it points the function. It is set in
> TemplateInstance::hasNestedArgs.
>
> - Module::importedFrom may represents the "root module" of import chain. it
> points one of the module that listed in command line.
>
> Good luck.
>
> Kenji Hara

I see no TemplateInstance::enclosing, only Scope::enclosing and TemplateInstance is not Scope descendant. Essentially I need to emit symbols not for "importedFrom" module but for module of top-most tinst in chain. I supposed that iterating tinst can do the trick, but it is not. For example, when map is used, it call templated range functions somewhere down the line and iterating up from those don't get you initial map call scope. (Same issue in printInstantiationTrace as it uses this method).

Is this a problem with setting correct tinst initialization in other code, my misunderstanding or, probably, this info may be relied upon only in semantic3, not semantic1?

TemplateInstance::enclosing, also does not get you the instantiating context, referring to the comment in the source code, I think for my tests it was null.

importedFrom was no solution either. The only thing that worked for me was walking up with tinst and then use loc. This way you get the instantiating file. (Got that from printInstantiationTrace) I think it should be possible to use this found file name to find the module by iterating Module::modules, but I have not even bothered trying, because I felt there has to be some better way, unfortunately I also got no replies.

At least it helps that I am not alone with this problem :-)

Maybe it is not entirely unlikely that there is in fact no better way, as apart from "where do I put the instantiated template" and error messages there is really no need to know where the template got instantiated.


Best regards,
Robert

On Wednesday, 10 April 2013 at 11:53:51 UTC, Robert wrote:
> TemplateInstance::enclosing, also does not get you the instantiating context, referring to the comment in the source code, I think for my tests it was null.
>
> importedFrom was no solution either. The only thing that worked for me was walking up with tinst and then use loc. This way you get the instantiating file. (Got that from printInstantiationTrace) I think it should be possible to use this found file name to find the module by iterating Module::modules, but I have not even bothered trying, because I felt there has to be some better way, unfortunately I also got no replies.
>
> At least it helps that I am not alone with this problem :-)
>
> Maybe it is not entirely unlikely that there is in fact no better way, as apart from "where do I put the instantiated template" and error messages there is really no need to know where the template got instantiated.

Excited to see I am not the only one caring about this! :) Module* can be found via tinst->scope->module, so it is pretty straightforward. I have tried to use this and it _almost_ worked but some symbols were still missing (I have mentioned example with map and range before, can reduce test case if you are interested).

importFrom is obvious hack instead of precisely tracking symbol source - "lets just emit weak symbols for every compiled file from all his imports and let linker clean up the mess".

Considering Kenji's explanation this seems to work as I have initially understood. Then I need to reduce that case and see why those std.range template symbols are not propagated to the top of the chain.

This is a major blocker for getting reliable and efficient separate compilation and bugzilla issue references in topic is just an observable side-effect.