Lars Ivar Igesund wrote:
> Walter Bright wrote:
> 
>> kris wrote:
>>> Would you perhaps explain why ELF does not have this problem? Thanks
>> The ar format doesn't care if there are multiple definitions of the same
>> symbol. The linker just picks one (presumably the first one it finds)
>> and pulls in that module along with whatever else happens to be in that
>> module.
> 
> That's not entirely true. Symbols in ELF can be said to be either LOCAL,
> GLOBAL or WEAK. The local ones are per module, the global ones you can only
> have one of, whereas the weak ones you can have multiple of and one
> (probably the first, depending on linker etc) is chosen.

The template instances are put in the .o file as GLOBAL ones.

Frits van Bommel wrote:
> Walter Bright wrote:
>> You'll care <g> once templates get complex enough. It's been a big problem with C++.
> 
> I'd prefer bloated object files over link errors as well...
> 
> Perhaps a command-line option could be added to emit templates in situations like this, so object files don't increase in size because of this unless necessary?

My experience with those command line options is almost nobody is able to use them successfully. (They were used in the early days of C++ before COMDATs.)

Walter Bright wrote:
> Sean Kelly wrote:
> 
>> Who cares if the object files are bloated, if that's the only workable option here?
> 
> You'll care <g> once templates get complex enough. It's been a big problem with C++.
> 
>> Or perhaps this suggests the need for a D linker that builds a catalog of symbols inside libraries before linking instead of the behavior you describe above?
> 
> Building a linker or object file with non-standard semantics has its own set of problems.

Thanks for the reply.  Your proposed solution finally reminded me that we had this exact conversation a few years ago :-)  It's a nasty hack, but what if the compiler performed your proposed solution during compilation:

> 1) have another global in module test that gets pulled in, thus also
> pulling the COMDAT along with it, and resolving the symbol.

In modules containing global template instances, generate a symbol, let's say _Dforcelink, which is implicitly referenced in importing modules.  Is this possible?


Sean

Sean Kelly wrote:
> Thanks for the reply.  Your proposed solution finally reminded me that we had this exact conversation a few years ago :-)  It's a nasty hack, but what if the compiler performed your proposed solution during compilation:
> 
>  > 1) have another global in module test that gets pulled in, thus also
>  > pulling the COMDAT along with it, and resolving the symbol.
> 
> In modules containing global template instances, generate a symbol, let's say _Dforcelink, which is implicitly referenced in importing modules.  Is this possible?

What happens if you have two such modules in a library?

On Tue, 20 Feb 2007 20:46:39 +0200, Walter Bright <newshound@digitalmars.com> wrote:
> Sean Kelly wrote:
>> Let's back up for a second.  First, in the situation Kris mentioned, how many instances of Test!(char) exist altogether?
>
> One.
>
>> I had expected there to be two: one in test.lib and one in tester.obj.  But in this case I wouldn't expect the link error to occur, so perhaps you're saying that when the compiler sees "Test!(char) Global" in module test, it doesn't bother to create one in tester.obj?
>
> That's right.
>
[snip]

For a person which doesn't know how linkers and object files work, this seems to be an 'interesting' problem. <g>

I mean, there is an instance of Test!(char) in test.lib, but the linker doesn't found it because there might be another instance in another object file? And at the same time, the compiler tries to ensure that there will be only one instance of Test!(char)...

What will be the number of instances of Test!(char) if test.d does not contain 'Test!(char) Global;', and I have another module tester2.d, similar to tester.d, which also uses the Test!(char) class? I except two, one for tester.d and one for tester2.d. That is, at least, when the files are compiled independently. And if the compiler would be smart enough to create only one instance in that case, doesn't it mean the same linker error will occur then too?

Walter Bright wrote:
> kris wrote:
> 
>> Would you perhaps explain why ELF does not have this problem? Thanks
> 
> 
> The ar format doesn't care if there are multiple definitions of the same symbol. The linker just picks one (presumably the first one it finds) and pulls in that module along with whatever else happens to be in that module.

For whatever reason, that approach appears to work quite effectively?

The problem here is that one cannot currently build a production quality lib where templates are involved. It's not reasonable or feasible to expect a user to understand and compensate for the issues involved, nor is it entirely feasible to construct the lib in such a manner that the rather delicate balance is maintained. We wish it were.

The reality problem is this: the more extensive the lib, and the more use it makes of templates, the fragility of it appears to increase exponentially. As does the difficulty of reverting that fragility.

At this time, it is just not possible for us to construct a reliable Tango.lib which, given our collective D 'expertise', seems entirely bogus :)

Can you resolve it in some reliable and user-friendly manner?

Walter Bright wrote:
> Frits van Bommel wrote:
>> Walter Bright wrote:
>>> You'll care <g> once templates get complex enough. It's been a big problem with C++.
>>
>> I'd prefer bloated object files over link errors as well...
>>
>> Perhaps a command-line option could be added to emit templates in situations like this, so object files don't increase in size because of this unless necessary?
> 
> My experience with those command line options is almost nobody is able to use them successfully. (They were used in the early days of C++ before COMDATs.)

That raises an interesting question: seeing as how it's using the same backend, how does DMC handle this problem for C++?

-- 
- EricAnderton at yahoo

Walter Bright wrote:
> Sean Kelly wrote:
>> Thanks for the reply.  Your proposed solution finally reminded me that we had this exact conversation a few years ago :-)  It's a nasty hack, but what if the compiler performed your proposed solution during compilation:
>>
>>  > 1) have another global in module test that gets pulled in, thus also
>>  > pulling the COMDAT along with it, and resolving the symbol.
>>
>> In modules containing global template instances, generate a symbol, let's say _Dforcelink, which is implicitly referenced in importing modules.  Is this possible?
> 
> What happens if you have two such modules in a library?

Oops... the name would have to be mangled to include the module name.  I suppose this still raises issues for some corner-cases, but it's all I can think of.


Sean

Pragma wrote:
> That raises an interesting question: seeing as how it's using the same backend, how does DMC handle this problem for C++?

Unlike D, a C++ compiler has no idea what templates are instantiated in other files. So it instantiates everything. This is one reason why C++ builds are so much slower than D.

Walter Bright wrote:

> Pragma wrote:
>> That raises an interesting question: seeing as how it's using the same backend, how does DMC handle this problem for C++?
> 
> Unlike D, a C++ compiler has no idea what templates are instantiated in other files. So it instantiates everything. This is one reason why C++ builds are so much slower than D.

Shouldn't this reasoning make it easier to provide working and efficient template instances together with libs in D?

Also, on the topic of OMF, you suggested once that OMF was a better format than ELF? I've yet to see any justification for such a claim, currently it looks a tad behind the times.

-- 
Lars Ivar Igesund
blog at http://larsivi.net
DSource, #d.tango & #D: larsivi
Dancing the Tango