dmd on Win32 has recently become much better at handling Templates and libs, but it still has some issues. I got lucky and isolated one of them, listed below:

Two modules, one called test.d and the other tester.d -- add test.d to a library called test.lib, and then compile+link tester.d against said lib. You should get a correctly linked app that runs and emits "hello".

============
module test;

extern (C) int printf (char*, ...);

class Test (T)
{
        final void show (T[] msg)
        {
                printf ("%.*s\n", msg);
        }
}

//Test!(char) Global;
==========


==========
module tester;

pragma (lib, "test.lib");

import test;

void main()
{
        auto t = new Test!(char);
        t.show ("hello");
}
==========

Now, remove the comment in test.d so that 'Global' is exposed, and rebuild the lib. You should now get linker errors of this nature:

>dmd tester
C:\d\dmd\bin\..\..\dm\bin\link.exe tester,,,user32+kernel32/noi;
OPTLINK (R) for Win32  Release 7.50B1
Copyright (C) Digital Mars 1989 - 2001  All Rights Reserved

tester.obj(tester)
 Error 42: Symbol Undefined _D4test11__T4TestTaZ4Test7__ClassZ
tester.obj(tester)
 Error 42: Symbol Undefined _D4test11__T4TestTaZ4Test4showMFAaZv
--- errorlevel 2


This error does not occur with dmd on linux, nor when using GDC on any supported platform that we've tried. It's only the dmd/Win32 combination

Please can we have this fixed soon?

The problem is that one of two kinds of COMDAT sections can be generated:

1) Only one COMDAT with a particular name may appear
2) Any number of COMDATs with the same name may appear, pick one and discard the others

Instantiating a template produces a COMDAT section. Since multiple modules may instantiate a template with the same arguments, without knowing about each other, option (2) is used.

When a module is put into a library, a dictionary is created for the library, essentially an associative array of object modules indexed by symbol names. COMDATs of option (1) get put into the dictionary, ones of option (2) do not. Why not? Because if there's more than one of (2), which object module do you pull in? No way to tell.

Thus, the problem you're seeing. The solution is:

1) have another global in module test that gets pulled in, thus also pulling the COMDAT along with it, and resolving the symbol.

2) explicitly link in module test

So, you might ask, why not just regenerate the template instantiation every time you use it? Because it would lead to a lot of object file bloat.

Mmm... yummy technical details :P

There seem to be quite a few problems/annoyances due to limitations in the object file formats.  It makes one wonder whether it wouldn't be a good idea to simply make a new one that properly supported D's feature set...

Of course, that would involve touching OPTLINK, something I gather no one is particularly keen on :P

	-- Daniel

Walter Bright wrote:
> The problem is that one of two kinds of COMDAT sections can be generated:
> 
> 1) Only one COMDAT with a particular name may appear
> 2) Any number of COMDATs with the same name may appear, pick one and
> discard the others
> 
> Instantiating a template produces a COMDAT section. Since multiple modules may instantiate a template with the same arguments, without knowing about each other, option (2) is used.
> 
> When a module is put into a library, a dictionary is created for the library, essentially an associative array of object modules indexed by symbol names. COMDATs of option (1) get put into the dictionary, ones of option (2) do not. Why not? Because if there's more than one of (2), which object module do you pull in? No way to tell.
> 
> Thus, the problem you're seeing. The solution is:
> 
> 1) have another global in module test that gets pulled in, thus also pulling the COMDAT along with it, and resolving the symbol.
> 
> 2) explicitly link in module test
> 
> So, you might ask, why not just regenerate the template instantiation every time you use it? Because it would lead to a lot of object file bloat.

-- 
Unlike Knuth, I have neither proven or tried the above; it may not even make sense.

v2sw5+8Yhw5ln4+5pr6OFPma8u6+7Lw4Tm6+7l6+7D i28a2Xs3MSr2e4/6+7t4TNSMb6HTOp5en5g6RAHCP  http://hackerkey.com/

"Daniel Keep" <daniel.keep.lists@gmail.com> wrote in message news:eret4n$2ors$1@digitalmars.com...
> Mmm... yummy technical details :P
>
> There seem to be quite a few problems/annoyances due to limitations in the object file formats.  It makes one wonder whether it wouldn't be a good idea to simply make a new one that properly supported D's feature set...

That would be awesome.  If D compilers could still generate some kind of "standard" object file for linking with other languages, and then a "D object" for D compilers that had extra features.  The D object format could even be standardized and made sure to be compatible across D compilers (something C++ sure can't promise!).

I always thought it would be cool if templates were sort of .. compiled almost into a scripting language intermediate representation.  Then, to instantiate a template, either one it just compiled or one it loaded from a D object, the compiler would just interpret the script.  This way a generic "how to instantiate template X" would be stored in the object file, and you wouldn't need the original definition.  This is probably a lot of work though.

> Of course, that would involve touching OPTLINK, something I gather no one is particularly keen on :P
>

Phh, if it means getting away from OMF, I'm all for it ;)

Walter Bright wrote:
> The problem is that one of two kinds of COMDAT sections can be generated:
> 
> 1) Only one COMDAT with a particular name may appear
> 2) Any number of COMDATs with the same name may appear, pick one and discard the others
> 
> Instantiating a template produces a COMDAT section. Since multiple modules may instantiate a template with the same arguments, without knowing about each other, option (2) is used.
> 
> When a module is put into a library, a dictionary is created for the library, essentially an associative array of object modules indexed by symbol names. COMDATs of option (1) get put into the dictionary, ones of option (2) do not. Why not? Because if there's more than one of (2), which object module do you pull in? No way to tell.

Let's back up for a second.  First, in the situation Kris mentioned, how many instances of Test!(char) exist altogether?  I had expected there to be two: one in test.lib and one in tester.obj.  But in this case I wouldn't expect the link error to occur, so perhaps you're saying that when the compiler sees "Test!(char) Global" in module test, it doesn't bother to create one in tester.obj?

Also, how do COMDATs differ from normal code blocks?  ie. Why is the linker able to resolve normal symbols in libraries but not templates? Wouldn't the symbol name in both cases be enough to sort things out?

> Thus, the problem you're seeing. The solution is:
> 
> 1) have another global in module test that gets pulled in, thus also pulling the COMDAT along with it, and resolving the symbol.
> 
> 2) explicitly link in module test
> 
> So, you might ask, why not just regenerate the template instantiation every time you use it? Because it would lead to a lot of object file bloat.

I guess this answers my question above: the compiler sees "Test!(char) Global" and doesn't bother to create another instance of the code.  But surely being able to create a functional application is preferable in this case.  Won't an optimizing linker throw out duplicates anyway?  Who cares if the object files are bloated, if that's the only workable option here?  Or perhaps this suggests the need for a D linker that builds a catalog of symbols inside libraries before linking instead of the behavior you describe above?


Sean

Walter Bright wrote:
> The problem is that one of two kinds of COMDAT sections can be generated:
> 
> 1) Only one COMDAT with a particular name may appear
> 2) Any number of COMDATs with the same name may appear, pick one and discard the others
> 
> Instantiating a template produces a COMDAT section. Since multiple modules may instantiate a template with the same arguments, without knowing about each other, option (2) is used.
> 
> When a module is put into a library, a dictionary is created for the library, essentially an associative array of object modules indexed by symbol names. COMDATs of option (1) get put into the dictionary, ones of option (2) do not. Why not? Because if there's more than one of (2), which object module do you pull in? No way to tell.
> 
> Thus, the problem you're seeing. The solution is:
> 
> 1) have another global in module test that gets pulled in, thus also pulling the COMDAT along with it, and resolving the symbol.
> 
> 2) explicitly link in module test
> 
> So, you might ask, why not just regenerate the template instantiation every time you use it? Because it would lead to a lot of object file bloat.


Would you perhaps explain why ELF does not have this problem? Thanks

Sean Kelly wrote:
> Let's back up for a second.  First, in the situation Kris mentioned, how many instances of Test!(char) exist altogether?

One.

> I had expected there to be two: one in test.lib and one in tester.obj.  But in this case I wouldn't expect the link error to occur, so perhaps you're saying that when the compiler sees "Test!(char) Global" in module test, it doesn't bother to create one in tester.obj?

That's right.

> Also, how do COMDATs differ from normal code blocks?

COMDATs are each placed in their own segment.

> ie. Why is the linker able to resolve normal symbols in libraries but not templates? 

Because normal symbols cannot appear multiple times (if they did, you get multiple definition errors at link time).

> Wouldn't the symbol name in both cases be enough to sort things out?

No.

> I guess this answers my question above: the compiler sees "Test!(char) Global" and doesn't bother to create another instance of the code.  But surely being able to create a functional application is preferable in this case.  Won't an optimizing linker throw out duplicates anyway?

The problem is not knowing which module to link in - after all, there are (possibly) other global symbols in the module.

> Who cares if the object files are bloated, if that's the only workable option here?

You'll care <g> once templates get complex enough. It's been a big problem with C++.

> Or perhaps this suggests the need for a D linker that builds a catalog of symbols inside libraries before linking instead of the behavior you describe above?

Building a linker or object file with non-standard semantics has its own set of problems.

kris wrote:
> Would you perhaps explain why ELF does not have this problem? Thanks

The ar format doesn't care if there are multiple definitions of the same symbol. The linker just picks one (presumably the first one it finds) and pulls in that module along with whatever else happens to be in that module.

Walter Bright wrote:

> kris wrote:
>> Would you perhaps explain why ELF does not have this problem? Thanks
> 
> The ar format doesn't care if there are multiple definitions of the same symbol. The linker just picks one (presumably the first one it finds) and pulls in that module along with whatever else happens to be in that module.

That's not entirely true. Symbols in ELF can be said to be either LOCAL, GLOBAL or WEAK. The local ones are per module, the global ones you can only have one of, whereas the weak ones you can have multiple of and one (probably the first, depending on linker etc) is chosen.

IIRC

-- 
Lars Ivar Igesund
blog at http://larsivi.net
DSource, #d.tango & #D: larsivi
Dancing the Tango

Walter Bright wrote:
> Sean Kelly wrote:
>> Who cares if the object files are bloated, if that's the only workable option here?
> 
> You'll care <g> once templates get complex enough. It's been a big problem with C++.

I'd prefer bloated object files over link errors as well...

Perhaps a command-line option could be added to emit templates in situations like this, so object files don't increase in size because of this unless necessary?