Sean Kelly wrote:
> Walter Bright wrote:
>> Sean Kelly wrote:
>>> The first block goes from line 410 to line 6719 in the map file, and the second block goes from line 8047 to line 14559 in the map file.  All told, that's 12821 separate constants or initializers in the manually linked version that seem absent from the library-based version.
>>
>> That means that, in the library based version, nothing in the explicitly linked .obj files referenced them.
> 
> If that's true then nothing in either version referenced them.  The application is identical in both cases but for some of the objects living in a library in the small case.

Just some further discussion.  After exhaustive experimentation yesterday with Tango the following seems to be true:

Using the Bud/Build approach of compiling all modules to object files and then linking them always works, however on Win32 it may result in a surplus of data in the executable.  This may actually be legitimate behavior based on how the linker works and how the OMF blocks are segmented in the object files at link time, but it seems odd.

Using the library approach of compiling all library code into a static library and then linking it does not always work, but it results in much leaner executables on Win32.  Little or no surplus data is added to the executable, including data that is added using the Bud/Build method, even for the exact same application.  This may again be legitimate behavior based on how the lib tool splits/joins/moves OMF blocks when assembling the library (assuming it does so), or it may have something to do with the linker having "all the information at its fingertips" when assembling the application.

Where the library approach fails is if the library contains template code.  Results are inconsistent, but yesterday we were seeing link errors in some cases and GPFs in others just for different library modules using the same template code, all of which work perfectly using the Bud/Build method.  For the link errors, as near as we could tell the object files contained all the relevant functions (many of the link errors referred to class TypeInfo instances) but for some reason they didn't appear to be available at link time.  It is possible that the lib tool is throwing them out for some reason, but I could not speculate on why.

This leaves us in an awkward situation for releasing Tango.  Suggesting everyone use Bud/Build seems the most reliable but some may not be happy with large executables or with having to use Bud/Build at all. Assembling everything into a single static library seems unreliable at best, but it will undoubtedly be preferred by many people.  At the moment, I suspect we will adopt a hybrid approach out of necessity, placing some of the larger packages (like the Win32 package) in static libraries and linking them either explicitly or implicitly via pragma(lib) (which is problematic in itself since it doesn't work if placed in include files, at least with DMD--Bud may be different, I have yet to test this and plan to do so today).

I am hoping that we can produce a small test case for the template issues so we can move to a full library approach once the problems have been addressed.  Alternately, perhaps we can find a way to address linking so that less unnecessary data is included while linking objects directly.  I'll admit that at this point I'm of half a mind to simply write an OMF linker that takes as long as necessary to produce a working application and be done with it.

Out of curiosity, I suppose it would be a substantial amount of work for DMD (and I suppose DMC) to produce COFF object files instead of OMF? And I don't suppose that some work in this area may be necessary for generating 64-bit executables?  If there's a chance we could move to a newer object file format for Win32, we would at least gain access to a broader array of tools, even if COFF itself is a bit problematic.


Sean