I'm referring to the issue raised by Tango developers about a TypeInfo for char[][] inflating the .EXE size by importing unneeded modules.

The solution given by Walter to this issue was carefully/painfully examination of object file symbols to determine the correct order of linking.

Are other solutions been planned or considered?

Because I don't think this is a long term solution for big projects, specially if an IDE is been used (Most of them don't even let you specify the compilation order of files).

I think it would be possible to build a tool that analyzes object files and determines the optimal order in most cases, but this seems like a hack on par with the moc compiler of the Qt project. A hack that's only needed to supply a deficiency of the compiler.

Nathan Petrelli wrote:
> I'm referring to the issue raised by Tango developers about a TypeInfo for char[][]
> inflating the .EXE size by importing unneeded modules.
> 
> The solution given by Walter to this issue was carefully/painfully examination
> of object file symbols to determine the correct order of linking.
> 
> Are other solutions been planned or considered?
> 
> Because I don't think this is a long term solution for big projects,
> specially if an IDE is been used (Most of them don't even let you specify the
> compilation order of files).
> 
> I think it would be possible to build a tool that analyzes object files and
> determines the optimal order in most cases, but this seems like a hack on par
> with the moc compiler of the Qt project. A hack that's only needed to supply
> a deficiency of the compiler.

This situation also only crops up when you're passing all the modules at once to dmd, and then putting the resulting object files into a library. Try compiling the modules independently when they are intended to be put in a library.

Walter Bright wrote:

> This situation also only crops up when you're passing all the modules at once to dmd, and then putting the resulting object files into a library. Try compiling the modules independently when they are intended to be put in a library.

Could you please be explicit about what the distinctions would be? And how it would affect template generation also?

Please assume I am a complete idiot, and lead me through step-by-step:

(a) what the implications are for discrete versus "batch" compilation

(b) how the different compilation approaches lead to differing results

(c) how templates are affected at each step

I'm hoping this will lead to a "comprehensive" set of instructions to help others create useful libs for WIn32 with DM tools. The longer and more detailed these intructions are, the better it will be for D

kris wrote:
> Walter Bright wrote:
> 
>> This situation also only crops up when you're passing all the modules at once to dmd, and then putting the resulting object files into a library. Try compiling the modules independently when they are intended to be put in a library.
> 
> Could you please be explicit about what the distinctions would be? And how it would affect template generation also?
> 
> Please assume I am a complete idiot, and lead me through step-by-step:
> 
> (a) what the implications are for discrete versus "batch" compilation
> 
> (b) how the different compilation approaches lead to differing results
> 
> (c) how templates are affected at each step
> 
> I'm hoping this will lead to a "comprehensive" set of instructions to help others create useful libs for WIn32 with DM tools. The longer and more detailed these intructions are, the better it will be for D
> 

When you compile:
	dmd -c a b
then dmd is assuming that a.obj and b.obj will be linked together, so it does not matter which object file something is placed in. In other words, it does not generate things twice.

On the other hand:
	dmd -c a
	dmd -c b
then dmd doesn't know, when compiling a.obj what will be in b.obj, so it assumes the worst and generates it.

In other words:
	dmd -c a b
	lib foo.lib a.obj b.obj
is not a good way to create a library, instead:
	dmd -c a
	dmd -c b
	lib foo.lib a.obj b.obj

Walter Bright wrote:
> kris wrote:
> 
>> Walter Bright wrote:
>>
>>> This situation also only crops up when you're passing all the modules at once to dmd, and then putting the resulting object files into a library. Try compiling the modules independently when they are intended to be put in a library.
>>
>>
>> Could you please be explicit about what the distinctions would be? And how it would affect template generation also?
>>
>> Please assume I am a complete idiot, and lead me through step-by-step:
>>
>> (a) what the implications are for discrete versus "batch" compilation
>>
>> (b) how the different compilation approaches lead to differing results
>>
>> (c) how templates are affected at each step
>>
>> I'm hoping this will lead to a "comprehensive" set of instructions to help others create useful libs for WIn32 with DM tools. The longer and more detailed these intructions are, the better it will be for D
>>
> 
> When you compile:
>     dmd -c a b
> then dmd is assuming that a.obj and b.obj will be linked together, so it does not matter which object file something is placed in. In other words, it does not generate things twice.
> 
> On the other hand:
>     dmd -c a
>     dmd -c b
> then dmd doesn't know, when compiling a.obj what will be in b.obj, so it assumes the worst and generates it.
> 
> In other words:
>     dmd -c a b
>     lib foo.lib a.obj b.obj
> is not a good way to create a library, instead:
>     dmd -c a
>     dmd -c b
>     lib foo.lib a.obj b.obj



What about (c) how templates are affected at each step ?

kris wrote:
> What about (c) how templates are affected at each step ?

It's the same algorithm - nothing special about templates.

Walter Bright wrote:
> kris wrote:
> 
>> What about (c) how templates are affected at each step ?
> 
> 
> It's the same algorithm - nothing special about templates.

Is it possible, do you think, to be just a little more forthcoming on this?

1) when you batch-compile code with multiple references to a template, there is just one instance generated.

2) when you compile the same code modules individually, there are presumably multiple template instances generated?

3) how does the linker resolve the multiple template instances to just one?

On Thu, 08 Mar 2007 12:36:24 -0800, Walter Bright wrote:

>> Walter Bright wrote:
>> 
>>> This situation also only crops up when you're passing all the modules at once to dmd, and then putting the resulting object files into a library. Try compiling the modules independently when they are intended to be put in a library.

> When you compile:
> 	dmd -c a b
> then dmd is assuming that a.obj and b.obj will be linked together, so it
> does not matter which object file something is placed in. In other
> words, it does not generate things twice.
> 
> On the other hand:
> 	dmd -c a
> 	dmd -c b
> then dmd doesn't know, when compiling a.obj what will be in b.obj, so it
> assumes the worst and generates it.
> 
> In other words:
> 	dmd -c a b
> 	lib foo.lib a.obj b.obj
> is not a good way to create a library, instead:
> 	dmd -c a
> 	dmd -c b
> 	lib foo.lib a.obj b.obj

One of the things that greatly impressed me was DMD's ability to quickly compile multiple files in one pass, rather than the make-like process on doing one file per DMD run. So when I came to write Bud, I made a lot of effort to ensure that I could compile as many as possible files in one call to the compiler.

It now seems that you are warning us against this feature of DMD, in the case of creating libraries. This is extremely disappointing.

I will add a new switch to Bud to force file-by-file compilation.

-- 
Derek Parnell
Melbourne, Australia
"Justice for David Hicks!"
skype: derek.j.parnell

Derek Parnell wrote:
> One of the things that greatly impressed me was DMD's ability to quickly
> compile multiple files in one pass, rather than the make-like process on
> doing one file per DMD run. So when I came to write Bud, I made a lot of
> effort to ensure that I could compile as many as possible files in one call
> to the compiler. 
> 
> It now seems that you are warning us against this feature of DMD, in the
> case of creating libraries. This is extremely disappointing.
> 
> I will add a new switch to Bud to force file-by-file compilation.

I don't see that as a hardship when buidling libs, since it perhaps doesn't happen as often as "regular" builds (assuming, of course, that this strategy actually resolves the underlying issue) ?

Having said that, the new switch will be *greatly* appreciated. Means we can avoid having to create and maintain the damn make-files. Thanks, Derek!

kris wrote:
> Walter Bright wrote:
>> kris wrote:
>>
>>> What about (c) how templates are affected at each step ?
>>
>>
>> It's the same algorithm - nothing special about templates.
> 
> Is it possible, do you think, to be just a little more forthcoming on this?
> 
> 1) when you batch-compile code with multiple references to a template, there is just one instance generated.

Yes.

> 2) when you compile the same code modules individually, there are presumably multiple template instances generated?

Yes.

> 3) how does the linker resolve the multiple template instances to just one?

The template instantiations are put into COMDAT sections, and the linker discards redundant ones.