Walter Bright wrote:
> kris wrote:
> 
>> Walter Bright wrote:
>>
>>> kris wrote:
>>>
>>>> There's no real surprise that this would happen, but it's the actual difference that is cause for a little concern -- it appears that each module consumes 512 bytes minimum. This may actually be a linker thing, but perhaps not?
>>>
>>>
>>>
>>> The way to see what's in an object file is to run obj2asm on it. It'll show exactly what's submitted to the linker.
>>>
>>
>> I'm aware of that, thanks.
> 
> 
> The words used suggested an unfamiliarity with the tools. 

Forgive me. They were chosen to cause the least amount of conflict? But, as far as OptLink goes -- yes, I have no clue about it, and there's more switches than a power-station. Best to ask the expert, it would seem?


> I believe it's well worth the effort to master what's going on with object files and linking, especially for professional developers, and the tools obj2asm, /MAP, and dumpexe are marvelous aids.

Thank you. Indeed they are. We've both been using these kinds of tools for approximately the same period of time.

> 
>> Do you think you could comment on why it might be the linker? And how to compensate? You're probably one of the two ppl in the world who know OptLink ...
> 
> 
> I'd first look at the contents of the .obj file, and see if that is what is expected. 

Did that first. There's nothing that jumps out. I should note that this was first noticed perhaps two years ago ... it's not something that suddenly changed.

But, it has become more important recently; vis-a-vis win32 headers


> I'd also check the optlink instructions http://www.digitalmars.com/ctg/ctgLinkSwitches.html#alignment
> as there are quite a lot of switches that offer a great deal of control over the linking process.

Yes, classic stuff! That's why I'm asking the expert.

> 
>> Also: are dmd obj files compatible with any other linker?
> 
> 
> Any linker that supports the Microsoft OMF format. I know Microsoft linkers dropped support for it when they went to 32 bits, but I am not very familiar with other linkers. Pharlap did, but I think they went out of business.

Thanks. This means it's not exactly feasible to check the concern via another linker.

Frits van Bommel wrote:
> Sean Kelly wrote:
>> Thomas Kuehne wrote:
>>> Every non-trivial module contains (numbers are for Linux)
>>> _D5module7__arrayZ
>>> 23 bytes code, 19 bytes stringtab, 18 bytes symtab
>>>
>>> _D5module8__assertFiZv
>>> 24 bytes code, 23 bytes stringtab, 18 bytes symtab
>>
>> These are the outstanding problem for exposing templates from library code.  And I don't understand why they are generated, since it seems like the code will be identical for each instance generated.  Couldn't they just have a static definition in the runtime?
> 
> In their current form they're not identical for each module, for the simple reason that the code (after linking) has the reference to the module name string hardcoded.
> For two extra instructions per call that could be avoided, though at that point you might as well just call _d_assert/_d_arraybounds directly instead of using an intermediary function...

Exactly my point.  Why not define _d_assert and _d_arraybounds somewhere and simply call those?  Since, as far as I can tell, the function bodies never vary.


Sean

Sean Kelly wrote:
> Frits van Bommel wrote:
>> Sean Kelly wrote:
>>> Thomas Kuehne wrote:
>>>> Every non-trivial module contains (numbers are for Linux)
>>>> _D5module7__arrayZ
>>>> 23 bytes code, 19 bytes stringtab, 18 bytes symtab
>>>>
>>>> _D5module8__assertFiZv
>>>> 24 bytes code, 23 bytes stringtab, 18 bytes symtab
>>>
>>> These are the outstanding problem for exposing templates from library code.  And I don't understand why they are generated, since it seems like the code will be identical for each instance generated.  Couldn't they just have a static definition in the runtime?
>>
>> In their current form they're not identical for each module, for the simple reason that the code (after linking) has the reference to the module name string hardcoded.
>> For two extra instructions per call that could be avoided, though at that point you might as well just call _d_assert/_d_arraybounds directly instead of using an intermediary function...
> 
> Exactly my point.  Why not define _d_assert and _d_arraybounds somewhere 

Please look at phobos/std/asserterror.d and phobos/std/array.d[1]. I didn't just pick those names out of a hat ;).
What the generated functions do is basically:
asm {
    push EAX;           // caller puts line number there
    push name_ptr;
    push name_length;
    call _d_assert;     // or _d_array_bounds
}
Like I said, this can easily be inlined. Replacing "mov EAX, linenr" with three pushes and using a different address is all it takes...

In fact, it would seem calls to _d_assert_msg are already done like this (for asserts with the optional char[] second argument).


[1]: It would seem I made a typo, it's _d_array_bounds.

> and simply call those?  Since, as far as I can tell, the function bodies never vary.

== Quote from kris (foo@bar.com)'s article
> >> Also: are dmd obj files compatible with any other linker?
> >
> >
> > Any linker that supports the Microsoft OMF format. I know Microsoft linkers dropped support for it when they went to 32 bits, but I am not very familiar with other linkers. Pharlap did, but I think they went out of business.
> Thanks. This means it's not exactly feasible to check the concern via another linker.

It's probably a longshot that wouldn't work, but it's possible that the Standalone
OpenWatcom Tools would be of some use:
http://cmeerw.org/prog/owtools/

I haven't tried using any of the tools myself (and the webpage implies that development stopped in November 2003), but I thought I'd point this page out in case it's of use to you. It mentions: "I am currently working on getting OpenWatcom's tools working with Digital Mars C++ to provide an alternative to the already dated Digital Mars tools." And one of the tools is called "Open Watcom OMF Dump Utility".

jcc7

jcc7 wrote:
> == Quote from kris (foo@bar.com)'s article
> 
>>>>Also: are dmd obj files compatible with any other linker?
>>>
>>>
>>>Any linker that supports the Microsoft OMF format. I know Microsoft
>>>linkers dropped support for it when they went to 32 bits, but I am not
>>>very familiar with other linkers. Pharlap did, but I think they went out
>>>of business.
>>
>>Thanks. This means it's not exactly feasible to check the concern via
>>another linker.
> 
> 
> It's probably a longshot that wouldn't work, but it's possible that the Standalone
> OpenWatcom Tools would be of some use:
> http://cmeerw.org/prog/owtools/
> 
> I haven't tried using any of the tools myself (and the webpage implies that
> development stopped in November 2003), but I thought I'd point this page out in
> case it's of use to you. It mentions: "I am currently working on getting
> OpenWatcom's tools working with Digital Mars C++ to provide an alternative to the
> already dated Digital Mars tools." And one of the tools is called "Open Watcom OMF
> Dump Utility".
> 
> jcc7

That's a good pointer -- thanks, jcc7 :)

On Thu, 25 Jan 2007 15:49:28 +0000, jcc7 wrote:

> == Quote from kris (foo@bar.com)'s article
>> >> Also: are dmd obj files compatible with any other linker?
>> >
>> >
>> > Any linker that supports the Microsoft OMF format. I know Microsoft linkers dropped support for it when they went to 32 bits, but I am not very familiar with other linkers. Pharlap did, but I think they went out of business.
>> Thanks. This means it's not exactly feasible to check the concern via another linker.
> 
> It's probably a longshot that wouldn't work, but it's possible that the Standalone
> OpenWatcom Tools would be of some use:
> http://cmeerw.org/prog/owtools/
> 
> I haven't tried using any of the tools myself (and the webpage implies that development stopped in November 2003), but I thought I'd point this page out in case it's of use to you. It mentions: "I am currently working on getting OpenWatcom's tools working with Digital Mars C++ to provide an alternative to the already dated Digital Mars tools." And one of the tools is called "Open Watcom OMF Dump Utility".
> 
> jcc7


That statement has been there for years.  I'm not sure if anywone has
found any benefit to using the modified openwatcom linker.  There doesn't
seem to be any ongoing development for dmc compatibility.  The one
benefit (and primary goal, I think) was the ability to link with
coff files, but since dmc/dmd spits out omf only, there's not much
advantage beyond that... and then even coff files from different compiler
venders are incompatible. Object formats have been a mess on windows for
well over a decade...

dmd's largest problem here continues to be the omf format.  On the other hand, dmd would do well to have a new linker and object system to help it support new language features.  Depending on old C technology is restrictive to D's progress.  Alas, easier said than done.

-JJR

Frits van Bommel wrote:
> Sean Kelly wrote:
>> Frits van Bommel wrote:
>>> Sean Kelly wrote:
>>>> Thomas Kuehne wrote:
>>>>> Every non-trivial module contains (numbers are for Linux)
>>>>> _D5module7__arrayZ
>>>>> 23 bytes code, 19 bytes stringtab, 18 bytes symtab
>>>>>
>>>>> _D5module8__assertFiZv
>>>>> 24 bytes code, 23 bytes stringtab, 18 bytes symtab
>>>>
>>>> These are the outstanding problem for exposing templates from library code.  And I don't understand why they are generated, since it seems like the code will be identical for each instance generated.  Couldn't they just have a static definition in the runtime?
>>>
>>> In their current form they're not identical for each module, for the simple reason that the code (after linking) has the reference to the module name string hardcoded.
>>> For two extra instructions per call that could be avoided, though at that point you might as well just call _d_assert/_d_arraybounds directly instead of using an intermediary function...
>>
>> Exactly my point.  Why not define _d_assert and _d_arraybounds somewhere 
> 
> Please look at phobos/std/asserterror.d and phobos/std/array.d[1]. I didn't just pick those names out of a hat ;).

It's been too long since I've messed with this portion of Phobos.  I knew they rang a bell! :-p

> What the generated functions do is basically:
> asm {
>     push EAX;           // caller puts line number there
>     push name_ptr;
>     push name_length;
>     call _d_assert;     // or _d_array_bounds
> }
> Like I said, this can easily be inlined. Replacing "mov EAX, linenr" with three pushes and using a different address is all it takes...
> 
> In fact, it would seem calls to _d_assert_msg are already done like this (for asserts with the optional char[] second argument).

Makes perfect sense.  Well... doing this would eliminate one of the last quantifiable issues with placing templates in a library, so it has my vote.


Sean

Sean Kelly wrote:
> Frits van Bommel wrote:
> 
>> Sean Kelly wrote:
>>
>>> Frits van Bommel wrote:
>>>
>>>> Sean Kelly wrote:
>>>>
>>>>> Thomas Kuehne wrote:
>>>>>
>>>>>> Every non-trivial module contains (numbers are for Linux)
>>>>>> _D5module7__arrayZ
>>>>>> 23 bytes code, 19 bytes stringtab, 18 bytes symtab
>>>>>>
>>>>>> _D5module8__assertFiZv
>>>>>> 24 bytes code, 23 bytes stringtab, 18 bytes symtab
>>>>>
>>>>>
>>>>> These are the outstanding problem for exposing templates from library code.  And I don't understand why they are generated, since it seems like the code will be identical for each instance generated.  Couldn't they just have a static definition in the runtime?
>>>>
>>>>
>>>> In their current form they're not identical for each module, for the simple reason that the code (after linking) has the reference to the module name string hardcoded.
>>>> For two extra instructions per call that could be avoided, though at that point you might as well just call _d_assert/_d_arraybounds directly instead of using an intermediary function...
>>>
>>>
>>> Exactly my point.  Why not define _d_assert and _d_arraybounds somewhere 
>>
>>
>> Please look at phobos/std/asserterror.d and phobos/std/array.d[1]. I didn't just pick those names out of a hat ;).
> 
> 
> It's been too long since I've messed with this portion of Phobos.  I knew they rang a bell! :-p
> 
>> What the generated functions do is basically:
>> asm {
>>     push EAX;           // caller puts line number there
>>     push name_ptr;
>>     push name_length;
>>     call _d_assert;     // or _d_array_bounds
>> }
>> Like I said, this can easily be inlined. Replacing "mov EAX, linenr" with three pushes and using a different address is all it takes...
>>
>> In fact, it would seem calls to _d_assert_msg are already done like this (for asserts with the optional char[] second argument).
> 
> 
> Makes perfect sense.  Well... doing this would eliminate one of the last quantifiable issues with placing templates in a library, so it has my vote.
> 
> 
> Sean


Mine too

John Reimer wrote:
>
> dmd's largest problem here continues to be the omf format.  On the
> other hand, dmd would do well to have a new linker and object system to
> help it support new language features.

As someone who has hacked away on OMF handling for a while now, I must agree. It has quite a few strikes against it for prospective D toolchain developers:

* Hard to follow documentation that contains errors and makes *lots* of assumptions
* Is unsupported by just about everyone else except DigitalMars (no offense intended)
* Is *not* 64-bit ready
* Contains gobs of legacy cruft (specification moreso than .obj files)
* Is limited in how it can represent module inter-dependencies

Don't get me wrong. I like DMD and OPTLINK in particular - I would love to see OPTLINK modified to work with ELF files or some new D object file format.

-- 
- EricAnderton at yahoo

Sean Kelly wrote:
> Makes perfect sense.  Well... doing this would eliminate one of the last quantifiable issues with placing templates in a library, so it has my vote.

Well, there's still another bug that seems to crop up in source files with only templates: Good ol' bug #22 (http://d.puremagic.com/issues/show_bug.cgi?id=22)
Ran into that one again today. But you should know about it, since you were the one to report it.
Though perhaps that one is more of a general templates problem, unrelated to libraries. Still annoying though.