January 18, 2012 Re: Program size, linking matter, and static this() | ||||
---|---|---|---|---|
| ||||
Posted in reply to Marco Leise | On 1/18/2012 1:43 AM, Marco Leise wrote: > It is back again! The following struct in my main module increases the > executable size by 10MB with DMD 2.075: > > struct Test { > byte abcd[10 * 1024 * 1024]; > } Compiling it and obj2asm'ing the result, and you'll see it goes into the BSS segment: _TEXT segment dword use32 public 'CODE' ;size is 0 _TEXT ends _DATA segment para use32 public 'DATA' ;size is 12 _DATA ends CONST segment para use32 public 'CONST' ;size is 0 CONST ends _BSS segment para use32 public 'BSS' ;size is 10485760 _BSS ends FLAT group extrn _D19TypeInfo_S3foo4Test6__initZ public _D3foo4Test6__initZ FMB segment dword use32 public 'DATA' ;size is 0 FMB ends FM segment dword use32 public 'DATA' ;size is 4 FM ends FME segment dword use32 public 'DATA' ;size is 0 FME ends extrn _D15TypeInfo_Struct6__vtblZ public _D3foo12__ModuleInfoZ _D19TypeInfo_S3foo4Test6__initZ COMDAT flags=x0 attr=x10 align=x0 _TEXT segment assume CS:_TEXT _TEXT ends _DATA segment _D3foo12__ModuleInfoZ: db 004h,000h,000h,0ffffff80h,000h,000h,000h,000h ;........ db 066h,06fh,06fh,000h ;foo. _DATA ends CONST segment CONST ends _BSS segment _BSS ends FMB segment FMB ends FM segment dd offset FLAT:_D3foo12__ModuleInfoZ FM ends FME segment FME ends _D19TypeInfo_S3foo4Test6__initZ comdat dd offset FLAT:_D15TypeInfo_Struct6__vtblZ db 000h,000h,000h,000h ;.... db 008h,000h,000h,000h ;.... dd offset FLAT:_D19TypeInfo_S3foo4Test6__initZ[03Ch] db 000h,000h,0ffffffa0h,000h,000h,000h,000h,000h ;........ db 000h,000h,000h,000h,000h,000h,000h,000h ;........ db 000h,000h,000h,000h,000h,000h,000h,000h ;........ db 000h,000h,000h,000h,000h,000h,000h,000h ;........ db 000h,000h,000h,000h,000h,000h,000h,000h ;........ db 001h,000h,000h,000h,066h,06fh,06fh,02eh ;....foo. db 054h,065h,073h,074h,000h ;Test. _D19TypeInfo_S3foo4Test6__initZ ends end ------------------------------------------------- Adding a void main(){} yields an executable of 145,948 bytes. |
January 18, 2012 Re: Program size, linking matter, and static this() | ||||
---|---|---|---|---|
| ||||
Posted in reply to Walter Bright | Am 18.01.2012, 11:18 Uhr, schrieb Walter Bright <newshound2@digitalmars.com>:
> On 1/18/2012 1:43 AM, Marco Leise wrote:
>> It is back again! The following struct in my main module increases the
>> executable size by 10MB with DMD 2.075:
>>
>> struct Test {
>> byte abcd[10 * 1024 * 1024];
>> }
>
> Compiling it and obj2asm'ing the result, and you'll see it goes into the BSS segment:
>
>
> [...]
>
> Adding a void main(){} yields an executable of 145,948 bytes.
Thanks for checking back. I'll have to experiment a bit to narrow this one down. It comes and goes like a ghost. I was using Linux 64-bit and the switches -O -release on a medium size code base.
|
January 18, 2012 Re: Program size, linking matter, and static this() | ||||
---|---|---|---|---|
| ||||
> Yes they are. static constructors completely chicken out on them. Not only is > there no real attempt to determine whether the static constructors are > actually dependent (which granted, isn't an easy problem), but there is _zero_ > support in the language for resolving such circular dependencies. There's no > way to say that they _aren't_ dependent even if you can clearly see that they > aren't. The solution used in Phobos (which won't work in std.datetime due to > the use of immutable and pure) is to create a C module which has the code from > the static constructor and then have a separate module which calls it in its > static constructor. Which is a hack because that C function is a compiler wall while the dependency persists. Btw. that stdiobase and datebase are obsolete the cycles have vanished. You will get this only if std.dateparse had a shared static ctor too. Cycle detected between modules with ctors/dtors: std.date -> std.dateparse -> std.date object.Exception@src/rt/minfo.d(309): Aborting! There is a cleaner hack to solve the issue but I really don't like it. It's two DAGs that are iterated one for "shared static this" and one for "static this". ---- module a; import b; shared static this() { } ---- module b; import a, core.atomic : cas; shared bool initialized; static this() { if (!cas(&initialized, false, true)) return; ... } ---- |
January 18, 2012 Re: Program size, linking matter, and static this() | ||||
---|---|---|---|---|
| ||||
On Wed, 18 Jan 2012 12:14:07 +0100, Martin Nowak <dawg@dawgfoto.de> wrote: >> Yes they are. static constructors completely chicken out on them. Not only is >> there no real attempt to determine whether the static constructors are >> actually dependent (which granted, isn't an easy problem), but there is _zero_ >> support in the language for resolving such circular dependencies. There's no >> way to say that they _aren't_ dependent even if you can clearly see that they >> aren't. The solution used in Phobos (which won't work in std.datetime due to >> the use of immutable and pure) is to create a C module which has the code from >> the static constructor and then have a separate module which calls it in its >> static constructor. > Which is a hack because that C function is a compiler wall while the dependency > persists. Btw. that stdiobase and datebase are obsolete the cycles have vanished. > > You will get this only if std.dateparse had a shared static ctor too. > Cycle detected between modules with ctors/dtors: > std.date -> std.dateparse -> std.date > object.Exception@src/rt/minfo.d(309): Aborting! > > There is a cleaner hack to solve the issue but I really don't like it. > It's two DAGs that are iterated one for "shared static this" and > one for "static this". > > ---- > module a; > import b; > > shared static this() > { > } > ---- > module b; > import a, core.atomic : cas; > > shared bool initialized; > static this() > { > if (!cas(&initialized, false, true)) return; > ... > } > ---- Forget about it. Immutable initialization shouldn't work from thread local ctors. But hey I found a bug and it already had a number http://d.puremagic.com/issues/show_bug.cgi?id=4923. |
January 18, 2012 Re: Program size, linking matter, and static this() | ||||
---|---|---|---|---|
| ||||
Posted in reply to Andrei Alexandrescu | > I'm not an expert in linkers, but my understanding is that linkers naturally remove unused object files. That, coupled with dmd's ability to break compilation output in many pseudo-object files, would take care of the matter. Truth be told, once you link in Object.factory(), bam - all classes are linked.
>
That's strange, because Object.factory should only require TypeInfo_Class
which only indirectly iterates through all modules.
The ModuleInfos do drag in all their classes so what we currently don't get
is a module with only some of it's classes.
What OS are you using? Can you bundle up some files that reproduce this?
|
January 19, 2012 Re: Program size, linking matter, and static this() | ||||
---|---|---|---|---|
| ||||
Posted in reply to Walter Bright | I tried different versions of DMD 2.057: - compiled from sources in the release zip (Gentoo ebuild) - using the 32-bit binaries in the release zip - compiling the latest 32-bit version of DMD from the repository I tried different compiler flags or no flags at all, compiled similar code in C++ to see if the linker is ok and tried -m32 and -m64, all to no avail. Then I found a solution that I can hardly imagine happening only on my unique snow-flake of a system ;) : struct Test { __gshared byte abcd[10 * 1024 * 1024]; } If it weren't for your own test results, I'd assume there is a small compiler bug in the code that decides what can go into .bss, that makes it look only for data explicitly flagged as __gshared, but not other immutable data. (Something like that anyway.) I back-tracked the compiler code to where it either calls obj_bytes (good case, goes into .bss) or obj_lidata (bad case) to write the 10 MB of zeros. But there were so many call sites, that I figured someone with inside knowledge would figure it out faster. As a side-effect of this experiment I found this combination to do funny things at runtime: -------------------------------------------------- struct Test { byte arr1[1024 * 1024 * 10]; __gshared byte arr2[1024 * 1024 * 10]; } int main() { Test test; return 0; } -------------------------------------------------- -- Marco Am 18.01.2012, 11:18 Uhr, schrieb Walter Bright <newshound2@digitalmars.com>: > On 1/18/2012 1:43 AM, Marco Leise wrote: >> It is back again! The following struct in my main module increases the >> executable size by 10MB with DMD 2.075: >> >> struct Test { >> byte abcd[10 * 1024 * 1024]; >> } > > Compiling it and obj2asm'ing the result, and you'll see it goes into the BSS segment: > > > _TEXT segment dword use32 public 'CODE' ;size is 0 > _TEXT ends > _DATA segment para use32 public 'DATA' ;size is 12 > _DATA ends > CONST segment para use32 public 'CONST' ;size is 0 > CONST ends > _BSS segment para use32 public 'BSS' ;size is 10485760 > _BSS ends > FLAT group > extrn _D19TypeInfo_S3foo4Test6__initZ > > public _D3foo4Test6__initZ > FMB segment dword use32 public 'DATA' ;size is 0 > FMB ends > FM segment dword use32 public 'DATA' ;size is 4 > FM ends > FME segment dword use32 public 'DATA' ;size is 0 > FME ends > extrn _D15TypeInfo_Struct6__vtblZ > > public _D3foo12__ModuleInfoZ > _D19TypeInfo_S3foo4Test6__initZ COMDAT flags=x0 attr=x10 align=x0 > > _TEXT segment > assume CS:_TEXT > _TEXT ends > _DATA segment > _D3foo12__ModuleInfoZ: > db 004h,000h,000h,0ffffff80h,000h,000h,000h,000h ;........ > db 066h,06fh,06fh,000h ;foo. > _DATA ends > CONST segment > CONST ends > _BSS segment > _BSS ends > FMB segment > FMB ends > FM segment > dd offset FLAT:_D3foo12__ModuleInfoZ > FM ends > FME segment > FME ends > _D19TypeInfo_S3foo4Test6__initZ comdat > dd offset FLAT:_D15TypeInfo_Struct6__vtblZ > db 000h,000h,000h,000h ;.... > db 008h,000h,000h,000h ;.... > dd offset FLAT:_D19TypeInfo_S3foo4Test6__initZ[03Ch] > db 000h,000h,0ffffffa0h,000h,000h,000h,000h,000h ;........ > db 000h,000h,000h,000h,000h,000h,000h,000h ;........ > db 000h,000h,000h,000h,000h,000h,000h,000h ;........ > db 000h,000h,000h,000h,000h,000h,000h,000h ;........ > db 000h,000h,000h,000h,000h,000h,000h,000h ;........ > db 001h,000h,000h,000h,066h,06fh,06fh,02eh ;....foo. > db 054h,065h,073h,074h,000h ;Test. > _D19TypeInfo_S3foo4Test6__initZ ends > end > ------------------------------------------------- > > Adding a void main(){} yields an executable of 145,948 bytes. |
January 19, 2012 Re: Program size, linking matter, and static this() | ||||
---|---|---|---|---|
| ||||
Posted in reply to Walter Bright | P.S.: I could have realized it earlier: DMD uses the Windows PE BSS section quite well! It is Linux where the .bss section is not used! I'll file a bug report about this after lunch and look forward to smaller executables under Linux any time soon :D |
Copyright © 1999-2021 by the D Language Foundation