On 1/18/2012 1:43 AM, Marco Leise wrote:
> It is back again! The following struct in my main module increases the
> executable size by 10MB with DMD 2.075:
>
> struct Test {
> byte abcd[10 * 1024 * 1024];
> }

Compiling it and obj2asm'ing the result, and you'll see it goes into the BSS segment:


_TEXT   segment dword use32 public 'CODE'       ;size is 0
_TEXT   ends
_DATA   segment para use32 public 'DATA'        ;size is 12
_DATA   ends
CONST   segment para use32 public 'CONST'       ;size is 0
CONST   ends
_BSS    segment para use32 public 'BSS' ;size is 10485760
_BSS    ends
FLAT    group
        extrn   _D19TypeInfo_S3foo4Test6__initZ

        public  _D3foo4Test6__initZ
FMB     segment dword use32 public 'DATA'       ;size is 0
FMB     ends
FM      segment dword use32 public 'DATA'       ;size is 4
FM      ends
FME     segment dword use32 public 'DATA'       ;size is 0
FME     ends
        extrn   _D15TypeInfo_Struct6__vtblZ

        public  _D3foo12__ModuleInfoZ
_D19TypeInfo_S3foo4Test6__initZ COMDAT flags=x0 attr=x10 align=x0

_TEXT   segment
        assume  CS:_TEXT
_TEXT   ends
_DATA   segment
_D3foo12__ModuleInfoZ:
        db      004h,000h,000h,0ffffff80h,000h,000h,000h,000h   ;........
        db      066h,06fh,06fh,000h     ;foo.
_DATA   ends
CONST   segment
CONST   ends
_BSS    segment
_BSS    ends
FMB     segment
FMB     ends
FM      segment
        dd      offset FLAT:_D3foo12__ModuleInfoZ
FM      ends
FME     segment
FME     ends
_D19TypeInfo_S3foo4Test6__initZ comdat
        dd      offset FLAT:_D15TypeInfo_Struct6__vtblZ
        db      000h,000h,000h,000h     ;....
        db      008h,000h,000h,000h     ;....
        dd      offset FLAT:_D19TypeInfo_S3foo4Test6__initZ[03Ch]
        db      000h,000h,0ffffffa0h,000h,000h,000h,000h,000h   ;........
        db      000h,000h,000h,000h,000h,000h,000h,000h ;........
        db      000h,000h,000h,000h,000h,000h,000h,000h ;........
        db      000h,000h,000h,000h,000h,000h,000h,000h ;........
        db      000h,000h,000h,000h,000h,000h,000h,000h ;........
        db      001h,000h,000h,000h,066h,06fh,06fh,02eh ;....foo.
        db      054h,065h,073h,074h,000h        ;Test.
_D19TypeInfo_S3foo4Test6__initZ ends
        end
-------------------------------------------------

Adding a void main(){} yields an executable of 145,948 bytes.

Am 18.01.2012, 11:18 Uhr, schrieb Walter Bright <newshound2@digitalmars.com>:

> On 1/18/2012 1:43 AM, Marco Leise wrote:
>> It is back again! The following struct in my main module increases the
>> executable size by 10MB with DMD 2.075:
>>
>> struct Test {
>> byte abcd[10 * 1024 * 1024];
>> }
>
> Compiling it and obj2asm'ing the result, and you'll see it goes into the BSS segment:
>
>
> [...]
>
> Adding a void main(){} yields an executable of 145,948 bytes.

Thanks for checking back. I'll have to experiment a bit to narrow this one down. It comes and goes like a ghost. I was using Linux 64-bit and the switches -O -release on a medium size code base.

> Yes they are. static constructors completely chicken out on them. Not only is
> there no real attempt to determine whether the static constructors are
> actually dependent (which granted, isn't an easy problem), but there is _zero_
> support in the language for resolving such circular dependencies. There's no
> way to say that they _aren't_ dependent even if you can clearly see that they
> aren't. The solution used in Phobos (which won't work in std.datetime due to
> the use of immutable and pure) is to create a C module which has the code from
> the static constructor and then have a separate module which calls it in its
> static constructor.
Which is a hack because that C function is a compiler wall while the dependency
persists. Btw. that stdiobase and datebase are obsolete the cycles have vanished.

You will get this only if std.dateparse had a shared static ctor too.
  Cycle detected between modules with ctors/dtors:
  std.date -> std.dateparse -> std.date
  object.Exception@src/rt/minfo.d(309): Aborting!

There is a cleaner hack to solve the issue but I really don't like it.
It's two DAGs that are iterated one for "shared static this" and
one for "static this".

----
module a;
import b;

shared static this()
{
}
----
module b;
import a, core.atomic : cas;

shared bool initialized;
static this()
{
    if (!cas(&initialized, false, true)) return;
    ...
}
----

On Wed, 18 Jan 2012 12:14:07 +0100, Martin Nowak <dawg@dawgfoto.de> wrote:

>> Yes they are. static constructors completely chicken out on them. Not only is
>> there no real attempt to determine whether the static constructors are
>> actually dependent (which granted, isn't an easy problem), but there is _zero_
>> support in the language for resolving such circular dependencies. There's no
>> way to say that they _aren't_ dependent even if you can clearly see that they
>> aren't. The solution used in Phobos (which won't work in std.datetime due to
>> the use of immutable and pure) is to create a C module which has the code from
>> the static constructor and then have a separate module which calls it in its
>> static constructor.
> Which is a hack because that C function is a compiler wall while the dependency
> persists. Btw. that stdiobase and datebase are obsolete the cycles have vanished.
>
> You will get this only if std.dateparse had a shared static ctor too.
>    Cycle detected between modules with ctors/dtors:
>    std.date -> std.dateparse -> std.date
>    object.Exception@src/rt/minfo.d(309): Aborting!
>
> There is a cleaner hack to solve the issue but I really don't like it.
> It's two DAGs that are iterated one for "shared static this" and
> one for "static this".
>
> ----
> module a;
> import b;
>
> shared static this()
> {
> }
> ----
> module b;
> import a, core.atomic : cas;
>
> shared bool initialized;
> static this()
> {
>      if (!cas(&initialized, false, true)) return;
>      ...
> }
> ----
Forget about it. Immutable initialization shouldn't work from thread local ctors.
But hey I found a bug and it already had a number
http://d.puremagic.com/issues/show_bug.cgi?id=4923.

> I'm not an expert in linkers, but my understanding is that linkers naturally remove unused object files. That, coupled with dmd's ability to break compilation output in many pseudo-object files, would take care of the matter. Truth be told, once you link in Object.factory(), bam - all classes are linked.
>
That's strange, because Object.factory should only require TypeInfo_Class
which only indirectly iterates through all modules.
The ModuleInfos do drag in all their classes so what we currently don't get
is a module with only some of it's classes.

What OS are you using? Can you bundle up some files that reproduce this?

I tried different versions of DMD 2.057:
- compiled from sources in the release zip (Gentoo ebuild)
- using the 32-bit binaries in the release zip
- compiling the latest 32-bit version of DMD from the repository
I tried different compiler flags or no flags at all, compiled similar code in C++ to see if the linker is ok and tried -m32 and -m64, all to no avail. Then I found a solution that I can hardly imagine happening only on my unique snow-flake of a system ;) :

struct Test {
    __gshared byte abcd[10 * 1024 * 1024];
}

If it weren't for your own test results, I'd assume there is a small compiler bug in the code that decides what can go into .bss, that makes it look only for data explicitly flagged as __gshared, but not other immutable data. (Something like that anyway.)
I back-tracked the compiler code to where it either calls obj_bytes (good case, goes into .bss) or obj_lidata (bad case) to write the 10 MB of zeros. But there were so many call sites, that I figured someone with inside knowledge would figure it out faster.

As a side-effect of this experiment I found this combination to do funny things at runtime:

--------------------------------------------------

struct Test {
	byte arr1[1024 * 1024 * 10];
	__gshared byte arr2[1024 * 1024 * 10];
}

int main() {
	Test test;
	return 0;
}

--------------------------------------------------

-- Marco

Am 18.01.2012, 11:18 Uhr, schrieb Walter Bright <newshound2@digitalmars.com>:

> On 1/18/2012 1:43 AM, Marco Leise wrote:
>> It is back again! The following struct in my main module increases the
>> executable size by 10MB with DMD 2.075:
>>
>> struct Test {
>> byte abcd[10 * 1024 * 1024];
>> }
>
> Compiling it and obj2asm'ing the result, and you'll see it goes into the BSS segment:
>
>
> _TEXT   segment dword use32 public 'CODE'       ;size is 0
> _TEXT   ends
> _DATA   segment para use32 public 'DATA'        ;size is 12
> _DATA   ends
> CONST   segment para use32 public 'CONST'       ;size is 0
> CONST   ends
> _BSS    segment para use32 public 'BSS' ;size is 10485760
> _BSS    ends
> FLAT    group
>          extrn   _D19TypeInfo_S3foo4Test6__initZ
>
>          public  _D3foo4Test6__initZ
> FMB     segment dword use32 public 'DATA'       ;size is 0
> FMB     ends
> FM      segment dword use32 public 'DATA'       ;size is 4
> FM      ends
> FME     segment dword use32 public 'DATA'       ;size is 0
> FME     ends
>          extrn   _D15TypeInfo_Struct6__vtblZ
>
>          public  _D3foo12__ModuleInfoZ
> _D19TypeInfo_S3foo4Test6__initZ COMDAT flags=x0 attr=x10 align=x0
>
> _TEXT   segment
>          assume  CS:_TEXT
> _TEXT   ends
> _DATA   segment
> _D3foo12__ModuleInfoZ:
>          db      004h,000h,000h,0ffffff80h,000h,000h,000h,000h   ;........
>          db      066h,06fh,06fh,000h     ;foo.
> _DATA   ends
> CONST   segment
> CONST   ends
> _BSS    segment
> _BSS    ends
> FMB     segment
> FMB     ends
> FM      segment
>          dd      offset FLAT:_D3foo12__ModuleInfoZ
> FM      ends
> FME     segment
> FME     ends
> _D19TypeInfo_S3foo4Test6__initZ comdat
>          dd      offset FLAT:_D15TypeInfo_Struct6__vtblZ
>          db      000h,000h,000h,000h     ;....
>          db      008h,000h,000h,000h     ;....
>          dd      offset FLAT:_D19TypeInfo_S3foo4Test6__initZ[03Ch]
>          db      000h,000h,0ffffffa0h,000h,000h,000h,000h,000h   ;........
>          db      000h,000h,000h,000h,000h,000h,000h,000h ;........
>          db      000h,000h,000h,000h,000h,000h,000h,000h ;........
>          db      000h,000h,000h,000h,000h,000h,000h,000h ;........
>          db      000h,000h,000h,000h,000h,000h,000h,000h ;........
>          db      001h,000h,000h,000h,066h,06fh,06fh,02eh ;....foo.
>          db      054h,065h,073h,074h,000h        ;Test.
> _D19TypeInfo_S3foo4Test6__initZ ends
>          end
> -------------------------------------------------
>
> Adding a void main(){} yields an executable of 145,948 bytes.

P.S.: I could have realized it earlier: DMD uses the Windows PE BSS section quite well! It is Linux where the .bss section is not used! I'll file a bug report about this after lunch and look forward to smaller executables under Linux any time soon :D