> Could you elaborate?

As I remember, these features rely heavily on linker support, namely on how it merges sections. Each pointer to static ctor (and unittest) is placed in a section, then these sections are merged by linker and you get an array of pointers to all static ctors and unittests. The startup code reads the pointers in sequence and executes them. If some section is removed, the corresponding static ctor won't be run.

> Real code, that i was testing with previously, contained static module ctors which
> worked, and i just tried a simple two module test w/ multiple shared_[cd]tors +
> static_[cd]tors + unittests -- all of them are called as expected.

If so then ok.

On 12/21/11 17:42, Kagamin wrote:
>> Could you elaborate?
> 
> As I remember, these features rely heavily on linker support, namely on how it merges sections. Each pointer to static ctor (and unittest) is placed in a section, then these sections are merged by linker and you get an array of pointers to all static ctors and unittests. The startup code reads the pointers in sequence and executes them. If some section is removed, the corresponding static ctor won't be run.
> 
>> Real code, that i was testing with previously, contained static module ctors which worked, and i just tried a simple two module test w/ multiple shared_[cd]tors + static_[cd]tors + unittests -- all of them are called as expected.
> 
> If so then ok.

I suspected that was what you were worried about and did that test [1],
which gives the expected output. But looking now at the list of linker
gc'ed sections, i notice ld does claim to have removed eg
".text._D11gcfuncdata218_sharedStaticCtor3FZv", which is one of the ctors.
The reason things still work is because the code is also inlined in
"_D11gcfuncdata215__modsharedctorFZv" together with the other ctor.
Aha, the compiler (gdc) actually inlined all the [cd]tors when run at -O3,
and the "real" constructor was then garbage collected by the linker. Phew.
Compiling with -O2 makes modsharedctor call all the ctors instead, and they
are then not gc'ed.

Why the compiler emits the unused outofline ctors is another question. Maybe i should start marking them as private?...

Turned out not to be a good idea, as the compiler silently accepts
"shared private static this() {...}" and does not emit the dead code anymore.
But.. this constructor is run together with the un-shared ones, ie possibly
_after_ them... (this is not section-gc specific, happens w/o it too)

artur

[1]

import std.stdio;

/* gcfuncdata2.d is a copy of this module, w/o main() */
import gcfuncdata2;

auto p(S...)(S args) { return stdout.writeln(__FILE__~": ", args); }

void main(string[] args){ p("main"); }

static this() { p("ctor1"); }
static this() { p("ctor2"); }
shared static this() { p("shared ctor1"); }
shared static this() { p("shared ctor2"); }

static ~this() { p("dtor1"); }
static ~this() { p("dtor2"); }
shared static ~this() { p("shared dtor1"); }
shared static ~this() { p("shared dtor2"); }

unittest { p("unittest1"); }
unittest { p("unittest2"); }

> Turned out not to be a good idea, as the compiler silently accepts
> "shared private static this() {...}" and does not emit the dead code anymore.
> But.. this constructor is run together with the un-shared ones, ie possibly
> _after_ them... (this is not section-gc specific, happens w/o it too)

try `private shared static this()`
AFAIK static ctors have some attribute ordering issue.

> import std.stdio;
>
> /* gcfuncdata2.d is a copy of this module, w/o main() */
> import gcfuncdata2;
>
> auto p(S...)(S args) { return stdout.writeln(__FILE__~": ", args); }
>
> void main(string[] args){ p("main"); }
>
> static this() { p("ctor1"); }
> static this() { p("ctor2"); }
> shared static this() { p("shared ctor1"); }
> shared static this() { p("shared ctor2"); }
>
> static ~this() { p("dtor1"); }
> static ~this() { p("dtor2"); }
> shared static ~this() { p("shared dtor1"); }
> shared static ~this() { p("shared dtor2"); }
>
> unittest { p("unittest1"); }
> unittest { p("unittest2"); }

Also add class static ctors and function static variables.

On 12/21/11 20:13, Kagamin wrote:
>> Turned out not to be a good idea, as the compiler silently accepts
>> "shared private static this() {...}" and does not emit the dead code anymore.
>> But.. this constructor is run together with the un-shared ones, ie possibly
>> _after_ them... (this is not section-gc specific, happens w/o it too)
> 
> try `private shared static this()`
> AFAIK static ctors have some attribute ordering issue.

Was going to follow up with exactly this.

"private shared static this() { p("shared ctor2"); }"

is a shared constructor

"shared private static this() { p("shared ctor2"); }"

is not, but the compiler won't even warn you about this.

(in this case gdc, but i assume it's not gdc specific)

>> import std.stdio;
>>
>> /* gcfuncdata2.d is a copy of this module, w/o main() */
>> import gcfuncdata2;
>>
>> auto p(S...)(S args) { return stdout.writeln(__FILE__~": ", args); }
>>
>> void main(string[] args){ p("main"); }
>>
>> static this() { p("ctor1"); }
>> static this() { p("ctor2"); }
>> shared static this() { p("shared ctor1"); }
>> shared static this() { p("shared ctor2"); }
>>
>> static ~this() { p("dtor1"); }
>> static ~this() { p("dtor2"); }
>> shared static ~this() { p("shared dtor1"); }
>> shared static ~this() { p("shared dtor2"); }
>>
>> unittest { p("unittest1"); }
>> unittest { p("unittest2"); }
> 
> Also add class static ctors and function static variables.

But what would be the point? They should be GCed if unused and emitted, and wont be if something references them.

artur

On Wednesday, 21 December 2011 at 19:04:21 UTC, Artur Skawina wrote:
> gc'ed sections, i notice ld does claim to have removed eg
> ".text._D11gcfuncdata218_sharedStaticCtor3FZv", which is one of the ctors.
> The reason things still work is because the code is also inlined in "_D11gcfuncdata215__modsharedctorFZv" together with the other ctor. Aha, the compiler (gdc) actually inlined all the [cd]tors when run at -O3,
> and the "real" constructor was then garbage collected by the linker. Phew.
> Compiling with -O2 makes modsharedctor call all the ctors instead, and they are then not gc'ed.
>
> Why the compiler emits the unused outofline ctors is another question.
> Maybe i should start marking them as private?...

Maybe they should be marked by the compiler as private by default?

Indeed, a couple small programs I wrote today behave erratically w/ gc-sections.  This only seems to occur on DMD, but I'm not sure if this is a bug in DMD or if differences in library build configurations between compilers (these are workarounds for bugs in GDC and LDC) explain it.

On Wednesday, 21 December 2011 at 04:15:21 UTC, Artur Skawina wrote:
> On 12/20/11 19:59, Trass3r wrote:
>> Seems like --gc-sections _can_ have its pitfalls:
>> http://blog.flameeyes.eu/2009/11/21/garbage-collecting-sections-is-not-for-production
>> 
>> Also I read somewhere that --gc-sections isn't always supported (no standard switch or something like that).
>
> The scenario in that link apparently involves a hack, where a completely unused symbol
> is used to communicate with another program/library (which checks for its presence with
> dlsym(3)).
> The linker will omit that symbol, as nothing else references it - the solution is to
> simply reference it from somewhere. Or explicitly place it in a used section. Or
> incrementally link in the unused symbols _after_ the gc pass. Or...
>
> If you use such hacks you have to handle them specially; there's no way for the compiler
> to magically know which unreferenced symbols are not really unused. (which is also why
> this optimization isn't very useful for shared libs - every visible symbol has to be
> assumed used, for obvious reasons)
>
> The one potential problematic case i mentioned in that gdc bug mentioned above is this:
> If the D runtime (most likely GC) needs to know the start/end of the data and bss
> sections _and_ does it in a way that can confuse it if some unreferenced parts of these
> sections disappear and/or are reordered, then turning on the section GC could uncover
> this bug. From the few simple tests i ran here everything seems to work fine, but I did
> not check the code to confirm there are no incorrect assumptions present.
>
>> I personally see no reason not to use -ffunction-sections and -fdata-sections for compiling phobos though, cause a test with gdc didn't even result in a much bigger lib file, nor did it take significantly longer to compile/link.
>
> 737k -> 320k executable size reduction is a compelling argument.
>
>> That site I linked claims though, that it does mean serious overhead even if --gc-sections is omitted then.
>
> ?
>
>> So we have to do tests with huge codebases first.
>
> yes.
>
> artur

http://d.puremagic.com/issues/show_bug.cgi?id=879

On Dec 21, 2011, at 2:24 PM, dsimcha wrote:

> Indeed, a couple small programs I wrote today behave erratically w/ gc-sections.  This only seems to occur on DMD, but I'm not sure if this is a bug in DMD or if differences in library build configurations between compilers (these are workarounds for bugs in GDC and LDC) explain it.
> 
> On Wednesday, 21 December 2011 at 04:15:21 UTC, Artur Skawina wrote:
>> On 12/20/11 19:59, Trass3r wrote:
>>> Seems like --gc-sections _can_ have its pitfalls:
>>> http://blog.flameeyes.eu/2009/11/21/garbage-collecting-sections-is-not-for-production
>>> Also I read somewhere that --gc-sections isn't always supported (no standard switch or something like that).
>> 
>> The scenario in that link apparently involves a hack, where a completely unused symbol
>> is used to communicate with another program/library (which checks for its presence with
>> dlsym(3)).
>> The linker will omit that symbol, as nothing else references it - the solution is to
>> simply reference it from somewhere. Or explicitly place it in a used section. Or
>> incrementally link in the unused symbols _after_ the gc pass. Or...
>> 
>> If you use such hacks you have to handle them specially; there's no way for the compiler to magically know which unreferenced symbols are not really unused. (which is also why this optimization isn't very useful for shared libs - every visible symbol has to be assumed used, for obvious reasons)
>> 
>> The one potential problematic case i mentioned in that gdc bug mentioned above is this: If the D runtime (most likely GC) needs to know the start/end of the data and bss sections _and_ does it in a way that can confuse it if some unreferenced parts of these sections disappear and/or are reordered, then turning on the section GC could uncover this bug. From the few simple tests i ran here everything seems to work fine, but I did not check the code to confirm there are no incorrect assumptions present.
>> 
>>> I personally see no reason not to use -ffunction-sections and -fdata-sections for compiling phobos though, cause a test with gdc didn't even result in a much bigger lib file, nor did it take significantly longer to compile/link.
>> 
>> 737k -> 320k executable size reduction is a compelling argument.
>> 
>>> That site I linked claims though, that it does mean serious overhead even if --gc-sections is omitted then.
>> 
>> ?
>> 
>>> So we have to do tests with huge codebases first.
>> 
>> yes.
>> 
>> artur
> 
> 

On 12/22/11 00:06, Sean Kelly wrote:
> http://d.puremagic.com/issues/show_bug.cgi?id=879

Thanks for that link.
The exception testcase (from one of the duplicates) seems to work here.
(at least w/ gdc, as i don't have dmd here to test)

-----
import std.stdio;

/* gcfuncdata2.d is a copy of this module, w/o main() */
import gcfuncdata2;

auto p(S...)(S args) { return stdout.writeln(__FILE__~": ", args); }

void main(string[] args){
   p("main");

   try {
      throw new Exception("muh");
   } catch (Exception e) {
      p(e);
   }
}

static this() { p("module ctor1"); }
static this() { p("module ctor2"); }
shared static this() { p("shared module ctor1"); }
shared static this() { p("shared module ctor2"); }

static ~this() { p("module dtor1"); }
static ~this() { p("module dtor2"); }
shared static ~this() { p("shared module dtor1"); }
shared static ~this() { p("shared module dtor2"); }

unittest { p("unittest1"); }
unittest { p("unittest2"); }

class blah {
   static this() { p("class static ctor1"); }
   shared static this() { p("class shared static ctor1"); }
   static ~this() { p("class static dtor1"); }
   shared static ~this() { p("class shared static dtor1"); }
}

-----

results in:

$ ./gcfuncdata1
gcfuncdata2.d: shared module ctor1
gcfuncdata2.d: shared module ctor2
gcfuncdata2.d: class shared static ctor1
gcfuncdata1.d: shared module ctor1
gcfuncdata1.d: shared module ctor2
gcfuncdata1.d: class shared static ctor1
gcfuncdata2.d: module ctor1
gcfuncdata2.d: module ctor2
gcfuncdata2.d: class static ctor1
gcfuncdata1.d: module ctor1
gcfuncdata1.d: module ctor2
gcfuncdata1.d: class static ctor1
gcfuncdata2.d: unittest1
gcfuncdata2.d: unittest2
gcfuncdata1.d: unittest1
gcfuncdata1.d: unittest2
gcfuncdata1.d: main
gcfuncdata1.d: object.Exception@gcfuncdata1.d(12): muh
gcfuncdata1.d: class static dtor1
gcfuncdata1.d: module dtor2
gcfuncdata1.d: module dtor1
gcfuncdata2.d: class static dtor1
gcfuncdata2.d: module dtor2
gcfuncdata2.d: module dtor1
gcfuncdata1.d: class shared static dtor1
gcfuncdata1.d: shared module dtor2
gcfuncdata1.d: shared module dtor1
gcfuncdata2.d: class shared static dtor1
gcfuncdata2.d: shared module dtor2
gcfuncdata2.d: shared module dtor1