This is disheartening;

Sooner or later, most D programmers will find the need for a static ctor. In the Mango libraries, they are used to initialize all manner of singletons; e.g. Stdout & Stdin, various lookup tables (Uri, ServletContext, etc), even instances of exceptions. This is done so the rest of the code can make certain valid assumptions about the environment, which leads to a faster, more manageable, and more maintainable code-base overall.

Unfortunately, I had assumed the D compiler had a solid foundation for such things ~ instead, this foundation is pretty much non-existent. I'll tell you why:

1) Static ctors will often form a dependency graph. Therefore, the order in which static ctors are invoked must respect any and all interdependencies. Without this behavior, the utility of static ctors is seriously crippled. Looking at the compiler source-code, there is precious little evidence that it does anything particularly useful with regard to this ordering. Hence, whilst the runtime code (moduleinit.d) tries valiantly to visit the constructors in a valid order, the data-structures it traverses are laid out in a fashion dictated by the order of compilation. This is farcical. As it turns out, Walter has recently added a delicate little footnote to the relevant documentation (“the order in which they are called is not defined”).

2) There’s also a little secret that you may not know about ~ in order to know anything about static ctors, the compiler must see them within the modules it compiles. To wit; the list of said ctors is managed by the compiler only, and not by the linker. Suppose you are a non-open-source corporate entity, and you wish to hide the implementation of your algorithmic crown jewels … you must still expose the source-code for all static constructors, which means all that precious data which you load up into runtime-containers must be exposed for all to see.

There’s lot’s of things to like about D, and there are some things that are pretty ugly. What bother me most is that working with D often feels like playing a game of Poker. IMO this is an unreasonable state of affairs, and here’s one developer who’s pretty fed up with it.

I make a humble request that (1) be resolved quickly; it can be done by building a simple dependency graph of the imports at compile-time (perhaps in Module::genmoduleinfo; all the relevant information is ready & accessible there), and emitting the module-structures in the resultant order. This would both resolve the issue, and greatly simplify the valiant, yet thoroughly borked, traversal performed at startup time. It would also permit the ModuleInfo to be placed in the CONST segment instead.

How about it, Walter?

As for (2), I don’t see a resolution without the linker being involved.

On Sun, 13 Feb 2005 23:43:01 +0000 (UTC), Kris wrote:

[snip]

> As for (2), I don’t see a resolution without the linker being involved.

Just clarifying this for myself, but are you saying that if the static ctor is linked in (and not compiled) that the ctor is not run?

-- 
Derek
Melbourne, Australia
14/02/2005 11:00:30 AM

I may have made a rather glaring booboo regarding that point, Derek. Looking at the compiler source-code again, I realized it was doing things rather differently that I had originally thought. So the answer is, I don't know. Although I thought I did :-(

Felt like I was on thin-ice already with this issue; I think one leg just went through

- Kris


In article <yelthh1kf26l$.i5r2b5jnerb3.dlg@40tude.net>, Derek Parnell says...
>
>On Sun, 13 Feb 2005 23:43:01 +0000 (UTC), Kris wrote:
>
>[snip]
> 
>> As for (2), I don’t see a resolution without the linker being involved.
>
>Just clarifying this for myself, but are you saying that if the static ctor is linked in (and not compiled) that the ctor is not run?
>
>-- 
>Derek
>Melbourne, Australia
>14/02/2005 11:00:30 AM

I made a tragic mistake in assessing the method in which ModuleInfo is constructed, so the original suggestion was nonsense. Armed with a bit more knowledge though, I'll have another go:

Each object file appears to contain a list of ModuleInfo descriptors, representing those imports which have static ctors ~ where the list order follows that of the module source-code. By maintaining such a list within each module object-file, the design apparently expects to resolve static-ctor dependencies at startup time by recursively traversing the ModuleInfo structures across all files linked. I'm guessing there's some voodoo that happens to link the lists together.

The problem occurs when one module acts as a 'proxy' for another in this regard. Specifically:

module C has a static ctor
module B imports, and references C, but has no static ctor of its own
module A imports B, and has a static ctor *indirectly* dependent upon C

In this case, A does not need to import C (doesn't even know it exists), though it must import B. However, the compiler mistakenly drops C from the list belonging to A because the import of B does not expose any static-ctors therin. C is orphaned via the lack of a static-ctor in B.

Thus, the chain of dependencies is broken. I checked this by adding a dummy static ctor (module-level) to B and, sure enough, the graph was restored.

This whole thing might be resolved by ensuring the compiler does not drop C, just because B does not have a static-ctor. The blunt way to do this would be to implicitly add a static-ctor to those modules which don't have one themselves, but which import modules that do (which is already done for static-class-ctors).

I may be wrong in this assessment also, but feel the ice is thicker where I'm standing now

- Kris


In article <cuoom4$2614$1@digitaldaemon.com>, Kris says...
>
>a simple dependency graph of the imports at compile-time (perhaps in
>Module::genmoduleinfo; all the relevant information is ready & accessible
>there), and emitting the module-structures in the resultant order. This would
>both resolve the issue, and greatly simplify the valiant, yet thoroughly borked, traversal performed at startup time.

On Mon, 14 Feb 2005 01:36:09 +0000 (UTC), Kris <Kris_member@pathlink.com> wrote:
> I made a tragic mistake in assessing the method in which ModuleInfo is
> constructed, so the original suggestion was nonsense. Armed with a bit more
> knowledge though, I'll have another go:
>
> Each object file appears to contain a list of ModuleInfo descriptors,
> representing those imports which have static ctors ~ where the list order
> follows that of the module source-code. By maintaining such a list within each
> module object-file, the design apparently expects to resolve static-ctor
> dependencies at startup time by recursively traversing the ModuleInfo structures
> across all files linked. I'm guessing there's some voodoo that happens to link
> the lists together.
>
> The problem occurs when one module acts as a 'proxy' for another in this regard.
> Specifically:
>
> module C has a static ctor
> module B imports, and references C, but has no static ctor of its own
> module A imports B, and has a static ctor *indirectly* dependent upon C
>
> In this case, A does not need to import C (doesn't even know it exists), though
> it must import B. However, the compiler mistakenly drops C from the list
> belonging to A because the import of B does not expose any static-ctors therin.
> C is orphaned via the lack of a static-ctor in B.
>
> Thus, the chain of dependencies is broken. I checked this by adding a dummy
> static ctor (module-level) to B and, sure enough, the graph was restored.
>
> This whole thing might be resolved by ensuring the compiler does not drop C,
> just because B does not have a static-ctor. The blunt way to do this would be to
> implicitly add a static-ctor to those modules which don't have one themselves,
> but which import modules that do (which is already done for static-class-ctors).

It seems to me that:
  "A imports B" && "B imports C" == "A imports C"

regardless of whether any module has a static ctor.

Can the compiler simply give each module a number which refers to the deepest level in the dependancy tree it occurs at, then process that list from the biggest entry first?

You can drop from the list an entry that does not have a static ctor itself, as long as you still count it as increasing the depth.

I'm not sure whether circular imports will cause trouble?

i.e.

A imports B imports C

A = 0
B = 1
C = 2

process C, process B, process A.

> I may be wrong in this assessment also, but feel the ice is thicker where I'm
> standing now

You're doing more than I am... I'm sitting inside where it's warm drinking hot cocoa dreaming about going out on the ice.

Regan

> module C has a static ctor
> module B imports, and references C, but has no static ctor of its own
> module A imports B, and has a static ctor *indirectly* dependent upon C


This thread might be related: http://www.digitalmars.com/d/archives/digitalmars/D/bugs/1773.html

In particular check out http://www.digitalmars.com/drn-bin/wwwnews?digitalmars.D.bugs/1780

I thought Walter had fixed all those problems but I guess not. -Ben

What about using the same way ctors are resolved by classes. This by adding sort of inheritance/dependancy to the module statement.

module std.c.stdio : std.stdio;

Which ensures that std.stdio ctors are called first. This way the initialisation is mandatory. If you have to do ctors in a specific order than the programmer becomes responsable and not a random algorithm dictated by the compiler. Or funky link options by reordering the linkage.

Daan

Kris schreef:
> I made a tragic mistake in assessing the method in which ModuleInfo is
> constructed, so the original suggestion was nonsense. Armed with a bit more
> knowledge though, I'll have another go:
> 
> Each object file appears to contain a list of ModuleInfo descriptors,
> representing those imports which have static ctors ~ where the list order
> follows that of the module source-code. By maintaining such a list within each
> module object-file, the design apparently expects to resolve static-ctor
> dependencies at startup time by recursively traversing the ModuleInfo structures
> across all files linked. I'm guessing there's some voodoo that happens to link
> the lists together.
> 
> The problem occurs when one module acts as a 'proxy' for another in this regard.
> Specifically:
> 
> module C has a static ctor
> module B imports, and references C, but has no static ctor of its own
> module A imports B, and has a static ctor *indirectly* dependent upon C
> 
> In this case, A does not need to import C (doesn't even know it exists), though
> it must import B. However, the compiler mistakenly drops C from the list
> belonging to A because the import of B does not expose any static-ctors therin.
> C is orphaned via the lack of a static-ctor in B.
> 
> Thus, the chain of dependencies is broken. I checked this by adding a dummy
> static ctor (module-level) to B and, sure enough, the graph was restored.
> 
> This whole thing might be resolved by ensuring the compiler does not drop C,
> just because B does not have a static-ctor. The blunt way to do this would be to
> implicitly add a static-ctor to those modules which don't have one themselves,
> but which import modules that do (which is already done for static-class-ctors).
> 
> I may be wrong in this assessment also, but feel the ice is thicker where I'm
> standing now
> 
> - Kris
> 
> 
> In article <cuoom4$2614$1@digitaldaemon.com>, Kris says...
> 
>>a simple dependency graph of the imports at compile-time (perhaps in
>>Module::genmoduleinfo; all the relevant information is ready & accessible
>>there), and emitting the module-structures in the resultant order. This would
>>both resolve the issue, and greatly simplify the valiant, yet thoroughly borked, traversal performed at startup time. 
> 
> 
> 

In article <cup7gp$2ibs$1@digitaldaemon.com>, Ben Hinkle says...
>
>> module C has a static ctor
>> module B imports, and references C, but has no static ctor of its own
>> module A imports B, and has a static ctor *indirectly* dependent upon C
>
>
>This thread might be related: http://www.digitalmars.com/d/archives/digitalmars/D/bugs/1773.html
>
>In particular check out http://www.digitalmars.com/drn-bin/wwwnews?digitalmars.D.bugs/1780
>
>I thought Walter had fixed all those problems but I guess not. -Ben

I thought this problem looked familiar.  Looks like it was only pushed /closer/ to completion than before, Walter *did* change the compiler a little bit.  From what I've gathered from here and on dsource, the problem is still much the same.

Here's a thought: Is it possible that we're seeing a regression instead of a new bug?

- EricAnderton at yahoo

"Kris" <Kris_member@pathlink.com> wrote in message news:cuova9$2biv$1@digitaldaemon.com...
> Each object file appears to contain a list of ModuleInfo descriptors, representing those imports which have static ctors ~ where the list order follows that of the module source-code. By maintaining such a list within
each
> module object-file, the design apparently expects to resolve static-ctor dependencies at startup time by recursively traversing the ModuleInfo
structures
> across all files linked.

That is how it works. The rule used is straightforward: before a module constructor can be run, all the module constructors in the modules it imports must have been run.

> I'm guessing there's some voodoo that happens to link
> the lists together.

It's not voodoo, but it is different between windows and linux. It uses the same technique used by C++ object files to find the static constructors.

> The problem occurs when one module acts as a 'proxy' for another in this
regard.
> Specifically:
>
> module C has a static ctor
> module B imports, and references C, but has no static ctor of its own
> module A imports B, and has a static ctor *indirectly* dependent upon C
>
> In this case, A does not need to import C (doesn't even know it exists),
though
> it must import B. However, the compiler mistakenly drops C from the list belonging to A because the import of B does not expose any static-ctors
therin.
> C is orphaned via the lack of a static-ctor in B.
>
> Thus, the chain of dependencies is broken. I checked this by adding a
dummy
> static ctor (module-level) to B and, sure enough, the graph was restored.
>
> This whole thing might be resolved by ensuring the compiler does not drop
C,
> just because B does not have a static-ctor. The blunt way to do this would
be to
> implicitly add a static-ctor to those modules which don't have one
themselves,
> but which import modules that do (which is already done for
static-class-ctors).

The problem with having the compiler insert implicit static ctors in modules that don't have them is that it will impose a likely unresolvable order of construction problem. In other words, it would become impossible to have circular imports (which is possible now if at least one of the modules in the circle doesn't have a static ctor).

Putting a static this(){} in B will resolve the problem. It isn't perfect, but it works. Note that C++ doesn't guarantee any order at all to static constructors.

In article <curj1o$2afu$1@digitaldaemon.com>, Walter says...
>
>
>"Kris" <Kris_member@pathlink.com> wrote in message
>
>> module C has a static ctor
>> module B imports, and references C, but has no static ctor of its own
>> module A imports B, and has a static ctor *indirectly* dependent upon C
>>
>> In this case, A does not need to import C (doesn't even know it exists),
>though
>> it must import B. However, the compiler mistakenly drops C from the list belonging to A because the import of B does not expose any static-ctors
>therin.
>> C is orphaned via the lack of a static-ctor in B.
>>
>> Thus, the chain of dependencies is broken. I checked this by adding a
>dummy
>> static ctor (module-level) to B and, sure enough, the graph was restored.
>>
>> This whole thing might be resolved by ensuring the compiler does not drop
>C,
>> just because B does not have a static-ctor. The blunt way to do this would
>be to
>> implicitly add a static-ctor to those modules which don't have one
>themselves,
>> but which import modules that do (which is already done for
>static-class-ctors).
>
>The problem with having the compiler insert implicit static ctors in modules that don't have them is that it will impose a likely unresolvable order of construction problem. In other words, it would become impossible to have circular imports (which is possible now if at least one of the modules in the circle doesn't have a static ctor).

Thanks for the reply;

As I understand it, such a case is detected by the runtime initialization. At this time, it throws an exception ~ could it be changed to simply terminate the tail-chasing instead?


>Putting a static this(){} in B will resolve the problem. It isn't perfect,
>but it works.

Right. The true problem with this is education ~ the issue is not even documented. If it were, then you could hardly expect just anyone to say "oh well .. of course that's the case". It's not a good place for D to be so, better to find a compiler-based resolution ~ if indeed there is one


> Note that C++ doesn't guarantee any order at all to static
>constructors.

Not interested ~ don't mean to be rude, but we're talking about D

- Kris