I know the runtime is supposed to work it all out, call thread module dtors, then shared dtors and then terminate.

But now I have a case where it is broken and I don't know if
we expect the runtime to figure it out, especially when it
comes to separate compilation and such. Basically on one end
I have a singleton template that uses a `shared static ~this()`
to destroy the object. On the other end I have a global list
of reference counted resources, also heavily templated and
relying on a `shared static ~this()` to free the memory
associated with the ref counts etc.
When I build something new from these parts, like a singleton
object that contains on of these ref counted resources, the
runtime is unable to figure out that it has to destroy the
singleton (and its resource with it), before it can call the
destructor of the resource list. For now I just wrote
`import Lib.Sys.Resource;` into the singleton destructor
(i.e. every singleton imports it, whether it needs it or not),
but that doesn't scale obviously.

Should the runtime be able to reliably figure out even such tough cases? The alternative, disallowing static dtors in templates isn't appealing.

Out of curiosity: What module do templated static dtors belong to anyways. And how does that effect when they are called?

-- 
Marco

On 11/10/14 9:57 AM, Marco Leise wrote:
> I know the runtime is supposed to work it all out, call
> thread module dtors, then shared dtors and then terminate.
>
> But now I have a case where it is broken and I don't know if
> we expect the runtime to figure it out, especially when it
> comes to separate compilation and such. Basically on one end
> I have a singleton template that uses a `shared static ~this()`
> to destroy the object. On the other end I have a global list
> of reference counted resources, also heavily templated and
> relying on a `shared static ~this()` to free the memory
> associated with the ref counts etc.
> When I build something new from these parts, like a singleton
> object that contains on of these ref counted resources, the
> runtime is unable to figure out that it has to destroy the
> singleton (and its resource with it), before it can call the
> destructor of the resource list. For now I just wrote
> `import Lib.Sys.Resource;` into the singleton destructor
> (i.e. every singleton imports it, whether it needs it or not),
> but that doesn't scale obviously.
>
> Should the runtime be able to reliably figure out even such
> tough cases? The alternative, disallowing static dtors in
> templates isn't appealing.
>
> Out of curiosity: What module do templated static dtors belong
> to anyways. And how does that effect when they are called?
>

I don't know about your specific issue. But I do know how the runtime calls static ctors/dtors because I rewrote that part a few years ago.

First, any ctors/dtors in a specific module are called in the order they appear in the file.

Second, the compiler records 3 things about a module:
1. Is it a standalone module? That is, does it not import any other modules.
2. If not, what modules does it import.
3. Does it have any dtors or ctors (it has a flag for each kind)

When run, the runtime uses this information to build a graph of the ordering to call the static ctors. If it detects any cycles in that graph, where module A imports directly or indirectly module B, and module B imports directly or indirectly module A, and both of them have the *same kind* of static ctor or dtor, then it errors and refuses to run the program (I'm sure all of you have seen this error).

Now, if that doesn't happen, it has a list of modules, of what imports whatever else. And it can call the static ctors in the order of that graph making sure dependent modules are constructed first. It does this separately for shared and unshared ctors/dtors. Standalone modules are called first, and are not included in the search for cycles.

On program/thread termination, it calls the dtors in the opposite order.

Two things that it CANNOT detect:

1. If you circumvent the module system to call functions in static ctor/dtors using extern(C).
2. If you establish any dependencies during runtime, such as pushing into module A's global array a reference to a module B global object, without module A or B importing each other. It won't know to call module A's dtor first.

I'm guessing the latter is what you are having issues with.

-Steve

Am Mon, 10 Nov 2014 10:06:01 -0500
schrieb Steven Schveighoffer <schveiguy@yahoo.com>:

> I don't know about your specific issue. But I do know how the runtime calls static ctors/dtors because I rewrote that part a few years ago.
> 
> First, any ctors/dtors in a specific module are called in the order they appear in the file.
> 
> Second, the compiler records 3 things about a module:
> 1. Is it a standalone module? That is, does it not import any other modules.
> 2. If not, what modules does it import.
> 3. Does it have any dtors or ctors (it has a flag for each kind)
> 
> When run, the runtime uses this information to build a graph of the ordering to call the static ctors. If it detects any cycles in that graph, where module A imports directly or indirectly module B, and module B imports directly or indirectly module A, and both of them have the *same kind* of static ctor or dtor, then it errors and refuses to run the program (I'm sure all of you have seen this error).
> 
> Now, if that doesn't happen, it has a list of modules, of what imports whatever else. And it can call the static ctors in the order of that graph making sure dependent modules are constructed first. It does this separately for shared and unshared ctors/dtors. Standalone modules are called first, and are not included in the search for cycles.
> 
> On program/thread termination, it calls the dtors in the opposite order.
> 
> Two things that it CANNOT detect:
> 
> 1. If you circumvent the module system to call functions in static
> ctor/dtors using extern(C).
> 2. If you establish any dependencies during runtime, such as pushing
> into module A's global array a reference to a module B global object,
> without module A or B importing each other. It won't know to call module
> A's dtor first.
> 
> I'm guessing the latter is what you are having issues with.
> 
> -Steve

Thank you for the explanation. I guess I'm dealing with case
2. But then, how is D supposed to deal with this situation?
Naturally the concepts of "external resources" and "singleton"
are orthogonal and the two modules _should_ not know about
each other.
Granted, a D singleton class is never going to be part of an
external system resource ;), but building on the idea,
whenever we have static dtors in templates, we risk that the
template arguments introduce a "hidden" dependency. The type
is `T` and Singleton.d does not import the module that `T`
came from, which would transitively import Resource.d and make
sure static dtors in there are deferred until after
Singleton.d's dtors were called.

Effectively templated module dtors are crashes waiting to be triggered!

The compiler must add any modules as dependencies that are passed in through template arguments as if the concrete type was directly used with any required import.

-- 
Marco

Filed as:
https://issues.dlang.org/show_bug.cgi?id=13712