This has come up in a private discussion with Luís Marques. Consider:

import std.stdio;

import openmethods;
mixin(registerMethods);

class Company
{
  Employee[] employees;

}

class Startup : Company
{

}

class Role
{
  Role[] companies;
}

class Founder : Role
{

}

class Employee : Role
{

}

void main()
{
    // blah blah blah
}

As the program grows bigger, I want to split it into several modules, e.g. one per class or one per hierarchy. Now I have cyclic dependencies between modules because role.d must 'import company' and 'company.d' must 'import role'.

It seems that at this point I have forgone a language feature: static constructors. They may be needed if I want to implement e.g. a global registry in Role and Company that tracks all the instances of the classes in their respective hierarchy.

At this point the only workaround I see is to add base interfaces under Role and Company and establish the bi-directional relationship in terms of the said interfaces.

...or I can throw in that flag that allows cyclic deps in presence of static constructors. Eventually I may run into trouble.

Have I overlooked something?

On Tuesday, 19 December 2017 at 17:43:46 UTC, Jean-Louis Leroy wrote:
> class Role
> {
>   Role[] companies;

I think he meant Company[] companies

On Tuesday, 19 December 2017 at 18:30:22 UTC, Luís Marques wrote:
> On Tuesday, 19 December 2017 at 17:43:46 UTC, Jean-Louis Leroy wrote:
>> class Role
>> {
>>   Role[] companies;
>
> I think he meant Company[] companies

Yes thanks.

On 12/19/17 12:43 PM, Jean-Louis Leroy wrote:

> As the program grows bigger, I want to split it into several modules, e.g. one per class or one per hierarchy. Now I have cyclic dependencies between modules because role.d must 'import company' and 'company.d' must 'import role'.
> 
> It seems that at this point I have forgone a language feature: static constructors. They may be needed if I want to implement e.g. a global registry in Role and Company that tracks all the instances of the classes in their respective hierarchy.

Yes, static constructor dependencies are a very crude mechanism. It assumes if you make an import, anything from that import could possibly be used in the static ctor, and anything in that module could depend on static ctors in that module.

There have been several ideas on how to make them more manageable. But the truth is, it's really hard to come up with ways to detect whether static ctors *really* depend on each other.

What you may consider is to implement the registration outside the module itself. That is, have a helper module that imports the class defining module, which isn't imported from anywhere. One issue there is that the compiler/linker may trim out that module!

It really is a bad mess. I don't know of a good way to solve it, without coming up with an attribute scheme to manually identify the dependencies.

-Steve

On Tuesday, December 19, 2017 17:43:46 Jean-Louis Leroy via Digitalmars-d wrote:
> This has come up in a private discussion with Luís Marques. Consider:
>
> import std.stdio;
>
> import openmethods;
> mixin(registerMethods);
>
> class Company
> {
>    Employee[] employees;
>
> }
>
> class Startup : Company
> {
>
> }
>
> class Role
> {
>    Role[] companies;
> }
>
> class Founder : Role
> {
>
> }
>
> class Employee : Role
> {
>
> }
>
> void main()
> {
>      // blah blah blah
> }
>
> As the program grows bigger, I want to split it into several modules, e.g. one per class or one per hierarchy. Now I have cyclic dependencies between modules because role.d must 'import company' and 'company.d' must 'import role'.
>
> It seems that at this point I have forgone a language feature: static constructors. They may be needed if I want to implement e.g. a global registry in Role and Company that tracks all the instances of the classes in their respective hierarchy.
>
> At this point the only workaround I see is to add base interfaces under Role and Company and establish the bi-directional relationship in terms of the said interfaces.
>
> ...or I can throw in that flag that allows cyclic deps in presence of static constructors. Eventually I may run into trouble.
>
> Have I overlooked something?

The fact that static constructors can't properly handle circular dependencies does mean that there's a tendancy to throw them under the bus at some point, which can be a serious problem. Andrei has been jumping through all kinds of insane hoops recently in an attempt to remove all static constructors from Phobos, which is going a bit far IMHO, but that's open for debate, and if the result works properly, I don't know how much I care, though I'm disinclined to jump through those sort of hoops in my own code. Personally, I tend to use static constructors where appropriate until I find that I can't, and then I refactor, but I end up needing them rarely enough that it usually isn't a problem (though occasionally, it is).

I think that the workarounds for static constructors tend to fall into one of two camps:

1. Lazily load whatever you were trying to initialize via a static constructor. It seems like sometimes this works somewhat cleanly, and sometimes the loops that you have to jump through to make it work are just crazy. In the case of std.datetime's LocalTime class, which is both immutable and a singleton, that meant having to carefully use casting to violate pure while not actually violating pure's guarantees. It's not pretty, but it works, and it's fairly straightforward. In other cases, things are much more complicated. Improvements to CTFE have mode this sort of thing less of an issue but haven't completely fixed it, since some stuff simply has to be done at runtime (e.g. UTC's singleton can now be constructed at compile time instead of using the same trick, but LocalTime can't, because it has to run some stuff at runtime when it's constructed).

2. Create another module which contains the static constructor and is imported by the module which is supposed to have the static constructor. std.stdio used to do this for its shared static constructor for initializing stdin, stdio, and stderr. We had std/stdiobase.d:

---------------------------------------
module std.stdiobase;

extern(C) void std_stdio_static_this();

shared static this()
{
    std_stdio_static_this();
}
---------------------------------------

and then inside of std.stdio, we had std_stdio_static_this with the actual implementation of the shared static constructor. This only works in cases where what you're trying to do in a static constructor doesn't _have_ to be done in a static constructor (e.g. initializing an immutable variable won't work), but it does allow you to work around the circular dependency problem in a lot of cases.

And of course, neither of these solutions work if what the static constructors are doing is actually circularly dependent. The type declarations can be circularly dependent, but if the order that the static constructors run in matters, you're screwed. And that's exactly what the runtime is telling you when it throws an Error about circular imports. It's just that it's not smart enough to figure out whether what the static constructors are actually doing is circularly dependent.

Depending on what your problem is, you may just need to put up with not splitting up your modules with static constructors - though if all you're really looking to do is split up the public API, then you can always have a larger module to avoid the circular imports and then have separate modules that publicly import the pieces to provide the broken up API. That's not terribly pleasant though.

In the past, I suggested that we have some sort of attribute for a static constructor that tells the compiler and runtime that there isn't really a circular dependency - either by indicating that it isn't circularly dependent on anything or by telling it what the dependency order is so that the load order for the modules can be dealt with correctly - but Walter rejected the idea on the grounds that it was too risky. Not only would it be easy to get wrong to begin with, but if the code changed later, actual, circular dependencies could go unnoticed and result in some pretty nasty bugs.

Unfortunately, ultimately, static constructors are a bit of a problem feature. They're required to make certain things in the language work (especially if you don't want to do hacky things with casts), but we've never been able to come up with a way to cleanly deal with modules that import each other (even indirectly) once static constructors are involved. It seems that what the compiler and runtime know about how circularly dependent static constructors actually are is just too poor. :(

However, for something like a global registry of classes, I expect that the old std.stdio solution would work just fine, since that doesn't have anything to do with initializing the stuff that's circularly dependent, just using it.

- Jonathan M Davis

On 12/19/17 2:04 PM, Jonathan M Davis wrote:
> 
> The fact that static constructors can't properly handle circular
> dependencies does mean that there's a tendancy to throw them under the bus
> at some point, which can be a serious problem. Andrei has been jumping
> through all kinds of insane hoops recently in an attempt to remove all
> static constructors from Phobos, which is going a bit far IMHO, but that's
> open for debate, and if the result works properly, I don't know how much I
> care, though I'm disinclined to jump through those sort of hoops in my own
> code.

But that's not because of circular dependencies, that's to allow those pieces of phobos to be independent of druntime.

However, it does (as a side effect) eliminate the need for proper construction order, at the cost of more expensive access to the global data.

> In the past, I suggested that we have some sort of attribute for a static
> constructor that tells the compiler and runtime that there isn't really a
> circular dependency - either by indicating that it isn't circularly
> dependent on anything or by telling it what the dependency order is so that
> the load order for the modules can be dealt with correctly - but Walter
> rejected the idea on the grounds that it was too risky. Not only would it be
> easy to get wrong to begin with, but if the code changed later, actual,
> circular dependencies could go unnoticed and result in some pretty nasty
> bugs.

But this doesn't even solve the problem of decoupling the grouping of types/data into a related module from the problem of static construction. That is, you may import other modules for reasons totally separate from static construction.

The only idea I had that makes things a bit better is to ignore imports inside unit tests (since those are only ever run after static construction anyway).

Maybe there are some other ways to do it. But none of them are going to be painless.

-Steve

On Tuesday, December 19, 2017 16:09:41 Steven Schveighoffer via Digitalmars- d wrote:
> On 12/19/17 2:04 PM, Jonathan M Davis wrote:
> > The fact that static constructors can't properly handle circular dependencies does mean that there's a tendancy to throw them under the bus at some point, which can be a serious problem. Andrei has been jumping through all kinds of insane hoops recently in an attempt to remove all static constructors from Phobos, which is going a bit far IMHO, but that's open for debate, and if the result works properly, I don't know how much I care, though I'm disinclined to jump through those sort of hoops in my own code.
>
> But that's not because of circular dependencies, that's to allow those pieces of phobos to be independent of druntime.
>
> However, it does (as a side effect) eliminate the need for proper construction order, at the cost of more expensive access to the global data.

Either way, it's bending over backwards to avoid a language feature that's supposed to perfectly fine to use. And I confess that I'm getting a bit tired of the stuff in druntime or Phobos being contorted to avoid certain language features just to satisfy some of the more extreme segments of the community.

> > In the past, I suggested that we have some sort of attribute for a
> > static
> > constructor that tells the compiler and runtime that there isn't really
> > a
> > circular dependency - either by indicating that it isn't circularly
> > dependent on anything or by telling it what the dependency order is so
> > that the load order for the modules can be dealt with correctly - but
> > Walter rejected the idea on the grounds that it was too risky. Not only
> > would it be easy to get wrong to begin with, but if the code changed
> > later, actual, circular dependencies could go unnoticed and result in
> > some pretty nasty bugs.
>
> But this doesn't even solve the problem of decoupling the grouping of types/data into a related module from the problem of static construction. That is, you may import other modules for reasons totally separate from static construction.

Having an attribute to kill the circular dependency complaint would solve the case where we have something like std/stdiobase.d with a static constructor calling a function that has the actual static constructor code - and it would do so without sacrificing the ability to do stuff like initialize immutable variables. I don't think that it would really solve anything beyond that. A "perfect" solution would involve the compiler and runtime figuring out the exact order to run static constructors in spite of the circular imports and only complaining when it's actually not possible to order them correctly, because they're truly circularly dependent, but unfortunately, that's really too hard to do, and as annoying as the current solution is, it's arguably a good solution given the constraints.

> The only idea I had that makes things a bit better is to ignore imports inside unit tests (since those are only ever run after static construction anyway).
>
> Maybe there are some other ways to do it. But none of them are going to be painless.

Yeah, I don't know. It might also be possible to figure something out based on which imports are local, but I'm not sure. It wouldn't take much for even a moderate improvement to get way too complicated to be feasible.

- Jonathan M Davis