View mode: basic / threaded / horizontal-split · Log in · Help
November 29, 2012
Re: Fixing cyclic import static construction problems
On Thursday, 29 November 2012 at 12:17:49 UTC, Jonathan M Davis 
wrote:
> On Thursday, November 29, 2012 12:39:19 Paulo Pinto wrote:
>> On Thursday, 29 November 2012 at 03:19:55 UTC, Andrei
>> 
>> Alexandrescu wrote:
>> > On 11/28/12 9:34 PM, Walter Bright wrote:
>> >> For discussion:
>> > [snip]
>> > 
>> > I'd say we better finish const, immutable, and shared first.
>> > 
>> > Andrei
>> 
>> +1
>> 
>> Fully agree.
>> 
>> Cyclic imports are a minor nuisance that can be easily solvable
>> with better code architecture.
>> 
>> Turbo Pascal/Delphi is the only language that I know fully 
>> allows
>> cyclic dependencies between modules. So this is not that
>> important for most people.
>
> Basic features in the language require static constructors 
> (e.g. static
> variables frequently do), and some things just can't be done 
> without them.
> Andrei's std.benchmark proposal actually doesn't work, because 
> in order to do
> what it does, in needs to mixin in a static constructor and 
> static destructor
> (to set up the benchmarking and record it at the end, I 
> believe). That
> completely falls apart due to cyclical imports. It doesn't 
> actually cause any
> true circular dependencies, but it's treated as such anyway. A 
> _lot_ of us
> have run into this problem and a number of us consider it to be 
> a huge design
> problem in the language. Personally, I'd put it towards the top 
> of design
> mistakes at this point, and I'm very glad to see Walter 
> actually finally being
> willing to do something about this. In the past when I've 
> brought up similar
> solutions, he's been completely opposed to them.
>
> Yes, we have other major issues that need to be resolved, and 
> those may very
> well be of a higher priority, but this is still a very import 
> issue, and it's
> the sort of issue that can probably be implemented with minimal 
> effort. The
> main thing is getting Walter to agree to it and how it will be 
> done. Once a
> decision is made, it wouldn't surprise me if someone like Kenji 
> were able to
> get it done very quickly.
>
> - Jonathan M Davis

Maybe I should keep my mouth shut, because actually I don't 
really use D besides some toy experiments, as there is no place 
for D in the type of work I currently do, either on the job or 
privately.

I like however to point people to D as a possible C++ successor 
as way to do some language publicity, and think that these type 
of discussions might scare D newbies away in what concerns 
language stability, hence my post.

--
Paulo
November 29, 2012
Re: Fixing cyclic import static construction problems
11/29/2012 4:17 PM, Jonathan M Davis пишет:
> On Thursday, November 29, 2012 12:39:19 Paulo Pinto wrote:
>> On Thursday, 29 November 2012 at 03:19:55 UTC, Andrei
>>
>> Alexandrescu wrote:
>>> On 11/28/12 9:34 PM, Walter Bright wrote:
>>>> For discussion:
>>> [snip]
>>>
>>> I'd say we better finish const, immutable, and shared first.
>>>
>>> Andrei
>>
>> +1
>>
>> Fully agree.
>>
>> Cyclic imports are a minor nuisance that can be easily solvable
>> with better code architecture.
>>
>> Turbo Pascal/Delphi is the only language that I know fully allows
>> cyclic dependencies between modules. So this is not that
>> important for most people.
>
> Basic features in the language require static constructors (e.g. static
> variables frequently do), and some things just can't be done without them.
> Andrei's std.benchmark proposal actually doesn't work, because in order to do
> what it does, in needs to mixin in a static constructor and static destructor
> (to set up the benchmarking and record it at the end, I believe). That
> completely falls apart due to cyclical imports.

I do suspect it could be healed by inverting the control flow (or rather 
bringing it back).

Basically there is this pattern:

module a;
...
mixin ScheduleForBenchmark;

Same for module b, c, d...

And then compiling all of this with -version benchmark should run the 
benchmarks.

It already has another problem: there has to be D main stuffed in there 
somehow. It can't be in std.benchmark as it then will have to be 
included in phobos(.lib|.a) and it can't be magically turned on/off with 
-version switch either unless std.benchmark is passed directly to the 
compiler.

The other solution would be: make a separate my_benchmark.d module that 
contains:
import a,b,c,d...;

void main(){
   runBenchmarks!(a,b,c,d...)();
}

And that's it.
In any case I usually like having some separation between different sets 
of modules to benchmark so that I can start them in a 'clean room' scenario.

> It doesn't actually cause any
> true circular dependencies, but it's treated as such anyway. A _lot_ of us
> have run into this problem and a number of us consider it to be a huge design
> problem in the language. Personally, I'd put it towards the top of design
> mistakes at this point, and I'm very glad to see Walter actually finally being
> willing to do something about this. In the past when I've brought up similar
> solutions, he's been completely opposed to them.
>
> Yes, we have other major issues that need to be resolved, and those may very
> well be of a higher priority, but this is still a very import issue, and it's
> the sort of issue that can probably be implemented with minimal effort. The
> main thing is getting Walter to agree to it and how it will be done. Once a
> decision is made, it wouldn't surprise me if someone like Kenji were able to
> get it done very quickly.
>
> - Jonathan M Davis
>
-- 
Dmitry Olshansky
November 29, 2012
Re: Fixing cyclic import static construction problems
On 11/29/12 10:54 AM, Dmitry Olshansky wrote:
> Basically there is this pattern:
>
> module a;
> ...
> mixin ScheduleForBenchmark;
>
> Same for module b, c, d...

Correct.

> And then compiling all of this with -version benchmark should run the
> benchmarks.
>
> It already has another problem: there has to be D main stuffed in there
> somehow. It can't be in std.benchmark as it then will have to be
> included in phobos(.lib|.a) and it can't be magically turned on/off with
> -version switch either unless std.benchmark is passed directly to the
> compiler.
>
> The other solution would be: make a separate my_benchmark.d module that
> contains:
> import a,b,c,d...;
>
> void main(){
> runBenchmarks!(a,b,c,d...)();
> }

That's workable. I'm hoping, however, to make benchmarks as easy and as 
trivial to define as unittests.


Andrei
November 29, 2012
Re: Fixing cyclic import static construction problems
On Thursday, 29 November 2012 at 15:18:11 UTC, Paulo Pinto wrote:
> On Thursday, 29 November 2012 at 12:04:28 UTC, Max Samukha 
> wrote:
>> On Thursday, 29 November 2012 at 11:39:20 UTC, Paulo Pinto 
>> wrote:
>>> On Thursday, 29 November 2012 at 03:19:55 UTC, Andrei 
>>> Alexandrescu wrote:
>>>> On 11/28/12 9:34 PM, Walter Bright wrote:
>>>>> For discussion:
>>>> [snip]
>>>>
>>>> I'd say we better finish const, immutable, and shared first.
>>>>
>>>> Andrei
>>>
>>> +1
>>>
>>> Fully agree.
>>>
>>> Cyclic imports are a minor nuisance that can be easily 
>>> solvable with better code architecture.
>>
>> Show me please how to solve that problem easily with 
>> acceptable results, would you?
>
> You just need to have a better architecture.
>
> In 20 years of software development experience I never found a 
> case were this wasn't possible.

That's an argument from authority, sorry.

>
> Maybe you care to provide an example?
>

The general problem is constructing global data structures based 
on data introspected at compile-time.

My specific problem is extending scarce runtime type information 
provided by the language with something usable for runtime 
reflection. With lots of detail omitted:

module reflect;

Meta[string] metas;
mixin template Reflect(alias object) {
    static this()
    {
        auto m = meta!(object);
        metas[m.fullName] ~= m;
    }
}


module a;
import reflect;

struct S
{
}
mixin Reflect!S;

The meta-object for S is automatically made available at runtime 
through the global metas array. Note that we do not want to force 
the user to register the meta-object manually because then it 
would not be a "better architecture".

The important (Andrei somehow thinks it is not) requirement is 
there must not be circular dependency issues for the users of the 
"reflect" module.
November 29, 2012
Re: Fixing cyclic import static construction problems
On Thursday, 29 November 2012 at 12:17:49 UTC, Jonathan M Davis 
wrote:
> Basic features in the language require static constructors 
> (e.g. static
> variables frequently do), and some things just can't be done 
> without them.
> Andrei's std.benchmark proposal actually doesn't work, because 
> in order to do
> what it does, in needs to mixin in a static constructor and 
> static destructor
> (to set up the benchmarking and record it at the end, I 
> believe). That
> completely falls apart due to cyclical imports. It doesn't 
> actually cause any
> true circular dependencies, but it's treated as such anyway. A 
> _lot_ of us
> have run into this problem and a number of us consider it to be 
> a huge design
> problem in the language. Personally, I'd put it towards the top 
> of design
> mistakes at this point, and I'm very glad to see Walter 
> actually finally being
> willing to do something about this. In the past when I've 
> brought up similar
> solutions, he's been completely opposed to them.
>
> Yes, we have other major issues that need to be resolved, and 
> those may very
> well be of a higher priority, but this is still a very import 
> issue, and it's
> the sort of issue that can probably be implemented with minimal 
> effort. The
> main thing is getting Walter to agree to it and how it will be 
> done. Once a
> decision is made, it wouldn't surprise me if someone like Kenji 
> were able to
> get it done very quickly.
>
> - Jonathan M Davis

That is understood, but Let me explain how I see things.

We are here adding yet a new feature, however small. Nothing 
stabilize when adding new feature all the time.

The annoyance exist, is real, but is not THAT bad, and 'm pretty 
sure the proposed solution is likely to backfire. Firstly because 
a module can have several constructors, and more can be added by 
mixins. Ensuring that constructor A will not create dependancy 
cycle error may silently break other constructor of the module.

Considering the problem static constructor solve, I'd be happy to 
see some think out of the box. The way things work here may be 
broken. Many people consider that D runtime should evolve in a 
way that promote fibers and map them over system threads (this 
have many benefit : the program adapt autmatically to the 
provided hardware, yields can be introduced directly into the 
runtime or some libraries to hide IO latencies, and many other 
things), and static constructor/variables really get into the way 
(you would have to run all static constructors each time you 
start a new fiber or reset one to reuse it).

I really wish we can focus on figuring out the uses cases we have 
for static constructor and start the reflection from theses sues 
case and not from the feature itself.
November 29, 2012
Re: Fixing cyclic import static construction problems
On 2012-11-29 16:58, Andrei Alexandrescu wrote:

> That's workable. I'm hoping, however, to make benchmarks as easy and as
> trivial to define as unittests.

Unit tests have the same problem. One need to create a module containing 
a main function and importing the modules one wants to benchmark/test.

One idea would be to add more support for this in the compiler.

Another idea, that might better and is easier to implement, is tool that:

1. Takes a couple of modules on the command line
2. Create a main module which imports the modules on the command line
3. Compile all the modules
4. Run the benchmarks or tests

-- 
/Jacob Carlborg
November 29, 2012
Re: Fixing cyclic import static construction problems
On 2012-11-29 19:17, deadalnix wrote:

> That is understood, but Let me explain how I see things.
>
> We are here adding yet a new feature, however small. Nothing stabilize
> when adding new feature all the time.
>
> The annoyance exist, is real, but is not THAT bad, and 'm pretty sure
> the proposed solution is likely to backfire. Firstly because a module
> can have several constructors, and more can be added by mixins. Ensuring
> that constructor A will not create dependancy cycle error may silently
> break other constructor of the module.
>
> Considering the problem static constructor solve, I'd be happy to see
> some think out of the box. The way things work here may be broken. Many
> people consider that D runtime should evolve in a way that promote
> fibers and map them over system threads (this have many benefit : the
> program adapt autmatically to the provided hardware, yields can be
> introduced directly into the runtime or some libraries to hide IO
> latencies, and many other things), and static constructor/variables
> really get into the way (you would have to run all static constructors
> each time you start a new fiber or reset one to reuse it).
>
> I really wish we can focus on figuring out the uses cases we have for
> static constructor and start the reflection from theses sues case and
> not from the feature itself.

BTW, how does Java handle this? And C# if it has something similar.

-- 
/Jacob Carlborg
November 29, 2012
Re: Fixing cyclic import static construction problems
Max Samukha wrote:

> there must not be circular dependency issues

The model shown does not force circular dependencies, because 
every receiver module only imports one module: `module reflect'. 
And I can not see, that this module imports any other module.

Therefore `module relect' and its `import' statements are not 
responsible for any circular dependency issue.

Where am I wrong?

-manfred
November 29, 2012
Re: Fixing cyclic import static construction problems
On Thursday, November 29, 2012 16:18:10 Paulo Pinto wrote:
> On Thursday, 29 November 2012 at 12:04:28 UTC, Max Samukha wrote:
> > On Thursday, 29 November 2012 at 11:39:20 UTC, Paulo Pinto
> > 
> > wrote:
> >> On Thursday, 29 November 2012 at 03:19:55 UTC, Andrei
> >> 
> >> Alexandrescu wrote:
> >>> On 11/28/12 9:34 PM, Walter Bright wrote:
> >>>> For discussion:
> >>> [snip]
> >>> 
> >>> I'd say we better finish const, immutable, and shared first.
> >>> 
> >>> Andrei
> >> 
> >> +1
> >> 
> >> Fully agree.
> >> 
> >> Cyclic imports are a minor nuisance that can be easily
> >> solvable with better code architecture.
> > 
> > Show me please how to solve that problem easily with acceptable
> > results, would you?
> 
> You just need to have a better architecture.
> 
> In 20 years of software development experience I never found a
> case were this wasn't possible.
> 
> Maybe you care to provide an example?

Considering that you need static constructors to initialize any static 
variables at runtime (and in the case of const and immutable variables, you 
_can't_ work around that by doing the initialization later - it _must_ be in 
the static constructor), and all it takes for the compiler to declare a 
circular import is to have two modules import each other - even indirectly - 
it becomes extremely easy to run into this problem. And if you're dealing with 
immutable static variables which are initialized at runtime, you can't fix it 
unless you can contort your modules so that they don't depend on each other, 
and with a lot of modules, that can become extremely difficult. We had to strip 
out all static constructors from std.datetime because of this, and in order to 
fix it, we had to do some nasty voodoo with casting in order to lazily 
initialize some variables which are supposed to be immutable. _None_ of that 
sort of thing should be necessary. As it stands, it's basically bad practice 
to use static constructors, because it's so easy to end up with circular 
dependencies, and they can involve modules which don't seem even vaguely 
related because of other stuff that they import and are often a royal pain to 
debug and fix, if you even can without seriously revamping your design. And for 
a library like Phobos which needs to avoid breaking backwards compatibility, 
redesigning things to avoid such circular dependencies isn't necessarily even 
possible, because those redesigns would break backwards compatibliity.

We _need_ a solution for this. Whether now is the best time tackle it is 
another matter, but the current situation with static constructors is 
ridiculous. There are features which require them, but you have to avoid them 
or you're going to run into circular dependency issues very easily.

This is a case of some great design decisions in D have leading to a very bad 
design, and it needs to be fixed. Fortunately, it shouldn't be all that hard to 
fix with an attribute of some kind. But prior to now, Walter wouldn't even 
consider it. I do think though that the idea of using only one pragma instead 
of an attribute/pragma per static constructor is a bad idea because of the 
silent breakage that it would cause, as has been pointed out in this thread. 
So, his proposed solution needs some tweaking.

- Jonathan M Davis
November 29, 2012
Re: Fixing cyclic import static construction problems
On Thursday, November 29, 2012 21:08:58 Jacob Carlborg wrote:
> BTW, how does Java handle this? And C# if it has something similar.

They just let you blow your foot off. All static variables can be directly 
initialized at runtime, so it's easy to use variables before they're actually 
initialized. I don't know how they decide what order to run static 
constructors in, but AFAIK, it never worries about circular dependencies. 
We're only running into this problem beacuse we're trying to provide higher 
safety and better guarantees with regards to when and how variables are 
initialized.

- Jonathan M Davis
1 2 3 4 5 6 7
Top | Discussion index | About this forum | D home