On 12/31/2016 12:20 PM, Martin Nowak wrote:
> On Saturday, 24 December 2016 at 10:54:08 UTC, Stefan Koch wrote:
>> If that were made more lazy, we could import half of the world with
>> noticing impact.
>>
>> (Which espcially in std.traits, would not make that much of a
>> difference since every template in there depends on nearly every other
>> template in there)
>
> Also the established technique of serializing precompiled AST (after
> semantic3) of modules to a cache should be applicable as well.
>
> Cross-posting from
> https://github.com/dlang/DIPs/pull/51#issuecomment-269107966, b/c it
> wasn't answered yet.

No worries, I'll make a pass (a rewrite, really) taking all feedback into account.

> Were any other means considered? This is proposing to add plenty of
> additional annotations only to speed up compilation, but none of the
> classical tools for pre-compilation were assessed.
> Since D's modules don't have the header problem, even pre-compilation
> and reuse of semantic3 should be possible, or not?

DIP1005 gives consideration to the speed of compilation aspect in larger proportion than speed's importance; the first and foremost benefit of DIP1005 is it closes the gap on dependency encapsulation, which had been very successfully narrowed by local imports. However, the DIP will keep the experiments and results on speed measurements because they are relevant to it and any related methods of lazily loading of modules.

Regarding the ongoing doubts about the advantages of inline imports: they are first and foremost a completion of the nested import feature. As such, most, if not all, arguments against inline imports apply equally to nested imports. Come to think of it, lazy imports vs nested imports:

* same improvement in compilation speed? check
* no language changes? check
* no nasty bugs in the aftermath (such as the infamous https://issues.dlang.org/show_bug.cgi?id=10378)? check
* scalable builds? check

Yet local imports are overwhelmingly superior to lazy imports because of one thing: they localize dependencies. They introduce modularity and its ancillary perks (fast and scalable builds, easier review and refactoring) not by engineering, but by organically placing dependencies spatially with their dependents. (The scope statement does the same thing with temporal dependencies.) That the DIP does not make it clear that it is a necessary and sufficient extension of local imports is a problem with it.

I now am really glad we slipped local imports before the formalization of DIPs. The feature could have been easily demeaned out of existence.

Allow me to make an appeal regarding the review of any DIP. There seems to be a tendency of some reviewers to get attached and emotionally invested to their opinion, to the extent they'd be hurt by being "wrong" and would go to great lengths to argue they're "right". This has obvious negative effects on the entire process. Please don't. There's no loss of face to worry about. The only commitment we all should have is to the good of the D language. If DIP1005 reaches the conclusion of its own uselessness, I'd be the first one to write it up and close the PR.


Thanks,

Andrei

On Monday, 2 January 2017 at 21:23:19 UTC, Andrei Alexandrescu wrote:
> DIP1005 gives consideration to the speed of compilation aspect in larger proportion than speed's importance; the first and foremost benefit of DIP1005 is it closes the gap on dependency encapsulation, which had been very successfully narrowed by local imports.
I love that idea. But I still can't see why this requires any new syntax. Simply extending the scope of local inports to include the function header is enough.
Only for .di-generation it may be useful to move all local imports to the declaration (maybe with this new syntax "with" before it) - but that should be done with ALL local imports, because today the .di-files are incomplete and will stay so if the new syntax is introduced but "old-style" local imports still valid and not exported to the .di.
Or the old local imports become deprecated together with the introduction of the new "with" syntax and vanish soon after that.

On Monday, 2 January 2017 at 21:23:19 UTC, Andrei Alexandrescu wrote:
> Regarding the ongoing doubts about the advantages of inline imports: they are first and foremost a completion of the nested import feature. As such, most, if not all, arguments against inline imports apply equally to nested imports. Come to think of it, lazy imports vs nested imports:
>
> * same improvement in compilation speed? check
> * no language changes? check
> * no nasty bugs in the aftermath (such as the infamous https://issues.dlang.org/show_bug.cgi?id=10378)? check
> * scalable builds? check
>
> Yet local imports are overwhelmingly superior to lazy imports because of one thing: they localize dependencies. They introduce modularity and its ancillary perks (fast and scalable builds, easier review and refactoring) not by engineering, but by organically placing dependencies spatially with their dependents. (The scope statement does the same thing with temporal dependencies.) That the DIP does not make it clear that it is a necessary and sufficient extension of local imports is a problem with it.
>
> I now am really glad we slipped local imports before the formalization of DIPs. The feature could have been easily demeaned out of existence.

Except that almost nobody has argued against local imports.  Rather, the argument is that local imports mostly solved this problem, so why bother adding new syntax for the dozen remaining symbols from 2-3 modules that are commonly used in template constraints?

Arguing that local imports have been successful so we should simply do more of it is not a good argument, as there comes a point of diminishing returns.  You need to show that there are still worthile gains to be made from changing the language again, which is why I want to benchmark this feature with Walter before deciding.

> Allow me to make an appeal regarding the review of any DIP. There seems to be a tendency of some reviewers to get attached and emotionally invested to their opinion, to the extent they'd be hurt by being "wrong" and would go to great lengths to argue they're "right". This has obvious negative effects on the entire process. Please don't. There's no loss of face to worry about. The only commitment we all should have is to the good of the D language. If DIP1005 reaches the conclusion of its own uselessness, I'd be the first one to write it up and close the PR.

We could level this analysis back at you: you consider this DIP so "obvious" that you are not engaging with our concerns, making flip, incorrect remarks about how we would have bikeshedded local imports also.

In the end, this is a minor DIP that is easily bikeshedded, as everybody can grasp it and have an opinion on it.  I have refrained from commenting recently because I will let benchmarking settle it for me.  Obviously, that won't suffice for others.

On Tue, 03 Jan 2017 07:19:41 +0000, Dominikus Dittes Scherkl wrote:

> On Monday, 2 January 2017 at 21:23:19 UTC, Andrei Alexandrescu wrote:
>> DIP1005 gives consideration to the speed of compilation aspect in larger proportion than speed's importance; the first and foremost benefit of DIP1005 is it closes the gap on dependency encapsulation, which had been very successfully narrowed by local imports.
> I love that idea. But I still can't see why this requires any new syntax. Simply extending the scope of local inports to include the function header is enough.

You mean:

  Socket connect()
  {
    import std.socket;
    return new Socket();
  }

Which should already work if you used auto for the return type.

On Monday, 2 January 2017 at 21:23:19 UTC, Andrei Alexandrescu wrote:
> On 12/31/2016 12:20 PM, Martin Nowak wrote:
>> On Saturday, 24 December 2016 at 10:54:08 UTC, Stefan Koch wrote:
>>> If that were made more lazy, we could import half of the world with
>>> noticing impact.
>>>
>>> (Which espcially in std.traits, would not make that much of a
>>> difference since every template in there depends on nearly every other
>>> template in there)
>>
>> Also the established technique of serializing precompiled AST (after
>> semantic3) of modules to a cache should be applicable as well.
>>
>> Cross-posting from
>> https://github.com/dlang/DIPs/pull/51#issuecomment-269107966, b/c it
>> wasn't answered yet.
>
> No worries, I'll make a pass (a rewrite, really) taking all feedback into account.
>
>> Were any other means considered? This is proposing to add plenty of
>> additional annotations only to speed up compilation, but none of the
>> classical tools for pre-compilation were assessed.
>> Since D's modules don't have the header problem, even pre-compilation
>> and reuse of semantic3 should be possible, or not?
>
> DIP1005 gives consideration to the speed of compilation aspect in larger proportion than speed's importance; the first and foremost benefit of DIP1005 is it closes the gap on dependency encapsulation, which had been very successfully narrowed by local imports. However, the DIP will keep the experiments and results on speed measurements because they are relevant to it and any related methods of lazily loading of modules.
>
> Regarding the ongoing doubts about the advantages of inline imports: they are first and foremost a completion of the nested import feature. As such, most, if not all, arguments against inline imports apply equally to nested imports. Come to think of it, lazy imports vs nested imports:
>
> * same improvement in compilation speed? check
> * no language changes? check
> * no nasty bugs in the aftermath (such as the infamous https://issues.dlang.org/show_bug.cgi?id=10378)? check
> * scalable builds? check
>
> Yet local imports are overwhelmingly superior to lazy imports because of one thing: they localize dependencies. They introduce modularity and its ancillary perks (fast and scalable builds, easier review and refactoring) not by engineering, but by organically placing dependencies spatially with their dependents. (The scope statement does the same thing with temporal dependencies.) That the DIP does not make it clear that it is a necessary and sufficient extension of local imports is a problem with it.
>
> I now am really glad we slipped local imports before the formalization of DIPs. The feature could have been easily demeaned out of existence.
>
> Allow me to make an appeal regarding the review of any DIP. There seems to be a tendency of some reviewers to get attached and emotionally invested to their opinion, to the extent they'd be hurt by being "wrong" and would go to great lengths to argue they're "right". This has obvious negative effects on the entire process. Please don't. There's no loss of face to worry about. The only commitment we all should have is to the good of the D language. If DIP1005 reaches the conclusion of its own uselessness, I'd be the first one to write it up and close the PR.
>
>
> Thanks,
>
> Andrei

Allow me just to share a worthless outsider opinion.

I never contributed anything worthwhile and decided it was better to just focus on JVM, .NET languages., alongside C++, as those are the skills I get paid for, thus stop polluting D forums.

Looking from the outside, and watching what was reached from 2016 roadmap, it is clear the DIPs evaluated thus dar aren't about fixing the library or runtime issues that prevent D's adoption at large as a systems programming language.

Meanwhile Swift, Go and Rust have a clear roadmap how their future is supposed to look like, and drive just in that direction, with C++ taking all remaining good D ideas.

This DIP discussion and the latest ones about splitting the runtime again, don't do anything to earn D any credibility it might still have left.

On Monday, 2 January 2017 at 21:23:19 UTC, Andrei Alexandrescu wrote:
>
> No worries, I'll make a pass (a rewrite, really) taking all feedback into account.

Are all the proposals mutually exclusive? When it comes to declarations, the DIP actually won me over! However I was hoping for it to solve a different issue inside the body with local imports. I tend to write short helper functions, with the result that in the common case, an imported symbol is often used only once per function.

Currently I consider it good style to list all imports in the beginning of a scope, but based on past experience with C89, this is not entirely optimal, however Timon Gehr's proposal could solve it.

import and directly invoke:
import std.traits : isArray(...)

=======================================================
[Proposed Style - analogous to C99]
body
{
  static if(...)
    import std.range : zip(...);
  static if(import std.traits : isArray(...)))
    import std.range : join(...);
}
=======================================================
[Old Style - analogous to C89 - Suffers from "unused" imports]
body
{
  import std.range : zip, join;
  import std.traits : isArray;

  static if(...)
    zip(...)
  static if(isArray(...)))
    join(...)
}
=======================================================
[Alternate Style - Suffers from DRY and can't limit scope of isArray]
body
{
  import std.traits : isArray;

  static if(...)
  {
    import std.range : zip;
    zip(...);
  }
  static if(is(isArray(...)))
  {
    import std.range : join;
    join(...);
  }
}

C89 vs C99 style I was referring to if it wasn't clear.
=======================================================
[C89]
void foo(void)
{
#ifdef ... /* needed to silence unused variable warnings */
  int x;
#endif
... lots of code ...
#ifdef ...
  x = bar();
#endif
}

[C99]
void foo(void)
{
#ifdef ...
  int x = bar();
#endif
}
=======================================================

On Tuesday, 3 January 2017 at 10:18:57 UTC, Daniel N wrote:
> static if(is(isArray(...)))
> {
>   import std.range : join;
>   join(...);
> }

PS Actually double {} is needed for the 'alternate' style to be meaningful, preventing the import from bleeding into the outer scope.

static if(isArray(...))
{{
  import std.range : join;
  join(...);
}}

On 01/03/2017 02:19 AM, Dominikus Dittes Scherkl wrote:
> On Monday, 2 January 2017 at 21:23:19 UTC, Andrei Alexandrescu wrote:
>> DIP1005 gives consideration to the speed of compilation aspect in
>> larger proportion than speed's importance; the first and foremost
>> benefit of DIP1005 is it closes the gap on dependency encapsulation,
>> which had been very successfully narrowed by local imports.
> I love that idea. But I still can't see why this requires any new
> syntax. Simply extending the scope of local inports to include the
> function header is enough.

I'll mention this possibility in the DIP.

> Only for .di-generation it may be useful to move all local imports to
> the declaration (maybe with this new syntax "with" before it) - but that
> should be done with ALL local imports, because today the .di-files are
> incomplete and will stay so if the new syntax is introduced but
> "old-style" local imports still valid and not exported to the .di.
> Or the old local imports become deprecated together with the
> introduction of the new "with" syntax and vanish soon after that.

Local imports don't pertain to the interface, they are an implementation detail. Why would those be made a part of the .di?


Andrei

On Tue, 03 Jan 2017 08:10:04 +0000, Joakim wrote:
> Except that almost nobody has argued against local imports. Rather, the argument is that local imports mostly solved this problem, so why bother adding new syntax for the dozen remaining symbols from 2-3 modules that are commonly used in template constraints?

That was my misunderstanding, and it only applied to the speed argument anyway.

This is usable with top-level functions, top-level templates, and classes with inheritance.

On 01/03/2017 03:10 AM, Joakim wrote:
> On Monday, 2 January 2017 at 21:23:19 UTC, Andrei Alexandrescu wrote:
>> Regarding the ongoing doubts about the advantages of inline imports:
>> they are first and foremost a completion of the nested import feature.
>> As such, most, if not all, arguments against inline imports apply
>> equally to nested imports. Come to think of it, lazy imports vs nested
>> imports:
>>
>> * same improvement in compilation speed? check
>> * no language changes? check
>> * no nasty bugs in the aftermath (such as the infamous
>> https://issues.dlang.org/show_bug.cgi?id=10378)? check
>> * scalable builds? check
>>
>> Yet local imports are overwhelmingly superior to lazy imports because
>> of one thing: they localize dependencies. They introduce modularity
>> and its ancillary perks (fast and scalable builds, easier review and
>> refactoring) not by engineering, but by organically placing
>> dependencies spatially with their dependents. (The scope statement
>> does the same thing with temporal dependencies.) That the DIP does not
>> make it clear that it is a necessary and sufficient extension of local
>> imports is a problem with it.
>>
>> I now am really glad we slipped local imports before the formalization
>> of DIPs. The feature could have been easily demeaned out of existence.
>
> Except that almost nobody has argued against local imports.

I don't have time to research this, but my recollection is that at least some framed the bugs regarding lookups as a fundamental problem of local imports, not a simple matter of getting it right.

Overall, yes, local imports have been a success (really saving scalability of large project builds which looked pretty bleak at a time), which should increase trust in the authors of the feature... hmmm... :o)

> Rather, the
> argument is that local imports mostly solved this problem, so why bother
> adding new syntax for the dozen remaining symbols from 2-3 modules that
> are commonly used in template constraints?

I understand. It is my humble opinion that we are "mostly pregnant" for as long as we require top-level imports. The real change of phase occurs when there are ZERO imports at top level. That's the prize DIP1005 is after.

> Arguing that local imports have been successful so we should simply do
> more of it is not a good argument, as there comes a point of diminishing
> returns.  You need to show that there are still worthile gains to be
> made from changing the language again, which is why I want to benchmark
> this feature with Walter before deciding.

You can be reasonably certain the benchmarks will improve as projected - starting in proportion to the the transitive fanout of top-level imports (10.5x for the standard library) and going down thereafter with the size, complexity, and actual dependencies of the module being compiled. All in all measurable but not dramatic. The same improvement will be brought about by lazy imports, and it won't be the deal maker/breaker. If you're waiting for the numbers to get convinced of anything, you already consider DIP1005 useless.

>> Allow me to make an appeal regarding the review of any DIP. There
>> seems to be a tendency of some reviewers to get attached and
>> emotionally invested to their opinion, to the extent they'd be hurt by
>> being "wrong" and would go to great lengths to argue they're "right".
>> This has obvious negative effects on the entire process. Please don't.
>> There's no loss of face to worry about. The only commitment we all
>> should have is to the good of the D language. If DIP1005 reaches the
>> conclusion of its own uselessness, I'd be the first one to write it up
>> and close the PR.
>
> We could level this analysis back at you: you consider this DIP so
> "obvious" that you are not engaging with our concerns, making flip,
> incorrect remarks about how we would have bikeshedded local imports also.

I apologize it my remarks seem flippant, though I honestly have difficulty finding evidence of that. All I did was note that many of the arguments pitching lazy imports as a better solution than DIP1005 (except of course for the Amdahl one) apply directly to local imports.

> In the end, this is a minor DIP that is easily bikeshedded, as everybody
> can grasp it and have an opinion on it.  I have refrained from
> commenting recently because I will let benchmarking settle it for me.
> Obviously, that won't suffice for others.

I do agree that if framed as a modest improvement in build economics, it is quite unimportant. But that's a problem with the DIP; its main strength is better encapsulation, which is no small thing.


Andrei