On Wednesday, 21 December 2016 at 15:16:34 UTC, Andrei Alexandrescu wrote:

On 12/20/2016 11:32 PM, Joakim wrote:

Second, as I noted above, most top-level imports have not been made
selective yet, because of the symbol leak bug that was recently fixed by
Martin.  You will see in my PRs that I only list those symbols as a
comment, because I could not turn those into selective imports yet.  If
the compiler is doing its job right, selective imports should greatly
reduce the cost of importing a module, even if your metric would still
show the module being imported.

That is not relevant to this section, which discusses the effectiveness of using local imports with the current compilation technology. Per the section's opening sentence:

A legitimate question to ask is whether consistent use of local
imports wherever possible would be an appropriate approximation of
the Dependency-Carrying Declarations goal with no change in the
language at all.

It is relevant because it could further reduce the cost from module-scope imports.  Whether your section only chooses to focus on locally scoped imports and ignore the impact of selective imports is irrelevant to me.

Finally, while it's nice to know the extent of the dependency graph,
what really matters is the _cost_ of each link of the graph, which is
what I keep hammering on.  If the cost of links is small, it doesn't
matter how entangled it is.  If minimizing the dependency graph through
scoping alone, ie without implementing this DIP, removes most of the
cost, that's all I care about.

In first approximation, whether a file gets opened or not makes a difference (filesystem operations (possibly including networking), necessity to rebuild if dependent code is changed. The analysis shows there is significant overhead remaining, on average 10.5 additional files per unused import.

Opening a file or 10 is extremely cheap compared to all the other costs of the compiler.  Purely from a technical cost perspective, I'm not sure even scoped imports were worth it, as my simple investigation below suggests.

If the current document could be clearer in explaining costs, please let me know.

Yes, please explain what significant "overhead" was reduced by scoping imports and would be futher reduced by this DIP.  Opening files doesn't cut it.

My point is that the dependency graph matters, but now that we're
getting down to the last entanglements, we need to know the cost of
those last links.  Your dependency analysis gives us some quantitative
idea of the size of the remaining graph, but tells us nothing about the
cost of those links. That's what I'm looking for.

I will spend some time now investigating those costs with sample code.
My request all along has been that you give us some idea of those costs,
if you know the answer already.

I don't know how to make matters much clearer than the current document. Any suggestions are welcome. The section "Workaround: Are Local Imports Good Enough?" discusses the material cost in terms of extra files that need to be opened and parsed (some unnecessarily) in order to complete a compilation. The "Rationale" part of the document discusses the costs in terms of maintainability, clarity, and documentation.

I don't know how to make it clearer that that's not good enough.  You seem to understand that I want more justification than hand-waving about "scalable" and "overhead," which is why you now give the cost of opening files as justification, but you don't seem to have anything more substantive than that flimsy claim.

I just tried compiling phobos and its unittests for dmd 2.066.1 and 2.067.1, the dmd releases from right before and right after Ilya's PRs linked above (compiling phobos from the older release with the newer dmd didn't work and I wasn't interested in porting it).  I found they took about the same amount of time to compile their respective phobos and the later version consistently took 15 seconds longer to compile the unittests on a single core.

This suggests that there has been essentially no technical benefit to scoped imports, that it is largely superficial.  That's fine, I think it's worth it just from the standpoint of understanding where most dependencies are coming from.  I don't think an additional syntax change is necessary for the remaining few module-scope dependencies for template constraints.

Note on investigation: Of course, some other relevant factors could have changed in dmd and phobos between those two releases, so the result is certainly not conclusive.  But the fact that there was no decrease in compile times while phobos took around the same amount of time is highly suggestive.

I'm uninterested in investigating further given the consistent hand-waving justifications for this DIP.  If somebody else had submitted this DIP, it would have been quickly shot down, and rightly so.