http://chopapp.com/#fvepfd8 shows the number of dependencies (plus 1) for each module in phobos. Those include druntime dependencies.

With https://github.com/D-Programming-Language/phobos/pull/1768 dependencies would be generated automatically, which should be a good step toward reining in.

The zsh command I ran was  wc -w generated/**/*.deps | sort -nr.


Andrei

19-Dec-2013 01:06, Andrei Alexandrescu пишет:
> http://chopapp.com/#fvepfd8 shows the number of dependencies (plus 1)
> for each module in phobos. Those include druntime dependencies.
>

So the bill of using pretty much anything in Phobos is pulling in 87 modules. Pretty much what I feared it is.

-- 
Dmitry Olshansky

On 12/18/13 1:16 PM, Dmitry Olshansky wrote:
> 19-Dec-2013 01:06, Andrei Alexandrescu пишет:
>> http://chopapp.com/#fvepfd8 shows the number of dependencies (plus 1)
>> for each module in phobos. Those include druntime dependencies.
>>
>
> So the bill of using pretty much anything in Phobos is pulling in 87
> modules. Pretty much what I feared it is.

There are several directions we can take this.

1. Improve the compiler to handle imports lazily, i.e. an unused import is never opened. That's unlikely to help a lot of uses because most unqualified name lookups require all imports to be loaded (even after the name if resolved, the compiler must still look for ambiguities).

2. Push imports from top level into the entities (functions, classes etc) that use them.

3. Apply classic dependency management (break larger modules in smaller ones, accept some code duplication etc).

I favor (2).


Andrei

19-Dec-2013 01:40, Andrei Alexandrescu пишет:
> On 12/18/13 1:16 PM, Dmitry Olshansky wrote:
>> 19-Dec-2013 01:06, Andrei Alexandrescu пишет:
>>> http://chopapp.com/#fvepfd8 shows the number of dependencies (plus 1)
>>> for each module in phobos. Those include druntime dependencies.
>>>
>>
>> So the bill of using pretty much anything in Phobos is pulling in 87
>> modules. Pretty much what I feared it is.
>
> There are several directions we can take this.
>
> 1. Improve the compiler to handle imports lazily, i.e. an unused import
> is never opened. That's unlikely to help a lot of uses because most
> unqualified name lookups require all imports to be loaded (even after
> the name if resolved, the compiler must still look for ambiguities).
>
> 2. Push imports from top level into the entities (functions, classes
> etc) that use them.
>
> 3. Apply classic dependency management (break larger modules in smaller
> ones, accept some code duplication etc).
>
> I favor (2).
>

I'd add something that might help make (2) possible in case of templates.

4. Split modules at least into a package of with at least 2 parts:
- Meta (traits) - constraints, type definitions etc.
- API - functions, globals and other "meat" of the module.

One prime example is std.range - isInputRange, isForwardRange etc. need not to be "bought together" with all of range adapters and std.algorithm dependency.

-- 
Dmitry Olshansky

On Thu, Dec 19, 2013 at 01:47:26AM +0400, Dmitry Olshansky wrote:
> 19-Dec-2013 01:40, Andrei Alexandrescu пишет:
> >On 12/18/13 1:16 PM, Dmitry Olshansky wrote:
> >>19-Dec-2013 01:06, Andrei Alexandrescu пишет:
> >>>http://chopapp.com/#fvepfd8 shows the number of dependencies (plus 1) for each module in phobos. Those include druntime dependencies.
> >>>
> >>
> >>So the bill of using pretty much anything in Phobos is pulling in 87 modules. Pretty much what I feared it is.
> >
> >There are several directions we can take this.
> >
> >1. Improve the compiler to handle imports lazily, i.e. an unused import is never opened. That's unlikely to help a lot of uses because most unqualified name lookups require all imports to be loaded (even after the name if resolved, the compiler must still look for ambiguities).
> >
> >2. Push imports from top level into the entities (functions, classes
> >etc) that use them.
> >
> >3. Apply classic dependency management (break larger modules in smaller ones, accept some code duplication etc).
> >
> >I favor (2).

I vote for (2) as well.

And for the first part of (3): I think std.algorithm is due for a breakup. There is no reason why using find, for example, should pull in sorting algorithms, and why using sorting algorithms should pull in cartesianProduct.


> I'd add something that might help make (2) possible in case of
> templates.
> 
> 4. Split modules at least into a package of with at least 2 parts:
> - Meta (traits) - constraints, type definitions etc.
> - API - functions, globals and other "meat" of the module.
> 
> One prime example is std.range - isInputRange, isForwardRange etc. need not to be "bought together" with all of range adapters and std.algorithm dependency.
[...]

I've been saying this over and over: now we have package.d, let's make use of it. Jonathan has been working on splitting up std.datetime (though that has yet to materialize); I think std.algorithm and std.range are due, too.


T

-- 
"Hi." "'Lo."

On 12/18/2013 1:40 PM, Andrei Alexandrescu wrote:
> 1. Improve the compiler to handle imports lazily, i.e. an unused import is never
> opened. That's unlikely to help a lot of uses because most unqualified name
> lookups require all imports to be loaded (even after the name if resolved, the
> compiler must still look for ambiguities).
>
> 2. Push imports from top level into the entities (functions, classes etc) that
> use them.
>
> 3. Apply classic dependency management (break larger modules in smaller ones,
> accept some code duplication etc).
>
> I favor (2).

(1) is impractical for the reasons you mentioned.

I favor (2), and also (4):

4. Break kitchen sink modules like std.algorithm into one module per algorithm. This should not result in code duplication.

On 12/18/13 3:08 PM, Walter Bright wrote:
> On 12/18/2013 1:40 PM, Andrei Alexandrescu wrote:
>> 1. Improve the compiler to handle imports lazily, i.e. an unused
>> import is never
>> opened. That's unlikely to help a lot of uses because most unqualified
>> name
>> lookups require all imports to be loaded (even after the name if
>> resolved, the
>> compiler must still look for ambiguities).
>>
>> 2. Push imports from top level into the entities (functions, classes
>> etc) that
>> use them.
>>
>> 3. Apply classic dependency management (break larger modules in
>> smaller ones,
>> accept some code duplication etc).
>>
>> I favor (2).
>
> (1) is impractical for the reasons you mentioned.
>
> I favor (2), and also (4):
>
> 4. Break kitchen sink modules like std.algorithm into one module per
> algorithm. This should not result in code duplication.

That's (3), which enumerates a number of possible effects that may or may not overlap/

Andrei

On Wednesday, 18 December 2013 at 23:08:21 UTC, Walter Bright wrote:
> I favor (2), and also (4):
>
> 4. Break kitchen sink modules like std.algorithm into one module per algorithm. This should not result in code duplication.

That seems a little over the top. While I guess it's not necessarily a bad thing to increase modularity, users would have to pull in a large number of imports to do anything nontrivial. For example, when you want to use sort, find, splitter and swap all together, you either have the choice of doing:

import std.algorithm;

And pull in everything, or doing:

import std.algorithm.sort, std.algorithm.find, std.algorithm.splitter, std.algorithm.swap;

Or is there something I'm missing here?

Of course, another large boon would be to correct the implementation of:

import std.algorithm: sort, find, splitter, swap;

So that it actually works in a sane way as opposed to pulling in everything.

On Wednesday, 18 December 2013 at 21:40:08 UTC, Andrei Alexandrescu wrote:
> 1. Improve the compiler to handle imports lazily, i.e. an unused import is never opened. That's unlikely to help a lot of uses because most unqualified name lookups require all imports to be loaded (even after the name if resolved, the compiler must still look for ambiguities).
>
> 2. Push imports from top level into the entities (functions, classes etc) that use them.
>
> 3. Apply classic dependency management (break larger modules in smaller ones, accept some code duplication etc).
>
> I favor (2).
>
>
> Andrei

2 + 3 (breaking modules into packages part) should do the tricky as far as I can see. Lot of current dependency bloat comes from importing whole std.range / std.algorithm to use one or two utility functions in few places.

On topic of (2) I propose to add it to Phobos style guide and start enforcing during PR review.

On Thu, 19 Dec 2013 00:51:04 +0100, Meta wrote:

> On Wednesday, 18 December 2013 at 23:08:21 UTC, Walter Bright wrote:
>> I favor (2), and also (4):
>>
>> 4. Break kitchen sink modules like std.algorithm into one module per algorithm. This should not result in code duplication.
> 
> That seems a little over the top. While I guess it's not necessarily a bad thing to increase modularity, users would have to pull in a large number of imports to do anything nontrivial. For example, when you want to use sort, find, splitter and swap all together, you either have the choice of doing:
> 
> import std.algorithm;
> 
> And pull in everything, or doing:
> 
> import std.algorithm.sort, std.algorithm.find, std.algorithm.splitter, std.algorithm.swap;
> 
> Or is there something I'm missing here?
> 
> Of course, another large boon would be to correct the implementation of:
> 
> import std.algorithm: sort, find, splitter, swap;
> 
> So that it actually works in a sane way as opposed to pulling in everything.

Presumably there would still be a std.algorithm package module;  the
questions is: does
  import std.algorithm : joiner;
still work if std.algorithm is actually a package module and joiner is in
a std.algorithm.joiner module?