Hello,

When I display the dependencies with "dmd -deps=depends", I see that
simply importing std.stdio imports dozens of modules, among which
std.ranges, std.datetime, std.c.windows.winsock, std.regex, etc
In fact, the depends file is 433 lines long.

I noticed that std.string imports quite a lot of stuff. Here is a copypasta of the code:

import core.exception : onRangeError;
import core.vararg, core.stdc.stdlib, core.stdc.string,
    std.algorithm, std.ascii, std.conv, std.exception, std.format,
std.functional,
    std.metastrings, std.range, std.regex, std.traits,
    std.typetuple, std.uni, std.utf;

//Remove when repeat is finally removed. They're only here as part of the
//deprecation of these functions in std.string.
public import std.algorithm : startsWith, endsWith, cmp, count;
public import std.array : join, split;

So as an experiment, I tried to remove a few imports:

std.range, std.regex, std.traits and std.algorithm
as well as the two lines of public import.

To my surprise, it still compiles !
The depends file goes down to 402 lines, but the executable size is
understandably not reduced because all the removed dependencies are
still used elsewhere.

Then I thought this may be due to public imports from other modules, which makes me think that public imports are a bad idea, as we can compile a module only because we imported unknowingly a namespace from imported modules.

My question is: how do we know if std.range, std.regex, std.traits and std.algorithm are spurious imports or if we can (and threfore should) remove them safely from std.string ?

Dude

Le 09/11/2011 10:14, Somedude a écrit :
> 
> My question is: how do we know if std.range, std.regex, std.traits and std.algorithm are spurious imports or if we can (and threfore should) remove them safely from std.string ?
> 
> Dude

I meant:
 how do we know if std.range, std.regex, std.traits and
> std.algorithm are necessary imports or if we can (and threfore should) remove them safely from std.string ?

On 2011-11-09 10:14, Somedude wrote:
> Hello,
>
> When I display the dependencies with "dmd -deps=depends", I see that
> simply importing std.stdio imports dozens of modules, among which
> std.ranges, std.datetime, std.c.windows.winsock, std.regex, etc
> In fact, the depends file is 433 lines long.
>
> I noticed that std.string imports quite a lot of stuff.
> Here is a copypasta of the code:
>
> import core.exception : onRangeError;
> import core.vararg, core.stdc.stdlib, core.stdc.string,
>      std.algorithm, std.ascii, std.conv, std.exception, std.format,
> std.functional,
>      std.metastrings, std.range, std.regex, std.traits,
>      std.typetuple, std.uni, std.utf;
>
> //Remove when repeat is finally removed. They're only here as part of the
> //deprecation of these functions in std.string.
> public import std.algorithm : startsWith, endsWith, cmp, count;
> public import std.array : join, split;
>
> So as an experiment, I tried to remove a few imports:
>
> std.range, std.regex, std.traits and std.algorithm
> as well as the two lines of public import.
>
> To my surprise, it still compiles !
> The depends file goes down to 402 lines, but the executable size is
> understandably not reduced because all the removed dependencies are
> still used elsewhere.
>
> Then I thought this may be due to public imports from other modules,
> which makes me think that public imports are a bad idea, as we can
> compile a module only because we imported unknowingly a namespace from
> imported modules.
>
> My question is: how do we know if std.range, std.regex, std.traits and
> std.algorithm are spurious imports or if we can (and threfore should)
> remove them safely from std.string ?
>
> Dude

Phobos contains a lot of templates and if a template isn't instantiated it won't be compiled. Meaning there can be hidden compile errors if you start to remove imports and they will not show until a template that uses something from the import is instantiate.

-- 
/Jacob Carlborg

Le 09/11/2011 13:15, Jacob Carlborg a écrit :
> Phobos contains a lot of templates and if a template isn't instantiated
> it won't be compiled. Meaning there can be hidden compile errors if you
> start to remove imports and they will not show until a template that
> uses something from the import is instantiate.
>

I see, but then
1. is there a systematic procedure to know if a an import is really needed ?
2. what is your opinion about public import ? In C++, "hidden" or "implicit" #includes is a common source of compilation problems (order of #includes), I tend to think it's a bad thing.

> 2. what is your opinion about public import ? In C++, "hidden" or "implicit" #includes is a common source of compilation problems (order of #includes), I tend to think it's a bad thing.

It can be quite useful. I use it often for C library wrappers. As soon as you import the wrapper code you automatically import the bindings to be able to use constants etc.

> Phobos contains a lot of templates and if a template isn't instantiated it won't be compiled. Meaning there can be hidden compile errors if you start to remove imports and they will not show until a template that uses something from the import is instantiate.

Wouldn't it be possible/better then to move the imports into those template functions?

On 2011-11-09 14:16, Trass3r wrote:
>> Phobos contains a lot of templates and if a template isn't
>> instantiated it won't be compiled. Meaning there can be hidden compile
>> errors if you start to remove imports and they will not show until a
>> template that uses something from the import is instantiate.
>
> Wouldn't it be possible/better then to move the imports into those
> template functions?

I probably would. Having imports in non-global scope is quite a new feature and I guess nobody has either thought of the idea or just haven't had the time yet.

-- 
/Jacob Carlborg

On 2011-11-09 13:45, Somedude wrote:
> Le 09/11/2011 13:15, Jacob Carlborg a écrit :
>> Phobos contains a lot of templates and if a template isn't instantiated
>> it won't be compiled. Meaning there can be hidden compile errors if you
>> start to remove imports and they will not show until a template that
>> uses something from the import is instantiate.
>>
>
> I see, but then
> 1. is there a systematic procedure to know if a an import is really
> needed ?

Not that I know of. You can make sure that every template is instantiated at least once.

> 2. what is your opinion about public import ? In C++, "hidden" or
> "implicit" #includes is a common source of compilation problems (order
> of #includes), I tend to think it's a bad thing.

Sometimes public imports are useful. It's possible to emulate Java's import foo.* using public imports:

// a._.d

public import a.foo;
public import a.bar;

// foobar.d
import a._;

It can also be useful to have public imports if you have a module with array functions and a module with string functions. Then the string module can publicly import the array module since all(most) array functions will work with strings as well.

-- 
/Jacob Carlborg

On Wednesday, November 09, 2011 05:42 Jacob Carlborg wrote:
> On 2011-11-09 14:16, Trass3r wrote:
> >> Phobos contains a lot of templates and if a template isn't instantiated it won't be compiled. Meaning there can be hidden compile errors if you start to remove imports and they will not show until a template that uses something from the import is instantiate.
> > 
> > Wouldn't it be possible/better then to move the imports into those template functions?
> 
> I probably would. Having imports in non-global scope is quite a new feature and I guess nobody has either thought of the idea or just haven't had the time yet.

You also get the question of what to do if a lot of templated functions use the same import. Sure, you could put the import in each individual function so that it doesn't get imported if you don't use any of those functions, but then you've duplicated the same import in a bunch of places. It's not an entirely clear issue.

And I don't think that it ever affects the size of the executable. If you're dealing with static libraries, then all of the functions that aren't used shouldn't be compiled in. And if you're using a shared library, everything in the library is going to need to be there anyway (which would just affect the size of the library, not the executably anyway). So, I don't see how unnecessary imports would be a problem with regards to executable size.

The bigger problem is static constructors and circular dependencies. If the wrong modules import one another when they don't need to, it can result in circular dependencies which will then cause the runtime to exit with an error when you try and run your program. So that - and the fact that it's just cleaner - is a good reason to not have unnecessary imports in a module. However, the circular dependency situation can be made _worse_ by putting imports within a template, since not only is there no way to detect the circular dependency at compile time, but whether you _have_ a circular dependency can depend on whether a particular template was instantiated or not. So, this is definitely not a clear-cut issue.

I'd definitely say that if there are only one or two functions in a module which need a particular import, then it's probably better to restrict that import to those functions, but beyond that, it's better to just stick them at the module level. But since (as you point out) local imports are quite new, I really don't think that much of Phobos is taking advantage of them even if it should be. That'll probably change over time, but it's not exactly a high priority.

- Jonathan M Davis

On 2011-11-09 19:14, Jonathan M Davis wrote:
> On Wednesday, November 09, 2011 05:42 Jacob Carlborg wrote:
>> On 2011-11-09 14:16, Trass3r wrote:
>>>> Phobos contains a lot of templates and if a template isn't
>>>> instantiated it won't be compiled. Meaning there can be hidden compile
>>>> errors if you start to remove imports and they will not show until a
>>>> template that uses something from the import is instantiate.
>>>
>>> Wouldn't it be possible/better then to move the imports into those
>>> template functions?
>>
>> I probably would. Having imports in non-global scope is quite a new
>> feature and I guess nobody has either thought of the idea or just
>> haven't had the time yet.
>
> You also get the question of what to do if a lot of templated functions use
> the same import. Sure, you could put the import in each individual function so
> that it doesn't get imported if you don't use any of those functions, but then
> you've duplicated the same import in a bunch of places. It's not an entirely
> clear issue.

Didn't think of that.

> And I don't think that it ever affects the size of the executable. If you're
> dealing with static libraries, then all of the functions that aren't used
> shouldn't be compiled in. And if you're using a shared library, everything in
> the library is going to need to be there anyway (which would just affect the
> size of the library, not the executably anyway). So, I don't see how
> unnecessary imports would be a problem with regards to executable size.
>
> The bigger problem is static constructors and circular dependencies. If the
> wrong modules import one another when they don't need to, it can result in
> circular dependencies which will then cause the runtime to exit with an error
> when you try and run your program. So that - and the fact that it's just
> cleaner - is a good reason to not have unnecessary imports in a module.
> However, the circular dependency situation can be made _worse_ by putting
> imports within a template, since not only is there no way to detect the
> circular dependency at compile time, but whether you _have_ a circular
> dependency can depend on whether a particular template was instantiated or
> not. So, this is definitely not a clear-cut issue.

That would be bad.

> I'd definitely say that if there are only one or two functions in a module
> which need a particular import, then it's probably better to restrict that
> import to those functions, but beyond that, it's better to just stick them at
> the module level. But since (as you point out) local imports are quite new, I
> really don't think that much of Phobos is taking advantage of them even if it
> should be. That'll probably change over time, but it's not exactly a high
> priority.
>
> - Jonathan M Davis

Yes, exactly.

-- 
/Jacob Carlborg