On 4/5/12 7:43 AM, Steven Schveighoffer wrote:
> I currently think DIP16 is invalid/worksforme (public imports allows
> splitting a module into a package). All that is left is how we could
> specifically import a package with one import statement.

Not entirely (I was aware of the way public import works). An issue does exist - there are "too many names", i.e. the alias pulled in the importing module and also the name being imported. This makes for odd synonyms such as std.algorithm_package.sort.sort being the same as std.algorithm.sort. The other issue is that obviously algorithm_package and algorithm must have distinct names, which makes the scheme a bit awkward at least until we define a compelling convention. I guess we can live with all that.

Andrei

On Thu, 05 Apr 2012 11:23:12 -0400, Andrei Alexandrescu <SeeWebsiteForEmail@erdani.org> wrote:

> On 4/5/12 7:43 AM, Steven Schveighoffer wrote:
>> I currently think DIP16 is invalid/worksforme (public imports allows
>> splitting a module into a package). All that is left is how we could
>> specifically import a package with one import statement.
>
> Not entirely (I was aware of the way public import works). An issue does exist - there are "too many names", i.e. the alias pulled in the importing module and also the name being imported. This makes for odd synonyms such as std.algorithm_package.sort.sort being the same as std.algorithm.sort.

Right, but if one only ever imports std.algorithm, who cares what the submodule FQNs are?

AIUI, DIP16 also doesn't really reduce the number of names either.

> The other issue is that obviously algorithm_package and algorithm must have distinct names, which makes the scheme a bit awkward at least until we define a compelling convention. I guess we can live with all that.

Agreed.  I think Don mentioned std.math already does something, we may want to look at that model.

A couple issues that still need consideration:

1. If std.algorithm the module becomes std.algorithm the package, what happens with ddoc?  We probably *do* need a compiler solution to this.
2. deadalnix pointed out that if we come up with a scheme where the package module and its submodules are in the same directory, the package accessibility qualifier can be used (hey look, a use for the package keyword!).

-Steve

Le 05/04/2012 17:23, Andrei Alexandrescu a écrit :
> On 4/5/12 7:43 AM, Steven Schveighoffer wrote:
>> I currently think DIP16 is invalid/worksforme (public imports allows
>> splitting a module into a package). All that is left is how we could
>> specifically import a package with one import statement.
>
> Not entirely (I was aware of the way public import works). An issue does
> exist - there are "too many names", i.e. the alias pulled in the
> importing module and also the name being imported. This makes for odd
> synonyms such as std.algorithm_package.sort.sort being the same as
> std.algorithm.sort. The other issue is that obviously algorithm_package
> and algorithm must have distinct names, which makes the scheme a bit
> awkward at least until we define a compelling convention. I guess we can
> live with all that.
>
> Andrei
>

The first one isn't a problem. It isn't too many names, it is 2 names, and it happen when explicitly told to do so.

The second is.

On Thursday, April 05, 2012 09:49:59 Steven Schveighoffer wrote:
> On Thu, 05 Apr 2012 09:23:25 -0400, Timon Gehr <timon.gehr@gmx.ch> wrote:
> > On 04/05/2012 02:58 PM, Steven Schveighoffer wrote:
> >> No, public imports simply mean that you can view the publicly imported module. It does *not* add aliases to the importing module.
> > 
> > Have you tried it?
> 
> I just did. OK, what the hell are we arguing about then?!
> 
> DIP16 is worksforme :)
> 
> See this part of the spec:
> 
> http://dlang.org/module.html
> 
> Read the part on public modules. You may understand why I didn't know about that "feature" (which seems to work on all the installed compilers I have, back to 2.033). I just read the public import part of TDPL, and there it is, all spelled out quite nicely. I'm going to file a bug against the spec... grrr...
> 
> I'm now firmly in the "we don't need to change anything, just refactor the modules and use public import" camp. We don't even need to change ANYTHING in the compiler! Forget everything I said before in this thread ;)
> 
> I suppose the only thing we don't get is being able to have a module and a package with the same FQN. I don't see that being a major issue.

What doesn't work is being able to turn a module into a package with the same name. Right now, we could create a std.alg package with sub-modules containing all of std.algorithm's functionality and change std.algorithm to pubicly import them all, but you can't turn std.algorithm itself into a package without breaking code. The package.d portion of the proposal makes it so that you can.

Now, public import's current behavior is _almost_ enough to make the second part completely unnecessary. The only change that would be required would be to make it so that std.algorithm.x looks in std/algorithm/package.d if there's no std.algorithm.x module, because otherwise the public imports in package.d would be putting all of the symbols in std.algorithm.package, not std.algorithm (that, and package.d isn't legal at present).

So, the whole point of this proposal - to seemlessly allow the transition of a module to a package in place - _does_ require a language/compiler change. But moving stuff from a module to a new package does work already.

- Jonathan M Davis

On Thursday, April 05, 2012 11:30:26 Steven Schveighoffer wrote:
> A couple issues that still need consideration:
> 
> 1. If std.algorithm the module becomes std.algorithm the package, what happens with ddoc? We probably *do* need a compiler solution to this.

That's assuming that you insist on keeping all of the documentation in one file. That arguably defeats the purpose of splitting up the modules. If there isn't enough in the module to split the documentation, then why do you need to split the module?

What _would_ be valuable and the package.d could provide is an overview of the package. The ddoc comment for the package.d module can become the documentation for the package as a whole.

> 2. deadalnix pointed out that if we come up with a scheme where the package module and its submodules are in the same directory, the package accessibility qualifier can be used (hey look, a use for the package keyword!).

Yes. std.datetime will need that once it's split. Without that, much of it can't be split and/or code would have to be needlessly duplicated.

- Jonathan M Davis

On Thu, 05 Apr 2012 14:33:22 -0400, Jonathan M Davis <jmdavisProg@gmx.com> wrote:

> On Thursday, April 05, 2012 11:30:26 Steven Schveighoffer wrote:
>> A couple issues that still need consideration:
>>
>> 1. If std.algorithm the module becomes std.algorithm the package, what
>> happens with ddoc? We probably *do* need a compiler solution to this.
>
> That's assuming that you insist on keeping all of the documentation in one
> file. That arguably defeats the purpose of splitting up the modules. If there
> isn't enough in the module to split the documentation, then why do you need to
> split the module?

I thought the whole point was code maintenance?  Not documentation splitting... I would have expected people to continue to treat std.algorithm like it was one module, even though it imports several sub-modules for its implementation.

>
> What _would_ be valuable and the package.d could provide is an overview of the
> package. The ddoc comment for the package.d module can become the
> documentation for the package as a whole.
>
>> 2. deadalnix pointed out that if we come up with a scheme where the
>> package module and its submodules are in the same directory, the package
>> accessibility qualifier can be used (hey look, a use for the package
>> keyword!).
>
> Yes. std.datetime will need that once it's split. Without that, much of it
> can't be split and/or code would have to be needlessly duplicated.

Hm.. I just thought of something, as long as the main "package" module imports everything from the same directory, and doesn't define anything, this isn't an issue.

-Steve

On Thu, 05 Apr 2012 14:24:10 -0400, Jonathan M Davis <jmdavisProg@gmx.com> wrote:

> On Thursday, April 05, 2012 09:49:59 Steven Schveighoffer wrote:
>> On Thu, 05 Apr 2012 09:23:25 -0400, Timon Gehr <timon.gehr@gmx.ch> I suppose the only thing we don't get is being able to have a module and a
>> package with the same FQN. I don't see that being a major issue.
>
> What doesn't work is being able to turn a module into a package with the same
> name. Right now, we could create a std.alg package with sub-modules containing
> all of std.algorithm's functionality and change std.algorithm to pubicly
> import them all, but you can't turn std.algorithm itself into a package
> without breaking code.

But so what?  nobody has any code like:

import std.algorithm.sort;

So who cares where that module goes?  I agree it would be ideal to put it there, but I don't think it's strictly necessary.  And there is no need for the shortcut for fully qualified names.

> So, the whole point of this proposal - to seemlessly allow the transition of a
> module to a package in place - _does_ require a language/compiler change.

I don't see how.  Just move the code into another module and publicly import that module from std/algorithm.d.  Problem pretty much solved.

BTW, importing a directory was already proposed in DIP15. http://prowiki.org/wiki4d/wiki.cgi?LanguageDevel/DIPs/DIP15

-Steve

On Thursday, April 05, 2012 15:30:17 Steven Schveighoffer wrote:
> On Thu, 05 Apr 2012 14:24:10 -0400, Jonathan M Davis <jmdavisProg@gmx.com>
> 
> wrote:
> > On Thursday, April 05, 2012 09:49:59 Steven Schveighoffer wrote:
> >> On Thu, 05 Apr 2012 09:23:25 -0400, Timon Gehr <timon.gehr@gmx.ch> I suppose the only thing we don't get is being able to have a module and a package with the same FQN. I don't see that being a major issue.
> > 
> > What doesn't work is being able to turn a module into a package with the
> > same
> > name. Right now, we could create a std.alg package with sub-modules
> > containing
> > all of std.algorithm's functionality and change std.algorithm to pubicly
> > import them all, but you can't turn std.algorithm itself into a package
> > without breaking code.
> 
> But so what? nobody has any code like:
> 
> import std.algorithm.sort;
> 
> So who cares where that module goes? I agree it would be ideal to put it there, but I don't think it's strictly necessary. And there is no need for the shortcut for fully qualified names.
>
> > So, the whole point of this proposal - to seemlessly allow the
> > transition of a
> > module to a package in place - _does_ require a language/compiler change.
> 
> I don't see how. Just move the code into another module and publicly import that module from std/algorithm.d. Problem pretty much solved.

The issue is code organization. If you want to split up std.algorithm (or std.datetime or whatever) into multiple modules, you have to create a new package with a completely different name with no connection to the original save for the fact that the original publicly imports it. For instance, with the work that I've done thus far on splitting std.datetime, I've had to create a std.dtime package to hold the modules and have std.datetime pubicly import them. This automatically creates the issue of what the difference between them is (for anyone new to Phobos) and does not indicate their relation at all in the hierarchy. It would be much cleaner to be able to turn std/datetime.d into std/datetime/ with a package.d in it along with the new modules.

No, we don't _have_ to do something to make it so that std.algorithm can be turned into std/algorithm/ while still not breaking code. But it would be very nice. Certainly, I don't understand why this DIP would ever have been proposed if Andrei didn't find it valuable.

> BTW, importing a directory was already proposed in DIP15. http://prowiki.org/wiki4d/wiki.cgi?LanguageDevel/DIPs/DIP15

Yes, but having a package.d with the public imports gives you much finer- grained control over what gets imported, and DIP15 doesn't solve the problem of fully qualified uses of std.alorgithm.sort not breaking when std.algorithm.sort gets moved to something like std.algorithm.sorting.d. So, DIP15 doesn't work as a means of seemlessly breaking up a module. It just _mostly_ works as one (since people usually don't fully qualify symbols). Also, package.d would give us a means for documenting a package, which I would very much like to be able to do. Having std.datetime give an overview of the std.dtime package is definitely worse than having a means of having the package document itself - which the package.d file should be able to give us.

The package.d portion of DIP16 allows a means of controlling what importing a package would mean, it provides a means of turning a module into package in place, and it potentially provides a way of documenting a package - all of which are valuable. Whether they're valuable enough to merit a language change is obviously up for debate, but certainly, as the designer and primary maintainer of one of the main targets for being split up, I very much like the idea of being able to split up a module in place rather than having to create a new package with a new name with no obvious relation to the original.

- Jonathan M Davis

On Thursday, April 05, 2012 15:26:14 Steven Schveighoffer wrote:
> On Thu, 05 Apr 2012 14:33:22 -0400, Jonathan M Davis <jmdavisProg@gmx.com>
> 
> wrote:
> > On Thursday, April 05, 2012 11:30:26 Steven Schveighoffer wrote:
> >> A couple issues that still need consideration:
> >> 
> >> 1. If std.algorithm the module becomes std.algorithm the package, what happens with ddoc? We probably *do* need a compiler solution to this.
> > 
> > That's assuming that you insist on keeping all of the documentation in
> > one
> > file. That arguably defeats the purpose of splitting up the modules. If
> > there
> > isn't enough in the module to split the documentation, then why do you
> > need to
> > split the module?
> 
> I thought the whole point was code maintenance? Not documentation splitting... I would have expected people to continue to treat std.algorithm like it was one module, even though it imports several sub-modules for its implementation.

If the module isn't large enough to be split for documentation, I find it hard to believe that it needs to be split for maintenance. And if all you care about is sub-modules for implementation and want all of the functions in the same module still, then this DIP is pointless. All you have to do is declare undocumented sub-modules which hold the various implementations and have the actual module call them. We already do this sort of thing in Phobos to get around static destructors screaming about circular dependencies.

I only see the need to split of a module as the DIP suggests when it's too big, and if the documentation isn't large enough to be an issue, then I don't see how the module itself is going to be too large unless it has a ton of helper stuff, and that stuff can be seemlessy put in another, undocumented module without needing to create a package.

- Jonathan M Davis

On Thu, 05 Apr 2012 17:00:56 -0400, Jonathan M Davis <jmdavisProg@gmx.com> wrote:

> On Thursday, April 05, 2012 15:30:17 Steven Schveighoffer wrote:

>> I don't see how. Just move the code into another module and publicly
>> import that module from std/algorithm.d. Problem pretty much solved.
>
> The issue is code organization. If you want to split up std.algorithm (or
> std.datetime or whatever) into multiple modules, you have to create a new
> package with a completely different name with no connection to the original
> save for the fact that the original publicly imports it.

My view is that people will not import the smaller modules, they will only ever import std.algorithm.

Look at std.bigint, which imports modules from std.internal.math.  Nobody ever imports std.internal.math.??? because they just import std.bigint.

>> BTW, importing a directory was already proposed in DIP15.
>> http://prowiki.org/wiki4d/wiki.cgi?LanguageDevel/DIPs/DIP15
>
> Yes, but having a package.d with the public imports gives you much finer-
> grained control over what gets imported, and DIP15 doesn't solve the problem
> of fully qualified uses of std.alorgithm.sort not breaking when
> std.algorithm.sort gets moved to something like std.algorithm.sorting.d.

I think you are forgetting that all current code imports std.algorithm, which will register the symbol std.algorithm.sort via public import.  No code will break, even if it uses FQN.

The only way to "break" code is to write new import statements.  I don't think anyone will do that unknowingly.

-Steve