April 05, 2012
On 2012-04-05 00:50:49 +0000, Michel Fortin <michel.fortin@michelf.com> said:

> I think we need a third option.
> 
> Here's an idea: we could allow modules having a single symbol with the same name as the module to behave as if they were the symbol itself, just like templates behaves. For instance:
> 
> 	module std.algorithm.sort;
> 
> 	void sort(int[] t);
> 
> Now you can import std.algorithm.sort and then use the std.algorithm.sort fully qualified name as if it was a function, even though it's the module name (std.algorithm.sort.sort would be the function's name).
> 
> Or maybe we could just allow "alias sort this" at module level? Or is it allowed already?

Forgot to follow through with what "import std.algorithm" would do.

If std.algorithm is a package, then it should only contain modules (which can each pose for a function/class/struct/variable thanks to the feature described above). Importing the package std.algorithm would open the std/algorithm.d file, read a list of package from the file, and proceed to import each of them (just as if you'd have imported each of them separately).

Here is an idea for declaring the list of modules to import when you import a package in std/algorithm.d:

	package std.algorithm;

	module std.algorithm.sort;
	module std.algorithm.map;
	...

(I'm _not_ using a package.d file because that'd incur two lookups: one for std/algorithm.d in case it's a module, and one for std/algorithm/package.d in case it's a package. It also removes the possibility of both files existing at the same time which would be confusing.)

This way, modules and packages stay exactly the same in the language as they are now, except that you can now import a package directly instead of importing each module separately. And modules with a single symbol with the same name as the module are treated as if they were that symbol. This allows migrating a module to a package without breaking the existing code base, but the drawback is that each symbol in the module becoming a package need to become its own submodule.

-- 
Michel Fortin
michel.fortin@michelf.com
http://michelf.com/

April 05, 2012
Le 05/04/2012 02:55, Martin Nowak a écrit :
>> What happen if both pkg.d and pkg/_.d exists ? If it is not in the
>> same path (think -I compiler option). In one case, this is an issue,
>> in the other this isn't.
> pkg.d would always be a module, hence result in a module/package conflict.
> We'd need to directly search for a subdirectory to decide whether it's a
> package.
>

This is true, but why would it be a conflict ?

Package already is a tree structure, so it shouldn't cause much trouble.

> Maybe someone with experience of big Python projects has some valuable
> insights, but I think one point of adding the file inside a folder is to
> prevent from unintended pickup of folders.

Using eponymous trick, you'd always pick up a file, never a folder. Hence, lookup rules get easier. Plus it is D-ish.

I have no doubt that this approach is working in python, and could work in D. This is exactly why this was my first proposal. But this doesn't means that we can't do better. I have tested quite a lot of way to achieve that, in several languages, and this is usually a messy topic, with no single solution.
April 05, 2012
On Wed, 04 Apr 2012 20:50:49 -0400, Michel Fortin <michel.fortin@michelf.com> wrote:

> On 2012-04-04 19:48:32 +0000, "Steven Schveighoffer" <schveiguy@yahoo.com> said:
>
>> On Wed, 04 Apr 2012 14:03:07 -0400, Timon Gehr <timon.gehr@gmx.ch> wrote:
>>
>>> No symbol is resolved until semantic, but I don't think hiding the  module/package symbol if any clashing symbol in the module/any  subpackage exists is a satisfactory solution.
>>  Then we must come up with a way to hide the submodules of a virtual module.
>>  I tried this, which fails on the current compiler:
>>  import x;
>>  int x;
>>  So it seems we have two choices here:
>>  1. DIP16 needs to get more complex to make package submodules not  accesible as individual modules.
>> 2. Start contextually interpreting identifiers at least in the case of  modules vs. non-modules.
>>  I'd suggest option 2 allows for better backwards compatibility and more  flexibility.
>
> I don't think option 2 is realistic (see my other post).

I saw your other post.  I see that a major issue with the way DIP16 intends to shortcut fully qualified names is demonstrated by this simple example:

a/b/c.d:

module a.b.c;

void foo() {}
struct c
{
   static void foo() {}
}

main.d:
import a.b.c;

void main()
{
   a.b.c.foo(); // cannot determine what this is.  Is it the global function, or a shortcut FQN to a.b.c.c.foo?
}

This should be counter-case enough to disqualify that idea -- it will break existing code without ever adding any package.d files.  Thanks for explaining.

>
> I don't think option 1 is an improvement over what we have. I mean, if you're going to hide the submodules, what is the benefit compared to just using a different package name for the implementation modules?

The advantage is you can split your logical module into submodules for maintenance purposes.

> You can already refactor std.algorithm this way with no change in the compiler:
>
> 	module std.algorithm;
>
> 	public import std.algorithm_impl.sort;
> 	public import std.algorithm_impl.map;
> 	public import std.algorithm_impl.blah_blah_blah;
> >
> If we add a language feature, it should be an noticeable improvement over this situation.

I agree, option 1 is not very convincing.

> I think we need a third option.
>
> Here's an idea: we could allow modules having a single symbol with the same name as the module to behave as if they were the symbol itself, just like templates behaves. For instance:
>
> 	module std.algorithm.sort;
>
> 	void sort(int[] t);
>
> Now you can import std.algorithm.sort and then use the std.algorithm.sort fully qualified name as if it was a function, even though it's the module name (std.algorithm.sort.sort would be the function's name).
>
> Or maybe we could just allow "alias sort this" at module level? Or is it allowed already?

I don't like this proposal, simply because this means one function/symbol per submodule.  It should be more flexible than that.  But I like the line of thinking.

Let's re-examine the issue.  We need to allow splitting of a module X.d into pieces for maintenance (or possibly accessibility -- disallowing friends).  But we don't want to break code which currently uses FQN to access X's symbols.

I think the following might work:

algorithm.d:

import this = std.algorithm_impl.sort;

Which then imports std.algorithm_impl.sort, and effectively aliases all its symbols into algorithm.d.  If std.algorithm_impl.sort defines a function called sort, then it's also aliased to std.algorithm.sort.  In essence, sort has *two* FQN, but there are no FQN that refer to more than one symbol.

I purposely left out the package.d idea because it's orthogonal to this.

-Steve
April 05, 2012
Le 05/04/2012 13:46, Steven Schveighoffer a écrit :
> I don't like this proposal, simply because this means one
> function/symbol per submodule. It should be more flexible than that. But
> I like the line of thinking.
>
> Let's re-examine the issue. We need to allow splitting of a module X.d
> into pieces for maintenance (or possibly accessibility -- disallowing
> friends). But we don't want to break code which currently uses FQN to
> access X's symbols.
>
> I think the following might work:
>
> algorithm.d:
>
> import this = std.algorithm_impl.sort;
>
> Which then imports std.algorithm_impl.sort, and effectively aliases all
> its symbols into algorithm.d. If std.algorithm_impl.sort defines a
> function called sort, then it's also aliased to std.algorithm.sort. In
> essence, sort has *two* FQN, but there are no FQN that refer to more
> than one symbol.
>
> I purposely left out the package.d idea because it's orthogonal to this.
>
> -Steve

The behavior you described has been proposed for public import, and have been discussed. This is interesting.

You propose an alternative syntax with is fine and have the advantage to not disturb already existing public imports, but have the drawback to create a new syntax, again.

If such a syntax is adopted, what would be the point of public imports ? If it is still useful, then your syntax is a better choice, otherwise, we'd better modify public import.
April 05, 2012
On Thu, 05 Apr 2012 08:49:24 -0400, deadalnix <deadalnix@gmail.com> wrote:

> Le 05/04/2012 13:46, Steven Schveighoffer a écrit :
>> I don't like this proposal, simply because this means one
>> function/symbol per submodule. It should be more flexible than that. But
>> I like the line of thinking.
>>
>> Let's re-examine the issue. We need to allow splitting of a module X.d
>> into pieces for maintenance (or possibly accessibility -- disallowing
>> friends). But we don't want to break code which currently uses FQN to
>> access X's symbols.
>>
>> I think the following might work:
>>
>> algorithm.d:
>>
>> import this = std.algorithm_impl.sort;
>>
>> Which then imports std.algorithm_impl.sort, and effectively aliases all
>> its symbols into algorithm.d. If std.algorithm_impl.sort defines a
>> function called sort, then it's also aliased to std.algorithm.sort. In
>> essence, sort has *two* FQN, but there are no FQN that refer to more
>> than one symbol.
>>
>> I purposely left out the package.d idea because it's orthogonal to this.
>>
>> -Steve
>
> The behavior you described has been proposed for public import, and have been discussed. This is interesting.
>
> You propose an alternative syntax with is fine and have the advantage to not disturb already existing public imports, but have the drawback to create a new syntax, again.
>
> If such a syntax is adopted, what would be the point of public imports ? If it is still useful, then your syntax is a better choice, otherwise, we'd better modify public import.

No, public imports simply mean that you can view the publicly imported module.  It does *not* add aliases to the importing module.

for example:

foo.d:

module foo;

int x;

bar.d:
module bar;
public import foo;

int y;

main.d:
import bar; // publicly imports foo.

void main()
{
   foo.x = 5; // ok, publicly imported via bar (non public, this would be an error)
   bar.y = 5; // ok
   bar.x = 5; // error, public import doesn't create new fully qualified name for foo.x
}

With the system I propose, using the specific technique would make it so bar.x would also be valid, and would refer to foo.x

I think we need new syntax, or a new language feature, because you don't want to alter the operation of existing code.  Simply changing public imports would cause lots of existing code to be reinterpreted, possibly in a way not intended, or that would be ambiguous.

-Steve
April 05, 2012
On Thu, 05 Apr 2012 08:49:24 -0400, deadalnix <deadalnix@gmail.com> wrote:

> If such a syntax is adopted, what would be the point of public imports ? If it is still useful, then your syntax is a better choice, otherwise, we'd better modify public import.

Another way to look at it:

A public import saves you from having to import dependent modules that you should be importing anyway.
A 'this' import treats the other modules as if they were actually part of the imported module in terms of namespace.

To give an example, std.range imports std.array.  std.array is a module all on its own, and has a specific set of functionality.  std.range has a different set of functionality, but you would never want to have std.range imported without also importing std.array.  Another example would be a derived class module publicly importing the base class module.

On the other hand, something like std.container could have 15 different container types in it.  Each of these container types should really live in their own module, in terms of maintenance and separation of private data.  For example, std.container.RedBlackTree has no business accessing the private members of std.container.Array.  But because they live in the same module, it can.  However, we don't want to define std.container.RedBlackTree.RedBlackTree in terms of namespace.  So we have one of these combining modules, and everything still lives in the std.container namespace, but we get all the benefits of separating the code into individual modules.

-Steve
April 05, 2012
On 04/05/2012 02:58 PM, Steven Schveighoffer wrote:
> No, public imports simply mean that you can view the publicly imported
> module. It does *not* add aliases to the importing module.
>

Have you tried it?
April 05, 2012
On Thu, 05 Apr 2012 09:23:25 -0400, Timon Gehr <timon.gehr@gmx.ch> wrote:

> On 04/05/2012 02:58 PM, Steven Schveighoffer wrote:
>> No, public imports simply mean that you can view the publicly imported
>> module. It does *not* add aliases to the importing module.
>>
>
> Have you tried it?

I just did.  OK, what the hell are we arguing about then?!

DIP16 is worksforme :)

See this part of the spec:

http://dlang.org/module.html

Read the part on public modules.  You may understand why I didn't know about that "feature" (which seems to work on all the installed compilers I have, back to 2.033).  I just read the public import part of TDPL, and there it is, all spelled out quite nicely.  I'm going to file a bug against the spec...  grrr...

I'm now firmly in the "we don't need to change anything, just refactor the modules and use public import" camp.  We don't even need to change ANYTHING in the compiler!  Forget everything I said before in this thread ;)

I suppose the only thing we don't get is being able to have a module and a package with the same FQN.  I don't see that being a major issue.

-Steve
April 05, 2012
Le 05/04/2012 14:58, Steven Schveighoffer a écrit :
> On Thu, 05 Apr 2012 08:49:24 -0400, deadalnix <deadalnix@gmail.com> wrote:
>
>> Le 05/04/2012 13:46, Steven Schveighoffer a écrit :
>>> I don't like this proposal, simply because this means one
>>> function/symbol per submodule. It should be more flexible than that. But
>>> I like the line of thinking.
>>>
>>> Let's re-examine the issue. We need to allow splitting of a module X.d
>>> into pieces for maintenance (or possibly accessibility -- disallowing
>>> friends). But we don't want to break code which currently uses FQN to
>>> access X's symbols.
>>>
>>> I think the following might work:
>>>
>>> algorithm.d:
>>>
>>> import this = std.algorithm_impl.sort;
>>>
>>> Which then imports std.algorithm_impl.sort, and effectively aliases all
>>> its symbols into algorithm.d. If std.algorithm_impl.sort defines a
>>> function called sort, then it's also aliased to std.algorithm.sort. In
>>> essence, sort has *two* FQN, but there are no FQN that refer to more
>>> than one symbol.
>>>
>>> I purposely left out the package.d idea because it's orthogonal to this.
>>>
>>> -Steve
>>
>> The behavior you described has been proposed for public import, and
>> have been discussed. This is interesting.
>>
>> You propose an alternative syntax with is fine and have the advantage
>> to not disturb already existing public imports, but have the drawback
>> to create a new syntax, again.
>>
>> If such a syntax is adopted, what would be the point of public imports
>> ? If it is still useful, then your syntax is a better choice,
>> otherwise, we'd better modify public import.
>
> No, public imports simply mean that you can view the publicly imported
> module. It does *not* add aliases to the importing module.
>
> for example:
>
> foo.d:
>
> module foo;
>
> int x;
>
> bar.d:
> module bar;
> public import foo;
>
> int y;
>
> main.d:
> import bar; // publicly imports foo.
>
> void main()
> {
> foo.x = 5; // ok, publicly imported via bar (non public, this would be
> an error)
> bar.y = 5; // ok
> bar.x = 5; // error, public import doesn't create new fully qualified
> name for foo.x
> }
>
> With the system I propose, using the specific technique would make it so
> bar.x would also be valid, and would refer to foo.x
>
> I think we need new syntax, or a new language feature, because you don't
> want to alter the operation of existing code. Simply changing public
> imports would cause lots of existing code to be reinterpreted, possibly
> in a way not intended, or that would be ambiguous.
>
> -Steve

I know. I think you answered too fast.

I wasn't stating that public import are the same as what you propose. I was discussing the fact that both probably answer the same need and that 2 different syntax is something to avoid.

Code wouldn't be broken in unexpected way, because thing will cause either a compile error (collision of names, but the problem exists with other solutions too) or is illegal in the current shape of the language.
April 05, 2012
On 2012-04-05 11:46:38 +0000, "Steven Schveighoffer" <schveiguy@yahoo.com> said:

> I don't like this proposal, simply because this means one function/symbo l
> per submodule.  It should be more flexible than that.  But I like the li ne
> of thinking.

I have the same reserve about it's lack of flexibility too. It would be a nice thing to have for all those libraries having one module per class as it'd reduce the length of the fully qualified name you have to use when you need to disambiguate. But for moving a module like std.algorithm to a package, it's cumbersome.


> Let's re-examine the issue.  We need to allow splitting of a module X.d
> into pieces for maintenance (or possibly accessibility -- disallowing
> friends).  But we don't want to break code which currently uses FQN to
> access X's symbols.
> 
> I think the following might work:
> 
> algorithm.d:
> 
> import this = std.algorithm_impl.sort;
> 
> Which then imports std.algorithm_impl.sort, and effectively aliases all
> its symbols into algorithm.d.  If std.algorithm_impl.sort defines a
> function called sort, then it's also aliased to std.algorithm.sort.  In
> essence, sort has *two* FQN, but there are no FQN that refer to more tha n
> one symbol.

This is what a public import should do.


-- 
Michel Fortin
michel.fortin@michelf.com
http://michelf.com/