The following post is about entity naming and scoping, i.e. how entities are accessed and made available. This is a reworked follow-up after the original discussion on this previous thread: http://www.digitalmars.com/drn-bin/wwwnews?digitalmars.D/28105 .

BACKGROUND TERMINOLOGY:

In D (and similar programming languages), I define the term "entity" as: a code entity that can be defined and named (this is not a recursive definition btw). That would be: variables, classes (and template instances), structs, typedefs, functions, enums, enum literals, modules, etc.. Labels too but nevermind them. Aliases are a special entity case. Packages currently aren't proper entities, they are instead part of the module base name (more on this below).

A scoping entity is an entity that introduces a new scope, in which new entities can be defined. Most entities are scoping entities.
An entity can be refered by it's base name, if available in the current scope, or (recursively) trough it's parent scoping entity with the "." operator, if the parent entity is available in the current scope (and if security and semantic restrictions allow it). Example:

  module packA.foobar;

  class Foo{
    static int var1 = 42;
    this() {
      // packA.foobar.Foo.var1 entity accessed by base name:
      int var2 = var1;
    }
  }

  void func() {
    // packA.foobar.Foo.var1 entity accessed by parent Foo:
    Foo.var1++;
  }

Thus an entity can be access/used/refered by several names.
The Fully Qualified Name of an entity is it's base name plus all parent entities names in its hierarchy. Example: packA.foobar.Foo.var1 . The FQN is unique.


CURRENT SITUATION:

Currenly, external entities are made accessible by the import <module>; statement. This statement processes the imported module and then does two things:
1) It brings the <module> entity to the current module scope.
2) It brings all entities in the <module> scope to the current module scope.
There is a security attribute in an import (public or private) that specifies the visibility of the imported entities when the current module is imported elsewhere. The default is public (i.e. propagate imports).

Problems/shortcomings:

1) It is conceptualy wrong to have this statement make entities acessible in two ways, since almost allways you will want to use only one way. It is especially worse when it is by FQN that you want to access the module's entities, as the unwanted base names start polluting the current module scope. In this situation, a more concrete problem arises for example when you have an IDE with code autocompletion.
2) You cannot import two modules that have each a (public) child entity with the same name. There is no workaround for this. [I think...]
3) The default behaviour of import is to do a public import, but private imports are by far more common. [are they not? In a test project of mine, and in some random source files of some projects I've looked up on dsource.org, *all* imports I saw were private]

I am pretty damn sure that the behaviour of this whole aspect is ill-conceived and needs to be rethinked and fixed.


PROPOSALS:
Here is a set of three sequential proposals (i.e. should be implemented only after the previous), plus a forth parallel proposal. They are not final, i.e. some points are still open to discussion.


PROPOSAL 1, FQN import:
Introduce a new statement, identical to import, :
  fqnimport <module>;
except that it only brings the <module> entity into scope. It's child entities will have to be accessed trough the <module> entity (by FQN then).
The keyword for this statement could be "fqnimport", or perhaps a shorter but still descriptive name like "mimport", "importm" (for import module)? [Right now I'm favoring "importm" but am not decided;]


PROPOSAL 2, import scopes statement(using):
Introduce a new statement, the using statement, that would import another entity's scope into the current scope. Usage:
  using <scoping entity>;
where scoping entity would be a module. Ideally the statement should be allowed on any block, not just at module-scope level. Also, the behaviour of using is pretty much the same as the with statement, only that this later has a different form:
  with ( <scoping entity or expression> ) /BlockStatement/
Thus it would be better if using was a generalization of the with statement, and to also allow it to have the same form as with, as well as allow the scoping entities (or expressions) allowed in with.

* Variant 1:
Should using also allow packages as a scoping entity? For example:

  importm renderer.foo;
  importm renderer.bar;
  importm renderer.baz;
  using renderer;
  ...
    bar.doStuff();

This does seems to make sense and be useful. However one must first ask what exactly a package is. Conceptualy it is clear that it is a collection of modules and sub-packages. However, in (D) code, how is then a package *defined*? Such package definition could the one extracted from the tree of packages&modules created by the import(and importm) statements so far. The problem is this "so far"... consider this:

  importm renderer.foo;
  importm renderer.bar;
  using renderer;
  importm renderer.baz;

Would baz be accessible? The problem is that packages cant be well-defined, or more precisely, to be defined in just one statement . Should "using renderer;" consider the tree created by only the previous import statements, or by the next imports as well?
I think this would be implementation defined, and considered bad code style; Also, proposal 3 offers a proper [and preferable IMO] solution to this.


PROPOSAL 3, Automatic Availability of all Entities by FQN:
Automatically search for all D source files in a given path (called the import path), specified in the command-line to the compiler, and fqn-import all such D files. The fqnimport statement (with the keyword "importm" or other) would then be ignored (and become deprecated). The normal import statement would do the same as the using statement (and become deprecated).

This is the most radical of the proposed changes, but overall this is a better conceptual and semantic approach for the whole naming environment. In pratice, besides removing the needless need for fqn-imports, it also has some other advantages and possibilities I can think of. One of them is that it well-defines the package entity, since packages would be defined instantly and have the same definition everywhere, and thus better suited for use (by using statements).
Sidenote: This feature can be somewhat emulated once you have the fqn-import statement, on a project basis by an external tool (the tool adds fqn-imports for all modules before compilation). However it has some natural shortcomings, and so it is only a second-best option to being a built-in language feature.

Compatibility:
There would be errors if a top-level package or module would have the same name as an entity defined in the current module scope. A workaround could be made I think, if one was interested in preserving full backwards compatibility.


PROPOSAL 0, private as default import visibility:
Since private imports are far more common than public imports, private should be the default visibility/security of the imported entities in whatever kind of import statement (normal import, fqn-import, using).

Compatibility:
Backwards compatibility would not be garanteed. Any program that used public imports might become broken, altough in a simple-to-fix way. (One could garantee backwards compatibility if only the new import statements (using, fqn-import) were changed to this behaviour)


- - - -

Pheew.., I'm fatigued from all this writing... :p

-- 
Bruno Medeiros
Computer Science/Engineering student

On Wed, 07 Sep 2005 00:31:49 +0000, Bruno Medeiros <daiphoenixNO@SPAMlycos.com> wrote:
<snip>

Nice post. I agree with much of what you say.

> Problems/shortcomings:
>
> 1) It is conceptualy wrong to have this statement make entities acessible in two ways, since almost allways you will want to use only one way. It is especially worse when it is by FQN that you want to access the module's entities, as the unwanted base names start polluting the current module scope. In this situation, a more concrete problem arises for example when you have an IDE with code autocompletion.

I find collisions in symbol names a pain (heck even phobos has these issues) and the alias workaround annoying. I see "import" as being redundant (except perhaps as documentation) in the case where you use FQN's.

Collisions I've encountered in Phobos:
  std.string and std.regexp  'find'
  std.c.stdio and std.stream 'stdin'

> 2) You cannot import two modules that have each a (public) child entity with the same name. There is no workaround for this. [I think...]

There is a workaround, "alias" eg.

[mod1.d]
int foo;

[mod2.d]
double foo;

[main.d]
import mod1;
import mod2;
alias mod2.foo foo;
void main() { foo = 5.5; }

> 3) The default behaviour of import is to do a public import, but private imports are by far more common. [are they not? In a test project of mine, and in some random source files of some projects I've looked up on dsource.org, *all* imports I saw were private]

I agree. Imports should be private by default. The most common argument against this is that everything else is public by default (i.e. class members) so it's consistent. To which I reply, you're comparing apples and oranges and expect them to be consistent (i.e. the same)? madness!

<snip>

> PROPOSAL 3, Automatic Availability of all Entities by FQN:
> Automatically search for all D source files in a given path (called the import path), specified in the command-line to the compiler, and fqn-import all such D files. The fqnimport statement (with the keyword "importm" or other) would then be ignored (and become deprecated). The normal import statement would do the same as the using statement (and become deprecated).
>
> This is the most radical of the proposed changes, but overall this is a better conceptual and semantic approach for the whole naming environment. In pratice, besides removing the needless need for fqn-imports, it also has some other advantages and possibilities I can think of. One of them is that it well-defines the package entity, since packages would be defined instantly and have the same definition everywhere, and thus better suited for use (by using statements).
> Sidenote: This feature can be somewhat emulated once you have the fqn-import statement, on a project basis by an external tool (the tool adds fqn-imports for all modules before compilation). However it has some natural shortcomings, and so it is only a second-best option to being a built-in language feature.
>
> Compatibility:
> There would be errors if a top-level package or module would have the same name as an entity defined in the current module scope. A workaround could be made I think, if one was interested in preserving full backwards compatibility.

I'd vote for "PROPOSAL 3" entities should be available by FQN without any import style statement.

However, I don't understand why "using" or "import" would be deprecated? I think we still need a statement which says "bring this module into the current scope". "import" is that statement, further "alias" would not change in behaviour and would be required to resolve the collisions created by "import".

This would mean that "import" and "alias" enable shorthand, thay are tools to lessen the amount of typing we need to do.

Another potentially useful idea is to have a statement, say "using" which does what import does but only after the line on which it appears, eg.

using a;

foo = 5; <- could be foo or a.foo

using b;

foo = 6; <- could be foo, a.foo or b.foo

> PROPOSAL 0, private as default import visibility:
> Since private imports are far more common than public imports, private should be the default visibility/security of the imported entities in whatever kind of import statement (normal import, fqn-import, using).

Agreed.

Regan

> Currenly, external entities are made accessible by the import <module>;
> statement. This statement processes the imported module and then does two
> things:
> 1) It brings the <module> entity to the current module scope.
> 2) It brings all entities in the <module> scope to the current module
> scope.
> There is a security attribute in an import (public or private) that
> specifies the visibility of the imported entities when the current module
> is imported elsewhere. The default is public (i.e. propagate imports).
>
> Problems/shortcomings:
>
> 1) It is conceptualy wrong to have this statement make entities acessible in two ways, since almost allways you will want to use only one way.

Given 2 above most D code I've seen uses the short name with occasional FQN or aliases. Similarly most Java code I've seen uses the short names. As you've posted earlier your style is different so I'd just avoid using short names and always use the FQN - what you don't use won't hurt you :-)

> It is especially worse when it is by FQN that you want to access the module's entities, as the unwanted base names start polluting the current module scope.

I'm not sure what the problem is. Can you give an example? Any definition in the current module scope hides the imported entities.

> In this situation, a more concrete problem arises for example when you have an IDE with code autocompletion.

I don't understand. Can you give more details?

> 2) You cannot import two modules that have each a (public) child entity with the same name. There is no workaround for this. [I think...]

I don't understand the problem. This works fine:
 module f1;
 int x;

 module f2;
 int x;

 module f3;
 import f1, f2;
 int main() { f1.x = 10; f2.x = 20; return 0; }

> 3) The default behaviour of import is to do a public import, but private imports are by far more common. [are they not? In a test project of mine, and in some random source files of some projects I've looked up on dsource.org, *all* imports I saw were private]

This is a reasonable request that I wouldn't have a problem with. Switching the default visiblity would be pretty simple and easy to modify user code. I'm sure you've read all the previous threads about this since it has come up before. I don't mind the current behavior but then I've gotten used to typing "private" before most of my imports.

Regan Heath wrote:
> On Wed, 07 Sep 2005 00:31:49 +0000, Bruno Medeiros  
>> 2) You cannot import two modules that have each a (public) child entity  with the same name. There is no workaround for this. [I think...]
> 
> 
> There is a workaround, "alias" eg.
> 
> [mod1.d]
> int foo;
> 
> [mod2.d]
> double foo;
> 
> [main.d]
> import mod1;
> import mod2;
> alias mod2.foo foo;
> void main() { foo = 5.5; }
> 
Indeed, seems I missed that one.

>> 3) The default behaviour of import is to do a public import, but private  imports are by far more common. [are they not? In a test project of  mine, and in some random source files of some projects I've looked up on  dsource.org, *all* imports I saw were private]
> 
> 
> I agree. Imports should be private by default. The most common argument  against this is that everything else is public by default (i.e. class  members) so it's consistent. To which I reply, you're comparing apples and  oranges and expect them to be consistent (i.e. the same)? madness!
> 
Agreed, the comparision is senseless. I would even go as far as saying that public imports might not be needed at all, especially if you have auto import (proposal 3). In fact that's the case with modern languages (C# and Java), they don't have such thing as a "public" import.

> 
> I'd vote for "PROPOSAL 3" entities should be available by FQN without any  import style statement.
> 
> However, I don't understand why "using" or "import" would be deprecated? I  think we still need a statement which says "bring this module into the  current scope". "import" is that statement, further "alias" would not  change in behaviour and would be required to resolve the collisions  created by "import".
> 
> This would mean that "import" and "alias" enable shorthand, thay are tools  to lessen the amount of typing we need to do.
> 
> Another potentially useful idea is to have a statement, say "using" which  does what import does but only after the line on which it appears, eg.
> 
> using a;
> 
> foo = 5; <- could be foo or a.foo
> 
> using b;
> 
> foo = 6; <- could be foo, a.foo or b.foo
> 

I agree, and I didn't meant to say using should be deprecated. In this sentence: "The normal import statement would do the same as the using statement (and  become deprecated)." , when I said become deprecated I was only refering to the normal import statement, not using.

-- 
Bruno Medeiros
Computer Science/Engineering student

On Wed, 07 Sep 2005 11:00:48 +0000, Bruno Medeiros <daiphoenixNO@SPAMlycos.com> wrote:
<snip>

>>  I'd vote for "PROPOSAL 3" entities should be available by FQN without any  import style statement.
>>  However, I don't understand why "using" or "import" would be deprecated?

<snip>

> I agree, and I didn't meant to say using should be deprecated. In this sentence: "The normal import statement would do the same as the using statement (and  become deprecated)." , when I said become deprecated I was only refering to the normal import statement, not using.

Ahh.. I see now. In that case I vote for "PROPOSAL 3".

Regan

Ben Hinkle wrote:
>>Currenly, external entities are made accessible by the import <module>; statement. This statement processes the imported module and then does two things:
>>1) It brings the <module> entity to the current module scope.
>>2) It brings all entities in the <module> scope to the current module scope.
>>There is a security attribute in an import (public or private) that specifies the visibility of the imported entities when the current module is imported elsewhere. The default is public (i.e. propagate imports).
>>
>>Problems/shortcomings:
>>
>>1) It is conceptualy wrong to have this statement make entities acessible in two ways, since almost allways you will want to use only one way.
> 
> 
> Given 2 above most D code I've seen uses the short name with occasional FQN or aliases. Similarly most Java code I've seen uses the short names. As you've posted earlier your style is different so I'd just avoid using short names and always use the FQN - what you don't use won't hurt you :-)
> 
> 
There is something else I must clarify. In theoretical terms it's not just by base name or FQ name that you can access a variable, it could be starting in the middle of the hierarchy. And so, what I actually want is not necessarily to access by FQN, but instead to access entities (not all btw) with at least *one* parent entity, one that specifies the "category", the conceptual module/part of the program. This may be the FQN if the FQN is short, but it can be only part of it.
That explained, I must contest that this style I've just described is that different or uncommon:
In D you don't have the facilities to work with this unobtrusively (you would have to use aliases). That's why I talked about allowing packages in the using statement).
In Java (and C#) things are not so linear that you can speak of "uses short names" (I'm assuming you mean base names). Because if you have a set of functions and variables you wish to group together, in Java you must encase them in a class as static members (or as normal members of a singleton class), and so when they are accessed they will allways have at least that class as a prefix.


>>It is especially worse when it is by FQN that you want to access the module's entities, as the unwanted base names start polluting the current module scope.
> 
> 
> I'm not sure what the problem is. Can you give an example? Any definition in the current module scope hides the imported entities.
> 
> 
>>In this situation, a more concrete problem arises for example when you have an IDE with code autocompletion.
> 
> 
> I don't understand. Can you give more details?
> 
> 
An example from the previous thread, slighty modified:

As for the IDE code-completion, to be more clear let me give an example of what would happen with D currently:

  module foobar;
    void DoStuff() { ... }
    void DoStuff2() { ... }
---------------------------------------
  module whatever;
  import foobar;

  void DoThings() { ... }
  void DoMoreThings() { ... }

  int func(){
    ...
    Do| <- press ctrl-space here for code-completion
  }

In this example you want to access entities from foobar by FQN, such as foobar.DoStuff() . However, the base name (DoStuff, DoStuff2, etc.) of the entities of foobar is also made availabe in the current module scope by the import. So when you want to use code-completion, (in this example with the cursor right after Do where the "|" symbol is), you will be presented the the local options (DoThings, DoMoreThings), plus DoStuff and DoStuff2, which you do not want to be presented.


>>2) You cannot import two modules that have each a (public) child entity with the same name. There is no workaround for this. [I think...]
> 
> 
> I don't understand the problem. This works fine:
>  module f1;
>  int x;
> 
>  module f2;
>  int x;
> 
>  module f3;
>  import f1, f2;
>  int main() { f1.x = 10; f2.x = 20; return 0; }
> 
> 
Yes, I was wrong, I was thinking that the clash ocurred only when you tried to use the conflicting var, not just by importing.

>>3) The default behaviour of import is to do a public import, but private imports are by far more common. [are they not? In a test project of mine, and in some random source files of some projects I've looked up on dsource.org, *all* imports I saw were private]
> 
> 
> This is a reasonable request that I wouldn't have a problem with. Switching the default visiblity would be pretty simple and easy to modify user code. I'm sure you've read all the previous threads about this since it has come up before. I don't mind the current behavior but then I've gotten used to typing "private" before most of my imports.
> 
If those threads are from some time ago, I probably haven't seen them since I'm still new around here. And now that you say that, it worries me that there have been some threads about this previously and nothing has changed :( . Where there many people who disagreed with such change? Or just Walter?

-- 
Bruno Medeiros
Computer Science/Engineering student

In article <dflce3$21vp$1@digitaldaemon.com>, Bruno Medeiros says...
>
>1) It is conceptualy wrong to have this statement make entities acessible in two ways, since almost allways you will want to use only one way. It is especially worse when it is by FQN that you want to access the module's entities, as the unwanted base names start polluting the current module scope.

But sometimes you want the base names to pollute the module's scope.  An obvious example are the standard C and POSIX headers.  Many of them are required to expose symbols defined in other modules.  Since private import/alias doesn't work (it's seen as a re-declaration from importing modules), public import is the only option.  I like your suggestions, but there must be a way to import one module's symbols into another, even if it is explicit (through 'using', for example).


Sean

In article <dfmvr0$drb$1@digitaldaemon.com>, Sean Kelly says...
>
>In article <dflce3$21vp$1@digitaldaemon.com>, Bruno Medeiros says...
>>
>>1) It is conceptualy wrong to have this statement make entities acessible in two ways, since almost allways you will want to use only one way. It is especially worse when it is by FQN that you want to access the module's entities, as the unwanted base names start polluting the current module scope.
>
>But sometimes you want the base names to pollute the module's scope.  An obvious example are the standard C and POSIX headers.  Many of them are required to expose symbols defined in other modules.  Since private import/alias doesn't work (it's seen as a re-declaration from importing modules), public import is the only option.  I like your suggestions, but there must be a way to import one module's symbols into another, even if it is explicit (through 'using', for example).
>

This got me thinking.  What if we keep the behavior of import, but allow for a more verbose syntax when we want less generalized behavior.

> import <module> as <namespace name>

It would function along the lines of the current guidelines for aliasing module definitions, but it would affect the entire import.  Plus, we could define this as saying that all the symbols imported this way *must* use the FQN prefixed by the 'namespace name' provided.

> module foobar;
> void DoSomething(){}

> module whatever;
> import foobar as foobar;
> import foobar as gorf;
>
> void main(){
>   foobar.DoSomething();
>   gorf.DoSomething(); // technically the same thing
>   DoSomething(); // illegal, does not exist as 'DoSomething()' is not in scope
> }

So by this we get a short-but-sweet workaround to namespace concflicts.

Personally, I think something like this would dovetail nicely with D's current behavior.  It also has the benefit of not breaking all the code that's already being maintained.  The 'as' token isn't presently a keyword, but AFAIK not many programmers use it as a variable or function name anyway.  At worst, 'as' could be replaced by another token like 'is' or '=', without changing the overall semantics of this proposal.

> import foobar is foobar;
> import foobar = foobar;

Does this solve the problem at hand?

- EricAnderton at yahoo

>>>In this situation, a more concrete problem arises for example when you have an IDE with code autocompletion.
>>
>> I don't understand. Can you give more details?
>>
> An example from the previous thread, slighty modified:
>
> As for the IDE code-completion, to be more clear let me give an example of what would happen with D currently:
>
>   module foobar;
>     void DoStuff() { ... }
>     void DoStuff2() { ... }
> ---------------------------------------
>   module whatever;
>   import foobar;
>
>   void DoThings() { ... }
>   void DoMoreThings() { ... }
>
>   int func(){
>     ...
>     Do| <- press ctrl-space here for code-completion
>   }

The IDE can have a preference to toggle between showing all symbols and symbols in the current module. By default it would show everything (since that's how D works normally) but if one wants to only work with local symbols one can restrict what gets shown.

"Bruno Medeiros" <daiphoenixNO@SPAMlycos.com> wrote in message news:dflce3$21vp$1@digitaldaemon.com...
> I am pretty damn sure that the behaviour of this whole aspect is ill-conceived and needs to be rethinked and fixed.

This sounds like a job for namespacing.

[module1.d]
module module1;

namespace foo // explicit namespace - MUST use FQN
{
    int bar;
}

[main.d]
import module1;
import std.stdio;

void main()
{
    writefln(foo.bar);
    writefln(bar); // Illegal, must use FQN
}


Or, perhaps some anonymous namespaces...

[module1.d]
module module1;

namespace // anonymous namespaces take namespace of module
{
    int foo;
}

int bar;

[main.d]
import module1;
import std.stdio;

void main()
{
    writefln(module1.foo);
    writefln(foo); // illegal, must use FQN
    writefln(module1.bar);
    writefln(bar); // OK, bar is not in explicit module namespace
                        // is imported into current namespace
}

Namespaces could also be more useful by allowing them in classes, when using inner classes or structs is too overkill (or just not an option, i.e. if you want to derive from a class and overload the methods of an inner class or struct, it's impossible, but it would be possible with a namespace's methods).