Ben Hinkle wrote:
> "Bruno Medeiros" <daiphoenixNO@SPAMlycos.com> wrote in message news:delf3c$2p1d$1@digitaldaemon.com...
> 
>>Concerning in D the whole aspect of imports, packages, modules and symbol name scopes, etc.(what to call this whole thing?) there are some things I find ill-conceived, so much that I think this aspect of D is in need of some rethinking.
> 
> [snip]
> 
>>But then I realized, why do we then even need the imports at all?
> 
> 
> One benefit is that it becomes much easier to figure out the dependencies between modules. One only has to look at import statements to decide if module foo depends on module bar. Without import statements one has to not only find all "bar." strings one also has to make sure bar isn't a local variable that's a structure or object where "bar." doesn't refer to the module bar at all.
> 
> 
Yes, there is that difference (if you're not using the /using/ statement), but what you ask can be done easily by a tool. In fact it *should* be done by a tool: because you can have unused imports, or you can be using a module that is not imported directly, but instead by other imported modules, if one is not careful with private/public imports and such.

>>Second: the module statement. The module statement specifies the name of the module, and the hierarchy of parent packages. The module name however must match the file name, and the packages must match the underlying directory structure, otherwise the source file will be unusable (un-importable actually).
> 
> [snip]
> 
> There are two points to consider:
> 1) like Java D source codes doesn't have to be stored in a file system with directories etc. See http://java.sun.com/docs/books/jls/third_edition/html/packages.html#7.2
> 2) by storing the package and module name in the file(s) the entire hierarchy can be recreated from the source code. That is, the contents of the source code defines the program completely, not the source code+directory structure. 
> 
> 
Hum, yes, upon further thinking I too agree: the source code should define the program completely. Thus option 2 should be de-favored, and we are left with option 1(no change) or option 3(dropping the restriction that dir+filename should match package+module). I'm now more inclined towards the third option.

-- 
Bruno Medeiros
Computer Science/Engineering student

Ok, best if I clarify the whole thing at once then. Suppose we have the following source files:

pack1/foobar.d
	Class Whatever
	function DoStuff()
	function DoStuff2()
pack2/xpto.d
	var xpto1
	var xpto2
pack2/asdf.d
	class Alpha
	class Beta


Now suppose that in another D file we want to use doStuff, we then would have to issue a:

  import pack1.foobar;

Currently, D import statement such as this do two things: it makes DoStuff (and other symbols of pack1.foobar) available in the current module with two names: fully-qualified (pack1.foobar.DoStuff) and stripped (just DoStuff). The symbols of public imports of pack1.foobar are also made available in these (both) ways.


Now let's examine the same situation in Java, let's suppose all those D files are Java source files. (technically, then the modules would have to be classes, but it wouldn't make any relevant difference for this discussion). Now suppose that in another Java file, we want to use DoStuff. Things are different, if we want to access DoStuff we can already do it by its FQN:

  pack2.Foobar.DoStuff();

It is only if we want to access it with a shorter name, that we issue a Java import statement;

  import pack2.Foobar;	// or import pack2.*;
  ...
  Foobar.DoStuff();

In Java you can only import the Foobar symbol (a module-like class) into scope, not the contents of Foobar. Thus you cannot access DoStuff() directly, or any external funcion for that matter (the wildcard "*" can only be used after packages).

In C# the code with FQN would be the same:

  pack2.Foobar.DoStuff();

With shorter names it is:

  using pack2;
  ...
  Foobar.DoStuff();

a behaviour quite similiar to C++ namespaces. Altough in C# there are no include statements and namespaces can only contain Classes, Structs or Enums, if I recall correctly.


Now with this in mind, let's look at your questions (you may also want to reread my original post):


Derek Parnell wrote:
>>Bruno Medeiros wrote:
>>
>>>Concerning in D the whole aspect of imports, packages, modules and symbol name scopes, etc.(what to call this whole thing?) there are some things I find ill-conceived, so much that I think this aspect of D is in need of some rethinking.
>>>
>>>First: more often than not, I like to access a symbol using its "fully qualified name" (what to call it?), like this:
>>>package1.module1.func(module2.var1);
>>>
>>>Currently, if I want to do this I have to import the respective modules, which unfortunately also brings the imported modules scope to the local module scope, which I do not want. I want to keep the local scope clean and access these imported modules only by its FQN.
> 
> 
> I haven't had a problem with this. What problems is this behaviour giving
> you?
> 

In terms of just being possible, there is no problem. The problem is the "side-effect" of bringing the stripped name (DoStuff) into local scope as well, not just the FQ name. Altough it may be hard for this to actually cause a bug, or code-inconsistency, it is conceptually an important behaviour. For example, down the road, if we have a D IDE that does code autocompletion (aka intellisense by MS), it becomes especially relevant, and problematic!

>[...]
> 
> Are you saying that if somewhere in my code I have a full qualified name
> for an external item, that the D compiler should assume that I had meant to
> also code the import statement for that module? If so, should it assume a
> private or public import?
> 
> For example:
> 
>      result = std.string.strip(theField);
> 
> would cause DMD to assume that I had also coded ...
> 
>   private import std.string;
> 
> 

Yes that would be it, *but* the import would be different, it would only import FQ names (let's call it fqimport) like I described before. And with such import there would be no difference in being public or private. (do you see why?)

>>>Second: the module statement. The module statement specifies the name of the module, and the hierarchy of parent packages. The module name however must match the file name, and the packages must match the underlying directory structure, otherwise the source file will be unusable (un-importable actually).
>>>This makes the module name declaration pretty much redundant, as well as the parent packages declaration (when given a common root dir for a set of source files), thus making the whole module statement redundant. I find this redundancy unnecessary and undesirable.
> 
> 
> I agree that the module statement is basically useless and an
> inconvenience. The only thing is seems to do is enforce which package a
> module is supposed to be in. 
> 
> 

The module statement is redundant, but that doesn't mean that it is the statement that should go away. See my reply to Ben Hinkle.

>>>I was originaly inclined to this second option, however, the introduction of automatic global import/availability of symbol names possibilitates a third option:
>>>
>>>* Based upon in-file declarations only, the file pathnames do not matter at all. This is the C# way (mostly!). In D we would have the compiler search the first statement of all project files, which would be the module definition, to create the table of packages+modules.
> 
> 
> What's a "project file"? Do just mean a source code file? How would DMD
> find all project files, that is, all files that belong to a project? One
> can't assume that all files in a specific directory belong to the same
> application, and some projects generate more than one application program.
> So I guess I'm not sure what you are asking for here. Can you give
> examples?
> 
By project file I meant source code file belonging to a project, "project" being something that would produce a *single* output executable or library (yes, likely not a clear term).
The method would indeed be to search all D files in a directory (recursively). So in the example in the beggining, supose those files are in a "C:/projects/test" dir. We would have:

  dmd ... -IC:/projects/test -IC:/projects/lib

then dmd would search all files in "C:/projects/test", which would be:
  pack1/foobar.d
  pack2/xpto.d
  pack2/asdf.d

and they could now have different packages+modules names other than pack1.foobar, pack2.xpto or pack2.asdf respectively. The files in "C:/projects/lib" would be searched in the same way. So this is basicly the behaviour of the third option, and so far I don't see any problem with it.
If you have the source code of several applications mingled in the same dir, you should have them named with apropriate and unconflicting modules, or possibly better, you should separate them in different source dirs. (Frankly I think one should do this regardless of whether the compiler searches for files or not, it is basic best-pratices to have a good, clear code structure in terms of files too)


-- 
Bruno Medeiros
Computer Science/Engineering student

Derek Parnell wrote:
> On Wed, 31 Aug 2005 00:56:32 +0000, Bruno Medeiros wrote:
> 
> 
>>Derek Parnell wrote:
>>
>>Hum.. I was under the impression that the post was clear. I will clarify, but before that let know this: are you familiar with (as in having some experience with) Java or C#?
> 
> 
> Just a little under 1% ;-) 
> 
> Could you use plain English instead?
> 
Oh, I thought so, that explains things.  And it is also somewhat disturbing. [I can explain this last phrase but it would take another thread, maybe later]

-- 
Bruno Medeiros
Computer Science/Engineering student

On Wed, 31 Aug 2005 16:14:59 +0000, Bruno Medeiros wrote:

> Ok, best if I clarify the whole thing at once then.

[snipped a good clarification]

>> I haven't had a problem with this. What problems is this behaviour giving you?
>> 
> 
> In terms of just being possible, there is no problem. The problem is the "side-effect" of bringing the stripped name (DoStuff) into local scope as well, not just the FQ name. Altough it may be hard for this to actually cause a bug, or code-inconsistency, it is conceptually an important behaviour. For example, down the road, if we have a D IDE that does code autocompletion (aka intellisense by MS), it becomes especially relevant, and problematic!

So another way of saying it is that public symbols in other modules should only become visible if I explicitly name them. Or ... I can only see them if I say I want to see them.

I would have thought that an IDE would want to show me symbols that I
haven't named yet so I could possibly choose them.

>>[...]
>> 
>> Are you saying that if somewhere in my code I have a full qualified name for an external item, that the D compiler should assume that I had meant to also code the import statement for that module? If so, should it assume a private or public import?
>> 
>> For example:
>> 
>>      result = std.string.strip(theField);
>> 
>> would cause DMD to assume that I had also coded ...
>> 
>>   private import std.string;
>> 
>> 
> 
> Yes that would be it, *but* the import would be different, it would only import FQ names (let's call it fqimport) like I described before. And with such import there would be no difference in being public or private. (do you see why?)

Got it.


[snip]

>> What's a "project file"?

>> 
> By project file I meant source code file belonging to a project, "project" being something that would produce a *single* output executable or library (yes, likely not a clear term).

[snip]

> If you have the source code of several applications mingled in the same dir, you should have them named with apropriate and unconflicting modules, or possibly better, you should separate them in different source dirs. (Frankly I think one should do this regardless of whether the compiler searches for files or not, it is basic best-pratices to have a good, clear code structure in terms of files too)

It's not always that practical. My company has a retail banking system that has over 5,000 applications in it. Many have three files or fewer in them. To have each application in its own folder would cause other problems to arise. We have then organized along functional areas, which keeps them down to about 100 folders; this has worked for us over the last twelve years.

-- 
Derek Parnell
Melbourne, Australia
1/09/2005 7:37:26 AM

On Wed, 31 Aug 2005 21:32:24 +0000, Bruno Medeiros wrote:

> Derek Parnell wrote:
>> On Wed, 31 Aug 2005 00:56:32 +0000, Bruno Medeiros wrote:
>> 
>> 
>>>Derek Parnell wrote:
>>>
>>>Hum.. I was under the impression that the post was clear. I will clarify, but before that let know this: are you familiar with (as in having some experience with) Java or C#?
>> 
>> 
>> Just a little under 1% ;-)
>> 
>> Could you use plain English instead?
>> 
> Oh, I thought so, that explains things.  And it is also somewhat disturbing. [I can explain this last phrase but it would take another thread, maybe later]

I can hardly wait ... :D

BTW, my programming background (in Chronological order) is

IBM COBOL
IBM 360/370 Assembler
PL/I
IBM 1440 Autocoder
MS BASIC v1.0
C
Intel Assembler
Motorola Assembler
Forth
UFO (a 4GL used to make CICS/VS coding easier)
RPG
Progress 4GL
C++
Pascal
E (on the beloved Amiga)
Visual Basic v3/4/5/6
IBM UniVerse
Euphoria
D


-- 
Derek Parnell
Melbourne, Australia
1/09/2005 8:01:56 AM

There was something ahead in your post that prompted me to change some of the terminology I used, and clear some more things. I realized that calling "symbols" to this that I have so far been calling "symbols", may not be very apropriate since it is a somewhat ambiguous term.
So I have come up with a better nomenclature for this subject: it is best to call these things entities, and to define them as: a code entity that can be defined and named (this is not a recursive definition btw).
That would be: variables, classes (and template instances), structs, typedefs, functions, enums, enum literals, modules. (technicaly labels too but nevermind them) Aliases are a special entity case. As for packages I right now don't consider them proper entities, they are instead part of the module base name.
A scoping entity is an entity that introduces a new scope, in which new entities can be defined. Most entities are scoping entities.
An entity can be refered by it's base name, if available in the current scope, or (recursively) trough it's parent entity with the "." operator, if the parent entity is available in the current scope (and if security and semantic restrictions allow it). Example:

  module packA.foobar;

  class Foo{
    static int var1 = 42;
    this() {
      // packA.foobar.Foo.var1 entity accessed by base name;
      int var2 = var1;
    }
  }

  void func() {
    // packA.foobar.Foo.var1 entity accessed by parent Foo;
    Foo.var1++;
  }

Thus an entity can be access/used/refered by several names.
The Fully Qualified Name of an entity is it's base name plus all parent entities names in its hierarchy. Example: packA.foobar.Foo.var1;

I originally called these symbols because I was thinking of the compiler symbol table, however, such symbols are more akin to names, than to entities themselves.

Derek Parnell wrote:
>>In terms of just being possible, there is no problem. The problem is the "side-effect" of bringing the stripped name (DoStuff) into local scope as well, not just the FQ name. Altough it may be hard for this to actually cause a bug, or code-inconsistency, it is conceptually an important behaviour. For example, down the road, if we have a D IDE that does code autocompletion (aka intellisense by MS), it becomes especially relevant, and problematic!
> 
> 
> So another way of saying it is that public symbols in other modules should
> only become visible if I explicitly name them. Or ... I can only see them
> if I say I want to see them.
> 
> I would have thought that an IDE would want to show me symbols that I
> haven't named yet so I could possibly choose them.
>  
> 
Hum..I didn't understand this you just said, so no, I don't now if it is another way of saying it. I'm not sure if it even makes sense. What is "naming a symbol"? Or "seeing a symbol"? (this was what prompted me to change the terminology)
I should note that in this particular paragraph of mine you just replied I didn't say how things "should be", only what was the current situation with D, and the shortcomings of that.

As for the IDE code-completion, to be more clear let me give an example of what would happen with D currently:

  import pack1.foobar;

  void DoThings() { ... }

  int func(){
	pack1.foobar.DoStuff();
	...
	pack1.foobar.DoStuff2();
	...
	Do| <- press ctrl-space here for code-completion
  }

In this example you want to access entities from pack1.foobar by FQN, such as pack1.foobar.DoStuff() . However, the base name (DoStuff, DoStuff2, etc.) of the entities of pack1.foobar is also made availabe in the current module scope by the import. So when you want to use code-completion, with the cursor right after Do where the "|" symbol is, you will be presented the the local options (DoThings), plus DoStuff and DoStuff2, which you do not want to be presented.

>>If you have the source code of several applications mingled in the same dir, you should have them named with apropriate and unconflicting modules, or possibly better, you should separate them in different source dirs. (Frankly I think one should do this regardless of whether the compiler searches for files or not, it is basic best-pratices to have a good, clear code structure in terms of files too)
> 
> 
> It's not always that practical. My company has a retail banking system that
> has over 5,000 applications in it. Many have three files or fewer in them.
> To have each application in its own folder would cause other problems to
> arise. We have then organized along functional areas, which keeps them down
> to about 100 folders; this has worked for us over the last twelve years.
> 
5000 programs ?! Whoa. I am tempted to ask why so many programs, and why would having each in its own source folder would cause other problems to arise. However I sense the answer may be too long/complicated to be of interest right now.
Still, two things at least I would like to know: in what language are those aplications coded? And would the other solution proposed ("naming them with apropriate and unconflicting modules") be fine, if the application was coded in D?

-- 
Bruno Medeiros
Computer Science/Engineering student

On Thu, 01 Sep 2005 15:21:56 +0000, Bruno Medeiros wrote:

> There was something ahead in your post that prompted me to change some
> of the terminology I used, and clear some more things. I realized that
> calling "symbols" to this that I have so far been calling "symbols", may
> not be very apropriate since it is a somewhat ambiguous term.
> So I have come up with a better nomenclature for this subject: it is
> best to call these things entities, ...

Your definition of entity is the same as I was thinking of too.

>> 
>> So another way of saying it is that public symbols in other modules should only become visible if I explicitly name them. Or ... I can only see them if I say I want to see them.
>> 
>> I would have thought that an IDE would want to show me symbols that I
>> haven't named yet so I could possibly choose them.
>> 
>> 
> Hum..I didn't understand this you just said, so no, I don't now if it is another way of saying it. I'm not sure if it even makes sense. What is "naming a symbol"?

I mean by that ... "referencing an entity in another module by using its fully qualified name".

> Or "seeing a symbol"?

I mean by "I can only see them if I say I want to see them." ... "the compiler can find the entity in scope if the coder has explicitly told the compiler the entity's fully qualified name"

>(this was what prompted me to
> change the terminology)
> I should note that in this particular paragraph of mine you just replied
> I didn't say how things "should be", only what was the current situation
> with D, and the shortcomings of that.
> 
> As for the IDE code-completion, to be more clear let me give an example of what would happen with D currently:
> 
>    import pack1.foobar;
> 
>    void DoThings() { ... }
> 
>    int func(){
> 	pack1.foobar.DoStuff();
> 	...
> 	pack1.foobar.DoStuff2();
> 	...
> 	Do| <- press ctrl-space here for code-completion
>    }
> 
> In this example you want to access entities from pack1.foobar by FQN, such as pack1.foobar.DoStuff() . However, the base name (DoStuff, DoStuff2, etc.) of the entities of pack1.foobar is also made availabe in the current module scope by the import. So when you want to use code-completion, with the cursor right after Do where the "|" symbol is, you will be presented the the local options (DoThings), plus DoStuff and DoStuff2, which you do not want to be presented.

How do you know that I don't want them presented?

>>>If you have the source code of several applications mingled in the same dir, you should have them named with apropriate and unconflicting modules, or possibly better, you should separate them in different source dirs. (Frankly I think one should do this regardless of whether the compiler searches for files or not, it is basic best-pratices to have a good, clear code structure in terms of files too)
>> 
>> 
>> It's not always that practical. My company has a retail banking system that has over 5,000 applications in it. Many have three files or fewer in them. To have each application in its own folder would cause other problems to arise. We have then organized along functional areas, which keeps them down to about 100 folders; this has worked for us over the last twelve years.
>> 
> 5000 programs ?! Whoa. I am tempted to ask why so many programs, and why would having each in its own source folder would cause other problems to arise. However I sense the answer may be too long/complicated to be of interest right now.

A retail banking system does a *lot* of things.

> Still, two things at least I would like to know: in what language are those aplications coded?

Progress 4GL

> And would the other solution proposed ("naming them with apropriate and unconflicting modules") be fine, if the application was coded in D?

Not really. It would not make any difference.

-- 
Derek Parnell
Melbourne, Australia
2/09/2005 7:23:47 AM

Derek Parnell wrote:
>>>So another way of saying it is that public symbols in other modules should
>>>only become visible if I explicitly name them. Or ... I can only see them
>>>if I say I want to see them.
>>>
>>>I would have thought that an IDE would want to show me symbols that I
>>>haven't named yet so I could possibly choose them.
>>> 
>>>
>>
>>Hum..I didn't understand this you just said, so no, I don't now if it is another way of saying it. I'm not sure if it even makes sense. What is "naming a symbol"? 
> 
> 
> I mean by that ... "referencing an entity in another module by using its
> fully qualified name".
> 
> 
>>Or "seeing a symbol"? 
> 
> 
> I mean by "I can only see them if I say I want to see them." ... "the
> compiler can find the entity in scope if the coder has explicitly told the
> compiler the entity's fully qualified name"
> 
> 
Ok, now I understand. And now I can say that no, that isn't quite the
same. It is not just a matter of "finding the entity in scope", it is
also *how* it is available, since it can be available directly (base
name in scope) or hierarchly (through one or more parent entities). So
it would be: "the compiler can find the entity in scope (by FQN, not
directly) if the coder has explicitly told the compiler the entity's
fully qualified name"
This is now correctly (altough I would say it in another way) the basis
of the "automatic availability of entities by FQN" feature (was:
"automatic global import/availability of symbol names")

BTW, I'm in the process of rewriting and formalizing my proposal.

>>As for the IDE code-completion, to be more clear let me give an example of what would happen with D currently:
>>
>>   import pack1.foobar;
>>
>>   void DoThings() { ... }
>>
>>   int func(){
>>	pack1.foobar.DoStuff();
>>	...
>>	pack1.foobar.DoStuff2();
>>	...
>>	Do| <- press ctrl-space here for code-completion
>>   }
>>
>>In this example you want to access entities from pack1.foobar by FQN, such as pack1.foobar.DoStuff() . However, the base name (DoStuff, DoStuff2, etc.) of the entities of pack1.foobar is also made availabe in the current module scope by the import. So when you want to use code-completion, with the cursor right after Do where the "|" symbol is, you will be presented the the local options (DoThings), plus DoStuff and DoStuff2, which you do not want to be presented.
> 
> 
> How do you know that I don't want them presented? 
> 
> 
It was a premise of the example: "In this example you want to access
entities from pack1.foobar by FQN". Should have added an "only" there at
the end, but it was somewhat implicit.

>>Still, two things at least I would like to know: in what language are those aplications coded? 
> 
> 
> Progress 4GL
> 
> 
>>And would the other solution proposed ("naming them with apropriate and unconflicting modules") be fine, if the application was coded in D?
> 
> 
> Not really. It would not make any difference.
> 
Well, I still have my doubts that a D modern application/system (that
is, one that produces several programs or libraries) could not be
structured in some (good) way, that the automatic processing of all D
source files in a dir would not be a problem.
But if I would be wrong it doesn't matter. It would be trivial, if the
need be, to have the compiler have an option to process a specified list
of files (instead of all in a dir) as an alternate search method to
implement the "automatic availability of entities by FQN" feature.

-- 
Bruno Medeiros
Computer Science/Engineering student