On Wednesday, 2 March 2016 at 08:50:50 UTC, Mike Parker wrote:
> I'm curious what sort of system you have in mind. How does the current system not work for you and what would you prefer to see?

First, you must know that I've been a C++ programmer for way too much time ;
and that my way of thinking about modules has probably been - badly - shaped accordingly :-).

However, it feels wrong that modules (foo, bar) inside the same package ("pkg") need to repeat the "absolute" package name when referencing each other, I mean "import pkg.foo" instead of "import foo". Not because of the extra typing, but because of the dependency it introduces. But as I said, maybe my vision is wrong.

I'm going to answer your first question, I hope you don't get bored before the end ;-)

First some generic stuff:
- The libraries I write are source-only, and I never install them system-wide, as many projects might depend on different revisions of one library.
- I only work with separate compilation, using gdc (Windows-dmd produces OMF, which is a dealbreaker ; but I use rdmd a lot for small programs).
- I often mix C code with D, sometimes, C++ code. I compile them using gcc/g++.
- My build system is composed of simple dependency-aware non-recursive makefiles (BTW, this is a great way to keep your code decoupled, otherwise you might not get feedback that you're making a dependency mess soon enough!)

This is the best way I found to be able to checkout one project, type 'make',
and have everything build properly, wether I'm doing it on GNU/Linux or on Windows (MSYS).

Everything is under version control using GNU Bazaar (which basically is to git what D is to C++).

I have a small library, which I call "lib_algo", containing semi-related utilities (containers, topological sort, etc.) which I use in many projects.

$ cd myLibs
$ find lib_algo -name "*.d" -or -name "*.mk" -or name "Makefile"
lib_algo/options.d
lib_algo/misc.d
lib_algo/pointer.d
lib_algo/set.d
lib_algo/queue.d
lib_algo/algebraic.d
lib_algo/vector.d
lib_algo/bresenham.d
lib_algo/topological.d
lib_algo/fix_array.d
lib_algo/test.d
lib_algo/pool.d
lib_algo/stack.d
lib_algo/list.d
lib_algo/array2d.d
lib_algo/project.mk
lib_algo/Makefile
(I know, there's some overlap with Phobos today)

The project.mk simply contains the list of source files of the library.
The test.d file actually contains a main function, which I use to run manual tests of the library functionalities. It's not included in the project.mk list. And it obviously must import the files needing to be tested.

With this flat directory structure, the only way test.d can import files is by saying, "import stack;", not  "import lib_algo.stack".
(adding ".." to the list of import directories might do the trick, but this would definitely feel wrong).

Now, let's see a project using this "lib_algo", I call it "myGame".

$ cd myProjects
$ find <files relevant to this discussion>
myGame/lib_algo/options.d
myGame/lib_algo/....
myGame/lib_algo/array2d.d
myGame/lib_algo/project.mk
myGame/lib_algo/Makefile
myGame/src/main.d
myGame/Makefile


"lib_algo" simply is an external (aka "git submodule"), pointing to a specific revision of the repository "lib_algo".
The top-level "Makefile" of the projects includes "lib_algo/project.mk", and all is well, I can compose the list of source files to compile without having to rely on "convenience/intermediate" libraries.

Now, if "src/main.d" wants to import "array2d.d", currently, I must write "import array2d.d", and add "-I myGame/lib_algo" to my compiler command line.
I said in a previous post that this would not scale.
That's because with this scheme, obviously, I can't have two modules having the same name, even if they belong to different libraries.
Let's say I have "lib_algo/utils.d" and "lib_geom/utils.d", if "main.d" says "import utils.d", it's ambiguous.

Clearly, "main.d" needs to say "import lib_algo.utils". However, if I do this,
I must add a module declaration in "lib_algo/utils.d", and I must change
"lib_algo/test.d" so it says "import lib_algo.utils".

This is where it begins to feel clumsy. Why should lib_algo need to be
modified in order to resolve an ambiguity happening in "myGame" ?

Moreover, we saw earlier that this modification of the import done by
"lib_algo/test.d" had consequences on the directory structure of lib_algo.
In order to be able to build the test executable for lib_algo, I now need to
have the following structure:
$ find lib_algo
lib_algo/lib_algo/options.d
lib_algo/lib_algo/....
lib_algo/lib_algo/array2d.d
lib_algo/test.d
(test.d can be put inside the inner "lib_algo")

Am I the only one to find this counterintuitive - and probably wrong?
What am I missing there?

(And please, don't tell me that I need to switch to dub+git+dmd+D_source_files_only ; I like my tools to be composable)

Thanks for reading!

On 02.03.2016 21:40, Sebastien Alaiwan wrote:
> - I only work with separate compilation, using gdc (Windows-dmd produces
> OMF, which is a dealbreaker ; but I use rdmd a lot for small programs).

dmd gives you COFF with -m64 or -m32mscoff (-m32 is the default is => OMF).

[...]
> Clearly, "main.d" needs to say "import lib_algo.utils". However, if I do
> this,
> I must add a module declaration in "lib_algo/utils.d", and I must change
> "lib_algo/test.d" so it says "import lib_algo.utils".

That's the way you're supposed to do it, yes. One source file <-> one module name.

> This is where it begins to feel clumsy. Why should lib_algo need to be
> modified in order to resolve an ambiguity happening in "myGame" ?

I don't know the exact reasons why the module system works like it does. Maybe you're on to something, maybe not. Anyway, here are some problems/surprises I see with your alternative idea (which basically means using relative file paths, if I get you right):

* You need to be aware of the directory structure when reading/writing code, as `import foo;` may mean different things depending on where you are. And you don't have a module declaration to tell you where you are.

* You're not able to copy imports from one file to another anymore.

* When moving a module, you have to update all its imports. You can introduce a subtle bug when you miss one, because the old import may still be valid but have a different meaning.

* The name of a module depends on the current working directory of the compiler run. You can't change directories between compiler calls and link the object files together.

* You can deliberately or accidentally import one source file under two different module names. Two modules from one source file is weird, especially when it happens accidentally.

* Can't just throw source files from different packages at the compiler anymore without thinking about their relative locations or the -I switch.

[...]
> Moreover, we saw earlier that this modification of the import done by
> "lib_algo/test.d" had consequences on the directory structure of lib_algo.
> In order to be able to build the test executable for lib_algo, I now
> need to
> have the following structure:
> $ find lib_algo
> lib_algo/lib_algo/options.d
> lib_algo/lib_algo/....
> lib_algo/lib_algo/array2d.d
> lib_algo/test.d
> (test.d can be put inside the inner "lib_algo")

Or, you know: `-I..` :P But changing the directory structure is probably nicer.

> Am I the only one to find this counterintuitive - and probably wrong?
> What am I missing there?

I don't think your alternative is obviously superior. The current way of having exactly one fixed name per module makes it easy to see what's going on at any point. You have to pay for that by typing a bit more, I guess.

On Wednesday, 2 March 2016 at 20:40:39 UTC, Sebastien Alaiwan wrote:

> However, it feels wrong that modules (foo, bar) inside the same package ("pkg") need to repeat the "absolute" package name when referencing each other, I mean "import pkg.foo" instead of "import foo". Not because of the extra typing, but because of the dependency it introduces. But as I said, maybe my vision is wrong.

The package name is part of the module name. If you use thislib.somemod and thatlib.somemod, how would you disambiguate them without the package name? Even in C and C++, libraries are often distributed with header files in a sub directory, where the top-level directory is intended to be added to the include path and not the subdirecotry, otherwise two includes with the same name conflict. Some libraries go further and prefix their header files with a namespace (like sdl_foo.h, sdl_bar.h), because they get no help from the compiler with this. Making the package name part of the fully qualified module name helps minimize the likelihood of such conflicts. It also gives a natural namespace for conflicting symbols in two modules named foo. You can use thispack.somemod.foo and thatpack.somemod.foo to disambiguate.


> - I only work with separate compilation, using gdc (Windows-dmd produces OMF, which is a dealbreaker ; but I use rdmd a lot for small programs).

-m64 or -m32mscoff

> - I often mix C code with D, sometimes, C++ code. I compile them using gcc/g++.

Assuming MinGW, the GDC is definitely your best bet here. The flags above will get DMD to produce COFF for the MS linker, but MinGW COFF and MS COFF are not always compatible.


>
> With this flat directory structure, the only way test.d can import files is by saying, "import stack;", not  "import lib_algo.stack".
> (adding ".." to the list of import directories might do the trick, but this would definitely feel wrong).

It is wrong. You should add ../mylibs :)


> Now, let's see a project using this "lib_algo", I call it "myGame".
>
> $ cd myProjects
> $ find <files relevant to this discussion>
> myGame/lib_algo/options.d
> myGame/lib_algo/....
> myGame/lib_algo/array2d.d
> myGame/lib_algo/project.mk
> myGame/lib_algo/Makefile
> myGame/src/main.d
> myGame/Makefile
>
>
> "lib_algo" simply is an external (aka "git submodule"), pointing to a specific revision of the repository "lib_algo".
> The top-level "Makefile" of the projects includes "lib_algo/project.mk", and all is well, I can compose the list of source files to compile without having to rely on "convenience/intermediate" libraries.
>
> Now, if "src/main.d" wants to import "array2d.d", currently, I must write "import array2d.d", and add "-I myGame/lib_algo" to my compiler command line.
> I said in a previous post that this would not scale.
> That's because with this scheme, obviously, I can't have two modules having the same name, even if they belong to different libraries.
> Let's say I have "lib_algo/utils.d" and "lib_geom/utils.d", if "main.d" says "import utils.d", it's ambiguous.

Assuming that all of the libraries you are going to use are set up the same way, then you can simply pass -ImyGame. Then you can use lib_algo.utils, lib_geom.utils, and so on. What's the problem with that?

>
> Clearly, "main.d" needs to say "import lib_algo.utils". However, if I do this,
> I must add a module declaration in "lib_algo/utils.d", and I must change
> "lib_algo/test.d" so it says "import lib_algo.utils".
>
> This is where it begins to feel clumsy. Why should lib_algo need to be
> modified in order to resolve an ambiguity happening in "myGame" ?

You shouldn't have to "change" it. This is how it should be set up from the beginning. The idea is that you decide on a module structure as soon as you start your project and then you stick to it.

The module declaration is necessary when using packages because otherwise DMD has no idea how far up the path to go to create a default package name. That's why the it uses only the file name when no module declaration is present. The package name is the solution to *prevent* ambiguities, not to resolve them after the fact. If you don't want lib_algo/util.d and lib_geom/util.d to conflict, then the use both lib_algo and lib_geom as package names in module declarations. How else would you propose resolving the ambiguity?

>
> Moreover, we saw earlier that this modification of the import done by
> "lib_algo/test.d" had consequences on the directory structure of lib_algo.
> In order to be able to build the test executable for lib_algo, I now need to
> have the following structure:
> $ find lib_algo
> lib_algo/lib_algo/options.d
> lib_algo/lib_algo/....
> lib_algo/lib_algo/array2d.d
> lib_algo/test.d
> (test.d can be put inside the inner "lib_algo")
>
> Am I the only one to find this counterintuitive - and probably wrong?
> What am I missing there?

No, you don't have to set it up that way. If your top-level directory is always named the same as your top-level package name, then you can just use its parent directory.

mylibs/lib_algo/*.d
mylibs/lib_geom/*.d

dmd -Imylibs main.d

Done. Does not the same issue exist in C++? You either have to keep all of your headers in a single directory or pass multiple directories on the command line with -I.

What I used to do before I started using DUB exclusively was model the system I used for C and C++ projects. For every new project I created, I had created an imports subdirectory in the project, then ran a script (written in D) to copy over the source tree of the specific version of each library I wanted to use. Then I added -Iimport on the command line when compiling. Done.

The package and module system is very intuitive to me and has been from the beginning, so I'm still not quite sure what it is about it that bothers you. The two major issues you bring up here, the need to import package.module and how to structure your projects, are easily resolved. Am I misunderstanding something?

On Thursday, 3 March 2016 at 01:47:08 UTC, Mike Parker wrote:

> The package and module system is very intuitive to me and has been from the beginning, so I'm still not quite sure what it is about it that bothers you. The two major issues you bring up here, the need to import package.module and how to structure your projects, are easily resolved. Am I misunderstanding something?

I should also say that I get the impression that you aren't considering the package name as part of the module name. Maybe that's the root of your problem. As I said in a previous post, your desire to use the default package for everything so that you can simply import foo is perfectly fine. The price of doing that, as you point out here, is that you can easily run into conflicts. Packages are the solution to avoiding them and package names are part of the module name, such that module foo is no longer just foo, but mypack.foo. Without a module declaration, the compiler would need some other way to know that C:\\src\libs\mylib\mypack\foo.d is actually mypack.foo and not c.src.lib.mylib.mypack.foo (which again, is why it only uses the file name when no module declaration is given). I can't think of a better way to do it than putting it right there in the source file. Using a command line switch would not be scalable.

Hi guys,

thanks a lot for your answers!

On Wednesday, 2 March 2016 at 22:42:18 UTC, ag0aep6g wrote:
> On 02.03.2016 21:40, Sebastien Alaiwan wrote:
>> - I only work with separate compilation, using gdc (Windows-dmd produces OMF, which is a dealbreaker ; but I use rdmd a lot for small programs).
> dmd gives you COFF with -m64 or -m32mscoff (-m32 is the default is => OMF).
Thanks for the tip, I didn't know that.
I have actually deeper reasons for not using dmd (seemless cross-compilation, e.g "make CROSS_COMPILE=i686-w64-mingw32" or "make CROSS_COMPILE=arm-linux-gnueabi" without any special case in the Makefile ; and the way its unusual command line (-oftarget) conflicts with MSYS's path conversion, etc. ).

> I don't know the exact reasons why the module system works like it does. Maybe you're on to something, maybe not. Anyway, here are some problems/surprises I see with your alternative idea (which basically means using relative file paths, if I get you right):
Yes, I think we can say that.

I 100% agree with all the limitations you listed ; which makes a forced absolute-import system appear to be a lot more comfortable!

There's also one big limitation I see that you didn't list:
* If you're importing lib_a and lib_b both depending on lib_c, you need them to expect lib_c to be at the same path/namespace location. The current module system guarantees this, because all package names are global.

(And linking with precompiled binaries might become a nightmare)

However, using global package names means you're relying on the library providers to avoid conflicts, in their package names, although they might not even know each other. Which basically implies to always put your name / company name in your package name, like "import alaiwan.lib_algo.array2d;", or "import apple.lib_algo.array2d".
Or rename my "lib_algo" to something meaningless / less common, like "import diamond.array2d" (e.g Derelict, Pegged, imaged, etc.).

If, unfortunately, I happen to run into a conflict, i.e my project uses two unrelated libraries sharing a name for their top-level namespace, there's no way out for me, right?
(Except relying on shared objects to isolate symbols)

> I don't think your alternative is obviously superior.
Me neither :-) I'm just sharing my doubts about the current system!

> The current way of having exactly one fixed name per module makes it easy to see what's going on at any point. You have to pay for that by typing a bit more, I guess.
Yes, I guess you're right.

I'm going to follow your advice and Mike's for my projects, this will allow me to see things more clearly. Anyway, thanks again to all of you for your answers!

On Thursday, 3 March 2016 at 06:53:33 UTC, Sebastien Alaiwan wrote:
> Hi guys,
>
> thanks a lot for your answers!
>
> On Wednesday, 2 March 2016 at 22:42:18 UTC, ag0aep6g wrote:
>> On 02.03.2016 21:40, Sebastien Alaiwan wrote:
>>> - I only work with separate compilation, using gdc (Windows-dmd produces OMF, which is a dealbreaker ; but I use rdmd a lot for small programs).
>> dmd gives you COFF with -m64 or -m32mscoff (-m32 is the default is => OMF).
> Thanks for the tip, I didn't know that.
> I have actually deeper reasons for not using dmd (seemless cross-compilation, e.g "make CROSS_COMPILE=i686-w64-mingw32" or "make CROSS_COMPILE=arm-linux-gnueabi" without any special case in the Makefile ; and the way its unusual command line (-oftarget) conflicts with MSYS's path conversion, etc. ).
>
>> I don't know the exact reasons why the module system works like it does. Maybe you're on to something, maybe not. Anyway, here are some problems/surprises I see with your alternative idea (which basically means using relative file paths, if I get you right):
> Yes, I think we can say that.
>
> I 100% agree with all the limitations you listed ; which makes a forced absolute-import system appear to be a lot more comfortable!
>
> There's also one big limitation I see that you didn't list:
> * If you're importing lib_a and lib_b both depending on lib_c, you need them to expect lib_c to be at the same path/namespace location. The current module system guarantees this, because all package names are global.
>
> (And linking with precompiled binaries might become a nightmare)
>
> However, using global package names means you're relying on the library providers to avoid conflicts, in their package names, although they might not even know each other. Which basically implies to always put your name / company name in your package name, like "import alaiwan.lib_algo.array2d;", or "import apple.lib_algo.array2d".
> Or rename my "lib_algo" to something meaningless / less common, like "import diamond.array2d" (e.g Derelict, Pegged, imaged, etc.).
>
> If, unfortunately, I happen to run into a conflict, i.e my project uses two unrelated libraries sharing a name for their top-level namespace, there's no way out for me, right?
> (Except relying on shared objects to isolate symbols)

See modules doc page off dlang.org.
import io = std.stdio;
void main()
{ io.writeln("hello!"); // ok, calls std.stdio.writeln
std.stdio.writeln("hello!"); // error, std is undefined
writeln("hello!"); // error, writeln is undefined
}

Also static import and scoped imports.

On Thursday, 3 March 2016 at 08:39:51 UTC, Laeeth Isharc wrote:
> On Thursday, 3 March 2016 at 06:53:33 UTC, Sebastien Alaiwan
>> If, unfortunately, I happen to run into a conflict, i.e my project uses two unrelated libraries sharing a name for their top-level namespace, there's no way out for me, right?
>> (Except relying on shared objects to isolate symbols)
>
> See modules doc page off dlang.org.
> import io = std.stdio;
> void main()
> { io.writeln("hello!"); // ok, calls std.stdio.writeln
> std.stdio.writeln("hello!"); // error, std is undefined
> writeln("hello!"); // error, writeln is undefined
> }
Thansk, but I was talking about module name collisions, e.g if my project uses two unrelated libraries, both - sloppily - defining a module called "geom.utils".

Can this be worked-around using renamed imports?

How is "import myUtils = geom.utils" going to know which of both "geom.utils" I'm talking about?
As far as I understand, _module_ renaming at import is rather a tool for defining shorthands, than for disambiguation.

I agree that it might not be a problem in practice - just wanting to make sure I'm not missing something.

On 03.03.2016 07:53, Sebastien Alaiwan wrote:
> However, using global package names means you're relying on the library
> providers to avoid conflicts, in their package names, although they
> might not even know each other. Which basically implies to always put
> your name / company name in your package name, like "import
> alaiwan.lib_algo.array2d;", or "import apple.lib_algo.array2d".
> Or rename my "lib_algo" to something meaningless / less common, like
> "import diamond.array2d" (e.g Derelict, Pegged, imaged, etc.).

Yeah, that's a thing. For published libraries one should choose a unique name.

I think reverse domain name notation [1] is a convention for exactly that. As far as I know, not many D projects do that, though, if any at all. But it would be a way out if collisions become a problem.

Also, dub/code.dlang.org plays a role here, as people may assume that a name is free as long as it's not on the dub list. This works well when dub is indeed the one place to go for libraries. If it isn't, then things may get messy.

> If, unfortunately, I happen to run into a conflict, i.e my project uses
> two unrelated libraries sharing a name for their top-level namespace,
> there's no way out for me, right?
> (Except relying on shared objects to isolate symbols)

Yeah, as far as I can tell, you would have to change the name of one of the libraries, and update all related module declarations and imports.


[1] https://en.wikipedia.org/wiki/Reverse_domain_name_notation

On Thursday, 3 March 2016 at 20:05:03 UTC, ag0aep6g wrote:
> [1] https://en.wikipedia.org/wiki/Reverse_domain_name_notation

LOL at the code section of that page.