Hi all,

I've came across the following problem number of times.
Let's say I have project A and B, sharing code but having a different tree structure.

$ find projectA
./projectA/internal/math/algo.d
./projectA/internal/math/lcp.d
./projectA/internal/math/optimize.d
./projectA/gui/main.d

$ find projectB
./projectB/app/render/gfx.d
./projectB/app/render/algo.d
./projectB/app/physics/math/algo.d
./projectB/app/physics/math/lcp.d
./projectB/app/physics/math/optimize.d
./projectB/main.d

The directory "math" is shared between projects. It actually is an external/submodule. So it has a standalone existence as a library, and might one day be used by projectC.
(In the context of this issue, I'm using separate compilation).

I'd like to be able to write, in projectA's main:

import internal.math.optimize;

This requires me to add, at the beginning of "optimize.d" file, this module definition:
module internal.math.optimize;

However, this "optimize.d" file is shared between projects, now it's becoming specific to projectA.

How am I supposed to share code between projects then?

Clearly, putting everyone back into the same "file namespace" by adding every subdirectory of my project as import paths through the command line is not going to scale.

After many projects spent thinking of this issue, I came to the conclusion that being able to specify, on the command line, the module name of the module currently being compiled, thus allowing to override it to be inferred to the basename of the file, would solve this problem.

As a side-problem related to this one, having to repeat the "absolute" module path to all module declarations/importations is tedious. What if I change the name of the package? Now I might have to change hundreds of module declarations from "module oldname.algo;" to "module newname.algo;".

Clearly, something must be wrong here (and I hope it's me!)
I'd be very happy if someone showed me how solve this problem, this has bothered me for ages.

On 29.02.2016 20:03, Sebastien Alaiwan wrote:
> $ find projectA
> ./projectA/internal/math/algo.d
> ./projectA/internal/math/lcp.d
> ./projectA/internal/math/optimize.d
> ./projectA/gui/main.d
>
> $ find projectB
> ./projectB/app/render/gfx.d
> ./projectB/app/render/algo.d
> ./projectB/app/physics/math/algo.d
> ./projectB/app/physics/math/lcp.d
> ./projectB/app/physics/math/optimize.d
> ./projectB/main.d
>
> The directory "math" is shared between projects. It actually is an
> external/submodule. So it has a standalone existence as a library, and
> might one day be used by projectC.
> (In the context of this issue, I'm using separate compilation).
[...]
> Clearly, putting everyone back into the same "file namespace" by adding
> every subdirectory of my project as import paths through the command
> line is not going to scale.

How is not going to scale? You have to add an import path for every location where packages are to be found. How many such locations are you going to have that this is unreasonable?

With your example, you'd have -Iinternal in project A, and -Iapp/physics in project B. The module declaration would of course be `module math.optimize;`.

On Monday, 29 February 2016 at 19:03:53 UTC, Sebastien Alaiwan wrote:
> Clearly, something must be wrong here (and I hope it's me!)
> I'd be very happy if someone showed me how solve this problem, this has bothered me for ages.

My solution would be to never have the same file be part of multiple projects. Share code using libraries rather than just sharing the files. Then modules won't be changing where they are in a project or anything like that. The shared code goes into a library, and whichever projects need it link against that library. You get better modularization and encapsulation that way and completely avoid the problem that you're having.

- Jonathan M Davis

On Monday, 29 February 2016 at 19:03:53 UTC, Sebastien Alaiwan wrote:
> Hi all,
>
> I've came across the following problem number of times.
> Let's say I have project A and B, sharing code but having a different tree structure.
>
> $ find projectA
> ./projectA/internal/math/algo.d
> ./projectA/internal/math/lcp.d
> ./projectA/internal/math/optimize.d
> ./projectA/gui/main.d
>
> $ find projectB
> ./projectB/app/render/gfx.d
> ./projectB/app/render/algo.d
> ./projectB/app/physics/math/algo.d
> ./projectB/app/physics/math/lcp.d
> ./projectB/app/physics/math/optimize.d
> ./projectB/main.d

I've used this pattern.

 ./projectA/lib/math/algo.d
 ./projectA/lib/math/lcp.d
 ./projectA/lib/math/optimize.d
 ./projectA/gui/main.d

 ./projectB/app/render/gfx.d
 ./projectB/app/render/algo.d
 ./projectB/lib/math/algo.d
 ./projectB/lib/math/lcp.d
 ./projectB/lib/math/optimize.d
 ./projectB/main.d


Dub doesn't like me too much, but in general it works.

Note that it doesn't matter where your modules live, if they have the declared module name in the file:

module math.optimize;

If DMD reads this file all your imports should look like:

import math.optimize;

Even if the file is in app/physics/math.

On Monday, 29 February 2016 at 19:23:08 UTC, Jonathan M Davis wrote:
> My solution would be to never have the same file be part of multiple projects. Share code using libraries rather than just sharing the files. Then modules won't be changing where they are in a project or anything like that. The shared code goes into a library, and whichever projects need it link against that library. You get better modularization and encapsulation that way and completely avoid the problem that you're having.
Thanks for your answer ; from what I understand, you're basically telling me to only import binary/precompiled libraries, right?

Although I had to admit I had never considered this solution, never have the same file be part of multiple projects seems extremely restrictive to me.
How is it going to work if I want to share heavily templated code, like a container library?

On Monday, 29 February 2016 at 19:56:20 UTC, Jesse Phillips wrote:
> I've used this pattern.
>
>  ./projectA/lib/math/algo.d
>  ./projectA/lib/math/lcp.d
>  ./projectA/lib/math/optimize.d
>  ./projectA/gui/main.d
>
>  ./projectB/app/render/gfx.d
>  ./projectB/app/render/algo.d
>  ./projectB/lib/math/algo.d
>  ./projectB/lib/math/lcp.d
>  ./projectB/lib/math/optimize.d
>  ./projectB/main.d
>
> Dub doesn't like me too much, but in general it works.
>
> Note that it doesn't matter where your modules live, if they have the declared module name in the file:
>
> module math.optimize;
>
> If DMD reads this file all your imports should look like:
>
> import math.optimize;
>
> Even if the file is in app/physics/math.
Yeah, I'm using this pattern too at the moment ;
Although, I'm trying to avoid having these redundant module declaration directives at the beginning of each of my library files.

On Monday, 29 February 2016 at 20:05:11 UTC, Sebastien Alaiwan wrote:
> Although, I'm trying to avoid having these redundant module declaration directives at the beginning of each of my library files.

Those module declarations aren't redundant - they are virtually required (I think it is a mistake that they aren't explicitly required in all cases, actually)

The file layout does not matter to the language itself. Only that module declaration does - it is NOT optional if you want a package name.

On Monday, 29 February 2016 at 20:59:45 UTC, Adam D. Ruppe wrote:
> On Monday, 29 February 2016 at 20:05:11 UTC, Sebastien Alaiwan wrote:
>> Although, I'm trying to avoid having these redundant module declaration directives at the beginning of each of my library files.
>
> Those module declarations aren't redundant - they are virtually required (I think it is a mistake that they aren't explicitly required in all cases, actually)
>
> The file layout does not matter to the language itself. Only that module declaration does - it is NOT optional if you want a package name.

$ find
main.d
foo/hello.d

$ cat main.d
import foo.hello;

$ cat foo/hello.d
module foo.hello;

Ok so now let's say I rename the directory "lib" to "foo". If I don't change the "import lib.hello" to "import foo.hello", how is the compiler going to find "hello.d"?
(As a reminder, as I said, I'm using separate compilation)

On Monday, 29 February 2016 at 20:00:48 UTC, Sebastien Alaiwan wrote:
> On Monday, 29 February 2016 at 19:23:08 UTC, Jonathan M Davis wrote:
>> My solution would be to never have the same file be part of multiple projects. Share code using libraries rather than just sharing the files. Then modules won't be changing where they are in a project or anything like that. The shared code goes into a library, and whichever projects need it link against that library. You get better modularization and encapsulation that way and completely avoid the problem that you're having.
> Thanks for your answer ; from what I understand, you're basically telling me to only import binary/precompiled libraries, right?
>
> Although I had to admit I had never considered this solution, never have the same file be part of multiple projects seems extremely restrictive to me.
> How is it going to work if I want to share heavily templated code, like a container library?

Phobos is a library, and templates work with it just fine. You're still importing the .d files. They're just compiled in as part of the library that you link against rather than compiled as part of your project - e.g. pretty much every D program is linked against libphobos, and none of them compile in modules like std.stdio. They just import them. They were compiled as part of libphobos are aside from templates are not compiled as part of your program, just linked.

As for templated code, the templates are generated when your code is built, so they're not really linked in in the normal sense, but those modules are still part of the library and not part of your code, so you'd never do something like add std/algorithm/searching.d to the list of files that you're compiling. You'd just import it.

If someone wants to hide code, then they'd use .di files and give you those to link against, in which case non-templated code could be hidden, but templated code would still be in those files, because the compiler has to see their source when they're used. And most folks will likely just use .d files, since it's simpler and allows stuff like CTFE and inlining to work.

Most of the projects on code.dlang.org are libraries, and you're using dub, they're easy to pull in and link against. You can do the same with your code.

- Jonathan M Davis

On Monday, 29 February 2016 at 21:04:52 UTC, Sebastien Alaiwan wrote:
> Ok so now let's say I rename the directory "lib" to "foo". If I don't change the "import lib.hello" to "import foo.hello", how is the compiler going to find "hello.d"?

You have to tell the compiler where it is. Either way, the module name is *not* optional.

> (As a reminder, as I said, I'm using separate compilation)

meh that's part of your problem, why are you doing it that way?