Is it just me that thinks the build system of D (DMD, Phobos, druntime, and so on) is really annoying. I'm mostly thinking of the directory structure that the makefiles expect.

For example, when building Phobos it will also, automatically, build druntime, which it expects to find in the parent directory.

Today I tried to build the dlang.org web site. The first thing that hit me was that it tried to build a complete release out of DMD by default. Sure that can be consider my fault since I didn't check what targets the makefile would run by default. But this makefile will also expect DMD, Phobos and so on to be in the parent directory. It will also expect that its own folder is named "d-programming-language.org". I renamed that to "dlang.org" since it's shorter and that's what the site is called these days, but that completely broke the makefile. The makefile is looking explicitly for a folder named "d-programming-language.org", i.e. "../d-programming-language.org/std.ddoc".

I think it's really annoying, and slightly scary, that the build system breaks out of its own directory, doing things in the parent directory and expects a certain parent directory structure.

-- 
/Jacob Carlborg

On 11/19/12, Jacob Carlborg <doob@me.com> wrote:
> Is it just me that thinks the build system of D (DMD, Phobos, druntime, and so on) is really annoying. I'm mostly thinking of the directory structure that the makefiles expect.

Well on win32 I have two directories on my drive, "dmd" and "dmd-git". "dmd" is an official release, while "dmd-git" is git-head. The dir structure is:

dmd-git\dmd2\src\dmd  <- git-head
dmd-git\dmd2\src\druntime <- git-head
dmd-git\dmd2\src\phobos <- git-head
dmd-git\dmd2\windows\  ... <- binaries, etc

"dmd-git" is in the PATH env variable before "dmd" so it always overrides it. When I want to switch between using git-head and an official release I append an underscore to "dmd-git" (a shortcut key can do the trick using autohotkey).

It's the simplest system I could think of..

On 11/19/12, Andrej Mitrovic <andrej.mitrovich@gmail.com> wrote:
> dmd-git\dmd2\src\dmd  <- git-head
> dmd-git\dmd2\src\druntime <- git-head
> dmd-git\dmd2\src\phobos <- git-head
> dmd-git\dmd2\windows\  ... <- binaries, etc

Also:
dmd-git\dmd2\src\d-programming-language.org

On 11/19/12 3:29 AM, Jacob Carlborg wrote:
> Is it just me that thinks the build system of D (DMD, Phobos, druntime,
> and so on) is really annoying. I'm mostly thinking of the directory
> structure that the makefiles expect.
>
> For example, when building Phobos it will also, automatically, build
> druntime, which it expects to find in the parent directory.

That's because phobos is dependent on druntime.

> Today I tried to build the dlang.org web site. The first thing that hit
> me was that it tried to build a complete release out of DMD by default.

That is because the current site docs must include the current druntime and phobos docs. In turn, those must be built with the currently released compiler. To automate all that, the build makes sure the latest released compiler is built and then the docs for the latest released libs are built.

Before that we were in the situation that updating the site was putting whatever was in the Phobos head (or worse, what was on the machine the site was built on).

> Sure that can be consider my fault since I didn't check what targets the
> makefile would run by default. But this makefile will also expect DMD,
> Phobos and so on to be in the parent directory. It will also expect that
> its own folder is named "d-programming-language.org". I renamed that to
> "dlang.org" since it's shorter and that's what the site is called these
> days, but that completely broke the makefile. The makefile is looking
> explicitly for a folder named "d-programming-language.org", i.e.
> "../d-programming-language.org/std.ddoc".

That's the name of the github project.

> I think it's really annoying, and slightly scary, that the build system
> breaks out of its own directory, doing things in the parent directory
> and expects a certain parent directory structure.

The makefiles have evolved organically out of clear and simple necessities. Also, they could use some love because there was more focus on solving specific automation issues, rather than on a globally nice system.

So if you have ideas on how to improve the above - great! Just don't fall for Chesterton's fence fallacy.


Andrei

On 2012-11-19 21:00, Andrei Alexandrescu wrote:

> That's the name of the github project.

Yeah, but I haven't seen any other project that is actually dependent on the name of its own folder.

> So if you have ideas on how to improve the above - great! Just don't
> fall for Chesterton's fence fallacy.

One solution that instantly pops in my mind is using git submodules. Create a new repository, say "dlang", add the other repositories as git submoduels to "dlang". Then the build system can do whatever it wants in the "dlang" directory and its sub directories, but it shouldn't escape the "dlang" directory.

Since Make is one of the worst thing I know, I'm not very enthusiastic about improving the build system.

-- 
/Jacob Carlborg

On 2012-11-19 11:34, Andrej Mitrovic wrote:

> Well on win32 I have two directories on my drive, "dmd" and "dmd-git".
> "dmd" is an official release, while "dmd-git" is git-head. The dir
> structure is:
>
> dmd-git\dmd2\src\dmd  <- git-head
> dmd-git\dmd2\src\druntime <- git-head
> dmd-git\dmd2\src\phobos <- git-head
> dmd-git\dmd2\windows\  ... <- binaries, etc
>
> "dmd-git" is in the PATH env variable before "dmd" so it always
> overrides it. When I want to switch between using git-head and an
> official release I append an underscore to "dmd-git" (a shortcut key
> can do the trick using autohotkey).
>
> It's the simplest system I could think of..

I think you misunderstood what I was trying to say, or I am misunderstanding you.

-- 
/Jacob Carlborg

On 11/20/12, Jacob Carlborg <doob@me.com> wrote:
> I think you misunderstood what I was trying to say, or I am misunderstanding you.

I'm showing the path of least resistance for how to build either dmd/druntime/phobos/dlang-org

> One solution that instantly pops in my mind is using git submodules.

They are buggy on Windows. The last time I've used them Git started showing thousands of error messages, crashed and erased my repo and commit history, but one of the last few commits were left in the reflog so I managed to recover my files. It also copied the submodule repo directory into randomly selected folders in an upper directory.

Maybe it's just an msysgit thing, but I'm never touching submodules again.

On 2012-11-20 09:21, Andrej Mitrovic wrote:

> I'm showing the path of least resistance for how to build either
> dmd/druntime/phobos/dlang-org

The build systems for the repositories are still breaking out of their top level directory.

> They are buggy on Windows. The last time I've used them Git started
> showing thousands of error messages, crashed and erased my repo and
> commit history, but one of the last few commits were left in the
> reflog so I managed to recover my files. It also copied the submodule
> repo directory into randomly selected folders in an upper directory.
>
> Maybe it's just an msysgit thing, but I'm never touching submodules again.

I haven't had any problems with submodules so far. I don't really understand why they wouldn't work as good on Windows as the other platforms.

-- 
/Jacob Carlborg

On 20-11-2012 09:21, Andrej Mitrovic wrote:
> On 11/20/12, Jacob Carlborg <doob@me.com> wrote:
>> I think you misunderstood what I was trying to say, or I am
>> misunderstanding you.
>
> I'm showing the path of least resistance for how to build either
> dmd/druntime/phobos/dlang-org
>
>> One solution that instantly pops in my mind is using git submodules.
>
> They are buggy on Windows. The last time I've used them Git started
> showing thousands of error messages, crashed and erased my repo and
> commit history, but one of the last few commits were left in the
> reflog so I managed to recover my files. It also copied the submodule
> repo directory into randomly selected folders in an upper directory.
>
> Maybe it's just an msysgit thing, but I'm never touching submodules again.
>

Er... what?

I certainly cannot say I've had the issues you describe. Submodules have always worked flawlessly on Windows for me.

-- 
Alex Rønne Petersen
alex@lycus.org
http://lycus.org