We should move to a D-based build system to build dmd/druntime/phobos/d-programming-language/tools.

Reasons, in case it's not obvious:

1) DRY: makefile is full of repetitions
2) cross platform: different makefiles for different architectures are
needed (even win32 vs win64!)
3) safety: makefile happily ignores that $VAR variables are not set, and
can create havoc; likewise tools/update.sh is a bash script and may not
stop on 1st error.
4) tools/update.sh doesn't work out of the box on OSX last i checked (I
made a pull request for that some time back:
https://github.com/timotheecour/d-programming-language.org/commit/557a2befa74ddfe99ee5c0e12e7c3d028f27d276)

Even a bad D-based build system will be better than a makefile, but it's not hard to write a reasonable one at least for the particular task of building dmd/druntime/phobos/d-programming-language/tools. It'll lower the entry point for people to contribute to dmd/phobos by making rebuilding trivial.

Here's how it'd work:

1) a mixed D-based/makefile build that'll work 'from scratch' via
bootstrapping:
1a) dmd: first uses a makefile to build dmd+druntime
1b) rdmd built from a D file using dmd
1c) phobos, d-programming-language, tools built from a D file using rdmd

2) a pure D-based build that'll work using an existing dmd binary (eg
distributed binary)
that's the one most will use.

On Thu, 30 May 2013 00:49:57 -0700
Timothee Cour <thelastmammoth@gmail.com> wrote:
> 
> Here's how it'd work:
> 
> 1) a mixed D-based/makefile build that'll work 'from scratch' via
> bootstrapping:
> 1a) dmd: first uses a makefile to build dmd+druntime
> 1b) rdmd built from a D file using dmd
> 1c) phobos, d-programming-language, tools built from a D file using
> rdmd
> 

Related: https://github.com/D-Programming-Language/installer/pull/18#issuecomment-18662826

On Thu, 30 May 2013 06:13:54 -0400
Nick Sabalausky <SeeWebsiteToContactMe@semitwist.com> wrote:

> On Thu, 30 May 2013 00:49:57 -0700
> Timothee Cour <thelastmammoth@gmail.com> wrote:
> > 
> > Here's how it'd work:
> > 
> > 1) a mixed D-based/makefile build that'll work 'from scratch' via
> > bootstrapping:
> > 1a) dmd: first uses a makefile to build dmd+druntime
> > 1b) rdmd built from a D file using dmd
> > 1c) phobos, d-programming-language, tools built from a D file using
> > rdmd
> > 
> 
> Related: https://github.com/D-Programming-Language/installer/pull/18#issuecomment-18662826
> 

Erm, I mean: https://github.com/D-Programming-Language/installer/pull/18

On Thu, 30 May 2013 00:49:57 -0700
Timothee Cour <thelastmammoth@gmail.com> wrote:

> We should move to a D-based build system to build dmd/druntime/phobos/d-programming-language/tools.
> 
> Reasons, in case it's not obvious:
> 
> 1) DRY: makefile is full of repetitions
> 2) cross platform: different makefiles for different architectures are
> needed (even win32 vs win64!)
> 3) safety: makefile happily ignores that $VAR variables are not set,
> and can create havoc; likewise tools/update.sh is a bash script and
> may not stop on 1st error.
> 4) tools/update.sh doesn't work out of the box on OSX last i checked
> (I made a pull request for that some time back:
> https://github.com/timotheecour/d-programming-language.org/commit/557a2befa74ddfe99ee5c0e12e7c3d028f27d276)
> 
> Even a bad D-based build system will be better than a makefile, but it's not hard to write a reasonable one at least for the particular task of building dmd/druntime/phobos/d-programming-language/tools. It'll lower the entry point for people to contribute to dmd/phobos by making rebuilding trivial.
> 
> Here's how it'd work:
> 
> 1) a mixed D-based/makefile build that'll work 'from scratch' via
> bootstrapping:
> 1a) dmd: first uses a makefile to build dmd+druntime
> 1b) rdmd built from a D file using dmd
> 1c) phobos, d-programming-language, tools built from a D file using
> rdmd
> 
> 2) a pure D-based build that'll work using an existing dmd binary (eg
> distributed binary)
> that's the one most will use.
> 

FWIW, phobos needs to be built before rdmd because rdmd relies on phobos.

Also, while I like the idea of replacing makefiles with a D solution, I'm unsure whether it's really a good idea because that may make it more difficult to bootstrap DMD to run on new platforms.

However, I do think that wrapping the invocations of all the makefiles is a good idea and would still have the benefit of making it harder to make mistakes such as (like you said) setting up the %VARs wrong.

In fact I've started working on such stuff already (And DVM can already compile DMD, too, although it's possible it may still have some room for improvement).

On Thursday, May 30, 2013 16:47:49 Nick Sabalausky wrote:
> Also, while I like the idea of replacing makefiles with a D solution, I'm unsure whether it's really a good idea because that may make it more difficult to bootstrap DMD to run on new platforms.

I would point out that we're going to have that problem anyway once we move the compiler to D. At some point, already having a fully functional D compiler _will_ be required in order to build dmd, druntime, and Phobos. If we switched to using D for writing the build scripts before switching the compiler to D, we'd simply be introducing that circular dependency earlier. So, that shouldn't ultimately be much of a factor in whether we make the switch.

Whether we switch to using D scripts for building dmd, druntime, and Phobos should depend on whether the result would be more maintainable and still be able to reasonably do everything that we need our build system to do. And while many of us really aren't very good at make and find the Makefiles that we have to be rather obtuse, folks that are more proficient at make might not think that making the switch to a D script would make sense. And I really don't know what the tradeoffs would be.

I did go with writing my build script in D for a recent project which was generating a document written in ddoc, and it's working out quite well to do that, but I don't know whether it ultimately makes sense to do the same for the dmd toolchain.

- Jonathan M Davis

On Thu, 30 May 2013 16:56:48 -0400
"Jonathan M Davis" <jmdavisProg@gmx.com> wrote:
> 
> And while many of us really aren't very good at make
> and find the Makefiles that we have to be rather obtuse, folks that
> are more proficient at make might not think that making the switch to
> a D script would make sense. And I really don't know what the
> tradeoffs would be.
> 

I normally despise make, but FWIW I do find DMD's makefiles to be *vastly* nicer than others, probably because they doesn't really use the DRY-ness features that usually cause make to be such a pain.

On Thursday, 30 May 2013 at 07:50:11 UTC, Timothee Cour wrote:
> We should move to a D-based build system to build
> dmd/druntime/phobos/d-programming-language/tools.
>
> Reasons, in case it's not obvious:
>
> 1) DRY: makefile is full of repetitions
> 2) cross platform: different makefiles for different architectures are
> needed (even win32 vs win64!)
> 3) safety: makefile happily ignores that $VAR variables are not set, and
> can create havoc; likewise tools/update.sh is a bash script and may not
> stop on 1st error.
> 4) tools/update.sh doesn't work out of the box on OSX last i checked (I
> made a pull request for that some time back:
> https://github.com/timotheecour/d-programming-language.org/commit/557a2befa74ddfe99ee5c0e12e7c3d028f27d276)
>
> Even a bad D-based build system will be better than a makefile, but it's
> not hard to write a reasonable one at least for the particular task of
> building dmd/druntime/phobos/d-programming-language/tools. It'll lower the
> entry point for people to contribute to dmd/phobos by making rebuilding
> trivial.
>
> Here's how it'd work:
>
> 1) a mixed D-based/makefile build that'll work 'from scratch' via
> bootstrapping:
> 1a) dmd: first uses a makefile to build dmd+druntime
> 1b) rdmd built from a D file using dmd
> 1c) phobos, d-programming-language, tools built from a D file using rdmd
>
> 2) a pure D-based build that'll work using an existing dmd binary (eg
> distributed binary)
> that's the one most will use.

You may want take a look at Qbs, it will the next build system of Qt in replacement of QMake.
http://qt-project.org/wiki/Qbs-Quick-Reference

Using a full featured language is good cause, there is no restriction and build aren't just build, it's also packaging/testing issues.
Doing something that just have to call compiler binaries is finaly the simple thing, and provide more control to users.