View mode: basic / threaded / horizontal-split · Log in · Help
July 13, 2011
Prototype buildsystem "Drake"
The recent discussions about package managers and buildsystems has prompted 
me to get off my ass (or rather, *on* my ass as per how programming is 
usually performed...) and start on the D-based rake-inspired buildtool I've 
been meaning to make for awhile now. It's not actually usable yet, but 
there's a sample drakefile demonstrating everything, and it does actually 
compile and run (but just doesn't build the dependency tree or actually run 
any of the build steps). *Should* work on Posix, but I only tested on 
Windows, so I may have fucked up the Posix version...

Apologies to Jacob Carlborg for the name being so close to "dake". Didn't 
really know what else to call it ("Duck", maybe?) Like dake, it's inspired 
by Ruby's Rake. But unlike dake, the buildscript is written in D instead of 
Ruby, which was my #1 reason for making a Rake alternative.

Before I go implemeting everything, I'd like to get input on it. Is it 
something that could be a major D tool?

Overview of Drake and links to all the code are here:

https://bitbucket.org/Abscissa256/drake/wiki/Home
July 13, 2011
Re Build tools for D [ was Re: Prototype buildsystem "Drake" ]
On Tue, 2011-07-12 at 21:02 -0400, Nick Sabalausky wrote:
[ . . . ]
> Before I go implemeting everything, I'd like to get input on it. Is it 
> something that could be a major D tool?
[ . . . ]

Given the nature of the debate, I will add to the mix that SCons, a
pre-existing -- and indeed working :-) -- system, has a D tool and can
compile and link D code.  What it needs is some love to bring it up to
the level of the C and C++ support.

SCons has a built-in D tool which needs work, but rather than fork SCons
to work on the tool I have created a separate tool that can be used with
any SCons installation.  See https://bitbucket.org/russel/scons_dmd_new.
Prior to any SCons release a patch between this version and the one in
the SCons core will be made and applied.

I started using SCons (and recently Waf -- which also has a D tool) in
preference to Rake because SCons has very superior support for C, C++,
Fortran and LaTeX.  Using Rake for these is like using assembly language
to write a GUI application:  with Rake you have to build everything
yourself, with SCons most of the work is done for you in the tool
infrastructure.  There was Rant which tried to replace Rake in the C, C
++, Fortran arena but the project died.  Rake appears to have almost no
traction outside the Ruby community.  Even Buildr (which tried to build
on Rake to combat Maven seems to have no headway in the market.

Clearly there is always a trend for a language to demand a specialist
build tool:  Scala has SBT, Clojure has Leiningen, Ruby has Rake, but
there is also a section of the universe that thinks the same build tool
should be usable across all languages.  Ant and Maven were Java specific
but now can handle anything targeting the JVM.  SCons and Waf come from
a C, C++, Fortran, LaTeX, D background but can now handle Java, Scala,
etc.  Gradle arose as a replacement for Maven on the JVM, but is now
(finally) branching out into C, C++, etc.

Note also that systems like Gradle and indeed Maven, are not just code
compiler and link systems, they also manage code review tools for
creating coverage reports, bug reports, documentation generation, even
whole websites.

Currently, from what I can tell, we have a number of individuals making
proposals for ideas.  Nothing wrong with that per se.  However I think
this is the third time this debate has occurred since I have been
(peripherally) involved with the core D community.  This does strike me
as wrong since it means that there is no momentum being created, the
energy associated with each debate simply dissipates.

I think that in order to create a momentum, a fundamental choice needs
to be made:

--  Should D have a specialist tool (à la SBT/Scala, Leiningen/Clojure
-- at the risk of massive NIH-ism, just as the Scala and Clojure folk
appear to have) or go with a generalist tool such as Gradle or SCons.

True there has to be debate about the possibilities in each category so
as to get the "lay of the land" and a feel for the "pluses" and
"minuses", but always the aim should be to answer the above question.

Then we can move to debating the possibilities.

Then we can create some momentum behind doing something by creating a
group of activists.

Then D build wins.


-- 
Russel.
=============================================================================
Dr Russel Winder      t: +44 20 7585 2200   voip: sip:russel.winder@ekiga.net
41 Buckmaster Road    m: +44 7770 465 077   xmpp: russel@russel.org.uk
London SW11 1EN, UK   w: www.russel.org.uk  skype: russel_winder
July 13, 2011
Re: Re Build tools for D [ was Re: Prototype buildsystem "Drake" ]
And, of course, I should have mentioned CMake and CMakeD.

The fact that I forgot, shows my prejudice against Makefile-based
systems and for direct DAG-based systems such as Gradle, SCons and Waf.
This though should not stop CMakeD being a part of this debate. 

-- 
Russel.
=============================================================================
Dr Russel Winder      t: +44 20 7585 2200   voip: sip:russel.winder@ekiga.net
41 Buckmaster Road    m: +44 7770 465 077   xmpp: russel@russel.org.uk
London SW11 1EN, UK   w: www.russel.org.uk  skype: russel_winder
July 13, 2011
Re: Re Build tools for D [ was Re: Prototype buildsystem "Drake" ]
On Wednesday 13 July 2011 06:12:58 Russel Winder wrote:
> And, of course, I should have mentioned CMake and CMakeD.
> 
> The fact that I forgot, shows my prejudice against Makefile-based
> systems and for direct DAG-based systems such as Gradle, SCons and Waf.
> This though should not stop CMakeD being a part of this debate.

>From previous discussions, it seems that one of the primary reasons for having 
a D build tool in many people's minds is to also handle package management of 
D libraries (like Haskell's cabal or rubygems for ruby). And as great as 
cmaked, scons, gradle, waf, and other such tools may be, they don't do that.

- Jonathan M Davis
July 13, 2011
Re: Re Build tools for D [ was Re: Prototype buildsystem "Drake" ]
On Tue, 2011-07-12 at 22:28 -0700, Jonathan M Davis wrote:
[ . . . ]
> From previous discussions, it seems that one of the primary reasons for having 
> a D build tool in many people's minds is to also handle package management of 
> D libraries (like Haskell's cabal or rubygems for ruby). And as great as 
> cmaked, scons, gradle, waf, and other such tools may be, they don't do that.

Go is currently using Make for this -- they have a structured Makefile
hierarchy that handles most compilation and linking in the context of a
rigidly enforced filestore structure, and goinstall for bringing in
packages from outside into the filestore hierarchy.  It is a bit
primitive at the minute, but is being worked on and rapidly improved.

Go actually has a plethora of build tools, include a couple of
SCons-based ones, most of which are falling by the wayside.  I think the
effort expended doing this has not been a waste as information is being
generated that is adding to the pool.  I think they will replace the
Makefile system shortly with one of the tools, possibly GoBuild, but I
can't remember exactly.

Haskell and Ruby both have inward looking approaches -- to put it
bluntly (at the risk of causing offense to some), Haskell build only
cares about Haskell source and Ruby build only cares about Ruby source.
Almost by definition D has to live in a mixed C, C++, Fortran, D
universe, so this has to be an issue from the outset.

I agree with the premise that package management must be a core part of
the build management, but I disagree that Gradle, SCons, and Waf cannot
handle this.  They cannot handle this today true, but in the same way
that there is currently no system that does it properly as yet.  So I
would suggest that package management is currently a "green field"
problem that can be handled by a D-specific tool or one of Gradle, SCons
or Waf.

-- 
Russel.
=============================================================================
Dr Russel Winder      t: +44 20 7585 2200   voip: sip:russel.winder@ekiga.net
41 Buckmaster Road    m: +44 7770 465 077   xmpp: russel@russel.org.uk
London SW11 1EN, UK   w: www.russel.org.uk  skype: russel_winder
July 13, 2011
Re: Re Build tools for D [ was Re: Prototype buildsystem "Drake" ]
On Wednesday 13 July 2011 06:53:28 Russel Winder wrote:
> On Tue, 2011-07-12 at 22:28 -0700, Jonathan M Davis wrote:
> [ . . . ]
> 
> > From previous discussions, it seems that one of the primary reasons for
> > having a D build tool in many people's minds is to also handle package
> > management of D libraries (like Haskell's cabal or rubygems for ruby).
> > And as great as cmaked, scons, gradle, waf, and other such tools may
> > be, they don't do that.
> 
> Go is currently using Make for this -- they have a structured Makefile
> hierarchy that handles most compilation and linking in the context of a
> rigidly enforced filestore structure, and goinstall for bringing in
> packages from outside into the filestore hierarchy.  It is a bit
> primitive at the minute, but is being worked on and rapidly improved.
> 
> Go actually has a plethora of build tools, include a couple of
> SCons-based ones, most of which are falling by the wayside.  I think the
> effort expended doing this has not been a waste as information is being
> generated that is adding to the pool.  I think they will replace the
> Makefile system shortly with one of the tools, possibly GoBuild, but I
> can't remember exactly.
> 
> Haskell and Ruby both have inward looking approaches -- to put it
> bluntly (at the risk of causing offense to some), Haskell build only
> cares about Haskell source and Ruby build only cares about Ruby source.
> Almost by definition D has to live in a mixed C, C++, Fortran, D
> universe, so this has to be an issue from the outset.
> 
> I agree with the premise that package management must be a core part of
> the build management, but I disagree that Gradle, SCons, and Waf cannot
> handle this.  They cannot handle this today true, but in the same way
> that there is currently no system that does it properly as yet.  So I
> would suggest that package management is currently a "green field"
> problem that can be handled by a D-specific tool or one of Gradle, SCons
> or Waf.

Feel free to propose whatever you'd like for a D build system (be it something 
entirely new or an improvement of an existing build tool). I'm just pointing 
out that one of the primary reasons that a number of the people on this list 
want a build tool is to handle package management of D libraries, so whatever 
solution you propose should incorporate that or a lot of people aren't going 
to be all that interested in it or will at least see it as inferior to another 
solution which does incorporate package management.

- Jonathan M Davis
July 13, 2011
Re: Re Build tools for D [ was Re: Prototype buildsystem "Drake" ]
From: "Russel Winder" <russel@russel.org.uk>
>
>I agree with the premise that package management must be a core part of
>the build management,

If I understand what you're saying correctly, then I strongly disagree. A 
package management tool, by necessity, must be a common agreed-upon standard 
(otherwise it won't serve it's purpose). But if that package management is 
tied into build management then we're *also* forcing a single standard build 
system on everyone. So then instead of being able to choose your build 
system, everyone now has to come to an agreement on the same build system, 
which #1: Is unlikely to ever happen, and #2: Serves no useful purpose.
July 13, 2011
Re: Re Build tools for D [ was Re: Prototype buildsystem "Drake" ]
"Russel Winder" <russel@russel.org.uk> wrote in message 
news:mailman.1585.1310533819.14074.digitalmars-d@puremagic.com...
>On Tue, 2011-07-12 at 21:02 -0400, Nick Sabalausky wrote:
>[ . . . ]
>> Before I go implemeting everything, I'd like to get input on it. Is it
>> something that could be a major D tool?
>[ . . . ]
>
>Given the nature of the debate, I will add to the mix that SCons, a
>pre-existing -- and indeed working :-) -- system, has a D tool and can
>compile and link D code.  What it needs is some love to bring it up to
>the level of the C and C++ support.
>
>SCons has a built-in D tool which needs work, but rather than fork SCons
>to work on the tool I have created a separate tool that can be used with
>any SCons installation.  See https://bitbucket.org/russel/scons_dmd_new.
>Prior to any SCons release a patch between this version and the one in
>the SCons core will be made and applied.
>
[...Waf, Rake, Ant, Maven, etc...]

First of all, just to be clear, Drake isn't D-specific. Like Rake, it's 
designed to work with anything you can programmatically invoke (and doesn't 
require any special module to support a particular language or tool). After 
having worked on projects that, by necessity, involved multiple languages, I 
find that quality essential in a build system.

Regarding SCons, Waf, Rake, etc, the point of Drake is that 1: it doesn't 
force D projects to be reliant on Python, Ruby, etc. And 2: It allows D 
users to use D for their buildscript instead of needing to switch to a 
different (and less preferable IMO) langauge. (And I *hate* language-soup 
projects - projects that use a totally different language for every little 
damn thing.)

The *make tools (whether they use makefiles directly or indirectly) aren't 
even worth consideration. Makefiles need to just die, period.

Regarding this topic's frequent ressurection, I see that as a direct result 
of it all being *merely* talk, with very little concrete progress. We don't 
even *have* a proper package management system (there's DSSS, but it's dead, 
bitrotted, and I don't think it handles dependencies or multiple versions of 
the same package). So naturally that topic's going to keep coming up until 
we have something. And as for build systems, everything either lacks 
multiple-target/configuration management (rdmd), or creates a dependency on 
(and a need to write in) some other langauge (SCons, Rake, etc). So 
naturally there's also going to be people that feel unsatisfied with the 
build systems until something fills that role. I'm trying to get past 
"talk", so I've started some actual code to address what I see as the 
current issues for build systems.

Of course, it doesn't help that everyone seems to want something completely 
different from what everyone else wants. ;)
July 13, 2011
Re: Re Build tools for D [ was Re: Prototype buildsystem "Drake" ]
On Wednesday 13 July 2011 03:01:04 Nick Sabalausky wrote:
> Regarding this topic's frequent ressurection, I see that as a direct result
> of it all being *merely* talk, with very little concrete progress. We don't
> even *have* a proper package management system (there's DSSS, but it's dead,
> bitrotted, and I don't think it handles dependencies or multiple versions
> of the same package). So naturally that topic's going to keep coming up
> until we have something.

I belive that Jacob Carlborg is working on a solution. So,  there _is_ work 
being done on it. By no means does that mean that you shouldn't work on a 
solution, but it's not the case that no one has been working on the problem - 
though it is true that it's been being brought up for far longer than it's 
been being worked on.

- Jonathan M Davis
July 13, 2011
Re: Re Build tools for D [ was Re: Prototype buildsystem "Drake" ]
Nick Sabalausky Wrote:

> From: "Russel Winder" <russel@russel.org.uk>
> >
> >I agree with the premise that package management must be a core part of
> >the build management,
> 
> If I understand what you're saying correctly, then I strongly disagree. A 
> package management tool, by necessity, must be a common agreed-upon standard 
> (otherwise it won't serve it's purpose). But if that package management is 
> tied into build management then we're *also* forcing a single standard build 
> system on everyone. So then instead of being able to choose your build 
> system, everyone now has to come to an agreement on the same build system, 
> which #1: Is unlikely to ever happen, and #2: Serves no useful purpose.

A build tool should be able to invoke a package manager and package manager should be able to invoke a build tool, this doesn't mean a tie, it means a working out of the box solution. This solution is meant to just work. If people don't want solution that just works, they can go with something else of course. For example I prefer makefiles.
« First   ‹ Prev
1 2 3 4 5
Top | Discussion index | About this forum | D home