July 13, 2011
The recent discussions about package managers and buildsystems has prompted me to get off my ass (or rather, *on* my ass as per how programming is usually performed...) and start on the D-based rake-inspired buildtool I've been meaning to make for awhile now. It's not actually usable yet, but there's a sample drakefile demonstrating everything, and it does actually compile and run (but just doesn't build the dependency tree or actually run any of the build steps). *Should* work on Posix, but I only tested on Windows, so I may have fucked up the Posix version...

Apologies to Jacob Carlborg for the name being so close to "dake". Didn't really know what else to call it ("Duck", maybe?) Like dake, it's inspired by Ruby's Rake. But unlike dake, the buildscript is written in D instead of Ruby, which was my #1 reason for making a Rake alternative.

Before I go implemeting everything, I'd like to get input on it. Is it something that could be a major D tool?

Overview of Drake and links to all the code are here:

https://bitbucket.org/Abscissa256/drake/wiki/Home


July 13, 2011
On Tue, 2011-07-12 at 21:02 -0400, Nick Sabalausky wrote:
[ . . . ]
> Before I go implemeting everything, I'd like to get input on it. Is it something that could be a major D tool?
[ . . . ]

Given the nature of the debate, I will add to the mix that SCons, a pre-existing -- and indeed working :-) -- system, has a D tool and can compile and link D code.  What it needs is some love to bring it up to the level of the C and C++ support.

SCons has a built-in D tool which needs work, but rather than fork SCons to work on the tool I have created a separate tool that can be used with any SCons installation.  See https://bitbucket.org/russel/scons_dmd_new. Prior to any SCons release a patch between this version and the one in the SCons core will be made and applied.

I started using SCons (and recently Waf -- which also has a D tool) in preference to Rake because SCons has very superior support for C, C++, Fortran and LaTeX.  Using Rake for these is like using assembly language to write a GUI application:  with Rake you have to build everything yourself, with SCons most of the work is done for you in the tool infrastructure.  There was Rant which tried to replace Rake in the C, C ++, Fortran arena but the project died.  Rake appears to have almost no traction outside the Ruby community.  Even Buildr (which tried to build on Rake to combat Maven seems to have no headway in the market.

Clearly there is always a trend for a language to demand a specialist build tool:  Scala has SBT, Clojure has Leiningen, Ruby has Rake, but there is also a section of the universe that thinks the same build tool should be usable across all languages.  Ant and Maven were Java specific but now can handle anything targeting the JVM.  SCons and Waf come from a C, C++, Fortran, LaTeX, D background but can now handle Java, Scala, etc.  Gradle arose as a replacement for Maven on the JVM, but is now (finally) branching out into C, C++, etc.

Note also that systems like Gradle and indeed Maven, are not just code compiler and link systems, they also manage code review tools for creating coverage reports, bug reports, documentation generation, even whole websites.

Currently, from what I can tell, we have a number of individuals making proposals for ideas.  Nothing wrong with that per se.  However I think this is the third time this debate has occurred since I have been (peripherally) involved with the core D community.  This does strike me as wrong since it means that there is no momentum being created, the energy associated with each debate simply dissipates.

I think that in order to create a momentum, a fundamental choice needs to be made:

--  Should D have a specialist tool (à la SBT/Scala, Leiningen/Clojure -- at the risk of massive NIH-ism, just as the Scala and Clojure folk appear to have) or go with a generalist tool such as Gradle or SCons.

True there has to be debate about the possibilities in each category so as to get the "lay of the land" and a feel for the "pluses" and "minuses", but always the aim should be to answer the above question.

Then we can move to debating the possibilities.

Then we can create some momentum behind doing something by creating a group of activists.

Then D build wins.


-- 
Russel. ============================================================================= Dr Russel Winder      t: +44 20 7585 2200   voip: sip:russel.winder@ekiga.net 41 Buckmaster Road    m: +44 7770 465 077   xmpp: russel@russel.org.uk London SW11 1EN, UK   w: www.russel.org.uk  skype: russel_winder


July 13, 2011
And, of course, I should have mentioned CMake and CMakeD.

The fact that I forgot, shows my prejudice against Makefile-based systems and for direct DAG-based systems such as Gradle, SCons and Waf. This though should not stop CMakeD being a part of this debate.

-- 
Russel. ============================================================================= Dr Russel Winder      t: +44 20 7585 2200   voip: sip:russel.winder@ekiga.net 41 Buckmaster Road    m: +44 7770 465 077   xmpp: russel@russel.org.uk London SW11 1EN, UK   w: www.russel.org.uk  skype: russel_winder


July 13, 2011
On Wednesday 13 July 2011 06:12:58 Russel Winder wrote:
> And, of course, I should have mentioned CMake and CMakeD.
> 
> The fact that I forgot, shows my prejudice against Makefile-based systems and for direct DAG-based systems such as Gradle, SCons and Waf. This though should not stop CMakeD being a part of this debate.

>From previous discussions, it seems that one of the primary reasons for having
a D build tool in many people's minds is to also handle package management of D libraries (like Haskell's cabal or rubygems for ruby). And as great as cmaked, scons, gradle, waf, and other such tools may be, they don't do that.

- Jonathan M Davis
July 13, 2011
On Tue, 2011-07-12 at 22:28 -0700, Jonathan M Davis wrote: [ . . . ]
> From previous discussions, it seems that one of the primary reasons for having a D build tool in many people's minds is to also handle package management of D libraries (like Haskell's cabal or rubygems for ruby). And as great as cmaked, scons, gradle, waf, and other such tools may be, they don't do that.

Go is currently using Make for this -- they have a structured Makefile hierarchy that handles most compilation and linking in the context of a rigidly enforced filestore structure, and goinstall for bringing in packages from outside into the filestore hierarchy.  It is a bit primitive at the minute, but is being worked on and rapidly improved.

Go actually has a plethora of build tools, include a couple of SCons-based ones, most of which are falling by the wayside.  I think the effort expended doing this has not been a waste as information is being generated that is adding to the pool.  I think they will replace the Makefile system shortly with one of the tools, possibly GoBuild, but I can't remember exactly.

Haskell and Ruby both have inward looking approaches -- to put it bluntly (at the risk of causing offense to some), Haskell build only cares about Haskell source and Ruby build only cares about Ruby source. Almost by definition D has to live in a mixed C, C++, Fortran, D universe, so this has to be an issue from the outset.

I agree with the premise that package management must be a core part of the build management, but I disagree that Gradle, SCons, and Waf cannot handle this.  They cannot handle this today true, but in the same way that there is currently no system that does it properly as yet.  So I would suggest that package management is currently a "green field" problem that can be handled by a D-specific tool or one of Gradle, SCons or Waf.

-- 
Russel. ============================================================================= Dr Russel Winder      t: +44 20 7585 2200   voip: sip:russel.winder@ekiga.net 41 Buckmaster Road    m: +44 7770 465 077   xmpp: russel@russel.org.uk London SW11 1EN, UK   w: www.russel.org.uk  skype: russel_winder


July 13, 2011
On Wednesday 13 July 2011 06:53:28 Russel Winder wrote:
> On Tue, 2011-07-12 at 22:28 -0700, Jonathan M Davis wrote: [ . . . ]
> 
> > From previous discussions, it seems that one of the primary reasons for having a D build tool in many people's minds is to also handle package management of D libraries (like Haskell's cabal or rubygems for ruby). And as great as cmaked, scons, gradle, waf, and other such tools may be, they don't do that.
> 
> Go is currently using Make for this -- they have a structured Makefile hierarchy that handles most compilation and linking in the context of a rigidly enforced filestore structure, and goinstall for bringing in packages from outside into the filestore hierarchy.  It is a bit primitive at the minute, but is being worked on and rapidly improved.
> 
> Go actually has a plethora of build tools, include a couple of SCons-based ones, most of which are falling by the wayside.  I think the effort expended doing this has not been a waste as information is being generated that is adding to the pool.  I think they will replace the Makefile system shortly with one of the tools, possibly GoBuild, but I can't remember exactly.
> 
> Haskell and Ruby both have inward looking approaches -- to put it bluntly (at the risk of causing offense to some), Haskell build only cares about Haskell source and Ruby build only cares about Ruby source. Almost by definition D has to live in a mixed C, C++, Fortran, D universe, so this has to be an issue from the outset.
> 
> I agree with the premise that package management must be a core part of the build management, but I disagree that Gradle, SCons, and Waf cannot handle this.  They cannot handle this today true, but in the same way that there is currently no system that does it properly as yet.  So I would suggest that package management is currently a "green field" problem that can be handled by a D-specific tool or one of Gradle, SCons or Waf.

Feel free to propose whatever you'd like for a D build system (be it something entirely new or an improvement of an existing build tool). I'm just pointing out that one of the primary reasons that a number of the people on this list want a build tool is to handle package management of D libraries, so whatever solution you propose should incorporate that or a lot of people aren't going to be all that interested in it or will at least see it as inferior to another solution which does incorporate package management.

- Jonathan M Davis
July 13, 2011
From: "Russel Winder" <russel@russel.org.uk>
>
>I agree with the premise that package management must be a core part of the build management,

If I understand what you're saying correctly, then I strongly disagree. A package management tool, by necessity, must be a common agreed-upon standard (otherwise it won't serve it's purpose). But if that package management is tied into build management then we're *also* forcing a single standard build system on everyone. So then instead of being able to choose your build system, everyone now has to come to an agreement on the same build system, which #1: Is unlikely to ever happen, and #2: Serves no useful purpose.


July 13, 2011
"Russel Winder" <russel@russel.org.uk> wrote in message news:mailman.1585.1310533819.14074.digitalmars-d@puremagic.com...
>On Tue, 2011-07-12 at 21:02 -0400, Nick Sabalausky wrote:
>[ . . . ]
>> Before I go implemeting everything, I'd like to get input on it. Is it something that could be a major D tool?
>[ . . . ]
>
>Given the nature of the debate, I will add to the mix that SCons, a pre-existing -- and indeed working :-) -- system, has a D tool and can compile and link D code.  What it needs is some love to bring it up to the level of the C and C++ support.
>
>SCons has a built-in D tool which needs work, but rather than fork SCons to work on the tool I have created a separate tool that can be used with any SCons installation.  See https://bitbucket.org/russel/scons_dmd_new. Prior to any SCons release a patch between this version and the one in the SCons core will be made and applied.
>
[...Waf, Rake, Ant, Maven, etc...]

First of all, just to be clear, Drake isn't D-specific. Like Rake, it's designed to work with anything you can programmatically invoke (and doesn't require any special module to support a particular language or tool). After having worked on projects that, by necessity, involved multiple languages, I find that quality essential in a build system.

Regarding SCons, Waf, Rake, etc, the point of Drake is that 1: it doesn't force D projects to be reliant on Python, Ruby, etc. And 2: It allows D users to use D for their buildscript instead of needing to switch to a different (and less preferable IMO) langauge. (And I *hate* language-soup projects - projects that use a totally different language for every little damn thing.)

The *make tools (whether they use makefiles directly or indirectly) aren't even worth consideration. Makefiles need to just die, period.

Regarding this topic's frequent ressurection, I see that as a direct result of it all being *merely* talk, with very little concrete progress. We don't even *have* a proper package management system (there's DSSS, but it's dead, bitrotted, and I don't think it handles dependencies or multiple versions of the same package). So naturally that topic's going to keep coming up until we have something. And as for build systems, everything either lacks multiple-target/configuration management (rdmd), or creates a dependency on (and a need to write in) some other langauge (SCons, Rake, etc). So naturally there's also going to be people that feel unsatisfied with the build systems until something fills that role. I'm trying to get past "talk", so I've started some actual code to address what I see as the current issues for build systems.

Of course, it doesn't help that everyone seems to want something completely different from what everyone else wants. ;)


July 13, 2011
On Wednesday 13 July 2011 03:01:04 Nick Sabalausky wrote:
> Regarding this topic's frequent ressurection, I see that as a direct result of it all being *merely* talk, with very little concrete progress. We don't even *have* a proper package management system (there's DSSS, but it's dead, bitrotted, and I don't think it handles dependencies or multiple versions of the same package). So naturally that topic's going to keep coming up until we have something.

I belive that Jacob Carlborg is working on a solution. So,  there _is_ work being done on it. By no means does that mean that you shouldn't work on a solution, but it's not the case that no one has been working on the problem - though it is true that it's been being brought up for far longer than it's been being worked on.

- Jonathan M Davis
July 13, 2011
Nick Sabalausky Wrote:

> From: "Russel Winder" <russel@russel.org.uk>
> >
> >I agree with the premise that package management must be a core part of the build management,
> 
> If I understand what you're saying correctly, then I strongly disagree. A package management tool, by necessity, must be a common agreed-upon standard (otherwise it won't serve it's purpose). But if that package management is tied into build management then we're *also* forcing a single standard build system on everyone. So then instead of being able to choose your build system, everyone now has to come to an agreement on the same build system, which #1: Is unlikely to ever happen, and #2: Serves no useful purpose.

A build tool should be able to invoke a package manager and package manager should be able to invoke a build tool, this doesn't mean a tie, it means a working out of the box solution. This solution is meant to just work. If people don't want solution that just works, they can go with something else of course. For example I prefer makefiles.
« First   ‹ Prev
1 2 3 4 5 6 7 8 9 10 11
Top | Discussion index | About this forum | D home