February 17, 2013
On Sat, 16 Feb 2013 23:58:41 +0100
"Rob T" <alanb@ucora.com> wrote:
> 
> Why use json (which is a subset of javascript), or ruby, or python, etc? Is there something fundamentally wrong with D that makes it unsuitable for this role?
> 

I largely agree, except:

1. For simple projects with trivial build system requirements, D is overkill compared to a purely data-only language.

2. If it's in D, it'll tend to end up tied to a particular range of compiler versions (at least for now). Granted, the same will be true of whatever D-based project it's building, but a script written in a more stable language (as data-only languages generally tend to be) could conceivably be able to actually detect the installed compiler and react accordingly.

Of course, #2 could be easily mitigated if something like DVM were
totally standard and then (at least for Linux, maybe not Windows) your
D-based buildscript started with something like:
#!dmd-2.058

February 17, 2013
On Saturday, 16 February 2013 at 22:58:42 UTC, Rob T wrote:
> I'm having good success using D itself as the build tool language, and I'm at the point now where I'm getting much better results than what I was getting out of using external build tools, so for me there's no looking back.
>
> I run rdmd on a .d "script" that I wrote that's setup to do exactly what I want.
>
> What is missing from D is a good library of functions that are generally useful for writing build scripts. Phobos already supplies a great deal of what is needed that can be used to construct what is missing.
>
> The benefit of using D is that I do not have to install and learn a secondary language or conform to a specific build format, or follow any weird restrictions. What I want to do is build my D code, not learn something new and perform acrobatics to get the job done. I can even use the same D process to build C/C++ code if I wanted to expand on it.
>
> What I'm saying here is that I see no reason to use a language other than D itself as the build tool. What D can use is an addition to Phobos that supplies the necessary generalized functions that all build tools should supply, and I don't think there's all that much that's missing.
>
> For a package manager, some standards may be required, but it too can be done completely with D.
>
> Why use json (which is a subset of javascript), or ruby, or python, etc? Is there something fundamentally wrong with D that makes it unsuitable for this role?
>
> --rt

Indeed, it does sound like a sweet idea to provide a complete library of D functions to help building a project and use D itself as the build tool.

Maybe dub could offer the possibility to call D as an integrated scripting language  ?
February 17, 2013
On Saturday, 16 February 2013 at 22:49:27 UTC, Nick Sabalausky wrote:

> I like a lot of what you've said, but my concern about this part is,
> does it support things like:
>
> - Dependencies between custom build steps (so that custom build steps
> aren't needlessly re-run when they're not out-of-date, and can
> possibly be parallelized)
>

One problem with parallelizing builds right now is, the risk is high they fail with out of memory errors before they finish. Unless it's possible to pararellize them on different build boxes, or one owns a build machine with 64 Gb of RAM.
February 17, 2013
On Sun, 17 Feb 2013 03:25:04 +0100
"SomeDude" <lovelydear@mailmetrash.com> wrote:

> On Saturday, 16 February 2013 at 22:49:27 UTC, Nick Sabalausky wrote:
> 
> > I like a lot of what you've said, but my concern about this
> > part is,
> > does it support things like:
> >
> > - Dependencies between custom build steps (so that custom build
> > steps
> > aren't needlessly re-run when they're not out-of-date, and can
> > possibly be parallelized)
> >
> 
> One problem with parallelizing builds right now is, the risk is high they fail with out of memory errors before they finish. Unless it's possible to pararellize them on different build boxes, or one owns a build machine with 64 Gb of RAM.

For compiling D sources yea, but not necessarily for other custom build steps.

February 17, 2013
> You might want to list all the dependencies needed for dub or distribute them in a zip.
> 

They are in the .zip now and I listed the dependencies on the download page. Sorry, the distribution stuff is still very much ad-hoc ATM. I'll make some installers once the build process is automated.

> Ah I didn't spot the download link http://registry.vibed.org/download
> 
> I guess this could be made more visible by adding a link to the download page from the github repository, and maybe putting the { * Using DUB * Download * Publishing packages * Helping developme } section at the top instead of the bottom.

There now is a link on the github page + a note for non-Windows that libevent/libssl are needed. I also added a short sentence how to build by hand. The dependencies will also likely change to just libcurl at some point with a make file or something to make bootstrapping as simple as possible.

I also agree regarding the navigation links. The page layout will have to be extended a bit to handle more packages anyway (pages, search function, possibly categories) so I'll keep it as a TODO item for a few days and then do both at once.
February 17, 2013
Am 16.02.2013 23:49, schrieb Nick Sabalausky:
> On Sat, 16 Feb 2013 22:21:55 +0100
> Sönke Ludwig <sludwig@outerproduct.org> wrote:
>>
>> My idea for the things you mentioned (swig, c, etc.) was to have a set of hooks that can be used to run external tools (invoked before build/project file generation, before build or after build). That together with your proposed interface should provide all the necessary flexibility while putting an emphasis on a standard way to describe the build process.
> 
> I like a lot of what you've said, but my concern about this part is, does it support things like:
> 
> - Dependencies between custom build steps (so that custom build steps aren't needlessly re-run when they're not out-of-date, and can possibly be parallelized)
> 
> - Multiple custom build targets
> 
> - Multiple custom build configurations
> 
> I think those are essential for a real general-purpose build tool.
> 

I'd hope that all of this (except multiple custom build commands) can be pushed off to the external build tool. So in essence those build steps would be just dump command lists that are always executed.

It would basically need to be that way, if the target builder doesn't support something like this anyway (e.g. the pre/post build command in a VisualD or Mono-D project). If this should turn out not to be enough, it could always be extended, but I think most of it should be easy to handle using "make" or something similar.
February 17, 2013
Am 16.02.2013 23:58, schrieb Rob T:
> 
> I'm having good success using D itself as the build tool language, and I'm at the point now where I'm getting much better results than what I was getting out of using external build tools, so for me there's no looking back.
> 
> I run rdmd on a .d "script" that I wrote that's setup to do exactly what I want.

I did that, too, for automated and Linux builds. But this means that if you also want to work with an IDE for debugging and such things, you have to keep additional project files in sync. The DUB approach just combines the two (+ the package management) and also has less overhead than doing it with D (main function, imports, possibly distributing helper modules etc.).

> 
> What is missing from D is a good library of functions that are generally useful for writing build scripts. Phobos already supplies a great deal of what is needed that can be used to construct what is missing.
> 
> The benefit of using D is that I do not have to install and learn a secondary language or conform to a specific build format, or follow any weird restrictions. What I want to do is build my D code, not learn something new and perform acrobatics to get the job done. I can even use the same D process to build C/C++ code if I wanted to expand on it.
> 

But you will have to learn the API of the build script helpers, too. I'm not sure if this is actually less to learn than the JSON alternative.

> What I'm saying here is that I see no reason to use a language other than D itself as the build tool. What D can use is an addition to Phobos that supplies the necessary generalized functions that all build tools should supply, and I don't think there's all that much that's missing.
> 
> For a package manager, some standards may be required, but it too can be done completely with D.

Well, DUB is fully written in D, so the question is less D or not, but more if an additional intermediate layer for the package description makes sense or not.

> 
> Why use json (which is a subset of javascript), or ruby, or python, etc? Is there something fundamentally wrong with D that makes it unsuitable for this role?
> 

I see the following points for the decision for data/JSON vs. D:

 - Meta information needs to be available to the package registry and
for managing dependencies in general. Executing a script/program would
imply a big performance risk, and worse, is a high security risk in case
of the registry. So a data-driven approach is needed at least for the
meta data anyway.

 - JSON is a nice standard format with generally low risk of errors. I
agree that it is not the prettiest language in the world (although IMHO
much better than something XML based). Generally the same recent
discussion on the beta list for the dmd.conf format applies here, I'm
not sure what is the best approach, but JSON at least is not too bad.

To me the most interesting open question is this: Do we actually gain from programmatic support for the build description, or does it suffice to have a good purely descriptive system? If the former should be true for more than 1% of the cases, that would definitely be a good argument against pure data.

February 17, 2013
> Another issue: I understand why you are using json but it is not the best suited format IMHO. D put some restriction on module names, thus the format can be simplified.
> 
> (...)
> 
> Exporting to other formats will probably needed anyway.
> 
> Peter

I agree about the non-optimal syntax, but it comes down to nicer syntax (where JSON at least is not _too_ bad) vs. standard format (familiarity, data exchange, proven design and implementation). And while data exchange could worked around with export/import functions, considering that these files are usually just 4 to 20 lines long, I personally would put less weight on the syntax issue than on the others.

BTW, I think YAML as a superset of JSON is also a good contender with nice syntax features, but also much more complex.
February 17, 2013
On Sunday, February 17, 2013 09:12:00 Sönke Ludwig wrote:
> BTW, I think YAML as a superset of JSON is also a good contender with nice syntax features, but also much more complex.

It's also whitespace-sensitive, which is downright evil IMHO. I'd take JSON over YAML any day.

- Jonathan M Davis
February 17, 2013
On Sat, 2013-02-16 at 21:02 +0100, Johannes Pfau wrote: […]

> The benefit of splitting them in this way: You're forced to provide interfaces for the build tool to communicate with the package manager and every other build tool can use those interfaces as well. This way there are no second class build systems.

Comment from a build veteran last Tuesday: "I will not use any build system that cannot be used as a library."

This is the modern way.

-- 
Russel. ============================================================================= Dr Russel Winder      t: +44 20 7585 2200   voip: sip:russel.winder@ekiga.net 41 Buckmaster Road    m: +44 7770 465 077   xmpp: russel@winder.org.uk London SW11 1EN, UK   w: www.russel.org.uk  skype: russel_winder