View mode: basic / threaded / horizontal-split · Log in · Help
February 17, 2013
Re: The DUB package manager
On Sat, 16 Feb 2013 23:58:41 +0100
"Rob T" <alanb@ucora.com> wrote:
> 
> Why use json (which is a subset of javascript), or ruby, or 
> python, etc? Is there something fundamentally wrong with D that 
> makes it unsuitable for this role?
> 

I largely agree, except:

1. For simple projects with trivial build system requirements, D is
overkill compared to a purely data-only language.

2. If it's in D, it'll tend to end up tied to a particular range of
compiler versions (at least for now). Granted, the same will be true of
whatever D-based project it's building, but a script written in a more
stable language (as data-only languages generally tend to be) could
conceivably be able to actually detect the installed compiler and react
accordingly.

Of course, #2 could be easily mitigated if something like DVM were
totally standard and then (at least for Linux, maybe not Windows) your
D-based buildscript started with something like:
#!dmd-2.058
February 17, 2013
Re: The DUB package manager
On Saturday, 16 February 2013 at 22:58:42 UTC, Rob T wrote:
> I'm having good success using D itself as the build tool 
> language, and I'm at the point now where I'm getting much 
> better results than what I was getting out of using external 
> build tools, so for me there's no looking back.
>
> I run rdmd on a .d "script" that I wrote that's setup to do 
> exactly what I want.
>
> What is missing from D is a good library of functions that are 
> generally useful for writing build scripts. Phobos already 
> supplies a great deal of what is needed that can be used to 
> construct what is missing.
>
> The benefit of using D is that I do not have to install and 
> learn a secondary language or conform to a specific build 
> format, or follow any weird restrictions. What I want to do is 
> build my D code, not learn something new and perform acrobatics 
> to get the job done. I can even use the same D process to build 
> C/C++ code if I wanted to expand on it.
>
> What I'm saying here is that I see no reason to use a language 
> other than D itself as the build tool. What D can use is an 
> addition to Phobos that supplies the necessary generalized 
> functions that all build tools should supply, and I don't think 
> there's all that much that's missing.
>
> For a package manager, some standards may be required, but it 
> too can be done completely with D.
>
> Why use json (which is a subset of javascript), or ruby, or 
> python, etc? Is there something fundamentally wrong with D that 
> makes it unsuitable for this role?
>
> --rt

Indeed, it does sound like a sweet idea to provide a complete 
library of D functions to help building a project and use D 
itself as the build tool.

Maybe dub could offer the possibility to call D as an integrated 
scripting language  ?
February 17, 2013
Re: The DUB package manager
On Saturday, 16 February 2013 at 22:49:27 UTC, Nick Sabalausky 
wrote:

> I like a lot of what you've said, but my concern about this 
> part is,
> does it support things like:
>
> - Dependencies between custom build steps (so that custom build 
> steps
> aren't needlessly re-run when they're not out-of-date, and can
> possibly be parallelized)
>

One problem with parallelizing builds right now is, the risk is 
high they fail with out of memory errors before they finish. 
Unless it's possible to pararellize them on different build 
boxes, or one owns a build machine with 64 Gb of RAM.
February 17, 2013
Re: The DUB package manager
On Sun, 17 Feb 2013 03:25:04 +0100
"SomeDude" <lovelydear@mailmetrash.com> wrote:

> On Saturday, 16 February 2013 at 22:49:27 UTC, Nick Sabalausky 
> wrote:
> 
> > I like a lot of what you've said, but my concern about this 
> > part is,
> > does it support things like:
> >
> > - Dependencies between custom build steps (so that custom build 
> > steps
> > aren't needlessly re-run when they're not out-of-date, and can
> > possibly be parallelized)
> >
> 
> One problem with parallelizing builds right now is, the risk is 
> high they fail with out of memory errors before they finish. 
> Unless it's possible to pararellize them on different build 
> boxes, or one owns a build machine with 64 Gb of RAM.

For compiling D sources yea, but not necessarily for other custom build
steps.
February 17, 2013
Re: The DUB package manager
> You might want to list all the dependencies needed for dub or
> distribute them in a zip.
> 

They are in the .zip now and I listed the dependencies on the download
page. Sorry, the distribution stuff is still very much ad-hoc ATM. I'll
make some installers once the build process is automated.

> Ah I didn't spot the download link http://registry.vibed.org/download
> 
> I guess this could be made more visible by adding a link to the
> download page from the github repository, and maybe putting the { *
> Using DUB * Download * Publishing packages * Helping developme }
> section at the top instead of the bottom.

There now is a link on the github page + a note for non-Windows that
libevent/libssl are needed. I also added a short sentence how to build
by hand. The dependencies will also likely change to just libcurl at
some point with a make file or something to make bootstrapping as simple
as possible.

I also agree regarding the navigation links. The page layout will have
to be extended a bit to handle more packages anyway (pages, search
function, possibly categories) so I'll keep it as a TODO item for a few
days and then do both at once.
February 17, 2013
Re: The DUB package manager
Am 16.02.2013 23:49, schrieb Nick Sabalausky:
> On Sat, 16 Feb 2013 22:21:55 +0100
> Sönke Ludwig <sludwig@outerproduct.org> wrote:
>>
>> My idea for the things you mentioned (swig, c, etc.) was to have a set
>> of hooks that can be used to run external tools (invoked before
>> build/project file generation, before build or after build). That
>> together with your proposed interface should provide all the necessary
>> flexibility while putting an emphasis on a standard way to describe
>> the build process.
> 
> I like a lot of what you've said, but my concern about this part is,
> does it support things like:
> 
> - Dependencies between custom build steps (so that custom build steps
> aren't needlessly re-run when they're not out-of-date, and can
> possibly be parallelized)
> 
> - Multiple custom build targets
> 
> - Multiple custom build configurations
> 
> I think those are essential for a real general-purpose build tool.
> 

I'd hope that all of this (except multiple custom build commands) can be
pushed off to the external build tool. So in essence those build steps
would be just dump command lists that are always executed.

It would basically need to be that way, if the target builder doesn't
support something like this anyway (e.g. the pre/post build command in a
VisualD or Mono-D project). If this should turn out not to be enough, it
could always be extended, but I think most of it should be easy to
handle using "make" or something similar.
February 17, 2013
Re: The DUB package manager
Am 16.02.2013 23:58, schrieb Rob T:
> 
> I'm having good success using D itself as the build tool language, and
> I'm at the point now where I'm getting much better results than what I
> was getting out of using external build tools, so for me there's no
> looking back.
> 
> I run rdmd on a .d "script" that I wrote that's setup to do exactly what
> I want.

I did that, too, for automated and Linux builds. But this means that if
you also want to work with an IDE for debugging and such things, you
have to keep additional project files in sync. The DUB approach just
combines the two (+ the package management) and also has less overhead
than doing it with D (main function, imports, possibly distributing
helper modules etc.).

> 
> What is missing from D is a good library of functions that are generally
> useful for writing build scripts. Phobos already supplies a great deal
> of what is needed that can be used to construct what is missing.
> 
> The benefit of using D is that I do not have to install and learn a
> secondary language or conform to a specific build format, or follow any
> weird restrictions. What I want to do is build my D code, not learn
> something new and perform acrobatics to get the job done. I can even use
> the same D process to build C/C++ code if I wanted to expand on it.
> 

But you will have to learn the API of the build script helpers, too. I'm
not sure if this is actually less to learn than the JSON alternative.

> What I'm saying here is that I see no reason to use a language other
> than D itself as the build tool. What D can use is an addition to Phobos
> that supplies the necessary generalized functions that all build tools
> should supply, and I don't think there's all that much that's missing.
> 
> For a package manager, some standards may be required, but it too can be
> done completely with D.

Well, DUB is fully written in D, so the question is less D or not, but
more if an additional intermediate layer for the package description
makes sense or not.

> 
> Why use json (which is a subset of javascript), or ruby, or python, etc?
> Is there something fundamentally wrong with D that makes it unsuitable
> for this role?
> 

I see the following points for the decision for data/JSON vs. D:

- Meta information needs to be available to the package registry and
for managing dependencies in general. Executing a script/program would
imply a big performance risk, and worse, is a high security risk in case
of the registry. So a data-driven approach is needed at least for the
meta data anyway.

- JSON is a nice standard format with generally low risk of errors. I
agree that it is not the prettiest language in the world (although IMHO
much better than something XML based). Generally the same recent
discussion on the beta list for the dmd.conf format applies here, I'm
not sure what is the best approach, but JSON at least is not too bad.

To me the most interesting open question is this: Do we actually gain
from programmatic support for the build description, or does it suffice
to have a good purely descriptive system? If the former should be true
for more than 1% of the cases, that would definitely be a good argument
against pure data.
February 17, 2013
Re: The DUB package manager
> Another issue: I understand why you are using json but it is
> not the best suited format IMHO. D put some restriction on
> module names, thus the format can be simplified.
> 
> (...)
> 
> Exporting to other formats will probably needed anyway.
> 
> Peter

I agree about the non-optimal syntax, but it comes down to nicer syntax
(where JSON at least is not _too_ bad) vs. standard format (familiarity,
data exchange, proven design and implementation). And while data
exchange could worked around with export/import functions, considering
that these files are usually just 4 to 20 lines long, I personally would
put less weight on the syntax issue than on the others.

BTW, I think YAML as a superset of JSON is also a good contender with
nice syntax features, but also much more complex.
February 17, 2013
Re: The DUB package manager
On Sunday, February 17, 2013 09:12:00 Sönke Ludwig wrote:
> BTW, I think YAML as a superset of JSON is also a good contender with
> nice syntax features, but also much more complex.

It's also whitespace-sensitive, which is downright evil IMHO. I'd take JSON 
over YAML any day.

- Jonathan M Davis
February 17, 2013
Re: The DUB package manager
On Sat, 2013-02-16 at 21:02 +0100, Johannes Pfau wrote:
[…]

> The benefit of splitting them in this way: You're forced to provide
> interfaces for the build tool to communicate with the package manager
> and every other build tool can use those interfaces as well. This way
> there are no second class build systems.

Comment from a build veteran last Tuesday: "I will not use any build
system that cannot be used as a library."

This is the modern way.

-- 
Russel.
=============================================================================
Dr Russel Winder      t: +44 20 7585 2200   voip: sip:russel.winder@ekiga.net
41 Buckmaster Road    m: +44 7770 465 077   xmpp: russel@winder.org.uk
London SW11 1EN, UK   w: www.russel.org.uk  skype: russel_winder
1 2 3 4 5 6 7
Top | Discussion index | About this forum | D home