View mode: basic / threaded / horizontal-split · Log in · Help
February 17, 2013
Re: The DUB package manager
On Sun, 2013-02-17 at 09:12 +0100, Sönke Ludwig wrote:

> I agree about the non-optimal syntax, but it comes down to nicer syntax
> (where JSON at least is not _too_ bad) vs. standard format (familiarity,
> data exchange, proven design and implementation). And while data
> exchange could worked around with export/import functions, considering
> that these files are usually just 4 to 20 lines long, I personally would
> put less weight on the syntax issue than on the others.
> 
> BTW, I think YAML as a superset of JSON is also a good contender with
> nice syntax features, but also much more complex.

Peter's point is more important than the above response indicates.

Using JSON (YAML, whatever) is a recipe for failure. The syntax is data
exchange facing, whereas writing and maintaining a build script (even if
it is only 4 lines long), is a human activity. Do not underestimate the
reaction "why do I have to write this data exchange format?" The rapid
demise of XML, Ant, Maven, etc. is testament to people awakening to the
fact that they were forced to use manually a language designed for
another purpose.

Where Gradle, and before it Gant, Buildr, etc., got it right was to use
an internal DSL mentality to ask what is a written form that has
affordance for the user that we can interpret in some way to deliver for
computation the data needed. Groovy provides the infrastructure for
Gradle, but people write Gradle scripts not Groovy. The build/packaging
system does the hard work, not the user.

The lesson from SBT is that by judicious cleverness, a trivially simple
internal DSL can be derived that does all the work using Scala. The only
downside to SBT scripts is the extra newline needed everywhere. Surely D
is better than Scala for this?

So if D is to contribute to build and package management, it should be
looking to replace Make, CMake, SCons, Waf, Gradle. It should be able to
build D systems yes, and manage D packages, but also look to build C, C
++, Fortran, and interlink all of them. Clearly starting small is good:
a successful large system always evolves from a successful small system;
designed large systems always fail. (cf. Gall, Systemantics, 1975)

If all the energy around DUB and Orbit leads to the situation where D
cannot be used to create an internal DSL for describing build and
package/artefact management, then D has failed.

-- 
Russel.
=============================================================================
Dr Russel Winder      t: +44 20 7585 2200   voip: sip:russel.winder@ekiga.net
41 Buckmaster Road    m: +44 7770 465 077   xmpp: russel@winder.org.uk
London SW11 1EN, UK   w: www.russel.org.uk  skype: russel_winder
February 17, 2013
Re: The DUB package manager
Am 17.02.2013, 10:24 Uhr, schrieb Russel Winder:
> On Sun, 2013-02-17 at 09:12 +0100, Sönke Ludwig wrote:
>
>> I agree about the non-optimal syntax, but it comes down to nicer syntax
>> (where JSON at least is not _too_ bad) vs. standard format (familiarity,
>> data exchange, proven design and implementation). And while data
>> [...]
>
> Peter's point is more important than the above response indicates.

I think so too. And I don't believe that it will stay by a few lines
only because the demands will increase in the course of time. Think
about including C/C++ compiler, a project manager etc.

With a syntax as I suggested or something similar everything is mapped
direct onto an AA and is easily scanned by the build system or other
tools.

Regarding json: It is designed for data exchange between different
sources, not to be edited by people. It is too annoying and error-
prone to wrap each string into quotes. And people have to edit the
configuration file. Better to start in a well suited format.

Peter
February 17, 2013
Re: The DUB package manager
Am 17.02.2013 11:21, schrieb Peter Sommerfeld:
> Am 17.02.2013, 10:24 Uhr, schrieb Russel Winder:
>> On Sun, 2013-02-17 at 09:12 +0100, Sönke Ludwig wrote:
>>
>>> I agree about the non-optimal syntax, but it comes down to nicer syntax
>>> (where JSON at least is not _too_ bad) vs. standard format (familiarity,
>>> data exchange, proven design and implementation). And while data
>>> [...]
>>
>> Peter's point is more important than the above response indicates.
> 
> I think so too. And I don't believe that it will stay by a few lines
> only because the demands will increase in the course of time. Think
> about including C/C++ compiler, a project manager etc.

If I were to vote for anything, that would be to never support C/C++ or
any other foreign language builds. There are enough tools for that and I
don't see enough value for the huge complexity that this route would add
(just consider autotools or cmake).

Not sure what you mean by project manager, but right now, I see no
reason why it shouldn't be possible to keep the package/build
description very compact. If something gets complex anyway, maybe it
should be broken up into multiple packages (e.g. one with the C/C++
stuff + bindings and one using the C++ stuff).

> 
> With a syntax as I suggested or something similar everything is mapped
> direct onto an AA and is easily scanned by the build system or other
> tools.
> 
> Regarding json: It is designed for data exchange between different
> sources, not to be edited by people. It is too annoying and error-
> prone to wrap each string into quotes. And people have to edit the
> configuration file. Better to start in a well suited format.

This is just wrong. JSON is just JavaScript, which is meant for human
editing just as D or a custom DSL is. There definitely are nicer
syntaxes, but this is putting it out of proportion, IMO. I'm surely not
against using a different format, but all arguments must be weighted
against each other here.
February 17, 2013
Re: The DUB package manager
Am 17.02.2013 10:24, schrieb Russel Winder:
> On Sun, 2013-02-17 at 09:12 +0100, Sönke Ludwig wrote:
> 
>> I agree about the non-optimal syntax, but it comes down to nicer syntax
>> (where JSON at least is not _too_ bad) vs. standard format (familiarity,
>> data exchange, proven design and implementation). And while data
>> exchange could worked around with export/import functions, considering
>> that these files are usually just 4 to 20 lines long, I personally would
>> put less weight on the syntax issue than on the others.
>>
>> BTW, I think YAML as a superset of JSON is also a good contender with
>> nice syntax features, but also much more complex.
> 
> Peter's point is more important than the above response indicates.
> 
> Using JSON (YAML, whatever) is a recipe for failure. The syntax is data
> exchange facing, whereas writing and maintaining a build script (even if
> it is only 4 lines long), is a human activity. Do not underestimate the
> reaction "why do I have to write this data exchange format?" The rapid
> demise of XML, Ant, Maven, etc. is testament to people awakening to the
> fact that they were forced to use manually a language designed for
> another purpose.

I don't agree with this, as already said in my response to Peter. We are
talking about rather small syntactic differences here (i.e. ',' vs ';'
and omitting quotes) and XML adds a lot more overhead than the quotes
around key names.

Ant and the like also have the awkward approach of not only using a
verbose format such as XML, but they also try to express procedural
build scripts in that language, which is insane. But my idea for dub is
to completely avoid the need for procedural build steps apart from
invoking an external tool (which should be needed only for a minority of
D projects).

> 
> Where Gradle, and before it Gant, Buildr, etc., got it right was to use
> an internal DSL mentality to ask what is a written form that has
> affordance for the user that we can interpret in some way to deliver for
> computation the data needed. Groovy provides the infrastructure for
> Gradle, but people write Gradle scripts not Groovy. The build/packaging
> system does the hard work, not the user.
> 
> The lesson from SBT is that by judicious cleverness, a trivially simple
> internal DSL can be derived that does all the work using Scala. The only
> downside to SBT scripts is the extra newline needed everywhere. Surely D
> is better than Scala for this?

If by internal DSL, you mean compiled as D source code directly or
indirectly, please also consider the server side issues (i.e. danger of
DoS and hijacking the system).

> 
> So if D is to contribute to build and package management, it should be
> looking to replace Make, CMake, SCons, Waf, Gradle. It should be able to
> build D systems yes, and manage D packages, but also look to build C, C
> ++, Fortran, and interlink all of them. Clearly starting small is good:
> a successful large system always evolves from a successful small system;
> designed large systems always fail. (cf. Gall, Systemantics, 1975)

I think building foreign languages are better left to specialized tools.
It should be possible to invoke those automatically, but adding their
functionality directly to a D build system would blow up the complexity
dramatically for an uncertain return of value (what would be the problem
invoking make to build a C dependency?).

My question if we can get by without procedural build scripts is still
open. If we could, it would give great benefits in simplicity of the
system and its usage. This may require a change in conventions for some
projects, but I think that can be worth it.

Maybe it would be good to have some concrete examples of complex build
scenarios to better judge what is possible and what may be problematic.
February 17, 2013
Re: The DUB package manager
Am 17.02.2013, 11:58 Uhr, schrieb Sönke Ludwig:
> Am 17.02.2013 11:21, schrieb Peter Sommerfeld:
> Not sure what you mean by project manager, but right now, I see no
> reason why it shouldn't be possible to keep the package/build
> description very compact.

If a good configuration format for D is established, it will (or
should) be used everywhere.

>> Regarding json: It is designed ...
> This is just wrong. JSON is just JavaScript, which is meant for human
> editing just as D or a custom DSL is.

OK, that is not important. The point is: Config files have to
be edited by humans and all the quotes are simply superfluous,
annoying and error-prone.

The appropriate file format has low-priority and may be discussed
when a general decision about using dub as build-tool has been
made. I am for it! So let us continue with more important issues
for now.

Peter
February 17, 2013
Re: The DUB package manager
On Sun, 17 Feb 2013 09:24:08 +0000
Russel Winder <russel@winder.org.uk> wrote:
> 
> Using JSON (YAML, whatever) is a recipe for failure. The syntax is
> data exchange facing, whereas writing and maintaining a build script
> (even if it is only 4 lines long), is a human activity. Do not
> underestimate the reaction "why do I have to write this data exchange
> format?" The rapid demise of XML, Ant, Maven, etc. is testament to
> people awakening to the fact that they were forced to use manually a
> language designed for another purpose.
> 
[...]
> 
> If all the energy around DUB and Orbit leads to the situation where D
> cannot be used to create an internal DSL for describing build and
> package/artefact management, then D has failed.
> 

I half-agree with you. Where I disagree is with the idea that it's an
issue of "embedded DSL (good) vs data language (bad)". I think the
real issue is just simply having good clean syntax for what you need
to accomplish.

Note that your examples of "rapid demise" (XML, Ant, Maven) are all
XML, which is notoriously over-verbose. JSON has much the same problem,
just to a lesser degree: It's a subset of human-intended JavaScript,
yes, but it just happens to be an unfortunately verbose subset
(although not to the extreme degree as XML).

Then on the flipside, we have the example of INI files: Definitely a
purely data-language, definitely not an embedded DSL, and yet that's
never been a hindrance for it: it's been a lasting success for many
things. And the only time anyone complains about it is when more power
is needed. (And no, I'm not suggesting DUB or Orbit use INI files.
Again, specifically because more power is needed here.)

So I agree JSON/YAML carries an unfortunate risk of turning people off -
but because of excess syntax, not because it's a data language or
because it's not embedded in D.

I think SDL (Simple Declarative Language) hits the sweet spot:
http://sdl.ikayzo.org/display/SDL/Language+Guide

FWIW, I just resumed working on a D parser for SDL. EVen if SDL
doesn't get used for DUB or Orbit or dmd.conf, I think it'll still be a
great thing to have available. It's such a simple grammar, I don't
think it'll take long to reach a usable point.
February 17, 2013
Re: The DUB package manager
Am 17.02.2013 12:47, schrieb Nick Sabalausky:
> I think SDL (Simple Declarative Language) hits the sweet spot:
> http://sdl.ikayzo.org/display/SDL/Language+Guide
> 
> FWIW, I just resumed working on a D parser for SDL. EVen if SDL
> doesn't get used for DUB or Orbit or dmd.conf, I think it'll still be a
> great thing to have available. It's such a simple grammar, I don't
> think it'll take long to reach a usable point.
> 

I have to say that it looks almost exactly like the DSLs I've been using
for all of my internal stuff, so I really like it (although I've never
heard of it before). It would require a bit of a rework because it
doesn't map to JSON 1-to-1, though.
February 17, 2013
Re: The DUB package manager
On 2013-02-16 20:35, Russel Winder wrote:

> Thus I suggest that it is not that the build tool is embedded in the
> package manager but that package and dependency management is part of
> the build system.

In general yes, but I think that there should be two separated tools and 
handle what they're designed to handled. On top of that they should have 
great integration with each other. I'm not talking about processes 
invoking each other. I'm talking about both tools are built as libraries 
and can be easily integrated with each other.

Say we have build tool A and package manager B. I don't want to force 
anyone using A just to use B, or the opposite.

-- 
/Jacob Carlborg
February 17, 2013
Re: The DUB package manager
On 2013-02-16 23:58, Rob T wrote:

> Why use json (which is a subset of javascript), or ruby, or python, etc?
> Is there something fundamentally wrong with D that makes it unsuitable
> for this role?

You generally will get a more verbose syntax using D. It's not really 
made to execute statements on the top level.

-- 
/Jacob Carlborg
February 17, 2013
Re: The DUB package manager
On 2013-02-17 02:49, Nick Sabalausky wrote:

> I largely agree, except:
>
> 1. For simple projects with trivial build system requirements, D is
> overkill compared to a purely data-only language.

It might be but the syntax is no that different:

Yaml:

flags: "-l-L. -release"
foo: "bar"

Json:

{
  "flags": "-l-L. -release"
  "foo": "bar"
}

D:

flags = "-l-L. -release";
foo = "bar";

In this example D has less syntax than Json.

-- 
/Jacob Carlborg
1 2 3 4 5 6 7 8
Top | Discussion index | About this forum | D home