February 17, 2013
On Sun, 2013-02-17 at 09:12 +0100, Sönke Ludwig wrote:

> I agree about the non-optimal syntax, but it comes down to nicer syntax (where JSON at least is not _too_ bad) vs. standard format (familiarity, data exchange, proven design and implementation). And while data exchange could worked around with export/import functions, considering that these files are usually just 4 to 20 lines long, I personally would put less weight on the syntax issue than on the others.
> 
> BTW, I think YAML as a superset of JSON is also a good contender with nice syntax features, but also much more complex.

Peter's point is more important than the above response indicates.

Using JSON (YAML, whatever) is a recipe for failure. The syntax is data exchange facing, whereas writing and maintaining a build script (even if it is only 4 lines long), is a human activity. Do not underestimate the reaction "why do I have to write this data exchange format?" The rapid demise of XML, Ant, Maven, etc. is testament to people awakening to the fact that they were forced to use manually a language designed for another purpose.

Where Gradle, and before it Gant, Buildr, etc., got it right was to use an internal DSL mentality to ask what is a written form that has affordance for the user that we can interpret in some way to deliver for computation the data needed. Groovy provides the infrastructure for Gradle, but people write Gradle scripts not Groovy. The build/packaging system does the hard work, not the user.

The lesson from SBT is that by judicious cleverness, a trivially simple internal DSL can be derived that does all the work using Scala. The only downside to SBT scripts is the extra newline needed everywhere. Surely D is better than Scala for this?

So if D is to contribute to build and package management, it should be looking to replace Make, CMake, SCons, Waf, Gradle. It should be able to build D systems yes, and manage D packages, but also look to build C, C ++, Fortran, and interlink all of them. Clearly starting small is good: a successful large system always evolves from a successful small system; designed large systems always fail. (cf. Gall, Systemantics, 1975)

If all the energy around DUB and Orbit leads to the situation where D cannot be used to create an internal DSL for describing build and package/artefact management, then D has failed.

-- 
Russel. ============================================================================= Dr Russel Winder      t: +44 20 7585 2200   voip: sip:russel.winder@ekiga.net 41 Buckmaster Road    m: +44 7770 465 077   xmpp: russel@winder.org.uk London SW11 1EN, UK   w: www.russel.org.uk  skype: russel_winder


February 17, 2013
Am 17.02.2013, 10:24 Uhr, schrieb Russel Winder:
> On Sun, 2013-02-17 at 09:12 +0100, Sönke Ludwig wrote:
>
>> I agree about the non-optimal syntax, but it comes down to nicer syntax
>> (where JSON at least is not _too_ bad) vs. standard format (familiarity,
>> data exchange, proven design and implementation). And while data
>> [...]
>
> Peter's point is more important than the above response indicates.

I think so too. And I don't believe that it will stay by a few lines
only because the demands will increase in the course of time. Think
about including C/C++ compiler, a project manager etc.

With a syntax as I suggested or something similar everything is mapped
direct onto an AA and is easily scanned by the build system or other
tools.

Regarding json: It is designed for data exchange between different
sources, not to be edited by people. It is too annoying and error-
prone to wrap each string into quotes. And people have to edit the
configuration file. Better to start in a well suited format.

Peter
February 17, 2013
Am 17.02.2013 11:21, schrieb Peter Sommerfeld:
> Am 17.02.2013, 10:24 Uhr, schrieb Russel Winder:
>> On Sun, 2013-02-17 at 09:12 +0100, Sönke Ludwig wrote:
>>
>>> I agree about the non-optimal syntax, but it comes down to nicer syntax
>>> (where JSON at least is not _too_ bad) vs. standard format (familiarity,
>>> data exchange, proven design and implementation). And while data
>>> [...]
>>
>> Peter's point is more important than the above response indicates.
> 
> I think so too. And I don't believe that it will stay by a few lines only because the demands will increase in the course of time. Think about including C/C++ compiler, a project manager etc.

If I were to vote for anything, that would be to never support C/C++ or any other foreign language builds. There are enough tools for that and I don't see enough value for the huge complexity that this route would add (just consider autotools or cmake).

Not sure what you mean by project manager, but right now, I see no reason why it shouldn't be possible to keep the package/build description very compact. If something gets complex anyway, maybe it should be broken up into multiple packages (e.g. one with the C/C++ stuff + bindings and one using the C++ stuff).

> 
> With a syntax as I suggested or something similar everything is mapped direct onto an AA and is easily scanned by the build system or other tools.
> 
> Regarding json: It is designed for data exchange between different sources, not to be edited by people. It is too annoying and error- prone to wrap each string into quotes. And people have to edit the configuration file. Better to start in a well suited format.

This is just wrong. JSON is just JavaScript, which is meant for human editing just as D or a custom DSL is. There definitely are nicer syntaxes, but this is putting it out of proportion, IMO. I'm surely not against using a different format, but all arguments must be weighted against each other here.
February 17, 2013
Am 17.02.2013 10:24, schrieb Russel Winder:
> On Sun, 2013-02-17 at 09:12 +0100, Sönke Ludwig wrote:
> 
>> I agree about the non-optimal syntax, but it comes down to nicer syntax (where JSON at least is not _too_ bad) vs. standard format (familiarity, data exchange, proven design and implementation). And while data exchange could worked around with export/import functions, considering that these files are usually just 4 to 20 lines long, I personally would put less weight on the syntax issue than on the others.
>>
>> BTW, I think YAML as a superset of JSON is also a good contender with nice syntax features, but also much more complex.
> 
> Peter's point is more important than the above response indicates.
> 
> Using JSON (YAML, whatever) is a recipe for failure. The syntax is data exchange facing, whereas writing and maintaining a build script (even if it is only 4 lines long), is a human activity. Do not underestimate the reaction "why do I have to write this data exchange format?" The rapid demise of XML, Ant, Maven, etc. is testament to people awakening to the fact that they were forced to use manually a language designed for another purpose.

I don't agree with this, as already said in my response to Peter. We are talking about rather small syntactic differences here (i.e. ',' vs ';' and omitting quotes) and XML adds a lot more overhead than the quotes around key names.

Ant and the like also have the awkward approach of not only using a verbose format such as XML, but they also try to express procedural build scripts in that language, which is insane. But my idea for dub is to completely avoid the need for procedural build steps apart from invoking an external tool (which should be needed only for a minority of D projects).

> 
> Where Gradle, and before it Gant, Buildr, etc., got it right was to use an internal DSL mentality to ask what is a written form that has affordance for the user that we can interpret in some way to deliver for computation the data needed. Groovy provides the infrastructure for Gradle, but people write Gradle scripts not Groovy. The build/packaging system does the hard work, not the user.
> 
> The lesson from SBT is that by judicious cleverness, a trivially simple internal DSL can be derived that does all the work using Scala. The only downside to SBT scripts is the extra newline needed everywhere. Surely D is better than Scala for this?

If by internal DSL, you mean compiled as D source code directly or indirectly, please also consider the server side issues (i.e. danger of DoS and hijacking the system).

> 
> So if D is to contribute to build and package management, it should be looking to replace Make, CMake, SCons, Waf, Gradle. It should be able to build D systems yes, and manage D packages, but also look to build C, C ++, Fortran, and interlink all of them. Clearly starting small is good: a successful large system always evolves from a successful small system; designed large systems always fail. (cf. Gall, Systemantics, 1975)

I think building foreign languages are better left to specialized tools. It should be possible to invoke those automatically, but adding their functionality directly to a D build system would blow up the complexity dramatically for an uncertain return of value (what would be the problem invoking make to build a C dependency?).

My question if we can get by without procedural build scripts is still open. If we could, it would give great benefits in simplicity of the system and its usage. This may require a change in conventions for some projects, but I think that can be worth it.

Maybe it would be good to have some concrete examples of complex build scenarios to better judge what is possible and what may be problematic.
February 17, 2013
Am 17.02.2013, 11:58 Uhr, schrieb Sönke Ludwig:
> Am 17.02.2013 11:21, schrieb Peter Sommerfeld:
> Not sure what you mean by project manager, but right now, I see no
> reason why it shouldn't be possible to keep the package/build
> description very compact.

If a good configuration format for D is established, it will (or
should) be used everywhere.

>> Regarding json: It is designed ...
> This is just wrong. JSON is just JavaScript, which is meant for human
> editing just as D or a custom DSL is.

OK, that is not important. The point is: Config files have to
be edited by humans and all the quotes are simply superfluous,
annoying and error-prone.

The appropriate file format has low-priority and may be discussed
when a general decision about using dub as build-tool has been
made. I am for it! So let us continue with more important issues
for now.

Peter
February 17, 2013
On Sun, 17 Feb 2013 09:24:08 +0000
Russel Winder <russel@winder.org.uk> wrote:
> 
> Using JSON (YAML, whatever) is a recipe for failure. The syntax is data exchange facing, whereas writing and maintaining a build script (even if it is only 4 lines long), is a human activity. Do not underestimate the reaction "why do I have to write this data exchange format?" The rapid demise of XML, Ant, Maven, etc. is testament to people awakening to the fact that they were forced to use manually a language designed for another purpose.
> 
[...]
> 
> If all the energy around DUB and Orbit leads to the situation where D cannot be used to create an internal DSL for describing build and package/artefact management, then D has failed.
> 

I half-agree with you. Where I disagree is with the idea that it's an issue of "embedded DSL (good) vs data language (bad)". I think the real issue is just simply having good clean syntax for what you need to accomplish.

Note that your examples of "rapid demise" (XML, Ant, Maven) are all XML, which is notoriously over-verbose. JSON has much the same problem, just to a lesser degree: It's a subset of human-intended JavaScript, yes, but it just happens to be an unfortunately verbose subset (although not to the extreme degree as XML).

Then on the flipside, we have the example of INI files: Definitely a purely data-language, definitely not an embedded DSL, and yet that's never been a hindrance for it: it's been a lasting success for many things. And the only time anyone complains about it is when more power is needed. (And no, I'm not suggesting DUB or Orbit use INI files. Again, specifically because more power is needed here.)

So I agree JSON/YAML carries an unfortunate risk of turning people off - but because of excess syntax, not because it's a data language or because it's not embedded in D.

I think SDL (Simple Declarative Language) hits the sweet spot: http://sdl.ikayzo.org/display/SDL/Language+Guide

FWIW, I just resumed working on a D parser for SDL. EVen if SDL doesn't get used for DUB or Orbit or dmd.conf, I think it'll still be a great thing to have available. It's such a simple grammar, I don't think it'll take long to reach a usable point.

February 17, 2013
Am 17.02.2013 12:47, schrieb Nick Sabalausky:
> I think SDL (Simple Declarative Language) hits the sweet spot: http://sdl.ikayzo.org/display/SDL/Language+Guide
> 
> FWIW, I just resumed working on a D parser for SDL. EVen if SDL doesn't get used for DUB or Orbit or dmd.conf, I think it'll still be a great thing to have available. It's such a simple grammar, I don't think it'll take long to reach a usable point.
> 

I have to say that it looks almost exactly like the DSLs I've been using for all of my internal stuff, so I really like it (although I've never heard of it before). It would require a bit of a rework because it doesn't map to JSON 1-to-1, though.
February 17, 2013
On 2013-02-16 20:35, Russel Winder wrote:

> Thus I suggest that it is not that the build tool is embedded in the
> package manager but that package and dependency management is part of
> the build system.

In general yes, but I think that there should be two separated tools and handle what they're designed to handled. On top of that they should have great integration with each other. I'm not talking about processes invoking each other. I'm talking about both tools are built as libraries and can be easily integrated with each other.

Say we have build tool A and package manager B. I don't want to force anyone using A just to use B, or the opposite.

-- 
/Jacob Carlborg
February 17, 2013
On 2013-02-16 23:58, Rob T wrote:

> Why use json (which is a subset of javascript), or ruby, or python, etc?
> Is there something fundamentally wrong with D that makes it unsuitable
> for this role?

You generally will get a more verbose syntax using D. It's not really made to execute statements on the top level.

-- 
/Jacob Carlborg
February 17, 2013
On 2013-02-17 02:49, Nick Sabalausky wrote:

> I largely agree, except:
>
> 1. For simple projects with trivial build system requirements, D is
> overkill compared to a purely data-only language.

It might be but the syntax is no that different:

Yaml:

flags: "-l-L. -release"
foo: "bar"

Json:

{
  "flags": "-l-L. -release"
  "foo": "bar"
}

D:

flags = "-l-L. -release";
foo = "bar";

In this example D has less syntax than Json.

-- 
/Jacob Carlborg