March 01, 2023
On Tuesday, 28 February 2023 at 16:32:42 UTC, Adam D Ruppe wrote:
> On Tuesday, 28 February 2023 at 16:10:19 UTC, Steven Schveighoffer wrote:
>> module buildfile;
>
> This isn't what I'm talking about at all.
>
> I don't want to run D to build, I want to *introspect* D for build info. You can read module names, dependencies (including platform-specific and configuration-specific ones) right out of D code. Author, description, license, etc., have standard ddoc sections in D code.
>
> Other things dub.json define like flags could easily be a UDA on the module declaration and be pulled out pretty easily by compilers or even relatively basic parsers.
>
> Even subconfigurations can... kinda be pulled out of D code by looking for version blocks though that's probably another thing I'd put on a UDA.

Yeah, this sounds like magic if it works and has a big "wow" effect potential for any new D user.
March 01, 2023

On Tuesday, 28 February 2023 at 17:27:24 UTC, CM wrote:

>

On Tuesday, 28 February 2023 at 14:29:28 UTC, Mathias LANG wrote:

>

[...]

Is obscurity SDL's biggest downside? When I was a D and DUB initiate, I had no issues acclimating to its usage, especially compared to SBT's DSL. Instead, the source of the friction was due to DUB's documentation, and there's already an effort to improve this.

The syntax will always be an outlier. Someone building a project nowadays need not only a language and a build system, but a wealth of tools around it. CI, editor, deployment configs... Many of which have adopted YAML, none of which use SDL.

March 01, 2023

On Wednesday, 1 March 2023 at 07:41:02 UTC, tastyminerals wrote:

>

It is safer to just go with whatever is used and accepted as a reliable format for configs. I like the idea to use plain D tbo.

JSON lacks comments, but it's fine for minimal default projects. We try to use TOML everywhere in a company, although I cannot say how well it works as a config for building projects. YAML is the most risky, since it has a high chance to alienate a sizeable amount of potential users.

Why would YAML be the most risky choice ? For me it's quite the opposite: It's a well known and accepted format, AND the implementation is already there. So I clearly don't see how it would alienate users, except if those users are also alienated by Github Actions, Circle CI, Kubernetes, Ansible, Docker...

March 01, 2023

On Tuesday, 28 February 2023 at 15:35:58 UTC, Steven Schveighoffer wrote:

>

Json5. It's json but with comments and allows key names without quotes.

https://json5.org/

I didn't knew it supported quoteless keys, that's pretty nice!
If it's a superset of JSON, we could potentially do it in place too.

>

I have to say, I hate YAML. With a passion. The "backwards compatibility with JSON" is pretty must just bolted on. And having multiple ways to write configurations might be nice for people writing configs, it's not nice for people reading them. Also, significant whitespace = bad.

I think the significant whitespace boat has sailed. I don't like it anymore than you do, but nowadays, editors align everything, and if they don't, the linter will complain, or the reviewer will. As much as I dislike it, I think significant whitespace just hasn't proved to be the barrier it was envisioned to be, as demonstrated by Python and YAML.

>

I would be OK with TOML. But then we have 3 formats. Changing to a json-compatible format means we still only have to maintain 2 parsers.

Note that going with YAML allows us to deprecate the SDL parser. Going with JSON5 includes writing a new parser, and a library to do what Configy currently does, on top.

March 01, 2023

On Tuesday, 28 February 2023 at 14:29:28 UTC, Mathias LANG wrote:

>

Currently, Dub uses two formats: JSON and SDL.
...
Here are the options that have been floated around:

  1. Use SDL everywhere;
  2. Add support for YAML;
  3. Add support for another format (E.g. TOML);

I would like to go with YAML, not because it is the greatest format on earth, but because SDLang is not well known (it means newcomer have to learn yet another format, and while not a big deal, it adds friction), while YAML is everywhere. In the thread were SDL was discussed, people even thought we invented it!

what about this:
https://news.ycombinator.com/item?id=34351503
do we risk having a future effort and discussion again?

I don't like SDL probably because I don't know it and have never seen it used besides D. With JSON esp. JSON5 I can live, besides its massively used and well understood.

I do not see a real benefit in yet another file format for configuration.

March 01, 2023

On Wednesday, 1 March 2023 at 09:53:08 UTC, Arjan wrote:

>

what about this:
https://news.ycombinator.com/item?id=34351503
do we risk having a future effort and discussion again?

I don't like SDL probably because I don't know it and have never seen it used besides D. With JSON esp. JSON5 I can live, besides its massively used and well understood.

I do not see a real benefit in yet another file format for configuration.

90% of this can be solved by simply quoting strings in value.
Also, once again, the plan is not "Yet another format", but "One format".

March 01, 2023

On Tuesday, 28 February 2023 at 16:32:42 UTC, Adam D Ruppe wrote:

>

On Tuesday, 28 February 2023 at 16:10:19 UTC, Steven Schveighoffer wrote:

>

module buildfile;

This isn't what I'm talking about at all.

I don't want to run D to build, I want to introspect D for build info. You can read module names, dependencies (including platform-specific and configuration-specific ones) right out of D code. Author, description, license, etc., have standard ddoc sections in D code.

Other things dub.json define like flags could easily be a UDA on the module declaration and be pulled out pretty easily by compilers or even relatively basic parsers.

Even subconfigurations can... kinda be pulled out of D code by looking for version blocks though that's probably another thing I'd put on a UDA.

On paper it sounds quite good, but I am not sure how well it works to derive the build instructions from the thing you are trying to build. Most of the time, when I build something, it fails. It's quite obvious why: Build something new, it will fail for a reason or another. Fix it, rebuild, and it either pass, or you discover another error. Things do succeed on the first try on some occasions, but the number of time they don't is much higher.

How do you ensure your build system is resilient in the presence of wrong modules ? How do you ensure that build system errors are not replaced by linker / compilation errors ?

March 01, 2023

On Wednesday, 1 March 2023 at 09:53:08 UTC, Arjan wrote:

>

I do not see a real benefit in yet another file format for configuration.

+1

And going SDL was called a "strategic mistake"

I'd be ok with removing one of SDL or JSON, but beyond that there is really nothing to win.
JSON5 idea sounds like the less worse.

My fav scenarii:

  1. SDL is removed, JSON comments reinstated
  2. JSON is removed
  3. JSON5
March 01, 2023
On Wednesday, 1 March 2023 at 07:45:26 UTC, tastyminerals wrote:
>
> Yeah, this sounds like magic if it works and has a big "wow" effect potential for any new D user.

For that to work you would still need version information for imports.

@dependency("~>2.0") import audioformats;

It also wouldn't work with configurations:


    mixin myTemplateThatDependUponAVersion!(stuff);


if myTemplateThatDependUponAVersion is declared in another dependency, that then depends upon a version, then the magic parser has no way to know that, or that this program would need a configuration, or that a configuration needs two different version.

Then you would need:

    @configuration(bubbleUp, "myConf") version("myVersionIdentifier")
    {
        // blah
    }


Now you need to parse all possible dependencies to know the configurations? The build description is still encoded in your program but distributed.
Not sure what to think of that.



March 01, 2023

On Tuesday, 28 February 2023 at 14:29:28 UTC, Mathias LANG wrote:

>

Currently, Dub uses two formats: JSON and SDL.

JSON was there from the beginning, I assume due to Dub's npm inspiration.
In fact, we still very much depend on JSON internally: until recently, JSON object were passed around in quite a few places. When we download a package, regardless of the format the package recipe is in, we store it as dub.json.

But JSON is a terrible format to write configurations in, given how verbose it is, and it lacking support for comments.
So around 2014, we added support for SDL (no, not this one, or this one, but this one, and it took me 10 minutes to find the link because Wikipedia doesn't even know about it).

At some point last year, I did some heavy refactoring on dub along with quite a few improvements. In the process of doing so, I took advantage of the fact that YAML is a subset of JSON to add a YAML parser, on which I have built a library to do automatic configuration parsing, because let's be honest, it's 2023 and we're using the language with the best compile-time reflection capabilities, so we really don't want to manually parse a standard file format to fill a struct.

This parser is used to parse all JSON files:

  • dub.json (but not dub.sdl);
  • dub.selections.json;
  • dub.settings.json;

Now, I think that having dub.settings.json being written in JSON doesn't make sense, and I would like to change this. In fact, I have been wanting this for more than 3 years, and the people that took part in the discussion broadly agreed.

Here are the options that have been floated around:

  1. Use SDL everywhere;
  2. Add support for YAML;
  3. Add support for another format (E.g. TOML);

I would like to go with YAML, not because it is the greatest format on earth, but because SDLang is not well known (it means newcomer have to learn yet another format, and while not a big deal, it adds friction), while YAML is everywhere. In the thread were SDL was discussed, people even thought we invented it!

In practice, this would mean adding support for the file format with a YAML extension, giving it priority over the old format, then after a number of releases (most likely 10, probably more for dub.json), deprecating the JSON / SDL equivalent.

Obviously such a change would not happen overnight, and would need broad support from the community. Opinions ?

I have several thoughts on this post and the replies to it.

At the time SDL was introduced, I didn't think it was a good idea. I'd still rather it hadn't happened, but the truth is that once one gets used to it, it's a lot nicer to write and read than JSON. But: it's obscure, and now we have both, and two different pages to look at for dub documentation. I don't think it's a great place to be.

Having said that, having 3 formats is even worse. I don't think we can talk about changing the format without having a plan for migrating every single dub.json/dub.sdl out there. I don't know how that'd work with issuing PRs to all of them, especially if the maintainers ignore them.

To all who doubt that using a turing-complete language is a bad idea: it's not. Any complicated project ends up needing it anyway and getting around the limitation by generating the declarative description or something like that. I've written enough CMake to know. I've worked around dub's limitations more times than I can count.

Using D sounds great, until one realises that reflecting can take a while, and unless there's a generation step followed by an actual simple build system, is too annoying to use. I might know what I'm talking about here: https://github.com/atilaneves/reggae/.
It's gotten to the point that I'm thinking of transitioning the build descriptions to Python at work to get around 2-3s of compile-time. The main reason I haven't done it yet is our builds are so complicated and dub takes so long to actually give me the information I need that it frequently dwarfs those seconds I'd save anyway.

I wish I had a conclusion but I don't.