On Tuesday, 28 February 2023 at 14:29:28 UTC, Mathias LANG wrote:
> Currently, Dub uses two formats: JSON and SDL.
JSON was there from the beginning, I assume due to Dub's npm inspiration.
In fact, we still very much depend on JSON internally: until recently, JSON object were passed around in quite a few places. When we download a package, regardless of the format the package recipe is in, we store it as dub.json
.
But JSON is a terrible format to write configurations in, given how verbose it is, and it lacking support for comments.
So around 2014, we added support for SDL (no, not this one, or this one, but this one, and it took me 10 minutes to find the link because Wikipedia doesn't even know about it).
At some point last year, I did some heavy refactoring on dub along with quite a few improvements. In the process of doing so, I took advantage of the fact that YAML is a subset of JSON to add a YAML parser, on which I have built a library to do automatic configuration parsing, because let's be honest, it's 2023 and we're using the language with the best compile-time reflection capabilities, so we really don't want to manually parse a standard file format to fill a struct.
This parser is used to parse all JSON files:
dub.json
(but not dub.sdl
);
dub.selections.json
;
dub.settings.json
;
Now, I think that having dub.settings.json
being written in JSON doesn't make sense, and I would like to change this. In fact, I have been wanting this for more than 3 years, and the people that took part in the discussion broadly agreed.
Here are the options that have been floated around:
- Use SDL everywhere;
- Add support for YAML;
- Add support for another format (E.g. TOML);
I would like to go with YAML, not because it is the greatest format on earth, but because SDLang is not well known (it means newcomer have to learn yet another format, and while not a big deal, it adds friction), while YAML is everywhere. In the thread were SDL was discussed, people even thought we invented it!
In practice, this would mean adding support for the file format with a YAML extension, giving it priority over the old format, then after a number of releases (most likely 10, probably more for dub.json
), deprecating the JSON / SDL equivalent.
Obviously such a change would not happen overnight, and would need broad support from the community. Opinions ?
I have several thoughts on this post and the replies to it.
At the time SDL was introduced, I didn't think it was a good idea. I'd still rather it hadn't happened, but the truth is that once one gets used to it, it's a lot nicer to write and read than JSON. But: it's obscure, and now we have both, and two different pages to look at for dub documentation. I don't think it's a great place to be.
Having said that, having 3 formats is even worse. I don't think we can talk about changing the format without having a plan for migrating every single dub.json/dub.sdl out there. I don't know how that'd work with issuing PRs to all of them, especially if the maintainers ignore them.
To all who doubt that using a turing-complete language is a bad idea: it's not. Any complicated project ends up needing it anyway and getting around the limitation by generating the declarative description or something like that. I've written enough CMake to know. I've worked around dub's limitations more times than I can count.
Using D sounds great, until one realises that reflecting can take a while, and unless there's a generation step followed by an actual simple build system, is too annoying to use. I might know what I'm talking about here: https://github.com/atilaneves/reggae/.
It's gotten to the point that I'm thinking of transitioning the build descriptions to Python at work to get around 2-3s of compile-time. The main reason I haven't done it yet is our builds are so complicated and dub takes so long to actually give me the information I need that it frequently dwarfs those seconds I'd save anyway.
I wish I had a conclusion but I don't.