I'm considering adding JSON5 support to std.json and want to know how well this would be received.

JSON5 is pretty much just modern JavaScript's object literal format, allowing things like comments, trailing commas, and single-quoted strings.

I only plan to add support for parsing JSON5, not emitting it. So there should be no compatibility concerns with what std.json emits.

Since it's technically a breaking change (people can use std.json currently to validate that a document is valid JSON), I am inclined to make JSON5 be off by default and add an option to parse JSON5 rather than JSON1. Anyone have strong feelings about this?

On Monday, 30 November 2015 at 00:30:07 UTC, Chris Wright wrote:
> I'm considering adding JSON5 support to std.json and want to know how well this would be received.
>
> JSON5 is pretty much just modern JavaScript's object literal format, allowing things like comments, trailing commas, and single-quoted strings.
>
> I only plan to add support for parsing JSON5, not emitting it. So there should be no compatibility concerns with what std.json emits.
>
> Since it's technically a breaking change (people can use std.json currently to validate that a document is valid JSON), I am inclined to make JSON5 be off by default and add an option to parse JSON5 rather than JSON1. Anyone have strong feelings about this?

Having a JSON 5 parser makes some sense, but I don't think that it makes any sense to have one which can parse JSON 5 but not emit it. Either you're dealing with JSON 5 or you're not, and the historical approach of the internet to be lax in what you accept and strict what you emit has proven to be a horrible approach IMHO.

So, creating a new module specifically for a new version of JSON seems fine, or maybe it would make sense for one module to support multiple versions by specifying which you want to use (assuming the the APIs are pretty much the same across versions, which they probably aren't), but let's please not have something that mucks around with multiple versions of JSON at once.

Also, note that std.json will probably be replaced at some point here. There's at least one replacement in the works that's had some review in the newsgroup:

http://code.dlang.org/packages/std_data_json

So, updating the current std.json to support other versions of JSON probably doesn't make a lot of sense regardless of how we want to handle multiple versions.

- Jonathan M Davis

On Monday, 30 November 2015 at 00:30:07 UTC, Chris Wright wrote:
> I'm considering adding JSON5 support to std.json and want to know how well this would be received.

Considering this is something that has apparently existed for more than three years and this is the first time I'm hearing of it, I have a hard time justifying it's inclusion into Phobos. I have never once heard of any project using JSON5 and googling it brings up one negative blog post, the HN post which is full of negative comments, and every other result are language bindings.

I have to agree with all the negativity, it seems like a poor idea and calling it "JSON" is inappropriate. Also, std.json is slated for deprecation.

If you want a JSON5 D parser, I'd say go ahead and throw it up on code.dlang.org. What would be much more beneficial IMO is a YAML parser based on the API of http://code.dlang.org/packages/std_data_json

On Monday, 30 November 2015 at 03:40:14 UTC, Jack Stouffer wrote:
> What would be much more beneficial IMO is a YAML parser based on the API of http://code.dlang.org/packages/std_data_json

I don't know what you'd think of its API, but there is already a library on code.dlang.org which deals with yaml:

http://code.dlang.org/packages/dyaml

- Jonathan M Davis

On Monday, 30 November 2015 at 01:02:38 UTC, Jonathan M Davis wrote:
> On Monday, 30 November 2015 at 00:30:07 UTC, Chris Wright wrote:
>> I'm considering adding JSON5 support to std.json and want to know how well this would be received.
>>
>> JSON5 is pretty much just modern JavaScript's object literal format, allowing things like comments, trailing commas, and single-quoted strings.
>>
>> I only plan to add support for parsing JSON5, not emitting it. So there should be no compatibility concerns with what std.json emits.
>>
>> Since it's technically a breaking change (people can use std.json currently to validate that a document is valid JSON), I am inclined to make JSON5 be off by default and add an option to parse JSON5 rather than JSON1. Anyone have strong feelings about this?
>
> Having a JSON 5 parser makes some sense, but I don't think that it makes any sense to have one which can parse JSON 5 but not emit it. Either you're dealing with JSON 5 or you're not, and the historical approach of the internet to be lax in what you accept and strict what you emit has proven to be a horrible approach IMHO.

It's pointless to emit most of the JSON5 relaxations - I see no reason why the emitter should specifically add trailing commas, and I don't see how it can emit comments...

The only JSON5 feature that is not syntactic sugar is the special floating point values, and the current std.json halfway supports it if you use `JSONOptions.specialFloatLiterals` - http://dpaste.dzfl.pl/42bbe53e00f9. "NaN" is emitted the same as JSON5, but for infinity we emit "Infinite" while JSON5 specifies "Infinity".

On Monday, 30 November 2015 at 09:43:58 UTC, Idan Arye wrote:
> http://dpaste.dzfl.pl/42bbe53e00f9. "NaN" is emitted the same as JSON5, but for infinity we emit "Infinite" while JSON5 specifies "Infinity".

That sounds like a bug. ECMAScript uses the mapping Number("Infinity"). Number("Infinite") would yield NaN...

On Monday, 30 November 2015 at 09:43:58 UTC, Idan Arye wrote:
> It's pointless to emit most of the JSON5 relaxations - I see no reason why the emitter should specifically add trailing commas, and I don't see how it can emit comments...

You might want to implement a pretty-print with the library? Generate the default config file including helpful comments?

On Mon, 30 Nov 2015 03:40:12 +0000, Jack Stouffer wrote:

> On Monday, 30 November 2015 at 00:30:07 UTC, Chris Wright wrote:
>> I'm considering adding JSON5 support to std.json and want to know how well this would be received.
> 
> Considering this is something that has apparently existed for more than three years and this is the first time I'm hearing of it, I have a hard time justifying it's inclusion into Phobos. I have never once heard of any project using JSON5 and googling it brings up one negative blog post, the HN post which is full of negative comments, and every other result are language bindings.
> 
> I have to agree with all the negativity, it seems like a poor idea and calling it "JSON" is inappropriate. Also, std.json is slated for deprecation.
> 
> If you want a JSON5 D parser, I'd say go ahead and throw it up on code.dlang.org. What would be much more beneficial IMO is a YAML parser based on the API of http://code.dlang.org/packages/std_data_json

JSON5 is pretty much just a codification of common relaxations to JSON parsers. For instance, a feature request for Newtonsoft JSON.NET (which is the most well known JSON library for .NET) to support JSON5 pretty much went "What's JSON5?" and then "Oh, we already support 90% of that".

The recent thread about DUB's config format would have been much more strongly in favor of JSON if std.json supported JSON5.

The reason to incorporate it into the standard library interpretation is because it's mostly identical to JSON, so sharing implementations is obvious.

On Monday, 30 November 2015 at 16:12:07 UTC, Chris Wright wrote:
> The reason to incorporate it into the standard library interpretation is because it's mostly identical to JSON, so sharing implementations is obvious.

That doesn't follow. Just because implementations are mostly shared doesn't mean that it should be included into Phobos. Someone still needs to maintain your new code and the cost benefit analysis still needs to happen.

> For instance, a feature request for Newtonsoft JSON.NET (which is the most
> well known JSON library for .NET) to support JSON5 pretty much went "What's JSON5?" and then "Oh, we already support 90% of that".

So the authors of one of the most well known JSON libraries has also never heard of it.

> The recent thread about DUB's config format would have been much more strongly
> in favor of JSON if std.json supported JSON5.

How much stronger can you get than almost everyone but like three people being in favor of JSON over SDL?

Again, I see no evidence that JSON5 is at all popular. I can list well over 2000 popular repos on Github that use JSON in some way, either with APIs or config files, etc. So show me at least 50 github repos with over 500 stars that use JSON5 (meaning they are not JSON5 parsers) and I will change my mind. 50 is not an unreasonable number to ask for, as you want this to go into Phobos, this needs a lot of ground swell and use.

JSON5 is also just a terrible idea. There is a very good reason why JSON does not have comments https://plus.google.com/+DouglasCrockfordEsq/posts/RK8qyGVaGSr and why it's strict. Data formats should have data, not anything else. It's bare bones structure and strictness makes standard following parsers extremely fast, in fact the fastest serializers ever. Loose data formats are a nightmare to work with, XML being the front runner in this regard, as parsing is slow and data can be stored and accessed in tons of different ways.

As I mentioned earlier, JSON5 is not JSON, it's something else. There is only one JSON and calling what ever this is JSON is disingenuous on the part of the creator. So even if this got into Phobos, it shouldn't go into std.json.

If you want a human readable format, just use YAML and use one of the many, many cross compilers of YAML to JSON.

On Mon, 30 Nov 2015 20:42:20 +0000, Jack Stouffer wrote:
> JSON5 is also just a terrible idea. There is a very good reason why JSON does not have comments https://plus.google.com/+DouglasCrockfordEsq/posts/RK8qyGVaGSr and why it's strict. Data formats should have data, not anything else.

I fully understand and sympathize with desires for simple formats for data generated by a program to be read by a program. JSON is just fine for REST interfaces. It's great as a storage format for document databases. I'm not advocating for JSON5 there.

But JSON is also used for human-edited configuration files.

Configuration files should allow for comments. I poke about /etc and see that nearly every file there has copious comments detailing which fields exist, what sorts of values are allowed, the effects that field has, what the default values are, and whether you need to coordinate values between them. I've probably used those comments several hundred times, and I'd spend many hours paging through Google and asking questions on stackoverflow if they didn't exist.

JSON forbids comments, which is less than ideal. But Ruby's JSON parser allows comments, as does Newtonsoft JSON.NET.

If a human is going to edit a document, it should ideally be easy to edit it. It would be friendly, for instance, to allow trailing commas, because that's one less thing that a human has to worry about. Java's org.json library, which is linked from and has its documentation hosted at json.org, allows trailing commas. Newtonsoft JSON.NET does likewise.

It would be friendlier to the fingers to be able to omit quotes on simple object keys since it's less typing and no less clarity. Many strings contain double quotes, so it would be nice to allow single-quoted strings. org.json allows single-quoted strings. JSON.NET allows unquoted object keys and single-quoted strings.