Am 28.07.2015 um 17:19 schrieb Etienne Cimon:
> On Tuesday, 28 July 2015 at 14:07:19 UTC, Atila Neves wrote:
>> Start of the two week process, folks.
>>
>> Code: https://github.com/s-ludwig/std_data_json
>> Docs: http://s-ludwig.github.io/std_data_json/
>>
>> Atila
>
> This is cool:
> https://github.com/s-ludwig/std_data_json/blob/aac6d846d596750623fd5c546343f4f9d19447fa/source/stdx/data/json/value.d#L183
>
>
> I was getting tired of programmatically checking for null, then checking
> for object type, before moving along in the object and doing the same
> recursively. Not quite as intuitive as the optional chaining ?. operator
> in swift but it gets pretty close
> https://blog.sabintsev.com/optionals-in-swift-c94fd231e7a4#5622

An idea might be to support something like this:

json_value.opt.foo.bar[2].baz
or
opt(json_value).foo.bar[2].baz

opt (name is debatable) would return a wrapper struct around the JSONValue that supports opDispatch/opIndex and propagates a missing field to the top gracefully. It could also keep track of the complete path to give a nice error message when a non-existent value is dereferenced.

On Tuesday, 28 July 2015 at 18:45:51 UTC, Sönke Ludwig wrote:
> An idea might be to support something like this:
>
> json_value.opt.foo.bar[2].baz
> or
> opt(json_value).foo.bar[2].baz
>
> opt (name is debatable) would return a wrapper struct around the JSONValue that supports opDispatch/opIndex and propagates a missing field to the top gracefully. It could also keep track of the complete path to give a nice error message when a non-existent value is dereferenced.

+1

This would solve the cumbersome access of something deeply nested that I've had to deal with when using stdx.data.json. Combine that with the Algebraic improvements you've mentioned before and it'll be just about as pleasant as it could be to use.

On 7/28/2015 7:07 AM, Atila Neves wrote:
> Start of the two week process, folks.

Thank you very much, Sönke, for taking this on. Thank you, Atila, for taking on the thankless job of being review manager.

Just looking at the documentation only, some general notes:

1. Not sure that 'JSON' needs to be embedded in the public names. 'parseJSONStream' should just be 'parseStream', etc. Name disambiguation, if needed, should be ably taken care of by a number of D features for that purpose. Additionally, I presume that the stdx.data package implies a number of different formats. These formats should all use the same names with as similar as possible APIs - this won't work too well if JSON is embedded in the APIs.

2. JSON is a trivial format, http://json.org/. But I count 6 files and 30 names in the public API.

3. Stepping back a bit, when I think of parsing JSON data, I think:

    auto ast = inputrange.toJSON();

where toJSON() accepts an input range and produces a container, the ast. The ast is just a JSON value. Then, I can just query the ast to see what kind of value it is (using overloading), and walk it as necessary. To create output:

    auto r = ast.toChars();  // r is an InputRange of characters
    writeln(r);

So, we'll need:
    toJSON
    toChars
    JSONException

The possible JSON values are:
    string
    number
    object (associative arrays)
    array
    true
    false
    null

Since these are D builtin types, they can actually be a simple union of D builtin types.

There is a decision needed about whether toJSON() allocates data or returns slices into its inputrange. This can be 'static if' tested by: if inputrange can return immutable slices. toChars() can take a compile time argument to determine if it is 'pretty' or not.

On Tue, Jul 28, 2015 at 03:29:02PM -0700, Walter Bright via Digitalmars-d wrote: [...]
> 3. Stepping back a bit, when I think of parsing JSON data, I think:
> 
>     auto ast = inputrange.toJSON();
> 
> where toJSON() accepts an input range and produces a container, the ast. The ast is just a JSON value. Then, I can just query the ast to see what kind of value it is (using overloading), and walk it as necessary.

+1. The API should be as simple as possible.

Ideally, I'd say hook it up to std.conv.to for maximum flexibility. Then
you can just use to() to convert between a JSON container and the value
that it represents (assuming the types are compatible).

OTOH, some people might want the option of parser-driven data processing instead (e.g. the JSON data is very large and we don't want to store the whole thing in memory at once). I'm not sure what a good API for that would be, though.


> To create output:
> 
>     auto r = ast.toChars();  // r is an InputRange of characters
>     writeln(r);
> 
> So, we'll need:
>     toJSON
>     toChars

Shouldn't it just be toString()?


[...]
> The possible JSON values are:
>     string
>     number
>     object (associative arrays)
>     array
>     true
>     false
>     null
> 
> Since these are D builtin types, they can actually be a simple union of D builtin types.
> 
> There is a decision needed about whether toJSON() allocates data or returns slices into its inputrange. This can be 'static if' tested by: if inputrange can return immutable slices. toChars() can take a compile time argument to determine if it is 'pretty' or not.

Whether or not toJSON() allocates *data*, it will have to allocate container nodes of some sort. At the minimum, it will need to use AA's, so it cannot be @nogc.


T

-- 
Recently, our IT department hired a bug-fix engineer. He used to work for Volkswagen.

On 7/28/2015 3:37 PM, H. S. Teoh via Digitalmars-d wrote:
> On Tue, Jul 28, 2015 at 03:29:02PM -0700, Walter Bright via Digitalmars-d wrote:
> Ideally, I'd say hook it up to std.conv.to for maximum flexibility. Then
> you can just use to() to convert between a JSON container and the value
> that it represents (assuming the types are compatible).

Well, I wouldn't want std.conv to be importing std.json.


> OTOH, some people might want the option of parser-driven data processing
> instead (e.g. the JSON data is very large and we don't want to store the
> whole thing in memory at once).

That is a good point.


> I'm not sure what a good API for that would be, though.

Probably simply returning an InputRange of JSON values.


>> To create output:
>>
>>      auto r = ast.toChars();  // r is an InputRange of characters
>>      writeln(r);
>>
>> So, we'll need:
>>      toJSON
>>      toChars
>
> Shouldn't it just be toString()?

No. toString() returns a string, which has to be allocated. toChars() (an upcoming convention) would return an InputRange instead, side-stepping allocation.


> Whether or not toJSON() allocates *data*, it will have to allocate
> container nodes of some sort. At the minimum, it will need to use AA's,
> so it cannot be @nogc.

That's right. At some point the API will need to add a parameter for Andrei's allocator system.

A speed optimization, since JSON parsing speed is critical:

If the parser is able to use slices of its input, store numbers as slices. Only convert them to numbers lazily, as the numeric conversion can take significant time.

On Tuesday, 28 July 2015 at 23:16:34 UTC, Walter Bright wrote:
> A speed optimization, since JSON parsing speed is critical:
>
> If the parser is able to use slices of its input, store numbers as slices. Only convert them to numbers lazily, as the numeric conversion can take significant time.

That's what it does (depending on which parser you use). The StAX style parser included is lazy and non-allocating.

On 7/28/2015 4:24 PM, Brad Anderson wrote:
> On Tuesday, 28 July 2015 at 23:16:34 UTC, Walter Bright wrote:
>> A speed optimization, since JSON parsing speed is critical:
>>
>> If the parser is able to use slices of its input, store numbers as slices.
>> Only convert them to numbers lazily, as the numeric conversion can take
>> significant time.
>
> That's what it does (depending on which parser you use). The StAX style parser
> included is lazy and non-allocating.

Great!

On Tue, Jul 28, 2015 at 03:55:22PM -0700, Walter Bright via Digitalmars-d wrote:
> On 7/28/2015 3:37 PM, H. S. Teoh via Digitalmars-d wrote:
> >On Tue, Jul 28, 2015 at 03:29:02PM -0700, Walter Bright via Digitalmars-d wrote:
> >Ideally, I'd say hook it up to std.conv.to for maximum flexibility.
> >Then you can just use to() to convert between a JSON container and
> >the value that it represents (assuming the types are compatible).
> 
> Well, I wouldn't want std.conv to be importing std.json.

I'm pretty sure std.conv has interfaces that allow you to keep JSON-specific stuff in std.json, so that you don't get the JSON conversion capability until you actually import std.json.


> >OTOH, some people might want the option of parser-driven data processing instead (e.g. the JSON data is very large and we don't want to store the whole thing in memory at once).
> 
> That is a good point.
> 
> 
> >I'm not sure what a good API for that would be, though.
> 
> Probably simply returning an InputRange of JSON values.

But how would you capture the nesting substructures?


> >>To create output:
> >>
> >>     auto r = ast.toChars();  // r is an InputRange of characters
> >>     writeln(r);
> >>
> >>So, we'll need:
> >>     toJSON
> >>     toChars
> >
> >Shouldn't it just be toString()?
> 
> No. toString() returns a string, which has to be allocated. toChars()
> (an upcoming convention) would return an InputRange instead,
> side-stepping allocation.
[...]

??!  Surely you have heard of the non-allocating overload of toString?

	void toString(scope void delegate(const(char)[]) dg);


T

-- 
When solving a problem, take care that you do not become part of the problem.

On Tuesday, 28 July 2015 at 18:45:51 UTC, Sönke Ludwig wrote:
> Am 28.07.2015 um 17:19 schrieb Etienne Cimon:
>> On Tuesday, 28 July 2015 at 14:07:19 UTC, Atila Neves wrote:
>>> Start of the two week process, folks.
>>>
>>> Code: https://github.com/s-ludwig/std_data_json
>>> Docs: http://s-ludwig.github.io/std_data_json/
>>>
>>> Atila
>>
>> This is cool:
>> https://github.com/s-ludwig/std_data_json/blob/aac6d846d596750623fd5c546343f4f9d19447fa/source/stdx/data/json/value.d#L183
>>
>>
>> I was getting tired of programmatically checking for null, then checking
>> for object type, before moving along in the object and doing the same
>> recursively. Not quite as intuitive as the optional chaining ?. operator
>> in swift but it gets pretty close
>> https://blog.sabintsev.com/optionals-in-swift-c94fd231e7a4#5622
>
> An idea might be to support something like this:
>
> json_value.opt.foo.bar[2].baz
> or
> opt(json_value).foo.bar[2].baz
>
> opt (name is debatable) would return a wrapper struct around the JSONValue that supports opDispatch/opIndex and propagates a missing field to the top gracefully. It could also keep track of the complete path to give a nice error message when a non-existent value is dereferenced.

I like it quite well. No, actually, a lot. Thinking about it some more... this could end up being the most convenient feature ever known to mankind and would likely push it towards a new age of grand discoveries, infinite fusion power and space colonization. Lets do it