On Sunday, 3 August 2014 at 15:14:43 UTC, Andrei Alexandrescu wrote:
> On 8/3/14, 2:38 AM, Sönke Ludwig wrote:
>> 3. Use of "opDispatch" for an open set of members has been
>> criticized for vibe.data.json before and I agree with that
>> criticism. The only advantage is saving a few keystrokes
>> (json.key instead of json["key"]), but I came to the conclusion
>> that the right approach to work with JSON values in D is to
>> always directly deserialize when/if possible anyway, which
>> mostly makes this is a moot point.
>
> Interesting. Well if experience with opDispatch is negative then
> it should probably not be used here, or only offered on an
> opt-in basis.
>
I suspect that depends on the circumstances.  I've been using this style (with Adam's jsvar), and I find it quite nice for decomposing my TOML parse trees to Variant-like structures that go several levels deep.  It makes reading (and, consequently, reasoning about) them much easier for me.

That said, I think the ideal would be that nesting Variant[] should work predictably such that users can just write a one-line opDispatch if they want it to behave that way.

-Wyatt

On Monday, 4 August 2014 at 07:34:19 UTC, Jacob Carlborg wrote:
> On Sunday, 3 August 2014 at 18:44:37 UTC, Dicebot wrote:
>
>> Before going this route one needs to have a good vision how it may interact with imaginary std.serialization to avoid later deprecation.
>
> I suggest only provide functions for serializing primitive types. A separate serialization module/package with a JSON archive type would use this module as its backend.

Do you consider structs primitive types? This is probably #1 use case for JSON conversion.

>> At the same time I have recently started to think that dedicated serialization module that decouples aggregate iteration from data storage format is in most cases impractical for performance reasons - different serialization methods imply very different efficient iteration strategies. Probably it is better to define serialization compile-time traits instead and require each `std.data.*` provider to implement those on its own in the most effective fashion.
>
> I'm not sure I agree with that. In my work on std.serialization I have not seen this to be a problem. What problems have you found?

http://forum.dlang.org/post/mzweposldwqdtmqoltiy@forum.dlang.org

On Monday, 4 August 2014 at 14:02:22 UTC, Dicebot wrote:

> Do you consider structs primitive types? This is probably #1 use case for JSON conversion.

No, only types that cannot be broken down in to smaller pieces, i.e. integral, floating points,  bool and strings.

> http://forum.dlang.org/post/mzweposldwqdtmqoltiy@forum.dlang.org

I don't understand exactly how that binary serialization works. I think I would need a code example.

--
/Jacob Carlborg

On Monday, 4 August 2014 at 09:10:46 UTC, Daniel Murphy wrote:

> This is exactly what I need in most projects.  Basic types, arrays, AAs, and structs are usually enough.

I was more thinking only types that cannot be broken down in to smaller pieces, i.e. integer, floating point, bool and string. The serializer would break down the other types in to smaller pieces.

--
/Jacob Carlborg

On Monday, 4 August 2014 at 14:18:41 UTC, Jacob Carlborg wrote:
> On Monday, 4 August 2014 at 14:02:22 UTC, Dicebot wrote:
>
>> Do you consider structs primitive types? This is probably #1 use case for JSON conversion.
>
> No, only types that cannot be broken down in to smaller pieces, i.e. integral, floating points,  bool and strings.

That is exactly the problem - if `structToJson` won't be provided, complaints are inevitable, it is too basic feature to wait for std.serialization :(

>> http://forum.dlang.org/post/mzweposldwqdtmqoltiy@forum.dlang.org
>
> I don't understand exactly how that binary serialization works. I think I would need a code example.

Simplified serialization algorithm:

1) write (cast(void*) &struct)[0..struct.sizeof] to target buffer
2) write any of array content to the same buffer after the struct
3.1) if array contains structs, recursion
3.2) go back to buffer[0..struct.sizeof] slice and update array fields to store an index in the same buffer instead of actual ptr

Simplified deserialization algorithm:

1) recursively traverse the struct and replace array index offsets with real slices to the buffer

(I don't want to bother with getting copyright permissions to publish actual code)

I am pretty sure that this is not the only optimized serialization approach out there that does not fit in a content-insensitive primitive-based traversal scheme. And we won't Phobos stuff to be blazingly fast which can lead to situation where new data module will circumvent the std.serialization API to get more performance.

On 8/4/14, 12:56 AM, Jacob Carlborg wrote:
> On Sunday, 3 August 2014 at 07:16:05 UTC, Andrei Alexandrescu wrote:
>> We need a better json library at Facebook. I'd discussed with Sönke
>> the possibility of taking vibe.d's json to std but he said it needs
>> some more work. So I took std.jgrandson to proof of concept state and
>> hence ready for destruction:
>>
>> http://erdani.com/d/jgrandson.d
>> http://erdani.com/d/phobos-prerelease/std_jgrandson.html

Thanks for your comments! A few responses within:

> * Could you please put it on Github to get syntax highlighting and all
> the other advantages

Quick workaround: http://dpaste.dzfl.pl/65f4dcc36ab8

> * It doesn't completely follow the Phobos naming conventions

What would be the places?

> * The indentation is off in some places

Xamarin/Mono-D is at fault here :o).

> * The unit tests is a bit lacking for the separate parsing functions

Agreed.

> * There are methods for getting the strings and numbers, what about
> booleans?

You mean for Token? Good point. Numbers and strings are somewhat special because they have a payload associated. In contrast Booleans are represented by two distinct tokens. Would be good to add a convenience method.

> * Shouldn't it be called TokenRange?

Yah.

> * Shouldn't this be built using the lexer generator you so strongly have
> been pushing for?

Of course, and in the beginning I've actually pasted some code from it. Then I regressed to minimizing dependencies.

> * The unit tests for TokenStream is very dense. I would prefer empty
> newlines for grouping "assert" and calls to "popFront" belonging together

De gustibus et the coloribus non est disputandum :o).


Andrei

On 8/4/14, 12:47 AM, Andrea Fontana wrote:
> On my bson library I found very useful to have some methods to know if a
> field exists or not, and to get a "defaulted" value. Something like:
>
> auto assume(T)(Value v, T default = T.init);

Nice. Probably "get" would be better to be in keep with built-in hashtables.

> Another good method could be something like xpath to get a deep value:
>
> Value v = value["/path/to/sub/object"];

Cool. Is it unlikely that a value contains an actual slash? If so would be value["path"]["to"]["sub"]["object"] more precise?

> Moreover in my library I actually have three different methods to read a
> value:
>
> T get(T)() // Exception if value is not a T or not valid or value
> doesn't exist
> T to(T)()  // Try to convert value to T using to!string. Exception if
> doesn't exists or not valid
>
> BsonField!T as(T)(lazy T default = T.init)  // Always return a value
>
> BsonField!T is an "alias this"-ed struct with two fields: T value and
> bool error(). T value is the aliased field, and error() tells you if
> value is defaulted (because of an error: field not exists or can't
> convert to T)
>
> So I can write something like this:
>
> int myvalue = json["/that/deep/property"].as!int;
>
> or
>
> auto myvalue = json["/that/deep/property"].as!int(10);
>
> if (myvalue.error) writeln("Property doesn't exists, I'm using default
> value);
>
> writeln("Property value: ", myvalue);
>
> I hope this can be useful...

Sure is, thanks. Listen, would you want to volunteer a std.data.json proposal?


Andrei

On 2014-08-04 16:55, Dicebot wrote:

> That is exactly the problem - if `structToJson` won't be provided,
> complaints are inevitable, it is too basic feature to wait for
> std.serialization :(

Hmm, yeah, that's a problem.

> Simplified serialization algorithm:
>
> 1) write (cast(void*) &struct)[0..struct.sizeof] to target buffer
> 2) write any of array content to the same buffer after the struct
> 3.1) if array contains structs, recursion
> 3.2) go back to buffer[0..struct.sizeof] slice and update array fields
> to store an index in the same buffer instead of actual ptr
>
> Simplified deserialization algorithm:
>
> 1) recursively traverse the struct and replace array index offsets with
> real slices to the buffer

I see. I need to think a bit about this.

> (I don't want to bother with getting copyright permissions to publish
> actual code)

Fair enough. The above was quite descriptive.

> I am pretty sure that this is not the only optimized serialization
> approach out there that does not fit in a content-insensitive
> primitive-based traversal scheme. And we won't Phobos stuff to be
> blazingly fast which can lead to situation where new data module will
> circumvent the std.serialization API to get more performance.

I don't like the idea of having to reimplement serialization for each data type that can be generalized.

-- 
/Jacob Carlborg

On 2014-08-04 18:55, Andrei Alexandrescu wrote:

> What would be the places?

That's why it's easier with Github ;) I can comment directly on a line.

I just had a quick look but "_true", "_false" and "_null" in Token.Kind. If I recall correctly we add an underscore as a suffix for symbols with the same name as keywords.

> You mean for Token? Good point.

Yes, in Token.

> Numbers and strings are somewhat special
> because they have a payload associated. In contrast Booleans are
> represented by two distinct tokens. Would be good to add a convenience
> method.

Right.

> De gustibus et the coloribus non est disputandum :o).

Please avoid these Latin sentences, I have no idea what they mean. This is an international community, please don't make it more complicated than it already is with language barriers.

-- 
/Jacob Carlborg

On 8/4/14, 11:46 AM, Jacob Carlborg wrote:
>
>
>> De gustibus et the coloribus non est disputandum :o).
>
> Please avoid these Latin sentences, I have no idea what they mean. This
> is an international community, please don't make it more complicated
> than it already is with language barriers.

"Favorite foods and colors are not to be disputed." 51,300 results on google... and please let's end this before it becomes another Epic Debate. -- Andrei