Wanted: Review manager for std.data.json (page 6)

Am 09.04.2015 um 23:56 schrieb weaselcat: > On Thursday, 9 April 2015 at 21:52:08 UTC, Sönke Ludwig wrote: >> Hm interesting, but for me LDC unfortunately crashes when adding >> -partial-inliner. What else did you specify? > > ldc2 -partial-inliner -boundscheck=off -singleobj -ofjson_d_new_lazy_ldc > -O5 -release -w -oq -Istd_data_json/source/ > std_data_json/source/stdx/data/json/*.d test_new_lazy.d > > it seems to segfault with -O0 optimization > > ./json_d_new_lazy_ldc 0.95s user 0.08s system 99% cpu 1.025 total > (clang)./json_rapid_cpp 0.98s user 0.30s system 99% cpu 1.282 total > (gcc)./json_rapid_cpp 0.80s user 0.31s system 99% cpu 1.108 total Okay, turned out that this was a 0.15.2-beta.1 issue. I've downgraded to 0.15.1 and now get a nice time of 1.30 seconds on my tablet PC!

what a difference. cool stuff Sönke! below results from a 3.1 GHz Intel Core i7 Broadwell mbp13" on os x: ldc2 0.15.1 from homebrew dmd v2.067 from homebrew optimized => " -partial-inliner -boundscheck=off -singleobj " D new lazy Ldc optimized 0.85 204.5Mb Crystal Schema 1.41 283.4Mb Crystal Pull 1.49 1.2Mb C++ Rapid 1.92 946.5Mb D new lazy dmd 2 204.5Mb D new lazy Ldc 2.12 204.6Mb D new Ldc optimized 3.19 1640.5Mb Crystal 3.23 1091.9Mb Javascript Node 3.48 905.3Mb D new Ldc 3.98 1640.5Mb Nim 4.38 1336.8Mb Go 5.34 420.3Mb D new - debug 6.34 1793.6Mb Python3 7.06 999.7Mb Rust 7.46 3041.7Mb Python Pypy 3 9.27 2710.3Mb Julia 10.09 3054.8Mb D 12.47 1219.1Mb Ruby 13.94 2167.2Mb C++ Boost 24.64 2699.4Mb D Ldc 27.38 928.1Mb y

On Thursday, 9 April 2015 at 20:01:10 UTC, weaselcat wrote: > On Thursday, 9 April 2015 at 19:44:57 UTC, weaselcat wrote: >> >> I saw that commit to the benchmark and changed it locally. >> They're about the same performance now comparing clang to LDC, with -inline -boundscheck=off -singleobj flags >> >> Nice. > > Also, since an LDC dev might read this - is there a reason -singleobj isn't on by default when creating an executable? I argued it should be the default a while ago and I vaguely remember agreement from at least one LDC dev. but I can't find the post now :/

On Friday, 10 April 2015 at 07:35:13 UTC, John Colvin wrote: > On Thursday, 9 April 2015 at 20:01:10 UTC, weaselcat wrote: >> Also, since an LDC dev might read this - is there a reason -singleobj isn't on by default when creating an executable? > > I argued it should be the default a while ago and I vaguely remember agreement from at least one LDC dev. but I can't find the post now :/ I still agree with you. ;) The only reason why I didn't just commit the change so far is backwards compatibility, but as we might overhaul some of the command line options soon anyway, I think we should just go ahead with that change. As noted previously, mixing one-object-per-file and all-at-once compilation (e.g. in incremental builds) does not work correctly in all cases due to frontend restrictions anyway. — David

On 2015-04-07 18:37, Sönke Ludwig wrote: > Anyone up to this? The issues of the previous discussion [1] have all > been addressed now more or less, so the package is ready for a more > thorough review. Is it possible to use "toJSON" or a similar method to generate JSON from a primitive type without going through JSONValue? -- /Jacob Carlborg

Am 16.04.2015 um 11:17 schrieb Jacob Carlborg: > On 2015-04-07 18:37, Sönke Ludwig wrote: >> Anyone up to this? The issues of the previous discussion [1] have all >> been addressed now more or less, so the package is ready for a more >> thorough review. > > Is it possible to use "toJSON" or a similar method to generate JSON from > a primitive type without going through JSONValue? > I'd like to let that be part of a more general serialization framework in top of this package instead of integrating a simplistic custom solution that will then later be obsoleted. Having said that, you can go through JSONToken instead of JSONValue for lower (computational) overhead.

On 2015-04-16 11:29, Sönke Ludwig wrote: > I'd like to let that be part of a more general serialization framework > in top of this package instead of integrating a simplistic custom > solution that will then later be obsoleted. I was thinking about some low level primitives that a serialization library could use. Types that cannot be broken in to smaller parts, integer, bool and so on. > Having said that, you can go through JSONToken instead of JSONValue for lower (computational) overhead. Something like? JSONToken token; token.number = 3; token.kind = JSONToken.Kind.number; toJSON(token); Or what would you recommend a serialization library used? -- /Jacob Carlborg

April 16, 2015

Re: Wanted: Review manager for std.data.json

Posted by Sönke Ludwig
in reply to Jacob Carlborg

Permalink

Sönke Ludwig

Posted in reply to Jacob Carlborg

Permalink

Am 16.04.2015 um 13:03 schrieb Jacob Carlborg:
> On 2015-04-16 11:29, Sönke Ludwig wrote:
>
>> I'd like to let that be part of a more general serialization framework
>> in top of this package instead of integrating a simplistic custom
>> solution that will then later be obsoleted.
>
> I was thinking about some low level primitives that a serialization
> library could use. Types that cannot be broken in to smaller parts,
> integer, bool and so on.
>
>> Having said that, you can go through JSONToken instead of JSONValue
>> for lower (computational) overhead.
>
> Something like?
>
> JSONToken token;
> token.number = 3;
> token.kind = JSONToken.Kind.number;
> toJSON(token);

Yes, just without setting the kind explicitly (it's set automatically by assigning to the number property). Instead of toJSON(), writeJSON() would probably be more appropriate to avoid additional allocations.

>
> Or what would you recommend a serialization library used?
>

The simplest target for a serialization library would be to generate a stream of JSONParserNodes. That way the serializer doesn't have to keep track of nesting levels and can reuse the pretty printing functionality of stdx.data.generator.

However, this currently requires an input range of JSONParserNodes, which looks like it is suboptimal for a serializer (either requires an explicitly allocated node array, or a strange serializer architecture that directly works as an input range). For that reason I'm thinking about providing an output range interface like so:

	auto dst = appender!string;
	auto rng = jsonOutputRange(dst);

	JSONParserNode n;
	// start an object
	n.kind = JSONParserNode.Kind.objectStart;
	rng.put(n);

	// write a single entry "foo": 10.5
	n.key = "foo";
	rng.put(n);
	rng.put(10.5);

	// write another entry "bar": true
	n.key = "bar";
	rng.put(n);
	rng.put(true);

	// finish the object
	n.kind = JSONParserNode.Kind.objectEnd;
	rng.put(n);

	writefln("JSON: %s", dst.data);

What do you think?

BTW, looks like JSONToken could use a set of constructors for more convenient in-place construction.

On Wednesday, 8 April 2015 at 18:56:00 UTC, Iain Buclaw wrote: > > Frankly, if we are not as fast (or elegant) as Python's json library, > it should be thrown out back to the drawing board. > > Iain. I'll leave the speed aside, as more recent posts show improvements and I think Sönke will be able to take what he has, tweak it, and get it close to the fastest libraries. I think we can beat the competition. For elegance, dynamic languages will always be able to deal with JSON with less syntax, as you get just the basic type back out of arrays and maps, without having to explictly request a string, integer, etc. Static languages will always have more verbose JSON libraries, but they might also catch bugs more easily. We just need the short properties for converting JSON values to basic types.

Forums