April 09, 2015 Re: Wanted: Review manager for std.data.json | ||||
---|---|---|---|---|
| ||||
Posted in reply to Martin Nowak | Am 09.04.2015 um 15:20 schrieb Martin Nowak: > On 04/09/2015 10:59 AM, Sönke Ludwig wrote: >> As far as the profiler results can be trusted, a good chunk of the time >> gets spent for reading individual bytes from memory, but there must be >> something else low-level going on that make things this bad. However, >> there is nothing fundamental in the structure/design that would cause >> this, so I think spending more time with the profiler is the only >> logical step now. Unfortunately my VTune license has expired and perf on >> Linux makes the task quite a bit more involved. > > I didn't found too many issues. > > Most of the time is spent in parseJSONValue (looks like there are some > expansive struct copies) > https://github.com/s-ludwig/std_data_json/blob/1da3f828ae6c4fd7cac7f7e13ae9e51ec93e6f02/source/stdx/data/json/parser.d#L148 I'll have a look at the DOM part later. At the moment I'm looking at low level issues of the lexer. There is a lot of potential there, but it's a lot of detail work. > > and skipWhitespace. This function could really take some optimization, > e.g. avoid UTF decoding. All of the lexing work happens on str.representation instead of the string directly, so no decoding should happen there. > > https://github.com/s-ludwig/std_data_json/blob/1da3f828ae6c4fd7cac7f7e13ae9e51ec93e6f02/source/stdx/data/json/lexer.d#L345 > > Libdparse has some optimized ASM function, might be useful. > https://github.com/Hackerpilot/libdparse/blob/51b7d9d321aac0fcc4a9be99bbbed5db3158326c/src/std/d/lexer.d#L2233 > |
April 09, 2015 Re: Wanted: Review manager for std.data.json | ||||
---|---|---|---|---|
| ||||
Posted in reply to John Colvin | On 4/9/15 5:10 AM, John Colvin wrote:
> On Thursday, 9 April 2015 at 11:49:00 UTC, Martin Nowak wrote:
>> On 04/08/2015 08:32 PM, tcha wrote:
>>
>> Now with release numbers.
>>
>>> D new - debug - 14.98s, 1782.0Mb
>> 8.53s, 1786.8Mb
>>> D new Gdc - debug - 29.08s, 1663.9Mb
>> GDC still misses @nogc support.
>>> D new Ldc - 16.99s, 1663.0Mb
>> 18.76s, 1664.1Mb
>>> D new lazy - debug - 11.50s, 213.2Mb
>> 4.57s, 206Mb
>>> D new lazy Gdc - 13.66s, 206.1Mb
>> Can't compile stdx.data.json with gdc-4.9.0 which doesn't yet support
>> @nogc.
>>> D new lazy Ldc - 3.59s, 205.4Mb
>> 4.0s, 205.4Mb
>>
>> LDC doesn't yet have the GC improvements, therefor is much slower for
>> the DOM parsing benchmarks.
>
> Still getting trounced across the board by rapidjson.
Agreed. Rapidjson is the most important baseline. -- Andrei
|
April 09, 2015 Re: Wanted: Review manager for std.data.json | ||||
---|---|---|---|---|
| ||||
Posted in reply to Martin Nowak | On Thursday, 9 April 2015 at 11:49:00 UTC, Martin Nowak wrote: > On 04/08/2015 08:32 PM, tcha wrote: > > Now with release numbers. > >> D new - debug - 14.98s, 1782.0Mb > 8.53s, 1786.8Mb >> D new Gdc - debug - 29.08s, 1663.9Mb > GDC still misses @nogc support. >> D new Ldc - 16.99s, 1663.0Mb > 18.76s, 1664.1Mb >> D new lazy - debug - 11.50s, 213.2Mb > 4.57s, 206Mb >> D new lazy Gdc - 13.66s, 206.1Mb > Can't compile stdx.data.json with gdc-4.9.0 which doesn't yet support @nogc. >> D new lazy Ldc - 3.59s, 205.4Mb > 4.0s, 205.4Mb > > LDC doesn't yet have the GC improvements, therefor is much slower for > the DOM parsing benchmarks. ldc -singleobj flag speeds it up dramatically ldc2 -ofjson_d_new_lazy_ldc -O5 -release -enable-inlining -w -oq -Istd_data_json/source/ std_data_json/source/stdx/data/json/*.d test_new_lazy.d ./json_d_new_lazy_ldc 2.79s user 0.08s system 99% cpu 2.868 total ldc2 -singleobj -ofjson_d_new_lazy_ldc -O5 -release -enable-inlining -w -oq -Istd_data_json/source/ std_data_json/source/stdx/data/json/*.d test_new_lazy.d ./json_d_new_lazy_ldc 1.78s user 0.08s system 99% cpu 1.869 total for comparison: ./json_rapid_cpp 0.80s user 0.30s system 99% cpu 1.106 total Bye. |
April 09, 2015 Re: Wanted: Review manager for std.data.json | ||||
---|---|---|---|---|
| ||||
Posted in reply to weaselcat | Am 09.04.2015 um 21:06 schrieb weaselcat:
> On Thursday, 9 April 2015 at 11:49:00 UTC, Martin Nowak wrote:
>> On 04/08/2015 08:32 PM, tcha wrote:
>>
>> Now with release numbers.
>>
>>> D new - debug - 14.98s, 1782.0Mb
>> 8.53s, 1786.8Mb
>>> D new Gdc - debug - 29.08s, 1663.9Mb
>> GDC still misses @nogc support.
>>> D new Ldc - 16.99s, 1663.0Mb
>> 18.76s, 1664.1Mb
>>> D new lazy - debug - 11.50s, 213.2Mb
>> 4.57s, 206Mb
>>> D new lazy Gdc - 13.66s, 206.1Mb
>> Can't compile stdx.data.json with gdc-4.9.0 which doesn't yet support
>> @nogc.
>>> D new lazy Ldc - 3.59s, 205.4Mb
>> 4.0s, 205.4Mb
>>
>> LDC doesn't yet have the GC improvements, therefor is much slower for
>> the DOM parsing benchmarks.
>
> ldc -singleobj flag speeds it up dramatically
>
> ldc2 -ofjson_d_new_lazy_ldc -O5 -release -enable-inlining -w -oq
> -Istd_data_json/source/ std_data_json/source/stdx/data/json/*.d
> test_new_lazy.d
>
> ./json_d_new_lazy_ldc 2.79s user 0.08s system 99% cpu 2.868 total
>
> ldc2 -singleobj -ofjson_d_new_lazy_ldc -O5 -release -enable-inlining -w
> -oq -Istd_data_json/source/ std_data_json/source/stdx/data/json/*.d
> test_new_lazy.d
>
> ./json_d_new_lazy_ldc 1.78s user 0.08s system 99% cpu 1.869 total
>
>
> for comparison:
> ./json_rapid_cpp 0.80s user 0.30s system 99% cpu 1.106 total
>
>
> Bye.
Not sure, but that may also have been my recent optimizations.
|
April 09, 2015 Re: Wanted: Review manager for std.data.json | ||||
---|---|---|---|---|
| ||||
Posted in reply to Sönke Ludwig | On Thursday, 9 April 2015 at 19:17:48 UTC, Sönke Ludwig wrote:
> Not sure, but that may also have been my recent optimizations.
Just tried it with your recent optimizations and it doesn't build with LDC 0.15.1.
|
April 09, 2015 Re: Wanted: Review manager for std.data.json | ||||
---|---|---|---|---|
| ||||
Posted in reply to tcha | Am 08.04.2015 um 20:32 schrieb tcha:
> On Wednesday, 8 April 2015 at 13:56:55 UTC, Sönke Ludwig wrote:
>> The problem is that even the pull parser alone is relatively slow.
>> Also, for some reason the linker reports unresolved symbols as soon as
>> I build without the -debug flag...
>
> Unfortunatelly I overlooked that I tested it with std_data_json v 0.12.0
> yesterday.
> So tested it again today on master:
> dmd and gdc failed to build in release
> ldc2 works
>
> I tried it also with ParseJSONStream and made fork from original
> benchmark with added tests (https://github.com/chalucha/benchmarks) so
> anyone can look (and PR)
>
> Some results:
> Ruby - 6.74s, 2106.2Mb
> Go - 5.92s, 420.9Mb
> D - 10.26s, 1322.5Mb
> D Gdc - 24.76s, 926.1Mb
> D Ldc - 26.31s, 925.0Mb
> D new - debug - 14.98s, 1782.0Mb
> D new Gdc - debug - 29.08s, 1663.9Mb
> D new Ldc - 16.99s, 1663.0Mb
> D new lazy - debug - 11.50s, 213.2Mb
> D new lazy Gdc - debug - 13.66s, 206.1Mb
> D new lazy Ldc - 3.59s, 205.4Mb
> Python - 5.89s, 1063.4Mb
> C++ Boost - 14.60s, 2884.8Mb
> C++ Rapid - 0.80s, 678.9Mb
>
>
> Also tried to dustmite the minimal failing version and here is a result:
> http://pastebin.com/YjdvT3G4
>
> It's my first use of it so I hope it can help to solve this problem. It
> outputs less errors, but also compiles fine in debug and fails to link
> in release.
I got the latest version using the pull parser down to the Crystal Pull parser speed*. This is about twice as slow as Rapid JSON.
* Using DMD to compile. LDC is for some reason much slower, but I didn't try the -singleobj switch yet.
|
April 09, 2015 Re: Wanted: Review manager for std.data.json | ||||
---|---|---|---|---|
| ||||
Posted in reply to weaselcat | Am 09.04.2015 um 21:26 schrieb weaselcat:
> On Thursday, 9 April 2015 at 19:17:48 UTC, Sönke Ludwig wrote:
>
>> Not sure, but that may also have been my recent optimizations.
>
> Just tried it with your recent optimizations and it doesn't build with
> LDC 0.15.1.
Should work now. I just tested LDC with -singleobj and it's faster than RapidJSON! Now the DOM parser gets interesting.
|
April 09, 2015 Re: Wanted: Review manager for std.data.json | ||||
---|---|---|---|---|
| ||||
Posted in reply to Sönke Ludwig | On Thursday, 9 April 2015 at 19:35:24 UTC, Sönke Ludwig wrote:
> Am 09.04.2015 um 21:26 schrieb weaselcat:
>> On Thursday, 9 April 2015 at 19:17:48 UTC, Sönke Ludwig wrote:
>>
>>> Not sure, but that may also have been my recent optimizations.
>>
>> Just tried it with your recent optimizations and it doesn't build with
>> LDC 0.15.1.
>
> Should work now. I just tested LDC with -singleobj and it's faster than RapidJSON! Now the DOM parser gets interesting.
What flags are you using? RapidJSON is still a lot faster for me.
also, I just realized the build script has the wrong inline flag for LDC(it's using -enable-inlining instead of -inline)
|
April 09, 2015 Re: Wanted: Review manager for std.data.json | ||||
---|---|---|---|---|
| ||||
Posted in reply to Sönke Ludwig | Am 09.04.2015 um 21:35 schrieb Sönke Ludwig:
> Am 09.04.2015 um 21:26 schrieb weaselcat:
>> On Thursday, 9 April 2015 at 19:17:48 UTC, Sönke Ludwig wrote:
>>
>>> Not sure, but that may also have been my recent optimizations.
>>
>> Just tried it with your recent optimizations and it doesn't build with
>> LDC 0.15.1.
>
> Should work now. I just tested LDC with -singleobj and it's faster than
> RapidJSON! Now the DOM parser gets interesting.
Okay let's say about the same speed, the numbers fluctuate quite a bit.
|
April 09, 2015 Re: Wanted: Review manager for std.data.json | ||||
---|---|---|---|---|
| ||||
Posted in reply to weaselcat | Am 09.04.2015 um 21:37 schrieb weaselcat:
> On Thursday, 9 April 2015 at 19:35:24 UTC, Sönke Ludwig wrote:
>> Am 09.04.2015 um 21:26 schrieb weaselcat:
>>> On Thursday, 9 April 2015 at 19:17:48 UTC, Sönke Ludwig wrote:
>>>
>>>> Not sure, but that may also have been my recent optimizations.
>>>
>>> Just tried it with your recent optimizations and it doesn't build with
>>> LDC 0.15.1.
>>
>> Should work now. I just tested LDC with -singleobj and it's faster
>> than RapidJSON! Now the DOM parser gets interesting.
>
> What flags are you using? RapidJSON is still a lot faster for me.
> also, I just realized the build script has the wrong inline flag for
> LDC(it's using -enable-inlining instead of -inline)
I'll have to test with that exact version, because I've used my own main() function for the test. The flags were: -singleobj -release -O5 -enable-inlining -disable-bondscheck (-disable-boundscheck is only a little faster).
|
Copyright © 1999-2021 by the D Language Foundation