October 15, 2015 Re: Fastest JSON parser in the world is a D project | ||||
---|---|---|---|---|
| ||||
Posted in reply to Daniel Kozak | Daniel Kozak via Digitalmars-d-announce píše v Čt 15. 10. 2015 v 11:07 +0200: > > > Gary Willoughby via Digitalmars-d-announce <digitalmars-d-announce@pu remagic.com> napsal Čt, říj 15, 2015 v 10∶08 : > > On Wednesday, 14 October 2015 at 07:01:49 UTC, Marco Leise wrote: > > fast: 0.34s, 226.7Mb (GDC) > > RapidJSON: 0.79s, 687.1Mb (GCC) > > > > (* Timings from my computer, Haswell CPU, Linux amd64.) > > > > Where's the code? > code.dlang.org https://github.com/mleise/fast |
October 15, 2015 Re: Fastest JSON parser in the world is a D project | ||||
---|---|---|---|---|
| ||||
Posted in reply to Marco Leise | On Wednesday, 14 October 2015 at 07:01:49 UTC, Marco Leise wrote:
> fast: 0.34s, 226.7Mb (GDC)
> RapidJSON: 0.79s, 687.1Mb (GCC)
Why not add this to std.experimental?
|
October 15, 2015 Re: Fastest JSON parser in the world is a D project | ||||
---|---|---|---|---|
| ||||
Posted in reply to Per Nordlöw | On 10/15/15 12:40 PM, Per Nordlöw wrote:
> On Wednesday, 14 October 2015 at 07:01:49 UTC, Marco Leise wrote:
>> fast: 0.34s, 226.7Mb (GDC)
>> RapidJSON: 0.79s, 687.1Mb (GCC)
>
> Why not add this to std.experimental?
Sure seems like a good question! At the least a more generic generalization (more character and range types etc) should start from Marco's core implementation. -- Andrei
|
October 15, 2015 Re: Fastest JSON parser in the world is a D project | ||||
---|---|---|---|---|
| ||||
Posted in reply to Marco Leise Attachments:
| In browser JSON.serialize is the usual way to serialize JSON values. The problem is that on D side if one does deserialization of an object or struct. If the types inside the JSON don't match exactly then vibe freaks out. Another problem with most D JSON implementations is that they don't support proper JSON, e.g. outputting nan as though it was a valid value etc... browsers don't like that stuff. For me the best D JSON implementation at the moment is actually jsvar by Adam and its not even a JSON parser, it just has that as a needed feature. It feels slow though I haven't benchmarked, but if I run it over a couple of Gigs of data(Paged by 1000) it takes a long while. On Thu, Oct 15, 2015 at 3:42 AM, Marco Leise via Digitalmars-d-announce < digitalmars-d-announce@puremagic.com> wrote: > Am Wed, 14 Oct 2015 10:22:37 +0200 > schrieb Rory McGuire via Digitalmars-d-announce > <digitalmars-d-announce@puremagic.com>: > > > Does this version handle real world JSON? > > > > I've keep getting problems with vibe and JSON because web browsers will automatically make a "1" into a 1 which then causes exceptions in vibe. > > > > Does yours do lossless conversions automatically? > > No I don't read numbers as strings. Could the client JavaScript be fixed? I fail to see why the conversion would happen automatically when the code could explicitly check for strings before doing math with the value "1". What do I miss? > > -- > Marco > > |
October 15, 2015 Re: Fastest JSON parser in the world is a D project | ||||
---|---|---|---|---|
| ||||
Posted in reply to Daniel Kozak | Am Thu, 15 Oct 2015 11:09:01 +0200 schrieb Daniel Kozak <kozzi@dlang.cz>: > Daniel Kozak via Digitalmars-d-announce píše v Čt 15. 10. 2015 v 11:07 +0200: > > > > > > Gary Willoughby via Digitalmars-d-announce <digitalmars-d-announce@pu remagic.com> napsal Čt, říj 15, 2015 v 10∶08 : > > > On Wednesday, 14 October 2015 at 07:01:49 UTC, Marco Leise wrote: > > > fast: 0.34s, 226.7Mb (GDC) > > > RapidJSON: 0.79s, 687.1Mb (GCC) > > > > > > (* Timings from my computer, Haswell CPU, Linux amd64.) > > > > > > Where's the code? > > code.dlang.org > > https://github.com/mleise/fast BTW: Is there a reason why the code is GPL licensed? I understand that people might want to use more restrictive licenses, but isn't LGPL a better replacement for GPL when writing library code? Doesn't the GPL force everybody _using_ fast.json to also use the GPL license? See: http://stackoverflow.com/a/10179181/471401 |
October 15, 2015 Re: Fastest JSON parser in the world is a D project | ||||
---|---|---|---|---|
| ||||
Posted in reply to Johannes Pfau | On Thursday, 15 October 2015 at 12:51:58 UTC, Johannes Pfau wrote:
> BTW: Is there a reason why the code is GPL licensed? I understand that people might want to use more restrictive licenses, but isn't LGPL a better replacement for GPL when writing library code? Doesn't the GPL force everybody _using_ fast.json to also use the GPL license?
>
> See: http://stackoverflow.com/a/10179181/471401
It also precludes any of this code being used in Phobos :/
|
October 15, 2015 Re: Fastest JSON parser in the world is a D project | ||||
---|---|---|---|---|
| ||||
Posted in reply to Rory McGuire | Am 15.10.2015 um 13:06 schrieb Rory McGuire via Digitalmars-d-announce: > In browser JSON.serialize is the usual way to serialize JSON values. > The problem is that on D side if one does deserialization of an object > or struct. If the types inside the JSON don't match exactly then vibe > freaks out. For float and double fields, the serialization code should actually accept both, floating point and integer numbers: https://github.com/rejectedsoftware/vibe.d/blob/2fffd94d8516cd6f81c75d45a54c655626d36c6b/source/vibe/data/json.d#L1603 https://github.com/rejectedsoftware/vibe.d/blob/2fffd94d8516cd6f81c75d45a54c655626d36c6b/source/vibe/data/json.d#L1804 Do you have a test case for your error? |
October 15, 2015 Re: Fastest JSON parser in the world is a D project | ||||
---|---|---|---|---|
| ||||
Posted in reply to Marco Leise | Am 14.10.2015 um 09:01 schrieb Marco Leise: > JSON parsing in D has come a long way, especially when you > look at it from the efficiency angle as a popular benchmark > does that has been forked by well known D contributers like > Martin Nowak or Sönke Ludwig. > > The test is pretty simple: Parse a JSON object, containing an > array of 1_000_000 3D coordinates in the range [0..1) and > average them. > > The performance of std.json in parsing those was horrible > still in the DMD 2.066 days*: > > DMD : 41.44s, 934.9Mb > Gdc : 29.64s, 929.7Mb > Python : 12.30s, 1410.2Mb > Ruby : 13.80s, 2101.2Mb > > Then with 2.067 std.json got a major 3x speed improvement and > rivaled the popular dynamic languages Ruby and Python: > > DMD : 13.02s, 1324.2Mb > > In the mean time several other D JSON libraries appeared with > varying focus on performance or API: > > Medea : 56.75s, 1753.6Mb (GDC) > libdjson : 24.47s, 1060.7Mb (GDC) > stdx.data.json: 2.76s, 207.1Mb (LDC) > > Yep, that's right. stdx.data.json's pull parser finally beats > the dynamic languages with native efficiency. (I used the > default options here that provide you with an Exception and > line number on errors.) > From when are the numbers for stdx.data.json? The latest results for the pull parser that I know of were faster than RapidJson: http://forum.dlang.org/post/wlczkjcawyteowjbbcpo@forum.dlang.org Judging by the relative numbers, your "fast" result is still a bit faster, but if that doesn't validate, it's hard to make an objective comparison. |
October 15, 2015 Re: Fastest JSON parser in the world is a D project | ||||
---|---|---|---|---|
| ||||
Posted in reply to Andrei Alexandrescu | On Thursday, 15 October 2015 at 10:34:16 UTC, Andrei Alexandrescu wrote:
> On 10/15/15 12:40 PM, Per Nordlöw wrote:
>> On Wednesday, 14 October 2015 at 07:01:49 UTC, Marco Leise wrote:
>>> fast: 0.34s, 226.7Mb (GDC)
>>> RapidJSON: 0.79s, 687.1Mb (GCC)
>>
>> Why not add this to std.experimental?
>
> Sure seems like a good question! At the least a more generic generalization (more character and range types etc) should start from Marco's core implementation. -- Andrei
Would it not be a better use of effort to attempt to merge the efforts here over to Sonkes new stdx.json? I didn't look at either codebase, so I dont know how difficult that'll be.
|
October 15, 2015 Re: Fastest JSON parser in the world is a D project | ||||
---|---|---|---|---|
| ||||
Posted in reply to Johannes Pfau | On 2015-10-15 14:51, Johannes Pfau wrote: > Doesn't the GPL force everybody _using_ fast.json to also use the GPL license? Yes, it does have that enforcement. -- /Jacob Carlborg |
Copyright © 1999-2021 by the D Language Foundation