Thread overview
Optimising JSON parsing leads to wierd execution timings
Mar 25, 2017
xtreak
Mar 27, 2017
Seb
Mar 29, 2017
Laeeth Isharc
March 25, 2017
Hi,

Sorry for the double post. I have asked a question at Stackoverflow regarding this : https://stackoverflow.com/questions/42992507/get-float-value-out-of-jsonvalue-in-dlang . I have a `rating` field that might have 3 which parses to JSONValue.integer or 3.4 which parses to JSONValue.floating and I need to use float. I was using exceptions that was causing the program to be slow as per the profile information. Now I have used an if clause as suggested by Adam in the answer. Though the execution time for the relevant function that used exceptions earlier reduced the program was still taking the same time.

So I profiled again and now the time for JSON parsing has suddenly increased as per the profile. I thought it was due to using .type and I returned a hardcoded number for the function get_number and the time to write to terminal increased. I don't know why reducing the time on one function leads to increase in another function.

JSON file : gist.github.com/tirkarthi/c4b2ea745f15dbc6a01327b86c1d059f
Compiler : ldc2 with -O3 and -release flags
LLVM : LLVM 4.0
Operating System : Mac OS (Early 2015 model) with 4GB RAM

Original program with exceptions :

import std.net.curl;
import std.exception;
import std.json;
import std.stdio;
import std.file;
import std.algorithm;
import std.array;
import std.conv;
import std.string;

auto fmt = "%35s | %10s | %10s | %10s | %15s";
auto value_fmt = "%35s | %10.2f | %10.0f | %10.2f | %15s";

void print_header() {
  writeln("-----------------------------------------------------------------------------------------------");
  writefln(fmt, "Name", "Price", "Window", "Rating", "Type");
  writeln("-----------------------------------------------------------------------------------------------");
}

float get_number(T)(T item) {
  float output = 0;
  try {
    output = to!float(item.integer);
  } catch(Exception e) {
    output = to!float(item.floating);
  }
  return to!float(output);
}

void print_item(T)(T item) {
  writefln(value_fmt, item["Tvs"].str, item["MinFare"].get_number(), item["WnSt"].get_number(), item["Rtg"]["totRt"].get_number(), item["BusCategory"]["IsSleeper"]);
}

void main() {
  auto content = readText("sample.json");
  auto parsed = parseJSON(content);
  auto raw = parsed["SRD"][0]["RIN"][0]["InvList"].array;
  auto filtered = raw.filter!(item => to!bool(to!string(item["BusCategory"]["IsAc"])));

  print_header();
  foreach(item; filtered) {
    print_item(item);
  }
}
March 27, 2017
On Saturday, 25 March 2017 at 06:53:58 UTC, xtreak wrote:
> Hi,
>
> Sorry for the double post. I have asked a question at Stackoverflow regarding this : https://stackoverflow.com/questions/42992507/get-float-value-out-of-jsonvalue-in-dlang . I have a `rating` field that might have 3 which parses to JSONValue.integer or 3.4 which parses to JSONValue.floating and I need to use float. I was using exceptions that was causing the program to be slow as per the profile information. Now I have used an if clause as suggested by Adam in the answer. Though the execution time for the relevant function that used exceptions earlier reduced the program was still taking the same time.
>
> [...]

If you need a fast JSON library, have a look at https://github.com/tamediadigital/asdf or https://github.com/s-ludwig/std_data_json
March 29, 2017
On Monday, 27 March 2017 at 09:05:00 UTC, Seb wrote:
> On Saturday, 25 March 2017 at 06:53:58 UTC, xtreak wrote:
>> Hi,
>>
>> Sorry for the double post. I have asked a question at Stackoverflow regarding this : https://stackoverflow.com/questions/42992507/get-float-value-out-of-jsonvalue-in-dlang . I have a `rating` field that might have 3 which parses to JSONValue.integer or 3.4 which parses to JSONValue.floating and I need to use float. I was using exceptions that was causing the program to be slow as per the profile information. Now I have used an if clause as suggested by Adam in the answer. Though the execution time for the relevant function that used exceptions earlier reduced the program was still taking the same time.
>>
>> [...]
>
> If you need a fast JSON library, have a look at https://github.com/tamediadigital/asdf or https://github.com/s-ludwig/std_data_json

Or search forum for fastest Json parser in the world.  I use asdf though, which is fast enough and has a nice interface.