View mode: basic / threaded / horizontal-split · Log in · Help
December 01, 2011
std.json dynamic initialization of JSONValue
I'm finding std.json extremely well written, with one glaring exception.

I can't seem to figure out how to do this:

JSONValue root = JSONValue(null, JSON_TYPE.OBJECT);
root.object["first_object"] = JSONValue(null, JSON_TYPE.OBJECT);
root.object["first_string"] = JSONValue("first_string", JSON_TYPE.STRING);

which would decode to:

{"first_object":{},"first_string":"first_string"}

What I end up having to do is:
JSONValue root;
root.type = JSON_TYPE.OBJECT;
root.object["first_object"] = JSONValue();
root.object["first_object"].type = JSON_TYPE.OBJECT;
root.object["first_string"] = JSON_Value();
root.object["first_string"].type = JSON_TYPE.STRING;
root.object["first_string"].str = "first_string";

That just feels like I'm doing it wrong. Is there a way to dynamically 
initialize a JSONValue struct? If I try to intialize the JSONValue 
object with anything other than simply null, or empty string, I either 
get a compile error or a segfault at run-time.

root.object["first_object"] = JSONValue(null, JSON_TYPE.OBJECT);

compile error:
Error: overlapping initialization for integer

root.object["first_string"] = JSONValue("first_string");
run-time segfault.

Any ideas?
March 31, 2012
Re: std.json dynamic initialization of JSONValue
On 12/01/2011 02:45 PM, Kai Meyer wrote:
> I'm finding std.json extremely well written, with one glaring exception.
>
> I can't seem to figure out how to do this:
>
> JSONValue root = JSONValue(null, JSON_TYPE.OBJECT);
> root.object["first_object"] = JSONValue(null, JSON_TYPE.OBJECT);
> root.object["first_string"] = JSONValue("first_string", 
JSON_TYPE.STRING);
>
> which would decode to:
>
> {"first_object":{},"first_string":"first_string"}
>
> What I end up having to do is:
> JSONValue root;
> root.type = JSON_TYPE.OBJECT;
> root.object["first_object"] = JSONValue();
> root.object["first_object"].type = JSON_TYPE.OBJECT;
> root.object["first_string"] = JSON_Value();
> root.object["first_string"].type = JSON_TYPE.STRING;
> root.object["first_string"].str = "first_string";
>
> That just feels like I'm doing it wrong. Is there a way to dynamically
> initialize a JSONValue struct?

std.json doesn't provide help with constructing different JSON values.

> If I try to intialize the JSONValue
> object with anything other than simply null, or empty string, I either
> get a compile error or a segfault at run-time.

That's because JSONValue has an anonymous union, which can only be 
initialized by their first member, and that happens to be 'str' for 
JSONValue:

  http://dlang.org/struct.html

<quote>
If there are anonymous unions in the struct, only the first member of 
the anonymous union can be initialized with a struct literal, and all 
subsequent non-overlapping fields are default initialized.
</quote>

> root.object["first_object"] = JSONValue(null, JSON_TYPE.OBJECT);
>
> compile error:
> Error: overlapping initialization for integer
>
> root.object["first_string"] = JSONValue("first_string");
> run-time segfault.
>
> Any ideas?

std.json have been discussed recently at a different forum.[1] There, I 
have come up with the following code which simplifies constructing JSON 
objects by defining to!JSONValue():

import std.stdio;
import std.conv;
import std.json;
import std.traits;
import std.exception;
import std.string;

/* Observation: Neither to() is a function of this module, nor
 * JSONValue is its type. Is such a function template that combines
 * the two legitimate? Yes modules bring namespaces, so name
 * collisions can be avoided by fully qualifying names, it still feels
 * like std.json should provide this function template.
 *
 * (Aside: Although C++ forbids defining functions in the std
 * namespace, it is sometimes necessary to define the << operator for
 * std::pair of user types.)
 *
 * BUG: This function is lacks supports for associative arrays.
 */
JSONValue to(Target : JSONValue, T)(T value)
{
    JSONValue json;

    static if (isSomeString!T) {
        json.type = JSON_TYPE.STRING;
        json.str = std.conv.to!string(value);

    } else static if (is (T : long)) {
        static if (is (T == ulong)) {
            /* Because std.json uses the long type for JSON
             * 'INTEGER's, we protect against data loss with ulong
             * values. */
            enforce(value <= long.max,
                    format("Loss of data: %s value %s cannot be"
                           " represented as long!",
                           T.stringof, value));
        }

        json.type = JSON_TYPE.INTEGER;
        json.integer = value;

    } else static if (is (T : real)) {
        json.type = JSON_TYPE.FLOAT;
        json.floating = value;

    } else static if (isArray!T) {
        json.type = JSON_TYPE.ARRAY;

        foreach (eleman; value) {
            json.array ~= to!JSONValue(eleman);
        }

    } else static if (__traits(compiles, cast(JSONValue)value)) {
        json = cast(JSONValue)(value);

    } else {
        static assert(false,
                      "Cannot convert this type to JSONValue: "
                      ~ T.stringof);
    }

    return json;
}

unittest
{
    import std.typetuple;

    /* Test that we are protected against data loss. */
    bool isThrown = false;
    try {
        to!JSONValue(ulong.max);

    } catch (Exception) {
        isThrown = true;
    }
    enforce(isThrown, "Exception for ulong.max has not been thrown!");

    /* These types must be supported by to!JSONValue(). */
    alias TypeTuple!(
        byte, ubyte, short, ushort, int, uint, long, ulong,
        float, double, real,
        string, wstring, dstring, char[], wchar[], dchar[],
        int[])
        Types;

    foreach (Type; Types) {
        to!JSONValue(Type.init);
    }
}

struct Student
{
    string name;
    ulong id;
    uint[] grades;

    JSONValue opCast(T : JSONValue)() const @property
    {
        /*
         * TODO: It should be possible to simplify this function by
         * taking advantage of __traits(allMembers) and perhaps
         * mixin() by excluding the member functions by isCallable().
         *
         * A question remains: What if one of the members is of a type
         * that defines the opCall() operator? Would isCallable()
         * produce true for that data member as well and exclude it
         * from JSON representation?
         */
        JSONValue[string] members;
        members["name"] = to!JSONValue(name);
        members["id"] = to!JSONValue(id);
        members["grades"] = to!JSONValue(grades);

        JSONValue json;
        json.object = members;
        json.type = JSON_TYPE.OBJECT;
        return json;
    }
}

JSONValue JSONRoot()
{
    JSONValue json;
    json.type = JSON_TYPE.OBJECT;
    return json;
}

void main()
{
    auto students = [ Student("Ayşe", 12, [ 90, 100 ]),
                      Student("Başak", 34, [ 95, 99 ]) ];

    JSONValue root = JSONRoot();
    root.object["students"] = to!JSONValue(students);

    writeln(toJSON(&root));
}

Ali

[1] Ddili Turkish forum: http://ddili.org/forum/forum
March 31, 2012
Re: std.json dynamic initialization of JSONValue
On 12/1/11, Kai Meyer <kai@unixlords.com> wrote:
> I'm finding std.json extremely well written, with one glaring exception.

I'm finding it to be crap. The last time I used it I just kept getting
access violations (or was that std.xml? They're both crap when I used
them.). ae.json beats its pants off for its simplicity + you get
toJson/jsonParse for serialization and a way to skip serializing
fields since a recent commit . It's easy to write your own
tree-walking routines as well.

But whatever works for people. :)
March 31, 2012
Re: std.json dynamic initialization of JSONValue
Andrej Mitrovic wrote:
> On 12/1/11, Kai Meyer<kai@unixlords.com>  wrote:
>> I'm finding std.json extremely well written, with one glaring exception.
>
> I'm finding it to be crap. The last time I used it I just kept getting
> access violations (or was that std.xml? They're both crap when I used
> them.). ae.json beats its pants off for its simplicity + you get
> toJson/jsonParse for serialization and a way to skip serializing
> fields since a recent commit . It's easy to write your own
> tree-walking routines as well.
>
> But whatever works for people. :)

I have written streaming json parser using ranges. It returns slices 
when possible. Benchmarked it and it's about 2.05x the speed of std.json.

It gives possibility to "dig" into the structure and stream (using 
ranges) by member fields, array elements, or characters of field names 
and string values. It's possible to parse JSON without a single 
allocation. For convenience, one can get objects, arrays and strings as 
a whole.

I plan to add a streaming json writer and release it (currently it 
outputs json using toString()). I've seen questions on stackoverflow 
about parsing 500 MB JSON... so streaming feature makes it really 
universal. This approach should be good for XML parser too.

Currently, I don't have time to improve it. But if someone is interested 
I can send it as is :-)
April 01, 2012
Re: std.json dynamic initialization of JSONValue
On Sat, Mar 31, 2012 at 12:25 PM, Piotr Szturmaj <bncrbme@jadamspam.pl> wrote:
> Andrej Mitrovic wrote:
>>
>> On 12/1/11, Kai Meyer<kai@unixlords.com>  wrote:
>>>
>>> I'm finding std.json extremely well written, with one glaring exception.
>>
>>
>> I'm finding it to be crap. The last time I used it I just kept getting
>> access violations (or was that std.xml? They're both crap when I used
>> them.). ae.json beats its pants off for its simplicity + you get
>> toJson/jsonParse for serialization and a way to skip serializing
>> fields since a recent commit . It's easy to write your own
>> tree-walking routines as well.
>>
>> But whatever works for people. :)
>
>
> I have written streaming json parser using ranges. It returns slices when
> possible. Benchmarked it and it's about 2.05x the speed of std.json.
>
> It gives possibility to "dig" into the structure and stream (using ranges)
> by member fields, array elements, or characters of field names and string
> values. It's possible to parse JSON without a single allocation. For
> convenience, one can get objects, arrays and strings as a whole.
>
> I plan to add a streaming json writer and release it (currently it outputs
> json using toString()). I've seen questions on stackoverflow about parsing
> 500 MB JSON... so streaming feature makes it really universal. This approach
> should be good for XML parser too.
>
> Currently, I don't have time to improve it. But if someone is interested I
> can send it as is :-)

I'm very interested in your json lib. I just started writing my own
but you have more advanced stuff already so I'd better build something
on top of your work. Is it on a repository somewhere on the web?
April 01, 2012
Re: std.json dynamic initialization of JSONValue
Nicolas Silva wrote:
> On Sat, Mar 31, 2012 at 12:25 PM, Piotr Szturmaj<bncrbme@jadamspam.pl>  wrote:
>> I have written streaming json parser using ranges. It returns slices when
>> possible. Benchmarked it and it's about 2.05x the speed of std.json.
>>
>> It gives possibility to "dig" into the structure and stream (using ranges)
>> by member fields, array elements, or characters of field names and string
>> values. It's possible to parse JSON without a single allocation. For
>> convenience, one can get objects, arrays and strings as a whole.
>>
>> I plan to add a streaming json writer and release it (currently it outputs
>> json using toString()). I've seen questions on stackoverflow about parsing
>> 500 MB JSON... so streaming feature makes it really universal. This approach
>> should be good for XML parser too.
>>
>> Currently, I don't have time to improve it. But if someone is interested I
>> can send it as is :-)
>
> I'm very interested in your json lib. I just started writing my own
> but you have more advanced stuff already so I'd better build something
> on top of your work. Is it on a repository somewhere on the web?

I just uploaded it here: 
https://github.com/pszturmaj/json-streaming-parser. Hope you like it :-)
Top | Discussion index | About this forum | D home