May 26, 2009
On Tue, May 26, 2009 at 2:35 AM, Denis Koroskin <2korden@gmail.com> wrote:
> On Tue, 26 May 2009 13:29:39 +0400, Qian Xu <quian.xu@stud.tu-ilmenau.de> wrote:
>
>> BCS wrote:
>>
>>> I'm planning on taking a crack at a Serialization template library and
>>> I'm
>>> looking for feed back. My thinking so far is up on my blog here:
>>>
>>>
>>
>> http://arrayboundserror.blogspot.com/2009/05/serialization-for-d-part-1-of-n.html
>>>
>>> Please comment! (here or there, doesn't matter, I think I'll see both)
>>
>> A question:
>> Should every object contain "mixin Serializable!()" in its declaration?
>> It is easy to add this in own classes, but not easy to 3rd-party
>> libraries.
>> for instance: tango
>>
>>
>> --Qian
>>
>
> Good serialization library supports external serialization via template
> specialization (or similar tricks) :)

I'll also add that you should be able to properly
serialize/deserialize a BaseClass pointer that actually points to a
DerivedClass.
This is pretty tricky to get working seamlessly when combined with the
external serialization requirement.
H3r3tic's xpose library has this working IIRC.

--bb
May 26, 2009
On Tue, 26 May 2009 21:44:49 +0400, Bill Baxter <wbaxter@gmail.com> wrote:

> On Tue, May 26, 2009 at 2:35 AM, Denis Koroskin <2korden@gmail.com> wrote:
>> On Tue, 26 May 2009 13:29:39 +0400, Qian Xu
>> <quian.xu@stud.tu-ilmenau.de>
>> wrote:
>>
>>> BCS wrote:
>>>
>>>> I'm planning on taking a crack at a Serialization template library and
>>>> I'm
>>>> looking for feed back. My thinking so far is up on my blog here:
>>>>
>>>>
>>>
>>> http://arrayboundserror.blogspot.com/2009/05/serialization-for-d-part-1-of-n.html
>>>>
>>>> Please comment! (here or there, doesn't matter, I think I'll see both)
>>>
>>> A question:
>>> Should every object contain "mixin Serializable!()" in its declaration?
>>> It is easy to add this in own classes, but not easy to 3rd-party
>>> libraries.
>>> for instance: tango
>>>
>>>
>>> --Qian
>>>
>>
>> Good serialization library supports external serialization via template
>> specialization (or similar tricks) :)
>
> I'll also add that you should be able to properly
> serialize/deserialize a BaseClass pointer that actually points to a
> DerivedClass.
> This is pretty tricky to get working seamlessly when combined with the
> external serialization requirement.
> H3r3tic's xpose library has this working IIRC.
>
> --bb

Sure! How about the following test:

struct A
{
    mixin Serializable!();

    B[] b;
}

struct B
{
    mixin Serializable!();

    A a;
    int i;
}

B[] b = new B[1];
b[0].a.b = b;
b[0].i = 42;

A* a = &b[0].a;

data = serialize(a); // should serialize outer struct (B), too, because it is accessible through a

a = deserialize(data);
assert(a.b.length == 1);
assert(a.b[0].i == 42);

BTW, thanks for reminding about xpose. I gotta compare it, doost.serialize and mine own one against each other in terms of efficiency and size (for binary output) someday.
May 26, 2009
On Tue, May 26, 2009 at 12:19 PM, Denis Koroskin <2korden@gmail.com> wrote:
> On Tue, 26 May 2009 21:44:49 +0400, Bill Baxter <wbaxter@gmail.com> wrote:
>
>> On Tue, May 26, 2009 at 2:35 AM, Denis Koroskin <2korden@gmail.com> wrote:
>>> On Tue, 26 May 2009 13:29:39 +0400, Qian Xu
>>> <quian.xu@stud.tu-ilmenau.de>
>>> wrote:
>>>
>>>> BCS wrote:
>>>>
>>>>> I'm planning on taking a crack at a Serialization template library and
>>>>> I'm
>>>>> looking for feed back. My thinking so far is up on my blog here:
>>>>>
>>>>>
>>>>
>>>> http://arrayboundserror.blogspot.com/2009/05/serialization-for-d-part-1-of-n.html
>>>>>
>>>>> Please comment! (here or there, doesn't matter, I think I'll see both)
>>>>
>>>> A question:
>>>> Should every object contain "mixin Serializable!()" in its declaration?
>>>> It is easy to add this in own classes, but not easy to 3rd-party
>>>> libraries.
>>>> for instance: tango
>>>>
>>>>
>>>> --Qian
>>>>
>>>
>>> Good serialization library supports external serialization via template
>>> specialization (or similar tricks) :)
>>
>> I'll also add that you should be able to properly
>> serialize/deserialize a BaseClass pointer that actually points to a
>> DerivedClass.
>> This is pretty tricky to get working seamlessly when combined with the
>> external serialization requirement.
>> H3r3tic's xpose library has this working IIRC.
>>
>> --bb
>
> Sure! How about the following test:
>
> struct A
> {
>    mixin Serializable!();
>
>    B[] b;
> }
>
> struct B
> {
>    mixin Serializable!();
>
>    A a;
>    int i;
> }
>
> B[] b = new B[1];
> b[0].a.b = b;
> b[0].i = 42;
>
> A* a = &b[0].a;
>
> data = serialize(a); // should serialize outer struct (B), too, because it is accessible through a
>
> a = deserialize(data);
> assert(a.b.length == 1);
> assert(a.b[0].i == 42);
>
> BTW, thanks for reminding about xpose. I gotta compare it, doost.serialize and mine own one against each other in terms of efficiency and size (for binary output) someday.

That's not quite what I meant.

I mean this:

/// 3RD PARTY LIB
class BaseClass
{
     string toString() { return "base"; }
}

class DerivedClass
{
     string toString() { return "derived"; }
}


/// MY CODE

[...some kind of class registration here....]

BaseClass b = new DerivedClass;

data = serialize(b);
b2 = deserialize(data);

assert(b2.toString() == "derived");
------------------

The data stream should contain enough info about the instance to reconstruct the exact derived type.

--bb
May 26, 2009
On Wed, 27 May 2009 01:23:58 +0400, Bill Baxter <wbaxter@gmail.com> wrote:

> On Tue, May 26, 2009 at 12:19 PM, Denis Koroskin <2korden@gmail.com> wrote:
>> On Tue, 26 May 2009 21:44:49 +0400, Bill Baxter <wbaxter@gmail.com> wrote:
>>
>>> On Tue, May 26, 2009 at 2:35 AM, Denis Koroskin <2korden@gmail.com>
>>> wrote:
>>>> On Tue, 26 May 2009 13:29:39 +0400, Qian Xu
>>>> <quian.xu@stud.tu-ilmenau.de>
>>>> wrote:
>>>>
>>>>> BCS wrote:
>>>>>
>>>>>> I'm planning on taking a crack at a Serialization template library and
>>>>>> I'm
>>>>>> looking for feed back. My thinking so far is up on my blog here:
>>>>>>
>>>>>>
>>>>>
>>>>> http://arrayboundserror.blogspot.com/2009/05/serialization-for-d-part-1-of-n.html
>>>>>>
>>>>>> Please comment! (here or there, doesn't matter, I think I'll see both)
>>>>>
>>>>> A question:
>>>>> Should every object contain "mixin Serializable!()" in its declaration?
>>>>> It is easy to add this in own classes, but not easy to 3rd-party
>>>>> libraries.
>>>>> for instance: tango
>>>>>
>>>>>
>>>>> --Qian
>>>>>
>>>>
>>>> Good serialization library supports external serialization via template
>>>> specialization (or similar tricks) :)
>>>
>>> I'll also add that you should be able to properly
>>> serialize/deserialize a BaseClass pointer that actually points to a
>>> DerivedClass.
>>> This is pretty tricky to get working seamlessly when combined with the
>>> external serialization requirement.
>>> H3r3tic's xpose library has this working IIRC.
>>>
>>> --bb
>>
>> Sure! How about the following test:
>>
>> struct A
>> {
>>    mixin Serializable!();
>>
>>    B[] b;
>> }
>>
>> struct B
>> {
>>    mixin Serializable!();
>>
>>    A a;
>>    int i;
>> }
>>
>> B[] b = new B[1];
>> b[0].a.b = b;
>> b[0].i = 42;
>>
>> A* a = &b[0].a;
>>
>> data = serialize(a); // should serialize outer struct (B), too, because it is accessible through a
>>
>> a = deserialize(data);
>> assert(a.b.length == 1);
>> assert(a.b[0].i == 42);
>>
>> BTW, thanks for reminding about xpose. I gotta compare it, doost.serialize and mine own one against each other in terms of efficiency and size (for binary output) someday.
>
> That's not quite what I meant.
>
> I mean this:
>
> /// 3RD PARTY LIB
> class BaseClass
> {
>      string toString() { return "base"; }
> }
>
> class DerivedClass
> {
>      string toString() { return "derived"; }
> }
>
>
> /// MY CODE
>
> [...some kind of class registration here....]
>
> BaseClass b = new DerivedClass;
>
> data = serialize(b);
> b2 = deserialize(data);
>
> assert(b2.toString() == "derived");
> ------------------
>
> The data stream should contain enough info about the instance to
> reconstruct the exact derived type.
>
> --bb

Yes, I understood and showed yet another feature which is quite hard to implement.

May 26, 2009
On Tue, May 26, 2009 at 2:26 PM, Denis Koroskin <2korden@gmail.com> wrote:
> On Wed, 27 May 2009 01:23:58 +0400, Bill Baxter <wbaxter@gmail.com> wrote:
>
>> On Tue, May 26, 2009 at 12:19 PM, Denis Koroskin <2korden@gmail.com> wrote:
>>>
>>> On Tue, 26 May 2009 21:44:49 +0400, Bill Baxter <wbaxter@gmail.com> wrote:
>>>
>>>> On Tue, May 26, 2009 at 2:35 AM, Denis Koroskin <2korden@gmail.com> wrote:
>>>>>
>>>>> On Tue, 26 May 2009 13:29:39 +0400, Qian Xu
>>>>> <quian.xu@stud.tu-ilmenau.de>
>>>>> wrote:
>>>>>
>>>>>> BCS wrote:
>>>>>>
>>>>>>> I'm planning on taking a crack at a Serialization template library
>>>>>>> and
>>>>>>> I'm
>>>>>>> looking for feed back. My thinking so far is up on my blog here:
>>>>>>>
>>>>>>>
>>>>>>
>>>>>>
>>>>>> http://arrayboundserror.blogspot.com/2009/05/serialization-for-d-part-1-of-n.html
>>>>>>>
>>>>>>> Please comment! (here or there, doesn't matter, I think I'll see
>>>>>>> both)
>>>>>>
>>>>>> A question:
>>>>>> Should every object contain "mixin Serializable!()" in its
>>>>>> declaration?
>>>>>> It is easy to add this in own classes, but not easy to 3rd-party
>>>>>> libraries.
>>>>>> for instance: tango
>>>>>>
>>>>>>
>>>>>> --Qian
>>>>>>
>>>>>
>>>>> Good serialization library supports external serialization via template
>>>>> specialization (or similar tricks) :)
>>>>
>>>> I'll also add that you should be able to properly
>>>> serialize/deserialize a BaseClass pointer that actually points to a
>>>> DerivedClass.
>>>> This is pretty tricky to get working seamlessly when combined with the
>>>> external serialization requirement.
>>>> H3r3tic's xpose library has this working IIRC.
>>>>
>>>> --bb
>>>
>>> Sure! How about the following test:
>>>
>>> struct A
>>> {
>>>   mixin Serializable!();
>>>
>>>   B[] b;
>>> }
>>>
>>> struct B
>>> {
>>>   mixin Serializable!();
>>>
>>>   A a;
>>>   int i;
>>> }
>>>
>>> B[] b = new B[1];
>>> b[0].a.b = b;
>>> b[0].i = 42;
>>>
>>> A* a = &b[0].a;
>>>
>>> data = serialize(a); // should serialize outer struct (B), too, because
>>> it is accessible through a
>>>
>>> a = deserialize(data);
>>> assert(a.b.length == 1);
>>> assert(a.b[0].i == 42);
>>>
>>> BTW, thanks for reminding about xpose. I gotta compare it, doost.serialize and mine own one against each other in terms of efficiency and size (for binary output) someday.
>>
>> That's not quite what I meant.
>>
>> I mean this:
>>
>> /// 3RD PARTY LIB
>> class BaseClass
>> {
>>     string toString() { return "base"; }
>> }
>>
>> class DerivedClass
>> {
>>     string toString() { return "derived"; }
>> }
>>
>>
>> /// MY CODE
>>
>> [...some kind of class registration here....]
>>
>> BaseClass b = new DerivedClass;
>>
>> data = serialize(b);
>> b2 = deserialize(data);
>>
>> assert(b2.toString() == "derived");
>> ------------------
>>
>> The data stream should contain enough info about the instance to reconstruct the exact derived type.
>>
>> --bb
>
> Yes, I understood and showed yet another feature which is quite hard to implement.

Ok.  I interpreted that as "Sure! How about the following test (of the
feature you just mentioned)".
Some segue words like "in addition" or "also" probably would have
tipped me off that you were changing the subject. :-)

--bb
May 27, 2009
Hello Bill,

> I'll also add that you should be able to properly
> serialize/deserialize a BaseClass pointer that actually points to a
> DerivedClass.
> This is pretty tricky to get working seamlessly when combined with the
> external serialization requirement.
> H3r3tic's xpose library has this working IIRC.
> --bb


The current version of the code has this working. It was fairly high on my feature list.  


May 27, 2009
Hello Qian,

> BCS wrote:
> 
>> I'm planning on taking a crack at a Serialization template library
>> and I'm looking for feed back. My thinking so far is up on my blog
>> here:
>> 
> http://arrayboundserror.blogspot.com/2009/05/serialization-for-d-part-
> 1-of-n.html
> 
>> Please comment! (here or there, doesn't matter, I think I'll see
>> both)
>> 
> A question:
> Should every object contain "mixin Serializable!()" in its
> declaration? It is easy to add this in own classes, but
> not easy to 3rd-party libraries. for instance: tango
> --Qian
> 

I've been mentally dodging that question because it's going to be ugly. If anyone has some ideas on how to make it work, I'm interested.

Right now the system has a hard coded list of basic types (byte, real, etc.) that it can work with and failing that, it assume you have added "mixin Serializable!()". The main problem is getting the third case is overload. Overloading doesn't work across mixins or modules last I checked. It would be a hack, but named function called via string mixins (mixin("Marshal"~ typeof(a).stringof ~ "(sink,a);")) might work.


June 01, 2009
And yet more progress. This time I've got "hard-links" working and I'm thinking about how to do 3rd party types.

I don't have any useable ideas I really /like/, but I have one I'm sure I can make work. I do have one idea down at the bottom that I'd love to use but it would need some new language features.

http://arrayboundserror.blogspot.com/2009/05/serialization-for-d-part-5-of-n.html

As always: Comments, Please! (particularly on the "feature request")


June 11, 2009
the latest and greatest:

http://arrayboundserror.blogspot.com/2009/06/serialization-for-d-part-6-of-n.html

This time I'm hoping for some feedback on how people want to interface with 3rd party types.


June 11, 2009
BCS wrote:
> the latest and greatest:
> 
> http://arrayboundserror.blogspot.com/2009/06/serialization-for-d-part-6-of-n.html 
> 
> 
> This time I'm hoping for some feedback on how people want to interface with 3rd party types.
> 
> 

Is there any real reason for all those mixins?