Reply to grauzone,

> BCS wrote:
> 
>> the latest and greatest:
>> 
>> http://arrayboundserror.blogspot.com/2009/06/serialization-for-d-part
>> -6-of-n.html
>> 
>> This time I'm hoping for some feedback on how people want to
>> interface with 3rd party types.
>> 
> Is there any real reason for all those mixins?
> 

which ones?

>> Is there any real reason for all those mixins?
>>
> 
> which ones?

All used by the user. That would be Serializable and SerializableRecuring.

Reply to grauzone,

>>> Is there any real reason for all those mixins?
>>> 
>> which ones?
>> 
> All used by the user. That would be Serializable and
> SerializableRecuring.
> 

What else would you use? I guess if I really wanted to I could use the same device as for 3rd party types, but that just pushes the mixin to global scope (the functions have to be defined in the same module as the type or they won't be able to access private members) and introduces unneeded performance issues from using function pointers.

If you can spot a better solution, I'm all for it, but I don't see one and don't think this solution is to bad.

BCS wrote:
> Reply to grauzone,
> 
>>>> Is there any real reason for all those mixins?
>>>>
>>> which ones?
>>>
>> All used by the user. That would be Serializable and
>> SerializableRecuring.
>>
> 
> What else would you use? I guess if I really wanted to I could use the same device as for 3rd party types, but that just pushes the mixin to global scope (the functions have to be defined in the same module as the type or they won't be able to access private members) and introduces unneeded performance issues from using function pointers.
> 
> If you can spot a better solution, I'm all for it, but I don't see one and don't think this solution is to bad.

tupleof _can_ access private members, even if the type is from a different module than the function that uses the tupleof. This changed somewhere between dmd 1.030 and 1.040, I think.

Reply to grauzone,

> BCS wrote:
> 
>> Reply to grauzone,
>> 
>>>>> Is there any real reason for all those mixins?
>>>>> 
>>>> which ones?
>>>> 
>>> All used by the user. That would be Serializable and
>>> SerializableRecuring.
>>> 
>> What else would you use? I guess if I really wanted to I could use
>> the same device as for 3rd party types, but that just pushes the
>> mixin to global scope (the functions have to be defined in the same
>> module as the type or they won't be able to access private members)
>> and introduces unneeded performance issues from using function
>> pointers.
>> 
>> If you can spot a better solution, I'm all for it, but I don't see
>> one and don't think this solution is to bad.
>> 
> tupleof _can_ access private members, even if the type is from a
> different module than the function that uses the tupleof. This changed
> somewhere between dmd 1.030 and 1.040, I think.

If it can, I would consider that a bug.

And even if Walter sanctions it, I wont use it because I'm very leery of generating a system that automaticly reaches in from the outside and mucks with private members. Whoever makes a type serializeable needs to put at least some thought into how to serialize it. If it is the type's author, I'm comfortable with them saying "just do everything", but if someone else is doing it, that is likely to get feet shot off. If I made a generic "any type" function, it would recur into any referenced types and either spew a big list of type that it's dealing with (that people won't look at) or give no warning that it's walking into new territory and that it may not be doing valid operations.

grauzone wrote:
>>> Is there any real reason for all those mixins?
>>>
>>
>> which ones?
> 
> All used by the user. That would be Serializable and SerializableRecuring.

Also, what is curing in this context, and why would you need to do it multiple times?

Hello Christopher,

>> SerializableRecuring.
> 
> Also, what is curing in this context, and why would you need to do it
> multiple times?
> 

My spelling sucks: s/Recuring/Recurring/

>> tupleof _can_ access private members, even if the type is from a
>> different module than the function that uses the tupleof. This changed
>> somewhere between dmd 1.030 and 1.040, I think.
> 
> If it can, I would consider that a bug.

It would be nice to get an official statement.

> And even if Walter sanctions it, I wont use it because I'm very leery of generating a system that automaticly reaches in from the outside and mucks with private members. Whoever makes a type serializeable needs to put at least some thought into how to serialize it. If it is the type's 

That's why I'd still require types to be marked as serializeable by the programmer. But using mixins and adding unknown members from the serialization machinery isn't that clean either. At leasr for all those structs, it's truly annoying and unneeded. Having to use different mixins for structs/classes sucks even more (wouldn't be required if you wanted).

By the way, how are you going to solve the other problems when serializing D types? Here are some troublesome types:
- pointers?
- function pointers?
- delegates?
- unions?
- deserializing members that are const/immutable?
- referential integrity across arrays and slices?

Also, it seems I just spotted a bug: floating point numbers loose precision as they are formatted with "%s".

> author, I'm comfortable with them saying "just do everything", but if someone else is doing it, that is likely to get feet shot off. If I made a generic "any type" function, it would recur into any referenced types and either spew a big list of type that it's dealing with (that people won't look at) or give no warning that it's walking into new territory and that it may not be doing valid operations.

I didn't understand the last sentence.

Hello grauzone,


>> And even if Walter sanctions it, I wont use it because I'm very leery
>> of generating a system that automaticly reaches in from the outside
>> and mucks with private members. Whoever makes a type serializeable
>> needs to put at least some thought into how to serialize it. If it is
>> the type's
>> 
> That's why I'd still require types to be marked as serializeable by
> the programmer.

How would you do that aside from mixins?

> But using mixins and adding unknown members from the
> serialization machinery isn't that clean either.

I'm not going to fix it now, but I have a few ideas how I can only add the public API to the types. That would be one non static serialization function and a static deserialization function.

> At least for all
> those structs, it's truly annoying and unneeded. Having to use
> different mixins for structs/classes sucks even more (wouldn't be
> required if you wanted).

It isn't required, the difference is not struct vs. class but "do you care about repeated references to instances of this type?"

> 
> By the way, how are you going to solve the other problems when
> serializing D types? Here are some troublesome types:
> - pointers?

class references are pointers, for struct and what not, I'll just use a similar approach

> - function pointers?
> - delegates?

I won't

> - unions?

for those I'll provide a way for the user to define the serialization themselves.

> - deserializing members that are const/immutable?

D1, when I get to that; the same way you generated them anywhere else? I might even go with "unsafe" casts as I can be sure to get that correct.

> - referential integrity across arrays and slices?

I won't

> Also, it seems I just spotted a bug: floating point numbers loose
> precision as they are formatted with "%s".

I suppose that should be "%A" I'll get to that at some point.

> 
>> author, I'm comfortable with them saying "just do everything", but if
>> someone else is doing it, that is likely to get feet shot off. If I
>> made a generic "any type" function, it would recur into any
>> referenced types and either spew a big list of type that it's dealing
>> with (that people won't look at) or give no warning that it's walking
>> into new territory and that it may not be doing valid operations.
>> 
> I didn't understand the last sentence.
> 

First off there is not enough information to correctly generate sterilizers for types. So the user has to do some thinking for most types. How do I make the user thinks about how and for what types code is being generated?

Option 1) have the template run a paramg(msg,T.stringof) for each type and hope the user don't just ignore the list.
Option 2) do nothing and be sure the user won't see anything.
Option 3) Make the user write per type code and error when they don't.

BCS wrote:
>> That's why I'd still require types to be marked as serializeable by
>> the programmer.
> 
> How would you do that aside from mixins?

Make the user implement a marker interface, or let him provide a (single) special class member which fulfill the same function, or introduce annotations into the language. As far as marking goes, a mixin would be OK too, but as I said, I don't like adding arbitrary members into the user's scope without the user knowing.

>> At least for all
>> those structs, it's truly annoying and unneeded. Having to use
>> different mixins for structs/classes sucks even more (wouldn't be
>> required if you wanted).
> 
> It isn't required, the difference is not struct vs. class but "do you care about repeated references to instances of this type?"

IMHO a relatively useless optimization, that will only confuse the user. It will introduce subtle bugs if objects accidentally are "repeated" instead of serialized only once.

>> By the way, how are you going to solve the other problems when
>> serializing D types? Here are some troublesome types:
>> - pointers?
> 
> class references are pointers, for struct and what not, I'll just use a similar approach

That's not really the problem here. You have no idea where a pointer points to. Is it a pointer to a new'ed memory block? Does it point into an object? Does it point into an array? Into the data segment?

The GC provides no API to find out. You may be able to handle some cases above, but not all.

>> - function pointers?
>> - delegates?
> 
> I won't

Forces the user to use interfaces instead of delegates.

>> - unions?
> 
> for those I'll provide a way for the user to define the serialization themselves.
> 
>> - deserializing members that are const/immutable?
> 
> D1, when I get to that; the same way you generated them anywhere else? I might even go with "unsafe" casts as I can be sure to get that correct.

In D1, it comes for free: you simply don't need to handle it.
But yes, it looks like you have to use dirty tricks in D2.

>> - referential integrity across arrays and slices?
> 
> I won't

I wonder if anyone (I mean, especially user of a serialization library) would disagree with that choice. Sure, there are valid D programs that would break with this, but is relying on this really clean?

>>> author, I'm comfortable with them saying "just do everything", but if
>>> someone else is doing it, that is likely to get feet shot off. If I
>>> made a generic "any type" function, it would recur into any
>>> referenced types and either spew a big list of type that it's dealing
>>> with (that people won't look at) or give no warning that it's walking
>>> into new territory and that it may not be doing valid operations.
>>>
>> I didn't understand the last sentence.
>>
> 
> First off there is not enough information to correctly generate sterilizers for types. So the user has to do some thinking for most types. How do I make the user thinks about how and for what types code is being generated?
> 
> Option 1) have the template run a paramg(msg,T.stringof) for each type and hope the user don't just ignore the list.
> Option 2) do nothing and be sure the user won't see anything.
> Option 3) Make the user write per type code and error when they don't.

Oh, you mean if there are types in the object graphs the serializer can't deal with it? But how would option 2) work?