On 2013-02-18 15:32, Steven Schveighoffer wrote:

> Then maybe I didn't understand deadalnix's point, or maybe you have
> solved his problem?  I don't know enough about your library.

I didn't really understand deadalnix's point. But I answered your comment. My answer to your comment is true but I don't know if that has anything to do with what deadalnix said.

> If there isn't a problem, then we don't need a solution.  But if there
> is a problem, we may need something like this.

I don't know, is there a problem?

-- 
/Jacob Carlborg

On Mon, 18 Feb 2013 10:09:30 -0500, Jacob Carlborg <doob@me.com> wrote:

> On 2013-02-18 15:32, Steven Schveighoffer wrote:
>
>> Then maybe I didn't understand deadalnix's point, or maybe you have
>> solved his problem?  I don't know enough about your library.
>
> I didn't really understand deadalnix's point. But I answered your comment. My answer to your comment is true but I don't know if that has anything to do with what deadalnix said.

Hm... maybe something like this:

class Widget
{
  Display d;
}

If you serialize Widget, then it will try to serialize d, but d is not owned by Widget, it's just referenced by Widget.

How does Orange deal with this?  What if whoever wrote Widget never intended it to be serialized, but wasn't aware of the serialization library, so didn't know to mark it as @nonSerialized

-Steve

On 2013-02-18 18:46, Steven Schveighoffer wrote:

> Hm... maybe something like this:
>
> class Widget
> {
>    Display d;
> }
>
> If you serialize Widget, then it will try to serialize d, but d is not
> owned by Widget, it's just referenced by Widget.
>
> How does Orange deal with this?

The first time Orange sees "d" it will just serialize it. If there's any other reference to "d", somewhere in the data structure that is being serialized, it will serialize a reference.

If "d" is not owned by Widget it will serialize it anyway. It doesn't care where it comes from.

This code:

http://pastebin.com/n1jwG9vE

Will serialize to this XML:

http://pastebin.com/tHbzVRqZ

> What if whoever wrote Widget never
> intended it to be serialized, but wasn't aware of the serialization
> library, so didn't know to mark it as @nonSerialized

I don't know if there is solution for that if the serialization library is using an opt-out approach. The one serializing the data structure is responsible.

-- 
/Jacob Carlborg

On Monday, 18 February 2013 at 01:27:17 UTC, Steven Schveighoffer wrote:
> On Sun, 17 Feb 2013 19:58:06 -0500, Nick Sabalausky <SeeWebsiteToContactMe@semitwist.com> wrote:
>
>> On Sun, 17 Feb 2013 13:18:05 -0800
>> Walter Bright <newshound2@digitalmars.com> wrote:
>>
>>> On 2/17/2013 12:51 PM, Jacob Carlborg wrote:
>>> > I just stripped out all D1 and Tango related code from Orange.
>>> > D1/Tango is still supported in the d1 branch. Hopefully this will
>>> > make it easier to integrate into Phobos.
>>> >
>>> > It also now supports UDA's for indicating a field/class/struct
>>> > shouldn't be serialized:
>>> >
>>> > class Foo
>>> > {
>>> >     @nonSerialized int a;
>>> > }
>>> >
>>> > @nonSerialized class Bar { }
>>>
>>> Hmm, shouldn't it be the other way around - marking the ones to be
>>> serialized?
>>>
>>
>> You're already opting-in to serialization anyway when you say
>> "serialize object foobar". @serializable would just be redundant.
>>
>
> I think Walter's point is that the author of foobar may not have opted in.
>
> My response to that is, so?  If someone is trying to serialize your class, and you didn't test for that, too bad for that person if it doesn't work.
>
> The reality is, an author may write foobar without intending it to be serializable, but it actually is.  In that case, it is still on the user, but if it works, great!  The user is taking the risk that later the serialization breaks, and could always submit a patch to the author of foobar if it somehow becomes not-working in a future version.
>
> To have to say:
>
> struct S
> {
>   int x;
>   int y;
> }
>
> is serializable seems like super-redundant info.  The D type system is one of the most advanced I've ever seen.  Let's try and use it!

I agree that for lots of code it is redundant (data structures in general). It might even be desirable for library code, whose author did not consider Orange. However, it might lead into subtle bugs in some cases.

For example, what happens with a mutex? Would Orange serialize it with all its state (locked or unlocked)?

What happens with data structures, which are used in parallel. For a consistent serialization it might be necessary to take a lock first. In this case, the data structure must be able to provide its own serialization method.

I have not seen a "skip on serialize, but re-initialize on deserialize" annotation, which would be the correct behavior for uncontested thread-safe data structures using locks.

The most complex case would probably be something like a GUI toolkit. If I serialize a GtkD app on Linux and deserialize it on Windows, will it be able to produce a working GUI? It must provide a custom deserialize method to do that.

Looking at all those edge cases, an opt-in approach seems not that stupid to me. It might be tedious to add all those @serializeable annotations, but at least you do not run into weird deadlocks.

On Mon, 18 Feb 2013 18:43:16 -0500, qznc <qznc@go.to> wrote:

> On Monday, 18 February 2013 at 01:27:17 UTC, Steven Schveighoffer wrote:
>> On Sun, 17 Feb 2013 19:58:06 -0500, Nick Sabalausky <SeeWebsiteToContactMe@semitwist.com> wrote:
>>
>>> On Sun, 17 Feb 2013 13:18:05 -0800
>>> Walter Bright <newshound2@digitalmars.com> wrote:
>>>
>>>> On 2/17/2013 12:51 PM, Jacob Carlborg wrote:
>>>> > I just stripped out all D1 and Tango related code from > Orange.
>>>> > D1/Tango is still supported in the d1 branch. Hopefully > this will
>>>> > make it easier to integrate into Phobos.
>>>> >
>>>> > It also now supports UDA's for indicating a > field/class/struct
>>>> > shouldn't be serialized:
>>>> >
>>>> > class Foo
>>>> > {
>>>> >     @nonSerialized int a;
>>>> > }
>>>> >
>>>> > @nonSerialized class Bar { }
>>>>
>>>> Hmm, shouldn't it be the other way around - marking the ones to be
>>>> serialized?
>>>>
>>>
>>> You're already opting-in to serialization anyway when you say
>>> "serialize object foobar". @serializable would just be redundant.
>>>
>>
>> I think Walter's point is that the author of foobar may not have opted in.
>>
>> My response to that is, so?  If someone is trying to serialize your class, and you didn't test for that, too bad for that person if it doesn't work.
>>
>> The reality is, an author may write foobar without intending it to be serializable, but it actually is.  In that case, it is still on the user, but if it works, great!  The user is taking the risk that later the serialization breaks, and could always submit a patch to the author of foobar if it somehow becomes not-working in a future version.
>>
>> To have to say:
>>
>> struct S
>> {
>>   int x;
>>   int y;
>> }
>>
>> is serializable seems like super-redundant info.  The D type system is one of the most advanced I've ever seen.  Let's try and use it!
>
> I agree that for lots of code it is redundant (data structures in general). It might even be desirable for library code, whose author did not consider Orange. However, it might lead into subtle bugs in some cases.
>
> For example, what happens with a mutex? Would Orange serialize it with all its state (locked or unlocked)?

Most likely a mutex is a core library feature, and we can mark it as not serializeable.  If (when?) phobos gets serialization, I would expect the core types to be marked as such.

> What happens with data structures, which are used in parallel. For a consistent serialization it might be necessary to take a lock first. In this case, the data structure must be able to provide its own serialization method.

For D, we have shared which indicates it may be viewed by more than one thread.  Shared data could be opt-in for serialization.

> I have not seen a "skip on serialize, but re-initialize on deserialize" annotation, which would be the correct behavior for uncontested thread-safe data structures using locks.
>
> The most complex case would probably be something like a GUI toolkit. If I serialize a GtkD app on Linux and deserialize it on Windows, will it be able to produce a working GUI? It must provide a custom deserialize method to do that.

I would expect that something that complex has to be specifically written to deal with serialization.

> Looking at all those edge cases, an opt-in approach seems not that stupid to me. It might be tedious to add all those @serializeable annotations, but at least you do not run into weird deadlocks.

Just because someone is stupid and tries to serialize a non-serializable construct such as a gui toolkit, it's not the serializer's fault.

I don't think opt-in is stupid, it's just a more conservative set of rules.  I think the number of constructs that WON'T be serializable will be far less than the ones that will be.

And multi-thread access items could be made opt-in.  I'm not sure what Orange does now.

-Steve

On Sunday, 17 February 2013 at 21:18:12 UTC, Walter Bright wrote:
> On 2/17/2013 12:51 PM, Jacob Carlborg wrote:
>> I just stripped out all D1 and Tango related code from Orange. D1/Tango is still
>> supported in the d1 branch. Hopefully this will make it easier to integrate into
>> Phobos.
>>
>> It also now supports UDA's for indicating a field/class/struct shouldn't be
>> serialized:
>>
>> class Foo
>> {
>>     @nonSerialized int a;
>> }
>>
>> @nonSerialized class Bar { }
>
> Hmm, shouldn't it be the other way around - marking the ones to be serialized?

The vast majority of objects can be serialized without any problems. Those that can't usually aren't referenced in objects that are being serialized and you know what they are ahead of time. Forcing opt-in serialization would make every single library out there at the moment unusable with std.serialize. In a Component system for example, you won't know ahead of time what needs to be serialized, and a single object not being marked Serialized will completely prevent serialization.

That being said, personally I think serialization in Phobos has been delayed far too long for something that's so important to many programs. I'd be happy with either opt-in or opt-out, however I believe that there should be a way to customize how an object gets serialized, such as an onSerialize method or interface.

On 2013-02-19 00:43, qznc wrote:

> I agree that for lots of code it is redundant (data structures in
> general). It might even be desirable for library code, whose author did
> not consider Orange. However, it might lead into subtle bugs in some cases.
>
> For example, what happens with a mutex? Would Orange serialize it with
> all its state (locked or unlocked)?

It would just serialize it. It serialize all internal sate, except for void pointers.

> What happens with data structures, which are used in parallel. For a
> consistent serialization it might be necessary to take a lock first. In
> this case, the data structure must be able to provide its own
> serialization method.

Yes, or perhaps the user can use a lock or similar when perform the serialization?

> I have not seen a "skip on serialize, but re-initialize on deserialize"
> annotation, which would be the correct behavior for uncontested
> thread-safe data structures using locks.

The library supports custom serialization, both for your own types and for third party types.

https://github.com/jacob-carlborg/orange/blob/master/tests/Custom.d
https://github.com/jacob-carlborg/orange/blob/master/tests/NonIntrusive.d

> The most complex case would probably be something like a GUI toolkit. If
> I serialize a GtkD app on Linux and deserialize it on Windows, will it
> be able to produce a working GUI? It must provide a custom deserialize
> method to do that.

Yes, it will most likely need a custom (de)serialization method. If we take DWT as an example. It could automatically serialize all internal state except for the native widget that is stored. So in this case it would probably be best to use custom serialization and only use the public API to set the properties. This would make sure that the underlying native widget is updated as well.

Theoretically it would be possible to serialize the native widget as well, at least on Mac OS X. That's how Xcode/InterfaceBuilder is working.

> Looking at all those edge cases, an opt-in approach seems not that
> stupid to me. It might be tedious to add all those @serializeable
> annotations, but at least you do not run into weird deadlocks.

I don't agree. I think that most data types _can_ be serialized.

-- 
/Jacob Carlborg

On 2013-02-19 04:26, Steven Schveighoffer wrote:

> Most likely a mutex is a core library feature, and we can mark it as not
> serializeable.  If (when?) phobos gets serialization, I would expect the
> core types to be marked as such.

Exactly. IO objects, threads are also included in this category.

> I would expect that something that complex has to be specifically
> written to deal with serialization.

Yes, exactly, see my other reply:

http://forum.dlang.org/thread/kfrfs4$2h6f$1@digitalmars.com?page=4#post-kfvard:242nfi:241:40digitalmars.com

> Just because someone is stupid and tries to serialize a non-serializable
> construct such as a gui toolkit, it's not the serializer's fault.
>
> I don't think opt-in is stupid, it's just a more conservative set of
> rules.  I think the number of constructs that WON'T be serializable will
> be far less than the ones that will be.

I agree.

> And multi-thread access items could be made opt-in.  I'm not sure what
> Orange does now.

Orange just serialize everything. I haven't actually tried with anything marked as "shared". Perhaps I should do that.

-- 
/Jacob Carlborg

On Monday, 18 February 2013 at 07:36:07 UTC, Jacob Carlborg wrote:
> On 2013-02-17 22:52, Adam Wilson wrote:
>
>> Every serialization library i've used is opt-out.
>
> Java - Requires annotation
> Ruby - Doesn't require anything

.NET languages as well.

On Sunday, 17 February 2013 at 21:31:31 UTC, Manu wrote:
> On 18 February 2013 07:18, Walter Bright <newshound2@digitalmars.com> wrote:
>
>> On 2/17/2013 12:51 PM, Jacob Carlborg wrote:
>>
>>> I just stripped out all D1 and Tango related code from Orange. D1/Tango
>>> is still
>>> supported in the d1 branch. Hopefully this will make it easier to
>>> integrate into
>>> Phobos.
>>>
>>> It also now supports UDA's for indicating a field/class/struct shouldn't
>>> be
>>> serialized:
>>>
>>> class Foo
>>> {
>>>      @nonSerialized int a;
>>> }
>>>
>>> @nonSerialized class Bar { }
>>>
>>
>> Hmm, shouldn't it be the other way around - marking the ones to be
>> serialized?
>>
>
> I think both are useful, but I would expect opt-in rather than opt out as
> Walter asserts. For instance:
>
> @serialise class Foo // everything in the class
> {
>      int a;
>      @noSerialise int cacheValue; // except this
> }

yes, I think this is most useful, and:

@noSerialise class Foo // nothing in the class
{
     int a;
     @Serialise int cacheValue; // except this
}