On Saturday, 15 June 2013 at 18:54:35 UTC, Baz wrote:
> The basic problem is that there is no flag in the D object model for the serialization
Yep, and std.serialization adds it, via UDA.

> In the same order of idea, the doc about RTI is null. In fact there's even no RTI usefull for an "academic" serialization system.

RTI (rtti) is hardly relevant here, as far as I can see, std.serialization does static reflection.

On Tuesday, 2 April 2013 at 18:50:04 UTC, Jacob Carlborg wrote:
> It probably makes sense if one sends the data over the network and the data is mostly value based. I usually have an object hierarchy with many reference types and objects passed around.

I think that's kinda the sticking point with Orange for me. Integrating it in my current project implied bringing in a huge amount of code when my needs are super-straightforward.

I actually ended up writing myself a super-light range-based serializer that'll handle any combination of struct/Array/AA thrown at it. (for reference: https://github.com/Chabsf/flyb/blob/SerializableHistory/serialize.d ). It also does not copy data around, and instead just casts the data as arrays of bytes. It's under 300 lines of D code and does everything I need from Orange, and does it very fast. Orange is just so very overkill for my needs.

My point is that serializing POD-based structures in D is so simple that using a one-size-fit-all serializer made to handle absolutely everything feels very wasteful.

On 2013-06-17 15:31, Francois Chabot wrote:

> I think that's kinda the sticking point with Orange for me. Integrating
> it in my current project implied bringing in a huge amount of code when
> my needs are super-straightforward.

Why is that a problem?

> I actually ended up writing myself a super-light range-based serializer
> that'll handle any combination of struct/Array/AA thrown at it. (for
> reference:
> https://github.com/Chabsf/flyb/blob/SerializableHistory/serialize.d ).
> It also does not copy data around, and instead just casts the data as
> arrays of bytes. It's under 300 lines of D code and does everything I
> need from Orange, and does it very fast. Orange is just so very overkill
> for my needs.

It's a lot easier to create a serializer which doesn't handle the full set of D. When you start to bring in reference types, pointers, arrays and slices it's start to get more complicated.

> My point is that serializing POD-based structures in D is so simple that
> using a one-size-fit-all serializer made to handle absolutely everything
> feels very wasteful.

You don't have to use it.

-- 
/Jacob Carlborg

>> My point is that serializing POD-based structures in D is so simple that
>> using a one-size-fit-all serializer made to handle absolutely everything
>> feels very wasteful.
>
> You don't have to use it.

But I do use it, quite a bit. Whenever I have any form of polymorphic serialization, Orange is excellent. I find myself effectively choosing between two serialization libraries based on my needs right now, which is actually a very good thing.

The issue here is not whether I should be using Orange or not in a given project, but whether it should become std.serialization. If that happens, then I will find myself "forced" to use it, the same way I should be "forced" to use std::vector in C++ unless I am dealing with a truly exceptional situation.

But serializing data to send on a socket efficiently should NOT be treated as an exceptional case. If anything, I think it should be the base case. If/When std.serialization exists, then it will effectively become the goto serialization mechanism for any D newcomer, and it really should handle that case efficiently and elegantly IMHO.

I will grant that making Orange part of Phobos will alleviate the project bloat issue, which is a huge deal. But as it stands, to me, it only handles a subset of what std.serialization should.

On 2013-06-17 16:39, Francois Chabot wrote:

> I will grant that making Orange part of Phobos will alleviate the
> project bloat issue, which is a huge deal. But as it stands, to me, it
> only handles a subset of what std.serialization should.

So what would you like it to handle? I assume you want a binary archive and you want faster serialization? You are free to add enhancement requests to the github project and comment in the official review thread.

The thing is with Orange is that it's possible to add new archive types. If Orange gets accepted as std.serialization we could later add a binary archive.

Do you want to stop std.serialization just because of it not having a binary archive? Not having a binary archive doesn't make the XML archive useless.

-- 
/Jacob Carlborg

On Monday, 17 June 2013 at 20:59:31 UTC, Jacob Carlborg wrote:
> On 2013-06-17 16:39, Francois Chabot wrote:
>
>> I will grant that making Orange part of Phobos will alleviate the
>> project bloat issue, which is a huge deal. But as it stands, to me, it
>> only handles a subset of what std.serialization should.
>
> So what would you like it to handle? I assume you want a binary archive and you want faster serialization? You are free to add enhancement requests to the github project and comment in the official review thread.

Well, the main thing that I want on my end is a O(1) memory footprint when dealing with hierarchical data with no cross-references. Even better would be that serialization being a lazy input range that can be easily piped into something like Walter's std.compress. I guess I could log an enhancement request to that effect, but I kinda felt that this was beyond the scope of Orange. It has a clear serialize, then deal with the data design. What I need really goes against the grain here.

> The thing is with Orange is that it's possible to add new archive types. If Orange gets accepted as std.serialization we could later add a binary archive.

Once again, the sticking point is not the serialization format, but the delivery method of said data.

Hmmm, come to think of it, running Orange in a fiber with a special archive type and wrapping the whole thing in a range could work. However, I'm not familiar enough with how aggressively Orange caches potential references to know if it would work. Also, due to the polymorphic nature of archives, archive types with partial serialization support would be a pain, as they would generate runtime errors, not compile-time.

> Do you want to stop std.serialization just because of it not having a binary archive? Not having a binary archive doesn't make the XML archive useless.

No! no no no. I just feel that Orange handles a big piece of the serialization puzzle, but that there's a lot more to it.

On 2013-06-18 00:06, Francois Chabot wrote:

> Well, the main thing that I want on my end is a O(1) memory footprint
> when dealing with hierarchical data with no cross-references.

That's kind of hard since it creates data. But if you mean except for the data then it stores some meta data in order to support reference types. That is, not serializing the same reference more than once. Theoretically a template parameter could be added to avoid this, but that defeats the purpose of a class/interface design. It could be a runtime parameter, don't know how well the compiler can optimize that out.

> Even better would be that serialization being a lazy input range that can be
> easily piped into something like Walter's std.compress. I guess I could
> log an enhancement request to that effect, but I kinda felt that this
> was beyond the scope of Orange. It has a clear serialize, then deal with
> the data design. What I need really goes against the grain here.

I'm no expert on ranges but I'm pretty sure it can work with std.compress. It returns an array, which is a random access range. Although it won't be lazy.

The deserialization would be a bit harder since it depends on std.xml which is not range based.

> Once again, the sticking point is not the serialization format, but the
> delivery method of said data.

Ok.

> No! no no no. I just feel that Orange handles a big piece of the
> serialization puzzle, but that there's a lot more to it.

Ok, I see. But I need to know about the rest, what you're missing to be able to improve it. I have made a couple of comments here.

-- 
/Jacob Carlborg

Any news on this?

On 06/10/14 12:24, "Daniele Bondì" <daniele.bondi.dev@gmail.com>" wrote:
> Any news on this?

No, I'm currently working on D/Objective-C [1].

[1] http://wiki.dlang.org/DIP43

-- 
/Jacob Carlborg