On Monday, 12 August 2013 at 13:27:45 UTC, Dicebot wrote:
> Stepping up to act as a Review Manager for Jacob Carlborg std.serialization
>
> ---- Input ----
>
> Code: https://github.com/jacob-carlborg/phobos/tree/serialization
>
> Documentation: https://dl.dropboxusercontent.com/u/18386187/docs/std.serialization/index.html
>
> Previous review thread: http://forum.dlang.org/thread/adyanbsdsxsfdpvoozne@forum.dlang.org
>
> ---- Changes since last review ----
>
> - Sources has been integrated into Phobos source tree
> - DDOC documentation has been provided in a form it should look like on dlang.org
> - Most utility functions/template code depends on have been inlined. Remaining `package` utility modules:
>     * std.serialization.archives.xmldocument
>     * std.serialization.attribute
>     * std.serialization.registerwrapper
>
> ---- Information for reviewers ----
>
> Goal of this thread is to detect if there are any outstanding issues that need to fixed before formal "yes"/"no" voting happens. If no critical objections will arise, voting will begin starting with a next week.
>
> Please take this seriously: "If you identify problems along the way, please note if they are minor, serious, or showstoppers." (http://wiki.dlang.org/Review/Process). This information later will be used to determine if library is ready for voting.
>
> If there are any frequent Phobos contributors / core developers please pay extra attention to submission code style and fitting into overall Phobos guidelines and structure.
>
> -------------------------------------
>
> Let the thread begin.
>
> Jacob, it is probably worth creating a pull request with latest rebased version of your proposal to simplify getting a quick overview of changes. Also please tell if there is anything you want/need to implement before merging.

OK, time to make a short summary.

There have been mentioned several issues / improvement possibilities. I don't think they prevent voting and it is up to Jacob to decide what he want to incorporate from it.

However, there are two things that do matter in my opinion - pre-UDA part of API and uncertainty about range-based lazy approach. Important thing here is that while library can be included with plenty of features lacking we can't really afford to break its API only few releases later just to add/remove these features.

So as a review manager, I think voting should be delayed until API is ready to address lazy range-based work model. No actual implementation is required but

1) it should be possible to do it later without breaking user code
2) library should not make an assumption about implementation being lazy or eager

That is my understanding based on current knowledge of Phobos modules, please correct me if I am wrong.

Jacob, please tell if you have any objections or, if this decision sounds reasonable - just contact me via e-mail when you will find std.serialization suitable for final voting. I think it is pretty clear that package itself is considered useful and welcome to Phobos.

On 2013-08-17 10:29, glycerine wrote:

> Huh? Do you know what thrift does? Summary: Everything that
> Orange/std.serialization does and more. To the point: Thrift
> provides data versioning, std.serialization does not. In my book:
> end of story, game over. Thrift is preffered choice. If you
> are going to standardize something, standardize the Thrift
> bindings so that the compiler doesn't introduce regressions
> that break them, like happened from dmd 2.062 - present.

Orange/std.serialization is capable of serializing more types than Thrift is. Example it will correctly serialize and deserialize slices, pointers and so on.

It's easy to implement versioning yourself, something like:

class Foo
{
    int version_;
    int a;
    int b;

    void toData (Serializer serializer, Serializer.Data key)
    {
        serializer.serialize(a, "a");
        serializer.serialize(version_, "version_");

        if (version_ == 2)
            serializer.serialize(b, "b");
    }

    // Do the corresponding in "fromData".
}

If versioning is crucial it can be added.

-- 
/Jacob Carlborg

On 2013-08-18 10:38, ilya-stromberg wrote:

> - use another xml library, for example from Tango.

The XML module from Tango excepts the content being in memory as well, at least the Document module.

-- 
/Jacob Carlborg

Andrej Mitrovic wrote:
>> I recently needed some way to serialize a data structure (in order by save the state of the app and restore it later) and was quite disappointed there is nothing like that in Phobos.
> 
> FWIW you could try out msgpack-d: https://github.com/msgpack/msgpack-d#usage
> 
> It's a very tiny and a fast library.

That's what I ended up using, but I would be much more happy to have something like this in Phobos.

-- 
Marek Janukowicz

On 8/18/2013 11:26 AM, Dicebot wrote:
> So as a review manager, I think voting should be delayed until API is ready to
> address lazy range-based work model.

I agree. Ranges are a very big deal for D, and libraries that can conceivably support it must do so.

On Sunday, August 18, 2013 21:45:59 Jacob Carlborg wrote:
> If versioning is crucial it can be added.

I don't know if it's crucial or not, but I know that the Java guys didn't have it initially but ended up adding it later, which would imply that they ran into problems that made them decide that it should be there. I'd certainly be inclined to think that it's better to have it, and it's probably easier to add it before it's merged than later. But I don't know how crucial it is.

- Jonathan M Davis

On Sunday, 18 August 2013 at 18:26:55 UTC, Dicebot wrote:
> On Monday, 12 August 2013 at 13:27:45 UTC, Dicebot wrote:
>> Stepping up to act as a Review Manager for Jacob Carlborg std.serialization
>>
> So as a review manager, I think voting should be delayed until API is ready to address lazy range-based work model. No actual implementation is required but
>
> 1) it should be possible to do it later without breaking user code
> 2) library should not make an assumption about implementation being lazy or eager
>

Can we path current std.xml to add file input/output, not only memory input/output? It can helps to serialize big data arrays directly in file.

On 2013-08-18 01:31, Jesse Phillips wrote:
> I'd like to start off by saying I don't really know what I want from a
> std.serialize module. I've done some work with a C# JSON serializer and
> dealt with Protocol Buffers.
>
> This library looks to be providing a means to serialize any D data
> structure. It deals with pointers/class/struct/arrays... It is export
> format agnostic, while currently only XML is available, allowing for
> export to JSON or some binary form. Afterwards the data can return to
> the program through deserialization.
>
> This is a use-case I don't think I've needed. Though I do see the value
> in it and would expect Phobos to provide such functionality.
>
> What I'm not finding in this library is a way to support a 3rd party
> protocol. Such as those used in Thrift or Protocol Buffers. These
> specify some aspects of data layout, for example in Protocol Buffers
> arrays of primitives can be laid out in two forms [ID][value][ID][value]
> or [ID][Length][value][value].

I have had a brief look at Protocol Buffers and I don't see why it wouldn't work as an archive. I would probably need to implement a Protocol Buffers archive type to see what the limitations of std.serialization and Protocol Buffers are.

> Thrift and Protocol Buffers use code generation to create the language
> data type, and at least for Protocol Buffers a method contains all the
> logic for deserializing a collection of bytes, and one for serializing.
> I'm not seeing how std.serialize would make this easier or more usable.

If a Thrift or Protocol Buffers archive would be used with std.serialization I'm thinking that one would skip that step and have the data types defined directly in D.

> When looking at the Archive module, I see that all the specific types
> get their own void function. I'm unclear on where these functions are
> supposed to archive to

The archive holds the data. When the serialization is complete the data can be accessed using "archive.data".

, and container types take a delegate which I
> suppose is a means for the archiver to place output around the field data.

Yes, exactly. It lets the archive know where a structured type begins and ends.

> In conclusion, I don't feel like I've said very much. I don't think
> std.serialize is applicable to Protocol Buffers or Thrift, and I don't
> know what benefit there would be if it was.


-- 
/Jacob Carlborg

On 2013-08-18 20:26, Dicebot wrote:

> OK, time to make a short summary.
>
> There have been mentioned several issues / improvement possibilities. I
> don't think they prevent voting and it is up to Jacob to decide what he
> want to incorporate from it.

I've been quite busy lately but I've tried to address the minor issues with regards of documentation. I've hit a new problem in the process:

http://forum.dlang.org/thread/kujcns$1quo$1@digitalmars.com

> However, there are two things that do matter in my opinion - pre-UDA
> part of API and uncertainty about range-based lazy approach. Important
> thing here is that while library can be included with plenty of features
> lacking we can't really afford to break its API only few releases later
> just to add/remove these features.

What do you mean with "pre-UDA part of API"?

I think it will be fairly easy to add support for ranges, at least for the output. I'll see what I can do.

> So as a review manager, I think voting should be delayed until API is
> ready to address lazy range-based work model. No actual implementation
> is required but
>
> 1) it should be possible to do it later without breaking user code
> 2) library should not make an assumption about implementation being lazy
> or eager
>
> That is my understanding based on current knowledge of Phobos modules,
> please correct me if I am wrong.
>
> Jacob, please tell if you have any objections or, if this decision
> sounds reasonable - just contact me via e-mail when you will find
> std.serialization suitable for final voting. I think it is pretty clear
> that package itself is considered useful and welcome to Phobos.


-- 
/Jacob Carlborg

On Monday, 19 August 2013 at 12:57:56 UTC, Jacob Carlborg wrote:
> I've been quite busy lately but I've tried to address the minor issues with regards of documentation. I've hit a new problem in the process:
>
> http://forum.dlang.org/thread/kujcns$1quo$1@digitalmars.com

I also expect that enhancement to dlang.org to support package.d documentation will also probably be needed at some point to get proper examples. Such issues can be worked on during actual merge process and are not worth blocking voting.

> What do you mean with "pre-UDA part of API"?

This thread: http://forum.dlang.org/post/xqklcesoguxujifijadp@forum.dlang.org

> I think it will be fairly easy to add support for ranges, at least for the output. I'll see what I can do.

Great! Are there any difficulties with the input?