On Saturday, 17 August 2013 at 11:20:17 UTC, Dicebot wrote:
> 1) Having bindings in standard library is discouraged, we have Deimos for that. There is only curl stuff and it is considered a bad solution as far as I am aware of.

The D implementation of Thrift is actually not a binding and does not necessarily rely on the Thrift code generator either – all the latter does is to generate a D struct definition for the types/method parameters in your .thrift file that is then handled at D compile-time via reflection. In fact, this even works the other way, allowing you to generate .thrift IDL files for existing D types. (And yes, in theory the code generator could be replaced by ImportExpressions and a CTFE parser.)

David

On Saturday, 17 August 2013 at 10:15:34 UTC, BS wrote:
> I'd rather that was left for a separate module (or two or three) built on top of std.serialization.

In an ideal world, Thrift could maybe be built on std.serialization, but in the current form that's not true (regardless of e.g. versioning, Orange is likely not fast enough), and I am not sure whether this is a desirable goal in the first place anyway.

David

On Wednesday, 14 August 2013 at 16:25:21 UTC, Andrei Alexandrescu wrote:
> On 8/14/13 1:48 AM, Jacob Carlborg wrote:
>> On 2013-08-14 10:19, Tyler Jameson Little wrote:
>>>   - I would to serialize to a range (file?) and deserialize from a
>>> range (file?)
>>
>> The serialized data is returned as an array, so that is compatible with
>> the range interface, it just won't be lazy.
>
> This seems like a major limitation. (Disclaimer: I haven't read the documentation yet.)
>
> Andrei

Shall we fix it before accept the std.serialization?

For example, if I have 10GB of data and 16GB operating memory, I can't use std.serialization. It saves all my data into string into operating memory, so I haven't got enough memory to save data in file. It's currently limited by std.xml.

In other hand, std.serialization can help in many other cases if I have enough memory to store copy of my data.

As I can see, we have a few options:
- accept std.serialization as is. If users can't use std.serialization due memory limitation, they should find another way.
- hold std.serialization until we will have new std.xml module with support of range/file input/output. Users should use Orange if they need std.serialization right now.
- hold std.serialization until we will have binary archive for serialization with support of range/file input/output. Users should use Orange if they need std.serialization right now.
- use another xml library, for example from Tango.

Ideas?

ilya-stromberg wrote:
>>> The serialized data is returned as an array, so that is
>>> compatible with
>>> the range interface, it just won't be lazy.
>>
>> This seems like a major limitation. (Disclaimer: I haven't read
>> the documentation yet.)
>>
>> Andrei
> 
> Shall we fix it before accept the std.serialization?
> 
> For example, if I have 10GB of data and 16GB operating memory, I can't use std.serialization. It saves all my data into string into operating memory, so I haven't got enough memory to save data in file. It's currently limited by std.xml.
> 
> In other hand, std.serialization can help in many other cases if I have enough memory to store copy of my data.
> 
> As I can see, we have a few options:
> - accept std.serialization as is. If users can't use
> std.serialization due memory limitation, they should find another
> way.
> - hold std.serialization until we will have new std.xml module
> with support of range/file input/output. Users should use Orange
> if they need std.serialization right now.
> - hold std.serialization until we will have binary archive for
> serialization with support of range/file input/output. Users
> should use Orange if they need std.serialization right now.
> - use another xml library, for example from Tango.

My opinion is - accept it as it is (if it's not completely broken). I recently needed some way to serialize a data structure (in order by save the state of the app and restore it later) and was quite disappointed there is nothing like that in Phobos. Although XML is not necessarily well suited to my particular use case, it's still better than nothing.

Binary archive would be a great plus, but allow me to point out that current state of affairs (std.serialization being in a pre-accepted state for a long time AFAIK) is probably the worst state we might have - on the one hand I would not use third party libs, because std.serialization is just around the corner, on the other I don't have std.serialization distributed with the compiler yet. Also binary archive is an extension, not a change, so I don't see any reason why it could not be added later (because it would be backward compatible).

-- 
Marek Janukowicz

On 8/18/13, Marek Janukowicz <marek@janukowicz.net> wrote:
> I recently needed some way to serialize a data structure (in order by save the state of the app and restore it later) and was quite disappointed there is nothing like that in Phobos.

FWIW you could try out msgpack-d: https://github.com/msgpack/msgpack-d#usage

It's a very tiny and a fast library.

On Sunday, 18 August 2013 at 08:38:53 UTC, ilya-stromberg wrote:
> As I can see, we have a few options:
> - accept std.serialization as is. If users can't use std.serialization due memory limitation, they should find another way.
> - hold std.serialization until we will have new std.xml module with support of range/file input/output. Users should use Orange if they need std.serialization right now.
> - hold std.serialization until we will have binary archive for serialization with support of range/file input/output. Users should use Orange if they need std.serialization right now.
> - use another xml library, for example from Tango.
>
> Ideas?

We should add a suitable range interface, even if it makes no sense with current std.xml and include std.serialization now. For many use cases it will be sufficient and the improvements can come when std.xml2 comes. Holding back std.serialization will only mean that we won't see any new backend from users and would be quite unfair to Jacob and may keep off other contributors.

On 8/18/13, David Nadlinger <code@klickverbot.at> wrote:
> On Saturday, 17 August 2013 at 08:29:37 UTC, glycerine wrote:
>> If you
>> are going to standardize something, standardize the Thrift
>> bindings so that the compiler doesn't introduce regressions
>> that break them, like happened from dmd 2.062 - present.
>
> On a related note, we desperately need to do something about this, especially since there seems to be an increased amount of interest in Thrift lately. For 2.061 and the previous releases, I always tested every beta against Thrift, and almost invariably found at least one bug/regression per release. However, for 2.062 and 2.063, I was busy with LDC (and other things) at the time and it seems like I forgot to run the tests.

I think it would be good if we added Thrift and other test-cases, for example from the D Templates Book, to the test machines. But since there's a lot of code maybe the test machines should run the tests sporadically (e.g. after every #N new commits), otherwise pull requests would take forever to test.

Alternatively we could at least try to test these major projects with release candidates Normally the project maintainers would do this themselves, but it's easy to run out of time or just to forget to test things, and then it's too late (well we have fixup DMD releases now so it's not too bad).

On Sunday, 18 August 2013 at 14:24:38 UTC, Tobias Pankrath wrote:
> On Sunday, 18 August 2013 at 08:38:53 UTC, ilya-stromberg wrote:
>> As I can see, we have a few options:
>> - accept std.serialization as is. If users can't use std.serialization due memory limitation, they should find another way.
>> - hold std.serialization until we will have new std.xml module with support of range/file input/output. Users should use Orange if they need std.serialization right now.
>> - hold std.serialization until we will have binary archive for serialization with support of range/file input/output. Users should use Orange if they need std.serialization right now.
>> - use another xml library, for example from Tango.
>>
>> Ideas?
>
> We should add a suitable range interface, even if it makes no sense with current std.xml and include std.serialization now. For many use cases it will be sufficient and the improvements can come when std.xml2 comes. Holding back std.serialization will only mean that we won't see any new backend from users and would be quite unfair to Jacob and may keep off other contributors.

I completely agree.

I'm the one that brought it up, and I mostly brought it up so the API doesn't have to change once std.xml is fixed. I don't think changing the return type to a range will be too difficult or memory expensive.

Also, since slices *are* ranges, shouldn't this just work?

On Sunday, 18 August 2013 at 14:52:04 UTC, Andrej Mitrovic wrote:
> Normally the project maintainers would do this
> themselves, but it's easy to run out of time or just to forget to test
> things, and then it's too late (well we have fixup DMD releases now so it's not too bad).

The big problem with this right now is that quite frequently, you run the tests and discover one regression in the beta, file it, fix it (or wait for it to get fixed), then run the tests again, discover that they still don't pass, etc.

This is not only an annoying and time-intensive job for the maintainer of the project (as during beta you have to pretty much always be on your toes for a new version to test lest Walter decide to make the final release), but this also increases beta duration.

One obvious reaction to this (as a project maintainer) would be to continuously track Git master and report regressions as they arise. However, this is also not always practical, as quite often, there is a regression/backwards-incompatible change early on in the development process that is not fixed until much later, so that multiple issues can still pile up unnoticed.

Having a system that regularly, automatically runs the test suites of several larger, well-known D projects with the results being readily available to the DMD/druntime/Phobos teams would certainly help. But it's also not ideal, since if a project starts to fail, the exact nature of the issue (regression in DMD or bug in the project, and if the former, a minimal test case) can often be hard to track down for somebody not already familiar with the code base.

David

On Sunday, 18 August 2013 at 16:33:51 UTC, David Nadlinger wrote:
> ...

Please, don't move too far from review topic ;) It is a separate issue to discuss.