Hello again folks,

My background in file-level read/write operations comes from Delphi and its handling of types.  So for reading and writing strings they always had to be written and read in a specific way, and a length integer prepended to the string.  For this reason you can't have a string, unless limited in length, in a record (Pascal's version of struct).

Can D read and write structs with dynamically sized strings to and from streams?

Here's an example of a struct I'd use:
> struct MyStruct {
>   int val1, val2;
>   char[] id;
>   uint somesuchvalue;
>   // Etc...
> }

Can D handle writing a variably-sized struct like this to a stream without any other steps?  Or, should I have a "write" method of the struct to write it to a stream?  If so, then how do I write an array of these structs efficiently?  (Like a batch read, ie: Stream.Read(myArrayOfStructs, theirLength))

I guess this is just a general question about how Tango handles streams and structs when writing, as well as any D specific quirks or advantages there may be.

I'm thinking of making a simple archive format, because I don't like the source control packages out there and was thinking of making my own.

Thanks for the answers in advance!

Regards,
Robert

Robert Kosek Wrote:

> Hello again folks,

I know I asked a really basic question, but I haven't found a real answer either in the Tango documentation or the D language documentation either.  This makes it really tough to know how I can save structures to a stream for later use.

Can someone please tell me if I can read/write structures with dynamic length strings from/to a stream?  And if not, then how can I read and write a string to the stream (without the helper stream class)?

Robert

"Robert Kosek"  wrote
> Robert Kosek Wrote:
>
>> Hello again folks,
>
> I know I asked a really basic question, but I haven't found a real answer either in the Tango documentation or the D language documentation either. This makes it really tough to know how I can save structures to a stream for later use.
>
> Can someone please tell me if I can read/write structures with dynamic length strings from/to a stream?  And if not, then how can I read and write a string to the stream (without the helper stream class)?

I am not completely knowledgable in the stream classes of Tango that do serialization, but I don't think there's anything to do what you want.  It would be difficult for the stream class to know the exact layout of your struct, so how would it know when to output strings?  I think DataStream is what you are looking for, but it will only output a block of data.

Again, not completely knowledgable.  You might try asking on the Tango forums to get the attention of some of the Tango devs that don't pay attention to the NG.

-Steve

On Wed, Oct 22, 2008 at 11:24 PM, Steven Schveighoffer <schveiguy@yahoo.com> wrote:
> "Robert Kosek"  wrote
>> Robert Kosek Wrote:
>>
>>> Hello again folks,
>>
>> I know I asked a really basic question, but I haven't found a real answer either in the Tango documentation or the D language documentation either. This makes it really tough to know how I can save structures to a stream for later use.
>>
>> Can someone please tell me if I can read/write structures with dynamic length strings from/to a stream?  And if not, then how can I read and write a string to the stream (without the helper stream class)?
>
> I am not completely knowledgable in the stream classes of Tango that do serialization, but I don't think there's anything to do what you want.  It would be difficult for the stream class to know the exact layout of your struct, so how would it know when to output strings?  I think DataStream is what you are looking for, but it will only output a block of data.
>
> Again, not completely knowledgable.  You might try asking on the Tango forums to get the attention of some of the Tango devs that don't pay attention to the NG.

Tom S. was working on a nifty ctfe/mixin serialization module.  But I
think it's been on the back burner for a while.  Aarti_pl has one too
in the doost libs.
Neither is really complete.

And yes you need some kind of library if you want to stream arbitrary structs which may or may not contain pointers to disk.   And no, neither Tango nor Phobos has such a library.

--bb

Bill Baxter wrote:
> On Wed, Oct 22, 2008 at 11:24 PM, Steven Schveighoffer
> <schveiguy@yahoo.com> wrote:
> [...] And yes you need some kind of library if you want to stream arbitrary
> structs which may or may not contain pointers to disk.   And no,
> neither Tango nor Phobos has such a library.
> 
> --bb


Okay, thanks.  To write a struct to a stream should really be like writing a buffer.  You point the writer at the memory and tell it how large it is, and it writes the block to the stream.  So it should to be able to write a pointer-less struct.

That much is good, I can do string mangling/storage another way, probably.

Thanks for the answers Bill and Steven.

Robert

On Thu, Oct 23, 2008 at 12:11 AM, Robert Kosek <robert.kosek@thewickedflea.com> wrote:
> Bill Baxter wrote:
>>
>> On Wed, Oct 22, 2008 at 11:24 PM, Steven Schveighoffer
>> <schveiguy@yahoo.com> wrote:
>> [...] And yes you need some kind of library if you want to stream
>> arbitrary
>> structs which may or may not contain pointers to disk.   And no,
>> neither Tango nor Phobos has such a library.
>>
>> --bb
>
>
> Okay, thanks.  To write a struct to a stream should really be like writing a buffer.  You point the writer at the memory and tell it how large it is, and it writes the block to the stream.  So it should to be able to write a pointer-less struct.

But if you care about portability you need to worry about things like endianness and the variable size of D's real, creal, and ireal types. Also difference of size in size_t from 32 bit to 64 bit platforms.  So even without pointers there can be more to it than just dumping a chunk of memory into a stream.

> That much is good, I can do string mangling/storage another way, probably.

Yep, if your needs are fairly simple then it's not that difficult to come up with something.  There is something like an EndianStream in Tango.  I think it does byteswapping on ints and floats and things for you.

--bb