Jump to page: 1 2
Thread overview
Stream to struct
Mar 09, 2005
Martin M. Pedersen
Mar 09, 2005
David Medlock
Mar 09, 2005
pragma
Re: Stream to struct (Serialization)
Mar 09, 2005
pragma
Mar 09, 2005
Kris
Mar 15, 2005
Charles Hixson
Mar 09, 2005
Ben Hinkle
March 09, 2005
Look at the following code:

 struct BitmapFileHeader {
   ushort bfType;
   uint   bfSize;
   ushort bfReserved1;
   ushort bfReserved2;
   uint   bfOffBits;
  }

  BitmapFileHeader fileHeader;

  //method one for reading

  stream.seekSet(0);
  stream.read(fileHeader.bfType);
  stream.read(fileHeader.bfSize);
  stream.read(fileHeader.bfReserved1);
  stream.read(fileHeader.bfReserved2);
  stream.read(fileHeader.bfOffBits);

  //method two for reading

  stream.seekSet(0);
  stream.readExact(&fileHeader, fileHeader.sizeof);

Can someone explain me the reason I am getting different results between method one and two.

Thanks

-- 
Miguel Ferreira Simões


March 09, 2005
"Miguel Ferreira Simões" <Kobold@netcabo.pt> skrev i en meddelelse news:d0mmad$20ta$1@digitaldaemon.com...
> Can someone explain me the reason I am getting different results between method one and two.

It is probably due to alignment, and I suspect that method 1 is the one that gives you the expected results. The first member of the struct is 16 bit, and the next is 32 bit. Therefore the compiler needs to insert two padding bytes in the struct to maintain 32 bit alignment of field two. Even if the compiler had (or has?) methods of controlling padding, it easily winds up being compiler-specific, which is something you definitely do not want when working with a non-compiler-specific format as BMP. So I would consider it bad practice to use method 2 even if applicable.

Regards,
Martin


March 09, 2005
I suppose is due to alignment too.
Structs have a property .alignof can we control alignment?



March 09, 2005
In article <d0mmad$20ta$1@digitaldaemon.com>, Miguel Ferreira Simões says...
>
>Look at the following code:
>
> struct BitmapFileHeader {
>   ushort bfType;
>   uint   bfSize;
>   ushort bfReserved1;
>   ushort bfReserved2;
>   uint   bfOffBits;
>  }
>
>  BitmapFileHeader fileHeader;
>
>  //method one for reading
>
>  stream.seekSet(0);
>  stream.read(fileHeader.bfType);
>  stream.read(fileHeader.bfSize);
>  stream.read(fileHeader.bfReserved1);
>  stream.read(fileHeader.bfReserved2);
>  stream.read(fileHeader.bfOffBits);
>
>  //method two for reading
>
>  stream.seekSet(0);
>  stream.readExact(&fileHeader, fileHeader.sizeof);
>
>Can someone explain me the reason I am getting different results between method one and two.
>
>Thanks
>
>-- 
>Miguel Ferreira Simões
>
>

You should read the same way you write. Meaning if the stream that you are writing to uses the first method then the stream you are reading should also use the first method. If the stream you are writing to uses the second then read using the second. If you are not writing to the stream sending the data then you need to be careful about alignment, the size and layout of the data - in general if that is the case I would recommend using the first method.


March 09, 2005
Miguel Ferreira Simões wrote:
> I suppose is due to alignment too.
> Structs have a property .alignof can we control alignment?
> 
> 
> 
Its like private/protected/public sections syntax.

align(1):  <members here>

align(2){
  some other members here.
}



March 09, 2005
Martin M. Pedersen wrote:

>>Can someone explain me the reason I am getting different results between method one and two.
> 
> It is probably due to alignment, and I suspect that method 1 is the one that gives you the expected results. The first member of the struct is 16 bit, and the next is 32 bit. Therefore the compiler needs to insert two padding bytes in the struct to maintain 32 bit alignment of field two. Even if the compiler had (or has?) methods of controlling padding, it easily winds up being compiler-specific, which is something you definitely do not want when working with a non-compiler-specific format as BMP. So I would consider it bad practice to use method 2 even if applicable.

And also, alignment is only half of the problem. Your program
could probably live with the bad performance of having things
aligned to non-optimal boundaries, but everything larger than
bytes (say short and float, for instance) are also *endian*...

Which means that if you read/write directly from memory,
you'll have a harder time to port to a different platform
(BigEndian/LittleEndian). If you read/write each field,
then you can add the required byteswap operations there ?

In portable storage formats, this quickly becomes a problem.
I suggest you fill out the struct one field at a time, and
use a disk buffer for reading (if required) ? This buffer
could use the align(1) struct... But you should have one
format for disk (storage), and one format for memory (usage)
This will also enable you to e.g. compress the storage format.


Speaking of this, where does D stand on Object Serialization ?
http://java.sun.com/j2se/1.4.2/docs/api/java/io/Serializable.html

--anders
March 09, 2005
In article <d0mv1u$29so$1@digitaldaemon.com>, =?ISO-8859-1?Q?Anders_F_Bj=F6rklund?= says...
>
>In portable storage formats, this quickly becomes a problem.
>I suggest you fill out the struct one field at a time, and
>use a disk buffer for reading (if required) ? This buffer
>could use the align(1) struct... But you should have one
>format for disk (storage), and one format for memory (usage)
>This will also enable you to e.g. compress the storage format.
>
>
>Speaking of this, where does D stand on Object Serialization ? http://java.sun.com/j2se/1.4.2/docs/api/java/io/Serializable.html

As far as I understand, the only way to perform serialization of objects is explicitly.  Its currently a burden placed on the developer.  For example, see Mango's Pickling interface for a solid example of object serialization in D.

The reason for this is simple: D lacks a solid reflection interface. In order to accomplish truely transparent serialization, one would need access to the following features:

- runtime lookup of class members and/or methods.
- runtime creation of an object via classname (akin to java's classloader)

(...which you could implement yourself via a family of mixins or somesuch, but at that point you're really implementing your own type metadata system)

D's type system has plenty of other missing features, which Walter acknowledged many moons ago.  Your guess is as good as mine as to why its been sitting on the back burner.

- EricAnderton at yahoo
March 09, 2005
pragma wrote:

>>Speaking of this, where does D stand on Object Serialization ?
>>http://java.sun.com/j2se/1.4.2/docs/api/java/io/Serializable.html
> 
> As far as I understand, the only way to perform serialization of objects is
> explicitly.  Its currently a burden placed on the developer.  For example, see
> Mango's Pickling interface for a solid example of object serialization in D.

Ah, thanks. Strange name for that particular interface, don't you think?
http://svn.dsource.org/svn/projects/mango/trunk/doc/html/structIPickle.html

> The reason for this is simple: D lacks a solid reflection interface. In order to
> accomplish truely transparent serialization, one would need access to the
> following features:
> 
> - runtime lookup of class members and/or methods.
> - runtime creation of an object via classname (akin to java's classloader)
> 
> (...which you could implement yourself via a family of mixins or somesuch, but
> at that point you're really implementing your own type metadata system)

Wonder if this will ever be possible in D, being fully native and all ?
At least not in http://www.prowiki.org/wiki4d/wiki.cgi?LanguagesVersusD

> D's type system has plenty of other missing features, which Walter acknowledged
> many moons ago.  Your guess is as good as mine as to why its been sitting on the
> back burner.

Probably has other bugs to fix first...

--anders
March 09, 2005
In article <d0n47c$2frl$1@digitaldaemon.com>, =?ISO-8859-1?Q?Anders_F_Bj=F6rklund?= says...
>
>> For example, see Mango's Pickling interface for a solid example of object serialization in D.
>
>Ah, thanks. Strange name for that particular interface, don't you think? http://svn.dsource.org/svn/projects/mango/trunk/doc/html/structIPickle.html

I have to admit, the first time I read through Kris' source, I dismissed 'Pickle' for having such an odd name.  However, the concept of actually 'pickling' an object is an apt metaphor for the process.  Also, it originated in the python language and has cropped up as a boost module.

However, I cant get rid of the mental image of floppy disks in sealed jars of vinegar, on shelves in a dark basement.

>> - runtime lookup of class members and/or methods.
>> - runtime creation of an object via classname (akin to java's classloader)
>
>Wonder if this will ever be possible in D, being fully native and all ? At least not in http://www.prowiki.org/wiki4d/wiki.cgi?LanguagesVersusD

I'd say its more than possible.  All the needed information exists at compile time, so all that's left is to drop it in the output somewhere.  I also feel that the door is open for features like creating types from scratch at runtime.

>> Your guess is as good as mine as to why its been sitting on the back burner.
>
>Probably has other bugs to fix first...

Agreed.  If only he accepted code patches, I'd write it *for* him. ;)

- EricAnderton at yahoo
March 09, 2005
In article <d0n5rg$2hq9$1@digitaldaemon.com>, pragma says...
>
>In article <d0n47c$2frl$1@digitaldaemon.com>, =?ISO-8859-1?Q?Anders_F_Bj=F6rklund?= says...
>>
>>> For example, see Mango's Pickling interface for a solid example of object serialization in D.
>>
>>Ah, thanks. Strange name for that particular interface, don't you think? http://svn.dsource.org/svn/projects/mango/trunk/doc/html/structIPickle.html
>
>I have to admit, the first time I read through Kris' source, I dismissed 'Pickle' for having such an odd name.  However, the concept of actually 'pickling' an object is an apt metaphor for the process.  Also, it originated in the python language and has cropped up as a boost module.
>
>However, I cant get rid of the mental image of floppy disks in sealed jars of vinegar, on shelves in a dark basement.


Truth be told: 'pickle' is a whole lot easier to type that 'serialization' :-)

But I agree; it is hard to forego the thought of a faint waft rising from the ethernet-packets as they traverse a cluster ...

FWIW ~ the Mango pickling is actually a whole lot faster than "managed" serialization (as Java has), and just about as convenient for non-trivial use (efficiency is often a notable factor in serialization). Would still be better to have full reflection though -- perhaps in 2.0?

- Kris


« First   ‹ Prev
1 2