View mode: basic / threaded / horizontal-split · Log in · Help
March 09, 2005
Stream to struct
Look at the following code:

struct BitmapFileHeader {
  ushort bfType;
  uint   bfSize;
  ushort bfReserved1;
  ushort bfReserved2;
  uint   bfOffBits;
 }

 BitmapFileHeader fileHeader;

 //method one for reading

 stream.seekSet(0);
 stream.read(fileHeader.bfType);
 stream.read(fileHeader.bfSize);
 stream.read(fileHeader.bfReserved1);
 stream.read(fileHeader.bfReserved2);
 stream.read(fileHeader.bfOffBits);

 //method two for reading

 stream.seekSet(0);
 stream.readExact(&fileHeader, fileHeader.sizeof);

Can someone explain me the reason I am getting different results between 
method one and two.

Thanks

-- 
Miguel Ferreira Simões
March 09, 2005
Re: Stream to struct
"Miguel Ferreira Simões" <Kobold@netcabo.pt> skrev i en meddelelse 
news:d0mmad$20ta$1@digitaldaemon.com...
> Can someone explain me the reason I am getting different results between 
> method one and two.

It is probably due to alignment, and I suspect that method 1 is the one that 
gives you the expected results. The first member of the struct is 16 bit, 
and the next is 32 bit. Therefore the compiler needs to insert two padding 
bytes in the struct to maintain 32 bit alignment of field two. Even if the 
compiler had (or has?) methods of controlling padding, it easily winds up 
being compiler-specific, which is something you definitely do not want when 
working with a non-compiler-specific format as BMP. So I would consider it 
bad practice to use method 2 even if applicable.

Regards,
Martin
March 09, 2005
Re: Stream to struct
I suppose is due to alignment too.
Structs have a property .alignof can we control alignment?
March 09, 2005
Re: Stream to struct
In article <d0mmad$20ta$1@digitaldaemon.com>, Miguel Ferreira Simões says...
>
>Look at the following code:
>
> struct BitmapFileHeader {
>   ushort bfType;
>   uint   bfSize;
>   ushort bfReserved1;
>   ushort bfReserved2;
>   uint   bfOffBits;
>  }
>
>  BitmapFileHeader fileHeader;
>
>  //method one for reading
>
>  stream.seekSet(0);
>  stream.read(fileHeader.bfType);
>  stream.read(fileHeader.bfSize);
>  stream.read(fileHeader.bfReserved1);
>  stream.read(fileHeader.bfReserved2);
>  stream.read(fileHeader.bfOffBits);
>
>  //method two for reading
>
>  stream.seekSet(0);
>  stream.readExact(&fileHeader, fileHeader.sizeof);
>
>Can someone explain me the reason I am getting different results between 
>method one and two.
>
>Thanks
>
>-- 
>Miguel Ferreira Simões 
>
>

You should read the same way you write. Meaning if the stream that you are
writing to uses the first method then the stream you are reading should also use
the first method. If the stream you are writing to uses the second then read
using the second. If you are not writing to the stream sending the data then you
need to be careful about alignment, the size and layout of the data - in general
if that is the case I would recommend using the first method.
March 09, 2005
Re: Stream to struct
Miguel Ferreira Simões wrote:
> I suppose is due to alignment too.
> Structs have a property .alignof can we control alignment?
> 
> 
> 
Its like private/protected/public sections syntax.

align(1):  <members here>

align(2){
  some other members here.
}
March 09, 2005
Re: Stream to struct
Martin M. Pedersen wrote:

>>Can someone explain me the reason I am getting different results between 
>>method one and two.
> 
> It is probably due to alignment, and I suspect that method 1 is the one that 
> gives you the expected results. The first member of the struct is 16 bit, 
> and the next is 32 bit. Therefore the compiler needs to insert two padding 
> bytes in the struct to maintain 32 bit alignment of field two. Even if the 
> compiler had (or has?) methods of controlling padding, it easily winds up 
> being compiler-specific, which is something you definitely do not want when 
> working with a non-compiler-specific format as BMP. So I would consider it 
> bad practice to use method 2 even if applicable.

And also, alignment is only half of the problem. Your program
could probably live with the bad performance of having things
aligned to non-optimal boundaries, but everything larger than
bytes (say short and float, for instance) are also *endian*...

Which means that if you read/write directly from memory,
you'll have a harder time to port to a different platform
(BigEndian/LittleEndian). If you read/write each field,
then you can add the required byteswap operations there ?

In portable storage formats, this quickly becomes a problem.
I suggest you fill out the struct one field at a time, and
use a disk buffer for reading (if required) ? This buffer
could use the align(1) struct... But you should have one
format for disk (storage), and one format for memory (usage)
This will also enable you to e.g. compress the storage format.


Speaking of this, where does D stand on Object Serialization ?
http://java.sun.com/j2se/1.4.2/docs/api/java/io/Serializable.html

--anders
March 09, 2005
Re: Stream to struct
In article <d0mv1u$29so$1@digitaldaemon.com>,
=?ISO-8859-1?Q?Anders_F_Bj=F6rklund?= says...
>
>In portable storage formats, this quickly becomes a problem.
>I suggest you fill out the struct one field at a time, and
>use a disk buffer for reading (if required) ? This buffer
>could use the align(1) struct... But you should have one
>format for disk (storage), and one format for memory (usage)
>This will also enable you to e.g. compress the storage format.
>
>
>Speaking of this, where does D stand on Object Serialization ?
>http://java.sun.com/j2se/1.4.2/docs/api/java/io/Serializable.html

As far as I understand, the only way to perform serialization of objects is
explicitly.  Its currently a burden placed on the developer.  For example, see
Mango's Pickling interface for a solid example of object serialization in D.

The reason for this is simple: D lacks a solid reflection interface. In order to
accomplish truely transparent serialization, one would need access to the
following features:

- runtime lookup of class members and/or methods.
- runtime creation of an object via classname (akin to java's classloader)

(...which you could implement yourself via a family of mixins or somesuch, but
at that point you're really implementing your own type metadata system)

D's type system has plenty of other missing features, which Walter acknowledged
many moons ago.  Your guess is as good as mine as to why its been sitting on the
back burner.

- EricAnderton at yahoo
March 09, 2005
Re: Stream to struct (Serialization)
pragma wrote:

>>Speaking of this, where does D stand on Object Serialization ?
>>http://java.sun.com/j2se/1.4.2/docs/api/java/io/Serializable.html
> 
> As far as I understand, the only way to perform serialization of objects is
> explicitly.  Its currently a burden placed on the developer.  For example, see
> Mango's Pickling interface for a solid example of object serialization in D.

Ah, thanks. Strange name for that particular interface, don't you think?
http://svn.dsource.org/svn/projects/mango/trunk/doc/html/structIPickle.html

> The reason for this is simple: D lacks a solid reflection interface. In order to
> accomplish truely transparent serialization, one would need access to the
> following features:
> 
> - runtime lookup of class members and/or methods.
> - runtime creation of an object via classname (akin to java's classloader)
> 
> (...which you could implement yourself via a family of mixins or somesuch, but
> at that point you're really implementing your own type metadata system)

Wonder if this will ever be possible in D, being fully native and all ?
At least not in http://www.prowiki.org/wiki4d/wiki.cgi?LanguagesVersusD

> D's type system has plenty of other missing features, which Walter acknowledged
> many moons ago.  Your guess is as good as mine as to why its been sitting on the
> back burner.

Probably has other bugs to fix first...

--anders
March 09, 2005
Re: Stream to struct (Serialization)
In article <d0n47c$2frl$1@digitaldaemon.com>,
=?ISO-8859-1?Q?Anders_F_Bj=F6rklund?= says...
>
>> For example, see Mango's Pickling interface for a solid example of object serialization in D.
>
>Ah, thanks. Strange name for that particular interface, don't you think?
>http://svn.dsource.org/svn/projects/mango/trunk/doc/html/structIPickle.html

I have to admit, the first time I read through Kris' source, I dismissed
'Pickle' for having such an odd name.  However, the concept of actually
'pickling' an object is an apt metaphor for the process.  Also, it originated in
the python language and has cropped up as a boost module. 

However, I cant get rid of the mental image of floppy disks in sealed jars of
vinegar, on shelves in a dark basement.

>> - runtime lookup of class members and/or methods.
>> - runtime creation of an object via classname (akin to java's classloader)
>
>Wonder if this will ever be possible in D, being fully native and all ?
>At least not in http://www.prowiki.org/wiki4d/wiki.cgi?LanguagesVersusD

I'd say its more than possible.  All the needed information exists at compile
time, so all that's left is to drop it in the output somewhere.  I also feel
that the door is open for features like creating types from scratch at runtime. 

>> Your guess is as good as mine as to why its been sitting on the
>> back burner.
>
>Probably has other bugs to fix first...

Agreed.  If only he accepted code patches, I'd write it *for* him. ;)

- EricAnderton at yahoo
March 09, 2005
Re: Stream to struct (Serialization)
In article <d0n5rg$2hq9$1@digitaldaemon.com>, pragma says...
>
>In article <d0n47c$2frl$1@digitaldaemon.com>,
>=?ISO-8859-1?Q?Anders_F_Bj=F6rklund?= says...
>>
>>> For example, see Mango's Pickling interface for a solid example of object serialization in D.
>>
>>Ah, thanks. Strange name for that particular interface, don't you think?
>>http://svn.dsource.org/svn/projects/mango/trunk/doc/html/structIPickle.html
>
>I have to admit, the first time I read through Kris' source, I dismissed
>'Pickle' for having such an odd name.  However, the concept of actually
>'pickling' an object is an apt metaphor for the process.  Also, it originated in
>the python language and has cropped up as a boost module. 
>
>However, I cant get rid of the mental image of floppy disks in sealed jars of
>vinegar, on shelves in a dark basement.


Truth be told: 'pickle' is a whole lot easier to type that 'serialization' :-)

But I agree; it is hard to forego the thought of a faint waft rising from the
ethernet-packets as they traverse a cluster ...

FWIW ~ the Mango pickling is actually a whole lot faster than "managed"
serialization (as Java has), and just about as convenient for non-trivial use
(efficiency is often a notable factor in serialization). Would still be better
to have full reflection though -- perhaps in 2.0?

- Kris
« First   ‹ Prev
1 2
Top | Discussion index | About this forum | D home