Regan Heath wrote:
> On Sun, 23 Jul 2006 17:26:57 -0700, Charles D Hixson <charleshixsn@earthlink.net> wrote:
>> Jarrett Billingsley wrote:
>>> "Charles D Hixson" <charleshixsn@earthlink.net> wrote in message
>>>> ...
>>>
>>> You don't even need a routine to do it.
>>>
>>> char[] sig = "help";
>>> uint s = *cast(uint*)sig.ptr;
>>>
>>> And the other way..
>>>
>>> uint s = 0xAABBCCDD;
>>> char[] sig = new char[4];
>>> *cast(uint*)sig.ptr = s;
>>>
>>>
>> O, dear.  Yes, I see it.
>> But one of the things that cause me to prefer D over C is
>> the ability to avoid pointer manipulation, which to me seem
>> extremely hazardous and, when one gets beyond simple cases,
>> quite confusing.  (And unmaintainable.)
> 
> If it works, then I say put it in a function and ignore 'how' it works. More than likely it will be inlined and you'll never need to worry about it again.
> 
> Regan
Right you are.  That's what I did...though it's a tiny bit ugly because D won't allow functions to return char[4] and won't allow them as out parameters.  (So I had to make it a char[], and that worked.)

xs0 wrote:
> 
>> P.S.:  Is there a standard library routine for converting between strings of length 4 and uint-s?  If so I wasn't able to find it.  If not, I wasn't able to determine that it didn't exist.  (That would have made writing the sig more efficient.)
> 
> union {
>     uint asUInt;
>     char[4] asChars;
> }
> 
> or something to that effect :)
For some reason when I tried that I was told that I was using "illegal utf8 characters".  But Jarrett's approach worked fine.

On Mon, 24 Jul 2006 13:34:12 -0700, Charles D Hixson wrote:

> xs0 wrote:
>> 
>>> P.S.:  Is there a standard library routine for converting between strings of length 4 and uint-s?  If so I wasn't able to find it.  If not, I wasn't able to determine that it didn't exist.  (That would have made writing the sig more efficient.)
>> 
>> union {
>>     uint asUInt;
>>     char[4] asChars;
>> }
>> 
>> or something to that effect :)
> For some reason when I tried that I was told that I was using "illegal utf8 characters".  But Jarrett's approach worked fine.

How about
 union {
     uint asUInt;
     ubyte[4] asChars;
 }

Are you really using *characters* or *bytes*?


-- 
Derek Parnell
Melbourne, Australia
"Down with mediocrity!"

"Regan Heath" <regan@netwin.co.nz> wrote in message news:optc56yyzv23k2f5@nrage...

> Option #1
> Note: .dup is required the local 'i' becomes invalid after the function
> returns
> char[] uint_to_char(uint i)
> {
> char[] str = new char[4];
> *cast(uint*)str.ptr = i;
> return str.dup;
> }

I don't think a .dup is required here; when you write "*cast(uint*)str.ptr = i", it's copying the data from i to the array's memory, not setting the array's pointer to point to the integer.  Option 2 is actually semantically the same.  Would be kind of interesting to see what machine code each version produces :)

On Mon, 24 Jul 2006 20:15:03 -0400, Jarrett Billingsley <kb3ctd2@yahoo.com> wrote:
> "Regan Heath" <regan@netwin.co.nz> wrote in message
> news:optc56yyzv23k2f5@nrage...
>
>> Option #1
>> Note: .dup is required the local 'i' becomes invalid after the function
>> returns
>> char[] uint_to_char(uint i)
>> {
>> char[] str = new char[4];
>> *cast(uint*)str.ptr = i;
>> return str.dup;
>> }
>
> I don't think a .dup is required here; when you write "*cast(uint*)str.ptr =
> i", it's copying the data from i to the array's memory, not setting the
> array's pointer to point to the integer.

Oops, you're right.

> Option 2 is actually semantically
> the same.  Would be kind of interesting to see what machine code each
> version produces :)

Regan

Derek wrote:
> On Mon, 24 Jul 2006 13:34:12 -0700, Charles D Hixson wrote:
> 
>> xs0 wrote:
>>>> P.S.:  Is there a standard library routine for converting between strings of length 4 and uint-s?  If so I wasn't able to find it.  If not, I wasn't able to determine that it didn't exist.  (That would have made writing the sig more efficient.)
>>> union {
>>>     uint asUInt;
>>>     char[4] asChars;
>>> }
>>>
>>> or something to that effect :)
>> For some reason when I tried that I was told that I was using "illegal utf8 characters".  But Jarrett's approach worked fine.
> 
> How about
>  union {
>      uint asUInt;
>      ubyte[4] asChars;
>  }
> 
> Are you really using *characters* or *bytes*?
> 
> 
characters.  My current (working) versions are:

uint  char4ToUint(char[4] item)
{   return  *cast(uint *)item.ptr;  }
char[] uintToChar4(uint item)
{  char[]  itmStr = new char[4];
   *cast(uint *)itmStr.ptr  =  item;
   return  itmStr[0..4];
}

Note that even with the version the "illegal utf8
characters" sneaks in unless I return a range from the
string.  I have no idea as to why, I've just tried various
things until something worked.
(Note the heavy influence of Jarrett's code.)

The test I used was to run a string through both of them in turn, and check that the output was the same as the input. I didn't check all possible strings, so there may be corner cases.  E.g., there may well be numbers that will generate illegal utf-8 characters.  (I can't see how, but then I don't know why this version works and several that appeared to me to be essentially equivalent didn't work.)

I didn't save the version using a union, and I didn't try a byte array.  (I don't know why...perhaps that seemed like a way of dodging the error message without dodging the error? Since currently a lot of what is happening feels like magic, I tend to be a bit conservative when an unexplained error message shows up.)

Charles D Hixson wrote:
> I'm thinking that I should build two methods, possibly
> called write and print where write would export a "binary"
> version of the class that is not intended to be human
> readable and print would export something with, e.g.,
> numbers translated into strings.
> 
> It appears as if the methods should be instance methods of
> the class.  It looks as if write should be implemented
> something like:
> void write(Stream stream)
> {  stream.writeExact(&this, this.sizeof);	}
> 
> Though probably in this version I'd want to include
> delimiters, a type id, and a length to facilitate writing
> the corresponding read routing (which would need to be a
> class method).
> 
> It this the best approach?  Also are there any suggestions
> as to a reasonable way to specify the type id, so that it
> would be the same from run to run?  Should I keep track of
> the ids myself (manually)?  If not, how would I know on an
> attempt to read which class the type id referred to?

I would define a struct that contains all of the trivial types and placeholders for the non trivial types (arrays would be replaced by an Array struct, etc.)

In the simple case all that is needed is to copy the elements into the struct and output it. If reference types are used, more complicated things need to be done. One option is to block out space for the struct and then output the referenced data (storing file pointers to the data in the struct), then backup and put the struct in the hole you left.

Referenced objects might still be an issue but they may be handled recursively (watch out for circular references).

Pulling data out of the file is much the same. I haven't tried anything like this with covariant classes. So I haven't any ideas on how to make that work.

This has the advantage over sterilization that the file can be navigated like the original data without having to walk the entire file.

Well, here's where I am now.
Granted the get and save aren't totally general...but it
demonstrates where I'm headed.  I plan to save several
different instances of different classes to the same file.
(And, of course, to keep an index...which I'll need to write
just as I close the file...or possibly to keep in a separate
file.)

just an FYI ~ Mango has supported this for years now:

class Vec : IReadable, IWritable
{
   float[] vec;
   char[]  sig;

   void read (IReader input)
   {
      input (sig) (vec);
   }

   void write (IWriter output)
   {
      output (sig) (vec);
   }
}


void main()
{
   auto vec = new Vec;

   auto f = new FileConduit ("myfile", FileConduit.ReadWriteCreate);
   auto write = new Writer (f);
   auto read = new Reader (f);

   // write and flush
   write (vec) ();

   // rewind and read
   f.seek (0);
   read (vec);
}

Reader/Writer has endian versions, and text versions ...




Charles D Hixson wrote:
> Well, here's where I am now.
> Granted the get and save aren't totally general...but it
> demonstrates where I'm headed.  I plan to save several
> different instances of different classes to the same file.
> (And, of course, to keep an index...which I'll need to write
> just as I close the file...or possibly to keep in a separate
> file.)
> 
> 
> 
> ------------------------------------------------------------------------
> 
> class Vec
> {  protected   float[]  vec;
>    const char[4]  sig   =  "nVec";
> 
> // constructors
> 
>    this(int len, float val)
>    {  initRand();
>       vec.length  =  len;
>       for   (int i = 0; i < vec.length;   i++)  vec[i]   =  val;
>    }
> 
> // a lot of stuff was removed here
> 
> 
>   /**
>    *  Export the object to a seekable stream.  The format is only
>    *  intended for computers, but it should be lossless.
>    *  The returned value is the stream address of the start of the object.
>    */
>    ulong write(Stream s)
>       in
>       {  assert (s.isOpen());
>          assert (s.writeable);
>          assert (s.seekable);
>       }
>       body
>       {  ulong oStart   =  s.position;
>          s.write(cast(uint)0);
>          s.write(char4ToUint(sig));
>          s.write(cast(uint)(this.vec.length));
>          foreach(float f;  this.vec)   s.write(f);
>          s.write(eor);
>          ulong oEnd  =  s.position;
>          s.position(oStart);
>          s.write(cast(uint)(oEnd - oStart) );
>          s.position(oEnd);
>          return   oStart;
>       }
> 
>   /**
>    *  Import the object from a seekable stream.  The format is only
>    *  intended for computers, but it should be lossless.
>    */
>    static   Vec   get(Stream s)
>       in
>       {  assert (s.isOpen());
>          assert (s.readable);
>          assert (s.seekable);
>       }
>       body
>       {  ulong oStart   =  s.position;
>          Vec   v  =  new   Vec();
>          uint  len, vlen, veor, tmpSig2;
>          char[]   sig2;
>          s.read(len);
>          s.read(tmpSig2);
>          sig2  =  uintToChar4(tmpSig2);
>          if (sig != sig2)
>          {  s.position(oStart);
>             writefln("original signature = <", sig, ">");
>             writefln("read in signature  = <", sig2, ">");
>             throw new   Exception ("Vec:get: Signature mismatch");
>          }
>          s.read(cast(uint)(vlen));
>          v.length =  vlen;
>          for(int i = 0; i < vlen;   i++)  s.read(v.vec[i]);
>          s.read(veor);
>          if (eor != veor)
>          {  s.position(oStart);
>             throw new   Exception ("Vec:get: eor not detected when expected");
>          }
>          ulong oEnd  =  s.position;
>          if (len != cast(uint)(oEnd - oStart) )
>          {  s.position(oStart);
>             writefln("length calculated = %d", len);
>             writefln("length read       = %d", cast(uint)(oEnd - oStart));
>             throw new   Exception
>                   ("Vec:get: length calculated does not match length read");
>          }
>          return   v;
>       }  // get
> 
> }

kris wrote:
> just an FYI ~ Mango has supported this for years now:
> 
> class Vec : IReadable, IWritable
> {
>    float[] vec;
>    char[]  sig;
> 
>    void read (IReader input)
>    {
>       input (sig) (vec);
>    }
> 
>    void write (IWriter output)
>    {
>       output (sig) (vec);
>    }
> }
> 
> 
> void main()
> {
>    auto vec = new Vec;
> 
>    auto f = new FileConduit ("myfile", FileConduit.ReadWriteCreate);
>    auto write = new Writer (f);
>    auto read = new Reader (f);
> 
>    // write and flush
>    write (vec) ();
> 
>    // rewind and read
>    f.seek (0);
>    read (vec);
> }
> 
> Reader/Writer has endian versions, and text versions ...
> 
> 
> 
> 
> Charles D Hixson wrote:
>> Well, here's where I am now.
>> Granted the get and save aren't totally general...but it
>> demonstrates where I'm headed.  I plan to save several
>> different instances of different classes to the same file.
>> (And, of course, to keep an index...which I'll need to write
>> just as I close the file...or possibly to keep in a separate
>> file.)
>>
>>
>>
>> ------------------------------------------------------------------------
>>
>> class Vec
>> {  protected   float[]  vec;
>>    const char[4]  sig   =  "nVec";
>>
>> // constructors
>>
>>    this(int len, float val)
>>    {  initRand();
>>       vec.length  =  len;
>>       for   (int i = 0; i < vec.length;   i++)  vec[i]   =  val;
>>    }
>>
>> // a lot of stuff was removed here
>>
>>
>>   /**
>>    *  Export the object to a seekable stream.  The format is only
>>    *  intended for computers, but it should be lossless.
>>    *  The returned value is the stream address of the start of the
>> object.
>>    */
>>    ulong write(Stream s)
>>       in
>>       {  assert (s.isOpen());
>>          assert (s.writeable);
>>          assert (s.seekable);
>>       }
>>       body
>>       {  ulong oStart   =  s.position;
>>          s.write(cast(uint)0);
>>          s.write(char4ToUint(sig));
>>          s.write(cast(uint)(this.vec.length));
>>          foreach(float f;  this.vec)   s.write(f);
>>          s.write(eor);
>>          ulong oEnd  =  s.position;
>>          s.position(oStart);
>>          s.write(cast(uint)(oEnd - oStart) );
>>          s.position(oEnd);
>>          return   oStart;
>>       }
>>
>>   /**
>>    *  Import the object from a seekable stream.  The format is only
>>    *  intended for computers, but it should be lossless.
>>    */
>>    static   Vec   get(Stream s)
>>       in
>>       {  assert (s.isOpen());
>>          assert (s.readable);
>>          assert (s.seekable);
>>       }
>>       body
>>       {  ulong oStart   =  s.position;
>>          Vec   v  =  new   Vec();
>>          uint  len, vlen, veor, tmpSig2;
>>          char[]   sig2;
>>          s.read(len);
>>          s.read(tmpSig2);
>>          sig2  =  uintToChar4(tmpSig2);
>>          if (sig != sig2)
>>          {  s.position(oStart);
>>             writefln("original signature = <", sig, ">");
>>             writefln("read in signature  = <", sig2, ">");
>>             throw new   Exception ("Vec:get: Signature mismatch");
>>          }
>>          s.read(cast(uint)(vlen));
>>          v.length =  vlen;
>>          for(int i = 0; i < vlen;   i++)  s.read(v.vec[i]);
>>          s.read(veor);
>>          if (eor != veor)
>>          {  s.position(oStart);
>>             throw new   Exception ("Vec:get: eor not detected when
>> expected");
>>          }
>>          ulong oEnd  =  s.position;
>>          if (len != cast(uint)(oEnd - oStart) )
>>          {  s.position(oStart);
>>             writefln("length calculated = %d", len);
>>             writefln("length read       = %d", cast(uint)(oEnd -
>> oStart));
>>             throw new   Exception
>>                   ("Vec:get: length calculated does not match length
>> read");
>>          }
>>          return   v;
>>       }  // get
>>
>> }
Nice!  Thanks.
I'll need to look into what else Mango provides...it may
save me a lot of work.