Thread overview
write struct as raw data to file
Sep 26, 2021
Paul
Sep 26, 2021
Ali Çehreli
Sep 26, 2021
Ali Çehreli
Sep 26, 2021
Paul
Sep 26, 2021
Paul
Sep 27, 2021
Paul
Sep 27, 2021
Vitaliy Fadeev
Sep 27, 2021
Paul
Sep 27, 2021
Vitaliy Fadeev
September 26, 2021

I'm building a binary file. I can write my 'short[] myArray' directly to the file using: File f = File( "myFile.wav", "wb" ); f.rawWrite(myArray); It doesn't write any array formatting stuff (i.e. '[ , , ]'); it just moves the data into myFile like I want.

I can't seem to do this with myStruct? myStruct is just a declaration and initialization of a dozen atomic types (e.g uint WAVE_Id = 0x45_56_41_57). When I issue f.rawWrite(myStruct), it writes "myStruct(...,...,etc. ) instead of just writing the data.

Is there way to write the myStruct data to the file in a single statement...or two?

Thanks for any assistance.

September 26, 2021
rawRead and rawWrite work with slices (arrays) of things. You need to use an array of your struct even if there is a single item. Here is an example:

import std.stdio;
import std.file;

struct S {
  int i;
  double d;
}

void readFrom(string name) {
  // We need an array Ses (length of 1 in this case)
  auto ses = new S[1];

  File(name, "r").rawRead(ses);

  // Here is the first item of those ses:
  writefln!"Just read: %s"(ses[0]);
}

void writeTo(string name) {
  // Note the array of S passed to rawWrite
  File(name, "w").rawWrite([S(42, 1.5)]);
}

void main() {
  enum name = "rawWrite_struct_test";

  if (name.exists) {
    readFrom(name);
  }

  writeTo(name);
}

The program will create a file when you run it the first time and then it will also read from the that file on subsequent runs.

Ali

September 26, 2021
Correcting my sloppy code. :)

On 9/26/21 8:53 AM, Ali Çehreli wrote:
>    // We need an array Ses (length of 1 in this case)
>    auto ses = new S[1];

Allocating a dynamic array there is egregious pessimization. The following works the same but this time the static array of a single S will be on the stack:

  S[1] ses;

>    // Note the array of S passed to rawWrite
>    File(name, "w").rawWrite([S(42, 1.5)]);

Similarly, a temporary dynamic array is created there. Instead, we can have single S and make a slice of length 1 from that element:

  // Note the array of S passed to rawWrite
  auto s = S(42, 1.5);
  File(name, "w").rawWrite((&s)[0..1]);

The expression passed to rawWrite is complicated. It takes advantage of D's "make a slice from a pointer" feature.

- &s is a pointer to variable s
- If p is a pointer, the syntax p[0..length] makes a slice of 'length' elements starting from 'p'.

In this case, we are treating the variable 's' as a slice of 1.

Ali


September 26, 2021

On 9/26/21 11:09 AM, Paul wrote:

>

I'm building a binary file.  I can write my 'short[] myArray' directly to the file using: File f = File( "myFile.wav", "wb" ); f.rawWrite(myArray); It doesn't write any array formatting stuff (i.e. '[ ,  , ]'); it just moves the data into myFile like I want.

I can't seem to do this with myStruct? myStruct is just a declaration and initialization of a dozen atomic types (e.g uint WAVE_Id = 0x45_56_41_57).  When I issue f.rawWrite(myStruct), it writes "myStruct(...,...,etc. ) instead of just writing the data.

What is happening is that myStruct must have an alias-this to an array, and it is writing this array.

That's the only reasonable explanation I can think of.

>

Is there way to write the myStruct data to the file in a single statement...or two?

f.rawWrite((&myStruct)[0 .. 1]);

This will change the myStruct into an array of one struct instance, and then it can deal with it.

-Steve

September 26, 2021

Hmm...well this is what I did and sort of got what I wanted; it did compile and write data!

auto myStruct = new mystruct[1];
File f = File("myFile.wav", "wb");
f.rawWrite(myStruct); //this is 44 bytes
f.rawWrite(shortWaveArray);

What I got in the file was this:
-my 44 byte myStruct data + (4 bytes of zero)!
-then, my shortWaveArray data

What's with the 4 bytes of zero?

Thanks

September 26, 2021
>

What's with the 4 bytes of zero?

I miss-counted. All is well!

Thanks Ali / Steven

September 27, 2021

Finished product...
~15k samples x 2 sin() waves/composite wave x 16 DTMF tones = 16 DTMF wave files in ~40ms!

I love D.

September 27, 2021

On Sunday, 26 September 2021 at 15:09:38 UTC, Paul wrote:

>

Is there way to write the myStruct data to the file in a single statement...or two?

Thanks for any assistance.

    Header[1] header;

    void readFileHeader( ref File f )
    {
        f.rawRead( header );
    }

Such i read raw data from .ttf file.

And will be logically if will to write:

    Header[1] header;

    void writeFileHeader( ref File f )
    {
        f.rawWrite( header );
    }

Declaration rawRead/rawWrite is useful, beauty and logical:

    void rawWrite(T)( in T[] buffer );

Reason for using array for return number of readed bytes:

    auto readed = f.rawRead( header );
    // readed.length
    // readed[0]
    // readed[1]

And logically and symmetrically use rawWrite() with array:

    Header[1] header;
    f.rawWrite( header );

You can write other implementation:

    void rawWrite( T )( ref FILE f, T data )
    {
        //
    }

Why not ?

And check the cerealed: https://code.dlang.org/packages/cerealed

    struct Foo
    {
        int i;
    }

    const foo = Foo(5);

    f.rawWrite( foo.cerealize ) // is ubyte[]
September 27, 2021
Vitaliy,

Thanks for your assistance.  I was looking at your serialization package.  Is your example correct?

struct MyStruct {
        ubyte mybyte1;
        @NoCereal uint nocereal1; //won't be serialised
        @Bits!4 ubyte nibble;
        @Bits!1 ubyte bit;
        @Bits!3 ubyte bits3;
        ubyte mybyte2; }

assert(MyStruct(3, 123, 14, 1, 42).cerealise == [ 3, 0xea /*1110 1 010*/, 42]);

       mybyte1   =   3 ?
       nocereal1 = 123 ?
       nibble    =  14 ?
       bit       =   1 ?
   ??? bits3     = ___ ???
       mybyte2   =  42 ?
September 27, 2021

On Monday, 27 September 2021 at 13:45:19 UTC, Paul wrote:

>

Vitaliy,

Thanks for your assistance. I was looking at your serialization package. Is your example correct?

struct MyStruct {
ubyte mybyte1;
@NoCereal uint nocereal1; //won't be serialised
@Bits!4 ubyte nibble;
@Bits!1 ubyte bit;
@Bits!3 ubyte bits3;
ubyte mybyte2; }

assert(MyStruct(3, 123, 14, 1, 42).cerealise == [ 3, 0xea /1110 1 010/, 42]);

   mybyte1   =   3 ?
   nocereal1 = 123 ?
   nibble    =  14 ?
   bit       =   1 ?

??? bits3 = ___ ???
mybyte2 = 42 ?

Author of package is https://github.com/atilaneves
Issues here: https://github.com/atilaneves/cerealed/issues

Yes, "Bits" not worked. No value for bits3.
Next is correct:

import std;
import cerealed;

    struct MyStruct
    {
        ubyte mybyte1;
        @NoCereal uint nocereal1; //won't be serialised
        @Bits!4 ubyte nibble;
        @Bits!1 ubyte bit;
        @Bits!3 ubyte bits3;
        ubyte mybyte2;
    }

    void main()
    {
        assert(  MyStruct(3, 123, 14, 1, 2, 42).cerealise == [ 3, 0xea /*1110 1 010*/, 42]);
        writeln( MyStruct(3, 123, 14, 1, 2, 42).cerealise );
    }
    ```

`[3, 234, 42]`

And check simple struct without attributes Bits, NoCereal:

import std;
import cerealed;

struct MyStruct
{
    ubyte mybyte1;
    ubyte mybyte2;
}

void main()
{
    writeln( MyStruct(3, 42).cerealise );
}
```

[3, 42]