I have to read a binary file.
I can use std.stdio.File.rawRead to do this (and it's even typesafe, awesome!!)

Currently, I'm doing this with a little helper function I've called Get:
    /// Creates a buffer to read the file. Handles a special case string
    T[] Get(T)(File f, size_t num=1){
	static if(is(T == char)){
	    ubyte[] strLen = new ubyte[1];
	    f.rawRead(strLen);
	    ubyte[] strBuf = new ubyte[strLen[0]];
	    return (f.rawRead(strBuf));
	} else {	
	    T[] buf = new T[num];
            return f.rawRead(buf);
	}
    }

Then to use it:
File f = File(filename, "rb");

auto versionNumber = f.Get!(ushort); // reads 2 bytes from file into a ushort[]
auto nextByte = f.Get!(byte); // reads 1 byte into a byte[]
auto next5Bytes = f.Get!(byte)(5); // reads 5 bytes and puts them into a byte[]

There is a lot of improvements to be made to the above, mostly around returning a Range. Also, reusing buffers so as not to allocate so much. That's probably Priority 1 actually.
That's all fine and dandy, will get to that.

There's some more complicated data in there too, like a string which in this binary file is defined as a byte representing the number of bytes in this string, followed by that number of bytes. (Similar to a char[] in C/D I imagine).

So, to read that, I'd write:
    auto myString = f.Get!(char)(f.Get!(byte)); // reads f.Get!byte number of bytes into a char array.

While doing this I had the idea of implementing a more general approach to this, using CTFE to build a struct / parserFunction (Similar to Pegged [1] ).
You describe at compile time how this binary file looks, and then the parser handles everything else.

Does anyone know of a good Binary Description Grammar out in the wild? I'd rather not re-invent the wheel on that front, and just use something standard. My Googling didn't come up with something that could be considered "standard" however.

Any ideas?

Thanks

[1] https://github.com/PhilippeSigaud/Pegged

it looks like you can use some serialization library instead, like Orange. structs that those library takes as input can be seen as a kind of grammar description. ;-)

On Monday, 10 August 2015 at 12:38:11 UTC, ketmar wrote:
> it looks like you can use some serialization library instead, like Orange. structs that those library takes as input can be seen as a kind of grammar description. ;-)

Trouble with that is what if there's some funny formats some data is in? Like I described above a string which starts with a number and continues for that number of bytes.
What if it's a string that goes on until you hit a '\0' ? A serialisation library like Orange couldnt do that, as it's merely for (de)serialising D objects, I think?

On 10/08/15 15:12, wobbles wrote:

> Trouble with that is what if there's some funny formats some data is in?
> Like I described above a string which starts with a number and continues
> for that number of bytes.
> What if it's a string that goes on until you hit a '\0' ? A
> serialisation library like Orange couldnt do that, as it's merely for
> (de)serialising D objects, I think?

Orange can handle D strings. If it sees something like char* it will serialize it as a pointer to a char, not a C string. Orange supports custom archivers if you need the data in some special format.

-- 
/Jacob Carlborg

On Monday, 10 August 2015 at 12:29:43 UTC, wobbles wrote:
>
> While doing this I had the idea of implementing a more general approach to this, using CTFE to build a struct / parserFunction (Similar to Pegged [1] ).
> You describe at compile time how this binary file looks, and then the parser handles everything else.
>
> Does anyone know of a good Binary Description Grammar out in the wild? I'd rather not re-invent the wheel on that front, and just use something standard. My Googling didn't come up with something that could be considered "standard" however.
>
> Any ideas?
>

You should look into Construct (Python) - https://github.com/construct/construct
If you are not afraid of Haskell, also take a look at https://github.com/bos/attoparsec

On Monday, 10 August 2015 at 12:29:43 UTC, wobbles wrote:
> I have to read a binary file.
> I can use std.stdio.File.rawRead to do this (and it's even typesafe, awesome!!)
>
> [...]

https://github.com/atilaneves/cerealed

If that doesn't do what you need, I've done something wrong.

Atila

On Tue, 11 Aug 2015 07:51:37 +0000, Atila Neves wrote:

> On Monday, 10 August 2015 at 12:29:43 UTC, wobbles wrote:
>> I have to read a binary file.
>> I can use std.stdio.File.rawRead to do this (and it's even typesafe,
>> awesome!!)
>>
>> [...]
> 
> https://github.com/atilaneves/cerealed
> 
> If that doesn't do what you need, I've done something wrong.

a strange thing: i myself used only cerealed, but somehow keep thinking that it was orange, and speaking about orange.

i should setup autoreplacement in my nntp client...

On Tuesday, 11 August 2015 at 07:51:39 UTC, Atila Neves wrote:
> On Monday, 10 August 2015 at 12:29:43 UTC, wobbles wrote:
>> I have to read a binary file.
>> I can use std.stdio.File.rawRead to do this (and it's even typesafe, awesome!!)
>>
>> [...]
>
> https://github.com/atilaneves/cerealed
>
> If that doesn't do what you need, I've done something wrong.
>
> Atila

Yep, this seems to do the job!

I'll investigate tonight, and pester you later :D

On Tuesday, 11 August 2015 at 13:40:52 UTC, wobbles wrote:
> On Tuesday, 11 August 2015 at 07:51:39 UTC, Atila Neves wrote:
>> On Monday, 10 August 2015 at 12:29:43 UTC, wobbles wrote:
>>> I have to read a binary file.
>>> I can use std.stdio.File.rawRead to do this (and it's even typesafe, awesome!!)
>>>
>>> [...]
>>
>> https://github.com/atilaneves/cerealed
>>
>> If that doesn't do what you need, I've done something wrong.
>>
>> Atila
>
> Yep, this seems to do the job!
>
> I'll investigate tonight, and pester you later :D

Pester away! That's what I get for putting it out there. :P

Atila