I'm trying to read in just the first part of a .png file to peek at it's width and height without loading in the whole file. I'm using FreeImage for reading the whole file but since it doesn't have a function to let me peek at the image size before loading it all in I'm rolling my own.

I've gotten as a far as reading in the first 16 bytes which includes an 8 byte signature, then the length and type of the first chunk:

struct Header {
    ubyte[8] signature;
    uint length;
    char[4] type;
}
union Un {
    Header h;
    ubyte[16] u;
}

auto f = File(path, "r");
foreach (ubyte[] buffer; f.byChunk(16)){
    Un fff;
    fff.u = buffer[0..16];
    writeln(fff.h);
    break;
}

It prints out:
Header([137, 80, 78, 71, 13, 10, 26, 10], 218103808, "IHDR")

The signature is correct, the type is correct, but the value I get for length should be 13, not 218103808. So I'm guessing my ubytes are in the wrong order in the uint... how should I put them around the correct way so that my code won't break on another machine with different endianness?

On 06/23/2017 07:52 PM, Felix wrote:

> So I'm guessing my ubytes are in the
> wrong order in the uint... how should I put them around the correct way
> so that my code won't break on another machine with different endianness?

Yes, that would happen when your system is little-endian. (According to spec, the length field is big-endian.)

You can detect what endianness the system has and swap the bytes of the length field:

  http://ddili.org/ders/d.en/union.html#ix_union.endian,%20std.system

import std.system;
import core.bitop;

// ...

    if (endian == Endian.littleEndian) {
        address.value = bswap(address.value);
    }

std.bitmanip may be useful as well:

  https://dlang.org/phobos/std_bitmanip.html

Ali

That works, thanks!

On 06/23/2017 09:26 PM, Felix wrote:
> That works, thanks!

I've just tried this, which seems cleaner:

import std.stdio;
import std.system;
import std.bitmanip;

void ensureBigEndian(T)(ref T value) {
    if (endian == Endian.littleEndian) {
        value = *cast(T*)nativeToBigEndian(value).ptr;
    }
}

void main() {
    ubyte[] bytes = [ 0, 0, 0, 13 ];
    uint u = *cast(uint*)bytes.ptr;

    writefln("Just read: %s", u);
    u.ensureBigEndian;
    writefln("Converted: %s", u);
}

Just read: 218103808
Converted: 13

Ali

On Fri, Jun 23, 2017 at 10:10:22PM -0700, Ali Çehreli via Digitalmars-d-learn wrote:
> On 06/23/2017 09:26 PM, Felix wrote:
> > That works, thanks!
> 
> I've just tried this, which seems cleaner:
> 
> import std.stdio;
> import std.system;
> import std.bitmanip;
> 
> void ensureBigEndian(T)(ref T value) {
>     if (endian == Endian.littleEndian) {
>         value = *cast(T*)nativeToBigEndian(value).ptr;
>     }

This is wrong, you should be detecting the endianness of the input and use {big,little}EndianToNative instead.  For example, if the input is big endian, you should use bigEndianToNative.  Internally, if native is already big endian, it will do nothing; otherwise it will swap the endianness. This way you don't have to check the current machine's endianness yourself; you can just recompile on a machine of different endianness and it will Just Work.


T

-- 
People who are more than casually interested in computers should have at least some idea of what the underlying hardware is like. Otherwise the programs they write will be pretty weird. -- D. Knuth

On 06/23/2017 10:18 PM, H. S. Teoh via Digitalmars-d-learn wrote:
> On Fri, Jun 23, 2017 at 10:10:22PM -0700, Ali Çehreli via Digitalmars-d-learn wrote:
>> On 06/23/2017 09:26 PM, Felix wrote:
>>> That works, thanks!
>>
>> I've just tried this, which seems cleaner:
>>
>> import std.stdio;
>> import std.system;
>> import std.bitmanip;
>>
>> void ensureBigEndian(T)(ref T value) {
>>     if (endian == Endian.littleEndian) {
>>         value = *cast(T*)nativeToBigEndian(value).ptr;
>>     }
>
> This is wrong, you should be detecting the endianness of the input and
> use {big,little}EndianToNative instead.  For example, if the input is
> big endian, you should use bigEndianToNative.  Internally, if native is
> already big endian, it will do nothing; otherwise it will swap the
> endianness. This way you don't have to check the current machine's
> endianness yourself; you can just recompile on a machine of different
> endianness and it will Just Work.
>
>
> T
>

Thanks. Something like this:

import std.stdio;
import std.system;
import std.bitmanip;

void ensureCorrectFromBigEndian(T)(ref T value) {
    value = bigEndianToNative!(T, T.sizeof)(*cast(ubyte[T.sizeof]*)&value);
}

struct S {
    uint u;
}

void main() {
    // Bytes on the wire
    ubyte[] bytes = [ 0, 0, 0, 13 ];

    // Overlaying an object on those bytes
    S s = *cast(S*)bytes.ptr;

    void checkValue(uint expectedOnLE, uint expectedOnBE) {
        if (endian == Endian.littleEndian) {
            assert(s.u == expectedOnLE);
        } else if (endian == Endian.bigEndian) {
            assert(s.u == expectedOnBE);
        } else {
            assert(false, "What is this?");
        }
    }

    // The value should be wrong no a little-endian system
    checkValue(218103808, 13);

    s.u.ensureCorrectFromBigEndian;

    // No matter what, now the result will be correct on any system
    checkValue(13, 13);
}

Ali

On 6/24/17 1:18 AM, H. S. Teoh via Digitalmars-d-learn wrote:
> On Fri, Jun 23, 2017 at 10:10:22PM -0700, Ali Çehreli via Digitalmars-d-learn wrote:
>> On 06/23/2017 09:26 PM, Felix wrote:
>>> That works, thanks!
>>
>> I've just tried this, which seems cleaner:
>>
>> import std.stdio;
>> import std.system;
>> import std.bitmanip;
>>
>> void ensureBigEndian(T)(ref T value) {
>>      if (endian == Endian.littleEndian) {
>>          value = *cast(T*)nativeToBigEndian(value).ptr;
>>      }
> 
> This is wrong, you should be detecting the endianness of the input and
> use {big,little}EndianToNative instead.  For example, if the input is
> big endian, you should use bigEndianToNative.  Internally, if native is
> already big endian, it will do nothing; otherwise it will swap the
> endianness. This way you don't have to check the current machine's
> endianness yourself; you can just recompile on a machine of different
> endianness and it will Just Work.

I would also point out that there are pre-defined versions for endianness: version(BigEndian) and version(LittleEndian). This makes static checking a lot easier when you are writing your own code that deals with endiannness.

For byte-swapping one field, I would use the standard tools as H.S. has advised.

-Steve

On Saturday, 24 June 2017 at 02:52:23 UTC, Felix wrote:
> I'm trying to read in just the first part of a .png file to peek at it's width and height without loading in the whole file. I'm using FreeImage for reading the whole file but since it doesn't have a function to let me peek at the image size before loading it all in I'm rolling my own.
>
> I've gotten as a far as reading in the first 16 bytes which includes an 8 byte signature, then the length and type of the first chunk:
>
> struct Header {
>     ubyte[8] signature;
>     uint length;
>     char[4] type;
> }
> union Un {
>     Header h;
>     ubyte[16] u;
> }
>
> auto f = File(path, "r");
> foreach (ubyte[] buffer; f.byChunk(16)){
>     Un fff;
>     fff.u = buffer[0..16];
>     writeln(fff.h);
>     break;
> }
>
> It prints out:
> Header([137, 80, 78, 71, 13, 10, 26, 10], 218103808, "IHDR")
>
> The signature is correct, the type is correct, but the value I get for length should be 13, not 218103808. So I'm guessing my ubytes are in the wrong order in the uint... how should I put them around the correct way so that my code won't break on another machine with different endianness?

I would advise a look at sqlite.d https://github.com/UplinkCoder/sqlite-d
which solves the same problem albeit for sqlite database files and for png.
But it provides the means of simply putting a BigEndian!uint into your struct and have conversion on stuff be automatically handled.