Is there a way to create a 24-bit int? One that for all practical purposes acts as such? This is for 24-bit stuff like audio. It would respect endianness, allow for arrays int24[] that work properly, etc.

On Friday, 1 September 2017 at 19:39:14 UTC, EntangledQuanta wrote:
> Is there a way to create a 24-bit int? One that for all practical purposes acts as such? This is for 24-bit stuff like audio. It would respect endianness, allow for arrays int24[] that work properly, etc.

I haven't looked at endianness beyond it working on my computer. If you have special needs in that regard, consider this a starting point:


struct int24 {
    ubyte[3] _payload;

    this(int x) {
        value = x;
    }

    @property
    int value() {
        int val = *cast(int*)&_payload & 0xFFFFFF;

        if (val & 0x800000) {
            val |= 0xFF000000;
        }

        return val;
    }

    @property
    int value(int x) {
        _payload = (cast(ubyte*)&x)[0..3];
        return value;
    }

    auto opUnary(string op)() {
        static if (op == "++") {
            value = value + 1;
        } else static if (op == "--") {
            value = value - 1;
        } else static if (op == "+") {
            return value;
        } else static if (op == "-") {
            return -value;
        } else static if (op == "~") {
            return ~value;
        } else {
            static assert(false, "Unary operator '"~op~"' is not supported by int24.");
        }
    }

    auto opOpAssign(string op)(int x) {
        static assert(__traits(compiles, {mixin("value = value "~op~" x;");}), "Binary operator '"~op~"' is not supported by int24.");

        mixin("value = value "~op~" x;");
        return this;
    }

    alias value this;
}

unittest {
    int24[3] a;
    assert(a.sizeof == 9);

    // Max value
    a[1] = 8388607;
    assert(a[1] == 8388607);
    // Test for buffer overflow:
    assert(a[0] == 0);
    assert(a[2] == 0);

    // Overflow
    a[1] = 8388608;
    assert(a[1] == -8388608);
    // Test for buffer overflow:
    assert(a[0] == 0);
    assert(a[2] == 0);

    // negative value
    a[1] = -1;
    assert(a[1] == -1);
    // Test for buffer overflow:
    assert(a[0] == 0);
    assert(a[2] == 0);

    // Unary operators
    a[1] = 0;
    assert(~a[1] == -1);
    a[1]--;
    assert(a[1] == -1);
    assert(-a[1] == 1);
    assert(+a[1] == -1);
    a[1]++;
    assert(a[1] == 0);

    // Binary operators
    a[1] = 0;
    a[1] = a[1] + 1;
    assert(a[1] == 1);
    a[1] += 1;
    assert(a[1] == 2);
    a[1] = a[1] - 1;
    assert(a[1] == 1);
    a[1] -= 1;
    assert(a[1] == 0);

    a[1] = 3;
    a[1] = a[1] * 2;
    assert(a[1] == 6);
    a[1] = a[1] / 2;
    assert(a[1] == 3);
    a[1] *= 2;
    assert(a[1] == 6);
    a[1] /= 2;
    assert(a[1] == 3);
    a[1] = a[1] << 1;
    assert(a[1] == 6);
    a[1] <<= 1;
    assert(a[1] == 12);
    a[1] = a[1] >> 1;
    assert(a[1] == 6);
    a[1] >>= 1;
    assert(a[1] == 3);

    a[1] |= 4;
    assert(a[1] == 7);
    a[1] &= 5;
    assert(a[1] == 5);
    a[1] = a[1] | 2;
    assert(a[1] == 7);
    a[1] = a[1] & 3;
    assert(a[1] == 3);
}

--
  Biotronic

On Friday, 1 September 2017 at 22:10:43 UTC, Biotronic wrote:
> On Friday, 1 September 2017 at 19:39:14 UTC, EntangledQuanta wrote:
>> [...]
>
> I haven't looked at endianness beyond it working on my computer. If you have special needs in that regard, consider this a starting point:
>
> [...]

Thanks, I'll check it out and see.

On Friday, 1 September 2017 at 22:10:43 UTC, Biotronic wrote:
> On Friday, 1 September 2017 at 19:39:14 UTC, EntangledQuanta wrote:
>> Is there a way to create a 24-bit int? One that for all practical purposes acts as such? This is for 24-bit stuff like audio. It would respect endianness, allow for arrays int24[] that work properly, etc.
>
> I haven't looked at endianness beyond it working on my computer. If you have special needs in that regard, consider this a starting point:
>
>
> struct int24 {
>     ubyte[3] _payload;
>
>     this(int x) {
>         value = x;
>     }
>
>     ...
> }
>
> --
>   Biotronic

You may also want to put an align(1) on it so that you dont waste 25% of the allocated memory in an array of int24's

On Saturday, 2 September 2017 at 00:43:00 UTC, Nicholas Wilson wrote:
> On Friday, 1 September 2017 at 22:10:43 UTC, Biotronic wrote:
>> On Friday, 1 September 2017 at 19:39:14 UTC, EntangledQuanta wrote:
>>> Is there a way to create a 24-bit int? One that for all practical purposes acts as such? This is for 24-bit stuff like audio. It would respect endianness, allow for arrays int24[] that work properly, etc.
>>
>> I haven't looked at endianness beyond it working on my computer. If you have special needs in that regard, consider this a starting point:
>>
>>
>> struct int24 {
>>     ubyte[3] _payload;
>>
>>     this(int x) {
>>         value = x;
>>     }
>>
>>     ...
>> }
>>
>> --
>>   Biotronic
>
> You may also want to put an align(1) on it so that you dont waste 25% of the allocated memory in an array of int24's

The whole point is so that there is no wasted space, so if it requires that then it's not a waste of space but a bug.

Audio that is in24 is 3 bytes per sample, not 4. Every 3 bytes are a sample, not every 3 out of 4.

Basically a byte[] cast to a int24 array should be 1/3 the size and every 3 bytes are the same as an int24.

Thanks for pointing this out if it is necessary.

On Saturday, 2 September 2017 at 01:19:52 UTC, EntangledQuanta wrote:
>
> The whole point is so that there is no wasted space, so if it requires that then it's not a waste of space but a bug.
>
> Audio that is in24 is 3 bytes per sample, not 4. Every 3 bytes are a sample, not every 3 out of 4.
>
> Basically a byte[] cast to a int24 array should be 1/3 the size and every 3 bytes are the same as an int24.
>
> Thanks for pointing this out if it is necessary.

It's not a bug, but a feature. Data structure alignment is important for efficient reads, so several languages (D, C, C++, Ada, and more) will automatically pad structs so that they can maintain specific byte alignments. On a 32-bit system, 4-byte boundaries are the default. So a struct with 3 ubytes is going to be padded with an extra byte at the end. Telling the compiler to align on a 1-byte boundary (essentially disabling alignment) will save you space, but will will generally cost you cycles in accessing the data.

On Friday, 1 September 2017 at 19:39:14 UTC, EntangledQuanta wrote:
> Is there a way to create a 24-bit int? One that for all practical purposes acts as such? This is for 24-bit stuff like audio. It would respect endianness, allow for arrays int24[] that work properly, etc.

Hi,

Probably you are looking for bitpack ndslice topology:
http://docs.algorithm.dlang.io/latest/mir_ndslice_topology.html#.bitpack

sizediff_t[] data;
// creates a packed signed integer slice with max allowed value equal to `2^^24 - 1`.
auto packs = data[].sliced.bitpack!24;

packs has the same API as D arrays

Package is Mir Algorithm
http://code.dlang.org/packages/mir-algorithm

Best,
Ilya

On Saturday, 2 September 2017 at 02:37:08 UTC, Mike Parker wrote:
> On Saturday, 2 September 2017 at 01:19:52 UTC, EntangledQuanta wrote:
>>
>> The whole point is so that there is no wasted space, so if it requires that then it's not a waste of space but a bug.
>>
>> Audio that is in24 is 3 bytes per sample, not 4. Every 3 bytes are a sample, not every 3 out of 4.
>>
>> Basically a byte[] cast to a int24 array should be 1/3 the size and every 3 bytes are the same as an int24.
>>
>> Thanks for pointing this out if it is necessary.
>
> It's not a bug, but a feature. Data structure alignment is important for efficient reads, so several languages (D, C, C++, Ada, and more) will automatically pad structs so that they can maintain specific byte alignments. On a 32-bit system, 4-byte boundaries are the default. So a struct with 3 ubytes is going to be padded with an extra byte at the end. Telling the compiler to align on a 1-byte boundary (essentially disabling alignment) will save you space, but will will generally cost you cycles in accessing the data.

You fail to read correctly. A bug in his code. If he is treating in24's as int32's and simply ignoring the upper byte then it is not a true int24 and all the work he did would be pointless. I can do that by simply reading an int32 and masking the high bit.

On Saturday, 2 September 2017 at 02:49:41 UTC, Ilya Yaroshenko wrote:
> On Friday, 1 September 2017 at 19:39:14 UTC, EntangledQuanta wrote:
>> Is there a way to create a 24-bit int? One that for all practical purposes acts as such? This is for 24-bit stuff like audio. It would respect endianness, allow for arrays int24[] that work properly, etc.
>
> Hi,
>
> Probably you are looking for bitpack ndslice topology:
> http://docs.algorithm.dlang.io/latest/mir_ndslice_topology.html#.bitpack
>
> sizediff_t[] data;
> // creates a packed signed integer slice with max allowed value equal to `2^^24 - 1`.
> auto packs = data[].sliced.bitpack!24;
>
> packs has the same API as D arrays
>
> Package is Mir Algorithm
> http://code.dlang.org/packages/mir-algorithm
>
> Best,
> Ilya

Thanks. Seems useful.

On Saturday, 2 September 2017 at 02:37:08 UTC, Mike Parker wrote:
> It's not a bug, but a feature. Data structure alignment is important for efficient reads, so several languages (D, C, C++, Ada, and more) will automatically pad structs so that they can maintain specific byte alignments. On a 32-bit system, 4-byte boundaries are the default. So a struct with 3 ubytes is going to be padded with an extra byte at the end. Telling the compiler to align on a 1-byte boundary (essentially disabling alignment) will save you space, but will will generally cost you cycles in accessing the data.

struct int24 {
    ubyte[3] _payload;
}

static assert(int24.sizeof == 3);
static assert(int24.alignof == 1);

Making absolute sense. ubytes don't need any specific alignment to be read efficiently.