i have an array of ubytes. how can i convert two adjacent ubytes from the array to an integer?

pseudocode example:
ubyte[5] array = createArray();
int value = array[2..3];

is there any 'memcpy' method or something else to do this?

On Wed, 16 May 2012 15:24:33 +0100, ref2401 <refactor24@gmail.com> wrote:

> i have an array of ubytes. how can i convert two adjacent ubytes from the array to an integer?
>
> pseudocode example:
> ubyte[5] array = createArray();
> int value = array[2..3];
>
> is there any 'memcpy' method or something else to do this?

You don't need to "copy" the data, just tell the compiler to "pretend" it's a short (in this case, for 2 bytes) then copy the value/assign to an int. e.g.

import std.stdio;

void main()
{
	ubyte[5] array = [ 0xFF, 0xFF, 0x01, 0x00, 0xFF ];
	int value = *cast(short*)array[2..3].ptr;
	writefln("Result = %s", value);
}

The line:
  int value = *cast(short*)array[2..3].ptr;

1. slices 2 bytes from the array.
2. obtains the ptr to them
3. casts the ptr to short*
4. copies the value pointed at by the short* ptr to an int

You may need to worry about little/big endian issues, see:
http://en.wikipedia.org/wiki/Endianness

The above code outputs "Result = 1" on my little-endian x86 desktop machine but would output "Result = 256" on a big-endian machine.

R

-- 
Using Opera's revolutionary email client: http://www.opera.com/mail/

On Wednesday, May 16, 2012 17:03:44 Regan Heath wrote:
> On Wed, 16 May 2012 15:24:33 +0100, ref2401 <refactor24@gmail.com> wrote:
> > i have an array of ubytes. how can i convert two adjacent ubytes from the array to an integer?
> > 
> > pseudocode example:
> > ubyte[5] array = createArray();
> > int value = array[2..3];
> > 
> > is there any 'memcpy' method or something else to do this?
> 
> You don't need to "copy" the data, just tell the compiler to "pretend" it's a short (in this case, for 2 bytes) then copy the value/assign to an int. e.g.
> 
> import std.stdio;
> 
> void main()
> {
> ubyte[5] array = [ 0xFF, 0xFF, 0x01, 0x00, 0xFF ];
> int value = *cast(short*)array[2..3].ptr;
> writefln("Result = %s", value);
> }
> 
> The line:
> int value = *cast(short*)array[2..3].ptr;
> 
> 1. slices 2 bytes from the array.
> 2. obtains the ptr to them
> 3. casts the ptr to short*
> 4. copies the value pointed at by the short* ptr to an int
> 
> You may need to worry about little/big endian issues, see: http://en.wikipedia.org/wiki/Endianness
> 
> The above code outputs "Result = 1" on my little-endian x86 desktop machine but would output "Result = 256" on a big-endian machine.

As long as you're going from big endian to little endian, std.bitmanip.bigEndianToNative will do the conversion fairly easily, but if they're in little endian, then the nasty casting is the way to go.

- Jonathan M Davis

On Wednesday, 16 May 2012 at 18:47:55 UTC, Jonathan M Davis wrote:
> As long as you're going from big endian to little endian,
> std.bitmanip.bigEndianToNative will do the conversion fairly easily, but if
> they're in little endian, then the nasty casting is the way to go.
>
> - Jonathan M Davis

Except they don't take slices. You need a helper function.

ubyte[2] _2(ubyte[] a){ubyte[2] b; assert(a.length==2); b[]=a; return b;}

On Wed, May 16, 2012 at 11:03 AM, Regan Heath <regan@netmail.co.nz> wrote:

> On Wed, 16 May 2012 15:24:33 +0100, ref2401 <refactor24@gmail.com> wrote:
>
>  i have an array of ubytes. how can i convert two adjacent ubytes from the
>> array to an integer?
>>
>> pseudocode example:
>> ubyte[5] array = createArray();
>> int value = array[2..3];
>>
>> is there any 'memcpy' method or something else to do this?
>>
>
> You don't need to "copy" the data, just tell the compiler to "pretend" it's a short (in this case, for 2 bytes) then copy the value/assign to an int. e.g.
>
> import std.stdio;
>
> void main()
> {
>        ubyte[5] array = [ 0xFF, 0xFF, 0x01, 0x00, 0xFF ];
>        int value = *cast(short*)array[2..3].ptr;
>        writefln("Result = %s", value);
> }
>
> The line:
>  int value = *cast(short*)array[2..3].ptr;
>
> 1. slices 2 bytes from the array.
> 2. obtains the ptr to them
> 3. casts the ptr to short*
> 4. copies the value pointed at by the short* ptr to an int
>
> You may need to worry about little/big endian issues, see: http://en.wikipedia.org/wiki/**Endianness<http://en.wikipedia.org/wiki/Endianness>
>
> The above code outputs "Result = 1" on my little-endian x86 desktop machine but would output "Result = 256" on a big-endian machine.
>
> R
>
>
Unfortunately, this is undefined behavior because you're breaking alignment
rules. On x86, this will just cause a slow load from memory. On ARM, this
will either crash your program with a bus error on newer hardware or give
you a gibberish value on ARMv6 and older.
Declaring a short, getting a pointer to it, and casting that pointer to a
ubyte* to copy into it is fine, but casting a ubyte* to a short* will cause
a 2-byte load from a 1-byte aligned address, which leads down the yellow
brick road to pain.

On Wed, May 16, 2012 at 10:25:51PM -0500, Andrew Wiley wrote:
> On Wed, May 16, 2012 at 11:03 AM, Regan Heath <regan@netmail.co.nz> wrote:
> 
> > On Wed, 16 May 2012 15:24:33 +0100, ref2401 <refactor24@gmail.com> wrote:
> >
> >  i have an array of ubytes. how can i convert two adjacent ubytes from the
> >> array to an integer?
> >>
> >> pseudocode example:
> >> ubyte[5] array = createArray();
> >> int value = array[2..3];
> >>
> >> is there any 'memcpy' method or something else to do this?
> >>
> >
> > You don't need to "copy" the data, just tell the compiler to "pretend" it's a short (in this case, for 2 bytes) then copy the value/assign to an int. e.g.
> >
> > import std.stdio;
> >
> > void main()
> > {
> >        ubyte[5] array = [ 0xFF, 0xFF, 0x01, 0x00, 0xFF ];
> >        int value = *cast(short*)array[2..3].ptr;
> >        writefln("Result = %s", value);
> > }
> >
> > The line:
> >  int value = *cast(short*)array[2..3].ptr;
> >
> > 1. slices 2 bytes from the array.
> > 2. obtains the ptr to them
> > 3. casts the ptr to short*
> > 4. copies the value pointed at by the short* ptr to an int
> >
> > You may need to worry about little/big endian issues, see: http://en.wikipedia.org/wiki/**Endianness<http://en.wikipedia.org/wiki/Endianness>
> >
> > The above code outputs "Result = 1" on my little-endian x86 desktop machine but would output "Result = 256" on a big-endian machine.
> >
> > R
> >
> >
> Unfortunately, this is undefined behavior because you're breaking
> alignment rules. On x86, this will just cause a slow load from memory.
> On ARM, this will either crash your program with a bus error on newer
> hardware or give you a gibberish value on ARMv6 and older.
> Declaring a short, getting a pointer to it, and casting that pointer
> to a ubyte* to copy into it is fine, but casting a ubyte* to a short*
> will cause a 2-byte load from a 1-byte aligned address, which leads
> down the yellow brick road to pain.

Do unions suffer from this problem? Could this prevent alignment problems:

	short bytesToShort(ubyte[] b)
	in { assert(b.length==2); }
	body {
		union U {
			short val;
			ubyte[2] b;
		}
		U u;

		u.b[] = b[];
		return u.val;
	}

?


T

-- 
Береги платье снову, а здоровье смолоду.

On Wed, May 16, 2012 at 11:07 PM, H. S. Teoh <hsteoh@quickfur.ath.cx> wrote:

> On Wed, May 16, 2012 at 10:25:51PM -0500, Andrew Wiley wrote:
> > On Wed, May 16, 2012 at 11:03 AM, Regan Heath <regan@netmail.co.nz>
> wrote:
> >
> > > On Wed, 16 May 2012 15:24:33 +0100, ref2401 <refactor24@gmail.com>
> wrote:
> > >
> > >  i have an array of ubytes. how can i convert two adjacent ubytes from
> the
> > >> array to an integer?
> > >>
> > >> pseudocode example:
> > >> ubyte[5] array = createArray();
> > >> int value = array[2..3];
> > >>
> > >> is there any 'memcpy' method or something else to do this?
> > >>
> > >
> > > You don't need to "copy" the data, just tell the compiler to "pretend" it's a short (in this case, for 2 bytes) then copy the value/assign to
> an
> > > int. e.g.
> > >
> > > import std.stdio;
> > >
> > > void main()
> > > {
> > >        ubyte[5] array = [ 0xFF, 0xFF, 0x01, 0x00, 0xFF ];
> > >        int value = *cast(short*)array[2..3].ptr;
> > >        writefln("Result = %s", value);
> > > }
> > >
> > > The line:
> > >  int value = *cast(short*)array[2..3].ptr;
> > >
> > > 1. slices 2 bytes from the array.
> > > 2. obtains the ptr to them
> > > 3. casts the ptr to short*
> > > 4. copies the value pointed at by the short* ptr to an int
> > >
> > > You may need to worry about little/big endian issues, see: http://en.wikipedia.org/wiki/**Endianness<
> http://en.wikipedia.org/wiki/Endianness>
> > >
> > > The above code outputs "Result = 1" on my little-endian x86 desktop machine but would output "Result = 256" on a big-endian machine.
> > >
> > > R
> > >
> > >
> > Unfortunately, this is undefined behavior because you're breaking
> > alignment rules. On x86, this will just cause a slow load from memory.
> > On ARM, this will either crash your program with a bus error on newer
> > hardware or give you a gibberish value on ARMv6 and older.
> > Declaring a short, getting a pointer to it, and casting that pointer
> > to a ubyte* to copy into it is fine, but casting a ubyte* to a short*
> > will cause a 2-byte load from a 1-byte aligned address, which leads
> > down the yellow brick road to pain.
>
> Do unions suffer from this problem? Could this prevent alignment problems:
>
>        short bytesToShort(ubyte[] b)
>        in { assert(b.length==2); }
>        body {
>                union U {
>                        short val;
>                        ubyte[2] b;
>                }
>                U u;
>
>                u.b[] = b[];
>                return u.val;
>        }
>
> ?
>
>
As I understand it, this should be fine because the compiler will guarantee that the union is aligned to the maximum alignment required by one of its members, which is the short. This is probably the safest solution.

On Thursday, 17 May 2012 at 04:16:10 UTC, Andrew Wiley wrote:
> On Wed, May 16, 2012 at 11:07 PM, H. S. Teoh <hsteoh@quickfur.ath.cx> wrote:
>> Do unions suffer from this problem? Could this prevent alignment
>> problems:
>>
>>        short bytesToShort(ubyte[] b)
>>        in { assert(b.length==2); }
>>        body {
>>                union U {
>>                        short val;
>>                        ubyte[2] b;
>>                }
>>                U u;
>>
>>                u.b[] = b[];
>>                return u.val;
>>        }
>>
>> ?
>>
>>
> As I understand it, this should be fine because the compiler will guarantee
> that the union is aligned to the maximum alignment required by one of its
> members, which is the short. This is probably the safest solution.

And what about the following code:

// This implementation is optimized for speed via swapping
endianness in-place
pure immutable(C)[] fixEndian(C, Endian blobEndian =
endian)(ubyte[] blob) if(is(CharTypeOf!C))
{
     import std.bitmanip, std.system;
     auto data = cast(C[]) blob;
     static if(blobEndian != endian)
     {
         static assert(!is(typeof(C) == char)); // UTF-8 doesn't
have endianness
         foreach(ref ch; data) ch = swapEndian(ch);
     }
     return cast(immutable) data;
}

On Thursday, 17 May 2012 at 07:07:58 UTC, Roman D. Boiko wrote:
> And what about the following code:
>
> // This implementation is optimized for speed via swapping
> endianness in-place
> pure immutable(C)[] fixEndian(C, Endian blobEndian =
> endian)(ubyte[] blob) if(is(CharTypeOf!C))
> {
>      import std.bitmanip, std.system;
>      auto data = cast(C[]) blob;
>      static if(blobEndian != endian)
>      {
>          static assert(!is(typeof(C) == char)); // UTF-8 doesn't
> have endianness
>          foreach(ref ch; data) ch = swapEndian(ch);
>      }
>      return cast(immutable) data;
> }
I mean, is it safe (assuming that we are allowed to mutate blob, and its length is a multiple of C.sizeof)?

I do casting from ubyte[] to C[].

On 05/17/12 10:15, Roman D. Boiko wrote:
> On Thursday, 17 May 2012 at 07:07:58 UTC, Roman D. Boiko wrote:
>> And what about the following code:
>>
>> // This implementation is optimized for speed via swapping
>> endianness in-place
>> pure immutable(C)[] fixEndian(C, Endian blobEndian =
>> endian)(ubyte[] blob) if(is(CharTypeOf!C))
>> {
>>      import std.bitmanip, std.system;
>>      auto data = cast(C[]) blob;
>>      static if(blobEndian != endian)
>>      {
>>          static assert(!is(typeof(C) == char)); // UTF-8 doesn't
>> have endianness
>>          foreach(ref ch; data) ch = swapEndian(ch);
>>      }
>>      return cast(immutable) data;
>> }
> I mean, is it safe (assuming that we are allowed to mutate blob, and its length is a multiple of C.sizeof)?
> 
> I do casting from ubyte[] to C[].

Only if C.ptr ends up properly aligned. There are also aliasing
issues, which i don't think are sufficiently defined for D (for
C, it would be legal only because char* is allowed to alias anything).

artur