Hi,

In network programming (and many other applications), you often want to map a layout onto an area of memory, allowing you to access protocol fields in a packet buffer, for example. In C you cannot portably do this by casting to a struct because of alignment & packing and because you cannot fix the length of an integer (to 3 octets, say). Messing with compiler pragmas is only a partial solution and is not portable, by definition. The problem is usually solved by using pre-processor macros to copy octects one by one between the fields of the packet and variables. This is ugly and non-optimal without custom macros for every structure/field/compiler/processor.

It would be nice if D had support for this common problem. Perhaps a special type of struct which is always packed and for which the compiler generates the neccessary code for accessing the fields that have an non-processor native length or which straddle word boundries that the (non-x86) processor cannot natively support.

Chris.

Chris Paulson-Ellis wrote:
> In network programming (and many other applications), you often want to map a layout onto an area of memory, allowing you to access protocol fields in a packet buffer, for example. In C you cannot portably do this by casting to a struct because of alignment & packing and because you cannot fix the length of an integer (to 3 octets, say). [...snip...]
> 
> It would be nice if D had support for this common problem. Perhaps a special type of struct which is always packed and for which the compiler generates the neccessary code for accessing the fields that have an non-processor native length or which straddle word boundries that the (non-x86) processor cannot natively support.

The guaranteed alignment problem is covered in D. See:

 http://www.digitalmars.com/d/declaration.html (struct declarations)
 http://www.digitalmars.com/d/attribute.html (align attribute)

It's unclear at this time whether a D compiler is expected to generate code to access words with nonstandard alignments (on x86, I believe this is done automatically, but on most modern RISC CPUs, accessing a word or long on an odd address yields an exception; since Walter's only working on the x86 compiler at this time, he can choose not to choose).

There's no D-specified solution to dealing with things like 3-octet/24-bit ints. You'll still have to bit-twiddle for things like this.

-Russell B

Russell Borogove wrote:

> There's no D-specified solution to dealing with things like 3-octet/24-bit ints. You'll still have to bit-twiddle for things like this.

A nice, portable (but not performance-optimal) programming trick for this would be to use the struct as the binary layout and then implement a simple wrapper class that did the bit twiddling for you:

class FooPacket
{
   struct FooPacketLayout
   {
       ....
    } binaryLayout;

public:
   int GetParamX() { binaryLayout.....bit twiddling }
};

Russ Lewis wrote:

> Russell Borogove wrote:
> 
>> There's no D-specified solution to dealing with things like 3-octet/24-bit ints. You'll still have to bit-twiddle for things like this.
> 
> A nice, portable (but not performance-optimal) programming trick for this would be to use the struct as the binary layout and then implement a simple wrapper class that did the bit twiddling for you:
> 
> class FooPacket
> {
>    struct FooPacketLayout
>    {
>        ....
>     } binaryLayout;
> 
> public:
>    int GetParamX() { binaryLayout.....bit twiddling }
> };
> 
> 

See it? What have I told what the output of seperated class and struct elements in the class is?

Why use a struct inside the class? Why not make the contents fields of the class itself?

Im my opinion seperated class and struct elements only tends to confuse people, and quickly results into suboptimal results during development.

- Axel

Ahh - align(1){fields...} is good, although I'm not sure of the value of the
colon syntax...

attribute:
    declaration;
    declaration;

... given the existance of the scoped syntax...

attribute {
    declaration;
    declaration;
}

Especially as the colon could easily be missed (or mistakely assumed to be present) by the fast scanning eye of a fatigued programmer.

It would be well worth pinning down the behavior of misaligned word accesses in the language spec before someone comes up with the first non-x86 compiler. I take it no-one has a valuable use for those bus errors? :-)

The nice-to-have 24-bit ints (or whatever) would be the icing on the cake... and debate is likely to be side tracked into the "can we specify a range in an int typedef?" question.

Chris.

"Russell Borogove" <kaleja@estarcion.com> wrote in message news:3BAF78FD.82DBF71C@estarcion.com...
> [snip]
> The guaranteed alignment problem is covered in D. See:
>
>  http://www.digitalmars.com/d/declaration.html (struct declarations)
>  http://www.digitalmars.com/d/attribute.html (align attribute)
>
> It's unclear at this time whether a D compiler is expected to generate code to access words with nonstandard alignments (on x86, I believe this is done automatically, but on most modern RISC CPUs, accessing a word or long on an odd address yields an exception; since Walter's only working on the x86 compiler at this time, he can choose not to choose).
>
> There's no D-specified solution to dealing with things like 3-octet/24-bit ints. You'll still have to bit-twiddle for things like this.
>
> -Russell B

There are many ways of encapsulating the bit twiddling. In C++ I've previously done obscure things with templates and even overloading of the specified address versions of new & delete! These solutions are even more obscure than using C pre-processor text manipulation. All these things are working around deficiencies in the language's ability to abstract a memory region with an externally determined layout. It is such a common problem that the language ought to support it (IMHO).

Chris.

"Axel Kittenberger" <axel@dtone.org> wrote in message news:9op7he$j2$1@digitaldaemon.com...
> Russ Lewis wrote:
> > [snip]
> > A nice, portable (but not performance-optimal) programming trick for
this
> > would be to use the struct as the binary layout and then implement a simple wrapper class that did the bit twiddling for you:
> >
> > class FooPacket
> > {
> >    struct FooPacketLayout
> >    {
> >        ....
> >     } binaryLayout;
> >
> > public:
> >    int GetParamX() { binaryLayout.....bit twiddling }
> > };
>
> See it? What have I told what the output of seperated class and struct elements in the class is?
>
> Why use a struct inside the class? Why not make the contents fields of the class itself?
>
> Im my opinion seperated class and struct elements only tends to confuse people, and quickly results into suboptimal results during development.

Axel Kittenberger wrote:

> See it? What have I told what the output of seperated class and struct elements in the class is?
>
> Why use a struct inside the class? Why not make the contents fields of the class itself?

Are you were arguing that this is an example why D should integrate structs and classes, or are you unclear about the code?  Sorry, I didn't quite understand.

If it's the former, I do agree that it would be nice to be able to integrate structs and classes somehow...though I don't know exactly what it should be. IMHO, having a struct at the top of the class that defines the binary layout isn't *too* bad, though it's certainly not great.

Chris Paulson-Ellis wrote:
> 
> Ahh - align(1){fields...} is good, although I'm not sure of the value of the
> colon syntax...

The value here I think is for "orthogonality" with public: and private: in C++ class declarations, though I'm not sure I accept that as a good thing. Ditto some other attributes.

> It would be well worth pinning down the behavior of misaligned word accesses in the language spec before someone comes up with the first non-x86 compiler. I take it no-one has a valuable use for those bus errors? :-)

The trouble is, there's rabid disagreement as to what the "correct" behavior of misaligned access should be.

If you define it as jumping through whatever hoops are required to read the data as if it were aligned, you get yelled at for the hidden performance hit by RISC[1] zealots porting x86 code. They would rather that the program crash, so they can rewrite the offending portions to execute more efficiently. This is the valuable use for those bus errors, weirdly enough.

If you define it as crashing on all platforms, you get yelled at by x86 users porting C code. They would rather that it read the data with a one-cycle hit, so they can read, for example, existing file formats that happen to include non-aligned header data, without jumping through hoops.

If you define it as crashing on RISC platforms and working on x86, you get yelled at by people who need to get their port to a RISC platform done ASAP and don't care about performance. On the bright side, people coming from a C background won't be surprised by this.

Thus, in C, under the C philosophy of "it works the way the machine says it works", well, it works the way the machine says it works.

-Russell B

[1] Yeah, RISC is a bit of a misnomer here, but it's shorter than "every processor family that I can think of developed since the introduction of the 68000".

I agree thats the same in many language decissions, you just can't make it 100%. It's the same bad vs. bad that people discussed about the throws statement, about brackets, etc.

> [1] Yeah, RISC is a bit of a misnomer here, but it's shorter than "every processor family that I can think of developed since the introduction of the 68000".

As far I know the new powerpc beeing marketed as a RISC processor is now again also able to read misaligned data.

"Axel Kittenberger" <axel@dtone.org> wrote in message news:9orr53$1gtm$1@digitaldaemon.com...
> I agree thats the same in many language decissions, you just can't make it 100%. It's the same bad vs. bad that people discussed about the throws statement, about brackets, etc.

I think that the compiler should generate the neccessary code. The programmer has documented that there may be a performance hit by putting align(1){...} in the source. Perhaps another attribute is needed to specify whether you want mis-aligned accesses to blow up or not. Or perhaps an extra 'argument' to the align attribute.

Chris.