D originally had a 'bit' type in place of 'bool' which could be packed in structs.  However, people didn't think very much of it, and fairly loudly complained that they wanted a "real" bool type.  Hence, 'bit' is now aliased to 'bool'.

The other problem was that 'bit' introduced various problems.  For instance, you can take the address of anything *except* a bitfield, since addresses only have byte-level granularity.  Bitfields also cannot have sensible sizeof properties, which could break generic code.

Off the top of my head, maybe it would suffice to give types a special 'bits' slice type.

int some_float;
int negative = some_float.bits[0..1];
int exponent = some_float.bits[1..9];
int mantissa = some_float.bits[9..$];

Basically just tarting up manual shifts and masking.  Just a thought :)

	-- Daniel

Daniel Keep wrote:
> D originally had a 'bit' type in place of 'bool' which could be packed
> in structs.  However, people didn't think very much of it, and fairly
> loudly complained that they wanted a "real" bool type.  Hence, 'bit' is
> now aliased to 'bool'.
> 
> The other problem was that 'bit' introduced various problems.  For
> instance, you can take the address of anything *except* a bitfield,
> since addresses only have byte-level granularity.  Bitfields also cannot
> have sensible sizeof properties, which could break generic code.
> 
> Off the top of my head, maybe it would suffice to give types a special
> 'bits' slice type.
> 
> int some_float;
> int negative = some_float.bits[0..1];
> int exponent = some_float.bits[1..9];
> int mantissa = some_float.bits[9..$];
> 
> Basically just tarting up manual shifts and masking.  Just a thought :)

You're not the first to think so :)

In fact, why not just slice the int directly:

int one;
int two;

one[0..4] = two[4..8];
one[4..8] = two[0..4];

In the meantime, in order to avoid having to re-code all the bit shifts in every struct with bitfields isn't there a mixin which could implement properties for accessing specific bits.

Regan

I don't like the idea of referring to e.g. n[3..4]. The way I see it, that's just as error prone as bitshifting. No, what you really want to do is separate interface from implementation. The interface is, you want to be able to manipulate bitfields as though they were integers. The implementation is how you want to store them in a struct. The whole business of using slice operators pulls implementation details into the algorithm, where they don't belong.

Howver, it seems to me that we already /have/ bitfields in D. Or at least, the ability to make them. In the example below, the D usage is identical to the C usage. All that's different is the way they're declared. Not only that, it seems to me that the D way offers far more precision and flexibility. Compare...

C/C++:

    /* define the structure */
    struct S
    {
        unsigned int a : 3;
        unsigned int b : 6;
        unsigned int c : 2;
        unsigned int d : 5;
    }

    /* declare an instance */
    S s;

    /* use the bits */
    s.b = 4;
    s.c = 3;
    s.d = s.b + 1;
    // etc....

D

    /* define the structure */
    struct S
    {
        ushort bits;

        uint a() { return (bits >> 13) & 0x07;
        uint b() { return (bits >>  7) & 0x3F;
        uint c() { return (bits >>  5) & 0x03;
        uint d() { return (bits >>  0) & 0x1F;

        void c(uint n) { bits &= 0x1FFF; bits |= (n << 13); }
        void b(uint n) { bits &= 0xE07F; bits |= (n <<  7); }
        void c(uint n) { bits &= 0xFF9F; bits |= (n <<  5); }
        void d(uint n) { bits &= 0xFFE0; bits |= (n <<  0); }
    }

    /* declare an instance */
    S s;

    /* use the bits */
    s.b = 4;
    s.c = 3;
    s.d = s.b + 1;
    // etc....

I'm sure some smart person can come up with a mixin to automate even this!

Yes, You are right Janice. We want to access the bitfields in a structure as if they were the integers.

Like this:

packet.ip.type_of_service.reliability = 1;

(See http://www.protocols.com/pbook/tcpip2.htm#IP for ip protocol.)

I whole heartedly disagree that your workaround is sufficient. It is a bit like saying C doesn't need native string support because you can just make a character array, or that you can just make a C++ class to handle strings. (We all know how well that works, everyone and their grandmother has a different string class.)

If you look at your example you are replacing the four parameters in the C++ version ( the four bitfield sizes) with twelve parameters in D ( 4 bitfield positions, and 8 masks.) not to mention the shifting and bitwise operations. And the need of the user to keep track of the “bits” members size. Plus, what if you wanted a structure 512 bytes long? Your ushort “bits” would have to be an array, and you would need to be very careful then of how your bitfields mapped into that array. This is adding a lot of accidental complexity to the problem. It's is okay if you are doing this once or twice. But I am literally talking about doing this on thousands of classes per project, on structures that can be quite large.

The D workaround adds quite a bit of error-prone complexity.

I would also like to suggest that this isn't a rarely-used feature. You run into this issue anytime you want to read or write to hardware, (or link to a library that talks to hardware) or access any sort of network protocol stack, or any type of communication protocol.

As an outsider looking in that works in these domains, this becomes a glaring hole in an otherwise wonderful language. One of my fundamental coding tasks is to talk to hardware, and I don't see that task as at all being rare.

I would agree that the way C++ does bitfields is “inefficient.” I know of no general-purpose language that does this well. The issues of portability (with respect to byte ordering) is valid, but kinda beside the point. If you are hacking hardware, it is by definition tied to the hardware. Just like the windows library is not portable outside windows. Plus, if we fixed that whole byte-order thing it would be a big justification to using the language, as the way C does it is just ugly.

There seems to be a lot of support for a feature along these lines. There has been a good discussion going on for quite some time. The underlying question is do we want D to live in this “low level bare metal “ environment? And if so, what do we (yes, I am including myself) need to do?

~Richard

Yes, You are right Janice. We want to access the bitfields in a structure as if they were the integers.

Like this:

packet.ip.type_of_service.reliability = 1;

(See http://www.protocols.com/pbook/tcpip2.htm#IP for ip protocol.)

I whole heartedly disagree that your workaround is sufficient. It is a bit like saying C doesn't need native string support because you can just make a character array, or that you can just make a C++ class to handle strings. (We all know how well that works, everyone and their grandmother has a different string class.)

If you look at your example you are replacing the four parameters in the C++ version ( the four bitfield sizes) with twelve parameters in D ( 4 bitfield positions, and 8 masks.) not to mention the shifting and bitwise operations. And the need of the user to keep track of the “bits” members size. Plus, what if you wanted a structure 512 bytes long? Your ushort “bits” would have to be an array, and you would need to be very careful then of how your bitfields mapped into that array. This is adding a lot of accidental complexity to the problem. It's is okay if you are doing this once or twice. But I am literally talking about doing this on thousands of classes per project, on structures that can be quite large.

The D workaround adds quite a bit of error-prone complexity.

I would also like to suggest that this isn't a rarely-used feature. You run into this issue anytime you want to read or write to hardware, (or link to a library that talks to hardware) or access any sort of network protocol stack, or any type of communication protocol.

As an outsider looking in that works in these domains, this becomes a glaring hole in an otherwise wonderful language. One of my fundamental coding tasks is to talk to hardware, and I don't see that task as at all being rare.

I would agree that the way C++ does bitfields is “inefficient.” I know of no general-purpose language that does this well. The issues of portability (with respect to byte ordering) is valid, but kinda beside the point. If you are hacking hardware, it is by definition tied to the hardware. Just like the windows library is not portable outside windows. Plus, if we fixed that whole byte-order thing it would be a big justification to using the language, as the way C does it is just ugly.

There seems to be a lot of support for a feature along these lines. There has been a good discussion going on for quite some time. The underlying question is do we want D to live in this “low level bare metal “ environment? And if so, what do we (yes, I am including myself) need to do?

~Richard

On 11/23/07, Richard Bradley <darthalias@yahoo.com> wrote:
> I whole heartedly disagree that your workaround is sufficient.

I did not claim that the workaround was sufficient, I only claimed that the workaround exists. You can't disagree with a claim I didn't make.


> If you look at your example you are replacing the four parameters in the C++ version ( the four bitfield sizes) with twelve parameters in D ( 4 bitfield positions, and 8 masks.) not to mention the shifting and bitwise operations. And the need of the user to keep track of the "bits" members size. Plus, what if you wanted a structure 512 bytes long?

Agreed. That would make it tricky to write getter and setter functions for each bitfield. I've certainly no objection to the compiler making things easier for the programmer. The shifting and bitwise operations will happen behind the scenes though, even if you're not aware of it.

I think the way to do it to have the compiler automagically write getter and setter functions for you. That way, the following would be allowed:

    n = s.a; // read a bitfield
    s.a = n; // write a bitfield

but things like s.a+=, &s.a, passing bitfields by reference, and so on, would still be disallowed. That would force the programmer to do all the calculations in "real" ints, using the bitfields only for storage.

Richard Bradley wrote:
> I would also like to suggest that this isn't a rarely-used feature. You run into this issue anytime you want to read or write to hardware, (or link to a library that talks to hardware) or access any sort of network protocol stack, or any type of communication protocol.

It's also used to reduce database throughput and in serialization of large structures to files, among other things. Although I'm not at all a "low-level"/"close-to-the-metal" programmer, I would definitely appreciate such a feature.

Richard Bradley wrote:
>There seems to be a lot of support for a feature along these lines. >There has been a good discussion going on for quite some time. The >underlying question is do we want D to live in this “low level bare >metal “ environment? And if so, what do we (yes, I am including myself) > need to do?

Richard


As no one else if volunteering at present,
I would like to suggest the following.

You create one document that covers the following:

1. Detail exactly the Bitfield functionality what you require (you
   seem to know this better than anyone on this list);
2. Address why the language _needs_  this functionality;	
2. Describe the target audience of the said feature(s);
3. Make a pitch as to the _number_ of additional users/projects who
   might join the "D" community as a result of said features.

When completed, ask for feedback from the "D" community, incorporate any feedback if required, and then, finally send it to Walter directly, asking for comment/feedback.

Good Luck

Nick B

Richard Bradley wrote:
>> It's not a priority, it's something that could be added later without
>>  breaking anything -- just syntactic sugar, really. We can try again later <g>.
> 
> Curious, Has there been any movement on this issue? Is there a less
> error-prone way to do this other then manual shifts and masks?
> 
> [...]
> 
> Comments, ideas welcome. Richard

FWIW I am quite sympathetic to the cause of C-like bitfields for D.
My take on D in the context of Kelleher's "What languages fix" was
always "D: C++ is a syntactic mess, and built-in GC is a win.".

OTOH, the accessor templates from the D.learn thread look very nice.

The question is probably: Do we really need native bitfield support,
or is D mighty enough that a sufficient solution can be implemented
without resorting to adding to the language itself?
I think the answer may in part depend on the result of the planned
"GCC bitfields" vs. "GDC templates" shootout as planned on D.learn
by bearophile.

Hi bearophile, any results yet? :)

regards, frank

0ffh:

> OTOH, the accessor templates from the D.learn thread look very nice.

I think they aren't good enough yet for practical usage, they do too much upcasting (to 64 bit numbers, in that thread I have shown it that using 32 bit makes those bitfields faster), so they require improvement. And they need more testing.


> I think the answer may in part depend on the result of the planned
> "GCC bitfields" vs. "GDC templates" shootout as planned on D.learn
> by bearophile.
> Hi bearophile, any results yet? :)

Hello, but sadly os far I haven't succed yet to install GDC (I'll try again), despite I already have MinGW installed. So I have tried with just the MinGW, the following:

#include <stdio.h>
#define N 25

typedef union {
  unsigned int i;
  struct {
    unsigned int i;

    unsigned int sign(void) { return (i >> 0x0) & 0x1; }
    void sign(unsigned int val) { i = (i & 0xfffffffffffffffeLL) | ((val & 0x1) << 0x0); }

    unsigned int biasedexponent(void) { return (i >> 0x1) & 0xff; }
    void biasedexponent(unsigned int val) { i = (i & 0xfffffffffffffe01LL) | ((val & 0xff) << 0x1); }

    unsigned int significand(void) { return (i >> 0x9) & 0x7fffff; }
    void significand(unsigned int val) { i = (i & 0x1ff) | ((val & 0x7fffff) << 0x9); }
  } bits;
} INTORFLOAT;

int main(void) {
  int i, j;
  unsigned long long tot = 0;
  INTORFLOAT f;

  unsigned int v[N] = {1628676761, 1620574103, 1237153253, 1098880307, 87513741,
             13181925, 14686126, 7429435, 16286706, 6474381,
             4879794, 7734725, 3745958, 13353858, 4236193,
             7587, 4309, 28846, 7313, 14516,
             126, 143, 171, 221, 156};

  for(j = 0; j < 4000000; j++)
    for(i = 0; i < N; i++) {
      f.i = v[i];
      f.bits.biasedexponent(f.bits.biasedexponent() / 2);
      tot += f.bits.biasedexponent() + f.bits.sign();
    }

   printf("%d\n", tot);
}

It runs even a bit faster than the normal C bitfields ;-) So they may run fast on GDC too.

Bye,
bearophile