I had written a detailed reply, but realized you and I were simply running around in the same circle saying the same things.

On Wednesday, 10 July 2024 at 07:09:10 UTC, Walter Bright wrote:
> I had written a detailed reply, but realized you and I were simply running around in the same circle saying the same things.

Maybe some input from 3rd party could help?

I use bitfields daily and never had any issues. What I do is to always use fix size types and then simply take all freedom away from the compiler.

uint32_t a;
uint32_t  :32; // Forced padding
uint64_t b:10;
uint64_t c:10;
uint64_t  :44; // Forced padding
uint32_t d;

I guess one can use 0 size bitfields also but I usually prefer to visualize how much padding remains for potential future use.

On 7/9/2024 5:44 PM, Timon Gehr wrote:
> On 7/9/24 02:29, Walter Bright wrote:
>>
>> Let's say Bob (poor Bob) needs to convert 20,000 lines of C code to D. I know you've done some of this yourself! Bob doesn't want to go through it line by line. Isn't it nice for Bob if it "just works"?
> 
> It won't, some edits will be necessary.

Yes, we know it is imperfect. The fewer nits, the better.


>> Then it doesn't just work, Bob has got some debugging to do (while Bob curses D and me), and Bob's got to figure out an alternative. Who wants to do that? Not Bob. Not me. Not nobody not nohow.
> As far as I am concerned, this is an irrelevant straw man. I don't want this. I never suggested anything that would cause this. It's pure FUD.

Having a D code with the same declarations as C code, but the code generated is different, is going to lead to subtle memory bugs. I.e. just another footgun.


> It's not nice to hand out a footgun disguised as candy.

Requiring an extern(C) to make it compatible with a C layout is just another footgun, and there's no way for the compiler to detect it.

The implementation-defined C layout has been there for what, 50 years? If it was so awful there'd be proposals to the C Standard to change it. People gripe about it now and then, but just go and fix their code and move on. Neither has C++ ever made any effort to change it, even though C++ has `extern "C"`.

I do not understand why this is such a problem, since C compilers change the struct member layout based on compiler switches (which I showed in another post), and elicits no complaint from anybody.

Having the default D struct member layout not line up with the associated C compiler layout is a memory safety issue. Not lining up with an externally imposed layout is not a memory safety issue.

The bottom line, whether D supports bitfields or not, whether extern(C) is applied or not, to conform to an externally specified layout, you're going to have to check and see if it matches. If it doesn't match, there are really simple ways to get it to match.

On Wednesday, 10 July 2024 at 07:43:40 UTC, Daniel N wrote:
> On Wednesday, 10 July 2024 at 07:09:10 UTC, Walter Bright wrote:
>> I had written a detailed reply, but realized you and I were simply running around in the same circle saying the same things.
>
> Maybe some input from 3rd party could help?
>
> I use bitfields daily and never had any issues. What I do is to always use fix size types and then simply take all freedom away from the compiler.
>
> uint32_t a;
> uint32_t  :32; // Forced padding
> uint64_t b:10;
> uint64_t c:10;
> uint64_t  :44; // Forced padding
> uint32_t d;
>
> I guess one can use 0 size bitfields also but I usually prefer to visualize how much padding remains for potential future use.

PS To avoid relying on convention, you could make an incomplete bitfield a compilation error in D, then D bitfields would have C layout *AND* be deterministic.

On 7/10/24 09:44, Walter Bright wrote:
> On 7/9/2024 5:44 PM, Timon Gehr wrote:
>> On 7/9/24 02:29, Walter Bright wrote:
>>>
>>> Let's say Bob (poor Bob) needs to convert 20,000 lines of C code to D. I know you've done some of this yourself! Bob doesn't want to go through it line by line. Isn't it nice for Bob if it "just works"?
>>
>> It won't, some edits will be necessary.
> 
> Yes, we know it is imperfect. The fewer nits, the better.
> ...

No, there are other considerations, otherwise D would be identical to C.

> 
>>> Then it doesn't just work, Bob has got some debugging to do (while Bob curses D and me), and Bob's got to figure out an alternative. Who wants to do that? Not Bob. Not me. Not nobody not nohow.
>> As far as I am concerned, this is an irrelevant straw man. I don't want this. I never suggested anything that would cause this. It's pure FUD.
> 
> Having a D code with the same declarations as C code, but the code generated is different, is going to lead to subtle memory bugs. I.e. just another footgun.
> ...

My position is: no footguns. It's easily achievable.

Your position is: one footgun or another footgun, does it really matter, let's just choose the footgun with the simpler design.

> 
>> It's not nice to hand out a footgun disguised as candy.
> 
> Requiring an extern(C) to make it compatible with a C layout is just another footgun, and there's no way for the compiler to detect it.
> ...

Again: You are arguing against something you made up yourself. Something that is not even on the table. I am however glad you agree there should not be footguns.

> The implementation-defined C layout has been there for what, 50 years? If it was so awful there'd be proposals to the C Standard to change it. 

I think you know very well that C has many design errors that were never fixed. Those people put up with C in the first place. They often even think it is a well-designed language.

> People gripe about it now and then, but just go and fix their code and move on. Neither has C++ ever made any effort to change it, even though C++ has `extern "C"`.
> 
> I do not understand why this is such a problem,

Because D prides itself on fixing mistakes, including underspecified layout.

> since C compilers change the struct member layout based on compiler switches (which I showed in another post),

I use D because it does not do stupid things like that.

You seem to think that "some C implementations did it that way" is in some way a good way to justify something. It just is not.

> and elicits no complaint from anybody.
> ...

I highly doubt it. I would complain about this.

> Having the default D struct member layout not line up with the associated C compiler layout is a memory safety issue.

I know. I care about memory safety. I do however not care about your argument, because it is a pure straw man. I am _not_ suggesting to make `extern(D)` bitfields silently have a different layout, just a bit more restrictions, rejecting sloppily written bit-field code.

Anyway, if you interoperate with C, you are on your own w.r.t. memory safety anyway, as you have no idea what kind of compiler extensions, attributes, and switches will change the struct layout in a way that ImportC does not yet understand, but will silently ignore.

```c
#include <stddef.h>
#include <stdio.h>
#include <stdlib.h>
struct __attribute((packed)) S{
    int x;
    int* y;
};

int main(){
    printf("%ld\n",offsetof(struct S, x));
    printf("%ld\n",offsetof(struct S, y));
}
```

```d
dmd -run test.c
0
8
```

```d
gcc test.c && ./a.out
0
4
```


> Not lining up with an externally imposed layout is not a memory safety issue.
> ...

It does not even need to be externally imposed. Consistency and reproducibility is important on its own.

> The bottom line, whether D supports bitfields or not, whether extern(C) is applied or not, to conform to an externally specified layout, you're going to have to check and see if it matches. If it doesn't match, there are really simple ways to get it to match.
> 

On the platform you happen to be using. It may not work on another platform. The entire point is that it should be sufficient to check once, and to match it once. If what you need to match is insane inconsistent C behavior, just be explicit about it. That is all I am asking.

On 7/10/24 08:53, Walter Bright wrote:
> On 7/9/2024 5:32 PM, Timon Gehr wrote:
>>> The offset of `y` does not even respect its alignment! This is insanity.
> 
> That's right. It's not a bug, it matches what the associated C compiler does.

Nonsense. The issue is the inconsistency between `S.y.alignof` and `S.y.offsetof`. In C, neither `offsetof` nor `alignof` work with bitfields in the first place, so the question does not even pose itself.

On 7/10/24 09:55, Daniel N wrote:
> On Wednesday, 10 July 2024 at 07:43:40 UTC, Daniel N wrote:
>> On Wednesday, 10 July 2024 at 07:09:10 UTC, Walter Bright wrote:
>>> I had written a detailed reply, but realized you and I were simply running around in the same circle saying the same things.
>>
>> Maybe some input from 3rd party could help?
>>
>> I use bitfields daily and never had any issues. What I do is to always use fix size types and then simply take all freedom away from the compiler.
>>
>> uint32_t a;
>> uint32_t  :32; // Forced padding
>> uint64_t b:10;
>> uint64_t c:10;
>> uint64_t  :44; // Forced padding
>> uint32_t d;
>>
>> I guess one can use 0 size bitfields also but I usually prefer to visualize how much padding remains for potential future use.
> 
> PS To avoid relying on convention, you could make an incomplete bitfield a compilation error in D, then D bitfields would have C layout *AND* be deterministic.
> 

Yes, something like that I think would be great, but I think Walter has a point that there should still be a way to match C bitfields even if the original author was less competent than you w.r.t. bitfield layout. Hence the proposal that anything goes if there is an `extern(C)` annotation, but for `extern(D)` bitfields, something like your approach would be enforced by the compiler.