On 4/29/2024 5:07 AM, Timon Gehr wrote:
> I am asking for example code how you would implement a function that gives you a "fat pointer" to a bitfield that lets you read and write from that bitfield.

The fat pointer in D is a delegate, and that's how I'd do it.

On 4/29/2024 8:54 AM, Jonathan M Davis wrote:
> Such code does already need to take unions into account (and there is _some_
> similarity between those and bitfields), but it's going to have done that by
> checking things like is(T == union), which won't help with bitfields at all.

Using `is(T==union)` is incomplete because anonymous unions are not fields. The compiler doesn't do a union check internally for that reason. The correct check would be:

```
if (S.a.offsetof + typeof(S.a).sizeof <= S.b.offsetof ||
    S.b.offsetof + typeof(S.b).sizeof <= S.a.offsetof)
{
    // S.a and S.b do not overlap
}
else
{
    // S.a and S.b overlap
}
```
This will work without change for bitfields.

On 4/30/24 05:14, Walter Bright wrote:
> I do not recall exactly why this `ref` behavior was done for foreach, but it was either a mistake or was done precisely to make generic code work.

You mean fail silently.

On 4/30/24 05:30, Walter Bright wrote:
> On 4/29/2024 5:04 AM, Timon Gehr wrote:
>>> If they are not checking for bitfields, but are just looking at .offsetof and the type, they'll interpret the bitfields as a union (which, in a way, is accurate).
>>> ...
>>
>> No, it is not accurate.
> 
> Getting and setting bit fields reads/writes all the bits in the underlying field, so it definitely is like a union.

No, more than one bitfield is valid at a time even if they have the same offsetof. This is definitely breaking expectations that used to be true.

> std.bitmanip.bitfields also implements it as a union,

No, this is not correct. It implements it as a field with accessors for different groups of bits. The only reason why `union` appears in that file is to support bitfields inside a union. This again highlights that those are not the same thing.

> because there is no other way. The CPU does not provide any instructions to access bit fields. (This is why atomics won't work on bitfields.)
> ...

Sure! I guess this opens the question what happens with bitfields and type qualifiers. The DIP currently says you can have `int`, `uint`, `long` and `ulong` bitfields.

Are e.g. `immutable(int)` bitfields allowed?
I'd expect `shared(int)` bitfields are not allowed?


> If the user of bitfields does not understand the underlying physical reality of bitfields, they will forever have problems with them. Just like programmers who do not understand the physical reality of pointers, floating point, 2s complement, etc.,

I understand the underlying reality of all of those concepts and I still disagree that interpreting bitfields as a union is correct. There are bitfields and there are unions.

> are always crippled and would probably be better off using Excel as their programming language :-/
> ...

However, this seems like an exaggeration. I think there are programmers who are gainfully employed and fall into neither of those categories.

> 
>>> Pointer to bitfields will work just the same as they do in C. I don't understand what you're asking for.
>> Well, you can't take a pointer to a bitfield.
> 
> Exactly what I meant!
> 
> 
>> Well, so far everything in `.tupleof` had an address.
> 
> When you mentioned enums not having an address, I had assumed you were talking about __traits(allMembers). .tupleof skips over enums.
> ...

But it will include bitfields, and not the underlying "physical" variables.

> 
>> It should at least be mentioned in the DIP, if nowhere else you should put it in the breaking language changes section.
> 
> I can mention it, sure.
> ...

Thanks!

> 
>> Well, if you are trying to deliberately make introspection unnecessarily complicated, I guess that's your prerogative.
> 
> __traits has an ugly syntax. The idea was to provide the ability, and the user (or Phobos) would put a pretty face on it.
> ...

In practice people often do use `__traits`, either because it is more efficient or wrapping is impossible. In any case, providing exactly what is needed is much simpler.

> 
>> All of those things are ugly hacks. This kind of brain teaser is how metaprogramming works (or increasingly: used to work) in C++, but I think it is not very wise to continue this tradition in D.
> 
> std.traits definitely continues the tradition. While I'm fine with ugly implementations in it, std.traits fails to document the behavior of the functions that supposedly put a pretty face on it. I've asked Adam Wilson to consider completely re-engineering std.traits.
> 
> As long as it is possible to put a pretty face on it, I'm ok with an underlying ugliness in the service of not having N>1 diverse ways to do X.
> 

I think the main thing is it should be immediately obvious to readers, as it is actually not hard.

On Tuesday, April 30, 2024 7:43:52 AM MDT Timon Gehr via dip.development wrote:
> Sure! I guess this opens the question what happens with bitfields and type qualifiers. The DIP currently says you can have `int`, `uint`, `long` and `ulong` bitfields.
>
> Are e.g. `immutable(int)` bitfields allowed?

I don't see why not. You shouldn't be able to mutate them, but reading them should be fine, since they're not going to change.

> I'd expect `shared(int)` bitfields are not allowed?

There should be no problem with shared bitfields existing. However, it shouldn't be legal to read them or write them so long as they're shared. But with the preview switch to lock down shared, that's true of any type, including int. Accessing shared data while it's not protected is always a problem.

Atomics shouldn't work with bitfields, since they can't, whereas if you protect them with a mutex, you can then temporarily cast away shared and operate on them just like you'd do with any other shared data. So, I don't see why bitfields would be at all special with regards to what needs to happen with shared.

- Jonathan M Davis

On 4/30/24 18:42, Jonathan M Davis wrote:
> On Tuesday, April 30, 2024 7:43:52 AM MDT Timon Gehr via dip.development
> wrote:
>> Sure! I guess this opens the question what happens with bitfields and
>> type qualifiers. The DIP currently says you can have `int`, `uint`,
>> `long` and `ulong` bitfields.
>>
>> Are e.g. `immutable(int)` bitfields allowed?
> 
> I don't see why not. You shouldn't be able to mutate them, but reading them
> should be fine, since they're not going to change.
> 
>> I'd expect `shared(int)` bitfields are not allowed?
> 
> There should be no problem with shared bitfields existing. However, it
> shouldn't be legal to read them or write them so long as they're shared. But
> with the preview switch to lock down shared, that's true of any type,
> including int. Accessing shared data while it's not protected is always a
> problem.
> 
> Atomics shouldn't work with bitfields, since they can't, whereas if you
> protect them with a mutex, you can then temporarily cast away shared and
> operate on them just like you'd do with any other shared data. So, I don't
> see why bitfields would be at all special with regards to what needs to
> happen with shared.
> 
> - Jonathan M Davis
> 
> 
> 

Well, I am bringing it up because the DIP draft ignores type qualifiers so far (and explicitly only lists unqualified types for support). What is happening with `shared` I think has not been fully pinned down, but last I heard the goal was to get implicit atomics.

On Tuesday, April 30, 2024 2:48:46 PM MDT Timon Gehr via dip.development wrote:
> Well, I am bringing it up because the DIP draft ignores type qualifiers so far (and explicitly only lists unqualified types for support). What is happening with `shared` I think has not been fully pinned down, but last I heard the goal was to get implicit atomics.

Atila was talking about possibly doing implicit atomics, and we may get that, but either way, for types that _can't_ use atomics (like bitfields), it's pretty clearly going to have to be the case that they can't be read or written to without casting if shared is actually going to protect against accessing shared data in a manner which isn't guaranteed to be thread-safe like it's theoretically supposed to and -preview=nosharedaccess is supposed to enforce.

So, the normal rules for type qualifiers should apply to bitfields exactly like they would with any other type, and there shouldn't be any surprises here, but yes, if we want to be thorough about things, then the DIP should probably mention what happens with type qualifiers.

- Jonathan M Davis

On Tuesday, 30 April 2024 at 03:30:15 UTC, Walter Bright wrote:
> On 4/29/2024 5:04 AM, Timon Gehr wrote:
>>> If they are not checking for bitfields, but are just looking at .offsetof and the type, they'll interpret the bitfields as a union (which, in a way, is accurate).
>>> ...
>> 
>> No, it is not accurate.
>
> Getting and setting bit fields reads/writes all the bits in the underlying field, so it definitely is like a union. std.bitmanip.bitfields also implements it as a union, because there is no other way. The CPU does not provide any instructions to access bit fields.
>
Not true. x86 provides BMI1 instructions which are present in x86 CPUs at least since 2013.
ARM also provides bit field instructions and quite a number of legacy CPU's also had bitfield instructions (m68k, NEC V30, Itanium, PowerPC, etc.).
Doesn't change the issues with language bitfields

On Friday, 3 May 2024 at 12:52:09 UTC, Patrick Schluter wrote:
> Not true. x86 provides BMI1 instructions which are present in x86 CPUs at least since 2013.
> ARM also provides bit field instructions and quite a number of legacy CPU's also had bitfield instructions (m68k, NEC V30, Itanium, PowerPC, etc.).
> Doesn't change the issues with language bitfields

About BMI/BMI2 it would interesting to see if optimizing compilers actually generate instructions of these extensions for c++ bitfields. I've tried for styx enum-sets, sure that's a bit a special case of bitfields, but so far the only difference visible is a BMI2 `shlxl` emitted. But once again very special case.