On Friday, 26 February 2021 at 22:29:09 UTC, Max Haughton wrote:
> On Friday, 26 February 2021 at 18:29:26 UTC, Nick Treleaven wrote:
>> Er, doesn't private mean module access? This DIP would help verify long modules that might inadvertently access unsafe fields from outside the struct definition, at least in @safe code.
>
> https://run.dlang.io/is/KalIzj
>
> Yes - however code subject to is supposed to be consumed in a different module for the most part, so this could be a nice side effect but it shouldn't (de jure) guide the actual DIP itself.

It's more than a nice side effect - it's actually necessary to enforce a @safe interface for structs that do unsafe things. Otherwise changing @safe code can cause memory-safety violations - that's contrary to the design goal of @safe. It also would in practice mean a programmer/reviewer would have to read every line of code in a module every time there was a change - that's not a sensible design.

On Saturday, 27 February 2021 at 08:23:38 UTC, Kagamin wrote:
> A simple workaround is an unsafe wrapper:
>
> struct Unsafe
> {
> 	private T a;
> 	T get() @system { return a; }
> }
>
> struct IntSlice
> {
> 	private Unsafe!(int*) ptr;
> 	private Unsafe!size_t length;
> ...

The compiler will still allow you to void-initialize/overlap/reinterpret-cast an Unsafe!size_t in @safe code, because it considers size_t to be a safe type.

You *can* get the compiler to treat any value as unsafe by overlapping it with a pointer; e.g.,

struct Unsafe(T)
{
    union Storage { T value; void* dummy; }
    Storage storage;
    ref T get() { return Storage.value; }
    /* ... */
}

But this introduces memory overhead for any T smaller than a pointer, which is not ideal.

On Saturday, 27 February 2021 at 13:32:24 UTC, Paul Backus wrote:
> The compiler will still allow you to void-initialize/overlap/reinterpret-cast an Unsafe!size_t in @safe code, because it considers size_t to be a safe type.

The wrapped value still won't be accessible to safe code.

On 2/27/21 8:32 AM, Paul Backus wrote:
> On Saturday, 27 February 2021 at 08:23:38 UTC, Kagamin wrote:
>> A simple workaround is an unsafe wrapper:
>>
>> struct Unsafe
>> {
>>     private T a;
>>     T get() @system { return a; }
>> }
>>
>> struct IntSlice
>> {
>>     private Unsafe!(int*) ptr;
>>     private Unsafe!size_t length;
>> ...
> 
> The compiler will still allow you to void-initialize/overlap/reinterpret-cast an Unsafe!size_t in @safe code, because it considers size_t to be a safe type.
> 
> You *can* get the compiler to treat any value as unsafe by overlapping it with a pointer; e.g.,
> 
> struct Unsafe(T)
> {
>      union Storage { T value; void* dummy; }
>      Storage storage;
>      ref T get() { return Storage.value; }
>      /* ... */
> }
> 
> But this introduces memory overhead for any T smaller than a pointer, which is not ideal.

Better mark that get as @system, or it can be used in @safe code.

-Steve

On Saturday, 27 February 2021 at 15:21:41 UTC, Steven Schveighoffer wrote:
> On 2/27/21 8:32 AM, Paul Backus wrote:
>> You *can* get the compiler to treat any value as unsafe by overlapping it with a pointer; e.g.,
>> 
>> struct Unsafe(T)
>> {
>>      union Storage { T value; void* dummy; }
>>      Storage storage;
>>      ref T get() { return Storage.value; }
>>      /* ... */
>> }
>> 
>> But this introduces memory overhead for any T smaller than a pointer, which is not ideal.
>
> Better mark that get as @system, or it can be used in @safe code.

It's in a template, so it will be inferred as @system.

On 2/27/21 10:55 AM, Paul Backus wrote:
> On Saturday, 27 February 2021 at 15:21:41 UTC, Steven Schveighoffer wrote:
>> On 2/27/21 8:32 AM, Paul Backus wrote:
>>> You *can* get the compiler to treat any value as unsafe by overlapping it with a pointer; e.g.,
>>>
>>> struct Unsafe(T)
>>> {
>>>      union Storage { T value; void* dummy; }
>>>      Storage storage;
>>>      ref T get() { return Storage.value; }
>>>      /* ... */
>>> }
>>>
>>> But this introduces memory overhead for any T smaller than a pointer, which is not ideal.
>>
>> Better mark that get as @system, or it can be used in @safe code.
> 
> It's in a template, so it will be inferred as @system.

D considers that @safe. It was a whole section of my Dconf online talk. I filed a bug report on it (in which you seem to think this is OK): https://issues.dlang.org/show_bug.cgi?id=21565

Even without templates:

struct DThinksThisIsSafe
{
   union Storage { int value; void *dummy; }
   Storage storage;
   @safe ref int get() return { return storage.value; } // compiles just fine
}

-Steve

On Saturday, 27 February 2021 at 19:05:01 UTC, Steven Schveighoffer wrote:
>
> D considers that @safe. It was a whole section of my Dconf online talk. I filed a bug report on it (in which you seem to think this is OK): https://issues.dlang.org/show_bug.cgi?id=21565
>
> Even without templates:
>
> struct DThinksThisIsSafe
> {
>    union Storage { int value; void *dummy; }
>    Storage storage;
>    @safe ref int get() return { return storage.value; } // compiles just fine
> }

Ah, right, because you're only accessing the int, not the pointer. Good catch.

On Saturday, 27 February 2021 at 20:07:30 UTC, Paul Backus wrote:
>
> Ah, right, because you're only accessing the int, not the pointer. Good catch.

...which means the whole approach doesn't actually work to begin with. D is perfectly within its rights to let you void-initialize the union (even though it currently doesn't), because @safe code can't access the pointer anyway, so it can never lead to undefined behavior.

On 2/27/21 3:12 PM, Paul Backus wrote:
> On Saturday, 27 February 2021 at 20:07:30 UTC, Paul Backus wrote:
>>
>> Ah, right, because you're only accessing the int, not the pointer. Good catch.

It's telling that you intuitively thought the system should prevent you from doing this (as you should!)

> 
> ...which means the whole approach doesn't actually work to begin with. D is perfectly within its rights to let you void-initialize the union (even though it currently doesn't), because @safe code can't access the pointer anyway, so it can never lead to undefined behavior.

D is perfectly within its rights to do whatever it wants for @safe code. It could let you write an array length without extending the array, and then only allow you accessing the single element pointed at. It could prevent dereferencing pointers, and still be considered memory-safe. But there is still the question of whether this is useful to programmers or not.

-Steve

On Saturday, 27 February 2021 at 20:54:34 UTC, Steven Schveighoffer wrote:
> D is perfectly within its rights to do whatever it wants for @safe code. It could let you write an array length without extending the array, and then only allow you accessing the single element pointed at. It could prevent dereferencing pointers, and still be considered memory-safe. But there is still the question of whether this is useful to programmers or not.

The general trend in the design of @safe, as exemplified by DIP 25 and DIP 1000, has been to lift restrictions when the compiler can prove they are not necessary to prevent UB. And I would argue that this is exactly what one wants in a proof system: the strongest result (memory safety) from the fewest axioms (restrictions on @safe code).

Another way to look at it is that automatically extending an array when its length is changed allows the compiler to prove memory safety for programs that would otherwise require @trusted, so it is a useful restriction. On the other hand, forbidding access to safe members of unions does not make any previously-@trusted programs @safe, so it is a useless restriction.

I understand from previous discussions that you have some less-rigorous ideas about what is "useful to programmers" and what is not, but I think this is an occasion where rigor is warranted.