On Sunday, 25 August 2024 at 20:02:01 UTC, Timon Gehr wrote:
> On 8/25/24 19:02, Steven Schveighoffer wrote:
>> 
>> I don't have time to play around with dip1000 normally -- it generally just results in bizarre and unfixable errors.
>> 
>> https://github.com/mysql-d/mysql-native/pull/283
>
>
>
> diff --git a/source/mysql/protocol/comms.d

Huh, this actually was it. I fully expected a deeper rabbit hole.

Now I have to figure out how to only turn on dip1000 for certain versions of D, as older compilers didn't have proper marking on std.socket...

-Steve

On 29/08/2024 2:48 AM, Paul Backus wrote:
> On Sunday, 25 August 2024 at 13:10:22 UTC, Mike Parker wrote:
>> Second, we'd like to get a number of examples of problems people have had with using DIP1000 that haven't shown up in Bugzilla. Walter wants to be as certain as he can whether such issues are fixable or if the design is fundamentally flawed.
> 
> Here's an issue that stems from a fundamental shortcoming of DIP1000's design: you can't write a `@safe` `swap` function.
> 
> Consider the following example:
> 
>      @safe void swap(ref /* ??? */ int* a, ref return scope int* b);
> 
>      @safe unittest
>      {
>          int local;
>          static int global;
> 
>          // Ensure both pointers have identical lexical scopes
>          static struct Pair { int* a, b; }
> 
>          auto p1 = Pair(&local, &local);
>          auto p2 = Pair(&global, &global);
>          auto p3 = Pair(&local, &global);
>          auto p4 = Pair(&global, &local);
> 
>          swap(p1.a, p1.b);
>          swap(p2.a, p2.b);
> 
>          static assert(!__traits(compiles, swap(p3.a, p3.b)));
>          static assert(!__traits(compiles, swap(p4.a, p4.b)));
>      }
> 
> A correct, `@safe` function signature for `swap` should be able to pass this unit test. However, under the DIP1000 system, there is no possible combination of attributes for the parameter `a` that will not cause one of the test cases to fail.
> 
> - `ref int* a` causes `swap(p1.a, p1.b)` to fail.
> - Both `ref scope int* a` and `ref return scope int* a` cause `swap(p3.a, p3.b)` to compile when it shouldn't.

This is a good example of what I mean by multiple outputs are not supported.

Both parameters are outputs, not just the first one.

```
onlineapp.d(9): Error: scope variable `temp` assigned to `ref` variable `a` with longer lifetime
onlineapp.d(25): Error: scope variable `p1` assigned to non-scope parameter `a` calling `swap`
onlineapp.d(8):        which is not `scope` because of `b = a`
```

```d
@safe void swap(ref /* ??? */ int* a, ref return scope int* b) {
    int* temp = b;
    b = a;
    a = temp;
}

 @safe unittest
 {
     int local;
     static int global;

     // Ensure both pointers have identical lexical scopes
     static struct Pair { int* a, b; }

     auto p1 = Pair(&local, &local);
     auto p2 = Pair(&global, &global);
     auto p3 = Pair(&local, &global);
     auto p4 = Pair(&global, &local);

     swap(p1.a, p1.b);
     swap(p2.a, p2.b);

     static assert(!__traits(compiles, swap(p3.a, p3.b)));
     static assert(!__traits(compiles, swap(p4.a, p4.b)));
 }
```

The line ``b = a;`` should also be erroring, but because ``b`` isn't seen as an error, its accepted.

I've been thinking about swap/move functions some more for my own proposal.

The following applies to swap just as much as move, but I'll use move as its simpler to understand here.

Lets say you don't have owner escape analysis (so just DIP1000).
And you have some stack memory that then gets borrowed from.
You can corrupt the borrowed memory, by calling a function like move.

Simply because a dependency exists of the thing being moved/swapped.

```d
int* value = ...;
int** ptr = &value; // ptr depends upon value

int* another = move(value);
// ugh oh, ptr is now corrupted
```

Now lets say you are able to annotate the escape sets fully (DIP1000 can do move).

```d
T move(T)(ref return T input);
```

Thing is, this signature just says the parameter contributes towards the return, it doesn't consume and invalidate the original input variable.

So a solution is to throw a new attribute into the mix like ``@escapeas(return)``. To show that it'll consume the input and it will go to a specific variable.

```d
void swap(T)(ref @escapeas(input2) T input1, ref @escapeas(input1) T input2);
```

Ok great, you modeled the escapes, congratulations.

Or you could use owner escape analysis to say "if have borrow, make effectively const" there cannot be assigned to.

Boom, neither swap nor move could modify the original and it all works.

So there are three scenarios here that I can think of:

1. Attribute the exact variable it'll escape to, to know that it will invalidate the input, preventing any relationships from it from being corrupted
2. Make these functions ``@system`` and say its a rare enough thing that it doesn't need to be made ``@safe``
3. Use owner escape analysis to provide the effectively const guarantees to prevent the mutation

Full example of a move in ``@safe`` code, that can corrupt memory (null, wrong length, wrong value ext.):

```d
@safe:

void main() {
    int*[1] value;
    int** ptr = &value[0];

    int* another = move(value[0]);
}

T move(T)(scope ref return T input) {
    T temp = input;
    input = T.init;
    return temp;
}
```

On 29/08/2024 8:53 PM, Richard (Rikki) Andrew Cattermole wrote:
> Thing is, this signature just says the parameter contributes towards the return, it doesn't consume and invalidate the original input variable.

To clarify, "invalidate the original input variable" means that from the caller function, after the function call the argument variables will no longer be the values that the compiler will be expecting. Potentially causing memory corruption that would otherwise have been caught.

On Thu, Aug 29, 2024 at 07:38:50PM +0000, Paul Backus via Digitalmars-d wrote: [...]
> To be clear, the issue here is not that the *implementation* requires `unsafe`/`@trusted`. The issue is that under the current system, it is impossible to give `swap` a `@safe` *function signature* that accepts `scope` arguments.
> 
> That's why I left the body of the `swap` function out of my example. It's completely irrelevant.

Exactly; this is why 'scope' is an inadequate solution to the wrong problem.

The whole discussion about lifetimes in D sounds like D really, badly, wants a Rust-like lifetime system, but, not wanting to ape Rust, which is considered too complex, we're trying instead to shoehorn various simple but incomplete solutions to try to pretend to solve the original problem.

I appreciate Walter's efforts to find simple solutions to problems -- that's the quality of a worthy engineer -- but some problems are simply inherently complex, and any solution that's simpler than the minimum complexity simply cannot be made to work, no matter how hard you try. More and more, lifetime tracking is appearing to be one of these problems. Any solution less complex than Rust's seems doomed to fail on some obscure corner case that will nevertheless compromise the entire solution.


T

-- 
Nearly all men can stand adversity, but if you want to test a man's character, give him power. -- Abraham Lincoln