You may have seen my previous dip1000 posts:
Consider this part 3 in the "fixing dip1000 series", but it's about a different bug.
Background
dip25 and dip1000 are supposed to provide simple lifetime tracking that's still good enough to be useful. In the previous thread Atila Neves mentioned that Lifetime Annotations like in Rust are to be avoided. Is it simple though?
On Wednesday, 26 May 2021 at 15:29:32 UTC, Paul Backus wrote:
>Of course, D's vision here is severely hampered in practice by
the poor quality of its documentation (raise your hand if you
can explain what "return ref parameter semantics with
additional scope parameter semantics" actually means). But
that's the idea.
Working on dip1000 made me finally able to "raise my hand", so here's how it works:
Function parameters of a type with pointers have three possible lifetimes: infinite, scope, or return scope. You might have heard that scope
is "not transitive" and think that there's only one layer to it. However, the key insight is that there's actually two layers when ref
comes into play: then the parameter's address itself also has a lifetime in addition to the value. It can be demonstrated with a linked list:
@safe:
struct Node {
int x;
Node* next;
}
// First layer: returning the address of the node
int* get0(return ref Node node) {
return &node.x;
}
// Second layer: returning a value of the node
int* get1(ref return scope Node node) {
return &node.next.x;
}
// Third layer and beyond: this is where scope checking ends
int* get2(ref scope Node node) {
return &node.next.next.x;
}
The lifetimes are determined as follows:
Lifetime | ref address |
value of pointer type |
---|---|---|
infinite | never | default |
current scope | default | with scope keyword |
return scope | with return keyword |
with return scope |
A few code examples:
@safe:
int* v0( int* x) {return x;} // allowed, no lifetime restrictions
int* v1(return int* x) {return x;} // allowed, returned value is `scope`
int* v2( scope int* x) {return x;} // not allowed, x is `scope`
int* v3(return scope int* x) {return x;} // allowed, equivalent to v1
int* r0( ref int x) {return &x;} // not allowed, `ref` is always scope
int* r1(scope ref int x) {return &x;} // not allowed, `scope` does nothing here
int* r2(return ref int x) {return &x;} // allowed, return applies to `ref`
As you can see, scope
always applies to the pointer value and not to the ref
, since ref
is inherently scope
. No ambiguity there. But what if we have a ref int*
: does return
apply to the address of the ref
or the int*
value?
That's where those confusing lines from the specification come in, which distinguishes "return ref semantics" and "return scope semantics". It turns out there are three important factors: whether the function's return type is ref
, whether the parameter is ref
, and whether the parameter is annotated scope
. Here's a table:
Does the return
attribute apply to the parameter's ref
or the pointer value?
scope |
no scope |
|
---|---|---|
ref return type / ref param |
ref |
ref |
value return type / ref param |
value | ref |
ref return type / value param |
value | value |
value return type / value param | value | value |
If you're still confused, I don't blame you: I'm still confusing myself regularly when reading signatures with return
and ref
. Anyway, is this difficulty problematic?
On Wednesday, 15 May 2019 at 08:32:09 UTC, Walter Bright wrote:
>On 5/15/2019 12:21 AM, Dukc wrote:
>Could be worth a try even without docs, but in the long run we
definitely need some explaining.
True, but I've tried fairly hard with the error messages.
Please post your experiences with them.
Also, there shouldn't be any caveats with using it. If it
passes the compiler, it should be good to go. (Much like const
and pure.)
All you need to do is see if the compiler complains, try adding return
and/or scope
, and see if the errors go away. Well...
@safe:
struct S {
int x;
}
int* f(ref return scope S s) {
return &s.x; // Error: returning `&s.x` escapes a reference to parameter `s`
// perhaps annotate the parameter with `return`
}
That's a confusing supplemental error, the parameter is annotated return
. The actual problem is that return
applies to the value, not the ref
parameter, since there is no ref
return.
struct T {
int x;
int* y; // <- pointer member added
}
int* g(ref return scope T t) {
return &t.x; // No error
}
And now the compiler accepts invalid code. Indeed, even the compiler doesn't always know what the return
storage class actually applies to. See bugzilla issue 21868.
The issue
While fixing issue 21868, the CI uncovered that dub package 'automem' relies on the current accepts-invalid behavior. Here's the reduced code:
struct Vector {
float[] _elements;
ref float opIndex(size_t i) scope return {
return this._elements[i];
}
}
With the patch I made, the error becomes:
source/automem/vector.d(212,25): Error: scope parameter `this` may not be returned
source/automem/vector.d(212,25): note that `return` applies to `ref`, not the value
My new supplemental error message is working, yay! But how to fix it?
One way is to pass the Vector
by value instead of by reference, but opIndex
must be a member function to work as an operator overload and member functions pass this
by reference. Another way is to return by value instead of by reference, but that means accessing array elements introduces a copy, and &vector[0]
won't work anymore.
dip1000 simply can't express a 'return scope' opIndex
returning by ref
.
So it turns out the double duty of the return
storage class is neither simple, nor expressive enough. Do you have any ideas how to move forward, and express the Vector.opIndex
method without making the attribute soup worse? Keep in mind that dip25 (with return ref
) is already in the language, but dip1000 (with return scope
) is still behind a preview switch.