On 5/4/2025 3:49 PM, Richard (Rikki) Andrew Cattermole wrote:
> On 05/05/2025 8:50 AM, Walter Bright wrote:
>> The point of the borrow checker is to ensure there is exactly one point of origin for a pointer and exactly one point of termination for it.
> 
> Hang on, that isn't the point of it at all.
> 
> A borrow checker has got the _side effect_ of only having one entry and exit point for an object.
> 
> But the _objective_ is to loosen the restrictions that an _ownership transfer system_ imposes on the type system whilst keeping its guarantees.

I don't see how that would be workable.


> We do not have an ownership transfer system in D.

Yes we do, in @live code. That's exactly what it does. A pointer assignment to another pointer transfers the ownership to the lvalue, and the rvalue becomes "dead".


> herefore we cannnot have a borrow checker.

The implementation shows it works.


> What we do have is a liveliness analysis with guaranteed modelability, and we do have a use for that, @restrict.
> 
> Enforcing @restrict could be an incredibly useful tool to those who do data processing with simd, right now they are doing this by hand.

Restricted pointers in C are guaranteed only by the user, the compiler does no checking.

You cannot vet restricted pointers without a borrow checker.

On 10/05/2025 4:21 PM, Walter Bright wrote:
> On 5/4/2025 3:49 PM, Richard (Rikki) Andrew Cattermole wrote:
>> On 05/05/2025 8:50 AM, Walter Bright wrote:
>>> The point of the borrow checker is to ensure there is exactly one point of origin for a pointer and exactly one point of termination for it.
>>
>> Hang on, that isn't the point of it at all.
>>
>> A borrow checker has got the _side effect_ of only having one entry and exit point for an object.
>>
>> But the _objective_ is to loosen the restrictions that an _ownership transfer system_ imposes on the type system whilst keeping its guarantees.
> 
> I don't see how that would be workable.

Okay that is interesting.

>> We do not have an ownership transfer system in D.
> 
> Yes we do, in @live code. That's exactly what it does. A pointer assignment to another pointer transfers the ownership to the lvalue, and the rvalue becomes "dead".

That is an owner state in the DFA.

Ownership transfer system operates at the type qualifier level and applies to a variable regardless of what function it is in.

See isolated from Midori, which was designed with D's type system in mind.

Currently people fake having one, by using a struct with a copy constructor that resets the old value.

>> herefore we cannnot have a borrow checker.
> 
> The implementation shows it works.
> 
> 
>> What we do have is a liveliness analysis with guaranteed modelability, and we do have a use for that, @restrict.
>>
>> Enforcing @restrict could be an incredibly useful tool to those who do data processing with simd, right now they are doing this by hand.
> 
> Restricted pointers in C are guaranteed only by the user, the compiler does no checking.
> 
> You cannot vet restricted pointers without a borrow checker.

Exactly, so what I've been thinking is tying @live to @restrict, which would make it a very useful tool.

This is something it could shine at, given its current implementation details.

----------------------------------------------------------------------

I want to go back to an earlier example I did elsewhere, where I have this type qualifier ``unique``, that is for an ownership transfer system.

If you have an ownership transfer system without a borrow checker:

```d
void func(scope ref int*) {}

unique(int*) a = ...;
assert(a !is null);

unique(int*) b = a;
assert(a is null);
assert(b !is null);

func(b); // error
```

Very annoying, it is what we have today if we fake it with a struct, except people will use alias this... and no more compiler error.

When you have both an ownership transfer system with a borrow checker:

```d
void func(scope ref int*) {}

unique(int*) a = ...;
assert(a !is null);

unique(int*) b = a;
assert(a is null);
assert(b !is null);

func(b); // ok
```

You may notice that unique may implicitly be removed in the type system.
It is not the type systems job to care about it being removed.

It is the job of the borrow checker to enforce the guarantee the origin outliving the borrow.

This is what I mean by: "But the _objective_ is to loosen the restrictions that an _ownership  transfer system_ imposes on the type system whilst keeping its guarantees."

In practice it just means that the type system doesn't need to solve a problem that a DFA is going to later on.

On 5/9/2025 9:40 PM, Richard (Rikki) Andrew Cattermole wrote:

> ```d
> void func(scope ref int*) {}
> 
> unique(int*) a = ...;
> assert(a !is null);
> 
> unique(int*) b = a;
> assert(a is null);
> assert(b !is null);
> 
> func(b); // ok
> ```

With a borrow checker like in @live, such asserts serve no purpose:
```
int* a = ...;
int* b = a;   // ownership transferred to b
*a = 3; // error, ownership was transferred out of a
```

On 10/05/2025 5:27 PM, Walter Bright wrote:
> On 5/9/2025 9:40 PM, Richard (Rikki) Andrew Cattermole wrote:
> 
>> ```d
>> void func(scope ref int*) {}
>>
>> unique(int*) a = ...;
>> assert(a !is null);
>>
>> unique(int*) b = a;
>> assert(a is null);
>> assert(b !is null);
>>
>> func(b); // ok
>> ```
> 
> With a borrow checker like in @live, such asserts serve no purpose:
> ```
> int* a = ...;
> int* b = a;   // ownership transferred to b
> *a = 3; // error, ownership was transferred out of a
> ```

Yes, they were there for demonstration purposes to show what the state is.

On Fri, 2 May 2025 at 23:10, Dennis via Digitalmars-d < digitalmars-d@puremagic.com> wrote:

> On Friday, 2 May 2025 at 07:44:52 UTC, Manu wrote:
> > On a slight tangent; why do I need to attribute the function at all for it check `scope` args don't escape?
>
> You don't need @live for that, only @safe and -preview=dip1000. @live / -preview=dip1021 checks for mutable aliasing, which is something added on top of `scope` escape analysis.
>

Okay, so then why should `scope` need `@safe`? It's an additional attribute added in its own right; there's no apparent value to requiring a SECOND attribute's presence in order to make the first attribute take effect. `scope` should work when you write `scope`. If you don't want scope checking, don't write `scope`...?

On Sun, 4 May 2025 at 05:15, Walter Bright via Digitalmars-d < digitalmars-d@puremagic.com> wrote:

> On 5/2/2025 12:44 AM, Manu wrote:
> > On a slight tangent; why do I need to attribute the function at all for
> it to
> > check `scope` args don't escape?
>
> Because otherwise the compiler has to examine the function implementation
> to
> verify this. Doing such is called attribute inference, and D does quite a
> bit of
> that. Problems are:
>
> 1. Function declarations don't have a body, so there's no way to inspect the body.
>

A prototype without a body must always trust the declaration. You must simply assume that presence of `scope` on an arg asserts that the parameter won't be retained by the call, and the caller can also bank on that fact; ie, using temp/stack memory to allocate an argument.

2. Inspecting all the bodies means compilation gets slowed down quite a bit.
>

If you don't want escape analysis, don't write `scope`; point is, why do you need to write TWO attributes together for the first one (which is itself strictly optional) to take effect?

3. Chicken-and-egg problems when inferring attributes when there are loops
> in
> the function call flow graph.
>

I don't follow.
Is it something like; a template which receives an arg but doesn't
explicitly state `scope` just spend time to try and infer `scope` because a
caller of that template may rely on that attribute inference?

> I do want to add `scope` to pointers and ref arguments, but I don't see any
> > reason that I should have to attribute `live`... why isn't `scope`
> enough to
> > enable escape analysis?
>
> The DFA (Data Flow Analysis) required to determine whether a pointer is
> the
> "owner" or not at each point in the function is considerably more complex
> than
> just checking for scope-ness. DFA equations are generated and then solved
> iteratively.
>
> The code required to do it is:
>
> https://github.com/dlang/dmd/blob/master/compiler/src/dmd/ob.d
>
> which is very similar to the DFA performed by the optimizer:
>
> https://github.com/dlang/dmd/blob/master/compiler/src/dmd/backend/gflow.d
>

Yeah but I still don't get it; if you don't want escape analysis, don't
write `scope`. If you do write it, you have clearly and unambiguously
indicated that you DO want escape analysis, so it should happen.
I can't see any reason a second unrelated attribute should be required in
conjunction to make the first one work... you write `scope` thinking it
will be effective, and then by not writing (or forgetting to write) the
other thing, your code is now lying to you. You won't know this until you
are bitten by the escape bug that you were trying to prevent; making the
sutuation actually *worse* than useless, because when you're trying to
track down your bug, you will see the attribute, and continue looking
elsewhere.
`scope` should work when you write it; period. If that is to say that
writing `scope` infers the other attribute, whatever... but it's not a
reasonable situation where `scope` silently does nothing and confuses or
misleads the author.

On 5/10/25 06:02, Walter Bright wrote:
> On 5/7/2025 4:21 PM, Timon Gehr wrote:
>> On 5/7/25 16:47, jmh530 wrote:
>>>
>>> I can think of a few different ways forward (by no means limited to just this)
>>
>> Well, the big issue here is that @live is still not transitive.
> 
> We are all well aware of the problems with transitive attributes. Making @live transitive means that nobody will ever use it, no matter how good it is. People keep asking for a non-transitive @nogc.
> ...

My point is not that `@live` should be a transitive function attribute. It should not be a function attribute in the first place.

For `@live` functions, not only is it a problem when they call non-`@live` functions, it is also a problem you call _them_ from non-`@live` functions.

And even then, you have not implemented checks that are even sufficient to guarantee anything if your entire program is `@live` with a custom `@live` druntime.


>> A function attribute is not really the right way to go about this anyway. Aliasing is a property of a pointer, not a function.
> 
> Attaching it to a pointer means nobody will use it, because they'll have to redo their entire program.
> 

If this is really your perspective, it is better to just drop borrow checking as a direction. It seems your design constraints are self-contradictory assuming that utility is one of them. The current design does not incentivize people to use it, it just makes it not very useful.

The entire point is allowing gradual adoption with gradual benefits. However, this has to be done one datastructure at as time, not one function at a time.

In any case, what you say is simply not true, you can e.g. pass borrowed things as `scope` parameters, as long as you are able to control accesses to the owner for the duration of that function call.

But sure, if your entire `@safe` program is based around passing raw, non-scope pointers around, there will _never_ be a way to use these functions from `@safe` code using manually allocated pointers.

Note that `int*` and `scope int*` are _already_ de facto different pointer types (even though `scope` is implemented as an attribute).
You cannot pass a `scope int*` to an `int*` parameter. Yet for DIP1000, having to "redo the entire program" to use `scope` and other annotations was not considered a deal-breaker.

Though neither DIP1000 nor `@live` have any meaningful story about multiple indirections.

On Friday, 9 May 2025 at 18:28:11 UTC, Richard (Rikki) Andrew Cattermole wrote:
> On 10/05/2025 5:17 AM, jmh530 wrote:
>> To your point about being a property of a pointer, the alternative approach is to introduce a reference that is borrow checked. Yes, it results in additional pointer types, but if that's the approach that Rust ultimately relies on and it's the only way to achieve what we want, then it's what should be done.
>
> I want to touch upon this because its misunderstood quite widely regarding the difference between Rust's borrow checker and ownership transfer system.
>
> The borrow checker loosens restrictions placed upon you by the ownership transfer system to give strong guarantees of aliasing and lifetimes. If you did not have the borrow checker, the ownership transfer systems restrictions would make things absolutely impossible to work with.

I can't say I'm 100% comfortable with these things, but anyway:

A characteristic of affine and linear types is that they can be 'used'/'consumed' at most one, with linear also being 'used' at least once.  Hence linear types must be 'used'.

This has some awkwardness in how that behaviour is then experienced by the programmer.

So Rust has something which is a form of affine types, but the borrowing allows one to make use of the type without consuming the value.  i.e. a use is not deemed a 'use'/'consumption'.

This 'borrowing' creates a restricted form of pointer, possibly mutable, and the checker is then validating / proving a set of constraints for how such borrowed references may be handled.

I view this sort of thing (along with the many different forms of pointer which Cyclone had) as completely different to the forms of near/far/huge/segment pointer which were available in various DOS compilers, and that (IMO) doesn't make for a sensible comparison.

Certain modern languages already have multiple forms of pointer (nullable, non-null, fat, thing, with and without arithmetic) and those do not create the same issues for the programmer as the DOS style pointers did.

D itself already has 3 different types of pointer: references, pointers, arrays/slices.

I don't see adding additional types as an issue, other then syntactical issues.  Obviously there will be implementation issues for the compiler, but depending upon the presented form, I do not see a reason why this should result in user issues.

On 5/10/2025 5:25 AM, Manu wrote:
>>     1. Function declarations don't have a body, so there's no way to inspect the
>>     body.
> A prototype without a body must always trust the declaration. You must simply assume that presence of `scope` on an arg asserts that the parameter won't be retained by the call, and the caller can also bank on that fact; ie, using temp/stack memory to allocate an argument.

Yes, of course, that's how dip1000 works.

>>     2. Inspecting all the bodies means compilation gets slowed down quite a bit.
> If you don't want escape analysis, don't write `scope`; point is, why do you need to write TWO attributes together for the first one (which is itself strictly optional) to take effect?

`scope` is needed to say that a pointer being passed does not escape. `return` is needed to specify that an escape via the function's return is allowed.


>>     3. Chicken-and-egg problems when inferring attributes when there are loops in
>>     the function call flow graph.
> I don't follow.

f calls g calls h calls f

You can't infer attributes of f because it depends on f. You can't infer attributes of g because it depends on f which depends  on g. And so on. To deal with this problem requires constructing Data Flow Equations and then solving those equations by iterating and converging on a unique solution. This is what the code in the link to gflow.d does. It's in the comments.

Unfortunately, building and solving the DFE's is slow. This is why the optimized builds are much slower than non-optimized builds. This is why the simplistic DFA is used to infer attributes rather than DFA, because we want fast non-optimizing builds.


> Is it something like; a template which receives an arg but doesn't explicitly state `scope` just spend time to try and infer `scope` because a caller of that template may rely on that attribute inference?

Of course.

> 
>      > I do want to add `scope` to pointers and ref arguments, but I don't see any
>      > reason that I should have to attribute `live`... why isn't `scope` enough to
>      > enable escape analysis?
> 
>     The DFA (Data Flow Analysis) required to determine whether a pointer is the
>     "owner" or not at each point in the function is considerably more complex than
>     just checking for scope-ness. DFA equations are generated and then solved
>     iteratively.
> 
>     The code required to do it is:
> 
>     https://github.com/dlang/dmd/blob/master/compiler/src/dmd/ob.d
>     <https://github.com/dlang/dmd/blob/master/compiler/src/dmd/ob.d>
> 
>     which is very similar to the DFA performed by the optimizer:
> 
>     https://github.com/dlang/dmd/blob/master/compiler/src/dmd/backend/gflow.d
>     <https://github.com/dlang/dmd/blob/master/compiler/src/dmd/backend/gflow.d>
> 
> 
> Yeah but I still don't get it; if you don't want escape analysis, don't write `scope`.

Yes, and then you cannot call that function with a scope pointer.


> If you do write it, you have clearly and unambiguously indicated that you DO want escape analysis, so it should happen.
> I can't see any reason a second unrelated attribute should be required in conjunction to make the first one work... you write `scope` thinking it will be effective, and then by not writing (or forgetting to write) the other thing, your code is now lying to you. You won't know this until you are bitten by the escape bug that you were trying to prevent; making the sutuation actually *worse* than useless, because when you're trying to track down your bug, you will see the attribute, and continue looking elsewhere.
> `scope` should work when you write it; period. If that is to say that writing `scope` infers the other attribute, whatever... but it's not a reasonable situation where `scope` silently does nothing and confuses or misleads the author.

It does work when you write it. The "second unrelated attribute" (which I assume is the "return" attribute) is necessary to indicate if the scope'd pointer returns or not. Not having the `return` attribute means the following code is impossible to write:

```
int* = foo(scope int* a) { return a; } // error: `a` is escaping via return!
scope int* p = ...;
scope int* q = foo(p);
```

On 5/10/2025 5:10 AM, Manu wrote:
> Okay, so then why should `scope` need `@safe`?

It doesn't. It needs -dip1000, though.

```
int* foo(scope int* q) { return q; }
```
```
./cc -c x.d -dip1000
x.d(1): Error: scope parameter `q` may not be returned
int* foo(scope int* q) { return q; }
                                ^
```