On Saturday, 22 October 2016 at 07:19:11 UTC, Walter Bright wrote:
> On 10/21/2016 1:30 AM, pineapple wrote:
>> There's this big language enhancement in the pipeline
>
> But it's not big.
>
> It's the same as 'return ref', which is already there, except it's 'return scope'. The examples on how it works all boil down to a couple lines of code.

Most people seem to disagree with that opinion.

I was looking at the DIP, and checked out the P.R. branch to test it, and I cannot see how it could work if `scope` is not transitive.
It is just pushing the problem one level down.

For example:

```
alias FunDG = void delegate () @safe;

struct Escaper
{
    FunDG DG;
}

FunDG escapeDg1 (scope FunDG d) @safe
{
    Escaper e;
    e.DG = d;
    return e.DG;
}
```

This compiles, and escapes a `scope delegate`.
Used `./src/dmd -transition=safe -dip25 -run test.d` to compile, on commit:
`48b7815 - (HEAD, walter/return-scope) add lifetime checks (9 days ago) <Walter Bright>`. Did I miss something ?


Also, as others already mentioned, the DIP is quite confusing.

I read the whole thread and AFAICS people got confused because of https://github.com/dlang/DIPs/blob/master/DIPs/DIP1000.md#escaping-via-return which mentions that "taking the address of a local [...] is restrictive." and continue with "The `scope` storage class was introduced which annotates the pointer.", which doesn't quite relate to the previous sentence if scope is not meant to remove this restriction.
The examples also use a mix of `@safe` and non-annotated function, which doesn't help understanding since it's explicitly stated that those checks are only done in @safe code (why?).
Finally, I agree with Dicebot that having scope infered when the type is specified is extremely confusing, and makes writing test cases, and code in general, much more complicated.

On 10/21/2016 09:43 PM, meppl wrote:
> On Friday, 21 October 2016 at 15:38:39 UTC, Dicebot wrote:
>>
>> 2) The whole thing is very disappointing ...
>> ... won't allow easy skipping of reference count inc/dec - ...
> 
> Isn't that - in the short term - impossible anyway? If a struct leaves
> the scope the reference counter has to be increased. But how does
> compiled code of a library know, if the foreign struct is an RC-object -
> without doing any runtime checks? So, any function has to exist twice,
> one time for RC and one time for no-RC?
> I mean, it's probably a big topic that doesn't have to be solved in that
> DIP, too.

It would be possible with even most simple borrowing semantics, see http://forum.dlang.org/post/nu00tf$u0a$1@digitalmars.com - if one can bind lifetime of a scope pointer to lifetime of stack variable, it becomes OK to implicitly convert reference counted struct to scope pointer for passing to other functions. So you only define function taking scope pointer if it doesn't store it (and it works with both RC and other data sourceS) or taking RC pointer explicitly if does indeed store it.

But with the system Walter is proposing it is indeed impossible.

On 10/22/2016 10:52 AM, Walter Bright wrote:
>> It won't allow to fix `Unique`, won't enable
>> new RNG design Joe has been going, won't allow easy skipping of
>> reference count inc/dec - the list goes on.
> 
> You're right, scope won't help at all in inc/dec skipping. But all is
> not lost. I have written up a short article on how the compiler can do
> inc/dec skipping, but it'll have to wait until we have an actual RC design.

Beauty of borrowing is that it allows to to inc/dec skipping with no special RC support from compiler and works the very same for stack allocated data. What you are going at is simply replacing GC with (compiler enhanced) RC and that is not good at all.

Would you at least be willing to consider an alternative proposal before `scope` branch is merged for release? I have a feeling that with relatively few changes to DIP1000 rules we can both have the cookie and eat it (i.e. get a way to express non-transitive borrowing), more in spirit of original proposal by Marc.

Ok, so I am revisiting DIP1000 right now based on recent comments from Walter. One thing remains unclear to me though, consider this example in DIP1000 document:

```
@safe struct RefCountedSlice(T) {
	private T[] payload;

	// ...

        // interesting fact #2: references to internals can be given away
	scope ref T opIndex(size_t i) {
		return payload[i];
	}
}
```

It looks like with current semantics this statement is very wrong. Assuming that you have meant `scope return` instead (because plain scope does nothing for `this` reference) Right now it does not matter how `opIndex` is annotated if variable holding `RefCountedSlice` itself is not annotated/deduced as scope:

```
@safe struct S
{
    // RefCountedSlice code reduced to bare minimum

    private int* payload;

    this (int) {
        this.payload = new int;
    }

    @trusted ~this () {
        delete this.payload;
    }

    int* get () scope return {
        return this.payload;
    }
}

int* global;

@safe unittest
{
    scope s = S(42);
    global = s.get(); // Error: scope variable s assigned to global with
longer lifetime
}

@safe unittest
{
    auto s = S(42);
    global = s.get(); // compiles, oops
}
```

And indeed, with your explanations it seems to work as intended - `scope return` annotation is simply forwarding lifetime of `S.payload` to return value, which happens to be infinity by lifetime rules (GC allocation). But that means that statement saying destructor can be @trusted is wrong.

Walter, would you mind showing reference counted example that actually works under your implementation?

On 10/22/2016 8:52 AM, Mathias Lang wrote:
> I was looking at the DIP, and checked out the P.R. branch to test it, and I
> cannot see how it could work if `scope` is not transitive.
> It is just pushing the problem one level down.
>
> For example:
>
> ```
> alias FunDG = void delegate () @safe;
>
> struct Escaper
> {
>     FunDG DG;
> }
>
> FunDG escapeDg1 (scope FunDG d) @safe
> {
>     Escaper e;
>     e.DG = d;
>     return e.DG;
> }
> ```
>
> This compiles, and escapes a `scope delegate`.

1. It is not an example of transitivity. e and d are at the same level, being a field does not make it a level down.

2. You're right that the above should error, I'll look into it.


> Finally, I agree with Dicebot that having scope infered when the type is
> specified is extremely confusing,

What is confusing about it?

   scope int* p = ...;
   int* q = p; // q becomes 'scope' as well

The compiler already does tons of inference for types and attributes. People should be well used to it by now :-)


> and makes writing test cases, and code in
> general, much more complicated.

Actually, it makes things much simpler allowing the compiler to take care of it. The whole scope thing would be fairly unusable if one had to manually add all the annotations.

On 10/22/2016 7:34 PM, Walter Bright wrote:
> 2. You're right that the above should error, I'll look into it.

Added the fix to the PR.

On 10/22/2016 4:22 PM, Dicebot wrote:
> Walter, would you mind showing reference counted example that actually
> works under your implementation?

I can't, because I'm waiting for Andrei to make available what he considers a correct ref counted design.

But you are correct in that the 'return' rules can be used to transfer the lifetime from an argument to the return value, even if the return value is not derived from the argument.

On 10/22/2016 3:50 PM, Dicebot wrote:
> Beauty of borrowing is that it allows to to inc/dec skipping with no
> special RC support from compiler and works the very same for stack
> allocated data. What you are going at is simply replacing GC with
> (compiler enhanced) RC and that is not good at all.

I don't see why supporting something beyond RC and GC is necessary.

> Would you at least be willing to consider an alternative proposal before
> `scope` branch is merged for release? I have a feeling that with
> relatively few changes to DIP1000 rules we can both have the cookie and
> eat it (i.e. get a way to express non-transitive borrowing), more in
> spirit of original proposal by Marc.

It'd have to be a big improvement over DIP1000, without sacrificing easy use. I find Rust's system to be seriously restrictive, off-putting, and hear many reports of users fighting its borrow checker. (I have not written any Rust code myself.)

At some point we need to move forward. We are already years late. Another month has slipped by.

I am also not convinced that you cannot do what you want to do with DIP1000. I do not understand why it is necessary to return the address of a value passed by ref, when one can return by ref.

On 10/24/2016 05:29 AM, Walter Bright wrote:
> On 10/22/2016 3:50 PM, Dicebot wrote:
>> Beauty of borrowing is that it allows to to inc/dec skipping with no special RC support from compiler and works the very same for stack allocated data. What you are going at is simply replacing GC with (compiler enhanced) RC and that is not good at all.
> 
> I don't see why supporting something beyond RC and GC is necessary.

Both GC and RC are forms of automatic memory management with unavoidable associated overhead. Both have their uses but are unacceptable in most of the system code. Which means that would would either need to duplicate all Phobos/druntime functions to work with both RC and plain pointers or it will become even less suitable for low level code than it is now.

>> Would you at least be willing to consider an alternative proposal before `scope` branch is merged for release? I have a feeling that with relatively few changes to DIP1000 rules we can both have the cookie and eat it (i.e. get a way to express non-transitive borrowing), more in spirit of original proposal by Marc.
> 
> It'd have to be a big improvement over DIP1000, without sacrificing easy use. I find Rust's system to be seriously restrictive, off-putting, and hear many reports of users fighting its borrow checker. (I have not written any Rust code myself.)

The problem is DIP1000 is a lie. I started to rewrite it to match your
implemented semantics but that would simply result in throwing much of
its content away. Some of crucial things DIP1000 promises to deliver but
are not possible with design as in https://github.com/dlang/dmd/pull/5972 :

- @system code like std.internal.scopebuffer can be made @safe
- reference counting systems need not adjust the count when passing
references that do not escape

That means that any counter-proposal that actually allows to do those things would already be considerably better.

Now, I do agree that we don't want to go with amount of complexity Rust has. Something that makes basic idioms possible at 10% of complexity price would be a much better trade-off. Thus I'd propose to keep most of the stuff in DIP1000 as it is (including non-transitivity), but enhance/tweak lifetime rules, for example:

- allow taking address of a `ref` and define lifetime of such expression
to be same of `ref` lifetime
- `return scope` of a struct methods binds returns value lifetime to one
of the struct instance itself, not composition of its fields

It is of similar complexity as existing proposal but is good enough to allow design patterns I have been trying to do.

> At some point we need to move forward. We are already years late. Another month has slipped by.

Current implementation does not bring us closer to feature rich compiler-verified safety in any way. What is even worse, going with it will close the option of using same syntax for something better and we will be stuck with it forever.

I have a feeling that you completely misunderstand why Rust system has all the hype. Would it help if I explained some of compiler-verified safety patterns there? I have put some time into studying Rust last year.

> I am also not convinced that you cannot do what you want to do with DIP1000. I do not understand why it is necessary to return the address of a value passed by ref, when one can return by ref.

You are author of the proposal. It is your obligation to convince everyone that it is possible to do what you claim possible, not mine. If you think it can be done, show me the code.

On Monday, 24 October 2016 at 02:29:35 UTC, Walter Bright wrote:

> At some point we need to move forward. We are already years late. Another month has slipped by.

That argument can go two ways.
Since we are already years late another month won't matter that much.
D's main advantage is hindsight and careful consideration of complexity versus power!
We should stick to those values and trying to beat someone in a marketing hype.