On Sunday, 23 October 2016 at 07:36:39 UTC, Walter Bright wrote:
> I can't, because I'm waiting for Andrei to make available what he considers a correct ref counted design.
>
> But you are correct in that the 'return' rules can be used to transfer the lifetime from an argument to the return value, even if the return value is not derived from the argument.

Glad to hear that. Even after 1 hour thinking in circles we couldn't figure out where scope could be used without this.
From my still limited understanding, the lifetime of the return value should be bound to the minimum (shortest) lifetime of all arguments that could potentially be aliased.
So if the return value is a struct containing an int* and a S*, it might alias any argument containing/or being an int or S (and maybe untyped memory).
This seems to be essential for any wrapper using malloced memory to safely return sth. (e.g. a range) with a pointer into that memory.

On Sunday, 23 October 2016 at 07:36:39 UTC, Walter Bright wrote:
> On 10/22/2016 4:22 PM, Dicebot wrote:
>> Walter, would you mind showing reference counted example that actually
>> works under your implementation?
>
> I can't, because I'm waiting for Andrei to make available what he considers a correct ref counted design.

We were working on that about more than a year ago when return ref landed. It didn't solve the escape problem for classes, but we have fairly solid understanding how a baseline implementation for unique, ref-counted, and weak-ref ownership should work.
Since those are primitives, we should be able to use them for the design of scope w/o a concrete RC.

https://github.com/dlang/phobos/pull/3259

If necessary, I can finish the rewrite to inform further discussion.

On Monday, 24 October 2016 at 19:32:45 UTC, Walter Bright wrote:
> For one thing, it would require all users to learn scope semantics and annotate them correctly. It would be like requiring const correctness.

Could we just turn off scope inference in @system code but escape check explicit scope usage?

On Tuesday, 25 October 2016 at 10:05:50 UTC, Walter Bright wrote:
> One way of fixing it besides doing multiple passes is noting its use inside dg, and setting a flag saying it cannot be made scope, and so the scoped assignment becomes an error.

A tristate tainting, unkown/scope/escape, should indeed work here.

On 10/24/2016 09:49 PM, Walter Bright wrote:
> ...

I think it is obvious that there is fundamental misunderstanding on both sides here about goals and need of the other one, is is better to make a step back and resolve that before continuing same discussion in circles.

Yesterday we had a very nice live discussion with Martin about current scope semantics and my concerns about it. He has mentioned a planned call with you later on same topic - let's wait for that and return to discussing details after if something will yet remain unclear.

On Saturday, 22 October 2016 at 07:38:18 UTC, Walter Bright wrote:
> If this is a class this then return scope is appropriate, because this is a pointer. But when this is a ref, scope then applies to the struct's fields.

Mmh, and if wanted to return a reference to the struct itself, we'd need to resort to return ref? Would it be possible to conflate return scope and return ref, at least for that use case?

>> Are you saying that existing system forgets that `instance` is `this`
>> the very moment `borrow` methods finishes? If yes, I am afraid it is a
>> rather useless feature, much worse than one may think reading DIP1000.
>
> By making borrow @trusted in order to defeat the checking, you defeated the connection between 'instance' the argument and the return value from the function.
>
> Do not convert 'ref' to '*'. It won't work.

As said further down in this thread, it seems that returning casted refs (or structs containing pointers) that alias internal memory, but are bound/scoped to the lifetime of the comtainer, is a major use-case. Why can't we use this explicit scope annotation on a trusted method to make this work?

>>> The practical result is that a container that is passed by ref can only
>>> control its uses by returning by ref, not by *.
>>
>> 1) Why? You still haven't answered how scope pointer is different from
>> ref in @safe code.
>
> I've been trying to answer. There are two things at work for variable 'v':
>
> 1. v's address
> 2. v's value
>
> 'ref' deals with the first, 'scope' with the second. It is very, very, very important to think about whether you are dealing with v's address or v's value, otherwise you will get hopelessly lost.

Could you elaborate a bit on that?

> Next, recall that 'this' for a struct is an address. 'this' for a class is a value.

The other way round, right?

> That's what scope variables are for - storing addresses in.
>
> I implore you to look at the test cases with the PR. They are all trivial and easy to follow.

I hope you agree that we must check the design against our actual high level goals. While the breaking down the rules is a good approach for a formal view, being able to puzzle together the pieces to useful idioms is essential.

On 10/25/2016 5:39 PM, Martin Nowak wrote:
> From my still limited understanding, the lifetime of the return value should be
> bound to the minimum (shortest) lifetime of all arguments that could potentially
> be aliased.

That's right.

> So if the return value is a struct containing an int* and a S*, it might alias
> any argument containing/or being an int or S (and maybe untyped memory).

If that parameter is marked "return scope", then yes.

I think I thought of a solution. Currently, the design ignores 'scope' if it is applied to a variable with no indirections. But we can make it apply, in that the 'scope' of a function's return value is tied to the 'scope' of the argument, even if that argument has no indirections.

This should make Dicebot's design work.

On Wednesday, 26 October 2016 at 09:59:07 UTC, Dicebot wrote:
> Yesterday we had a very nice live discussion with Martin about current scope semantics and my concerns about it. He has mentioned a planned call with you later on same topic - let's wait for that and return to discussing details after if something will yet remain unclear.

Yes, let's do that on tuesday if that suits all of you, will send around an invite. I think we're all in the topic now and finally close to mutual understanding ;).

On Monday, 24 October 2016 at 19:49:10 UTC, Walter Bright wrote:
> On 10/23/2016 11:49 PM, Dicebot wrote:
>> On 10/24/2016 05:29 AM, Walter Bright wrote:
>>> On 10/22/2016 3:50 PM, Dicebot wrote:
>>>> Beauty of borrowing is that it allows to to inc/dec skipping with no
>>>> special RC support from compiler and works the very same for stack
>>>> allocated data. What you are going at is simply replacing GC with
>>>> (compiler enhanced) RC and that is not good at all.
>>>
>>> I don't see why supporting something beyond RC and GC is necessary.
>>
>> Both GC and RC are forms of automatic memory management with unavoidable
>> associated overhead. Both have their uses but are unacceptable in most
>> of the system code. Which means that would would either need to
>> duplicate all Phobos/druntime functions to work with both RC and plain
>> pointers or it will become even less suitable for low level code than it
>> is now.
>
> I'm afraid that is a bit vague. Can you give a more concrete example?

Also important are Unique ownership (handing out scoped references to internal (untyped) memory) and ranges that cannot outlive their containers.

>> - allow taking address of a `ref` and define lifetime of such expression
>> to be same of `ref` lifetime
>
> Again, why? Why can't the function return by 'ref'?

While returning by ref is good enough to prevent escaping, and we do use it in RefCounted for structs, it doesn't allow you to bind that returned value anywhere.
So you could only chain method calls s.get.call.getNested.doSome.foo.bar;.

While that might be fine as a first step, it doesn't seem to need scope/return scope, and is fairly limiting in the mid-term.

>> - `return scope` of a struct methods binds returns value lifetime to one
>> of the struct instance itself, not composition of its fields
>
> 'ref' return already does that.

Would it be possible to conflate the two different annotatons with similar but slightly different semantic? Couldn't return scope on the method mean, that a reference to the whole struct can be returned.