On 10/24/2016 2:11 PM, Mathias Lang wrote:
> 1) As highlighted in the previous post, we should not change the STC / type of a
> declaration after it is declared. It's a can of worms and will break in many ways.

The definition of a storage class is a bit fuzzy. 'scope' does not say where the variable is stored - it does not say "put it on the stack", "put it in the code segment", "put it on the heap". It's more of an attribute, just like nothrow, pure, etc.


> 2) I don't know a single place in the language where a single "something" (type
> / attributes / STC) is infered. It's either all or nothing (e.g. you cannot have
> a function infering safety but not infering nothrow / @nogc, short of tricking
> the compiler by changing the function's code). Also, I don't know of a single
> place in the language where a type provided explicitly is infered. For example,
> if you do `char[] s = "Hello World";`, `const` or `immutable` is not infered,
> and you get an error, and that's a good thing. Changing it would make code way
> less readable and harder to reason about.

'scope' is not a type, although it does affect function type signatures the same as @safe, pure, nothrow and @nogc do, as well as affecting covariance/contravariance in an equivalent manner.

On 10/24/2016 2:02 PM, Mathias Lang wrote:
> On Monday, 24 October 2016 at 19:30:16 UTC, Walter Bright wrote:
>> On 10/24/2016 2:13 AM, Mathias Lang wrote:
>>> So it means that the storage class of 'e' changes during semantic analysis of
>>> the function. I don't think that's something we want to ever do.
>>
>> Why? D already infers lots of things.
>
> To answer your question: Because it isn't even inference.
> The specs explicitly says "Unlike module level declarations, declarations within
> function scope are processed in order." (https://dlang.org/spec/function.html)
> Changing the storage class of a previous declaration is then explicitly breaking
> that line of the spec, and breaking a key assumption the specs / compiler relies
> on.

I don't believe it breaks anything the language relies on.


> I pointed that problem using delegate, I could have done so using templates,
> nested functions, or whatever part of the language that relies on said assumption.

Please wait until I investigate that one.


> On Monday, 24 October 2016 at 19:32:45 UTC, Walter Bright wrote:
>> On 10/24/2016 9:49 AM, Mathias Lang wrote:
>>> Why is this necessary / what would be the problem in allowing it in @system ?
>>
>> For one thing, it would require all users to learn scope semantics and
>> annotate them correctly. It would be like requiring const correctness.
>
> Users will have to learn `scope` semantic to write `@safe` code.
> If one doesn't want to learn `scope` semantics, one just doesn't use it, `scope`
> is opt-in after all.
> I fail to see any benefit for someone that doesn't want to learn `scope`,

Writing @system code can be done without paying attention to that stuff, and there is value in that. @safe code comes into play when writing larger scale, longer lived (!) code, and code that is worked on by a team.


> and find it extremely confusing for users that want to use it but can't make their
> code `@safe` (because for example, they're accepting a delegate which can be
> @system so they can't just wrap code in `@trusted`).

That's the reason for inference. (I'd also like to do inference for 'const'. Inference of pure, nothrow, etc., has been very successful, and I don't see any fundamental issue with taking it further.)

On Monday, 24 October 2016 at 21:30:50 UTC, Walter Bright wrote:
> On 10/24/2016 2:11 PM, Mathias Lang wrote:
>> 1) As highlighted in the previous post, we should not change the STC / type of a
>> declaration after it is declared. It's a can of worms and will break in many ways.
>
> The definition of a storage class is a bit fuzzy. 'scope' does not say where the variable is stored - it does not say "put it on the stack", "put it in the code segment", "put it on the heap". It's more of an attribute, just like nothrow, pure, etc.

`scope o = new Object;`
If you prefer to compare it to `nothrow` or `pure`, that's also fine by me.
I still don't know of any place where an attribute can be added to a pre-existing variable declaration.


> 'scope' is not a type, although it does affect function type signatures the same as @safe, pure, nothrow and @nogc do, as well as affecting covariance/contravariance in an equivalent manner.

The DIP is very clear that `scope` is not a type qualifier.
The parallel with `const` inference was drawn as a reference to https://forum.dlang.org/post/nubdqu$v6u$1@digitalmars.com (and your later post).
I see both inference as getting in the way of code readability if not requested by the user explicitly.


On Monday, 24 October 2016 at 21:40:48 UTC, Walter Bright wrote:
> 
> Please wait until I investigate that one.

Fair enough, then I'll leave the point of adding attributes for later.


>> On Monday, 24 October 2016 at 19:32:45 UTC, Walter Bright wrote:
>>> On 10/24/2016 9:49 AM, Mathias Lang wrote:
>> Users will have to learn `scope` semantic to write `@safe` code.
>> If one doesn't want to learn `scope` semantics, one just doesn't use it, `scope`
>> is opt-in after all.
>> I fail to see any benefit for someone that doesn't want to learn `scope`,
>
> Writing @system code can be done without paying attention to that stuff, and there is value in that. @safe code comes into play when writing larger scale, longer lived (!) code, and code that is worked on by a team.

Could you explain the value there is in not checking `scope` in `@system` ?
Also, how does that value offset the user confusion and possible bugs caused by people trying to use `scope` in `@system` code without realizing it doesn't do any check ?
Quick note: I notice you said `@system`. The DIP said "only checked in @safe code". Will @trusted by checked ?

Also, I don't quite see what @safe has to do with code lifetime, or collaboration.

On 10/24/2016 3:10 PM, Mathias Lang wrote:
> `scope o = new Object;`
> If you prefer to compare it to `nothrow` or `pure`, that's also fine by me.
> I still don't know of any place where an attribute can be added to a
> pre-existing variable declaration.

You're right, that is a bit of a special case, inherited from D1. But storing it on the stack is an optimization, it behaves "as if" it was allocated on the heap.


> I see both inference as getting in the way of code readability if not requested
> by the user explicitly.

At this point, we'll just have to disagree on that.


> Could you explain the value there is in not checking `scope` in `@system` ?

Quick and dirty code, for one.


> Also, how does that value offset the user confusion and possible bugs caused by
> people trying to use `scope` in `@system` code without realizing it doesn't do
> any check ?

It still checks the interface.


> Quick note: I notice you said `@system`. The DIP said "only checked in @safe
> code". Will @trusted by checked ?

No, because @trusted code is system code.


> Also, I don't quite see what @safe has to do with code lifetime,

Wouldn't you want to take more care when you write code that will be used for years, rather than some throwaway piece of code?


>  or collaboration.

Because it enables your collaborators to understand your code better.

On 10/24/2016 2:13 AM, Mathias Lang wrote:
> Crafting a test case that exploit this was trivial:
> void* escape3 (scope void* p) @safe
> {
>     Escaper e;
>     scope dg = () { return e.e; };
>     e.e = p;
>     return dg();
> }

A complete example:

--------
struct Escaper { void* e; }

void* escape3 (scope void* p) @safe {
    Escaper e;
    e.e = p;
    scope dg = () { return e.e; }; // Error: scope variable e may not be returned
    return dg();
}
-------

But changing the order of things:

--------
struct Escaper { void* e; }

void* escape3 (scope void* p) @safe {
    Escaper e;
    scope dg = () { return e.e; }; // no error
    e.e = p;
    return dg();
}
-------

This definitely is a bug, but is fixable. Thanks for posting it!

On Monday, 24 October 2016 at 09:13:58 UTC, Mathias Lang wrote:
> void* escape3 (scope void* p) @safe
> {
>     Escaper e;
>     scope dg = () { return e.e; };
>     e.e = p;
>     return dg();
> }

Seems like this would require that the scope tainting is done in a separate pass, before going through the statements in lexical order. The dg needs to know that e will escape in order to change to return scope.
Not sure the compiler is ready for 2-pass function semantics.
While it's clearly powerful, my intuition also goes a bit against it.
- Does the compiler architecture allow for the necessary semantics.
- What's a use-case that requires this?
- Might the scope inference graph have cycles?
- Can humans understand scope inference, and can we print proper error messages?

It might be a bit too ambituous, and turn out to be an unforced error creating too many scope holes.

On Monday, 24 October 2016 at 23:10:46 UTC, Walter Bright wrote:
> This definitely is a bug, but is fixable. Thanks for posting it!

Let's please talk about how this needs fixing beforehand, see my post below.
It seems to me this requires 2 semantic passes through function statememts :o.

On Saturday, 22 October 2016 at 15:52:17 UTC, Mathias Lang wrote:
>> It's the same as 'return ref', which is already there, except it's 'return scope'. The examples on how it works all boil down to a couple lines of code.
>
> Most people seem to disagree with that opinion.
>
> I was looking at the DIP, and checked out the P.R. branch to test it, and I cannot see how it could work if `scope` is not transitive.

Let's clarify the term transitive, it's quite possible different people have different understandings of that term.

Only drawing on my conclusion from rewriting the RC/Unique implementations.
What's needed is a "container" to wrap return values so that they cannot be aliased or escaped, a bit similar to how you can't take the address of an rvalue.
If you could get a named alias (e.g. scope ref/ptr) into the returned scope value, then the lifetimes of the two variables are entangled, and it would be trivial to end up with a dangling scoped ref/ptr.

scope S s:
scope int* p = s.getInternalRef; // could be a malloced class ref instead of a pointer
if (flipCoin)
  s.destroy; // or s = S();

To make this work, we'd need to disallow any mutation of s, while references to it are reachable.
I wonder if that's worth the trouble over simply disallowing any binding to a named (and thus reachable) variable. Would require chaining of return values though, s.get.cannot.put.that.hot.potatoe.anywhere :/.

On Tuesday, 25 October 2016 at 07:51:21 UTC, Martin Nowak wrote:
> To make this work, we'd need to disallow any mutation of s, while references to it are reachable.

Didn't mention this explicitly. Binding the returned scope value as named scope ref argument to a chained function call must of course be possible, but that still wouldn't allow to bind to a variable that outlives the expression.
So it's really somewhat similar to rvalues.

On 10/25/2016 12:09 AM, Martin Nowak wrote:
> Let's please talk about how this needs fixing beforehand, see my post below.
> It seems to me this requires 2 semantic passes through function statememts :o.

One way of fixing it besides doing multiple passes is noting its use inside dg, and setting a flag saying it cannot be made scope, and so the scoped assignment becomes an error.

General flow analysis generally requires an arbitrary number of passes to converge on a solution (the global optimizer does this), but we can do it in one pass with the conservative approach above, and just accept that it's conservative.