On Friday, 12 December 2014 at 08:44:56 UTC, Walter Bright wrote:
> On 12/12/2014 12:16 AM, deadalnix wrote:
>> On Friday, 12 December 2014 at 07:48:21 UTC, Walter Bright wrote:
>>> Are you suggesting two kinds of scope - transitive and non-transitive?
>>>
>>
>> Non transitive scope can be added without any language extension.
>
> In order for it to work, the holes in the language that enabled escapes had to be plugged. That's what this proposal does - plug the holes.
>

The holes covered by a non transitive scope are already undefined behavior.

>> ref counting is just one form of ownership.
>
> My point is if that works with this proposal, then the other forms can work, too.

No, unless we wrap every single indirection (pointer, slice, classes, delegates) into a wrapper. That is not going to fly very far.

On Friday, 12 December 2014 at 00:13:10 UTC, deadalnix wrote:
> On Thursday, 11 December 2014 at 13:55:55 UTC, Marc Schütz wrote:
>> This is a point that most people don't seem to understand yet, and which wasn't obvious for me either, at the beginning:
>>
>> * A `ref` parameter means that it cannot escape the function, _except_ by return.
>> * A `scope ref` parameter means that it cannot escape the function  _ever_, not even by return.
>> * A `scope ref` return means that it cannot leave the current statement.
>>
>> Therefore, a `scope ref` return value can be passed on to the next function as a `ref` argument. If that function again returns a reference (even if not explicitly designated as `scope`), the compiler will treat it as if it were `scope ref`.
>>
>
> No, it understood.

Steven hadn't, evidently.

> It is simply not useful.

I'm not convinced either.

>
>> I agree, this is important. In my proposal, this works without transitivity. The wrapper stores the pointer as a `scope` member, then by copying the wrapper, the pointer gets copied implicitly, to which the normal scope restrictions apply (`scope` on members means "will not outlive the aggregate"). If it stored it as normal non-scope pointer, it couldn't get assigned in the first place.
>
> Wut ? You cante store anything with grear lifetime, including non scope things (as they'll have infinite lifetime). Meaning the only thing you know, is that thing are possibly scoped.
>
> Meaning you have to assume infinite lifetime with every indirection, which make this proposal useless.

My comments above don't refer to this proposal, but my original one:

    struct Wrapper(T) {
        scope T payload;
        // used to be:
        // scope!this T payload;
    }

    scope int a;
    auto w = Wrapper!(int*)(&a); // ok
    scope int b;
    w.payload = &b;              // error, w (and therefore w.payload)
                                 // lives longer than b
    Wrapper!(int*) w2 = w;       // ok, lifetime(w2) < lifetime(w)
    w = w2;                      // error, equivalent to
                                 // w.payload = w2.payload

Therefore, wrapping is safe without transitivity of scope.

On Friday, 12 December 2014 at 08:42:22 UTC, Manu via Digitalmars-d wrote:
> On 11 December 2014 at 23:55, via Digitalmars-d
> <digitalmars-d@puremagic.com> wrote:
>> This is a point that most people don't seem to understand yet, and which
>> wasn't obvious for me either, at the beginning:
>>
>> * A `ref` parameter means that it cannot escape the function, _except_ by
>> return.
>> * A `scope ref` parameter means that it cannot escape the function  _ever_,
>> not even by return.
>> * A `scope ref` return means that it cannot leave the current statement.
>>
>> Therefore, a `scope ref` return value can be passed on to the next function
>> as a `ref` argument. If that function again returns a reference (even if not
>> explicitly designated as `scope`), the compiler will treat it as if it were
>> `scope ref`.
>
> Ummm. I reckon there's a good reason that people don't seem to
> understand this... because it would have difficulty being any more
> unintuitive!
> It's contrary to the behaviour of basically everything else in D!
> const, pure, nothrow, etc... they all work a reliable and consistent way.

Unfortunately I have to agree.

>
> Also, I still don't think I get it... a scope-ref return, which has a
> restriction as you say, can be subverted by being piped through a
> function that receives and returns it as ref?
> What is the point?

It cannot be subverted. As soon as you pass something into a function G() as `ref` that's been returned from a function F() via `scope ref`, the compiler needs to treat what is returned from G() as `scope ref`, even if it's only declared as `ref`.

As you say, it's unintuitive and non-obvious.

>
>>> I'm also quite uneasy with the fact that scope would not be transitive
>>> as a storage class. What happens when it's applied to a value type,
>>> like a struct, that contains some pointers? An adaptation wrapper for
>>> a single pointer is super common; ie, a tiny struct passed by value
>>> with scope needs to have it's contained pointer receive the scope-ness
>>> of the argument.
>>
>>
>> I agree, this is important. In my proposal, this works without transitivity.
>> The wrapper stores the pointer as a `scope` member, then by copying the
>> wrapper, the pointer gets copied implicitly, to which the normal scope
>> restrictions apply (`scope` on members means "will not outlive the
>> aggregate"). If it stored it as normal non-scope pointer, it couldn't get
>> assigned in the first place. (Additionally, some higher level tricks are
>> possible if we allow scope for value types and overloading on scope.)
>
> How can a member be marked scope? Apparently it's a storage class, not
> a type constructor... we can only attribute storage classes to
> function args/results(?).

I was talking about my proposal here, where it's a type modifier, not a storage class.

deadalnix also brought up the problem of transitivity. This, too, would simply go away if it's a type modifier.

On 12 December 2014 at 07:41, Walter Bright via Digitalmars-d <digitalmars-d@puremagic.com> wrote:
> On 12/11/2014 4:47 AM, Manu via Digitalmars-d wrote:
>>
>> On 8 December 2014 at 07:29, Walter Bright via Digitalmars-d <digitalmars-d@puremagic.com> wrote:
>>>
>>> On 12/7/2014 6:12 AM, Dicebot wrote:
>>>>
>>>>
>>>> But from existing cases it doesn't seem working good enough. For
>>>> example,
>>>> not
>>>> being able to represent idiom of `scope ref int foo(scope ref int x) {
>>>> return x;
>>>> }` seems very limiting.
>>>
>>>
>>>
>>>    scope ref int foo(ref int x);
>>>
>>> will do it.
>>
>>
>> Will it? It looks like foo can't be called with scope data?
>
>
> Yes, it can be.
>
>
>> I don't have the perfect proposal, but I feel very strongly about 2
>> things:
>> 1. It must not be a storage class; the concept was a disaster with
>> ref, and I struggle with this more frequently than any other 'feature'
>> in D.
>
>
> I simply do not understand why distinguishing beteen ref and not-ref is a cornerstone of everything you do.

I've said so many times, it's the single greatest regular frustration
I encounter, by far.
That is of course a relative measure. It's not the 'cornerstone of
everything I do'; I don't start discussions about things that are not
broken and otherwise painless.
It is the thing that is *the most broken*, and leads to the most edge
cases, general bloat, and text mixins.
It comes up the most frequently, and by that metric alone, I consider
it highest priority on my list.

I've also said many times before, it possibly stems from the fact that
one of the key cornerstones of almost everything I do in D is interact
with other languages.
This is a practical reality, I can't write all my code in D, and have
mountains of existing code to interact with.
Boilerplate is the result. D has powerful systems to automate
boilerplate, like a carrot dangling right in front of my face, but
it's almost always thwarted by ref, and almost exclusively so.

My recent work updating LuaD to support all the features I required
wasted about 80% of my time dealing with ref issues.
In my past C++ bindings solutions, much code, which was otherwise
simple, readable, and elegant, turned into a mess of text mixins,
almost exclusively because ref is broken.


>> 2. I feel it's a big mistake to separate it from the type system, which I think most agree, is D's greatest asset by far. Manipulating types is simple and convenient using the type system, and I think manipulating scope will be just as important as any other attribute as soon as even slightly complex use cases begin to arise.
>
>
> Consider a ref counted type, RC!T. If scope were transitive, then you could not have, say, a tree where the edges were RC!T. I.e., the payload of an RC type should not be forced to be scope.

I'm not sure I quite visualise this correctly...

It sounds like you're talking about the same problem where people
complain with respect to 'const' and you happily tell them to stop
whingeing and deal with it?
(FWIW, I think you made the right call on const. It's never caused me
any hair loss.)

So you're saying that a pointer itself shouldn't be able to escape a
call tree, but the thing it points to should be able to escape just
fine?
It feels like that kinda defeats the purpose... or at least makes the
concept about as reliable as 'const' is in C++.

I guess you're seeing a situation where 'scope' is almost exclusively
useful as a mechanism to tell the RC that it doesn't need to worry
about ref-fiddling, and the thing we're passing isn't interested in
scope restrictions at any level other than that RC optimisation?
I guess I can see your angle... but my reaction is if that's all you
are concerned about, maybe scope is the wrong tool for eliding ref
fiddling in that case :/

I can see how you arrived at the opinion from your other comment
(wherever it was) that you are kinda likening this whole problem to
the RC case, if that's how you're looking at it.
It sounds like what you want is something like head-scope, in the same
way that people often want head-const?

On 12 December 2014 at 08:53, John Colvin via Digitalmars-d <digitalmars-d@puremagic.com> wrote:
> On Thursday, 11 December 2014 at 21:41:11 UTC, Walter Bright wrote:
>>
>> On 12/11/2014 4:47 AM, Manu via Digitalmars-d wrote:
>>>
>>> On 8 December 2014 at 07:29, Walter Bright via Digitalmars-d <digitalmars-d@puremagic.com> wrote:
>>>>
>>>> On 12/7/2014 6:12 AM, Dicebot wrote:
>>>>>
>>>>>
>>>>> But from existing cases it doesn't seem working good enough. For
>>>>> example,
>>>>> not
>>>>> being able to represent idiom of `scope ref int foo(scope ref int x) {
>>>>> return x;
>>>>> }` seems very limiting.
>>>>
>>>>
>>>>
>>>>   scope ref int foo(ref int x);
>>>>
>>>> will do it.
>>>
>>>
>>> Will it? It looks like foo can't be called with scope data?
>>
>>
>> Yes, it can be.
>>
>>
>>> I don't have the perfect proposal, but I feel very strongly about 2
>>> things:
>>> 1. It must not be a storage class; the concept was a disaster with
>>> ref, and I struggle with this more frequently than any other 'feature'
>>> in D.
>>
>>
>> I simply do not understand why distinguishing beteen ref and not-ref is a cornerstone of everything you do.
>
>
> Because he requires control over function ABIs for both inter-language communication and performance.
>
> In binding D to IDL (Interactive Data, not Interface Description) I found ref often required special casing.

Thank you. I feel like I'm just repeating myself over and over again.

On 12 December 2014 at 22:18, via Digitalmars-d <digitalmars-d@puremagic.com> wrote:
> On Friday, 12 December 2014 at 08:42:22 UTC, Manu via Digitalmars-d wrote:
>>
>> How can a member be marked scope? Apparently it's a storage class, not a type constructor... we can only attribute storage classes to function args/results(?).
>
>
> I was talking about my proposal here, where it's a type modifier, not a storage class.

Oh okay. Is that still on the table?


> deadalnix also brought up the problem of transitivity. This, too, would simply go away if it's a type modifier.

Just for clarity, does type modifier mean the same thing as type constructor?

On Friday, 12 December 2014 at 12:34:40 UTC, Manu via Digitalmars-d wrote:
> On 12 December 2014 at 22:18, via Digitalmars-d
> <digitalmars-d@puremagic.com> wrote:
>> On Friday, 12 December 2014 at 08:42:22 UTC, Manu via Digitalmars-d wrote:
>>>
>>> How can a member be marked scope? Apparently it's a storage class, not
>>> a type constructor... we can only attribute storage classes to
>>> function args/results(?).
>>
>>
>> I was talking about my proposal here, where it's a type modifier, not a
>> storage class.
>
> Oh okay. Is that still on the table?

Dunno, maybe? DIP69 doesn't look very well-received at the moment...

>
>
>> deadalnix also brought up the problem of transitivity. This, too, would
>> simply go away if it's a type modifier.
>
> Just for clarity, does type modifier mean the same thing as type constructor?

Yes. "Type modifier" = const, shared, immutable, etc.; "type constructor" = the syntactical construct by which a type is constructed/modified. Andrei suggested this differentiation some time ago (sorry, don't have a reference at hand).

On Friday, 12 December 2014 at 12:34:40 UTC, Manu via
Digitalmars-d wrote:
>> deadalnix also brought up the problem of transitivity. This, too, would
>> simply go away if it's a type modifier.
>
> Just for clarity, does type modifier mean the same thing as type constructor?

It appears to me that scopeness is a completely different beast
than type qualifier, and type qualifier won't work.

I think type modifier was just made up here. And why not ? It is
more like some metadata (lifetime) being associated with value
and symbols, like a type would, but that isn't really per se a
type as D understands it right now.

On Friday, 12 December 2014 at 13:38:10 UTC, Marc Schütz wrote:
> Yes. "Type modifier" = const, shared, immutable, etc.; "type constructor" = the syntactical construct by which a type is constructed/modified. Andrei suggested this differentiation some time ago (sorry, don't have a reference at hand).

That won't fly. scope turtle left, type qualifier turtle right.

On 12/12/2014 2:28 AM, deadalnix wrote:
> On Friday, 12 December 2014 at 08:44:56 UTC, Walter Bright wrote:
>> On 12/12/2014 12:16 AM, deadalnix wrote:
>>> On Friday, 12 December 2014 at 07:48:21 UTC, Walter Bright wrote:
>>>> Are you suggesting two kinds of scope - transitive and non-transitive?
>>>>
>>>
>>> Non transitive scope can be added without any language extension.
>>
>> In order for it to work, the holes in the language that enabled escapes had to
>> be plugged. That's what this proposal does - plug the holes.
>>
>
> The holes covered by a non transitive scope are already undefined behavior.

Right, but since the current compiler cannot detect them, it is a hole.


>>> ref counting is just one form of ownership.
> > My point is if that works with this proposal, then the other forms can work, too.
> No, unless we wrap every single indirection (pointer, slice, classes, delegates)
> into a wrapper. That is not going to fly very far.

The beauty of GC is that you don't have to do any of this. To have safe other methods of allocation, there has to be annotation everywhere or use a wrapped type in the same places.