On Tue, 23 Apr 2013 14:01:14 -0400, Diggory <diggsey@googlemail.com> wrote:

>
>> So no more returning ref?  Because if you allow returning ref, you lose any notion of safety, unless you plan on changing the entire compilation model?
>
> The rules from DIP25/35 show how you can return refs while still maintaining safety.

Those rules disallow the following valid code:

struct S
{
    int x;
    ref S opOpAssign(string op : "+")(ref S other) { x += other.x; return this;}
}

ref S add5(ref S s)
{
    auto o = S(5);
    return s += o;
}

void main()
{
    auto s = S(5);
    S s2 = add5(s);
}

Because opOpAssign takes two refs to S, and in add5, we bind one parameter to a local, we cannot return the result, even though it's perfectly safe and valid.

The point is simply that there exist valid and safe cases which will be disallowed by these rules.  While maybe it's OK in your mind to restrict the above code from validity, it's currently valid and compiling code.  You will be breaking existing code for pretty much no reason.

-Steve

On 24 April 2013 03:33, Andrei Alexandrescu <SeeWebsiteForEmail@erdani.org>wrote:

> On 4/23/13 12:04 PM, Manu wrote:
>
>> On 24 April 2013 00:24, Andrei Alexandrescu
>> <SeeWebsiteForEmail@erdani.org <mailto:SeeWebsiteForEmail@**erdani.org<SeeWebsiteForEmail@erdani.org>
>> >>
>>
>> wrote:
>>
>>
>>         The very point of this DIP is to not create
>>         syntax-driven features, instead better define existing ones that
>>         make
>>         sense on their own so they can be used for same purpose.
>>
>>
>>     It's a new feature, no two ways about it. It overlaps ref and auto
>>     ref without any palpable benefit and defines yet another way to
>>     achieve the same thing as auto ref. On this ground alone the
>>     proposal has a large problem.
>>
>>
>> How does it overlap ref? It simply justifies the argument with an extra constraint and isn't tied to 'ref' at all, it's just useful in conjunction.
>>
>
> The best setup would be:
>
> 1. To take lvalues by reference, write "ref".
>
> 2. To take lvalues and rvalues by reference, write "auto ref".
>

That's not a good setup at all. It still doesn't make sense.
There's nothing 'automatic' about it, I've specified ref, it is ref,
there's no other choice.
And it relies on a major breaking change to ref, which restricts the
functionality of ref by default.

Everything else is superfluous and puts the burden of justification on the
> proposer. With DIP36, the setup would be:
>
> 1. To take lvalues by reference, write "ref".
>
> 2. To take lvalues and rvalues by reference:
>
> 2.1. Is it a template? Then write "auto ref".
>
> 2.2. Is it a non-template? Then write "scope ref".


Stop talking about r-values, rather, consider safety of passing
temporaries. This problem has nothing to do with r-values, this is equally
unsafe:
  void f(ref int x);
  int x;
  f(x);

It's precisely the same problem, and should be fixed with the same solution.

I would rewrite your list as such:

1. 'un-safely' pass a value (may not be a local), write 'ref'
2. safely pass a value (may be a local), write 'scope ref'
3. if you are concerned with templates do you:
3.a. want unsafe auto-ref, type 'auto ref' (only non-locals would generate
'ref')
3.b. want safe auto-ref, type 'scope auto ref'

The automatic selection of ref for templates is fairly unrelated to this issue.

 I can't agree that it overlaps auto-ref at all. They're fundamentally
>> different concepts. auto-ref is a template concept; it selects the
>> ref-ness based on the received arg. 'auto ref', ie, 'automatic
>> ref-ness'. It makes no sense on a non-template situation.
>> I'm still completely amazed that the very reason this DIP makes perfect
>> sense to me(/us) is the same reason you have a problem with it.
>>
>
> I don't know how to respond to this. To me is it painfully obvious DIP 36 is poor language design and fails to solve a variety of issues, such as clarifying lifetime of temporaries, safety, and returning ref from functions.
>

Lifetime of temporaries is the most basic of principles. A local lives the
life of the function in which it is defined.
Safety is the whole point, and intrinsic to the proposal; safety by
explicit specification, thus the programmer retains the option.

If a function receives 'scope ref', and wants to return it, I think it should also return 'scope ref' (I guess this wasn't defined in the DIP), otherwise it would be considered an escape.

             2. The proposal is sketchy and does not give many details,
>>             such as the
>>             lifetime of temporaries bound to scope ref objects.
>>
>>
>>         It can't because lifetime of temporaries is not defined in D at
>>         all and
>>         suck stuff needs to be consistent. It is not really different
>> from a
>>         lifetime of struct literal temporary used for pass-by-value.
>>
>>
>>     A proposal aimed at binding rvalues to references must address
>>     lifetime of temporaries as a central concern.
>>
>>
>> It's not an r-value, it's a standard stack-allocated temporary. It's
>> lifetime is identical to any other local.
>> The reason it's not detailed in the proposal is because it adds no such
>> new feature, and makes no changes. The lifetime of a local is well
>> understood.
>>
>
> Currently rvalues are destroyed immediately after the call they are passed into. DIP 36 would need to change that, but fails to specify it.
>

Again, I think it was presumed (I can't conceive any other approach), and certainly Kenji read it that way, because his code appears to do just that.

             3. The relationship with auto ref is insufficiently
>>             described, e.g.
>>             there should be clarification on why auto ref cannot be
>>             improved to
>>             fulfill the desired role.
>>
>>
>>         auto ref is a template-world entity. If by "improved" you mean
>>         "completely reworked" than sure, I can add this rationale. Will
>>         do today.
>>
>>
>>     I think we should focus on
>>     http://d.puremagic.com/issues/**__show_bug.cgi?id=9238<http://d.puremagic.com/issues/__show_bug.cgi?id=9238>
>>
>>     <http://d.puremagic.com/**issues/show_bug.cgi?id=9238<http://d.puremagic.com/issues/show_bug.cgi?id=9238>>
>> and on making
>>     ref safe.
>>
>>
>> I don't believe it's possible to make ref safe. Can you suggest any vision for this?
>>
>
> http://d.puremagic.com/issues/**show_bug.cgi?id=9238<http://d.puremagic.com/issues/show_bug.cgi?id=9238>
>
>
>  It's unsafe by definition... you are passing a pointer of unknown origin
>> to a function that could do anything with that pointer.
>> Hence 'scope ref', which appropriately restricts what the callee is able
>> to do with it.
>>
>
> Our intent is to make "ref" always scoped and reserve non-scoped uses to pointers. We consider this good language design: we have unrestricted pointers for code that doesn't care much about safety, and we have "ref" which is almost as powerful but sacrifices a teeny bit of that power for the sake of guaranteed safety. Safety is guaranteed by making sure "ref" is always scoped (references can be passed down but never escape their bound value).
>

That's a massive breaking change...

[response in other thread, we need to stop repeating in 2 threads I think]

On 24 April 2013 04:01, Andrei Alexandrescu <SeeWebsiteForEmail@erdani.org>wrote:

> On 4/23/13 1:57 PM, Steven Schveighoffer wrote:
>
>> On Tue, 23 Apr 2013 13:33:31 -0400, Andrei Alexandrescu <SeeWebsiteForEmail@erdani.org**> wrote:
>>
>>  Our intent is to make "ref" always scoped and reserve non-scoped uses
>>> to pointers.
>>>
>>
>> So no more returning ref?
>>
>
> If we can't return ref, ref has failed. (This is what makes the entire
> thing difficult btw.)


I think the key that's not in this DIP is that it should also return 'scope
ref' to maintain the safety.
This brings the proposal in-line with your plans, except the safety is
explicit, and the option is available to the programmer.

Making 'safe' ref the default is a major breaking change.

On 4/23/13 2:29 PM, Manu wrote:
> On 24 April 2013 03:33, Andrei Alexandrescu
> <SeeWebsiteForEmail@erdani.org <mailto:SeeWebsiteForEmail@erdani.org>>
> wrote:
>     The best setup would be:
>
>     1. To take lvalues by reference, write "ref".
>
>     2. To take lvalues and rvalues by reference, write "auto ref".
>
>
> That's not a good setup at all. It still doesn't make sense.
> There's nothing 'automatic' about it,

There is - it means ref is inferred.

> I've specified ref, it is ref,
> there's no other choice.

Well "auto ref" means "I don't care whether this is ref or not".

> And it relies on a major breaking change to ref, which restricts the
> functionality of ref by default.

This is a confusion. There's no reliance on any change to ref. The breaking changes are related to adding safety. If safety weren't a concern, a lot of things would be simple.

> Stop talking about r-values, rather, consider safety of passing
> temporaries. This problem has nothing to do with r-values, this is
> equally unsafe:
>    void f(ref int x);
>    int x;
>    f(x);
>
> It's precisely the same problem, and should be fixed with the same solution.

We should make the code above safe as is.

> I would rewrite your list as such:
>
> 1. 'un-safely' pass a value (may not be a local), write 'ref'
> 2. safely pass a value (may be a local), write 'scope ref'
> 3. if you are concerned with templates do you:
> 3.a. want unsafe auto-ref, type 'auto ref' (only non-locals would
> generate 'ref')
> 3.b. want safe auto-ref, type 'scope auto ref'

This a terrible, terrible list. I'd be ashamed to have to explain this with a straight face.

>     I don't know how to respond to this. To me is it painfully obvious
>     DIP 36 is poor language design and fails to solve a variety of
>     issues, such as clarifying lifetime of temporaries, safety, and
>     returning ref from functions.
>
>
> Lifetime of temporaries is the most basic of principles. A local lives
> the life of the function in which it is defined.
> Safety is the whole point, and intrinsic to the proposal; safety by
> explicit specification, thus the programmer retains the option.

The missing point here is inflicting more complexity on the user.


Andrei

On 4/23/13 2:32 PM, Manu wrote:
> On 24 April 2013 04:01, Andrei Alexandrescu
> <SeeWebsiteForEmail@erdani.org <mailto:SeeWebsiteForEmail@erdani.org>>
> wrote:
>
>     On 4/23/13 1:57 PM, Steven Schveighoffer wrote:
>
>         On Tue, 23 Apr 2013 13:33:31 -0400, Andrei Alexandrescu
>         <SeeWebsiteForEmail@erdani.org
>         <mailto:SeeWebsiteForEmail@erdani.org>__> wrote:
>
>             Our intent is to make "ref" always scoped and reserve
>             non-scoped uses
>             to pointers.
>
>
>         So no more returning ref?
>
>
>     If we can't return ref, ref has failed. (This is what makes the
>     entire thing difficult btw.)
>
>
> I think the key that's not in this DIP is that it should also return
> 'scope ref' to maintain the safety.
> This brings the proposal in-line with your plans, except the safety is
> explicit, and the option is available to the programmer.

We prefer to simplify. Use ref safely for scoped pass-down of resources, use pointers for unrestricted escapes. What you see as "offering an option" I see as "adding a burden".

> Making 'safe' ref the default is a major breaking change.

How do you assess the size of the breakage?


Andrei

In that example you could make opOpAssign take a "const ref" and then the compiler could see that it could not return that parameter, but this is slightly hacky and I can see how there may be obscure corner cases where the rules are not sufficient.

Actually this could also be solved by using the (original) meaning of "scope" to mark parameters that are not going to be returned. (I still think both "ref" and "scope ref" should accept rvalues)

On 04/23/2013 07:33 PM, Andrei Alexandrescu wrote:
> On 4/23/13 12:04 PM, Manu wrote:
>> On 24 April 2013 00:24, Andrei Alexandrescu
>> <SeeWebsiteForEmail@erdani.org <mailto:SeeWebsiteForEmail@erdani.org>>
>> wrote:
>>
>>
>>         The very point of this DIP is to not create
>>         syntax-driven features, instead better define existing ones that
>>         make
>>         sense on their own so they can be used for same purpose.
>>
>>
>>     It's a new feature, no two ways about it. It overlaps ref and auto
>>     ref without any palpable benefit and defines yet another way to
>>     achieve the same thing as auto ref. On this ground alone the
>>     proposal has a large problem.
>>
>>
>> How does it overlap ref? It simply justifies the argument with an extra
>> constraint and isn't tied to 'ref' at all, it's just useful in
>> conjunction.
>
> The best setup would be:
>
> 1. To take lvalues by reference, write "ref".
>
> 2. To take lvalues and rvalues by reference, write "auto ref".
>
> Everything else is superfluous and puts the burden of justification on
> the proposer. With DIP36, the setup would be:
>
> 1. To take lvalues by reference, write "ref".
>
> 2. To take lvalues and rvalues by reference:
>
> 2.1. Is it a template? Then write "auto ref".
>

or "scope ref", to be fair. scope ref on a template would behave differently than auto ref on a template.

> 2.2. Is it a non-template? Then write "scope ref".
>
>> I can't agree that it overlaps auto-ref at all. They're fundamentally
>> different concepts. auto-ref is a template concept; it selects the
>> ref-ness based on the received arg. 'auto ref', ie, 'automatic
>> ref-ness'. It makes no sense on a non-template situation.
>> I'm still completely amazed that the very reason this DIP makes perfect
>> sense to me(/us) is the same reason you have a problem with it.
>
> I don't know how to respond to this. To me is it painfully obvious DIP
> 36 is poor language design and fails to solve a variety of issues, such
> as clarifying lifetime of temporaries, safety, and returning ref from
> functions.
>

+1.

> ...
>
> Our intent is to make "ref" always scoped and reserve non-scoped uses to
> pointers. We consider this good language design: we have unrestricted
> pointers for code that doesn't care much about safety, and we have "ref"
> which is almost as powerful but sacrifices a teeny bit of that power for
> the sake of guaranteed safety. Safety is guaranteed by making sure "ref"
> is always scoped (references can be passed down but never escape their
> bound value).
> ...

Safety should be guaranteed in @safe code. There is no point in conservatively disallowing perfectly fine @system code on the basis of safety concerns, unless, of course, the type system is advanced enough to prove safe all relevant use cases.

On Tue, 23 Apr 2013 15:40:35 -0400, Timon Gehr <timon.gehr@gmx.ch> wrote:


> Safety should be guaranteed in @safe code. There is no point in conservatively disallowing perfectly fine @system code on the basis of safety concerns, unless, of course, the type system is advanced enough to prove safe all relevant use cases.

I agree with this.  If @safe code is all that is affected, we will be well off.  We can also disallow known bad cases in non-@safe code.

-Steve

On 04/23/2013 09:51 PM, Steven Schveighoffer wrote:
> On Tue, 23 Apr 2013 15:40:35 -0400, Timon Gehr <timon.gehr@gmx.ch> wrote:
>
>
>> Safety should be guaranteed in @safe code. There is no point in
>> conservatively disallowing perfectly fine @system code on the basis of
>> safety concerns, unless, of course, the type system is advanced enough
>> to prove safe all relevant use cases.
>
> I agree with this.  If @safe code is all that is affected, we will be
> well off.  We can also disallow known bad cases in non-@safe code.
>
> -Steve

Another possibility I consider fine is to implement what Andrei suggests and then allow @system code to cast around the conservative escape checks using something like eg. cast(scope).


struct CodeGen{
    struct Label{
        CodeGen* outer;
    }
    Label makeLabel(){ return Label(&this); } // error
}

struct CodeGen{
    struct Label{
        CodeGen* outer;
    }
    Label makeLabel(){ return Label(cast(scope)&this); } // ok
}

The best way to fix this would of course be to introduce lifetimes as an explicit type system feature. (Though if I was allowed to add that kind of complexity to the type system, I'd do many things very differently.)

First of all, we should define words for the discussion.

1. Let's name current `auto ref` "parameterized-ref", which works only with
template functions and the `auto ref` parameter is instantiated to both
ref/non-ref by the actual function argument.
2. Let's call it "rvalue-ref(erence)", which currently discussed, that can
receive both lvalues and rvalues by one function body.

2013/4/24 Andrei Alexandrescu <SeeWebsiteForEmail@erdani.org>

>
> The best setup would be:
>
> 1. To take lvalues by reference, write "ref".
>
> 2. To take lvalues and rvalues by reference, write "auto ref".
>
> Everything else is superfluous and puts the burden of justification on the proposer. With DIP36, the setup would be:
>
> 1. To take lvalues by reference, write "ref".
>
> 2. To take lvalues and rvalues by reference:
>
> 2.1. Is it a template? Then write "auto ref".
>
> 2.2. Is it a non-template? Then write "scope ref".


The difference between 2.1 and 2.2 is not superfluous. Because:

1. "parameterized-ref" is already there with the syntax `auto ref`. Removing it introduces breaking language change.

2. If we remove "parameterized-ref" from D, we will lose the ability to write certain kind of template functions. For example:

    auto ref forward(T)(auto ref T arg) { return arg; }

The parameter `arg` is instantiated with both ref/non-ref, and forward `function` returns also both ref/non-ref value. In such case, you cannot replace "parameterized-ref" to "rvalue-ref".

These are why DIP36 is necessary. Repeatedly, I think removing current `auto ref` is not good.


> Currently rvalues are destroyed immediately after the call they are passed into. DIP 36 would need to change that, but fails to specify it.
>

DIP36 does not change it.


> Our intent is to make "ref" always scoped and reserve non-scoped uses to pointers. We consider this good language design: we have unrestricted pointers for code that doesn't care much about safety, and we have "ref" which is almost as powerful but sacrifices a teeny bit of that power for the sake of guaranteed safety. Safety is guaranteed by making sure "ref" is always scoped (references can be passed down but never escape their bound value).
>

I'm basically agreeing with the DIP25. Making `ref` more safety is possible and should.

I can guess that you are intended applying DIP25 to "rvalue-ref" concept. It is sane design, but DIP36 also accepts such applying DIP25. There is no issue.


>
>> Andrei
>

Kenji Hara