On 5/4/2013 6:44 PM, Jonathan M Davis wrote:
> On Saturday, May 04, 2013 20:37:36 Andrei Alexandrescu wrote:
>> On 5/4/13 7:31 PM, Walter Bright wrote:
>>> On 5/4/2013 3:51 PM, w0rp wrote:
>>>> Does all of this also mean that a
>>>> function with a ref parameter will automagically work with r-values?
>>>
>>> Yes.
>>
>> This is new to me. My understanding is that the discussed design
>> addresses safety, and leaves the rvalue discussion for a future iteration.
>
> That is definitely where things were when we ended the discussion on Wednesday
> night. Walter favored making ref accept rvalues, but we never agreed on that.
> Manu was still in favor of scop ref (and David Nadlinger agreed with him
> IIRC), and you and I were arguing for auto ref to designate that a function
> accepts rvalues. We all agreed on the bounds check solution for @safety, but
> we explicitly tabled the discussion about accepting rvalues, because it was
> getting late, and we'd already been discussing it / arguing about it for quite
> some time. So, unless further discussion occurred after that which I missed,
> there is still no agreement on how to handle having a parameter accept both
> lvalues and rvalues by ref.

That wasn't my understanding. I thought we agreed that since rvalues would be copied to locals, and then the issue was one of escaping local references.

We did explicitly defer discussion about what happens with "nop" rvalue conversions.

On 5/4/2013 5:42 PM, bearophile wrote:
> Andrei Alexandrescu:
>
>> http://static.rust-lang.org/doc/tutorial-borrowed-ptr.html
>
> The management of pointers is one of the most refined parts of the Rust design.
> It offers safety, allows per-thread GCs, and more. It's powerful but it also
> adds some complexity to the language.

Years ago, Bartosz proposed an ownership system for pointers. While sound, it was rather complicated.

I don't think a complex system is going to gain wide adoption.

On 5/4/13 10:04 PM, Walter Bright wrote:
> That wasn't my understanding. I thought we agreed that since rvalues
> would be copied to locals, and then the issue was one of escaping local
> references.

The short answer is no.

Andrei

On 5/4/2013 5:28 PM, Andrei Alexandrescu wrote:
> Couple amendments:
>
> On 5/4/13 2:33 PM, Walter Bright wrote:
>> Case B:
>> ref T foob(ref U u) { return u.t; } // note that T is derivable from U
>> ref U bar() { T t; return foob(t); }
>
> That's not derivable, it's embedded: type U transitively has a member of type T.
>
> Same case applies to statically-sized arrays:
>
> ref T foob(ref T[42] u) { return u[13]; }
> ref T[42] bar() { T[42] t; return foob(t); }
>
> Here the notion that a statically-sized arrays behaves much like a struct is
> applicable. This case probably deserves notice too:
>
> ref T fooa(ref T t) { return t; }
> ref T bar() { T[42] t; return fooa(t[13]); }

Yes.

>> 1. Always involves a return statement.
>
> Except if pointers are used, which leaves the question of what we do when people
> take the address of refs returned by functions.

Ref is a restricted form of pointer, the whole point of them is so we can do more reasoning about them. If we throw into the mix allowing converting them to pointers in safe code, everything falls apart.

>> 2. The return type must always be the type of the stack variable or a
>> type type derived from a stack variable's type via safe casting or
>> subtyping.
>
> That's not subtyping, it's transitive member access. Here transitive goes
> through members but not through indirections. Not sure how to call that to not
> make it confusing.

I know what you mean. I don't know what word to use, either.


>> 3. Returning rvalues is the same issue, as rvalues are always turned
>> into local stack temporaries.
>
> The complicating factor here is that lvalues have well-understood lifetimes
> whereas rvalues are more subtle and opened to subtleties and interpretations. I
> think right now D destroys temporaries too early.

Considering that we only have to deal with return statement expressions here, where the lifetime of those temporaries would be restricted to those expressions regardless, that shouldn't be an issue.


>> 4. Whether a function returns a ref derived from a parameter or not is
>> not reflected in the function signature.
>
> Yes! That's why any static solution is either conservative or complicates the
> language.
>
>> 5. Always involves passing a local by ref to a function that returns by
>> ref, and that function gets called in a return statement.
>
> There's also the case of e.g. "return *p;"

Again, allowing ref <=> pointer conversions makes it impractical to reason about refs.

> and "return a[13];".

If 'a' is a local array allocated on the stack, this is trivially disallowed, just like:

    S s;
    return s.t;

would be.

On Saturday, May 04, 2013 19:04:21 Walter Bright wrote:
> On 5/4/2013 6:44 PM, Jonathan M Davis wrote:
> > On Saturday, May 04, 2013 20:37:36 Andrei Alexandrescu wrote:
> >> On 5/4/13 7:31 PM, Walter Bright wrote:
> >>> On 5/4/2013 3:51 PM, w0rp wrote:
> >>>> Does all of this also mean that a
> >>>> function with a ref parameter will automagically work with r-values?
> >>> 
> >>> Yes.
> >> 
> >> This is new to me. My understanding is that the discussed design addresses safety, and leaves the rvalue discussion for a future iteration.
> > 
> > That is definitely where things were when we ended the discussion on Wednesday night. Walter favored making ref accept rvalues, but we never agreed on that. Manu was still in favor of scop ref (and David Nadlinger agreed with him IIRC), and you and I were arguing for auto ref to designate that a function accepts rvalues. We all agreed on the bounds check solution for @safety, but we explicitly tabled the discussion about accepting rvalues, because it was getting late, and we'd already been discussing it / arguing about it for quite some time. So, unless further discussion occurred after that which I missed, there is still no agreement on how to handle having a parameter accept both lvalues and rvalues by ref.
> 
> That wasn't my understanding. I thought we agreed that since rvalues would be copied to locals, and then the issue was one of escaping local references.

The @safety issue is one of escaping local references, but Andrei and I were arguing that it's a maintenance issue for ref to always accept rvalues. If ref does not accept rvalues, then you can look at a function signature like

auto foo(ref int i);

and know that it's intended to alter its argument. However, if ref accepted rvalues, you couldn't know that anymore. People would be using ref all over the place for the efficiency gain - just like they do with const ref in C++ - so the fact that a parameter was ref would mean nothing about how it was used. So, you could see code like

[5, 6, 7].popFrontN(5);

and not know that it was effectively a no-op (in this case, it's fairly obvious, but if you're not already familiar with the function, it generally wouldn't be).

However, if we had an attribute which explicitly designated that a function accepted both rvalues and lvalues (which is what auto ref was originally supposed to do as Andrei proposed it), then if you saw

auto foo(ref int i);
auto bar(auto ref int i);

then you could be reasonably certain that foo was intending to alter its arguments and bar was not. And if you want the full guarantee that bar _can't_ alter its arguments, you use const

auto bar(auto ref const int i);

But given how restrictive D's const is, we can't really go with C++'s solution of const& for that. However, auto ref is then very similar to C++'s const&, except that it doesn't require const to do it (and it's @safe thanks to the new @safety solution for ref).

So, the primary difference between ref and auto ref would then be simply that auto ref accepted rvalues and ref wouldn't (though, the difference would be somewhat greater with templates, since in that case, it generates different templates for lvalues and rvalues in order to accept both, whereas the non- templated version would effectively create a local variable to assign the rvalue to so that it could be passed to the function as an lvalue). But the distinction between ref and auto ref is very important when trying to understand what code does and therefore will have a definite impact on how maintainable code is.

> We did explicitly defer discussion about what happens with "nop" rvalue conversions.

I'm not sure what you mean by nop rvalue conversions, at least not by name.

- Jonathan M Davis

On 5/4/2013 7:16 PM, Andrei Alexandrescu wrote:
> On 5/4/13 10:04 PM, Walter Bright wrote:
>> That wasn't my understanding. I thought we agreed that since rvalues
>> would be copied to locals, and then the issue was one of escaping local
>> references.
>
> The short answer is no.

Please explain your understanding of what we agreed on.

On Saturday, May 04, 2013 19:07:25 Walter Bright wrote:
> On 5/4/2013 5:42 PM, bearophile wrote:
> > Andrei Alexandrescu:
> >> http://static.rust-lang.org/doc/tutorial-borrowed-ptr.html
> > 
> > The management of pointers is one of the most refined parts of the Rust design. It offers safety, allows per-thread GCs, and more. It's powerful but it also adds some complexity to the language.
> 
> Years ago, Bartosz proposed an ownership system for pointers. While sound, it was rather complicated.
> 
> I don't think a complex system is going to gain wide adoption.

The trick is balancing it so that it's powerful enough and yet not too complicated to be useable by normal programmers. I think that we're okay, but I also think that we're pushing it as it is. Going with Bartosz proposal would almost certainly have been too much.

As it is, we arguably didn't choose the best defaults with the attributes that we have (e.g. @system is the default instead of @safe, and impure is the default instead of pure). The result is that we have to use a lot of annotations if we want to properly take advantage of the various language features, whereas ideally, having to use annotations for stuff like @safety or purity would be the exception. Don was complaining that one reason that moving to D2 at Sociomantic looks unappealing in spite of the benefits is the fact that they're going to have to add so many extra annotations to their code.

- Jonathan M Davis

On 5/4/2013 7:30 PM, Jonathan M Davis wrote:
> On Saturday, May 04, 2013 19:04:21 Walter Bright wrote:
>> On 5/4/2013 6:44 PM, Jonathan M Davis wrote:
>>> On Saturday, May 04, 2013 20:37:36 Andrei Alexandrescu wrote:
>>>> On 5/4/13 7:31 PM, Walter Bright wrote:
>>>>> On 5/4/2013 3:51 PM, w0rp wrote:
>>>>>> Does all of this also mean that a
>>>>>> function with a ref parameter will automagically work with r-values?
>>>>>
>>>>> Yes.
>>>>
>>>> This is new to me. My understanding is that the discussed design
>>>> addresses safety, and leaves the rvalue discussion for a future
>>>> iteration.
>>>
>>> That is definitely where things were when we ended the discussion on
>>> Wednesday night. Walter favored making ref accept rvalues, but we never
>>> agreed on that. Manu was still in favor of scop ref (and David Nadlinger
>>> agreed with him IIRC), and you and I were arguing for auto ref to
>>> designate that a function accepts rvalues. We all agreed on the bounds
>>> check solution for @safety, but we explicitly tabled the discussion about
>>> accepting rvalues, because it was getting late, and we'd already been
>>> discussing it / arguing about it for quite some time. So, unless further
>>> discussion occurred after that which I missed, there is still no
>>> agreement on how to handle having a parameter accept both lvalues and
>>> rvalues by ref.
>>
>> That wasn't my understanding. I thought we agreed that since rvalues would
>> be copied to locals, and then the issue was one of escaping local
>> references.
>
> The @safety issue is one of escaping local references, but Andrei and I were
> arguing that it's a maintenance issue for ref to always accept rvalues. If ref
> does not accept rvalues, then you can look at a function signature like
>
> auto foo(ref int i);
>
> and know that it's intended to alter its argument. However, if ref accepted
> rvalues, you couldn't know that anymore. People would be using ref all over
> the place for the efficiency gain - just like they do with const ref in C++ - so
> the fact that a parameter was ref would mean nothing about how it was used.
> So, you could see code like
>
> [5, 6, 7].popFrontN(5);
>
> and not know that it was effectively a no-op (in this case, it's fairly
> obvious, but if you're not already familiar with the function, it generally
> wouldn't be).
>
> However, if we had an attribute which explicitly designated that a function
> accepted both rvalues and lvalues (which is what auto ref was originally
> supposed to do as Andrei proposed it), then if you saw
>
> auto foo(ref int i);
> auto bar(auto ref int i);
>
> then you could be reasonably certain that foo was intending to alter its
> arguments and bar was not. And if you want the full guarantee that bar _can't_
> alter its arguments, you use const
>
> auto bar(auto ref const int i);
>
> But given how restrictive D's const is, we can't really go with C++'s solution
> of const& for that. However, auto ref is then very similar to C++'s const&,
> except that it doesn't require const to do it (and it's @safe thanks to the
> new @safety solution for ref).
>
> So, the primary difference between ref and auto ref would then be simply that
> auto ref accepted rvalues and ref wouldn't (though, the difference would be
> somewhat greater with templates, since in that case, it generates different
> templates for lvalues and rvalues in order to accept both, whereas the non-
> templated version would effectively create a local variable to assign the
> rvalue to so that it could be passed to the function as an lvalue). But the
> distinction between ref and auto ref is very important when trying to
> understand what code does and therefore will have a definite impact on how
> maintainable code is.
>
>> We did explicitly defer discussion about what happens with "nop" rvalue
>> conversions.
>
> I'm not sure what you mean by nop rvalue conversions, at least not by name.

I meant exactly what you said: "and not know that it was effectively a no-op".

On Saturday, May 04, 2013 20:12:24 Walter Bright wrote:
> > I'm not sure what you mean by nop rvalue conversions, at least not by name.
> 
> I meant exactly what you said: "and not know that it was effectively a no-op".

Oh, okay. LOL. I was thinking you meant something lower level like than that, and it didn't click. Yeah, distinguishing between functions that are meant to mutate their arguments and those that just want to pass them efficiently is the core issue with naked ref accepting rvalues, and we didn't come to an agreement on that.

- Jonathan M Davis

On Sunday, 5 May 2013 at 02:36:45 UTC, Jonathan M Davis wrote:
> Don was complaining that one reason that moving
> to D2 at Sociomantic looks unappealing in spite of the benefits is the fact
> that they're going to have to add so many extra annotations to their code.

When did he mention that? If I had noticed, I would have been interested in a closer rationale, as D2's extra annotations are pretty much opt-in only, even more so if you are using your own library anyway.

David