On 24 April 2013 00:30, Andrei Alexandrescu <SeeWebsiteForEmail@erdani.org>wrote:

> On 4/23/13 10:05 AM, Manu wrote:
>
>> I can't see the fault in DIP36's reasoning. It just makes sense. Why is everyone so against it? I'm yet to understand a reason...
>>
>
> 1. It defines a new language feature instead of improving the existing ones. At this point in the development of the language, our preference should be putting the existing features in good order.
>

I see it in exactly the opposite way.
This does put an existing feature in good order, ie, scope, which is
defined but barely implemented, and might as well have been invented for
this purpose as far as I can tell from what little information is available
about it.

2. The proposal is sketchy and does not give many details, such as the
> lifetime of temporaries bound to scope ref objects.
>

Is that the only detail missing? An r-value passed this way produces a temp, which is a stack variable. It's life is identical to any other stack variable, ie, it lives for the life of the function where it appears.

3. The relationship with auto ref is insufficiently described, e.g. there
> should be clarification on why auto ref cannot be improved to fulfill the desired role.
>

auto-ref is a template concept (necessary because ref is not part of the type like in c++). It doesn't make sense to me at all being forced into this concept. I see people saying it over and over, but I just can't see it. How does 'auto' apply conceptually? It's a template concept by definition. It feels like auto-ref is deliberately contriving a concept to fit a problem, not DIP36.

4. Above all this is a new language feature and again we want to resort to
> adding new feature only if it is clear that the existing features are insufficient and cannot be made sufficient.


Again, I can't see how this is a new feature at all. It's precisely what
you say; making an existing feature (scope) that already exists and is
defined actually work, and elegantly solve the problem.
auto-ref on the other hand IS a new feature (in this context), and it also
makes no sense if you ask me. It's a template concept which is not
applicable here.

In particular we are much more inclined to impart real, demonstrable safety
> to "ref"


ref is unsafe by definition. I don't believe this is possible without some
further justification.
DIP36 however creates a situation where it's known that passing a temp is
actually safe.

and to make "auto ref" work as a reference that can bind to rvalues as well
> as lvalues.


What does it mean to make a reference bind to r-values aswell as l-values?
Lots of people keep saying this too, but it doesn't really make sense to me
either.
No reference can bind to r-values, r-values can not be addressed. It's
really a temp copy of said r-value that we're dealing with, which is an
l-value, ie, a local with a lifetime that's unsuitable for passing by
non-scope-ref.
scope-ref would promise that it won't escape the callee, and thus is safe
to pass a temp.

ref is fundamentally broken in D right now. DIP36 creates a situation where
it could be fixed. I would personally take DIP36 one step further, and ban
all local's from being passed to non-scope ref.
Yes, a breaking change, but you could argue that any code that passes a
stack variable to any ref arg is already broken. But this can be addressed
in a future DIP.


...perhaps I'm missing something fundamental in DIP36, or about 'auto ref'? I can't understand why there seem to be 2 polarised parties on this issue, which appear to see the problem completely differently, and can't visualise the counter perspective at all.

> That's on the mark. We need to make auto ref work with non-templates as well as templates, safely. The implementation mechanics may be different. Lifetime of temporaries is an important detail. I discuss such in http://d.puremagic.com/issues/show_bug.cgi?id=9238 but I never had the time to finish it.
>
> Andrei

> 3. The relationship with auto ref is insufficiently described, e.g. there should be clarification on why auto ref cannot be improved to fulfill the desired role.

I and probably many other have thought that auto ref cannot work for non-templates because that would mean, that auto ref would work differently for templates and non-templates. That was the main reason to search for an alternative.
But have you any ideas how auto ref should do this for non-templates?
E.g. with creating temporary lvalues, doubled the function etc.? That would make it easier to support you.

On 24 April 2013 01:01, deadalnix <deadalnix@gmail.com> wrote:

> On Tuesday, 23 April 2013 at 14:38:12 UTC, Dicebot wrote:
>
>> On Tuesday, 23 April 2013 at 14:28:54 UTC, Andrei Alexandrescu wrote:
>>
>>> If it were about scope it would be very careful with lifetime of temporaries.
>>>
>>> Andrei
>>>
>>
>> Please explain. In its current form DIP36 only cares that passed temporary exists while called function is executed. I can't imagine any sane lifetime rules that would result in violating this.
>>
>
> This isn't enough for a DIP. You have to explicitly define the lifetime, as it is required to know what is safe and what isn't, to get deterministic destruction, or whatever.
>

"The r-value being passed is assigned to a stack allocated temporary, which
has a lifetime that is identical to any other local variable, ie, the
lifetime of the function in which it appears."
There, I defined it.

> No it is not the only difference. "scope ref" (as proposed in DIP35) is more restrictive in usage - can't take address of it, can't return it, can't implicitly cast it to normal ref. It is "scope" primarily and "rvalue ref solution" only secondarily.

With DIP25 these restrictions apply to "ref" as well (with the exception that you CAN return a "ref", but it turns out it would also be safe to return a "scope ref" in those circumstances).

> DIP25 imposes a number of code-breaking restrictions even in @system code, if DIP36 was in place, one could consider imposing the DIP25 restrictions only in SafeD.

Yes, the "addressof" part of DIP25 would cause problems, but the part to do with safety of "ref" will not. IMO we shouldn't start trying to add rvalue references until we've solved the safety of normal references. We should therefore split DIP25 into two parts:

- The part that guarantees safety of "ref" can be implemented right away (just make taking the address of a reference unsafe instead of adding "addressof"). There is simply no avoiding implementing something like this if you want safe references.

- The part to do with fixing syntactic ambiguities of &, adding "addressof" and making "ref" more restrictive can be a separate DIP. This is the one that would potentially break code.

If we do this first, the issue of rvalue references becomes a non-issue. Even without explicitly defining what the lifetime of an rvalue reference is it becomes safe to pass them as "ref":

- In DIP25 it is the caller who enforces the safety of "ref", the callee just has to follow the same simple rules.

- The caller will always know the lifetime of anything it passes in, so it is simple to ensure that it doesn't last beyond that.

> Furthermore if one day the compiler would be sufficiently smart to infer scope automatically, there still would be an important difference between 'ref' and 'scope ref'.

That's not what I'm saying, even with no inference on behalf of the compiler there is still no difference between "scope ref" and "ref".

On 4/23/13 10:38 AM, Dicebot wrote:
> On Tuesday, 23 April 2013 at 14:28:54 UTC, Andrei Alexandrescu wrote:
>> If it were about scope it would be very careful with lifetime of
>> temporaries.
>>
>> Andrei
>
> Please explain. In its current form DIP36 only cares that passed
> temporary exists while called function is executed. I can't imagine any
> sane lifetime rules that would result in violating this.

One matter to look into is returning scope ref from functions (which is useful).

Andrei

On 4/23/13 11:27 AM, Manu wrote:
> On 24 April 2013 00:30, Andrei Alexandrescu
> <SeeWebsiteForEmail@erdani.org <mailto:SeeWebsiteForEmail@erdani.org>>
> wrote:
>
>     On 4/23/13 10:05 AM, Manu wrote:
>
>         I can't see the fault in DIP36's reasoning. It just makes sense.
>         Why is
>         everyone so against it? I'm yet to understand a reason...
>
>
>     1. It defines a new language feature instead of improving the
>     existing ones. At this point in the development of the language, our
>     preference should be putting the existing features in good order.
>
>
> I see it in exactly the opposite way.
> This does put an existing feature in good order, ie, scope, which is
> defined but barely implemented, and might as well have been invented for
> this purpose as far as I can tell from what little information is
> available about it.

"scope" is a keyword, not a language feature. In case you are referring to scope variables, the feature "scope ref" has little to do with it.

>     2. The proposal is sketchy and does not give many details, such as
>     the lifetime of temporaries bound to scope ref objects.
>
>
> Is that the only detail missing?

Many details are missing. This is not a simple problem.

> An r-value passed this way produces a
> temp, which is a stack variable. It's life is identical to any other
> stack variable, ie, it lives for the life of the function where it appears.

That's a possibility, but it's a departure from current semantics and is not mentioned in the DIP.

>     3. The relationship with auto ref is insufficiently described, e.g.
>     there should be clarification on why auto ref cannot be improved to
>     fulfill the desired role.
>
>
> auto-ref is a template concept (necessary because ref is not part of the
> type like in c++). It doesn't make sense to me at all being forced into
> this concept. I see people saying it over and over, but I just can't see it.
> How does 'auto' apply conceptually? It's a template concept by
> definition. It feels like auto-ref is deliberately contriving a concept
> to fit a problem, not DIP36.

Well feeling a certain way is fine but it doesn't help rigorous definitions.

The "scope ref" feature is adding no value over what we have and need to get in good order. People would come and ask, "how can a function accept both rvalues and lvalues by reference?" and then experts will answer with, "well is that a template or a non-template? because the means are entirely different". "Why are they different?" And the experts will answer just like you: "By definition" - thus closing the circle.

>     4. Above all this is a new language feature and again we want to
>     resort to adding new feature only if it is clear that the existing
>     features are insufficient and cannot be made sufficient.
>
>
> Again, I can't see how this is a new feature at all. It's precisely what
> you say; making an existing feature (scope) that already exists and is
> defined actually work, and elegantly solve the problem.

This is a new feature. There is no "existing feature scope".

> auto-ref on the other hand IS a new feature (in this context), and it
> also makes no sense if you ask me. It's a template concept which is not
> applicable here.

It is a feature that has been implemented and works, just not in all cases.

>     In particular we are much more inclined to impart real, demonstrable
>     safety to "ref"
>
>
> ref is unsafe by definition.

We want to aim at making ref safe, thus making it useful as restricted pass-down pointers. For full possibilities, one should use pointers.

> I don't believe this is possible without
> some further justification.

The justification is that unsafe uses of ref are few and uninteresting (they can be replaced with pointers). It would be very powerful to be able to guarantee that safe code can use ref.

> DIP36 however creates a situation where it's known that passing a temp
> is actually safe.
>
>     and to make "auto ref" work as a reference that can bind to rvalues
>     as well as lvalues.
>
>
> What does it mean to make a reference bind to r-values aswell as
> l-values? Lots of people keep saying this too, but it doesn't really
> make sense to me either.

I don't understand the question as the answer is in it.

> No reference can bind to r-values, r-values can not be addressed.

But auto ref and scope ref do bind to r-values.

> It's
> really a temp copy of said r-value that we're dealing with, which is an
> l-value, ie, a local with a lifetime that's unsuitable for passing by
> non-scope-ref.
> scope-ref would promise that it won't escape the callee, and thus is
> safe to pass a temp.

Our aim is to have ref make that promise.

> ref is fundamentally broken in D right now. DIP36 creates a situation
> where it could be fixed.

A new feature is not a fix.

> I would personally take DIP36 one step further,
> and ban all local's from being passed to non-scope ref.
> Yes, a breaking change, but you could argue that any code that passes a
> stack variable to any ref arg is already broken. But this can be
> addressed in a future DIP.
>
>
> ...perhaps I'm missing something fundamental in DIP36, or about 'auto ref'?
> I can't understand why there seem to be 2 polarised parties on this
> issue, which appear to see the problem completely differently, and can't
> visualise the counter perspective at all.

DIP36 should be closed. We must focus on making ref safe and on making auto ref work with non-templates.


Andrei

On 4/23/13 11:31 AM, Namespace wrote:
>> That's on the mark. We need to make auto ref work with non-templates
>> as well as templates, safely. The implementation mechanics may be
>> different. Lifetime of temporaries is an important detail. I discuss
>> such in http://d.puremagic.com/issues/show_bug.cgi?id=9238 but I never
>> had the time to finish it.
>>
>> Andrei
>
>> 3. The relationship with auto ref is insufficiently described, e.g.
>> there should be clarification on why auto ref cannot be improved to
>> fulfill the desired role.
>
> I and probably many other have thought that auto ref cannot work for
> non-templates because that would mean, that auto ref would work
> differently for templates and non-templates. That was the main reason to
> search for an alternative.

I recall Walter has been the first to put this forward, but has been convinced otherwise. We can make auto ref work for non-templates, even if the implementation is different. The main issues are returning ref from functions and lifetime of temporaries, as discussed in http://d.puremagic.com/issues/show_bug.cgi?id=9238.

> But have you any ideas how auto ref should do this for non-templates?
> E.g. with creating temporary lvalues, doubled the function etc.? That
> would make it easier to support you.

There would be one implementation of the function. Binding rules, typechecking the function, and lifetime of temporaries may be changed.


Andrei

On Tuesday, 23 April 2013 at 17:15:14 UTC, Andrei Alexandrescu wrote:
> DIP36 should be closed. We must focus on making ref safe and on making auto ref work with non-templates.

It is marked as such already, since the very moment you have stated your opinion.

On 4/23/13 1:53 PM, Dicebot wrote:
> On Tuesday, 23 April 2013 at 17:15:14 UTC, Andrei Alexandrescu wrote:
>> DIP36 should be closed. We must focus on making ref safe and on making
>> auto ref work with non-templates.
>
> It is marked as such already, since the very moment you have stated your
> opinion.

I'd feel uncomfortable wielding this much power. I don't think this DIP was ready for a formal review (hence there should be no rejecting it yet), but there has been good discussion and experimental implementation as well. I suggest we keep it in draft format. If our attempts at working on ref/auto ref fail, we should revisit this.

Andrei

On 24 April 2013 03:15, Andrei Alexandrescu <SeeWebsiteForEmail@erdani.org>wrote:

> On 4/23/13 11:27 AM, Manu wrote:
>
>> On 24 April 2013 00:30, Andrei Alexandrescu
>> <SeeWebsiteForEmail@erdani.org <mailto:SeeWebsiteForEmail@**erdani.org<SeeWebsiteForEmail@erdani.org>
>> >>
>>
>> wrote:
>>
>>     On 4/23/13 10:05 AM, Manu wrote:
>>
>>         I can't see the fault in DIP36's reasoning. It just makes sense.
>>         Why is
>>         everyone so against it? I'm yet to understand a reason...
>>
>>
>>     1. It defines a new language feature instead of improving the
>>     existing ones. At this point in the development of the language, our
>>     preference should be putting the existing features in good order.
>>
>>
>> I see it in exactly the opposite way.
>> This does put an existing feature in good order, ie, scope, which is
>> defined but barely implemented, and might as well have been invented for
>> this purpose as far as I can tell from what little information is
>> available about it.
>>
>
> "scope" is a keyword, not a language feature. In case you are referring to scope variables, the feature "scope ref" has little to do with it.
>

How so? 'scope' simply promises that a variable may not escape its scope,
no?
I think it's important to recognise it as 'scope' + 'ref', the 2 don't have
any special meaning when put together, just the logical compound, which
allows for a safe situation for temporaries that wasn't previously
available.

     2. The proposal is sketchy and does not give many details, such as
>>     the lifetime of temporaries bound to scope ref objects.
>>
>>
>> Is that the only detail missing?
>>
>
> Many details are missing. This is not a simple problem.
>

So what are some others?

 An r-value passed this way produces a
>> temp, which is a stack variable. It's life is identical to any other
>>
>> stack variable, ie, it lives for the life of the function where it appears.
>>
>
> That's a possibility, but it's a departure from current semantics and is not mentioned in the DIP.
>

I think it's presumed in the DIP, and it's certainly how Kenji implemented
it.
What 'current' semantic is it a departure from? The one where passing a
literal produces a compile error? Certainly, that's the point.

 auto-ref on the other hand IS a new feature (in this context), and it
>> also makes no sense if you ask me. It's a template concept which is not applicable here.
>>
>
> It is a feature that has been implemented and works, just not in all cases.
>

This isn't a 'case'. It's a separate issue.
Safely passing a temp to a ref function arg, and whether a template
argument is automatically determined to be ref or not are barely related
problems.
I still can't see how auto-ref has any business in this context.

     In particular we are much more inclined to impart real, demonstrable
>>     safety to "ref"
>>
>>
>> ref is unsafe by definition.
>>
>
> We want to aim at making ref safe, thus making it useful as restricted pass-down pointers. For full possibilities, one should use pointers.


Okay, I'm good with that too, but how is that intended to work?
If the intent is to make ref escaping disallowed by default, that is a
major breaking change...
Can we start talking about virtual-by-default again while we're at it?

 I don't believe this is possible without
>> some further justification.
>>
>
> The justification is that unsafe uses of ref are few and uninteresting (they can be replaced with pointers). It would be very powerful to be able to guarantee that safe code can use ref.
>

Again, this sounds like a major breaking change.
Why is scope-ref inferior? It's more informative, and offers more
flexibility (ie, the option of ref with or without scope)

 DIP36 however creates a situation where it's known that passing a temp
>> is actually safe.
>>
>>     and to make "auto ref" work as a reference that can bind to rvalues
>>     as well as lvalues.
>>
>>
>> What does it mean to make a reference bind to r-values aswell as l-values? Lots of people keep saying this too, but it doesn't really make sense to me either.
>>
>
> I don't understand the question as the answer is in it.
>
>
>  No reference can bind to r-values, r-values can not be addressed.
>>
>
> But auto ref and scope ref do bind to r-values.
>
>
>  It's
>> really a temp copy of said r-value that we're dealing with, which is an
>> l-value, ie, a local with a lifetime that's unsuitable for passing by
>> non-scope-ref.
>> scope-ref would promise that it won't escape the callee, and thus is
>> safe to pass a temp.
>>
>
> Our aim is to have ref make that promise.
>
>
>  ref is fundamentally broken in D right now. DIP36 creates a situation
>> where it could be fixed.
>>
>
> A new feature is not a fix.


If scope is a new feature, then the keyword shouldn't compile and pretend
that it does stuff.
It's an incomplete/unimplemented feature, not a new one.
People are aware of it, they can write code that presumes it's present and
working. It compiles successfully.

 I would personally take DIP36 one step further,
>> and ban all local's from being passed to non-scope ref.
>> Yes, a breaking change, but you could argue that any code that passes a
>> stack variable to any ref arg is already broken. But this can be
>> addressed in a future DIP.
>>
>>
>> ...perhaps I'm missing something fundamental in DIP36, or about 'auto
>> ref'?
>> I can't understand why there seem to be 2 polarised parties on this
>> issue, which appear to see the problem completely differently, and can't
>> visualise the counter perspective at all.
>>
>
> DIP36 should be closed. We must focus on making ref safe and on making auto ref work with non-templates.


I'm fine with that, but it sounds like a massive breaking change.
However upon the presumption of this new goal, I don't see the relevance of
auto-ref anymore? Why continue to bring it up?
If ref is safe, nothing else is needed.