On 4/23/13 4:24 PM, Timon Gehr wrote:
> On 04/23/2013 07:15 PM, Andrei Alexandrescu wrote:
>> ...
>>
>> The justification is that unsafe uses of ref are few and uninteresting
>> (they can be replaced with pointers).
>
> I don't get this. Uses cannot be "unsafe". They are either correct or
> wrong.

Unfortunately the safe/unsafe and correct/wrong sets don't overlap perfectly.

>> It would be very powerful to be
>> able to guarantee that safe code can use ref.
>> ...
>
> It is not required to make ref guaranteed safe in @system code to
> guarantee this.

Yah, agreed. It would be nice to have ref offer guarantees to all code while also being powerful, but if we can't make that @safe is always a possibility.


Andrei

On 04/23/2013 10:44 PM, Andrei Alexandrescu wrote:
> On 4/23/13 4:24 PM, Timon Gehr wrote:
>> On 04/23/2013 07:15 PM, Andrei Alexandrescu wrote:
>>> ...
>>>
>>> The justification is that unsafe uses of ref are few and uninteresting
>>> (they can be replaced with pointers).
>>
>> I don't get this. Uses cannot be "unsafe". They are either correct or
>> wrong.
>
> Unfortunately the safe/unsafe and correct/wrong sets don't overlap
> perfectly.
>...

I now see what you mean.

(It is important to keep in mind that only the latter sets are an intrinsic property of the use. The first ones depend on the implemented solution. Hence, in order to be able to provide the above justification, an appropriate efficient safe/unsafe classification scheme must be devised first.)

2013/4/24 Manu <turkeyman@gmail.com>

> "The r-value being passed is assigned to a stack allocated temporary,
> which has a lifetime that is identical to any other local variable, ie, the
> lifetime of the function in which it appears."
> There, I defined it.
>

Good definition. If add more,
"getting address of "scope" parameter would be disallowed, at least in
@safe code, because it would be regarded as the escape of stack allocated
temporary."

Kenji Hara

On Wednesday, 24 April 2013 at 00:54:12 UTC, kenji hara wrote:
> 2013/4/24 Manu <turkeyman@gmail.com>
>
>> "The r-value being passed is assigned to a stack allocated temporary,
>> which has a lifetime that is identical to any other local variable, ie, the
>> lifetime of the function in which it appears."
>> There, I defined it.
>>
>
> Good definition. If add more,
> "getting address of "scope" parameter would be disallowed, at least in
> @safe code, because it would be regarded as the escape of stack allocated
> temporary."
>
> Kenji Hara

Why does the temporary need to exist any longer than the current statement? (the current lifetime of temporaries are the statement or expression). Surely any longer is just wasting stack space.

Temporaries which created on stack to pass an address to `scope ref` parameter would be destroyed after the "current statement/expression".

T rvalue();

void foo(scope ref T) { ... }
foo(rvalue());   // here is "current statement/expression"
// lowered to: { auto __tmp = rvalue(); foo(__tmp);  __tmp.~this(); };

Note that the lifetime of taken rvalue is mostly same as non-ref parameter
case.
void bar(T arg) { ... arg.~this(); }
bar(rvalue());
// the rvalue argument is moved into the function 'bar', and destroyed
inside function.

Kenji Hara


2013/4/24 Diggory <diggsey@googlemail.com>

> On Wednesday, 24 April 2013 at 00:54:12 UTC, kenji hara wrote:
>
>> 2013/4/24 Manu <turkeyman@gmail.com>
>>
>>  "The r-value being passed is assigned to a stack allocated temporary,
>>> which has a lifetime that is identical to any other local variable, ie,
>>> the
>>> lifetime of the function in which it appears."
>>> There, I defined it.
>>>
>>>
>> Good definition. If add more,
>> "getting address of "scope" parameter would be disallowed, at least in
>> @safe code, because it would be regarded as the escape of stack allocated
>> temporary."
>>
>> Kenji Hara
>>
>
> Why does the temporary need to exist any longer than the current statement? (the current lifetime of temporaries are the statement or expression). Surely any longer is just wasting stack space.
>

On Tuesday, April 23, 2013 14:58:50 Andrei Alexandrescu wrote:
> On 4/23/13 2:45 PM, Namespace wrote:
> >> auto ref is needed to accept rvalues.
> > 
> > I'm curious as to when that is implemented. Or if I purely watch here again in a year and the issue still exists. :)
> 
> It is implemented, but currently only for templates.

I think that the obvious thing to do for auto ref and non-templates is to just make it so that underneath the hood it's ref, and if you pass an rvalue to it, a variable is created on the stack (presumably for the lifetime of the statement that the function call is in) and passed to the function (so then it's an lvalue as ref requires).

However, the problem with simply making auto ref do this for non-templated functions is that then it functions fundamentally differently for templated and non-templated functions. And we can't change auto ref to work that way for templated functions, because auto ref's current behavior has proven useful for forwarding the refness of an argument (which auto won't do on its own), which improves D's forwarding capabilities. But given the combinatorial explosion of template instantiations if you have many auto ref variables for a templated function, most uses of auto ref with templated functions really should be using what I just described for non-templated functions. But to allow both behaviors, we pretty much need a new attribute. And if we're using a new attribute, we either need to make it do what auto ref currently does for templates (and therefore change auto ref to do the new thing and break some amount of code out there), or we need to make the new attribute do the new thing and never implement auto ref for non-templated functions.

So, probably the best route at this point is to come up with a new attribute which is replace with ref in the function definition and transparently creates variables for rvalues so that they can be passed to the function as lvalues, and then that attribute works with both templated and non-templated functions. But it _is_ kind of annoying to not be able to use auto ref for that given that that's basically what it was intended for in the first place.

Maybe I'm missing something here, but that's how the situation seems to me.

- Jonathan M Davis

On 24 April 2013 04:29, Andrei Alexandrescu <SeeWebsiteForEmail@erdani.org>wrote:

> On 4/23/13 2:00 PM, Manu wrote:
>
>> On 24 April 2013 03:15, Andrei Alexandrescu
>>
>      Many details are missing. This is not a simple problem.
>>
>>
>> So what are some others?
>>
>
> Returning a reference is an important topic.
>

I think that's easily solved. We've already touched on this.
It just extends this same rule to the return value, only 'scope ref' may
return a 'scope ref'.

         An r-value passed this way produces a
>>         temp, which is a stack variable. It's life is identical to any
>> other
>>
>>         stack variable, ie, it lives for the life of the function where
>>         it appears.
>>
>>
>>     That's a possibility, but it's a departure from current semantics
>>     and is not mentioned in the DIP.
>>
>>
>> I think it's presumed in the DIP, and it's certainly how Kenji
>> implemented it.
>> What 'current' semantic is it a departure from? The one where passing a
>> literal produces a compile error? Certainly, that's the point.
>>
>
> Currently, rvalues exist until they have been consumed by a call. By DIP 36, some rvalues exist through the end of the function.
>

I don't think this is quite right. r-values never 'exist' at all, and
they're not 'consumed'. They only come into existence when being copied to
the callee's argument list, this makes them local to, and gives them the
life of the callee.
ref args must receive an existing object however, so in the ref case, a
local temp must be created in the caller, which logically has the life of
any other local.

         auto-ref on the other hand IS a new feature (in this context),
>>         and it
>>         also makes no sense if you ask me. It's a template concept which
>>         is not
>>         applicable here.
>>
>>
>>     It is a feature that has been implemented and works, just not in all
>>     cases.
>>
>>
>> This isn't a 'case'. It's a separate issue.
>> Safely passing a temp to a ref function arg, and whether a template
>> argument is automatically determined to be ref or not are barely related
>> problems.
>> I still can't see how auto-ref has any business in this context.
>>
>
> They are related inasmuch they solve the same problem (define a function that accepts both lvalues and rvalues). They are distinct because currently in a template you could at least in theory figure out whether the function has been called with an lvalue on rvalue. The code below does not currently work but could be made to work:
>
> void fun(T)(auto ref T t)
> {
>     static if (is(t == ref)) {}
> }
>
> If we decide this feature is unnecessary (as I suspect is the case), we should change the implementation of auto ref to only use one body for both ref and non-ref versions.
>

I think you're conflating 2 problems to make them appear related.
auto-ref is well defined as is when used with templates, if you change
that, it'll break the purpose it was designed for.
And again, it's not about accepting 'both lvalues and rvalues', it's got
nothing to do with r-values.
r-values can NEVER be accepted by a ref argument, they don't 'exist', we're
talking about implicitly generated temporaries (which also addresses the
issues with receiving _explicit_ temp's).

              In particular we are much more inclined to impart real,
>>         demonstrable
>>              safety to "ref"
>>
>>
>>         ref is unsafe by definition.
>>
>>
>>     We want to aim at making ref safe, thus making it useful as
>>     restricted pass-down pointers. For full possibilities, one should
>>     use pointers.
>>
>>
>> Okay, I'm good with that too, but how is that intended to work?
>> If the intent is to make ref escaping disallowed by default, that is a
>> major breaking change...
>>
>
> Walter and I are inclined to take the hit because we believe the upside is worth it.
>
>
>  Can we start talking about virtual-by-default again while we're at it?
>>
>
> There are no plans to change that.
>

Well we need to start discussing it then.
The magnitude of the breakage is less than making ref safe, and if that's
on the table, then this surely deserves some attention.

         I don't believe this is possible without
>>         some further justification.
>>
>>
>>     The justification is that unsafe uses of ref are few and
>>     uninteresting (they can be replaced with pointers). It would be very
>>     powerful to be able to guarantee that safe code can use ref.
>>
>>
>> Again, this sounds like a major breaking change.
>> Why is scope-ref inferior? It's more informative, and offers more
>> flexibility (ie, the option of ref with or without scope)
>>
>
> Whether scope ref is inferior to the ref/auto ref combo is a judgment call. On the face of it, any new feature has to prove its utility so it starts from a somewhat disadvantaged position.


Your proposal is an equally new feature, and surely stands on even ground. I still see that auto-ref is completely irrelevant if you make ref safe. I'm not sure why auto-ref keeps coming up under your proposal. It's fine applied to templates, and it's no longer required when applied to functions in the event ref can safely receive a temp.

I'm worried that your proposal significantly changes the meaning of multiple existing features (both ref, and auto-ref), at the cost of major breaking changes, while the DIP36 approach requires no changes to existing features, and makes perfect sense  logically. It also keeps control in the programmers hands, which I would never consider a problem.

         DIP36 however creates a situation where it's known that passing
>>         a temp
>>         is actually safe.
>>
>>              and to make "auto ref" work as a reference that can bind to
>>         rvalues
>>              as well as lvalues.
>>
>>
>>         What does it mean to make a reference bind to r-values aswell as
>>         l-values? Lots of people keep saying this too, but it doesn't
>> really
>>         make sense to me either.
>>
>>
>>     I don't understand the question as the answer is in it.
>>
>>
>>         No reference can bind to r-values, r-values can not be addressed.
>>
>
> This is a matter of language definition. Rvalues can be bound to references today, and the bound references can be addressed.
>
> struct S { void fun() { writeln(&this); } }
> unittest { S().fun(); }
>

S() must just be an implicit local (just like every other instance of
passing an rvalue-by-ref should be).
If this code is possible, then a serious non-uniformity exists where this
is not applied to all r-values, and that should be fixed on this basis
alone.

     But auto ref and scope ref do bind to r-values.
>>
>>
>>         It's
>>         really a temp copy of said r-value that we're dealing with,
>>         which is an
>>         l-value, ie, a local with a lifetime that's unsuitable for
>>         passing by
>>         non-scope-ref.
>>         scope-ref would promise that it won't escape the callee, and thus
>> is
>>         safe to pass a temp.
>>
>>
>>     Our aim is to have ref make that promise.
>>
>>
>>         ref is fundamentally broken in D right now. DIP36 creates a
>>         situation
>>         where it could be fixed.
>>
>>
>>     A new feature is not a fix.
>>
>>
>> If scope is a new feature, then the keyword shouldn't compile and pretend that it does stuff.
>>
>
> You are confusing a feature with a keyword. A given keyword may support many features, e.g. static, final etc.
>

Okay, so what does this add to scope that it doesn't already allege to
promise?
You're saying that "scope doesn't work now, therefore it's not a feature,
only a plan, with reserved syntax"?

 It's an incomplete/unimplemented feature, not a new one.
>> People are aware of it, they can write code that presumes it's present and working. It compiles successfully.
>>
>>         I would personally take DIP36 one step further,
>>         and ban all local's from being passed to non-scope ref.
>>         Yes, a breaking change, but you could argue that any code that
>>         passes a
>>         stack variable to any ref arg is already broken. But this can be
>>         addressed in a future DIP.
>>
>>
>>         ...perhaps I'm missing something fundamental in DIP36, or about
>>         'auto ref'?
>>         I can't understand why there seem to be 2 polarised parties on
>> this
>>         issue, which appear to see the problem completely differently,
>>         and can't
>>         visualise the counter perspective at all.
>>
>>
>>     DIP36 should be closed. We must focus on making ref safe and on
>>     making auto ref work with non-templates.
>>
>>
>> I'm fine with that, but it sounds like a massive breaking change.
>> However upon the presumption of this new goal, I don't see the relevance
>> of auto-ref anymore? Why continue to bring it up?
>> If ref is safe, nothing else is needed.
>>
>
> auto ref is needed to accept rvalues.


If I were to start arguing upon the basis of your proposal, I would then
start arguing that auto-ref in this context is pointless, and automatic
creation of a temp to hold any r-value should be the universal/default
behaviour.
You assert that requiring explicit use of 'scope' is a burden. I assert
that requiring a completely meaningless instance of 'auto' is a greater
burden, when it could easily be made the default behaviour.
If you're gonna go about making ref safe, then abandon thoughts of auto-ref
on non-templates; they can now receive a temporary safely without any
justification.

On 24 April 2013 04:44, Walter Bright <newshound2@digitalmars.com> wrote:

> On 4/23/2013 8:33 AM, Manu wrote:
>
>> "The r-value being passed is assigned to a stack allocated temporary,
>> which has
>> a lifetime that is identical to any other local variable, ie, the
>> lifetime of
>> the function in which it appears."
>> There, I defined it.
>>
>
> Locals have a lifetime that is terminated by the closing } of the scope they appear in. There can be many such scopes in a function.
>
> There's also the issue of:
>
>   a || b || c
>
> If b creates a temporary, it's life ends at the end of the expression or statement - it's complicated.
>

Is it actually complicated?
Enclosing scope seems fine too. Can you suggest a case where it could
escalate to an outer scope via a scope-ref argument?
So let's say then, that lifetime should be identical to a local declared in
the same location. No change of any rules is required, it will work as
expected.

scope ref a(scope ref int x) { return x; }
void b(scope ref int x);


On 24 April 2013 11:02, Diggory <diggsey@googlemail.com> wrote:

> On Wednesday, 24 April 2013 at 00:54:12 UTC, kenji hara wrote:
>
>> 2013/4/24 Manu <turkeyman@gmail.com>
>>
>>  "The r-value being passed is assigned to a stack allocated temporary,
>>> which has a lifetime that is identical to any other local variable, ie,
>>> the
>>> lifetime of the function in which it appears."
>>> There, I defined it.
>>>
>>>
>> Good definition. If add more,
>> "getting address of "scope" parameter would be disallowed, at least in
>> @safe code, because it would be regarded as the escape of stack allocated
>> temporary."
>>
>> Kenji Hara
>>
>
> Why does the temporary need to exist any longer than the current statement? (the current lifetime of temporaries are the statement or expression). Surely any longer is just wasting stack space.
>

>
>
> On 24 April 2013 11:02, Diggory <diggsey@googlemail.com> wrote:
>
>> On Wednesday, 24 April 2013 at 00:54:12 UTC, kenji hara wrote:
>>
>>> 2013/4/24 Manu <turkeyman@gmail.com>
>>>
>>>  "The r-value being passed is assigned to a stack allocated temporary,
>>>> which has a lifetime that is identical to any other local variable, ie,
>>>> the
>>>> lifetime of the function in which it appears."
>>>> There, I defined it.
>>>>
>>>>
>>> Good definition. If add more,
>>> "getting address of "scope" parameter would be disallowed, at least in
>>> @safe code, because it would be regarded as the escape of stack allocated
>>> temporary."
>>>
>>> Kenji Hara
>>>
>>
>> Why does the temporary need to exist any longer than the current statement? (the current lifetime of temporaries are the statement or expression). Surely any longer is just wasting stack space.
>
>
scope ref a(scope ref int x) { return x; }
void b(scope ref int x);

b(a(10));

The temp produced from '10' needs to last longer than a(), because it can
be returned to b(). Ie, the temp needs the life of any normal local
declared within that scope.

How do you describe 'wasting' stack space? Does D attempt to recycle stack space from previous short-lived locals within a single function?