On 4/23/13 2:00 PM, Manu wrote:
> On 24 April 2013 03:15, Andrei Alexandrescu
> <SeeWebsiteForEmail@erdani.org <mailto:SeeWebsiteForEmail@erdani.org>>
> wrote:
>     "scope" is a keyword, not a language feature. In case you are
>     referring to scope variables, the feature "scope ref" has little to
>     do with it.
>
>
> How so? 'scope' simply promises that a variable may not escape its
> scope, no?
> I think it's important to recognise it as 'scope' + 'ref', the 2 don't
> have any special meaning when put together, just the logical compound,
> which allows for a safe situation for temporaries that wasn't previously
> available.

I understand. This could surely be used as an appeal to intuition for the added feature, but it's in no way a justification that it's not a new feature.

>     Many details are missing. This is not a simple problem.
>
>
> So what are some others?

Returning a reference is an important topic.

>         An r-value passed this way produces a
>         temp, which is a stack variable. It's life is identical to any other
>
>         stack variable, ie, it lives for the life of the function where
>         it appears.
>
>
>     That's a possibility, but it's a departure from current semantics
>     and is not mentioned in the DIP.
>
>
> I think it's presumed in the DIP, and it's certainly how Kenji
> implemented it.
> What 'current' semantic is it a departure from? The one where passing a
> literal produces a compile error? Certainly, that's the point.

Currently, rvalues exist until they have been consumed by a call. By DIP 36, some rvalues exist through the end of the function.

>         auto-ref on the other hand IS a new feature (in this context),
>         and it
>         also makes no sense if you ask me. It's a template concept which
>         is not
>         applicable here.
>
>
>     It is a feature that has been implemented and works, just not in all
>     cases.
>
>
> This isn't a 'case'. It's a separate issue.
> Safely passing a temp to a ref function arg, and whether a template
> argument is automatically determined to be ref or not are barely related
> problems.
> I still can't see how auto-ref has any business in this context.

They are related inasmuch they solve the same problem (define a function that accepts both lvalues and rvalues). They are distinct because currently in a template you could at least in theory figure out whether the function has been called with an lvalue on rvalue. The code below does not currently work but could be made to work:

void fun(T)(auto ref T t)
{
    static if (is(t == ref)) {}
}

If we decide this feature is unnecessary (as I suspect is the case), we should change the implementation of auto ref to only use one body for both ref and non-ref versions.

>              In particular we are much more inclined to impart real,
>         demonstrable
>              safety to "ref"
>
>
>         ref is unsafe by definition.
>
>
>     We want to aim at making ref safe, thus making it useful as
>     restricted pass-down pointers. For full possibilities, one should
>     use pointers.
>
>
> Okay, I'm good with that too, but how is that intended to work?
> If the intent is to make ref escaping disallowed by default, that is a
> major breaking change...

Walter and I are inclined to take the hit because we believe the upside is worth it.

> Can we start talking about virtual-by-default again while we're at it?

There are no plans to change that.

>         I don't believe this is possible without
>         some further justification.
>
>
>     The justification is that unsafe uses of ref are few and
>     uninteresting (they can be replaced with pointers). It would be very
>     powerful to be able to guarantee that safe code can use ref.
>
>
> Again, this sounds like a major breaking change.
> Why is scope-ref inferior? It's more informative, and offers more
> flexibility (ie, the option of ref with or without scope)

Whether scope ref is inferior to the ref/auto ref combo is a judgment call. On the face of it, any new feature has to prove its utility so it starts from a somewhat disadvantaged position.

>         DIP36 however creates a situation where it's known that passing
>         a temp
>         is actually safe.
>
>              and to make "auto ref" work as a reference that can bind to
>         rvalues
>              as well as lvalues.
>
>
>         What does it mean to make a reference bind to r-values aswell as
>         l-values? Lots of people keep saying this too, but it doesn't really
>         make sense to me either.
>
>
>     I don't understand the question as the answer is in it.
>
>
>         No reference can bind to r-values, r-values can not be addressed.

This is a matter of language definition. Rvalues can be bound to references today, and the bound references can be addressed.

struct S { void fun() { writeln(&this); } }
unittest { S().fun(); }

>     But auto ref and scope ref do bind to r-values.
>
>
>         It's
>         really a temp copy of said r-value that we're dealing with,
>         which is an
>         l-value, ie, a local with a lifetime that's unsuitable for
>         passing by
>         non-scope-ref.
>         scope-ref would promise that it won't escape the callee, and thus is
>         safe to pass a temp.
>
>
>     Our aim is to have ref make that promise.
>
>
>         ref is fundamentally broken in D right now. DIP36 creates a
>         situation
>         where it could be fixed.
>
>
>     A new feature is not a fix.
>
>
> If scope is a new feature, then the keyword shouldn't compile and
> pretend that it does stuff.

You are confusing a feature with a keyword. A given keyword may support many features, e.g. static, final etc.

> It's an incomplete/unimplemented feature, not a new one.
> People are aware of it, they can write code that presumes it's present
> and working. It compiles successfully.
>
>         I would personally take DIP36 one step further,
>         and ban all local's from being passed to non-scope ref.
>         Yes, a breaking change, but you could argue that any code that
>         passes a
>         stack variable to any ref arg is already broken. But this can be
>         addressed in a future DIP.
>
>
>         ...perhaps I'm missing something fundamental in DIP36, or about
>         'auto ref'?
>         I can't understand why there seem to be 2 polarised parties on this
>         issue, which appear to see the problem completely differently,
>         and can't
>         visualise the counter perspective at all.
>
>
>     DIP36 should be closed. We must focus on making ref safe and on
>     making auto ref work with non-templates.
>
>
> I'm fine with that, but it sounds like a massive breaking change.
> However upon the presumption of this new goal, I don't see the relevance
> of auto-ref anymore? Why continue to bring it up?
> If ref is safe, nothing else is needed.

auto ref is needed to accept rvalues.

Andrei

On 4/23/2013 8:33 AM, Manu wrote:
> "The r-value being passed is assigned to a stack allocated temporary, which has
> a lifetime that is identical to any other local variable, ie, the lifetime of
> the function in which it appears."
> There, I defined it.

Locals have a lifetime that is terminated by the closing } of the scope they appear in. There can be many such scopes in a function.

There's also the issue of:

  a || b || c

If b creates a temporary, it's life ends at the end of the expression or statement - it's complicated.

> auto ref is needed to accept rvalues.

I'm curious as to when that is implemented. Or if I purely watch here again in a year and the issue still exists. :)

On 4/23/13 2:45 PM, Namespace wrote:
>> auto ref is needed to accept rvalues.
>
> I'm curious as to when that is implemented. Or if I purely watch here
> again in a year and the issue still exists. :)

It is implemented, but currently only for templates.

Andrei

On Tuesday, 23 April 2013 at 18:58:50 UTC, Andrei Alexandrescu wrote:
> On 4/23/13 2:45 PM, Namespace wrote:
>>> auto ref is needed to accept rvalues.
>>
>> I'm curious as to when that is implemented. Or if I purely watch here
>> again in a year and the issue still exists. :)
>
> It is implemented, but currently only for templates.
>
> Andrei

I know that. But I meant when it will be "fixed". In other words: when it will be implemented for both, templates and non-templates.

On Tuesday, 23 April 2013 at 14:25:36 UTC, Andrei Alexandrescu wrote:
>> Now, this may be too naive, but what about allowing the compiler to pass
>> regular (not declared ref) const arguments per reference as it sees fit?
>> It would be just another optimization.
>>
>> I'd declare function(const arg) and that would be it.
>
> Aliasing would kill that. We considered it.
>
> Andrei

I assume you're talking about this:
http://forum.dlang.org/post/4F8C93A4.10007@erdani.com

I guess, "difficult to figure" means that the notorious sufficiently smart compiler is unrealistic. Too bad, because I don't like the idea of adding verbosity (auto ref) for the common case. pure and @safe already have that flaw, and I'd rather see no more semi-compulsory keywords.

On 4/23/13 3:01 PM, Namespace wrote:
> On Tuesday, 23 April 2013 at 18:58:50 UTC, Andrei Alexandrescu wrote:
>> On 4/23/13 2:45 PM, Namespace wrote:
>>>> auto ref is needed to accept rvalues.
>>>
>>> I'm curious as to when that is implemented. Or if I purely watch here
>>> again in a year and the issue still exists. :)
>>
>> It is implemented, but currently only for templates.
>>
>> Andrei
>
> I know that. But I meant when it will be "fixed". In other words: when
> it will be implemented for both, templates and non-templates.

Oh, I see. Well, we better get working on http://d.puremagic.com/issues/show_bug.cgi?id=9238 and http://wiki.dlang.org/DIP25.

Andrei

Based on DIP25/35 but slightly simplified, these rules would make "ref" safe, would minimise code breakage, and only marginally restricts the usage of "ref". The notion of "lifetime" is purely a tool for simplifying the explanation of the rules.

Lifetimes
0 = temporary/rvalue
1 = local variable/non-ref parameter/scope-ref parameter
2 = ref parameter
3 = static/heap allocated value

A value with any lifetime may be passed as any parameter into a function.

The & operator is safe only for lifetime >= 3, in other cases it is unsafe or system

Only values with lifetime >= 2 can be returned by ref

The lifetime of a return value is determined by the caller as follows:
    - Take the lifetimes of all values passed in using "ref" without the "scope" qualifier
    - The lifetime of the return value is the minimum lifetime of these values

OPTIONAL:
- A return value marked as "out" is guaranteed to have a lifetime >= 3 (equivalent to all ref parameters being marked "scope")
- Marking a method as "scope" is equivalent to marking the "this" parameter as such

On Tuesday, 23 April 2013 at 03:30:35 UTC, Walter Bright wrote:
> Previous discussions:
>
> http://forum.dlang.org/thread/4F84D6DD.5090405@digitalmars.com#post-4F84D6DD.5090405:40digitalmars.com
>
> http://d.puremagic.com/issues/show_bug.cgi?id=9238

This is of course an important issue. I don't know if it can be divorced from the larger ref safety issue, the reason being that you don't want to allow rvalue references which aren't safe. And the ref safety issue in itself has some subtleties.

With this in mind, my analysis so far distinguishes three (unimplemented) features which it might be desirable for any given ref parameter to have. Let's say a ref parameter which accepts an rvalue be tagged with '@rval' for the purpose of this analysis. And one which may not return itself as a reference as '@noreturn'. And one which does not assign by reference to any heap, static, or other ref parameter as '@noassign'.

I believe an important goal is to see how much mileage can be gotten out of existing syntaxes. The two existing syntaxes in question are 'ref' and 'auto ref'. I will leave 'auto ref' aside for now.

DIP25: Sealed References is divided into two parts, the first part dealing with 'ref' parameters, and the second with the '&' operator. In the first part ('DIP25A'), 'ref' is defined in such a way as to make it completely safe. This makes perfect sense as a beginning, and although it imposes significant restrictions, I don't have a great way to know just how much existing code would be affected by them. My ideal way to procede would be to implement DIP25A and to field test it on existing code with willing participants, with an eye toward finding the most common and general use cases where DIP25A was indeed too restrictive for everyday use. In my DIP35, I suggested the 'copy' function, as opposed to Andrei's 'identity' function, which I thought might be sufficiently common to warrant an addition to DIP25A, but I'm shooting somewhat in the dark without real world data.

The parameter attribute which would alleviate the problem with the 'copy' function is '@noreturn' (or some variant), which is pretty straightforward as to its functionality.

ref T func(ref T x, @noreturn ref T y) {
  return x; // pass
  return y; // error
  static T* t = &y; // pass!?
}

As you can see, '@noret' is pretty limited in that it doesn't actually prevent assigning to a global reference, but it is good for the purposes of this analysis. '@noassign' is the opposite.

ref T func(ref T x, @noassign ref T y, ref T* z) {
  return x; // pass
  return y; // pass
  z = &x; // pass
  z = &y; // error
  static T* t = &x; // pass
  t = &y; // error
}

My current experience leads me to look at the above code and think that '@noassign' should not be a separate attribute, but rather the default for *all* @safe 'ref' parameters. Assigning the address of a 'ref', which in itself does not indicate whether it is stored on the stack or on the heap, seems inherently dangerous, and I can't think of any easy way to verify that it's safe. The existing way to allow it would be to mark functions which do this '@trusted' or '@system', but there has also been suggested the inverse of '@noassign', '@global', which verifies at the call site that a non-stack reference will be passed.

void func(ref T a, @global ref T b) {
  static T* t = &a; // error with '@noassign' default
  t = &b; // pass
}

void fun(ref T a, @global ref T b) {
  T c;
  static T d;
  func(c, d); // pass
  func(d, c); // error, 'c' local

  func(a, b); // pass
  func(b, a); // error, 'a' assumed local
}

'@global' would only be useful if it proved necessary to distinguish the kind of assignments illustrated above from '@trusted' code, but it would require use case data.

'@rval' accepts rvalue references. Safety, in this case, means assuming the reference comes from the stack. Clearly, '@noassign', but what else? If the reference is returned, and some form of '@noreturn' is implemented, the calling function can know perfectly well just how local the return by reference is. So for me the dilemma is whether to merge '@rval' with '@noreturn' or not.

ref int func(@rval ref int x, @noreturn ref int y, @rval @noreturn ref int z) {
  return x; // pass
  return y; // error
  return z; // error
}

ref T fun() {
  T t;
  return func(1, t, 2); //pass, safety enforceable
  return func(1, 2, t); // error, 2 not lvalue
  return func(t, 1, 2); // error, t local, so func() is local
}

One possible definition of 'scope ref' is equivalent to '@rval @noreturn @noassign'. The advantage of this definition is that there is no need for yet another parameter attribute, '@noreturn', because it can be folded into 'scope ref'.

Another definition is simply '@rval @noassign'. The advantage here is that 'scope ref' parameters can still allow the 'identity' function and call chains.

My goal with this analysis has been to express the extent of my current thought process. I understand the powerful dilemma between the desirability of concise and usable syntax and the desirability of a powerful and flexible system which covers all of the common use cases for the features involved. The best analysis uses actual data from the field.

On 04/23/2013 07:15 PM, Andrei Alexandrescu wrote:
> ...
>
> The justification is that unsafe uses of ref are few and uninteresting
> (they can be replaced with pointers).

I don't get this. Uses cannot be "unsafe". They are either correct or wrong.

> It would be very powerful to be
> able to guarantee that safe code can use ref.
>...

It is not required to make ref guaranteed safe in @system code to guarantee this.