On 4/12/12 6:40 AM, Michel Fortin wrote:
> Speaking of ref escapes, perhaps it'd be a good time to remind ourselves of this hole:
>
> 	ref int foo(ref int i)
> 	{
> 		return i;
> 	}
>
> 	ref int bar()
> 	{
> 		int i;
> 		return foo(i);
> 	}
>
> 	void main()
> 	{
> 		bar() += 1; // what are we incrementing again?
> 	}
>
> The way ref will work now makes makes it possible to define bar like this instead, which would result in the same generated code:
>
> 	ref int bar()
> 	{
> 		return foo(1);
> 	}
>
> Now the problem is a little more hidden from sight.
>
> I don't see that as a problem of the new ref behaviour (which I like), but since we're looking at ref semantics I thought it'd be good to have a reminder that there's still an important hole related to ref in need of some attention.
>
> <http://d.puremagic.com/issues/show_bug.cgi?id=3925>

I hate this problem. It makes the necessary analysis quite a bit more complicated and less precise. Essentially most functions that take a ref to a local and return a ref must be analyzed for the possibility they return the argument.

Let me look over Kenji's proposal, hopefully it takes care of this automatically :o).

Andrei

_______________________________________________
dmd-beta mailing list
dmd-beta@puremagic.com
http://lists.puremagic.com/mailman/listinfo/dmd-beta

This part of the proposal is sound but I fear it could cause confusion. It is a special rule, and special rules may be surprising.

Also I don't know when people have bugs because of it. I mean when is the last time anyone called swap(10, 20)?

Let me see the second part.


Andrei

On 4/13/12 12:58 AM, kenji hara wrote:
> I think to avoid some useless case, I'd like to add a rule.
> 
> In basic, literals (e.g. 10, 3.14, "hello") in D are typed as mutable.
> typeof(10) == int
> typeof(3.14) == double
> typeof("hello") == immutable(char)[]   // array itself is mutable
> 
> Then, with new rvalue reference mechanism, literals are assigned to mutable temporaries, and be bound to ref.
> 
> void foo(ref int n){}
> foo(10);
> // same as int tmp = 10; foo(tmp);
> 
> But it is less useful behavior.
> 
> I propose that if rvalue reference binding is needed, and the rvalue is built-in literal, its type is treated as const.
> 
> void foo(ref int n){}
> foo(10);
> // same as const(int) tmp = 10; foo(tmp);
> // tmp is const, and binding const variable to mutable ref is illegal,
> then raises an error.
> 
> Finally, I don't know we should apply this rule to struct literal too. It sounds good, but I afraid that is too restrict.
> 
> Kenji Hara
> 
> 2012$BG/(B4$B7n(B11$BF|(B13:33 Andrei Alexandrescu<andrei@erdani.com>:
>> On 4/10/12 7:57 PM, Walter Bright wrote:
>>>
>>> 2.
>>> double&  d;
>>> int i;
>>> void foo() {
>>> d = i;
>>> }
>>
>>
>> This example is off; a reference can't be rebound. The relevant example is:
>>
>> void increment(double&  d)
>> {
>>     ++d;
>> }
>> ...
>> int i;
>> increment(i);
>>
>> People think the int has been incremented, but in fact a useless temporary has.
>>
>> The discussion about what to do in D has been a bit longer and more far-reaching than Walter mentioned.
>>
>> The long-term plan is to never let the address of a ref escape the expression in which the ref occurs. That means in essence that user code can't take the address of a ref.
>>
>> Once that is in place, we will know for sure that all ref passed into and returned by functions will not escape the immediate expression in which that happens - great for safe code.
>>
>> People who need to take&this and escape it (e.g in linked lists implemented with struct) will not be able to; they'll have to use static functions and pointers for that. Generally any work that involves escaping pointers will have to use pointers, not references.
>>
>> I think this puts us in a very good spot:
>>
>> 1. Safe code will be able to use ref liberally
>>
>> 2. Functions will be able to return ref knowing the ref won't survive the current expression. This is awesome for sealed containers - safe and fast.
>>
>> What does this have to do with rvalues and lvalues? It means that with the appropriate precautions, we _can_ transform rvalues into lvalues, because we know their address can't unsafely escape.
>>
>> There is one precautions to take: we should never convert a value of type T to a ref of another type U. That would cause the problems we learned from C++. There are 3 cases of such implicit conversions:
>>
>> 1. built-in numerics, e.g. an int should not convert to a ref double.
>>
>> 2. Class inheritance, e.g. a Widget should not convert to a ref Object.
>>
>> 3. alias this, e.g.:
>>
>> struct T {}
>> struct A { @property T fun(); alias fun this; }
>> void fun(ref T);
>> ...
>> A a;
>> fun(a); // should not work
>>
>>
>> I think this all holds water. Destroy!
>>
>> Andrei
>>
>>
>> _______________________________________________
>> dmd-beta mailing list
>> dmd-beta@puremagic.com
>> http://lists.puremagic.com/mailman/listinfo/dmd-beta
> _______________________________________________
> dmd-beta mailing list
> dmd-beta@puremagic.com
> http://lists.puremagic.com/mailman/listinfo/dmd-beta
_______________________________________________
dmd-beta mailing list
dmd-beta@puremagic.com
http://lists.puremagic.com/mailman/listinfo/dmd-beta

On 4/13/12 3:47 AM, kenji hara wrote:
> I think the main purpose of rvalue reference is:
> "Called function requires the address of given value, but not need to
> change its value semantics."

I think it's a good way to look at it, but there's sometimes something like this: "This function doesn't care rvalue vs. lvalue, but doesn't want to mess with const and would prefer pass by ref wherever possible for efficiency reasons."

> We can never rewrite basic literals, therefore we can say that they
> have constant value semantics, even if their types are mutable. And
> then, it seems that rewriting literal through rvalue reference (==
> temporary variable) is counter-intuitive, and less useful.
> On the other hand, compound literals usually have mutable
> elements/fields, so called function may use and rewrite them. It is
> often useful.

Correct. One detail is that basic literals are obviously not modifiable at the call site: foo(2, 3.4) is visibly unable to modify its arguments. But then, in fairness, foo(a, b) is more confusing if a and b are enum values.

> More details of my proposal with an actual code:
> 
> struct S{}
> 
> void foo(const ref int v){}
> void foo(const ref double v){}
> void foo(const ref cdouble v){}
> void foo(const ref int[] v){}
> void foo(const ref int[int] v){}
> void foo(const ref S v){}
> void bar(      ref int v){}
> void bar(      ref double v){}
> void bar(      ref cdouble v){}
> void bar(      ref int[] v){}
> void bar(      ref int[int] v){}
> void bar(      ref S v){}
> 
> void hoo(T)(const ref T v){}
> void var(T)(      ref T v){}
> 
> void main()
> {
>      // cannot bind basic literals to mutable ref
>      enum int ei = 1024;
>      foo(1024);  static assert(!__traits(compiles, bar(1024)));
>      hoo(1024);  static assert(!__traits(compiles, var(1024)));
>      foo(ei);    static assert(!__traits(compiles, bar(ei)));
>      hoo(ei);    static assert(!__traits(compiles, var(ei)));

Fine.

>      enum double ef = 3.14;
>      foo(3.14);  static assert(!__traits(compiles, bar(3.14)));
>      hoo(3.14);  static assert(!__traits(compiles, var(3.14)));
>      foo(ef);    static assert(!__traits(compiles, bar(ef)));
>      hoo(ef);    static assert(!__traits(compiles, var(ef)));
> 
>      enum cdouble ec = 1+1i;
>    /*foo(1+1i);*/static assert(!__traits(compiles, bar(1+1i)));
>    /*hoo(1+1i);*/static assert(!__traits(compiles, var(1+1i)));
>      foo(ec);    static assert(!__traits(compiles, bar(ec)));
>      hoo(ec);    static assert(!__traits(compiles, var(ec)));

So far so good.

>      // can bind compound literals to mutable ref
>      enum int[] ea = [1,2];
>      foo([1,2]); bar([1,2]);
>      hoo([1,2]); var([1,2]);
>      foo(ea);    bar(ea);
>      hoo(ea);    var(ea);
> 
>      enum int[int] eaa = [1:1];
>      foo([1:1]); bar([1:1]);
>      hoo([1:1]); var([1:1]);
>      foo(eaa);   bar(eaa);
>      hoo(eaa);   var(eaa);
> 
>      enum S es = S();
>      foo(S());   bar(S());
>      hoo(S());   var(S());
>      foo(es);    bar(es);
>      hoo(es);    var(es);
> }

This makes sense, but there are two issues:

1. How about generic code? Generic code should have a "use the most efficient passing convention for this argument, I'm okay with either value or reference".

2. What if the rvalue is the result of calling a function? Does it bind to mutable ref or not?

It became clear to me that Walter and my initial proposal is not good. But I'm not sure we have a clear and simple vision yet.



Andrei
_______________________________________________
dmd-beta mailing list
dmd-beta@puremagic.com
http://lists.puremagic.com/mailman/listinfo/dmd-beta

On Apr 13, 2012, at 1:48 PM, "Jonathan M Davis" <jmdavisProg@gmx.com> wrote:

> On Friday, April 13, 2012 22:18:16 Don Clugston wrote:
>> Interestingly, the ref int[3] idiom is documented as working, under
>> 'interfacing with C'.
>> Anything beyond that seems to be undefined.
>> That page seems to document dynamic arrays as _not_ working --
>> certainly as having no C equivalent.
> 
> Which seems really backwards considering that in both case, you'd be using T* in C, and if anything, dynamic arrays are closer to that than static arrays. Personally, I'd have expected arr.ptr to be required in both cases.

The problem is mostly with stuff like:

extern (C):
alias int[2] foo;
void fn(foo);

Now make the alias platform-dependent (as in the Posix package) and tell me what the prototype for fn() should be. Fortunately, static array args are almost nonexistent in C99 and Posix.
_______________________________________________
dmd-beta mailing list
dmd-beta@puremagic.com
http://lists.puremagic.com/mailman/listinfo/dmd-beta

On Sunday, April 15, 2012 09:07:03 Sean Kelly wrote:
> On Apr 13, 2012, at 1:48 PM, "Jonathan M Davis" <jmdavisProg@gmx.com> wrote:
> > On Friday, April 13, 2012 22:18:16 Don Clugston wrote:
> >> Interestingly, the ref int[3] idiom is documented as working, under
> >> 'interfacing with C'.
> >> Anything beyond that seems to be undefined.
> >> That page seems to document dynamic arrays as _not_ working --
> >> certainly as having no C equivalent.
> > 
> > Which seems really backwards considering that in both case, you'd be using T* in C, and if anything, dynamic arrays are closer to that than static arrays. Personally, I'd have expected arr.ptr to be required in both cases.
> The problem is mostly with stuff like:
> 
> extern (C):
> alias int[2] foo;
> void fn(foo);
> 
> Now make the alias platform-dependent (as in the Posix package) and tell me
> what the prototype for fn() should be. Fortunately, static array args are
> almost nonexistent in C99 and Posix.

It was my understanding that you arrays in C are _always_ passed as a pointer and that even if you use [] instead of * on the parameter, it's the same as using * and that there was no way to specifically pass a static array differently from a dynamic one. But I may remember that incorrectly, since I almost never use static arrays.

If I'm right though, then fn will just always take an int*.

- Jonathan M Davis
_______________________________________________
dmd-beta mailing list
dmd-beta@puremagic.com
http://lists.puremagic.com/mailman/listinfo/dmd-beta

Disallowing property results to bind to ref is not a good solution IMO.  Note that one of the *worst* problems with rvalue/lvalue ref semantics right now is operator overloading.  Frequently, one may want to pass by ref for overloading arithmetic operations or other operators due to the size of the struct.  But then simple combining operations like a + (b - c) don't work.

Note that this varies *greatly* with the issue of the compiler doing implicit casting.  In the compiler's case, it's creating behind-the-scenes temporaries that you cannot see or access.  But in the case of the return value of a function or property, everything is spelled out in the declaration of the function or property.

I think part of our issue here is, there are multiple reasons to pass by ref.  So it's difficult to establish the correct rules.

My thought is, we need different markers for different situations.  I don't think one keyword is enough.

What 'auto ref' was supposed to do was allow both rvalue and lvalue binding.  However, maybe it's lvalue-only binding which is the less common beast?  What about ref taking both rvalues and lvalues (unless a non-ref overload is present) and something else designating that lvalues are the only acceptable binding?

Regardless of the solution, I think we need to designate two different situations:

1. I want to pass this by reference because it's more efficient 2. I want to pass this by reference because I want to change it.

And I really can't see any way the compiler can infer one over the other.

-Steve




>________________________________
> From: Andrei Alexandrescu <andrei@erdani.com>
>To: Discuss the dmd beta releases for D <dmd-beta@puremagic.com>
>Sent: Thursday, April 12, 2012 5:50 PM
>Subject: Re: [dmd-beta] rvalue references
> 
>On 4/12/12 3:11 PM, Dmitry Olshansky wrote:
>> Interesting thing with non-escaping ref is that making truly unsealed containers is hard while writing sealed ones made easier(and that's a good thing btw).
>
>There is a liability here however. Today, people who don't want to allow changes to their containers or ranges would routinely return rvalues from e.g. front().
>
>If we allow function results to bind to ref parameters, people would think they modify stuff when in fact they don't do anything. Consider:
>
>void swap(T)(ref T lhs, ref T rhs);
>...
>swap(r1.front, r2.front);
>
>The user thinks the fronts of the ranges are swapped but nothing happens.
>
>So we don't want to allow function results (including property results) to bind to ref parameters.
>
>
>Andrei
>
>_______________________________________________
>dmd-beta mailing list
>dmd-beta@puremagic.com
>http://lists.puremagic.com/mailman/listinfo/dmd-beta
>
>
>

On Apr 15, 2012, at 5:29 PM, Jonathan M Davis wrote:

> On Sunday, April 15, 2012 09:07:03 Sean Kelly wrote:
>> On Apr 13, 2012, at 1:48 PM, "Jonathan M Davis" <jmdavisProg@gmx.com> wrote:
>>> On Friday, April 13, 2012 22:18:16 Don Clugston wrote:
>>>> Interestingly, the ref int[3] idiom is documented as working, under
>>>> 'interfacing with C'.
>>>> Anything beyond that seems to be undefined.
>>>> That page seems to document dynamic arrays as _not_ working --
>>>> certainly as having no C equivalent.
>>> 
>>> Which seems really backwards considering that in both case, you'd be using T* in C, and if anything, dynamic arrays are closer to that than static arrays. Personally, I'd have expected arr.ptr to be required in both cases.
>> The problem is mostly with stuff like:
>> 
>> extern (C):
>> alias int[2] foo;
>> void fn(foo);
>> 
>> Now make the alias platform-dependent (as in the Posix package) and tell me
>> what the prototype for fn() should be. Fortunately, static array args are
>> almost nonexistent in C99 and Posix.
> 
> It was my understanding that you arrays in C are _always_ passed as a pointer and that even if you use [] instead of * on the parameter, it's the same as using * and that there was no way to specifically pass a static array differently from a dynamic one. But I may remember that incorrectly, since I almost never use static arrays.
> 
> If I'm right though, then fn will just always take an int*.

Sure… as long as you're sure "foo" is a static array.  But if it's an alias that the spec allows to be an opaque type...
_______________________________________________
dmd-beta mailing list
dmd-beta@puremagic.com
http://lists.puremagic.com/mailman/listinfo/dmd-beta

On Apr 16, 2012, at 6:32 AM, Steve Schveighoffer wrote:
> 
> Regardless of the solution, I think we need to designate two different situations:
> 
> 1. I want to pass this by reference because it's more efficient 2. I want to pass this by reference because I want to change it

Ideally, no one should ever specify "ref" simply for efficiency.  The programmer should choose semantics for logical reasons, and leave optimization up to the compiler.  That may still not be practical however.
_______________________________________________
dmd-beta mailing list
dmd-beta@puremagic.com
http://lists.puremagic.com/mailman/listinfo/dmd-beta

Well, there are many times I'd rather have 'this' passed by value for efficiency!

Const ref would fit the bill nicely, if it didn't affect the type.


What would be interesting is if there was a way to say "this won't change in this function", and then the compiler could choose to pass by ref or value depending on which was more efficient, while still enforcing the property that the value doesn't change.

Hm.. could there be a library solution?


-Steve




>________________________________
> From: Sean Kelly <sean@invisibleduck.org>
>To: Steve Schveighoffer <schveiguy@yahoo.com>; Discuss the dmd beta releases for D <dmd-beta@puremagic.com>
>Sent: Monday, April 16, 2012 2:31 PM
>Subject: Re: [dmd-beta] rvalue references
> 
>On Apr 16, 2012, at 6:32 AM, Steve Schveighoffer wrote:
>> 
>> Regardless of the solution, I think we need to designate two different situations:
>> 
>> 1. I want to pass this by reference because it's more efficient 2. I want to pass this by reference because I want to change it
>
>Ideally, no one should ever specify "ref" simply for efficiency.  The programmer should choose semantics for logical reasons, and leave optimization up to the compiler.  That may still not be practical however.
>
>
>

On Monday, April 16, 2012 11:31:19 Sean Kelly wrote:
> On Apr 16, 2012, at 6:32 AM, Steve Schveighoffer wrote:
> > Regardless of the solution, I think we need to designate two different situations:
> > 
> > 1. I want to pass this by reference because it's more efficient 2. I want to pass this by reference because I want to change it
> 
> Ideally, no one should ever specify "ref" simply for efficiency. The programmer should choose semantics for logical reasons, and leave optimization up to the compiler. That may still not be practical however.

In many cases, the compiler can't know when a copy is actually required. If there's a way for the programmer to indicate that they don't care whether a copy is made or not, letting the compiler decide, then that solves the problem (which was the idea behind auto ref). Without that though, the compiler will have to assume that a copy has to be made in cases where it doesn't. And actually, since without auto ref, there's no way to overload a function such that the compiler chooses ref or non-ref based on what it thinks is the most efficient, I don't see how the compiler could elide copies except in cases where inlining eliminates the function call entirely. We need a way for the programmer to be able to tell the compiler that they don't care whether the argument is passed by ref or not. Barring that, making it possible for const ref (and possibly ref) to take rvalues is probably the best that we can do.

- Jonathan M Davis
_______________________________________________
dmd-beta mailing list
dmd-beta@puremagic.com
http://lists.puremagic.com/mailman/listinfo/dmd-beta