> Honestly though, I'm inclined to argue that functions which return by ref and
> have a ref parameter of that same type just be considered @system.

What about this:

struct Foo
{
    int a;
}

ref int bar(ref Foo foo)
{
    return foo.a;
}

the parameter type and the return type here are different, but bar still returns a reference to its parameter. I guess you should consider all functions that return ref and have at least one ref parameter @system (unless they are marked @trusted).

On Sunday, December 30, 2012 23:18:43 jerro wrote:
> > Honestly though, I'm inclined to argue that functions which
> > return by ref and
> > have a ref parameter of that same type just be considered
> > @system.
> 
> What about this:
> 
> struct Foo
> {
>      int a;
> }
> 
> ref int bar(ref Foo foo)
> {
>      return foo.a;
> }
> 
> the parameter type and the return type here are different, but bar still returns a reference to its parameter. I guess you should consider all functions that return ref and have at least one ref parameter @system (unless they are marked @trusted).

Good point. Member variables of parameters also cause problems. So, it very quickly devolves to any function which accepts a user-defined type by ref and returns anything by ref would have to be @system, which is far from pleasant.

- Jonathan M Davis

On Sunday, 30 December 2012 at 22:30:24 UTC, Jonathan M Davis wrote:
> On Sunday, December 30, 2012 23:18:43 jerro wrote:
>> > Honestly though, I'm inclined to argue that functions which
>> > return by ref and
>> > have a ref parameter of that same type just be considered
>> > @system.
>> 
>> What about this:
>> 
>> struct Foo
>> {
>>      int a;
>> }
>> 
>> ref int bar(ref Foo foo)
>> {
>>      return foo.a;
>> }
>> 
>> the parameter type and the return type here are different, but
>> bar still returns a reference to its parameter. I guess you
>> should consider all functions that return ref and have at least
>> one ref parameter @system (unless they are marked @trusted).
>
> Good point. Member variables of parameters also cause problems. So, it very
> quickly devolves to any function which accepts a user-defined type by ref and
> returns anything by ref would have to be @system, which is far from pleasant.
>
> - Jonathan M Davis

This may be far fetched, but consider this,

If a function returns a ref that is the same as what was passed in by ref, then the passed and return addresses would match, which means that it still may be possible for the compiler to detect the situation.

This is more complicated in the case where a user defined struct was passed by ref, and the ref return type is a member from that struct, but it still may be possible to detect it.

If address matching is possible (or is that determined only at link time?) then it may be possible to detect a situation that should be illegal and flagged as a compiler error.

--rt

On Monday, December 31, 2012 02:37:56 Rob T wrote:
> This may be far fetched, but consider this,
> 
> If a function returns a ref that is the same as what was passed in by ref, then the passed and return addresses would match, which means that it still may be possible for the compiler to detect the situation.
> 
> This is more complicated in the case where a user defined struct was passed by ref, and the ref return type is a member from that struct, but it still may be possible to detect it.
> 
> If address matching is possible (or is that determined only at link time?) then it may be possible to detect a situation that should be illegal and flagged as a compiler error.

Addresses would only be known at runtime, and it's far too late at that point.

- Jonathan M Davis

/*
The implementation of delegates has solved an analogous problem. e.g.
*/
import std.stdio;

auto getfun( int x) {
	int y = x*x;
	int ysquared() {
		return y*y;
	}
	return &ysquared;
}

void main() {
	auto f1 = getfun(2);
	auto f2 = getfun(3);
	writeln( f1() );
	writeln( f2() );
}
/*
The variable y no longer exists local to getfun, but its existence is prolonged making its use safe.

Just like _immutable_, some clever compiler inference that is transitive plus a delegate-like solution may do the job with ref without imposing constraints upon the sane. Plus, it may lead to new terrain when the liberation of local variables as above occurs in this context.

Warning: this is all speculation.
*/

On Monday, December 31, 2012 03:34:18 Carl Sturtivant wrote:
> The implementation of delegates has solved an analogous problem.

Delegates use closures. The stack of the calling function is copied onto the heap so that it will continue to be valid for the delegate after the function returns. We don't want to be doing anything like that with ref, and it's generally, completely unnecessary. It's just that there are cases where you can escape such references if you're not careful, and the compiler currently considers those to be @safe, when they're not actually safe. So, we presumably need to do one of

1. Limit what can be legally done with ref to get rid of the problem.

2. Make ref @system in cases where we can't prove that it's @safe and try and prove it to be @safe in as many situations as possible.

3. Create a new attribute which has to be used when a function returns a ref to a parameter and use that to make it illegal to pass a ref to a local variable to such functions.

4. Something else that similarly protects against this at compile time without any extra overhead at runtime.

I really don't think that something like closures would be acceptable for ref parameters.

- Jonathan M Davis

On 2012-12-30 22:29:33 +0000, Jonathan M Davis <jmdavisProg@gmx.com> said:

> Good point. Member variables of parameters also cause problems. So, it very
> quickly devolves to any function which accepts a user-defined type by ref and
> returns anything by ref would have to be @system, which is far from pleasant.

Note that the above definition includes every struct member function that returns a ref because of the implicit this parameter.

Also, it's not just functions returning by ref, it could be a function returning a delegate too, if the delegate happens to make use of the reference:

void delegate() foo(ref int a) {
   return { writeln(a); };
}

void delegate() bar() {
   int a;
   return foo(a); // leaking reference to a beyond bar's scope
}

And similar to passing a value by ref: you can pass a slice to a static array, then return the slice:

int[] foo(int[] a) {
	return a;
}

int[] bar() {
	int[2] a;
	return foo(a[]);
}

Three variations on the same theme.

-- 
Michel Fortin
michel.fortin@michelf.ca
http://michelf.ca/

On Monday, 31 December 2012 at 02:47:46 UTC, Jonathan M Davis wrote:

> 3. Create a new attribute which has to be used when a function returns a ref
> to a parameter and use that to make it illegal to pass a ref to a local
> variable to such functions.

If this is the way to go, maybe "@saferef" could double as both @safe and @inoutref.

[OT] I've not been here for a while, but I've been reading up on the D boards again. I might want to help with the standard library lexer and parser. Happy New Year...

On 30/12/2012 22:01, Jonathan M Davis wrote:
> On Sunday, December 30, 2012 17:32:40 Nick Treleaven wrote:
>> I think the compiler needs to be able to mark foo as a function that
>> returns its input reference. Then, any arguments to foo that are locals
>> should cause an error at the call site (e.g. in baz). So legal calls to
>> foo can always be @safe.
>>
>> To extend the above code:
>>
>> ref int quux(ref int i)
>> {
>>       return foo(i);
>> }
>>
>> Here the compiler already knows that foo returns its input reference. So
>> it checks whether foo is being passed a local - no; but it also has to
>> check if foo is passed any ref parameters of quux, which it is. The
>> compiler now has to mark quux as a function that returns its input
>> reference.
>>
>> Works?
>
> No. There's no guarantee that the compiler has access to the function's body,
> and the function being called could be compiled after the function which calls
> it. There's a reason that attribute inferrence only works with templated
> functions. In every other case, the programmer has to mark it. We're _not_
> going to get any kind inferrence without templates. D's compilation model
> doesn't allow it.

I was aware attributes would be needed for .di files. I suppose attribute inference for non-template functions is not doable.

> The closest that we could get to what you suggest would be to add a new
> attribute similar to nothrow but which guarantees that the function does not
> return a ref to a parameter. So, you'd have to mark your functions that way
> (e.g. with @norefparamreturn). Maybe the compiler could infer it for templated
> ones, but this attribute would basically have to work like other inferred
> attributes and be marked manually in all other cases. Certainly, you can't
> have the compiler figuring it out for you in general, because D's compilation
> model allows the function being called to be compiled separately from (and
> potentially after) the function calling it.

As you suggested below that, I would have the attribute mean the opposite, @refparamreturn. Functions that need it but don't have it can be detected by recompiling them. The syntax could be 'in ref':

in ref int quux(ref int i);

> And when you think about what this attribute would be needed for, it gets a
> bit bizarre to have it. The _only_ time that it's applicable is when a
> function takes an argument by ref and returns the same type by ref. In all
> other cases, the compiler can guarantee it just based on the type system.

As jerro and Michel Fortin pointed out, 'in ref' would be needed for returning input struct members, and capturing inputs with delegates and slices. So the feature might pull its weight.

I think detecting all these situations would be essentially the same as the checks needed for a scope parameter, so if/when scope parameters get implemented, 'in ref' might not be hard to bolt on.

If a simpler solution doesn't disallow any sensible uses of ref returns, that would be preferable. But I don't think we've found it yet.

> I suppose that we could have an attribute that indicated that a function _did_
> return a ref to one of its params and then have the compiler give an error if
> it were missing, which means that the foo function
>
> ref int foo(ref int i)
> {
>      return i;
> }
>
> would end up with an error for not having the attribute, whereas a function
> like baz
>
> ref int baz(int i)
> {
>      return foo(i);
> }
>
> would not end up with the error unless foo had the attribute on it. But that's
> very different from any attribute that we currently have. It would be like
> having a throw attribute instead of a nothrow attribute. I suppose that it is
> a possible solution though. I could also see an argument that the attribute
> should go on the parameter rather than the function, in which case you could
> have more fine-grained control over it, but it does complicate things further.

I suppose a parameter attribute might be useful to allow passing locals to other ref parameters which aren't returned, and as documentation.

On 31/12/2012 14:44, Nick Treleaven wrote:
> On 30/12/2012 22:01, Jonathan M Davis wrote:
>> The closest that we could get to what you suggest would be to add a new
>> attribute similar to nothrow but which guarantees that the function
>> does not
>> return a ref to a parameter. So, you'd have to mark your functions
>> that way
>> (e.g. with @norefparamreturn). Maybe the compiler could infer it for
>> templated
>> ones, but this attribute would basically have to work like other inferred
>> attributes and be marked manually in all other cases. Certainly, you
>> can't
>> have the compiler figuring it out for you in general, because D's
>> compilation
>> model allows the function being called to be compiled separately from
>> (and
>> potentially after) the function calling it.
>
> As you suggested below that, I would have the attribute mean the
> opposite, @refparamreturn. Functions that need it but don't have it can
> be detected by recompiling them. The syntax could be 'in ref':
>
> in ref int quux(ref int i);

Actually overloading 'in ref' with a different meaning on return type from parameter (const scope) is probably a bad idea. Instead:

ref int quux(@escape ref int i);

[...]
> I suppose a parameter attribute might be useful to allow passing locals
> to other ref parameters which aren't returned, and as documentation.
>