View mode: basic / threaded / horizontal-split · Log in · Help
December 30, 2012
Re: ref is unsafe
> Honestly though, I'm inclined to argue that functions which 
> return by ref and
> have a ref parameter of that same type just be considered 
> @system.

What about this:

struct Foo
{
    int a;
}

ref int bar(ref Foo foo)
{
    return foo.a;
}

the parameter type and the return type here are different, but 
bar still returns a reference to its parameter. I guess you 
should consider all functions that return ref and have at least 
one ref parameter @system (unless they are marked @trusted).
December 30, 2012
Re: ref is unsafe
On Sunday, December 30, 2012 23:18:43 jerro wrote:
> > Honestly though, I'm inclined to argue that functions which
> > return by ref and
> > have a ref parameter of that same type just be considered
> > @system.
> 
> What about this:
> 
> struct Foo
> {
>      int a;
> }
> 
> ref int bar(ref Foo foo)
> {
>      return foo.a;
> }
> 
> the parameter type and the return type here are different, but
> bar still returns a reference to its parameter. I guess you
> should consider all functions that return ref and have at least
> one ref parameter @system (unless they are marked @trusted).

Good point. Member variables of parameters also cause problems. So, it very 
quickly devolves to any function which accepts a user-defined type by ref and 
returns anything by ref would have to be @system, which is far from pleasant.

- Jonathan M Davis
December 31, 2012
Re: ref is unsafe
On Sunday, 30 December 2012 at 22:30:24 UTC, Jonathan M Davis 
wrote:
> On Sunday, December 30, 2012 23:18:43 jerro wrote:
>> > Honestly though, I'm inclined to argue that functions which
>> > return by ref and
>> > have a ref parameter of that same type just be considered
>> > @system.
>> 
>> What about this:
>> 
>> struct Foo
>> {
>>      int a;
>> }
>> 
>> ref int bar(ref Foo foo)
>> {
>>      return foo.a;
>> }
>> 
>> the parameter type and the return type here are different, but
>> bar still returns a reference to its parameter. I guess you
>> should consider all functions that return ref and have at least
>> one ref parameter @system (unless they are marked @trusted).
>
> Good point. Member variables of parameters also cause problems. 
> So, it very
> quickly devolves to any function which accepts a user-defined 
> type by ref and
> returns anything by ref would have to be @system, which is far 
> from pleasant.
>
> - Jonathan M Davis

This may be far fetched, but consider this,

If a function returns a ref that is the same as what was passed 
in by ref, then the passed and return addresses would match, 
which means that it still may be possible for the compiler to 
detect the situation.

This is more complicated in the case where a user defined struct 
was passed by ref, and the ref return type is a member from that 
struct, but it still may be possible to detect it.

If address matching is possible (or is that determined only at 
link time?) then it may be possible to detect a situation that 
should be illegal and flagged as a compiler error.

--rt
December 31, 2012
Re: ref is unsafe
On Monday, December 31, 2012 02:37:56 Rob T wrote:
> This may be far fetched, but consider this,
> 
> If a function returns a ref that is the same as what was passed
> in by ref, then the passed and return addresses would match,
> which means that it still may be possible for the compiler to
> detect the situation.
> 
> This is more complicated in the case where a user defined struct
> was passed by ref, and the ref return type is a member from that
> struct, but it still may be possible to detect it.
> 
> If address matching is possible (or is that determined only at
> link time?) then it may be possible to detect a situation that
> should be illegal and flagged as a compiler error.

Addresses would only be known at runtime, and it's far too late at that point.

- Jonathan M Davis
December 31, 2012
Re: ref is unsafe
/*
The implementation of delegates has solved an analogous problem. 
e.g.
*/
import std.stdio;

auto getfun( int x) {
	int y = x*x;
	int ysquared() {
		return y*y;
	}
	return &ysquared;
}

void main() {
	auto f1 = getfun(2);
	auto f2 = getfun(3);
	writeln( f1() );
	writeln( f2() );
}
/*
The variable y no longer exists local to getfun, but its 
existence is prolonged making its use safe.

Just like _immutable_, some clever compiler inference that is 
transitive plus a delegate-like solution may do the job with ref 
without imposing constraints upon the sane. Plus, it may lead to 
new terrain when the liberation of local variables as above 
occurs in this context.

Warning: this is all speculation.
*/
December 31, 2012
Re: ref is unsafe
On Monday, December 31, 2012 03:34:18 Carl Sturtivant wrote:
> The implementation of delegates has solved an analogous problem.

Delegates use closures. The stack of the calling function is copied onto the 
heap so that it will continue to be valid for the delegate after the function 
returns. We don't want to be doing anything like that with ref, and it's 
generally, completely unnecessary. It's just that there are cases where you 
can escape such references if you're not careful, and the compiler currently 
considers those to be @safe, when they're not actually safe. So, we presumably 
need to do one of

1. Limit what can be legally done with ref to get rid of the problem.

2. Make ref @system in cases where we can't prove that it's @safe and try and 
prove it to be @safe in as many situations as possible.

3. Create a new attribute which has to be used when a function returns a ref 
to a parameter and use that to make it illegal to pass a ref to a local 
variable to such functions.

4. Something else that similarly protects against this at compile time without 
any extra overhead at runtime.

I really don't think that something like closures would be acceptable for ref 
parameters.

- Jonathan M Davis
December 31, 2012
Re: ref is unsafe
On 2012-12-30 22:29:33 +0000, Jonathan M Davis <jmdavisProg@gmx.com> said:

> Good point. Member variables of parameters also cause problems. So, it very
> quickly devolves to any function which accepts a user-defined type by ref and
> returns anything by ref would have to be @system, which is far from pleasant.

Note that the above definition includes every struct member function 
that returns a ref because of the implicit this parameter.

Also, it's not just functions returning by ref, it could be a function 
returning a delegate too, if the delegate happens to make use of the 
reference:

void delegate() foo(ref int a) {
   return { writeln(a); };
}

void delegate() bar() {
   int a;
   return foo(a); // leaking reference to a beyond bar's scope
}

And similar to passing a value by ref: you can pass a slice to a static 
array, then return the slice:

int[] foo(int[] a) {
	return a;
}

int[] bar() {
	int[2] a;
	return foo(a[]);
}

Three variations on the same theme.

-- 
Michel Fortin
michel.fortin@michelf.ca
http://michelf.ca/
December 31, 2012
Re: ref is unsafe
On Monday, 31 December 2012 at 02:47:46 UTC, Jonathan M Davis 
wrote:

> 3. Create a new attribute which has to be used when a function 
> returns a ref
> to a parameter and use that to make it illegal to pass a ref to 
> a local
> variable to such functions.

If this is the way to go, maybe "@saferef" could double as both 
@safe and @inoutref.

[OT] I've not been here for a while, but I've been reading up on 
the D boards again. I might want to help with the standard 
library lexer and parser. Happy New Year...
December 31, 2012
Re: ref is unsafe
On 30/12/2012 22:01, Jonathan M Davis wrote:
> On Sunday, December 30, 2012 17:32:40 Nick Treleaven wrote:
>> I think the compiler needs to be able to mark foo as a function that
>> returns its input reference. Then, any arguments to foo that are locals
>> should cause an error at the call site (e.g. in baz). So legal calls to
>> foo can always be @safe.
>>
>> To extend the above code:
>>
>> ref int quux(ref int i)
>> {
>>       return foo(i);
>> }
>>
>> Here the compiler already knows that foo returns its input reference. So
>> it checks whether foo is being passed a local - no; but it also has to
>> check if foo is passed any ref parameters of quux, which it is. The
>> compiler now has to mark quux as a function that returns its input
>> reference.
>>
>> Works?
>
> No. There's no guarantee that the compiler has access to the function's body,
> and the function being called could be compiled after the function which calls
> it. There's a reason that attribute inferrence only works with templated
> functions. In every other case, the programmer has to mark it. We're _not_
> going to get any kind inferrence without templates. D's compilation model
> doesn't allow it.

I was aware attributes would be needed for .di files. I suppose 
attribute inference for non-template functions is not doable.

> The closest that we could get to what you suggest would be to add a new
> attribute similar to nothrow but which guarantees that the function does not
> return a ref to a parameter. So, you'd have to mark your functions that way
> (e.g. with @norefparamreturn). Maybe the compiler could infer it for templated
> ones, but this attribute would basically have to work like other inferred
> attributes and be marked manually in all other cases. Certainly, you can't
> have the compiler figuring it out for you in general, because D's compilation
> model allows the function being called to be compiled separately from (and
> potentially after) the function calling it.

As you suggested below that, I would have the attribute mean the 
opposite, @refparamreturn. Functions that need it but don't have it can 
be detected by recompiling them. The syntax could be 'in ref':

in ref int quux(ref int i);

> And when you think about what this attribute would be needed for, it gets a
> bit bizarre to have it. The _only_ time that it's applicable is when a
> function takes an argument by ref and returns the same type by ref. In all
> other cases, the compiler can guarantee it just based on the type system.

As jerro and Michel Fortin pointed out, 'in ref' would be needed for 
returning input struct members, and capturing inputs with delegates and 
slices. So the feature might pull its weight.

I think detecting all these situations would be essentially the same as 
the checks needed for a scope parameter, so if/when scope parameters get 
implemented, 'in ref' might not be hard to bolt on.

If a simpler solution doesn't disallow any sensible uses of ref returns, 
that would be preferable. But I don't think we've found it yet.

> I suppose that we could have an attribute that indicated that a function _did_
> return a ref to one of its params and then have the compiler give an error if
> it were missing, which means that the foo function
>
> ref int foo(ref int i)
> {
>      return i;
> }
>
> would end up with an error for not having the attribute, whereas a function
> like baz
>
> ref int baz(int i)
> {
>      return foo(i);
> }
>
> would not end up with the error unless foo had the attribute on it. But that's
> very different from any attribute that we currently have. It would be like
> having a throw attribute instead of a nothrow attribute. I suppose that it is
> a possible solution though. I could also see an argument that the attribute
> should go on the parameter rather than the function, in which case you could
> have more fine-grained control over it, but it does complicate things further.

I suppose a parameter attribute might be useful to allow passing locals 
to other ref parameters which aren't returned, and as documentation.
December 31, 2012
Re: ref is unsafe
On 31/12/2012 14:44, Nick Treleaven wrote:
> On 30/12/2012 22:01, Jonathan M Davis wrote:
>> The closest that we could get to what you suggest would be to add a new
>> attribute similar to nothrow but which guarantees that the function
>> does not
>> return a ref to a parameter. So, you'd have to mark your functions
>> that way
>> (e.g. with @norefparamreturn). Maybe the compiler could infer it for
>> templated
>> ones, but this attribute would basically have to work like other inferred
>> attributes and be marked manually in all other cases. Certainly, you
>> can't
>> have the compiler figuring it out for you in general, because D's
>> compilation
>> model allows the function being called to be compiled separately from
>> (and
>> potentially after) the function calling it.
>
> As you suggested below that, I would have the attribute mean the
> opposite, @refparamreturn. Functions that need it but don't have it can
> be detected by recompiling them. The syntax could be 'in ref':
>
> in ref int quux(ref int i);

Actually overloading 'in ref' with a different meaning on return type 
from parameter (const scope) is probably a bad idea. Instead:

ref int quux(@escape ref int i);

[...]
> I suppose a parameter attribute might be useful to allow passing locals
> to other ref parameters which aren't returned, and as documentation.
>
1 2 3 4 5 6
Top | Discussion index | About this forum | D home