View mode: basic / threaded / horizontal-split · Log in · Help
January 03, 2013
Re: ref is unsafe
On Sunday, 30 December 2012 at 08:38:27 UTC, Jonathan M Davis 
wrote:
> And maybe another solution which I can't think of at the moment 
> would be
> better. But my point is that we currently have a _major_ hole 
> in SafeD thanks
> to the combination of ref parameters and ref return types, and 
> we need to find
> a solution.
>
> - Jonathan M Davis
>
>
> Related: http://d.puremagic.com/issues/show_bug.cgi?id=8838

I've thought about how I think the attributes should work if D is 
forced to use them. This was the first system I came up with, but 
as you'll see below, the system can be simplified by ignoring 
@safe-ty altogether:

Two attributes: @saferef and @inoutref

// "@saferef" is semantically equivalent to "@safe @inoutref"
@saferef ref int fupz(ref int a)
{
  somethingUnsafe(); // Error
  return a;  //Okay
}

// The same function won't work with just @safe
@safe ref int fuz(ref int a)
{
  return a; // Error: a @safe function which returns a reference 
to
            // a variable deriving from one of its parameters 
must be
            // marked @saferef
}

// Basic rule against using it when not necessary:
// a @saferef or @inoutref function must both accept and return a 
ref
@saferef int validate1(ref int a) { return a; } // Error
@inoutref ref int validate2(int a) { return a; } // Error

// @saferef's are chained by compiler enforcement:
@saferef ref int fonz(ref int a) { return a; }
@safe ref int frooz(ref int a)
{
  return fonz(a); // Error: a function which returns the result 
of one of
                  // its parameters being passed to a @saferef or 
@inoutref
                  // function must itself be marked @saferef or 
@inoutref
}

// The problem of escaping local variables:
@saferef ref int fonz(ref int a) { return a; }
ref int dollop()
{
  int local;
  return fonz(local); // Error: a function may not return the 
result of a local variable passed to a @saferef or an @inoutref 
function
}

// @inoutref may be used when you have otherwise un-safe code:
@inoutref ref int froes(ref int a)
{
  /+…some unsafe code…+/
  return a;
}
ref int f()
{
  int local;
  return froes(local); // Bug caught now even in @system code
}

// An enhancement: mark harmless parameters as @saferef
@saferef ref int twoParams(@saferef ref int a, ref int b)
{
  return a; // Error: a @saferef or @inoutref function may not 
return a reference derived from a parameter marked @saferef
  return b; // Fine
}

// Only @saferef or @inoutref functions would be able to use 
@saferef parameters:
ref int zorf(@saferef ref int a, ref int b) {} // Error


So I typed all of that out and realized that a simpler 
alternative would be to ignore @safe altogether and have the 
@inoutref functionality be on by default. The only attribute now 
required would be @outref, which could be simplified to just 
"out" so long as it appeared *before* the parameter list, since 
it could be confused for an out contract if it came afterwards.

So:

"@saferef" <=> "@safe @outref" is unnecessary because all 
functions are checked, not just @safe ones.

ref int lugs(ref int a)
{
  return a; // Okay
}

ref int h(ref int a)
{
  return lugs(a); // Okay

  int local;
  return lugs(local); // Error: may not return the result of a 
local variable
                      // passed to a function which both accepts 
and returns a
		      // ref unless that function is marked "@outref"
}

int d;
@outref ref int saml(ref int a)
{
  return *(new int); // Fine
  return d; // Fine

  return a; // Error: a function marked "@outref" may not return 
a reference
            // deriving from one of its parameters
}

ref int lugs(ref int a) { return a; }

@outref ref int druh(ref int a)
{
  return lugs(a); // Error: a function marked @outref may not 
return the result
                  // of one of its parameters being passed to a 
function unless
                  // that function is itself marked @outref
}

// Must both accept and return a reference
@outref int boops(ref int a) {} // Error
@outref ref int bop(int a) {} // Error

// Harmless parameters may be marked @trusted:
@outref ref int lit(@trusted ref int a, ref int b)
{
  return a; // Passes based on the honor system
  return b; // Error
}

The second system is much simpler, and it's only a little more 
computationally expensive than the first, since the signature of 
all functions called with local variables must be scanned for ref 
output and input, not just safe ones.
January 03, 2013
Re: ref is unsafe
On 01/03/2013 01:52 PM, Jason House wrote:
> On Thursday, 3 January 2013 at 05:56:27 UTC, Timon Gehr wrote:
>> On 01/03/2013 12:48 AM, Jason House wrote:
>>> ...
>>>
>>>>
>>>> ref int bar()
>>>> {
>>>>    int i = 7;
>>>>    return foo(i);
>>>> }
>>>
>>> If @safe, this code will not compile.
>>> Error: foo may return a local stack variable
>>> Since "i" is a local variable, "foo(i)" might return it.
>>>
>>>
>>>>
>>>> ref int baz(int i)
>>>> {
>>>>    return foo(i);
>>>> }
>>>
>>> This function is fine. "i" is an input argument so "foo(i)" is
>>> considered to be equivalent to an input argument.
>>>
>>
>> Those two cases are pretty much the same.
>
> If what I suggest is done, they must be differentiated. If you replace
> "return foo(i)" with "return i", the compiler will already issue an
> error for the local variable case.

Obviously _both_ examples result in memory corruption. i is not a ref 
parameter.
January 03, 2013
Re: ref is unsafe
On Sunday, 30 December 2012 at 08:38:27 UTC, Jonathan M Davis 
wrote:
> After some recent discussions relating to auto ref and const 
> ref, I have come
> to the conlusion that as it stands, ref is not @safe. It's 
> @system. And I
> think that we need to take a serious look at it to see what we 
> can do to make
> it @safe. The problem is combining code that takes ref 
> parameters with code
> that returns by ref. Take this code for example:
>
> ref int foo(ref int i)
> {
>     return i;
> }
>
> ref int bar()
> {
>     int i = 7;
>     return foo(i);
> }
>
> ref int baz(int i)
> {
>     return foo(i);
> }
>
> void main()
> {
>     auto a = bar();
>     auto b = baz(5);
> }

I must admit that I haven't read the rest of the thread yet, but 
I think the obvious and correct solution is to disallow passing 
locals (including non-ref parameters, which are effectively 
locals in D) as non-scope ref arguments.

The scope attribute, once properly implemented, would make sure 
that the reference is not escaped. For now, we could just make it 
behave overly conservative in @safe code.

David
January 03, 2013
Re: ref is unsafe
On Thursday, 3 January 2013 at 21:56:22 UTC, David Nadlinger 
wrote:
> I must admit that I haven't read the rest of the thread yet, 
> but I think the obvious and correct solution is to disallow 
> passing locals (including non-ref parameters, which are 
> effectively locals in D) as non-scope ref arguments.

The problem with that idea, is that a ref return with no 
arguments may call another ref return that returns something that 
escapes the scope it was created in. If the source code is not 
available, then there's no way for the compiler to determine that 
this is going on.

I would suggest to disallow all ref returns that make use of a 
ref return function call *unless* the code portion is marked as 
@trusted, and to to that requires following the ideas presented 
for changing how @trusted should be implemented, ie allowing 
selected portions of otherwise unsafe code to be marked as 
trusted by a programmer who has verified the use of the code to 
be safe given the context.

> The scope attribute, once properly implemented, would make sure 
> that the reference is not escaped. For now, we could just make 
> it behave overly conservative in @safe code.
>
> David

My understanding was that in some cases that source code is not 
available to the compiler, which I would think means that 
preventing scope escaping cannot be 100% guaranteed, correct?

--rt
January 03, 2013
Re: ref is unsafe
On Thursday, 3 January 2013 at 21:56:22 UTC, David Nadlinger 
wrote:
> I must admit that I haven't read the rest of the thread yet, 
> but I think the obvious and correct solution is to disallow 
> passing locals (including non-ref parameters, which are 
> effectively locals in D) as non-scope ref arguments.
>
> The scope attribute, once properly implemented, would make sure 
> that the reference is not escaped. For now, we could just make 
> it behave overly conservative in @safe code.
>

This seems to me like the sane thing to do.
January 03, 2013
Re: ref is unsafe
On Thursday, 3 January 2013 at 22:50:38 UTC, Rob T wrote:
> On Thursday, 3 January 2013 at 21:56:22 UTC, David Nadlinger 
> wrote:
>> I must admit that I haven't read the rest of the thread yet, 
>> but I think the obvious and correct solution is to disallow 
>> passing locals (including non-ref parameters, which are 
>> effectively locals in D) as non-scope ref arguments.
>
> The problem with that idea, is that a ref return with no 
> arguments may call another ref return that returns something 
> that escapes the scope it was created in. If the source code is 
> not available, then there's no way for the compiler to 
> determine that this is going on.

I am not quite sure what you are trying to say. If the compiler 
never sees the source code for the functions, then codegen is 
going to be difficult. ;)

Yes, if you just see "void iPromiseNotToEscapeMyParameter(scope 
ref int a) @safe;", then there is no way to directly check that 
the function actually does not leak the parameter address. 
However, you can be sure that the compiler checked that when 
generating the code for the function.

David
January 03, 2013
Re: ref is unsafe
On Thursday, 3 January 2013 at 22:50:38 UTC, Rob T wrote:
> The problem with that idea, is that a ref return with no 
> arguments may call another ref return that returns something 
> that escapes the scope it was created in. If the source code is 
> not available, then there's no way for the compiler to 
> determine that this is going on.
>

You can't return a scope ref in @safe code, so that is not an 
issue.

> I would suggest to disallow all ref returns that make use of a 
> ref return function call *unless* the code portion is marked as 
> @trusted, and to to that requires following the ideas presented 
> for changing how @trusted should be implemented, ie allowing 
> selected portions of otherwise unsafe code to be marked as 
> trusted by a programmer who has verified the use of the code to 
> be safe given the context.
>
>> The scope attribute, once properly implemented, would make 
>> sure that the reference is not escaped. For now, we could just 
>> make it behave overly conservative in @safe code.
>>
>> David
>
> My understanding was that in some cases that source code is not 
> available to the compiler, which I would think means that 
> preventing scope escaping cannot be 100% guaranteed, correct?
>

This is why the scope qualifier exists.
January 03, 2013
Re: ref is unsafe
On Thursday, January 03, 2013 23:50:37 Rob T wrote:
> My understanding was that in some cases that source code is not
> available to the compiler, which I would think means that
> preventing scope escaping cannot be 100% guaranteed, correct?

The source code is always available when compiling the function itself. So 
(assuming that scope is fully implemented - which it's not right now), the 
compiler will be able to verify that a scope parameter does not escape the 
function when it compiles that function.

What doesn't work is inferring function attributes at the call site, because 
that requires that the full code be available at the call site. And that's not 
necessarily true unless you're dealing with a templated function (which is 
part of why attribute inferrence only works with templated functions). But as 
long as you're talking about stuff that can be verified when the function itself 
is compiled, then the fact that the source code isn't necessarily available to 
the caller isn't an issue.

- Jonathan M Davis
January 03, 2013
Re: ref is unsafe
OK, I understand what you mean by "scope" and how that can be 
used to prevent leaking a local ref out.

Don't forget to consider this kind of scenario, which has no ref 
arguments to consider

struct X
{
   int _i;
   ref int f()
   {
      return _i;
   }
}

ref int F()
{

   X x;
   return x.f();
}

int main()
{
    // example uses that currently compile
    F = 1000;
    writeln(F());
}

Is this valid? Does local x remain defined up until the function 
call terminates completely, ie until after the reference is no 
longer valid?

I can also mark everything as @safe and it will compile, and also 
scope x

@safe ref int F()
{

   scope X x;
   return x.f();

   // this compiles too

   return x._i;

}

--rt
January 04, 2013
Re: ref is unsafe
On Thursday, 3 January 2013 at 23:06:03 UTC, David Nadlinger 
wrote:
>> The problem with that idea, is that a ref return with no 
>> arguments may call another ref return that returns something 
>> that escapes the scope it was created in. If the source code 
>> is not available, then there's no way for the compiler to 
>> determine that this is going on.
>
> I am not quite sure what you are trying to say. If the compiler 
> never sees the source code for the functions, then codegen is 
> going to be difficult. ;)
>

See my post directly above this one that has an example.

--rt
1 2 3 4 5 6 7
Top | Discussion index | About this forum | D home