On Sunday, 30 December 2012 at 08:38:27 UTC, Jonathan M Davis wrote:
> And maybe another solution which I can't think of at the moment would be
> better. But my point is that we currently have a _major_ hole in SafeD thanks
> to the combination of ref parameters and ref return types, and we need to find
> a solution.
>
> - Jonathan M Davis
>
>
> Related: http://d.puremagic.com/issues/show_bug.cgi?id=8838

I've thought about how I think the attributes should work if D is forced to use them. This was the first system I came up with, but as you'll see below, the system can be simplified by ignoring @safe-ty altogether:

Two attributes: @saferef and @inoutref

// "@saferef" is semantically equivalent to "@safe @inoutref"
@saferef ref int fupz(ref int a)
{
  somethingUnsafe(); // Error
  return a;  //Okay
}

// The same function won't work with just @safe
@safe ref int fuz(ref int a)
{
  return a; // Error: a @safe function which returns a reference to
            // a variable deriving from one of its parameters must be
            // marked @saferef
}

// Basic rule against using it when not necessary:
// a @saferef or @inoutref function must both accept and return a ref
@saferef int validate1(ref int a) { return a; } // Error
@inoutref ref int validate2(int a) { return a; } // Error

// @saferef's are chained by compiler enforcement:
@saferef ref int fonz(ref int a) { return a; }
@safe ref int frooz(ref int a)
{
  return fonz(a); // Error: a function which returns the result of one of
                  // its parameters being passed to a @saferef or @inoutref
                  // function must itself be marked @saferef or @inoutref
}

// The problem of escaping local variables:
@saferef ref int fonz(ref int a) { return a; }
ref int dollop()
{
  int local;
  return fonz(local); // Error: a function may not return the result of a local variable passed to a @saferef or an @inoutref function
}

// @inoutref may be used when you have otherwise un-safe code:
@inoutref ref int froes(ref int a)
{
  /+…some unsafe code…+/
  return a;
}
ref int f()
{
  int local;
  return froes(local); // Bug caught now even in @system code
}

// An enhancement: mark harmless parameters as @saferef
@saferef ref int twoParams(@saferef ref int a, ref int b)
{
  return a; // Error: a @saferef or @inoutref function may not return a reference derived from a parameter marked @saferef
  return b; // Fine
}

// Only @saferef or @inoutref functions would be able to use @saferef parameters:
ref int zorf(@saferef ref int a, ref int b) {} // Error


So I typed all of that out and realized that a simpler alternative would be to ignore @safe altogether and have the @inoutref functionality be on by default. The only attribute now required would be @outref, which could be simplified to just "out" so long as it appeared *before* the parameter list, since it could be confused for an out contract if it came afterwards.

So:

"@saferef" <=> "@safe @outref" is unnecessary because all functions are checked, not just @safe ones.

ref int lugs(ref int a)
{
  return a; // Okay
}

ref int h(ref int a)
{
  return lugs(a); // Okay

  int local;
  return lugs(local); // Error: may not return the result of a local variable
                      // passed to a function which both accepts and returns a
		      // ref unless that function is marked "@outref"
}

int d;
@outref ref int saml(ref int a)
{
  return *(new int); // Fine
  return d; // Fine

  return a; // Error: a function marked "@outref" may not return a reference
            // deriving from one of its parameters
}

ref int lugs(ref int a) { return a; }

@outref ref int druh(ref int a)
{
  return lugs(a); // Error: a function marked @outref may not return the result
                  // of one of its parameters being passed to a function unless
                  // that function is itself marked @outref
}

// Must both accept and return a reference
@outref int boops(ref int a) {} // Error
@outref ref int bop(int a) {} // Error

// Harmless parameters may be marked @trusted:
@outref ref int lit(@trusted ref int a, ref int b)
{
  return a; // Passes based on the honor system
  return b; // Error
}

The second system is much simpler, and it's only a little more computationally expensive than the first, since the signature of all functions called with local variables must be scanned for ref output and input, not just safe ones.

On 01/03/2013 01:52 PM, Jason House wrote:
> On Thursday, 3 January 2013 at 05:56:27 UTC, Timon Gehr wrote:
>> On 01/03/2013 12:48 AM, Jason House wrote:
>>> ...
>>>
>>>>
>>>> ref int bar()
>>>> {
>>>>    int i = 7;
>>>>    return foo(i);
>>>> }
>>>
>>> If @safe, this code will not compile.
>>> Error: foo may return a local stack variable
>>> Since "i" is a local variable, "foo(i)" might return it.
>>>
>>>
>>>>
>>>> ref int baz(int i)
>>>> {
>>>>    return foo(i);
>>>> }
>>>
>>> This function is fine. "i" is an input argument so "foo(i)" is
>>> considered to be equivalent to an input argument.
>>>
>>
>> Those two cases are pretty much the same.
>
> If what I suggest is done, they must be differentiated. If you replace
> "return foo(i)" with "return i", the compiler will already issue an
> error for the local variable case.

Obviously _both_ examples result in memory corruption. i is not a ref parameter.

On Sunday, 30 December 2012 at 08:38:27 UTC, Jonathan M Davis wrote:
> After some recent discussions relating to auto ref and const ref, I have come
> to the conlusion that as it stands, ref is not @safe. It's @system. And I
> think that we need to take a serious look at it to see what we can do to make
> it @safe. The problem is combining code that takes ref parameters with code
> that returns by ref. Take this code for example:
>
> ref int foo(ref int i)
> {
>     return i;
> }
>
> ref int bar()
> {
>     int i = 7;
>     return foo(i);
> }
>
> ref int baz(int i)
> {
>     return foo(i);
> }
>
> void main()
> {
>     auto a = bar();
>     auto b = baz(5);
> }

I must admit that I haven't read the rest of the thread yet, but I think the obvious and correct solution is to disallow passing locals (including non-ref parameters, which are effectively locals in D) as non-scope ref arguments.

The scope attribute, once properly implemented, would make sure that the reference is not escaped. For now, we could just make it behave overly conservative in @safe code.

David

On Thursday, 3 January 2013 at 21:56:22 UTC, David Nadlinger wrote:
> I must admit that I haven't read the rest of the thread yet, but I think the obvious and correct solution is to disallow passing locals (including non-ref parameters, which are effectively locals in D) as non-scope ref arguments.

The problem with that idea, is that a ref return with no arguments may call another ref return that returns something that escapes the scope it was created in. If the source code is not available, then there's no way for the compiler to determine that this is going on.

I would suggest to disallow all ref returns that make use of a ref return function call *unless* the code portion is marked as @trusted, and to to that requires following the ideas presented for changing how @trusted should be implemented, ie allowing selected portions of otherwise unsafe code to be marked as trusted by a programmer who has verified the use of the code to be safe given the context.

> The scope attribute, once properly implemented, would make sure that the reference is not escaped. For now, we could just make it behave overly conservative in @safe code.
>
> David

My understanding was that in some cases that source code is not available to the compiler, which I would think means that preventing scope escaping cannot be 100% guaranteed, correct?

--rt

On Thursday, 3 January 2013 at 21:56:22 UTC, David Nadlinger wrote:
> I must admit that I haven't read the rest of the thread yet, but I think the obvious and correct solution is to disallow passing locals (including non-ref parameters, which are effectively locals in D) as non-scope ref arguments.
>
> The scope attribute, once properly implemented, would make sure that the reference is not escaped. For now, we could just make it behave overly conservative in @safe code.
>

This seems to me like the sane thing to do.

On Thursday, 3 January 2013 at 22:50:38 UTC, Rob T wrote:
> On Thursday, 3 January 2013 at 21:56:22 UTC, David Nadlinger wrote:
>> I must admit that I haven't read the rest of the thread yet, but I think the obvious and correct solution is to disallow passing locals (including non-ref parameters, which are effectively locals in D) as non-scope ref arguments.
>
> The problem with that idea, is that a ref return with no arguments may call another ref return that returns something that escapes the scope it was created in. If the source code is not available, then there's no way for the compiler to determine that this is going on.

I am not quite sure what you are trying to say. If the compiler never sees the source code for the functions, then codegen is going to be difficult. ;)

Yes, if you just see "void iPromiseNotToEscapeMyParameter(scope ref int a) @safe;", then there is no way to directly check that the function actually does not leak the parameter address. However, you can be sure that the compiler checked that when generating the code for the function.

David

On Thursday, 3 January 2013 at 22:50:38 UTC, Rob T wrote:
> The problem with that idea, is that a ref return with no arguments may call another ref return that returns something that escapes the scope it was created in. If the source code is not available, then there's no way for the compiler to determine that this is going on.
>

You can't return a scope ref in @safe code, so that is not an issue.

> I would suggest to disallow all ref returns that make use of a ref return function call *unless* the code portion is marked as @trusted, and to to that requires following the ideas presented for changing how @trusted should be implemented, ie allowing selected portions of otherwise unsafe code to be marked as trusted by a programmer who has verified the use of the code to be safe given the context.
>
>> The scope attribute, once properly implemented, would make sure that the reference is not escaped. For now, we could just make it behave overly conservative in @safe code.
>>
>> David
>
> My understanding was that in some cases that source code is not available to the compiler, which I would think means that preventing scope escaping cannot be 100% guaranteed, correct?
>

This is why the scope qualifier exists.

On Thursday, January 03, 2013 23:50:37 Rob T wrote:
> My understanding was that in some cases that source code is not available to the compiler, which I would think means that preventing scope escaping cannot be 100% guaranteed, correct?

The source code is always available when compiling the function itself. So (assuming that scope is fully implemented - which it's not right now), the compiler will be able to verify that a scope parameter does not escape the function when it compiles that function.

What doesn't work is inferring function attributes at the call site, because that requires that the full code be available at the call site. And that's not necessarily true unless you're dealing with a templated function (which is part of why attribute inferrence only works with templated functions). But as long as you're talking about stuff that can be verified when the function itself is compiled, then the fact that the source code isn't necessarily available to the caller isn't an issue.

- Jonathan M Davis

OK, I understand what you mean by "scope" and how that can be used to prevent leaking a local ref out.

Don't forget to consider this kind of scenario, which has no ref arguments to consider

struct X
{
   int _i;
   ref int f()
   {
      return _i;
   }
}

ref int F()
{

   X x;
   return x.f();
}

int main()
{
    // example uses that currently compile
    F = 1000;
    writeln(F());
}

Is this valid? Does local x remain defined up until the function call terminates completely, ie until after the reference is no longer valid?

I can also mark everything as @safe and it will compile, and also scope x

@safe ref int F()
{

   scope X x;
   return x.f();

   // this compiles too

   return x._i;

}

--rt

On Thursday, 3 January 2013 at 23:06:03 UTC, David Nadlinger wrote:
>> The problem with that idea, is that a ref return with no arguments may call another ref return that returns something that escapes the scope it was created in. If the source code is not available, then there's no way for the compiler to determine that this is going on.
>
> I am not quite sure what you are trying to say. If the compiler never sees the source code for the functions, then codegen is going to be difficult. ;)
>

See my post directly above this one that has an example.

--rt