View mode: basic / threaded / horizontal-split · Log in · Help
January 04, 2013
Re: ref is unsafe
On Wednesday, 2 January 2013 at 23:33:16 UTC, Thiez wrote:
> On Wednesday, 2 January 2013 at 22:53:04 UTC, Jonathan M Davis 
> wrote:
>> Then we're going to have to disagree, and I believe that 
>> Walter and Andrei are
>> completely with me on this one. If all of the constructs that 
>> you use are
>> @safe, then it should be _guaranteed_ that your program is 
>> memory-safe. That's
>> what @safe is for. Yes, it can be gotten around if the 
>> programmer marks
>> @system code as @trusted when it's not really memory-safe, but 
>> that's the
>> programmer's problem. @safe is not doing it's job and is 
>> completely pointless
>> if it has any holes in it beyond programmers mislabeling 
>> functions as @trusted.
>> - Jonathan M Davis
>
> Perhaps it is worth looking at Rust for this problem?

You can also look at how Algol solved this over 40 years ago: 
Insert a runtime check that the escaping reference does not point 
to the current stack frame which is about to be destroyed. The 
check should be very cheap at runtime but it can be deactivated 
in a release build for efficiency just like it is done for array 
indexing.

FYI Nimrod has the same problem and it's planned to prevent these 
cases statically with a type based alias analysis; however at 
least the first versions will still keep the dynamic check as 
these kind of static analyses cry for correctness proofs IMO.
January 04, 2013
Re: ref is unsafe
On Friday, 4 January 2013 at 00:46:33 UTC, Araq wrote:
> On Wednesday, 2 January 2013 at 23:33:16 UTC, Thiez wrote:
>> On Wednesday, 2 January 2013 at 22:53:04 UTC, Jonathan M Davis 
>> wrote:
>>> Then we're going to have to disagree, and I believe that 
>>> Walter and Andrei are
>>> completely with me on this one. If all of the constructs that 
>>> you use are
>>> @safe, then it should be _guaranteed_ that your program is 
>>> memory-safe. That's
>>> what @safe is for. Yes, it can be gotten around if the 
>>> programmer marks
>>> @system code as @trusted when it's not really memory-safe, 
>>> but that's the
>>> programmer's problem. @safe is not doing it's job and is 
>>> completely pointless
>>> if it has any holes in it beyond programmers mislabeling 
>>> functions as @trusted.
>>> - Jonathan M Davis
>>
>> Perhaps it is worth looking at Rust for this problem?
>
> You can also look at how Algol solved this over 40 years ago: 
> Insert a runtime check that the escaping reference does not 
> point to the current stack frame which is about to be 
> destroyed. The check should be very cheap at runtime but it can 
> be deactivated in a release build for efficiency just like it 
> is done for array indexing.
>
> FYI Nimrod has the same problem and it's planned to prevent 
> these cases statically with a type based alias analysis; 
> however at least the first versions will still keep the dynamic 
> check as these kind of static analyses cry for correctness 
> proofs IMO.

I did suggest something like that, and it may be a good idea to 
implement as a debugging aid (like runtime range checking). I 
wonder how difficult it would be to implement?

Unfortunately, it does not help solve the @safe compile time 
checks.

--rt
January 04, 2013
Re: ref is unsafe
Am 03.01.2013 00:48, schrieb Jason House:
> On Sunday, 30 December 2012 at 08:38:27 UTC, Jonathan M Davis wrote:
>> After some recent discussions relating to auto ref and const ref, I have come
>> to the conlusion that as it stands, ref is not @safe. It's @system. And I
>> think that we need to take a serious look at it to see what we can do to make
>> it @safe. The problem is combining code that takes ref parameters with code
>> that returns by ref.
> 
> The best solution I can think of is for the @safe code to require a ref return value is treated with
> the same care as all the function input arguments. I'll try to annotate the example code you gave to
> explain.
> 

+1

In other words, references returned by a function call that took any references to locals would be
tainted as possibly local (in the function local data flow) and thus are not allowed to escape the
scope. References derived from non-local refs could still be returned and returning references to
fields from a struct method also works.

---
@safe ref int test(ref int v) {
	return v; // fine
}

@safe ref int test2() {
	int local;
	return test(local); // error: (possibly) returning ref to local
}

@safe ref int test3() {
	int local;
	int* ptr = &test(local); // fine, ptr is tainted 'local'
	return *ptr; // error: (possibly) returning ref to local
}

@safe ref int test4(ref int val) {
	return test(val); // fine, can only be a ref to the external 'val' or to a global
}
---
January 04, 2013
Re: ref is unsafe
On Friday, 4 January 2013 at 06:30:55 UTC, Sönke Ludwig wrote:
> In other words, references returned by a function call that 
> took any references to locals would be
> tainted as possibly local (in the function local data flow) and 
> thus are not allowed to escape the
> scope. References derived from non-local refs could still be 
> returned and returning references to
> fields from a struct method also works.
>
> ---
> @safe ref int test(ref int v) {
> 	return v; // fine
> }
>

v should be scope here. If not, other function have no guarantee 
that the reference will not escape.

> @safe ref int test2() {
> 	int local;
> 	return test(local); // error: (possibly) returning ref to local
> }
>
> @safe ref int test3() {
> 	int local;
> 	int* ptr = &test(local); // fine, ptr is tainted 'local'
> 	return *ptr; // error: (possibly) returning ref to local
> }
>
> @safe ref int test4(ref int val) {
> 	return test(val); // fine, can only be a ref to the external 
> 'val' or to a global
> }
> ---

Given the modification mentioned above, this look like the way to 
go.
January 04, 2013
Re: ref is unsafe
On Friday, 4 January 2013 at 06:30:55 UTC, Sönke Ludwig wrote:
> In other words, references returned by a function call that 
> took any references to locals would be
> tainted as possibly local (in the function local data flow) and 
> thus are not allowed to escape the
> scope. References derived from non-local refs could still be 
> returned and returning references to
> fields from a struct method also works.
>
> ---
> @safe ref int test(ref int v) {
> 	return v; // fine
> }
>
> @safe ref int test2() {
> 	int local;
> 	return test(local); // error: (possibly) returning ref to local
> }
>
> @safe ref int test3() {
> 	int local;
> 	int* ptr = &test(local); // fine, ptr is tainted 'local'
> 	return *ptr; // error: (possibly) returning ref to local
> }
>
> @safe ref int test4(ref int val) {
> 	return test(val); // fine, can only be a ref to the external 
> 'val' or to a global
> }
> ---

Trying to say that formally:

Definitions:
'Tainter function':
    A function that:
    1. takes at least one of its parameters by reference
    and
    2. returns by reference
	
'Tainting function call':
    A call to a 'tainter function' where at least one of the
    arguments passed by reference is ref to a local variable

Then the rules become:
Function may not return a reference to:
Rule 1: a function-local variable
Rule 2. a value returned by a 'tainting function call'

@safe:

ref int tfun(ref int v) { // tfun tagged 'tainter function'
    ...
}

ref int test1() {
    int local;
    return local; // error by Rule 1
}

ref int test2() {
    int local;
    return tfun(local); // error by Rule 2
}

ref int test3() {
    int local;
    int* ptr = &tfun(local); // ptr tagged 'local'
    return *ptr; // error by Rule 2
}

ref int test4(ref int val) {
    return tfun(val); // fine
}

int global;

ref int test5() {
    int local;
    int* ptr = &tfun(local); // ptr tagged 'local'
    ptr = &global; // ptr's 'local' tag removed
    return *ptr; // fine
}
January 04, 2013
Re: ref is unsafe
On Friday, 4 January 2013 at 14:15:01 UTC, Tommi wrote:
> 'Tainting function call':
>     A call to a 'tainter function' where at least one of the
>     arguments passed by reference is ref to a local variable

I forgot to point out that the return value of a 'tainting 
function call' is considered to be a "reference to a 
function-local variable" (even if it's not in reality).
January 04, 2013
Re: ref is unsafe
On Sunday, 30 December 2012 at 22:02:16 UTC, Jonathan M Davis 
wrote:
> The closest that we could get to what you suggest would be to 
> add a new
> attribute similar to nothrow but which guarantees that the 
> function does not
> return a ref to a parameter. So, you'd have to mark your 
> functions that way
> (e.g. with @norefparamreturn). Maybe the compiler could infer 
> it for templated
> ones, but this attribute would basically have to work like 
> other inferred
> attributes and be marked manually in all other cases. 
> Certainly, you can't
> have the compiler figuring it out for you in general, because 
> D's compilation
> model allows the function being called to be compiled 
> separately from (and
> potentially after) the function calling it.
>
> And when you think about what this attribute would be needed 
> for, it gets a
> bit bizarre to have it. The _only_ time that it's applicable is 
> when a
> function takes an argument by ref and returns the same type by 
> ref. In all
> other cases, the compiler can guarantee it just based on the 
> type system.

I realized just now that it's also applicable to member functions:
struct F
{
  int _i;
  ref int ser() { return _i; } // Needs to be marked as well
}

A struct's fields are implicit parameters in anything it returns.

> Honestly though, I'm inclined to argue that functions which 
> return by ref and
> have a ref parameter of that same type just be considered 
> @system.

Structs mess that up as well:
struct S { int i; }
ref int d(ref S s)
{
  return s.i;
}
January 04, 2013
Re: ref is unsafe
On Friday, January 04, 2013 17:26:59 Zach the Mystic wrote:
> > Honestly though, I'm inclined to argue that functions which
> > return by ref and
> > have a ref parameter of that same type just be considered
> > @system.
> 
> Structs mess that up as well:
> struct S { int i; }
> ref int d(ref S s)
> {
> return s.i;
> }

Yes. That's a function which takes a ref and returns by ref just like I said. 
It's just that in this case, the ref returned isn't the full object that was 
passed by ref but just a portion of it. What that means is that you can't 
assume that the ref being returned is safe just because the type of the 
parameter and the return type aren't the same. But it doesn't change the 
statement that a function which takes a parameter by ref and returns by ref 
can't be considered @safe without additional constraints of some kind. It just 
shows why you don't have an easy way out to make many of them @safe based on 
the differing types involved.

- Jonathan M Davis
January 05, 2013
Re: ref is unsafe
On Friday, 4 January 2013 at 20:20:08 UTC, Jonathan M Davis wrote:
> ... But it doesn't change the
> statement that a function which takes a parameter by ref and 
> returns by ref
> can't be considered @safe without additional constraints of 
> some kind. It just
> shows why you don't have an easy way out to make many of them 
> @safe based on
> the differing types involved.

Well, I've been working on just that. I'll have it for you 
tomorrow, I think.
January 05, 2013
Re: ref is unsafe
I've here formalized how I think the constraints on a non-scope 
ref taking and ref returning function should work. This 
represents a whole addition to the type system. The attribute 
"@outref" from my previous post has been shortened to keyword 
"out" (must come before parentheses). This is all I have left to 
say about this topic:

ref int lugs(ref int a) { return a; }
ref int h(ref int a)
{
  return lugs(a); // Okay

  int local;
  return lugs(local); // Error: the result of a function which 
accepts a local as non-scope ref and returns ref is treated as 
local and cannot be escaped unless that function is marked "out"

  int* p = &lugs(local); // Same error
}

int d;
out ref int saml(ref int a)
{
  return *(new int); // Fine
  return d; // Fine

  return a; // Error: a function marked "out" may not escape a 
non-scope ref parameter
}

ref int lugh(ref int a) { return a; }
out ref int druh(ref int a)
{
  return lugh(a); // Error: a function marked "out" may not 
escape the result of a function which accepts its non-scope ref 
parameter and returns a ref unless that function is also marked 
"out"
}

out int boops(ref int a) {} // Error: a function marked "out" 
must return a reference
out ref int bop(int a, in ref b, scope ref c) {} // Error: a 
non-member function marked "out" must accept at least one 
non-scope ref parameter

// "cast(out)" provides all needed flexibility:
out ref int lit(ref int a)
{
  return cast(out) a; // Not @safe

  // But with @trusted blocks, we could do:
  @trusted { return cast(out) a; } // @safe

  // And with @trusted statements, the brackets are gone:
  @trusted return cast(out) a; // @safe

  // Otherwise, this function must be marked "@trusted"
}

// You can use cast(out) anywhere:
ref int hugs(ref int a) { return a; }
ref int g(ref int a)
{
  int local;
  return cast(out) (hugs(local)); // Okay
  return cast(out) local; // Okay??
  return hugs(cast(out) local); // Won't know what hit 'em
}

// Nor did I forget about structs:
struct S
{
  int _i;
  static int _s;
  out ref int club() { return _i; } // Error: a member function 
marked "out" may not escape a non-static field
  out ref int trob() { return _s; } // Okay
  out ref int blub() { return cast(out) _i; } //Okay
}

struct B { int _i; ref int snub() { return _i; } }
ref int bub()
{
  B b;
  return b.snub(); // Error: the result of a local instance's 
non-static method which returns ref is considered local and may 
not be escaped unless that method is marked "out"

  int* i = &b.snub(); // Same error
}
2 3 4 5 6 7
Top | Discussion index | About this forum | D home