View mode: basic / threaded / horizontal-split · Log in · Help
September 13, 2012
Re: Would like to see ref and out required for function calls
On 9/13/12 6:58 AM, Namespace wrote:
> In D:
>
> class A { // or struct
> A(ref int host) : _host(&host) { }
> private:
> int* _host;
> }
>
> Since when do we have initialization lists in D? ;)

Oops, sorry :o).

> at topic: +1

One question Walter and I thought about a lot was whether we should 
disallow escaping addresses of ref parameters in general or only in 
@safe code. We decided to go for the latter in order to avoid breakage 
of existing code. Thoughts?

Andrei
September 13, 2012
Re: Would like to see ref and out required for function calls
On Thursday, 13 September 2012 at 10:34:00 UTC, Andrei 
Alexandrescu wrote:
> The thing is, however, that in this matter reasonable people 
> may disagree.


Sounds exactly like the C# lambda syntax situation.
September 13, 2012
Re: Would like to see ref and out required for function calls
> I don't think there would be problems with allowing ref/out 
> optionally at the call site. The thing is, however, that in 
> this matter reasonable people may disagree.
> I'd be unable to identify any pattern in engineers choosing one 
> preference over the other.

Maybe C++ fans prefer pointers or implicit ref, C# fans prefer 
call-site ref?

> Now that the subject has been broken, we do have good evidence 
> of a pattern that generates significant and difficult bugs: 
> escaping the address of a reference. In C++:
>
> struct A {
>     A(int& host) : host_(host) {}
> private:
>     int& host_;
> };
>
> In D:
>
> class A { // or struct
>     A(ref int host) : _host(&host) {}
> private:
>     int* _host;
> }
>
> A solution we use for C++ is to require escaped addresses to be 
> always passed as pointers or smart pointers.
>
> Walter and I have discussed this for quite a while. We have 
> recently decided to disallow, at least in SafeD, escaping the 
> address of a ref parameter. In the beginning we'll be overly 
> conservative by disallowing taking the address of a ref 
> altogether. I'll write a DIP on that soon.

Err, wouldn't that break a lot of stuff, a lot of which is 
actually safe code?

void a(ref int x) { b(&x); }
void b(int* x) { if(x != null) (*x)++; }

Escaping the addresses of stack variables, not just ref 
parameters, is a general problem in "safe" D. Do you have any 
ideas about that?
September 13, 2012
Re: Would like to see ref and out required for function calls
On 9/13/12 10:53 AM, David Piepgrass wrote:
>> Walter and I have discussed this for quite a while. We have recently
>> decided to disallow, at least in SafeD, escaping the address of a ref
>> parameter. In the beginning we'll be overly conservative by
>> disallowing taking the address of a ref altogether. I'll write a DIP
>> on that soon.
>
> Err, wouldn't that break a lot of stuff, a lot of which is actually safe
> code?
>
> void a(ref int x) { b(&x); }
> void b(int* x) { if(x != null) (*x)++; }

Yes. Disallowing taking the address of a local is conservative and would 
disallow a number of valid programs.

Arguably, such programs are in poor style anyway. A good program takes 
pointers only if it needs to keep them around; if all that's needed is 
to use the parameter transitorily or pass it down, ref is best.

> Escaping the addresses of stack variables, not just ref parameters, is a
> general problem in "safe" D. Do you have any ideas about that?

Same thing. By and large safe programs will need to make more use of the 
garbage collector than others. It's the way things work; stack 
allocation can be made safer if we add typed regions, but that's a very 
significant escalation of complication. There is no simple solution to 
this today.


Andrei
September 13, 2012
Re: Would like to see ref and out required for function calls
On Thursday, September 13, 2012 12:34:34 Andrei Alexandrescu wrote:
> I don't think there would be problems with allowing ref/out optionally
> at the call site. The thing is, however, that in this matter reasonable
> people may disagree.

I really think that optionally allowing ref and out at the call site is more 
damaging than beneficial. _Requiring_ it could be beneficial, since then you 
know that the arguments are being taken by ref, but if it's optional, it gives 
you a false sense of security and can be misleading. Presumabyl, any time that 
ref or out is used at a call site, presumably the compiler would check that it 
really was passed by ref our out, which would be of some benefit, but it would 
mean _nothing_ if ref or out wasn't used, since it could be an argument being 
passed by ref but which wasn't marked or an argument which wasn't passed by 
ref at all. So, if you have code where ref and out is used at the call site 
semi-religiously, then it'll become very easy to falsely think that the lack 
of them means that ref or out _isn't_ being used when it really is, misleading 
people about what the code is doing.

So, if it wasn't going to break so much code, I'd see some argument for 
changing things so that ref and out were required at the call site, but I 
think that _allowing_ them but not requiring them would actually be 
detrimental.

- Jonathan M Davis
September 13, 2012
Re: Would like to see ref and out required for function calls
On Thursday, 13 September 2012 at 15:01:28 UTC, Andrei 
Alexandrescu wrote:
> On 9/13/12 10:53 AM, David Piepgrass wrote:
>>> Walter and I have discussed this for quite a while. We have 
>>> recently
>>> decided to disallow, at least in SafeD, escaping the address 
>>> of a ref
>>> parameter. In the beginning we'll be overly conservative by
>>> disallowing taking the address of a ref altogether. I'll 
>>> write a DIP
>>> on that soon.
>>
>> Err, wouldn't that break a lot of stuff, a lot of which is 
>> actually safe
>> code?
>>
>> void a(ref int x) { b(&x); }
>> void b(int* x) { if(x != null) (*x)++; }
>
> Yes. Disallowing taking the address of a local is conservative 
> and would disallow a number of valid programs.
>
> Arguably, such programs are in poor style anyway. A good 
> program takes pointers only if it needs to keep them around; if 
> all that's needed is to use the parameter transitorily or pass 
> it down, ref is best.

Another common reason to use a pointer (instead of ref) is if 
it's optional (nullable). If the parameter is ref then the caller 
must go to the trouble of creating a variable.

However, this could be solved with a feature like the following:

int* find(string searchString, out int index) { ... }
// _ means "don't care", assuming no variable "_" is defined
void caller() { find("foo", out _); }

In fact this is arguably better for 'out' variables since the 
callee (find) no longer has to check whether 'index' is null 
before assigning it. However this doesn't totally solve the 
problem for 'ref' parameters, since such parameters are both 
output and input parameters and the programmer may want 'null' to 
have some special meaning as an input.

>> Escaping the addresses of stack variables, not just ref 
>> parameters, is a
>> general problem in "safe" D. Do you have any ideas about that?
Btw just a simple illustrative example:
int* unsafe1() { int x = 1; return unsafe2(&x); }
int* unsafe2(int* x) { return x; }
int unsafe3() { int y = 7; *unsafe1() = 8; return y; }
enum gaff = unsafe3(); // ICE, no line number given

> Same thing. By and large safe programs will need to make more 
> use of the garbage collector than others. It's the way things 
> work; stack allocation can be made safer if we add typed 
> regions, but that's a very significant escalation of 
> complication. There is no simple solution to this today.

Same thing meaning that you'd propose disallowing taking the 
address of a stack variable in SafeD? (I guess this would include 
escaping 'this' within a struct.)
September 13, 2012
Re: Would like to see ref and out required for function calls
> I really think that optionally allowing ref and out at the call 
> site is more
> damaging than beneficial. _Requiring_ it could be beneficial, 
> since then you
> know that the arguments are being taken by ref, but if it's 
> optional, it gives
> you a false sense of security and can be misleading.

It gives *who* a false sense of security? If it's optional then I 
*know* lack of ref/out doesn't imply that the parameter won't 
change. Only people who don't know the rules would have this 
false sense of security.

I think it would be nice to have it required, but it's very bad 
to break everyone's code. It could only be reasonably enforced 
with a compiler switch--or, wait, come to think of it, a pragma 
would probably be better way to introduce language changes like 
this:

module foo;
pragma(callSiteRef);
// and would it make sense to offer an alternative to -property 
too?
pragma(property);

Now you can tell whether a program uses ref/out religiously or 
not.
September 13, 2012
Re: Would like to see ref and out required for function calls
On 9/13/12 11:47 AM, David Piepgrass wrote:
>> I really think that optionally allowing ref and out at the call site
>> is more
>> damaging than beneficial. _Requiring_ it could be beneficial, since
>> then you
>> know that the arguments are being taken by ref, but if it's optional,
>> it gives
>> you a false sense of security and can be misleading.
>
> It gives *who* a false sense of security?

It's "whom".

Andrei

P.S. All of my life I've been waiting for this moment!!!
September 13, 2012
Re: Would like to see ref and out required for function calls
On Thursday, 13 September 2012 at 16:29:19 UTC, Andrei 
Alexandrescu wrote:
>> It gives *who* a false sense of security?
>
> It's "whom".
>
> Andrei
>
> P.S. All of my life I've been waiting for this moment!!!

LOL
September 13, 2012
Re: Would like to see ref and out required for function calls
On Sep 13, 2012, at 7:21 AM, Andrei Alexandrescu <SeeWebsiteForEmail@erdani.org> wrote:
> 
> One question Walter and I thought about a lot was whether we should disallow escaping addresses of ref parameters in general or only in @safe code. We decided to go for the latter in order to avoid breakage of existing code. Thoughts?

I love the idea.  However, this could complicate working with C APIs, particularly regarding structs which are often passed by reference as a matter of course.  For example:

struct some_c_struct {}

void fn(ref some_c_struct s) {
   some_c_func(&s);
}

I guess the only way to know if this will turn out to be a real issue is to give it a try though.
1 2 3 4 5 6 7
Top | Discussion index | About this forum | D home