On 9/13/12 6:58 AM, Namespace wrote:
> In D:
>
> class A { // or struct
> A(ref int host) : _host(&host) { }
> private:
> int* _host;
> }
>
> Since when do we have initialization lists in D? ;)

Oops, sorry :o).

> at topic: +1

One question Walter and I thought about a lot was whether we should disallow escaping addresses of ref parameters in general or only in @safe code. We decided to go for the latter in order to avoid breakage of existing code. Thoughts?

Andrei

On Thursday, 13 September 2012 at 10:34:00 UTC, Andrei Alexandrescu wrote:
> The thing is, however, that in this matter reasonable people may disagree.


Sounds exactly like the C# lambda syntax situation.

> I don't think there would be problems with allowing ref/out optionally at the call site. The thing is, however, that in this matter reasonable people may disagree.
> I'd be unable to identify any pattern in engineers choosing one preference over the other.

Maybe C++ fans prefer pointers or implicit ref, C# fans prefer call-site ref?

> Now that the subject has been broken, we do have good evidence of a pattern that generates significant and difficult bugs: escaping the address of a reference. In C++:
>
> struct A {
>     A(int& host) : host_(host) {}
> private:
>     int& host_;
> };
>
> In D:
>
> class A { // or struct
>     A(ref int host) : _host(&host) {}
> private:
>     int* _host;
> }
>
> A solution we use for C++ is to require escaped addresses to be always passed as pointers or smart pointers.
>
> Walter and I have discussed this for quite a while. We have recently decided to disallow, at least in SafeD, escaping the address of a ref parameter. In the beginning we'll be overly conservative by disallowing taking the address of a ref altogether. I'll write a DIP on that soon.

Err, wouldn't that break a lot of stuff, a lot of which is actually safe code?

void a(ref int x) { b(&x); }
void b(int* x) { if(x != null) (*x)++; }

Escaping the addresses of stack variables, not just ref parameters, is a general problem in "safe" D. Do you have any ideas about that?

On 9/13/12 10:53 AM, David Piepgrass wrote:
>> Walter and I have discussed this for quite a while. We have recently
>> decided to disallow, at least in SafeD, escaping the address of a ref
>> parameter. In the beginning we'll be overly conservative by
>> disallowing taking the address of a ref altogether. I'll write a DIP
>> on that soon.
>
> Err, wouldn't that break a lot of stuff, a lot of which is actually safe
> code?
>
> void a(ref int x) { b(&x); }
> void b(int* x) { if(x != null) (*x)++; }

Yes. Disallowing taking the address of a local is conservative and would disallow a number of valid programs.

Arguably, such programs are in poor style anyway. A good program takes pointers only if it needs to keep them around; if all that's needed is to use the parameter transitorily or pass it down, ref is best.

> Escaping the addresses of stack variables, not just ref parameters, is a
> general problem in "safe" D. Do you have any ideas about that?

Same thing. By and large safe programs will need to make more use of the garbage collector than others. It's the way things work; stack allocation can be made safer if we add typed regions, but that's a very significant escalation of complication. There is no simple solution to this today.


Andrei

On Thursday, September 13, 2012 12:34:34 Andrei Alexandrescu wrote:
> I don't think there would be problems with allowing ref/out optionally at the call site. The thing is, however, that in this matter reasonable people may disagree.

I really think that optionally allowing ref and out at the call site is more damaging than beneficial. _Requiring_ it could be beneficial, since then you know that the arguments are being taken by ref, but if it's optional, it gives you a false sense of security and can be misleading. Presumabyl, any time that ref or out is used at a call site, presumably the compiler would check that it really was passed by ref our out, which would be of some benefit, but it would mean _nothing_ if ref or out wasn't used, since it could be an argument being passed by ref but which wasn't marked or an argument which wasn't passed by ref at all. So, if you have code where ref and out is used at the call site semi-religiously, then it'll become very easy to falsely think that the lack of them means that ref or out _isn't_ being used when it really is, misleading people about what the code is doing.

So, if it wasn't going to break so much code, I'd see some argument for changing things so that ref and out were required at the call site, but I think that _allowing_ them but not requiring them would actually be detrimental.

- Jonathan M Davis

On Thursday, 13 September 2012 at 15:01:28 UTC, Andrei Alexandrescu wrote:
> On 9/13/12 10:53 AM, David Piepgrass wrote:
>>> Walter and I have discussed this for quite a while. We have recently
>>> decided to disallow, at least in SafeD, escaping the address of a ref
>>> parameter. In the beginning we'll be overly conservative by
>>> disallowing taking the address of a ref altogether. I'll write a DIP
>>> on that soon.
>>
>> Err, wouldn't that break a lot of stuff, a lot of which is actually safe
>> code?
>>
>> void a(ref int x) { b(&x); }
>> void b(int* x) { if(x != null) (*x)++; }
>
> Yes. Disallowing taking the address of a local is conservative and would disallow a number of valid programs.
>
> Arguably, such programs are in poor style anyway. A good program takes pointers only if it needs to keep them around; if all that's needed is to use the parameter transitorily or pass it down, ref is best.

Another common reason to use a pointer (instead of ref) is if it's optional (nullable). If the parameter is ref then the caller must go to the trouble of creating a variable.

However, this could be solved with a feature like the following:

int* find(string searchString, out int index) { ... }
// _ means "don't care", assuming no variable "_" is defined
void caller() { find("foo", out _); }

In fact this is arguably better for 'out' variables since the callee (find) no longer has to check whether 'index' is null before assigning it. However this doesn't totally solve the problem for 'ref' parameters, since such parameters are both output and input parameters and the programmer may want 'null' to have some special meaning as an input.

>> Escaping the addresses of stack variables, not just ref parameters, is a
>> general problem in "safe" D. Do you have any ideas about that?
Btw just a simple illustrative example:
int* unsafe1() { int x = 1; return unsafe2(&x); }
int* unsafe2(int* x) { return x; }
int unsafe3() { int y = 7; *unsafe1() = 8; return y; }
enum gaff = unsafe3(); // ICE, no line number given

> Same thing. By and large safe programs will need to make more use of the garbage collector than others. It's the way things work; stack allocation can be made safer if we add typed regions, but that's a very significant escalation of complication. There is no simple solution to this today.

Same thing meaning that you'd propose disallowing taking the address of a stack variable in SafeD? (I guess this would include escaping 'this' within a struct.)

> I really think that optionally allowing ref and out at the call site is more
> damaging than beneficial. _Requiring_ it could be beneficial, since then you
> know that the arguments are being taken by ref, but if it's optional, it gives
> you a false sense of security and can be misleading.

It gives *who* a false sense of security? If it's optional then I *know* lack of ref/out doesn't imply that the parameter won't change. Only people who don't know the rules would have this false sense of security.

I think it would be nice to have it required, but it's very bad to break everyone's code. It could only be reasonably enforced with a compiler switch--or, wait, come to think of it, a pragma would probably be better way to introduce language changes like this:

module foo;
pragma(callSiteRef);
// and would it make sense to offer an alternative to -property too?
pragma(property);

Now you can tell whether a program uses ref/out religiously or not.

On 9/13/12 11:47 AM, David Piepgrass wrote:
>> I really think that optionally allowing ref and out at the call site
>> is more
>> damaging than beneficial. _Requiring_ it could be beneficial, since
>> then you
>> know that the arguments are being taken by ref, but if it's optional,
>> it gives
>> you a false sense of security and can be misleading.
>
> It gives *who* a false sense of security?

It's "whom".

Andrei

P.S. All of my life I've been waiting for this moment!!!

On Thursday, 13 September 2012 at 16:29:19 UTC, Andrei Alexandrescu wrote:
>> It gives *who* a false sense of security?
>
> It's "whom".
>
> Andrei
>
> P.S. All of my life I've been waiting for this moment!!!

LOL

On Sep 13, 2012, at 7:21 AM, Andrei Alexandrescu <SeeWebsiteForEmail@erdani.org> wrote:
> 
> One question Walter and I thought about a lot was whether we should disallow escaping addresses of ref parameters in general or only in @safe code. We decided to go for the latter in order to avoid breakage of existing code. Thoughts?

I love the idea.  However, this could complicate working with C APIs, particularly regarding structs which are often passed by reference as a matter of course.  For example:

struct some_c_struct {}

void fn(ref some_c_struct s) {
    some_c_func(&s);
}

I guess the only way to know if this will turn out to be a real issue is to give it a try though.