On Thursday, September 13, 2012 17:47:45 David Piepgrass wrote:
> It gives *who* a false sense of security? If it's optional then I *know* lack of ref/out doesn't imply that the parameter won't change. Only people who don't know the rules would have this false sense of security.

More like anyone reading the code but especially when someone _other_ than you is reading your code. I'd fully expect that there would be quite a few programmers who would assume that the lack of ref at the call site would mean that it's not passed by ref when the code uses ref and out at the call site all over the place.

And because the lack of ref or out at the call site means nothing, you're always going to have to either know whether the function takes its arguments by ref or not, or you're going to have to look up the function anyway. And if that's the case, actually using ref or out at the call site buys you almost nothing.

- Jonathan M Davis

On Thursday, 13 September 2012 at 15:18:06 UTC, Jonathan M Davis wrote:
> On Thursday, September 13, 2012 12:34:34 Andrei Alexandrescu wrote:
>> I don't think there would be problems with allowing ref/out optionally
>> at the call site. The thing is, however, that in this matter reasonable
>> people may disagree.
>
> I really think that optionally allowing ref and out at the call site is more
> damaging than beneficial. _Requiring_ it could be beneficial, since then you
> know that the arguments are being taken by ref, but if it's optional, it gives
> you a false sense of security and can be misleading. Presumabyl, any time that
> ref or out is used at a call site, presumably the compiler would check that it
> really was passed by ref our out, which would be of some benefit, but it would
> mean _nothing_ if ref or out wasn't used, since it could be an argument being
> passed by ref but which wasn't marked or an argument which wasn't passed by
> ref at all. So, if you have code where ref and out is used at the call site
> semi-religiously, then it'll become very easy to falsely think that the lack
> of them means that ref or out _isn't_ being used when it really is, misleading
> people about what the code is doing.
>
> So, if it wasn't going to break so much code, I'd see some argument for
> changing things so that ref and out were required at the call site, but I
> think that _allowing_ them but not requiring them would actually be
> detrimental.
>
> - Jonathan M Davis

I see your point here but, as David already argued in his post, it would only be detrimental if people would not realise that these annotations are not obligatory. If that is clearly stated, i don't see so much of a problem here.

E.g. if we have a function / method call:

transmogrify(myValueType1, myValueType2, myValueType3);

Right now we don't know whether any of these values is going to be modified or not. And with _optional_ annotations allowed we couldn't say neither. That situation is not worse than the situation we have now. Things would only be bad if people would wrongly think that call site annotations are obligatory.


But if we would see the function call:

transmogrify(myValueType1, myValueType2, out myValueType3);

in some unknown piece of code we could at least surely know that myValueType3 is going to be modified. We couldn't say anything about myValueType1 or myValueType2 with optional annotations. The other parameters could or couldn't be modified. But we have at least a bit more context / information (e.g. that the function has side-effects). Normally functions / methods have better names so we would have even more context.


If it would be possible to allow call site annotations for the people who want to use them as an additional documentation / safety feature and maybe make the compiler emit warnings (through an optional compiler switch) if they are omitted or just allow them, as long as it's clear that they are optional, i don't see any problem with it.

On Thursday, 13 September 2012 at 10:34:00 UTC, Andrei Alexandrescu wrote:
> On 9/13/12 1:02 AM, Manuel wrote:
>> If a bigger part of the D community would like to have these annotations
>> added to the language, at least as an optional feature, then that might
>> persuade our "benevolent dictators" (just joking - you are great!)
>> Walther and Andrei to add them or we could just make a fork of the
>> language and add them ourselves (more joking - no, just skip this last
>> part - it's late) :-)
>>
>> There might be other problems, maybe with the compiler internals or
>> breakage of parts of the language. These were severe. But i think only
>> Walther and Andrei might tell. If these wouldn't exist it would at least
>> be possible, how David proposed it in his post, to make them optional.
>
> I don't think there would be problems with allowing ref/out optionally at the call site. The thing is, however, that in this matter reasonable people may disagree. In C++, the equivalent argument (should we pass everything modifiable by pointer or not?) erupts every six months on internal fora at Facebook. No winning argument is made by either part. For example, it is often brought up that it's good to see "&" at the call site when debugging some urgent matter at 3am. Yet there are other people who are just as apt at debugging urgent matters at 3am, and the absence of "&" doesn't seem to be a handicap for them at all. I'd be unable to identify any pattern in engineers choosing one preference over the other. As a consequence, our code has a mix of pass-by-pointer and pass-by-reference-to-nonconst, all engineers manage either style just as well, and we've never been able to find any evidence pointing one way or another.
>
> Now that the subject has been broken,

Thanks Andrei for sharing your experience with this matter in your working environment (after all, it's "Facebook" and not "John Nobody and Sons Inc.").

To be honest, i and i think even Kevin, who started this thread, didn't expect such lively reactions. But Kevins thread seems to struck a nerve.

And i think you made a very important point here. Whether someone would like to have this feature in the language or not is, despite all technical reasons / merits, also a psychological thing or a matter of taste.


> people who are just as apt at debugging urgent matters at 3am, and the absence of "&" doesn't seem to be a handicap for them at all


I think every competent programmer can handle both cases well (even at 3am). It's more, and here you are right, where you want to spend your time. Do you want to spend your time looking up all the function/method signatures you don't know / remember or spend it writing endless call site annotations. Do you want to spend your time debugging strange side-effects or want to write annoying annotations just to please the compiler. At the end, at 3am, it might result in the same level of productivity.


(1) transmogrify(MyValueType1, MyValueType2, MyValueType3);

The disadvantage here is, that if you don't know / have written the function you don't know whether if and which parameters might be modified. So you have to look the function signature up, no other choice here. There is no way for a quick read over the code. The advantage is that you save yourself writing endless annoying anotations, safe some typing and have less noise.


(2) transmogrify(MyValueType1, MyValueType2, out MyValueType3);

The disadvantage here is more typing, more noise. The advantage is that you as a programmer are forced to confirm that you know that this call will modify the value type and you want it this way, so nothing happens accidently. It's basically a double-check. You say i know this function will modify things and i want it that way. It gives people who read / review the code also more informations at a quick glance so they don't have too look things up constantly.

Whether you want something double checked might really depend on your personality (maybe i am more the anxious type and you are the bungee jumper type ;-) ).


So i think it basically boils down to this. I think it would fit do D because of the safety concept inside the language (as it is similar to contracts).

Since you don't seem to be overzealous regarding this (if i interpreted your confession rightly ;-) ) then it might be the best to leave all as it is.


I would like to have heared more arguments like: it breaks down certain language constructs, there are problems with templates or UCFS, or maybe from Walther that it makes too much work on the compiler side which could be spend better for other things or such.

But as i have written i haved lived without call site annotations in D and can live on without them. And honestly, i even think there are also more important things to be adressed in D (the classic one like bugs, toolchain, etc.).

So maybe the best contribution so far came from Paulo Pinto. To take things as they are and concentrate on more important things. But it's at least a good thing that we discussed this ;-)





> we do have good evidence of a pattern that generates significant and difficult bugs: escaping the address of a reference. In C++:
>
> struct A {
>     A(int& host) : host_(host) {}
> private:
>     int& host_;
> };
>
> In D:
>
> class A { // or struct
>     A(ref int host) : _host(&host) {}
> private:
>     int* _host;
> }
>
> A solution we use for C++ is to require escaped addresses to be always passed as pointers or smart pointers.
>
> Walter and I have discussed this for quite a while. We have recently decided to disallow, at least in SafeD, escaping the address of a ref parameter. In the beginning we'll be overly conservative by disallowing taking the address of a ref altogether. I'll write a DIP on that soon.
>
>
> Andrei

Couldn't this easily be supported by an IDE by coloured function arguments based on their type? Then the information is clearly visible without cluttering the code.

On 2012-09-13 18:28, Sean Kelly wrote:

> I love the idea.  However, this could complicate working with C APIs, particularly regarding structs which are often passed by reference as a matter of course.  For example:
>
> struct some_c_struct {}
>
> void fn(ref some_c_struct s) {
>      some_c_func(&s);
> }
>
> I guess the only way to know if this will turn out to be a real issue is to give it a try though.

This applies to static arrays as well:

extern (C) void foo (ref int[3] a);

Since in D static arrays have value semantics.

-- 
/Jacob Carlborg

On Thursday, 13 September 2012 at 22:21:34 UTC, ixid wrote:
> Couldn't this easily be supported by an IDE by coloured function arguments based on their type? Then the information is clearly visible without cluttering the code.

no because... wait... that... that could work.

On 13/09/2012 16:29, David Piepgrass wrote:
> On Thursday, 13 September 2012 at 15:01:28 UTC, Andrei Alexandrescu wrote:
>> On 9/13/12 10:53 AM, David Piepgrass wrote:
>>>> Walter and I have discussed this for quite a while. We have recently
>>>> decided to disallow, at least in SafeD, escaping the address of a ref
>>>> parameter. In the beginning we'll be overly conservative by
>>>> disallowing taking the address of a ref altogether. I'll write a DIP
>>>> on that soon.
>>>
>>> Err, wouldn't that break a lot of stuff, a lot of which is actually safe
>>> code?
>>>
>>> void a(ref int x) { b(&x); }
>>> void b(int* x) { if(x != null) (*x)++; }
>>
>> Yes. Disallowing taking the address of a local is conservative and
>> would disallow a number of valid programs.

What about:

void b(scope int* x) { if(x != null) (*x)++; }

Then 'a' could compile safely.

>> Arguably, such programs are in poor style anyway. A good program takes
>> pointers only if it needs to keep them around; if all that's needed is
>> to use the parameter transitorily or pass it down, ref is best.
>
> Another common reason to use a pointer (instead of ref) is if it's
> optional (nullable). If the parameter is ref then the caller must go to
> the trouble of creating a variable.

Maybe std.typecons.Nullable or something like it helps there.

On 12/09/2012 13:55, Nick Treleaven wrote:
> On 12/09/2012 13:39, Nick Treleaven wrote:
>>> class Bar { int b; }
>>> void changeBar(ref Bar b) { b = new Bar(); }
>>> void warning()
>>> {
>>>      Bar bar = new Bar();
>>>      bar.b = 10;
>>>      bar.changeBar(); // Warning: 'bar' is implicitly passed by
>>> reference. To eliminate this warning, use 'changeBar(ref bar)' instead
>>> or do not compile with '-callSiteRef'
>>> }
>>>
>>> Again, this problem only applies to classes, since it is understood that
>>> structs are normally passed by reference.
>>
>> I had only thought about UFCS and ref parameters for value types. You
>> are right that requiring callsite ref for class ref parameters would be
>> a useful idea, as modifying the ref itself is unusual behavior. And
>> because of that, disallowing UFCS for functions that have a class ref
>> parameter as the first parameter might be an elegant solution.
>
> Also the same applies to ref parameters with any reference types:
>
> On 08/09/2012 14:05, Chris Nicholson-Sauls wrote:
>  > void func (ref int[], int)
>  >
>  > If ref/out were required at the call site, this destroys UFCS.
>  >
>  > int[] array;
>  > array.func(0); // error, ref not specified by caller
>
> So the above poster was right

Ignore that last reply, slices are value types. The syntax array.func() looks like it might modify array's contents and/or length, so UFCS is OK. It is only ref class and ref pointer arguments that UFCS should be disallowed on to enforce call-site ref.

On Friday, 14 September 2012 at 06:55:00 UTC, F i L wrote:
> On Thursday, 13 September 2012 at 22:21:34 UTC, ixid wrote:
>> Couldn't this easily be supported by an IDE by coloured function arguments based on their type? Then the information is clearly visible without cluttering the code.
>
> no because... wait... that... that could work.

Ok. Then ... who is writing this IDE ?

On Friday, 14 September 2012 at 22:20:16 UTC, Manuel wrote:
> On Friday, 14 September 2012 at 06:55:00 UTC, F i L wrote:
>> On Thursday, 13 September 2012 at 22:21:34 UTC, ixid wrote:
>>> Couldn't this easily be supported by an IDE by coloured function arguments based on their type? Then the information is clearly visible without cluttering the code.
>>
>> no because... wait... that... that could work.
>
> Ok. Then ... who is writing this IDE ?

I'm just saying it's a solution that doesn't require adding a breaking feature to D. I know MonoDevelop has the ability to color user-types and such, maybe eventually Alex will be able to add such a feature.

If the compiler developers can't or won't add 'foo(ref x)' semantics to the compiler, then this is really the only option left available that gives the same level of insight to developers that C# does.