View mode: basic / threaded / horizontal-split · Log in · Help
September 13, 2012
Re: Would like to see ref and out required for function calls
On Thursday, September 13, 2012 17:47:45 David Piepgrass wrote:
> It gives *who* a false sense of security? If it's optional then I
> *know* lack of ref/out doesn't imply that the parameter won't
> change. Only people who don't know the rules would have this
> false sense of security.

More like anyone reading the code but especially when someone _other_ than you 
is reading your code. I'd fully expect that there would be quite a few 
programmers who would assume that the lack of ref at the call site would mean 
that it's not passed by ref when the code uses ref and out at the call site 
all over the place.

And because the lack of ref or out at the call site means nothing, you're 
always going to have to either know whether the function takes its arguments 
by ref or not, or you're going to have to look up the function anyway. And if 
that's the case, actually using ref or out at the call site buys you almost 
nothing.

- Jonathan M Davis
September 13, 2012
Re: Would like to see ref and out required for function calls
On Thursday, 13 September 2012 at 15:18:06 UTC, Jonathan M Davis 
wrote:
> On Thursday, September 13, 2012 12:34:34 Andrei Alexandrescu 
> wrote:
>> I don't think there would be problems with allowing ref/out 
>> optionally
>> at the call site. The thing is, however, that in this matter 
>> reasonable
>> people may disagree.
>
> I really think that optionally allowing ref and out at the call 
> site is more
> damaging than beneficial. _Requiring_ it could be beneficial, 
> since then you
> know that the arguments are being taken by ref, but if it's 
> optional, it gives
> you a false sense of security and can be misleading. 
> Presumabyl, any time that
> ref or out is used at a call site, presumably the compiler 
> would check that it
> really was passed by ref our out, which would be of some 
> benefit, but it would
> mean _nothing_ if ref or out wasn't used, since it could be an 
> argument being
> passed by ref but which wasn't marked or an argument which 
> wasn't passed by
> ref at all. So, if you have code where ref and out is used at 
> the call site
> semi-religiously, then it'll become very easy to falsely think 
> that the lack
> of them means that ref or out _isn't_ being used when it really 
> is, misleading
> people about what the code is doing.
>
> So, if it wasn't going to break so much code, I'd see some 
> argument for
> changing things so that ref and out were required at the call 
> site, but I
> think that _allowing_ them but not requiring them would 
> actually be
> detrimental.
>
> - Jonathan M Davis

I see your point here but, as David already argued in his post, 
it would only be detrimental if people would not realise that 
these annotations are not obligatory. If that is clearly stated, 
i don't see so much of a problem here.

E.g. if we have a function / method call:

transmogrify(myValueType1, myValueType2, myValueType3);

Right now we don't know whether any of these values is going to 
be modified or not. And with _optional_ annotations allowed we 
couldn't say neither. That situation is not worse than the 
situation we have now. Things would only be bad if people would 
wrongly think that call site annotations are obligatory.


But if we would see the function call:

transmogrify(myValueType1, myValueType2, out myValueType3);

in some unknown piece of code we could at least surely know that 
myValueType3 is going to be modified. We couldn't say anything 
about myValueType1 or myValueType2 with optional annotations. The 
other parameters could or couldn't be modified. But we have at 
least a bit more context / information (e.g. that the function 
has side-effects). Normally functions / methods have better names 
so we would have even more context.


If it would be possible to allow call site annotations for the 
people who want to use them as an additional documentation / 
safety feature and maybe make the compiler emit warnings (through 
an optional compiler switch) if they are omitted or just allow 
them, as long as it's clear that they are optional, i don't see 
any problem with it.
September 13, 2012
Re: Would like to see ref and out required for function calls
On Thursday, 13 September 2012 at 10:34:00 UTC, Andrei 
Alexandrescu wrote:
> On 9/13/12 1:02 AM, Manuel wrote:
>> If a bigger part of the D community would like to have these 
>> annotations
>> added to the language, at least as an optional feature, then 
>> that might
>> persuade our "benevolent dictators" (just joking - you are 
>> great!)
>> Walther and Andrei to add them or we could just make a fork of 
>> the
>> language and add them ourselves (more joking - no, just skip 
>> this last
>> part - it's late) :-)
>>
>> There might be other problems, maybe with the compiler 
>> internals or
>> breakage of parts of the language. These were severe. But i 
>> think only
>> Walther and Andrei might tell. If these wouldn't exist it 
>> would at least
>> be possible, how David proposed it in his post, to make them 
>> optional.
>
> I don't think there would be problems with allowing ref/out 
> optionally at the call site. The thing is, however, that in 
> this matter reasonable people may disagree. In C++, the 
> equivalent argument (should we pass everything modifiable by 
> pointer or not?) erupts every six months on internal fora at 
> Facebook. No winning argument is made by either part. For 
> example, it is often brought up that it's good to see "&" at 
> the call site when debugging some urgent matter at 3am. Yet 
> there are other people who are just as apt at debugging urgent 
> matters at 3am, and the absence of "&" doesn't seem to be a 
> handicap for them at all. I'd be unable to identify any pattern 
> in engineers choosing one preference over the other. As a 
> consequence, our code has a mix of pass-by-pointer and 
> pass-by-reference-to-nonconst, all engineers manage either 
> style just as well, and we've never been able to find any 
> evidence pointing one way or another.
>
> Now that the subject has been broken,

Thanks Andrei for sharing your experience with this matter in 
your working environment (after all, it's "Facebook" and not 
"John Nobody and Sons Inc.").

To be honest, i and i think even Kevin, who started this thread, 
didn't expect such lively reactions. But Kevins thread seems to 
struck a nerve.

And i think you made a very important point here. Whether someone 
would like to have this feature in the language or not is, 
despite all technical reasons / merits, also a psychological 
thing or a matter of taste.


> people who are just as apt at debugging urgent matters at 3am, 
> and the absence of "&" doesn't seem to be a handicap for them 
> at all


I think every competent programmer can handle both cases well 
(even at 3am). It's more, and here you are right, where you want 
to spend your time. Do you want to spend your time looking up all 
the function/method signatures you don't know / remember or spend 
it writing endless call site annotations. Do you want to spend 
your time debugging strange side-effects or want to write 
annoying annotations just to please the compiler. At the end, at 
3am, it might result in the same level of productivity.


(1) transmogrify(MyValueType1, MyValueType2, MyValueType3);

The disadvantage here is, that if you don't know / have written 
the function you don't know whether if and which parameters might 
be modified. So you have to look the function signature up, no 
other choice here. There is no way for a quick read over the 
code. The advantage is that you save yourself writing endless 
annoying anotations, safe some typing and have less noise.


(2) transmogrify(MyValueType1, MyValueType2, out MyValueType3);

The disadvantage here is more typing, more noise. The advantage 
is that you as a programmer are forced to confirm that you know 
that this call will modify the value type and you want it this 
way, so nothing happens accidently. It's basically a 
double-check. You say i know this function will modify things and 
i want it that way. It gives people who read / review the code 
also more informations at a quick glance so they don't have too 
look things up constantly.

Whether you want something double checked might really depend on 
your personality (maybe i am more the anxious type and you are 
the bungee jumper type ;-) ).


So i think it basically boils down to this. I think it would fit 
do D because of the safety concept inside the language (as it is 
similar to contracts).

Since you don't seem to be overzealous regarding this (if i 
interpreted your confession rightly ;-) ) then it might be the 
best to leave all as it is.


I would like to have heared more arguments like: it breaks down 
certain language constructs, there are problems with templates or 
UCFS, or maybe from Walther that it makes too much work on the 
compiler side which could be spend better for other things or 
such.

But as i have written i haved lived without call site annotations 
in D and can live on without them. And honestly, i even think 
there are also more important things to be adressed in D (the 
classic one like bugs, toolchain, etc.).

So maybe the best contribution so far came from Paulo Pinto. To 
take things as they are and concentrate on more important things. 
But it's at least a good thing that we discussed this ;-)





> we do have good evidence of a pattern that generates 
> significant and difficult bugs: escaping the address of a 
> reference. In C++:
>
> struct A {
>     A(int& host) : host_(host) {}
> private:
>     int& host_;
> };
>
> In D:
>
> class A { // or struct
>     A(ref int host) : _host(&host) {}
> private:
>     int* _host;
> }
>
> A solution we use for C++ is to require escaped addresses to be 
> always passed as pointers or smart pointers.
>
> Walter and I have discussed this for quite a while. We have 
> recently decided to disallow, at least in SafeD, escaping the 
> address of a ref parameter. In the beginning we'll be overly 
> conservative by disallowing taking the address of a ref 
> altogether. I'll write a DIP on that soon.
>
>
> Andrei
September 13, 2012
Re: Would like to see ref and out required for function calls
Couldn't this easily be supported by an IDE by coloured function 
arguments based on their type? Then the information is clearly 
visible without cluttering the code.
September 14, 2012
Re: Would like to see ref and out required for function calls
On 2012-09-13 18:28, Sean Kelly wrote:

> I love the idea.  However, this could complicate working with C APIs, particularly regarding structs which are often passed by reference as a matter of course.  For example:
>
> struct some_c_struct {}
>
> void fn(ref some_c_struct s) {
>      some_c_func(&s);
> }
>
> I guess the only way to know if this will turn out to be a real issue is to give it a try though.

This applies to static arrays as well:

extern (C) void foo (ref int[3] a);

Since in D static arrays have value semantics.

-- 
/Jacob Carlborg
September 14, 2012
Re: Would like to see ref and out required for function calls
On Thursday, 13 September 2012 at 22:21:34 UTC, ixid wrote:
> Couldn't this easily be supported by an IDE by coloured 
> function arguments based on their type? Then the information is 
> clearly visible without cluttering the code.

no because... wait... that... that could work.
September 14, 2012
Re: Would like to see ref and out required for function calls
On 13/09/2012 16:29, David Piepgrass wrote:
> On Thursday, 13 September 2012 at 15:01:28 UTC, Andrei Alexandrescu wrote:
>> On 9/13/12 10:53 AM, David Piepgrass wrote:
>>>> Walter and I have discussed this for quite a while. We have recently
>>>> decided to disallow, at least in SafeD, escaping the address of a ref
>>>> parameter. In the beginning we'll be overly conservative by
>>>> disallowing taking the address of a ref altogether. I'll write a DIP
>>>> on that soon.
>>>
>>> Err, wouldn't that break a lot of stuff, a lot of which is actually safe
>>> code?
>>>
>>> void a(ref int x) { b(&x); }
>>> void b(int* x) { if(x != null) (*x)++; }
>>
>> Yes. Disallowing taking the address of a local is conservative and
>> would disallow a number of valid programs.

What about:

void b(scope int* x) { if(x != null) (*x)++; }

Then 'a' could compile safely.

>> Arguably, such programs are in poor style anyway. A good program takes
>> pointers only if it needs to keep them around; if all that's needed is
>> to use the parameter transitorily or pass it down, ref is best.
>
> Another common reason to use a pointer (instead of ref) is if it's
> optional (nullable). If the parameter is ref then the caller must go to
> the trouble of creating a variable.

Maybe std.typecons.Nullable or something like it helps there.
September 14, 2012
Re: Would like to see ref and out required for function calls
On 12/09/2012 13:55, Nick Treleaven wrote:
> On 12/09/2012 13:39, Nick Treleaven wrote:
>>> class Bar { int b; }
>>> void changeBar(ref Bar b) { b = new Bar(); }
>>> void warning()
>>> {
>>>      Bar bar = new Bar();
>>>      bar.b = 10;
>>>      bar.changeBar(); // Warning: 'bar' is implicitly passed by
>>> reference. To eliminate this warning, use 'changeBar(ref bar)' instead
>>> or do not compile with '-callSiteRef'
>>> }
>>>
>>> Again, this problem only applies to classes, since it is understood that
>>> structs are normally passed by reference.
>>
>> I had only thought about UFCS and ref parameters for value types. You
>> are right that requiring callsite ref for class ref parameters would be
>> a useful idea, as modifying the ref itself is unusual behavior. And
>> because of that, disallowing UFCS for functions that have a class ref
>> parameter as the first parameter might be an elegant solution.
>
> Also the same applies to ref parameters with any reference types:
>
> On 08/09/2012 14:05, Chris Nicholson-Sauls wrote:
>  > void func (ref int[], int)
>  >
>  > If ref/out were required at the call site, this destroys UFCS.
>  >
>  > int[] array;
>  > array.func(0); // error, ref not specified by caller
>
> So the above poster was right

Ignore that last reply, slices are value types. The syntax array.func() 
looks like it might modify array's contents and/or length, so UFCS is 
OK. It is only ref class and ref pointer arguments that UFCS should be 
disallowed on to enforce call-site ref.
September 14, 2012
Re: Would like to see ref and out required for function calls
On Friday, 14 September 2012 at 06:55:00 UTC, F i L wrote:
> On Thursday, 13 September 2012 at 22:21:34 UTC, ixid wrote:
>> Couldn't this easily be supported by an IDE by coloured 
>> function arguments based on their type? Then the information 
>> is clearly visible without cluttering the code.
>
> no because... wait... that... that could work.

Ok. Then ... who is writing this IDE ?
September 14, 2012
Re: Would like to see ref and out required for function calls
On Friday, 14 September 2012 at 22:20:16 UTC, Manuel wrote:
> On Friday, 14 September 2012 at 06:55:00 UTC, F i L wrote:
>> On Thursday, 13 September 2012 at 22:21:34 UTC, ixid wrote:
>>> Couldn't this easily be supported by an IDE by coloured 
>>> function arguments based on their type? Then the information 
>>> is clearly visible without cluttering the code.
>>
>> no because... wait... that... that could work.
>
> Ok. Then ... who is writing this IDE ?

I'm just saying it's a solution that doesn't require adding a 
breaking feature to D. I know MonoDevelop has the ability to 
color user-types and such, maybe eventually Alex will be able to 
add such a feature.

If the compiler developers can't or won't add 'foo(ref x)' 
semantics to the compiler, then this is really the only option 
left available that gives the same level of insight to developers 
that C# does.
2 3 4 5 6 7
Top | Discussion index | About this forum | D home