January 15, 2012
On Sun, 15 Jan 2012 07:36:53 -0600, Alex Rønne Petersen <xtzgzorex@gmail.com> wrote:
> Hi,
>
> I don't know how many times I've made the mistake of passing a local
> variable to a function which takes a 'ref' parameter. Suddenly, local
> variables/fields are just mutating out of nowhere, because it's not at
> all obvious that a function you're calling is taking a 'ref' parameter.
> This is particularly true for std.utf.decode().
>
> Yes, I realize I could look at the function declaration. Yes, I could
> read the docs too. But that doesn't prevent me from forgetting that a
> function takes a 'ref' parameter, and then doing the mistake again. The
> damage is done, and the time is wasted.
>
> I think D should allow 'ref' on call sites to prevent these mistakes.
> For example:
>
> string str = ...;
> size_t pos;
> auto chr = std.utf.decode(str, ref pos);
>
> Now it's much more obvious that the parameter is passed by reference and
> is going to be mutated.
>
> Ideally, this would not be optional, but rather *required*, but I
> realize that such a change would break a *lot* of code, so that's
> probably not a good idea.
>
> Thoughts?

In no particular order:

1) Adding ref to the call site is lexically similar to Hungarian notation and has all the drawbacks and advantages thereof. I know invoking Hungarian notation is almost an invocation of Godwin's law when it comes to programming syntax discussions, but Hungarian notation was introduced for a reason; there was a time when for large software projects it dramatically increases code comprehension (and thus quality) for the code reviewer and/or code maintainer. And that argument still stand today for anyone _not_ using a modern IDE. The primary reason Hungarian notation is disparaged today is that IDEs evolved beyond emacs, vim and notepad. Once they could tell the programmer with a tooltip what every variable's type was, the need to encode the type in the variable name vanished. Or more to the original point, modern IDEs already list function parameter's type and type modifiers (i.e. aka) as you type the function in so it's _always_ obvious what is ref/const/etc and what is not. And for the code reviewer, ref parameters could be auto highlighted/underlined/etc to easy their job. Yes, this issue does need to be addressed, but I don't think that this is fundamentally a language problem; it more a lack of modern tools for D. That reminds me, I need to check out the latest revision of Visual D. :)

2) It is perfectly possible to write functions in D that require explicit demarcation at the call site:

int foo( int* v ) { return *x; }

int y;
int z = foo(@y);

3) Mandating ref at the call is currently how C# handles reference parameters. Anytime someone suggests a feature from another language, particularly C# or Java, two questions are immediately raised in my mind.
3a) Does the asker simply want D to be more like language X?
3b) What do programmers experienced in X and in D/C++/C/etc think about that particular feature (good/bad/ugly)?

4) When presenting problematic language use case and an associated proposed solution, being light on details/analysis and missing the second most obvious drawbacks of your proposed solution will cause your argument to be dismissed by many people. In short, if you're not willing to fully think through your idea, why should we take the time to too? That aside, if you are serious about this topic, you might want to have a look at/submit a D improvement proposal (http://prowiki.org/wiki4d/wiki.cgi?LanguageDevel/DIPs).
January 15, 2012
On 15/01/12 5:57 PM, Robert Jacques wrote:
> On Sun, 15 Jan 2012 07:36:53 -0600, Alex Rønne Petersen
>> Thoughts?
>
> In no particular order:
>
> 1) Adding ref to the call site is lexically similar to Hungarian
> notation and has all the drawbacks and advantages thereof. I know
> invoking Hungarian notation is almost an invocation of Godwin's law when
> it comes to programming syntax discussions, but Hungarian notation was
> introduced for a reason; there was a time when for large software
> projects it dramatically increases code comprehension (and thus quality)
> for the code reviewer and/or code maintainer. And that argument still
> stand today for anyone _not_ using a modern IDE. The primary reason
> Hungarian notation is disparaged today is that IDEs evolved beyond
> emacs, vim and notepad. Once they could tell the programmer with a
> tooltip what every variable's type was, the need to encode the type in
> the variable name vanished. Or more to the original point, modern IDEs
> already list function parameter's type and type modifiers (i.e. aka) as
> you type the function in so it's _always_ obvious what is ref/const/etc
> and what is not. And for the code reviewer, ref parameters could be auto
> highlighted/underlined/etc to easy their job. Yes, this issue does need
> to be addressed, but I don't think that this is fundamentally a language
> problem; it more a lack of modern tools for D. That reminds me, I need
> to check out the latest revision of Visual D. :)

A fair point, but:

a) D doesn't have much in terms of *free* highly-featured IDEs.
b) Even if it did, lots of people still use emacs/vim.
c) That does not help when glancing at code to find where a variable is modified.


> 2) It is perfectly possible to write functions in D that require
> explicit demarcation at the call site:
>
> int foo( int* v ) { return *x; }
>
> int y;
> int z = foo(@y);

True, but these aren't the same. Pointers can be reseated, ref parameters cannot. Pointers may also be null, encouraging needless null pointer checks.


> 3) Mandating ref at the call is currently how C# handles reference
> parameters. Anytime someone suggests a feature from another language,
> particularly C# or Java, two questions are immediately raised in my mind.
> 3a) Does the asker simply want D to be more like language X?

Perhaps, perhaps not. I think it is better to evaluate a suggestion on its merit and not on the suggester's motivation.


> 3b) What do programmers experienced in X and in D/C++/C/etc think about
> that particular feature (good/bad/ugly)?

Mandated ref is well-received in C# as far as I'm aware.

I also know that people dislike using reference parameters in C++ due to the lack of visibility at the call site (causing them to use pointers).

January 15, 2012
On 1/15/2012 5:36 AM, Alex Rønne Petersen wrote:
> Hi,
>
> I don't know how many times I've made the mistake of passing a local variable to a function which takes a 'ref' parameter. Suddenly, local variables/fields are just mutating out of nowhere, because it's not at all obvious that a function you're calling is taking a 'ref' parameter. This is particularly true for std.utf.decode().
>
> Yes, I realize I could look at the function declaration. Yes, I could read the docs too. But that doesn't prevent me from forgetting that a function takes a 'ref' parameter, and then doing the mistake again. The damage is done, and the time is wasted.
>
> I think D should allow 'ref' on call sites to prevent these mistakes. For example:
>
> string str = ...;
> size_t pos;
> auto chr = std.utf.decode(str, ref pos);
>
> Now it's much more obvious that the parameter is passed by reference and is going to be mutated.
>
> Ideally, this would not be optional, but rather *required*, but I realize that such a change would break a *lot* of code, so that's probably not a good idea.
>
> Thoughts?
>
I suggested this a while ago but people acted as though I was talking about aliens...
January 15, 2012
> I also know that people dislike using reference parameters in C++ due to the lack of visibility at the call site (causing them to use pointers).

Which is true for me. Every single one of my reference parameters is const ref or pointer.
January 15, 2012
Robert Jacques:

> 1) Adding ref to the call site is lexically similar to Hungarian notation and has all the drawbacks and advantages thereof.

The presence of "ref" at the call site is meant to be enforced by the type system of the compiler.


> Or more to the original point, modern IDEs already list function parameter's type and type modifiers (i.e. aka) as you type the function in so it's _always_ obvious what is ref/const/etc and what is not.

C# is meant to be used with a modern IDE, yet it requires "ref" and "out" at the call site (but in some cases, where the use of ref is pervasive).

Bye,
bearophile
January 15, 2012
Tobias Pankrath:

> > I also know that people dislike using reference parameters in C++ due to the lack of visibility at the call site (causing them to use pointers).
> 
> Which is true for me. Every single one of my reference parameters is const ref or pointer.

(What you do is also required by the Google C++ style guide.)
So your experience favors the use of callsite "ref" for nonconst ref arguments :-)

Bye,
bearophile
January 15, 2012
"Alex Rønne Petersen" <xtzgzorex@gmail.com> wrote in message news:jeukpm$168v$1@digitalmars.com...
> Hi,
>
> I don't know how many times I've made the mistake of passing a local variable to a function which takes a 'ref' parameter. Suddenly, local variables/fields are just mutating out of nowhere, because it's not at all obvious that a function you're calling is taking a 'ref' parameter. This is particularly true for std.utf.decode().
>
> Yes, I realize I could look at the function declaration. Yes, I could read the docs too. But that doesn't prevent me from forgetting that a function takes a 'ref' parameter, and then doing the mistake again. The damage is done, and the time is wasted.
>
> I think D should allow 'ref' on call sites to prevent these mistakes. For example:
>
> string str = ...;
> size_t pos;
> auto chr = std.utf.decode(str, ref pos);
>
> Now it's much more obvious that the parameter is passed by reference and is going to be mutated.
>
> Ideally, this would not be optional, but rather *required*, but I realize that such a change would break a *lot* of code, so that's probably not a good idea.
>
> Thoughts?
>

Yes, this is one of the few things I always thought C# got right and D got wrong.

That said though, I haven't personally run into this problem, and I've gotten used to not having "ref" or "out" on the caller's side. I wouldn't be opposed to the change though, even just as an optional warning.


January 15, 2012
On 01/15/2012 02:36 PM, Alex Rønne Petersen wrote:
> Hi,
>
> I don't know how many times I've made the mistake of passing a local
> variable to a function which takes a 'ref' parameter. Suddenly, local
> variables/fields are just mutating out of nowhere, because it's not at
> all obvious that a function you're calling is taking a 'ref' parameter.
> This is particularly true for std.utf.decode().
>

Odd. I always thought the fact that std.utf.decode modifies the second parameter is very intuitive. (decoding cannot work nicely without it)

> Yes, I realize I could look at the function declaration. Yes, I could
> read the docs too. But that doesn't prevent me from forgetting that a
> function takes a 'ref' parameter, and then doing the mistake again. The
> damage is done, and the time is wasted.
>

This is true for every part of a function interface. How can a programmer even understand what the function does if he forgets what the calling conventions are? Conversely, does a reminder of the calling conventions usually restore full knowledge of the semantics of the called function?

> I think D should allow 'ref' on call sites to prevent these mistakes.
> For example:
>
> string str = ...;
> size_t pos;
> auto chr = std.utf.decode(str, ref pos);
>
> Now it's much more obvious that the parameter is passed by reference and
> is going to be mutated.
>
> Ideally, this would not be optional, but rather *required*, but I
> realize that such a change would break a *lot* of code, so that's
> probably not a good idea.
>
> Thoughts?
>

I personally think 'ref' at call site is pure syntax noise but I see that it might be useful to some in some cases. I'd prefer to leave it as-is, but I don't feel very strongly for either way.
My preference would be
[not allowed > enforced > ... > optional > optional with switch]

However, it is very important not to do this to 'lazy'.

Another thing that would have to be discussed: what happens to const ref parameters? It is very reasonable that someone will decide to change calling conventions from by value to by const ref or the other way round after profiling. It is very convenient that such a change is syntactically transparent to the caller and this should stay.

It is even realistic that someone will decide to interchange by ref/by value, because there is no way to express 'I will not change the head level' in the current type system other than taking a parameter by value. Therefore introducing call-site ref would perhaps necessitate re-introducing 'final' for variables.

January 15, 2012
On 15/01/12 10:10 PM, Timon Gehr wrote:
> Another thing that would have to be discussed: what happens to const ref
> parameters? It is very reasonable that someone will decide to change
> calling conventions from by value to by const ref or the other way round
> after profiling. It is very convenient that such a change is
> syntactically transparent to the caller and this should stay.

Actually, that would be very dangerous. const ref does not have the privilege of being able to bind to rvalues in D like it does in C++.

void foo(const ref int x) {...}

foo(1); // this is legal in C++, but illegal in D

In case, const ref would not require the ref at call site because there's no danger of it being modified.

January 15, 2012
On 01/15/2012 11:10 PM, Timon Gehr wrote:
> On 01/15/2012 02:36 PM, Alex Rønne Petersen wrote:
>> Hi,
>>
>> I don't know how many times I've made the mistake of passing a local
>> variable to a function which takes a 'ref' parameter. Suddenly, local
>> variables/fields are just mutating out of nowhere, because it's not at
>> all obvious that a function you're calling is taking a 'ref' parameter.
>> This is particularly true for std.utf.decode().
>>
>
> Odd. I always thought the fact that std.utf.decode modifies the second
> parameter is very intuitive. (decoding cannot work nicely without it)
>
>> Yes, I realize I could look at the function declaration. Yes, I could
>> read the docs too. But that doesn't prevent me from forgetting that a
>> function takes a 'ref' parameter, and then doing the mistake again. The
>> damage is done, and the time is wasted.
>>
>
> This is true for every part of a function interface. How can a
> programmer even understand what the function does if he forgets what the
> calling conventions are? Conversely, does a reminder of the calling
> conventions usually restore full knowledge of the semantics of the
> called function?
>
>> I think D should allow 'ref' on call sites to prevent these mistakes.
>> For example:
>>
>> string str = ...;
>> size_t pos;
>> auto chr = std.utf.decode(str, ref pos);
>>
>> Now it's much more obvious that the parameter is passed by reference and
>> is going to be mutated.
>>
>> Ideally, this would not be optional, but rather *required*, but I
>> realize that such a change would break a *lot* of code, so that's
>> probably not a good idea.
>>
>> Thoughts?
>>
>
> I personally think 'ref' at call site is pure syntax noise but I see
> that it might be useful to some in some cases. I'd prefer to leave it
> as-is, but I don't feel very strongly for either way.
> My preference would be
> [not allowed > enforced > ... > optional > optional with switch]
>
> However, it is very important not to do this to 'lazy'.
>
> Another thing that would have to be discussed: what happens to const ref
> parameters? It is very reasonable that someone will decide to change
> calling conventions from by value to by const ref or the other way round
> after profiling. It is very convenient that such a change is
> syntactically transparent to the caller and this should stay.
>
> It is even realistic that someone will decide to interchange by ref/by
> value, because there is no way to express 'I will not change the head
> level' in the current type system other than taking a parameter by
> value. Therefore introducing call-site ref would perhaps necessitate
> re-introducing 'final' for variables.
>

Oh, and furthermore it would give rise to abominations such as:

auto ref wrap(alias foo,T...)(auto ref T args){
    return foo(auto ref args);
}

*shudder*. What does 'auto ref' even mean if 'ref' is required for ref parameters at call site? I think the best answers would silently break existing code.