On Saturday, 21 July 2018 at 04:09:25 UTC, Jonathan M Davis wrote:
> Honestly, I think we're just coming from points of view that are too different. IMHO, the core use case for ref is for a function to mutate an argument and have that result progagate to the argument,

I think the critical point here is that the the caller is free to ignore the refness of the arg by creating a temporary and not using it further.

> and having ref accept rvalues is not only counter to that, but it risks bugs that are currently impossible.

A class of bugs I believe to be self-evident and incredibly sparse.

>I think that having a way to accept rvalues by ref
> for functions where you want to avoid copying is potentially useful but not particularly critical. On the other hand, you seem to see little or no value in having parameters that are intended to only accept lvalues

As the API author you can @disable the rvalue overload, but nothing you do can stop the caller supplying an unused temporary to the ref argument. Yes you can discourage them with docs/@disabled overload sets/ whatever, but you can't stop them. That is no different to today. This DIP may lessen your ability to discourage them from misusing it, but you can't stop a stubborn idiot, and that is not the audience D is targeting.

> and see great value in having functions that accept rvalues by ref in order to avoid copying.

It is not just the avoiding copying, if it were I'm not sure I'd support it. For me the greatest benefit is the increase in readability due to not having useless temporaries everywhere in ref heavy code (that may not be under API user's control).

On Fri, 20 Jul 2018 at 22:45, Nicholas Wilson via Digitalmars-d <digitalmars-d@puremagic.com> wrote:
>
> On Saturday, 21 July 2018 at 04:09:25 UTC, Jonathan M Davis wrote:
> > Honestly, I think we're just coming from points of view that are too different. IMHO, the core use case for ref is for a function to mutate an argument and have that result progagate to the argument,
>
> I think the critical point here is that the the caller is free to ignore the refness of the arg by creating a temporary and not using it further.
>
> > and having ref accept rvalues is not only counter to that, but it risks bugs that are currently impossible.
>
> A class of bugs I believe to be self-evident and incredibly sparse.
>
> >I think that having a way to accept rvalues by ref
> > for functions where you want to avoid copying is potentially useful but not particularly critical. On the other hand, you seem to see little or no value in having parameters that are intended to only accept lvalues
>
> As the API author you can @disable the rvalue overload, but nothing you do can stop the caller supplying an unused temporary to the ref argument. Yes you can discourage them with docs/@disabled overload sets/ whatever, but you can't stop them. That is no different to today. This DIP may lessen your ability to discourage them from misusing it, but you can't stop a stubborn idiot, and that is not the audience D is targeting.
>
> > and see great value in having functions that accept rvalues by ref in order to avoid copying.
>
> It is not just the avoiding copying, if it were I'm not sure I'd support it. For me the greatest benefit is the increase in readability due to not having useless temporaries everywhere in ref heavy code (that may not be under API user's control).

For me, the biggest win is reducing the amount of broken meta because
authors didn't think about ref up front (or ever).
Handling heaps of edge cases in complex meta really sucks; it's hard
to get right, and it's hard to test for. Most people NEVER test
anything with ref.
Removing unnecessary and awkward edge cases that result in brittle
code is always a good thing!

Also removing frustrating bloat and badly named temporaries (t, t1, t2, etc) from places they never should have existed is a massive win. Calling into C++, and via polymorphic interfaces also gain disproportionate advantage.

On Saturday, 21 July 2018 at 05:59:37 UTC, Manu wrote:
> [...]

Let me give a concrete example of one of the situations Jonathan is describing. Consider the following code:


struct Secret
{
public:
    string key;
}

/* ... */

genrateRandomKey(ref string key) {
    key = /*  some code to actually generate the key */
}

Secret secret;
generateRandomKey(secret.key);


Now somebody else working on the project who sees the definition of Secret might think "Having public access to member variables is bad, so lets use property methods instead. This even allows us to do some contract checks!" and he changes the definition of Secret to the following:


struct Secret
{
private:
    string _key;

public:
    string key() @property const
    {
        return this._key;
    }

    void key(string key) @property
    in
    {
        assert(key.length == 256)
    }
    do
    {
        this._key = key;
    }
}


Now with DIP 1016, the use of generateRandomKey silently "fails", i.e. secret._key will not be changed by it, which in this case is a big problem as the key is still default initialized!

Of course one might argue that genrateRandomKey should not take its argument by reference and rather just return the key instead. But in my experience, there is quite a lot of code like this out there (e.g. in order to avoid copying, string is probably not the best example here...).

In one of your earlier answers you argued, that in cases like this, the @property function should probably have returned by reference and that not doing so is the real bug. Do you also think this is true in this case? I don't think so. One reason is for example, that you can not put contracts on your setter @property method then...

In my opinion, there is no bug with the definition of the @property methods here. The bug only arises through interactions with functions which take its parameters by reference.

Do you think this example in contrived? If yes, why?

On Saturday, 21 July 2018 at 07:10:51 UTC, Johannes Loher wrote:
> [...]

By the way, this does not mean I am totally against DIP 1016. It has a lot of benefits in my opinion. But I also think that Jonathan has a very valid point which definitely needs to be considered.

On Saturday, 21 July 2018 at 05:40:24 UTC, Nicholas Wilson wrote:

> It is not just the avoiding copying, if it were I'm not sure I'd support it. For me the greatest benefit is the increase in readability due to not having useless temporaries everywhere in ref heavy code (that may not be under API user's control).

Explicit is better than implicit.
(The Zen of Python)

Frankly speaking, my feeling is that D is becoming a horrible mess for the programmer...

And, BTW, I'm totally with Jonathan also on @implicit for copy ctor... I really really don't like it, another nail in the coffin of a beauty, symmetry, and intuitive syntax.

I'm starting to think that only a D3, with a lot of thing reorganised without the obsession of breaking changes can safe that beautiful language: Python was _almost_ killed in the 2-3 transaction, but kudos to Guido and the core time, it resurrected more strong than ever.

/Paolo

On Sat, 21 Jul 2018 at 00:15, Johannes Loher via Digitalmars-d <digitalmars-d@puremagic.com> wrote:
>
> On Saturday, 21 July 2018 at 05:59:37 UTC, Manu wrote:
> > [...]
>
> Let me give a concrete example of one of the situations Jonathan is describing. Consider the following code:
>
>
> struct Secret
> {
> public:
>      string key;
> }
>
> /* ... */
>
> genrateRandomKey(ref string key) {
>      key = /*  some code to actually generate the key */
> }
>
> Secret secret;
> generateRandomKey(secret.key);
>
>
> Now somebody else working on the project who sees the definition of Secret might think "Having public access to member variables is bad, so lets use property methods instead. This even allows us to do some contract checks!" and he changes the definition of Secret to the following:
>
>
> struct Secret
> {
> private:
>      string _key;
>
> public:
>      string key() @property const
>      {
>          return this._key;
>      }
>
>      void key(string key) @property
>      in
>      {
>          assert(key.length == 256)
>      }
>      do
>      {
>          this._key = key;
>      }
> }
>
>
> Now with DIP 1016, the use of generateRandomKey silently "fails", i.e. secret._key will not be changed by it, which in this case is a big problem as the key is still default initialized!
>
> Of course one might argue that genrateRandomKey should not take its argument by reference and rather just return the key instead. But in my experience, there is quite a lot of code like this out there (e.g. in order to avoid copying, string is probably not the best example here...).
>
> In one of your earlier answers you argued, that in cases like this, the @property function should probably have returned by reference and that not doing so is the real bug. Do you also think this is true in this case? I don't think so. One reason is for example, that you can not put contracts on your setter @property method then...
>
> In my opinion, there is no bug with the definition of the @property methods here. The bug only arises through interactions with functions which take its parameters by reference.
>
> Do you think this example in contrived? If yes, why?

So, to be clear; the 'gotcha' moment as proposed is this:
  1. Function mutates an input received by ref.
  2. Existing code is structured so the function is called with a
member of some lvalue.
  3. Member is _changed_ to be an accessor property for some reason
*and* the property returns the value by-val.
  4. Gotcha!

It is definitely pretty contrived.

1. The function in this scenario is clearly an 'out' parameter, so it
should use 'out', not ref.
2. A function like that would almost certainly return its result, not
mutate an argument. Using ref this way is a poor choice and subverts
move semantics.
3. Scenario depends on introduction of a getter, but any getter
property that returns a member by-val is almost certainly a primitive
type. A getter for a struct member would necessarily return a ref, or
it would experience large copy semantics every time the get is called!
4. A struct-type getter that returns by-val exhibits this gotcha in a
variety of ways; you 'get' the member (a by-val copy), then mutate a
member in any way, (ie, call a method), and you've accidentally
modified the copy, not the source value! (`ref` is not the bug)
  - note, in all other constructions of this same 'bug', existing
language semantics find it acceptable to silently accept the
accidental mutation of an expiring rvalue. Nobody is losing sleep at
night.
5. It's super-unlikely a function returns a primitive type via a
mutating parameter; primitive results would virtually always be
`return`ed, so instances of this pattern are only meaningfully
applicable to structs (see above).
6. Failing all that, I have accepted prior that there may exist
legitimate occurrences, but when you filter out all the cases above,
what is left? It's super rare at very least, and the scenario depends
on 2 things: code is CHANGED to use properties that return by-val, and
mutating function is niche enough that it doesn't fall into any cases
above. I don't know what that case is; at this stage, it's still
hypothetical.
Recommend: use the @disable technique defensively if you suspect the
use case is likely to be used incorrectly.

As I've said prior; this case is quite contrived, and it's clear that
the bug is actually in the property, not the use of 'ref'.
The accidental mutation of an rvalue as suggested can occur in a
variety of ways - only one of which is using ref.

The same 'bug' expressed in a simpler and more likely way:

// a struct that shall be the member
struct M {
  int x;
  void mutate() { ++x; }
}

// the original (working) code
struct S {
  M member;
}
S s;
s.member.mutate();

// the user adds a property in place of an existing member, but the
property incorrectly returns by-val
struct S {
  private M _m;
  M member() { return _m; }
}
S s;
s.member.mutate(); // <- oops, mutated an rvalue!

There are countless ways you can construct the same bug. ref doesn't
contact this problem in a general way, so a solution to this class of
problem shouldn't be ref's responsibility.
It may be possible to argue that the existing ref semantics may
provide defense against a very small surface area of this class of
bug, but I'll never find that narrow advantage weighs favourably
against all the other advantages of this DIP.


...all that said, we understand that there is value in inhibiting calls with rvalues in some cases. We address this in a very nice way with @disable, which is also nicely symmetrical such that the limitation may by applied to rval or lval's.


> By the way, this does not mean I am totally against DIP 1016. It has a lot of benefits in my opinion. But I also think that Jonathan has a very valid point which definitely needs to be considered.

Trust me, I've been considering Jonathan's case painstakingly for 10 years!!
I think it's a contrived, and fairly weak argument. If you consider
the problem as a *class of problem*, which it is, I don't understand
how the existing rule can be so zealously defended in the face of a
bunch of advantages, when all other constructions of the exact same
problem are silently allowed, and literally nobody complains about
them ever!

On Friday, 20 July 2018 at 23:19:08 UTC, Nicholas Wilson wrote:
> On Friday, 20 July 2018 at 16:39:46 UTC, Dukc wrote:
>> How so? It could be made it act exactly as if the temporary was made just before the function call, meaning the lifetime would end at the end of current scope.
>>
>
> ... which is exactly what this DIP proposes ...
>

... except the compiler would do that only when appending .byRef.

>> Of course, this required compiler magic. A library solution would have exactly the limits you said.
>
> ... and why its a DIP, and not a phobos PR.

With that I agree.

Thanks a lot, Manu, I'm a huge fan of this.

Wrt. binding rvalues to mutable refs, we could introduce something like `-transition=rval_to_mutable_ref` to have the compiler list all matching call sites.

Wrt. `auto ref`, I'd very much like to see its semantics change to 'pass this argument in the most efficient way' (depending on type and ABI, not just lvalue-ness of argument) at some point in the future.

On Saturday, 21 July 2018 at 08:55:59 UTC, Paolo Invernizzi wrote:
> Frankly speaking, my feeling is that D is becoming a horrible mess for the programmer...

> I'm starting to think that only a D3, with a lot of thing reorganised without the obsession of breaking changes can safe that beautiful language...

If a transition from D to D2 is one cause of that mess, why a new transition wouldn't continue that mess.

And yes as new fellow playing with this language since 2012, I think it's becoming more like a C++ in madness.

John.

On Saturday, 21 July 2018 at 12:54:28 UTC, JohnB wrote:
> On Saturday, 21 July 2018 at 08:55:59 UTC, Paolo Invernizzi wrote:
>> Frankly speaking, my feeling is that D is becoming a horrible mess for the programmer...
>
>> I'm starting to think that only a D3, with a lot of thing reorganised without the obsession of breaking changes can safe that beautiful language...
>
> If a transition from D to D2 is one cause of that mess, why a new transition wouldn't continue that mess.
>
> And yes as new fellow playing with this language since 2012, I think it's becoming more like a C++ in madness.
>
> John.

Not to talk about the recently revived rcstring, that by default are not a range... guess it...

/P