On Friday, March 30, 2018 11:01:23 Manu via Digitalmars-d wrote:
> On 29 March 2018 at 21:08, Jonathan M Davis via Digitalmars-d
>
> <digitalmars-d@puremagic.com> wrote:
> > On Thursday, March 29, 2018 23:28:54 Nick Sabalausky  via Digitalmars-d
> >
> > wrote:
> >> On 03/23/2018 09:06 PM, Jonathan M Davis wrote:
> >> > My biggest concern in all of this is that I don't want to see ref
> >> > start
> >> > accepting rvalues as has been occasionally discussed. It needs to be
> >> > clear when a function is accept an argument by ref because it's going
> >> > to mutate the object and when it's accepting by ref because it wants
> >> > to
> >> > avoid a copy.
> >>
> >> That ship sailed ages ago: It's already unclear. If we want to fix
> >> that,
> >> we can fix it, but blocking rvalue ref does nothing for that cause.
> >
> > Really? And how often does ref get used just to avoid copying? You can almost always look at a function, see that it accepts ref, and know that it's supposed to mutate the argument. Functions that want to avoid copying lvalues usually use auto ref, not ref, whereas if ref accepted rvalues, a number of folks would start using it all over the place to avoid copying. Right now, folks rarely use ref that way, because it becomes too annoying to call the function with an rvalue. So, while it might not be the case 100% of the time right now that ref is used with the purpose of mutating the argument, it almost always is. As such, you can pretty reliably look at a function signature and expect that if one of its parameters is ref, it's going to be mutating that argument. The function that accepts an argument by ref with no intention of mutating it is very much the exception, and I really don't want to see that change.
>
> Interesting. Just understand that you're trading that feeling for a
> suite of edge cases though. Accepting asymmetric calling rules is a
> pretty big cost to pay for that 'nice thought'.
> That idea also dismisses the existence of the set of cases where ref
> is genuinely useful/correct. Your sentiment effectively puts those use
> cases into the position of 2nd-class citizens.
> I understand your sentiment, but I think the cost is not balanced, or
> even particularly fair with respect to users in those niche groups :/

I'm not arguing against having a way to indicate that a parameter accepts both rvalues and lvalues and that the rvalues get copied to an invisible variable so that they can be passed as an lvalue. I'm arguing against simply making ref have that behavior. Assuming that the details of how that worked internally didn't involve problematic stuff like the rvalue reference stuff that Andrei and Walter are so against, having an attribute such as @rvalue to attach to a ref parameter to allow it to accept rvalues would be fine with me. I just don't want ref by itself to lose its current semantics, because that would dilute its meaning and increase its amiguity.

I want to be able to look at a function signature, see ref without other qualifiers, and be reasonably certain that the function is supposed to be mutating that argument, whereas if ref by itself accepted rvalues, then we lose that. If an attribute were used to make it allow rvalues, then we wouldn't.

- Jonathan M Davis

On 30 March 2018 at 12:24, Jonathan M Davis via Digitalmars-d <digitalmars-d@puremagic.com> wrote:
> On Friday, March 30, 2018 11:01:23 Manu via Digitalmars-d wrote:
>> On 29 March 2018 at 21:08, Jonathan M Davis via Digitalmars-d
>>
>> <digitalmars-d@puremagic.com> wrote:
>> > On Thursday, March 29, 2018 23:28:54 Nick Sabalausky  via Digitalmars-d
>> >
>> > wrote:
>> >> On 03/23/2018 09:06 PM, Jonathan M Davis wrote:
>> >> > My biggest concern in all of this is that I don't want to see ref
>> >> > start
>> >> > accepting rvalues as has been occasionally discussed. It needs to be
>> >> > clear when a function is accept an argument by ref because it's going
>> >> > to mutate the object and when it's accepting by ref because it wants
>> >> > to
>> >> > avoid a copy.
>> >>
>> >> That ship sailed ages ago: It's already unclear. If we want to fix
>> >> that,
>> >> we can fix it, but blocking rvalue ref does nothing for that cause.
>> >
>> > Really? And how often does ref get used just to avoid copying? You can almost always look at a function, see that it accepts ref, and know that it's supposed to mutate the argument. Functions that want to avoid copying lvalues usually use auto ref, not ref, whereas if ref accepted rvalues, a number of folks would start using it all over the place to avoid copying. Right now, folks rarely use ref that way, because it becomes too annoying to call the function with an rvalue. So, while it might not be the case 100% of the time right now that ref is used with the purpose of mutating the argument, it almost always is. As such, you can pretty reliably look at a function signature and expect that if one of its parameters is ref, it's going to be mutating that argument. The function that accepts an argument by ref with no intention of mutating it is very much the exception, and I really don't want to see that change.
>>
>> Interesting. Just understand that you're trading that feeling for a
>> suite of edge cases though. Accepting asymmetric calling rules is a
>> pretty big cost to pay for that 'nice thought'.
>> That idea also dismisses the existence of the set of cases where ref
>> is genuinely useful/correct. Your sentiment effectively puts those use
>> cases into the position of 2nd-class citizens.
>> I understand your sentiment, but I think the cost is not balanced, or
>> even particularly fair with respect to users in those niche groups :/
>
> I'm not arguing against having a way to indicate that a parameter accepts both rvalues and lvalues and that the rvalues get copied to an invisible variable so that they can be passed as an lvalue. I'm arguing against simply making ref have that behavior. Assuming that the details of how that worked internally didn't involve problematic stuff like the rvalue reference stuff that Andrei and Walter are so against, having an attribute such as @rvalue to attach to a ref parameter to allow it to accept rvalues would be fine with me. I just don't want ref by itself to lose its current semantics, because that would dilute its meaning and increase its amiguity.

https://imgflip.com/i/27gjv9


> I want to be able to look at a function signature, see ref without other qualifiers, and be reasonably certain that the function is supposed to be mutating that argument, whereas if ref by itself accepted rvalues, then we lose that. If an attribute were used to make it allow rvalues, then we wouldn't.

I don't know any reason why someone wouldn't attribute the argument
'const' if it doesn't intend to write to it. Likewise 'return ref' if
it's going to be modified and returned.
I'm not sure your concern is actually a thing...?

On Friday, March 30, 2018 14:47:06 Manu via Digitalmars-d wrote:
> On 30 March 2018 at 12:24, Jonathan M Davis via Digitalmars-d
> > I want to be able to look at a function signature, see ref without other
> > qualifiers, and be reasonably certain that the function is supposed to
> > be
> > mutating that argument, whereas if ref by itself accepted rvalues, then
> > we lose that. If an attribute were used to make it allow rvalues, then
> > we wouldn't.
>
> I don't know any reason why someone wouldn't attribute the argument
> 'const' if it doesn't intend to write to it. Likewise 'return ref' if
> it's going to be modified and returned.
> I'm not sure your concern is actually a thing...?

const would do it, but given how restrictive const is in D, I don't see how it would be very reasonable to restrict passing rvalues to const. And honestly, I don't really want to encourage the use of const. It's fine if folks use it where it really works, and it's potentially even quite valuable there, but it seems like too often folks try to add const in places where it's just going to cause problems. It's particularly bad when folks try to add const to generic code - e.g. we recently had to revert a commit to std.random which worked with dynamic arrays but not other ranges because of const. And there have been other cases where folks have wanted to try to make stuff in Phobos "const-correct", which would cause problems. So, I'm not a big fan of the idea of doing anything that would make folks want to use const more.

Now, if you can convince Walter and Andrei to allow const ref to accept rvalues, then fine. I think that that's definitely worse than an attribute specifically for that, given how limiting const is, but it wouldn't screw up normal ref in the process, which is what I'm most worried about here. So, I don't think that going with const would be the best solution to the problem, but it's far better than making ref in general accept rvalues.

- Jonathan M Davis

On 30 March 2018 at 16:09, Jonathan M Davis via Digitalmars-d <digitalmars-d@puremagic.com> wrote:
> On Friday, March 30, 2018 14:47:06 Manu via Digitalmars-d wrote:
>> On 30 March 2018 at 12:24, Jonathan M Davis via Digitalmars-d
>> > I want to be able to look at a function signature, see ref without other
>> > qualifiers, and be reasonably certain that the function is supposed to
>> > be
>> > mutating that argument, whereas if ref by itself accepted rvalues, then
>> > we lose that. If an attribute were used to make it allow rvalues, then
>> > we wouldn't.
>>
>> I don't know any reason why someone wouldn't attribute the argument
>> 'const' if it doesn't intend to write to it. Likewise 'return ref' if
>> it's going to be modified and returned.
>> I'm not sure your concern is actually a thing...?
>
> const would do it, but given how restrictive const is in D, I don't see how it would be very reasonable to restrict passing rvalues to const.

I'm actually coming around to the idea of not restricting it to const...
But what I'm trying to say is, the number of cases where your
principle will feel violated should be as infrequent as the cases
where const is insufficient for 'reasons' ;)
As far as I know, in usages that I've ever encountered, I can't
imagine a case where you would feel violated :)


> And honestly, I don't really want to encourage the use of const. It's fine if folks use it where it really works, and it's potentially even quite valuable there, but it seems like too often folks try to add const in places where it's just going to cause problems. It's particularly bad when folks try to add const to generic code - e.g. we recently had to revert a commit to std.random which worked with dynamic arrays but not other ranges because of const. And there have been other cases where folks have wanted to try to make stuff in Phobos "const-correct", which would cause problems. So, I'm not a big fan of the idea of doing anything that would make folks want to use const more.

It's interesting... I recognise the general backlash against const. I
personally just make myself a `struct Mutable(T) { ... }` which I use
when I want to violate const ;)
But if it turns out that const is useless, then we really need to
reconsider the basic design of const >_<
Like, what's the point? You're advocating active discouragement of
const... why is there a feature which it's accepted is a loaded-gun?
Handing it to people creates a high probability they'll shoot
themselves in the feet.


> Now, if you can convince Walter and Andrei to allow const ref to accept rvalues, then fine. I think that that's definitely worse than an attribute specifically for that, given how limiting const is, but it wouldn't screw up normal ref in the process, which is what I'm most worried about here. So, I don't think that going with const would be the best solution to the problem, but it's far better than making ref in general accept rvalues.

Useful examples using 'return ref' have been presented, which I find quite compelling too. That's an interesting case of non-const ref. Safe to say I'm not convincing anyone of anything in any way other than DIP form.


But I'd like to understand your concern better. You say it's about
scanning an API and understanding some details from it based on seeing
'ref' written there... how does a function accepting an rvalue
interact with your visibility of the API?
Like, your criticism is with respect to understanding the API at a
glance... I don't understand how this proposal interferes with that in
any way?

Sadly, I don't think I'll be able to make it to DConf this year...
which is probably a reason for rejoice of literally everybody
attending! :P
It would be nice to workshop it in person though.

On 3/30/2018 6:03 PM, Manu wrote:
> Sadly, I don't think I'll be able to make it to DConf this year...

:-(

> which is probably a reason for rejoice of literally everybody
> attending! :P

You'll be missed.

> It would be nice to workshop it in person though.

On Friday, March 30, 2018 18:03:49 Manu via Digitalmars-d wrote:
> On 30 March 2018 at 16:09, Jonathan M Davis via Digitalmars-d
> > And honestly, I don't really want to encourage the use of const. It's fine if folks use it where it really works, and it's potentially even quite valuable there, but it seems like too often folks try to add const in places where it's just going to cause problems. It's particularly bad when folks try to add const to generic code - e.g. we recently had to revert a commit to std.random which worked with dynamic arrays but not other ranges because of const. And there have been other cases where folks have wanted to try to make stuff in Phobos "const-correct", which would cause problems. So, I'm not a big fan of the idea of doing anything that would make folks want to use const more.
>
> It's interesting... I recognise the general backlash against const. I personally just make myself a `struct Mutable(T) { ... }` which I use when I want to violate const ;)

Just don't do anything where you cast away const and then mutate. That violates the type system and results in undefined behavior. And depending on what the compiler chooses to do based on const, you could get some rather subtle and nasty bugs.

> But if it turns out that const is useless, then we really need to
> reconsider the basic design of const >_<
> Like, what's the point? You're advocating active discouragement of
> const... why is there a feature which it's accepted is a loaded-gun?
> Handing it to people creates a high probability they'll shoot
> themselves in the feet.

const makes some sense when you want to write code that accepts both mutable and immutable arguments, and there are times where it works perfectly fine to use const. I see no problem with using const in such situations. The problem is that there are a _lot_ cases where const simply can't be used in D, because you need to mutate _something_ involved, even if it's not something that's part of the "logical" state of the object - e.g. in D, you can't put a mutex in a type and then have it protect anything in a const member function, because locking the mutex would require mutating it, and D provides no backdoors like C++'s mutable. What's const is actually const.

Also, const tends to interact very badly with generic code. Whatever API you're duck typing has to include const as part of it, or you can't use const - e.g. ranges can't be const, because they have to be mutated to be iterated, and because the range API does not require that properties like front or empty be const, no generic code can assume that even those work with const (and for many ranges, they _can_'t be const - especially when one range wraps another). In general, it can be quite difficult to even do something like have a tail-const range over a container. Sometimes, const can be made to work in more complex situations, but often it can't, and when it can't, it's often a royal pain.

For non-generic code that can clearly treat an object as fully const without needing any backdoors, const is just fine, and it may even help prevent bugs. But for a _lot_ of code - especially idiomatic D code that does a lot with templates and ranges - const simply doesn't work.

So, const can be used on some level, but anyone who tries to be "const-correct" in D like many of us usually try to do in C++ is in for a world of frustration.

If you haven't, I'd suggest that you read this article I recently wrote on the topic:

http://jmdavisprog.com/articles/why-const-sucks.html

And here's the newsgroup thread discussing it:

https://forum.dlang.org/thread/mailman.581.1520247475.3374.digitalmars-d-announce@puremagic.com

> > Now, if you can convince Walter and Andrei to allow const ref to accept rvalues, then fine. I think that that's definitely worse than an attribute specifically for that, given how limiting const is, but it wouldn't screw up normal ref in the process, which is what I'm most worried about here. So, I don't think that going with const would be the best solution to the problem, but it's far better than making ref in general accept rvalues.
>
> Useful examples using 'return ref' have been presented, which I find quite compelling too. That's an interesting case of non-const ref. Safe to say I'm not convincing anyone of anything in any way other than DIP form.

Yes, at this point, convincing Walter or Andrei will require writing a DIP. A well-written proposal that clearly doesn't have the downsides that they're so against may have a chance with them, but without something that formally and clearly provides the arguments, I don't think that they're even going to pay much attention at this point. The topic has been debated to death previously, and without a DIP, it really doesn't matter what Walter and Andrei think, since at this point, they wouldn't introduce a language change like that without a DIP. So, they'll just do like Andrei did in response to this thread and tell you to write a DIP rather than be willing to discuss it further on its own. But no matter the topic, if it involves a major change to the language, since that now requires a DIP, there's not much point in arguing the matter with Walter or Andrei beforehand except to help iron out the DIP. Increasingly, they don't get involved in such discussions outside of the DIP process, because it eats up too much of their time.

> But I'd like to understand your concern better. You say it's about
> scanning an API and understanding some details from it based on seeing
> 'ref' written there... how does a function accepting an rvalue
> interact with your visibility of the API?
> Like, your criticism is with respect to understanding the API at a
> glance... I don't understand how this proposal interferes with that in
> any way?

The only thing that interfers with that is making ref with no other qualifiers accept rvalues. If it's const or inout or immutable or something like @rvalue, then you can look at the API and know that ref is on the parameter so that lvalues will not be copied, and if it accepts rvalues too (be by it copying them to invisible variables to pass as lvalues or whatever), then that may mean a performance hit due to the lack of a move, but you know that the function is not mutating the argument with the intention of giving the result to the caller. If it's const, then mutation wouldn't be possible, and if it's @rvalue, then it simply wouldn't make sense for the function to be providing the result back to the caller, because that would only work with lvalues. @rvalue would indicate that the function _could_ mutate the argument (so the restrictions imposed by const would not apply), and relying on the argument not being mutated would be asking for bugs, but it would clearly be ref for the purpose of avoiding copying lvalues and not because the purpose is to mutate the argument and provide the result.

As long as naked ref does not accept rvalues, then you can look it at and reasonably assume that the function is intended to mutate the argument and provide the result to the caller. It also prevents accidentally passing an rvalue and thus avoids the bugs that would come from passing an rvalue to a function that's supposed to be mutating its arguments.

As it stands, folks do still sometimes write functions which accept arguments by ref to avoid copying lvalues, but because of the annoyances with passing rvalues, most folks don't do that. And if the solution were to add @rvalue to make ref accept rvalues, then anyone who wanted to use ref for efficiency reasons would use @rvalue on the ref, and it would make it that much more reliable that a naked ref indicated that the function was supposed to mutate the argument. It also wouldn't introduce the problem of accidentally passing rvalues to ref parameters like making ref in general accept rvalues would.

So, unless you're proposing that ref in general accept rvalues, I'm not necessarily criticizing what you're proposing. I do have reservations about using const instead of something like @rvalue to indicate that rvalues should be accepted due to the restrictions on const, but I also don't care enough about being able to pass rvalues by ref to make a big stink about const ref and auto ref being the only way to accept rvalues without copying lvalues. I think that it's worth pointing out that making const part of the solution is problematic, but if you write a DIP that Walter and Andrei accept that uses const to indicate that ref will accept rvalues, then it's generally not going to cause problems for me, since I won't be using it enough for the restrictions on const to matter to me. So, I think that it's a worse solution, but a subpar solution to this problem only affects me with regards to the fact that I want D to be a great language (and thus would prefer not to have subpar solutions) and insofar as I run into other people's code that uses the technique. I might end up using it occasonally, but for the most part, the few cases where I'd care, I've been able to use auto ref to solve the problem.

Most of that I write works just fine passing stuff by value (and if it doesn't, I tend to make the type a reference type to solve the problem), and I'm not likely to do much with calling extern(C++) functions outside of auto-generated code. So, while I acknowledge the problem and have no problem with it being solved, it simply isn't an issue that affects me much. As such, it will be that much more annoying if the solution makes existing parts of the language worse.

> Sadly, I don't think I'll be able to make it to DConf this year...
> which is probably a reason for rejoice of literally everybody
> attending! :P
> It would be nice to workshop it in person though.

LOL. I don't know that anyone would rejoice about it. If they really don't want to talk to you, they probably just won't talk to you.

But it's not always easy to get away, and it's not necessarily cheap, so it's understandable if you can't come. Fortunately, I generally haven't had a problem with getting the time off to go. It's the cost that generally makes the trip problematic for me, though I've managed to make it works thus far, and this year, I'm speaking, so it won't be anywhere near as expensive for me as the last couple of years were.

If you can't make it this year, then hopefully you'll be able to make it next year, wherever it happens to end up being.

- Jonathan M Davis

On Friday, March 30, 2018 18:52:38 Walter Bright via Digitalmars-d wrote:
> On 3/30/2018 6:03 PM, Manu wrote:
> > Sadly, I don't think I'll be able to make it to DConf this year...
> :
> :-(
> :
> > which is probably a reason for rejoice of literally everybody attending! :P
>
> You'll be missed.

My usual response to that is that it's better to be missed than to be hit. ;)

But yes, it will be a shame if it's unable to come.

- Jonathan M Davis

On 30 March 2018 at 18:52, Walter Bright via Digitalmars-d <digitalmars-d@puremagic.com> wrote:
> On 3/30/2018 6:03 PM, Manu wrote:
>>
>> Sadly, I don't think I'll be able to make it to DConf this year...
>
>
> :-(
>
>> which is probably a reason for rejoice of literally everybody attending! :P
>
>
> You'll be missed.

Thank your stars... I wouldn't be able to resist heckling everyone on this the whole time! :P

On 30.03.2018 11:06, Atila Neves wrote:
> 
> Right, and I was assuming (perhaps incorrectly) that this existing code was C++, hence me being on board with binding rvalues to const ref there.

That must not happen. D const and C++ const don't even mean the same thing, and now suddenly you will see people use extern(C++) just to get the binding of rvalues to ref. Add to this the confusing overload behavior if const signifies anything other than read-only.

On 3/31/18 9:28 AM, Timon Gehr wrote:
> On 30.03.2018 11:06, Atila Neves wrote:
>>
>> Right, and I was assuming (perhaps incorrectly) that this existing code was C++, hence me being on board with binding rvalues to const ref there.
> 
> That must not happen. D const and C++ const don't even mean the same thing, and now suddenly you will see people use extern(C++) just to get the binding of rvalues to ref. Add to this the confusing overload behavior if const signifies anything other than read-only.

This is a very important aspect that must not be overlooked.

Allow me to reiterate: the one way to get this going is convert the forum dialog into a team and a DIP.


Andrei