On 11/16/15 3:48 PM, Andrei Alexandrescu wrote:

>
> I think the main problem is the difficulty of getting from "I want to
> make this method work with mutable and nonconst data" to "I have a
> working solution using inout".

Again, it's not that hard. I have had little trouble with using it instead of const where it is appropriate. There are a couple rough edges, but you haven't hit them yet.

>
> The semantics is complex and difficult. Error messages are horrifying.
> Even explaining code that works is hard. Once a solution works, trying
> to change it in meaningful ways again makes the code not work, and again
> with uninformative error messages. So there's resistance to changing
> working code.

I just did the naive thing and slapped inout on your functions. This is what I got:

persistent_list2.d(146): Error: cannot implicitly convert expression (( List!(immutable(int)) __slList1681 = List(null, null);
 , __slList1681).this(n, this.allocator())) of type List!(immutable(int)) to inout(List!(immutable(int)))

WTF!

This is not how it used to be. What the hell is __slList1681? And what is List(null, null)?

This has to be some sort of lowering that was added to the compiler, I have never seen such horrible messages for inout. How it should look:

Error: cannot implicitly convert expression List(n, allocator) of type List!(immutable(int)) to inout(List!(immutable(int)))

Note, this is not an inout problem. If I put immutable(List) as the return type, I get:

persistent_list2.d(146): Error: cannot implicitly convert expression (( List!(immutable(int)) __slList1681 = List(null, null);
 , __slList1681).this(n, this.allocator())) of type List!(immutable(int)) to immutable(List!(immutable(int)))

We can and should fix this. I'll file an issue if none has been filed.

> At the same time, we have a wonderful language ready to help with
> features that didn't exist when we introduced inout. Search
> http://dpaste.dzfl.pl/52a3013efe34 for QList - it's restricted properly,
> works very well, and is easy to explain.

Actually, it's not easier to explain or maintain, and the solution existed before inout was introduced (in fact one of the reasons *to* use inout was to avoid QList-like solutions).

Example:

static QList cons(QList)(T head, auto ref QList lst)
    if (is(Unqual!QList == List))

Tell me how cons guarantees not to modify lst? Is it convention or compiler guarantees? find the bug:

static QList cons(QList)(T head, auto ref QList lst)
    if (is(Unqual!QList == List))
{
List result;
result._allocator = either(lst.allocator, theAllocator);
import std.conv : emplace;
void[] buf = result.allocator.allocate(Node.sizeof);
auto n = emplace!(const Node)(buf, head, lst.root, 1);
incRef(lst.root);
result.root = n;
static if(!is(QList == const)) lst.root = lst.root.next;
return result;
}

The point of const is to provide a mechanism to those reading the signature to say "this function will NOT change the parameter, and the compiler guarantees it."

I remember reading originally why Walter put const into the language -- because C++ developers who used const extensively to provide guarantees simply wouldn't use D if it didn't provide a mechanism to do this. I don't recall the exact wording, but I'm sure it's somewhere in the ng.

The point of inout was to provide what const does, but allow you to hook the output const flavor to the input const flavor. That's all.

> Walter and I think inout didn't turn out well. It became a little
> monster mastering a swamp. Template solutions can drain that swamp, make
> the monster disappear, and build nice things on that field.

I disagree, and appeals to authority is not very convincing, no matter how colorful. I think inout works great for what it's good at. It has some finishing to do, but the concept is sound and works fine. It's a perfect fit for containers in general. In this particular case it's not necessary.

We absolutely should fix the error messages, they are not very helpful.

> I think we should slowly marginalize inout - discourage it from new
> code, document and publicize better alternatives, the works.

Because we have a poor error message? I can assure you this is not endemic to inout.

>> In this case, it appears to work only because of your cast of _allocator
>> to mutable whenever you access it via allocator. Other than that, the
>> only other member is the node pointer, which is const. Effectively, A
>> list's data is always const, so there is no reason to make the struct
>> itself const.
>
> Plenty of reason. The list may be member in an object. Also, if you
> don't have const List you can't compose List with itself.

Huh? Mutable List casts to const List just fine. If I always return a mutable List from a composing operation, then it works for const List as well (both as the receiver of the return value and as the parameter).

What I meant to say is that a const(List) isn't particularly needed to guard the List's data. That is already const.

-Steve

On 11/16/2015 04:18 PM, Jonathan M Davis wrote:
> When you can templatize the function on the type that would otherwise be
> inout, then there really isn't any reason to use inout. But what do you
> do when you're dealing with a member function - especially on a class? I
> don't know of any way to templatize a member function on the type of the
> this pointer/reference, but maybe there's a template feature with which
> I'm not familiar enough. But even if there _is_ such a feature, it won't
> work for virtual functions.

It does. Write one-liners that forward to the template function, as shown in http://dpaste.dzfl.pl/52a3013efe34 (search the page for QList). -- Andrei

On 11/16/2015 04:36 PM, Steven Schveighoffer wrote:
> We can and should fix this. I'll file an issue if none has been filed.

That's great! -- Andrei

On Monday, 16 November 2015 at 21:25:52 UTC, Timon Gehr wrote:
> On 11/16/2015 10:18 PM, Jonathan M Davis wrote:
>> I don't know of any way to templatize a member function on the type of
>> the this pointer/reference, but maybe there's a template feature with
>> which I'm not familiar enough.
>
> Indeed there is.
>
> class C{
>     void foo(this T)(){ pragma(msg, T); }
> }
>
> class D: C{}
>
> void main(){
>     auto c=new const(C)();
>     c.foo();
>     auto d=new immutable(D)();
>     d.foo();
> }

Cool. That's good to know about. Thanks.

- Jonathan M Davis

On Monday, 16 November 2015 at 21:47:07 UTC, Andrei Alexandrescu wrote:
> On 11/16/2015 04:18 PM, Jonathan M Davis wrote:
>> When you can templatize the function on the type that would otherwise be
>> inout, then there really isn't any reason to use inout. But what do you
>> do when you're dealing with a member function - especially on a class? I
>> don't know of any way to templatize a member function on the type of the
>> this pointer/reference, but maybe there's a template feature with which
>> I'm not familiar enough. But even if there _is_ such a feature, it won't
>> work for virtual functions.
>
> It does. Write one-liners that forward to the template function, as shown in http://dpaste.dzfl.pl/52a3013efe34 (search the page for QList). -- Andrei

Well, that does reduce the code duplication, but you're still forced to have up to three separate functions to get each of the different return types (even if they are then simply wrapper functions). And compare that with simply declaring the same function once with its implementation internal to itself. It's just that you use inout instead of const. How is that worse?

There are cases where inout doesn't work as well as const does due to limitations with inout, but in many (most?) cases, the only difference between using const and inout is that you're typing const instead of inout. So, if anything, that makes it sound like fixing inout so that it works in those other cases would be better than avoiding it in favor of splitting out the implementation into a separate function and having multiple wrapper functions in the API.

- Jonathan M Davis

On Monday, 16 November 2015 at 21:36:41 UTC, Steven Schveighoffer wrote:
> I just did the naive thing and slapped inout on your functions. This is what I got:
>
> persistent_list2.d(146): Error: cannot implicitly convert expression (( List!(immutable(int)) __slList1681 = List(null, null);
>  , __slList1681).this(n, this.allocator())) of type List!(immutable(int)) to inout(List!(immutable(int)))
>
> WTF!

Wow is that hideous. I don't recall ever seeing error messages that obtuse.

> We can and should fix this. I'll file an issue if none has been filed.

Please do.

- Jonathan M Davis

On 11/16/15 5:02 PM, Jonathan M Davis wrote:
> On Monday, 16 November 2015 at 21:47:07 UTC, Andrei Alexandrescu wrote:
>> On 11/16/2015 04:18 PM, Jonathan M Davis wrote:
>>> When you can templatize the function on the type that would otherwise be
>>> inout, then there really isn't any reason to use inout. But what do you
>>> do when you're dealing with a member function - especially on a class? I
>>> don't know of any way to templatize a member function on the type of the
>>> this pointer/reference, but maybe there's a template feature with which
>>> I'm not familiar enough. But even if there _is_ such a feature, it won't
>>> work for virtual functions.
>>
>> It does. Write one-liners that forward to the template function, as
>> shown in http://dpaste.dzfl.pl/52a3013efe34 (search the page for
>> QList). -- Andrei
>
> Well, that does reduce the code duplication, but you're still forced to
> have up to three separate functions to get each of the different return
> types (even if they are then simply wrapper functions). And compare that
> with simply declaring the same function once with its implementation
> internal to itself. It's just that you use inout instead of const. How
> is that worse?

I'll reiterate what I said elsewhere: inout is better, safer, and easier to maintain than the template solution.

Example:

static QList cons(QList)(T head, auto ref QList lst)
    if (is(Unqual!QList == List))

Tell me how cons guarantees not to modify lst? Is it convention or compiler guarantees? find the bug:

static QList cons(QList)(T head, auto ref QList lst)
    if (is(Unqual!QList == List))
{
List result;
result._allocator = either(lst.allocator, theAllocator);
import std.conv : emplace;
void[] buf = result.allocator.allocate(Node.sizeof);
auto n = emplace!(const Node)(buf, head, lst.root, 1);
incRef(lst.root);
result.root = n;
static if(!is(QList == const)) lst.root = lst.root.next;
return result;
}

The point of const is to provide a mechanism to those reading the signature to say "this function will NOT change the parameter, and the compiler guarantees it." The point of inout is the same, but allows returning a portion of the parameters without resorting to casts or wrappers.

-Steve

On 11/16/2015 05:02 PM, Jonathan M Davis wrote:
> It's just that you use inout instead of const. How is that worse?

The short answer is my perception is inout is too complicated for what it does. -- Andrei

On Monday, 16 November 2015 at 16:58:24 UTC, Lionello Lunesu wrote:
> If it's RC we want, then @mutable is an axe when what we need is a scalpel.
>
> The non-observability comes from the fact the refcount is changed when the caller has lost its (const) reference and constness is a moot point. Changing refcount is fine now, provided the object is not immutable. As far as other outstanding const references go, these already expect changes to happen.
>
> This is what makes refcount special and provably safe to mutate. As long as we can ensure the object is not immutable, the object is allowed to change its own refcount. But refcount needs to be special cased, somehow, or else we'll end up with some C++ like `mutable`.

I'm not a compiler or language-design guy, but something strikes
me as odd about this whole discussion.  It seems to me that a lot
of it depends on a presumption that one wants the refcount field
to be user-visible, and that therefore it must also have some
user-visible type and perhaps some attributes.  But at bottom,
doesn't a refcount field behave more like a vptr for a class
instance, which is there in the underlying storage model but is
not directly user-visible?  Or similar to whatever metadata is
used for the start/end pointers of a dynamic array?  As such, is
there any reason for a refcount to be user-visible in the first
place, other than to the compiler-internals implementor?  I have
to wonder whether a refcount field could be appended to the data
payload via some sort of @rc attribute instead of as an explicit
class/struct member, and perhaps referenced via some implicit
pseudo member when the field needs to be manipulated.  Then there
perhaps wouldn't need to be any requirement to define any new
kinds of storage categories for this field, in the same way that
there isn't an explicit storage category for a vptr field.

I could be all wet here with such ideas, but I'm just wondering ...

On 11/16/2015 05:30 PM, Steven Schveighoffer wrote:
> I'll reiterate what I said elsewhere: inout is better, safer, and easier
> to maintain than the template solution.

Do you have a working solution in dpaste? No change to the return types please. -- Andrei