On Tuesday, 17 November 2015 at 01:49:05 UTC, Steven Schveighoffer wrote:
[...]
>
> I think inout is worth saving, it needs a few tweaks and a good explanation.
>
> -Steve

Sounds interesting. Might your ideas help to fix the difficulties inout has with composability? That would be excellent.

Joseph

On Tuesday, 17 November 2015 at 01:49:05 UTC, Steven Schveighoffer wrote:
> I think it's quite clear that inout in its current form is a point of huge confusion. I want to work to fix this perception (and fix the broken corners of the implementation).
>
> What I would ask is for those who at the moment dislike it keep an open mind, and hold back on your pitchforks. I need to prepare a suitable defense, and it's difficult to shake off the "confusing and complex" albatross without having sufficient time to create something that is easy to understand/explain for something that is conceptually simple, but requires a complex proof.
>
> What I am afraid of is that someone makes a decision that something is bad, and by the time a good defense, or workable proposal is mounted, the answer is "we already discussed that, it's bad. Let's move on."
>
> At least with this, we have a feature that is part of the language, so is unlikely to go away. But I'd hate to see a mob of PR requests that remove inout from all of phobos/druntime before I can counter this.
>
> I think inout is worth saving, it needs a few tweaks and a good explanation.
>
> -Steve

Regarding my question on StackOverflow, I understand that I had an inaccurate picture of how inout works, which was causing my problem. These are also problems with const and immutable to a degree. However, inout is like a fourth qualifier that you have to deal with that is separate from const and immutable, due to how it currently works. In that light, I still don't think it pulls its weight. What user will have the presence of mind to define their Foobar type with an inout/const/immutable toString function? Almost none, and so Nullable!Foobar.opEquals cannot be inout. I first encountered this problem trying to get Nullable.toString to work with inout, and couldn't. That's my main gripe with inout.

On 11/16/15 9:00 PM, Meta wrote:
> On Tuesday, 17 November 2015 at 01:49:05 UTC, Steven Schveighoffer wrote:
>> I think it's quite clear that inout in its current form is a point of
>> huge confusion. I want to work to fix this perception (and fix the
>> broken corners of the implementation).
>>
>> What I would ask is for those who at the moment dislike it keep an
>> open mind, and hold back on your pitchforks. I need to prepare a
>> suitable defense, and it's difficult to shake off the "confusing and
>> complex" albatross without having sufficient time to create something
>> that is easy to understand/explain for something that is conceptually
>> simple, but requires a complex proof.
>>
>> What I am afraid of is that someone makes a decision that something is
>> bad, and by the time a good defense, or workable proposal is mounted,
>> the answer is "we already discussed that, it's bad. Let's move on."
>>
>> At least with this, we have a feature that is part of the language, so
>> is unlikely to go away. But I'd hate to see a mob of PR requests that
>> remove inout from all of phobos/druntime before I can counter this.
>>
>> I think inout is worth saving, it needs a few tweaks and a good
>> explanation.
>>
>
> Regarding my question on StackOverflow, I understand that I had an
> inaccurate picture of how inout works, which was causing my problem.
> These are also problems with const and immutable to a degree.

I can understand the confusion.

However, here is the truth: you only need to define opEquals for const or mutable. You don't need both (if your wrapped type defines both, then the const one can be called in both cases), and you don't need an inout version. Here is why: inout is simply *equivalent to const* (but without the implicit casting) when there is no inout on the return type. Originally this was flagged as an error, but because of templates and IFTI, it made writing template functions much harder.

But here is a working system that has no inout:

struct Wrapper(T)
{
    T t;

    static if(is(typeof(const(T).init == const(T).init)))
    {
        bool opEquals(const(Wrapper) other) const
        {
            return t == other.t;
        }
        bool opEquals(const(T) val) const
        {
            return t == val;
        }
    }
    else
    {
        bool opEquals(Wrapper other)
        {
            return t == other.t;
        }
        bool opEquals(T val)
        {
            return t == val;
        }
    }
}

struct Test
{
    bool opEquals(Test t)
    {
        return true;
    }
}

struct Test2
{
    bool opEquals(inout(Test2) t) inout
    {
        return true;
    }
}

struct Test3
{
    bool opEquals(const(Test3) t) const
    {
        return true;
    }
}

void foo(T)()
{
    Wrapper!T a, b;
    assert(a == b);
    assert(a == T());
}

void main()
{
    foo!Test();
    foo!Test2();
    foo!Test3();
}

This doesn't solve the case that has only a mutable and immutable opEquals, and no const version, but you get the idea.

> However,
> inout is like a fourth qualifier that you have to deal with that is
> separate from const and immutable, due to how it currently works. In
> that light, I still don't think it pulls its weight. What user will have
> the presence of mind to define their Foobar type with an
> inout/const/immutable toString function? Almost none, and so
> Nullable!Foobar.opEquals cannot be inout.

In almost all cases I've come across, inout isn't another thing that needs handling. It's simply an alternative to const (when applicable). In cases where it is applicable, you write MUCH LESS code. Where it breaks down is composition (i.e. creating a type with an inout member) and nested functions.

> I first encountered this
> problem trying to get Nullable.toString to work with inout, and
> couldn't. That's my main gripe with inout.

This may have hit one of the warts of inout.

-Steve

On Monday, 16 November 2015 at 00:00:18 UTC, Andrei Alexandrescu wrote:
> This illustrates a simple procedural problem. The entire point of my posting a simple example of a complete container was to avoid hypotheticals such as this. The code is there and shows that no, physical const does not suffice. At least I don't know how to do it. If you think it does there's one way to show it, it's easy - write the code that does it.
>
> We could sit on our testes all day long, speculate how things ought to work, and feel awfully smart in the process. It's very easy to do. Hell, I've done it more than too many times. What's more difficult is have a positive, constructive approach that builds on a weak solution to improve it. This is the kind of dialog we need to foster.

Sadly, design I would love to see also doesn't seem to be implementable but because of different issue - missing scope control. I gave a quick go to implement something that shows my desired semantics - https://gist.github.com/mihails-strasuns-sociomantic/1d7529eef723b1132564 (only essential functionality to show stuff). As you may see it is totally different from your initial proposal - this is why making constructive feedback is rather hard :)

But chosing between breaking physical const via @mutable and waiting until we finally get lond wanted scope control tools I am very much in favor of the latter.

On Monday, 16 November 2015 at 14:45:35 UTC, Andrei Alexandrescu wrote:
> The challenge is proving that a mutation is not observable. Got an attack on that? -- Andrei

Here's what I wrote a few days ago, without having read the current discussion, so it's not a perfect fit:
http://wiki.dlang.org/DIP85

For the case of lazy initialization, assigning a member exactly once is safe, if it hasn't been read and returned before. The compiler can check this by automatically inserting a few asserts. The rules in the DIP aren't completely water-tight, but they are a good approximation.

But for members that have to be mutated several times (e.g. refcount), this obviously isn't useful. I think we can make this @system, thereby forcing implementors to write @trusted code and (hopefully) consider the consequences thoroughly. But the compiler can also assist with that. For example, it could detect whether a "privately mutable" value, or something depending on it, is returned from the function, and reject that.

Maybe the implementation for the two kinds of use-cases (write once vs write multiple times) can be unified, too.

On Monday, 16 November 2015 at 16:58:24 UTC, Lionello Lunesu wrote:
> On 16/11/15 22:45, Andrei Alexandrescu wrote:
>> The challenge is proving that a mutation is not observable. Got an
>> attack on that? -- Andrei
>
> Forgive me, I haven't followed the RC discussions closely, so I miss a lot of context. Feel free to point me to existing threads/articles.
>
> If it's RC we want, then @mutable is an axe when what we need is a scalpel.

It has additional uses, though, e.g. memoizing and  lazy initialization of members.

>
> The non-observability comes from the fact the refcount is changed when the caller has lost its (const) reference and constness is a moot point. Changing refcount is fine now, provided the object is not immutable. As far as other outstanding const references go, these already expect changes to happen.

Refcount also needs to change when you get an additional reference. But it is non-observable, because the caller can't access the refcount directly (or we could allow it, but as @system). The only potentially observable side-effect would be if the object is destroyed, but this only happens if there are no references left. Then the only way this can be achieved is if the destructor modifies state that is otherwise accessible (global, or by references), which however is not a problem, because the normal rules of const-ness apply to it.

>
> This is what makes refcount special and provably safe to mutate. As long as we can ensure the object is not immutable, the object is allowed to change its own refcount. But refcount needs to be special cased, somehow, or else we'll end up with some C++ like `mutable`.

But the point is that we _want_ to have refcounted immutables, too, not just refcounted consts.

On Monday, 16 November 2015 at 17:12:06 UTC, Steven Schveighoffer wrote:
> One thing, however, is that if you can mark an island of space within an object as ALWAYS mutable, the compiler can know this and avoid optimizing away access to those pieces. It would have to be clearly marked as such, so the optimizations on non-marked data could still happen.
>

Yes, that's a necessity, simply to stop the compiler from applying breaking optimizations. However, then we would lose the guarantees immutable provides, in particular implicit sharing, and the implications for purity. That's why this mutability needs to be restricted, so that it has no observable effects.

> I think it could be done, because logical const is possible via a global lookup table. Any time you go through a cast, however, this could easily break down.

That's a really good argument: We don't actually introduce new semantics, we only want make existing techniques more accessible and efficient.

On 11/17/2015 10:54 AM, Marc Schütz wrote:
> On Monday, 16 November 2015 at 17:12:06 UTC, Steven Schveighoffer wrote:
>> ...
>> I think it could be done, because logical const is possible via a
>> global lookup table. Any time you go through a cast, however, this
>> could easily break down.
>
> That's a really good argument: We don't actually introduce new
> semantics, we only want make existing techniques more accessible and
> efficient.

That's only strictly true if access to @mutable members is impure.
One also needs to be careful about 'shared'.

On 11/17/2015 04:50 AM, Dicebot wrote:
> ...
> https://gist.github.com/mihails-strasuns-sociomantic/1d7529eef723b1132564

Again, this idiom does not do anything:

static assert(
    !is(typeof(this) == immutable),
    "PersistentList itself must be mutable for allocation / RC purposes"
);

The condition is always true.

Anyway, any design that does not allow e.g. nested lists is inadequate IMO.

On Tuesday, 17 November 2015 at 12:32:03 UTC, Timon Gehr wrote:
> On 11/17/2015 04:50 AM, Dicebot wrote:
>> ...
>> https://gist.github.com/mihails-strasuns-sociomantic/1d7529eef723b1132564
>
> Again, this idiom does not do anything:
>
> static assert(
>     !is(typeof(this) == immutable),
>     "PersistentList itself must be mutable for allocation / RC purposes"
> );
>
> The condition is always true.

Acknowledged, I have simply copied it from original sample to show that intention os the same. It isn't as critical in my case though as the code simply won't compile as immutable (no const casts).

> Anyway, any design that does not allow e.g. nested lists is inadequate IMO.

But here I have to disagree. There simply isn't anything generic in immutable containers with RC, each requires own tweaked solution. It all works nice with GC because all memory tracking becomes exclusively external - trying to ignore and hide that fact makes no sense to me.

Main use case for such containers is efficient thread sharing and that totally justifies specialized container (or even allocator) for each dataset.