On 11/15/15 2:09 PM, Andrei Alexandrescu wrote:
> On 11/13/2015 06:10 PM, Andrei Alexandrescu wrote:
> [snip]
>
> Thanks all for the feedback. I've uploaded an updated version to
> http://dpaste.dzfl.pl/52a3013efe34. It doesn't use "inout", but doesn't
> duplicate code either; instead, it relies on private internal functions
> to do the deed.

This still doesn't compile for me.

> While revising feedback I realized I'd missed Steve's version that works
> with inout: http://dpaste.dzfl.pl/3fbc786a50c1. It works and the
> inout-related parts don't look difficult; the only thing is making sure
> the right incantations are in place, and that's the difficult part.

I would say the difficult part is figuring out the incantations from the given error messages. When you are depending on something to compile because it can call a template function, it's often difficult to figure out where it went wrong. The error messages are frequently unhelpful.

inout is actually pretty simple to use. It has a couple of severe limitations that make it difficult to work with in certain situations -- one of them being composition (you can't compose a type with inout members). We can work on fixing these problems.

> So one important question is what style we use for Phobos and endorse as
> idiomatic. My sense after working on this for a while is - forget inout.
> Qualifiers are rather complex, and of them inout is the most. So I'd
> rather marginalize it. Please chime in with any thoughts.

I disagree. inout is no more complex than const in most cases. In my experience changing to inout is simply changing the term "const" to "inout", and everything pretty much works. This was my experience when updating dcollections after inout was finally made to work.

I'm going to work on writing an article about inout. It frequently confuses the shit out of everyone who uses it, and I don't think it should.

-Steve

On 11/15/15 9:54 AM, Andrei Alexandrescu wrote:
> On 11/15/2015 09:34 AM, Dicebot wrote:
>> On Sunday, 15 November 2015 at 14:23:05 UTC, Jonathan M Davis wrote:
>>>> As I mentioned, he's okay with changing the language to make the
>>>> casts well defined. -- Andrei
>>>
>>> Well, that's a big change, since it pretty much means that D's const
>>> isn't physical const anymore, and Walter has been _very_ insistent
>>> about that in the past - to the point that he's argued that C++'s
>>> const is outright useless because it isn't physical const. If casting
>>> away const and mutating is well-defined behavior, then we basically
>>> have C++'s const except that it's transitive ...
>>
>> Casting away _const_ is already legal if programmer can himself
>> guarantee underlying object has mutable origin (i.e. not available via
>> immutable reference), by the very definition of const. It is casting
>> away immutable and mutating that is strictly UB.
>
> Correct. I'm not sure whether that's clarified in the language
> documentation yet. -- Andrei

I argued this way, and eventually lost. I don't think it's feasible to have a const that can be cast away, and have optimizations based on const or pure.

See this discussion:

forum.dlang.org/post/riiehqozpkyluhhifwha@forum.dlang.org

One thing, however, is that if you can mark an island of space within an object as ALWAYS mutable, the compiler can know this and avoid optimizing away access to those pieces. It would have to be clearly marked as such, so the optimizations on non-marked data could still happen.

I think it could be done, because logical const is possible via a global lookup table. Any time you go through a cast, however, this could easily break down. I hate to say it, but you would likely need another modifier like "tainted const" or something in order for that to work.

-Steve

On 11/16/2015 11:03 AM, Steven Schveighoffer wrote:
> On 11/14/15 10:44 AM, Andrei Alexandrescu wrote:
>
>> That doesn't work for me with my unittests, I get:
>
>
> I'm such an idiot, I depended on dpaste to run the unit test, but forgot
> to check the unit test button.
>
> So technically, my updated version *parses* correctly :)
>
> Looking in more depth...

Thanks for letting us know. -- Andrei

On 11/16/2015 11:58 AM, Lionello Lunesu wrote:
> On 16/11/15 22:45, Andrei Alexandrescu wrote:
>> On 11/16/2015 08:51 AM, Marc Schütz wrote:
>>> On Monday, 16 November 2015 at 02:26:29 UTC, Andrei Alexandrescu
>>> wrote:
>>>> Yah, I agree with that argument. Probably @mutable is a more
>>>> principled way to go about things.
>>>
>>> Glad to here that. I think the current transitive const system
>>> is really good and shouldn't be watered down beyond necessity.
>>> And a @mutable keyword, too, shouldn't just mean "immutability
>>> or const-ness end here", thus allowing any kind of mutation. What
>>> we actually need for immutable/const refcounting etc. is
>>> _non-observable mutation_, i.e. physical mutability, but without
>>> observable effects outside of the type's implementation (better
>>> yet, restricted to very short parts of it, just like @trusted).
>>
>> The challenge is proving that a mutation is not observable. Got an
>>  attack on that? -- Andrei
>
> Forgive me, I haven't followed the RC discussions closely, so I miss
> a lot of context. Feel free to point me to existing
> threads/articles.

The best place to start for absorbing context is look at the casts in
http://dpaste.dzfl.pl/52a3013efe34, understand the necessity of each,
and figure out how they can be either made legal or (preferably) eliminated.

> If it's RC we want, then @mutable is an axe when what we need is a
> scalpel.

There's more than RC in there.


Andrei

On 11/16/2015 11:58 AM, Steven Schveighoffer wrote:
> On 11/15/15 2:09 PM, Andrei Alexandrescu wrote:
>> On 11/13/2015 06:10 PM, Andrei Alexandrescu wrote:
>> [snip]
>>
>> Thanks all for the feedback. I've uploaded an updated version to
>> http://dpaste.dzfl.pl/52a3013efe34. It doesn't use "inout", but doesn't
>> duplicate code either; instead, it relies on private internal functions
>> to do the deed.
>
> This still doesn't compile for me.

You need my emplace bugfixes, so get the latest toolchain. -- Andrei

On 11/16/15 11:03 AM, Steven Schveighoffer wrote:
> On 11/14/15 10:44 AM, Andrei Alexandrescu wrote:
>
>> That doesn't work for me with my unittests, I get:
>
>
> I'm such an idiot, I depended on dpaste to run the unit test, but forgot
> to check the unit test button.
>
> So technically, my updated version *parses* correctly :)
>
> Looking in more depth...
>

I can't get your original code to work, so it's hard for me to figure out what is wrong. Using 2.069.1, I have the following error (after factoring out `either`, which I'm assuming is a new function):

Stevens-MacBook-Pro:testd steves$ dmd -unittest persistent_list.d
/Users/steves/.dvm/compilers/dmd-2.069.1/osx/bin/../../src/phobos/std/conv.d(4161): Error: template std.conv.emplaceImpl!(const(Node)).emplaceImpl cannot deduce function from argument types !()(const(Node), immutable(int), const(Node)*, int), candidates are:
/Users/steves/.dvm/compilers/dmd-2.069.1/osx/bin/../../src/phobos/std/conv.d(3932):        std.conv.emplaceImpl!(const(Node)).emplaceImpl()(ref UT chunk)
/Users/steves/.dvm/compilers/dmd-2.069.1/osx/bin/../../src/phobos/std/conv.d(4037):        std.conv.emplaceImpl!(const(Node)).emplaceImpl(Args...)(ref UT chunk, auto ref Args args)
/Users/steves/.dvm/compilers/dmd-2.069.1/osx/bin/../../src/phobos/std/conv.d(5087): Error: template instance std.conv.emplace!(const(Node), immutable(int), const(Node)*, int) error instantiating
persistent_list.d(169):        instantiated from here: emplace!(const(Node), immutable(int), const(Node)*, int)
persistent_list.d(195):        instantiated from here: opBinaryRight!"~"
persistent_list.d(266):        instantiated from here: List!(immutable(int))

However, examining the code, I think you don't even need inout here, just a const function will do fine, since nothing on the original is changing, and all your nodes are flavored as pointing to const nodes.

I see you have updated the code, and that doesn't work for me either.

I'm afraid without a working example, I can't figure out what's *supposed* to work with inout/const.

-Steve

On 11/16/15 12:35 PM, Steven Schveighoffer wrote:

> I'm afraid without a working example, I can't figure out what's
> *supposed* to work with inout/const.
>

Update, saw your other note. Working now, trying to figure out how to do this correctly without any duplication (or wrapping).

-Steve

On 11/16/15 12:42 PM, Steven Schveighoffer wrote:
> On 11/16/15 12:35 PM, Steven Schveighoffer wrote:
>
>> I'm afraid without a working example, I can't figure out what's
>> *supposed* to work with inout/const.
>>
>
> Update, saw your other note. Working now, trying to figure out how to do
> this correctly without any duplication (or wrapping).
>
> -Steve
>

OK, I have figured it out.

inout isn't necessary here at all because everything is const internally.

For example, you had this:

        List tail()
	{
		assert(root);
		auto n = root.next;
		incRef(n);
		return List(n, allocator);
	}
	/// ditto
	const(List) tail() const
	{
		assert(root);
		auto n = root.next;
		incRef(n);
		return const(List)(n, allocator);
	}

Which I was able to replace with this:

    List tail() const
    {
        assert(root);
        auto n = root.next;
        incRef(n);
        return List(n, allocator);
    }

In any case, I think the choice between const and inout is confusing, and warrants a more in-depth explanation (when should I choose const or inout?)

-Steve

On 16-Nov-2015 19:44, Steven Schveighoffer wrote:
> On 11/14/15 3:42 AM, Dmitry Olshansky wrote:
>> On 14-Nov-2015 02:10, Andrei Alexandrescu wrote:
>
>>> * Lines 141-152: I couldn't make tail() work with inout. Generally I'm
>>> very unhappy about inout. I don't know how to use it. Everything I read
>>> about it is extremely complicated compared to its power. I wish we
>>> removed it from the language and replaced it with an understandable
>>> idiom.
>>>
>>
>> I can't agree more. Every time dealing with inout when I finally think I
>> grok how it works the next instant I see that it doesn't do what I
>> expect.
>>
>> For me inout inevitably stops at the boundary of being unnable to have
>> List!(inout(T)) and the like.
>
> One thing we could allow is types that contain inout member variables,
> but ONLY in the context of inout functions. In other words, you can
> create a type for it, but can only use it in the context of an inout
> function.

inout should be indistinguishable of const but the moment you pass inout(T) to a template it becomes murky.

Appender!(inout(int)) - which qualifier to instantiate?

>
> A List!(inout(T)) outside an inout function has no meaning. This is why
> we disallowed it.
>
> However, my belief is that it's the lack of a mechanism to apply an
> attribute to the tail of some struct that prevents the most utility
> here. If you returned a List!(inout(T)) there is no way to magically
> convert it to a List!(const(T)) or List!(immutable(T)).
>
> Consider that Node in this code has a "const(Node)* next" as its tail.
>
> I believe we should be able to achieve both.

Looks a lot like tail-const problem. That is immutable T!E has no way to convert to T!(immutable E) and similarly to T!(const E).

Overall I see const/immutable a fine thing at low-level scope but not applicable at all to high-level constructs.

If something is conceptually const - its the task of implementation which may rely on physical const qualifier somewhere deep inside.

> -Steve


-- 
Dmitry Olshansky

On 11/16/15 1:15 PM, Dmitry Olshansky wrote:
> On 16-Nov-2015 19:44, Steven Schveighoffer wrote:
>> On 11/14/15 3:42 AM, Dmitry Olshansky wrote:
>>> On 14-Nov-2015 02:10, Andrei Alexandrescu wrote:
>>
>>>> * Lines 141-152: I couldn't make tail() work with inout. Generally I'm
>>>> very unhappy about inout. I don't know how to use it. Everything I read
>>>> about it is extremely complicated compared to its power. I wish we
>>>> removed it from the language and replaced it with an understandable
>>>> idiom.
>>>>
>>>
>>> I can't agree more. Every time dealing with inout when I finally think I
>>> grok how it works the next instant I see that it doesn't do what I
>>> expect.
>>>
>>> For me inout inevitably stops at the boundary of being unnable to have
>>> List!(inout(T)) and the like.
>>
>> One thing we could allow is types that contain inout member variables,
>> but ONLY in the context of inout functions. In other words, you can
>> create a type for it, but can only use it in the context of an inout
>> function.
>
> inout should be indistinguishable of const but the moment you pass
> inout(T) to a template it becomes murky.
>
> Appender!(inout(int)) - which qualifier to instantiate?

The idea is, you would instantiate Appender!(inout(int)) :) Should be fine to use inside an inout function.

The problem is, you couldn't use it outside inout, so you couldn't use this as the return type. But inside the function, you could use it.

Right now, you can effectively do this with arrays, since inout(T)[] works fine, and this can essentially map to some fictional template instantiation ArrayStruct!(inout(T)) (or some other way to instantiate ArrayStruct!T and wrap this in an attribute that means tail). But arrays enjoy both auto casting that tail member to other flavors, and also the compiler allows you to create inout arrays inside an inout function.

We can at least solve the latter pretty easily. The closer we can move to have an actual ArrayStruct template that works just like a real array, the closer we are to having a complete solution.

>
>>
>> A List!(inout(T)) outside an inout function has no meaning. This is why
>> we disallowed it.
>>
>> However, my belief is that it's the lack of a mechanism to apply an
>> attribute to the tail of some struct that prevents the most utility
>> here. If you returned a List!(inout(T)) there is no way to magically
>> convert it to a List!(const(T)) or List!(immutable(T)).
>>
>> Consider that Node in this code has a "const(Node)* next" as its tail.

Note, I think I was wrong with this. The Node has a Node *next, but root is labelled as const.

In the end, this code is a bad example of what is wrong with inout, because inout shouldn't be used on it anyway.

> Looks a lot like tail-const problem. That is immutable T!E has no way to
> convert to T!(immutable E) and similarly to T!(const E).

It all depends on how E is used in T. Tail-const is a specific problem that *could* be solved if we could provide a mechanism to do this, but providing the mechanism isn't *required*.

-Steve