On 11/16/15 4:49 PM, Andrei Alexandrescu wrote:
> On 11/16/2015 04:36 PM, Steven Schveighoffer wrote:
>> We can and should fix this. I'll file an issue if none has been filed.
>
> That's great! -- Andrei

https://issues.dlang.org/show_bug.cgi?id=15347

Note that the trigger is the ctor and dtor of the struct. Each adds worse layers of error message horror :)

-Steve

On 11/16/15 5:35 PM, Andrei Alexandrescu wrote:
> On 11/16/2015 05:30 PM, Steven Schveighoffer wrote:
>> I'll reiterate what I said elsewhere: inout is better, safer, and easier
>> to maintain than the template solution.
>
> Do you have a working solution in dpaste? No change to the return types
> please. -- Andrei

Ah, where it breaks down is emplace uses an internal struct with a T, which is inout(const(Node)*).

So you *are* hitting one of the limitations of inout, one that Dmitry brought up. I stand corrected.

I think it can be fixed, we need to make structs that contain inout member variables legal as long as they only exist on inout function stacks. It may be possible to fix emplace for inout by changing the ctor of the internal struct to inout instead of typing T as inout, but I'm not sure.

In this case, in order to work around, you have to cast when you use emplace, somewhat defeating the safety of emplace :(

http://dpaste.dzfl.pl/182e2ac5da2d

The only other weird part is where you see inout(const(Node)*) in the fixed ctor. This is because it's a different type than inout(Node*), it's like const(immutable(Node)*), the pointer itself is const, but the data it points at is still immutable.

-Steve

On Sunday, 15 November 2015 at 20:11:02 UTC, Dicebot wrote:
> On Sunday, 15 November 2015 at 19:51:09 UTC, Andrei Alexandrescu wrote:
>> Just to clarify - is that referring to the part "We need to change that if we want things like composable containers that work with const." or to the "I think it's a good thing to want" part? -- Andrei
>
> Second part. I don't see a case for const containers at all. Fully immutable functional style ones - sure, I have actually experimented implementing cache this way in https://github.com/Dicebot/mood/blob/master/source/mood/storage/generic_cache.d (thing that powers blog.dicebot.lv). But what would you use const containers for? Mixing mutable and immutable elements in one container? That sounds like a source of much trouble.

For the variance. If you wan't write in the container, you make make it so that

ConstContainer<B> implicitly convert to ConstContainer<A> if B inherit from A.

You can have code that accept mutable and const container as well, which is useful, because you can cram immutable container in the ro section.

On Monday, 16 November 2015 at 22:30:55 UTC, Andrei Alexandrescu wrote:
> On 11/16/2015 05:02 PM, Jonathan M Davis wrote:
>> It's just that you use inout instead of const. How is that worse?
>
> The short answer is my perception is inout is too complicated for what it does. -- Andrei

I'm happy to read this. inout has the wrong cost/complexity ratio. It doesn't solve the problem generally (one want to  return type depending on argument qualifier in general, not only for type qualifiers) while having a high complexity.

On 11/16/15 6:56 PM, deadalnix wrote:
> On Monday, 16 November 2015 at 22:30:55 UTC, Andrei Alexandrescu wrote:
>> On 11/16/2015 05:02 PM, Jonathan M Davis wrote:
>>> It's just that you use inout instead of const. How is that worse?
>>
>> The short answer is my perception is inout is too complicated for what
>> it does. -- Andrei
>
> I'm happy to read this. inout has the wrong cost/complexity ratio. It
> doesn't solve the problem generally (one want to return type depending
> on argument qualifier in general, not only for type qualifiers) while
> having a high complexity.
>

The problem it solves is to provide a mechanism that allows one to safely apply the same qualifiers to the return type, but have the inputs be constant *throughout the function*.

A templated function doesn't do this, you can have conditional code, or unchecked method calls that can modify the data when possible.

Of course, if this isn't important, then the template solution is usable. My take is that whenever you use inout is when you would normally use const, but const doesn't allow what you need. If your function doesn't need const inside the function, then you shouldn't use inout.

To me, it feels like inout is simple, but the implementation has warts that make it seem inconsistent. There is also the issue that nested inout is super-confusing, because now you have layers of inout, each layer meaning possibly something different. If we could somehow fix the nested part (and allow types with inout members inside there), I think inout would be less intimidating.

-Steve

On Tuesday, 17 November 2015 at 00:30:39 UTC, Steven Schveighoffer wrote:
> The problem it solves is to provide a mechanism that allows one to safely apply the same qualifiers to the return type, but have the inputs be constant *throughout the function*.
>
> A templated function doesn't do this, you can have conditional code, or unchecked method calls that can modify the data when possible.
>
> Of course, if this isn't important, then the template solution is usable. My take is that whenever you use inout is when you would normally use const, but const doesn't allow what you need. If your function doesn't need const inside the function, then you shouldn't use inout.
>
> To me, it feels like inout is simple, but the implementation has warts that make it seem inconsistent. There is also the issue that nested inout is super-confusing, because now you have layers of inout, each layer meaning possibly something different. If we could somehow fix the nested part (and allow types with inout members inside there), I think inout would be less intimidating.
>
> -Steve

I don't mean to derail this thread, but I also am not really crazy about inout in its current form. Its transitive nature is one issue (though that can't be changed due to const and immutable also being transitive), but also the fact that inout is basically a fourth qualifier, which causes a lot of pain when you're wrapping other types.

For example, calling opEquals on a wrapped struct inside the opEquals of the wrapper struct (which is marked as inout) when the wrapped struct does not also define an inout opEquals, will not call the mutable opEquals of the wrapped struct. It *will not* compile unless the wrapped struct also defines an inout opEquals, no matter the constness of the wrapper or the wrappee. Basically, wrapper types cannot with inout.

http://stackoverflow.com/questions/31516713/escaping-from-inout-hell

On Tuesday, 17 November 2015 at 00:30:39 UTC, Steven Schveighoffer wrote:
> On 11/16/15 6:56 PM, deadalnix wrote:
>> On Monday, 16 November 2015 at 22:30:55 UTC, Andrei Alexandrescu wrote:
>>> On 11/16/2015 05:02 PM, Jonathan M Davis wrote:
>>>> It's just that you use inout instead of const. How is that worse?
>>>
>>> The short answer is my perception is inout is too complicated for what
>>> it does. -- Andrei
>>
>> I'm happy to read this. inout has the wrong cost/complexity ratio. It
>> doesn't solve the problem generally (one want to return type depending
>> on argument qualifier in general, not only for type qualifiers) while
>> having a high complexity.
>>
>
> The problem it solves is to provide a mechanism that allows one to safely apply the same qualifiers to the return type, but have the inputs be constant *throughout the function*.
>
> A templated function doesn't do this, you can have conditional code, or unchecked method calls that can modify the data when possible.
>
> Of course, if this isn't important, then the template solution is usable. My take is that whenever you use inout is when you would normally use const, but const doesn't allow what you need. If your function doesn't need const inside the function, then you shouldn't use inout.
>
> To me, it feels like inout is simple, but the implementation has warts that make it seem inconsistent. There is also the issue that nested inout is super-confusing, because now you have layers of inout, each layer meaning possibly something different. If we could somehow fix the nested part (and allow types with inout members inside there), I think inout would be less intimidating.
>
> -Steve

Here is, IMO, the point you are missing.

There many qualifier you'd like to have depend on other qualifiers. For instance :

void doSomeThingAndCallBack(maybe_pure void function() callback) pure_if_maybe_pure_argument_is_pure {
    callback();
}

You'll note that this is the same problem as inout solves. Thing is inout is complex, but only solve a small subset of the problem.

Thus, the power/complexity ratio is not good.

Also, yes, the implementation is B0rken :) Still it has gotten much better over time (or I learned the hard way to stay into the patterns that works ?).

On 11/16/15 8:11 PM, Meta wrote:
> On Tuesday, 17 November 2015 at 00:30:39 UTC, Steven Schveighoffer wrote:
>> The problem it solves is to provide a mechanism that allows one to
>> safely apply the same qualifiers to the return type, but have the
>> inputs be constant *throughout the function*.
>>
>> A templated function doesn't do this, you can have conditional code,
>> or unchecked method calls that can modify the data when possible.
>>
>> Of course, if this isn't important, then the template solution is
>> usable. My take is that whenever you use inout is when you would
>> normally use const, but const doesn't allow what you need. If your
>> function doesn't need const inside the function, then you shouldn't
>> use inout.
>>
>> To me, it feels like inout is simple, but the implementation has warts
>> that make it seem inconsistent. There is also the issue that nested
>> inout is super-confusing, because now you have layers of inout, each
>> layer meaning possibly something different. If we could somehow fix
>> the nested part (and allow types with inout members inside there), I
>> think inout would be less intimidating.
>>
>> -Steve
>
> I don't mean to derail this thread, but I also am not really crazy about
> inout in its current form. Its transitive nature is one issue (though
> that can't be changed due to const and immutable also being transitive),
> but also the fact that inout is basically a fourth qualifier, which
> causes a lot of pain when you're wrapping other types.
>
> For example, calling opEquals on a wrapped struct inside the opEquals of
> the wrapper struct (which is marked as inout) when the wrapped struct
> does not also define an inout opEquals, will not call the mutable
> opEquals of the wrapped struct. It *will not* compile unless the wrapped
> struct also defines an inout opEquals, no matter the constness of the
> wrapper or the wrappee. Basically, wrapper types cannot with inout.
>
> http://stackoverflow.com/questions/31516713/escaping-from-inout-hell

I think it's quite clear that inout in its current form is a point of huge confusion. I want to work to fix this perception (and fix the broken corners of the implementation).

What I would ask is for those who at the moment dislike it keep an open mind, and hold back on your pitchforks. I need to prepare a suitable defense, and it's difficult to shake off the "confusing and complex" albatross without having sufficient time to create something that is easy to understand/explain for something that is conceptually simple, but requires a complex proof.

What I am afraid of is that someone makes a decision that something is bad, and by the time a good defense, or workable proposal is mounted, the answer is "we already discussed that, it's bad. Let's move on."

At least with this, we have a feature that is part of the language, so is unlikely to go away. But I'd hate to see a mob of PR requests that remove inout from all of phobos/druntime before I can counter this.

I think inout is worth saving, it needs a few tweaks and a good explanation.

-Steve

On 11/16/15 8:21 PM, deadalnix wrote:
> On Tuesday, 17 November 2015 at 00:30:39 UTC, Steven Schveighoffer wrote:
>> On 11/16/15 6:56 PM, deadalnix wrote:
>>> On Monday, 16 November 2015 at 22:30:55 UTC, Andrei Alexandrescu wrote:
>>>> On 11/16/2015 05:02 PM, Jonathan M Davis wrote:
>>>>> It's just that you use inout instead of const. How is that worse?
>>>>
>>>> The short answer is my perception is inout is too complicated for what
>>>> it does. -- Andrei
>>>
>>> I'm happy to read this. inout has the wrong cost/complexity ratio. It
>>> doesn't solve the problem generally (one want to return type depending
>>> on argument qualifier in general, not only for type qualifiers) while
>>> having a high complexity.
>>>
>>
>> The problem it solves is to provide a mechanism that allows one to
>> safely apply the same qualifiers to the return type, but have the
>> inputs be constant *throughout the function*.
>>
>> A templated function doesn't do this, you can have conditional code,
>> or unchecked method calls that can modify the data when possible.
>>
>> Of course, if this isn't important, then the template solution is
>> usable. My take is that whenever you use inout is when you would
>> normally use const, but const doesn't allow what you need. If your
>> function doesn't need const inside the function, then you shouldn't
>> use inout.
>>
>> To me, it feels like inout is simple, but the implementation has warts
>> that make it seem inconsistent. There is also the issue that nested
>> inout is super-confusing, because now you have layers of inout, each
>> layer meaning possibly something different. If we could somehow fix
>> the nested part (and allow types with inout members inside there), I
>> think inout would be less intimidating.
> Here is, IMO, the point you are missing.
>
> There many qualifier you'd like to have depend on other qualifiers. For
> instance :
>
> void doSomeThingAndCallBack(maybe_pure void function() callback)
> pure_if_maybe_pure_argument_is_pure {
>      callback();
> }

This already works with templates. If all you care about is writing a function that has the *ability* to take any kind of input, then templates are perfect for that.

What inout solves above templates (and note, you can combine inout AND templates), is that during the function call, the parameter is not modified -- guaranteed by the compiler (as much as it can). Templates cannot offer that. To me the maybe_pure solution doesn't even come close to inout's utility (what does it guarantee beside being able to take different types of parameters?).

For those who care about const or try to use const (and immutable) to provide API guarantees on parameters, it becomes unwieldy to both specify "yes! I can take any qualified type of parameter" and "no! I promise I won't modify them." Const fills this void, but has the the issue of blindly coloring everything const, locking up all your data for things like function chaining. Inout is supposed to fix this one problem, and should be a seamless replacement for const in those cases. It obviously is not.

> Thus, the power/complexity ratio is not good.

inout is for one specific thing, I see it to be unrelated to templates. It's more of an extension of const. The complexity, IMO, is not necessarily huge (perhaps in the current implementation, I'm not sure), but the consistency is terrible. There are things you just should be able to do, and the compiler won't do them. Inevitably, the templates people write end up trying to do these things (because they are logically correct), and the compiler spits out cryptic "cannot call this function" errors. These can be difficult to figure out.

> Also, yes, the implementation is B0rken :) Still it has gotten much
> better over time (or I learned the hard way to stay into the patterns
> that works ?).
>

It has definitely gotten better over time. It used to be that if you used any kind of template with an inout parameter, it wouldn't compile.

-Steve

On Sunday, 15 November 2015 at 12:56:27 UTC, Andrei Alexandrescu wrote:
> On 11/14/2015 05:49 PM, Timon Gehr wrote:
>> List uses RC internally. I don't think the UB casts will stay for the
>> final version, unless you are willing to completely dilute the meaning
>> of const in ways that Walter has explicitly expressed will not be done
>> in the past.
>
> As I mentioned, he's okay with changing the language to make the casts well defined. -- Andrei

Ok Here is what I propose as a spec. This is rough brain dump, but that is something I think can be a good start.

There are mutable data that are thread local and immutable data that are shared. const is a wildcard type qualifier that can refers to mutable or immutable data.

immutable is supposed to be effectively immutable. This means that the compiler is allowed to store these data in ro segments and optimize redundant loads without having to prove that no aliasing write occur in between.

One can cast away const or immutable. It is a @system operation and this cast needs to be explicit.

If the underlying data is in ro segment, any write is effectively undefined behavior (most likely a fault).
If the underlying data is allocated on the head (using the GC or malloc for instance) then writing to it will effectively update the underlying memory.

Any read from a const or immutable reference after these write can is undefined.
Granted the write was not UB :
1/ Any read from a mutable reference in another thread is undefined behavior unless the writing thread release after write and the reading one acquire (or any stronger memory barrier).
2/ Any read from a mutable reference in the same thread will see the updates in a sequentially consistent manner.

Sounds good ? This definitively allow to do RC for const/immutable without throwing away optimization opportunities.