On 04/15/2016 02:24 PM, jmh530 wrote:
> I had not realized that the main reason that inout was added was because
> of not being able to use templates as virtual functions in classes.

The main reason is actually avoiding code duplication is such situations (i.e. the problem has a solution just not ideal). -- Andrei

On 4/15/16 1:11 PM, Andrei Alexandrescu wrote:
> On 04/15/2016 12:19 PM, Kenji Hara via Digitalmars-d wrote:
>>
>> Didn't you use array.dup until now? It's a good example to handle
>> qualifiers with inout.
>
> Would it be difficult to make it work without inout?

T[] dup(T)(T[] arr)

this should be doable. Problem is, you get identical instantiations for each of the different qualifiers, unless you take different actions based on the mutability. Instead:

inout(T)[] dup(T)(inout(T)[] arr)

This is one instantiation, no matter the mutability of the parameter. And it guarantees no molestation of arr, even if it's mutable.

This would be much more difficult with a struct/class, whereas with inout it's pretty simple.

But I think dup as it is defined is a bad example, because as defined by the language, it implies that you want a mutable version where the source may be const/immutable. In the general sense, a duplication function where you get the *same* mutability is more applicable for inout.

>> It's not sensible at all, inout is already well-defined and has much
>> power in D's type system.
>> Removing it is just a foolish idea.
>
> What are a few examples of the power of inout?

One thing to understand is that inout is only valuable if you think const is valuable. That is, you think the advertisement of "no, I will not modify this parameter, give me whatever you got" is of value. If you don't care, then templates suffice.

void popFront(T)(inout(T)[] arr)

This is possible only with inout. const will not cut it. The current definition is:

void popFront(T)(T[] arr)

which is OK, but there is no advertisement that popFront doesn't change any data in arr.

> What things does inout afford us, that would be otherwise not achievable?

advertisement of const without having to cast the type to const (and all the limitations that entail).

>> If you worry the future of Phobos written in D, first you would need to
>> think about @safe.
>> It's yet not well defined/implemented by compiler, and many Phobos code
>> are marked as @trusted.
>> Does it has lower priority than complain to a small hack for the
>> *current* inout limitation?
>
> The thing about @safe is it does enable things that otherwise would not
> be possible. Overall I agree there are plenty of things that deserve a
> revisit, but just putting in competition things against one another is
> unlikely to shed light on their technical merit.

I think the point of Kenji's argument is that inout's current limitations are what you are bumping into, and those limitations are unnecessary and arbitrary. We can make inout better and more consistent. Pretty easily actually. We can certainly fix the inout int = 0 problem.

> To restate my arguments:
>
> 1. The motivation of inout is clear and simple - have it as a
> placeholder for any other qualifier so as to avoid the method
> duplication observed in C++. However, it has over time grown into a
> feature of which complexity way transcends its small usefulness.

It has some rough edges. These can be smoothed out.

> 2. Attempting to make inout useful have created their own problems,
> solving which in turn have increased its complexity. This cycle of
> accretions has led over time to a vortex of oddity in the middle of the
> type system.

This is not what you are railing against. The current limitations (and their workarounds) are self-imposed, we just need to get rid of them.

An interesting thing is that when you involve generic code, certain "good ideas" become very inconvenient. For example, this whole discussion is based on the idea that "if you don't have any inout parameters, why are you declaring inout variables? It makes no sense!" A very sensible and logical idea. But when you want code that works with or without inout variables, all of a sudden the same code is throwing an error in some cases because of an unforeseen situation.

Another example is code like this:

int firstInt(T...)(T t)
{
  foreach(ref x; t)
  {
      static if(typeof(x) == int)
         return x;
  }
  return -1;
}

This throws an error because if there is an int in T..., then return -1 statement is "unreachable". This is similar to the inout int = 0 case. In non-generic code, it makes sense to flag code that won't be reached. But in generic code, it's actually not a true statement for all instantiations of the template! It's another limitation we should relax.

> 3. For all problems that inout is purported to solve, I know of idioms
> that are definitely simpler and overall almost as good if not better. So
> a hard question is whether the existence is justified.

I know of no "better" idioms, in terms of code generation and specificity, to replace inout (or at least, inout as I envision it should be). Care to elaborate?

-Steve

On Friday, 15 April 2016 at 12:54:11 UTC, ixid wrote:
> On Friday, 15 April 2016 at 03:10:12 UTC, Andrei Alexandrescu wrote:
>> We want Phobos to be beautiful, a prime example of good D code. Admittedly, it also needs to be very general and efficient, which sometimes gets in the way. But we cannot afford an accumulation of mad tricks to obscure the large design.
>>
>>
>> Andrei
>
> Why didn't we go with all functions being able to infer const, pure etc rather than just templates?

The problem is essentially untractable when there are loops in the call graph.

That being said, it would make sense to do it for auto functions.

On 4/15/16 2:39 PM, Steven Schveighoffer wrote:
> On 4/15/16 1:11 PM, Andrei Alexandrescu wrote:
>> On 04/15/2016 12:19 PM, Kenji Hara via Digitalmars-d wrote:
>>>
>>> Didn't you use array.dup until now? It's a good example to handle
>>> qualifiers with inout.
>>
>> Would it be difficult to make it work without inout?
>
> T[] dup(T)(T[] arr)
>
> this should be doable. Problem is, you get identical instantiations for
> each of the different qualifiers, unless you take different actions
> based on the mutability. Instead:
>
> inout(T)[] dup(T)(inout(T)[] arr)
>
> This is one instantiation, no matter the mutability of the parameter.
> And it guarantees no molestation of arr, even if it's mutable.
>
> This would be much more difficult with a struct/class, whereas with
> inout it's pretty simple.
>
> But I think dup as it is defined is a bad example, because as defined by
> the language, it implies that you want a mutable version where the
> source may be const/immutable. In the general sense, a duplication
> function where you get the *same* mutability is more applicable for inout.

A better support for this argument is std.array.replaceSlice at https://github.com/D-Programming-Language/phobos/blob/master/std/array.d#L2594:

inout(T)[] replaceSlice(T)(inout(T)[] s, in T[] slice, in T[] replacement);

So here we are guaranteed that (a) the result type is the same as the first argument, and (b) the first argument is never modified even if a mutable slice is passed.

I agree this is a nice property, and one that C++ does not enjoy. The questions are of course how important it is, how useful it is, and what price it is worth.

Getting back to replaceSlice I'd define it in more flexible terms anyway. So I'd eliminate the use of inout there even if it did add value.

>>> It's not sensible at all, inout is already well-defined and has much
>>> power in D's type system.
>>> Removing it is just a foolish idea.
>>
>> What are a few examples of the power of inout?
>
> One thing to understand is that inout is only valuable if you think
> const is valuable. That is, you think the advertisement of "no, I will
> not modify this parameter, give me whatever you got" is of value. If you
> don't care, then templates suffice.

This is correct but incomplete. The omitted part is that inout is valuable when $EVERYTHING_YOU_SAID and also you want to transport the qualifier from an argument to the result. This diminishes its importance considerably for all folks, including those for whom const is important.

> void popFront(T)(inout(T)[] arr)

void popFront(T)(ref inout(T)[] arr);

(point stays)

> This is possible only with inout. const will not cut it. The current
> definition is:
>
> void popFront(T)(T[] arr)
>
> which is OK, but there is no advertisement that popFront doesn't change
> any data in arr.

Should this work as well?

void popFront(ref const(T)[] arr);

In other words conversion of ref T[] to ref const(T)[] is sound.

>> What things does inout afford us, that would be otherwise not achievable?
>
> advertisement of const without having to cast the type to const (and all
> the limitations that entail).
>
>>> If you worry the future of Phobos written in D, first you would need to
>>> think about @safe.
>>> It's yet not well defined/implemented by compiler, and many Phobos code
>>> are marked as @trusted.
>>> Does it has lower priority than complain to a small hack for the
>>> *current* inout limitation?
>>
>> The thing about @safe is it does enable things that otherwise would not
>> be possible. Overall I agree there are plenty of things that deserve a
>> revisit, but just putting in competition things against one another is
>> unlikely to shed light on their technical merit.
>
> I think the point of Kenji's argument is that inout's current
> limitations are what you are bumping into, and those limitations are
> unnecessary and arbitrary. We can make inout better and more consistent.
> Pretty easily actually. We can certainly fix the inout int = 0 problem.

I'm not sure - at all in fact. This is at the tail of a sequence of changes of inout, and there is a history of such mini-designs leading to complications, regressions, and further mini-designs. It's the wolf that eats the dog that eats the cat that eats the mouse. We fix this and there's another and then another and then another. It has already happened.

In fact Walter's and my design of inout is a prime example of the failings of that approach. We underdesigned it to work in a few small examples. We knew we lost when Kenji figured inout must be a type qualifier (we were hoping to getting away with it being a sort of a placeholder/wildcard). I think we should have pulled it back then. It's become a mini-monster that only wants more attention, more language changes, more work.

> An interesting thing is that when you involve generic code, certain
> "good ideas" become very inconvenient. For example, this whole
> discussion is based on the idea that "if you don't have any inout
> parameters, why are you declaring inout variables? It makes no sense!" A
> very sensible and logical idea. But when you want code that works with
> or without inout variables, all of a sudden the same code is throwing an
> error in some cases because of an unforeseen situation.

This is an example of the mini-monster: the use of inout is necessary to satisfy the existence of inout.

> Another example is code like this:
>
> int firstInt(T...)(T t)
> {
>    foreach(ref x; t)
>    {
>        static if(typeof(x) == int)
>           return x;
>    }
>    return -1;
> }
>
> This throws an error because if there is an int in T..., then return -1
> statement is "unreachable". This is similar to the inout int = 0 case.

Not at all! There are very nice solutions for this, including looking at T or recursion. I think we can safely strike this one.

>> 3. For all problems that inout is purported to solve, I know of idioms
>> that are definitely simpler and overall almost as good if not better. So
>> a hard question is whether the existence is justified.
>
> I know of no "better" idioms, in terms of code generation and
> specificity, to replace inout (or at least, inout as I envision it
> should be). Care to elaborate?

I thought I did. Use one-liner functions that forward to templates.


Andrei

On 15-Apr-2016 08:38, Andrei Alexandrescu wrote:
> On 04/15/2016 12:23 AM, Jonathan M Davis via Digitalmars-d wrote:
>> On Thursday, April 14, 2016 23:10:12 Andrei Alexandrescu via
>> Digitalmars-d
>> wrote:
[snip]
>>
>> - Jonathan M Davis
>
> I think we should deprecate inout. For real. It costs way too much for
> what it does. For all I can tell most of D's proponents don't know how
> it works. -- Andrei

Love this trend! inout is a monstrously complicated feature solving minor inconvenience.

-- 
Dmitry Olshansky

On 4/15/16 3:28 PM, Andrei Alexandrescu wrote:
> On 4/15/16 2:39 PM, Steven Schveighoffer wrote:
>> On 4/15/16 1:11 PM, Andrei Alexandrescu wrote:
>>>
>>> Would it be difficult to make it work without inout?
>>
>> T[] dup(T)(T[] arr)
>>
>> this should be doable. Problem is, you get identical instantiations for
>> each of the different qualifiers, unless you take different actions
>> based on the mutability. Instead:
>>
>> inout(T)[] dup(T)(inout(T)[] arr)
>>
>> This is one instantiation, no matter the mutability of the parameter.
>> And it guarantees no molestation of arr, even if it's mutable.
>>
>> This would be much more difficult with a struct/class, whereas with
>> inout it's pretty simple.
>>
>> But I think dup as it is defined is a bad example, because as defined by
>> the language, it implies that you want a mutable version where the
>> source may be const/immutable. In the general sense, a duplication
>> function where you get the *same* mutability is more applicable for
>> inout.
>
> A better support for this argument is std.array.replaceSlice at
> https://github.com/D-Programming-Language/phobos/blob/master/std/array.d#L2594:
>
>
> inout(T)[] replaceSlice(T)(inout(T)[] s, in T[] slice, in T[] replacement);
>
> So here we are guaranteed that (a) the result type is the same as the
> first argument, and (b) the first argument is never modified even if a
> mutable slice is passed.

In fact, a mutable slice must be passed (slice is not marked as inout).

>
> I agree this is a nice property, and one that C++ does not enjoy. The
> questions are of course how important it is, how useful it is, and what
> price it is worth.
>
> Getting back to replaceSlice I'd define it in more flexible terms
> anyway. So I'd eliminate the use of inout there even if it did add value.

You can add more flexibility and keep inout at the same time. The nice feature here is that inout protects the elements of s that aren't in slice, and there is no requirement to verify s and slice are the same underlying type, etc.

>> One thing to understand is that inout is only valuable if you think
>> const is valuable. That is, you think the advertisement of "no, I will
>> not modify this parameter, give me whatever you got" is of value. If you
>> don't care, then templates suffice.
>
> This is correct but incomplete. The omitted part is that inout is
> valuable when $EVERYTHING_YOU_SAID and also you want to transport the
> qualifier from an argument to the result. This diminishes its importance
> considerably for all folks, including those for whom const is important.

Nope. Actually inout has proven to have a more interesting property of not requiring casting.

>
>> void popFront(T)(inout(T)[] arr)
>
> void popFront(T)(ref inout(T)[] arr);
>
> (point stays)

Yes, thanks, that is what I meant.

>
>> This is possible only with inout. const will not cut it. The current
>> definition is:
>>
>> void popFront(T)(T[] arr)
>>
>> which is OK, but there is no advertisement that popFront doesn't change
>> any data in arr.
>
> Should this work as well?
>
> void popFront(ref const(T)[] arr);
>
> In other words conversion of ref T[] to ref const(T)[] is sound.

No, because it's not sound. you cannot cast to const through 2+ indirections. However inout works there.

>> I think the point of Kenji's argument is that inout's current
>> limitations are what you are bumping into, and those limitations are
>> unnecessary and arbitrary. We can make inout better and more consistent.
>> Pretty easily actually. We can certainly fix the inout int = 0 problem.
>
> I'm not sure - at all in fact. This is at the tail of a sequence of
> changes of inout, and there is a history of such mini-designs leading to
> complications, regressions, and further mini-designs. It's the wolf that
> eats the dog that eats the cat that eats the mouse. We fix this and
> there's another and then another and then another. It has already happened.

I assure you, these limitations were self-imposed. I insisted on them, without realizing that they would cause problems with generic code. I thought they would be good "lint" detection.

https://issues.dlang.org/show_bug.cgi?id=3748

> In fact Walter's and my design of inout is a prime example of the
> failings of that approach. We underdesigned it to work in a few small
> examples. We knew we lost when Kenji figured inout must be a type
> qualifier (we were hoping to getting away with it being a sort of a
> placeholder/wildcard). I think we should have pulled it back then. It's
> become a mini-monster that only wants more attention, more language
> changes, more work.

It must be a type qualifier or it doesn't work.

>> An interesting thing is that when you involve generic code, certain
>> "good ideas" become very inconvenient. For example, this whole
>> discussion is based on the idea that "if you don't have any inout
>> parameters, why are you declaring inout variables? It makes no sense!" A
>> very sensible and logical idea. But when you want code that works with
>> or without inout variables, all of a sudden the same code is throwing an
>> error in some cases because of an unforeseen situation.
>
> This is an example of the mini-monster: the use of inout is necessary to
> satisfy the existence of inout.

I don't know what this argument is saying?

>> Another example is code like this:
>>
>> int firstInt(T...)(T t)
>> {
>>    foreach(ref x; t)
>>    {
>>        static if(typeof(x) == int)
>>           return x;
>>    }
>>    return -1;
>> }
>>
>> This throws an error because if there is an int in T..., then return -1
>> statement is "unreachable". This is similar to the inout int = 0 case.
>
> Not at all! There are very nice solutions for this, including looking at
> T or recursion. I think we can safely strike this one.

Absolutely similar. There are ways around it, but at the end of the day, the compiler is complaining that the line can never be reached, when in fact it can (just pass a short in). Just like there are ways around the inout local variable requirement.

>>> 3. For all problems that inout is purported to solve, I know of idioms
>>> that are definitely simpler and overall almost as good if not better. So
>>> a hard question is whether the existence is justified.
>>
>> I know of no "better" idioms, in terms of code generation and
>> specificity, to replace inout (or at least, inout as I envision it
>> should be). Care to elaborate?
>
> I thought I did. Use one-liner functions that forward to templates.

I didn't see it?

-Steve

On 15.04.2016 17:22, Steven Schveighoffer wrote:
> On 4/14/16 11:10 PM, Andrei Alexandrescu wrote:
>> Consider:
>>
>> https://github.com/D-Programming-Language/phobos/blob/master/std/range/primitives.d#L152
>>
>
> It works around a limitation of inout that isn't necessary (even though
> I thought it was being helpful when I suggested it). That is, functions
> without inout parameters cannot declare local inout variables. But this
> isn't really necessary, and should be fixed. I will discuss this in my
> talk in a few weeks.
> ...

That's potentially dangerous. What about cases like the following?

void main(){
    inout(int)[] x=[1,2,3];
    immutable(int) a;
    int b;
    inout(int)[] foo(inout int){
        return x;
    }
    immutable(int)[] y=foo(a);
    int[] z=foo(b);
}



> Note, the =0 part isn't necessary right now, since it's not called. It's
> just used to test if the function can compile.
>
> In short, my opinion on inout is that it has some unnecessary
> limitations, which can be removed, and inout will work as mostly
> expected. These requirements to work around the limitations will go away.
> ...

Other important limitations of inout are e.g.:
- inout variables cannot be fields.
- There can be only one inout in scope.


> I expect after reading this thread that the Q&A will be.. interesting.
>
> -Steve

On 04/15/2016 03:44 PM, Steven Schveighoffer wrote:
> No, because it's not sound. you cannot cast to const through 2+
> indirections. However inout works there.

That is correct, a classic... thanks for getting me straight.

>>> I think the point of Kenji's argument is that inout's current
>>> limitations are what you are bumping into, and those limitations are
>>> unnecessary and arbitrary. We can make inout better and more consistent.
>>> Pretty easily actually. We can certainly fix the inout int = 0 problem.
>>
>> I'm not sure - at all in fact. This is at the tail of a sequence of
>> changes of inout, and there is a history of such mini-designs leading to
>> complications, regressions, and further mini-designs. It's the wolf that
>> eats the dog that eats the cat that eats the mouse. We fix this and
>> there's another and then another and then another. It has already
>> happened.
>
> I assure you, these limitations were self-imposed. I insisted on them,
> without realizing that they would cause problems with generic code. I
> thought they would be good "lint" detection.
>
> https://issues.dlang.org/show_bug.cgi?id=3748

Problem is we could have other problems once we fix those. As you just showed, it has already happened.

We should really do away with the cowboy style of designing language, which sadly Walter and I have been guilty of too often in the past. The slow but sure accretion of complexity of inout is a textbook example of where that leads.


Andrei

On 4/15/16 3:48 PM, Timon Gehr wrote:
> On 15.04.2016 17:22, Steven Schveighoffer wrote:
>> On 4/14/16 11:10 PM, Andrei Alexandrescu wrote:
>>> Consider:
>>>
>>> https://github.com/D-Programming-Language/phobos/blob/master/std/range/primitives.d#L152
>>>
>>>
>>
>> It works around a limitation of inout that isn't necessary (even though
>> I thought it was being helpful when I suggested it). That is, functions
>> without inout parameters cannot declare local inout variables. But this
>> isn't really necessary, and should be fixed. I will discuss this in my
>> talk in a few weeks.
>> ...
>
> That's potentially dangerous. What about cases like the following?
>
> void main(){
>      inout(int)[] x=[1,2,3];
>      immutable(int) a;
>      int b;
>      inout(int)[] foo(inout int){
>          return x;
>      }
>      immutable(int)[] y=foo(a);
>      int[] z=foo(b);
> }

We don't need to guess:

void foo (inout int)
{
    inout(int)[] x=[1,2,3];
    immutable(int) a;
    int b;
    inout(int)[] foo(inout int){
        return x;
    }
    immutable(int)[] y=foo(a); // line 9
    int[] z=foo(b);  // line 10
}

testinout.d(9): Error: modify inout to immutable is not allowed inside inout function
testinout.d(10): Error: modify inout to mutable is not allowed inside inout function

>
>
>
>> Note, the =0 part isn't necessary right now, since it's not called. It's
>> just used to test if the function can compile.
>>
>> In short, my opinion on inout is that it has some unnecessary
>> limitations, which can be removed, and inout will work as mostly
>> expected. These requirements to work around the limitations will go away.
>> ...
>
> Other important limitations of inout are e.g.:
> - inout variables cannot be fields.

I have a way to make this work. This is actually the most major sticking point in inout.

The only correct thing is to keep is that globals/static variables cannot be typed inout.

> - There can be only one inout in scope.

This is not so much a problem I think.

-Steve

On 04/15/2016 04:03 PM, Steven Schveighoffer wrote:
> I have a way to make this work. This is actually the most major sticking
> point in inout.
>
> The only correct thing is to keep is that globals/static variables
> cannot be typed inout.

Another special case? The only correct thing is to simplify the language to everybody's benefit. -- Andrei