On 4/15/16 4:03 PM, Andrei Alexandrescu wrote:
> On 04/15/2016 03:44 PM, Steven Schveighoffer wrote:
>>
>> I assure you, these limitations were self-imposed. I insisted on them,
>> without realizing that they would cause problems with generic code. I
>> thought they would be good "lint" detection.
>>
>> https://issues.dlang.org/show_bug.cgi?id=3748
>
> Problem is we could have other problems once we fix those. As you just
> showed, it has already happened.

If you look at the core of what inout actually is (a type modifier placeholder), we can simplify how to think about it, and how to implement it. These "requirements" were just extra helpful things that would flag valid, but (naively assumed) pointless code. Turns out, templates generate lots of pointless code (that is typically optimized away in the type system or the optimizer).

I hope to make this clear in my talk at dconf.

> We should really do away with the cowboy style of designing language,
> which sadly Walter and I have been guilty of too often in the past. The
> slow but sure accretion of complexity of inout is a textbook example of
> where that leads.

I actually agree, someone with as little experience as I had should not have been listened to when making such a feature :)

If I could do it over again, I would still want this feature, but obviously, we could make sane simple rules for it. They are actually quite easy to understand.

-Steve

On 4/15/16 4:08 PM, Andrei Alexandrescu wrote:
> On 04/15/2016 04:03 PM, Steven Schveighoffer wrote:
>> I have a way to make this work. This is actually the most major sticking
>> point in inout.
>>
>> The only correct thing is to keep is that globals/static variables
>> cannot be typed inout.
>
> Another special case? The only correct thing is to simplify the language
> to everybody's benefit. -- Andrei
>

This is not a special case any more than disallowing access of shared data from a pure function is a special case.

In fact, you could allow inout variables as static or globals, but just couldn't copy them to local inout variables (except full value types).

The point is that between 2 different calls to inout, the wrapper means something different. A global/static persists between calls.

-Steve

On 15.04.2016 22:03, Steven Schveighoffer wrote:
> On 4/15/16 3:48 PM, Timon Gehr wrote:
>> On 15.04.2016 17:22, Steven Schveighoffer wrote:
>>> On 4/14/16 11:10 PM, Andrei Alexandrescu wrote:
>>>> Consider:
>>>>
>>>> https://github.com/D-Programming-Language/phobos/blob/master/std/range/primitives.d#L152
>>>>
>>>>
>>>>
>>>
>>> It works around a limitation of inout that isn't necessary (even though
>>> I thought it was being helpful when I suggested it). That is, functions
>>> without inout parameters cannot declare local inout variables. But this
>>> isn't really necessary, and should be fixed. I will discuss this in my
>>> talk in a few weeks.
>>> ...
>>
>> That's potentially dangerous. What about cases like the following?
>>
>> void main(){
>>      inout(int)[] x=[1,2,3];
>>      immutable(int) a;
>>      int b;
>>      inout(int)[] foo(inout int){
>>          return x;
>>      }
>>      immutable(int)[] y=foo(a);
>>      int[] z=foo(b);
>> }
>
> We don't need to guess:
>
> void foo (inout int)
> {
>      inout(int)[] x=[1,2,3];
>      immutable(int) a;
>      int b;
>      inout(int)[] foo(inout int){
>          return x;
>      }
>      immutable(int)[] y=foo(a); // line 9
>      int[] z=foo(b);  // line 10
> }
>
> testinout.d(9): Error: modify inout to immutable is not allowed inside
> inout function
> testinout.d(10): Error: modify inout to mutable is not allowed inside
> inout function
>

I'm very aware of that: https://issues.dlang.org/show_bug.cgi?id=10758
main is not an inout function in the example above. I.e. if we just change the compiler minimally such as to allow making inout local variables, the above example will violate immutability guarantees. You can also imagine cases where the inout local variable is defined only after the local inout function has been declared and called with a non-inout argument. At which point does the constraint become active? etc. We cannot _simply_ allow declaring inout locals.

>>
>>
>>
>>> Note, the =0 part isn't necessary right now, since it's not called. It's
>>> just used to test if the function can compile.
>>>
>>> In short, my opinion on inout is that it has some unnecessary
>>> limitations, which can be removed, and inout will work as mostly
>>> expected. These requirements to work around the limitations will go
>>> away.
>>> ...
>>
>> Other important limitations of inout are e.g.:
>> - inout variables cannot be fields.
>
> I have a way to make this work.

Without syntax changes?
Can the struct/class instances with inout fields be returned from the enclosing inout function?

> This is actually the most major sticking
> point in inout.
>  The only correct thing is to keep is that globals/static variables
> cannot be typed inout.
>
>> - There can be only one inout in scope.
>
> This is not so much a problem I think.
> ...

I think it is. It's just not the prevalent limitation one runs in at the moment. It will be more of a problem once functions can return structs with inout fields. IMHO compositionality should be ensured for a language feature from the start.

On 4/15/16 4:27 PM, Timon Gehr wrote:
> On 15.04.2016 22:03, Steven Schveighoffer wrote:
>> On 4/15/16 3:48 PM, Timon Gehr wrote:
>>> On 15.04.2016 17:22, Steven Schveighoffer wrote:
>>>> On 4/14/16 11:10 PM, Andrei Alexandrescu wrote:
>>>>> Consider:
>>>>>
>>>>> https://github.com/D-Programming-Language/phobos/blob/master/std/range/primitives.d#L152
>>>>>
>>>>>
>>>>>
>>>>>
>>>>
>>>> It works around a limitation of inout that isn't necessary (even though
>>>> I thought it was being helpful when I suggested it). That is, functions
>>>> without inout parameters cannot declare local inout variables. But this
>>>> isn't really necessary, and should be fixed. I will discuss this in my
>>>> talk in a few weeks.
>>>> ...
>>>
>>> That's potentially dangerous. What about cases like the following?
>>>
>>> void main(){
>>>      inout(int)[] x=[1,2,3];
>>>      immutable(int) a;
>>>      int b;
>>>      inout(int)[] foo(inout int){
>>>          return x;
>>>      }
>>>      immutable(int)[] y=foo(a);
>>>      int[] z=foo(b);
>>> }
>>
>> We don't need to guess:
>>
>> void foo (inout int)
>> {
>>      inout(int)[] x=[1,2,3];
>>      immutable(int) a;
>>      int b;
>>      inout(int)[] foo(inout int){
>>          return x;
>>      }
>>      immutable(int)[] y=foo(a); // line 9
>>      int[] z=foo(b);  // line 10
>> }
>>
>> testinout.d(9): Error: modify inout to immutable is not allowed inside
>> inout function
>> testinout.d(10): Error: modify inout to mutable is not allowed inside
>> inout function
>>
>
> I'm very aware of that: https://issues.dlang.org/show_bug.cgi?id=10758
> main is not an inout function in the example above. I.e. if we just
> change the compiler minimally such as to allow making inout local
> variables, the above example will violate immutability guarantees. You
> can also imagine cases where the inout local variable is defined only
> after the local inout function has been declared and called with a
> non-inout argument. At which point does the constraint become active?
> etc. We cannot _simply_ allow declaring inout locals.

There's no difference between a function that declares its variables inout within its parameters or one that declares them locally.

They should be treated the same once the function starts compiling.

>>>
>>>
>>>
>>>> Note, the =0 part isn't necessary right now, since it's not called.
>>>> It's
>>>> just used to test if the function can compile.
>>>>
>>>> In short, my opinion on inout is that it has some unnecessary
>>>> limitations, which can be removed, and inout will work as mostly
>>>> expected. These requirements to work around the limitations will go
>>>> away.
>>>> ...
>>>
>>> Other important limitations of inout are e.g.:
>>> - inout variables cannot be fields.
>>
>> I have a way to make this work.
>
> Without syntax changes?
> Can the struct/class instances with inout fields be returned from the
> enclosing inout function?

Yes, as long as inout is wrapping inout. We run into this currently with inout functions that create local types. e.g. emplace.

Obviously inout cannot be unwrapped if it is typed on a field of a struct or class. So the unwrapping has to result in inout.

>>> - There can be only one inout in scope.
>>
>> This is not so much a problem I think.
>> ...
>
> I think it is. It's just not the prevalent limitation one runs in at the
> moment. It will be more of a problem once functions can return structs
> with inout fields. IMHO compositionality should be ensured for a
> language feature from the start.

At the point where we need to tag multiple pools of inout parameters, the complexity of the language doesn't justify the benefits.

We could make it possible, for instance, to templatize the mutability modifier instead of using a specific keyword. Then you could have foo(int)[]. Then I think you could do all this (and scrap inout), but I wouldn't want to work in that language.

-Steve

On 04/15/2016 04:47 PM, Steven Schveighoffer wrote:
> There's no difference between a function that declares its variables
> inout within its parameters or one that declares them locally.

So now we get to things like:

void fun() {
  inout int ohHello = 42;
  ...
}

How to explain such a construct? Not to mention globals of that type are not allowed.


Andrei

On 15.04.2016 22:47, Steven Schveighoffer wrote:
> On 4/15/16 4:27 PM, Timon Gehr wrote:
>> On 15.04.2016 22:03, Steven Schveighoffer wrote:
>>> On 4/15/16 3:48 PM, Timon Gehr wrote:
>>>> On 15.04.2016 17:22, Steven Schveighoffer wrote:
>>>>> On 4/14/16 11:10 PM, Andrei Alexandrescu wrote:
>>>>>> Consider:
>>>>>>
>>>>>> https://github.com/D-Programming-Language/phobos/blob/master/std/range/primitives.d#L152
>>>>>>
>>>>>>
>>>>>>
>>>>>>
>>>>>>
>>>>>
>>>>> It works around a limitation of inout that isn't necessary (even
>>>>> though
>>>>> I thought it was being helpful when I suggested it). That is,
>>>>> functions
>>>>> without inout parameters cannot declare local inout variables. But
>>>>> this
>>>>> isn't really necessary, and should be fixed. I will discuss this in my
>>>>> talk in a few weeks.
>>>>> ...
>>>>
>>>> That's potentially dangerous. What about cases like the following?
>>>>
>>>> void main(){
>>>>      inout(int)[] x=[1,2,3];
>>>>      immutable(int) a;
>>>>      int b;
>>>>      inout(int)[] foo(inout int){
>>>>          return x;
>>>>      }
>>>>      immutable(int)[] y=foo(a);
>>>>      int[] z=foo(b);
>>>> }
>>>
>>> We don't need to guess:
>>>
>>> void foo (inout int)
>>> {
>>>      inout(int)[] x=[1,2,3];
>>>      immutable(int) a;
>>>      int b;
>>>      inout(int)[] foo(inout int){
>>>          return x;
>>>      }
>>>      immutable(int)[] y=foo(a); // line 9
>>>      int[] z=foo(b);  // line 10
>>> }
>>>
>>> testinout.d(9): Error: modify inout to immutable is not allowed inside
>>> inout function
>>> testinout.d(10): Error: modify inout to mutable is not allowed inside
>>> inout function
>>>
>>
>> I'm very aware of that: https://issues.dlang.org/show_bug.cgi?id=10758
>> main is not an inout function in the example above. I.e. if we just
>> change the compiler minimally such as to allow making inout local
>> variables, the above example will violate immutability guarantees. You
>> can also imagine cases where the inout local variable is defined only
>> after the local inout function has been declared and called with a
>> non-inout argument. At which point does the constraint become active?
>> etc. We cannot _simply_ allow declaring inout locals.
>
> There's no difference between a function that declares its variables
> inout within its parameters or one that declares them locally.
> ...

Yes, there is. Semantic analysis sees the parameter types before it sees the body.

> They should be treated the same once the function starts compiling.
> ...

I think I have stated clearly why this is impossible. :P

>>>>
>>>>
>>>>
>>>>> Note, the =0 part isn't necessary right now, since it's not called.
>>>>> It's
>>>>> just used to test if the function can compile.
>>>>>
>>>>> In short, my opinion on inout is that it has some unnecessary
>>>>> limitations, which can be removed, and inout will work as mostly
>>>>> expected. These requirements to work around the limitations will go
>>>>> away.
>>>>> ...
>>>>
>>>> Other important limitations of inout are e.g.:
>>>> - inout variables cannot be fields.
>>>
>>> I have a way to make this work.
>>
>> Without syntax changes?
>> Can the struct/class instances with inout fields be returned from the
>> enclosing inout function?
>
> Yes, as long as inout is wrapping inout. We run into this currently with
> inout functions that create local types. e.g. emplace.
>
> Obviously inout cannot be unwrapped if it is typed on a field of a
> struct or class. So the unwrapping has to result in inout.
> ...

Ok.

>>>> - There can be only one inout in scope.
>>>
>>> This is not so much a problem I think.
>>> ...
>>
>> I think it is. It's just not the prevalent limitation one runs in at the
>> moment. It will be more of a problem once functions can return structs
>> with inout fields. IMHO compositionality should be ensured for a
>> language feature from the start.
>
> At the point where we need to tag multiple pools of inout parameters,
> the complexity of the language doesn't justify the benefits.
> ...

I think this is a funny place to draw the line, but I guess this is a matter of taste.


> We could make it possible, for instance, to templatize the mutability
> modifier instead of using a specific keyword. Then you could have
> foo(int)[]. Then I think you could do all this (and scrap inout), but I
> wouldn't want to work in that language.
>
> -Steve

Well, that is precisely the way that languages with real type systems address issues like this one. D has many others like it.

Note that for type systems, complexity and expressiveness do not necessarily correlate.

On 04/15/2016 05:17 PM, Timon Gehr wrote:
> Well, that is precisely the way that languages with real type systems
> address issues like this one. D has many others like it.
>
> Note that for type systems, complexity and expressiveness do not
> necessarily correlate.

Nicely put on both counts. (Well "real" is semantically sarcastic a bit.) I don't even disagree :o). -- Andrei

On 15.04.2016 23:03, Andrei Alexandrescu wrote:
> On 04/15/2016 04:47 PM, Steven Schveighoffer wrote:
>> There's no difference between a function that declares its variables
>> inout within its parameters or one that declares them locally.
>
> So now we get to things like:
>
> void fun() {
>    inout int ohHello = 42;
>    ...
> }
>
> How to explain such a construct? Not to mention globals of that type are
> not allowed.
>
>
> Andrei
>

It's an int that has not decided yet whether it wants to be mutable, const or immutable and goes out of scope before it is able to make up its mind.

On 4/15/16 5:17 PM, Timon Gehr wrote:
> On 15.04.2016 22:47, Steven Schveighoffer wrote:
>>
>> There's no difference between a function that declares its variables
>> inout within its parameters or one that declares them locally.
>> ...
>
> Yes, there is. Semantic analysis sees the parameter types before it sees
> the body.

I don't know what the current implementation sees the function as doing. The way I look at it, the function context is like an extra parameter to the inner function. If it contains inout data, then that needs to be taken into account if the inner function has additional inout parameters.

For example:

inout(int) *x;

inout(int) *foo(inout int) {return x;}

foo is really taking 2 parameters: y and the context pointer that contains x. It almost looks like this:

inout(int) *foo(inout int, ref inout(int) *x) { return x;}

call this with: foo(1, x)

And it won't compile.

However, call it with: foo(inout(int)(1), x);

and it should be fine, returning an inout(int)*.

>
>> They should be treated the same once the function starts compiling.
>> ...
>
> I think I have stated clearly why this is impossible. :P

Impossible or difficult to do with the current implementation?

>> At the point where we need to tag multiple pools of inout parameters,
>> the complexity of the language doesn't justify the benefits.
>> ...
>
> I think this is a funny place to draw the line, but I guess this is a
> matter of taste.

I may have said this incorrectly. The language itself wouldn't really be that much more complex. It's the cost of understanding what each of the different inout pools mean. The benefit would be quite small, whereas there are obvious places inout makes sense -- the 'this' parameter and the return value.

Then there is the syntax that would be required, I'm not sure what that looks like.

>> We could make it possible, for instance, to templatize the mutability
>> modifier instead of using a specific keyword. Then you could have
>> foo(int)[]. Then I think you could do all this (and scrap inout), but I
>> wouldn't want to work in that language.
>
> Well, that is precisely the way that languages with real type systems
> address issues like this one. D has many others like it.

Aye, solutions like Rebindable, which is pretty much a failure IMO, show how lack of expressiveness in the core language can't be easily substituted.

> Note that for type systems, complexity and expressiveness do not
> necessarily correlate.

Humans are creatures of habit and familiarity. To allow each library to define what words mean what for modifiers would be really difficult to deal with.

-Steve

On 15.04.2016 23:56, Steven Schveighoffer wrote:
> On 4/15/16 5:17 PM, Timon Gehr wrote:
>> On 15.04.2016 22:47, Steven Schveighoffer wrote:
>>>
>>> There's no difference between a function that declares its variables
>>> inout within its parameters or one that declares them locally.
>>> ...
>>
>> Yes, there is. Semantic analysis sees the parameter types before it sees
>> the body.
>
> I don't know what the current implementation sees the function as doing.
> The way I look at it, the function context is like an extra parameter to
> the inner function. If it contains inout data,

Rather, when the inout data is actually accessed.

> then that needs to be
> taken into account if the inner function has additional inout parameters.
> ...

That's sensible, of course. The current implementation is a lot more conservative though.

> For example:
>
> inout(int) *x;
>
> inout(int) *foo(inout int) {return x;}
>
> foo is really taking 2 parameters: y and the context pointer that
> contains x. It almost looks like this:
>
> inout(int) *foo(inout int, ref inout(int) *x) { return x;}
>
> call this with: foo(1, x)
>
> And it won't compile.
>
> However, call it with: foo(inout(int)(1), x);
>
> and it should be fine, returning an inout(int)*.
>
>>
>>> They should be treated the same once the function starts compiling.
>>> ...
>>
>> I think I have stated clearly why this is impossible. :P
>
> Impossible or difficult to do with the current implementation?
> ...

What I'm saying is that the check that is currently implemented is not adequate for the new case. This is not terribly important though. The point was that there needs to be a design effort beyond lifting the limitation, and you seemed to claim that the current implementation already takes care of the presented issue, which is not the case.

Your ideas are sound though.


>>> At the point where we need to tag multiple pools of inout parameters,
>>> the complexity of the language doesn't justify the benefits.
>>> ...
>>
>> I think this is a funny place to draw the line, but I guess this is a
>> matter of taste.
>
> I may have said this incorrectly. The language itself wouldn't really be
> that much more complex. It's the cost of understanding what each of the
> different inout pools mean.

They would just be named parameters to the function on the type system level, similar to template arguments but not causing repeated instantiation.

> The benefit would be quite small, whereas
> there are obvious places inout makes sense -- the 'this' parameter and
> the return value.
> ...

The return value might contain more than one pointer, and functions often have more than one parameter.

> Then there is the syntax that would be required, I'm not sure what that
> looks like.
> ...

Anything that is analogous to template parameters, e.g. an additional set of arguments with a different delimiter. Many workable possibilities. Anyway, it is unlikely to happen.


>>> We could make it possible, for instance, to templatize the mutability
>>> modifier instead of using a specific keyword. Then you could have
>>> foo(int)[]. Then I think you could do all this (and scrap inout), but I
>>> wouldn't want to work in that language.
>>
>> Well, that is precisely the way that languages with real type systems
>> address issues like this one. D has many others like it.
>
> Aye, solutions like Rebindable, which is pretty much a failure IMO, show
> how lack of expressiveness in the core language can't be easily
> substituted.
>
>> Note that for type systems, complexity and expressiveness do not
>> necessarily correlate.
>
> Humans are creatures of habit and familiarity. To allow each library to
> define what words mean what for modifiers would be really difficult to
> deal with.
>
> -Steve

I don't see how the library would do that.