Please consider the following program:

struct V(T) {
    int opApply(scope int delegate(ref T value) dg) {
        return 0;
    }
    int opApply(scope int delegate(ref const T value) dg) const {
        return 0;
    }
}

struct V1(T) {
    int opApply(scope int delegate(ref inout T value) dg) inout {
        return 0;
    }
}

void main() {
    const V!int mytype1;
    V!int mytype2;
    V1!int mytype3;

    foreach( v; mytype1 ) {
    }
    foreach( v; mytype2 ) {
    }
    foreach( v; mytype3 ) { // <- This line doesn't compile: cannot uniquely infer foreach argument types
    }
}

So the only way to get inout to work is... not to use it?

Shachar

On 06/20/2016 04:47 PM, Shachar Shemesh wrote:
> Please consider the following program:
> 
> struct V(T) {
>     int opApply(scope int delegate(ref T value) dg) {
>         return 0;
>     }
>     int opApply(scope int delegate(ref const T value) dg) const {
>         return 0;
>     }
> }
> 
> struct V1(T) {
>     int opApply(scope int delegate(ref inout T value) dg) inout {
>         return 0;
>     }
> }
> 
> void main() {
>     const V!int mytype1;
>     V!int mytype2;
>     V1!int mytype3;
> 
>     foreach( v; mytype1 ) {
>     }
>     foreach( v; mytype2 ) {
>     }
>     foreach( v; mytype3 ) { // <- This line doesn't compile: cannot
> uniquely infer foreach argument types
>     }
> }
> 
> So the only way to get inout to work is... not to use it?
> 
> Shachar

I think Steven has mentioned this issue during his DConf inout talk.

On 6/20/16 9:55 AM, Dicebot wrote:
> On 06/20/2016 04:47 PM, Shachar Shemesh wrote:
>> Please consider the following program:
>>
>> struct V(T) {
>>     int opApply(scope int delegate(ref T value) dg) {
>>         return 0;
>>     }
>>     int opApply(scope int delegate(ref const T value) dg) const {
>>         return 0;
>>     }
>> }
>>
>> struct V1(T) {
>>     int opApply(scope int delegate(ref inout T value) dg) inout {
>>         return 0;
>>     }
>> }
>>
>> void main() {
>>     const V!int mytype1;
>>     V!int mytype2;
>>     V1!int mytype3;
>>
>>     foreach( v; mytype1 ) {
>>     }
>>     foreach( v; mytype2 ) {
>>     }
>>     foreach( v; mytype3 ) { // <- This line doesn't compile: cannot
>> uniquely infer foreach argument types
>>     }
>> }
>>
>> So the only way to get inout to work is... not to use it?
>
> I think Steven has mentioned this issue during his DConf inout talk.
>

Yes, it's something I'm going to post about soon.

In essence, the delegate which takes inout parameter is currently its own wrapping of inout. Meaning, the implicit delegate the compiler creates does not participate in the inout wrapping for the opApply call.

Consider how this works for the compiler. I'll give you an example:

V1!int mutableX;

foreach(ref int a; mutableX)
{
   a = 5;
}

Here, I've removed the type inference to set that issue aside for now.

What the compiler does is convert your foreach body into a local delegate. This local delegate looks like this:

int __foreachBody(ref int a)
{
   a = 5;

   // this is added by the compiler for flow control
   return 0;
}

and then calls:

switch(mutableX.opApply(&__foreachBody))
{
   // process specialized returns from delegate
}

So, ignoring the special opApply machinery, we can see the fundamental issue here is that we are passing a function pointer of type int (ref int a) into the opApply. However, currently, the compiler treats a function pointer with inout parameters to be simply a pointer to an inout function. The constructed delegate cannot be this, since a is ref int, not ref inout(int). So it cannot compile. We could change the body to not modify a, and possibly we could create a solution, but it's not ideal.

What I would like the compiler to do (and I went over this in my talk), is to allow the compiler to inout-wrap a delegate along with the other inout prameters to the function. That is, for:

int opApply(scope int delegate(ref inout T value) dg) inout

The inout inside the delegate is wrapped just like the inout of the 'this' parameter. effectively, this becomes equivalent to several function signatures:

int opApply(scope int delegate(ref T value) dg)
int opApply(scope int delegate(ref const T value) dg) const
int opApply(scope int delegate(ref immutable T value) dg) immutable
int opApply(scope int delegate(ref inout T value) dg) inout

And interestingly enough, the rules are kind of backwards for delegates -- while inout doesn't cast to anything, delegates with inout parameters can cast to any type of mutability modifier (I believe this is called contravariance). This is because the actual function is inout, so it cannot harm the mutability. So something like this:

foreach(inout a; anyV1)
{
}

should work for any flavor of V1.

I think this should work with existing code, and would simply open up things like opApply to inout support.

The only issue is that the compiler has to assume that while calling the delegate, the inout parameters passed COULD change value, because it doesn't know whether the passed delegate is truly inout, or just matches the constancy of the type, which could potentially be mutable. This differs from current expectations of inout delegate or function pointers.

It's a lot of complex explanation, but the end result is that you could simply tag your opApply's with inout and have one version for all modifiers.

-Steve

On 20.06.2016 17:09, Steven Schveighoffer wrote:
>
> What I would like the compiler to do (and I went over this in my talk),
> is to allow the compiler to inout-wrap a delegate along with the other
> inout prameters to the function. That is, for:
>
> int opApply(scope int delegate(ref inout T value) dg) inout
>
> The inout inside the delegate is wrapped just like the inout of the
> 'this' parameter. effectively, this becomes equivalent to several
> function signatures:
>
> int opApply(scope int delegate(ref T value) dg)
> int opApply(scope int delegate(ref const T value) dg) const
> int opApply(scope int delegate(ref immutable T value) dg) immutable
> int opApply(scope int delegate(ref inout T value) dg) inout
>
> And interestingly enough, the rules are kind of backwards for delegates
> -- while inout doesn't cast to anything, delegates with inout parameters
> can cast to any type of mutability modifier (I believe this is called
> contravariance). This is because the actual function is inout, so it
> cannot harm the mutability. So something like this:
>
> foreach(inout a; anyV1)
> {
> }
>
> should work for any flavor of V1.
>
> I think this should work with existing code, and would simply open up
> things like opApply to inout support.
>
> The only issue is that the compiler has to assume that while calling the
> delegate, the inout parameters passed COULD change value, because it
> doesn't know whether the passed delegate is truly inout, or just matches
> the constancy of the type, which could potentially be mutable. This
> differs from current expectations of inout delegate or function pointers.
>
> It's a lot of complex explanation, but the end result is that you could
> simply tag your opApply's with inout and have one version for all modifiers.

The problem here is that both variants make sense depending on context and there is no syntax to distinguish between them. This proposal interacts in a weird way with IFTI.

On Monday, 20 June 2016 at 15:09:22 UTC, Steven Schveighoffer wrote:
> int opApply(scope int delegate(ref inout T value) dg) inout
>
> The inout inside the delegate is wrapped just like the inout of the 'this' parameter. effectively, this becomes equivalent to several function signatures:
>
> int opApply(scope int delegate(ref T value) dg)
> int opApply(scope int delegate(ref const T value) dg) const
> int opApply(scope int delegate(ref immutable T value) dg) immutable
> int opApply(scope int delegate(ref inout T value) dg) inout

AFAIK, there was a bugzilla issue for it. Wasn't it implemented?

On 6/21/16 5:19 PM, Timon Gehr wrote:

>
> The problem here is that both variants make sense depending on context
> and there is no syntax to distinguish between them. This proposal
> interacts in a weird way with IFTI.

I know you are probably right, but can you explain maybe via an example what you mean? I'm not understanding your point.

-Steve

On 6/22/16 6:28 AM, Kagamin wrote:
> On Monday, 20 June 2016 at 15:09:22 UTC, Steven Schveighoffer wrote:
>> int opApply(scope int delegate(ref inout T value) dg) inout
>>
>> The inout inside the delegate is wrapped just like the inout of the
>> 'this' parameter. effectively, this becomes equivalent to several
>> function signatures:
>>
>> int opApply(scope int delegate(ref T value) dg)
>> int opApply(scope int delegate(ref const T value) dg) const
>> int opApply(scope int delegate(ref immutable T value) dg) immutable
>> int opApply(scope int delegate(ref inout T value) dg) inout
>
> AFAIK, there was a bugzilla issue for it. Wasn't it implemented?

No, this hasn't happened yet. There may be a bugzilla issue for it, I'm not sure.

-Steve

On 23.06.2016 14:58, Steven Schveighoffer wrote:
> On 6/21/16 5:19 PM, Timon Gehr wrote:
>
>>
>> The problem here is that both variants make sense depending on context
>> and there is no syntax to distinguish between them. This proposal
>> interacts in a weird way with IFTI.
>
> I know you are probably right, but can you explain maybe via an example
> what you mean? I'm not understanding your point.
>
> -Steve

One might legitimately want to pass a inout delegate to some inout function and then use the delegate on data with arbitrary mutability qualifiers, not just inout.


The IFTI comment was about for example something like this:

struct S{
    void bar(inout int){}
    void foo(T)(T arg)inout{
        // this looks like it can be called with any argument
        arg(&bar);
    }
}

void main(){
    immutable(S) s;
    void delegate(void delegate(inout(int))) dg = x=>x(2);
    s.foo(dg); // today, this works
    // with the proposed semantics, IFTI deduces
    // T = void delegate(void delegate(inout(int)))
    // but then cannot convert dg to a T after
    // inout resolution
}

But I don't feel strongly about which of the two variants it should be. It's weird either way, as long as there can be only one inout in scope.

On 6/23/16 3:20 PM, Timon Gehr wrote:
> On 23.06.2016 14:58, Steven Schveighoffer wrote:
>> On 6/21/16 5:19 PM, Timon Gehr wrote:
>>
>>>
>>> The problem here is that both variants make sense depending on context
>>> and there is no syntax to distinguish between them. This proposal
>>> interacts in a weird way with IFTI.
>>
>> I know you are probably right, but can you explain maybe via an example
>> what you mean? I'm not understanding your point.
>>
>
> One might legitimately want to pass a inout delegate to some inout
> function and then use the delegate on data with arbitrary mutability
> qualifiers, not just inout.

As you say, we don't have a lot of options. We only have one designation for inout, and we must treat it the same no matter where it is.

A possible workaround is to store the inout function pointer/delegate as a global variable. I don't like that idea at all, but it would work.

Perhaps we could find a way to mark a function parameter as not being part of the inout wrapping, or perhaps mark it as *being* part of the inout wrapping (to be backwards compatible). But this is a special thing for delegates and function pointers only, since it makes no sense for plain parameters.

The more I think about it, the more I think we need to have a clear demarcation of when you want the inout-containing delegate to participate in the wrapping or not. We can't eliminate existing behavior, and without special marking, the rules are going to be ugly and unintuitive.

> The IFTI comment was about for example something like this:
>
> struct S{
>     void bar(inout int){}
>     void foo(T)(T arg)inout{
>         // this looks like it can be called with any argument
>         arg(&bar);
>     }
> }
>
> void main(){
>     immutable(S) s;
>     void delegate(void delegate(inout(int))) dg = x=>x(2);
>     s.foo(dg); // today, this works
>     // with the proposed semantics, IFTI deduces
>     // T = void delegate(void delegate(inout(int)))
>     // but then cannot convert dg to a T after
>     // inout resolution
> }

Considering the possible use cases for something like this, I think we are better off having this limitation than not being able to use opApply with inout.

-Steve

On 24/06/16 01:22, Steven Schveighoffer wrote:

> Considering the possible use cases for something like this, I think we
> are better off having this limitation than not being able to use opApply
> with inout.
>
> -Steve

I am not sure I followed the discussion correctly. If I did, however, it seems to me the following would also become impossible:

size_t numElements(T)(const ref T val) {
  size_t num;

  foreach(i; val) {
    num++;
  }

  return num;
}

If "val" being const means that the delegate passed to opApply must also be const, the meaning is that that delegate may not change "num", which seems, to me, unworkable.

All assuming I correctly understood your suggestion.

Shachar