On Monday, 17 June 2019 at 20:11:32 UTC, Manu wrote:
> Everything I need to make a strong case revolves around shared; and
> since nobody on this forum seems to know what to do with shared (we've
> been arguing about it and doing exactly nothing for the 10 years I've
> been watching)

shared works fine, your irrational unwillingness to work with it is not a valid reason to introduce an abysmal change to the language.

On 18.06.19 11:10, Walter Bright wrote:
> On 6/13/2019 6:55 PM, Timon Gehr wrote:
>> It's not pretty that the qualifiers have a different meaning for member functions and nested functions (because the nested function meaning could be useful for member functions too), but it's what we have.
> 
> Keep in mind that the address of a nested function and the address of a member function have the same ABI. In fact, closures are implemented as member functions under the hood.

I know. This is how it should be, and it is not a problem. Basically, I was complaining about the aesthetics of the fact that there is no trivial way to write a member function for only e.g. the immutable part of a struct instance. As I said, I don't think this part needs fixing.

On Tuesday, 18 June 2019 at 00:17:06 UTC, Timon Gehr wrote:
> Unfortunately I have another paper deadline in a few weeks, but maybe I can set aside a few weekends this summer to try to fix function qualifiers.

I intend to beat you to it :)

Suleyman Sahmi has located where things go right for the struct case[1], so it shouldn't be too hard to manufacture a fix for the closure case.

[1]: https://github.com/dlang/dmd/blob/f455995f234a2091c4482bff222a2977dbfa186b/src/dmd/expressionsem.d#L885-L886

On Tue, Jun 18, 2019 at 7:40 PM Kagamin via Digitalmars-d <digitalmars-d@puremagic.com> wrote:
>
> On Monday, 17 June 2019 at 20:11:32 UTC, Manu wrote:
> > Everything I need to make a strong case revolves around shared;
> > and
> > since nobody on this forum seems to know what to do with shared
> > (we've
> > been arguing about it and doing exactly nothing for the 10
> > years I've
> > been watching)
>
> shared works fine, your irrational unwillingness to work with it is not a valid reason to introduce an abysmal change to the language.

shared int x;
x = 10; // <- not an error;

Don't tell me shared works fine.

On Tue, Jun 18, 2019 at 7:15 PM Walter Bright via Digitalmars-d <digitalmars-d@puremagic.com> wrote:
>
> On 6/13/2019 6:55 PM, Timon Gehr wrote:
> > It's not pretty that the qualifiers have a different meaning for
> > member functions and nested functions (because the nested function meaning could
> > be useful for member functions too), but it's what we have.
>
> Keep in mind that the address of a nested function and the address of a member function have the same ABI. In fact, closures are implemented as member functions under the hood.

Right, as it should. But it genuinely is weird that the object is not qualified by the qualifier. If that were the case, it would all wring out in the wash rather than all these bugs and oddities.

On 18.06.19 14:00, Nicholas Wilson wrote:
> On Tuesday, 18 June 2019 at 00:17:06 UTC, Timon Gehr wrote:
>> Unfortunately I have another paper deadline in a few weeks, but maybe I can set aside a few weekends this summer to try to fix function qualifiers.
> 
> I intend to beat you to it :)
> ...

Great. :)

> Suleyman Sahmi has located where things go right for the struct case[1], so it shouldn't be too hard to manufacture a fix for the closure case.
> 
> [1]: https://github.com/dlang/dmd/blob/f455995f234a2091c4482bff222a2977dbfa186b/src/dmd/expressionsem.d#L885-L886 
> 

I believe it might be easier to locate where things go right for `shared` and `immutable` capturing and fix the logic there. I provide tests below.

Note that a few missing checks and const-promotions when capturing are not the only issue with nested function qualifiers. If the following isses are fixed, we should be in pretty good shape (but probably I forgot about something):

0. Qualified capturing is not implemented fully correctly, it is particularly bad for `const`. Full test case:

void fun(inout(int)*){
    int* x;
    const(int*) cx;
    immutable(int*) ix;
    shared(int*) sx;
    shared(const(int*)) scx;
    inout(int*) wx;
    shared(inout(int*)) swx;
    const(inout(int*)) cwx;
    shared(const(inout(int*))) scwx;
    void foo(){
        int* x=x;
        const(int)* cx=cx; // ok
        immutable(int)* ix=ix; // ok
        shared(int)* sx=sx; // ok
        shared(const(int*)) scx=scx; // ok
        inout(int)* wx=wx; // ok
        shared(inout(int))* swx=swx; // ok
        const(inout(int))* cwx=cwx; // ok
        shared(const(inout(int)))* scwx=scwx; // ok
    }
    void fooc()const{
        int* x=x; // currently ok, shouldn't compile
        const(int)* x2=x; // ok
        const(int)* cx=cx; // ok
        immutable(int)* ix=ix; // ok
        shared(int)* sx=sx; // currently ok, shouldn't compile
        const(shared(int))* sx2=sx; // ok
        shared(const(int*)) scx=scx; // ok
        inout(int)* wx=wx; // currently ok, shouldn't compile
        const(inout(int))* wx2=wx; // ok
        shared(inout(int))* swx=swx; // currently ok, shouldn't compile
        shared(const(inout(int)))* swx2=swx; // ok
        const(inout(int))* cwx=cwx; // ok
        shared(const(inout(int)))* scwx=scwx; // ok
    }
    void fooi()immutable{
        //int* x=x; // error, correct
        //const(int)* cx=cx; // error, correct
        immutable(int)* ix=ix; // ok
        //shared(int)* sx=sx; // error, correct
        //shared(const(int*)) scx=scx; // error, correct
        //inout(int)* wx=wx; // error, correct
        //shared(inout(int))* swx=swx; // error, correct
        //const(inout(int))* cwx=cwx; // error, correct
        //shared(const(inout(int)))* scwx=scwx; // error, correct
    }
    void foos()shared{
        //int* x=x; // error, correct
        //const(int)* cx=cx; // error, correct
        immutable(int)* ix=ix; // ok
        shared(int)* sx=sx; // ok
        shared(const(int*)) scx=scx; // ok
        //inout(int)* wx=wx; // error, correct
        //shared(inout(int))* swx=swx; // currently error, should work
        //const(inout(int))* cwx=cwx; // error, correct
        //shared(const(inout(int)))* scwx=scwx; // currently error, should work
    }
    void foosc()shared const{
        //int* x=x; // error, correct
        //const(int)* cx=cx; // error, correct
        immutable(int)* ix=ix; // ok
        //shared(int)* sx=sx; // error, correct
        //const(shared(int))* sx2=sx; // currently error, should work
        shared(const(int*)) scx=scx; // ok
        //inout(int)* wx=wx; // error, correct
        //const(inout(int))* wx2=wx; // currently error, should work
        //shared(inout(int))* swx=swx; // error, correct
        //const(shared(inout(int)))* swx2=swx; // currently error, should work
        //const(inout(int))* cwx=cwx; // error, correct
        //shared(const(inout(int)))* scwx=scwx; // currently error, should work
    }
    void foow()inout{
        int* x=x; // currently ok, shouldn't compile
        immutable(int)* ix=ix; // ok
        shared(int)* sx=sx; // currently ok, shouldn't compile
        inout(int)* wx=wx; // ok
        shared(inout(int))* swx=swx; // ok
        const(inout(int))* cwx=cwx; // ok
        shared(const(inout(int)))* scwx=scwx; // ok
    }
    void foosw()shared inout{
        //int* x=x; // error, correct
        immutable(int)* ix=ix; // ok
        //shared(int)* sx=sx; // error, correct
        //inout(int)* wx=wx; // error, correct
        shared(inout(int))* swx=swx; // ok
        //const(inout(int))* cwx=cwx; // error, correct
        shared(const(inout(int)))* scwx=scwx; // ok
    }
    void fooscw()shared const inout{
        //int* x=x; // error, correct
        immutable(int)* ix=ix; // ok
        //shared(int)* sx=sx; // error, correct
        //inout(int)* wx=wx; // error, correct
        //shared(inout(int))* swx=swx; // error, correct
        //const(shared(inout(int)))* swx2=swx; // currently error, should compile
        //const(inout(int))* cwx=cwx; // error, correct
        shared(const(inout(int)))* scwx=scwx; // ok
    }
}



1. I can break the type system like this:

void main(){
    int* x=new int;
    struct S{
        int* delegate()pure dg1;
        int* dg2()pure immutable{
            return dg1(); // this shouldn't compile
        }
    }
    // you may think the next line is the problem, but this is actually ok:
    auto s=immutable(S)(()=>x);
    immutable(int*) y=s.dg2();
    assert(x is y); // mutable/immutable aliasing
}

The problem is that it is possible to call a immutable(T delegate(Args)). This is wrong. It should only be possible to call a qualified delegate if the delegate context has the respective (or a stronger) qualifier.

So

const(int* delegate()) dg = ...;
dg(); // should be error

const(int* delegate()immutable) dg = ...;
dg(); // this is fine

Note that the obvious idea of saying that the qualifier of the delegate transitively applies to the opaque delegate context does not work, because e.g. the delegate context cannot implicitly pick up a `const`, as the function pointer will access the context in a way that is typed mutable.


2. nested functions of `pure` functions are forced to be `pure`, but this is nonsense, they should just infer purity, like it is done for @safe:

int foo(){ return 2; }

void main()pure{
    enum x=(()=>foo())(); // Error: `pure` delegate `tt.main.__lambda1` cannot call impure function `tt.foo`
}
(I picked this example because here it is obvious that the error is nonsense, but this is a more general problem and not restricted to CTFE.)

In contrast, it works perfectly fine for @safe:

int foo(){ return 2; }

void main()@safe{
    enum x=(()=>foo())(); // ok!
}

I think it is terrible that the implementations of `@safe` inference and `pure` inference were allowed to diverge at all.


3. The same problem exists for @nogc, but the fix is easy. Here it is:
https://github.com/dlang/dmd/pull/9922#issuecomment-499544100

(I will create a pull request eventually, unless someone beats me to it. I would still need to write some tests.)


4. nested functions should infer `const`, `immutable`, `shared` and `inout` qualifiers based on the qualifiers of variables that they capture.


5. it should be possible to drop delegate context qualifiers:

int delegate()const dgc;
int delegate() dgc2=dgc; // this is correctly accepted
int delegate()immutable dgi;
int delegate() dgi2=dgi; // this is rejected incorrectly
int delegate() dgi3=()=>dgi(); // ugly workaround
int delegate()shared dgs;
int delegate() dgs2=dgs; // this is rejected incorrectly
int delegate() dgs3=()=>dgs(); // ugly workaround


6. `inout` should be implemented in a way that prevents scope confusion by design.

On 18.06.19 14:09, Manu wrote:
> On Tue, Jun 18, 2019 at 7:15 PM Walter Bright via Digitalmars-d
> <digitalmars-d@puremagic.com> wrote:
>>
>> On 6/13/2019 6:55 PM, Timon Gehr wrote:
>>> It's not pretty that the qualifiers have a different meaning for
>>> member functions and nested functions (because the nested function meaning could
>>> be useful for member functions too), but it's what we have.
>>
>> Keep in mind that the address of a nested function and the address of a member
>> function have the same ABI. In fact, closures are implemented as member
>> functions under the hood.
> 
> Right, as it should. But it genuinely is weird that the object is not
> qualified by the qualifier.

It is, the compiler just saves an allocation. I think this optimization would be valid in general.

> If that were the case, it would all wring
> out in the wash rather than all these bugs and oddities.
> 

Again, this is untrue. Most bugs around function qualifiers are not restricted to nested functions. There might have been a couple fewer bugs if closures were type checked after lowering, but this has other issues (error message quality).

On 18.06.19 15:24, Timon Gehr wrote:
> On 18.06.19 14:00, Nicholas Wilson wrote:
>> On Tuesday, 18 June 2019 at 00:17:06 UTC, Timon Gehr wrote:
>>> Unfortunately I have another paper deadline in a few weeks, but maybe I can set aside a few weekends this summer to try to fix function qualifiers.
>>
>> I intend to beat you to it :)
>> ...
> 
> Great. :)
> 
>> Suleyman Sahmi has located where things go right for the struct case[1], so it shouldn't be too hard to manufacture a fix for the closure case.
>>
>> [1]: https://github.com/dlang/dmd/blob/f455995f234a2091c4482bff222a2977dbfa186b/src/dmd/expressionsem.d#L885-L886 
>>
> 
> I believe it might be easier to locate where things go right for `shared` and `immutable` capturing and fix the logic there. I provide tests below.
> 
> Note that a few missing checks and const-promotions when capturing are not the only issue with nested function qualifiers. If the following isses are fixed, we should be in pretty good shape (but probably I forgot about something):
> 
> ...
> 

On 18.06.19 15:24, Timon Gehr wrote:
> On 18.06.19 14:00, Nicholas Wilson wrote:
>> On Tuesday, 18 June 2019 at 00:17:06 UTC, Timon Gehr wrote:
>>> Unfortunately I have another paper deadline in a few weeks, but maybe I can set aside a few weekends this summer to try to fix function qualifiers.
>>
>> I intend to beat you to it :)
>> ...
> 
> Great. :)
> 
>> Suleyman Sahmi has located where things go right for the struct case[1], so it shouldn't be too hard to manufacture a fix for the closure case.
>>
>> [1]: https://github.com/dlang/dmd/blob/f455995f234a2091c4482bff222a2977dbfa186b/src/dmd/expressionsem.d#L885-L886 
>>
> 
> I believe it might be easier to locate where things go right for `shared` and `immutable` capturing and fix the logic there. I provide tests below.
> 
> Note that a few missing checks and const-promotions when capturing are not the only issue with nested function qualifiers. If the following isses are fixed, we should be in pretty good shape (but probably I forgot about something):
> 
> ...

In fact I did.

7. Local function calls need to be checked as if they were delegate captures.

The general rule is that the code

void fun(){
    void bar()qualifiers1{ ... }
    void foo()qualifiers2{
        bar();
    }
}

should compile exactly if the following code compiles:

void fun(){
    void bar()qualifiers1{ ... }
    qualifiers2 dg=&bar; // conversion always ok
    void foo()qualifiers2{
        dg(); // capturing of variable 'dg' always ok, but call may not be
    }
}

This yields a simple way to implement the check. Tests:

void fun(inout(int)*){
    void bar(){}
    void barc()const{}
    void bari()immutable{}
    void bars()shared{}
    void barsc()shared const{}
    void barw()inout{}
    void barsw()shared inout{}
    void barcw()const inout{}
    void barscw()shared const inout{}
    void foo(){
        bar(); // ok
        barc(); // ok
        bari(); // ok
        bars(); // ok
        barsc(); // ok
        barsw(); // ok
        barcw(); // ok
        barscw(); // ok
    }
    void fooc()const{
        bar(); // currently ok, shouldn't compile
        barc(); // ok
        bari(); // ok
        bars(); // currently ok, shouldn't compile
        barsc(); // ok
        barsw(); // currently ok, shouldn't compile
        barcw(); // ok
        barscw(); // ok
    }
    void fooi()immutable{
        bar(); // currently ok, shouldn't compile
        barc(); // currently ok, shouldn't compile
        bari(); // ok
        bars(); // currently ok, shouldn't compile
        barsc(); // currently ok, shouldn't compile
        barsw(); // currently ok, shouldn't compile
        barcw(); // currently ok, shouldn't compile
        barscw(); // currently ok, shouldn't compile
    }
    void foos()shared{
        bar(); // currently ok, shouldn't compile
        barc(); // currently ok, shouldn't compile
        bari(); // ok
        bars(); // ok
        barsc(); // ok
        barsw(); // ok
        barcw(); // currently ok, shouldn't compile
        barscw(); // ok
    }
    void foosc()shared const{
        bar(); // currently ok, shouldn't compile
        barc(); // currently ok, shouldn't compile
        bari(); // ok
        bars(); // currently ok, shouldn't compile
        barsc(); // ok
        barsw(); // currently ok, shouldn't compile
        barcw(); // currently ok, shouldn't compile
        barscw(); // ok
    }
    void foow()inout{
        bar(); // currently ok, shouldn't compile
        barc(); // currently ok, shouldn't compile
        bari(); // ok
        bars(); // currently ok, shouldn't compile
        barsc(); // currently ok, shouldn't compile
        barsw(); // ok
        barcw(); // ok
        barscw(); // ok
    }
    void foosw()shared inout{
        bar(); // currently ok, shouldn't compile
        barc(); // currently ok, shouldn't compile
        bari(); // ok
        bars(); // currently ok, shouldn't compile
        barsc(); // currently ok, shouldn't compile
        barsw(); // ok
        barcw(); // currently ok, shouldn't compile
        barscw(); // ok
    }
    void fooscw()shared const inout{
        bar(); // currently ok, shouldn't compile
        barc(); // currently ok, shouldn't compile
        bari(); // ok
        bars(); // currently ok, shouldn't compile
        barsc(); // currently ok, shouldn't compile
        barsw(); // currently ok, shouldn't compile
        barcw(); // currently ok, shouldn't compile
        barscw(); // ok
    }
}

On 18.06.19 15:37, Timon Gehr wrote:
> ...

(Sorry, thunderbird hiccup.)