On Monday, 15 May 2017 at 21:06:57 UTC, ag0aep6g wrote:
> On 05/15/2017 10:34 PM, kinke wrote:
>> If you just want to append an extra context arg by passing it as last
>> actual arg, it'll end up in the stack sooner or later, and that, I
>> guess, is where bad things may happen by just pushing an additional arg,
>> not matching the function signature.
>
> I'm certainly reaching the boundaries of my limited knowledge here, but wouldn't it work when we push the context arg first, before any other arguments, and pop it in the caller?

It really depends on the target platform.  There may be ways to optimize it for particular systems with particular argument sets, but in the long run if you want to support this in the general case on all functions, your proposal would require every function to use the same ABI as it's delegate counterpart, which includes the code to unwind the stack if the context pointer was passed in there or any extra setup code in the caller.

If this proposal was integrated I could also imagine applications wanting to create functions that are not delegate compatible so they don't have to incur the delegate ABI overhead, i.e.

extern(NoContextPointer) void foo(int x)
{
}

Of course I suppose extern(C) would already do this.

In short, this proposal modifies existing the function ABI to use the delegate ABI which will have a runtime cost in the general case (even though some cases may have zero cost).  It's sacrificing performance for ALL functions when in practice applications will only need this for a small subset of their functions.

I like the concept but I wouldn't want it to be the default for all functions, I think it should be an "opt-in" feature for every function.  My DIP is one way you could cover this use case (a bit differently) or you could introduce a new extern like 'extern(delegate)' to say that the function should use the same ABI as it's delegate counterpart.

extern(delegate) void foo(int x)
{
}

void delegate(int) x = &foo;
x();

I'd definitely be ok with this.  It doesn't handle the use cases where you actually want the function to use the context pointer, but it handles your situation well.

On 05/15/2017 11:56 PM, Jonathan Marler wrote:
> your proposal would require every function to use the same ABI as it's
> delegate counterpart, which includes the code to unwind the stack if the
> context pointer was passed in there or any extra setup code in the caller.

I don't see how that's true. Stack cleanup of the context pointer would be done in the caller, not the callee. And no extra setup (or cleanup) is needed in the caller when the callee is a non-delegate function.

And that's the point. The goal is to keep normal, non-delegate function calls exactly as they are. That's why the context pointer is passed in a spot where it doesn't affect the rest of the call (i.e. in a free register or before the other args on the stack).

> If this proposal was integrated I could also imagine applications
> wanting to create functions that are not delegate compatible so they
> don't have to incur the delegate ABI overhead, i.e.

There should be zero overhead.

[...]
> In short, this proposal modifies existing the function ABI to use the
> delegate ABI which will have a runtime cost in the general case (even
> though some cases may have zero cost).

No. At least, that's not how it's supposed to work. The idea is to modify the delegate ABI to be compatible with the function ABI. I wouldn't touch the function ABI.

Every call to a delegate would contain code that's exactly the same as a call to a non-delegate function with the same visible parameters. That piece of code would look the same as it does today.

For example, if `f(1, 2);` results in machine code `foo` today, then `void delegate(int, int) dg = &f; dg(1, 2);` would be made to result in

    mov parameter_register_not_used_in_foo, context_pointer;
    foo;

or if `foo` already uses all registers

    push context_pointer;
    foo;
    pop;

If this doesn't work for some specific `foo`, I'd appreciate an example where it falls apart.

It doesn't work with `extern(C++)` methods, because we have to follow established calling conventions and put the `this` pointer somewhere else. So, implicit conversion wouldn't work here. That's ok.

I suspect that variadic functions might also be problematic. But I haven't checked, because I don't care much about them. Worst case, you can't implicitly convert a variadic function to a variadic delegate. Not a big issue.

On Monday, 15 May 2017 at 23:10:00 UTC, ag0aep6g wrote:
> On 05/15/2017 11:56 PM, Jonathan Marler wrote:
>> your proposal would require every function to use the same ABI as it's
>> delegate counterpart, which includes the code to unwind the stack if the
>> context pointer was passed in there or any extra setup code in the caller.
>
> I don't see how that's true. Stack cleanup of the context pointer would be done in the caller, not the callee. And no extra setup (or cleanup) is needed in the caller when the callee is a non-delegate function.
>
> And that's the point. The goal is to keep normal, non-delegate function calls exactly as they are. That's why the context pointer is passed in a spot where it doesn't affect the rest of the call (i.e. in a free register or before the other args on the stack).
>
>> If this proposal was integrated I could also imagine applications
>> wanting to create functions that are not delegate compatible so they
>> don't have to incur the delegate ABI overhead, i.e.
>
> There should be zero overhead.
>
> [...]
>> In short, this proposal modifies existing the function ABI to use the
>> delegate ABI which will have a runtime cost in the general case (even
>> though some cases may have zero cost).
>
> No. At least, that's not how it's supposed to work. The idea is to modify the delegate ABI to be compatible with the function ABI. I wouldn't touch the function ABI.
>
> Every call to a delegate would contain code that's exactly the same as a call to a non-delegate function with the same visible parameters. That piece of code would look the same as it does today.
>
> For example, if `f(1, 2);` results in machine code `foo` today, then `void delegate(int, int) dg = &f; dg(1, 2);` would be made to result in
>
>     mov parameter_register_not_used_in_foo, context_pointer;
>     foo;
>
> or if `foo` already uses all registers
>
>     push context_pointer;
>     foo;
>     pop;
>
> If this doesn't work for some specific `foo`, I'd appreciate an example where it falls apart.
>
> It doesn't work with `extern(C++)` methods, because we have to follow established calling conventions and put the `this` pointer somewhere else. So, implicit conversion wouldn't work here. That's ok.
>
> I suspect that variadic functions might also be problematic. But I haven't checked, because I don't care much about them. Worst case, you can't implicitly convert a variadic function to a variadic delegate. Not a big issue.

Ok I got your proposal backwards.  Now that I understand I think it's worth looking into.  The obvious questions would be the practical performance implications of changing how the context pointer is passed in, and also how close the current delegate ABI is to working with this.  I'm not alot of help here so I hope others respond.

It adds some restriction to the delegate ABI that didn't exist before so the next question would be is the decrease in ABI flexibility going to be an issue?  Some platforms may have very fast ways of passing the context pointer that aren't compatible with these restrictions in which case D delegates/member-functions would have to use an inferior ABI by default.  That could be addressed by adding something like "extern(optimized)" that allows the ABI to be flexible at the cost of being incompatible with a function pointer.

An interesting idea, it's sort of the reverse of my proposal hence why I was making the bad assumptions.  Will wait to hear others thoughts.

On Monday, 15 May 2017 at 10:41:55 UTC, ag0aep6g wrote:
> Now, what if we changed the ABI of delegates so that the context pointer is passed after the explicit arguments? That is, `dg(1, 2)` would pass (2, 1, 13). Then `f` would see 2 in b and 1 in a. It would ignore 13. Seems everything would just work then.

You can't pass 3 arguments to a function that takes 2 arguments. It's a violation of ABI and it won't work.

On 05/16/2017 11:35 AM, Kagamin wrote:
> On Monday, 15 May 2017 at 10:41:55 UTC, ag0aep6g wrote:
>> Now, what if we changed the ABI of delegates so that the context
>> pointer is passed after the explicit arguments? That is, `dg(1, 2)`
>> would pass (2, 1, 13). Then `f` would see 2 in b and 1 in a. It would
>> ignore 13. Seems everything would just work then.
>
> You can't pass 3 arguments to a function that takes 2 arguments. It's a
> violation of ABI and it won't work.

Can you give an example where it doesn't work? Are there requirements for registers which have no corresponding argument?

Note that the additional argument is not supposed to be used by the function. There's also no requirement that the argument survives the call. That is, if the context pointer is passed in a register, the called function may use that register for its own purposes, stomping over the context pointer.

My understanding is that we want to pass functions to functions which takes delegates transparently, or is it something more involved here?

Anyway if that is everything, then why not add a template to the D runtime which casts the function to a delegate?

And if that wrapper function is only called by the compiler it does not even have to be a template. Templetation just provides type safety in this case.

On 05/16/2017 02:57 PM, Jerry wrote:
> My understanding is that we want to pass functions to functions which
> takes delegates transparently, or is it something more involved here?

That's about it. It's not a big thing. Would just be nice to have.

> Anyway if that is everything, then why not add a template to the D
> runtime which casts the function to a delegate?

You can't currently cast a function (pointer) to a delegate. You have to create a wrapper delegate that calls the function. std.functional.toDelegate does this.

> And if that wrapper function is only called by the compiler it does not
> even have to be a template. Templetation just provides type safety in
> this case.

Walter doesn't like the idea of doing toDelegate automatically. He says it has "negative performance implications that users could find very surprising" [1].



[1] https://issues.dlang.org/show_bug.cgi?id=17156