Jump to page: 1 2
Thread overview
simple ABI change to enable implicit conversion of functions to delegates?
May 15, 2017
ag0aep6g
May 15, 2017
kinke
May 15, 2017
Jonathan Marler
May 15, 2017
ag0aep6g
May 15, 2017
Jonathan Marler
May 15, 2017
ag0aep6g
May 15, 2017
kinke
May 15, 2017
ag0aep6g
May 15, 2017
Jonathan Marler
May 15, 2017
ag0aep6g
May 16, 2017
Jonathan Marler
May 15, 2017
ag0aep6g
May 15, 2017
kinke
May 16, 2017
Kagamin
May 16, 2017
ag0aep6g
May 16, 2017
Jerry
May 16, 2017
ag0aep6g
May 15, 2017
TL;DR: Changing the ABI of delegates so that the context pointer is passed last would make functions implicitly convertible to delegates, no?

In the discussion of issue 17156 [1], Eyal asks why functions (function pointers?) don't convert implicitly to delegates. Walter's answer is that their ABIs differ and that a wrapper would have to be generated to treat a function transparently as a delegate.

As far as I understand, the problem is that the hidden context pointer of a delegate takes the first register, pushing the other parameters back. That means the visible arguments are passed in different registers than when calling a function.

Some code to show this:

----
void delegate(int a, int b) dg;
void f(int a, int b) { import std.stdio; writeln(a, " ", b); }

void main()
{
    dg.funcptr = &f; /* This line should probably not compile, but that's
        another story. */
    dg.ptr = cast(void*) 13;
    f(1, 2); /* prints "1 2" - no surprise */
    dg(1, 2); /* prints "2 13" */
}
----

Arguments are put into registers in reverse order. I.e., in a sense, the call `f(1, 2)` passes (2, 1) to f. And the call `dg(1, 2)` passes (13, 2, 1), because a delegate has a hidden last parameter: the context pointer. But `f` isn't compiled with such a hidden parameter, so it sees 13 in `b` and 2 in `a`. The register that holds 1 is simply ignored because there's no corresponding parameter.

Now, what if we changed the ABI of delegates so that the context pointer is passed after the explicit arguments? That is, `dg(1, 2)` would pass (2, 1, 13). Then `f` would see 2 in b and 1 in a. It would ignore 13. Seems everything would just work then.

This seems quite simple. But I'm most probably just too ignorant to see the problems. Why wouldn't this work? Maybe there's a reason why the context pointer has to be passed first?



[1] https://issues.dlang.org/show_bug.cgi?id=17156
May 15, 2017
On Monday, 15 May 2017 at 10:41:55 UTC, ag0aep6g wrote:
> TL;DR: Changing the ABI of delegates so that the context pointer is passed last would make functions implicitly convertible to delegates, no?
>
> In the discussion of issue 17156 [1], Eyal asks why functions (function pointers?) don't convert implicitly to delegates. Walter's answer is that their ABIs differ and that a wrapper would have to be generated to treat a function transparently as a delegate.
>
> As far as I understand, the problem is that the hidden context pointer of a delegate takes the first register, pushing the other parameters back. That means the visible arguments are passed in different registers than when calling a function.
>
> Some code to show this:
>
> ----
> void delegate(int a, int b) dg;
> void f(int a, int b) { import std.stdio; writeln(a, " ", b); }
>
> void main()
> {
>     dg.funcptr = &f; /* This line should probably not compile, but that's
>         another story. */
>     dg.ptr = cast(void*) 13;
>     f(1, 2); /* prints "1 2" - no surprise */
>     dg(1, 2); /* prints "2 13" */
> }
> ----
>
> Arguments are put into registers in reverse order. I.e., in a sense, the call `f(1, 2)` passes (2, 1) to f. And the call `dg(1, 2)` passes (13, 2, 1), because a delegate has a hidden last parameter: the context pointer. But `f` isn't compiled with such a hidden parameter, so it sees 13 in `b` and 2 in `a`. The register that holds 1 is simply ignored because there's no corresponding parameter.
>
> Now, what if we changed the ABI of delegates so that the context pointer is passed after the explicit arguments? That is, `dg(1, 2)` would pass (2, 1, 13). Then `f` would see 2 in b and 1 in a. It would ignore 13. Seems everything would just work then.
>
> This seems quite simple. But I'm most probably just too ignorant to see the problems. Why wouldn't this work? Maybe there's a reason why the context pointer has to be passed first?
>
>
>
> [1] https://issues.dlang.org/show_bug.cgi?id=17156

First of all, please don't forget that we're not only targeting X86, and that the args, according to the docs, shouldn't actually be reversed (incl. extern(D) - just on Win32, everywhere else the C ABI is to be followed).
Then some ABIs, like Microsoft's, treat ` this` in a special way, not just like any other argument (in combination with struct-return), which would apply to method calls via a delegate with context = object reference.

Some additional context: https://github.com/dlang/dmd/pull/5232
May 15, 2017
On Monday, 15 May 2017 at 12:27:10 UTC, kinke wrote:
> On Monday, 15 May 2017 at 10:41:55 UTC, ag0aep6g wrote:
>> TL;DR: Changing the ABI of delegates so that the context pointer is passed last would make functions implicitly convertible to delegates, no?
>>
>> In the discussion of issue 17156 [1], Eyal asks why functions (function pointers?) don't convert implicitly to delegates. Walter's answer is that their ABIs differ and that a wrapper would have to be generated to treat a function transparently as a delegate.
>>
>> As far as I understand, the problem is that the hidden context pointer of a delegate takes the first register, pushing the other parameters back. That means the visible arguments are passed in different registers than when calling a function.
>>
>> Some code to show this:
>>
>> ----
>> void delegate(int a, int b) dg;
>> void f(int a, int b) { import std.stdio; writeln(a, " ", b); }
>>
>> void main()
>> {
>>     dg.funcptr = &f; /* This line should probably not compile, but that's
>>         another story. */
>>     dg.ptr = cast(void*) 13;
>>     f(1, 2); /* prints "1 2" - no surprise */
>>     dg(1, 2); /* prints "2 13" */
>> }
>> ----
>>
>> Arguments are put into registers in reverse order. I.e., in a sense, the call `f(1, 2)` passes (2, 1) to f. And the call `dg(1, 2)` passes (13, 2, 1), because a delegate has a hidden last parameter: the context pointer. But `f` isn't compiled with such a hidden parameter, so it sees 13 in `b` and 2 in `a`. The register that holds 1 is simply ignored because there's no corresponding parameter.
>>
>> Now, what if we changed the ABI of delegates so that the context pointer is passed after the explicit arguments? That is, `dg(1, 2)` would pass (2, 1, 13). Then `f` would see 2 in b and 1 in a. It would ignore 13. Seems everything would just work then.
>>
>> This seems quite simple. But I'm most probably just too ignorant to see the problems. Why wouldn't this work? Maybe there's a reason why the context pointer has to be passed first?
>>
>>
>>
>> [1] https://issues.dlang.org/show_bug.cgi?id=17156
>
> First of all, please don't forget that we're not only targeting X86, and that the args, according to the docs, shouldn't actually be reversed (incl. extern(D) - just on Win32, everywhere else the C ABI is to be followed).
> Then some ABIs, like Microsoft's, treat ` this` in a special way, not just like any other argument (in combination with struct-return), which would apply to method calls via a delegate with context = object reference.
>
> Some additional context: https://github.com/dlang/dmd/pull/5232

Not sure if members in this conversation were aware of my DIP to address this issue:

https://github.com/dlang/DIPs/pull/61

The DIP uses a different approach to solve this same problem. It adds new semantics to specify whether a function should use the same ABI as a delegate, called a "delegateable function".

The solution you have proposed would remove the need for my DIP by reconciling the difference between the function ABI and delegate ABI.  You have two ways to go about this, either modify the delegate ABI to match the function ABI, or the reverse.  The specific change could be stated as:

Modify the function/delegate ABI so that the first parameter of every function is passed in the same way as the context pointer of a delegate.

The questions is whether or not this restriction is reasonable.  Currently I don't believe the language restricts any target machine to use a particular ABI. The only real restriction would be that functions with the same signature use the same ABI, but functions that add a parameter or use a different width on one of them can drastically change the ABI however they want.  This restriction could circumvent optimizations in the ABI since every function would HAVE TO be compatible with delegates. Take the following example.

Say you had a 32-bit machine with one 8-bit register and two 32-bit registers.  Since it's a 32-bit machine, the obvious ABI for delegates would be to use one of the 32-bit registers for the context pointer.  Now say you had a function like this:

void foo(byte a, int b, int c)
{
}

The obviously optimized ABI would be to pass the three parameters in the 3 registers, however, if you require that the first parameter of every function must use the same ABI as delegates then you would have to pass the 8-bit argument "a" in the 32-bit register.  This means you could not longer pass all the parameters in the registers so either "b" or "c" would need to be passed in some other way, maybe on the stack.

If the application never intends on passing foo to a delegate, you've just removed an optimization for no benefit.


May 15, 2017
On 05/15/2017 02:27 PM, kinke wrote:
> First of all, please don't forget that we're not only targeting X86, and
> that the args, according to the docs, shouldn't actually be reversed
> (incl. extern(D) - just on Win32, everywhere else the C ABI is to be
> followed).

As far as I see, it doesn't matter if the arguments are reversed or not. The idea is to pass the context pointer in a way that doesn't affect the other arguments. Put it in a spot that would otherwise be unused. I.e., if the arguments are not reversed, add the context pointer as a new last parameter, and if the arguments are reversed, make it a new first parameter.

> Then some ABIs, like Microsoft's, treat ` this` in a special way, not
> just like any other argument (in combination with struct-return), which
> would apply to method calls via a delegate with context = object reference.

This __thiscall thing, right? https://msdn.microsoft.com/en-us/library/ek8tkfbw.aspx

So, `this` always goes in ECX in that case. Yeah, that's a problem, I guess.

But __thiscall is for C++, and we don't have to adhere to it in D, do we? I mean, I know that D is supposed to (mostly) follow the C calling convention of the system, but matching C++ is not a goal, or is it?

`extern(C++)` functions/delegates have to follow it, obviously. But then we can just say that implicit conversion doesn't work with those.

> Some additional context: https://github.com/dlang/dmd/pull/5232

What I take from that is that changing the way arguments are passed (particularly if they're reversed or not) is going to break a ton of stuff.
May 15, 2017
On 05/15/2017 05:44 PM, Jonathan Marler wrote:
> Not sure if members in this conversation were aware of my DIP to address
> this issue:
>
> https://github.com/dlang/DIPs/pull/61
>
> The DIP uses a different approach to solve this same problem. It adds
> new semantics to specify whether a function should use the same ABI as a
> delegate, called a "delegateable function".
>
> The solution you have proposed would remove the need for my DIP by
> reconciling the difference between the function ABI and delegate ABI.

I was aware of your DIP. I think they're related, but don't really overlap that much.

You want to add a special kind of parameter that is taken as the context pointer when making a delegate of the function. You also add syntax to combine such a function with a matching `this` into a delegate.

I'd like the context pointer to be passed differently, so that a normal function doesn't mistake it for one of its parameters.

I don't think my thing would make your thing obsolete.

> You have two ways to go about this, either modify the delegate ABI to
> match the function ABI, or the reverse.  The specific change could be
> stated as:
>
> Modify the function/delegate ABI so that the first parameter of every
> function is passed in the same way as the context pointer of a delegate.

If I'm reading it right, with that rule this would be valid:

----
struct Foo { int x; }
void baz(Foo* foo, int y) { import std.stdio; writeln(foo.x, " ", y); }
void main()
{
    void delegate(int y) dg;
    dg.funcptr = &baz; /* parameter `foo` is taken as context pointer */
    dg.ptr = new Foo(1);
    dg(2); /* would print "1 2" */
}
----

That's not what I'm after. With my (crude) idea, that code would be just as invalid as it is now. A function's parameters would have to match the (visible) parameters of the delegate type. Any context pointer would be passed in a spot where the function doesn't look.

The point is to allow using functions as delegates that don't use the context pointer.
May 15, 2017
On Monday, 15 May 2017 at 17:03:20 UTC, ag0aep6g wrote:
> On 05/15/2017 02:27 PM, kinke wrote:
>> Some additional context: https://github.com/dlang/dmd/pull/5232
>
> What I take from that is that changing the way arguments are passed (particularly if they're reversed or not) is going to break a ton of stuff.

Well, when I experimentally didn't reverse the args for extern(D) back then for LDC (after patching druntime/Phobos inline asm accordingly...), that single issue prevented a fully green testsuite.
The problem is that druntime there goes the other way and invokes a method via a function pointer, so in essence the inverse of what you're after.

The problem there is that this/context may be passed differently on Win64; I checked, and LDC only does it for `extern(C++)` for Visual C++ compatibility, not for extern(D), so OTOH the (absolutely unintuitive) resulting arguments order for Win64 should currently be:

extern(C++) BigStruct freeFunC(Object this, int b, int c)
  => __sret, this, b, c
extern(D)   BigStruct freeFunD(Object this, int b, int c)
  => __sret, c, b, this
extern(C++) BigStruct Object.funC(int b, int c)
  => __this, __sret, b, c
extern(D)   BigStruct Object.funD(int b, int c)
  => __sret, __this, c, b

And yes, for Win32 there's the __thiscall convention, but also only for extern(C++).

> `extern(C++)` functions/delegates have to follow it, obviously. But then we can just say that implicit conversion doesn't work with those.

Doesn't sound that bad as long as the front-end enforces it.
May 15, 2017
On Monday, 15 May 2017 at 17:06:34 UTC, ag0aep6g wrote:
> On 05/15/2017 05:44 PM, Jonathan Marler wrote:
>> Not sure if members in this conversation were aware of my DIP to address
>> this issue:
>>
>> https://github.com/dlang/DIPs/pull/61
>>
>> The DIP uses a different approach to solve this same problem. It adds
>> new semantics to specify whether a function should use the same ABI as a
>> delegate, called a "delegateable function".
>>
>> The solution you have proposed would remove the need for my DIP by
>> reconciling the difference between the function ABI and delegate ABI.
>
> I was aware of your DIP. I think they're related, but don't really overlap that much.
>
> You want to add a special kind of parameter that is taken as the context pointer when making a delegate of the function. You also add syntax to combine such a function with a matching `this` into a delegate.
>
> I'd like the context pointer to be passed differently, so that a normal function doesn't mistake it for one of its parameters.
>
> I don't think my thing would make your thing obsolete.
>
>> You have two ways to go about this, either modify the delegate ABI to
>> match the function ABI, or the reverse.  The specific change could be
>> stated as:
>>
>> Modify the function/delegate ABI so that the first parameter of every
>> function is passed in the same way as the context pointer of a delegate.
>
> If I'm reading it right, with that rule this would be valid:
>
> ----
> struct Foo { int x; }
> void baz(Foo* foo, int y) { import std.stdio; writeln(foo.x, " ", y); }
> void main()
> {
>     void delegate(int y) dg;
>     dg.funcptr = &baz; /* parameter `foo` is taken as context pointer */
>     dg.ptr = new Foo(1);
>     dg(2); /* would print "1 2" */
> }
> ----
>
> That's not what I'm after. With my (crude) idea, that code would be just as invalid as it is now. A function's parameters would have to match the (visible) parameters of the delegate type. Any context pointer would be passed in a spot where the function doesn't look.
>
> The point is to allow using functions as delegates that don't use the context pointer.

Ah ok, you're proposing that the language guarantees that functions with the same "visible" parameters as delegates use the same ABI for those parameters.  So in this example, foo and bar would use the same ABI to pass in x:

void foo(int x)
{
}
struct Bar
{
    void bar(int x)
    {
    }
}

The drawback I can see is that it restricts the ABI that functions can use.  Say that our x86 ABI passes all context pointers using the EAX register (no matter what the other parameters are).  That would imply that function's couldn't use the EAX register to accept arguments.  Note that this would apply to ALL functions, not just the ones that are going to be called through delegates, which is a small subset of ALL functions.  An interesting idea but IMO it seems like an odd restriction to put on all functions.

I do understand wanting a solution to this use case though (passing a function pointer to a delegate).  That is why I created my DIP after all.  However, one could argue that even if the context pointer is ignored, it might be worth it to explicitly put it in your function which makes it clear that you want your function to be ABI compatible with a delegate:


void foo(void* this, int x)
{
    // the function doesn't really need the 'this' parameter.  That's
    // kind of passing a function to a delegate anyway.
}
struct Bar
{
    void bar(int x)
    {
    }
}

The only drawback here is that you have a 'this' parameter that you should ignore, but I would actually prefer the more verbose syntax of adding a "void* this" parameter than forcing all functions to not use whatever mechanism the equivalent delegate would use to pass in context pointer.

May 15, 2017
On 05/15/2017 09:34 PM, Jonathan Marler wrote:
> Ah ok, you're proposing that the language guarantees that functions with
> the same "visible" parameters as delegates use the same ABI for those
> parameters.

Yup.

[...]
> The drawback I can see is that it restricts the ABI that functions can
> use.  Say that our x86 ABI passes all context pointers using the EAX
> register (no matter what the other parameters are).  That would imply
> that function's couldn't use the EAX register to accept arguments.  Note
> that this would apply to ALL functions, not just the ones that are going
> to be called through delegates, which is a small subset of ALL
> functions.  An interesting idea but IMO it seems like an odd restriction
> to put on all functions.

What's "our ABI" here? The one we come up with, or one that we have to follow for some reason?

If we have to follow an ABI where all context pointers or `this`s are passed in a specific register, then that's a problem, yes. As far as the spec goes, it says that D follows the C conventions of the system. As C doesn't have method/delegate functionality, I think we're free to put our hidden parameters wherever we want.

`extern(C++)` and such is different story. I'd just not allow implicit conversion for those functions.

If "our ABI" is the one we come up with, then using a fixed register for the context pointer is not what I'm going for. My approach is to assign the visible arguments first, then pass the context pointer in whatever spot is left.

Say, the function ABI uses EAX, EBX, and ECX for the first three arguments (in that order). For a function call `f(1, 2)` that means:

    EAX: 1
    EBX: 2
    ECX: not used

For a delegate call `dg(1, 2)` I'd also put 1 and 2 into EAX and EBX. Additionally, the context pointer would be passed in ECX.

Calls to normal functions are supposed to stay as they are. Only method/delegate calls should be affected.
May 15, 2017
On Monday, 15 May 2017 at 20:14:49 UTC, ag0aep6g wrote:
> Say, the function ABI uses EAX, EBX, and ECX for the first three arguments (in that order). For a function call `f(1, 2)` that means:
>
>     EAX: 1
>     EBX: 2
>     ECX: not used
>
> For a delegate call `dg(1, 2)` I'd also put 1 and 2 into EAX and EBX. Additionally, the context pointer would be passed in ECX.
>
> Calls to normal functions are supposed to stay as they are. Only method/delegate calls should be affected.

If you just want to append an extra context arg by passing it as last actual arg, it'll end up in the stack sooner or later, and that, I guess, is where bad things may happen by just pushing an additional arg, not matching the function signature.
May 15, 2017
On 05/15/2017 10:34 PM, kinke wrote:
> If you just want to append an extra context arg by passing it as last
> actual arg, it'll end up in the stack sooner or later, and that, I
> guess, is where bad things may happen by just pushing an additional arg,
> not matching the function signature.

I'm certainly reaching the boundaries of my limited knowledge here, but wouldn't it work when we push the context arg first, before any other arguments, and pop it in the caller?
« First   ‹ Prev
1 2