I've managed to get a few functions working before mostly by copying whatever Phobos was doing for a similar purpose, but now that I'm trying to do something different I am really hitting a wall.

My issue is that I can't figure out how to access a function's arguments from within inline asm or how to ensure that the correct value is returned. I haven't found a single piece of documentation about this so far!

I am mainly concerned about doing this for naked asm functions, but any documentation on the subject would be helpful.

On Thursday, 15 November 2018 at 20:57:59 UTC, pineapple wrote:
> My issue is that I can't figure out how to access a function's arguments from within inline asm or how to ensure that the correct value is returned. I haven't found a single piece of documentation about this so far!

It would be part of the abi:
https://dlang.org/spec/abi.html#function_calling_conventions

though it references C so you might need to look that up too.

On Thursday, 15 November 2018 at 21:00:10 UTC, Adam D. Ruppe wrote:
> It would be part of the abi:
> https://dlang.org/spec/abi.html#function_calling_conventions
>
> though it references C so you might need to look that up too.

That's helpful, thank you!

> For other sized structs and static arrays, the return value is stored through a hidden pointer passed as an argument to the function.

Is there a way to access this pointer?

Also, the calling convention documentation there doesn't mention anything about 64-bit targets, are 64-bit registers just not used?

On Thursday, 15 November 2018 at 21:07:51 UTC, pineapple wrote:
> Is there a way to access this pointer?

It is passed as.. I think the final argument to the function. (unless it is the first, do a quick test to find out).

> Also, the calling convention documentation there doesn't mention anything about 64-bit targets, are 64-bit registers just not used?

It uses the C rules there, so look up the 64 bit C abi. I think this is accurate https://msdn.microsoft.com/en-us/library/ms235286.aspx

On Thursday, 15 November 2018 at 21:12:39 UTC, Adam D. Ruppe wrote:
> On Thursday, 15 November 2018 at 21:07:51 UTC, pineapple wrote:
>> Is there a way to access this pointer?
>
> It is passed as.. I think the final argument to the function. (unless it is the first, do a quick test to find out).
>
>> Also, the calling convention documentation there doesn't mention anything about 64-bit targets, are 64-bit registers just not used?
>
> It uses the C rules there, so look up the 64 bit C abi. I think this is accurate https://msdn.microsoft.com/en-us/library/ms235286.aspx

On the positive side: I see now how to return a 64-bit value from a function in Windows x64! And I understand how the arguments are coming in. This is very helpful.

On the less positive side: I still have no clue how to return my 16-byte struct. The Microsoft x64 ABI documentation I've seen so far explains how return values of 8 bytes and fewer work, but haven't explained how larger return values work. The obvious answer of "RAX or EAX contains a pointer" is either not working or my asm is wrong. (The latter is certainly a possibility.)


    // Returns 128 (0x80)
    ulong retTest() {
        version(D_InlineAsm_X86_64) asm {
            naked;
            mov RAX, 0x80;
            ret;
        }
    }

    // Crashes
    struct Result {
        ulong low;
        ulong high;
    }
    Result retTest() {
        version(D_InlineAsm_X86_64) asm {
            naked;
            mov [RAX + 0], 0x80;
            mov [RAX + 8], 0x80;
            ret;
        }
    }

    // Also crashes
    struct Result {
        ulong low;
        ulong high;
    }
    Result retTest() {
        version(D_InlineAsm_X86_64) asm {
            naked;
            mov [R9 + 0], 0x80;
            mov [R9 + 8], 0x80;
            ret;
        }
    }

On Thursday, 15 November 2018 at 21:32:10 UTC, pineapple wrote:
> On the less positive side: I still have no clue how to return my 16-byte struct. The Microsoft x64 ABI documentation I've seen so far explains how return values of 8 bytes and fewer work, but haven't explained how larger return values work. The obvious answer of "RAX or EAX contains a pointer" is either not working or my asm is wrong. (The latter is certainly a possibility.)

The MS docs are complete IIRC. The pointer to the pre-allocated result of your 16-bytes struct is passed in RCX.
If unsure, just reverse-engineer what you need: type it down in normal D and analyze the generated assembly. You can even do so online via run.dlang.io (https://run.dlang.io/is/rhsDBF); just select LDC and add `-mtriple=x86_64-pc-windows-msvc` to generate Win64 assembly.
Note that run.dlang.io displays AT&T-style asm, not the Intel one. You can use LDC offline via `-output-s -x86-asm-syntax=intel` to generate a .s file with Intel syntax.

Ah, I've got something working!

It's not exactly what I wanted, but it's good enough for now. Instead of using an invisible output pointer, the output pointer is passed in explicitly.

    struct Result {
        ulong low;
        ulong high;
    }
    void retTest(Result* result) {
        version(D_InlineAsm_X86_64) asm {
            naked;
            mov [RAX + 0], 0x80;
            mov [RAX + 8], 0xff;
            ret;
        }
    }

On Thursday, 15 November 2018 at 21:48:46 UTC, kinke wrote:
> The MS docs are complete IIRC. The pointer to the pre-allocated result of your 16-bytes struct is passed in RCX.
> If unsure, just reverse-engineer what you need: type it down in normal D and analyze the generated assembly. You can even do so online via run.dlang.io (https://run.dlang.io/is/rhsDBF); just select LDC and add `-mtriple=x86_64-pc-windows-msvc` to generate Win64 assembly.
> Note that run.dlang.io displays AT&T-style asm, not the Intel one. You can use LDC offline via `-output-s -x86-asm-syntax=intel` to generate a .s file with Intel syntax.

When I tested by writing to the pointer given by RCX, the program didn't crash but I did get different values every time and never the ones I wanted.

Well, for anyone who is tangling with similar mysteries, I finally got something to work the way I wanted it to. Thank you for the help, Adam and kinke!

The first "x" argument was stored in R8. The second "y" argument was stored in RDX. The invisible return value pointer was stored in RCX.

Here's what I was hoping to accomplish. The multiply function returns the low bits of the product in the `low` attribute of the struct and the high bits in the `high` attribute of the struct. The divide function returns the quotient in `low` and the remainder in `high`.

    struct Result {
        ulong low;
        ulong high;
    }

    Result unsignedMultiply(ulong x, ulong y) {
        version(D_InlineAsm_X86_64) asm {
            naked;
            mov RAX, R8;
            mul RDX;
            mov qword ptr [RCX], RAX;
            mov qword ptr [RCX + 8], RDX;
            ret;
        }
    }

    Result unsignedDivide(ulong x, ulong y) {
        version(D_InlineAsm_X86_64) asm {
            naked;
            mov RAX, R8;
            mov R9, RDX;
            mov RDX, 0;
            div R9;
            mov qword ptr [RCX], RAX;
            mov qword ptr [RCX + 8], RDX;
            ret;
        }
    }