Thread overview
Unexpected behavior when misusing inline assembly
Nov 12, 2013
Noah
Nov 12, 2013
Noah
Nov 12, 2013
Noah
November 12, 2013
When running the following code:

__gshared void* return_ptr;
__gshared void* injected_fn = &fn;

void main() {
    buffer(7);
    printf("End main\n");
}

void buffer() {
    test(x);
    printf("End buffer\n");
}

void test() {
    printf("This is a test!\n");
    inject();
    printf("End of the test!\n");
}

void fn() {
    printf("Hello, world!\n");
    asm {
        mov RAX, return_ptr;
        mov [RBP+8], RAX;
    }
}

void inject() {
    asm {
        naked;
        push RAX;
        mov RAX, [RBP+8];
        mov return_ptr, RAX;
        mov RAX, injected_fn;
        mov [RBP+8], RAX;
        pop RAX;
        ret;
    }
}

The program behaves as expected, that is, the program prints
This is a test!
End of the test!
Hello, world!
End buffer
End main

However, if I call test directly from main, it results in a segfault.
If I call inject directly from main, it works.
And, oddly, if I add any amount of inline assembler to main, and call test there, it works.

Could anyone explain to me what's going on?
November 12, 2013
Just realized I forgot to delete the 7 and the x from inside the
function calls.  Question is the same, though.

On Tuesday, 12 November 2013 at 18:22:46 UTC, Noah wrote:
> When running the following code:
>
> __gshared void* return_ptr;
> __gshared void* injected_fn = &fn;
>
> void main() {
>     buffer(7);
>     printf("End main\n");
> }
>
> void buffer() {
>     test(x);
>     printf("End buffer\n");
> }
>
> void test() {
>     printf("This is a test!\n");
>     inject();
>     printf("End of the test!\n");
> }
>
> void fn() {
>     printf("Hello, world!\n");
>     asm {
>         mov RAX, return_ptr;
>         mov [RBP+8], RAX;
>     }
> }
>
> void inject() {
>     asm {
>         naked;
>         push RAX;
>         mov RAX, [RBP+8];
>         mov return_ptr, RAX;
>         mov RAX, injected_fn;
>         mov [RBP+8], RAX;
>         pop RAX;
>         ret;
>     }
> }
>
> The program behaves as expected, that is, the program prints
> This is a test!
> End of the test!
> Hello, world!
> End buffer
> End main
>
> However, if I call test directly from main, it results in a segfault.
> If I call inject directly from main, it works.
> And, oddly, if I add any amount of inline assembler to main, and call test there, it works.
>
> Could anyone explain to me what's going on?
November 12, 2013
Of course after I post this I realize the answer.  The function only works when the surrounding function has at least one argument.  Probably has something to do with the way the compiler emits no-argument functions, I'll have to look at some disassembly.