http://d.puremagic.com/issues/show_bug.cgi?id=3717

           Summary: Stack corruption when returning a struct from a C
                    function in a MinGW generated shared library
           Product: D
           Version: 1.053
          Platform: Other
        OS/Version: Windows
            Status: NEW
          Severity: normal
          Priority: P2
         Component: DMD
        AssignedTo: nobody@puremagic.com
        ReportedBy: siegelords_abode@yahoo.com


--- Comment #0 from siegelords_abode@yahoo.com 2010-01-17 16:57:15 PST ---
When generating a shared C library using MinGW 4.4.0 on Windows and then generating an import library using implib, the resultant binary seems to have an improper default calling convention. This is evident when the C functions return structures. For example, given two files, test.d and test.c with the contents as follows:

//test.d
module test.d;

import tango.io.Stdout;

extern (C)
{
    struct COLOR
    {
        float r,g,b,a;
    }
    COLOR make_color(float r, float g, float b, float a);
}

void main()
{
    float[5] arr1 = 0;
    for(int ii = 0; ii < 10; ii++)
    {
        make_color(1, 1, 1, 1);
        Stdout.formatln("{},", arr1);
    }
}

//test.c
typedef struct COLOR
{
    float r,g,b,a;
} COLOR;

COLOR make_color(float r, float g, float b, float a)
{
    COLOR col;
    col.r = r;
    col.g = g;
    col.b = b;
    col.a = a;
    return col;
}

The command line instructions to reproduce this bug are then as follows:

gcc -shared -o test.dll test.c
implib /s test.lib test.dll
dmd test.d test.lib -L+tango.lib
test.exe

Outputs:
[0.00, 0.00, 0.00, 0.00, 0.00],
[0.00, 0.00, 0.00, 0.00, 0.00],
[0.00, 0.00, 0.00, 0.00, 0.00],
[1.00, 0.00, 0.00, 0.00, 0.00],
[1.00, 1.00, 0.00, 0.00, 0.00],
[1.00, 1.00, 1.00, 0.00, 0.00],
[1.00, 1.00, 1.00, 1.00, 0.00],
[0.17e-38, 1.00, 1.00, 1.00, 1.00],
[0.70e-44, 0.17e-38, 1.00, 1.00, 1.00]

As can be seen, the arr1 array gets slowly stomped over. This does not happen if I compile the C file with dmc, nor does it happen if I specify the linkage for the make_color function to be Windows (for both the C file in the D file). The issue here is with the default linkage type which seems to be incompatible in this case.

-- 
Configure issuemail: http://d.puremagic.com/issues/userprefs.cgi?tab=email
------- You are receiving this mail because: -------

http://d.puremagic.com/issues/show_bug.cgi?id=3717


Walter Bright <bugzilla@digitalmars.com> changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
             Status|NEW                         |RESOLVED
                 CC|                            |bugzilla@digitalmars.com
         Resolution|                            |WONTFIX


--- Comment #1 from Walter Bright <bugzilla@digitalmars.com> 2010-01-18 21:20:15 PST ---
Dmd for Windows was designed to work with the Digital Mars C compiler (dmc), not any other. dmc follows the conventional C abi for Windows (also followed by Microsoft C and Borland C). Apparently, MinGW does not.

I suggest using the Windows calling convention when working with MinGW.

-- 
Configure issuemail: http://d.puremagic.com/issues/userprefs.cgi?tab=email
------- You are receiving this mail because: -------