http://d.puremagic.com/issues/show_bug.cgi?id=3372

           Summary: optlink silently mistreats object files with more than
                    16384 symbols
           Product: D
           Version: 2.033
          Platform: Other
        OS/Version: Windows
            Status: NEW
          Severity: major
          Priority: P2
         Component: Optlink
        AssignedTo: nobody@puremagic.com
        ReportedBy: r.sagitario@gmx.de


--- Comment #0 from Rainer Schuetze <r.sagitario@gmx.de> 2009-10-07 23:50:29 PDT ---
Larger projects (like qtd) are suffering from problems with large object files with a lot of symbols, where the linker creates bad fixups, creating executables that are calling wrong functions.

Here's a test case that creates a lot of symbols

//////////////////////////////////////

import std.stdio;

class genfuns(int n, int m)
{
    static void run() {
        genfuns!(n - 1, m).run();
    }
}

class genfuns(int n : 0, int m)
{
    static void run() { writefln("=== 0, %d ===", m); }
}

void main()
{
    // dmd-release: stack overflow with n=700
    // dmd-debug:   stack overflow with n=500
    genfuns!(300, 0).run();
    genfuns!(300, 1).run();
    genfuns!(300, 2).run();
    genfuns!(300, 3).run();
    genfuns!(300, 4).run();
    genfuns!(300, 5).run();
    genfuns!(300, 6).run();
    genfuns!(300, 7).run();
    genfuns!(300, 8).run();
    genfuns!(300, 9).run();
    genfuns!(300, 10).run();
    genfuns!(300, 11).run();
}
////////////////////////////////////////////

The output looks like this:

=== 0, 0 ===
=== 0, 1 ===
=== 0, 2 ===
=== 0, 3 ===
=== 0, 4 ===
=== 0, 5 ===
=== 0, 6 ===
=== 0, 7 ===
=== 0, 8 ===
=== 0, 9 ===
=== 0, 0 ===
=== 0, 0 ===

That shows, that the two last functions are calling wrong functions. The symbols don't even show up in the map file, but are still in the debug info. There you can see, that the adress is plain wrong, and searching the symbol in the object file dump, it seems that symbol #16384 + n gets the address of symbol #n.

-- 
Configure issuemail: http://d.puremagic.com/issues/userprefs.cgi?tab=email
------- You are receiving this mail because: -------

http://d.puremagic.com/issues/show_bug.cgi?id=3372


Walter Bright <bugzilla@digitalmars.com> changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
             Status|NEW                         |RESOLVED
                 CC|                            |bugzilla@digitalmars.com
         Resolution|                            |WORKSFORME


--- Comment #1 from Walter Bright <bugzilla@digitalmars.com> 2010-09-01 18:57:27 PDT ---
I had to remove the compiler's recursion detection circuits to get this to compile, but I can't get it to fail after linking with several versions of optlink. In any case, the 8.00.6 linker that solves 4009 may resolve this one, too, they sound similar.

-- 
Configure issuemail: http://d.puremagic.com/issues/userprefs.cgi?tab=email
------- You are receiving this mail because: -------

http://d.puremagic.com/issues/show_bug.cgi?id=3372


Rainer Schuetze <r.sagitario@gmx.de> changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
             Status|RESOLVED                    |REOPENED
         Resolution|WORKSFORME                  |


--- Comment #2 from Rainer Schuetze <r.sagitario@gmx.de> 2010-09-01 23:45:20 PDT ---
Sorry, but the error is still there with optlink 8.00.7 and dmd svn r638.

Here's an updated test case that does not need a compiler change:


import std.stdio;

class genfuns(int n, int m)
{
    static void run() {
        genfuns!(n - 1, m).run();
    }
}

class genfuns(int n : 0, int m)
{
    static void run() { writefln("=== 0, %d ===", m); }
}

version = crash;

void main()
{
    // dmd-release: stack overflow with n=700
    // dmd-debug:   stack overflow with n=500
    // dmd-2.037: error "recursive expansion" for n=300
    genfuns!(200, 0).run();
    genfuns!(200, 1).run();
    genfuns!(200, 2).run();
    genfuns!(200, 3).run();
    genfuns!(200, 4).run();
    genfuns!(200, 5).run();
    genfuns!(200, 6).run();
    genfuns!(200, 7).run();
    genfuns!(200, 8).run();
    genfuns!(200, 9).run();
    genfuns!(200, 10).run();
    genfuns!(200, 11).run();
    genfuns!(200, 12).run();
    genfuns!(200, 13).run();
    genfuns!(200, 14).run();
    genfuns!(200, 15).run();
    genfuns!(200, 16).run();
    genfuns!(200, 17).run();
    genfuns!(200, 18).run();
    genfuns!(200, 19).run(); // 16224 symbols up to here
version(crash)
    genfuns!(200, 20).run(); // 17028 symbols with this line
}

The actual bad behaviour of the executable can vary a little due to changed symbols in libraries and resulting different relocations. For me it is currently a long loop repeating the expected output followed by access violations or failed memory allocations.

If you uncomment the version=crash statement, you'll get the expected result.

-- 
Configure issuemail: http://d.puremagic.com/issues/userprefs.cgi?tab=email
------- You are receiving this mail because: -------

http://d.puremagic.com/issues/show_bug.cgi?id=3372


Walter Bright <bugzilla@digitalmars.com> changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
             Status|REOPENED                    |RESOLVED
         Resolution|                            |FIXED


--- Comment #3 from Walter Bright <bugzilla@digitalmars.com> 2011-02-22 14:24:24 PST ---
Fixed optlink 8.00.10

The limit on the number of symbols per object file should now be 0x7FFF. This limit is hardcoded in the OMF object file format.

-- 
Configure issuemail: http://d.puremagic.com/issues/userprefs.cgi?tab=email
------- You are receiving this mail because: -------

http://d.puremagic.com/issues/show_bug.cgi?id=3372


Andrei Alexandrescu <andrei@metalanguage.com> changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
                 CC|                            |andrei@metalanguage.com


--- Comment #4 from Andrei Alexandrescu <andrei@metalanguage.com> 2011-02-22 14:38:22 PST ---
We should mark this day as a federal holiday :o).

-- 
Configure issuemail: http://d.puremagic.com/issues/userprefs.cgi?tab=email
------- You are receiving this mail because: -------

http://d.puremagic.com/issues/show_bug.cgi?id=3372


changlon <changlon@gmail.com> changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
                 CC|                            |changlon@gmail.com


--- Comment #5 from changlon <changlon@gmail.com> 2011-02-22 18:50:03 PST ---
I use latest dmd 1.067, optlink  8.00.10 .

latest tango and dwtx ,  re builad all lib.

in on console run. link.exe it tell me the version is 8.00.10 .

use xfbuild for the dwtx lib .

the optlink throw a error like:


dwtx is big,  the lib is 30M .

http://gool.googlecode.com/files/optlink_bug.jpg

-- 
Configure issuemail: http://d.puremagic.com/issues/userprefs.cgi?tab=email
------- You are receiving this mail because: -------

http://d.puremagic.com/issues/show_bug.cgi?id=3372



--- Comment #6 from Rainer Schuetze <r.sagitario@gmx.de> 2011-02-22 23:29:12 PST ---
I can confirm this bug is fixed.

Even if it's only the limit that has been pushed to 32768 without other changes, you won't be able to stuff an object file with more symbols into optlink because dmd 2.052 asserts with

Internal error: ..\ztc\cgobj.c 425

This could be a bit more friendly, though.

There might be projects that need more symbols still, so a work around is to not compile into a single object file, but to multiple object files. You currently have to do this by calling dmd twice, once for compilation and then for linking as the output filename will otherwise also be used for the single object file.

The problem described in comment 5 is probably not related to this bug, but the crasher bugs, e.g. 2436.

-- 
Configure issuemail: http://d.puremagic.com/issues/userprefs.cgi?tab=email
------- You are receiving this mail because: -------

http://d.puremagic.com/issues/show_bug.cgi?id=3372



--- Comment #7 from Walter Bright <bugzilla@digitalmars.com> 2011-02-23 02:42:31 PST ---
(In reply to comment #5)
> the optlink throw a error like:
> dwtx is big,  the lib is 30M .
> http://gool.googlecode.com/files/optlink_bug.jpg

This isn't 2436, which has been fixed.

If you can make available the files that are passed to the linker, I can see about fixing it. I don't need source files. Just the obj's and lib's passed to the linker.

-- 
Configure issuemail: http://d.puremagic.com/issues/userprefs.cgi?tab=email
------- You are receiving this mail because: -------

http://d.puremagic.com/issues/show_bug.cgi?id=3372



--- Comment #8 from changlon <changlon@gmail.com> 2011-02-23 04:24:35 PST ---
http://gool.googlecode.com/files/optlink_bug.rar

lib with obj will crash the optlink 8.0.0.10 .

-- 
Configure issuemail: http://d.puremagic.com/issues/userprefs.cgi?tab=email
------- You are receiving this mail because: -------

http://d.puremagic.com/issues/show_bug.cgi?id=3372



--- Comment #9 from Walter Bright <bugzilla@digitalmars.com> 2011-02-23 12:12:23 PST ---
(In reply to comment #8)
> http://gool.googlecode.com/files/optlink_bug.rar
> lib with obj will crash the optlink 8.0.0.10 .

Thanks, I can reproduce it, though there is no tango.lib included.

-- 
Configure issuemail: http://d.puremagic.com/issues/userprefs.cgi?tab=email
------- You are receiving this mail because: -------