http://d.puremagic.com/issues/show_bug.cgi?id=3274

           Summary: dmd fails to emit code for templates into object file
                    if several files are compiled at once
           Product: D
           Version: 1.046
          Platform: Other
        OS/Version: Linux
            Status: NEW
          Severity: major
          Priority: P2
         Component: DMD
        AssignedTo: nobody@puremagic.com
        ReportedBy: nfxjfg@gmail.com


If you compile several files at once, and the files instantiate the same template, dmd outputs the code for that template only into one object file.

This breaks incremental compilation with tools like rebuild/dsss.

ldc doesn't suffer from this bug, and outputs the template code into all corresponding object files.

Reproduction of that bug:

First put the files listed at the end of the bug report into a.d, b.d and c.d.
Then execute these steps (order of files is important):
1. dmd a.d b.d c.d -c -version=XX
2. dmd a.d -c
   (without version XX -> like a change to the source code; because b.d and c.d
don't depend from a.d, you shouldn't need to recompile it.)
3. dmd a.o b.o c.o

Step 3 gives the error:
b.o: In function `_D1b1xFZv':
b.d:(.text._D1b1xFZv+0x4): undefined reference to
`_D1c13__T6from_cTiZ6from_cFZv'
collect2: ld returned 1 exit status
--- errorlevel 1

If Step 2. is left out, the error doesn't happen. If you compile all files separately in Step 1, the error doesn't happen.

-- file a.d:
module a;

import b;
import c;

void main() {
    version (XX) {
        from_c!(int)();
    }
}

-- file b.d:
module b;

import c;

void x() {
    from_c!(int)();
}

-- file c.d:
module c;

void from_c(T)() {
}

-- 
Configure issuemail: http://d.puremagic.com/issues/userprefs.cgi?tab=email
------- You are receiving this mail because: -------

http://d.puremagic.com/issues/show_bug.cgi?id=3274


nfxjfg@gmail.com changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
             Status|NEW                         |RESOLVED
         Resolution|                            |WONTFIX


-- 
Configure issuemail: http://d.puremagic.com/issues/userprefs.cgi?tab=email
------- You are receiving this mail because: -------

http://d.puremagic.com/issues/show_bug.cgi?id=3274


Brad Roberts <braddr@puremagic.com> changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
             Status|RESOLVED                    |REOPENED
                 CC|                            |braddr@puremagic.com
         Resolution|WONTFIX                     |


-- 
Configure issuemail: http://d.puremagic.com/issues/userprefs.cgi?tab=email
------- You are receiving this mail because: -------

http://d.puremagic.com/issues/show_bug.cgi?id=3274


Walter Bright <bugzilla@digitalmars.com> changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
             Status|REOPENED                    |RESOLVED
                 CC|                            |bugzilla@digitalmars.com
         Resolution|                            |WONTFIX


--- Comment #1 from Walter Bright <bugzilla@digitalmars.com> 2011-04-04 23:17:05 PDT ---
The compiler does this as a compile performance optimization. Otherwise, template instantiations have to be done over and over, filling the object files, and then the linker has to remove the duplicates.

The workaround for an incremental build system is straightforward - put only one source module on a command to dmd at a time.

-- 
Configure issuemail: http://d.puremagic.com/issues/userprefs.cgi?tab=email
------- You are receiving this mail because: -------

http://d.puremagic.com/issues/show_bug.cgi?id=3274


jfanatiker@gmx.at changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
                 CC|                            |jfanatiker@gmx.at


--- Comment #2 from jfanatiker@gmx.at 2013-04-05 04:17:13 PDT ---
It seems to work correctly with DMD v2.062 at least on Fedora 64 bit.

Can anyone confirm this?

-- 
Configure issuemail: http://d.puremagic.com/issues/userprefs.cgi?tab=email
------- You are receiving this mail because: -------

http://d.puremagic.com/issues/show_bug.cgi?id=3274


Martin Nowak <code@dawg.eu> changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
                 CC|                            |code@dawg.eu


--- Comment #3 from Martin Nowak <code@dawg.eu> 2013-04-07 09:04:10 PDT ---
(In reply to comment #1)
> The compiler does this as a compile performance optimization. Otherwise, template instantiations have to be done over and over, filling the object files, and then the linker has to remove the duplicates.
> 
It remains an incorrect optimization that just works most of the time.

> The workaround for an incremental build system is straightforward - put only one source module on a command to dmd at a time.

The performance impact of this is also huge. The only reason to do this in the first place is to avoid duplicate parsing and semantic. However we can't make any assumptions how the generated object files are linked therefor each of them needs to get the required TypeInfo, templates et.al.

To put this into perspective, copying is only required when building multiple object files. It is not required when building an executable, a shared library or a single object (which may contain multiple modules, i.e. -c -of).

Maybe we can move the copying more to the backend so that we don't have to rerun IRgen and codegen.

-- 
Configure issuemail: http://d.puremagic.com/issues/userprefs.cgi?tab=email
------- You are receiving this mail because: -------

http://d.puremagic.com/issues/show_bug.cgi?id=3274



--- Comment #4 from Martin Nowak <code@dawg.eu> 2013-04-07 12:02:53 PDT ---
One idea to reduce the cost of copying is partial linking.
AFAIU that would allow us to generate a single temporary object containing the
template/TypeInfo instantiation and link it into every object file that
references it.

ld -r -o combined1.o template.o module1.o
mv combined1.o module1.o
ld -r -o combined2.o template.o module2.o
mv combined2.o module2.o

http://sourceware.org/binutils/docs-2.23.1/ld/Options.html#index-partial-link-88

Though I'm not sure how portable this or an equivalent dmd backend functionality would be.

-- 
Configure issuemail: http://d.puremagic.com/issues/userprefs.cgi?tab=email
------- You are receiving this mail because: -------

http://d.puremagic.com/issues/show_bug.cgi?id=3274


Martin Nowak <code@dawg.eu> changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
                 CC|                            |code@klickverbot.at


--- Comment #5 from Martin Nowak <code@dawg.eu> 2013-04-07 12:04:53 PDT ---
*** Issue 8769 has been marked as a duplicate of this issue. ***

-- 
Configure issuemail: http://d.puremagic.com/issues/userprefs.cgi?tab=email
------- You are receiving this mail because: -------