Thread overview
[Issue 23722] Lambdas are mangled incorrectly when using multiple compilation units, resulting in incorrect code
Feb 27, 2023
Iain Buclaw
May 05, 2023
kinke
Jun 23, 2023
Dlang Bot
Jun 23, 2023
Dennis
1 day ago
kinke
1 day ago
kinke
February 27, 2023
https://issues.dlang.org/show_bug.cgi?id=23722

Iain Buclaw <ibuclaw@gdcproject.org> changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
           Priority|P1                          |P3

--
May 05, 2023
https://issues.dlang.org/show_bug.cgi?id=23722

kinke <kinke@gmx.net> changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
           Priority|P3                          |P1
                 CC|                            |kinke@gmx.net

--- Comment #1 from kinke <kinke@gmx.net> ---
Bumping priority to P1, this is a horrible bug. I'm not sure whether the lambda mangles can be really made unique & stable even with separate compilation, but we could probably enforce a C-style `static` object-file visibility for these function literals, as they are codegen'd into each referencing object file (well, at least with LDC).

--
June 23, 2023
https://issues.dlang.org/show_bug.cgi?id=23722

Dlang Bot <dlang-bot@dlang.rocks> changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
           Keywords|                            |pull

--- Comment #2 from Dlang Bot <dlang-bot@dlang.rocks> ---
@dkorpel created dlang/dmd pull request #15343 "Fix 23722 - Lambdas are mangled incorrectly when using multiple compiā€¦" fixing this issue:

- Fix 23722 - Lambdas are mangled incorrectly when using multiple compilation units, resulting in incorrect code

https://github.com/dlang/dmd/pull/15343

--
June 23, 2023
https://issues.dlang.org/show_bug.cgi?id=23722

--- Comment #3 from Dennis <dkorpel@live.nl> ---
> as they are codegen'd into each referencing object file (well, at least with LDC)

Looks like dmd codegen's it in neither object file, I get an undefined reference error when changing `generateId` to `generateIdWithLoc` for `FuncExp`.

--
1 day ago
https://issues.dlang.org/show_bug.cgi?id=23722

ilya.yanok@gmail.com changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
                 CC|                            |ilya.yanok@gmail.com

--- Comment #4 from ilya.yanok@gmail.com ---
The provided example is a bit confusing since it triggers two independent bugs, I believe :)

First bug is the one that's fixed by @dkorpel's PR: it's not about mangling actually, but rather about assigning names to lambdas and friends. The code simply uses the respective symtab size as a suffix, but in different compilation units we may check things in different order, resulting in different suffixes.

I have another example of this:

$ cat mod.d
import lib;
bool f() {
    return b;
}

$ cat lib.d
auto f(string s, alias g)() {
    return true;
}

alias a = f!("a", output => output);
alias b = f!("b", output => true);

$ nm lib.o | ddemangle
0000000000000010 s EH_frame0
0000000000000000 S lib.__ModuleInfo
0000000000000078 S pure nothrow @nogc @safe bool lib.f!("a", lib.__lambda5).f()
0000000000000088 S pure nothrow @nogc @safe bool lib.f!("b", lib.__lambda6).f()
0000000000000028 S _lib.f!("a", (output) => output).f.eh
0000000000000050 S _lib.f!("b", (output) => true).f.eh

$ nm mod.o | ddemangle
0000000000000010 s EH_frame0
                 U pure nothrow @nogc @safe bool lib.f!("b", lib.__lambda5).f()
0000000000000000 S mod.__ModuleInfo
0000000000000050 S bool mod.f()
0000000000000028 S _mod.f.eh

Note that a wrong lambda is referenced. @dkorpel's PR does indeed fix this and I believe it should be merged ASAP.

On top of that, the example from this bug hits another issue: the code for `A.y` is not generated at all, if the files are compiled separately.

I don't quite understand the latter issue yet. Are aliases supposed to be lazy? Meaning the code is only generated once an alias is used? There is a problem then:   if an alias definition needs to generate some code, at the use site we generate a reference to a symbol that might be not generated at all.

I think we could either codegen aliases/enums eagerly, or codegen using weak symbols at use sites, like we do for templates.

--
1 day ago
https://issues.dlang.org/show_bug.cgi?id=23722

--- Comment #5 from ilya.yanok@gmail.com ---
(In reply to kinke from comment #1)
> visibility for these function literals, as they are codegen'd into each referencing object file (well, at least with LDC).

Interesting, then the second issue doesn't exist with LDC and we just need the PR to get merged.

--
1 day ago
https://issues.dlang.org/show_bug.cgi?id=23722

--- Comment #6 from kinke <kinke@gmx.net> ---
(In reply to ilya.yanok from comment #5)
> (In reply to kinke from comment #1)
> > visibility for these function literals, as they are codegen'd into each referencing object file (well, at least with LDC).
> 
> Interesting, then the second issue doesn't exist with LDC and we just need the PR to get merged.

Yes, I fixed the first issue (different lambdas getting the same mangle and incorrectly being folded at link-time) in LDC v1.33: https://github.com/ldc-developers/ldc/pull/4415

--
1 day ago
https://issues.dlang.org/show_bug.cgi?id=23722

--- Comment #7 from ilya.yanok@gmail.com ---
(In reply to kinke from comment #6)
> Yes, I fixed the first issue (different lambdas getting the same mangle and incorrectly being folded at link-time) in LDC v1.33: https://github.com/ldc-developers/ldc/pull/4415

Cool! But that won't help with lambda names leaking into template instantiations' symbol via arguments unfortunately. So I think we really need lambda names to be stable (which the PR does).

I was actually curious about the second issue: LDC seems to generate the code for

alias f = () { ... }

for all modules that use `f`, while dmd only generates it if a module that uses `f` is compiled together with the module that defines it (or if that's the same module). Do you happen to know where does this difference come from?

--
1 day ago
https://issues.dlang.org/show_bug.cgi?id=23722

--- Comment #8 from kinke <kinke@gmx.net> ---
(In reply to ilya.yanok from comment #7)
> (In reply to kinke from comment #6)
> > Yes, I fixed the first issue (different lambdas getting the same mangle and incorrectly being folded at link-time) in LDC v1.33: https://github.com/ldc-developers/ldc/pull/4415
> 
> Cool! But that won't help with lambda names leaking into template instantiations' symbol via arguments unfortunately. So I think we really need lambda names to be stable (which the PR does).

Oh right, that's a remaining problem indeed.

> I was actually curious about the second issue: LDC seems to generate the code for
> 
> alias f = () { ... }
> 
> for all modules that use `f`, while dmd only generates it if a module that uses `f` is compiled together with the module that defines it (or if that's the same module). Do you happen to know where does this difference come from?

Different code emission strategies in the glue layer, here for lambdas. There are also significant emission strategy differences for non-class TypeInfos across DMD and LDC. I prefer defining these symbols on a lazy basis in each referencing object file, for robustness and inline-ability (without needing LTO).

--