https://issues.dlang.org/show_bug.cgi?id=15831

--- Comment #6 from ZombineDev <petar.p.kirov@gmail.com> ---
For anyone who want to try a new scheme for reducing the symbol size: have a look at the test case for issue 16039 where a simple change makes the binary size explode from 18MB to over 400MB.

https://issues.dlang.org/show_bug.cgi?id=16039

--

https://issues.dlang.org/show_bug.cgi?id=15831

--- Comment #7 from Rainer Schuetze <r.sagitario@gmx.de> ---
https://github.com/dlang/dmd/pull/5855 yields these symbols:

_D13testexpansion__T1sTiZ1@FiZ6Result3fooMFNaNfZv _D13testexpansion__T1sTS0@__T1@TiZ1@FiZ6ResultZ1@F%0Z2@3fooMFNaNfZv _D13testexpansion__T1sTS0@__T1@TS0@__T1@TiZ1@FiZ6ResultZ1@F%1Z2@Z1@F%0Z2@3fooMFNaNfZv _D13testexpansion__T1sTS0@__T1@TS0@__T1@TS0@__T1@TiZ1@FiZ6ResultZ1@F%2Z2@Z1@F%1Z2@Z1@F%0Z2@3fooMFNaNfZv _D13testexpansion__T1sTS0@__T1@TS0@__T1@TS0@__T1@TS0@__T1@TiZ1@FiZ6ResultZ1@F%3Z 2@Z1@F%2Z2@Z1@F%1Z2@Z1@F%0Z2@3fooMFNaNfZv

This is a bit longer than the binary compression, but should be about the same when base64 encoding is applied to the latter. Please also note that the encoding has strictly linear size, while the binary compression is limited to a constant factor by maximum encodable match length and dictionary size.

Starting with a recursion depth of 16, compiling for win64 crashes the compiler
(issue 16229)
Starting with a recursion depth of 19, the current compiler also bails out with

testexpansion.d(6): Error: function testexpansion.s!(Result).s excessive length
66394404789887526 for symbol, possible recursive expansion?

Compilation of the example for win32 with recursion depth 18 takes 3.1 seconds with the current mangling, but is hardly measurable with the mangling in PR 5855.

--

https://issues.dlang.org/show_bug.cgi?id=15831

--- Comment #8 from Rainer Schuetze <r.sagitario@gmx.de> ---
> For anyone who want to try a new scheme for reducing the symbol size: have a look at the test case for issue 16039 where a simple change makes the binary size explode from 18MB to over 400MB.

With https://github.com/dlang/dmd/pull/5904 applied to avoid the crash when building for Win64, I get a 250 MB object file. With PR https://github.com/dlang/dmd/pull/5855, it shrinks to 645 kB.

--

https://issues.dlang.org/show_bug.cgi?id=15831

--- Comment #9 from ZombineDev <petar.p.kirov@gmail.com> ---
Thanks, Rainer! That's really great. What's left to be done, apart from PR 5904, before PR 5855 can be merged?

--

https://issues.dlang.org/show_bug.cgi?id=15831

--- Comment #10 from Rainer Schuetze <r.sagitario@gmx.de> ---
> What's left to be done, apart from PR 5904, before PR 5855 can be merged?

The PR fails because it seems there are some issues with building shared libraries. In addition, a number of compiler tests and unittests check the current mangling explicitely, so these have to be adjusted. I'm not sure how useful these tests are, maybe they should check the mangle-demangle pair instead.

Unfortunately, changing the mangling is a not-so-small breaking change that affects a number of functions in druntime and phobos (e.g. demangle, core.internal.traits.externDFunc and std.traits.mangledName) that don't have a feasable replacement yet (my best guess is to add some __trait).

It's also possible that people have taken advantage of the current mangling in their code.

In addition, we should verify a new mangling addresses reported ambiguities of the existing mangling.

--

https://issues.dlang.org/show_bug.cgi?id=15831

--- Comment #11 from Steven Schveighoffer <schveiguy@yahoo.com> ---
(In reply to Rainer Schuetze from comment #10)
> The PR fails because it seems there are some issues with building shared libraries.

This is something I can't really help, but maybe we can enlist some experts from the forums.

> In addition, a number of compiler tests and unittests check the
> current mangling explicitely, so these have to be adjusted. I'm not sure how
> useful these tests are, maybe they should check the mangle-demangle pair
> instead.

Since the mangling is part of the ABI, I think it's valid to check for them. Perhaps what we need is a versioned-out version that checks for your expected mangling, and then when we "flip the switch" we can switch to that version.

> Unfortunately, changing the mangling is a not-so-small breaking change that affects a number of functions in druntime and phobos (e.g. demangle, core.internal.traits.externDFunc and std.traits.mangledName) that don't have a feasable replacement yet (my best guess is to add some __trait).

Again, we should use versioned code. The D compiler can define whether the new mangling scheme is enabled or not.

> It's also possible that people have taken advantage of the current mangling in their code.

This is something we can't really help. However, I highly doubt that anyone is taking advantage of giant voldemort symbol names. Mangling is something I doubt we can deprecate. However, their code could employ the version scheme to at least avoid expectations that won't be met.

> In addition, we should verify a new mangling addresses reported ambiguities of the existing mangling.

Links?

(In reply to ZombineDev from comment #9)
> Thanks, Rainer! That's really great.

+100, this is awesome work!

--

https://issues.dlang.org/show_bug.cgi?id=15831

--- Comment #12 from Rainer Schuetze <r.sagitario@gmx.de> ---
(In reply to Steven Schveighoffer from comment #11)
> Since the mangling is part of the ABI, I think it's valid to check for them. Perhaps what we need is a versioned-out version that checks for your expected mangling, and then when we "flip the switch" we can switch to that version.

I started to just allow both versions in the tests, but that started to spread across a lot of files. Versioning on some condition should be ok, too.

> However, I highly doubt that anyone
> is taking advantage of giant voldemort symbol names. Mangling is something I
> doubt we can deprecate. However, their code could employ the version scheme
> to at least avoid expectations that won't be met.

The decision cannot be made just for user code, phobos and druntime must be compiled with the same mangling.

> > In addition, we should verify a new mangling addresses reported ambiguities of the existing mangling.
> 
> Links?

I skipped over a search for "mangling" and "mangl" in bugzilla:

Trouble with scoped variables/functions:

- Issue 4699 - Functions in peer scopes cannot have the same name
- Issue 14831 - Each function local symbols should have unique mangled name

Ambiguous name mangling:

- Issue 3043 - Template symbol arg cannot be demangled
- Issue 5957 - Ambiguous mangling of module and template in template symbol
args
- Issue 6525 - Numeric literals should always be mangled with a leading 'i'
- Issue 6526 - Disambiguate mangling of AA literals
- Issue 6527 - Ambiguous mangling of inout parameters
- Issue 11273 - inequable template alias parameters can be mangled identically
- Issue 14576 - [ddemangle] core.demangle unable to handle ambiguity in symbols

Bogus mangling/demangling:

- Issue 8789 - mangling of const member function
- Issue 14319 - core.demangle does not support member function attributes
- Issue 14563 - core.demangle: Does not demangle type modifers

- Issue 10393 - demangle doesn't work for unicode symbol names
- Issue 14410 - core.demangle doesn't handle typeof(null) correctly

- Issue 11131 - variables without linkage shouldn't have a mangling (.mangleof)
- Issue 11248 - template value parameter cause too long mangling
- Issue 14894 - mangling of mixins and lambdas can change w/ -unittest or other
versions

Compiler generated special symbols don't follow the ABI:

- Issue 5644 - The mangling of constructors is not documented by the ABI
- Issue 6045 - Unable to demangle symbols
- Issue 11586 - core.demangle should understand _D16TypeInfo_HAyayAa6__initZ
etc.
- Issue 10189 - demangle doesn't work with __ModuleInfoZ __initZ __arrayZ

Different number of underscores for _D:

- Issue 8207 - OS X: Should extern(D) symbols include another underscore? - Issue 15748 - inconsistent symbols generated by dmd vs ldc

--

https://issues.dlang.org/show_bug.cgi?id=15831

Martin Nowak <code@dawg.eu> changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
                 CC|                            |code@dawg.eu

--- Comment #13 from Martin Nowak <code@dawg.eu> ---
It's an important issue and should be addressed. Let's work on that for 2.073 which is scheduled for the end of 2016.

Because this will impact many tools in D's ecosystem, we need to proper plan how to mitigate the impact.

A few points.

- Need to talk to Iain to update gdb's D demangler.

  This will take ages to upstream and end up in distributions.
  Maybe we can should the backref mangling to only very long names in the
beginning? Those are unreadable already and not showing them would rather fix a
few GUIs.

- Should we also try to address other mangling issues?

  Some things like the ambiguity are annoying but not that problematic.
  Also mixing in other changes will only increase the impact and make it harder
to release.
  FWIW this is also not a "new" mangling but an abbreviation of the existing
one.

- We need to fix all our toolings and libraries.

  There are some functions in druntime&phobos that build mangled names (e.g.
for loading shared library symbols).
  At best we can just implement the backref scheme in core.demangle.mangle and
reuse the in phobos.
  We also ship the ddemangle tool and there are other debuggers (at least the
maintained ones need updates).


- OT: I'd like to see a similar abbreviation scheme for core.demangle.demangle,
b/c the full template names in backtraces aren't readable either.
  Would be nicer to have something along the line of `func!(Symbol!(Foo, @2))`.

--

https://issues.dlang.org/show_bug.cgi?id=15831

--- Comment #14 from anonymous4 <dfj1esp02@sneakemail.com> ---
For stack trace it should be enough (for this example) mymodule.s.Result.foo without parameters even, file and line number should be enough to locate it.

--

https://issues.dlang.org/show_bug.cgi?id=15831

--- Comment #15 from Martin Nowak <code@dawg.eu> ---
(In reply to anonymous4 from comment #14)
> For stack trace it should be enough (for this example) mymodule.s.Result.foo without parameters even, file and line number should be enough to locate it.

Let's continue that part of the discussion in issue 14885.
We really just need someone to spec a feasible implementation.

--