String Switch Lowering (page 2)

On Sat, Jan 27, 2018 at 09:22:07PM +0000, timotheecour via Digitalmars-d wrote: [...] > ``` > 28 dscanner 0x000000010d59f428 @safe void > std.getopt.getoptImpl!(std.getopt.config, immutable(char)[], bool*, > immutable(char)[], bool*, immutable(char)[], bool*, immutable(char)[], > bool*, immutable(char)[], bool*, immutable(char)[], bool*, > immutable(char)[], bool*, immutable(char)[], bool*, immutable(char)[], > bool*, immutable(char)[], bool*, immutable(char)[], bool*, > immutable(char)[], bool*, immutable(char)[], bool*, immutable(char)[], > bool*, immutable(char)[], bool*, immutable(char)[], immutable(char)[]*, > immutable(char)[], immutable(char)[]*, immutable(char)[], bool*, > immutable(char)[], immutable(char)[][]*, immutable(char)[], bool*, > immutable(char)[], bool*, immutable(char)[], bool*, immutable(char)[], > bool*).getoptImpl(ref immutable(char)[][], ref std.getopt.configuration, ref > std.getopt.GetoptResult, ref std.getopt.GetOptException, > void[][immutable(char)[]], void[][immutable(char)[]], std.getopt.config, > immutable(char)[], bool*, immutable(char)[], bool*, immutable(char)[], > bool*, immutable(char)[], bool*, immutable(char)[], bool*, > immutable(char)[], bool*, immutable(char)[], bool*, immutable(char)[], > bool*, immutable(char)[], bool*, immutable(char)[], bool*, > immutable(char)[], bool*, immutable(char)[], bool*, immutable(char)[], > bool*, immutable(char)[], bool*, immutable(char)[], bool*, > immutable(char)[], immutable(char)[]*, immutable(char)[], > immutable(char)[]*, immutable(char)[], bool*, immutable(char)[], > immutable(char)[][]*, immutable(char)[], bool*, immutable(char)[], bool*, > immutable(char)[], bool*, immutable(char)[], bool*) + 460 > ``` > > https://dlang.org/blog/2017/12/20/ds-newfangled-name-mangling/ doesn't seem to help in cases like that This has nothing to do with name mangling. The mangling itself may be relatively small (and probably is, judging from the amount of repetition in the signature above), but what you're looking at is the *demangled* identifier. That's going to be big no matter what, unless we fundamentally change the way getopt() is implemented. I proposed a compile-time introspected getopt() replacement before, only to get laughed at by Andrei. So I guess that means, don't expect to see that in Phobos anytime soon. But I might post the code on github sometime for those who would benefit from it. Basically, instead of taking a ridiculously long argument list, you create a struct whose members (together with some UDAs) define what the options are, any associated help text, etc., and just call it with the struct type as argument. It does its thing, and returns the struct populated with the values retrieved from the command-line. There are a few more features, but that's the gist of it. T -- There is no gravity. The earth sucks.

but this should be handled at the compiler level, with no change in standard library getopt, eg using a hashing scheme (cf `ldc -hashthres`) On Sat, Jan 27, 2018 at 2:38 PM, Kagamin via Digitalmars-d <digitalmars-d@puremagic.com> wrote: > IIRC several years ago somebody created a dub package with DbI getopt. I think it wouldn't suffer from this issue.

January 27, 2018

Re: String Switch Lowering

Posted by Timothee Cour

Permalink

Timothee Cour

Permalink

* This has nothing to do with name mangling.

Yes and no, these things are coupled. We can improve the situation by forcing the size of mangled and demangled symbols to be < threshold, eg `ldc -hash-threshold` would be 1 option.

example:

current mangled: _D8analysis3run__T9shouldRunVAyaa20_666c6f61745f6f70657261746f725f636865636bZQChFQCaKxSQDh6config20StaticAnalysisConfigZ__T9__lambda4TQEbZQpFNaNbNiNfQEqZQEu

current demangled:
pure nothrow @nogc @safe immutable(char)[]
analysis.run.shouldRun!("float_operator_check").shouldRun(immutable(char)[],
ref const(analysis.config.StaticAnalysisConfig)).__lambda4!(immutable(char)[]).__lambda4(immutable(char)[])

with a small threshold:

mangled:
_D8analysis3run__T9shouldRunℂ0abf2284dd3

demangled:
pure nothrow @nogc @safe immutable(char)[] analysis.run.shouldRun.ℂ0abf2284dd3

The `ℂ` symbol indicating hashing was applied because symbol size
exceed threshold.
The demangled version also would have that. A separate file (dmd
-mangle_map=file) could be produced in the rare case a user wants to
see the full 17KB mangled and demangled symbols mapped by ℂ0abf2284dd3


On Sat, Jan 27, 2018 at 3:12 PM, H. S. Teoh via Digitalmars-d <digitalmars-d@puremagic.com> wrote:
> On Sat, Jan 27, 2018 at 09:22:07PM +0000, timotheecour via Digitalmars-d wrote: [...]
>> ```
>> 28  dscanner                            0x000000010d59f428 @safe void
>> std.getopt.getoptImpl!(std.getopt.config, immutable(char)[], bool*,
>> immutable(char)[], bool*, immutable(char)[], bool*, immutable(char)[],
>> bool*, immutable(char)[], bool*, immutable(char)[], bool*,
>> immutable(char)[], bool*, immutable(char)[], bool*, immutable(char)[],
>> bool*, immutable(char)[], bool*, immutable(char)[], bool*,
>> immutable(char)[], bool*, immutable(char)[], bool*, immutable(char)[],
>> bool*, immutable(char)[], bool*, immutable(char)[], immutable(char)[]*,
>> immutable(char)[], immutable(char)[]*, immutable(char)[], bool*,
>> immutable(char)[], immutable(char)[][]*, immutable(char)[], bool*,
>> immutable(char)[], bool*, immutable(char)[], bool*, immutable(char)[],
>> bool*).getoptImpl(ref immutable(char)[][], ref std.getopt.configuration, ref
>> std.getopt.GetoptResult, ref std.getopt.GetOptException,
>> void[][immutable(char)[]], void[][immutable(char)[]], std.getopt.config,
>> immutable(char)[], bool*, immutable(char)[], bool*, immutable(char)[],
>> bool*, immutable(char)[], bool*, immutable(char)[], bool*,
>> immutable(char)[], bool*, immutable(char)[], bool*, immutable(char)[],
>> bool*, immutable(char)[], bool*, immutable(char)[], bool*,
>> immutable(char)[], bool*, immutable(char)[], bool*, immutable(char)[],
>> bool*, immutable(char)[], bool*, immutable(char)[], bool*,
>> immutable(char)[], immutable(char)[]*, immutable(char)[],
>> immutable(char)[]*, immutable(char)[], bool*, immutable(char)[],
>> immutable(char)[][]*, immutable(char)[], bool*, immutable(char)[], bool*,
>> immutable(char)[], bool*, immutable(char)[], bool*) + 460
>> ```
>>
>> https://dlang.org/blog/2017/12/20/ds-newfangled-name-mangling/ doesn't seem to help in cases like that
>
> This has nothing to do with name mangling. The mangling itself may be relatively small (and probably is, judging from the amount of repetition in the signature above), but what you're looking at is the *demangled* identifier. That's going to be big no matter what, unless we fundamentally change the way getopt() is implemented.
>
> I proposed a compile-time introspected getopt() replacement before, only to get laughed at by Andrei.  So I guess that means, don't expect to see that in Phobos anytime soon.  But I might post the code on github sometime for those who would benefit from it.  Basically, instead of taking a ridiculously long argument list, you create a struct whose members (together with some UDAs) define what the options are, any associated help text, etc., and just call it with the struct type as argument.  It does its thing, and returns the struct populated with the values retrieved from the command-line.  There are a few more features, but that's the gist of it.
>
>
> T
>
> --
> There is no gravity. The earth sucks.

On Sat, Jan 27, 2018 at 03:48:19PM -0800, Timothee Cour wrote: [...] > eg `ldc -hash-threshold` would be 1 option. [...] > with a small threshold: > > mangled: > _D8analysis3run__T9shouldRunℂ0abf2284dd3 > > demangled: > pure nothrow @nogc @safe immutable(char)[] analysis.run.shouldRun.ℂ0abf2284dd3 > > The `ℂ` symbol indicating hashing was applied because symbol size > exceed threshold. > The demangled version also would have that. A separate file (dmd > -mangle_map=file) could be produced in the rare case a user wants to > see the full 17KB mangled and demangled symbols mapped by ℂ0abf2284dd3 [...] This gives me an idea. A lot of the recent complaints about symbol size appears to be coming from templates with string template arguments, and encoding strings inside a symbol tend to quickly make its length explode. What if we changed the mangling scheme such that if a string template argument exceeds a certain length, or if the number of string arguments (or their accumulated length) exceeds a certain size, we hash the string arguments instead, and use the hash in the symbol instead of encoding the raw strings themselves? Regardless, *some* form of symbol compression is necessary in D, especially now that druntime is also moving in the direction of more templated code. Rainer's backref patch helped in one common use case (chained range functions), but the string argument case remains a source of major symbol bloat. I was also talking with Stefan Koch on github that we need to take up a project to improve the way dmd handles templates. There's much room for improvement, and with D's focus on heavy-duty compile-time features, this is a major area where we can reap large benefits for the effort invested. T -- Freedom of speech: the whole world has no right *not* to hear my spouting off!

On Saturday, 27 January 2018 at 23:12:01 UTC, H. S. Teoh wrote: > On Sat, Jan 27, 2018 at 09:22:07PM +0000, timotheecour via Digitalmars-d wrote: [...] >> ``` >> 28 dscanner 0x000000010d59f428 @safe void >> ... > > I proposed a compile-time introspected getopt() replacement before, only to get laughed at by Andrei. Do you remember his motivation? FYI: since Nov 2017 Druntime has exactly what you are proposing https://github.com/dlang/druntime/blob/master/src/core/internal/parseoptions.d However, it's not really sophisticated as it's intended only to be used for druntime, but then again, no one is stopping you from using it: ``` struct Config { int a; string b; void help() @nogc nothrow {} } void main() { import core.internal.parseoptions, std.stdio; string args = "a=42 b=foo"; Config conf; conf.parseOptions(args); conf.writeln; } ``` https://run.dlang.io/is/GVclu2

Forums