Thread overview | |||||||||||
---|---|---|---|---|---|---|---|---|---|---|---|
|
February 07, 2016 voldemort stack traces (and bloat) | ||||
---|---|---|---|---|
| ||||
I have a library where I was using very many voldemort types a la std.range. While debugging I had an exception triggered, but I found that the library *pauses* significantly while printing the exception. What I found is essentially that using voldemort types results in horrible stack traces. To demonstrate the problem: struct S(T) { void foo(){ throw new Exception("1");} } auto s(T)(T t) { struct Result { void foo(){ throw new Exception("2");} } return Result(); } void main(string[] args) { version(bad) auto x = 1.s.s.s.s.s; else S!(S!(S!(S!(S!(int))))) x; x.foo; } Building without bad version, and running, I get this as the stack frame for the foo call: 4 testexpansion 0x0000000103c3fc14 pure @safe void testexpansion.S!(testexpansion.S!(testexpansion.S!(testexpansion.S!(testexpansion.S!(int).S).S).S).S).S.foo() + 144 Now, if I compile with version=bad: 4 testexpansion 0x000000010fb5dbec pure @safe void testexpansion.s!(testexpansion.s!(testexpansion.s!(testexpansion.s!(testexpansion.s!(int).s(int).Result).s(testexpansion.s!(int).s(int).Result).Result).s(testexpansion.s!(testexpansion.s!(int).s(int).Result).s(testexpansion.s!(int).s(int).Result).Result).Result).s(testexpansion.s!(testexpansion.s!(testexpansion.s!(int).s(int).Result).s(testexpansion.s!(int).s(int).Result).Result).s(testexpansion.s!(testexpansion.s!(int).s(int).Result).s(testexpansion.s!(int).s(int).Result).Result).Result).Result).s(testexpansion.s!(testexpansion.s!(testexpansion.s!(testexpansion.s!(int).s(int).Result).s(testexpansion.s!(int).s(int).Result).Result).s(testexpansion.s!(testexpansion.s!(int).s(int).Result).s(testexpansion.s!(int).s(int).Result).Result).Result).s(testexpansion.s!(testexpansion.s!(testexpansion.s!(int).s(int).Result).s(testexpansion.s!(int).s(int).Result).Result).s(testexpansion.s!(testexpansion.s!(int).s(int).Result).s(testexpansion.s!(int).s(int).Result).Result).Result).Result).Result.foo () + 144 I believe what is happening is both the template parameter and the argument type are being printed, but both are the same! And each level of nesting results in another doubling of the printouts. So you have an exponential effect, and the resulting stack trace is horrendously useless. what's more, the template bloat factor skyrockets: dmd -c testexpansion.d ls -l testexpansion.o -rw-r--r--+ 1 steves staff 5664 Feb 7 00:06 testexpansion.o dmd -c -version=bad testexpansion.d ls -l testexpansion.o -rw-r--r--+ 1 steves staff 15312 Feb 7 00:07 testexpansion.o as a final test, I tried this: auto s(T)(T t) { return S!(T)(); } And the resulting .o file: -rw-r--r--+ 1 steves staff 7104 Feb 7 00:11 testexpansion.o With obviously the exception code printing in the less verbose form. So the cost in template bloat of using a voldemort type over a private type is 8k here, more than double the existing size. With more nesting, I'm sure that factor gets worse. Is there a better way we should be doing this? I'm wondering if voldemort types are really worth it. They offer a lot of convenience, and are much DRYer than separate private template types. But the bloat cost is not really worth the convenience IMO. Thoughts? -Steve |
February 07, 2016 Re: voldemort stack traces (and bloat) | ||||
---|---|---|---|---|
| ||||
Posted in reply to Steven Schveighoffer | On Sunday, 7 February 2016 at 05:18:39 UTC, Steven Schveighoffer wrote:
> Thoughts?
And no line number. But hey, these are convenience for youngsters. We real program, who type on the keyboard using our balls, don't need such distractions.
|
February 07, 2016 Re: voldemort stack traces (and bloat) | ||||
---|---|---|---|---|
| ||||
Posted in reply to Steven Schveighoffer | On Sunday, 7 February 2016 at 05:18:39 UTC, Steven Schveighoffer wrote: > 4 testexpansion 0x000000010fb5dbec pure @safe void testexpansion.s!(testexpansion.s!(testexpansion.s!(testexpansion.s!(testexpansion.s! Why "bad" foo is void? > Is there a better way we should be doing this? I'm wondering if Yeah would by nice to auto-repacle with testexpansion.S!(...)(...).Result.foo or even with ...Result.foo |
February 07, 2016 Re: voldemort stack traces (and bloat) | ||||
---|---|---|---|---|
| ||||
Posted in reply to deadalnix | On 2/7/16 5:20 AM, deadalnix wrote:
> On Sunday, 7 February 2016 at 05:18:39 UTC, Steven Schveighoffer wrote:
>> Thoughts?
>
> And no line number. But hey, these are convenience for youngsters. We
> real program, who type on the keyboard using our balls, don't need such
> distractions.
>
Remind me never to borrow your laptop.
But really, if you can't figure out what +144 is, *eyeroll*.
-Steve
|
February 07, 2016 Re: voldemort stack traces (and bloat) | ||||
---|---|---|---|---|
| ||||
Posted in reply to Iakh | On 2/7/16 10:42 AM, Iakh wrote: > On Sunday, 7 February 2016 at 05:18:39 UTC, Steven Schveighoffer wrote: >> 4 testexpansion 0x000000010fb5dbec pure @safe >> void >> testexpansion.s!(testexpansion.s!(testexpansion.s!(testexpansion.s!(testexpansion.s! >> > > Why "bad" foo is void? Huh? foo returns void in both instances. > Yeah would by nice to auto-repacle with > testexpansion.S!(...)(...).Result.foo > or even with ...Result.foo A possible fix for the stack printing is to use the template parameter placeholders: testexpansion.s!(T = testexpansion.s!...)(T).Result.foo But this doesn't fix the object-file bloat. -Steve |
February 08, 2016 Re: voldemort stack traces (and bloat) | ||||
---|---|---|---|---|
| ||||
Posted in reply to Steven Schveighoffer | On Monday, 8 February 2016 at 01:48:32 UTC, Steven Schveighoffer wrote:
> On 2/7/16 10:42 AM, Iakh wrote:
>> On Sunday, 7 February 2016 at 05:18:39 UTC, Steven Schveighoffer wrote:
>>> 4 testexpansion 0x000000010fb5dbec pure @safe
>>> void
>>> testexpansion.s!(testexpansion.s!(testexpansion.s!(testexpansion.s!(testexpansion.s!
>>>
>>
>> Why "bad" foo is void?
>
> Huh? foo returns void in both instances.
>
>> Yeah would by nice to auto-repacle with
>> testexpansion.S!(...)(...).Result.foo
>> or even with ...Result.foo
>
> A possible fix for the stack printing is to use the template parameter placeholders:
>
> testexpansion.s!(T = testexpansion.s!...)(T).Result.foo
>
> But this doesn't fix the object-file bloat.
>
> -Steve
I think it was Manu who was complaining about symbol length some time ago and we ended up discussing symbol compression as a possible solution. Did anything ever come of that? If so this seems like an obvious candidate for recursive compression.
Nic
|
February 08, 2016 Re: voldemort stack traces (and bloat) | ||||
---|---|---|---|---|
| ||||
Posted in reply to Steven Schveighoffer | On 2/7/16 12:18 AM, Steven Schveighoffer wrote: > I have a library where I was using very many voldemort types a la > std.range. > [snip] > Is there a better way we should be doing this? I'm wondering if > voldemort types are really worth it. They offer a lot of convenience, > and are much DRYer than separate private template types. But the bloat > cost is not really worth the convenience IMO. I modified all my voldemort-returning functions to return module-level types. One of my example programs (compiled optimized/inline) went from 10MB to 1MB. The other example program went from 2.1MB to 900k. -Steve |
February 08, 2016 Re: voldemort stack traces (and bloat) | ||||
---|---|---|---|---|
| ||||
Posted in reply to Steven Schveighoffer | On Monday, 8 February 2016 at 13:01:44 UTC, Steven Schveighoffer wrote:
> On 2/7/16 12:18 AM, Steven Schveighoffer wrote:
>> I have a library where I was using very many voldemort types a la
>> std.range.
>>
>
> [snip]
>
>> Is there a better way we should be doing this? I'm wondering if
>> voldemort types are really worth it. They offer a lot of convenience,
>> and are much DRYer than separate private template types. But the bloat
>> cost is not really worth the convenience IMO.
>
> I modified all my voldemort-returning functions to return module-level types.
>
> One of my example programs (compiled optimized/inline) went from 10MB to 1MB. The other example program went from 2.1MB to 900k.
>
> -Steve
Just to be sure, you replaced something like this:
auto myFunc(int x){
struct MyStruct{ int a; }
return MyStruct(x);
}
with?
private struct MyStruct{ int a; }
auto myFunc(int x){
return MyStruct(x);
}
|
February 08, 2016 Re: voldemort stack traces (and bloat) | ||||
---|---|---|---|---|
| ||||
Posted in reply to wobbles | On 2/8/16 8:19 AM, wobbles wrote:
> On Monday, 8 February 2016 at 13:01:44 UTC, Steven Schveighoffer wrote:
>> On 2/7/16 12:18 AM, Steven Schveighoffer wrote:
>>> I have a library where I was using very many voldemort types a la
>>> std.range.
>>>
>>
>> [snip]
>>
>>> Is there a better way we should be doing this? I'm wondering if
>>> voldemort types are really worth it. They offer a lot of convenience,
>>> and are much DRYer than separate private template types. But the bloat
>>> cost is not really worth the convenience IMO.
>>
>> I modified all my voldemort-returning functions to return module-level
>> types.
>>
>> One of my example programs (compiled optimized/inline) went from 10MB
>> to 1MB. The other example program went from 2.1MB to 900k.
>>
>
> Just to be sure, you replaced something like this:
>
> auto myFunc(int x){
> struct MyStruct{ int a; }
> return MyStruct(x);
> }
>
> with?
>
> private struct MyStruct{ int a; }
> auto myFunc(int x){
> return MyStruct(x);
> }
Yes, but with template parameters. It's not so much the moving of the struct that reduced the bloat, but the nature of how the template parameters are used. Each function that returns one of these structs wraps another such struct, so the template bloat is exponential because of the repeat of the template parameter for the function argument. By moving the struct into the module, there is only one specification of the template parameter for the name mangling. Basically, I changed a.b.c bloat factor from 2^3 to 1^3.
-Steve
|
Copyright © 1999-2021 by the D Language Foundation