November 18, 2014 Re: Would you trade 0.1% in performance for a better debugging experience? | ||||
---|---|---|---|---|
| ||||
Posted in reply to Vladimir Panteleev | On 11/18/14 12:03 PM, Vladimir Panteleev wrote: > On Tuesday, 18 November 2014 at 13:18:10 UTC, Steven Schveighoffer wrote: >> On 11/17/14 6:14 PM, Vladimir Panteleev wrote: >>> I proposed to build Phobos and Druntime with stack frames enabled: >> >> Sure, why not 3 versions of phobos/runtime in installation? Space is >> cheap. > > You have to consider the combinatorial explosion of our target > platforms, and also the impact on the size of the distribution. > > - In 2.066.0, phobos.lib is around 9MB. > - *2 for x86/x64. > - *4 for the number of supported platforms. > - *3 for the proposed number of build configurations. I refuse to let the fact that we package every platform's download into one zip to be some sort of argument :) DMD is the only distribution that I've ever seen do this. > That comes out to over 200 MB. Might be bigger due to debug builds of > Phobos taking more space. I have 0 problem making the user choose his configuration at download time. Each download should be 10MB, possibly x3 for debug and stack frame builds. -Steve |
November 18, 2014 Re: Would you trade 0.1% in performance for a better debugging experience? | ||||
---|---|---|---|---|
| ||||
Posted in reply to Vladimir Panteleev | Am Tue, 18 Nov 2014 17:15:12 +0000 schrieb "Vladimir Panteleev" <vladimir@thecybershadow.net>: > On Tuesday, 18 November 2014 at 16:49:55 UTC, Marco Leise wrote: > > Without fully understanding the issue, omitting the frame pointer on GNU amd64 systems is the default and is supposed to work using DWARF debug information. So there should be no need for a stack frame pointer, right? > > Firstly, does DMD generate DWARF debug information in practice that's correct enough to substitute stack frames? As far as I can tell, it doesn't. Is it fair if I argue that fixing DWARF info generation is a better solution then? > Second, there's still the argument that not every debugger and profiler can take advantage of the DWARF debug information. It's certainly nowhere as easy: from the technical point of view, but also from a legal one, considering that (IIRC) most libraries dealing with DWARF debug information are GPL or LGPL, meaning we can't use them in the D runtime. And indeed, for printing the stack trace for an unhandled exception, Druntime currently walks the stack frames: > > https://github.com/D-Programming-Language/druntime/blob/master/src/core/runtime.d#L452-L478 I don't need to open the link to know the code you linked to. Ever since I had weird seg faults when an exception was thrown from my D callback in Gtk code I wondered when this code will be updated. Consider that "no frame pointers" is the default for GCC on amd64 and thus practically all libraries on Linux can't be stack traced by druntime. It needs to use libunwind in the long run when C++ interfacing becomes more viable and D programmers start to expect exception handling and stack traces to work as per system ABI. At least that's what I believe. > There is also the libc backtrace function, which the D runtime apparently used at some point: > > http://www.gnu.org/software/libc/manual/html_node/Backtraces.html > > It was removed in this commit: > > https://github.com/D-Programming-Language/druntime/commit/9dca50fe65402fb3cdfbb689f1aca58dc835dce4 > > The commit message doesn't seem to mention why the use of the backtrace function was removed. But from reading the function's documentation, it looks like it works in the same way (walking the stack frame pointers, i.e. also reliant on stack frames). It probably fell short when amd64 was introduced. Although libunwind is not part of the GNU project, it is the de facto standard on Linux/GNU for stack unwinding with projects like Mono or perf using it. The license is MIT and it is easy to make it a runtime dependency of dmd/Phobos packages on Linux. -- Marco |
November 18, 2014 Re: Would you trade 0.1% in performance for a better debugging experience? | ||||
---|---|---|---|---|
| ||||
Posted in reply to Vladimir Panteleev | On Monday, 17 November 2014 at 23:14:32 UTC, Vladimir Panteleev wrote: > I proposed to build Phobos and Druntime with stack frames enabled: > > https://issues.dlang.org/show_bug.cgi?id=13726 > > Stack frames add three CPU instructions to each function (two in the prolog, and one in the epilog). This creates a linked list which debuggers, profilers, etc. can easily walk to find which function called which. They would allow debugging certain classes of bugs much more easily (e.g. the recurring InvalidMemoryOperationError - there's a thread about it in D.learn just today), and profiling your code with polling (non-instrumenting) profilers. Having run into that error myself on Android/x86 (https://github.com/D-Programming-Language/druntime/pull/784#issuecomment-42768642), it would be nice if it was easier to trace. Losing 0.1% performance certainly seems worth it, but you'd think anyone tracking such an issue could recompile druntime/phobos themselves. I was unaware that the "-gs" flag would help though, and nobody seems to have mentioned it in the forums or github until you just did. I guess the main issue is whether we're sure it doesn't get much worse than 0.1%, ie that there aren't degenerate cases where we get worse performance declines. Also, there's the principle of whether we want to force a small performance penalty, however minute, onto release builds of druntime/phobos for better debugging, which is what Walter and Martin seem to be against. |
November 18, 2014 Re: Would you trade 0.1% in performance for a better debugging experience? | ||||
---|---|---|---|---|
| ||||
Posted in reply to Steven Schveighoffer | On 11/18/14 1:49 PM, Steven Schveighoffer wrote:
> I refuse to let the fact that we package every platform's download into
> one zip to be some sort of argument :) DMD is the only distribution that
> I've ever seen do this.
Note: I can't imagine the build process for this is simple either.
-Steve
|
November 18, 2014 Re: Would you trade 0.1% in performance for a better debugging experience? | ||||
---|---|---|---|---|
| ||||
Posted in reply to Marco Leise | On Tuesday, 18 November 2014 at 18:55:43 UTC, Marco Leise wrote: > Is it fair if I argue that fixing DWARF info generation is a > better solution then? I don't know. Probably. Not all tools may read DWARF info. Also, I'm not sure if this is related but Phobos/Druntime are currently built without -g, although enabling it should be a much less controversial change. > I don't need to open the link to know the code you linked to. > Ever since I had weird seg faults when an exception was thrown > from my D callback in Gtk code I wondered when this code will > be updated. Consider that "no frame pointers" is the default > for GCC on amd64 and thus practically all libraries on Linux > can't be stack traced by druntime. It needs to use libunwind > in the long run when C++ interfacing becomes more viable and D > programmers start to expect exception handling and stack > traces to work as per system ABI. At least that's what I > believe. That sounds reasonable. I guess it remains to be seen how much we can rely on libunwind (with regards to availability, stability and compatibility with DMD-emitted DWARF info). |
November 18, 2014 Re: Would you trade 0.1% in performance for a better debugging experience? | ||||
---|---|---|---|---|
| ||||
Posted in reply to Vladimir Panteleev | Am Tue, 18 Nov 2014 19:35:06 +0000 schrieb "Vladimir Panteleev" <vladimir@thecybershadow.net>: > On Tuesday, 18 November 2014 at 18:55:43 UTC, Marco Leise wrote: > > Is it fair if I argue that fixing DWARF info generation is a better solution then? > > I don't know. Probably. Not all tools may read DWARF info. Also, I'm not sure if this is related but Phobos/Druntime are currently built without -g, although enabling it should be a much less controversial change. Were the tools just not upgraded for amd64 or is there a specific reason they require stack frame pointers? How do they work on 64-bit Linux distributions in general that have no stack frame pointers? If they are good tools, but never worked on amd64 reliably, well ... D executables without frame pointers don't make the situation any worse, right? Regarding the -g switch, I guess we could call the debug information that is required for proper exception handling on Linux "part of the language" and always generate it independently of the -g switch. Similar to how we currently require the --export-dynamic linker switch in dmd.conf. I'm really not knowledgeable about this stuff. Are we trying to do more here than C++? Does C++ also require DWARF debug information for stack unwinding? -- Marco |
November 19, 2014 Re: Would you trade 0.1% in performance for a better debugging experience? | ||||
---|---|---|---|---|
| ||||
Posted in reply to Marco Leise | On Tuesday, 18 November 2014 at 18:55:43 UTC, Marco Leise wrote:
> Is it fair if I argue that fixing DWARF info generation is a
> better solution then?
I'm afraid, DWARF is not designed to unwind segfaults, it works only with DWARF exceptions.
|
November 19, 2014 Re: Would you trade 0.1% in performance for a better debugging experience? | ||||
---|---|---|---|---|
| ||||
Posted in reply to Vladimir Panteleev | On Tuesday, 18 November 2014 at 17:15:13 UTC, Vladimir Panteleev wrote:
> Second, there's still the argument that not every debugger and profiler can take advantage of the DWARF debug information. It's certainly nowhere as easy: from the technical point of view, but also from a legal one, considering that (IIRC) most libraries dealing with DWARF debug information are GPL or LGPL, meaning we can't use them in the D runtime. And indeed, for printing the stack trace for an unhandled exception, Druntime currently walks the stack frames:
>
> https://github.com/D-Programming-Language/druntime/blob/master/src/core/runtime.d#L452-L478
If it's able to throw exceptions, it means code dealing with DWARF is already used.
|
November 19, 2014 Re: Would you trade 0.1% in performance for a better debugging experience? | ||||
---|---|---|---|---|
| ||||
Posted in reply to Vladimir Panteleev | I am personally willing to pay the small performance cost for better debugging information. |
November 19, 2014 Re: Would you trade 0.1% in performance for a better debugging experience? | ||||
---|---|---|---|---|
| ||||
Posted in reply to Vladimir Panteleev | On 18.11.2014 00:14, Vladimir Panteleev wrote: > I proposed to build Phobos and Druntime with stack frames enabled: > [...] > Personally, I think the 0.1% is practically negligible considering the > advantages. My proposal was rejected, so I'd like to hear more opinions > about this. What do you think? > I'm in favor of this change, but wanted to test the effect myself. I ran the druntime/benchmark tests with and without the -gs option. These tests mostly exercise the GC and dynamic and associative array implementations. Please note that I'm running a mobile i7 and benchmarks can vary quite a bit (average and standard deviation of 5 runs): Win32 master: 33.8s +- 0.5s Win32 master -gs: 35.8s +- 0.6s => +5.9% Being slighlty shocked by the difference, I reran the tests, but without a significant different result. Can someone else reproduce this difference? On Win64 the tests need quite a bit more memory so they get slower, but the difference is a lot less: Win64 master: 46.1s +- 0.8s Win64 master -gs: 46.3s +- 0.4s => +0.4% This can be compensated with tiny optimizations, e.g. https://github.com/D-Programming-Language/druntime/pull/1027, which are lurking everywhere. |
Copyright © 1999-2021 by the D Language Foundation