Thread overview | |||||||||||||||
---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
|
May 02, 2005 Stack Tracing | ||||
---|---|---|---|---|
| ||||
Attachments: | (Walter, I have some questions/comments for you down below.) I finally implemented stack tracing for D. Example =========================== D:\trunk\src\test>stacktrace Exception:"created exception" at .test.stacktrace.methodC (test\stacktrace.d:16) at .test.stacktrace.methodB (test\stacktrace.d:13) at .test.stacktrace.methodA (test\stacktrace.d:10) at main (test\stacktrace.d:7) at _main+008F CustomException:"you idiot throwing exceptions around!" at .test.stacktrace.methodC (test\stacktrace.d:19) at .test.stacktrace.methodB (test\stacktrace.d:13) at .test.stacktrace.methodA (test\stacktrace.d:10) at main (test\stacktrace.d:7) at _main+008F AccessViolationException:"Access Violation" at .test.stacktrace.methodB (test\stacktrace.d:13) at .test.stacktrace.methodA (test\stacktrace.d:10) at main (test\stacktrace.d:7) at _main+008F DivisionByZeroException:"Integer Divide by Zero" at .test.stacktrace.methodC (test\stacktrace.d:32) at .test.stacktrace.methodB (test\stacktrace.d:13) at .test.stacktrace.methodA (test\stacktrace.d:10) at main (test\stacktrace.d:7) at _main+008F Code that produces the above output ================================ module test.stacktrace; class CustomException : Exception { this(char[] msg) { super(msg); } } void main() { methodA(); } void methodA() { methodB(); } void methodB() { methodC(); } void methodC() { Exception e = new Exception("created exception"); e.print(); try { throw new CustomException("you idiot throwing exceptions around!"); } catch (Exception e) { e.print(); } try { Object o; o.print(); // access violation } catch (Exception e) { // methodC may be missing from the output... e.print(); } try { int divideByZero(int a) { return a/0; } int i = divideByZero(5); } catch (Exception e) { // methodC may be missing from the output... e.print(); } } Notes ================== Altough it does not currently work, the Exception could also be able to contain a 'cause' (i.e. another exception). In this case, the stack trace would contain the cause exception, as in: Exception1:"message" ... ... Caused by: Exception2:"message" ... ... This is all already coded, but DMD itself currently expects Exception to have only one parameter... Walter could fix this, I can't. If you want to use stack tracing, you will have to recompile phobos.lib for now. Hopefully, Walter can include it in the next release of DMD so it will not have to be done manually anymore. See below for instructions. General Usage/Caveats: ========================== - Only implemented on Windows x86 for now. However the framework is in place to support other platforms and only one method (per platform) needs to be implemented to do so (and some aliases defined). Unfortunately, it is that method that does pretty much all the work... :-/ AMD64 and Itanium (with Windows) could be supported easily with minor mods if I only knew how to do a try-except with DMC... - For best results compile with: -g -debug (and no -O or -inline). It will work with any flags though, as long as you include -g. (But obviously, some stack frames will be missing from the output.) You will need to link your application with dbghelp.lib. See below for more info. - Some Exceptions (e.g. StreamError) do not generate a stack frame at construction (even in debug version). Consequently, the stack trace reported is erroneous by one method call. I.e. it is not the reported function that has the exception but a function called by the reported function. There is unfortunately not much that I can do, it's Walter who's pulling the strings here. I suppose in debug version all functions should properly generate stack frames. - When in the MSVC 6.0 debugger, stack tracing doesn't work. I suppose it is because it is MSVC that starts the application and "owns" it. However, I'd be interested to know if this situation applies to all debuggers or only this one. The resulting stack trace I get is something like: AccessViolationException:"Access Violation" Stack symbol not found (err=487) Stack symbol not found (err=487) at ProcessIdToSessionId+017D To use stack tracing ======================= Find attached the new or modified files: phobos/win32.mak phobos/internal/deh.c phobos/internal/object.d phobos/std/c/windows/dbghelp.d dm/lib/dbghelp.lib - given here for convenience, but it can be easily reconstructed from lib/dbghelp.lib (from Windows platform SDK) with coff2omf. (Be sure to "lib /convert" it before with MS lib) So, in a nutshell: 1. Unzip and place each file to the directory where it belongs 2. Edit the attached win32.mak (change tool paths), or modify your own according to instructions below. 3. Recompile phobos: make -f win32.mak 4. Copy dmd/src/phobos/phobos.lib to dmd/lib (you should make a backup of the old phobos.lib just in case) 5. Compile your app with -g and link with dbghelp.lib Example with Derek's build utility: build test\stacktrace.d -debug -g dbghelp.lib How to modify win32.mak --------------------------- Your win32.mak should be modified thus (or use the one attached and modify the tool paths): ... unittest.exe : unittest.d phobos.lib $(DMD) unittest -g -L+dbghelp sc unittest.obj -g dbghelp.lib OBJS= dbghelp.obj ... ... dbghelp.obj : std\c\windows\dbghelp.d $(DMD) -c $(DFLAGS) std\c\windows\dbghelp.d ... Let me know if you have any problems! Comments/possible TODOs for Walter/me/others (in that order ;) including some questions for Walter: =============================================== - I cannot presently test on other platforms, so it is possible the versioning has bugs (i.e. will not compile). I did my best to ensure it was not so, sorry for any inconvenience. - The Exception class seems to be tied-in to the DMD core. This causes a number of problems and my current hierarchy is probably less than optimal and/or doesn't work 100% like intended. Ex 1: In Object.d: class Exception... ... this(char[] msg, Exception cause) // new ctor with 'cause' param { this.msg = msg; this.cause = cause; stackTrace = new StackTrace(); } ... In unittest.d: ... catch (Exception e) { // new ctor not recognized by DMD throw new Exception("Unable to perform operation", e); } Compiler complains: unittest.d(75): constructor object.Exception.this (char[]) does not match argument types (char[27],Exception ) unittest.d(75): Error: expected 1 arguments, not 2 Also (probably a side effect of #1), classes added to object.d are not publicly visible to other D source files. Because of this and #1 the "Caused by: + stack trace" printout could not be tested. (The new "system" Exceptions (AccessViolationException, DivisionByZeroException, StackOverflowException) should be made visible to the outside world so they can be specifically caught.) Ex 2: In Exception.this(), I called another member function: fillStackTrace(). This compiled fine, but generated an access violation at run-time. Stepping through the disassembly showed that the generated address for the call to fillStackTrace() was invalid. It seems that calling a member method from Exception.this() causes an access violation... In the end, I added another class altogether to take care of the stack trace (which I now believe to be a good thing.) - In a number of methods, I use the trick "asm { nop; }" to force the compiler to generate stack frames. (I typically use this in Exception constructors to have a valid starting point for the stack trace.) Maybe there is a better way to do that. At any rate, I believe that in these places stack frames should be generated even in release version. - Some Windows aliases in std.c.windows.dbghelp.d should probably go elsewhere. I'm not too familiar with the grand scheme, so I simply put them there for now. Given instructions, I could place them in the right file(s) myself. - It would be good to move exception-related classes out of object.d and in their own file. I tried to some extend, but because Exception is so tied to the core (see above), this seemed perilous for me to attempt. Again, given instructions I could do it myself. - It would probably be a good idea to make sure that in debug version all functions generate a stack frame. - I don't know how to generate try-except block with DMC. This would help x86 stack traces and is necessary to generate stack traces for other architectures. For instance, other platform supporting Windows could be rapidly supported by doing some minor modifications. - When an exception causes the program to exit, the main handler prefixes "Error: " to the message. This prefix should be removed (the stack trace is clear enough.) - IMO, either Error should be at the top of it's own hierarchy or it should be dropped, since Exception now has a cause (so the stack trace can be displayed as Exception1:... Caused by: Exception2... Caused by: Exception3...) Note the "Caused by" couldn't be tested (see above). - The library dbghelp.lib would pretty much have to be included by default from now on. - At this point, there is not much more I can do myself, and I hope you will take the time to integrate stack tracing in the next release of DMD (knowing, if the mythos in this newsgroup is true, that you don't have much ;). I implemented it for my own benefit but I hope it will also benefit others. Of course feel free to rearrange and rewrite portions you are uncomfortable with. My own knowledge of DMD internals is very limited and there are probably better ways to do things than the way I did. As far as redistribution is concerned, I don't care who uses/redistributes this. However, I did put my name in the file where I wrotea significant amount of new code: object.d. I would like my name to remain there (or in another file if you rearrange/rewrite). That's because I'm a vain person. ;) Thanks, Max |
May 02, 2005 Re: Stack Tracing | ||||
---|---|---|---|---|
| ||||
Posted in reply to Maxime Larose | Well done! That was fast, too. [snip] > Notes > ================== > Altough it does not currently work, the Exception could also be able > to contain a 'cause' (i.e. another exception). In this case, the stack > trace > would contain the cause exception, as in: > Exception1:"message" > ... > ... > Caused by: > Exception2:"message" > ... > ... > > This is all already coded, but DMD itself currently expects Exception to have only one parameter... Walter could fix this, I can't. agreed. Exception and Error need to be merged (or at least rationalized). [snip] > - Some Exceptions (e.g. StreamError) do not generate a stack frame at > construction (even in debug version). Consequently, the stack trace > reported is erroneous by one method call. I.e. it is not the reported > function that has the exception but a function called > by the reported function. There is unfortunately not much that I can do, > it's Walter who's pulling the strings here. I suppose in debug version > all functions should properly generate stack frames. I'm curious about the StreamError (a "deprecated" alias of StreamException). Can you give more details? Is there something std.stream should do differently to get a stack trace? |
May 02, 2005 Re: Stack Tracing | ||||
---|---|---|---|---|
| ||||
Posted in reply to Ben Hinkle | > > > - Some Exceptions (e.g. StreamError) do not generate a stack frame at > > construction (even in debug version). Consequently, the stack trace > > reported is erroneous by one method call. I.e. it is not the reported > > function that has the exception but a function called > > by the reported function. There is unfortunately not much that I can do, > > it's Walter who's pulling the strings here. I suppose in debug version > > all functions should properly generate stack frames. > > I'm curious about the StreamError (a "deprecated" alias of StreamException). > Can you give more details? Is there something std.stream should do differently to get a stack trace? > Well, I don't know if it's deprecated or not, but it was used in unittest.d, that's how I found out. The problem is that it doesn't generate a stack frame. A stack frame is generated with: asm { enter 0,0; } or the more traditional: asm { push ebp; mov ebp,esp; } Without having the frame, there is not way we can know who the caller of the method was later on. In this case, you have the following sequence (more or less): main call methodA methodA create a stack frame // we will know who the caller to A was (main) call methodB methodB new StreamError // we will not know who the caller to B was (methodA) StreamError.ctor new Error Error.ctor create frame // we will know who the caller was, i.e. StreamError.ctor Notice that in the above sequence, you will never know who the caller to StreamError.ctor was (methodB). As far as the stack frame walker is concerned, the calling sequence is: main, methodA, StrearError.ctor, Error.ctor (no methodB) (the last one, Error.ctor, is found by directly checking the EIP register) If stack frames are not created when optimizing, well, that makes sense. In debug version however, each function should properly generate a stack frame so no caller is missing from the stack trace. I could sift through all the phobos files and all the methods and generate the stack frames myself manually (I did that in some of the new classes). However, there would be a ton of files impacted and I figure there is probably a way for the compiler itself to better behave. So, this one is in Walter-land... ;) There is one thing worth mentioning. The hardware exceptions (access violation, division by 0) get trapped by the OS when they occur, and eventually D's handler is called. When that happens, we always miss the top caller (the one that did the actual violation). This is most inconvenient. I'm pretty sure there would be a way for us to get that info, thought I haven't found out how yet... I'm waiting for Walter (or anyone) to tell me how to do a try-except with DMC, then I'll make some more tests. |
May 02, 2005 Re: Stack Tracing | ||||
---|---|---|---|---|
| ||||
Posted in reply to Maxime Larose | "Maxime Larose" <mlarose@broadsoft.com> wrote in message news:d55pt2$28km$1@digitaldaemon.com... > > >> > - Some Exceptions (e.g. StreamError) do not generate a stack frame at >> > construction (even in debug version). Consequently, the stack trace >> > reported is erroneous by one method call. I.e. it is not the reported >> > function that has the exception but a function called >> > by the reported function. There is unfortunately not much that I can > do, >> > it's Walter who's pulling the strings here. I suppose in debug version >> > all functions should properly generate stack frames. >> >> I'm curious about the StreamError (a "deprecated" alias of > StreamException). >> Can you give more details? Is there something std.stream should do differently to get a stack trace? >> > > Well, I don't know if it's deprecated or not, but it was used in > unittest.d, > that's how I found out. ok. I remember that unittest. It does that in order to force the std.stream unittests to get compiled. When I changed the names to StreamException I didn't want to touch other parts of phobos so that use stayed. A few other places refer to StreamError just like there are bunches of Error subclasses in phobos. Anyway, it's all waiting for a spring cleaning... > The problem is that it doesn't generate a stack frame. A stack frame is > generated with: > asm { enter 0,0; } > or the more traditional: > asm { push ebp; mov ebp,esp; } > > Without having the frame, there is not way we can know who the caller of > the > method was later on. > > In this case, you have the following sequence (more or less): > main > call methodA > methodA > create a stack frame // we will know who the caller to A was (main) > call methodB > methodB > new StreamError // we will not know who the caller to B was (methodA) > StreamError.ctor > new Error > Error.ctor > create frame // we will know who the caller was, i.e. StreamError.ctor > > Notice that in the above sequence, you will never know who the caller to > StreamError.ctor was (methodB). As far as the stack frame walker is > concerned, the calling sequence is: > main, methodA, StrearError.ctor, Error.ctor (no methodB) > (the last one, Error.ctor, is found by directly checking the EIP register) Is the stack captured at the exception ctor or at the throw statement? Is it possible to trace from the throw? For example by modifying _d_throw in src/phobos/internal or something? > If stack frames are not created when optimizing, well, that makes sense. > In > debug version however, each function should properly generate a stack > frame > so no caller is missing from the stack trace. > I could sift through all the phobos files and all the methods and generate > the stack frames myself manually (I did that in some of the new classes). > However, there would be a ton of files impacted and I figure there is > probably a way for the compiler itself to better behave. So, this one is > in > Walter-land... ;) hmm. I don't understand why StreamError is different than other exceptions, but it sounds complicated. > There is one thing worth mentioning. The hardware exceptions (access > violation, division by 0) get trapped by the OS when they occur, and > eventually D's handler is called. When that happens, we always miss the > top > caller (the one that did the actual violation). This is most inconvenient. > I'm pretty sure there would be a way for us to get that info, thought I > haven't found out how yet... I'm waiting for Walter (or anyone) to tell me > how to do a try-except with DMC, then I'll make some more tests. |
May 03, 2005 Re: Stack Tracing | ||||
---|---|---|---|---|
| ||||
Posted in reply to Ben Hinkle | > Is the stack captured at the exception ctor or at the throw statement? Is it > possible to trace from the throw? For example by modifying _d_throw in src/phobos/internal or something? > It is best to get the stack trace when creating the Exception. That is because it then becomes a quick and easy way to get a stack trace and print it (without the need for a dummy throw-catch). A possibility would be to add the ability to re-compute the stack trace on an existing exception. However, that can lead non-careful developers making it very difficult to find the root cause to a problem. > > hmm. I don't understand why StreamError is different than other exceptions, > but it sounds complicated. > It is not just StreamError. StreamError was just an example. I haven't heard from Walter yet, it's a bit annoying. The more I read this newsgroup, the more I think he really underestimate the service we are all making to D... As relatively young and immature as D is right now, having its main/only architect being responsive is pretty much required to support confidence in people investing time and energies in this project... The more I read this newsgroup, the more I understand the "please walter respond" pleas litering the place. Honestly, this frightens me quite a bit. I am not sure anymore I was right to be so enthiusastic about D as I first was. It shows great promise, of course, but Walter's focus remains unclear (or plain wrong, IMO) on a number of topics. Anyways, I realize having a GC'ed language might not be good for what I have in mind. One idea I had to rewrite phobos now seems like a major overkill. At any rate, I'll stick around a little more to see what unfolds... |
May 03, 2005 Re: Stack Tracing | ||||
---|---|---|---|---|
| ||||
Posted in reply to Maxime Larose | "Maxime Larose" <mlarose@broadsoft.com> wrote in message news:d58c31$2mal$1@digitaldaemon.com... > >> Is the stack captured at the exception ctor or at the throw statement? Is > it >> possible to trace from the throw? For example by modifying _d_throw in src/phobos/internal or something? >> > It is best to get the stack trace when creating the Exception. That is > because it then becomes a quick and easy way to get a stack trace and > print > it (without the need for a dummy throw-catch). A possibility would be to > add > the ability to re-compute the stack trace on an existing exception. > However, > that can lead non-careful developers making it very difficult to find the > root cause to a problem. If a goal of the stack-trace feature is to dump stacks without throwing then why make it necessary to instantiate an exception? I'm thinking of an API like Java's Thread.dumpStack which dumps the stack of the code calling dumpStack. I think in the case of a thrown exception the stack should be at the throw since that is where the "error" truely happens and that's where a programmer would start debugging the issue. >> hmm. I don't understand why StreamError is different than other exceptions, but it sounds complicated. >> > It is not just StreamError. StreamError was just an example. ok > I haven't heard from Walter yet, it's a bit annoying. The more I read this > newsgroup, the more I think he really underestimate the service we are all > making to D... As relatively young and immature as D is right now, having > its main/only architect being responsive is pretty much required to > support > confidence in people investing time and energies in this project... The > more > I read this newsgroup, the more I understand the "please walter respond" > pleas litering the place. Honestly, this frightens me quite a bit. I am > not > sure anymore I was right to be so enthiusastic about D as I first was. It > shows great promise, of course, but Walter's focus remains unclear (or > plain > wrong, IMO) on a number of topics. yeah - I think his posting rate has gone down lately. I didn't remember it being this bad before. One can sympathize with the desire to have a compiler that doesn't crash but IMO the rather esoteric bugs (eg crashes on unusual invalid code) he's tracking could be prioritized after some of the rather huge API changes people are talking about. There's almost no visibility into the development process so we're left to make our own assumptions about where this train is headed (and the speed at which it is traveling). > Anyways, I realize having a GC'ed language might not be good for what I > have > in mind. One idea I had to rewrite phobos now seems like a major overkill. > At any rate, I'll stick around a little more to see what unfolds... please do! You stack trace stuff should be very valuable. |
May 03, 2005 Re: Stack Tracing | ||||
---|---|---|---|---|
| ||||
Posted in reply to Ben Hinkle | "Ben Hinkle" <bhinkle@mathworks.com> wrote in message news:d58db5$2o2h$1@digitaldaemon.com... > > > If a goal of the stack-trace feature is to dump stacks without throwing then > why make it necessary to instantiate an exception? I'm thinking of an API like Java's Thread.dumpStack which dumps the stack of the code calling dumpStack. I think in the case of a thrown exception the stack should be at > the throw since that is where the "error" truely happens and that's where a > programmer would start debugging the issue. > Yes, I see your point and I agree. It would not be difficult to change what I did for what you propose. |
May 03, 2005 Re: Stack Tracing | ||||
---|---|---|---|---|
| ||||
Posted in reply to Ben Hinkle | In article <d58db5$2o2h$1@digitaldaemon.com>, Ben Hinkle says... > >> Anyways, I realize having a GC'ed language might not be good for what I >> have >> in mind. One idea I had to rewrite phobos now seems like a major overkill. >> At any rate, I'll stick around a little more to see what unfolds... > >please do! You stack trace stuff should be very valuable. If I may say so, I'd add that this contribution is an *invaluable* service to every last one of us here. This feature was long overdue. Thank you Maxime. - EricAnderton at yahoo |
May 03, 2005 Re: Stack Tracing | ||||
---|---|---|---|---|
| ||||
Posted in reply to Maxime Larose | In article <d58djs$2o9o$1@digitaldaemon.com>, Maxime Larose says... > > >"Ben Hinkle" <bhinkle@mathworks.com> wrote in message news:d58db5$2o2h$1@digitaldaemon.com... >> >> >> If a goal of the stack-trace feature is to dump stacks without throwing >then >> why make it necessary to instantiate an exception? I'm thinking of an API like Java's Thread.dumpStack which dumps the stack of the code calling dumpStack. I think in the case of a thrown exception the stack should be >at >> the throw since that is where the "error" truely happens and that's where >a >> programmer would start debugging the issue. >> > >Yes, I see your point and I agree. It would not be difficult to change what I did for what you propose. Please do. This would be a great feature to have. >Anyways, I realize having a GC'ed language might not be good for what I have in mind. One idea I had to rewrite phobos now seems like a major overkill. Been there, doing that ;) Your stack trace feature will definately be in Ares, whether it ends up in Phobos or not. Sean |
May 03, 2005 Re: Stack Tracing | ||||
---|---|---|---|---|
| ||||
Posted in reply to Ben Hinkle | On Tue, 2005-05-03 at 13:45 -0400, Ben Hinkle wrote: > "Maxime Larose" <mlarose@broadsoft.com> wrote in message news:d58c31$2mal$1@digitaldaemon.com... > > ... > If a goal of the stack-trace feature is to dump stacks without throwing then why make it necessary to instantiate an exception? I'm thinking of an API like Java's Thread.dumpStack which dumps the stack of the code calling dumpStack. I think in the case of a thrown exception the stack should be at the throw since that is where the "error" truely happens and that's where a programmer would start debugging the issue. I disagree, as I've done things like this in Java: try { someMethod(); } catch (MyException e) { //Examine exception more closely if (ICanHandle(e)) { handle(e); } else { throw e; } } Now given, this is really just a work around for an extremely poor Exception hierarchy, but I think it shows that the error doesn't necessarily occur at the throw. I'd say a stack trace should be generated by the constructor (or not, depending on parameters) and also have a method .generateST() to (re-)generate the ST for maximum flexibility. John Demme |
Copyright © 1999-2021 by the D Language Foundation