Jump to page: 1 2
Thread overview
Stack Tracing
May 02, 2005
Maxime Larose
May 02, 2005
Ben Hinkle
May 02, 2005
Maxime Larose
May 02, 2005
Ben Hinkle
May 03, 2005
Maxime Larose
May 03, 2005
Ben Hinkle
May 03, 2005
Maxime Larose
May 03, 2005
Sean Kelly
May 04, 2005
Maxime Larose
May 04, 2005
Sean Kelly
May 03, 2005
pragma
May 03, 2005
John Demme
May 03, 2005
Ben Hinkle
May 02, 2005
(Walter, I have some questions/comments for you down below.)

I finally implemented stack tracing for D.

Example
===========================
D:\trunk\src\test>stacktrace
Exception:"created exception"
  at .test.stacktrace.methodC (test\stacktrace.d:16)
  at .test.stacktrace.methodB (test\stacktrace.d:13)
  at .test.stacktrace.methodA (test\stacktrace.d:10)
  at main (test\stacktrace.d:7)
  at _main+008F
CustomException:"you idiot throwing exceptions around!"
  at .test.stacktrace.methodC (test\stacktrace.d:19)
  at .test.stacktrace.methodB (test\stacktrace.d:13)
  at .test.stacktrace.methodA (test\stacktrace.d:10)
  at main (test\stacktrace.d:7)
  at _main+008F
AccessViolationException:"Access Violation"
  at .test.stacktrace.methodB (test\stacktrace.d:13)
  at .test.stacktrace.methodA (test\stacktrace.d:10)
  at main (test\stacktrace.d:7)
  at _main+008F
DivisionByZeroException:"Integer Divide by Zero"
  at .test.stacktrace.methodC (test\stacktrace.d:32)
  at .test.stacktrace.methodB (test\stacktrace.d:13)
  at .test.stacktrace.methodA (test\stacktrace.d:10)
  at main (test\stacktrace.d:7)
  at _main+008F


Code that produces the above output
================================
module test.stacktrace;
class CustomException : Exception
{ this(char[] msg) { super(msg); }
}

void main() {
  methodA();
}
void methodA() {
  methodB();
}
void methodB() {
  methodC();
}
void methodC() {
  Exception e = new Exception("created exception");
  e.print();
  try {
    throw new CustomException("you idiot throwing exceptions around!");
  } catch (Exception e)    {
    e.print();
  }
  try {
    Object o;
    o.print(); // access violation
  } catch (Exception e) {
    // methodC may be missing from the output...
    e.print();
  }
  try {
    int divideByZero(int a) { return a/0; }
    int i = divideByZero(5);
  } catch (Exception e) {
    // methodC may be missing from the output...
    e.print();
  }
}

Notes
==================
Altough it does not currently work, the Exception could also be able
to contain a 'cause' (i.e. another exception). In this case, the stack trace
would contain the cause exception, as in:
Exception1:"message"
  ...
  ...
Caused by:
Exception2:"message"
  ...
  ...

This is all already coded, but DMD itself currently expects Exception to have only one parameter... Walter could fix this, I can't.

If you want to use stack tracing, you will have to recompile phobos.lib
for now. Hopefully, Walter can include it in the next release of DMD
so it will not have to be done manually anymore.
See below for instructions.


General Usage/Caveats:
==========================
- Only implemented on Windows x86 for now. However the framework is in place
  to support other platforms and only one method (per platform) needs to be
  implemented to do so (and some aliases defined). Unfortunately,
  it is that method that does pretty much all the work... :-/
  AMD64 and Itanium (with Windows) could be supported easily with minor mods
  if I only knew how to do a try-except with DMC...

- For best results compile with: -g -debug (and no -O or -inline).
  It will work with any flags though, as long as you include -g. (But
  obviously, some stack frames will be missing from the output.)
  You will need to link your application with dbghelp.lib.
  See below for more info.

- Some Exceptions (e.g. StreamError) do not generate a stack frame at
  construction (even in debug version). Consequently, the stack trace
  reported is erroneous by one method call. I.e. it is not the reported
  function that has the exception but a function called
  by the reported function. There is unfortunately not much that I can do,
  it's Walter who's pulling the strings here. I suppose in debug version
  all functions should properly generate stack frames.

- When in the MSVC 6.0 debugger, stack tracing doesn't work. I suppose
  it is because it is MSVC that starts the application and "owns" it.
  However, I'd be interested to know if this situation applies to all
  debuggers or only this one.
  The resulting stack trace I get is something like:
  AccessViolationException:"Access Violation"
    Stack symbol not found (err=487)
    Stack symbol not found (err=487)
    at ProcessIdToSessionId+017D



To use stack tracing
=======================
Find attached the new or modified files:
phobos/win32.mak
phobos/internal/deh.c
phobos/internal/object.d
phobos/std/c/windows/dbghelp.d
dm/lib/dbghelp.lib - given here for convenience, but it can be easily
                     reconstructed from lib/dbghelp.lib (from Windows
                     platform SDK) with coff2omf. (Be sure to "lib /convert"
                     it before with MS lib)

So, in a nutshell:
1. Unzip and place each file to the directory where it belongs
2. Edit the attached win32.mak (change tool paths), or modify your own
   according to instructions below.
3. Recompile phobos: make -f win32.mak
4. Copy dmd/src/phobos/phobos.lib to dmd/lib (you should make a backup
   of the old phobos.lib just in case)
5. Compile your app with -g and link with dbghelp.lib
   Example with Derek's build utility:
   build test\stacktrace.d -debug -g dbghelp.lib

How to modify win32.mak
---------------------------
Your win32.mak should be modified thus (or use the one attached and
modify the tool paths):
...
unittest.exe : unittest.d phobos.lib
 $(DMD) unittest -g -L+dbghelp
 sc unittest.obj -g dbghelp.lib

OBJS= dbghelp.obj ...
...
dbghelp.obj : std\c\windows\dbghelp.d
 $(DMD) -c $(DFLAGS) std\c\windows\dbghelp.d
...


Let me know if you have any problems!



Comments/possible TODOs for Walter/me/others (in that order ;)
including some questions for Walter:
===============================================
- I cannot presently test on other platforms, so it is possible the
  versioning has bugs (i.e. will not compile). I did my best to ensure it
was
  not so, sorry for any inconvenience.

- The Exception class seems to be tied-in to the DMD core. This causes
  a number of problems and my current hierarchy is probably less than
  optimal and/or doesn't work 100% like intended.
  Ex 1:
In Object.d:
  class Exception...
  ...
  this(char[] msg, Exception cause)  // new ctor with 'cause' param
  {
    this.msg = msg;
    this.cause = cause;
    stackTrace = new StackTrace();
  }
  ...

In unittest.d:
  ...
  catch (Exception e)
  {
    // new ctor not recognized by DMD
    throw new Exception("Unable to perform operation", e);
  }

  Compiler complains:
  unittest.d(75): constructor object.Exception.this (char[]) does not match
  argument types (char[27],Exception )
  unittest.d(75): Error: expected 1 arguments, not 2

  Also (probably a side effect of #1), classes added to object.d are not
  publicly visible to other D source files. Because of this and #1 the
  "Caused by: + stack trace" printout could not be tested.
  (The new "system" Exceptions (AccessViolationException,
  DivisionByZeroException, StackOverflowException) should be made visible
  to the outside world so they can be specifically caught.)

  Ex 2:
  In Exception.this(), I called another member function: fillStackTrace().
  This compiled fine, but generated an access violation at run-time.
  Stepping through the disassembly showed that the generated address for
  the call to fillStackTrace() was invalid. It seems that calling a member
  method from Exception.this() causes an access violation... In the end,
  I added another class altogether to take care of the stack trace (which
  I now believe to be a good thing.)

- In a number of methods, I use the trick "asm { nop; }" to force
  the compiler to generate stack frames. (I typically use this in Exception
  constructors to have a valid starting point for the stack trace.)
  Maybe there is a better way to do that. At any rate, I believe that
  in these places stack frames should be generated even in release
  version.

- Some Windows aliases in std.c.windows.dbghelp.d should probably go
elsewhere.
  I'm not too familiar with the grand scheme, so I simply put them there
  for now. Given instructions, I could place them in the right file(s)
myself.

- It would be good to move exception-related classes out of object.d
  and in their own file. I tried to some extend, but because Exception is
  so tied to the core (see above), this seemed perilous for me to attempt.
  Again, given instructions I could do it myself.

- It would probably be a good idea to make sure that in debug version all
  functions generate a stack frame.

- I don't know how to generate try-except block with DMC.
  This would help x86 stack traces and is necessary to generate stack traces
  for other architectures. For instance, other platform supporting Windows
  could be rapidly supported by doing some minor modifications.

- When an exception causes the program to exit, the main handler prefixes
  "Error: " to the message. This prefix should be removed (the stack trace
is
  clear enough.)

- IMO, either Error should be at the top of it's own hierarchy or it should
  be dropped, since Exception now has a cause (so the stack trace can be
  displayed as Exception1:... Caused by: Exception2... Caused by:
Exception3...)
  Note the "Caused by" couldn't be tested (see above).

- The library dbghelp.lib would pretty much have to be included by default
  from now on.

- At this point, there is not much more I can do myself, and I hope you will
  take the time to integrate stack tracing in the next release of DMD
  (knowing, if the mythos in this newsgroup is true, that you don't
  have much ;).  I implemented it for my own benefit but I hope it will
  also benefit others. Of course feel free to rearrange and rewrite
  portions you are uncomfortable with. My own knowledge of DMD internals
  is very limited and there are probably better ways to do things than
  the way I did.

  As far as redistribution is concerned, I don't care who uses/redistributes
  this. However, I did put my name in the file where I wrotea significant
  amount of new code: object.d. I would like my name to remain
  there (or in another file if you rearrange/rewrite). That's because
  I'm a vain person. ;)

Thanks,

Max







May 02, 2005
Well done! That was fast, too.

[snip]
> Notes
> ==================
> Altough it does not currently work, the Exception could also be able
> to contain a 'cause' (i.e. another exception). In this case, the stack
> trace
> would contain the cause exception, as in:
> Exception1:"message"
>  ...
>  ...
> Caused by:
> Exception2:"message"
>  ...
>  ...
>
> This is all already coded, but DMD itself currently expects Exception to have only one parameter... Walter could fix this, I can't.

agreed. Exception and Error need to be merged (or at least rationalized).

[snip]

> - Some Exceptions (e.g. StreamError) do not generate a stack frame at
>  construction (even in debug version). Consequently, the stack trace
>  reported is erroneous by one method call. I.e. it is not the reported
>  function that has the exception but a function called
>  by the reported function. There is unfortunately not much that I can do,
>  it's Walter who's pulling the strings here. I suppose in debug version
>  all functions should properly generate stack frames.

I'm curious about the StreamError (a "deprecated" alias of StreamException). Can you give more details? Is there something std.stream should do differently to get a stack trace?


May 02, 2005
>
> > - Some Exceptions (e.g. StreamError) do not generate a stack frame at
> >  construction (even in debug version). Consequently, the stack trace
> >  reported is erroneous by one method call. I.e. it is not the reported
> >  function that has the exception but a function called
> >  by the reported function. There is unfortunately not much that I can
do,
> >  it's Walter who's pulling the strings here. I suppose in debug version
> >  all functions should properly generate stack frames.
>
> I'm curious about the StreamError (a "deprecated" alias of
StreamException).
> Can you give more details? Is there something std.stream should do differently to get a stack trace?
>

Well, I don't know if it's deprecated or not, but it was used in unittest.d, that's how I found out.

The problem is that it doesn't generate a stack frame. A stack frame is
generated with:
asm { enter 0,0; }
or the more traditional:
asm { push ebp; mov ebp,esp; }

Without having the frame, there is not way we can know who the caller of the method was later on.

In this case, you have the following sequence (more or less):
main
  call methodA
methodA
  create a stack frame   // we will know who the caller to A was (main)
  call methodB
methodB
  new StreamError  // we will not know who the caller to B was (methodA)
StreamError.ctor
  new Error
Error.ctor
  create frame // we will know who the caller was, i.e. StreamError.ctor

Notice that in the above sequence, you will never know who the caller to
StreamError.ctor was (methodB). As far as the stack frame walker is
concerned, the calling sequence is:
main, methodA, StrearError.ctor, Error.ctor   (no methodB)
(the last one, Error.ctor, is found by directly checking the EIP register)

If stack frames are not created when optimizing, well, that makes sense. In
debug version however, each function should properly generate a stack frame
so no caller is missing from the stack trace.
I could sift through all the phobos files and all the methods and generate
the stack frames myself manually (I did that in some of the new classes).
However, there would be a ton of files impacted and I figure there is
probably a way for the compiler itself to better behave. So, this one is in
Walter-land... ;)


There is one thing worth mentioning. The hardware exceptions (access violation, division by 0) get trapped by the OS when they occur, and eventually D's handler is called. When that happens, we always miss the top caller (the one that did the actual violation). This is most inconvenient. I'm pretty sure there would be a way for us to get that info, thought I haven't found out how yet... I'm waiting for Walter (or anyone) to tell me how to do a try-except with DMC, then I'll make some more tests.




May 02, 2005
"Maxime Larose" <mlarose@broadsoft.com> wrote in message news:d55pt2$28km$1@digitaldaemon.com...
> >
>> > - Some Exceptions (e.g. StreamError) do not generate a stack frame at
>> >  construction (even in debug version). Consequently, the stack trace
>> >  reported is erroneous by one method call. I.e. it is not the reported
>> >  function that has the exception but a function called
>> >  by the reported function. There is unfortunately not much that I can
> do,
>> >  it's Walter who's pulling the strings here. I suppose in debug version
>> >  all functions should properly generate stack frames.
>>
>> I'm curious about the StreamError (a "deprecated" alias of
> StreamException).
>> Can you give more details? Is there something std.stream should do differently to get a stack trace?
>>
>
> Well, I don't know if it's deprecated or not, but it was used in
> unittest.d,
> that's how I found out.

ok. I remember that unittest. It does that in order to force the std.stream unittests to get compiled. When I changed the names to StreamException I didn't want to touch other parts of phobos so that use stayed. A few other places refer to StreamError just like there are bunches of Error subclasses in phobos. Anyway, it's all waiting for a spring cleaning...

> The problem is that it doesn't generate a stack frame. A stack frame is
> generated with:
> asm { enter 0,0; }
> or the more traditional:
> asm { push ebp; mov ebp,esp; }
>
> Without having the frame, there is not way we can know who the caller of
> the
> method was later on.
>
> In this case, you have the following sequence (more or less):
> main
>  call methodA
> methodA
>  create a stack frame   // we will know who the caller to A was (main)
>  call methodB
> methodB
>  new StreamError  // we will not know who the caller to B was (methodA)
> StreamError.ctor
>  new Error
> Error.ctor
>  create frame // we will know who the caller was, i.e. StreamError.ctor
>
> Notice that in the above sequence, you will never know who the caller to
> StreamError.ctor was (methodB). As far as the stack frame walker is
> concerned, the calling sequence is:
> main, methodA, StrearError.ctor, Error.ctor   (no methodB)
> (the last one, Error.ctor, is found by directly checking the EIP register)

Is the stack captured at the exception ctor or at the throw statement? Is it possible to trace from the throw? For example by modifying _d_throw in src/phobos/internal or something?

> If stack frames are not created when optimizing, well, that makes sense.
> In
> debug version however, each function should properly generate a stack
> frame
> so no caller is missing from the stack trace.
> I could sift through all the phobos files and all the methods and generate
> the stack frames myself manually (I did that in some of the new classes).
> However, there would be a ton of files impacted and I figure there is
> probably a way for the compiler itself to better behave. So, this one is
> in
> Walter-land... ;)

hmm. I don't understand why StreamError is different than other exceptions, but it sounds complicated.

> There is one thing worth mentioning. The hardware exceptions (access
> violation, division by 0) get trapped by the OS when they occur, and
> eventually D's handler is called. When that happens, we always miss the
> top
> caller (the one that did the actual violation). This is most inconvenient.
> I'm pretty sure there would be a way for us to get that info, thought I
> haven't found out how yet... I'm waiting for Walter (or anyone) to tell me
> how to do a try-except with DMC, then I'll make some more tests.


May 03, 2005
> Is the stack captured at the exception ctor or at the throw statement? Is
it
> possible to trace from the throw? For example by modifying _d_throw in src/phobos/internal or something?
>
It is best to get the stack trace when creating the Exception. That is because it then becomes a quick and easy way to get a stack trace and print it (without the need for a dummy throw-catch). A possibility would be to add the ability to re-compute the stack trace on an existing exception. However, that can lead non-careful developers making it very difficult to find the root cause to a problem.


>
> hmm. I don't understand why StreamError is different than other
exceptions,
> but it sounds complicated.
>
It is not just StreamError. StreamError was just an example.


I haven't heard from Walter yet, it's a bit annoying. The more I read this newsgroup, the more I think he really underestimate the service we are all making to D... As relatively young and immature as D is right now, having its main/only architect being responsive is pretty much required to support confidence in people investing time and energies in this project... The more I read this newsgroup, the more I understand the "please walter respond" pleas litering the place. Honestly, this frightens me quite a bit. I am not sure anymore I was right to be so enthiusastic about D as I first was. It shows great promise, of course, but Walter's focus remains unclear (or plain wrong, IMO) on a number of topics.

Anyways, I realize having a GC'ed language might not be good for what I have in mind. One idea I had to rewrite phobos now seems like a major overkill. At any rate, I'll stick around a little more to see what unfolds...




May 03, 2005
"Maxime Larose" <mlarose@broadsoft.com> wrote in message news:d58c31$2mal$1@digitaldaemon.com...
>
>> Is the stack captured at the exception ctor or at the throw statement? Is
> it
>> possible to trace from the throw? For example by modifying _d_throw in src/phobos/internal or something?
>>
> It is best to get the stack trace when creating the Exception. That is
> because it then becomes a quick and easy way to get a stack trace and
> print
> it (without the need for a dummy throw-catch). A possibility would be to
> add
> the ability to re-compute the stack trace on an existing exception.
> However,
> that can lead non-careful developers making it very difficult to find the
> root cause to a problem.

If a goal of the stack-trace feature is to dump stacks without throwing then why make it necessary to instantiate an exception? I'm thinking of an API like Java's Thread.dumpStack which dumps the stack of the code calling dumpStack. I think in the case of a thrown exception the stack should be at the throw since that is where the "error" truely happens and that's where a programmer would start debugging the issue.

>> hmm. I don't understand why StreamError is different than other exceptions, but it sounds complicated.
>>
> It is not just StreamError. StreamError was just an example.

ok

> I haven't heard from Walter yet, it's a bit annoying. The more I read this
> newsgroup, the more I think he really underestimate the service we are all
> making to D... As relatively young and immature as D is right now, having
> its main/only architect being responsive is pretty much required to
> support
> confidence in people investing time and energies in this project... The
> more
> I read this newsgroup, the more I understand the "please walter respond"
> pleas litering the place. Honestly, this frightens me quite a bit. I am
> not
> sure anymore I was right to be so enthiusastic about D as I first was. It
> shows great promise, of course, but Walter's focus remains unclear (or
> plain
> wrong, IMO) on a number of topics.

yeah - I think his posting rate has gone down lately. I didn't remember it being this bad before. One can sympathize with the desire to have a compiler that doesn't crash but IMO the rather esoteric bugs (eg crashes on unusual invalid code) he's tracking could be prioritized after some of the rather huge API changes people are talking about. There's almost no visibility into the development process so we're left to make our own assumptions about where this train is headed (and the speed at which it is traveling).

> Anyways, I realize having a GC'ed language might not be good for what I
> have
> in mind. One idea I had to rewrite phobos now seems like a major overkill.
> At any rate, I'll stick around a little more to see what unfolds...

please do! You stack trace stuff should be very valuable.


May 03, 2005
"Ben Hinkle" <bhinkle@mathworks.com> wrote in message news:d58db5$2o2h$1@digitaldaemon.com...
>
>
> If a goal of the stack-trace feature is to dump stacks without throwing
then
> why make it necessary to instantiate an exception? I'm thinking of an API like Java's Thread.dumpStack which dumps the stack of the code calling dumpStack. I think in the case of a thrown exception the stack should be
at
> the throw since that is where the "error" truely happens and that's where
a
> programmer would start debugging the issue.
>

Yes, I see your point and I agree. It would not be difficult to change what I did for what you propose.



May 03, 2005
In article <d58db5$2o2h$1@digitaldaemon.com>, Ben Hinkle says...
>
>> Anyways, I realize having a GC'ed language might not be good for what I
>> have
>> in mind. One idea I had to rewrite phobos now seems like a major overkill.
>> At any rate, I'll stick around a little more to see what unfolds...
>
>please do! You stack trace stuff should be very valuable.

If I may say so, I'd add that this contribution is an *invaluable* service to every last one of us here.  This feature was long overdue.

Thank you Maxime.

- EricAnderton at yahoo
May 03, 2005
In article <d58djs$2o9o$1@digitaldaemon.com>, Maxime Larose says...
>
>
>"Ben Hinkle" <bhinkle@mathworks.com> wrote in message news:d58db5$2o2h$1@digitaldaemon.com...
>>
>>
>> If a goal of the stack-trace feature is to dump stacks without throwing
>then
>> why make it necessary to instantiate an exception? I'm thinking of an API like Java's Thread.dumpStack which dumps the stack of the code calling dumpStack. I think in the case of a thrown exception the stack should be
>at
>> the throw since that is where the "error" truely happens and that's where
>a
>> programmer would start debugging the issue.
>>
>
>Yes, I see your point and I agree. It would not be difficult to change what I did for what you propose.

Please do.  This would be a great feature to have.

>Anyways, I realize having a GC'ed language might not be good for what I have in mind. One idea I had to rewrite phobos now seems like a major overkill.

Been there, doing that ;)  Your stack trace feature will definately be in Ares, whether it ends up in Phobos or not.


Sean


May 03, 2005
On Tue, 2005-05-03 at 13:45 -0400, Ben Hinkle wrote:
> "Maxime Larose" <mlarose@broadsoft.com> wrote in message news:d58c31$2mal$1@digitaldaemon.com...
> >
...
> If a goal of the stack-trace feature is to dump stacks without throwing then why make it necessary to instantiate an exception? I'm thinking of an API like Java's Thread.dumpStack which dumps the stack of the code calling dumpStack. I think in the case of a thrown exception the stack should be at the throw since that is where the "error" truely happens and that's where a programmer would start debugging the issue.
I disagree, as I've done things like this in Java:
try {
	someMethod();
} catch (MyException e) {
	//Examine exception more closely
	if (ICanHandle(e)) {
		handle(e);
	} else {
		throw e;
	}
}

Now given, this is really just a work around for an extremely poor Exception hierarchy, but I think it shows that the error doesn't necessarily occur at the throw.  I'd say a stack trace should be generated by the constructor (or not, depending on parameters) and also have a method .generateST() to (re-)generate the ST for maximum flexibility.

John Demme

« First   ‹ Prev
1 2