View mode: basic / threaded / horizontal-split · Log in · Help
April 27, 2005
Stack traced exceptions
Walter,
(or anyone else who can respond)

I finally got the time to investigate built-in stack tracing in the last few
days. FYI, I want to deal with Windows first.

(I'll say bluntly and off the bat that I'm probably not the best person to
tackle this issue. I have no particular knowledge of stack frames, debugging
schemes, symbol tables, omf/coff, etc. I'd whish someone more knowledgeable
than me took up the task and maybe what I say below will sound naive. At any
rate, I believe built-in stack tracing is very important for any new
language. I've made my case in another thread, I will not repeat myself
here.)

So, my initial approach went like this:
- I use MSVC to debug D executables and it usually works fine (templates are
the exception, somtimes the debugger will have a hard time with them -
understandable).
- So, I figured that MSVC probably eat its own dog food, i.e. use
IMAGEHLP.DLL to do its stack walking and symbol referencing.
- The idea is to use StalkWalk() in imagehlp.dll to walk the stack frames.
Digged up old articles on the net about imagehlp.dll and tried to implement
something in phobos.
- There are some undocumented requirements for StackWalk and I need a
CONTEXT. Apparently, a valid CONTEXT can only be obtained when: (1) the
thread is sleeping or (2) an exception is raised.
- Function _d_framehandler in deh.c receives a CONTEXT as a parameter.So, I
hacked Object.d so that a specific (new) exception was raised (w/
RaiseException) in Exception.this(). The _d_framehandler  function traps
that exception and deals with it in a special manner, .i.e. call StackWalk
etc.

Unfortunately, at this point it's not working. The first stack frame I get
back from the library is wrong somehow. I didn't have time to investigate
more yesterday, but I was a bit worried that I'd wasted my time. What if D
(or the digital mars C++ compiler) don't follow the standard stack frame
format. But than how can MSVC debug D apps?  So, yes, D must follow the
standard stack frame format, and MSVC can work with it. However, perhaps
MSVC doesn't use imagehlp.


Again, I may sound naive and ignorant, but I would like to know if imagehlp
_should_ work with D apps. If not, I'd rather know now than engulf countless
hours in a futile attempt. If imagehlp can't work, than that's a whole
different story for me. I'd have to walk the stack myself and that involves
a *lot* more effort (mainly because the learning curve would be as steep as
it gets.)

So:
1. Should imagehlp work with D apps?
2. Perhaps with some tweakings?
3. Anyone (Walter?) that can provide pointers on how stack frames/symbols
are implemented in D, the overall design (ex: this file contains methods
that do this, this other file does this), etc.

The feeling I have is it would be so much simpler for Walter to implement
this... Anyway, I'm not complaining, just trying to save myself a lot of
time (which I don't have - but aren't we all in this same situation? ;).

Thanks,

Max
April 27, 2005
Re: Stack traced exceptions
I think it's just done "by hand". On Windows you get the EBP register (or 
EIP) register for the stack frame. That then has the address of the calling 
function and the pointer to the parent stack frame. I'm not sure how to map 
the function address to a string to print. I'd recommend Googling for "Win32 
stack trace" or something for more info. I expect the code to be pretty 
platform-dependent. See for example 
http://www.eptacom.net/pubblicazioni/pub_eng/except.html

"Maxime Larose" <mlarose@broadsoft.com> wrote in message 
news:d4ntes$tm5$1@digitaldaemon.com...
> Walter,
> (or anyone else who can respond)
>
> I finally got the time to investigate built-in stack tracing in the last 
> few
> days. FYI, I want to deal with Windows first.
>
> (I'll say bluntly and off the bat that I'm probably not the best person to
> tackle this issue. I have no particular knowledge of stack frames, 
> debugging
> schemes, symbol tables, omf/coff, etc. I'd whish someone more 
> knowledgeable
> than me took up the task and maybe what I say below will sound naive. At 
> any
> rate, I believe built-in stack tracing is very important for any new
> language. I've made my case in another thread, I will not repeat myself
> here.)
>
> So, my initial approach went like this:
> - I use MSVC to debug D executables and it usually works fine (templates 
> are
> the exception, somtimes the debugger will have a hard time with them -
> understandable).
> - So, I figured that MSVC probably eat its own dog food, i.e. use
> IMAGEHLP.DLL to do its stack walking and symbol referencing.
> - The idea is to use StalkWalk() in imagehlp.dll to walk the stack frames.
> Digged up old articles on the net about imagehlp.dll and tried to 
> implement
> something in phobos.
> - There are some undocumented requirements for StackWalk and I need a
> CONTEXT. Apparently, a valid CONTEXT can only be obtained when: (1) the
> thread is sleeping or (2) an exception is raised.
> - Function _d_framehandler in deh.c receives a CONTEXT as a parameter.So, 
> I
> hacked Object.d so that a specific (new) exception was raised (w/
> RaiseException) in Exception.this(). The _d_framehandler  function traps
> that exception and deals with it in a special manner, .i.e. call StackWalk
> etc.
>
> Unfortunately, at this point it's not working. The first stack frame I get
> back from the library is wrong somehow. I didn't have time to investigate
> more yesterday, but I was a bit worried that I'd wasted my time. What if D
> (or the digital mars C++ compiler) don't follow the standard stack frame
> format. But than how can MSVC debug D apps?  So, yes, D must follow the
> standard stack frame format, and MSVC can work with it. However, perhaps
> MSVC doesn't use imagehlp.
>
>
> Again, I may sound naive and ignorant, but I would like to know if 
> imagehlp
> _should_ work with D apps. If not, I'd rather know now than engulf 
> countless
> hours in a futile attempt. If imagehlp can't work, than that's a whole
> different story for me. I'd have to walk the stack myself and that 
> involves
> a *lot* more effort (mainly because the learning curve would be as steep 
> as
> it gets.)
>
> So:
> 1. Should imagehlp work with D apps?
> 2. Perhaps with some tweakings?
> 3. Anyone (Walter?) that can provide pointers on how stack frames/symbols
> are implemented in D, the overall design (ex: this file contains methods
> that do this, this other file does this), etc.
>
> The feeling I have is it would be so much simpler for Walter to implement
> this... Anyway, I'm not complaining, just trying to save myself a lot of
> time (which I don't have - but aren't we all in this same situation? ;).
>
> Thanks,
>
> Max
>
>
April 27, 2005
Re: Stack traced exceptions
I might be completely naive here, but why can't we build stack tracking 
functionality into the language?  For example, alter the D compiler to 
have an internal stack of strings (or an index into a string table if 
you are really concerned about speed), and make it so the D compiler 
emits a "push(function_name)" when you call a function and "pop" when 
you return.
That way you get stack tracing for every architechture without having to 
know anything about the backend binary format.

Brad
April 27, 2005
Re: Stack traced exceptions
"brad beveridge" <brad@nowhere.com> wrote in message 
news:d4ords$1tth$1@digitaldaemon.com...
>I might be completely naive here, but why can't we build stack tracking 
>functionality into the language?  For example, alter the D compiler to have 
>an internal stack of strings (or an index into a string table if you are 
>really concerned about speed), and make it so the D compiler emits a 
>"push(function_name)" when you call a function and "pop" when you return.
> That way you get stack tracing for every architechture without having to 
> know anything about the backend binary format.
>
> Brad

It would slow down function calls uniformly instead of just paying the price 
of determining the stack at throw-time. The performance hit would be too big 
(I assume).

Now that I look more closely at the MSDN doc for StackWalk64 I think Maxime 
is on the right track. MSDN talks about DebugHlp.dll instead of 
ImageHlp.dll, though.
April 27, 2005
Re: Stack traced exceptions
Ben Hinkle wrote:
> It would slow down function calls uniformly instead of just paying the price 
> of determining the stack at throw-time. The performance hit would be too big 
> (I assume).

Now that sounds like premature optimisation :)
I think you'd get away with
 ** function start **
stack_trace[depth++] = function_ID;
....
 ** function return **
depth--;
Where function_ID is a pointer to a string, or something that can be 
looked up later.
Which isn't much overhead.


Pros of inbuilt stack tracing
1) Shouldn't be too hard to implement in the compiler (I could be 
completly wrong here!)
2) Will work on all platforms.
3) Can be enabled/disabled with a compiler switch
4) Don't have to go poking into stack frames, etc at throw time.

Cons of inbuilt stack tracing
1) Small uniform overhead for each function called.

Brad
April 28, 2005
Re: Stack traced exceptions
In article <d4p1u1$24mm$1@digitaldaemon.com>, Brad Beveridge says...
>
>Ben Hinkle wrote:
>> It would slow down function calls uniformly instead of just paying the price 
>> of determining the stack at throw-time. The performance hit would be too big 
>> (I assume).
>
>Now that sounds like premature optimisation :)
>I think you'd get away with
>  ** function start **
>stack_trace[depth++] = function_ID;
>....
>  ** function return **
>depth--;
>Where function_ID is a pointer to a string, or something that can be 
>looked up later.
>Which isn't much overhead.
>
>
>Pros of inbuilt stack tracing
>1) Shouldn't be too hard to implement in the compiler (I could be 
>completly wrong here!)
>2) Will work on all platforms.
>3) Can be enabled/disabled with a compiler switch
>4) Don't have to go poking into stack frames, etc at throw time.
>
>Cons of inbuilt stack tracing
>1) Small uniform overhead for each function called.
>
>Brad

While this isn't too much overhead for heavy functions, I think for getters and
setters it would swamp you.

But, the functions always return where they are supposed to, right?  That is due
to the fact that the "stack" already does what you are proposing, only its idea
of a "function id" is the function pointer.

I am guessing (from a POV of no facts) that the complex part of this would be
trying to work out which "inlined" function you are in based on ranges of insns.
This can be done if you have debug data (I think) because C++ programs in gdb
show this.

Funny thing is, the code seems to go in-out-in of inlined and optimized methods
because it all gets mixed and reordered by the optimizer.

If you want to do this portably, though, a better way that inserting "push/pop"
on every call, might be (for the compiler) to insert something like:

try {
.. function contents
}
catch(TracedException e) {
e.push_frame_name("function name");
throw e;
}

..around every function body.  Then you only pay the piper in the sense that
you have increased the bloat factor, and neutered the inlining phase of the
optimizer.  Of course if you can (at compile time) determine that this function
can't/won't throw a traced exception, then you can omit the try{}catch{}.

Another downside is that you only get stack tracing from throw-point to
catch-point, but I suspect if you want a stack trace, it is probably thrown "all
the way" to main or further, right?

Kevin
April 28, 2005
Re: Stack traced exceptions
Pushing function names on the stack and the try-catch method have one
important drawback: you lose granularity. You know in which function the
exception occured, but you don't know where inside the function. The only
way to know where exactly the exception was thrown is by inspecting EIP/EBP
and walking the stack from there.

As Ben said, walking the stack is not that difficult really. I was mentally
preparing myself to do it manually, but in the process of stepping through
my imagehlp code, I found out that some structures were badly filled by
Windows. I now fill them myself with some tricks. Anyways, to make a long
story short, I made a lot of progress yesterday and the overall design will
be a lot more elegant.

I now have a stack trace that lists:
Method name 1 + displacement
Method name 2 + displacement
Method name 3 + displacement
...

My idea is to not limit such stack traces to instances where Exception is
created manually, but also to cases where you have null pointers and other
such "system" exceptions. (BTW Kevin, that would eliminate the possibility
for the compiler to eliminate the try-catch at compile time, since any code
can throw). I little nudging in deh.c will do the trick.

The problem now is the displacement. I don't want a displacement, I want a
line number. I had trouble making it work though as the function that
resolves line numbers on Windows seems flaky.

I'm going to spend more time on it, but Ben mentionned something good:
debughlp vs imagehlp. Debughlp is only available on Windows XP and Windows
2000 professional. I assumed that was not acceptable, but now I'm
reconsidering. I guess everyone is on Windows XP or 2000 by now (time
flies!).

So, I will use debughlp and support AMD64 out of the box.

Thanks for your comments,

Max




"Kevin Bealer" <Kevin_member@pathlink.com> wrote in message
news:d4q251$l5$1@digitaldaemon.com...
> In article <d4p1u1$24mm$1@digitaldaemon.com>, Brad Beveridge says...
> >
> >Ben Hinkle wrote:
> >> It would slow down function calls uniformly instead of just paying the
price
> >> of determining the stack at throw-time. The performance hit would be
too big
> >> (I assume).
> >
> >Now that sounds like premature optimisation :)
> >I think you'd get away with
> >  ** function start **
> >stack_trace[depth++] = function_ID;
> >....
> >  ** function return **
> >depth--;
> >Where function_ID is a pointer to a string, or something that can be
> >looked up later.
> >Which isn't much overhead.
> >
> >
> >Pros of inbuilt stack tracing
> >1) Shouldn't be too hard to implement in the compiler (I could be
> >completly wrong here!)
> >2) Will work on all platforms.
> >3) Can be enabled/disabled with a compiler switch
> >4) Don't have to go poking into stack frames, etc at throw time.
> >
> >Cons of inbuilt stack tracing
> >1) Small uniform overhead for each function called.
> >
> >Brad
>
> While this isn't too much overhead for heavy functions, I think for
getters and
> setters it would swamp you.
>
> But, the functions always return where they are supposed to, right?  That
is due
> to the fact that the "stack" already does what you are proposing, only its
idea
> of a "function id" is the function pointer.
>
> I am guessing (from a POV of no facts) that the complex part of this would
be
> trying to work out which "inlined" function you are in based on ranges of
insns.
> This can be done if you have debug data (I think) because C++ programs in
gdb
> show this.
>
> Funny thing is, the code seems to go in-out-in of inlined and optimized
methods
> because it all gets mixed and reordered by the optimizer.
>
> If you want to do this portably, though, a better way that inserting
"push/pop"
> on every call, might be (for the compiler) to insert something like:
>
> try {
> .. function contents
> }
> catch(TracedException e) {
> e.push_frame_name("function name");
> throw e;
> }
>
> ..around every function body.  Then you only pay the piper in the sense
that
> you have increased the bloat factor, and neutered the inlining phase of
the
> optimizer.  Of course if you can (at compile time) determine that this
function
> can't/won't throw a traced exception, then you can omit the try{}catch{}.
>
> Another downside is that you only get stack tracing from throw-point to
> catch-point, but I suspect if you want a stack trace, it is probably
thrown "all
> the way" to main or further, right?
>
> Kevin
>
>
>
April 28, 2005
Re: Stack traced exceptions
> I'm going to spend more time on it, but Ben mentionned something good:
> debughlp vs imagehlp. Debughlp is only available on Windows XP and Windows
> 2000 professional. I assumed that was not acceptable, but now I'm
> reconsidering. I guess everyone is on Windows XP or 2000 by now (time
> flies!).
>
> So, I will use debughlp and support AMD64 out of the box.

I searched for StackWalk on MSDN and didn't find anything except the 
debughlp version. I notice, though, that the include headers in 
dm/include/win32 only talk about StackWalk and not StackWalk64. I don't know 
which is better to use - maybe the 64 version would be fine behind a 
version(X86_64) or something.
April 28, 2005
Re: Stack traced exceptions
The StalkWalk64 version deals with 32-bit apps also.

My statement regarding StalkWalk64 as only being available under
Windows2000+ is false. You can have it for previous versions as well if you
get your hands on the dbghelp.dll redistributable. So, that's settled!

I don't want to jump the gun too much, but if all goes well, we will have
stack traced D in a few days/weeks. For windows that is. For Linux, I
unfortunately don't have it installed at home. I was considering getting a
unix server up for other reasons, but I pretty much decided against that for
the moment. So, if anyone wants to step in for linux...





"Ben Hinkle" <ben.hinkle@gmail.com> wrote in message
news:d4qj84$kjr$1@digitaldaemon.com...
> > I'm going to spend more time on it, but Ben mentionned something good:
> > debughlp vs imagehlp. Debughlp is only available on Windows XP and
Windows
> > 2000 professional. I assumed that was not acceptable, but now I'm
> > reconsidering. I guess everyone is on Windows XP or 2000 by now (time
> > flies!).
> >
> > So, I will use debughlp and support AMD64 out of the box.
>
> I searched for StackWalk on MSDN and didn't find anything except the
> debughlp version. I notice, though, that the include headers in
> dm/include/win32 only talk about StackWalk and not StackWalk64. I don't
know
> which is better to use - maybe the 64 version would be fine behind a
> version(X86_64) or something.
>
>
April 30, 2005
Re: Stack traced exceptions
In article <d4ntes$tm5$1@digitaldaemon.com>, Maxime Larose says...
>
>- There are some undocumented requirements for StackWalk and I need a
>CONTEXT. Apparently, a valid CONTEXT can only be obtained when: (1) the
>thread is sleeping or (2) an exception is raised.

Look at std.thread and/or internal.gc.gcx.  One of those grabs a CONTEXT
structure in Windows builds.  This is used to get stack pointers for garbage
collection.


Sean
« First   ‹ Prev
1 2
Top | Discussion index | About this forum | D home