April 27, 2005
Walter,
(or anyone else who can respond)

I finally got the time to investigate built-in stack tracing in the last few days. FYI, I want to deal with Windows first.

(I'll say bluntly and off the bat that I'm probably not the best person to tackle this issue. I have no particular knowledge of stack frames, debugging schemes, symbol tables, omf/coff, etc. I'd whish someone more knowledgeable than me took up the task and maybe what I say below will sound naive. At any rate, I believe built-in stack tracing is very important for any new language. I've made my case in another thread, I will not repeat myself here.)

So, my initial approach went like this:
- I use MSVC to debug D executables and it usually works fine (templates are
the exception, somtimes the debugger will have a hard time with them -
understandable).
- So, I figured that MSVC probably eat its own dog food, i.e. use
IMAGEHLP.DLL to do its stack walking and symbol referencing.
- The idea is to use StalkWalk() in imagehlp.dll to walk the stack frames.
Digged up old articles on the net about imagehlp.dll and tried to implement
something in phobos.
- There are some undocumented requirements for StackWalk and I need a
CONTEXT. Apparently, a valid CONTEXT can only be obtained when: (1) the
thread is sleeping or (2) an exception is raised.
- Function _d_framehandler in deh.c receives a CONTEXT as a parameter.So, I
hacked Object.d so that a specific (new) exception was raised (w/
RaiseException) in Exception.this(). The _d_framehandler  function traps
that exception and deals with it in a special manner, .i.e. call StackWalk
etc.

Unfortunately, at this point it's not working. The first stack frame I get back from the library is wrong somehow. I didn't have time to investigate more yesterday, but I was a bit worried that I'd wasted my time. What if D (or the digital mars C++ compiler) don't follow the standard stack frame format. But than how can MSVC debug D apps?  So, yes, D must follow the standard stack frame format, and MSVC can work with it. However, perhaps MSVC doesn't use imagehlp.


Again, I may sound naive and ignorant, but I would like to know if imagehlp _should_ work with D apps. If not, I'd rather know now than engulf countless hours in a futile attempt. If imagehlp can't work, than that's a whole different story for me. I'd have to walk the stack myself and that involves a *lot* more effort (mainly because the learning curve would be as steep as it gets.)

So:
1. Should imagehlp work with D apps?
2. Perhaps with some tweakings?
3. Anyone (Walter?) that can provide pointers on how stack frames/symbols
are implemented in D, the overall design (ex: this file contains methods
that do this, this other file does this), etc.

The feeling I have is it would be so much simpler for Walter to implement this... Anyway, I'm not complaining, just trying to save myself a lot of time (which I don't have - but aren't we all in this same situation? ;).

Thanks,

Max


April 27, 2005
I think it's just done "by hand". On Windows you get the EBP register (or EIP) register for the stack frame. That then has the address of the calling function and the pointer to the parent stack frame. I'm not sure how to map the function address to a string to print. I'd recommend Googling for "Win32 stack trace" or something for more info. I expect the code to be pretty platform-dependent. See for example http://www.eptacom.net/pubblicazioni/pub_eng/except.html

"Maxime Larose" <mlarose@broadsoft.com> wrote in message news:d4ntes$tm5$1@digitaldaemon.com...
> Walter,
> (or anyone else who can respond)
>
> I finally got the time to investigate built-in stack tracing in the last
> few
> days. FYI, I want to deal with Windows first.
>
> (I'll say bluntly and off the bat that I'm probably not the best person to
> tackle this issue. I have no particular knowledge of stack frames,
> debugging
> schemes, symbol tables, omf/coff, etc. I'd whish someone more
> knowledgeable
> than me took up the task and maybe what I say below will sound naive. At
> any
> rate, I believe built-in stack tracing is very important for any new
> language. I've made my case in another thread, I will not repeat myself
> here.)
>
> So, my initial approach went like this:
> - I use MSVC to debug D executables and it usually works fine (templates
> are
> the exception, somtimes the debugger will have a hard time with them -
> understandable).
> - So, I figured that MSVC probably eat its own dog food, i.e. use
> IMAGEHLP.DLL to do its stack walking and symbol referencing.
> - The idea is to use StalkWalk() in imagehlp.dll to walk the stack frames.
> Digged up old articles on the net about imagehlp.dll and tried to
> implement
> something in phobos.
> - There are some undocumented requirements for StackWalk and I need a
> CONTEXT. Apparently, a valid CONTEXT can only be obtained when: (1) the
> thread is sleeping or (2) an exception is raised.
> - Function _d_framehandler in deh.c receives a CONTEXT as a parameter.So,
> I
> hacked Object.d so that a specific (new) exception was raised (w/
> RaiseException) in Exception.this(). The _d_framehandler  function traps
> that exception and deals with it in a special manner, .i.e. call StackWalk
> etc.
>
> Unfortunately, at this point it's not working. The first stack frame I get back from the library is wrong somehow. I didn't have time to investigate more yesterday, but I was a bit worried that I'd wasted my time. What if D (or the digital mars C++ compiler) don't follow the standard stack frame format. But than how can MSVC debug D apps?  So, yes, D must follow the standard stack frame format, and MSVC can work with it. However, perhaps MSVC doesn't use imagehlp.
>
>
> Again, I may sound naive and ignorant, but I would like to know if
> imagehlp
> _should_ work with D apps. If not, I'd rather know now than engulf
> countless
> hours in a futile attempt. If imagehlp can't work, than that's a whole
> different story for me. I'd have to walk the stack myself and that
> involves
> a *lot* more effort (mainly because the learning curve would be as steep
> as
> it gets.)
>
> So:
> 1. Should imagehlp work with D apps?
> 2. Perhaps with some tweakings?
> 3. Anyone (Walter?) that can provide pointers on how stack frames/symbols
> are implemented in D, the overall design (ex: this file contains methods
> that do this, this other file does this), etc.
>
> The feeling I have is it would be so much simpler for Walter to implement this... Anyway, I'm not complaining, just trying to save myself a lot of time (which I don't have - but aren't we all in this same situation? ;).
>
> Thanks,
>
> Max
>
> 


April 27, 2005
I might be completely naive here, but why can't we build stack tracking functionality into the language?  For example, alter the D compiler to have an internal stack of strings (or an index into a string table if you are really concerned about speed), and make it so the D compiler emits a "push(function_name)" when you call a function and "pop" when you return.
That way you get stack tracing for every architechture without having to know anything about the backend binary format.

Brad
April 27, 2005
"brad beveridge" <brad@nowhere.com> wrote in message news:d4ords$1tth$1@digitaldaemon.com...
>I might be completely naive here, but why can't we build stack tracking functionality into the language?  For example, alter the D compiler to have an internal stack of strings (or an index into a string table if you are really concerned about speed), and make it so the D compiler emits a "push(function_name)" when you call a function and "pop" when you return.
> That way you get stack tracing for every architechture without having to know anything about the backend binary format.
>
> Brad

It would slow down function calls uniformly instead of just paying the price of determining the stack at throw-time. The performance hit would be too big (I assume).

Now that I look more closely at the MSDN doc for StackWalk64 I think Maxime is on the right track. MSDN talks about DebugHlp.dll instead of ImageHlp.dll, though.


April 27, 2005
Ben Hinkle wrote:
> It would slow down function calls uniformly instead of just paying the price of determining the stack at throw-time. The performance hit would be too big (I assume).

Now that sounds like premature optimisation :)
I think you'd get away with
 ** function start **
stack_trace[depth++] = function_ID;
....
 ** function return **
depth--;
Where function_ID is a pointer to a string, or something that can be looked up later.
Which isn't much overhead.


Pros of inbuilt stack tracing
1) Shouldn't be too hard to implement in the compiler (I could be completly wrong here!)
2) Will work on all platforms.
3) Can be enabled/disabled with a compiler switch
4) Don't have to go poking into stack frames, etc at throw time.

Cons of inbuilt stack tracing
1) Small uniform overhead for each function called.

Brad
April 28, 2005
In article <d4p1u1$24mm$1@digitaldaemon.com>, Brad Beveridge says...
>
>Ben Hinkle wrote:
>> It would slow down function calls uniformly instead of just paying the price of determining the stack at throw-time. The performance hit would be too big (I assume).
>
>Now that sounds like premature optimisation :)
>I think you'd get away with
>  ** function start **
>stack_trace[depth++] = function_ID;
>....
>  ** function return **
>depth--;
>Where function_ID is a pointer to a string, or something that can be
>looked up later.
>Which isn't much overhead.
>
>
>Pros of inbuilt stack tracing
>1) Shouldn't be too hard to implement in the compiler (I could be
>completly wrong here!)
>2) Will work on all platforms.
>3) Can be enabled/disabled with a compiler switch
>4) Don't have to go poking into stack frames, etc at throw time.
>
>Cons of inbuilt stack tracing
>1) Small uniform overhead for each function called.
>
>Brad

While this isn't too much overhead for heavy functions, I think for getters and setters it would swamp you.

But, the functions always return where they are supposed to, right?  That is due to the fact that the "stack" already does what you are proposing, only its idea of a "function id" is the function pointer.

I am guessing (from a POV of no facts) that the complex part of this would be trying to work out which "inlined" function you are in based on ranges of insns. This can be done if you have debug data (I think) because C++ programs in gdb show this.

Funny thing is, the code seems to go in-out-in of inlined and optimized methods because it all gets mixed and reordered by the optimizer.

If you want to do this portably, though, a better way that inserting "push/pop" on every call, might be (for the compiler) to insert something like:

try {
.. function contents
}
catch(TracedException e) {
e.push_frame_name("function name");
throw e;
}

..around every function body.  Then you only pay the piper in the sense that you have increased the bloat factor, and neutered the inlining phase of the optimizer.  Of course if you can (at compile time) determine that this function can't/won't throw a traced exception, then you can omit the try{}catch{}.

Another downside is that you only get stack tracing from throw-point to catch-point, but I suspect if you want a stack trace, it is probably thrown "all the way" to main or further, right?

Kevin



April 28, 2005
Pushing function names on the stack and the try-catch method have one important drawback: you lose granularity. You know in which function the exception occured, but you don't know where inside the function. The only way to know where exactly the exception was thrown is by inspecting EIP/EBP and walking the stack from there.

As Ben said, walking the stack is not that difficult really. I was mentally preparing myself to do it manually, but in the process of stepping through my imagehlp code, I found out that some structures were badly filled by Windows. I now fill them myself with some tricks. Anyways, to make a long story short, I made a lot of progress yesterday and the overall design will be a lot more elegant.

I now have a stack trace that lists:
Method name 1 + displacement
Method name 2 + displacement
Method name 3 + displacement
...

My idea is to not limit such stack traces to instances where Exception is created manually, but also to cases where you have null pointers and other such "system" exceptions. (BTW Kevin, that would eliminate the possibility for the compiler to eliminate the try-catch at compile time, since any code can throw). I little nudging in deh.c will do the trick.

The problem now is the displacement. I don't want a displacement, I want a line number. I had trouble making it work though as the function that resolves line numbers on Windows seems flaky.

I'm going to spend more time on it, but Ben mentionned something good: debughlp vs imagehlp. Debughlp is only available on Windows XP and Windows 2000 professional. I assumed that was not acceptable, but now I'm reconsidering. I guess everyone is on Windows XP or 2000 by now (time flies!).

So, I will use debughlp and support AMD64 out of the box.

Thanks for your comments,

Max




"Kevin Bealer" <Kevin_member@pathlink.com> wrote in message news:d4q251$l5$1@digitaldaemon.com...
> In article <d4p1u1$24mm$1@digitaldaemon.com>, Brad Beveridge says...
> >
> >Ben Hinkle wrote:
> >> It would slow down function calls uniformly instead of just paying the
price
> >> of determining the stack at throw-time. The performance hit would be
too big
> >> (I assume).
> >
> >Now that sounds like premature optimisation :)
> >I think you'd get away with
> >  ** function start **
> >stack_trace[depth++] = function_ID;
> >....
> >  ** function return **
> >depth--;
> >Where function_ID is a pointer to a string, or something that can be
> >looked up later.
> >Which isn't much overhead.
> >
> >
> >Pros of inbuilt stack tracing
> >1) Shouldn't be too hard to implement in the compiler (I could be
> >completly wrong here!)
> >2) Will work on all platforms.
> >3) Can be enabled/disabled with a compiler switch
> >4) Don't have to go poking into stack frames, etc at throw time.
> >
> >Cons of inbuilt stack tracing
> >1) Small uniform overhead for each function called.
> >
> >Brad
>
> While this isn't too much overhead for heavy functions, I think for
getters and
> setters it would swamp you.
>
> But, the functions always return where they are supposed to, right?  That
is due
> to the fact that the "stack" already does what you are proposing, only its
idea
> of a "function id" is the function pointer.
>
> I am guessing (from a POV of no facts) that the complex part of this would
be
> trying to work out which "inlined" function you are in based on ranges of
insns.
> This can be done if you have debug data (I think) because C++ programs in
gdb
> show this.
>
> Funny thing is, the code seems to go in-out-in of inlined and optimized
methods
> because it all gets mixed and reordered by the optimizer.
>
> If you want to do this portably, though, a better way that inserting
"push/pop"
> on every call, might be (for the compiler) to insert something like:
>
> try {
> .. function contents
> }
> catch(TracedException e) {
> e.push_frame_name("function name");
> throw e;
> }
>
> ..around every function body.  Then you only pay the piper in the sense
that
> you have increased the bloat factor, and neutered the inlining phase of
the
> optimizer.  Of course if you can (at compile time) determine that this
function
> can't/won't throw a traced exception, then you can omit the try{}catch{}.
>
> Another downside is that you only get stack tracing from throw-point to catch-point, but I suspect if you want a stack trace, it is probably
thrown "all
> the way" to main or further, right?
>
> Kevin
>
>
>


April 28, 2005
> I'm going to spend more time on it, but Ben mentionned something good: debughlp vs imagehlp. Debughlp is only available on Windows XP and Windows 2000 professional. I assumed that was not acceptable, but now I'm reconsidering. I guess everyone is on Windows XP or 2000 by now (time flies!).
>
> So, I will use debughlp and support AMD64 out of the box.

I searched for StackWalk on MSDN and didn't find anything except the debughlp version. I notice, though, that the include headers in dm/include/win32 only talk about StackWalk and not StackWalk64. I don't know which is better to use - maybe the 64 version would be fine behind a version(X86_64) or something.


April 28, 2005
The StalkWalk64 version deals with 32-bit apps also.

My statement regarding StalkWalk64 as only being available under Windows2000+ is false. You can have it for previous versions as well if you get your hands on the dbghelp.dll redistributable. So, that's settled!

I don't want to jump the gun too much, but if all goes well, we will have stack traced D in a few days/weeks. For windows that is. For Linux, I unfortunately don't have it installed at home. I was considering getting a unix server up for other reasons, but I pretty much decided against that for the moment. So, if anyone wants to step in for linux...





"Ben Hinkle" <ben.hinkle@gmail.com> wrote in message news:d4qj84$kjr$1@digitaldaemon.com...
> > I'm going to spend more time on it, but Ben mentionned something good: debughlp vs imagehlp. Debughlp is only available on Windows XP and
Windows
> > 2000 professional. I assumed that was not acceptable, but now I'm reconsidering. I guess everyone is on Windows XP or 2000 by now (time flies!).
> >
> > So, I will use debughlp and support AMD64 out of the box.
>
> I searched for StackWalk on MSDN and didn't find anything except the debughlp version. I notice, though, that the include headers in dm/include/win32 only talk about StackWalk and not StackWalk64. I don't
know
> which is better to use - maybe the 64 version would be fine behind a
> version(X86_64) or something.
>
>


April 30, 2005
In article <d4ntes$tm5$1@digitaldaemon.com>, Maxime Larose says...
>
>- There are some undocumented requirements for StackWalk and I need a
>CONTEXT. Apparently, a valid CONTEXT can only be obtained when: (1) the
>thread is sleeping or (2) an exception is raised.

Look at std.thread and/or internal.gc.gcx.  One of those grabs a CONTEXT structure in Windows builds.  This is used to get stack pointers for garbage collection.


Sean


« First   ‹ Prev
1 2
Top | Discussion index | About this forum | D home