View mode: basic / threaded / horizontal-split · Log in · Help
December 30, 2012
[Issue 9242] New: Add stack smashing code to flush out heisenbugs
http://d.puremagic.com/issues/show_bug.cgi?id=9242

          Summary: Add stack smashing code to flush out heisenbugs
          Product: D
          Version: D2
         Platform: All
       OS/Version: All
           Status: NEW
         Severity: enhancement
         Priority: P2
        Component: DMD
       AssignedTo: nobody@puremagic.com
       ReportedBy: bugzilla@digitalmars.com


--- Comment #0 from Walter Bright <bugzilla@digitalmars.com> 2012-12-29 16:42:28 PST ---
We've lately had some very hard to track down heisenbugs that ultimately turned
out to be references to stack frames that have gone out of scope. This
particularly is happening when there are bugs in the lambda implementation, but
it is quite possible that such can still happen with user code.

It's not possible to always detect these at runtime, but their incidence can be
reduced, and bugs should be easier to track down because those references will
not randomly appear to work.

The first part is to replace the stack frame cleanup code:

   mov ESP,EBP
   pop EBP
   ret

with:

   call __stack_frame_smash
   mov ESP,EBP
   pop EBP
   ret

What __stack_frame_smash does is:

   1. set all memory [ESP..EBP] to something like 0xDEADBEEF
   2. set to 0xDEADBEEF all registers that are not guaranteed to be preserved
   across function calls.

Unfortunately, this won't smash the parameter stack, and it can't because the
callee cannot know how many parameters are on that stack (according to the
ABI). But, ya can't have everything.

The second part is, when a pointer, reference, dynamic array, or delegate is
returned from a function, add the following code to the epilog before the call
to __stack_frame_smash:

   cmp EAX,EBP
   ja  Ok
   cmp EAX,ESP
   jb  Ok
   halt
Ok:

or EDX in the case of dynamic arrays. This will halt the machine if a pointer
into the deallocated stack frame is returned.

Insertion of this code is done if the -gh switch is thrown.

-- 
Configure issuemail: http://d.puremagic.com/issues/userprefs.cgi?tab=email
------- You are receiving this mail because: -------
December 30, 2012
[Issue 9242] Add stack smashing code to flush out heisenbugs
http://d.puremagic.com/issues/show_bug.cgi?id=9242


bearophile_hugs@eml.cc changed:

          What    |Removed                     |Added
----------------------------------------------------------------------------
                CC|                            |bearophile_hugs@eml.cc


--- Comment #1 from bearophile_hugs@eml.cc 2012-12-29 17:28:45 PST ---
(In reply to comment #0)

All this sounds very nice.

> Insertion of this code is done if the -gh switch is thrown.

Maybe it's better to name/syntax it differently, to make it more future-proof
in case we'll want to add other related runtime safeties.

Example: in FreePascal there is an option for stack checking:

http://www.freepascal.org/docs-html/prog/progsu101.html#x108-1090001.2.25

-- 
Configure issuemail: http://d.puremagic.com/issues/userprefs.cgi?tab=email
------- You are receiving this mail because: -------
December 30, 2012
[Issue 9242] Add stack smashing code to flush out heisenbugs
http://d.puremagic.com/issues/show_bug.cgi?id=9242



--- Comment #2 from bearophile_hugs@eml.cc 2012-12-29 18:02:38 PST ---
(In reply to comment #1)

> Maybe it's better to name/syntax it differently, to make it more future-proof
> in case we'll want to add other related runtime safeties.

A simpler possibility is not add a switch, and just turn on this feature (and
the feature in Issue 9243 ) when the "-debug" switch is used.

-- 
Configure issuemail: http://d.puremagic.com/issues/userprefs.cgi?tab=email
------- You are receiving this mail because: -------
December 30, 2012
[Issue 9242] Add stack smashing code to flush out heisenbugs
http://d.puremagic.com/issues/show_bug.cgi?id=9242


Jonathan M Davis <jmdavisProg@gmx.com> changed:

          What    |Removed                     |Added
----------------------------------------------------------------------------
                CC|                            |jmdavisProg@gmx.com


--- Comment #3 from Jonathan M Davis <jmdavisProg@gmx.com> 2012-12-29 18:08:16 PST ---
> A simpler possibility is not add a switch, and just turn on this feature (and
the feature in Issue 9243 ) when the "-debug" switch is used.

Let's not overload the -debug flag. All it does is enable debug blocks which
are used primarily for debug output. That's fundamentally different from
something like stack smashing, and depending, someone might actually want this
feature with -release, and while you _can_ use -debug with -release, you
generally don't want to, and I wouldn't expect anyone who wanted this feature
with -release to also want debug blocks to be enabled.

-- 
Configure issuemail: http://d.puremagic.com/issues/userprefs.cgi?tab=email
------- You are receiving this mail because: -------
January 01, 2013
[Issue 9242] Add stack smashing code to flush out heisenbugs
http://d.puremagic.com/issues/show_bug.cgi?id=9242



--- Comment #4 from bearophile_hugs@eml.cc 2013-01-01 05:46:06 PST ---
This is also useful against certain kinds of attacks. This seems useful for a
language like D that tries to be "safer" than C/C++. Some info:

http://en.wikipedia.org/wiki/Buffer_overflow_protection#StackGuard

http://www.research.ibm.com/trl/projects/security/ssp/

-- 
Configure issuemail: http://d.puremagic.com/issues/userprefs.cgi?tab=email
------- You are receiving this mail because: -------
January 24, 2013
[Issue 9242] Add stack smashing code to flush out heisenbugs
http://d.puremagic.com/issues/show_bug.cgi?id=9242



--- Comment #5 from Walter Bright <bugzilla@digitalmars.com> 2013-01-23 21:33:17 PST ---
https://github.com/D-Programming-Language/dmd/pull/1542

-- 
Configure issuemail: http://d.puremagic.com/issues/userprefs.cgi?tab=email
------- You are receiving this mail because: -------
January 24, 2013
[Issue 9242] Add stack smashing code to flush out heisenbugs
http://d.puremagic.com/issues/show_bug.cgi?id=9242



--- Comment #6 from Walter Bright <bugzilla@digitalmars.com> 2013-01-23 21:42:13 PST ---
(In reply to comment #1)
> Example: in FreePascal there is an option for stack checking:

Stack overflow checking is common on CPUs with no virtual memory, such as 16
bit DOS. Virtual memory systems get stack overflow checking "for free", by
marking the memory page beyond the end of the stack as neither readable nor
writeable. Then the hardware does the check for you.

The stack smashing thing is completely different.

-- 
Configure issuemail: http://d.puremagic.com/issues/userprefs.cgi?tab=email
------- You are receiving this mail because: -------
January 24, 2013
[Issue 9242] Add stack smashing code to flush out heisenbugs
http://d.puremagic.com/issues/show_bug.cgi?id=9242



--- Comment #7 from bearophile_hugs@eml.cc 2013-01-24 01:33:36 PST ---
(In reply to comment #6)

> Stack overflow checking is common on CPUs with no virtual memory, such as 16
> bit DOS. Virtual memory systems get stack overflow checking "for free", by
> marking the memory page beyond the end of the stack as neither readable nor
> writeable. Then the hardware does the check for you.

OK. (I'd like an error message plus a stack trace when D programs overflow the
stack.)


> The stack smashing thing is completely different.

I am aware of this.


See also those pages on StackGuard, that is a third different thing.

-- 
Configure issuemail: http://d.puremagic.com/issues/userprefs.cgi?tab=email
------- You are receiving this mail because: -------
January 24, 2013
[Issue 9242] Add stack smashing code to flush out heisenbugs
http://d.puremagic.com/issues/show_bug.cgi?id=9242



--- Comment #8 from Walter Bright <bugzilla@digitalmars.com> 2013-01-24 02:10:33 PST ---
(In reply to comment #7)
> OK. (I'd like an error message plus a stack trace when D programs overflow the
> stack.)

Why not try it and see what happens? In any case, discussing stack overflow
here is not the right place, as this issue has nothing in common with it.


> See also those pages on StackGuard, that is a third different thing.

I know what stackguard is. This is not stackguard, and has nothing to do with
it.

-- 
Configure issuemail: http://d.puremagic.com/issues/userprefs.cgi?tab=email
------- You are receiving this mail because: -------
January 25, 2013
[Issue 9242] Add stack stomping code to flush out heisenbugs
http://d.puremagic.com/issues/show_bug.cgi?id=9242



--- Comment #9 from bearophile_hugs@eml.cc 2013-01-24 18:24:04 PST ---
(In reply to comment #8)

> Why not try it and see what happens?

I have just tried this code with and without -gx, and on Windows32 the program
segfaults with no error message and no stack trace:


import std.c.stdio;
void recurse(in uint i=0) {
   printf("%u ", i);
   recurse(i + 1);
}
void main() {
   recurse();
}


> In any case, discussing stack overflow
> here is not the right place, as this issue has nothing in common with it.

OK.

-- 
Configure issuemail: http://d.puremagic.com/issues/userprefs.cgi?tab=email
------- You are receiving this mail because: -------
« First   ‹ Prev
1 2
Top | Discussion index | About this forum | D home