When you accidentally put endless recursion in your code, by default on linux, you get this printed to the console once the stack overflows:

[1]    37856 segmentation fault (core dumped)  ./program

Not very informative. After this, you typically open a debugger like gdb and use the backtrace command to find where this happened. Can't we just print a backtrace when it happens, just like an assertion failure?

Well, there exists a etc.linux.memoryerror module, which provides the registerMemoryErrorHandler function. After that is called, segfaults are caught using the posix function sigaction(SIGSEGV, ...), and it throws a NullPointerError or InvalidPointerError object with a stack trace.

This doesn't work with stack overflow however, because the signal callback function uses stack memory itself, so you get a stack overflow while handling the stack overflow, and the program just aborts again.

This can be solved by using sigaltstack, which provides alternative stack memory for the signal handler. I tried integrating this into the existing code, but it hijacks RIP (the instruction pointer) to an assembly routine and continues, so custom X86 assembly gets executed in the context of the segfault. In the case of stack overflow, this results in a loop where the signal handler is called endlessly.

As far as I can see, the assembly tricks are only needed to support catching the Error object, which is bad practice any way, so I thought it would be simpler to make a new handler using assert(0). I also think it can be enabled by default, at the very least in debug builds. Here's a PR with my work so far (at the time of writing):

https://github.com/dlang/dmd/pull/15331

It seems to works well, but I'm not experienced with signal handling, so review comments are welcome. Is it bad for performance? Is it unsafe? Can this break existing code? Hopefully not!

Better error messages is always nice to have thanks for your PR

I wrote a piece of D code few month ago that does print stacktrace on various segfaults (both windows/linux), would be nice to consolidate everything and have this built-in, so D gets a nice debugging experience out of the box

Zig goes this as well, and it pretty much is a nicer experience overall

https://github.com/ryuukk/backtraced/ (not pretty, needs to be cleaned up, but does the job)

On Saturday, 17 June 2023 at 22:52:42 UTC, ryuukk_ wrote:

Nice work, though I can't use it, since the GPL license isn't compatible with D's boost license.

On Saturday, 17 June 2023 at 23:01:30 UTC, Dennis wrote:

It's supposed to be public domain, i picked what ever github suggested at the time, i just changed it to CC0