Jump to page: 1 26  
Page
Thread overview
June 06

Over the past few months, more than half of the critical bug we've encountered which were not due to our own code are related to destructors. Specifically, destructors being invoked by the GC.

In case you need a refresher, allocating from a destructor that is called from the GC leads to an InvalidMemoryOperationError. Which is pretty bad, because you don't even know WHERE the allocation happen, because InvalidMemoryOperation does not give you a stack trace. Why ? Because generating a stack trace allocates. This also mean you can't debug other failures in destructors, such as when an assert fails (see below).

(Stack trace allocating can be fixed BTW, but requires a breaking change, because the interface we use provides you with already-formatted const(char)[] instead of structured data, but that's another discussion).

When such an InvalidMemoryOperation happens on your computer and is easy to debug, great. But when it happens once every blue moon on the production server... Not so great.

So what are the critical bugs we've seen over the last 6 months ?

AA.remove allocate(d)

We use Vibe.d extensively in our server app, and Vibe.d tries to clean after itself.
I think it's a reasonable assumption that, if you have an associative array, you should be able to call remove on it from a destructor, provided you can ensure it hasn't been collected.
Except, before v2.095.0, you couldn't, because remove might have called shrink, which allocates. This bug has been fixed now.

assert failure allocate(d)

I'm not talking about message formatting here, but pure assert(a != a) allocating.
This bug has been known for almost a decade and was finally fixed in v2.097.0 (it wasn't that hard)

throwing from the constructor of GC-allocated class is broken since 2.096.0

Why did we have an assertion failure in our destructor ?
Well turns out there was an assert checking a magic, and it was being triggered. Why ? Because of a regression introduced in v2.096.0.

This is the top 3 that we could track down. We currently have a random crash which seems to trigger when a C++ object is destructed by the GC (to be precise: the GC finalizes a class that holds an std::vector), but that might just be our bindings...

June 06

On Sunday, 6 June 2021 at 11:54:40 UTC, Mathias LANG wrote:

>

Over the past few months, more than half of the critical bug we've encountered which were not due to our own code are related to destructors. Specifically, destructors being invoked by the GC.

[...]

What would be some sensible solutions to these issues?

June 06

On Sunday, 6 June 2021 at 11:54:40 UTC, Mathias LANG wrote:

>

Over the past few months, more than half of the critical bug we've encountered which were not due to our own code are related to destructors. Specifically, destructors being invoked by the GC.

Is there anyway to build runtime with pre-allocated buffer on startup and use it to return error-text/stack-trace?
This is how Delphi does for out of memory exception & when using debug build runtime

June 06

On Sunday, 6 June 2021 at 15:01:31 UTC, apz28 wrote:

>

On Sunday, 6 June 2021 at 11:54:40 UTC, Mathias LANG wrote:

>

Over the past few months, more than half of the critical bug we've encountered which were not due to our own code are related to destructors. Specifically, destructors being invoked by the GC.

Is there anyway to build runtime with pre-allocated buffer on startup and use it to return error-text/stack-trace?
This is how Delphi does for out of memory exception & when using debug build runtime

That's a good idea, actually even a limited sized buffer is good enough, i.e the few top stack frames is mostly useful for the developers to identify the problem (even for stack overflow, e.g. recursion without termination condition).

@Mathias, hope this approach will take less work than you mentioned in the other post:

https://forum.dlang.org/post/hyayanlrcxuyljgetfms@forum.dlang.org

June 06

On Sunday, 6 June 2021 at 17:42:40 UTC, mw wrote:

>

On Sunday, 6 June 2021 at 15:01:31 UTC, apz28 wrote:

>

On Sunday, 6 June 2021 at 11:54:40 UTC, Mathias LANG wrote:

>

[...]

Is there anyway to build runtime with pre-allocated buffer on startup and use it to return error-text/stack-trace?
This is how Delphi does for out of memory exception & when using debug build runtime

That's a good idea, actually even a limited sized buffer is

We can even add a compiler flag: let the developer specify the buffer size, e.g. in debug build: 1024 * 100, and 1024 or 0 in release build.

>

good enough, i.e the few top stack frames is mostly useful for the developers to identify the problem (even for stack overflow, e.g. recursion without termination condition).

@Mathias, hope this approach will take less work than you mentioned in the other post:

https://forum.dlang.org/post/hyayanlrcxuyljgetfms@forum.dlang.org

June 07
On 07/06/2021 5:48 AM, mw wrote:
> We can even add a compiler flag: let the developer specify the buffer size, e.g. in debug build: 1024 * 100, and 1024 or 0 in release build.

It could be allocated using the DRT argument handling allowing it to be set at runtime rather than just at compilation.
June 06
On Sunday, 6 June 2021 at 18:24:45 UTC, rikki cattermole wrote:
>
> On 07/06/2021 5:48 AM, mw wrote:
>> We can even add a compiler flag: let the developer specify the buffer size, e.g. in debug build: 1024 * 100, and 1024 or 0 in release build.
>
> It could be allocated using the DRT argument handling allowing it to be set at runtime rather than just at compilation.

I've thought about it, but think it's better be controlled by the compiler: e.g. in release mode, commercial developers may not want the software end-user to peek into the trade secret by specifying a bigger buffer size flag at runtime, so s/he can see the stack frames the developers not intended to publish.

June 07
On 07/06/2021 6:37 AM, mw wrote:
> I've thought about it, but think it's better be controlled by the compiler: e.g. in release mode, commercial developers may not want the software end-user to peek into the trade secret by specifying a bigger buffer size flag at runtime, so s/he can see the stack frames the developers not intended to publish.

If the information exists in the binary, it can always be acquired by those that want it in that scenario. I.e. by attaching a debugger.
June 06

On Sunday, 6 June 2021 at 14:19:39 UTC, Imperatorn wrote:

>

On Sunday, 6 June 2021 at 11:54:40 UTC, Mathias LANG wrote:

>

Over the past few months, more than half of the critical bug we've encountered which were not due to our own code are related to destructors. Specifically, destructors being invoked by the GC.

[...]

What would be some sensible solutions to these issues?

Two first issues can be avoided by not letting the GC call class destructors, and call them deterministically instead.

June 06

On Sunday, 6 June 2021 at 18:55:47 UTC, Guillaume Piolat wrote:

>

On Sunday, 6 June 2021 at 14:19:39 UTC, Imperatorn wrote:

>

On Sunday, 6 June 2021 at 11:54:40 UTC, Mathias LANG wrote:

>

Over the past few months, more than half of the critical bug we've encountered which were not due to our own code are related to destructors. Specifically, destructors being invoked by the GC.

[...]

What would be some sensible solutions to these issues?

Two first issues can be avoided by not letting the GC call class destructors, and call them deterministically instead.

Also, you knew it was coming: http://p0nce.github.io/d-idioms/#GC-proof-resource-class

is all you need and has been an existing solution for 6 years.

« First   ‹ Prev
1 2 3 4 5 6