On Thursday, 18 March 2021 at 09:21:27 UTC, Per Nordlöw wrote:
> On Thursday, 4 March 2021 at 13:54:48 UTC, Mike Parker wrote:
>> The blog:
>> https://dlang.org/blog/2021/03/04/symphony-of-destruction-structs-classes-and-the-gc-part-one/
>
> Btw, what is the motive behind D's GC not being able to correctly handle GC allocations in  class destructors.
>
> Is it by design or because of limitations in D's current GC implementation?

Just implementation deficiency. I think it is fixable with some refactoring of the GC pipeline. One approach would be, (similar to other language implementations - see below), that GC-allocated objects with destructors should be placed on a queue and their destructors be called when the GC has finished the collection. Afterwards, the GC can release their memory during the next collection.

> And how does this relate to exception-throwing destructors in other managed languages such as C# and Go; are they forbidden or allowed and safe thanks to a more resilient GC?

TL;DR
* Go doesn't have exceptions or destructors. You can attach a finalizer function to an object via [0] which will be called before the object will be collected. After the associated finalizer is called, the object is marked as reachable again and the finalizer function is unset. Since all finalizers are called in a separate goroutine, it is not an issue to allocate memory from them, as technically this happens separately from the actual garbage collection.

* There is something like destructors (aka finalizers) in C#, but they can't be used to implement the RAII design pattern. They are even less deterministic than destructors of GC-allocated classes in D, as they're only called automatically by the runtime and by an arbitrary thread. Their runtime designed in such a way that memory allocation in destructors is not a problem at all, however the default policy is that thrown exceptions terminate the process, though that could be configured differently.

---

Instead of destructors, the recommended idiom in Go is to wrap resources in wrapper structs and implement a Close() method for those types, which the user of the code must not forget to call manually and sometimes check for error. They have `defer`, which is similar to D's `scope (exit)`. Similar to C#, finalizers in Go are not reliable and should probably be only used as a safety net to detect whether an object was forgotten to be closed manually. If a finalizer takes a long time to complete a clean-up task, it is recommended that it spawns a separate goroutine.

---

C# has 2 concepts: finalizers and the IDisposable interface.

C# finalizers [1][2] are defined using the C++ destructor syntax `~T()` (rather than D's `~this()`) which is lowered to a method that overrides the Object.Finalize() base method like so:

class Resource { ~Resource() { /* custom code */ } } // user code

// gets lowered to:

class Resource
{
  protected override Finalize()
  {
    try { /* custom code */ } finally { base.Finalize(); }
  }
}

Which means that finalization happens automatically from the most-derived class to the least derived one. This lowering also implies that the implementation is tolerant to exceptions. It is an compile-time error to manually define a `Finalize` method. Finalizers can only be defined by classes (reference types) and not structs (value types). Finalizers are only be called automatically (there's no analog to D's `destroy` or C++'s `delete`) and the only way to force that is using `System.GC.Collect()`, which is almost always bad idea. Finalizers used to be called at the end of the application when targeting .NET Framework, but the docs say that this is no longer the case with the newer the .NET Core, though this may have been addressed after the docs were written. The implementation may call finalizers from any thread, so your code must be prepared to handle that.

Given that finalizers are unsuitable for deterministic resource management, it is strongly recommended that class authors should implement the IDisposable [3] interface. Users of classes that implement IDisposable can either manually call IDisposable.Dispose() or they can use the `using` statement [4], which is lowered something like this:

using var r1 = new Resource1();
using var r2 = new Resource1();
/* some code */

// vvvvvvvvvvvvvvv

{
    Resource1 r1 = new Resource1();
    try
    {
        {
            Resource2 r2 = expression;
            try
            {
                /* some code */
            }
            finally
            {
                if (r2 != null) ((IDisposable)r2).Dispose();
            }
        }
    }
    finally
    {
        if (r1 != null) ((IDisposable)r1).Dispose();
    }
}

IDisposable.Dispose() can be called multiple times (though this is discouraged), so your implementation of this interface must be able to handle this. Finalizer should call the Dispose() function as a safety net.

[0]: https://golang.org/pkg/runtime/#SetFinalizer
[1]: https://docs.microsoft.com/en-us/dotnet/csharp/language-reference/language-specification/classes#destructors
[2]: https://docs.microsoft.com/en-us/dotnet/csharp/programming-guide/classes-and-structs/destructors
[3]: https://docs.microsoft.com/en-us/dotnet/api/system.idisposable?view=net-5.0
[4]: https://docs.microsoft.com/en-us/dotnet/csharp/language-reference/keywords/using-statement

On Thursday, 18 March 2021 at 12:27:56 UTC, Petar Kirov [ZombineDev] wrote:

>
> Just implementation deficiency. I think it is fixable with some refactoring of the GC pipeline. One approach would be, (similar to other language implementations - see below), that GC-allocated objects with destructors should be placed on a queue and their destructors be called when the GC has finished the collection. Afterwards, the GC can release their memory during the next collection.

As I understand, finalizers in D are run because the GC needs more memory *now*. Deferring release of memory until the next collection would defeat the purpose. We would need to decouple collection cycles from allocation. Am I missing something?

On 3/18/21 8:55 AM, Mike Parker wrote:
> On Thursday, 18 March 2021 at 12:27:56 UTC, Petar Kirov [ZombineDev] wrote:
> 
>>
>> Just implementation deficiency. I think it is fixable with some refactoring of the GC pipeline. One approach would be, (similar to other language implementations - see below), that GC-allocated objects with destructors should be placed on a queue and their destructors be called when the GC has finished the collection. Afterwards, the GC can release their memory during the next collection.
> 
> As I understand, finalizers in D are run because the GC needs more memory *now*. Deferring release of memory until the next collection would defeat the purpose. We would need to decouple collection cycles from allocation. Am I missing something?

I think this is the proper way to look at it. We are running a collection cycle because more memory is needed. If you allocate inside the GC, likely you would trigger another GC.

However, there are probably ways around this. For instance, you can allocate memory without triggering a GC, and we can probably try that instead.

AIUI, the stop-the-world phase is only for scanning. Once scanning is done, there is nothing to say we can't change the pool data while looking for blocks to finalize.

Most likely, some of them will free up blocks that then can be used by a finalizer allocation.

Would be a good SAOC project.

-Steve

On Thursday, 18 March 2021 at 12:21:46 UTC, Mike Parker wrote:
> I actually don't agree with that. I'll be discussion the solution in the next article:
>
> if(!GC.inFinalizer) {
> ...
> }
>
> It's perfectly fine to perform GC operations in destructors when they aren't invoked by the GC.

Could we at least add some guard in the GC that notifies the user of the reason for getting an exception, preferrably including a source position, when trying to allocate in a destructor run during finalization? Not getting an explanation has stolen hours of my development time on several occasion. And likely happen in the future for other users aswell potentially making them abandon D for other languages.

On Thursday, 18 March 2021 at 12:27:56 UTC, Petar Kirov [ZombineDev] wrote:
> On Thursday, 18 March 2021 at 09:21:27 UTC, Per Nordlöw wrote:
>> [...]
>
> Just implementation deficiency. I think it is fixable with some refactoring of the GC pipeline. One approach would be, (similar to other language implementations - see below), that GC-allocated objects with destructors should be placed on a queue and their destructors be called when the GC has finished the collection. Afterwards, the GC can release their memory during the next collection.
>
> [...]

Small correction, since .NET 5 / C# 9, implementing IDisposable isn't required if the Dispose() method is available.

This is done as performance improvement for using structs with determistic destruction and avoid implicit convertions to references when interfaces are used.

On Thursday, 18 March 2021 at 12:55:17 UTC, Mike Parker wrote:
> On Thursday, 18 March 2021 at 12:27:56 UTC, Petar Kirov [ZombineDev] wrote:
>
>>
>> Just implementation deficiency. I think it is fixable with some refactoring of the GC pipeline. One approach would be, (similar to other language implementations - see below), that GC-allocated objects with destructors should be placed on a queue and their destructors be called when the GC has finished the collection. Afterwards, the GC can release their memory during the next collection.

This need to improve which allow to call free memory in destructor
For realloc..., when size = 0, it needs to call free....

https://github.com/dlang/druntime/blob/3a32cc0305d4dd066f719d4c2df97337c86ea7ff/src/core/internal/gc/impl/conservative/gc.d#L444

vs

https://github.com/dlang/druntime/blob/3a32cc0305d4dd066f719d4c2df97337c86ea7ff/src/core/internal/gc/impl/conservative/gc.d#L678

On 3/18/21 5:21 AM, Per Nordlöw wrote:
> On Thursday, 4 March 2021 at 13:54:48 UTC, Mike Parker wrote:
>> The blog:
>> https://dlang.org/blog/2021/03/04/symphony-of-destruction-structs-classes-and-the-gc-part-one/ 
>>
> 
> Btw, what is the motive behind D's GC not being able to correctly handle GC allocations in  class destructors.
> 
> Is it by design or because of limitations in D's current GC implementation?
> 
> And how does this relate to exception-throwing destructors in other managed languages such as C# and Go; are they forbidden or allowed and safe thanks to a more resilient GC?

It is a frustrating rough edge, esp for non-experts; I cut myself when I tried to use a custom logging function (which of course GC allocates) in class destructors.