Hi all,

I was looking at D as the next language to use in my hobby projects, but the
"conservative GC" part in the language spec
(http://dlang.org/spec/garbage.html) looks a bit concerning. I'm wondering what
are the implications of the fact that current GC is a Boehm-style conservative
GC rather than a precise one, I've never worked with a conservative GC before.
Are there any disallowed memory operations? Can I break things by not following
some unchecked rules etc. ? How often does it leak? Do I need to be careful
with some operations to avoid leaks? Is a precise GC in the roadmap? any kind
of comments on the GC would be really appreciated.

Thanks

On Wednesday, 1 February 2017 at 06:58:43 UTC, osa1 wrote:
> I'm wondering what
> are the implications of the fact that current GC is a Boehm-style conservative
> GC rather than a precise one, I've never worked with a conservative GC before.

The GC isn't competitive with the ones you find in GC languages (Java, Go etc). E.g. Go is now aiming for GC pauses in the microseconds range.

Resource management in D is rather lacklustre, even C++ does better imho. D seems to move towards using thread local ref-counting and making GC optional. I guess that would be ok on cpus with few cores, but not really adequate on many core CPUs.

On Wednesday, 1 February 2017 at 09:40:17 UTC, Ola Fosheim Grøstad wrote:
> On Wednesday, 1 February 2017 at 06:58:43 UTC, osa1 wrote:
>> I'm wondering what
>> are the implications of the fact that current GC is a Boehm-style conservative
>> GC rather than a precise one, I've never worked with a conservative GC before.
>
> The GC isn't competitive with the ones you find in GC languages (Java, Go etc). E.g. Go is now aiming for GC pauses in the microseconds range.
>
> Resource management in D is rather lacklustre, even C++ does better imho. D seems to move towards using thread local ref-counting and making GC optional. I guess that would be ok on cpus with few cores, but not really adequate on many core CPUs.

Thanks for the answer. Could you elaborate on the lacklustre part? It's fine if I have to do manual memory management, but I don't want any leaks. Ideally I'd have a precise GC + RAII style resource management when needed.

On Wednesday, 1 February 2017 at 09:50:42 UTC, osa1 wrote:
> Thanks for the answer. Could you elaborate on the lacklustre part? It's fine if I have to do manual memory management, but I don't want any leaks. Ideally I'd have a precise GC + RAII style resource management when needed.

Rust, Go, Java, Swift etc have a single memory management scheme which is used by libraries and mostly enforced by the compiler.

In C++ you tend to go with unique ownership and occasional shared ownership with the ability to have weak pointers and swap out objects without updating pointer, and there is an easy transition form old ad-hoc ownership to shared (the reference counter is in a separate object). It is not enforced by the compiler, but C++ is moving towards having dedicated tools for checking correctness.

In D the goal is to have safety enforced by the compiler, but it isn't quite there yet and what is on the map for leak free resource management seems a simple reference counting mechanism (simpler than swift?) with refcount embedded in objects (like intrusive_ptr in Boost except the compiler is intended to be better at optimizing unnecessary updates of the reference count).

On Wednesday, 1 February 2017 at 06:58:43 UTC, osa1 wrote:
> Are there any disallowed memory operations?

Currently can't touch GC from destructor during collection. Another concern is interoperability with C-allocated memory: GC knows nothing about C heap.

> How often does it leak?

Leaks are likely in 32-bit processes and unlikely in 64-bit processes. See e.g. https://issues.dlang.org/show_bug.cgi?id=15723

> Do I need to be careful with some operations to avoid leaks?

Leaks happen only due to false pointers. But data allocated in GC with new operator and known to have no pointers (e.g. strings) is not scanned. Premature collection happen when GC doesn't see a pointer to the allocated data, happens when such pointer is put in a memory GC doesn't see, like C heap.

> Is a precise GC in the roadmap?

There's an effort on it: https://forum.dlang.org/post/hdwwkzqswwtffjehenjt@forum.dlang.org

> It's fine if I have to do manual memory management, but I don't want any leaks.

If you manually deallocate memory, it gets deallocated for sure, shouldn't leak.
Comparing to java, D GC trades GC performance for code execution performance, which can result in better overall performance when you don't allocate much and worse performance for allocation-heavy code that java GC is optimized for.

On Friday, 3 February 2017 at 10:49:00 UTC, Kagamin wrote:
> Leaks are likely in 32-bit processes and unlikely in 64-bit processes. See e.g. https://issues.dlang.org/show_bug.cgi?id=15723

This looks pretty bad. I think I'll consider something else until D's memory management story gets better. This is sad because the language otherwise looks quite good, and I'd love to try assertions, contracts, scope guards, macros etc.

On Friday, 3 February 2017 at 11:36:26 UTC, osa1 wrote:
> On Friday, 3 February 2017 at 10:49:00 UTC, Kagamin wrote:
>> Leaks are likely in 32-bit processes and unlikely in 64-bit processes. See e.g. https://issues.dlang.org/show_bug.cgi?id=15723
>
> This looks pretty bad. I think I'll consider something else until D's memory management story gets better. This is sad because the language otherwise looks quite good, and I'd love to try assertions, contracts, scope guards, macros etc.

you can use less auto GC. use the RC to replace the GC.
https://github.com/huntlabs/SmartRef

On Wednesday, 1 February 2017 at 06:58:43 UTC, osa1 wrote:
> I'm wondering what
> are the implications of the fact that current GC is a Boehm-style conservative
> GC rather than a precise one, I've never worked with a conservative GC before.
> Are there any disallowed memory operations? Can I break things by not following
> some unchecked rules etc. ? How often does it leak? Do I need to be careful
> with some operations to avoid leaks?

Here's some practical perspective from someone who released a 32-bit video processing app in D with thousands of users.
When developing with GC in D you need to keep in mind 3 key things:

1) The GC will treat some random stack data as possible pointers, and some of those false pointers will accidentally point to some places in the heap, so for any object in GC heap there is a probability that GC will think it's alive (used) even when it's not, and this probability is directly proportional to the size of your object.

2) Each GC iteration scans the whole GC heap, so the larger your managed heap, the slower it gets.

Main consequence of 1 and 2: don't store large objects (images, big file chunks etc.) in the GC heap, use other allocators for them. Leave GC heap just for the small litter. This way you practically don't leak and keep GC pauses short.

3) GC will call destructors (aka finalizers) for the objects that have them, and during the GC phase no allocations are allowed. Also, since you don't know in which order objects are collected, accessing other objects from a destructor is a bad idea, those objects might be collected already.

Main consequence of 3: don't do silly things in destructor (like throwing exceptions or doing other operations that might allocate), try avoiding using the destructors at all, if possible. They may be used to ensure you release your resources, but don't make it the primary and only way to release them, since some objects might leak and their destructors won't be called at all.

If you follow these principles, your app will be fine, it's not hard really.

On Saturday, 4 February 2017 at 11:09:21 UTC, thedeemon wrote:
> On Wednesday, 1 February 2017 at 06:58:43 UTC, osa1 wrote:
>> I'm wondering what
>> are the implications of the fact that current GC is a Boehm-style conservative
>> GC rather than a precise one, I've never worked with a conservative GC before.
>> Are there any disallowed memory operations? Can I break things by not following
>> some unchecked rules etc. ? How often does it leak? Do I need to be careful
>> with some operations to avoid leaks?
>
> Here's some practical perspective from someone who released a 32-bit video processing app in D with thousands of users.
> When developing with GC in D you need to keep in mind 3 key things:
>
> 1) The GC will treat some random stack data as possible pointers, and some of those false pointers will accidentally point to some places in the heap, so for any object in GC heap there is a probability that GC will think it's alive (used) even when it's not, and this probability is directly proportional to the size of your object.
>
> 2) Each GC iteration scans the whole GC heap, so the larger your managed heap, the slower it gets.
>
> Main consequence of 1 and 2: don't store large objects (images, big file chunks etc.) in the GC heap, use other allocators for them. Leave GC heap just for the small litter. This way you practically don't leak and keep GC pauses short.
>
> 3) GC will call destructors (aka finalizers) for the objects that have them, and during the GC phase no allocations are allowed. Also, since you don't know in which order objects are collected, accessing other objects from a destructor is a bad idea, those objects might be collected already.
>
> Main consequence of 3: don't do silly things in destructor (like throwing exceptions or doing other operations that might allocate), try avoiding using the destructors at all, if possible. They may be used to ensure you release your resources, but don't make it the primary and only way to release them, since some objects might leak and their destructors won't be called at all.
>
> If you follow these principles, your app will be fine, it's not hard really.

Honestly this still sounds horrible. I'd be OK with any of these two:

- Percise GC, no manual management, no RAII or destructors etc.
- Manual GC, RAII and destructors, smart pointers.

but this:

- Automatic but conservative. Can leak at any time. You have to implement manual management (managed heaps) to avoid leaks. Leaks are hard to find as any heap value may be causing it.

is the worst of both worlds. I'm surprised that D was able to come this far with this.

On Saturday, 4 February 2017 at 12:56:55 UTC, osa1 wrote:
> I'm surprised that D was able to come this far with this.

It's used mostly for server software. Things are moving to 64 bit, so this will be less of an issue.