Incremental garbage collection (page 3)

January 21, 2022

Re: Incremental garbage collection

Posted by Ola Fosheim Grøstad
in reply to rikki cattermole

Permalink

Ola Fosheim Grøstad

Posted in reply to rikki cattermole

Permalink

On Friday, 21 January 2022 at 11:38:10 UTC, rikki cattermole wrote:

On 22/01/2022 12:27 AM, Ola Fosheim Grøstad wrote:

On Friday, 21 January 2022 at 11:06:58 UTC, rikki cattermole wrote:

On 21/01/2022 11:58 PM, Ola Fosheim Grøstad wrote:

It would work for D, I guess.

Not likely, it would need a lot of work to bring that up to todays requirements.

What are the requirements?

That project was a masters thesis in 2009 running on 2002 era hardware.

Yes, but Notam and CCRMA tend to attract programmers that take audio seriously (electro acoustic computer music). So, I would not dismiss it because it is a mater thesis.

Size of data has changed, hardware has changed. It'll need work to prove it can even do the sort of workloads D has even in audio space.

Well, in his master thesis he covers the Boehm copy-heap collector and a collector based on barriers (which seems to use a lot of memory, but he claims it is essentially wait-free IIRC): https://www.duo.uio.no/handle/10852/8805

I haven't read the details, and maybe chapter 19 in the GC handbook is more interesting for you, but it is at least an example of a practical real time oriented approach to garbage collection. Which is in some sense is better than the high flying theories about what can be done in the GC department for D that tend to "poison" the forum discussions on the topic.

On Friday, 21 January 2022 at 02:17:34 UTC, Brad Roberts wrote: > > To make any progress towards a better GC it's going to have to be demonstrated. The GC is not as slow and annoying as it used to be. When I went to Dconf 2019 I was surprised to learn that it had been speed-up during the year (with both fork-GC and speed-up efforts by Rainer IIRC) but this wasn't advertised anywhere. IT's easy to be under the impression that the GC is the same than 10 years ago when it's not written anywhere what changes happened, you'd have to peruse all the release notes. Who knows the real timeline of GC improvement in D? What happened?

On Friday, 21 January 2022 at 12:45:42 UTC, Guillaume Piolat wrote: > On Friday, 21 January 2022 at 02:17:34 UTC, Brad Roberts wrote: >> >> To make any progress towards a better GC it's going to have to be demonstrated. > > The GC is not as slow and annoying as it used to be. When I went to Dconf 2019 I was surprised to learn that it had been speed-up during the year (with both fork-GC and speed-up efforts by Rainer IIRC) but this wasn't advertised anywhere. I agree. The GC run-time options are poorly documented. It's written nowhere that druntime can take specific arguments that change the GC behavior.

On Friday, 21 January 2022 at 12:26:42 UTC, Ola Fosheim Grøstad wrote: > It could, in theory, if you change the semantics of @nogc and annotate everything with @nogc that the collector must leave alone. You do not. See my other post where I clarified what I meant with this approach. > You also need to prevent scannable pointers to be available through unions You do not. You do need to do _something_ about unions, but you do need to disallow pointers in them. The spec says to pin objects which are in unions, which is a perfectly acceptable solution. There are also cleverer solutions which violate that clause of the spec but do not break any actual code. > There is also no way for D to force barriers on linked code, hence the need for a breaking language level change at some level. You can do it with name mangling. D is not binary compatible between releases. ------------------------------------ Bottom line: no source-level changes are necessary.

On Friday, 21 January 2022 at 12:52:02 UTC, Elronnd wrote: > You do need to do _something_ about unions, but you do need to disallow pointers in them. Err, this should read: 'you do _not_ need to disallow...'

On Friday, 21 January 2022 at 11:27:51 UTC, Ola Fosheim Grøstad wrote:

Makes sense to me for that particular niche. Might also work for embedded with the caveat that you need extra memory.

While it must be interesting, another solution you have in D is to deregister audio threads so that a GC pause would let them run free. The audio threads rarely owns GC root anyway.

Despite popular conception, it's indeed common to allocate from the audio-thread at first buffer, it can avoid some pessimization depending on incoming parameters that are only known at play time.

Amortized malloc is surprisingly cheap, you can merge allocation buffers, and you would do it only once at initialization. A lot of the audio buffers are also scoped in nature, with a lifetime that is the one of the processing treatment. So you can also leverage specialized allocators to mass release those buffers rapidly.

In Dplug plugins we got a GC back :) by integrating Wren as UI scripting language. It is expected to cause very few problems, as it has a max-heap size and doesn't touch audio.

On Friday, 21 January 2022 at 12:51:11 UTC, user1234 wrote: > I agree. The GC run-time options are poorly documented. It's written nowhere that druntime can take specific arguments that change the GC behavior. the page dlang.org/garbage has a header called "nowhere": https://dlang.org/spec/garbage.html#gc_config

Forums