On Mon, Jun 19, 2017 at 10:35:42PM +0000, Dmitry Olshansky via Digitalmars-d wrote:
> My take on D's GC problem, also spoiler - I'm going to build a new one soonish.
> 
> http://olshansky.me/gc/runtime/dlang/2017/06/14/inside-d-gc.html
[...]

Very interesting indeed!

One question about killing the no interior pointer attribute: would this be problematic for 32-bit platforms? And if so, what do you plan to do about it?  Keep the current GC as version(32bit) and your new version as version(64bit)?

One (potentially crazy) idea that occurred to me while reading your post is TLS allocations. I haven't thought through the details of how this would interact with the existing language yet, but would it make sense for some allocations that you know will never be shared across threads to be allocated in a thread-local pool instead of the global pool? I.e., in addition to the global set of memory pools you also have thread-local memory pools. Then you could potentially run collections per-thread rather than stop-the-world.

For example, if you have a bunch of threads that call a function that does a bunch of short-lived allocations that are not shared across threads, it seems to wasteful to have these allocations add to the global GC load. Why not have them go into a local pool that can be collected per-thread?  Of course, whether the current language can take advantage of this is another matter.  Perhaps if the function is pure and returns scope, then you know any allocation it makes can't possibly be shared with other threads, or something like that...


On Mon, Jun 19, 2017 at 10:50:05PM +0000, Adam D. Ruppe via Digitalmars-d wrote:
> What is it about Windows that makes you call it a distant possibility? Is it just that you are unfamiliar with it or is there some specific OS level feature you plan on needing?

He mentioned the "fork trick", which I assume refers to how Linux's implementation of fork() uses copy-on-write rather than immediately duplicating the parent process' memory structures.  There was a D1 GC some time ago that depended on this behaviour to speed up the collection cycle.  AFAIK, Windows does not have equivalent functionality to this.

(Well, for that matter, I'm not sure Posix in general has this feature either, since AFAIK it's Linux-specific. But I surmise that modern-day *nix flavors probably have adopted this in one way or another, since otherwise the very common pattern of fork-and-exec would be inordinately expensive -- copying all the parent's pages only to replace them all pretty much immediately.)


T

-- 
Give me some fresh salted fish, please.

H. S. Teoh wrote:

> He mentioned the "fork trick", which I assume refers to how Linux's
> implementation of fork() uses copy-on-write rather than immediately
> duplicating the parent process' memory structures.  There was a D1 GC
> some time ago that depended on this behaviour to speed up the collection
> cycle.

and it was even ported to D2, and worked. sadly, using `fork()` has it's own set of problems -- `fork()` itself is in no way  a flawless expirience. like you can fork while other thread is inside glibc's `malloc()`, and BOOM! alot of glibc is locked forever, as `malloc()` lock is never released in child process. some other libraries may try to intercept `fork()` to do unnecessary "cleanup", and so on.

so using "forking GC" require alot of discipline in coding and library use, or it will be an endless source of heisenbugs.

new linux kernels got userfaultfd API (so code can simply `select()` on fd, and process protection violation from `mprotect()` without tricks with signals), but... to much of my joy and hapiness, the proposed API was just fine to create GC with mprotect barriers, and the final API that was included gladly omited that exactly necessary API call which allows to make it happen. great work, yeah. it may changed since then, tho, i didn't rechecked.

On Tuesday, 20 June 2017 at 04:35:27 UTC, ketmar wrote:
> H. S. Teoh wrote:
>
>> He mentioned the "fork trick", which I assume refers to how Linux's
>> implementation of fork() uses copy-on-write rather than immediately
>> duplicating the parent process' memory structures.  There was a D1 GC
>> some time ago that depended on this behaviour to speed up the collection
>> cycle.
>
> and it was even ported to D2, and worked. sadly, using `fork()` has it's own set of problems -- `fork()` itself is in no way  a flawless expirience. like you can fork while other thread is inside glibc's `malloc()`, and BOOM! alot of glibc is locked forever, as `malloc()` lock is never released in child process. some other libraries may try to intercept `fork()` to do unnecessary "cleanup", and so on.

Since we are in control of what child does I see this as no issue. Just call mmap and do bump a pointer allocation.