On Tuesday, 20 June 2017 at 02:23:48 UTC, safety0ff wrote:
> On Monday, 19 June 2017 at 22:35:42 UTC, Dmitry Olshansky wrote:
>> My take on D's GC problem, also spoiler - I'm going to build a new one soonish.
>>
>> http://olshansky.me/gc/runtime/dlang/2017/06/14/inside-d-gc.html
>>
>> ---
>> Dmitry Olshansky
>
> Good overview, however:
> the binary search pool lookup is used because it naturally supports variable sized pools.
> IMHO, simply concluding "A hash table could have saved quite a few cycles." glosses over the issue of handling variable sizes.

Pools are granular to 256kb irc, so the trick is to keep them 256kb aligned in memory. Then a map from 256kb chunks to pools is easily created.


---
Dmitry Olshansky

On Monday, 19 June 2017 at 22:50:05 UTC, Adam D. Ruppe wrote:
> What is it about Windows that makes you call it a distant possibility? Is it just that you are unfamiliar with it or is there some specific OS level feature you plan on needing?

This is mostly because I wanted to abuse lazy commit of POSIX. Now that I think of it Windows is mostly ok, except for the fork trick used in concurrent GC. As Vladimir pointed out on Windows there are other ways to do it but they are more involved.

---
Dmitry Olshansky

On Monday, 19 June 2017 at 23:52:16 UTC, Vladimir Panteleev wrote:
> On Monday, 19 June 2017 at 22:35:42 UTC, Dmitry Olshansky wrote:
>> My take on D's GC problem, also spoiler - I'm going to build a new one soonish.
>
> Looks like I'm not the only one itching to have a go at D's GC :) This will very likely be my DConf 2018 project. However, I have slightly different plans:

I see no problem in eventually uniting our efforts.

>
> - The GC should be usable as a library (mainly to facilitate testing).
> - Support for all platforms D already supports from the start.
> - Use design-by-introspection when applicable and design-by-contract elsewhere to split the design into modular components.

Nice. A pool could have many different structures, the collector could then introspect on that. Sadly this almost doubles the effort so I will not go there.

> - Make the GC configurable (using policies) and swappable at runtime. (No need to get clever, just treat previous implementation's pools as opaque void[]).
> - Support concurrency on Windows via anonymous memory-mapped files.

Yeah I recall Rainer and myself discussing this approach, it had some downside such as you need to remap each pool individually. Still doable.

> - Support generational collection using write barriers implemented through memory protection.

Super slow sadly. That being said I belive D is just fine without generational GC. The generational hypothesis just doesn't hold to the extent it holds in say Java. My hypothesis is that most performance minded applications already allocate temporaries using region allocator of sorts (or using C heap).

> - Integrate existing GC work - don't reinvent the wheel.
> - More, much more debugging facilities! Integrate Diamond and Valgrind interoperability.

 I could use help on thus one.

> - Gray-marking and compacting.
> - Still need to look at immix.
>
> I have some past work that I'd like to integrate (an experimental generational GC I wrote like 9 years ago for D1, Diamond, and Valgrind integration I have in a fork somewhere.)

---
Dmitry Olshansky

On Monday, 19 June 2017 at 23:10:43 UTC, Ali Çehreli wrote:
> On 06/19/2017 03:35 PM, Dmitry Olshansky wrote:
>> My take on D's GC problem, also spoiler - I'm going to build a new one
>> soonish.
>>
>> http://olshansky.me/gc/runtime/dlang/2017/06/14/inside-d-gc.html
>>
>> ---
>> Dmitry Olshansky
>
> Very informative, thanks.
>
> However, I can think of many reasons like appreciation the efforts of the original authors to tone it down a little bit like changing "mistake" to "optimization opportunity", "criticism" to "observation", etc. :)
>

I could call it a problem :) Still one reason I didn't go to D blog to post this is because it's a critique followed by a promise of action though.

> Ali

---
Dmitry Olshansky

On Monday, 19 June 2017 at 23:39:54 UTC, H. S. Teoh wrote:
> On Mon, Jun 19, 2017 at 10:35:42PM +0000, Dmitry Olshansky via Digitalmars-d wrote:
>> My take on D's GC problem, also spoiler - I'm going to build a new one soonish.
>> 
>> http://olshansky.me/gc/runtime/dlang/2017/06/14/inside-d-gc.html
> [...]
>
> Very interesting indeed!
>
> One question about killing the no interior pointer attribute: would this be problematic for 32-bit platforms? And if so, what do you plan to do about it?  Keep the current GC as version(32bit) and your new version as version(64bit)?

Yeah if said 32-bit application makes use of no interior pointer attribute then using old gc is an option. I have no plans for this broken attribute.

>
> One (potentially crazy) idea that occurred to me while reading your post is TLS allocations. I haven't thought through the details of how this would interact with the existing language yet, but would it make sense for some allocations that you know will never be shared across threads to be allocated in a thread-local pool instead of the global pool? I.e., in addition to the global set of memory pools you also have thread-local memory pools. Then you could potentially run collections per-thread rather than stop-the-world.
>
This needs spec updateon interaction between TLS and shared, in particular the current trend of lock + cast away shared is problematic. Also the implicit cast to immutable of a result of unique expression.

>
> On Mon, Jun 19, 2017 at 10:50:05PM +0000, Adam D. Ruppe via Digitalmars-d wrote:
>> What is it about Windows that makes you call it a distant possibility? Is it just that you are unfamiliar with it or is there some specific OS level feature you plan on needing?
>
> He mentioned the "fork trick", which I assume refers to how Linux's implementation of fork() uses copy-on-write rather than immediately duplicating the parent process' memory structures.  There was a D1 GC some time ago that depended on this behaviour to speed up the collection cycle.  AFAIK, Windows does not have equivalent functionality to this.

To the best of my knowledge all of D's current target OSes support this save for Windows.

>
> T

On 6/20/2017 12:04 AM, Nicholas Wilson wrote:
> On Monday, 19 June 2017 at 22:35:42 UTC, Dmitry Olshansky wrote:
>> http://olshansky.me/gc/runtime/dlang/2017/06/14/inside-d-gc.html
> This was posted on reddit: https://www.reddit.com/r/programming/comments/6ic52d/inside_ds_gc/

Also on hacker news.

On 2017-06-20 06:37, ketmar wrote:

> it is higly depends of undocumented windows internals, and not portable between windows versions. more-or-less working implementations of `fork()` were existed at least since NT3 era, but nobody considered 'em as more than a PoC, and even next service pack can break everything.

I'm wondering what Windows 10 is using to implement "fork" for Windows Subsystem for Linux. If it's using these internal functions or something else.

-- 
/Jacob Carlborg

On 2017-06-20 01:52, Vladimir Panteleev wrote:

> - More, much more debugging facilities! Integrate Diamond and Valgrind interoperability.

Don't for get the Clang sanitizers, assuming they work using LDC.

-- 
/Jacob Carlborg

On 20/06/2017 12:41 PM, Jacob Carlborg wrote:
> On 2017-06-20 06:37, ketmar wrote:
> 
>> it is higly depends of undocumented windows internals, and not portable between windows versions. more-or-less working implementations of `fork()` were existed at least since NT3 era, but nobody considered 'em as more than a PoC, and even next service pack can break everything.
> 
> I'm wondering what Windows 10 is using to implement "fork" for Windows Subsystem for Linux. If it's using these internal functions or something else.

It wouldn't surprise me to learn that it was a posix layer specific syscall, meaning we can't from a native Windows process.

On 2017-06-20 06:54, ketmar wrote:

> "...the dubious optimization of no interior pointers..."
> 
> this is the ONLY (i emphasise it!) way i were able to make my e-mail and irc clients to not leak memory, and keep using GC. on 32-bit systems false pointers *is* a problem, and NO_INTERIOR really helps.
> 
> turning NO_INTERIOR into something dog-slow (or noop) will make D unusable on 32-bit systems for anything more complex than helloworld and throwaway scripts. particularly, any app that should work for weeks or monthes without restart (yep, i want my mail client to Just Work, and i'm not rebooting my PC that often) will be *forced* to ditch GC.
> 
> while NO_INTERIOR requires some coding discipline, it is invaluable in IRL apps.

You need to move to 64bit. Apple is already deprecating support for 32bit apps and after the next version of macOS (High Sierra) they're going to remove the support for 32bit apps.

-- 
/Jacob Carlborg