January 25, 2022
On Tuesday, 25 January 2022 at 07:22:36 UTC, Elronnd wrote:
> On Tuesday, 25 January 2022 at 07:13:41 UTC, Paulo Pinto wrote:
>> ARC will also not compete, unless one goes the extra mile of making the compiler ARC aware, elide retain/release calls, do cascade deletions in background threads, take care on cascade deletions to avoid stack overflows on destructor calls, provide multicore friendly versions of them,.....
>
> Indeed.  See Bacon et al, 'Unified Theory of Garbage Collection': increasingly sophisticated RC approaches tracing (and vice versa).  So it's a bit strange to assume we can do one but not the other.  And tracing makes a better starting point due to the generational hypothesis.

Only if you take the "deferred" RC route, which Swift/Rust/C++/Nim do not! Without the "deferred" aspect RC remains quite a different beast. Different algorithm, different runtime profiles, different memory consumptions, enables different optimizations and of course different problems.
January 25, 2022
On Tuesday, 25 January 2022 at 09:42:25 UTC, Araq wrote:
> On Tuesday, 25 January 2022 at 07:22:36 UTC, Elronnd wrote:
>> On Tuesday, 25 January 2022 at 07:13:41 UTC, Paulo Pinto wrote:
>>> ARC will also not compete, unless one goes the extra mile of making the compiler ARC aware, elide retain/release calls, do cascade deletions in background threads, take care on cascade deletions to avoid stack overflows on destructor calls, provide multicore friendly versions of them,.....
>>
>> Indeed.  See Bacon et al, 'Unified Theory of Garbage Collection': increasingly sophisticated RC approaches tracing (and vice versa).  So it's a bit strange to assume we can do one but not the other.  And tracing makes a better starting point due to the generational hypothesis.
>
> Only if you take the "deferred" RC route, which Swift/Rust/C++/Nim do not! Without the "deferred" aspect RC remains quite a different beast. Different algorithm, different runtime profiles, different memory consumptions, enables different optimizations and of course different problems.

Indeed, that was kind of my point, unless one is willing to invest the required resources, a bit like you guys are doing with Nim, a RC implementation will not magically outperform modern tracing GC, only naive implementations of tracing GCs.
January 25, 2022

On Tuesday, 25 January 2022 at 09:42:25 UTC, Araq wrote:

>

Only if you take the "deferred" RC route, which Swift/Rust/C++/Nim do not!

What do you mean by "deferred"? RC increment when taking a reference from the heap, but not when the reference is taken from the stack + periodic stack scanning?

Actually, in C++ (and to some extent in Objective-C) you minimize reference counting by using programmer knowledge. You increment when you take it from the heap and from thereon you use a borrowed (raw) pointer down the call tree.

Anyway, the key problem is not solved by "deferred RC". The key problem can only be solved by segmenting the heap in the type system.

January 25, 2022
On Tuesday, 25 January 2022 at 03:37:57 UTC, Elronnd wrote:
> Apropos recent discussion, here is a serious question: would you pay for either of these?
>
> - High-throughput/scalable gc.  High sustained allocation rates, large heaps, many cores, compacting&generational
>
> - Concurrent gc.  No pauses

If it was delivered the foundation would probably be happy to give at least some money (not that we have an unlimited supply), on the condition that it were open-sourced.

Speaking as a user of D, I wouldn't use a forked compiler should one be required.
January 25, 2022
On Tuesday, 25 January 2022 at 10:19:48 UTC, max haughton wrote:
> On Tuesday, 25 January 2022 at 03:37:57 UTC, Elronnd wrote:
>> Apropos recent discussion, here is a serious question: would you pay for either of these?
>>
>> - High-throughput/scalable gc.  High sustained allocation rates, large heaps, many cores, compacting&generational
>>
>> - Concurrent gc.  No pauses
>
> If it was delivered the foundation would probably be happy to give at least some money (not that we have an unlimited supply), on the condition that it were open-sourced.
>

A contract for this sort of work is always a possibility. That's what the HR fund is for:

https://www.flipcause.com/secure/cause_pdetails/NTUxOTc=

Anyone serious about doing a project like this (any relatively complex project, not just a new GC) can get in touch and we can discuss it. I'm not saying the foundation *would* pay for any particular project, but the discussion and possibly a meeting could lead to that if agreement is reached that it's worth doing.
January 25, 2022

On Tuesday, 25 January 2022 at 09:58:14 UTC, Ola Fosheim Grøstad wrote:

>

On Tuesday, 25 January 2022 at 09:42:25 UTC, Araq wrote:

>

Only if you take the "deferred" RC route, which Swift/Rust/C++/Nim do not!

What do you mean by "deferred"? RC increment when taking a reference from the heap, but not when the reference is taken from the stack + periodic stack scanning?

That's what it means, yes.

>

Actually, in C++ (and to some extent in Objective-C) you minimize reference counting by using programmer knowledge. You increment when you take it from the heap and from thereon you use a borrowed (raw) pointer down the call tree.

Anyway, the key problem is not solved by "deferred RC". The key problem can only be solved by segmenting the heap in the type system.

I didn't claim that deferred RC is a "solution". My post was a reply to another post.

January 25, 2022

On Tuesday, 25 January 2022 at 09:58:14 UTC, Ola Fosheim Grøstad wrote:

>

On Tuesday, 25 January 2022 at 09:42:25 UTC, Araq wrote:

>

Only if you take the "deferred" RC route, which Swift/Rust/C++/Nim do not!

What do you mean by "deferred"? RC increment when taking a reference from the heap, but not when the reference is taken from the stack + periodic stack scanning?

Actually, in C++ (and to some extent in Objective-C) you minimize reference counting by using programmer knowledge. You increment when you take it from the heap and from thereon you use a borrowed (raw) pointer down the call tree.

...

Pity that programmer knowledge can't do much to ABI requirements.

http://www.open-std.org/jtc1/sc22/wg21/docs/papers/2019/p1116r0.pdf

Also that trick only works in single developer code bases, good luck not introducing a memory corruption some months/years down the line.

My experience with COM proves that is generally what happens when one decides to be smart about manually optimizing AddRef/Release calls.

January 25, 2022

On Tuesday, 25 January 2022 at 10:56:50 UTC, Paulo Pinto wrote:

>

Pity that programmer knowledge can't do much to ABI requirements.

C++ has an ABI?

>

http://www.open-std.org/jtc1/sc22/wg21/docs/papers/2019/p1116r0.pdf

So this is basically about giving a shared_ptr the semantics of a unique_ptr.

That is not required for what we are talking about here.

>

Also that trick only works in single developer code bases, good luck not introducing a memory corruption some months/years down the line.

No, this is not a big issue if you create proper ADTs. The issue is that it is very difficult for a compiler to distinguish between objects that "wrap ownership" around a data-structure and nodes within a datastructure; in particular what happens to ownership when those nodes are rearranged.

However, the programmer should have good and solid knowledge about this, so you only need to increment on the root-object if you know that nodes do not escape below a point in the call tree. (And you might be able to wrap this in a reference-type specific to the ADT).

Anyway, in C++ you tend almost always to use unique_ptr, shared_ptr is the exception. So you usually have very few shared_ptrs and therefore they are not all that hard to reason about.

For a language like D you could have ARC + borrow checker + the ability to constrain ARC-pointers (to get a unique_ptr) for shared objects and something GC-like for objects local to actors/tasks.

January 25, 2022

On Tuesday, 25 January 2022 at 11:47:26 UTC, Ola Fosheim Grøstad wrote:

>

On Tuesday, 25 January 2022 at 10:56:50 UTC, Paulo Pinto wrote:

>

Pity that programmer knowledge can't do much to ABI requirements.

C++ has an ABI?

Yes, the one from the compiler and OS vendor shipping their C++ compilers on their platform.

> >

http://www.open-std.org/jtc1/sc22/wg21/docs/papers/2019/p1116r0.pdf

So this is basically about giving a shared_ptr the semantics of a unique_ptr.

That is not required for what we are talking about here.

>

Also that trick only works in single developer code bases, good luck not introducing a memory corruption some months/years down the line.

No, this is not a big issue if you create proper ADTs. The issue is that it is very difficult for a compiler to distinguish between objects that "wrap ownership" around a data-structure and nodes within a datastructure; in particular what happens to ownership when those nodes are rearranged.

However, the programmer should have good and solid knowledge about this, so you only need to increment on the root-object if you know that nodes do not escape below a point in the call tree. (And you might be able to wrap this in a reference-type specific to the ADT).

Anyway, in C++ you tend almost always to use unique_ptr, shared_ptr is the exception. So you usually have very few shared_ptrs and therefore they are not all that hard to reason about.

As someone that does security as part of DevOps assignments, what the programmers should be able to do, and what they actually deploy into production isn't always the same.

That is how we end up with the 70% magical number being quoted from several security reports.

>

For a language like D you could have ARC + borrow checker + the ability to constrain ARC-pointers (to get a unique_ptr) for shared objects and something GC-like for objects local to actors/tasks.

In theory yes, in practice someone has to put down the money to make it happen and ensure that the performance gains are worth the money spent into it.

January 25, 2022
On Tuesday, 25 January 2022 at 03:37:57 UTC, Elronnd wrote:
> Apropos recent discussion, here is a serious question: would you pay for either of these?

No. D's GC is already plenty good enough right now.