On 2013-10-09 14:06:51 +0000, Johannes Pfau <nospam@example.com> said:

> Am Wed, 9 Oct 2013 09:38:12 -0400
> schrieb Michel Fortin <michel.fortin@michelf.ca>:
> 
>> I think the only way to make that work sanely is to create
>> another root object for ref-counted classes.
> 
> The problem here is we need a way to know that type X is refcounted,
> right? Couldn't we just use an attribute with the class or interface
> declaration:
> 
> @refcounted interface X{};
> @refcounted class Y {};
> class X1 : X;
> class Y1 : Y;
> 
> Now we know for sure that all these classes and SubClasses are
> refcounted.

The problem is that you can't cast Y to its base class Object (because you'd be casting an ARC pointer to a GC one and the reference count would not be maintained). Even calling Object's functions should be prohibited (because the "this" pointer is a GC pointer inside of Object's functions, and it could be leaked). So, de facto, by introducing all the necessary safety limitations, you've practically made Y a root class. The only things you can safely access from the base non-ref-counted class are its variables and its static functions.

And the same apply to interface: you can't mix them.

On top of that all that, if your goal is to go GC-free, you have to reinvent the whole standard library (so it uses ref-counted classes) and you still have to avoid the common pitfalls of implicit allocations (array concat for instance). So it's not terribly useful for someone whose goal is to avoid the GC.

-- 
Michel Fortin
michel.fortin@michelf.ca
http://michelf.ca

On 9 October 2013 23:06, Michel Fortin <michel.fortin@michelf.ca> wrote:

> On 2013-10-09 07:33:29 +0000, Manu <turkeyman@gmail.com> said:
>
>  Is there more to it? Cleaning up circular references I guess... what does
>> Apple do?
>>
>
> Apple implemented auto-nulling weak pointers for ARC (as well as __unsafe_unretained ones if you are not afraid of dangling pointers).
>
>  It's an uncommon edge case, so there's gotta be heaps of room for
>> efficient
>> solutions to that (afaik) one edge case. Are there others?
>>
>
> I don't know about you, but circular references are not rare at all in my code.
>
> Another solution that could be used for circular references is to have the stop-the-world GC we have now collect them. Reference counting could be used to free GC memory in advance (in the absence of circular references), so the GC would have less garbage to collect when it runs.
>
> Both solutions (week-autonulling and last-resort GC) are not mutually
> exclusive and can coexist.


I suspect there are plenty of creative possibilities that could be applied
to reducing the pool of potential circular-references to as small a pool as
possible.
It's simply not feasible to have machine scanning 4-6 gigabytes of
allocated memory looking for garbage.

On Wednesday, 9 October 2013 at 13:03:33 UTC, Michel Fortin wrote:
> On 2013-10-09 04:07:53 +0000, "deadalnix" <deadalnix@gmail.com> said:
>
>> That being said, isolated is probably something we want to add in the future.
>
> You'll hit a problem with immutable though. Immutable is implicitly shared, and immutable strings are everywhere!

Collecting immutable do not require to stop the application.

On Wednesday, 9 October 2013 at 14:17:44 UTC, Sean Kelly wrote:
> On Oct 9, 2013, at 4:30 AM, Jacob Carlborg <doob@me.com> wrote:
>> 
>>> On 2013-10-09 05:31, Walter Bright wrote:
>>> 
>>> Making this work is fraught with difficulty. It is normal behavior in D
>>> to create local data with new(), build a data structure, and then cast
>>> it to shared so it can be transferred to another thread. This will fail
>>> miserably if the data is allocated on a thread local heap.
>> 
>> I agree with Andrei here. Alternatively perhaps the runtime can move the data to a global pool if it's casted to shared.
>
> Generally not, since even D's precise GC is partially conservative.  It's also way more expensive than any cast should be. For better or worse, I think being able to cast data to shared means that we can't have thread-local pools. Unless a new attribute were introduced like "local" that couldn't ever be cast to shared, and that sounds like a disaster.

That isn't accurant. Allocator like tcmalloc use thread local info to allocate shared chunk of memory. What does matter is that the block is tagged as shared as far as the GC is oncerned.

Casting qualifier is a NOOP at machine level, so that won't be any slower.

On 9 October 2013 17:31, Walter Bright <newshound2@digitalmars.com> wrote:

> On 10/9/2013 12:29 AM, Manu wrote:
>
>> Does anyone here REALLY believe that a bunch of volunteer contributors can
>> possibly do what apple failed to do with their squillions of dollars and
>> engineers?
>> I haven't heard anybody around here propose the path to an acceptable
>> solution.
>> It's perpetually in the too-hard basket, hence we still have the same GC
>> as
>> forever and it's going nowhere.
>>
>
> What do you propose?
>

ARC. I've been here years now, and I see absolutely no evidence that the GC
is ever going to improve. I can trust ARC, it's predictable, I can control
it.
Also, proper support for avoiding the GC without severe inconvenience as
constantly keeps coming up. But I don't think there's any debate on that
one. Everyone seems to agree.

Am 09.10.2013 16:52, schrieb Manu:
> On 9 October 2013 23:06, Michel Fortin <michel.fortin@michelf.ca
> <mailto:michel.fortin@michelf.ca>> wrote:
>
>     On 2013-10-09 07:33:29 +0000, Manu <turkeyman@gmail.com
>     <mailto:turkeyman@gmail.com>> said:
>
>         Is there more to it? Cleaning up circular references I guess...
>         what does
>         Apple do?
>
>
>     Apple implemented auto-nulling weak pointers for ARC (as well as
>     __unsafe_unretained ones if you are not afraid of dangling pointers).
>
>         It's an uncommon edge case, so there's gotta be heaps of room
>         for efficient
>         solutions to that (afaik) one edge case. Are there others?
>
>
>     I don't know about you, but circular references are not rare at all
>     in my code.
>
>     Another solution that could be used for circular references is to
>     have the stop-the-world GC we have now collect them. Reference
>     counting could be used to free GC memory in advance (in the absence
>     of circular references), so the GC would have less garbage to
>     collect when it runs.
>
>     Both solutions (week-autonulling and last-resort GC) are not
>     mutually exclusive and can coexist.
>
>
> I suspect there are plenty of creative possibilities that could be
> applied to reducing the pool of potential circular-references to as
> small a pool as possible.
> It's simply not feasible to have machine scanning 4-6 gigabytes of
> allocated memory looking for garbage.

The Azul VM does not have a problem with it, as it does pauseless concurrent GC, while being used on online trading systems.

--
Paulo

Am 09.10.2013 16:30, schrieb Manu:
> On 9 October 2013 17:31, Walter Bright <newshound2@digitalmars.com> wrote:
>
>> On 10/9/2013 12:29 AM, Manu wrote:
>>
>>> Does anyone here REALLY believe that a bunch of volunteer contributors can
>>> possibly do what apple failed to do with their squillions of dollars and
>>> engineers?
>>> I haven't heard anybody around here propose the path to an acceptable
>>> solution.
>>> It's perpetually in the too-hard basket, hence we still have the same GC
>>> as
>>> forever and it's going nowhere.
>>>
>>
>> What do you propose?
>>
>
> ARC. I've been here years now, and I see absolutely no evidence that the GC
> is ever going to improve. I can trust ARC, it's predictable, I can control
> it.
> Also, proper support for avoiding the GC without severe inconvenience as
> constantly keeps coming up. But I don't think there's any debate on that
> one. Everyone seems to agree.
>

just a question:

how should ref count locking be done - only for shared?
all non shared pointers don't need to be thread-safe - or?

Am Wed, 9 Oct 2013 10:36:36 -0400
schrieb Michel Fortin <michel.fortin@michelf.ca>:

> > 
> > @refcounted interface X{};
> > @refcounted class Y {};
> > class X1 : X;
> > class Y1 : Y;
> > 
> > Now we know for sure that all these classes and SubClasses are refcounted.
> 
> The problem is that you can't cast Y to its base class Object (because you'd be casting an ARC pointer to a GC one and the reference count would not be maintained). Even calling Object's functions should be prohibited (because the "this" pointer is a GC pointer inside of Object's functions, and it could be leaked). So, de facto, by introducing all the necessary safety limitations, you've practically made Y a root class. The only things you can safely access from the base non-ref-counted class are its variables and its static functions.

I see. That's a good point why a separate object hirarchy is necessary.

> On top of that all that, if your goal is to go GC-free, you have to reinvent the whole standard library (so it uses ref-counted classes) and you still have to avoid the common pitfalls of implicit allocations (array concat for instance). So it's not terribly useful for someone whose goal is to avoid the GC.
> 

I don't think the standard lib is not that much of a problem, at least there's a way to avoid it (and some modules, std.algorithm, std.digest, std.uuid already are 99% GC free).

But if someone really wants to strip the GC _completely_ there's a huge issue with memory management of Exceptions.

Am 09.10.2013 16:30, schrieb Manu:
> On 9 October 2013 17:31, Walter Bright <newshound2@digitalmars.com
> <mailto:newshound2@digitalmars.com>> wrote:
>
>     On 10/9/2013 12:29 AM, Manu wrote:
>
>         Does anyone here REALLY believe that a bunch of volunteer
>         contributors can
>         possibly do what apple failed to do with their squillions of
>         dollars and engineers?
>         I haven't heard anybody around here propose the path to an
>         acceptable solution.
>         It's perpetually in the too-hard basket, hence we still have the
>         same GC as
>         forever and it's going nowhere.
>
>
>     What do you propose?
>
>
> ARC. I've been here years now, and I see absolutely no evidence that the
> GC is ever going to improve. I can trust ARC, it's predictable, I can
> control it.
> Also, proper support for avoiding the GC without severe inconvenience as
> constantly keeps coming up. But I don't think there's any debate on that
> one. Everyone seems to agree.

As someone that is in the sidelines and doesn't really use D, my opinion should not count that much, if at all.

However, rewriting D's memory management to be ARC based will have performance impact if the various D compilers aren't made ARC aware.

Then there is the whole point of rewriting phobos and druntime to use ARC instead of GC.

Will the return on investment pay off, instead of fixing the existing GC?

What will be the message sent to the outsiders wondering if D is stable enough to be adopted, and see these constant rewrites?

--
Paulo

On Oct 9, 2013, at 7:35 AM, Jacob Carlborg <doob@me.com> wrote:

> On 2013-10-09 15:51, Sean Kelly wrote:
> 
>> Generally not, since even D's precise GC is partially conservative.  It's also way more expensive than any cast should be. For better or worse, I think being able to cast data to shared means that we can't have thread-local pools. Unless a new attribute were introduced like "local" that couldn't ever be cast to shared, and that sounds like a disaster.
> 
> Since casting breaks the type system to begin with and is an advanced feature. How about providing a separate function that moves the object? It will be up to the user to call the function.

Okay so following that… it might be reasonable if the location of data keyed off the attribute set at construction.  So "new shared(X)" puts it in the shared pool.  Strings are a bit trickier though, since there's no way to specify the locality of the result of a string operation:

shared string x = a ~ b ~ c;

And I'm inclined to think that solving the issue for strings actually gains us more than for classes in terms of performance.