Am 09.10.2013 17:46, schrieb Paulo Pinto:
> Am 09.10.2013 16:30, schrieb Manu:
>> On 9 October 2013 17:31, Walter Bright <newshound2@digitalmars.com
>> <mailto:newshound2@digitalmars.com>> wrote:
>>
>>     On 10/9/2013 12:29 AM, Manu wrote:
>>
>>         Does anyone here REALLY believe that a bunch of volunteer
>>         contributors can
>>         possibly do what apple failed to do with their squillions of
>>         dollars and engineers?
>>         I haven't heard anybody around here propose the path to an
>>         acceptable solution.
>>         It's perpetually in the too-hard basket, hence we still have the
>>         same GC as
>>         forever and it's going nowhere.
>>
>>
>>     What do you propose?
>>
>>
>> ARC. I've been here years now, and I see absolutely no evidence that the
>> GC is ever going to improve. I can trust ARC, it's predictable, I can
>> control it.
>> Also, proper support for avoiding the GC without severe inconvenience as
>> constantly keeps coming up. But I don't think there's any debate on that
>> one. Everyone seems to agree.
>
> As someone that is in the sidelines and doesn't really use D, my opinion
> should not count that much, if at all.
>
> However, rewriting D's memory management to be ARC based will have
> performance impact if the various D compilers aren't made ARC aware.
>
> Then there is the whole point of rewriting phobos and druntime to use
> ARC instead of GC.
>
> Will the return on investment pay off, instead of fixing the existing GC?
>
> What will be the message sent to the outsiders wondering if D is stable
> enough to be adopted, and see these constant rewrites?
>
> --
> Paulo
>

not manual ARC - compiler generated ARC - so there is no need for rewrite

On 2013-10-09 15:18:23 +0000, "deadalnix" <deadalnix@gmail.com> said:

> On Wednesday, 9 October 2013 at 13:03:33 UTC, Michel Fortin wrote:
>> On 2013-10-09 04:07:53 +0000, "deadalnix" <deadalnix@gmail.com> said:
>> 
>>> That being said, isolated is probably something we want to add in the future.
>> 
>> You'll hit a problem with immutable though. Immutable is implicitly shared, and immutable strings are everywhere!
> 
> Collecting immutable do not require to stop the application.

All I'm pointing out is that as long as immutable is implicitly shared, immutable memory has to be allocated and collected the same way as shared memory "just in case" someone passes it to another thread. The type system can't prevent you from sharing your immutable strings with other threads.

-- 
Michel Fortin
michel.fortin@michelf.ca
http://michelf.ca

On Oct 9, 2013, at 8:23 AM, deadalnix <deadalnix@gmail.com> wrote:

> On Wednesday, 9 October 2013 at 14:17:44 UTC, Sean Kelly wrote:
>> On Oct 9, 2013, at 4:30 AM, Jacob Carlborg <doob@me.com> wrote:
>>>> On 2013-10-09 05:31, Walter Bright wrote:
>>>> Making this work is fraught with difficulty. It is normal behavior in D
>>>> to create local data with new(), build a data structure, and then cast
>>>> it to shared so it can be transferred to another thread. This will fail
>>>> miserably if the data is allocated on a thread local heap.
>>> I agree with Andrei here. Alternatively perhaps the runtime can move the data to a global pool if it's casted to shared.
>> 
>> Generally not, since even D's precise GC is partially conservative.  It's also way more expensive than any cast should be. For better or worse, I think being able to cast data to shared means that we can't have thread-local pools. Unless a new attribute were introduced like "local" that couldn't ever be cast to shared, and that sounds like a disaster.
> 
> That isn't accurant. Allocator like tcmalloc use thread local info to allocate shared chunk of memory. What does matter is that the block is tagged as shared as far as the GC is concerned.

If the GC can determine whether a block is shared at allocation time, it can allocate the block from a thread-local pool in the unshared case.  So if a collection is necessary, no global stop the world need occur.  Only the current thread's roots are scanned.  It's a huge performance gain in a concurrent program.  Particularly in a language like D where data is thread-local by default.

Am 09.10.2013 17:49, schrieb dennis luehring:
> Am 09.10.2013 17:46, schrieb Paulo Pinto:
>> Am 09.10.2013 16:30, schrieb Manu:
>>> On 9 October 2013 17:31, Walter Bright <newshound2@digitalmars.com
>>> <mailto:newshound2@digitalmars.com>> wrote:
>>>
>>>     On 10/9/2013 12:29 AM, Manu wrote:
>>>
>>>         Does anyone here REALLY believe that a bunch of volunteer
>>>         contributors can
>>>         possibly do what apple failed to do with their squillions of
>>>         dollars and engineers?
>>>         I haven't heard anybody around here propose the path to an
>>>         acceptable solution.
>>>         It's perpetually in the too-hard basket, hence we still have the
>>>         same GC as
>>>         forever and it's going nowhere.
>>>
>>>
>>>     What do you propose?
>>>
>>>
>>> ARC. I've been here years now, and I see absolutely no evidence that the
>>> GC is ever going to improve. I can trust ARC, it's predictable, I can
>>> control it.
>>> Also, proper support for avoiding the GC without severe inconvenience as
>>> constantly keeps coming up. But I don't think there's any debate on that
>>> one. Everyone seems to agree.
>>
>> As someone that is in the sidelines and doesn't really use D, my opinion
>> should not count that much, if at all.
>>
>> However, rewriting D's memory management to be ARC based will have
>> performance impact if the various D compilers aren't made ARC aware.
>>
>> Then there is the whole point of rewriting phobos and druntime to use
>> ARC instead of GC.
>>
>> Will the return on investment pay off, instead of fixing the existing GC?
>>
>> What will be the message sent to the outsiders wondering if D is stable
>> enough to be adopted, and see these constant rewrites?
>>
>> --
>> Paulo
>>
>
> not manual ARC - compiler generated ARC - so there is no need for rewrite
>

There will be a need for rewrite, if the code happens to have cyclic references.

--
Paulo

On Wednesday, 9 October 2013 at 15:48:58 UTC, Sean Kelly wrote:
> On Oct 9, 2013, at 7:35 AM, Jacob Carlborg <doob@me.com> wrote:
>
>> On 2013-10-09 15:51, Sean Kelly wrote:
>> 
>>> Generally not, since even D's precise GC is partially conservative.  It's also way more expensive than any cast should be. For better or worse, I think being able to cast data to shared means that we can't have thread-local pools. Unless a new attribute were introduced like "local" that couldn't ever be cast to shared, and that sounds like a disaster.
>> 
>> Since casting breaks the type system to begin with and is an advanced feature. How about providing a separate function that moves the object? It will be up to the user to call the function.
>
> Okay so following that… it might be reasonable if the location of data keyed off the attribute set at construction.  So "new shared(X)" puts it in the shared pool.  Strings are a bit trickier though, since there's no way to specify the locality of the result of a string operation:
>
> shared string x = a ~ b ~ c;
>
> And I'm inclined to think that solving the issue for strings actually gains us more than for classes in terms of performance.

The heap allocated part here is immutable. The slice itself is a value type.

On Oct 9, 2013, at 8:35 AM, Paulo Pinto <pjmlp@progtools.org> wrote:
> 
> The Azul VM does not have a problem with it, as it does pauseless concurrent GC, while being used on online trading systems.

Incremental GCs are awesome.  Have you looked at IBM's Metronome for Java?  Maybe this would work with SafeD, but I don't think it's an option with the full D language.

On Wednesday, 9 October 2013 at 15:53:13 UTC, Sean Kelly wrote:
> If the GC can determine whether a block is shared at allocation time, it can allocate the block from a thread-local pool in the unshared case.  So if a collection is necessary, no global stop the world need occur.  Only the current thread's roots are scanned.  It's a huge performance gain in a concurrent program.
>  Particularly in a language like D where data is thread-local by default.

Yes, we have this awesome mecanism and we aren't using it.

On Wednesday, 9 October 2013 at 13:38:12 UTC, Michel Fortin wrote:
> On 2013-10-09 11:58:33 +0000, Jacob Carlborg <doob@me.com> said:
>
>> On 2013-10-09 09:33, Manu wrote:
>> 
>>> It sounds pretty easy to reach to me. Compiler generating inc/dec ref
>>> calls can't possibly be difficult. An optimisation that simplifies
>>> redundant inc/dec sequences doesn't sound hard either... :/
>>> Is there more to it? Cleaning up circular references I guess... what
>>> does Apple do?
>>> It's an uncommon edge case, so there's gotta be heaps of room for
>>> efficient solutions to that (afaik) one edge case. Are there others?
>> 
>> See my reply to one of your other posts:
>> 
>> http://forum.dlang.org/thread/bsqqfmhgzntryyaqrtky@forum.dlang.org?page=10#post-l33gah:24ero:241:40digitalmars.com
>
> I
>> 
>> don't recall the exact issues but there were several issues that were brought up in the email conversation.
>
> Here's a quick summary:
>
> Walter's idea was to implement ARC for classes implementing IUnknown. So if you wanted a ARC'ed class, you implement IUnknown and you're done. Problems were on the edges of ARC-controlled pointers and GC-controlled ones (which cannot be allowed to be mixed), and also with wanting to use the GC to free cycles for these objects (which would make COM AddRef and Release unusable when called from non-D code). I think the only way to make that work sanely is to create another root object for ref-counted classes.
>
> Another idea was to make *everything* in D ref-counted. ARC simply becomes another GC implementation. There can be no confusion between what's ref-counted and what isn't (everything is). It's much simpler really. But Walter isn't keen on the idea of having to call a function at every pointer assignment to keep the reference count up to date (for performance reasons), so that idea was rejected. This makes some sense, because unlike Objective-C ARC where only Objective-C object pointers are ref-counted, in D you'd have to do that with all pointers, and some will point to external data that does not need to be ref-counted at all.
>
> It was also pointed out that concurrent GCs require some work to be done on pointer assignment (and also when moving a pointer). So it seems to me that advances on the GC front are going to be limited without that.
>
> And now, things seems to have stalled again. It's a little disappointing.

I found no summary and stuff seems to get lost,
so I created a page on the wiki.

http://wiki.dlang.org/Versus_the_garbage_collector

On 2013-10-09 17:48, Sean Kelly wrote:

> Okay so following that… it might be reasonable if the location of data keyed off the attribute set at construction.  So "new shared(X)" puts it in the shared pool.

I thought that was obvious. Is there a problem with that approach?

-- 
/Jacob Carlborg

On 10 October 2013 01:40, Johannes Pfau <nospam@example.com> wrote:

> Am Wed, 9 Oct 2013 10:36:36 -0400
> schrieb Michel Fortin <michel.fortin@michelf.ca>:
>
> > >
> > > @refcounted interface X{};
> > > @refcounted class Y {};
> > > class X1 : X;
> > > class Y1 : Y;
> > >
> > > Now we know for sure that all these classes and SubClasses are refcounted.
> >
> > The problem is that you can't cast Y to its base class Object (because you'd be casting an ARC pointer to a GC one and the reference count would not be maintained). Even calling Object's functions should be prohibited (because the "this" pointer is a GC pointer inside of Object's functions, and it could be leaked). So, de facto, by introducing all the necessary safety limitations, you've practically made Y a root class. The only things you can safely access from the base non-ref-counted class are its variables and its static functions.
>
> I see. That's a good point why a separate object hirarchy is necessary.
>
> > On top of that all that, if your goal is to go GC-free, you have to reinvent the whole standard library (so it uses ref-counted classes) and you still have to avoid the common pitfalls of implicit allocations (array concat for instance). So it's not terribly useful for someone whose goal is to avoid the GC.
> >
>
> I don't think the standard lib is not that much of a problem, at least there's a way to avoid it (and some modules, std.algorithm, std.digest, std.uuid already are 99% GC free).
>
> But if someone really wants to strip the GC _completely_ there's a huge issue with memory management of Exceptions.
>

Exceptions have a pretty well defined lifetime... can't they be manually cleaned up by the exception handler after the catching scope exits?