December 19, 2011
On Sun, 18 Dec 2011 23:19:41 -0800, Vladimir Panteleev <vladimir@thecybershadow.net> wrote:

> On Monday, 19 December 2011 at 07:11:10 UTC, Adam Wilson wrote:
>> It seems to me that no one else is interested...
>
> Not true :)

Hehe, I mean't outside the D community. I can think of five people off the top of my head inside the D community who are interested in it, myself excluded. ;-P

-- 
Adam Wilson
Project Coordinator
The Horizon Project
http://www.thehorizonproject.org/
December 19, 2011
On Monday, 19 December 2011 at 07:27:14 UTC, Adam Wilson wrote:
> On Sun, 18 Dec 2011 23:19:41 -0800, Vladimir Panteleev <vladimir@thecybershadow.net> wrote:
>
>> On Monday, 19 December 2011 at 07:11:10 UTC, Adam Wilson wrote:
>>> It seems to me that no one else is interested...
>>
>> Not true :)
>
> Hehe, I mean't outside the D community. I can think of five people off the top of my head inside the D community who are interested in it, myself excluded. ;-P

Ah. D'oh. Yeah, I think the thing closest to D is the Boehm GC (which has a good amount of research associated with it, I believe).
December 19, 2011
On 2011-12-19 00:19, Adam Wilson wrote:
> On Sun, 18 Dec 2011 15:02:17 -0800, Timon Gehr <timon.gehr@gmx.ch> wrote:
>
>> On 12/18/2011 11:53 PM, Vladimir Panteleev wrote:
>>> On Sunday, 18 December 2011 at 20:32:18 UTC, Andrei Alexandrescu wrote:
>>>> That is an interesting opportunity. At any rate, I am 100% convinced
>>>> precise GC is the only way to go, and I think I've convinced Walter to
>>>> a good extent as well.
>>>
>>> Sacrificing something (performance, executable size) for something else
>>> is not an unilateral improvement.
>>
>> It is an unilateral improvement if both options are kept open. I don't
>> see a reason to cease support for the current GC model.
>>
>> Furthermore, a generational GC performs much better than a simple
>> mark-sweep GC.
>
> It seems to that we are really dancing around the potential solution. A
> pluggable GC interface that allowed the developer to choose the right GC
> for the task, or no GC at all. Imagine if all the developer had to do is
> set a compiler switch and the compiler automatically linked in the
> correct GC for the job. D could ship with a default GC then others could
> write different GC's based on different paradigms or their own needs. It
> would be a piece of work to get the interfaces right, but definitely
> worth it in the long run.
>
> Theoretically this would also give the developer the ability to link in
> multiple collectors and switch between them during program execution,
> provided the working set data was stored in a common format; although I
> have never heard of a use case for such a capability. I imagine that it
> could be useful if your application needs a low power state and you
> wanted to use a more power-efficient GC over a more precise one. I'm
> just theorizing here though. I'm sure their are technical snags that
> would prevent this from happening.
>
> I think supporting a NoGC environment is a good idea in the long run as
> their are cases to be made for manual memory management, but it also
> shouldn't be our first goal. GC's are were the value is.
>

It's already possible to plug different GC's at link time.

-- 
/Jacob Carlborg
December 19, 2011
On Sun, 18 Dec 2011 23:43:04 -0800, Jacob Carlborg <doob@me.com> wrote:

> On 2011-12-19 00:19, Adam Wilson wrote:
>> On Sun, 18 Dec 2011 15:02:17 -0800, Timon Gehr <timon.gehr@gmx.ch> wrote:
>>
>>> On 12/18/2011 11:53 PM, Vladimir Panteleev wrote:
>>>> On Sunday, 18 December 2011 at 20:32:18 UTC, Andrei Alexandrescu wrote:
>>>>> That is an interesting opportunity. At any rate, I am 100% convinced
>>>>> precise GC is the only way to go, and I think I've convinced Walter to
>>>>> a good extent as well.
>>>>
>>>> Sacrificing something (performance, executable size) for something else
>>>> is not an unilateral improvement.
>>>
>>> It is an unilateral improvement if both options are kept open. I don't
>>> see a reason to cease support for the current GC model.
>>>
>>> Furthermore, a generational GC performs much better than a simple
>>> mark-sweep GC.
>>
>> It seems to that we are really dancing around the potential solution. A
>> pluggable GC interface that allowed the developer to choose the right GC
>> for the task, or no GC at all. Imagine if all the developer had to do is
>> set a compiler switch and the compiler automatically linked in the
>> correct GC for the job. D could ship with a default GC then others could
>> write different GC's based on different paradigms or their own needs. It
>> would be a piece of work to get the interfaces right, but definitely
>> worth it in the long run.
>>
>> Theoretically this would also give the developer the ability to link in
>> multiple collectors and switch between them during program execution,
>> provided the working set data was stored in a common format; although I
>> have never heard of a use case for such a capability. I imagine that it
>> could be useful if your application needs a low power state and you
>> wanted to use a more power-efficient GC over a more precise one. I'm
>> just theorizing here though. I'm sure their are technical snags that
>> would prevent this from happening.
>>
>> I think supporting a NoGC environment is a good idea in the long run as
>> their are cases to be made for manual memory management, but it also
>> shouldn't be our first goal. GC's are were the value is.
>>
>
> It's already possible to plug different GC's at link time.

Cool, I didn't know that. Thanks for the heads up, i'm still working hard at understanding D's rather large set of capabilities. :-)

-- 
Adam Wilson
Project Coordinator
The Horizon Project
http://www.thehorizonproject.org/
December 19, 2011
On Sun, 18 Dec 2011 23:29:38 -0800, Vladimir Panteleev <vladimir@thecybershadow.net> wrote:

> On Monday, 19 December 2011 at 07:27:14 UTC, Adam Wilson wrote:
>> On Sun, 18 Dec 2011 23:19:41 -0800, Vladimir Panteleev <vladimir@thecybershadow.net> wrote:
>>
>>> On Monday, 19 December 2011 at 07:11:10 UTC, Adam Wilson wrote:
>>>> It seems to me that no one else is interested...
>>>
>>> Not true :)
>>
>> Hehe, I mean't outside the D community. I can think of five people off the top of my head inside the D community who are interested in it, myself excluded. ;-P
>
> Ah. D'oh. Yeah, I think the thing closest to D is the Boehm GC (which has a good amount of research associated with it, I believe).

Ahh yes, Boehm. According to this wikipedia page http://en.wikipedia.org/wiki/Boehm_garbage_collector it is also the GC that is used by D, with some minor modifications of course. If this is really the case then we really would be pushing the boundaries of GC research. My finger has been hovering over the buy link for the GC book for days ... I'm getting closer. I may not be much of compiler guy, but memory management and GC's have been a subject near and dear to my heart for some years now.

-- 
Adam Wilson
Project Coordinator
The Horizon Project
http://www.thehorizonproject.org/
December 19, 2011
On 12/19/2011 05:23 AM, a wrote:
>
>> In case of passing GC memory to C functions, I would prefer to just
>> disallow the C code to capture the reference, and to disable GC while
>> the C function runs.
>
> Wouldn't this be a problem with non concurrent GC? If you have multiple threads that spend most of the time in C function calls (for example performing IO), garbage collection would be impossible most of the time. In the extreme case garbage would never be collected.
>

Garbage collection is triggered by allocations. The C function does not perform GC allocations. While the memory is allocated, the GC is enabled. With non-concurrent GC there is certainly no issue, but the GC would have to be disabled and re-enabled on by thread basis.

> Couldn't we just require that the programmer keeps a reference to the memory passed to C in a D variable? AFAIK python does it that way, but
> I don't use python enough to know if it causes serious problems there.
>

The issue is that a copying GC must be able to move the memory. It cannot do that if there are references it does not know about, because it has to update all references to the moved memory. If you just keep around a reference to the memory, your reference will be updated, but the reference from the C code will be left dangling.
December 19, 2011
On 12/19/2011 01:00 AM, Vladimir Panteleev wrote:
> On Sunday, 18 December 2011 at 23:55:17 UTC, Timon Gehr wrote:
>> On 12/19/2011 12:45 AM, Vladimir Panteleev wrote:
>>> On Sunday, 18 December 2011 at 23:31:03 UTC, Timon Gehr wrote:
>>>> On 12/19/2011 12:24 AM, Vladimir Panteleev wrote:
>>>>> On Sunday, 18 December 2011 at 23:18:22 UTC, Timon Gehr wrote:
>>>>>> You are right. I have had in mind a generational GC that uses a
>>>>>> copying collector for the nursery as this is what most
>>>>>> state-of-the-art VM GCs do.
>>>>> These changes are too invasive for the language at this point, I
>>>>> believe. We need to work with what we have.
>>>>
>>>> I disagree. Code that relies on other semantics would just have to use
>>>> conservative GC.
>>>
>>> Please elaborate on how you would hypothetically change D to accustom
>>> such changes. I am having trouble imagining such an implementation that
>>> would not have a considerable impact on D's performance.
>>
>> If you have an union
>>
>> union X{
>> int x;
>> int* y;
>> }
>>
>> The compiler would just lay out x and y sequentially instead of at the
>> same memory location. Alternatively, it could add a tag to each union.
>>
>> In case of passing GC memory to C functions, I would prefer to just
>> disallow the C code to capture the reference, and to disable GC while
>> the C function runs. (disabling and enabling GC is just a matter of
>> modifying a counter somewhere, that should not visibly impact
>> performance.) If C code was to capture a reference to the memory then
>> it would probably take ownership anyway, which would necessitate to
>> allocate the memory with malloc even with conservative GC.
>>
>> I am not sure what to do about void[] though.
>
> OK... but what about the "generational GC that uses a copying collector
> for the nursery"?

I think we should attempt an implementation, find out where the main performance issues lie and then find a suitable set of trade-offs.
December 19, 2011
On 18-12-2011 15:40, Somedude wrote:
> Le 18/12/2011 15:07, Ruslan Mullakhmetov a écrit :
>>
>> GC is just a mater of implementation. In presence of resources implement
>> good GC algorithm offers no difficulty.
>>
>
> Oh really ? Then please make us a favor and write one for D. Also I'm
> sure the C++ guys will be pleased to hear that it's such an easy task.

Yeah, unfortunately, we can't just keep saying "it's an implementation issue". It's very much a real problem; D programmers are *avoiding* the GC because it just isn't comparable to, say, the .NET and Java GCs in performance at all.

- Alex
December 19, 2011
On Monday, 19 December 2011 at 08:28:52 UTC, Adam Wilson wrote:
> According to this wikipedia page http://en.wikipedia.org/wiki/Boehm_garbage_collector it is also the GC that is used by D, with some minor modifications of course.

I'm not sure if that's true... I believe that they both use the same basic idea, but AFAIK the D garbage collector is a D port of a C rewrite which was originally written for something else. The D GC has been optimized a lot since its first versions.
December 19, 2011
On 12/19/2011 07:50 PM, Vladimir Panteleev wrote:
> On Monday, 19 December 2011 at 08:28:52 UTC, Adam Wilson wrote:
>> According to this wikipedia page
>> http://en.wikipedia.org/wiki/Boehm_garbage_collector it is also the GC
>> that is used by D, with some minor modifications of course.
>
> I'm not sure if that's true... I believe that they both use the same
> basic idea, but AFAIK the D garbage collector is a D port of a C rewrite
> which was originally written for something else. The D GC has been
> optimized a lot since its first versions.

It would probably be interesting to test the mostly concurrent generational Boehm GC with D. I'd expect it to perform a lot better than the simple mark and sweep GC we have in druntime.