Congratulations, Antti-Ville!  This project creates a better implementation of precise GC heap scanning than anything that's been created so far for D.  The goal is to eventually integrate it into standard D distributions.  Any volunteers for beta testing?

Code:

https://github.com/Tuna-Fish/druntime/tree/gc_poolwise_bitmap

The code for this project is a fork of druntime.  The master branch was a failed (or less successful) experiment.  The version we're going with for integration is the gc_poolwise_bitmap branch.

Congrats!

On Tue, 21 Aug 2012 15:15:48 +0200, dsimcha <dsimcha@yahoo.com> wrote:

> Congratulations, Antti-Ville!  This project creates a better implementation of precise GC heap scanning than anything that's been created so far for D.  The goal is to eventually integrate it into standard D distributions.  Any volunteers for beta testing?
>
> Code:
>
> https://github.com/Tuna-Fish/druntime/tree/gc_poolwise_bitmap
>
> The code for this project is a fork of druntime.  The master branch was a failed (or less successful) experiment.  The version we're going with for integration is the gc_poolwise_bitmap branch.

Congratulations!

Any numbers on how this is better than the status quo?

-- 
Simen

On Tuesday, 21 August 2012 at 13:15:49 UTC, dsimcha wrote:
> Congratulations, Antti-Ville!  This project creates a better implementation of precise GC heap scanning than anything that's been created so far for D.  The goal is to eventually integrate it into standard D distributions.

Great! I've been pretty much out of touch with the GSoC projects this summers, as I had a rather intense exam session at university – are there any progress reports/blog posts with actual numbers in them?

David

On Wednesday, 22 August 2012 at 20:04:12 UTC, David Nadlinger wrote:
> On Tuesday, 21 August 2012 at 13:15:49 UTC, dsimcha wrote:
>> Congratulations, Antti-Ville!  This project creates a better implementation of precise GC heap scanning than anything that's been created so far for D.  The goal is to eventually integrate it into standard D distributions.
>
> Great! I've been pretty much out of touch with the GSoC projects this summers, as I had a rather intense exam session at university – are there any progress reports/blog posts with actual numbers in them?
>
> David

Unfortunately not yet.  Antti-ville ran into a lot if difficulty getting things to work and spent the first half of GSoC on an approach that turned out to be the wrong one in hindsight.  His project was originally supposed to be improving concurrency support in the GC, but when rtinfo was added to TypeInfo, precise heap scanning seemed like too good an opportunity to pass up.  To do this project he had to learn the D template system from scratch.  Therefore we didn't have the luxury of the luxury of doing extensive benchmarking and documentation.  I plan to work on that with him before he creates a pull request to integrate his new and improved GC upstream.

On 08/22/2012 05:41 PM, dsimcha wrote:
> On Wednesday, 22 August 2012 at 20:04:12 UTC, David Nadlinger wrote:
>> On Tuesday, 21 August 2012 at 13:15:49 UTC, dsimcha wrote:
>>> Congratulations, Antti-Ville! This project creates a better
>>> implementation of precise GC heap scanning than anything that's been
>>> created so far for D. The goal is to eventually integrate it into
>>> standard D distributions.
>>
>> Great! I've been pretty much out of touch with the GSoC projects this
>> summers, as I had a rather intense exam session at university – are
>> there any progress reports/blog posts with actual numbers in them?
>>
>> David
>
> Unfortunately not yet. Antti-ville ran into a lot if difficulty getting
> things to work and spent the first half of GSoC on an approach that
> turned out to be the wrong one in hindsight. His project was originally
> supposed to be improving concurrency support in the GC, but when rtinfo
> was added to TypeInfo, precise heap scanning seemed like too good an
> opportunity to pass up. To do this project he had to learn the D
> template system from scratch. Therefore we didn't have the luxury of the
> luxury of doing extensive benchmarking and documentation. I plan to work
> on that with him before he creates a pull request to integrate his new
> and improved GC upstream.

Poolwise bitmap... what an interesting name.  I'll look forward to learning about the concepts behind it!

On Thu, Aug 23, 2012 at 4:01 AM, Chad J <chadjoan@__spam.is.bad__gmail.com>wrote:

>
> Poolwise bitmap... what an interesting name.  I'll look forward to learning about the concepts behind it!
>

+1

On Thursday, 23 August 2012 at 11:40:22 UTC, Rory McGuire wrote:
> On Thu, Aug 23, 2012 at 4:01 AM, Chad J
> <chadjoan@__spam.is.bad__gmail.com>wrote:
>
>>
>> Poolwise bitmap... what an interesting name.  I'll look forward to
>> learning about the concepts behind it!
>>
>
> +1

Basically, the idea is to store information about what is and isn't a pointer at the pool level instead of at the block level.  My attempt from a long time ago at precise heap scanning, and Antti-Ville's first attempt, stored meta-data at the end of every allocated block.  This worked well for large arrays, but was terribly inefficient for smaller allocations and made the GC code even messier than it already is.  The overhead was a fixed (void*).sizeof bits per block.  Now, each pool has a bit array that contains one bit for every possible aligned pointer.  The overhead is always 1 bit for every (void*).sizeof bytes no matter how large or small the block is.

On Thu, Aug 23, 2012 at 2:51 PM, dsimcha <dsimcha@yahoo.com> wrote:

> Basically, the idea is to store information about what is and isn't a pointer at the pool level instead of at the block level.  My attempt from a long time ago at precise heap scanning, and Antti-Ville's first attempt, stored meta-data at the end of every allocated block.  This worked well for large arrays, but was terribly inefficient for smaller allocations and made the GC code even messier than it already is.  The overhead was a fixed (void*).sizeof bits per block.  Now, each pool has a bit array that contains one bit for every possible aligned pointer.  The overhead is always 1 bit for every (void*).sizeof bytes no matter how large or small the block is.
>

Am I correct in thinking that this is still single threaded stop the world?

Any chance of the code being documented extensively in the hopes that it would encourage participation/experimentation?

Thanks for all the work you guys have put in.

On 23-08-2012 15:21, Rory McGuire wrote:
> On Thu, Aug 23, 2012 at 2:51 PM, dsimcha <dsimcha@yahoo.com
> <mailto:dsimcha@yahoo.com>> wrote:
>
>     Basically, the idea is to store information about what is and isn't
>     a pointer at the pool level instead of at the block level.  My
>     attempt from a long time ago at precise heap scanning, and
>     Antti-Ville's first attempt, stored meta-data at the end of every
>     allocated block.  This worked well for large arrays, but was
>     terribly inefficient for smaller allocations and made the GC code
>     even messier than it already is.  The overhead was a fixed
>     (void*).sizeof bits per block.  Now, each pool has a bit array that
>     contains one bit for every possible aligned pointer.  The overhead
>     is always 1 bit for every (void*).sizeof bytes no matter how large
>     or small the block is.
>
>
> Am I correct in thinking that this is still single threaded stop the world?

Yes, but parallelization of the mark phase is fairly trivial, and something we should probably look into.

The GC will probably always be STW unless we get compiler support for inserting GC barriers.

>
> Any chance of the code being documented extensively in the hopes that it
> would encourage participation/experimentation?
>
> Thanks for all the work you guys have put in.

-- 
Alex Rønne Petersen
alex@lycus.org
http://lycus.org