As I've mentioned previously, D's partially conservative garbage collection is
often a significant limitation for me when working with large amounts of data.
 As a workaround, I'd like to suggest that the GC be given a
noInternalPointers function.  When called with a pointer to the start of a
block, it would tell the GC to only consider pointers to the start of the
block, not internal pointers.  This would prevent false pointers from wreaking
havoc on large data structures.  In exchange, you'd have to be careful that a
reference to the start of the data structure was somewhere on the stack.  Of
course, you could still pass internal around all you wanted as long as you
didn't try to return an internal pointer from the function that allocated the
data structure.

The noInternalPointers property could also be a property of builtin dynamic arrays.  Appending a large amount of times to a large array breaks the GC in the current form because, when the array gets reallocated, the old memory blocks are not necessarily cleaned up due to the false pointer problem.  If an array has the noInternalPointers property, every time it is reallocated due to resizing, the pointer to the start of it would be automatically marked with the noInternalPointers attribute.

Does all this sound feasible/reasonable?

== Quote from dsimcha (dsimcha@yahoo.com)'s article
> As I've mentioned previously, D's partially conservative garbage collection is
> often a significant limitation for me when working with large amounts of data.
>  As a workaround, I'd like to suggest that the GC be given a
> noInternalPointers function.  When called with a pointer to the start of a
> block, it would tell the GC to only consider pointers to the start of the
> block, not internal pointers.  This would prevent false pointers from wreaking
> havoc on large data structures.  In exchange, you'd have to be careful that a
> reference to the start of the data structure was somewhere on the stack.  Of
> course, you could still pass internal around all you wanted as long as you
> didn't try to return an internal pointer from the function that allocated the
> data structure.
> The noInternalPointers property could also be a property of builtin dynamic
> arrays.  Appending a large amount of times to a large array breaks the GC in
> the current form because, when the array gets reallocated, the old memory
> blocks are not necessarily cleaned up due to the false pointer problem.  If an
> array has the noInternalPointers property, every time it is reallocated due to
> resizing, the pointer to the start of it would be automatically marked with
> the noInternalPointers attribute.
> Does all this sound feasible/reasonable?

It means that slices of an array won't keep the array data alive, but other than that I think it's reasonable.


Sean

Reply to Sean,

> == Quote from dsimcha (dsimcha@yahoo.com)'s article
> 
>> As I've mentioned previously, D's partially conservative garbage
>> collection is
>> often a significant limitation for me when working with large amounts
>> of data.
>> As a workaround, I'd like to suggest that the GC be given a
>> noInternalPointers function.  When called with a pointer to the start
>> of a
>> block, it would tell the GC to only consider pointers to the start of
>> the
>> block, not internal pointers.  This would prevent false pointers from
>> wreaking
>> havoc on large data structures.  In exchange, you'd have to be
>> careful that a
>> reference to the start of the data structure was somewhere on the
>> stack.  Of
>> course, you could still pass internal around all you wanted as long
>> as you
>> didn't try to return an internal pointer from the function that
>> allocated the
>> data structure.
>> The noInternalPointers property could also be a property of builtin
>> dynamic
>> arrays.  Appending a large amount of times to a large array breaks
>> the GC in
>> the current form because, when the array gets reallocated, the old
>> memory
>> blocks are not necessarily cleaned up due to the false pointer
>> problem.  If an
>> array has the noInternalPointers property, every time it is
>> reallocated due to
>> resizing, the pointer to the start of it would be automatically
>> marked with
>> the noInternalPointers attribute.
>> Does all this sound feasible/reasonable?
> It means that slices of an array won't keep the array data alive, but
> other than that I think it's reasonable.
> 
> Sean
> 

IIRC there is a contains pointers flag in the GC (and that's all I remember)

Yeah, I was actually just looking at that today.  That flag specifies whether the GC-allocated block could contain pointers to *other* GC-allocated blocks, not whether there could be pointers, i.e. from the stack, pointing internally *into* the block.