Consider the following code

void* mem = malloc(500);
GC.addRange(mem, 500);
mem = realloc(mem, 512); // assume the pointer didn't change
GC.removeRange(mem);
// if the GC kicks in here we're f*****
GC.addRange(mem, 512);



I digged into GC.addRange to find out if I simply can skip the call to GC.removeRange when the pointer doesn't change and ended up with

void addRange(void *pbot, void *ptop, const TypeInfo ti) nothrow @nogc
{
    ranges.insert(Range(pbot, ptop));
}

where ranges is defined as: Treap!Range ranges

"insert" of Treap is implemented as:

Node* insert(Node* node, E element) @nogc
{
    if (!node)
        return allocNode(element);
    else if (element < node.element)
    {
        node.left = insert(node.left, element);
        if (node.left.priority < node.priority)
            node = rotateR(node);
    }
    else if (element > node.element)
    {
        node.right = insert(node.right, element);
        if (node.right.priority < node.priority)
            node = rotateL(node);
    }
    else
    {} // ignore duplicate

    return node;
}

The problem is the line that says "ignore duplicate". Because that way I can not safly update a GC range.

How are you supposed to safely update a GC range that changed its size but not its address? The documentation doesn't say anything about it.

Either I'm missing something or this is a bug.

Kind Regards
Benjamin Thaut

On 8/17/15 3:27 PM, Benjamin Thaut wrote:
> Consider the following code
>
> void* mem = malloc(500);
> GC.addRange(mem, 500);
> mem = realloc(mem, 512); // assume the pointer didn't change
> GC.removeRange(mem);

This is actually unsafe, you have to remove the range first, or else if it *does* change the pointer, your GC is using free'd memory. Plus, if it does change the pointer, how do you remove the original range?

> // if the GC kicks in here we're f*****
> GC.addRange(mem, 512);

Can't you GC.disable around this whole thing?

-Steve

On 8/17/2015 12:38 PM, Steven Schveighoffer wrote:
> On 8/17/15 3:27 PM, Benjamin Thaut wrote:
>> void* mem = malloc(500);
>> GC.addRange(mem, 500);
>> mem = realloc(mem, 512); // assume the pointer didn't change
>> GC.removeRange(mem);
>
> This is actually unsafe, you have to remove the range first, or else if it
> *does* change the pointer, your GC is using free'd memory. Plus, if it does
> change the pointer, how do you remove the original range?

Good catch, quite right.


>> // if the GC kicks in here we're f*****
>> GC.addRange(mem, 512);
>
> Can't you GC.disable around this whole thing?

I agree that should work.

// if the GC kicks in here we're f*****

Why?

static nothrow @nogc void removeRange(in void* p);
Removes the memory range starting at p from an internal list of ranges to be scanned during a collection. <...>

On 8/17/15 3:57 PM, welkam wrote:
> // if the GC kicks in here we're f*****
>
> Why?
>
> static nothrow @nogc void removeRange(in void* p);
> Removes the memory range starting at p from an internal list of ranges
> to be scanned during a collection. <...>

Because presumably the reason why you have added the range is because it's not GC memory (as in this case). This means that if a GC run kicks in right then, that range of data will not be scanned, and the GC memory it may have been pointing at could potentially be freed.

-Steve

On Monday, 17 August 2015 at 20:07:08 UTC, Steven Schveighoffer wrote:
> On 8/17/15 3:57 PM, welkam wrote:
>> // if the GC kicks in here we're f*****
>>
>> Why?
>>
>> static nothrow @nogc void removeRange(in void* p);
>> Removes the memory range starting at p from an internal list of ranges
>> to be scanned during a collection. <...>
>
> Because presumably the reason why you have added the range is because it's not GC memory (as in this case). This means that if a GC run kicks in right then, that range of data will not be scanned, and the GC memory it may have been pointing at could potentially be freed.
>
> -Steve

I might be wrong, but he should worry about GC before he removes that memory range from GC managed list not after. And his code smells to me. He gives full memory control to GC, but then wants to take it away, fiddle and give it back. I would allocate more than I need to so avoiding a need to extend the memory. If not then either allocate new chunk of memory, copy data and forget about old one or have a data structure where I could add new chunks of memory.
Its best to minimise system calls such as malloc and similar because they hit OS and slow down execution.

On Monday, 17 August 2015 at 19:38:21 UTC, Steven Schveighoffer wrote:
> On 8/17/15 3:27 PM, Benjamin Thaut wrote:
>> Consider the following code
>>
>> void* mem = malloc(500);
>> GC.addRange(mem, 500);
>> mem = realloc(mem, 512); // assume the pointer didn't change
>> GC.removeRange(mem);
>
> This is actually unsafe, you have to remove the range first, or else if it *does* change the pointer, your GC is using free'd memory. Plus, if it does change the pointer, how do you remove the original range?

I specifically asked for the case where the pointer doesn't change. Obvisouly the case where it does change is easy, you first add the new range and then remove the old one. But if you do this and the pointer didn't change, the addRange doesn't do anything because its a duplicate and the removeRange then removes the range, because the pointer is still the same. You then end up with the GC not knowing anything about the range anymore.

>
>> // if the GC kicks in here we're f*****
>> GC.addRange(mem, 512);
>
> Can't you GC.disable around this whole thing?
>
> -Steve

Yes, this would work, but It seems kind of broken to me, that you have to make 4 API Calls to the gc to handle something as simple as a realloc.

Kind Regards
Benjamin Thaut

On Monday, 17 August 2015 at 20:33:45 UTC, welkam wrote:
> I might be wrong, but he should worry about GC before he removes that memory range from GC managed list not after. And his code smells to me. He gives full memory control to GC, but then wants to take it away, fiddle and give it back. I would allocate more than I need to so avoiding a need to extend the memory. If not then either allocate new chunk of memory, copy data and forget about old one or have a data structure where I could add new chunks of memory.
> Its best to minimise system calls such as malloc and similar because they hit OS and slow down execution.

The memory in question is never controlled by the GC. But it may contain pointers into GC memory. You need to update your view on what addRange and removeRange does. Also when you bind a different language to D you don't have the option to change the code, you have to work with whats already there.

Kind Regards
Benjamin Thaut

On Monday, 17 August 2015 at 19:38:21 UTC, Steven Schveighoffer wrote:
>> // if the GC kicks in here we're f*****
>> GC.addRange(mem, 512);
>
> Can't you GC.disable around this whole thing?

GC.collect can still be called from another thread.

On Tuesday, 18 August 2015 at 10:27:14 UTC, Casper Færgemand wrote:
> On Monday, 17 August 2015 at 19:38:21 UTC, Steven Schveighoffer wrote:
>>> // if the GC kicks in here we're f*****
>>> GC.addRange(mem, 512);
>>
>> Can't you GC.disable around this whole thing?
>
> GC.collect can still be called from another thread.

Good point, also GC.disable doesn't guarantee that the GC will never run. The documentation says that the GC may still run in out of memory or similar situations where it absolutely needs to run. So there isn't a safe way to do this after all.