For wrappers for C libs, that use callbacks, it is very handy to pass in callback data reference to some object.
But D garbage collector can suddenly move objects in memory, thus invalidating callback data.

What is a right way to handle this?

Ilya Zaitseff schrieb:
> For wrappers for C libs, that use callbacks, it is very handy to pass in  callback data reference to some object.
> But D garbage collector can suddenly move objects in memory, thus  invalidating callback data.

No, it cannot move anything. But you should make sure you have this data also pointed-to somewhere in the program so that it doesn't get collected.

> What is a right way to handle this?

I'd say the best way is to allocate this memory out of the reach of garbage colector, for example using std.c.stdlib.malloc, then it also has to be freed using std.c.stdlib.free.

-eye

In article <opsbpwigsaaaezs2@ilya.tec.amursk.ru>, Ilya Zaitseff says...
>
>For wrappers for C libs, that use callbacks, it is very handy to pass in
>callback data reference to some object.
>But D garbage collector can suddenly move objects in memory, thus
>invalidating callback data.
>
>What is a right way to handle this?

I don't know the right way but I use:

std.gc
void addRoot(void* p)
Add p to list of roots. Roots are references to memory allocated by the
collector that are maintained in memory outside the collector pool. The garbage
collector will by default look for roots in the stacks of each thread, the
registers, and the default static data segment. If roots are held elsewhere, use
addRoot() or addRange() to tell the collector not to free the memory it points
to.

void removeRoot(void* p)
Remove p from list of roots

http://www.digitalmars.com/d/phobos.html#gc

seems to work.
Is it the right way?

Ant

In article <ce2qef$quf$2@digitaldaemon.com>, Ilya Minkov says...
>
>No, it cannot move anything.

From the D manual, Garbage Collection section: "A copying garbage collector can arbitrarily move objects around in memory".

Jill

Arcane Jill schrieb:

> In article <ce2qef$quf$2@digitaldaemon.com>, Ilya Minkov says...
> 
>>No, it cannot move anything.
> 
> From the D manual, Garbage Collection section: "A copying garbage collector can
> arbitrarily move objects around in memory".

In principle you're right and i'm aware of it, but it's not copying now, and i hardly see the possibility that such a GC could possibly take place after a while. I can't remember exactly what i was thinking of, but most likely what we would have is a further optimized non-copying GC. There are some obstacles to copying, i just don't exactly remember what. And what i also thought was that a completely strict GC is not possible, it would always have a conservative component to it.

If i recall correctly the current GC can even track the references of application's memory within the memory allocated by foreign DLLs, but i think it's even worse to rely on. :)

-eye

In article <ce33lt$12bh$1@digitaldaemon.com>, Ilya Minkov says...
>
>Arcane Jill schrieb:
>
>> From the D manual, Garbage Collection section: "A copying garbage collector can arbitrarily move objects around in memory".
>
>In principle you're right and i'm aware of it, but it's not copying now, and i hardly see the possibility that such a GC could possibly take place after a while.

Well, my life would be S-O-O-O much easier if that sentence I quoted from the D manual could be removed, and replaced with "We guarantee that at no future time will the garbage collector ever move objects around in memory". But it doesn't. It says what it says, so I have to assume that the gc will move things, even if it doesn't.

The key word here is "guarantee". If you can't /guarantee/ that things will stay put, then you have no choice but to assume the possibility that they might not. Unfortunately.

Arcane Jill

Ilya Zaitseff wrote:

> For wrappers for C libs, that use callbacks, it is very handy to pass in  callback data reference to some object.
> But D garbage collector can suddenly move objects in memory, thus  invalidating callback data.
> 
> What is a right way to handle this?

dfbth (yet another windowing toolkit; not important except as an example) maps HWNDs/GtkWidget*s to object instances using an associative array.  This way, D object references stay on the D side and the GC doesn't need any extra information to work correctly. (copying/compacting or not)

 -- andy

"Ant" <Ant_member@pathlink.com> wrote in message news:ce2qjv$r5g$1@digitaldaemon.com...
> In article <opsbpwigsaaaezs2@ilya.tec.amursk.ru>, Ilya Zaitseff says...
> >
> >For wrappers for C libs, that use callbacks, it is very handy to pass in
> >callback data reference to some object.
> >But D garbage collector can suddenly move objects in memory, thus
> >invalidating callback data.
> >
> >What is a right way to handle this?
>
> I don't know the right way but I use:
>
> std.gc
> void addRoot(void* p)
> Add p to list of roots. Roots are references to memory allocated by the
> collector that are maintained in memory outside the collector pool. The
garbage
> collector will by default look for roots in the stacks of each thread, the registers, and the default static data segment. If roots are held
elsewhere, use
> addRoot() or addRange() to tell the collector not to free the memory it
points
> to.
>
> void removeRoot(void* p)
> Remove p from list of roots
>
> http://www.digitalmars.com/d/phobos.html#gc
>
> seems to work.
> Is it the right way?

Almost. The gc scans the stacks for roots. A function call's parameters are on the stack, and so are active until the C function returns. So, just passing a pointer to a C function does not in itself mean you need to notify the gc about it. The problem comes in if the pointer is pointing to a pool of data, say allocated by malloc(), and within that pool are pointers to gc'd data. The gc doesn't know about those roots, and so needs to be informed about them.

In article <ce3hrt$1bk9$1@digitaldaemon.com>, Walter says...
>
>
>"Ant" <Ant_member@pathlink.com> wrote in message news:ce2qjv$r5g$1@digitaldaemon.com...
>> In article <opsbpwigsaaaezs2@ilya.tec.amursk.ru>, Ilya Zaitseff says...
>> >
>> >For wrappers for C libs, that use callbacks, it is very handy to pass in
>> >callback data reference to some object.
>> >But D garbage collector can suddenly move objects in memory, thus
>> >invalidating callback data.
>> >
>> >What is a right way to handle this?
>>
>> I don't know the right way but I use:
>>
>> std.gc
>> void addRoot(void* p)
>> Add p to list of roots. Roots are references to memory allocated by the
>> collector that are maintained in memory outside the collector pool. The
>garbage
>> collector will by default look for roots in the stacks of each thread, the registers, and the default static data segment. If roots are held
>elsewhere, use
>> addRoot() or addRange() to tell the collector not to free the memory it
>points
>> to.
>>
>> void removeRoot(void* p)
>> Remove p from list of roots
>>
>> http://www.digitalmars.com/d/phobos.html#gc
>>
>> seems to work.
>> Is it the right way?
>
>Almost. The gc scans the stacks for roots. A function call's parameters are on the stack, and so are active until the C function returns. So, just passing a pointer to a C function does not in itself mean you need to notify the gc about it. The problem comes in if the pointer is pointing to a pool of data, say allocated by malloc(), and within that pool are pointers to gc'd data. The gc doesn't know about those roots, and so needs to be informed about them.
>
>

ok, I think I fully understand the root things.
well, I think because as I understand it it confirms the usage of
addRoot in this case.
What's going on is that my C external function has a pointer to
my root that the GC doesn't know about, exactly as you say.

I don't seem to have problems with objects being moved or collect
(I used gc.fullCollect() extensivly when testing,
maybe I should test with gc.minimize() also).

the root is removed when my external C object notifies it's own removal.

The method Andy told us might fail if the external application objects would be moved by it's how GC ... (?)

My external application (GTK+) collects it's own objects,
I have no idea of it's internals
(except that I have to use the addRoot equivalent on some operations).

Ant