From the current page:

"If pointers to D garbage collector allocated memory are passed to C functions, it's critical to ensure that that memory will not be collected by the garbage collector before the C function is done with it."

Various suggestions on how to prevent this are offered. Well and good, for now. What happens if and when D gets a copying GC implementation?

The first suggestion - copying to non-GCed memory - is not appealing for large data blocks. The others would presumably throw up hard-to-diagnose wild pointer errors in rare and hard-to-reproduce circumstances.

In article <dcorea$2pov$1@digitaldaemon.com>, Mike Capp says...
>
>Various suggestions on how to prevent this are offered. Well and good, for now. What happens if and when D gets a copying GC implementation?
>

If the GC changes in such a way, all non D code must be completely rewritten.

The usual way for a copying garbage collector is to offer indirect pointers to the referrenced value, while non copying collectors simply pass a direct pointer.

In article <dcorea$2pov$1@digitaldaemon.com>, Mike Capp says...
>
>From the current page:
>
>"If pointers to D garbage collector allocated memory are passed to C functions, it's critical to ensure that that memory will not be collected by the garbage collector before the C function is done with it."
>
>Various suggestions on how to prevent this are offered. Well and good, for now. What happens if and when D gets a copying GC implementation?

The GC would offer a method to "pin" memory that should not be moved.


Sean

In article <dcosu3$2r13$1@digitaldaemon.com>, llothar says...
>
>If the GC changes in such a way, all non D code must be completely rewritten.

Hence the "eek". Completely rewriting win32, for example, is not a viable option. And yet the docs and many previous postings have indicated that a copying GC at some point is possible.

cheers
Mike

In article <dcou45$2s21$1@digitaldaemon.com>, Sean Kelly says...
>
>>What happens if and when D gets a copying GC implementation?
>
>The GC would offer a method to "pin" memory that should not be moved.

Exactly. Existing D code rebuilt with such an implementation won't ever call that method and will _silently_ become horribly unsafe. That's going to be popular.

Plus, of course, many if not most calls to C APIs will have to be wrapped with pin/unpin calls. Verbose, ugly and easy to forget. Even worse with asynchronous operations or userdata pointers for callbacks.

cheers
Mike

In article <dcovni$2svo$1@digitaldaemon.com>, Mike Capp says...
>
>In article <dcou45$2s21$1@digitaldaemon.com>, Sean Kelly says...
>>
>>>What happens if and when D gets a copying GC implementation?
>>
>>The GC would offer a method to "pin" memory that should not be moved.
>
>Exactly. Existing D code rebuilt with such an implementation won't ever call that method and will _silently_ become horribly unsafe. That's going to be popular.

Once I get threading finished for Ares I'm planning to look at the GC interface. I'll add 'pin' functions in the process, even if they do nothing with the current GC.  I'll try and remember to ask that the same be done for Phobos.

>Plus, of course, many if not most calls to C APIs will have to be wrapped with pin/unpin calls. Verbose, ugly and easy to forget. Even worse with asynchronous operations or userdata pointers for callbacks.

So what would you suggest?  Auto classes might help to streamline the process, but I can't think of any way completely automate it.


Sean

In article <dcp03v$2t8k$1@digitaldaemon.com>, Sean Kelly says...
>
>So what would you suggest?

Shrug. I'd suggest that GC is a lousy official memory management strategy for a language like D, but I've already done that and seem to be in a minority of one. (Cue 'Hearts and Flowers'...)

>but I can't think of any way completely automate it.

Not at the library level, no. It might be partially possible at the compiler level - "whenever you see a pointer argument being passed to a function declared as extern (C), insert pin/unpin calls for that pointer around the function invocation" - but that doesn't help for the async and userdata-callback cases. Still, those are maybe rare enough that requiring a manual pin is acceptable.

cheers
Mike

In article <dcp2vg$2v6p$1@digitaldaemon.com>, Mike Capp says...
>
>In article <dcp03v$2t8k$1@digitaldaemon.com>, Sean Kelly says...
>>
>>but I can't think of any way completely automate it.
>
>Not at the library level, no. It might be partially possible at the compiler level - "whenever you see a pointer argument being passed to a function declared as extern (C), insert pin/unpin calls for that pointer around the function invocation" - but that doesn't help for the async and userdata-callback cases.

There are two cases where a pointer may be passed to a C function:

- it is used for the function call, and no references persist after return
- the function stores this pointer in a static location and continues using it
after return (callbacks and such)

For the latter case, auto-unpinning is incorrect behavior.  Also, pin/unpin is currently only necessary in the first case if the application is multithreaded, as the current D GC will only cleanup during a call to 'new'.

>Still, those are maybe rare enough that requiring a manual pin is acceptable.

Perhaps an auto class would facilitate this a tad.  Not ideal, but it's something.


Sean

In article <dcp6pf$b5$1@digitaldaemon.com>, Sean Kelly says...
>
>There are two cases where a pointer may be passed to a C function:
>
>- it is used for the function call, and no references persist after return
>- the function stores this pointer in a static location and continues using it
>after return (callbacks and such)
>
>For the latter case, auto-unpinning is incorrect behavior.

Is it? I don't how pinning is implemented in garbage collectors; I was imagining that you could stick multiple pins in a chunk of memory. Auto-unpinning would then be pointless but harmless as long as the user had also done a manual pin.

>Also, pin/unpin is
>currently only necessary in the first case if the application is multithreaded,
>as the current D GC will only cleanup during a call to 'new'.

Not _strictly_ true. If you call a C function (fc) which takes both a data
pointer (pd) and a function pointer (pf), and you supply a function in your D
module (fd) as the value of pf, and fc calls pf before accessing pd, and fd
happens to call 'new'...

>Perhaps an auto class would facilitate this [async case] a tad. Not ideal, but it's something.

The more I look at auto the less I'm convinced it facilitates anything. As currently implemented it appears to be a variant syntax for try-finally, only with added limitations and extra cost. I don't see how it could help here, since autos are limited to local scope and an asynchronous function guaranteed to complete by the end of a local scope doesn't sound very asynchronous to me.

cheers
Mike

I think that it depends on the pin/unpin syntax.  Obviously you should be able to pin/unpin a chunk of memory at any time, but how about also being able to pin the memory at variable assign time.

pinned char myBuffer[];

which tells the GC to treat myBuffer in exactly the same way that the current GC treats it.  Ie, it can move if you reallocated it, but the copying collector will not move it.

Brad