Hello!

I have written some D code that I need to link to :C++ huge project. Let it be just one function that uses GC. The question is: if C++ code creates several threads and runs this :D function simultaneously, will GC work correctly?

p.s. Of course the druntime is initialized before it.

Igor Shirkalin

On Monday, 10 July 2017 at 20:03:32 UTC, Igor Shirkalin wrote:
> Hello!
>
> I have written some D code that I need to link to :C++ huge project. Let it be just one function that uses GC. The question is: if C++ code creates several threads and runs this :D function simultaneously, will GC work correctly?
>
> p.s. Of course the druntime is initialized before it.
>
> Igor Shirkalin

If DRuntime is not made aware of the thread's existence, the thread will not be stopped by the GC, and the GC might collect memory that the thread is referencing on the stack or in non-GC memory. Anything allocated by the GC would still be scanned.

To inform DRuntime about your thread, you should call thread_attachThis:

https://dlang.org/phobos/core_thread.html#.thread_attachThis

As pointed out in the documentation of thread_attachThis, you might also want to call rt_moduleTlsCtor, to run thread local static constructors. Depending on your usage, this might not be necessary.

--
  Biotronic

On Tuesday, 11 July 2017 at 06:18:44 UTC, Biotronic wrote:
> On Monday, 10 July 2017 at 20:03:32 UTC, Igor Shirkalin wrote:
>> [...]
>
> If DRuntime is not made aware of the thread's existence, the thread will not be stopped by the GC, and the GC might collect memory that the thread is referencing on the stack or in non-GC memory. Anything allocated by the GC would still be scanned.
>
> To inform DRuntime about your thread, you should call thread_attachThis:
>
> https://dlang.org/phobos/core_thread.html#.thread_attachThis
>
> As pointed out in the documentation of thread_attachThis, you might also want to call rt_moduleTlsCtor, to run thread local static constructors. Depending on your usage, this might not be necessary.
>
> --
>   Biotronic

Thanks for very useful information!

On 2017-07-11 08:18, Biotronic wrote:

> If DRuntime is not made aware of the thread's existence, the thread will not be stopped by the GC, and the GC might collect memory that the thread is referencing on the stack or in non-GC memory.

Are you sure? Wouldn't that make malloc or any other custom allocators completely useless and the D GC would completely break the C standard library because it could collect memory allocated by the C standard library?

From "How Garbage Collection Works":

"5. Freeing all **GC** allocated memory that has no active pointers to it and do not need destructors to run" [1]. I added the emphasize on "GC".

From "Interfacing Garbage Collected Objects With Foreign Code"

"If the only pointer to an object is held outside of these areas, then the collector will miss it and free the memory. To avoid this from happening, either
* reallocate and copy the object using the foreign code's storage allocator or using the C runtime library's malloc/free." [2].

[1] http://dlang.org/spec/garbage.html#how_gc_works
[2] http://dlang.org/spec/garbage.html#gc_foreign_obj

-- 
/Jacob Carlborg

On Wednesday, 12 July 2017 at 09:10:07 UTC, Jacob Carlborg wrote:
> On 2017-07-11 08:18, Biotronic wrote:
>
>> If DRuntime is not made aware of the thread's existence, the thread will not be stopped by the GC, and the GC might collect memory that the thread is referencing on the stack or in non-GC memory.
>
> Are you sure? Wouldn't that make malloc or any other custom allocators completely useless and the D GC would completely break the C standard library because it could collect memory allocated by the C standard library?
>
> From "How Garbage Collection Works":
>
> "5. Freeing all **GC** allocated memory that has no active pointers to it and do not need destructors to run" [1]. I added the emphasize on "GC".
>
> From "Interfacing Garbage Collected Objects With Foreign Code"
>
> "If the only pointer to an object is held outside of these areas, then the collector will miss it and free the memory. To avoid this from happening, either
> * reallocate and copy the object using the foreign code's storage allocator or using the C runtime library's malloc/free." [2].
>
> [1] http://dlang.org/spec/garbage.html#how_gc_works
> [2] http://dlang.org/spec/garbage.html#gc_foreign_obj

That's basically what I tried to say - the GC may collect memory *it has allocated* if the only pointers to it are in memory the GC doesn't scan (i.e. on the stack of an unregistered thread or in memory not allocated via the GC).

It will not collect memory allocated by other means, but that Foo* you got from D and are using in C++ might point to a Bar soon after the GC runs.

--
  Biotronic

On Tuesday, 11 July 2017 at 22:59:42 UTC, Igor Shirkalin wrote:
> On Tuesday, 11 July 2017 at 06:18:44 UTC, Biotronic wrote:
>> On Monday, 10 July 2017 at 20:03:32 UTC, Igor Shirkalin wrote:
>>> [...]
>>
>> If DRuntime is not made aware of the thread's existence, the thread will not be stopped by the GC, and the GC might collect memory that the thread is referencing on the stack or in non-GC memory. Anything allocated by the GC would still be scanned.
>>
>> To inform DRuntime about your thread, you should call thread_attachThis:
>>
>> https://dlang.org/phobos/core_thread.html#.thread_attachThis
>>
>> As pointed out in the documentation of thread_attachThis, you might also want to call rt_moduleTlsCtor, to run thread local static constructors. Depending on your usage, this might not be necessary.
>>
>> --
>>   Biotronic
>
> Thanks for very useful information!

Just one small note.
If you don't know the foreign thread lifetime, it's cleaner to detach it from the runtime upon exit.

Else may fall in the following scenario.
1. you register thread A
2. thread A is destroyed later on, in the C++ code
3. another thread B come into your callback and allocate. The GC triggers and try to pause a non-existing thread A.

On 2017-07-12 11:28, Biotronic wrote:

> That's basically what I tried to say 

It wasn't very clear to me at least.

> - the GC may collect memory *it has allocated* if the only pointers to it are in memory the GC doesn't scan (i.e. on the stack of an unregistered thread or in memory not allocated via the GC).
> 
> It will not collect memory allocated by other means, but that Foo* you got from D and are using in C++ might point to a Bar soon after the GC runs.

Yes, that can happen.

-- 
/Jacob Carlborg

On Wednesday, 12 July 2017 at 12:08:35 UTC, Jacob Carlborg wrote:
> On 2017-07-12 11:28, Biotronic wrote:
>
>> That's basically what I tried to say
>
> It wasn't very clear to me at least.

Yeah, I see it in retrospect. "might collect memory that the thread is referencing on the stack or in non-GC memory" doesn't convey that the collected memory is only that which is allocated by the GC.

--
  Biotronic

On Wednesday, 12 July 2017 at 09:49:32 UTC, Guillaume Piolat wrote:
> On Tuesday, 11 July 2017 at 22:59:42 UTC, Igor Shirkalin wrote:
>>>> [...]
>>>
>>> --
>>>   Biotronic
>>
>> Thanks for very useful information!
>
> Just one small note.
> If you don't know the foreign thread lifetime, it's cleaner to detach it from the runtime upon exit.
>
> Else may fall in the following scenario.
> 1. you register thread A
> 2. thread A is destroyed later on, in the C++ code
> 3. another thread B come into your callback and allocate. The GC triggers and try to pause a non-existing thread A.

This is important note. Yes, usually the lifetime of foreign thread is unknown.

You, guys, helped me a lot.