Thread overview | ||||||||||
---|---|---|---|---|---|---|---|---|---|---|
|
March 25, 2012 GC collecting "too much".. | ||||
---|---|---|---|---|
| ||||
I'm doing some coding against a c library, and Ds GC keeps collecting c owned objects (I think - disabling the GC makes everything work) But how can I figure out what the GC is (read: I am) fucking up? I have some to!string(c_struct_field) and format("%s", c_struct_field) and field = c_fields[i] etc etc |
March 25, 2012 Re: GC collecting "too much".. | ||||
---|---|---|---|---|
| ||||
Posted in reply to simendsjo | On Sunday, 25 March 2012 at 19:15:05 UTC, simendsjo wrote:
> I'm doing some coding against a c library, and Ds GC keeps collecting c owned objects (I think - disabling the GC makes everything work)
Three alternative solutions:
- Allocate from the C heap the memory that C will need to use, and free it manually or in a struct destructor (RAII) or with scope(exit).
- Keep a pointer to the D-GC memory in the D code too.
- In core.memory there are ways to disable scanning of a memory zone. Maybe it's usable for your purposes too.
Bye,
bearophile
|
March 26, 2012 Re: GC collecting "too much".. | ||||
---|---|---|---|---|
| ||||
Posted in reply to bearophile | On Sun, 25 Mar 2012 22:18:02 +0200, bearophile <bearophileHUGS@lycos.com> wrote: > On Sunday, 25 March 2012 at 19:15:05 UTC, simendsjo wrote: >> I'm doing some coding against a c library, and Ds GC keeps collecting c owned objects (I think - disabling the GC makes everything work) > > Three alternative solutions: > - Allocate from the C heap the memory that C will need to use, and free it manually or in a struct destructor (RAII) or with scope(exit). > - Keep a pointer to the D-GC memory in the D code too. > - In core.memory there are ways to disable scanning of a memory zone. Maybe it's usable for your purposes too. > > Bye, > bearophile I've been able to find where the code fails, but now I don't understand what's happening at all. Is calling GC.collect() from an extern(C) function undefined? The following code just starts a mongoose web server and tries to run GC.collect in the handler. extern(C) void* cb(mg_event event, mg_connection* conn, mg_request_info* request_info) { GC.collect(); // segfault return null; } void main() { auto opts = ["listening_ports", "6969"].map!(toUTFz!(char*))().array(); mg_start(&cb, null, cast(const(char**))opts); GC.collect(); // no problem readln(); } If I collect memory from main(), it works as expected (removing the collect from cb of course) while(readln().chomp() != "q") { GC.collect(); } The documentation in core.memory is a bit sparse on how the GC works. Are there any articles on the D GC? |
March 26, 2012 Re: GC collecting "too much".. | ||||
---|---|---|---|---|
| ||||
Posted in reply to simendsjo | On Mon, 26 Mar 2012 10:13:35 +0200, simendsjo <simendsjo@gmail.com> wrote:
> On Sun, 25 Mar 2012 22:18:02 +0200, bearophile <bearophileHUGS@lycos.com> wrote:
>
>> On Sunday, 25 March 2012 at 19:15:05 UTC, simendsjo wrote:
>>> I'm doing some coding against a c library, and Ds GC keeps collecting c owned objects (I think - disabling the GC makes everything work)
>>
>> Three alternative solutions:
>> - Allocate from the C heap the memory that C will need to use, and free it manually or in a struct destructor (RAII) or with scope(exit).
>> - Keep a pointer to the D-GC memory in the D code too.
>> - In core.memory there are ways to disable scanning of a memory zone. Maybe it's usable for your purposes too.
>>
>> Bye,
>> bearophile
>
>
> I've been able to find where the code fails, but now I don't understand what's happening at all.
> Is calling GC.collect() from an extern(C) function undefined?
>
> The following code just starts a mongoose web server and tries to run GC.collect in the handler.
>
> extern(C) void* cb(mg_event event, mg_connection* conn, mg_request_info* request_info)
> {
> GC.collect(); // segfault
> return null;
> }
>
> void main()
> {
> auto opts = ["listening_ports", "6969"].map!(toUTFz!(char*))().array();
> mg_start(&cb, null, cast(const(char**))opts);
> GC.collect(); // no problem
> readln();
> }
>
> If I collect memory from main(), it works as expected (removing the collect from cb of course)
> while(readln().chomp() != "q") { GC.collect(); }
>
> The documentation in core.memory is a bit sparse on how the GC works. Are there any articles on the D GC?
It seems threads created in the c library is totally unknown to D. How can I make D aware of these threads when there is no library support for it?
|
March 26, 2012 Re: GC collecting "too much".. | ||||
---|---|---|---|---|
| ||||
Posted in reply to simendsjo | On 03/26/2012 11:55 AM, simendsjo wrote: > It seems threads created in the c library is totally unknown to D. How > can I make D aware of these threads when there is no library support for > it? You may be looking for this: http://dlang.org/phobos/core_thread.html#thread_attachThis |
March 26, 2012 Re: GC collecting "too much".. | ||||
---|---|---|---|---|
| ||||
Posted in reply to Timon Gehr | On Mon, 26 Mar 2012 17:10:34 +0200, Timon Gehr <timon.gehr@gmx.ch> wrote:
> On 03/26/2012 11:55 AM, simendsjo wrote:
>> It seems threads created in the c library is totally unknown to D. How
>> can I make D aware of these threads when there is no library support for
>> it?
>
> You may be looking for this:
> http://dlang.org/phobos/core_thread.html#thread_attachThis
Thanks, but I tried that too and couldn't get it to work.
I added the following:
extern(C) handler()
{
synchronized // needed here to avoid the GC to collect while attaching thread?
{
if(!Thread.getThis()) // thread unknown to D
{
thread_attachThis();
assert(Thread.getThis()); // now D knows about it
}
}
GC.collect(); // still segfaults
}
Actually, using attachThis segfaults GC.collect() outside the thread handling code too.
|
March 29, 2012 Re: GC collecting "too much".. | ||||
---|---|---|---|---|
| ||||
Posted in reply to simendsjo | On Mon, 26 Mar 2012 20:15:40 +0200, simendsjo <simendsjo@gmail.com> wrote: > On Mon, 26 Mar 2012 17:10:34 +0200, Timon Gehr <timon.gehr@gmx.ch> wrote: > >> On 03/26/2012 11:55 AM, simendsjo wrote: >>> It seems threads created in the c library is totally unknown to D. How >>> can I make D aware of these threads when there is no library support for >>> it? >> >> You may be looking for this: >> http://dlang.org/phobos/core_thread.html#thread_attachThis > > Thanks, but I tried that too and couldn't get it to work. > > I added the following: > > extern(C) handler() > { > synchronized // needed here to avoid the GC to collect while attaching thread? > { > if(!Thread.getThis()) // thread unknown to D > { > thread_attachThis(); > assert(Thread.getThis()); // now D knows about it > } > } > GC.collect(); // still segfaults > } > > Actually, using attachThis segfaults GC.collect() outside the thread handling code too. Some info from gdb: [New Thread 0xf7cfcb70 (LWP 18614)] [New Thread 0xf74fbb70 (LWP 18615)] Program received signal SIGUSR1, User defined signal 1. 0xf7fdd430 in __kernel_vsyscall () (gdb) info threads Id Target Id Frame 3 Thread 0xf74fbb70 (LWP 18615) "main" 0xf7fdd430 in __kernel_vsyscall () 2 Thread 0xf7cfcb70 (LWP 18614) "main" 0xf7fdd430 in __kernel_vsyscall () * 1 Thread 0xf7dfd9c0 (LWP 18611) "main" 0xf7fdd430 in __kernel_vsyscall () (gdb) c Continuing. Program received signal SIGSEGV, Segmentation fault. [Switching to Thread 0xf74fbb70 (LWP 18615)] 0x080855b4 in gc.gcx.Gcx.mark() () (gdb) info threads Id Target Id Frame * 3 Thread 0xf74fbb70 (LWP 18615) "main" 0x080855b4 in gc.gcx.Gcx.mark() () 2 Thread 0xf7cfcb70 (LWP 18614) "main" 0xf7fdd430 in __kernel_vsyscall () 1 Thread 0xf7dfd9c0 (LWP 18611) "main" 0xf7fdd430 in __kernel_vsyscall () ------------ And if I change the code to: extern(C) handler() { GC.collect(); } [New Thread 0xf7cfcb70 (LWP 21454)] [New Thread 0xf74fbb70 (LWP 21455)] Program received signal SIGSEGV, Segmentation fault. [Switching to Thread 0xf74fbb70 (LWP 21455)] 0x0808970e in core.thread.thread_suspendAll() () (gdb) info threads Id Target Id Frame * 3 Thread 0xf74fbb70 (LWP 21455) "main" 0x0808970e in core.thread.thread_suspendAll() () 2 Thread 0xf7cfcb70 (LWP 21454) "main" 0xf7fdd430 in __kernel_vsyscall () 1 Thread 0xf7dfd9c0 (LWP 21451) "main" 0xf7fdd430 in __kernel_vsyscall () ----- Any idea where I should go from here? |
March 29, 2012 Re: GC collecting "too much".. | ||||
---|---|---|---|---|
| ||||
Posted in reply to simendsjo | On Thu, 29 Mar 2012 14:14:10 +0200, simendsjo <simendsjo@gmail.com> wrote: > On Mon, 26 Mar 2012 20:15:40 +0200, simendsjo <simendsjo@gmail.com> wrote: > >> On Mon, 26 Mar 2012 17:10:34 +0200, Timon Gehr <timon.gehr@gmx.ch> wrote: >> >>> On 03/26/2012 11:55 AM, simendsjo wrote: >>>> It seems threads created in the c library is totally unknown to D. How >>>> can I make D aware of these threads when there is no library support for >>>> it? >>> >>> You may be looking for this: >>> http://dlang.org/phobos/core_thread.html#thread_attachThis >> >> Thanks, but I tried that too and couldn't get it to work. >> >> I added the following: >> >> extern(C) handler() >> { >> synchronized // needed here to avoid the GC to collect while attaching thread? >> { >> if(!Thread.getThis()) // thread unknown to D >> { >> thread_attachThis(); >> assert(Thread.getThis()); // now D knows about it >> } >> } >> GC.collect(); // still segfaults >> } >> >> Actually, using attachThis segfaults GC.collect() outside the thread handling code too. I compiled with a debug build of druntime, and it fails on dereferencing a pointer, pbot in gcx.d:2511: void mark(void *pbot, void *ptop, int nRecurse) void **p1 = cast(void **)pbot; (...) for (; p1 < p2; p1++) { auto p = cast(byte *)(*p1); // segfault! (...) gdb) print p1 $10 = (void **) 0xf74fc000 (gdb) print *p1 $11 = (void *) 0x0 Some more dbg info: (gdb) bt #0 0x08098753 in gc.gcx.Gcx.mark() (this=0x80df030, nRecurse=64, ptop=0xffffd190, pbot=0xf74f4df4) at src/gc/gcx.d:2511 #1 0x080986d6 in gc.gcx.Gcx.mark() (this=0x80df030, ptop=0xffffd190, pbot=0xf74f4df4) at src/gc/gcx.d:2494 #2 0x0809ef95 in core.thread.thread_scanAll() (this=0xf74f4cf8, p2=0xffffd190, p1=0xf74f4df4, type=<incomplete type>) at src/core/thread.d:2743 #3 0x080a6488 in thread_scanAllType (scan=..., curStackTop=0xf74f4df4) at src/core/thread.d:2698 #4 0x0809ef68 in thread_scanAll (scan=..., curStackTop=0xf74f4df4) at src/core/thread.d:2746 #5 0x08098c34 in gc.gcx.Gcx.fullcollect() (this=0x80df030, stackTop=0xf74f4df4) at src/gc/gcx.d:2776 #6 0x080989f9 in gc.gcx.Gcx.fullcollectshell() (this=0x80df030) at src/gc/gcx.d:2662 #7 0x08096da1 in gc.gcx.GC.fullCollect() (this=0x80df018) at src/gc/gcx.d:1372 #8 0x08086da6 in gc_collect () at src/gc/gc.d:154 #9 0x08085380 in core.memory.GC.collect() () at src/core/memory.d:101 #10 0x08080e2c in c_callback (event=<incomplete type>, conn=0x80ebec0, request_info=0x80ebec0) at main.d:54 #11 0xf7fd28c1 in handle_request(mg_connection*) () from ./libmongoose.so #12 0xf7fd4b7d in worker_thread(mg_context*) () from ./libmongoose.so #13 0xf7f8dd1a in start_thread (arg=0xf74fbb70) at pthread_create.c:304 #14 0xf7ed5e4e in clone () at ../sysdeps/unix/sysv/linux/i386/clone.S:130 Backtrace stopped: Not enough registers or memory available to unwind further (gdb) l 2511 2506 uint changes = 0; 2507 2508 //printf("marking range: %p -> %p\n", pbot, ptop); 2509 for (; p1 < p2; p1++) 2510 { 2511 auto p = cast(byte *)(*p1); 2512 2513 //if (log) debug(PRINTF) printf("\tmark %p\n", p); 2514 if (p >= minAddr && p < maxAddr) 2515 { > And if I change the code to: > extern(C) handler() > { > GC.collect(); > } (snip) Checking with debug build of druntime, and I see it tries to suspend(thread_getThis()). This gives a null reference error as thread_getThis() returns null when it's not attached. |
Copyright © 1999-2021 by the D Language Foundation