March 25, 2012
I'm doing some coding against a c library, and Ds GC keeps collecting c owned objects (I think - disabling the GC makes everything work)
But how can I figure out what the GC is (read: I am) fucking up?
I have some to!string(c_struct_field) and format("%s", c_struct_field) and field = c_fields[i] etc etc
March 25, 2012
On Sunday, 25 March 2012 at 19:15:05 UTC, simendsjo wrote:
> I'm doing some coding against a c library, and Ds GC keeps collecting c owned objects (I think - disabling the GC makes everything work)

Three alternative solutions:
- Allocate from the C heap the memory that C will need to use, and free it manually or in a struct destructor (RAII) or with scope(exit).
- Keep a pointer to the D-GC memory in the D code too.
- In core.memory there are ways to disable scanning of a memory zone. Maybe it's usable for your purposes too.

Bye,
bearophile
March 26, 2012
On Sun, 25 Mar 2012 22:18:02 +0200, bearophile <bearophileHUGS@lycos.com> wrote:

> On Sunday, 25 March 2012 at 19:15:05 UTC, simendsjo wrote:
>> I'm doing some coding against a c library, and Ds GC keeps collecting c owned objects (I think - disabling the GC makes everything work)
>
> Three alternative solutions:
> - Allocate from the C heap the memory that C will need to use, and free it manually or in a struct destructor (RAII) or with scope(exit).
> - Keep a pointer to the D-GC memory in the D code too.
> - In core.memory there are ways to disable scanning of a memory zone. Maybe it's usable for your purposes too.
>
> Bye,
> bearophile


I've been able to find where the code fails, but now I don't understand what's happening at all.
Is calling GC.collect() from an extern(C) function undefined?

The following code just starts a mongoose web server and tries to run GC.collect in the handler.

extern(C) void* cb(mg_event event, mg_connection* conn, mg_request_info* request_info)
{
    GC.collect(); // segfault
    return null;
}

void main()
{
    auto opts = ["listening_ports", "6969"].map!(toUTFz!(char*))().array();
    mg_start(&cb, null, cast(const(char**))opts);
    GC.collect(); // no problem
    readln();
}

If I collect memory from main(), it works as expected (removing the collect from cb of course)
while(readln().chomp() != "q") { GC.collect(); }

The documentation in core.memory is a bit sparse on how the GC works. Are there any articles on the D GC?
March 26, 2012
On Mon, 26 Mar 2012 10:13:35 +0200, simendsjo <simendsjo@gmail.com> wrote:

> On Sun, 25 Mar 2012 22:18:02 +0200, bearophile <bearophileHUGS@lycos.com> wrote:
>
>> On Sunday, 25 March 2012 at 19:15:05 UTC, simendsjo wrote:
>>> I'm doing some coding against a c library, and Ds GC keeps collecting c owned objects (I think - disabling the GC makes everything work)
>>
>> Three alternative solutions:
>> - Allocate from the C heap the memory that C will need to use, and free it manually or in a struct destructor (RAII) or with scope(exit).
>> - Keep a pointer to the D-GC memory in the D code too.
>> - In core.memory there are ways to disable scanning of a memory zone. Maybe it's usable for your purposes too.
>>
>> Bye,
>> bearophile
>
>
> I've been able to find where the code fails, but now I don't understand what's happening at all.
> Is calling GC.collect() from an extern(C) function undefined?
>
> The following code just starts a mongoose web server and tries to run GC.collect in the handler.
>
> extern(C) void* cb(mg_event event, mg_connection* conn, mg_request_info* request_info)
> {
>      GC.collect(); // segfault
>      return null;
> }
>
> void main()
> {
>      auto opts = ["listening_ports", "6969"].map!(toUTFz!(char*))().array();
>      mg_start(&cb, null, cast(const(char**))opts);
>      GC.collect(); // no problem
>      readln();
> }
>
> If I collect memory from main(), it works as expected (removing the collect from cb of course)
> while(readln().chomp() != "q") { GC.collect(); }
>
> The documentation in core.memory is a bit sparse on how the GC works. Are there any articles on the D GC?

It seems threads created in the c library is totally unknown to D. How can I make D aware of these threads when there is no library support for it?
March 26, 2012
On 03/26/2012 11:55 AM, simendsjo wrote:
> It seems threads created in the c library is totally unknown to D. How
> can I make D aware of these threads when there is no library support for
> it?

You may be looking for this:
http://dlang.org/phobos/core_thread.html#thread_attachThis
March 26, 2012
On Mon, 26 Mar 2012 17:10:34 +0200, Timon Gehr <timon.gehr@gmx.ch> wrote:

> On 03/26/2012 11:55 AM, simendsjo wrote:
>> It seems threads created in the c library is totally unknown to D. How
>> can I make D aware of these threads when there is no library support for
>> it?
>
> You may be looking for this:
> http://dlang.org/phobos/core_thread.html#thread_attachThis

Thanks, but I tried that too and couldn't get it to work.

I added the following:

extern(C) handler()
{
    synchronized // needed here to avoid the GC to collect while attaching thread?
    {
        if(!Thread.getThis()) // thread unknown to D
        {
            thread_attachThis();
            assert(Thread.getThis()); // now D knows about it
        }
    }
    GC.collect(); // still segfaults
}

Actually, using attachThis segfaults GC.collect() outside the thread handling code too.
March 29, 2012
On Mon, 26 Mar 2012 20:15:40 +0200, simendsjo <simendsjo@gmail.com> wrote:

> On Mon, 26 Mar 2012 17:10:34 +0200, Timon Gehr <timon.gehr@gmx.ch> wrote:
>
>> On 03/26/2012 11:55 AM, simendsjo wrote:
>>> It seems threads created in the c library is totally unknown to D. How
>>> can I make D aware of these threads when there is no library support for
>>> it?
>>
>> You may be looking for this:
>> http://dlang.org/phobos/core_thread.html#thread_attachThis
>
> Thanks, but I tried that too and couldn't get it to work.
>
> I added the following:
>
> extern(C) handler()
> {
>      synchronized // needed here to avoid the GC to collect while attaching thread?
>      {
>          if(!Thread.getThis()) // thread unknown to D
>          {
>              thread_attachThis();
>              assert(Thread.getThis()); // now D knows about it
>          }
>      }
>      GC.collect(); // still segfaults
> }
>
> Actually, using attachThis segfaults GC.collect() outside the thread handling code too.


Some info from gdb:

[New Thread 0xf7cfcb70 (LWP 18614)]
[New Thread 0xf74fbb70 (LWP 18615)]

Program received signal SIGUSR1, User defined signal 1.
0xf7fdd430 in __kernel_vsyscall ()

(gdb) info threads
  Id   Target Id         Frame
  3    Thread 0xf74fbb70 (LWP 18615) "main" 0xf7fdd430 in __kernel_vsyscall ()
  2    Thread 0xf7cfcb70 (LWP 18614) "main" 0xf7fdd430 in __kernel_vsyscall ()
* 1    Thread 0xf7dfd9c0 (LWP 18611) "main" 0xf7fdd430 in __kernel_vsyscall ()

(gdb) c
Continuing.

Program received signal SIGSEGV, Segmentation fault.
[Switching to Thread 0xf74fbb70 (LWP 18615)]
0x080855b4 in gc.gcx.Gcx.mark() ()
(gdb) info threads
  Id   Target Id         Frame
* 3    Thread 0xf74fbb70 (LWP 18615) "main" 0x080855b4 in gc.gcx.Gcx.mark() ()
  2    Thread 0xf7cfcb70 (LWP 18614) "main" 0xf7fdd430 in __kernel_vsyscall ()
  1    Thread 0xf7dfd9c0 (LWP 18611) "main" 0xf7fdd430 in __kernel_vsyscall ()

------------

And if I change the code to:
extern(C) handler()
{
    GC.collect();
}

[New Thread 0xf7cfcb70 (LWP 21454)]
[New Thread 0xf74fbb70 (LWP 21455)]

Program received signal SIGSEGV, Segmentation fault.
[Switching to Thread 0xf74fbb70 (LWP 21455)]
0x0808970e in core.thread.thread_suspendAll() ()
(gdb) info threads
  Id   Target Id         Frame
* 3    Thread 0xf74fbb70 (LWP 21455) "main" 0x0808970e in core.thread.thread_suspendAll() ()
  2    Thread 0xf7cfcb70 (LWP 21454) "main" 0xf7fdd430 in __kernel_vsyscall ()
  1    Thread 0xf7dfd9c0 (LWP 21451) "main" 0xf7fdd430 in __kernel_vsyscall ()


-----

Any idea where I should go from here?
March 29, 2012
On Thu, 29 Mar 2012 14:14:10 +0200, simendsjo <simendsjo@gmail.com> wrote:

> On Mon, 26 Mar 2012 20:15:40 +0200, simendsjo <simendsjo@gmail.com> wrote:
>
>> On Mon, 26 Mar 2012 17:10:34 +0200, Timon Gehr <timon.gehr@gmx.ch> wrote:
>>
>>> On 03/26/2012 11:55 AM, simendsjo wrote:
>>>> It seems threads created in the c library is totally unknown to D. How
>>>> can I make D aware of these threads when there is no library support for
>>>> it?
>>>
>>> You may be looking for this:
>>> http://dlang.org/phobos/core_thread.html#thread_attachThis
>>
>> Thanks, but I tried that too and couldn't get it to work.
>>
>> I added the following:
>>
>> extern(C) handler()
>> {
>>      synchronized // needed here to avoid the GC to collect while attaching thread?
>>      {
>>          if(!Thread.getThis()) // thread unknown to D
>>          {
>>              thread_attachThis();
>>              assert(Thread.getThis()); // now D knows about it
>>          }
>>      }
>>      GC.collect(); // still segfaults
>> }
>>
>> Actually, using attachThis segfaults GC.collect() outside the thread handling code too.

I compiled with a debug build of druntime, and it fails on dereferencing a pointer, pbot in gcx.d:2511:

void mark(void *pbot, void *ptop, int nRecurse)
        void **p1 = cast(void **)pbot;
(...)
        for (; p1 < p2; p1++)
        {
            auto p = cast(byte *)(*p1); // segfault!
(...)

gdb) print p1
$10 = (void **) 0xf74fc000
(gdb) print *p1
$11 = (void *) 0x0

Some more dbg info:

(gdb) bt
#0  0x08098753 in gc.gcx.Gcx.mark() (this=0x80df030, nRecurse=64, ptop=0xffffd190, pbot=0xf74f4df4) at src/gc/gcx.d:2511
#1  0x080986d6 in gc.gcx.Gcx.mark() (this=0x80df030, ptop=0xffffd190, pbot=0xf74f4df4) at src/gc/gcx.d:2494
#2  0x0809ef95 in core.thread.thread_scanAll() (this=0xf74f4cf8, p2=0xffffd190, p1=0xf74f4df4, type=<incomplete type>)
    at src/core/thread.d:2743
#3  0x080a6488 in thread_scanAllType (scan=..., curStackTop=0xf74f4df4) at src/core/thread.d:2698
#4  0x0809ef68 in thread_scanAll (scan=..., curStackTop=0xf74f4df4) at src/core/thread.d:2746
#5  0x08098c34 in gc.gcx.Gcx.fullcollect() (this=0x80df030, stackTop=0xf74f4df4) at src/gc/gcx.d:2776
#6  0x080989f9 in gc.gcx.Gcx.fullcollectshell() (this=0x80df030) at src/gc/gcx.d:2662
#7  0x08096da1 in gc.gcx.GC.fullCollect() (this=0x80df018) at src/gc/gcx.d:1372
#8  0x08086da6 in gc_collect () at src/gc/gc.d:154
#9  0x08085380 in core.memory.GC.collect() () at src/core/memory.d:101
#10 0x08080e2c in c_callback (event=<incomplete type>, conn=0x80ebec0, request_info=0x80ebec0) at main.d:54
#11 0xf7fd28c1 in handle_request(mg_connection*) () from ./libmongoose.so
#12 0xf7fd4b7d in worker_thread(mg_context*) () from ./libmongoose.so
#13 0xf7f8dd1a in start_thread (arg=0xf74fbb70) at pthread_create.c:304
#14 0xf7ed5e4e in clone () at ../sysdeps/unix/sysv/linux/i386/clone.S:130
Backtrace stopped: Not enough registers or memory available to unwind further
(gdb) l 2511
2506            uint changes = 0;
2507
2508            //printf("marking range: %p -> %p\n", pbot, ptop);
2509            for (; p1 < p2; p1++)
2510            {
2511                auto p = cast(byte *)(*p1);
2512
2513                //if (log) debug(PRINTF) printf("\tmark %p\n", p);
2514                if (p >= minAddr && p < maxAddr)
2515                {



> And if I change the code to:
> extern(C) handler()
> {
>      GC.collect();
> }
(snip)
Checking with debug build of druntime, and I see it tries to suspend(thread_getThis()). This gives a null reference error as thread_getThis() returns null when it's not attached.
Top | Discussion index | About this forum | D home