September 13, 2021

On Monday, 13 September 2021 at 18:45:22 UTC, jfondren wrote:

>

Instead of using a temporary EpollEvent array in EventQueue.wait, you could make the array an instance variable and have registerEventSource populate it directly

Actually, initial version of all that was using array,
allocated in constructor, but then (when struggling with GC)
I thought that array in stack will press GC less...

... It seems I said something stupid just now )

September 14, 2021

On Monday, 13 September 2021 at 18:42:47 UTC, Steven Schveighoffer wrote:

>

On 9/13/21 1:54 PM, eugene wrote:

>

[...]

The GC only scans things that it knows about.

Inside your EventQueue you have this code:

[...]

Umm is it okay that he declared variables init and idle of type Stage inside the constructor? Maybe that has something to do with this? Also, calling a variable init could be problematic since the compiler assigns a property of the same name to every single type?

September 14, 2021

On Tuesday, 14 September 2021 at 05:49:58 UTC, Tejas wrote:

>

Umm is it okay that he declared variables init and idle of type Stage inside the constructor?

States of a machine are in associative array.
All other machines create their states in constructor,
local variables are for using addReflex() method.

But this stopper machine is 'special' for GC somehow.

September 14, 2021

On Monday, 13 September 2021 at 18:45:22 UTC, jfondren wrote:

>
        auto p = cast(EpollEvent*) pureMalloc(EpollEvent.sizeof);

What? Allocate struct epoll_event on the heap?
It is a feeble joke ;)

static int ecap__add(int fd, void *dptr)
{
        struct epoll_event waitfor = {0};
           int flags, r;

        waitfor.data.ptr = dptr;

        r = epoll_ctl(epfd, EPOLL_CTL_ADD, fd, &waitfor);
        if (-1 == r) {

All fd's (sockets, timers etc) are added the same way
and corresponding EventSources are not destroyed by GC.

September 14, 2021

On Monday, 13 September 2021 at 17:18:30 UTC, eugene wrote:

>

Then after pressing ^C (SIGINT) the program gets SIGSEGV, since references to sg0 and sg1 are no longer valid (they are "sitting" in epoll_event structure).

... forget to mention, crashes here:

    bool wait() {

        const int maxEvents = 8;
        EpollEvent[maxEvents] events;

        if (done)
            return false;

        int n = epoll_wait(id, events.ptr, maxEvents, -1);
        if (-1 == n)
            return false;

        foreach (k; 0 .. n) {
            EventSource s = events[k].es;
            ulong ecode = s.eventCode(events[k].event_mask); // <<<<< SIGSEGV

sg0/sg1 are destroyed, so s points to wrong location.

September 14, 2021

On 9/14/21 1:49 AM, Tejas wrote:

>

On Monday, 13 September 2021 at 18:42:47 UTC, Steven Schveighoffer wrote:

>

On 9/13/21 1:54 PM, eugene wrote:

>

[...]

The GC only scans things that it knows about.

Inside your EventQueue you have this code:

[...]

Umm is it okay that he declared variables init and idle of type Stage inside the constructor? Maybe that has something to do with this? Also, calling a variable init could be problematic since the compiler assigns a property of the same name to every single type?

Declaring a member/field named init is likely a bad idea, but this is not a member, it's just a variable. That's fine. idle doesn't mean anything special to D.

This project is too big and complex for me to diagnose by just reading, it would take some effort, and I don't have the time, sorry. Though as I have learned helping C converts before, most of the time things like this have to do with forgetting to store a GC reference somewhere. It can be subtle too...

I still recommend pinning the object when adding the epoll event and seeing if that helps.

-Steve

September 14, 2021

On 9/14/21 7:31 AM, eugene wrote:

>

On Monday, 13 September 2021 at 17:18:30 UTC, eugene wrote:

>

Then after pressing ^C (SIGINT) the program gets SIGSEGV, since references to sg0 and sg1 are no longer valid (they are "sitting" in epoll_event structure).

... forget to mention, crashes here:

     bool wait() {

         const int maxEvents = 8;
         EpollEvent[maxEvents] events;

         if (done)
             return false;

         int n = epoll_wait(id, events.ptr, maxEvents, -1);
         if (-1 == n)
             return false;

         foreach (k; 0 .. n) {
             EventSource s = events[k].es;
             ulong ecode = s.eventCode(events[k].event_mask); // <<<<< SIGSEGV

sg0/sg1 are destroyed, so s points to wrong location.

Note that s likely still points at a valid memory address. However, when an object is destroyed, its vtable is nulled out (precisely to cause a segfault if you try to use an already-freed object). There is also the possibility the memory block has been reallocated to something else, and that is causing the segfault. But if the segfault is consistent, most likely it's the former problem.

-Steve

September 14, 2021

On Tuesday, 14 September 2021 at 12:09:03 UTC, Steven Schveighoffer wrote:

>

Though as I have learned helping C converts before, most of the time things like this have to do with forgetting to store a GC reference somewhere.

Yeah, in my first version I had

    foreach (k; 0 .. nConnections) {
        auto sm = new EchoClient(rxPool, txPool);
        sm.run();
    }

instead of

    EchoClient[] wrkMachines;
    foreach (k; 0 .. nConnections) {
        auto sm = new EchoClient(rxPool, txPool);
        wrkMachines ~= sm;
        sm.run();
    }

and even

    {
        auto stopper = new Stopper();
        stopper.run();
    }

:)

>

I still recommend pinning the object when adding the epoll event and seeing if that helps.

I understand your idea, but even if this will help, the question
remains - why that particular object is so special for GC.

September 14, 2021

On Tuesday, 14 September 2021 at 12:13:15 UTC, Steven Schveighoffer wrote:

>

On 9/14/21 7:31 AM, eugene wrote:

>

On Monday, 13 September 2021 at 17:18:30 UTC, eugene wrote:
            EventSource s = events[k].es;
            ulong ecode = s.eventCode(events[k].event_mask); // <<<<< SIGSEGV
Note that s likely still points at a valid memory address.

yeah, this address is obtained from OS (epoll_event struct),
compiler can not zero it.

>

However, when an object is destroyed, its vtable is nulled out (precisely to cause a segfault if you try to use an already-freed object).

that's right - calling eventCode() method results in segfault.

September 14, 2021

On 9/14/21 8:42 AM, eugene wrote:

>

On Tuesday, 14 September 2021 at 12:09:03 UTC, Steven Schveighoffer wrote:

> >

I still recommend pinning the object when adding the epoll event and seeing if that helps.

I understand your idea, but even if this will help, the question
remains - why that particular object is so special for GC.

Philosophically, it places the responsibility of making sure the object is valid while using it on the thing that chooses to store it outside the GC's view.

Looking at your examples, you are having to store these object references elsewhere, surrounding seemingly innocuous and normal D usage of objects. You have put the burden on the caller to make sure the implementation details are sound.

But I agree that a superficial reading of your code seems like it ought to not be collected, and that problem is also worth figuring out. I have high confidence that it's probably not a design flaw in the GC, but rather some misunderstanding of GC-allocated lifetimes in your code. But that doesn't mean it's not actually a bug somewhere in D.

-Steve