I created the simplest possible example as explained by the Vibe-D community in [1]. The exact source code of what I run is in [2].

On Windows I get a socket handle leak warning on shutdown with crtl+c from terminal after running the executable.

> [main(----) INF] Listening for requests on http://[::1]:8080/
> [main(----) INF] Listening for requests on http://127.0.0.1:8080/
> [main(----) INF] Please open http://127.0.0.1:8080/ in your browser.
> [00000000(----) INF] Received signal 2. Shutting down.
> Warning: 2 socket handles leaked at driver^ Cshutdown

On Ubuntu 20.04 I get leaking drivers warning with the same process.
> [main(----) INF] Listening for requests on http://[::1]:8080/
> [main(----) INF] Listening for requests on http://127.0.0.1:8080/
> [main(----) INF] Please open http://127.0.0.1:8080/ in your browser.
> ^C[main(----) INF] Received signal 2. Shutting down.
> Warning (thread: main): leaking eventcore driver because there are still active handles
>    FD 6 (streamListen)
>    FD 7 (streamListen)
> 
> Warning (thread: main): leaking eventcore driver because there are still active handles
>    FD 6 (streamListen)
>    FD 7 (streamListen)

I really don't know what this is all about but it is at the core of my Vibe-D development. So any pointers you might have would be very helpful to me.

Thanks in advance.

S

[1] https://vibed.org/blog/posts/a-scalable-chat-room-service-in-d
[2] https://github.com/SelimOzel/vibe_noLeaks

On 1/1/21 5:07 PM, Selim Ozel wrote:
> I created the simplest possible example as explained by the Vibe-D community in [1]. The exact source code of what I run is in [2].
> 
> On Windows I get a socket handle leak warning on shutdown with crtl+c from terminal after running the executable.
> 
>> [main(----) INF] Listening for requests on http://[::1]:8080/
>> [main(----) INF] Listening for requests on http://127.0.0.1:8080/
>> [main(----) INF] Please open http://127.0.0.1:8080/ in your browser.
>> [00000000(----) INF] Received signal 2. Shutting down.
>> Warning: 2 socket handles leaked at driver^ Cshutdown
> 
> On Ubuntu 20.04 I get leaking drivers warning with the same process.
>> [main(----) INF] Listening for requests on http://[::1]:8080/
>> [main(----) INF] Listening for requests on http://127.0.0.1:8080/
>> [main(----) INF] Please open http://127.0.0.1:8080/ in your browser.
>> ^C[main(----) INF] Received signal 2. Shutting down.
>> Warning (thread: main): leaking eventcore driver because there are still active handles
>>    FD 6 (streamListen)
>>    FD 7 (streamListen)
>>
>> Warning (thread: main): leaking eventcore driver because there are still active handles
>>    FD 6 (streamListen)
>>    FD 7 (streamListen)
> 
> I really don't know what this is all about but it is at the core of my Vibe-D development. So any pointers you might have would be very helpful to me.
> 
> Thanks in advance.
>

1. the sockets are leaked for a reason that is pretty obscure -- namely, the GC might need to access those sockets as the process is shut down. Prior to this, the end result of vibe.d server was frequently a segfault.
2. The reason they are leaking is most likely because you still have a listening socket somewhere. I wish it would tell you how that socket was allocated, but it doesn't.

To fix, make sure all your listening sockets are closed. In my vibe.d app, I have the following:

auto listener = listenHTTP(settings, router);
scope(exit) listener.stopListening();

I also clean up my session store connection (something I had to add support for in vibe.d), which was a different source of leaking handles.

I also clean up database connections, which might be cached.

And finally, even with all this, I still get leaking driver messages if an HTTP keepalive socket is open.

I really feel like vibe.d should give you the option of not printing this message, as most of the time, it's something you can ignore.

-Steve

On Saturday, 2 January 2021 at 00:28:43 UTC, Steven Schveighoffer wrote:
> On 1/1/21 5:07 PM, Selim Ozel wrote:
>> I created the simplest possible example as explained by the Vibe-D community in [1]. The exact source code of what I run is in [2].
>> 
>> On Windows I get a socket handle leak warning on shutdown with crtl+c from terminal after running the executable.
>> 
>>> [...]
>> 
>> On Ubuntu 20.04 I get leaking drivers warning with the same process.
>>> [...]
>> 
>> I really don't know what this is all about but it is at the core of my Vibe-D development. So any pointers you might have would be very helpful to me.
>> 
>> Thanks in advance.
>>
>
> 1. the sockets are leaked for a reason that is pretty obscure -- namely, the GC might need to access those sockets as the process is shut down. Prior to this, the end result of vibe.d server was frequently a segfault.
> 2. The reason they are leaking is most likely because you still have a listening socket somewhere. I wish it would tell you how that socket was allocated, but it doesn't.
>
> To fix, make sure all your listening sockets are closed. In my vibe.d app, I have the following:
>
> auto listener = listenHTTP(settings, router);
> scope(exit) listener.stopListening();
>
> I also clean up my session store connection (something I had to add support for in vibe.d), which was a different source of leaking handles.
>
> I also clean up database connections, which might be cached.
>
> And finally, even with all this, I still get leaking driver messages if an HTTP keepalive socket is open.
>
> I really feel like vibe.d should give you the option of not printing this message, as most of the time, it's something you can ignore.
>
> -Steve

Hey Steve. Thanks a ton for all the tips. FWIW I am writing down my findings below because just maybe they might be helpful for someone else later on.

The scope guard seems to have fixed some of the leak complaints. Unfortunately there is still a leak after a single connection.

Without connection Windows 10:
> Running .\vibe_noleaks.exe
> [main(----) INF] Listening for requests on http://[::1]:8080/
> [main(----) INF] Listening for requests on http://127.0.0.1:8080/
> [main(----) INF] Please open http://127.0.0.1:8080/ in your browser.
> [00000000(----) INF] Received signal 2. Shutting down.
> [main(----) INF] Stopped to listen for HTTP requests on ::1:8080
> [main(----) INF] Stopped to listen for HTTP requests on 127.0.0.1:8080

Without connection Ubuntu 20.04:
> Running ./vibe_noleaks
> [main(----) INF] Listening for requests on http://[::1]:8080/
> [main(----) INF] Listening for requests on http://127.0.0.1:8080/
> [main(----) INF] Please open http://127.0.0.1:8080/ in your browser.
> ^C[main(----) INF] Received signal 2. Shutting down.
> [main(----) INF] Stopped to listen for HTTP requests on ::1:8080
> [main(----) INF] Stopped to listen for HTTP requests on 127.0.0.1:8080

After logging into to 127.0.0.1 for a single time in my browser, if I do a ctrl+c it still leaks two socket handles.

With connection Windows 10:
> Running .\vibe_noleaks.exe
> [main(----) INF] Listening for requests on http://[::1]:8080/
> [main(----) INF] Listening for requests on http://127.0.0.1:8080/
> [main(----) INF] Please open http://127.0.0.1:8080/ in your browser.
> [00000000(----) INF] Received signal 2. Shutting down.
> [main(----) INF] Stopped to listen for HTTP^ requests on C::1:8080
> [main(----
> ) INFC:\Software\vibe_noLeaks>] Stopped to listen for HTTP requests on 127.0.0.1:8080
> Warning: 2 socket handles leaked at driver shutdown.
> Warning: 2 socket handles leaked at driver shutdown.


I think vibe-d is also leaking more sockets when there is a web interface attached (not included in my toy repository). These haven't stopped me from developing but they are just things I wanted to write down and learn more before building even more infrastructure with it.

I might dive into vibe-d codebase at some point too.

Best,
S

On Friday, 1 January 2021 at 22:07:28 UTC, Selim Ozel wrote:
> I created the simplest possible example as explained by the Vibe-D community in [1]. The exact source code of what I run is in [2].
>
> On Windows I get a socket handle leak warning on shutdown with crtl+c from terminal after running the executable.
>
>> [...]
>
> On Ubuntu 20.04 I get leaking drivers warning with the same process.
>>    [...]
>
> I really don't know what this is all about but it is at the core of my Vibe-D development. So any pointers you might have would be very helpful to me.
>
> Thanks in advance.
>
> S
>
> [1] https://vibed.org/blog/posts/a-scalable-chat-room-service-in-d
> [2] https://github.com/SelimOzel/vibe_noLeaks

For further reference, I also went through this issue [1]. It seems like I am seeing the same behavior as kinexis-uk.

S

[1] https://github.com/vibe-d/vibe.d/issues/2245

On Friday, 1 January 2021 at 22:07:28 UTC, Selim Ozel wrote:
> I created the simplest possible example as explained by the Vibe-D community in [1]. The exact source code of what I run is in [2].
>
> On Windows I get a socket handle leak warning on shutdown with crtl+c from terminal after running the executable.
>
>> [...]
>
> On Ubuntu 20.04 I get leaking drivers warning with the same process.
>>    [...]
>
> I really don't know what this is all about but it is at the core of my Vibe-D development. So any pointers you might have would be very helpful to me.
>
> Thanks in advance.
>
> S
>
> [1] https://vibed.org/blog/posts/a-scalable-chat-room-service-in-d
> [2] https://github.com/SelimOzel/vibe_noLeaks

Add this to your dub.json file to fix it

"versions": [ "VibeHighEventPriority" ]


This issue should be fixed in next vibe.d release

On 1/2/21 12:52 PM, Selim Ozel wrote:

> 
> After logging into to 127.0.0.1 for a single time in my browser, if I do a ctrl+c it still leaks two socket handles.
> 
> With connection Windows 10:
>> Running .\vibe_noleaks.exe
>> [main(----) INF] Listening for requests on http://[::1]:8080/
>> [main(----) INF] Listening for requests on http://127.0.0.1:8080/
>> [main(----) INF] Please open http://127.0.0.1:8080/ in your browser.
>> [00000000(----) INF] Received signal 2. Shutting down.
>> [main(----) INF] Stopped to listen for HTTP^ requests on C::1:8080
>> [main(----
>> ) INFC:\Software\vibe_noLeaks>] Stopped to listen for HTTP requests on 127.0.0.1:8080
>> Warning: 2 socket handles leaked at driver shutdown.
>> Warning: 2 socket handles leaked at driver shutdown.

This is normal. The server uses keepalive connections, so that in case any more requests arrive on the same connection, the initial connection setup does not need to be established. Well, at least that is what I think is happening.

If you want a few seconds (I think 5 or so), then you won't get these.

It would be good if vibe-d could provide a way to shut down any keepalive connections when the server is shutting down.

-Steve

On 1/3/21 6:53 PM, aberba wrote:
> On Friday, 1 January 2021 at 22:07:28 UTC, Selim Ozel wrote:
>> I created the simplest possible example as explained by the Vibe-D community in [1]. The exact source code of what I run is in [2].
>>
>> On Windows I get a socket handle leak warning on shutdown with crtl+c from terminal after running the executable.
>>
>>> [...]
>>
>> On Ubuntu 20.04 I get leaking drivers warning with the same process.
>>>    [...]
>>
>> I really don't know what this is all about but it is at the core of my Vibe-D development. So any pointers you might have would be very helpful to me.
>>
>> Thanks in advance.
>>
>> S
>>
>> [1] https://vibed.org/blog/posts/a-scalable-chat-room-service-in-d
>> [2] https://github.com/SelimOzel/vibe_noLeaks
> 
> Add this to your dub.json file to fix it
> 
> "versions": [ "VibeHighEventPriority" ]
> 
> 
> This issue should be fixed in next vibe.d release

No, this is a different issue. That issue is when the server doesn't shut down so you can't re-bind to the port without killing it with a SIGKILL signal.

And thankfully it should be fixed in the latest release (not vibe.d, but eventcore I think).

-Steve

On 1/4/21 12:17 PM, Steven Schveighoffer wrote:
> If you want a few seconds

*wait* a few seconds

-Steve

On Sunday, 3 January 2021 at 23:53:54 UTC, aberba wrote:
> On Friday, 1 January 2021 at 22:07:28 UTC, Selim Ozel wrote:
>> I created the simplest possible example as explained by the Vibe-D community in [1]. The exact source code of what I run is in [2].
>>
>> On Windows I get a socket handle leak warning on shutdown with crtl+c from terminal after running the executable.
>>
>>> [...]
>>
>> On Ubuntu 20.04 I get leaking drivers warning with the same process.
>>>    [...]
>>
>> I really don't know what this is all about but it is at the core of my Vibe-D development. So any pointers you might have would be very helpful to me.
>>
>> Thanks in advance.
>>
>> S
>>
>> [1] https://vibed.org/blog/posts/a-scalable-chat-room-service-in-d
>> [2] https://github.com/SelimOzel/vibe_noLeaks
>
> Add this to your dub.json file to fix it
>
> "versions": [ "VibeHighEventPriority" ]
>
>
> This issue should be fixed in next vibe.d release

Thanks. Not sure if relevant to this one but I came across that one as well before [1]. The symptom was "The simple hello world app I build with vibe-d does not seem to work on the second compile+execution." on an Ubuntu 20.04 EC2.

[1] https://forum.dlang.org/post/mailman.6758.1605999004.31109.digitalmars-d@puremagic.com

On Monday, 4 January 2021 at 17:17:10 UTC, Steven Schveighoffer wrote:
> On 1/2/21 12:52 PM, Selim Ozel wrote:
>
>> 
>> After logging into to 127.0.0.1 for a single time in my browser, if I do a ctrl+c it still leaks two socket handles.
>> 
>> With connection Windows 10:
>>> Running .\vibe_noleaks.exe
>>> [main(----) INF] Listening for requests on http://[::1]:8080/
>>> [main(----) INF] Listening for requests on http://127.0.0.1:8080/
>>> [main(----) INF] Please open http://127.0.0.1:8080/ in your browser.
>>> [00000000(----) INF] Received signal 2. Shutting down.
>>> [main(----) INF] Stopped to listen for HTTP^ requests on C::1:8080
>>> [main(----
>>> ) INFC:\Software\vibe_noLeaks>] Stopped to listen for HTTP requests on 127.0.0.1:8080
>>> Warning: 2 socket handles leaked at driver shutdown.
>>> Warning: 2 socket handles leaked at driver shutdown.
>
> This is normal. The server uses keepalive connections, so that in case any more requests arrive on the same connection, the initial connection setup does not need to be established. Well, at least that is what I think is happening.
>
> If you want a few seconds (I think 5 or so), then you won't get these.
>
> It would be good if vibe-d could provide a way to shut down any keepalive connections when the server is shutting down.
>
> -Steve

That's interesting. I actually started to dive deeper into those and tried to pinpoint the lines of code that result in additional open sockets upon new http connections; although my understanding of vibe-d is a bit too low at this point to figure out what's exactly happening.

I think from a user perspective having something a bit friendlier on warning side would be helpful. Do you have any suggestions in mind towards that? I have a bit of time this week and I could take a stab at it.

Best,
S