What is the fastest / most scalable way to implement a server (using a Socket) which can handle large numbers of incoming connections? Like, at least 10K, but probably up to 1 million connections.

More specifically:

1) How do I efficiently select the connections (client Socket instances) which have data which is ready to read?

2) How do I efficiently select the connection that are ready to accept data sent to them?
(which are write ready - in other words) ?


I read in the D Cookbook that using the SocketSet is not the fastest way to do this, as it has to iterate through all Socket instances in it, and check a flag on each Socket.

How about https://github.com/dcarp/asynchronous ? Asyncio Socket handling is sometimes quite nice. It's performance is okay for nearly no effort and the code looks clean.
Details here: http://dcarp.github.io/asynchronous/asynchronous/streams/startServer.html

vibe.d also offers a fiber based asyncio way of dealing with sockets.
http://vibed.org/docs#tcp-server

Maybe it fits your needs.

On Monday, 21 December 2015 at 20:20:44 UTC, Stefan wrote:
> How about https://github.com/dcarp/asynchronous ? Asyncio Socket handling is sometimes quite nice. It's performance is okay for nearly no effort and the code looks clean.
> Details here: http://dcarp.github.io/asynchronous/asynchronous/streams/startServer.html
>
> vibe.d also offers a fiber based asyncio way of dealing with sockets.
> http://vibed.org/docs#tcp-server
>
> Maybe it fits your needs.

Thanks - but I am primarily looking for a solution without external frameworks. Frameworks have a way of bloating over time.

On Monday, 21 December 2015 at 20:53:14 UTC, Jakob Jenkov wrote:
> On Monday, 21 December 2015 at 20:20:44 UTC, Stefan wrote:
>> How about https://github.com/dcarp/asynchronous ? Asyncio Socket handling is sometimes quite nice. It's performance is okay for nearly no effort and the code looks clean.
>> Details here: http://dcarp.github.io/asynchronous/asynchronous/streams/startServer.html
>>
>> vibe.d also offers a fiber based asyncio way of dealing with sockets.
>> http://vibed.org/docs#tcp-server
>>
>> Maybe it fits your needs.
>
> Thanks - but I am primarily looking for a solution without external frameworks. Frameworks have a way of bloating over time.

My server uses "poll" for that.

> My server uses "poll" for that.

Okay, how does that work? How do I use "poll" in D?

Link?
Code example?

On Monday, 21 December 2015 at 21:32:55 UTC, Jakob Jenkov wrote:
>> My server uses "poll" for that.
>
> Okay, how does that work? How do I use "poll" in D?
>
> Link?
> Code example?

The same as in C [1].
Just change
#include <poll.h>
to
import core.sys.posix.poll;

[1] http://linux.die.net/man/2/poll

V Mon, 21 Dec 2015 20:53:14 +0000
Jakob Jenkov via Digitalmars-d-learn
<digitalmars-d-learn@puremagic.com> napsáno:

> On Monday, 21 December 2015 at 20:20:44 UTC, Stefan wrote:
> > How about https://github.com/dcarp/asynchronous ? Asyncio
> > Socket handling is sometimes quite nice. It's performance is
> > okay for nearly no effort and the code looks clean.
> > Details here:
> > http://dcarp.github.io/asynchronous/asynchronous/streams/startServer.html
> >
> > vibe.d also offers a fiber based asyncio way of dealing with
> > sockets.
> > http://vibed.org/docs#tcp-server
> >
> > Maybe it fits your needs.
> 
> Thanks - but I am primarily looking for a solution without external frameworks. Frameworks have a way of bloating over time.

Use vibe.d or other library for async io (libasync). If you want to
reinvent the wheel you can use
kqueue(https://github.com/D-Programming-Language/druntime/blob/1f957372e5dadb92ab1d621d68232dbf8a2dbccf/src/core/sys/freebsd/sys/event.d)
for bsd,
epoll(https://github.com/D-Programming-Language/druntime/blob/master/src/core/sys/linux/epoll.d)
for linux and "I do not use windows" for windows.

On Monday, 21 December 2015 at 23:17:45 UTC, Daniel Kozák wrote:
> If you want to reinvent the wheel you can use

It isn't really reinventing the wheel to just use an alternate library... it isn't like the bundled functions with the OS are hard to use and you really should understand how they work anyway to write efficient programs.

V Mon, 21 Dec 2015 23:29:14 +0000
"Adam D. Ruppe via Digitalmars-d-learn"
<digitalmars-d-learn@puremagic.com> napsáno:

> On Monday, 21 December 2015 at 23:17:45 UTC, Daniel Kozák wrote:
> > If you want to reinvent the wheel you can use
> 
> It isn't really reinventing the wheel to just use an alternate library...
I guess you are right

> it isn't like the bundled functions with the OS are hard to use and you really should understand how they work anyway

I agree, if something go wrong it is a nice to know why

> to write efficient programs.
>

I can write efficient programs with vibe.d(libevent,libuv,libasync...)
too :).

Am Mon, 21 Dec 2015 23:29:14 +0000
schrieb Adam D. Ruppe <destructionator@gmail.com>:

> On Monday, 21 December 2015 at 23:17:45 UTC, Daniel Kozák wrote:
> > If you want to reinvent the wheel you can use
> 
> [...] it isn't like the bundled functions with the OS are hard to use [...]
> 

epoll and similar interfaces are not difficult to use. But you
need to be careful to handle all error conditions caused by low level
posix io calls (read/write) correctly. (Partial reads/writes, How
do you handle EINTR? How do you handle error codes returned by the close
function*? ...)

* http://lwn.net/Articles/576478/