Hello I have seen the wiki page https://wiki.dlang.org/Event_system and would like to know the current status. Is there a working group for this subject ? This is a topic I'm interested in and did some modest work on some years ago.

At the bottom of the wiki page there is an innocent question regarding TLS which is quite devastating. A worker thread pool system would not support affinity between threads and callback context. Unfortunately, D relies on Thread Local Storage for semi global data. This would be error prone. I saw such error case with people using TLS with Corba.

One way out of this apparent deadlock is if D would provide its own TLS that can be switched between threads. This would allow to preserve affinity between threads and callback execution context. Unfortunately, it would introduce an overhead to access the data in the local storage due to the required indirection. It would also require that the compiler is adapted.

When fibers and multithreading support is built in the language, as in Go, the compiler can do the magic.

I would like to underline that the server side of software development is the easiset side to conquer because the client side as to many different GUIs and execution contexts. But it requieres that the performances are on par with other languages like C, C++, Go or Java.

I would like to add that the switchable TLS is only a half backed solution. It would't work in a multi core context where threads are truly executing in parallel. Two such threads might get the same TLS context which would invalidate its implicit predicate.

Another strategy would be to forbit use of TLS with threads using the event loop. But this might break existing code.

Event loops needs to be thread local not per process.
So many API's such as WinAPI for e.g. GUI's have this requirement in it that its just not worth fighting over.

On Thursday, 5 May 2016 at 08:19:26 UTC, chmike wrote:
> Hello I have seen the wiki page https://wiki.dlang.org/Event_system and would like to know the current status. Is there a working group for this subject ? This is a topic I'm interested in and did some modest work on some years ago.
>
> [...]

We has one: Collie, now is in develop, and use in our server. Use for TCP and http.
It like facebook/wangle: https://github.com/putao-dev/collie

On Thursday, 5 May 2016 at 09:21:04 UTC, rikki cattermole wrote:
> Event loops needs to be thread local not per process.
> So many API's such as WinAPI for e.g. GUI's have this requirement in it that its just not worth fighting over.

I don't understand. Do you mean that these event loops are single threaded and thus don't allow multi threaded use and parallel event handling ?

Single threaded model avoids the overhead of synchronization. That would be another strong argument in favor of single threaded event loop. And another one is that single threaded application is much easier to get right than multi threaded applications.

On the other side, WinAPI is old and the actual hardware evolution goes toward multi core computers and massive true parallelism. At CERN we use 16 core computers. Of course it's good to be backward compatible with existing APIs but D should be designed to best match the future of computing I think.

So it seam the question boils down to determine if it's possible to have the best in both worlds.

I agree that event loops working in isolation is the most simple API from the user perspective and is the most efficient since synchronization can be avoided. But worker thread pools has also its advantages when the app is running on a multicore computer.

On 06/05/2016 9:40 PM, chmike wrote:
> On Thursday, 5 May 2016 at 09:21:04 UTC, rikki cattermole wrote:
>> Event loops needs to be thread local not per process.
>> So many API's such as WinAPI for e.g. GUI's have this requirement in
>> it that its just not worth fighting over.
>
> I don't understand. Do you mean that these event loops are single
> threaded and thus don't allow multi threaded use and parallel event
> handling ?
>
> Single threaded model avoids the overhead of synchronization. That would
> be another strong argument in favor of single threaded event loop. And
> another one is that single threaded application is much easier to get
> right than multi threaded applications.
>
> On the other side, WinAPI is old and the actual hardware evolution goes
> toward multi core computers and massive true parallelism. At CERN we use
> 16 core computers. Of course it's good to be backward compatible with
> existing APIs but D should be designed to best match the future of
> computing I think.
>
> So it seam the question boils down to determine if it's possible to have
> the best in both worlds.
>
> I agree that event loops working in isolation is the most simple API
> from the user perspective and is the most efficient since
> synchronization can be avoided. But worker thread pools has also its
> advantages when the app is running on a multicore computer.

Not even close :)

API's such as WinAPI are designed so that an event loop is per thread. This limitation is quite useful. You can see these limitations in X11 as well.

E.g. you can't go around drawing in another thread for a window.

For example WinAPI's GetMessage function has "Retrieves a message from the calling thread's message queue." https://msdn.microsoft.com/en-nz/library/windows/desktop/ms644936(v=vs.85).aspx

We can't fight stuff like this, its just not possible :)

On Thursday, 5 May 2016 at 08:19:26 UTC, chmike wrote:
> At the bottom of the wiki page there is an innocent question regarding TLS which is quite devastating. A worker thread pool system would not support affinity between threads and callback context. Unfortunately, D relies on Thread Local Storage for semi global data. This would be error prone. I saw such error case with people using TLS with Corba.

You can declare global data with shared qualifier:

int data; //TLS
shared int data; //global

On Thursday, 5 May 2016 at 08:28:36 UTC, chmike wrote:
> I would like to add that the switchable TLS is only a half backed solution. It would't work in a multi core context where threads are truly executing in parallel. Two such threads might get the same TLS context which would invalidate its implicit predicate.

If TLS doesn't work, it's a bug in TLS implementation. I don't think such bug exists. AFAIK TLS works fine on multicore systems.

On Thursday, 5 May 2016 at 08:19:26 UTC, chmike wrote:
> At the bottom of the wiki page there is an innocent question regarding TLS which is quite devastating. A worker thread pool system would not support affinity between threads and callback context. Unfortunately, D relies on Thread Local Storage for semi global data. This would be error prone. I saw such error case with people using TLS with Corba.

It is possible to set thread CPU affinity. Usage of TLS is also crucial in high performance fiber-based async systems (as soon as you have multiple threads) - for example, to implement lock-free TLS cache for all fibers running on that worker thread.

On Thursday, 5 May 2016 at 09:21:04 UTC, rikki cattermole wrote:
> Event loops needs to be thread local not per process.
> So many API's such as WinAPI for e.g. GUI's have this requirement in it that its just not worth fighting over.

It is implementation detail. You can have global event loop and internally distribute work between per-thread event loops - only event callbacks defined within existing task need to be bound to same worker thread. From the developer convenience PoV scheduler / event loop abstraction has to be process-global, I wouldn't consider anything else.

On 06/05/2016 11:21 PM, Dicebot wrote:
> On Thursday, 5 May 2016 at 09:21:04 UTC, rikki cattermole wrote:
>> Event loops needs to be thread local not per process.
>> So many API's such as WinAPI for e.g. GUI's have this requirement in
>> it that its just not worth fighting over.
>
> It is implementation detail. You can have global event loop and
> internally distribute work between per-thread event loops - only event
> callbacks defined within existing task need to be bound to same worker
> thread. From the developer convenience PoV scheduler / event loop
> abstraction has to be process-global, I wouldn't consider anything else.

If you do it per process, it sounds rather messy with synchronization ext.