On Mon, 2013-03-11 at 17:06 +0100, Martin Drasar wrote:
> Hi,
> 
> I started to work on a networking heavy project and I found out that there is nothing like boost.asio in D's standard library. So I looked around and discovered that vibe.d has nice fiber-based asynchronous library, but sadly it does not play along well with threads. Then I looked some more and found nothing else... (this can of course mean that I can't search)
> 
> I have hacked a solution that implements more or less what I need, but it got me thinking that such thing would be very useful in standard library. So I have some questions (in descending order of importance):
> 
> - Is there some asynchronous networking library for D out there?
> - Is somebody working on such library?
> - Is there a plan to work on such library that may be held back by
> insufficient manpower or by yet-to-be-added libraries like the new I/O?
> - If there is nothing and nobody is working on it, what would be the
> best course of action if I wanted to do it?
> DIP -> implementation -> experimetnal -> std?
> list discussion -> implementation -> experimental -> std?
> implementation <-> discussion -> experimental -> std?
> some other permutation?
> 
> Any other thoughts?
> 
> Cheers,
> Martin

I had plans to implement something like this, but at the moment I am stuck with other important things. I absolutely agree that something like boost::asio::io_service should be part of the standard library, so the situation we have in C++ where every library implements its own event queue could be avoided. (E.g. the only really working way to use Qt and boost::asio together is by using multiple threads) I would love to avoid this in D.

Ideally we would have asynchronous primitives and a synchronous interface compatible to regular streams like the upcoming std.stream based on fibers and an event queue. This way also higher level libraries building on std.stream like a db abstraction library would automatically offer asynchronous operation by simply replacing the used stream with an asynchronous fiber based one.

Just in a nutshell what I would try to implement, the moment I get to this item on my todo list. Although I would contact Soenke first, as he already has a lot of experience with fiber based IO.

Regarding your problems with vibe, what is your particular problem with threads? Have you asked about your problems on http://news.rejectedsoftware.com/groups/rejectedsoftware.vibed/ ?

Maybe they are easily solved? I don't use vibe multi-threaded currently, but if it does not work now, it seems that it is worked on. Also Soenke is usually glad about pull requests ;-)

Best regards,

Robert

On 11.3.2013 17:57, Robert wrote:
> On Mon, 2013-03-11 at 17:06 +0100, Martin Drasar wrote:
> 
> I had plans to implement something like this, but at the moment I am stuck with other important things. I absolutely agree that something like boost::asio::io_service should be part of the standard library, so the situation we have in C++ where every library implements its own event queue could be avoided. (E.g. the only really working way to use Qt and boost::asio together is by using multiple threads) I would love to avoid this in D.
> 
> Ideally we would have asynchronous primitives and a synchronous interface compatible to regular streams like the upcoming std.stream based on fibers and an event queue. This way also higher level libraries building on std.stream like a db abstraction library would automatically offer asynchronous operation by simply replacing the used stream with an asynchronous fiber based one.

Yup, I would like something like that as well.

> Just in a nutshell what I would try to implement, the moment I get to this item on my todo list. Although I would contact Soenke first, as he already has a lot of experience with fiber based IO.

I hope he will chime in and says how the things are going. In the last concurrency post on the forum he said that the concurrency should be ready this or next month.

> Regarding your problems with vibe, what is your particular problem with
> threads? Have you asked about your problems on
> http://news.rejectedsoftware.com/groups/rejectedsoftware.vibed/ ?
> Maybe they are easily solved? I don't use vibe multi-threaded currently,
> but if it does not work now, it seems that it is worked on. Also Soenke
> is usually glad about pull requests ;-)

It was related to blocking operations like readln stopping the event
loop. See
http://news.rejectedsoftware.com/groups/rejectedsoftware.vibed/thread/693/.
No real solution was found as this requires either the fiber support in
std.concurrency or std.concurrency support in vibe.d.

Martin

On Tue, 2013-03-12 at 11:07 +0100, Martin Drasar wrote:
> It was related to blocking operations like readln stopping the event
> loop. See
> http://news.rejectedsoftware.com/groups/rejectedsoftware.vibed/thread/693/.
> No real solution was found as this requires either the fiber support
> in
> std.concurrency or std.concurrency support in vibe.d.
Your problem is exactly the reason why I would love async operation support in the standard library including a standard event loop. The moment you have to wait for something ( a second event loop or just a standard synchronous operation ) you are basically screwed. A single event loop which is capable of waiting for all kind of events (also std.concurrency messages from other threads) would solve the problem.

Currently we have many different event loop implementations like:
- boost::asio::io_service
- libevent
- glib/Qt
- every other GUI toolkit out there

seriously hampering interoperability between libraries. I hope it is possible to establish a standard event loop for at least D in the future, although I still have to check out whether this is realistic or if there are in fact serious technical differences between existing event loops due to different implementation goals. (libevent targeting high performance servers, might not be suitable for a GUI toolkit and vice versa, but I don't know.)


Best regards,

Robert