So a very basic question about using vibed for a REST service.  I am serving data using REST to another application.  For the time being it is internal so it is not a disaster if the fiber blocks.  But I wanted to understand what I should be doing - the small server app calls library code to retrieve data for a selected series from a large data store (several files, each up to 45G).  This library code uses the standard C/posix APIs for file I/O so isn't written with asynchronous access in mind

What do I need to do to make sure that if the library code to retrieve the data takes a long time to return that the whole vibed event loop does not block?  Should I start a worker task on another thread and wait for it to return?  Or will vibed start another thread to serve a new incoming connection if I am still waiting for data in the meantime.


Thanks.


Laeeth,

Am 06.04.2015 um 17:45 schrieb Laeeth Isharc:
> So a very basic question about using vibed for a REST service.  I am
> serving data using REST to another application.  For the time being it
> is internal so it is not a disaster if the fiber blocks.  But I wanted
> to understand what I should be doing - the small server app calls
> library code to retrieve data for a selected series from a large data
> store (several files, each up to 45G). This library code uses the
> standard C/posix APIs for file I/O so isn't written with asynchronous
> access in mind
>
> What do I need to do to make sure that if the library code to retrieve
> the data takes a long time to return that the whole vibed event loop
> does not block?  Should I start a worker task on another thread and wait
> for it to return?  Or will vibed start another thread to serve a new
> incoming connection if I am still waiting for data in the meantime.
>
>
> Thanks.
>
>
> Laeeth,

You should start a worker thread for this and then use message passing (vibe.core.concurrency) to wait for the results (or alternatively a TaskCondition). I was thinking about including a generic thread proxy in vibe.d that I've used to encapsulate these details when working with libgit in a vibe.d based application, but didn't get to it so so far. I could post a quick gist on GitHub if you want, but I can't remember right now how good/generic the code actually was.

> You should start a worker thread for this and then use message passing (vibe.core.concurrency) to wait for the results (or alternatively a TaskCondition). I was thinking about including a generic thread proxy in vibe.d that I've used to encapsulate these details when working with libgit in a vibe.d based application, but didn't get to it so so far. I could poa quick gist on GitHub if you want, but I can't remember right now how good/generic the code actually was.

Thanks, Sonke.  I appreciate this very much (and of course also your work in building vibe.d in the first place).

I will have a crack at it myself, but if you have time would certainly like to see a gist (but don't bother if the code isn't immediately to hand).  I think compiling a set of snippets of how to do common things in vibe.d would be very helpful in the longer term.  (But maybe this is a task for others to build out).