How difficult would it be for you to split the async IO parts into a
separate library? This would be very much like gevent (python:
http://www.gevent.org/).

That would basically be vibe.core and vibe.stream. Those have no dependencies to other modules so they can already be used without the rest of the code adding overhead of any kind.