On 9/9/10 12:19 AM, Masahiro Nakagawa wrote:
> On Wed, 08 Sep 2010 23:53:13 +0900, Johannes Pfau <johannespfau at googlemail.com> wrote:
>
>> On 08.09.2010 16:02, Masahiro Nakagawa wrote:
>>>
>>> I am thinking about std.event.
>>> This module supports system-dependent APIs(kqueue, epoll, etc...) and
>>> abstraction layer.
>>>
>> I think the best solution would be a wrapper / abstraction layer for libev. All of the mentioned APIs (kqueue, epoll, select...) are broken in some way, so writing a new event loop system in D will be painful and it will take a long time until it gets stable. Also libev already has support for more event sources (timer, periodic, signal, stat, ...).
>
> libev is a good library. I often use rev in Ruby programming.
>
> Can Phobos include this library?

To clarify my position: I think it would be great if Phobos included a popular event library, but I'm not an expert in any so I can't initiate such an effort.

Andrei

OK, now I see libev seems to be a tad more general. Could we have a champion for a Phobos binding layer?

Andrei

On 9/9/10 10:43 AM, Johannes Pfau wrote:
>   On 09.09.2010 15:57, Andrei Alexandrescu wrote:
>> I don't know much about licence, but I have this question: which is better, libev or libevent?
>>
> Libev is faster than libevent1 (http://libev.schmorp.de/bench.html) But
> libevent2 (currently in RC state) is said to be as fast as libev, so
> that doesn't really matter. The biggest difference between the libraries
> is the set of event sources they support: Libevent supports i/o and
> timeouts, libev additionally supports "wall clock timers", events on
> posix signals, child process changes, stat events (file changed /
> directory changed,...) and a few more.
> The licenses are almost the same (3 clause BSD for libevent, 2 clause
> BSD for libev).
>
> I've read through the documentation of libev and think it has a nice api and feature set. I have to admit that I have not yet used libev or libevent personally though, so I can't really give you a good comparison. Sean posted somewhere in the newsgroup that he had used libevent, so maybe he can give more information.
>
> Leandro Lucarella has a libev binding&  wrapper for D1, maybe that could
> be adapted for phobos:
> http://www.digitalmars.com/pnews/read.php?server=news.digitalmars.com&group=digitalmars.D&artnum=112967
> <http://www.digitalmars.com/pnews/read.php?server=news.digitalmars.com&group=digitalmars.D&artnum=112967>
>

On Mon, 03 Jan 2011 07:51:56 +0900, Andrei Alexandrescu <andrei at erdani.com> wrote:

> Two questions:
>
> 1. Which is better, libev of libevent?

I think libev is better. libevent becomes libevent2, but performance is
still even.
See this mail: http://lists.schmorp.de/pipermail/libev/2010q4/001246.html

In addition, Node.js, Some MessagePack implementations and many projects
use libev.
libev has a good record :)

> 2. Which of them is available on Windows?

libev on Windows has some limitations but works.
However, I don't know the detail of limitations...


Masahiro

> On 9/8/10 9:53 AM, Johannes Pfau wrote:
>> On 08.09.2010 16:02, Masahiro Nakagawa wrote:
>>>
>>> I am thinking about std.event.
>>> This module supports system-dependent APIs(kqueue, epoll, etc...) and
>>> abstraction layer.
>>>
>> I think the best solution would be a wrapper / abstraction layer for libev. All of the mentioned APIs (kqueue, epoll, select...) are broken in some way, so writing a new event loop system in D will be painful and it will take a long time until it gets stable. Also libev already has support for more event sources (timer, periodic, signal, stat, ...).
>>
>> Quoting libev documentation:
>>
>> EVBACKEND_SELECT
>> Not/completely/standard, as libev tries to roll its own fd_set with no
>> limits on the number of fds, but if that fails, expect a fairly low
>> limit on the number of fds when using this backend.
>> This backend maps|EV_READ|to the|readfds|set and|EV_WRITE|to
>> the|writefds|set (and to work around Microsoft Windows bugs, also onto
>> the|exceptfds|set on that platform).
>>
>> EVBACKEND_POLL
>> It's more complicated than select, but handles sparse fds better and has
>> no artificial limit on the number of fds you can use (except it will
>> slow down considerably with a lot of inactive fds).
>>
>> EVBACKEND_EPOLL
>>
>> The epoll mechanism deserves honorable mention as the most misdesigned of the more advanced event mechanisms: mere annoyances include silently dropping file descriptors, requiring a system call per change per file descriptor (and unnecessary guessing of parameters), problems with dup and so on. The biggest issue is fork races, however - if a program forks then/both/parent and child process have to recreate the epoll set, which can take considerable time (one syscall per file descriptor) and is of course hard to detect.
>>
>> Epoll is also notoriously buggy - embedding epoll fds/should/work, but of course/doesn't/, and epoll just loves to report events for totally/different/file descriptors (even already closed ones, so one cannot even remove them from the set) than registered in the set (especially on SMP systems). Libev tries to counter these spurious notifications by employing an additional generation counter and comparing that against the events to filter out spurious ones, recreating the set when required.
>>
>> EVBACKEND_KQUEUE
>> Kqueue deserves special mention, as at the time of this writing, it was
>> broken on all BSDs except NetBSD (usually it doesn't work reliably with
>> anything but sockets and pipes, except on Darwin, where of course it's
>> completely useless). Unlike epoll, however, whose brokenness is by
>> design, these kqueue bugs can (and eventually will) be fixed without API
>> changes to existing programs. For this reason it's not being
>> "auto-detected" unless you explicitly specify it in the flags (i.e.
>> using|EVBACKEND_KQUEUE|) or libev was compiled on a known-to-be-good
>> (-enough) system like NetBSD.
>>
>> While stopping, setting and starting an I/O watcher does never cause an extra system call as with|EVBACKEND_EPOLL|, it still adds up to two event changes per incident. Support for|fork ()|is very bad (but sane, unlike epoll) and it drops fds silently in similarly hard-to-detect cases
>>
>> EVBACKEND_PORT
>>
>> This uses the Solaris 10 event port mechanism. As with everything on
>> Solaris, it's really slow, but it still scales very well
>> (O(active_fds)).
>>
>> Please note that Solaris event ports can deliver a lot of spurious notifications, so you need to use non-blocking I/O or other means to avoid blocking when no data (or space) is available.
>>
>> While this backend scales well, it requires one system call per active file descriptor per loop iteration. For small and medium numbers of file descriptors a "slow"|EVBACKEND_SELECT|or|EVBACKEND_POLL|backend might perform better
>>
>>
>> --
>> Johannes Pfau
>>
>>
>>
>> _______________________________________________
>> phobos mailing list
>> phobos at puremagic.com
>> http://lists.puremagic.com/mailman/listinfo/phobos
> _______________________________________________
> phobos mailing list
> phobos at puremagic.com
> http://lists.puremagic.com/mailman/listinfo/phobos

Would you want to be the leader on Phobos+libev?

Andrei

On 1/2/11 5:27 PM, Masahiro Nakagawa wrote:
> On Mon, 03 Jan 2011 07:51:56 +0900, Andrei Alexandrescu <andrei at erdani.com> wrote:
>
>> Two questions:
>>
>> 1. Which is better, libev of libevent?
>
> I think libev is better. libevent becomes libevent2, but performance is
> still even.
> See this mail: http://lists.schmorp.de/pipermail/libev/2010q4/001246.html
>
> In addition, Node.js, Some MessagePack implementations and many projects
> use libev.
> libev has a good record :)
>
>> 2. Which of them is available on Windows?
>
> libev on Windows has some limitations but works.
> However, I don't know the detail of limitations...
>
>
> Masahiro
>
>> On 9/8/10 9:53 AM, Johannes Pfau wrote:
>>> On 08.09.2010 16:02, Masahiro Nakagawa wrote:
>>>>
>>>> I am thinking about std.event.
>>>> This module supports system-dependent APIs(kqueue, epoll, etc...) and
>>>> abstraction layer.
>>>>
>>> I think the best solution would be a wrapper / abstraction layer for libev. All of the mentioned APIs (kqueue, epoll, select...) are broken in some way, so writing a new event loop system in D will be painful and it will take a long time until it gets stable. Also libev already has support for more event sources (timer, periodic, signal, stat, ...).
>>>
>>> Quoting libev documentation:
>>>
>>> EVBACKEND_SELECT
>>> Not/completely/standard, as libev tries to roll its own fd_set with no
>>> limits on the number of fds, but if that fails, expect a fairly low
>>> limit on the number of fds when using this backend.
>>> This backend maps|EV_READ|to the|readfds|set and|EV_WRITE|to
>>> the|writefds|set (and to work around Microsoft Windows bugs, also onto
>>> the|exceptfds|set on that platform).
>>>
>>> EVBACKEND_POLL
>>> It's more complicated than select, but handles sparse fds better and has
>>> no artificial limit on the number of fds you can use (except it will
>>> slow down considerably with a lot of inactive fds).
>>>
>>> EVBACKEND_EPOLL
>>>
>>> The epoll mechanism deserves honorable mention as the most misdesigned of the more advanced event mechanisms: mere annoyances include silently dropping file descriptors, requiring a system call per change per file descriptor (and unnecessary guessing of parameters), problems with dup and so on. The biggest issue is fork races, however - if a program forks then/both/parent and child process have to recreate the epoll set, which can take considerable time (one syscall per file descriptor) and is of course hard to detect.
>>>
>>> Epoll is also notoriously buggy - embedding epoll fds/should/work, but of course/doesn't/, and epoll just loves to report events for totally/different/file descriptors (even already closed ones, so one cannot even remove them from the set) than registered in the set (especially on SMP systems). Libev tries to counter these spurious notifications by employing an additional generation counter and comparing that against the events to filter out spurious ones, recreating the set when required.
>>>
>>> EVBACKEND_KQUEUE
>>> Kqueue deserves special mention, as at the time of this writing, it was
>>> broken on all BSDs except NetBSD (usually it doesn't work reliably with
>>> anything but sockets and pipes, except on Darwin, where of course it's
>>> completely useless). Unlike epoll, however, whose brokenness is by
>>> design, these kqueue bugs can (and eventually will) be fixed without API
>>> changes to existing programs. For this reason it's not being
>>> "auto-detected" unless you explicitly specify it in the flags (i.e.
>>> using|EVBACKEND_KQUEUE|) or libev was compiled on a known-to-be-good
>>> (-enough) system like NetBSD.
>>>
>>> While stopping, setting and starting an I/O watcher does never cause an extra system call as with|EVBACKEND_EPOLL|, it still adds up to two event changes per incident. Support for|fork ()|is very bad (but sane, unlike epoll) and it drops fds silently in similarly hard-to-detect cases
>>>
>>> EVBACKEND_PORT
>>>
>>> This uses the Solaris 10 event port mechanism. As with everything on
>>> Solaris, it's really slow, but it still scales very well
>>> (O(active_fds)).
>>>
>>> Please note that Solaris event ports can deliver a lot of spurious notifications, so you need to use non-blocking I/O or other means to avoid blocking when no data (or space) is available.
>>>
>>> While this backend scales well, it requires one system call per active file descriptor per loop iteration. For small and medium numbers of file descriptors a "slow"|EVBACKEND_SELECT|or|EVBACKEND_POLL|backend might perform better
>>>
>>>
>>> --
>>> Johannes Pfau
>>>
>>>
>>>
>>> _______________________________________________
>>> phobos mailing list
>>> phobos at puremagic.com
>>> http://lists.puremagic.com/mailman/listinfo/phobos
>> _______________________________________________
>> phobos mailing list
>> phobos at puremagic.com
>> http://lists.puremagic.com/mailman/listinfo/phobos
> _______________________________________________
> phobos mailing list
> phobos at puremagic.com
> http://lists.puremagic.com/mailman/listinfo/phobos

I will implement std.event after finishing std.socket if someone doesn't
implement this module.
But, I left university and went to work. Now, I am busy...


Masahiro

On Mon, 03 Jan 2011 08:34:53 +0900, Andrei Alexandrescu <andrei at erdani.com> wrote:

> Would you want to be the leader on Phobos+libev?
>
> Andrei
>
> On 1/2/11 5:27 PM, Masahiro Nakagawa wrote:
>> On Mon, 03 Jan 2011 07:51:56 +0900, Andrei Alexandrescu <andrei at erdani.com> wrote:
>>
>>> Two questions:
>>>
>>> 1. Which is better, libev of libevent?
>>
>> I think libev is better. libevent becomes libevent2, but performance is
>> still even.
>> See this mail:
>> http://lists.schmorp.de/pipermail/libev/2010q4/001246.html
>>
>> In addition, Node.js, Some MessagePack implementations and many projects
>> use libev.
>> libev has a good record :)
>>
>>> 2. Which of them is available on Windows?
>>
>> libev on Windows has some limitations but works.
>> However, I don't know the detail of limitations...
>>
>>
>> Masahiro
>>
>>> On 9/8/10 9:53 AM, Johannes Pfau wrote:
>>>> On 08.09.2010 16:02, Masahiro Nakagawa wrote:
>>>>>
>>>>> I am thinking about std.event.
>>>>> This module supports system-dependent APIs(kqueue, epoll, etc...) and
>>>>> abstraction layer.
>>>>>
>>>> I think the best solution would be a wrapper / abstraction layer for
>>>> libev. All of the mentioned APIs (kqueue, epoll, select...) are broken
>>>> in some way, so writing a new event loop system in D will be painful
>>>> and
>>>> it will take a long time until it gets stable. Also libev already has
>>>> support for more event sources (timer, periodic, signal, stat, ...).
>>>>
>>>> Quoting libev documentation:
>>>>
>>>> EVBACKEND_SELECT
>>>> Not/completely/standard, as libev tries to roll its own fd_set with no
>>>> limits on the number of fds, but if that fails, expect a fairly low
>>>> limit on the number of fds when using this backend.
>>>> This backend maps|EV_READ|to the|readfds|set and|EV_WRITE|to
>>>> the|writefds|set (and to work around Microsoft Windows bugs, also onto
>>>> the|exceptfds|set on that platform).
>>>>
>>>> EVBACKEND_POLL
>>>> It's more complicated than select, but handles sparse fds better and
>>>> has
>>>> no artificial limit on the number of fds you can use (except it will
>>>> slow down considerably with a lot of inactive fds).
>>>>
>>>> EVBACKEND_EPOLL
>>>>
>>>> The epoll mechanism deserves honorable mention as the most misdesigned
>>>> of the more advanced event mechanisms: mere annoyances include
>>>> silently
>>>> dropping file descriptors, requiring a system call per change per file
>>>> descriptor (and unnecessary guessing of parameters), problems with dup
>>>> and so on. The biggest issue is fork races, however - if a program
>>>> forks
>>>> then/both/parent and child process have to recreate the epoll set,
>>>> which
>>>> can take considerable time (one syscall per file descriptor) and is of
>>>> course hard to detect.
>>>>
>>>> Epoll is also notoriously buggy - embedding epoll fds/should/work, but of course/doesn't/, and epoll just loves to report events for totally/different/file descriptors (even already closed ones, so one cannot even remove them from the set) than registered in the set (especially on SMP systems). Libev tries to counter these spurious notifications by employing an additional generation counter and comparing that against the events to filter out spurious ones, recreating the set when required.
>>>>
>>>> EVBACKEND_KQUEUE
>>>> Kqueue deserves special mention, as at the time of this writing, it
>>>> was
>>>> broken on all BSDs except NetBSD (usually it doesn't work reliably
>>>> with
>>>> anything but sockets and pipes, except on Darwin, where of course it's
>>>> completely useless). Unlike epoll, however, whose brokenness is by
>>>> design, these kqueue bugs can (and eventually will) be fixed without
>>>> API
>>>> changes to existing programs. For this reason it's not being
>>>> "auto-detected" unless you explicitly specify it in the flags (i.e.
>>>> using|EVBACKEND_KQUEUE|) or libev was compiled on a known-to-be-good
>>>> (-enough) system like NetBSD.
>>>>
>>>> While stopping, setting and starting an I/O watcher does never cause
>>>> an
>>>> extra system call as with|EVBACKEND_EPOLL|, it still adds up to two
>>>> event changes per incident. Support for|fork ()|is very bad (but sane,
>>>> unlike epoll) and it drops fds silently in similarly hard-to-detect
>>>> cases
>>>>
>>>> EVBACKEND_PORT
>>>>
>>>> This uses the Solaris 10 event port mechanism. As with everything on
>>>> Solaris, it's really slow, but it still scales very well
>>>> (O(active_fds)).
>>>>
>>>> Please note that Solaris event ports can deliver a lot of spurious notifications, so you need to use non-blocking I/O or other means to avoid blocking when no data (or space) is available.
>>>>
>>>> While this backend scales well, it requires one system call per active
>>>> file descriptor per loop iteration. For small and medium numbers of
>>>> file
>>>> descriptors a "slow"|EVBACKEND_SELECT|or|EVBACKEND_POLL|backend might
>>>> perform better
>>>>
>>>>
>>>> --
>>>> Johannes Pfau
>>>>
>>>>
>>>>
>>>> _______________________________________________
>>>> phobos mailing list
>>>> phobos at puremagic.com
>>>> http://lists.puremagic.com/mailman/listinfo/phobos
>>> _______________________________________________
>>> phobos mailing list
>>> phobos at puremagic.com
>>> http://lists.puremagic.com/mailman/listinfo/phobos
>> _______________________________________________
>> phobos mailing list
>> phobos at puremagic.com
>> http://lists.puremagic.com/mailman/listinfo/phobos
> _______________________________________________
> phobos mailing list
> phobos at puremagic.com
> http://lists.puremagic.com/mailman/listinfo/phobos

On 1/2/11 6:24 PM, Masahiro Nakagawa wrote:
> I will implement std.event after finishing std.socket if someone doesn't
> implement this module.
> But, I left university and went to work. Now, I am busy...

Congratulations!!

Andrei

Thanks!


Masahiro

On Mon, 03 Jan 2011 10:09:35 +0900, Andrei Alexandrescu <andrei at erdani.com> wrote:

> On 1/2/11 6:24 PM, Masahiro Nakagawa wrote:
>> I will implement std.event after finishing std.socket if someone doesn't
>> implement this module.
>> But, I left university and went to work. Now, I am busy...
>
> Congratulations!!
>
> Andrei
> _______________________________________________
> phobos mailing list
> phobos at puremagic.com
> http://lists.puremagic.com/mailman/listinfo/phobos