On Wed, 08 Sep 2010 23:58:53 +0900, Andrei Alexandrescu <andrei at erdani.com> wrote:

> On 09/08/2010 09:02 AM, Masahiro Nakagawa wrote:
> [snip]
>
> All sounds good! One funny detail:
>
>>> * 2310: what does byes mean?
>>
>> I don't know, but I think byes stands for "blocking yes".
>
> This is cool :o). I had two hypotheses: typo for "bytes" or some sort of closing ack (plural for "bye").
>
>>> * 2934: select() has well-known issues. Any plans to support the newer system-dependent APIs or libevent?
>>
>> I will remove select() from Socket in new module.
>>
>> I am thinking about std.event.
>> This module supports system-dependent APIs(kqueue, epoll, etc...) and
>> abstraction layer.
>
> That would be awesome. Sean was also thinking of it and could provide code review. At best we'd integrate std.event with thread messages.

Yeah. Event is a important feature for inter/inner messaging and RPC :)


Masahiro

On Wed, 08 Sep 2010 23:53:13 +0900, Johannes Pfau <johannespfau at googlemail.com> wrote:

>  On 08.09.2010 16:02, Masahiro Nakagawa wrote:
>>
>> I am thinking about std.event.
>> This module supports system-dependent APIs(kqueue, epoll, etc...) and
>> abstraction layer.
>>
> I think the best solution would be a wrapper / abstraction layer for libev. All of the mentioned APIs (kqueue, epoll, select...) are broken in some way, so writing a new event loop system in D will be painful and it will take a long time until it gets stable. Also libev already has support for more event sources (timer, periodic, signal, stat, ...).

libev is a good library. I often use rev in Ruby programming.

Can Phobos include this library?


Masahiro

 On 09.09.2010 07:19, Masahiro Nakagawa wrote:
> On Wed, 08 Sep 2010 23:53:13 +0900, Johannes Pfau <johannespfau at googlemail.com> wrote:
>
>>  On 08.09.2010 16:02, Masahiro Nakagawa wrote:
>>>
>>> I am thinking about std.event.
>>> This module supports system-dependent APIs(kqueue, epoll, etc...) and
>>> abstraction layer.
>>>
>> I think the best solution would be a wrapper / abstraction layer for libev. All of the mentioned APIs (kqueue, epoll, select...) are broken in some way, so writing a new event loop system in D will be painful and it will take a long time until it gets stable. Also libev already has support for more event sources (timer, periodic, signal, stat, ...).
>
> libev is a good library. I often use rev in Ruby programming.
>
> Can Phobos include this library?
>
libev is using the "Simplified BSD License"
(http://en.wikipedia.org/wiki/BSD_licenses#2-clause_license_.28.22Simplified_BSD_License.22_or_.22FreeBSD_License.22.29).
This license puts no restrictions on using / linking to the library, so
using boost phobos + libev + (any license) user code should not be a
problem. The following clause could be a problem though:
------------------------------------------------------------------------------------
Redistributions in binary form must reproduce the above copyright notice, this list of conditions and the following disclaimer in the documentation and/or other materials provided with the distribution.
------------------------------------------------------------------------------------
I know Andrei and Walter want to avoid such clauses for the standard
library.
But I think libev could be shipped independent of phobos, so maybe
something like "If you want to use std.event you also need to have the
libev library installed" could work. Then Phobos would not have to
include the libev license. Projects using std.event and libev would
still have to include it, though.

There has also been talk about including libcurl which has a similar clause in the license. So maybe a general decision is needed.

-- 
Johannes Pfau

I don't know much about licence, but I have this question: which is better, libev or libevent?

Andrei

On 9/9/10 3:56 CDT, Johannes Pfau wrote:
>   On 09.09.2010 07:19, Masahiro Nakagawa wrote:
>> On Wed, 08 Sep 2010 23:53:13 +0900, Johannes Pfau <johannespfau at googlemail.com>  wrote:
>>
>>>   On 08.09.2010 16:02, Masahiro Nakagawa wrote:
>>>>
>>>> I am thinking about std.event.
>>>> This module supports system-dependent APIs(kqueue, epoll, etc...) and
>>>> abstraction layer.
>>>>
>>> I think the best solution would be a wrapper / abstraction layer for libev. All of the mentioned APIs (kqueue, epoll, select...) are broken in some way, so writing a new event loop system in D will be painful and it will take a long time until it gets stable. Also libev already has support for more event sources (timer, periodic, signal, stat, ...).
>>
>> libev is a good library. I often use rev in Ruby programming.
>>
>> Can Phobos include this library?
>>
> libev is using the "Simplified BSD License"
> (http://en.wikipedia.org/wiki/BSD_licenses#2-clause_license_.28.22Simplified_BSD_License.22_or_.22FreeBSD_License.22.29).
> This license puts no restrictions on using / linking to the library, so
> using boost phobos + libev + (any license) user code should not be a
> problem. The following clause could be a problem though:
> ------------------------------------------------------------------------------------
> Redistributions in binary form must reproduce the above copyright notice, this list of conditions and the following disclaimer in the documentation and/or other materials provided with the distribution.
> ------------------------------------------------------------------------------------
> I know Andrei and Walter want to avoid such clauses for the standard
> library.
> But I think libev could be shipped independent of phobos, so maybe
> something like "If you want to use std.event you also need to have the
> libev library installed" could work. Then Phobos would not have to
> include the libev license. Projects using std.event and libev would
> still have to include it, though.
>
> There has also been talk about including libcurl which has a similar clause in the license. So maybe a general decision is needed.
>

 On 09.09.2010 15:57, Andrei Alexandrescu wrote:
> I don't know much about licence, but I have this question: which is better, libev or libevent?
>
Libev is faster than libevent1 (http://libev.schmorp.de/bench.html) But
libevent2 (currently in RC state) is said to be as fast as libev, so
that doesn't really matter. The biggest difference between the libraries
is the set of event sources they support: Libevent supports i/o and
timeouts, libev additionally supports "wall clock timers", events on
posix signals, child process changes, stat events (file changed /
directory changed,...) and a few more.
The licenses are almost the same (3 clause BSD for libevent, 2 clause
BSD for libev).

I've read through the documentation of libev and think it has a nice api and feature set. I have to admit that I have not yet used libev or libevent personally though, so I can't really give you a good comparison. Sean posted somewhere in the newsgroup that he had used libevent, so maybe he can give more information.

Leandro Lucarella has a libev binding & wrapper for D1, maybe that could
be adapted for phobos:
http://www.digitalmars.com/pnews/read.php?server=news.digitalmars.com&group=digitalmars.D&artnum=112967
<http://www.digitalmars.com/pnews/read.php?server=news.digitalmars.com&group=digitalmars.D&artnum=112967>

-- 
Johannes Pfau

On Sep 9, 2010, at 8:43 AM, Johannes Pfau wrote:

> On 09.09.2010 15:57, Andrei Alexandrescu wrote:
>> I don't know much about licence, but I have this question: which is better, libev or libevent?
>> 
> Libev is faster than libevent1 (http://libev.schmorp.de/bench.html) But
> libevent2 (currently in RC state) is said to be as fast as libev, so
> that doesn't really matter. The biggest difference between the libraries
> is the set of event sources they support: Libevent supports i/o and
> timeouts, libev additionally supports "wall clock timers", events on
> posix signals, child process changes, stat events (file changed /
> directory changed,...) and a few more.
> The licenses are almost the same (3 clause BSD for libevent, 2 clause
> BSD for libev).
> 
> I've read through the documentation of libev and think it has a nice api and feature set. I have to admit that I have not yet used libev or libevent personally though, so I can't really give you a good comparison. Sean posted somewhere in the newsgroup that he had used libevent, so maybe he can give more information.

libev looks nicer than libevent, but either one would be fine.  I'm more interested in finding one with a compatible license, really.

  On 09.09.2010 07:19, Masahiro Nakagawa wrote:
> On Wed, 08 Sep 2010 23:53:13 +0900, Johannes Pfau <johannespfau at googlemail.com> wrote:
>
>>  On 08.09.2010 16:02, Masahiro Nakagawa wrote:
>>>
>>> I am thinking about std.event.
>>> This module supports system-dependent APIs(kqueue, epoll, etc...) and
>>> abstraction layer.
>>>
>> I think the best solution would be a wrapper / abstraction layer for libev. All of the mentioned APIs (kqueue, epoll, select...) are broken in some way, so writing a new event loop system in D will be painful and it will take a long time until it gets stable. Also libev already has support for more event sources (timer, periodic, signal, stat, ...).
>
> libev is a good library. I often use rev in Ruby programming.
>
> Can Phobos include this library?
>
>
> Masahiro
> _______________________________________________
> phobos mailing list
> phobos at puremagic.com
> http://lists.puremagic.com/mailman/listinfo/phobos

Isn't it possible to add a new library not part of the standard where
such modules could be included?
xtrabos? Containing xtb.event, xtb.curl and other libraries with
incompatible licenses? Could also act as an incubator project for phobos.

On Thu, 9 Sep 2010, Simen Endsj? wrote:

>  On 09.09.2010 07:19, Masahiro Nakagawa wrote:
> > On Wed, 08 Sep 2010 23:53:13 +0900, Johannes Pfau <johannespfau at googlemail.com> wrote:
> > 
> > >  On 08.09.2010 16:02, Masahiro Nakagawa wrote:
> > > > 
> > > > I am thinking about std.event.
> > > > This module supports system-dependent APIs(kqueue, epoll, etc...) and
> > > > abstraction layer.
> > > > 
> > > I think the best solution would be a wrapper / abstraction layer for libev. All of the mentioned APIs (kqueue, epoll, select...) are broken in some way, so writing a new event loop system in D will be painful and it will take a long time until it gets stable. Also libev already has support for more event sources (timer, periodic, signal, stat, ...).
> > 
> > libev is a good library. I often use rev in Ruby programming.
> > 
> > Can Phobos include this library?
> > 
> > 
> > Masahiro
> 
> Isn't it possible to add a new library not part of the standard where such
> modules could be included?
> xtrabos? Containing xtb.event, xtb.curl and other libraries with incompatible
> licenses? Could also act as an incubator project for phobos.

Easy enough for minimal integration.  The problem really starts when parts of phobos code want to depend on that library.  At that point it stops becoming optional and adds one more layer of friction to people who want to play with D (and hopefully later use for real).  Things like libc and libm are easy, they're a base part of the OS.  Things like libcurl and libev are _often_ already installed on posix systems but not always, and almost never on windows systems.

On 9/9/10 15:03 CDT, Simen Endsj? wrote:
> Isn't it possible to add a new library not part of the standard where
> such modules could be included?
> xtrabos? Containing xtb.event, xtb.curl and other libraries with
> incompatible licenses? Could also act as an incubator project for phobos.

We have etc/ for such purposes.

Andrei

Two questions:

1. Which is better, libev of libevent?

2. Which of them is available on Windows?


Andrei

On 9/8/10 9:53 AM, Johannes Pfau wrote:
> On 08.09.2010 16:02, Masahiro Nakagawa wrote:
>>
>> I am thinking about std.event.
>> This module supports system-dependent APIs(kqueue, epoll, etc...) and
>> abstraction layer.
>>
> I think the best solution would be a wrapper / abstraction layer for libev. All of the mentioned APIs (kqueue, epoll, select...) are broken in some way, so writing a new event loop system in D will be painful and it will take a long time until it gets stable. Also libev already has support for more event sources (timer, periodic, signal, stat, ...).
>
> Quoting libev documentation:
>
> EVBACKEND_SELECT
> Not/completely/standard, as libev tries to roll its own fd_set with no
> limits on the number of fds, but if that fails, expect a fairly low
> limit on the number of fds when using this backend.
> This backend maps|EV_READ|to the|readfds|set and|EV_WRITE|to
> the|writefds|set (and to work around Microsoft Windows bugs, also onto
> the|exceptfds|set on that platform).
>
> EVBACKEND_POLL
> It's more complicated than select, but handles sparse fds better and has
> no artificial limit on the number of fds you can use (except it will
> slow down considerably with a lot of inactive fds).
>
> EVBACKEND_EPOLL
>
> The epoll mechanism deserves honorable mention as the most misdesigned of the more advanced event mechanisms: mere annoyances include silently dropping file descriptors, requiring a system call per change per file descriptor (and unnecessary guessing of parameters), problems with dup and so on. The biggest issue is fork races, however - if a program forks then/both/parent and child process have to recreate the epoll set, which can take considerable time (one syscall per file descriptor) and is of course hard to detect.
>
> Epoll is also notoriously buggy - embedding epoll fds/should/work, but of course/doesn't/, and epoll just loves to report events for totally/different/file descriptors (even already closed ones, so one cannot even remove them from the set) than registered in the set (especially on SMP systems). Libev tries to counter these spurious notifications by employing an additional generation counter and comparing that against the events to filter out spurious ones, recreating the set when required.
>
> EVBACKEND_KQUEUE
> Kqueue deserves special mention, as at the time of this writing, it was
> broken on all BSDs except NetBSD (usually it doesn't work reliably with
> anything but sockets and pipes, except on Darwin, where of course it's
> completely useless). Unlike epoll, however, whose brokenness is by
> design, these kqueue bugs can (and eventually will) be fixed without API
> changes to existing programs. For this reason it's not being
> "auto-detected" unless you explicitly specify it in the flags (i.e.
> using|EVBACKEND_KQUEUE|) or libev was compiled on a known-to-be-good
> (-enough) system like NetBSD.
>
> While stopping, setting and starting an I/O watcher does never cause an extra system call as with|EVBACKEND_EPOLL|, it still adds up to two event changes per incident. Support for|fork ()|is very bad (but sane, unlike epoll) and it drops fds silently in similarly hard-to-detect cases
>
> EVBACKEND_PORT
>
> This uses the Solaris 10 event port mechanism. As with everything on
> Solaris, it's really slow, but it still scales very well (O(active_fds)).
>
> Please note that Solaris event ports can deliver a lot of spurious notifications, so you need to use non-blocking I/O or other means to avoid blocking when no data (or space) is available.
>
> While this backend scales well, it requires one system call per active file descriptor per loop iteration. For small and medium numbers of file descriptors a "slow"|EVBACKEND_SELECT|or|EVBACKEND_POLL|backend might perform better
>
>
> --
> Johannes Pfau
>
>
>
> _______________________________________________
> phobos mailing list
> phobos at puremagic.com
> http://lists.puremagic.com/mailman/listinfo/phobos