Jump to page: 1 2 3
Thread overview
[phobos] Breaking changes for std.socket improvement
Jul 05, 2010
Masahiro Nakagawa
Jul 07, 2010
Sean Kelly
Jul 08, 2010
Masahiro Nakagawa
Jul 08, 2010
Sean Kelly
Jul 09, 2010
Masahiro Nakagawa
Sep 08, 2010
Masahiro Nakagawa
[phobos] std.event / event loop for phobos
Sep 08, 2010
Johannes Pfau
Sep 09, 2010
Masahiro Nakagawa
Sep 09, 2010
Johannes Pfau
Sep 09, 2010
Johannes Pfau
Sep 09, 2010
Sean Kelly
Sep 09, 2010
Simen Endsjø
Sep 09, 2010
Brad Roberts
Jan 02, 2011
Masahiro Nakagawa
Jan 03, 2011
Masahiro Nakagawa
Jan 03, 2011
Masahiro Nakagawa
Sep 09, 2010
Masahiro Nakagawa
July 06, 2010
std.socket lacks some features and I don't like current design. So, I rewrote this module.

Summary:

- Change API of Protocol, Service, and InternetHost.

-----
/* old */
Protocol proto = new Protocol;  // construct before check
enforce(proto.getProtocolByType(ProtocolType.TCP));  // check
// do stuff

/* new */
Protocol proto = enforce(Protocol.getByType(ProtocolType.TCP));  //
construct after check
// do stuff
-----

I leaves these classes for aliases(getaddrinfo can't get aliases).

- Add AddressInfo class(wraps C's addrinfo struct)

AddressInfo resolves some address familes. I added AF_UNIX support. This class is useful for Socket and Endpoint initialization.

In the future, above classes should be moved to std.net(or std.net.dns).

- Endpoint

I think Address should not have port. Address is a address, not a endpoint. So I added Endpoint structs(derived their name from Asio) and Endpoints use corresponding Addresses.

-- IPEndpoint

This struct supports IPv4 and IPv6 using IPAddress.

-- LocalEndpoint

This struct supports local path using LocalAddress.

- Socket

Socket is a template bacause Socket should support any address familes. Old Socket uses class-inheritance, but newFamilyObject is bad(newFamilyObject can't use user-defined class).

Some methods of old Socket used return-value checking :(

- SocketSet

add and remove methods are typesafe-variadic function.

- Add const and attribute

Sources:

- socket.d http://bitbucket.org/repeatedly/scrap/src/tip/socket.d

unittest is a simple example.

- socketstream.d http://bitbucket.org/repeatedly/scrap/src/tip/socketstream.d

std.stream will be replaced or eliminated. This module need?

- LocalEndpoint sample http://bitbucket.org/repeatedly/scrap/src/tip/sample/echo_server.d http://bitbucket.org/repeatedly/scrap/src/tip/sample/echo_client.d

Contribution from satoru_h


I would like to commit this change. What do you think?


Masahiro
July 06, 2010
Sounds great.

Sent from my iPhone

On Jul 5, 2010, at 12:08 PM, "Masahiro Nakagawa" <repeatedly at gmail.com> wrote:

> std.socket lacks some features and I don't like current design. So, I rewrote this module.
> 
> Summary:
> 
> - Change API of Protocol, Service, and InternetHost.
> 
> -----
> /* old */
> Protocol proto = new Protocol;  // construct before check
> enforce(proto.getProtocolByType(ProtocolType.TCP));  // check
> // do stuff
> 
> /* new */
> Protocol proto = enforce(Protocol.getByType(ProtocolType.TCP));  // construct after check
> // do stuff
> -----
> 
> I leaves these classes for aliases(getaddrinfo can't get aliases).
> 
> - Add AddressInfo class(wraps C's addrinfo struct)
> 
> AddressInfo resolves some address familes. I added AF_UNIX support. This class is useful for Socket and Endpoint initialization.
> 
> In the future, above classes should be moved to std.net(or std.net.dns).
> 
> - Endpoint
> 
> I think Address should not have port. Address is a address, not a endpoint.
> So I added Endpoint structs(derived their name from Asio) and Endpoints use corresponding Addresses.
> 
> -- IPEndpoint
> 
> This struct supports IPv4 and IPv6 using IPAddress.
> 
> -- LocalEndpoint
> 
> This struct supports local path using LocalAddress.
> 
> - Socket
> 
> Socket is a template bacause Socket should support any address familes.
> Old Socket uses class-inheritance, but newFamilyObject is bad(newFamilyObject can't use user-defined class).
> 
> Some methods of old Socket used return-value checking :(
> 
> - SocketSet
> 
> add and remove methods are typesafe-variadic function.
> 
> - Add const and attribute
> 
> Sources:
> 
> - socket.d http://bitbucket.org/repeatedly/scrap/src/tip/socket.d
> 
> unittest is a simple example.
> 
> - socketstream.d http://bitbucket.org/repeatedly/scrap/src/tip/socketstream.d
> 
> std.stream will be replaced or eliminated. This module need?
> 
> - LocalEndpoint sample http://bitbucket.org/repeatedly/scrap/src/tip/sample/echo_server.d http://bitbucket.org/repeatedly/scrap/src/tip/sample/echo_client.d
> 
> Contribution from satoru_h
> 
> 
> I would like to commit this change. What do you think?
> 
> 
> Masahiro
> _______________________________________________
> phobos mailing list
> phobos at puremagic.com
> http://lists.puremagic.com/mailman/listinfo/phobos
July 09, 2010
Thanks.

I have something on my mind in committing this change. std.socket doesn't seem to be Boost license. Why? Can be changed?


Masahiro


On Wed, 07 Jul 2010 12:24:04 +0900, Sean Kelly <sean at invisibleduck.org> wrote:

> Sounds great.
>
> Sent from my iPhone
>
> On Jul 5, 2010, at 12:08 PM, "Masahiro Nakagawa" <repeatedly at gmail.com> wrote:
>
>> std.socket lacks some features and I don't like current design. So, I rewrote this module.
>>
>> Summary:
>>
>> - Change API of Protocol, Service, and InternetHost.
>>
>> -----
>> /* old */
>> Protocol proto = new Protocol;  // construct before check
>> enforce(proto.getProtocolByType(ProtocolType.TCP));  // check
>> // do stuff
>>
>> /* new */
>> Protocol proto = enforce(Protocol.getByType(ProtocolType.TCP));  //
>> construct after check
>> // do stuff
>> -----
>>
>> I leaves these classes for aliases(getaddrinfo can't get aliases).
>>
>> - Add AddressInfo class(wraps C's addrinfo struct)
>>
>> AddressInfo resolves some address familes. I added AF_UNIX support. This class is useful for Socket and Endpoint initialization.
>>
>> In the future, above classes should be moved to std.net(or std.net.dns).
>>
>> - Endpoint
>>
>> I think Address should not have port. Address is a address, not a
>> endpoint.
>> So I added Endpoint structs(derived their name from Asio) and Endpoints
>> use corresponding Addresses.
>>
>> -- IPEndpoint
>>
>> This struct supports IPv4 and IPv6 using IPAddress.
>>
>> -- LocalEndpoint
>>
>> This struct supports local path using LocalAddress.
>>
>> - Socket
>>
>> Socket is a template bacause Socket should support any address familes. Old Socket uses class-inheritance, but newFamilyObject is bad(newFamilyObject can't use user-defined class).
>>
>> Some methods of old Socket used return-value checking :(
>>
>> - SocketSet
>>
>> add and remove methods are typesafe-variadic function.
>>
>> - Add const and attribute
>>
>> Sources:
>>
>> - socket.d http://bitbucket.org/repeatedly/scrap/src/tip/socket.d
>>
>> unittest is a simple example.
>>
>> - socketstream.d http://bitbucket.org/repeatedly/scrap/src/tip/socketstream.d
>>
>> std.stream will be replaced or eliminated. This module need?
>>
>> - LocalEndpoint sample http://bitbucket.org/repeatedly/scrap/src/tip/sample/echo_server.d http://bitbucket.org/repeatedly/scrap/src/tip/sample/echo_client.d
>>
>> Contribution from satoru_h
>>
>>
>> I would like to commit this change. What do you think?
>>
>>
>> Masahiro
>> _______________________________________________
>> phobos mailing list
>> phobos at puremagic.com
>> http://lists.puremagic.com/mailman/listinfo/phobos
> _______________________________________________
> phobos mailing list
> phobos at puremagic.com
> http://lists.puremagic.com/mailman/listinfo/phobos
July 08, 2010
Good question, and it's even better that you plan to throw away the code :o).

Walter?

Masahiro, I'll get back to you with a review soon. Thanks for the great work!


Andrei

On 07/08/2010 12:39 PM, Masahiro Nakagawa wrote:
> Thanks.
>
> I have something on my mind in committing this change. std.socket doesn't seem to be Boost license. Why? Can be changed?
>
>
> Masahiro
>
>
> On Wed, 07 Jul 2010 12:24:04 +0900, Sean Kelly <sean at invisibleduck.org> wrote:
>
>> Sounds great.
>>
>> Sent from my iPhone
>>
>> On Jul 5, 2010, at 12:08 PM, "Masahiro Nakagawa" <repeatedly at gmail.com> wrote:
>>
>>> std.socket lacks some features and I don't like current design. So, I rewrote this module.
>>>
>>> Summary:
>>>
>>> - Change API of Protocol, Service, and InternetHost.
>>>
>>> -----
>>> /* old */
>>> Protocol proto = new Protocol; // construct before check
>>> enforce(proto.getProtocolByType(ProtocolType.TCP)); // check
>>> // do stuff
>>>
>>> /* new */
>>> Protocol proto = enforce(Protocol.getByType(ProtocolType.TCP)); //
>>> construct after check
>>> // do stuff
>>> -----
>>>
>>> I leaves these classes for aliases(getaddrinfo can't get aliases).
>>>
>>> - Add AddressInfo class(wraps C's addrinfo struct)
>>>
>>> AddressInfo resolves some address familes. I added AF_UNIX support. This class is useful for Socket and Endpoint initialization.
>>>
>>> In the future, above classes should be moved to std.net(or std.net.dns).
>>>
>>> - Endpoint
>>>
>>> I think Address should not have port. Address is a address, not a
>>> endpoint.
>>> So I added Endpoint structs(derived their name from Asio) and
>>> Endpoints use corresponding Addresses.
>>>
>>> -- IPEndpoint
>>>
>>> This struct supports IPv4 and IPv6 using IPAddress.
>>>
>>> -- LocalEndpoint
>>>
>>> This struct supports local path using LocalAddress.
>>>
>>> - Socket
>>>
>>> Socket is a template bacause Socket should support any address familes. Old Socket uses class-inheritance, but newFamilyObject is bad(newFamilyObject can't use user-defined class).
>>>
>>> Some methods of old Socket used return-value checking :(
>>>
>>> - SocketSet
>>>
>>> add and remove methods are typesafe-variadic function.
>>>
>>> - Add const and attribute
>>>
>>> Sources:
>>>
>>> - socket.d http://bitbucket.org/repeatedly/scrap/src/tip/socket.d
>>>
>>> unittest is a simple example.
>>>
>>> - socketstream.d http://bitbucket.org/repeatedly/scrap/src/tip/socketstream.d
>>>
>>> std.stream will be replaced or eliminated. This module need?
>>>
>>> - LocalEndpoint sample http://bitbucket.org/repeatedly/scrap/src/tip/sample/echo_server.d http://bitbucket.org/repeatedly/scrap/src/tip/sample/echo_client.d
>>>
>>> Contribution from satoru_h
>>>
>>>
>>> I would like to commit this change. What do you think?
>>>
>>>
>>> Masahiro
>>> _______________________________________________
>>> phobos mailing list
>>> phobos at puremagic.com
>>> http://lists.puremagic.com/mailman/listinfo/phobos
>> _______________________________________________
>> phobos mailing list
>> phobos at puremagic.com
>> http://lists.puremagic.com/mailman/listinfo/phobos
> _______________________________________________
> phobos mailing list
> phobos at puremagic.com
> http://lists.puremagic.com/mailman/listinfo/phobos
July 08, 2010
When I converted the Phobos license, I only changed the modules where the author could be contacted for permission.  The exceptions were:

base64 - Chris Miller
loader - Matt Wilson
md5 - RSA Data Security
openrj - Matt Wilson
perf - Matt Wilson
socket - Chris Miller
socketstream - Chris Miller
stream - Pavel Minayev

Since most/all of these modules are destined for a rewrite or removal, I hope it doesn't prove to be much of an issue.  By the way, I do think I could reach Matt Wilson if need be.  Neither of the other two though.

On Jul 8, 2010, at 10:39 AM, Masahiro Nakagawa wrote:

> Thanks.
> 
> I have something on my mind in committing this change. std.socket doesn't seem to be Boost license. Why? Can be changed?
> 
> 
> Masahiro
> 
> 
> On Wed, 07 Jul 2010 12:24:04 +0900, Sean Kelly <sean at invisibleduck.org> wrote:
> 
>> Sounds great.
>> 
>> Sent from my iPhone
>> 
>> On Jul 5, 2010, at 12:08 PM, "Masahiro Nakagawa" <repeatedly at gmail.com> wrote:
>> 
>>> std.socket lacks some features and I don't like current design. So, I rewrote this module.
>>> 
>>> Summary:
>>> 
>>> - Change API of Protocol, Service, and InternetHost.
>>> 
>>> -----
>>> /* old */
>>> Protocol proto = new Protocol;  // construct before check
>>> enforce(proto.getProtocolByType(ProtocolType.TCP));  // check
>>> // do stuff
>>> 
>>> /* new */
>>> Protocol proto = enforce(Protocol.getByType(ProtocolType.TCP));  // construct after check
>>> // do stuff
>>> -----
>>> 
>>> I leaves these classes for aliases(getaddrinfo can't get aliases).
>>> 
>>> - Add AddressInfo class(wraps C's addrinfo struct)
>>> 
>>> AddressInfo resolves some address familes. I added AF_UNIX support. This class is useful for Socket and Endpoint initialization.
>>> 
>>> In the future, above classes should be moved to std.net(or std.net.dns).
>>> 
>>> - Endpoint
>>> 
>>> I think Address should not have port. Address is a address, not a endpoint.
>>> So I added Endpoint structs(derived their name from Asio) and Endpoints use corresponding Addresses.
>>> 
>>> -- IPEndpoint
>>> 
>>> This struct supports IPv4 and IPv6 using IPAddress.
>>> 
>>> -- LocalEndpoint
>>> 
>>> This struct supports local path using LocalAddress.
>>> 
>>> - Socket
>>> 
>>> Socket is a template bacause Socket should support any address familes.
>>> Old Socket uses class-inheritance, but newFamilyObject is bad(newFamilyObject can't use user-defined class).
>>> 
>>> Some methods of old Socket used return-value checking :(
>>> 
>>> - SocketSet
>>> 
>>> add and remove methods are typesafe-variadic function.
>>> 
>>> - Add const and attribute
>>> 
>>> Sources:
>>> 
>>> - socket.d http://bitbucket.org/repeatedly/scrap/src/tip/socket.d
>>> 
>>> unittest is a simple example.
>>> 
>>> - socketstream.d http://bitbucket.org/repeatedly/scrap/src/tip/socketstream.d
>>> 
>>> std.stream will be replaced or eliminated. This module need?
>>> 
>>> - LocalEndpoint sample http://bitbucket.org/repeatedly/scrap/src/tip/sample/echo_server.d http://bitbucket.org/repeatedly/scrap/src/tip/sample/echo_client.d
>>> 
>>> Contribution from satoru_h
>>> 
>>> 
>>> I would like to commit this change. What do you think?
>>> 
>>> 
>>> Masahiro
>>> _______________________________________________
>>> phobos mailing list
>>> phobos at puremagic.com
>>> http://lists.puremagic.com/mailman/listinfo/phobos
>> _______________________________________________
>> phobos mailing list
>> phobos at puremagic.com
>> http://lists.puremagic.com/mailman/listinfo/phobos
> _______________________________________________
> phobos mailing list
> phobos at puremagic.com
> http://lists.puremagic.com/mailman/listinfo/phobos

July 10, 2010
On Fri, 09 Jul 2010 04:12:12 +0900, Sean Kelly <sean at invisibleduck.org> wrote:

> When I converted the Phobos license, I only changed the modules where the author could be contacted for permission.  The exceptions were:
>
> base64 - Chris Miller
> loader - Matt Wilson
> md5 - RSA Data Security
> openrj - Matt Wilson
> perf - Matt Wilson
> socket - Chris Miller
> socketstream - Chris Miller
> stream - Pavel Minayev
>
> Since most/all of these modules are destined for a rewrite or removal, I hope it doesn't prove to be much of an issue.  By the way, I do think I could reach Matt Wilson if need be.  Neither of the other two though.
>

Thank you for letting me know.

Hmm... I don't use these modules.
But, I think Phobos needs std.crypt like dcrypt. std.md5 is poor.


Masahiro
September 05, 2010
I already wrote the following code review for http://bitbucket.org/repeatedly/scrap/src/tip/socket.d before seeing Masahiro's new message that we should scrap that review. I'm sending this in case it applies to his new work based on Asio.

* Line 67: don't use typedef - it will go away

* 87: feel free to insert a static assert(sockaddr_storage.sizeof == ...) to make sure the compiler did as you expected.

* 270: enforce(val, new SocketException(...))

* 304-309: lowercase? We have visibility protection because of AddressFamily.

* 362: Why is Protocol a class? It has all public members and no overridable methods. (Also, constructors and public members often are questionable.)

* 433: Same question about Service.

* 539: And same question about InternetHost.

* 546: why not a ref hostent?

* 553: no need for the .idup

* 663: I'm weary about every module adding its own exception type, but this is subject to a separate investigation we should do for all of Phobos.

* 694-698: Public members of a class mean that I can change any of them to random values without breaking the consistency of the object. Is that true for AddressInfo?

* 752: ref addrinfo

* 777 and others: which of these methods do you think should be overridable? Probably a small subset if any. Then they should be final.

* 947: Is there something that naturally differentiates the two types of addresses? If so, you could get rid of Type.

* 1294: I'm not an IPv6 pro, but looks like a little hierarchy would work here (with e.g. IPv4 and IPv6 inheriting a common base). Maybe LocalAddress could be snuck in too.

* 2310: what does byes mean?

* 2520, 2541 etc.: Return size_t.

* 2934: select() has well-known issues. Any plans to support the newer system-dependent APIs or libevent?


Andrei

On 07/05/2010 02:08 PM, Masahiro Nakagawa wrote:
> std.socket lacks some features and I don't like current design. So, I rewrote this module.
>
> Summary:
>
> - Change API of Protocol, Service, and InternetHost.
>
> -----
> /* old */
> Protocol proto = new Protocol; // construct before check
> enforce(proto.getProtocolByType(ProtocolType.TCP)); // check
> // do stuff
>
> /* new */
> Protocol proto = enforce(Protocol.getByType(ProtocolType.TCP)); //
> construct after check
> // do stuff
> -----
>
> I leaves these classes for aliases(getaddrinfo can't get aliases).
>
> - Add AddressInfo class(wraps C's addrinfo struct)
>
> AddressInfo resolves some address familes. I added AF_UNIX support. This class is useful for Socket and Endpoint initialization.
>
> In the future, above classes should be moved to std.net(or std.net.dns).
>
> - Endpoint
>
> I think Address should not have port. Address is a address, not a endpoint. So I added Endpoint structs(derived their name from Asio) and Endpoints use corresponding Addresses.
>
> -- IPEndpoint
>
> This struct supports IPv4 and IPv6 using IPAddress.
>
> -- LocalEndpoint
>
> This struct supports local path using LocalAddress.
>
> - Socket
>
> Socket is a template bacause Socket should support any address familes. Old Socket uses class-inheritance, but newFamilyObject is bad(newFamilyObject can't use user-defined class).
>
> Some methods of old Socket used return-value checking :(
>
> - SocketSet
>
> add and remove methods are typesafe-variadic function.
>
> - Add const and attribute
>
> Sources:
>
> - socket.d http://bitbucket.org/repeatedly/scrap/src/tip/socket.d
>
> unittest is a simple example.
>
> - socketstream.d http://bitbucket.org/repeatedly/scrap/src/tip/socketstream.d
>
> std.stream will be replaced or eliminated. This module need?
>
> - LocalEndpoint sample http://bitbucket.org/repeatedly/scrap/src/tip/sample/echo_server.d http://bitbucket.org/repeatedly/scrap/src/tip/sample/echo_client.d
>
> Contribution from satoru_h
>
>
> I would like to commit this change. What do you think?
>
>
> Masahiro
> _______________________________________________
> phobos mailing list
> phobos at puremagic.com
> http://lists.puremagic.com/mailman/listinfo/phobos
September 08, 2010
Thanks for the review.


On Sun, 05 Sep 2010 15:29:36 +0900, Andrei Alexandrescu <andrei at erdani.com> wrote:

> I already wrote the following code review for http://bitbucket.org/repeatedly/scrap/src/tip/socket.d before seeing Masahiro's new message that we should scrap that review. I'm sending this in case it applies to his new work based on Asio.
>
> * Line 67: don't use typedef - it will go away

OK.

> * 87: feel free to insert a static assert(sockaddr_storage.sizeof == ...) to make sure the compiler did as you expected.

I think static assert doesn't need because the detail of sockaddr_storage is not important.

I will add sockaddr_storage to std.c.windows.

> * 270: enforce(val, new SocketException(...))

I forgot to rewrite.

> * 304-309: lowercase? We have visibility protection because of AddressFamily.

OK.

> * 362: Why is Protocol a class? It has all public members and no overridable methods. (Also, constructors and public members often are questionable.)
>
> * 433: Same question about Service.
>
> * 539: And same question about InternetHost.
>
> * 546: why not a ref hostent?
>
> * 553: no need for the .idup

I don't know the detail. I will remove those classes in new module.

> * 663: I'm weary about every module adding its own exception type, but this is subject to a separate investigation we should do for all of Phobos.

I agree. I think Phobos guide should describe the exception design.

> * 694-698: Public members of a class mean that I can change any of them to random values without breaking the consistency of the object. Is that true for AddressInfo?

I think yes for AddressInfo bacause AddressInfo is a container for
getaddrinfo.
I will change AddressInfo to struct.

> * 752: ref addrinfo

OK.

> * 777 and others: which of these methods do you think should be overridable? Probably a small subset if any. Then they should be final.

I mentioned above.

> * 947: Is there something that naturally differentiates the two types of addresses? If so, you could get rid of Type.

It's a matter of taste.
In general, User uses AddressInfo for Socket.
So, IPv4Address and IPv6Address are almost invisible.
I think IPAddress only is not handy if User uses IPv4 or IPv6
representation directly.

I will add some functions to IPv4 and IPv6 in new module.

> * 1294: I'm not an IPv6 pro, but looks like a little hierarchy would work here (with e.g. IPv4 and IPv6 inheriting a common base). Maybe LocalAddress could be snuck in too.

What is a common base?
IP Address contains IPv4 and IPv6, but IPv4 and IPv6 doesn't inherit IP
Address.
In addition, IPv6 doesn't have the compatibility to IPv4.

> * 2310: what does byes mean?

I don't know, but I think byes stands for "blocking yes".

> * 2520, 2541 etc.: Return size_t.

Oops.

> * 2934: select() has well-known issues. Any plans to support the newer system-dependent APIs or libevent?

I will remove select() from Socket in new module.

I am thinking about std.event.
This module supports system-dependent APIs(kqueue, epoll, etc...) and
abstraction layer.


Masahiro
September 08, 2010
 On 08.09.2010 16:02, Masahiro Nakagawa wrote:
>
> I am thinking about std.event.
> This module supports system-dependent APIs(kqueue, epoll, etc...) and
> abstraction layer.
>
I think the best solution would be a wrapper / abstraction layer for libev. All of the mentioned APIs (kqueue, epoll, select...) are broken in some way, so writing a new event loop system in D will be painful and it will take a long time until it gets stable. Also libev already has support for more event sources (timer, periodic, signal, stat, ...).

Quoting libev documentation:

EVBACKEND_SELECT
Not /completely/ standard, as libev tries to roll its own fd_set with no
limits on the number of fds, but if that fails, expect a fairly low
limit on the number of fds when using this backend.
This backend maps |EV_READ| to the |readfds| set and |EV_WRITE| to
the |writefds| set (and to work around Microsoft Windows bugs, also onto
the |exceptfds| set on that platform).

EVBACKEND_POLL
It's more complicated than select, but handles sparse fds better and has
no artificial limit on the number of fds you can use (except it will
slow down considerably with a lot of inactive fds).

EVBACKEND_EPOLL

The epoll mechanism deserves honorable mention as the most misdesigned of the more advanced event mechanisms: mere annoyances include silently dropping file descriptors, requiring a system call per change per file descriptor (and unnecessary guessing of parameters), problems with dup and so on. The biggest issue is fork races, however - if a program forks then /both/ parent and child process have to recreate the epoll set, which can take considerable time (one syscall per file descriptor) and is of course hard to detect.

Epoll is also notoriously buggy - embedding epoll fds /should/ work, but
of course /doesn't/, and epoll just loves to report events for
totally /different/ file descriptors (even already closed ones, so one
cannot even remove them from the set) than registered in the set
(especially on SMP systems). Libev tries to counter these spurious
notifications by employing an additional generation counter and
comparing that against the events to filter out spurious ones,
recreating the set when required.

EVBACKEND_KQUEUE
Kqueue deserves special mention, as at the time of this writing, it was
broken on all BSDs except NetBSD (usually it doesn't work reliably with
anything but sockets and pipes, except on Darwin, where of course it's
completely useless). Unlike epoll, however, whose brokenness is by
design, these kqueue bugs can (and eventually will) be fixed without API
changes to existing programs. For this reason it's not being
"auto-detected" unless you explicitly specify it in the flags (i.e.
using |EVBACKEND_KQUEUE|) or libev was compiled on a known-to-be-good
(-enough) system like NetBSD.

While stopping, setting and starting an I/O watcher does never cause an extra system call as with |EVBACKEND_EPOLL|, it still adds up to two event changes per incident. Support for |fork ()| is very bad (but sane, unlike epoll) and it drops fds silently in similarly hard-to-detect cases

EVBACKEND_PORT

This uses the Solaris 10 event port mechanism. As with everything on
Solaris, it's really slow, but it still scales very well (O(active_fds)).

Please note that Solaris event ports can deliver a lot of spurious notifications, so you need to use non-blocking I/O or other means to avoid blocking when no data (or space) is available.

While this backend scales well, it requires one system call per active file descriptor per loop iteration. For small and medium numbers of file descriptors a "slow" |EVBACKEND_SELECT| or |EVBACKEND_POLL| backend might perform better


-- 
Johannes Pfau

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.puremagic.com/pipermail/phobos/attachments/20100908/b2a36397/attachment-0001.html>
September 08, 2010
On 09/08/2010 09:02 AM, Masahiro Nakagawa wrote:
[snip]

All sounds good! One funny detail:

>> * 2310: what does byes mean?
>
> I don't know, but I think byes stands for "blocking yes".

This is cool :o). I had two hypotheses: typo for "bytes" or some sort of closing ack (plural for "bye").

>> * 2934: select() has well-known issues. Any plans to support the newer system-dependent APIs or libevent?
>
> I will remove select() from Socket in new module.
>
> I am thinking about std.event.
> This module supports system-dependent APIs(kqueue, epoll, etc...) and
> abstraction layer.

That would be awesome. Sean was also thinking of it and could provide code review. At best we'd integrate std.event with thread messages.


Thanks,

Andrei
« First   ‹ Prev
1 2 3