Jump to page: 1 2
Thread overview
[phobos] Fwd: [Issue 4025] New: Making network with the std.stdio.File interface
Mar 29, 2010
Walter Bright
Mar 29, 2010
Sean Kelly
Mar 29, 2010
Walter Bright
Mar 29, 2010
Don Clugston
Mar 29, 2010
Walter Bright
Apr 08, 2010
Adam D. Ruppe
Apr 08, 2010
Adam D. Ruppe
March 28, 2010
I'm quite hopeful about this development. If we have anything for the network in D2, that would be great, and integration with File will reuse a ton of work.

I'm considering extending Adam an offer to join Phobos. He has donated a Windows machine for building and running Phobos and generally seems to know what he's talking about. If he'll join the project, my hope is to attract him to dedicate more time to it.

Please cast your vote by replying (silence = abstaining, i.e. you're not for or against).


Andrei

-------- Original Message --------
Subject: [Issue 4025] New: Making network with the std.stdio.File interface
Date: Sun, 28 Mar 2010 22:51:45 +0000 (UTC)
From: d-bugmail at puremagic.com
Organization: Digital Mars
To: digitalmars-d-bugs at puremagic.com
Newsgroups: digitalmars.D.bugs

http://d.puremagic.com/issues/show_bug.cgi?id=4025

            Summary: Making network with the std.stdio.File interface
            Product: D
            Version: 2.041
           Platform: Other
         OS/Version: Linux
             Status: NEW
           Severity: enhancement
           Priority: P2
          Component: Phobos
         AssignedTo: nobody at puremagic.com
         ReportedBy: destructionator at gmail.com


--- Comment #0 from Adam D. Ruppe <destructionator at gmail.com> 2010-03-28 
15:51:43 PDT ---
I've written a small module that opens a network connection, then wraps
it in
the File struct, allowing you to use the byLine, etc., ranges on it, as
well as
writef/readln/etc.

It is Linux only, but should be pretty easy to port to other operating
systems.
Ideally, I'd like to eventually be able to use File for talking
to processes too, on all D platforms.

Here's the code I have for now:

==============



public import std.stdio;
import std.string;

import std.conv;

version(linux):

import std.c.linux.linux;
import std.c.linux.socket;

alias std.c.linux.socket sock;
alias std.c.linux.linux linux;

enum int PF_INET = 2;
enum int AF_INET = PF_INET;

extern(C) FILE* fdopen(int, const(char)*);

File openNetwork(string host, ushort port) {
     hostent* h;
     sockaddr_in addr;

     h = gethostbyname(std.string.toStringz(host));
     if(h is null)
         throw new StdioException("gethostbyname");

     int s = socket(PF_INET, SOCK_STREAM, 0);
     if(s == -1)
         throw new StdioException("socket");

     scope(failure)
         close(s);

     addr.sin_family = AF_INET;
     addr.sin_port = htons(port);
     std.c.string.memcpy(&addr.sin_addr.s_addr, h.h_addr, h.h_length);

     if(sock.connect(s, cast(sockaddr*) &addr, addr.sizeof) == -1)
         throw new StdioException("Connect failed");

     FILE* fp = fdopen(s, "w+".ptr);

     File f;

     auto imp = new File.Impl(fp, 1, host ~ ":" ~ to!string(port));

     f.p = imp;

     return f;
}
March 28, 2010
Yes.

Andrei Alexandrescu wrote:
> I'm quite hopeful about this development. If we have anything for the network in D2, that would be great, and integration with File will reuse a ton of work.
>
> I'm considering extending Adam an offer to join Phobos. He has donated a Windows machine for building and running Phobos and generally seems to know what he's talking about. If he'll join the project, my hope is to attract him to dedicate more time to it.
>
> Please cast your vote by replying (silence = abstaining, i.e. you're not for or against).
>
>
> Andrei
>
> -------- Original Message --------
> Subject: [Issue 4025] New: Making network with the std.stdio.File
> interface
> Date: Sun, 28 Mar 2010 22:51:45 +0000 (UTC)
> From: d-bugmail at puremagic.com
> Organization: Digital Mars
> To: digitalmars-d-bugs at puremagic.com
> Newsgroups: digitalmars.D.bugs
>
> http://d.puremagic.com/issues/show_bug.cgi?id=4025
>
>            Summary: Making network with the std.stdio.File interface
>            Product: D
>            Version: 2.041
>           Platform: Other
>         OS/Version: Linux
>             Status: NEW
>           Severity: enhancement
>           Priority: P2
>          Component: Phobos
>         AssignedTo: nobody at puremagic.com
>         ReportedBy: destructionator at gmail.com
>
>
> --- Comment #0 from Adam D. Ruppe <destructionator at gmail.com> 
> 2010-03-28 15:51:43 PDT ---
> I've written a small module that opens a network connection, then
> wraps it in
> the File struct, allowing you to use the byLine, etc., ranges on it,
> as well as
> writef/readln/etc.
>
> It is Linux only, but should be pretty easy to port to other operating
> systems.
> Ideally, I'd like to eventually be able to use File for talking
> to processes too, on all D platforms.
>
> Here's the code I have for now:
>
> ==============
>
>
>
> public import std.stdio;
> import std.string;
>
> import std.conv;
>
> version(linux):
>
> import std.c.linux.linux;
> import std.c.linux.socket;
>
> alias std.c.linux.socket sock;
> alias std.c.linux.linux linux;
>
> enum int PF_INET = 2;
> enum int AF_INET = PF_INET;
>
> extern(C) FILE* fdopen(int, const(char)*);
>
> File openNetwork(string host, ushort port) {
>     hostent* h;
>     sockaddr_in addr;
>
>     h = gethostbyname(std.string.toStringz(host));
>     if(h is null)
>         throw new StdioException("gethostbyname");
>
>     int s = socket(PF_INET, SOCK_STREAM, 0);
>     if(s == -1)
>         throw new StdioException("socket");
>
>     scope(failure)
>         close(s);
>
>     addr.sin_family = AF_INET;
>     addr.sin_port = htons(port);
>     std.c.string.memcpy(&addr.sin_addr.s_addr, h.h_addr, h.h_length);
>
>     if(sock.connect(s, cast(sockaddr*) &addr, addr.sizeof) == -1)
>         throw new StdioException("Connect failed");
>
>     FILE* fp = fdopen(s, "w+".ptr);
>
>     File f;
>
>     auto imp = new File.Impl(fp, 1, host ~ ":" ~ to!string(port));
>
>     f.p = imp;
>
>     return f;
> }
> _______________________________________________
> phobos mailing list
> phobos at puremagic.com
> http://lists.puremagic.com/mailman/listinfo/phobos
>
>
March 28, 2010
Aye.

On Mar 28, 2010, at 4:21 PM, Andrei Alexandrescu wrote:

> I'm quite hopeful about this development. If we have anything for the network in D2, that would be great, and integration with File will reuse a ton of work.
> 
> I'm considering extending Adam an offer to join Phobos. He has donated a Windows machine for building and running Phobos and generally seems to know what he's talking about. If he'll join the project, my hope is to attract him to dedicate more time to it.
> 
> Please cast your vote by replying (silence = abstaining, i.e. you're not for or against).
> 
> 
> Andrei
> 
> -------- Original Message --------
> Subject: [Issue 4025] New: Making network with the std.stdio.File interface
> Date: Sun, 28 Mar 2010 22:51:45 +0000 (UTC)
> From: d-bugmail at puremagic.com
> Organization: Digital Mars
> To: digitalmars-d-bugs at puremagic.com
> Newsgroups: digitalmars.D.bugs
> 
> http://d.puremagic.com/issues/show_bug.cgi?id=4025
> 
>           Summary: Making network with the std.stdio.File interface
>           Product: D
>           Version: 2.041
>          Platform: Other
>        OS/Version: Linux
>            Status: NEW
>          Severity: enhancement
>          Priority: P2
>         Component: Phobos
>        AssignedTo: nobody at puremagic.com
>        ReportedBy: destructionator at gmail.com
> 
> 
> --- Comment #0 from Adam D. Ruppe <destructionator at gmail.com> 2010-03-28 15:51:43 PDT ---
> I've written a small module that opens a network connection, then wraps it in the File struct, allowing you to use the byLine, etc., ranges on it, as well as writef/readln/etc.
> 
> It is Linux only, but should be pretty easy to port to other operating systems.
> Ideally, I'd like to eventually be able to use File for talking
> to processes too, on all D platforms.
> 
> Here's the code I have for now:
> 
> ==============
> 
> 
> 
> public import std.stdio;
> import std.string;
> 
> import std.conv;
> 
> version(linux):
> 
> import std.c.linux.linux;
> import std.c.linux.socket;
> 
> alias std.c.linux.socket sock;
> alias std.c.linux.linux linux;
> 
> enum int PF_INET = 2;
> enum int AF_INET = PF_INET;
> 
> extern(C) FILE* fdopen(int, const(char)*);
> 
> File openNetwork(string host, ushort port) {
>    hostent* h;
>    sockaddr_in addr;
> 
>    h = gethostbyname(std.string.toStringz(host));
>    if(h is null)
>        throw new StdioException("gethostbyname");
> 
>    int s = socket(PF_INET, SOCK_STREAM, 0);
>    if(s == -1)
>        throw new StdioException("socket");
> 
>    scope(failure)
>        close(s);
> 
>    addr.sin_family = AF_INET;
>    addr.sin_port = htons(port);
>    std.c.string.memcpy(&addr.sin_addr.s_addr, h.h_addr, h.h_length);
> 
>    if(sock.connect(s, cast(sockaddr*) &addr, addr.sizeof) == -1)
>        throw new StdioException("Connect failed");
> 
>    FILE* fp = fdopen(s, "w+".ptr);
> 
>    File f;
> 
>    auto imp = new File.Impl(fp, 1, host ~ ":" ~ to!string(port));
> 
>    f.p = imp;
> 
>    return f;
> }
> _______________________________________________
> phobos mailing list
> phobos at puremagic.com
> http://lists.puremagic.com/mailman/listinfo/phobos

March 28, 2010

Sean Kelly wrote:
> Aye.
>
> 

Arrrr, me maties!
March 29, 2010
Aye, aye, cap'n.

On 29 March 2010 05:14, Walter Bright <walter at digitalmars.com> wrote:
>
>
> Sean Kelly wrote:
>>
>> Aye.
>>
>>
>
> Arrrr, me maties!
March 29, 2010
I have no problem with Adam joining.  As far as the network library, do we want to increase our dependency on the C FILE* structure/api?  I would not want that.  I don't know a lot of network solutions that put a socket fd inside a FILE *, there's probably good reason for that.

As long as there's a non-buffered solution that allows socket-like functions accessible, I'm cool with File wrapping a socket.  That is, there should be a way to create a socket with the API that doesn't use File.

I hope we will eventually get a D-only solution for File.  Good news is, any future improvements to File automatically translate to what Adam is working on.

-Steve



----- Original Message ----
> From: Andrei Alexandrescu <andrei at erdani.com>
> To: Discuss the phobos library for D <phobos at puremagic.com>
> Sent: Sun, March 28, 2010 7:21:20 PM
> Subject: [phobos] Fwd: [Issue 4025] New: Making network with the std.stdio.File interface
> 
> I'm quite hopeful about this development. If we have anything for the network in D2, that would be great, and integration with File will reuse a ton of work.

I'm considering extending Adam an offer to join Phobos. He has
> donated a Windows machine for building and running Phobos and generally seems to know what he's talking about. If he'll join the project, my hope is to attract him to dedicate more time to it.

Please cast your vote by replying
> (silence = abstaining, i.e. you're not for or against).


Andrei

-------- Original Message 
> --------
Subject: [Issue 4025] New: Making network with the std.stdio.File
> interface
Date: Sun, 28 Mar 2010 22:51:45 +0000 (UTC)
From:
> ymailto="mailto:d-bugmail at puremagic.com" href="mailto:d-bugmail at puremagic.com">d-bugmail at puremagic.com
Organization:
> Digital Mars
To:
> href="mailto:digitalmars-d-bugs at puremagic.com">digitalmars-d-bugs at puremagic.com
Newsgroups:
> digitalmars.D.bugs

http://d.puremagic.com/issues/show_bug.cgi?id=4025


>          Summary: Making network with the std.stdio.File
> interface
           Product: D

>        Version: 2.041

> Platform: Other
        OS/Version: Linux

>           Status: NEW

>   Severity: enhancement
          Priority:
> P2
         Component: Phobos

>   AssignedTo:
> href="mailto:nobody at puremagic.com">nobody at puremagic.com

>     ReportedBy:
> href="mailto:destructionator at gmail.com">destructionator at gmail.com


--- 
> Comment #0 from Adam D. Ruppe <
> href="mailto:destructionator at gmail.com">destructionator at gmail.com>
> 2010-03-28 15:51:43 PDT ---
I've written a small module that opens a network
> connection, then wraps it in
the File struct, allowing you to use the byLine,
> etc., ranges on it, as well as
writef/readln/etc.

It is Linux only,
> but should be pretty easy to port to other operating systems.
Ideally, I'd
> like to eventually be able to use File for talking
to processes too, on all D
> platforms.

Here's the code I have for
> now:

==============



public import std.stdio;
import
> std.string;

import std.conv;

version(linux):

import
> std.c.linux.linux;
import std.c.linux.socket;

alias std.c.linux.socket
> sock;
alias std.c.linux.linux linux;

enum int PF_INET = 2;
enum int
> AF_INET = PF_INET;

extern(C) FILE* fdopen(int, const(char)*);

File
> openNetwork(string host, ushort port) {
    hostent* h;

>   sockaddr_in addr;

    h =
> gethostbyname(std.string.toStringz(host));
    if(h is
> null)
        throw new
> StdioException("gethostbyname");

    int s = socket(PF_INET,
> SOCK_STREAM, 0);
    if(s == -1)

> throw new StdioException("socket");


> scope(failure)
        close(s);


> addr.sin_family = AF_INET;
    addr.sin_port =
> htons(port);
    std.c.string.memcpy(&addr.sin_addr.s_addr,
> h.h_addr, h.h_length);

    if(sock.connect(s, cast(sockaddr*)
> &addr, addr.sizeof) == -1)
        throw new
> StdioException("Connect failed");

    FILE* fp = fdopen(s,
> "w+".ptr);

    File f;

    auto imp = new
> File.Impl(fp, 1, host ~ ":" ~ to!string(port));

    f.p =
> imp;

    return
> f;
}
_______________________________________________
phobos mailing
> list

> href="mailto:phobos at puremagic.com">phobos at puremagic.com
http://lists.puremagic.com/mailman/listinfo/phobos



March 29, 2010
I had a long conversation with Andrei about a similar topic, i.e. dealing with zip files. We concluded that the zip file library should know nothing about files or file streams. It should work with ranges. Similarly, designs like "make this memory buffer look like a file stream" are completely wrong for modern designs. It should be "make a file look like a range" and "make a memory buffer look like a range" and then have the reader/writer operate on ranges.

The same should hold for network access.

Steve Schveighoffer wrote:
> I have no problem with Adam joining.  As far as the network library, do we want to increase our dependency on the C FILE* structure/api?  I would not want that.  I don't know a lot of network solutions that put a socket fd inside a FILE *, there's probably good reason for that.
>
> As long as there's a non-buffered solution that allows socket-like functions accessible, I'm cool with File wrapping a socket.  That is, there should be a way to create a socket with the API that doesn't use File.
>
> I hope we will eventually get a D-only solution for File.  Good news is, any future improvements to File automatically translate to what Adam is working on.
>
> -Steve
>
>
>
> ----- Original Message ----
> 
>> From: Andrei Alexandrescu <andrei at erdani.com>
>> To: Discuss the phobos library for D <phobos at puremagic.com>
>> Sent: Sun, March 28, 2010 7:21:20 PM
>> Subject: [phobos] Fwd: [Issue 4025] New: Making network with the std.stdio.File interface
>>
>> I'm quite hopeful about this development. If we have anything for the network in
>> D2, that would be great, and integration with File will reuse a ton of
>> work.
>> 
>
> I'm considering extending Adam an offer to join Phobos. He has
> 
>> donated a Windows machine for building and running Phobos and generally seems to
>> know what he's talking about. If he'll join the project, my hope is to attract
>> him to dedicate more time to it.
>> 
>
> Please cast your vote by replying
> 
>> (silence = abstaining, i.e. you're not for or
>> against).
>> 
>
>
> Andrei
>
> -------- Original Message 
> 
>> --------
>> 
> Subject: [Issue 4025] New: Making network with the std.stdio.File
> 
>> interface
>> 
> Date: Sun, 28 Mar 2010 22:51:45 +0000 (UTC)
> From:
> 
>> ymailto="mailto:d-bugmail at puremagic.com"
>> href="mailto:d-bugmail at puremagic.com">d-bugmail at puremagic.com
>> 
> Organization:
> 
>> Digital Mars
>> 
> To:
> 
>> href="mailto:digitalmars-d-bugs at puremagic.com">digitalmars-d-bugs at puremagic.com
>> 
> Newsgroups:
> 
>> digitalmars.D.bugs
>> 
>
> http://d.puremagic.com/issues/show_bug.cgi?id=4025
>
> 
> 
>>          Summary: Making network with the std.stdio.File
>> interface
>> 
>            Product: D
> 
> 
>>        Version: 2.041
>> 
> 
> 
>> Platform: Other
>> 
>         OS/Version: Linux
> 
> 
>>           Status: NEW
>> 
> 
> 
>>   Severity: enhancement
>> 
>           Priority:
> 
>> P2
>> 
>          Component: Phobos
> 
> 
>>   AssignedTo:
>> href="mailto:nobody at puremagic.com">nobody at puremagic.com
>> 
> 
> 
>>     ReportedBy:
>> href="mailto:destructionator at gmail.com">destructionator at gmail.com
>> 
>
>
> --- 
> 
>> Comment #0 from Adam D. Ruppe <
>> href="mailto:destructionator at gmail.com">destructionator at gmail.com>
>> 2010-03-28 15:51:43 PDT ---
>> 
> I've written a small module that opens a network
> 
>> connection, then wraps it in
>> 
> the File struct, allowing you to use the byLine,
> 
>> etc., ranges on it, as well as
>> 
> writef/readln/etc.
>
> It is Linux only,
> 
>> but should be pretty easy to port to other operating systems.
>> 
> Ideally, I'd
> 
>> like to eventually be able to use File for talking
>> 
> to processes too, on all D
> 
>> platforms.
>> 
>
> Here's the code I have for
> 
>> now:
>> 
>
> ==============
>
>
>
> public import std.stdio;
> import
> 
>> std.string;
>> 
>
> import std.conv;
>
> version(linux):
>
> import
> 
>> std.c.linux.linux;
>> 
> import std.c.linux.socket;
>
> alias std.c.linux.socket
> 
>> sock;
>> 
> alias std.c.linux.linux linux;
>
> enum int PF_INET = 2;
> enum int
> 
>> AF_INET = PF_INET;
>> 
>
> extern(C) FILE* fdopen(int, const(char)*);
>
> File
> 
>> openNetwork(string host, ushort port) {
>> 
>     hostent* h;
> 
> 
>>   sockaddr_in addr;
>> 
>
>     h =
> 
>> gethostbyname(std.string.toStringz(host));
>> 
>     if(h is
> 
>> null)
>> 
>         throw new
> 
>> StdioException("gethostbyname");
>> 
>
>     int s = socket(PF_INET,
> 
>> SOCK_STREAM, 0);
>> 
>     if(s == -1)
> 
> 
>> throw new StdioException("socket");
>> 
>
> 
> 
>> scope(failure)
>> 
>         close(s);
>
> 
> 
>> addr.sin_family = AF_INET;
>> 
>     addr.sin_port =
> 
>> htons(port);
>> 
>     std.c.string.memcpy(&addr.sin_addr.s_addr,
> 
>> h.h_addr, h.h_length);
>> 
>
>     if(sock.connect(s, cast(sockaddr*)
> 
>> &addr, addr.sizeof) == -1)
>> 
>         throw new
> 
>> StdioException("Connect failed");
>> 
>
>     FILE* fp = fdopen(s,
> 
>> "w+".ptr);
>> 
>
>     File f;
>
>     auto imp = new
> 
>> File.Impl(fp, 1, host ~ ":" ~ to!string(port));
>> 
>
>     f.p =
> 
>> imp;
>> 
>
>     return
> 
>> f;
>> 
> }
> _______________________________________________
> phobos mailing
> 
>> list
>> 
>
> 
>> href="mailto:phobos at puremagic.com">phobos at puremagic.com
>> 
> http://lists.puremagic.com/mailman/listinfo/phobos
>
>
> 
> _______________________________________________
> phobos mailing list
> phobos at puremagic.com
> http://lists.puremagic.com/mailman/listinfo/phobos
>
>
> 
March 30, 2010
Can you detail the reader/writer operations?  Because popNext and front ain't gonna cut it for streams.  Most of the time the element size you want is defined by the application (and not easily abstracted), not the range, but the range is in charge of the element size (via front).

While I believe ranges can be useful for streams, they are not the best interface for all applications.  For example, if I have a protocol that reads 2 bytes to get a length, and then reads length bytes from the stream, a range of those units is probably a good abstraction.  But I don't want to resort to C calls to create that abstraction -- there should be a nice D layer in between.  I should not have to create my own buffering solution.  I/O performance is more important IMO than interface when it comes to streams.  This does not mean big-O complexity, I'm talking about raw performance.

I hope we can see a design before you commit to doing it this way.  For example, a zip library uses a range as a source, what does the file range look like that satisfies the range properties and also is efficient?  Just seeing the API should be enough to judge.

And are there plans to make a good abstracted library for streams that custom ranges can be built upon?

-Steve



----- Original Message ----
> From: Walter Bright <walter at digitalmars.com>
> 
> I had a long conversation with Andrei about a similar topic, i.e. dealing with zip files. We concluded that the zip file library should know nothing about files or file streams. It should work with ranges. Similarly, designs like "make this memory buffer look like a file stream" are completely wrong for modern designs. It should be "make a file look like a range" and "make a memory buffer look like a range" and then have the reader/writer operate on ranges.

The
> same should hold for network access.

Steve Schveighoffer wrote:
> I have no problem with Adam joining.  As far as the network library, do we want to increase our dependency on the C FILE* structure/api?  I would not want that.  I don't know a lot of network solutions that put a socket fd inside a FILE *, there's probably good reason for that.
> 
> As long as there's a non-buffered solution that allows socket-like functions accessible, I'm cool with File wrapping a socket.  That is, there should be a way to create a socket with the API that doesn't use File.
> 
> I hope we will eventually get a D-only solution for File.  Good news is, any future improvements to File automatically translate to what Adam is working on.
> 
> 
> -Steve
> 
> 
> 
> ----- Original Message 
> ----
> 
>> From: Andrei Alexandrescu <
> ymailto="mailto:andrei at erdani.com" href="mailto:andrei at erdani.com">andrei at erdani.com>
>> To:
> Discuss the phobos library for D <
> href="mailto:phobos at puremagic.com">phobos at puremagic.com>
>> 
> Sent: Sun, March 28, 2010 7:21:20 PM
>> Subject: [phobos] Fwd: [Issue
> 4025] New: Making network with the std.stdio.File interface
>> 
> 
>> I'm quite hopeful about this development. If we have anything for
> the network in D2, that would be great, and integration with File will reuse a ton of work.
>> 
> 
> I'm considering extending Adam an offer to join Phobos. He has
>> donated a
> Windows machine for building and running Phobos and generally seems to know what he's talking about. If he'll join the project, my hope is to attract him to dedicate more time to it.
>> 
> 
> Please cast your vote by replying
>> (silence = abstaining, i.e.
> you're not for or against).
>> 
> 
> 
> 
> Andrei
> 
> -------- Original Message  
>> 
> --------
>> 
> Subject: [Issue 4025] New: Making network with the std.stdio.File
>> interface
>> 
> 
> Date: Sun, 28 Mar 2010 22:51:45 +0000 (UTC)
> From:
> 
>> ymailto="mailto:
> href="mailto:d-bugmail at puremagic.com">d-bugmail at puremagic.com"
> href="mailto:
> href="mailto:d-bugmail at puremagic.com">d-bugmail at puremagic.com">
> ymailto="mailto:d-bugmail at puremagic.com"
> href="mailto:d-bugmail at puremagic.com">d-bugmail at puremagic.com
>> 
> 
> Organization:
>> Digital
> Mars
>> 
> To:
>> href="mailto:
> ymailto="mailto:digitalmars-d-bugs at puremagic.com"
> href="mailto:digitalmars-d-bugs at puremagic.com">digitalmars-d-bugs at puremagic.com">
> ymailto="mailto:digitalmars-d-bugs at puremagic.com"
> href="mailto:digitalmars-d-bugs at puremagic.com">digitalmars-d-bugs at puremagic.com
>> 
> 
> Newsgroups:
>> 
> digitalmars.D.bugs
>> 
> 
> 
> http://d.puremagic.com/issues/show_bug.cgi?id=4025
> 
> 
> 
>>          Summary: Making network with the
> std.stdio.File interface
>> 
> 
>       Product: D
> 
>> 
>       Version: 2.041
>> 
> 
> 
>> Platform:
> Other
>> 
>         OS/Version:
> Linux
> 
>> 
> Status: NEW
>> 
> 
> 
>>   Severity: enhancement
>> 
> 
>           Priority:
>> 
> P2
>> 
> 
> Component: Phobos
> 
>> 
> AssignedTo: href="mailto:
> href="mailto:nobody at puremagic.com">nobody at puremagic.com">
> ymailto="mailto:nobody at puremagic.com"
> href="mailto:nobody at puremagic.com">nobody at puremagic.com
>> 
> 
> 
>>     ReportedBy:
> href="mailto:
> href="mailto:destructionator at gmail.com">destructionator at gmail.com">
> ymailto="mailto:destructionator at gmail.com"
> href="mailto:destructionator at gmail.com">destructionator at gmail.com
>> 
> 
> 
> 
> ---  
>> Comment #0 from Adam
> D. Ruppe <
>> href="mailto:
> ymailto="mailto:destructionator at gmail.com" href="mailto:destructionator at gmail.com">destructionator at gmail.com"> ymailto="mailto:destructionator at gmail.com" href="mailto:destructionator at gmail.com">destructionator at gmail.com> 2010-03-28 15:51:43 PDT ---
>> 
> I've written a small module that opens a network
>> connection, then wraps it
> in
>> 
> the File struct, allowing you to use the byLine,
>> etc., ranges on it, as well as
>> 
> 
> writef/readln/etc.
> 
> It is Linux only,
> 
>> but should be pretty easy to port to other operating
> systems.
>> 
> Ideally, I'd
>> 
> like to eventually be able to use File for talking
>> 
> 
> to processes too, on all D
>> 
> platforms.
>> 
> 
> Here's the code I have for
>> now:
>> 
> 
> 
> ==============
> 
> 
> 
> public import
> std.stdio;
> import
>> std.string;
>> 
> 
> 
> import std.conv;
> 
> 
> version(linux):
> 
> import
>> 
> std.c.linux.linux;
>> 
> import std.c.linux.socket;
> 
> alias std.c.linux.socket
> 
>> sock;
>> 
> alias std.c.linux.linux linux;
> 
> enum int PF_INET = 2;
> enum int
> 
>> AF_INET = PF_INET;
>> 
> 
> 
> extern(C) FILE* fdopen(int, const(char)*);
> 
> File
> 
>> openNetwork(string host, ushort port) {
>> 
> 
>     hostent* h;
> 
>> 
> sockaddr_in addr;
>> 
> 
>     h
> =
>> 
> gethostbyname(std.string.toStringz(host));
>> 
> 
>     if(h is
>> null)
>> 
> 
>         throw new
>> 
> StdioException("gethostbyname");
>> 
> 
> 
>     int s = socket(PF_INET,
>> SOCK_STREAM,
> 0);
>> 
>     if(s == -1)
> 
> 
>> throw new
> StdioException("socket");
>> 
> 
> 
> 
>> scope(failure)
>> 
> 
>         close(s);
> 
> 
> 
>> addr.sin_family = AF_INET;
>> 
> 
>     addr.sin_port =
>> 
> htons(port);
>> 
> 
> std.c.string.memcpy(&addr.sin_addr.s_addr,
>> h.h_addr,
> h.h_length);
>> 
> 
> 
> if(sock.connect(s, cast(sockaddr*)
>> &addr, addr.sizeof)
> == -1)
>> 
>         throw
> new
>> StdioException("Connect failed");
>> 
> 
> 
>     FILE* fp = fdopen(s,
>> 
> "w+".ptr);
>> 
> 
>     File
> f;
> 
>     auto imp = new
>> 
> File.Impl(fp, 1, host ~ ":" ~ to!string(port));
>> 
> 
> 
>     f.p =
>> imp;
>> 
> 
> 
>     return
>> 
> f;
>> 
> }
> 
> _______________________________________________
> phobos mailing
> 
>> list
>> 
> 
> 
> 
>> href="mailto:
> href="mailto:phobos at puremagic.com">phobos at puremagic.com">
> ymailto="mailto:phobos at puremagic.com"
> href="mailto:phobos at puremagic.com">phobos at puremagic.com
>> 
> 
> http://lists.puremagic.com/mailman/listinfo/phobos
> 
> 
> 
> 
> _______________________________________________
> phobos mailing
> list
> 
> href="mailto:phobos at puremagic.com">phobos at puremagic.com
> 
> href="http://lists.puremagic.com/mailman/listinfo/phobos" target=_blank
> >http://lists.puremagic.com/mailman/listinfo/phobos
> 
> 
> 
> 
_______________________________________________
phobos
> mailing list

> href="mailto:phobos at puremagic.com">phobos at puremagic.com

> href="http://lists.puremagic.com/mailman/listinfo/phobos" target=_blank
> >http://lists.puremagic.com/mailman/listinfo/phobos



April 07, 2010
Sorry, I totally forgot to answer this one.  I don't know if I've earned the right to vote here yet, but if so I vote 'yes'.  (Or was it 'yarrrr'?)

Regarding the use of std.stdio.File:  Though I agree with Steve that a native D solution for buffered IO would be way better than relying on FILE*, I really like the *interface* of File.  (In particular, I like the byLine and byChunk ranges.)

And that's really all we need to worry about at the moment -- find a good interface now, improve the implementation later.  A good interface should be as implementation independent as possible, so there will be minimal breakage to user code later.

-Lars



Andrei Alexandrescu wrote:
> I'm quite hopeful about this development. If we have anything for the network in D2, that would be great, and integration with File will reuse a ton of work.
> 
> I'm considering extending Adam an offer to join Phobos. He has donated a Windows machine for building and running Phobos and generally seems to know what he's talking about. If he'll join the project, my hope is to attract him to dedicate more time to it.
> 
> Please cast your vote by replying (silence = abstaining, i.e. you're not for or against).
> 
> 
> Andrei
> 
> -------- Original Message --------
> Subject: [Issue 4025] New: Making network with the std.stdio.File interface
> Date: Sun, 28 Mar 2010 22:51:45 +0000 (UTC)
> From: d-bugmail at puremagic.com
> Organization: Digital Mars
> To: digitalmars-d-bugs at puremagic.com
> Newsgroups: digitalmars.D.bugs
> 
> http://d.puremagic.com/issues/show_bug.cgi?id=4025
> 
>            Summary: Making network with the std.stdio.File interface
>            Product: D
>            Version: 2.041
>           Platform: Other
>         OS/Version: Linux
>             Status: NEW
>           Severity: enhancement
>           Priority: P2
>          Component: Phobos
>         AssignedTo: nobody at puremagic.com
>         ReportedBy: destructionator at gmail.com
> 
> 
> --- Comment #0 from Adam D. Ruppe <destructionator at gmail.com> 2010-03-28 
> 15:51:43 PDT ---
> I've written a small module that opens a network connection, then wraps
> it in
> the File struct, allowing you to use the byLine, etc., ranges on it, as
> well as
> writef/readln/etc.
> 
> It is Linux only, but should be pretty easy to port to other operating
> systems.
> Ideally, I'd like to eventually be able to use File for talking
> to processes too, on all D platforms.
> 
> Here's the code I have for now:
> 
> ==============
> 
> 
> 
> public import std.stdio;
> import std.string;
> 
> import std.conv;
> 
> version(linux):
> 
> import std.c.linux.linux;
> import std.c.linux.socket;
> 
> alias std.c.linux.socket sock;
> alias std.c.linux.linux linux;
> 
> enum int PF_INET = 2;
> enum int AF_INET = PF_INET;
> 
> extern(C) FILE* fdopen(int, const(char)*);
> 
> File openNetwork(string host, ushort port) {
>     hostent* h;
>     sockaddr_in addr;
> 
>     h = gethostbyname(std.string.toStringz(host));
>     if(h is null)
>         throw new StdioException("gethostbyname");
> 
>     int s = socket(PF_INET, SOCK_STREAM, 0);
>     if(s == -1)
>         throw new StdioException("socket");
> 
>     scope(failure)
>         close(s);
> 
>     addr.sin_family = AF_INET;
>     addr.sin_port = htons(port);
>     std.c.string.memcpy(&addr.sin_addr.s_addr, h.h_addr, h.h_length);
> 
>     if(sock.connect(s, cast(sockaddr*) &addr, addr.sizeof) == -1)
>         throw new StdioException("Connect failed");
> 
>     FILE* fp = fdopen(s, "w+".ptr);
> 
>     File f;
> 
>     auto imp = new File.Impl(fp, 1, host ~ ":" ~ to!string(port));
> 
>     f.p = imp;
> 
>     return f;
> }
> _______________________________________________
> phobos mailing list
> phobos at puremagic.com
> http://lists.puremagic.com/mailman/listinfo/phobos
April 07, 2010
On 03/29/2010 05:40 AM, Steve Schveighoffer wrote:
> As far as the network library, do we want to increase our dependency on the C FILE* structure/api?  I would not want that.  I don't know a lot of network solutions that put a socket fd inside a FILE *, there's probably good reason for that.

That's a good question. I agree that we shouldn't make things dependent on the old FILE* API, but on the other hand File doesn't have a lot to do with it. I think it still offers a way to tease the handle out, but under an unsavory name.

> As long as there's a non-buffered solution that allows socket-like functions accessible, I'm cool with File wrapping a socket.  That is, there should be a way to create a socket with the API that doesn't use File.
>
> I hope we will eventually get a D-only solution for File.  Good news is, any future improvements to File automatically translate to what Adam is working on.

Exactly. There are some good ranges already and we plan more.

The problem with File and FILE* is the connection with stdio. Truth is, people will want to use stdin and stdout in D programs. Also, people will want to call C functions without being afraid that I/O will be garbled.

C++ has the sync_with_stdio solution, which is beyond horrible:

http://www.cplusplus.com/reference/iostream/ios_base/sync_with_stdio/

By default synchronization is on, which reduces the speed of iostreams by a large factor. The fact that they are slow to begin with doesn't help either.

We want something that's as far to that design as possible. Walter and I talked at length about this. After having investigated the problem, I concluded that we can efficiently implement File and its related ranges with FILE* and regular stdio by using nonstandard functionality. That functionality is in Gnu's stdio for Linux and dmc's stdio implementation for Windows. We'll also keep some less efficient generic implementation to ease porting to new systems. I don't know how to optimize for OSX.


Andrei
« First   ‹ Prev
1 2