Hehe.. std.stdio.File is actually one of the new parts of Phobos, introduced with DMD 2.029 less than a year ago.  (std.stream, on the other hand, seems to be a relic from D1. It doesn't look like it's seen major updates since 2005...)

Though the underlying code of std.stdio.File may be suboptimal, I have taken a liking to its interface.  But then again, the notion of any input/output stream being a 'file' may not be natural for many users.

-Lars


Steve Schveighoffer wrote:
> Will there be an unbuffered interface to file handles?  std.stdio.File wraps a FILE* which I find substandard.  I don't think D should be relying on C buffering for D-only constructs.
> 
> I actually find the whole notion of relying on libc a little suspect, even for the standard handles.
> 
> Is there a document somewhere which lists which parts of phobos will be trimmed?  I am not very familiar with the library, and I'd like to know what parts to avoid when contributing.
> 
> -Steve
> 
> 
> 
> ----- Original Message ----
>> From: Andrei Alexandrescu <andrei at erdani.com>
>>
>> I plan to throw away all of std.stream.
>>
>> Andrei
> 
> 
> 
> 
> _______________________________________________
> phobos mailing list
> phobos at puremagic.com
> http://lists.puremagic.com/mailman/listinfo/phobos


-- 
Lars Tandle Kyllingstad
@: lars at kyllingen.net
#: 40233221
w: http://www.kyllingen.net

Steve Schveighoffer wrote:
> 
> 
> 
> ----- Original Message ----
>> From: Lars Tandle Kyllingstad <lars at kyllingen.net>
>>
>> Steve Schveighoffer wrote:
>>> Some things about my experience with Tango's Process:
>>>
>>> 1. I included a similar "redirect flags" option for the process, with the
>> added ability to redirect stdout to stderr and vice versa.
>>
>> That's a good idea.
>>
>> I've actually considered making it even more general, by allowing the user to provide File objects to replace the standard streams.  This would make it easy to use the output from one process as input to another, or to pass the contents of a file to a process' input stream. A simplified example that emulates "foo | bar":
>>
>>   Pid spawnProcess(string executable, File useThisAsInput);
>>
>>   // foo | bar
>>   auto fooPid = spawnProcess("foo");
>>   auto barPid = spawnProcess("bar", foo.stdout);
>>
>>   // baz < myfile.txt
>>   auto bazPid = spawnProcess("baz", File("myfile.txt");
>>
>> What do you think?
> 
> That sounds like a good idea.  Hm... I read the std.stream code, and I wasn't aware that Phobos used its own buffering layer in place of C's for everything but the standard handles.  That is good news!  However, you are still using FILE * in your code, you should get rid of that (I don't even know where the File.wrapFile function is).
> 
>>> 2. About the searchPath flag -- Windows' function to create a process already
>> takes into account the PATH variable, so I'm guessing that flag will be a noop on Windows?  Also, execlp and execvp already take into account the PATH.  I can't imagine you would need to build your own PATH parsing/searching method, or to run a command without taking the PATH into account (you can do this anyways by prepending a './').  My advice is to simply use one of those versions of exec, and get rid of the searchPath flag.
>>
>> I really can't decide what I think is best.  On one hand, having the function always search the path is convenient, on the other hand, if your program is going to execute another, there should be as little chance as possible of it executing the wrong program.  (The latter may be of small concern, though, since one can just include a directory in the executable name.)
>>
>> Importantly, though, I think code in Phobos should work in the same way on both Windows and POSIX, so that more or less settles it for me.
>>
>> The main reason I wrote my own path-searching mechanism is that the execvp function not only searches the path, but if that fails it also tries to run the command through the shell.  For some reason I find that annoying.
> 
> Hm... that is annoying.  Now that I look at Tango, it actually uses execve, and does the path parsing (only on Linux/Mac/BSD however).  As you say, it should work the same on both Windows and POSIX, so I think that path search should be implicit.
> 
>>> 3. Tango's Process object included a means to specify the environment for the
>> child process (a useful feature).  A simple argument with a string[string] for the environment variables would be useful.
>>
>> That's coming soon, just haven't gotten around to it yet. :)  In fact I have a few ideas regarding environment variables.  The getenv() function has no business being in std.process.  Rather, the following functions should be in std.system:
>>
>>   string getEnv(string varName);
>>   string[string] getEnv();
> 
> Sounds good.
> 
>>> 5. Some version that parses a command line would be useful.  For example, it
>> would be nice to be able to simply do spawn("ls -l");
>>
>> (That being the default on Windows, right?)  I have considered a spawnShell() function, would that be anything like what you're suggesting?
> 
> What I mean is, having a method that parses the command line according to simple sane rules and calls the array-style version.  I don't suggest simply passing the command to Windows, as the parsing and quote escaping is so ridiculous, nobody should have to deal with it.  Wait until you see it :)
> 
>>> 7. Windows has an additional flag to specify that no console should be
>> created.  This is very essential for things like plugins (where you don't want a black console box to pop up while running a child process).  The way I handled this in Tango is to add a gui flag which was a noop in Linux.
>>
>> My biggest problem here, which I mentioned in my first e-mail, is that I have no experience whatsoever with the Windows API.  I don't even have a computer with Windows on it.  (Though I may be able to get a Windows licence through work, I'll have to check that out.  Then I'll set up a virtual machine or something.)
>>
>> In the event that I'm able to get Windows, we are left with three options, either of which is fine by me:
>>
>> 1. I spend some time on it, and write it myself, from scratch.  This will take a while.
>>
>> 2. You, as the author of Tango's Process class, if possible, give me permission to take the Windows part of that code and adapt it to Phobos.  So far I haven't looked at it, due to those dreaded licensing issues.
>>
>> 3. Someone else writes the Windows version.  This is the fastest option by far, but I guess manpower is in short demand.
>>
> 
> As long as we agree on the interface, I can write the Windows version.  I am not the original author of Tango's Process class, that was Reagan Heath, and then Juan Comellas.  I took over because I needed some functionality that wasn't there, and I found some bugs, and Juan had stopped maintaining it.  Therefore, I can't really give you full permission to copy Tango's Process class.
> 
> But I did write the Windows quote parsing part of it, so I have no problems copying that over.  The rest is pretty straightforward code that can easily be rewritten from scratch.

That sounds like a great plan, and will most likely by far be the fastest way to get it up and running on both systems.  I'll keep working on the POSIX stuff, implementing and fixing the things we have discussed, and try to work the interface into something we can all agree is good.

I would also like some feedback on how well you guys think the interface
will fit with the new concurrency model.  The idea was, as I said
before, to write it in the same style, and when the time comes it should
  (API-wise) be a simple, non-breaking change to just add a MessageBox
to the Pid struct.

-Lars

-- 
Lars Tandle Kyllingstad
@: lars at kyllingen.net
#: 40233221
w: http://www.kyllingen.net

On Thu, Mar 4, 2010 at 3:40 PM, Steve Schveighoffer <schveiguy at yahoo.com>wrote:

> Will there be an unbuffered interface to file handles?  std.stdio.File wraps a FILE* which I find substandard.  I don't think D should be relying on C buffering for D-only constructs.
>
> I actually find the whole notion of relying on libc a little suspect, even for the standard handles.
>

You will find it a lot suspect if you try to work with files over 2 GB.  See bugs 2409 and 2410.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.puremagic.com/pipermail/phobos/attachments/20100304/95972d7d/attachment.htm>

I think ideally we'd separate the notion of opening/closing/buffering streams from fetching their data or writing to them. Data communication should be done via ranges. So instead of having types File and class BufferedFile, we should have File and then ranges that are buffered or not etc.

That won't play great with C's stdio which sets buffers at FILE* level, but I think we can come up with a good design. I've been very happy with using File.byLine and File.byChunk as two efficient methods of reading data from files.


Andrei

Lars Tandle Kyllingstad wrote:
> Hehe.. std.stdio.File is actually one of the new parts of Phobos, introduced with DMD 2.029 less than a year ago.  (std.stream, on the other hand, seems to be a relic from D1. It doesn't look like it's seen major updates since 2005...)
> 
> Though the underlying code of std.stdio.File may be suboptimal, I have taken a liking to its interface.  But then again, the notion of any input/output stream being a 'file' may not be natural for many users.
> 
> -Lars
> 
> 
> Steve Schveighoffer wrote:
>> Will there be an unbuffered interface to file handles?  std.stdio.File wraps a FILE* which I find substandard.  I don't think D should be relying on C buffering for D-only constructs.
>>
>> I actually find the whole notion of relying on libc a little suspect, even for the standard handles.
>>
>> Is there a document somewhere which lists which parts of phobos will be trimmed?  I am not very familiar with the library, and I'd like to know what parts to avoid when contributing.
>>
>> -Steve
>>
>>
>>
>> ----- Original Message ----
>>> From: Andrei Alexandrescu <andrei at erdani.com>
>>>
>>> I plan to throw away all of std.stream.
>>>
>>> Andrei
>>
>>
>>
>>       _______________________________________________
>> phobos mailing list
>> phobos at puremagic.com
>> http://lists.puremagic.com/mailman/listinfo/phobos
> 
> 

I'm confused. 2409 is about delegates and 2410 also looks unrelated. I must be missing something.

FWIW I recall I did solve the large file issue.


Andrei

David Simcha wrote:
> 
> 
> On Thu, Mar 4, 2010 at 3:40 PM, Steve Schveighoffer <schveiguy at yahoo.com <mailto:schveiguy at yahoo.com>> wrote:
> 
>     Will there be an unbuffered interface to file handles?
>      std.stdio.File wraps a FILE* which I find substandard.  I don't
>     think D should be relying on C buffering for D-only constructs.
> 
>     I actually find the whole notion of relying on libc a little
>     suspect, even for the standard handles.
> 
> 
> You will find it a lot suspect if you try to work with files over 2 GB. See bugs 2409 and 2410.
> 
> 
> ------------------------------------------------------------------------
> 
> _______________________________________________
> phobos mailing list
> phobos at puremagic.com
> http://lists.puremagic.com/mailman/listinfo/phobos

(Bangs head against wall.)  Sorry, meant 3409, 3410.

On Thu, Mar 4, 2010 at 4:39 PM, Andrei Alexandrescu <andrei at erdani.com>wrote:

> I'm confused. 2409 is about delegates and 2410 also looks unrelated. I must be missing something.
>
> FWIW I recall I did solve the large file issue.
>
>
> Andrei
>
> David Simcha wrote:
>
>>
>>
>> On Thu, Mar 4, 2010 at 3:40 PM, Steve Schveighoffer <schveiguy at yahoo.com<mailto: schveiguy at yahoo.com>> wrote:
>>
>>    Will there be an unbuffered interface to file handles?
>>     std.stdio.File wraps a FILE* which I find substandard.  I don't
>>    think D should be relying on C buffering for D-only constructs.
>>
>>    I actually find the whole notion of relying on libc a little
>>    suspect, even for the standard handles.
>>
>>
>> You will find it a lot suspect if you try to work with files over 2 GB.
>>  See bugs 2409 and 2410.
>>
>>
>> ------------------------------------------------------------------------
>>
>>
>> _______________________________________________
>> phobos mailing list
>> phobos at puremagic.com
>> http://lists.puremagic.com/mailman/listinfo/phobos
>>
> _______________________________________________
> phobos mailing list
> phobos at puremagic.com
> http://lists.puremagic.com/mailman/listinfo/phobos
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.puremagic.com/pipermail/phobos/attachments/20100304/d90f6c76/attachment.htm>

I see. At a point there was a long discussion between Sean and me about how we can fix that problem. I forgot the conclusion. Anyway, I know the matter can be fixed and at some point I even had a draft fix.

Andrei

David Simcha wrote:
> (Bangs head against wall.)  Sorry, meant 3409, 3410.
> 
> On Thu, Mar 4, 2010 at 4:39 PM, Andrei Alexandrescu <andrei at erdani.com <mailto:andrei at erdani.com>> wrote:
> 
>     I'm confused. 2409 is about delegates and 2410 also looks unrelated.
>     I must be missing something.
> 
>     FWIW I recall I did solve the large file issue.
> 
> 
>     Andrei
> 
>     David Simcha wrote:
> 
> 
> 
>         On Thu, Mar 4, 2010 at 3:40 PM, Steve Schveighoffer
>         <schveiguy at yahoo.com <mailto:schveiguy at yahoo.com>
>         <mailto:schveiguy at yahoo.com <mailto:schveiguy at yahoo.com>>> wrote:
> 
>            Will there be an unbuffered interface to file handles?
>             std.stdio.File wraps a FILE* which I find substandard.  I don't
>            think D should be relying on C buffering for D-only constructs.
> 
>            I actually find the whole notion of relying on libc a little
>            suspect, even for the standard handles.
> 
> 
>         You will find it a lot suspect if you try to work with files
>         over 2 GB.  See bugs 2409 and 2410.
> 
> 
>         ------------------------------------------------------------------------
> 
> 
>         _______________________________________________
>         phobos mailing list
>         phobos at puremagic.com <mailto:phobos at puremagic.com>
>         http://lists.puremagic.com/mailman/listinfo/phobos
> 
>     _______________________________________________
>     phobos mailing list
>     phobos at puremagic.com <mailto:phobos at puremagic.com>
>     http://lists.puremagic.com/mailman/listinfo/phobos
> 
> 
> 
> ------------------------------------------------------------------------
> 
> _______________________________________________
> phobos mailing list
> phobos at puremagic.com
> http://lists.puremagic.com/mailman/listinfo/phobos

Of course it's fixable in principle, but is it fixable in the context of File just being a wrapper around the C stdlib file IO?  I personally don't care if File wraps OS-specific APIs instead of the C stdlib API, but I thought that for whatever reason (maybe the reason was just ease of implementation) File was supposed to only be a thin wrapper over standard C I/O.

On Fri, Mar 5, 2010 at 3:53 PM, Andrei Alexandrescu <andrei at erdani.com>wrote:

> I see. At a point there was a long discussion between Sean and me about how we can fix that problem. I forgot the conclusion. Anyway, I know the matter can be fixed and at some point I even had a draft fix.
>
> Andrei
>
> David Simcha wrote:
>
>> (Bangs head against wall.)  Sorry, meant 3409, 3410.
>>
>> On Thu, Mar 4, 2010 at 4:39 PM, Andrei Alexandrescu <andrei at erdani.com<mailto: andrei at erdani.com>> wrote:
>>
>>    I'm confused. 2409 is about delegates and 2410 also looks unrelated.
>>    I must be missing something.
>>
>>    FWIW I recall I did solve the large file issue.
>>
>>
>>    Andrei
>>
>>    David Simcha wrote:
>>
>>
>>
>>        On Thu, Mar 4, 2010 at 3:40 PM, Steve Schveighoffer
>>        <schveiguy at yahoo.com <mailto:schveiguy at yahoo.com>
>>        <mailto:schveiguy at yahoo.com <mailto:schveiguy at yahoo.com>>> wrote:
>>
>>           Will there be an unbuffered interface to file handles?
>>            std.stdio.File wraps a FILE* which I find substandard.  I don't
>>           think D should be relying on C buffering for D-only constructs.
>>
>>           I actually find the whole notion of relying on libc a little
>>           suspect, even for the standard handles.
>>
>>
>>        You will find it a lot suspect if you try to work with files
>>        over 2 GB.  See bugs 2409 and 2410.
>>
>>
>>
>>  ------------------------------------------------------------------------
>>
>>
>>        _______________________________________________
>>        phobos mailing list
>>        phobos at puremagic.com <mailto:phobos at puremagic.com>
>>
>>        http://lists.puremagic.com/mailman/listinfo/phobos
>>
>>    _______________________________________________
>>    phobos mailing list
>>    phobos at puremagic.com <mailto:phobos at puremagic.com>
>>
>>    http://lists.puremagic.com/mailman/listinfo/phobos
>>
>>
>>
>> ------------------------------------------------------------------------
>>
>> _______________________________________________
>> phobos mailing list
>> phobos at puremagic.com
>> http://lists.puremagic.com/mailman/listinfo/phobos
>>
> _______________________________________________
> phobos mailing list
> phobos at puremagic.com
> http://lists.puremagic.com/mailman/listinfo/phobos
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.puremagic.com/pipermail/phobos/attachments/20100305/8462b4f3/attachment.htm>

Yes, File should support large files whether or not it wraps FILE* or some other OS-dependent handle.

Andrei

David Simcha wrote:
> Of course it's fixable in principle, but is it fixable in the context of File just being a wrapper around the C stdlib file IO?  I personally don't care if File wraps OS-specific APIs instead of the C stdlib API, but I thought that for whatever reason (maybe the reason was just ease of implementation) File was supposed to only be a thin wrapper over standard C I/O.
> 
> On Fri, Mar 5, 2010 at 3:53 PM, Andrei Alexandrescu <andrei at erdani.com <mailto:andrei at erdani.com>> wrote:
> 
>     I see. At a point there was a long discussion between Sean and me
>     about how we can fix that problem. I forgot the conclusion. Anyway,
>     I know the matter can be fixed and at some point I even had a draft fix.
> 
>     Andrei
> 
>     David Simcha wrote:
> 
>         (Bangs head against wall.)  Sorry, meant 3409, 3410.
> 
>         On Thu, Mar 4, 2010 at 4:39 PM, Andrei Alexandrescu
>         <andrei at erdani.com <mailto:andrei at erdani.com>
>         <mailto:andrei at erdani.com <mailto:andrei at erdani.com>>> wrote:
> 
>            I'm confused. 2409 is about delegates and 2410 also looks
>         unrelated.
>            I must be missing something.
> 
>            FWIW I recall I did solve the large file issue.
> 
> 
>            Andrei
> 
>            David Simcha wrote:
> 
> 
> 
>                On Thu, Mar 4, 2010 at 3:40 PM, Steve Schveighoffer
>                <schveiguy at yahoo.com <mailto:schveiguy at yahoo.com>
>         <mailto:schveiguy at yahoo.com <mailto:schveiguy at yahoo.com>>
>                <mailto:schveiguy at yahoo.com <mailto:schveiguy at yahoo.com>
>         <mailto:schveiguy at yahoo.com <mailto:schveiguy at yahoo.com>>>> wrote:
> 
>                   Will there be an unbuffered interface to file handles?
>                    std.stdio.File wraps a FILE* which I find
>         substandard.  I don't
>                   think D should be relying on C buffering for D-only
>         constructs.
> 
>                   I actually find the whole notion of relying on libc a
>         little
>                   suspect, even for the standard handles.
> 
> 
>                You will find it a lot suspect if you try to work with files
>                over 2 GB.  See bugs 2409 and 2410.
> 
> 
> 
>          ------------------------------------------------------------------------
> 
> 
>                _______________________________________________
>                phobos mailing list
>                phobos at puremagic.com <mailto:phobos at puremagic.com>
>         <mailto:phobos at puremagic.com <mailto:phobos at puremagic.com>>
> 
>                http://lists.puremagic.com/mailman/listinfo/phobos
> 
>            _______________________________________________
>            phobos mailing list
>            phobos at puremagic.com <mailto:phobos at puremagic.com>
>         <mailto:phobos at puremagic.com <mailto:phobos at puremagic.com>>
> 
>            http://lists.puremagic.com/mailman/listinfo/phobos
> 
> 
> 
>         ------------------------------------------------------------------------
> 
>         _______________________________________________
>         phobos mailing list
>         phobos at puremagic.com <mailto:phobos at puremagic.com>
>         http://lists.puremagic.com/mailman/listinfo/phobos
> 
>     _______________________________________________
>     phobos mailing list
>     phobos at puremagic.com <mailto:phobos at puremagic.com>
>     http://lists.puremagic.com/mailman/listinfo/phobos
> 
> 
> 
> ------------------------------------------------------------------------
> 
> _______________________________________________
> phobos mailing list
> phobos at puremagic.com
> http://lists.puremagic.com/mailman/listinfo/phobos

I'm not sure if this is supported in Windows, but if so, it might be nice if I could set a callback that would be executed when the process completes.

On Mar 3, 2010, at 9:44 AM, Lars Tandle Kyllingstad wrote:

> Hi,
> 
> Recently, I found myself in need of the functionality that is (or should have been) in std.process, but unfortunately I found it lacking in many respects.  Assuming that improving it currently isn't at the top of the to-do list for Phobos, I decided not to wait, and rather to write my own version.  If you want you can check it out here:
> 
> Code:   http://github.com/kyllingstad/ltk/blob/master/ltk/process.d Docs:   http://kyllingen.net/code/ltk/doc/process.html
> 
> I don't know if any of it is usable for Phobos, but if it is, I'd be happy to contribute.
> 
> I've tried to write it in the style of std.concurrency, with a function spawnProcess() that returns a Pid struct.  Currently it is for POSIX only, since I have no experience at all with the Windows API.
> 
> -Lars
> _______________________________________________
> phobos mailing list
> phobos at puremagic.com
> http://lists.puremagic.com/mailman/listinfo/phobos