On 08/18/2010 04:44 AM, Lars Tandle Kyllingstad wrote:
> On Wed, 2010-08-18 at 11:10 +0200, Lars Tandle Kyllingstad wrote:
>> On Wed, 2010-08-18 at 10:17 +0200, Lars Tandle Kyllingstad wrote:
>>> On Tue, 2010-08-17 at 23:46 -0500, Andrei Alexandrescu wrote:
>>>> Incidentally a colleague of mine mentioned today a much faster
>>>> implementation of popen():
>>>>
>>>> http://blog.famzah.net/2009/11/20/a-much-faster-popen-and-system-implementation-for-linux/
>>>>
>>>> http://code.google.com/p/popen-noshell/
>>>>
>>>> We should get that in Phobos.
>>>
>>> This is very useful information!  I'll see if this can be incorporated in the new std.process.
>>
>> I had a more thorough look at it now, and from what I can understand, this trick trades safety for speed.
>
> Forget it.  execve() takes care of creating a new memory space.  I'll definitely add this to std.process.

There we go. Sorry for the noise.

Andrei

On Wed, 2010-08-18 at 04:44 -0500, Andrei Alexandrescu wrote:
> On 08/18/2010 03:10 AM, Lars Tandle Kyllingstad wrote:
> > On Tue, 2010-08-17 at 22:03 -0500, Andrei Alexandrescu wrote:
> >> I understand that, and the motivation has been thoroughly explained in the Unix lore. My question is, why define a separate type? Wouldn't creating a file and calling setbuf(null, 0) suffice?
> >
> >
> > Well, for one thing, I didn't know about setbuf() until just now. :)
> >
> > I still think I prefer the UnbufferedFile solution, though.  setbuf() should be called before anything is read or written to the stream.  This means it will be up to the user to do so, and this is easily forgotten. Compare
> >
> >          auto f = UnbufferedFile("myfile.txt");
> >          ...
> >          auto p = spawnProcess("myapp", f);
> >
> > to
> >
> >          auto f = File("myfile.txt");
> >          setbuf(f.getFP(), null);      // DON'T FORGET THIS!
> >          ...
> >          auto p = spawnProcess("myapp", f);
> 
> spawnProcess could call setbuf itself and fail if that doesn't go through.


On my computer, setvbuf() doesn't fail when you turn off the buffer after reading something.  In particular, the following program runs fine:

        import core.stdc.stdio, std.string;

        void main(string[] args)
        {
            auto file = fopen(toStringz(args[1]), "r");

            auto data = new char[10];
            fread(data.ptr, 1, data.length, file);

            // This should fail, but doesn't:
            assert (setvbuf(file, null, _IONBF, 0) == 0);

            fclose(file);
        }

This means that it either silently ignores the request to turn off buffering, or it ditches a whole lot of data. :(

-Lars

On 08/18/2010 06:38 AM, Lars Tandle Kyllingstad wrote:
> On my computer, setvbuf() doesn't fail when you turn off the buffer
> after reading something.

Same on mine. Further testing suggests that the request is ignored.

Andrei

Or it updates the read offset when the buffer is adjusted.

On Aug 18, 2010, at 4:38 AM, Lars Tandle Kyllingstad wrote:

> On Wed, 2010-08-18 at 04:44 -0500, Andrei Alexandrescu wrote:
>> On 08/18/2010 03:10 AM, Lars Tandle Kyllingstad wrote:
>>> On Tue, 2010-08-17 at 22:03 -0500, Andrei Alexandrescu wrote:
>>>> I understand that, and the motivation has been thoroughly explained in the Unix lore. My question is, why define a separate type? Wouldn't creating a file and calling setbuf(null, 0) suffice?
>>> 
>>> 
>>> Well, for one thing, I didn't know about setbuf() until just now. :)
>>> 
>>> I still think I prefer the UnbufferedFile solution, though.  setbuf() should be called before anything is read or written to the stream.  This means it will be up to the user to do so, and this is easily forgotten. Compare
>>> 
>>>         auto f = UnbufferedFile("myfile.txt");
>>>         ...
>>>         auto p = spawnProcess("myapp", f);
>>> 
>>> to
>>> 
>>>         auto f = File("myfile.txt");
>>>         setbuf(f.getFP(), null);      // DON'T FORGET THIS!
>>>         ...
>>>         auto p = spawnProcess("myapp", f);
>> 
>> spawnProcess could call setbuf itself and fail if that doesn't go through.
> 
> 
> On my computer, setvbuf() doesn't fail when you turn off the buffer after reading something.  In particular, the following program runs fine:
> 
>        import core.stdc.stdio, std.string;
> 
>        void main(string[] args)
>        {
>            auto file = fopen(toStringz(args[1]), "r");
> 
>            auto data = new char[10];
>            fread(data.ptr, 1, data.length, file);
> 
>            // This should fail, but doesn't:
>            assert (setvbuf(file, null, _IONBF, 0) == 0);
> 
>            fclose(file);
>        }
> 
> This means that it either silently ignores the request to turn off buffering, or it ditches a whole lot of data. :(
> 
> -Lars
> 
> _______________________________________________
> phobos mailing list
> phobos at puremagic.com
> http://lists.puremagic.com/mailman/listinfo/phobos

On 8/16/10 8:36 CDT, David Simcha wrote:
> I know std.stdio.File is buggy as anything for >2GB on 32-bit (and also just generally buggy). See bugs 3409, 3410, 3425. I also know std.stream had some, but I never filed them b/c I figured it was going to be deprecated anyhow. IIRC as of about October of last year it would just stop reading after 2GB on Linux.

Getting synced with older emails... I think I've fixed the 2GB issue for real in Linux. Is anyone still experiencing issues on Linux?

Andrei