I just noticed that std.stdio.File.popen() is POSIX-only.  Taking that
into account, along with the fact that it is buggy (issue 3157), does
anyone mind if I remove it?

It's only used once in Phobos, namely in std.process.shell(), which is
easily implemented by using core.sys.posix.stdio.popen() together with
core.sys.posix.stdio.fdopen() and std.stdio.File.wrapFile().

-Lars

Yet another reason to get rid of FILE * dependency.  Any modern i/o library with opaque reference types uses methods which know how to close it.

-Steve




----- Original Message ----
> From: Lars Tandle Kyllingstad <lars at kyllingen.net>
> To: Phobos mailing list <phobos at puremagic.com>
> Sent: Mon, August 16, 2010 4:11:01 AM
> Subject: [phobos] Removing std.stdio.File.popen()
> 
> I just noticed that std.stdio.File.popen() is POSIX-only.  Taking  that
> into account, along with the fact that it is buggy (issue 3157),  does
> anyone mind if I remove it?
> 
> It's only used once in Phobos, namely  in std.process.shell(), which is
> easily implemented by using  core.sys.posix.stdio.popen() together with
> core.sys.posix.stdio.fdopen() and  std.stdio.File.wrapFile().
> 
> -Lars
> 
> _______________________________________________
> phobos  mailing list
> phobos at puremagic.com
> http://lists.puremagic.com/mailman/listinfo/phobos
> 

Though I agree with you, the fix for the popen-close bug is utterly simple regardless of whether FILE* is used.  The problem is that it requires adding more POSIX-only stuff, which I assume we wish to avoid.


Speaking of std.stdio.File and FILE*, I noticed that Phobos actually does have buffered I/O that doesn't depend on FILE* in std.stream.  I'm not saying we should keep std.stream, but at least we have something we could build a native D implementation of std.stdio.File from.

I seem to recall someone (David?) saying it has some bugs with large files on Linux, but I think that's just a matter of enabling the kernel's LFS API.  (If I'm not mistaken, it's simply a matter of adding declarations to enable the kernel's LFS syscalls to druntime, along with a version() statement to enable or disable them.  Then, people who want large file support can just compile Phobos with that enabled.

Also, I think FILE* has the same problem...

-Lars



On Mon, 2010-08-16 at 04:38 -0700, Steve Schveighoffer wrote:
> Yet another reason to get rid of FILE * dependency.  Any modern i/o library with opaque reference types uses methods which know how to close it.
> 
> -Steve
> 
> 
> 
> 
> ----- Original Message ----
> > From: Lars Tandle Kyllingstad <lars at kyllingen.net>
> > To: Phobos mailing list <phobos at puremagic.com>
> > Sent: Mon, August 16, 2010 4:11:01 AM
> > Subject: [phobos] Removing std.stdio.File.popen()
> > 
> > I just noticed that std.stdio.File.popen() is POSIX-only.  Taking  that
> > into account, along with the fact that it is buggy (issue 3157),  does
> > anyone mind if I remove it?
> > 
> > It's only used once in Phobos, namely  in std.process.shell(), which is
> > easily implemented by using  core.sys.posix.stdio.popen() together with
> > core.sys.posix.stdio.fdopen() and  std.stdio.File.wrapFile().
> > 
> > -Lars
> > 
> > _______________________________________________
> > phobos  mailing list
> > phobos at puremagic.com
> > http://lists.puremagic.com/mailman/listinfo/phobos
> > 
> 
> 
> 
> _______________________________________________
> phobos mailing list
> phobos at puremagic.com
> http://lists.puremagic.com/mailman/listinfo/phobos

Further investigation shows that druntime does indeed have the LFS declarations. :)

-Lars



On Mon, 2010-08-16 at 14:29 +0200, Lars Tandle Kyllingstad wrote:
> Though I agree with you, the fix for the popen-close bug is utterly simple regardless of whether FILE* is used.  The problem is that it requires adding more POSIX-only stuff, which I assume we wish to avoid.
> 
> 
> Speaking of std.stdio.File and FILE*, I noticed that Phobos actually does have buffered I/O that doesn't depend on FILE* in std.stream.  I'm not saying we should keep std.stream, but at least we have something we could build a native D implementation of std.stdio.File from.
> 
> I seem to recall someone (David?) saying it has some bugs with large files on Linux, but I think that's just a matter of enabling the kernel's LFS API.  (If I'm not mistaken, it's simply a matter of adding declarations to enable the kernel's LFS syscalls to druntime, along with a version() statement to enable or disable them.  Then, people who want large file support can just compile Phobos with that enabled.
> 
> Also, I think FILE* has the same problem...
> 
> -Lars
> 
> 
> 
> On Mon, 2010-08-16 at 04:38 -0700, Steve Schveighoffer wrote:
> > Yet another reason to get rid of FILE * dependency.  Any modern i/o library with opaque reference types uses methods which know how to close it.
> > 
> > -Steve
> > 
> > 
> > 
> > 
> > ----- Original Message ----
> > > From: Lars Tandle Kyllingstad <lars at kyllingen.net>
> > > To: Phobos mailing list <phobos at puremagic.com>
> > > Sent: Mon, August 16, 2010 4:11:01 AM
> > > Subject: [phobos] Removing std.stdio.File.popen()
> > > 
> > > I just noticed that std.stdio.File.popen() is POSIX-only.  Taking  that
> > > into account, along with the fact that it is buggy (issue 3157),  does
> > > anyone mind if I remove it?
> > > 
> > > It's only used once in Phobos, namely  in std.process.shell(), which is
> > > easily implemented by using  core.sys.posix.stdio.popen() together with
> > > core.sys.posix.stdio.fdopen() and  std.stdio.File.wrapFile().
> > > 
> > > -Lars
> > > 
> > > _______________________________________________
> > > phobos  mailing list
> > > phobos at puremagic.com
> > > http://lists.puremagic.com/mailman/listinfo/phobos
> > > 
> > 
> > 
> > 
> > _______________________________________________
> > phobos mailing list
> > phobos at puremagic.com
> > http://lists.puremagic.com/mailman/listinfo/phobos
> 
> 
> _______________________________________________
> phobos mailing list
> phobos at puremagic.com
> http://lists.puremagic.com/mailman/listinfo/phobos

I actually use it (which is why I duplicated your bug), but am OK with
removing it, since it is easy enough to get at anyway. For a while, I
did a separate extern(C) for pclose anyway!

However, I don't think something being POSIX only is a good reason to remove something. D should take advantages of whatever platform it is on. Portability is good when you can have it, but it shouldn't be a function killer alone.

But again, in this specific case, it is easy enough to call the C directly, so it could go.

On Mon, 2010-08-16 at 09:04 -0400, Adam Ruppe wrote:
> I actually use it (which is why I duplicated your bug), but am OK with
> removing it, since it is easy enough to get at anyway. For a while, I
> did a separate extern(C) for pclose anyway!
> 
> However, I don't think something being POSIX only is a good reason to remove something. D should take advantages of whatever platform it is on. Portability is good when you can have it, but it shouldn't be a function killer alone.

Two comments:

1. I disagree with you. :)  I think that Phobos' user-visible interface should be completely platform agnostic.  Code that depends only on Phobos should compile and run on any platform.

2. Steve and I have been working on a new version of std.process, which
will at some point, hopefully, obviate the need for popen().  See
pipeProcess() here:

  http://www.kyllingen.net/code/ltk/doc/process.html

The POSIX implementation is more or less complete, but its inclusion in Phobos is currently being blocked by bug 3979.  Also, Steve has run into some very tricky issues with pipes on Windows, fundamentally caused by D's dependence on the DMC runtime.  I don't know how (or if) that is working out.

-Lars

On 8/16/2010 8:29 AM, Lars Tandle Kyllingstad wrote:
> Though I agree with you, the fix for the popen-close bug is utterly simple regardless of whether FILE* is used.  The problem is that it requires adding more POSIX-only stuff, which I assume we wish to avoid.
>
>
> Speaking of std.stdio.File and FILE*, I noticed that Phobos actually does have buffered I/O that doesn't depend on FILE* in std.stream.  I'm not saying we should keep std.stream, but at least we have something we could build a native D implementation of std.stdio.File from.
>
> I seem to recall someone (David?) saying it has some bugs with large files on Linux, but I think that's just a matter of enabling the kernel's LFS API.  (If I'm not mistaken, it's simply a matter of adding declarations to enable the kernel's LFS syscalls to druntime, along with a version() statement to enable or disable them.  Then, people who want large file support can just compile Phobos with that enabled.
>
> Also, I think FILE* has the same problem...
>
> -Lars
> 

I know std.stdio.File is buggy as anything for >2GB on 32-bit (and also just generally buggy).  See bugs 3409, 3410, 3425.  I also know std.stream had some, but I never filed them b/c I figured it was going to be deprecated anyhow.  IIRC as of about October of last year it would just stop reading after 2GB on Linux.

On 8/16/10, Lars Tandle Kyllingstad <lars at kyllingen.net> wrote:
> 1. I disagree with you. :)  I think that Phobos' user-visible interface should be completely platform agnostic.  Code that depends only on Phobos should compile and run on any platform.

That would be nice, and usually can be done. But anyway, we can just have our difference in opinion here. It isn't hard to use the C library directly in most the platform specific places, including here.

> 2. Steve and I have been working on a new version of std.process, which
> will at some point, hopefully, obviate the need for popen().  See
> pipeProcess() here:

That looks awesome!

This looks terrific.  I've always found the old std.process to be way underpowered, especially on Windows.  Does your statement about cross-platformness imply that Windows will eventually be supported, too?

On Mon, Aug 16, 2010 at 9:20 AM, Lars Tandle Kyllingstad <lars at kyllingen.net
> wrote:

> On Mon, 2010-08-16 at 09:04 -0400, Adam Ruppe wrote:
> > I actually use it (which is why I duplicated your bug), but am OK with
> > removing it, since it is easy enough to get at anyway. For a while, I
> > did a separate extern(C) for pclose anyway!
> >
> > However, I don't think something being POSIX only is a good reason to remove something. D should take advantages of whatever platform it is on. Portability is good when you can have it, but it shouldn't be a function killer alone.
>
> Two comments:
>
> 1. I disagree with you. :)  I think that Phobos' user-visible interface should be completely platform agnostic.  Code that depends only on Phobos should compile and run on any platform.
>
> 2. Steve and I have been working on a new version of std.process, which
> will at some point, hopefully, obviate the need for popen().  See
> pipeProcess() here:
>
>  http://www.kyllingen.net/code/ltk/doc/process.html
>
> The POSIX implementation is more or less complete, but its inclusion in Phobos is currently being blocked by bug 3979.  Also, Steve has run into some very tricky issues with pipes on Windows, fundamentally caused by D's dependence on the DMC runtime.  I don't know how (or if) that is working out.
>
> -Lars
>
> _______________________________________________
> phobos mailing list
> phobos at puremagic.com
> http://lists.puremagic.com/mailman/listinfo/phobos
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.puremagic.com/pipermail/phobos/attachments/20100816/32c2d366/attachment.html>

It implies that if we can overcome the obstacles we're facing on Windows, it will definitely be supported.  However, it also implies that if it proves impossible to solve the Windows issues, the design will have to be abandoned or severely reduced.

-Lars



On Mon, 2010-08-16 at 10:11 -0400, David Simcha wrote:
> This looks terrific.  I've always found the old std.process to be way underpowered, especially on Windows.  Does your statement about cross-platformness imply that Windows will eventually be supported, too?
> 
> On Mon, Aug 16, 2010 at 9:20 AM, Lars Tandle Kyllingstad
> <lars at kyllingen.net> wrote:
>         On Mon, 2010-08-16 at 09:04 -0400, Adam Ruppe wrote:
>         > I actually use it (which is why I duplicated your bug), but
>         am OK with
>         > removing it, since it is easy enough to get at anyway. For a
>         while, I
>         > did a separate extern(C) for pclose anyway!
>         >
>         > However, I don't think something being POSIX only is a good
>         reason to
>         > remove something. D should take advantages of whatever
>         platform it is
>         > on. Portability is good when you can have it, but it
>         shouldn't be a
>         > function killer alone.
> 
> 
>         Two comments:
> 
>         1. I disagree with you. :)  I think that Phobos' user-visible
>         interface
>         should be completely platform agnostic.  Code that depends
>         only on
>         Phobos should compile and run on any platform.
> 
>         2. Steve and I have been working on a new version of
>         std.process, which
>         will at some point, hopefully, obviate the need for popen().
>          See
>         pipeProcess() here:
> 
>          http://www.kyllingen.net/code/ltk/doc/process.html
> 
>         The POSIX implementation is more or less complete, but its
>         inclusion in
>         Phobos is currently being blocked by bug 3979.  Also, Steve
>         has run into
>         some very tricky issues with pipes on Windows, fundamentally
>         caused by
>         D's dependence on the DMC runtime.  I don't know how (or if)
>         that is
>         working out.
> 
>         -Lars
> 
> 
>         _______________________________________________
>         phobos mailing list
>         phobos at puremagic.com
>         http://lists.puremagic.com/mailman/listinfo/phobos
> 
> 
> _______________________________________________
> phobos mailing list
> phobos at puremagic.com
> http://lists.puremagic.com/mailman/listinfo/phobos