On Fri, 25 Jun 2004 07:52:21 -0400, Ben Hinkle <bhinkle4@juno.com> wrote:

> Arcane Jill wrote:
>
>> In article <cbf0lp$1lvi$1@digitaldaemon.com>, Ben Hinkle says...
>>
>>> I'd first like to see what Sean does with std.stream plus I tend to agree
>>> with Matthew that more discussion is needed before jumping too soon in any
>>> one direction. There's a lot of cool stuff in mango that I'd love to see
>>> somehow merged with std.stream if possible. Maybe I'll give a poke at
>>> porting Kris's tokenizers and endian stuff to std.stream just to see what
>>> it looks like.
>>
>> I'd be quite happy if std.stream were to be improved. Here are some
>> suggestions. You'll probably think that many of them are trivial, but
>> each, in their own way, contributes a small amount of annoyance, and I'm
>> sure these things could be easily got rid of.
>>
>> 1) Since it is more normal to want buffered file access than non-buffered
>> file access (in C, fopen() is called more often than open()), it makes
>> sense that File should be buffered by default, and there should be a
>> separate class, maybe called RawFileStream or something, for the
>> unbuffered case.
>
> Funny you should suggest RawFileStream because the first version of stream.d
> that I sent Walter had "RawFile" and "File" instead of "File" and
> "BufferedFile". I decided to go with File and BufferedFile for backwards
> compatiblity and to avoid buffering stdin/out (unless the type of stdin/out
> was changed to Stream instead of File). Going back to buffering by default
> is probably better long-term.

i.e.

template BuffereStream(T) {
..etc..
}
class RawFile {
  read();
  write();
}
typedef BufferedStream!(RawFile) File;


>> 2) File should in any case be renamed FileStream
>
> but what else would a File be? ;-)
> Personally I like the analogy with stdio's FILE.

Using my new found knowledge that whatever implements read() and write() is a stream, I agree.
Why waste time typing FileStream when we can type File.

>> 3) FileMode.In and FileMode.Out should be renamed Filemode.IN and
>> Filemode.OUT respectively.
>
> why not FileMode? The dmd "style guide" page indicates FileMode would be
> preferred. The style guide also says all enums should be caps so IN/OUT
> seems right (though I tend to think we should move away from the historical
> baggage of C's preprocessor since FileMode.In doesn't look to me like a
> variable anything else besides a constant).

I find having to type FileMode. all the time annoying. After all the variable required is a FileMode so logically anything I pass must be a FileMode. so why type it?

I'd rather...

File f;
f.open("foo.txt",READ|WRITE|APPEND);

Surely this is possible?

>> 4) It should be possible to construct a File object in create mode, in one
>> step. As in File f = new File(filename, FileMode.CREATE);
>
> yup. or'able with In/Out.

Definately. See my other post, I think we need:

"r"  - READ  - read, fails if file does not exist.
"w"  - CREATE - write, overwrite existing.
"a"  - APPEND - write, create if not exist.
"r+" - READ|WRITE - read, write, fails if file does not exist.
"w+" - READ|CREATE - read, write, overwrite existing.
"a+" - READ|APPEND - read, append, create if not exist.

to emulte all the fopen options, which IMO is a basic requirement, and

WRITE|NEW - write, fail if file exist.

to give us more options than fopen does. :)

>> 5) In fact, all possible combinations of file opening supported by fopen()
>> should be supported by File. It should be possible to assert that the file
>> does or does not exist before opening it (atomically), to truncate or not
>> truncate, to position the file pointer at the start or end of the file, to
>> allow append-only access, etc.
>>
>> 6) The destructor should always close the file
>>
>> 7) EITHER Stream classes should be auto (likely to be an unpopular
>> suggestion, I know), OR there should be an auto wrapper class that you can
>> construct from a Stream, in order to guarantee that the file will be
>> closed in the event of an exception (which could of course be thrown by
>> ANY piece of code). Currently we have to either roll our own auto wrapper,
>> or use a try/catch block.
>>
>> 8) Documentation should be complete and accurate.
>>
>> 9) There should be a FilterStream class, from which BufferedStream
>> inherits, so that we can write our own stream filters. (Java does this.
>> It's neat).
>>
>> 10) Streams don't necessarily have to do transcoding (see - I learnt a new
>> word), but nonetheless it should be POSSIBLE to construct them from a
>> Reader/Writer in order to make such extensions possible in the future.
>>
>> 11) I want the function available(), as Java has. A buffered stream always
>> knows how much it's got left in its buffer, and I have no problem with an
>> unbuffered stream returning zero.
>>
>> 12) stdin, stdout and stderr should be globally available D streams.
>> (Maybe they are already, but point (8) means there's a lot I don't know
>> about existing capabilities)
>
> All look very reasonable.
>
>> 13) Streams should overload the << and >> operators. (Someone suggested
>> using ~. That would be fine too).
>
> I think Walter is hoping the typesafe varargs changes will remove a
> important motivation for adding << and >> (though there is the question of
> run-time vs compile-time safety). I have been playing around with making
> std.stream's printf typesafe and that's why I was trying to rebuild the
> phobos unittests.
>
>> None of these is particularly difficult in and of itself, but together
>> they add up to a frustrating gripe list. But I'm fairly confident that if
>> these flaws are fixed (along with any other gripes which others may
>> mention in the course of this thread) then I imagine that most people will
>> be pretty happy with new improved std.stream.
>>
>>
>>
>>
>>
>>> This will probably open up rat-holes, but two quick examples of things to
>>> discuss:
>>>
>>> 1) in mango it looks like to open a file and read it you need to create a
>>> FileConduit and pass that to a Reader constructor. So you have to grok the
>>> difference between Conduits and Readers/Writers (and maybe Buffers? I
>>> notice IConduit has a createBuffer method so is it not buffered by
>>> default? I'm not sure). In std.stream you make one object and there is
>>> less to grok. The flexibility of mango is probably nice but it adds
>>> complexity. Each person has a different notion of where to draw the
>>> boundaries.
>>
>> But there is logic behind it. Currently, D does no transcoding - that is,
>> writeLine() will spit out raw UTF-8. Now that's fine if your output is
>> going to a text file, but if it's going to a console, you're screwed. Now
>> you COULD simplify this a bit by "automatically" encoding the output in
>> the operating system default encoding - but that would just reverse the
>> problem. Now, output to the console would be fine, but output destined to
>> leave your machine and end up on someone else's machine (e.g. text file,
>> socket, etc.) would also be similarly munged. UTF-8 is pretty much the
>> best portable format, so ideally you only want to encode at the last
>> minute, just before the stream hits the user.
>>
>>
>>
>>> 2) in mango to use object serialization/deserialization you register an
>>> instance of a class so that means at startup you basically have to
>>> instantiate one of every class that might want to be deserialized. Seems
>>> wastful and it could affect class design to avoid having classes that have
>>> interdependencies.
>>
>> I'm not convinced that serialization necessarily has anything to do with
>> streams. You could serialize to a string, or an in-memory buffer. I guess
>> that would be faster for small objects but disadvantageous for very large
>> ones. In any case, you don't need to decide on a firm serialization policy
>> in order to make streams feel nice. That can come later, once we're happy
>> with the basics.
>>
>> Arcane Jill
>



-- 
Using M2, Opera's revolutionary e-mail client: http://www.opera.com/m2/

"Regan Heath" <regan@netwin.co.nz> escribió en el mensaje
news:opr959lrqc5a2sq9@digitalmars.com
|
| ...
|
| I find having to type FileMode. all the time annoying. After all the
| variable required is a FileMode so logically anything I pass must be a
| FileMode. so why type it?
|
| I'd rather...
|
| File f;
| f.open("foo.txt",READ|WRITE|APPEND);
|
| Surely this is possible?
|
| ...
|

The problem is when you have different modules, where each can define READ, etc., in there own particular, conflicting way (sorry, can't think of an example). If you fully qualify them, then there'll be no conflict at all (unless they define FileMode too, but that wouldn't be too likely).

-----------------------
Carlos Santander Bernal

Arcane Jill wrote:
> In this context, a Stream would implement TStream!(ubyte).InputStream and
> TStream!(ubyte).OutputStream; a Reader would implement
> TStream!(ubyte).InputStream and TStream!(dchar).OutputStream; a Writer would
> implement TStream!(dchar).InputStream and TStream!(ubyte).OutputStream; and a
> dchar-to-dchar filter would implement TStream!(dchar).InputStream and
> TStream!(dchar).OutputStream.
> 
> Now THAT's how I'd LIKE to see it done.

Why parameterize Stream?  It seems much simpler to me if all Streams do is read and write arrays of bytes.  Readers and Writers convert bytes to or frome other formats. (floats, strings, xml, whatever)  Filters could merely be decorator objects which wrap a Stream, Reader, or Writer and do some useful permutation.

As a convenience, Readers and Writers could have constructors which accept filenames, saving the programmer the trouble of creating the stream separately.

 -- andy

On Fri, 25 Jun 2004 17:56:01 -0500, Carlos Santander B. <carlos8294@msn.com> wrote:

> "Regan Heath" <regan@netwin.co.nz> escribió en el mensaje
> news:opr959lrqc5a2sq9@digitalmars.com
> |
> | ...
> |
> | I find having to type FileMode. all the time annoying. After all the
> | variable required is a FileMode so logically anything I pass must be a
> | FileMode. so why type it?
> |
> | I'd rather...
> |
> | File f;
> | f.open("foo.txt",READ|WRITE|APPEND);
> |
> | Surely this is possible?
> |
> | ...
> |
>
> The problem is when you have different modules, where each can define READ,
> etc., in there own particular, conflicting way (sorry, can't think of an
> example). If you fully qualify them, then there'll be no conflict at all
> (unless they define FileMode too, but that wouldn't be too likely).

I see what you're saying, but I don't think there is a conflict, because...

enum { READ,WRITE } FileMode;
enum { READ,WRITE } ReganMode;

void foo(FileMode a);
void bar(ReganMode a);

foo(READ);
bar(READ);

when the compiler see's foo, it knows it needs a FileMode, thus READ is interpreted as FileMode.READ.
when the compiler see's bar, it knows it needs a ReganMode, thus READ is interpreted as ReganMode.READ.

Regan.

> -----------------------
> Carlos Santander Bernal
>
>



-- 
Using M2, Opera's revolutionary e-mail client: http://www.opera.com/m2/

On Sat, 26 Jun 2004 11:08:44 +1200, Regan Heath <regan@netwin.co.nz> wrote:

> On Fri, 25 Jun 2004 17:56:01 -0500, Carlos Santander B. <carlos8294@msn.com> wrote:
>
>> "Regan Heath" <regan@netwin.co.nz> escribió en el mensaje
>> news:opr959lrqc5a2sq9@digitalmars.com
>> |
>> | ...
>> |
>> | I find having to type FileMode. all the time annoying. After all the
>> | variable required is a FileMode so logically anything I pass must be a
>> | FileMode. so why type it?
>> |
>> | I'd rather...
>> |
>> | File f;
>> | f.open("foo.txt",READ|WRITE|APPEND);
>> |
>> | Surely this is possible?
>> |
>> | ...
>> |
>>
>> The problem is when you have different modules, where each can define READ,
>> etc., in there own particular, conflicting way (sorry, can't think of an
>> example). If you fully qualify them, then there'll be no conflict at all
>> (unless they define FileMode too, but that wouldn't be too likely).
>
> I see what you're saying, but I don't think there is a conflict, because...
>
> enum { READ,WRITE } FileMode;
> enum { READ,WRITE } ReganMode;
>
> void foo(FileMode a);
> void bar(ReganMode a);
>
> foo(READ);
> bar(READ);
>
> when the compiler see's foo, it knows it needs a FileMode, thus READ is interpreted as FileMode.READ.
> when the compiler see's bar, it knows it needs a ReganMode, thus READ is interpreted as ReganMode.READ.

This is kinda like but not quite the same as the 'with' statement, and actually more similar to the C++ namespace idea. When processing the 'a' variable for 'foo' the namespace is set to that of the 'FileMode' so READ matches the one in FileMode.

So you could concievable say...

foo(READ|cast(FileMode)ReganMode.WRITE);

as you're explicitly stating the namespace of WRITE. the cast above is required as ReganMode is not a FileMode.

Regan

-- 
Using M2, Opera's revolutionary e-mail client: http://www.opera.com/m2/

On Fri, 25 Jun 2004 11:31:12 +0000 (UTC), Arcane Jill wrote:

> In article <opr95cp4td5a2sq9@digitalmars.com>, Regan Heath says...
>>
>>> A File /IS/ a stream. How could it not be? Sorry, I just didn't
>>> understand you
>>> here.
>>
>>That's my point. I dont think it should be a stream. fopen etc is not a stream. We want something as a drop in replacement for that, then, we write a stream class, one that will take any class that support read() write() etc.
> 
> That's what most of us mean by "stream". A stream is simply something that does
> read() and write().


I regard streams as something that allow reads/writes of arbitary number of
bytes. Other file-types only allow read/write of records, rather than
bytes. I only became aware of streams when I left the IBM mainframe world.



-- 
Derek
Melbourne, Australia

Sam McCall wrote:

> Ben Hinkle wrote:
> 
>>>2) File should in any case be renamed FileStream
>> 
>> 
>> but what else would a File be? ;-)
>> Personally I like the analogy with stdio's FILE.
> 
> Well, Java uses File to represent a path. I like the idea of
> File f=new File("C:\\something.txt");
> Tests:
> f.exists()
> f.isFile() // exists and is regular file
> f.isDirectory() // exists and is directory
> f.isDevice() // unix devices, (windows COM1 etc?)
> f.canRead()
> f.canWrite()
> and then
> f.open("r+") // returns a FileStream
> f.open("a",false)      // returns a RawFileStream,
> //optional arguments win again
> Sam

I don't think this is one of Java's highlights, though. A File class shouldn't include directories. Users have a pretty good understanding of what the word "File" means and it doesn't include directories. I think the general consensus is that Java's File class should have been called something like Path.

"Ben Hinkle" <bhinkle4@juno.com> wrote in message news:cbiove$12pm$1@digitaldaemon.com...
> Sam McCall wrote:
>
> > Ben Hinkle wrote:
> >
> >>>2) File should in any case be renamed FileStream
> >>
> >>
> >> but what else would a File be? ;-)
> >> Personally I like the analogy with stdio's FILE.
> >
> > Well, Java uses File to represent a path. I like the idea of
> > File f=new File("C:\\something.txt");
> > Tests:
> > f.exists()
> > f.isFile() // exists and is regular file
> > f.isDirectory() // exists and is directory
> > f.isDevice() // unix devices, (windows COM1 etc?)
> > f.canRead()
> > f.canWrite()
> > and then
> > f.open("r+") // returns a FileStream
> > f.open("a",false)      // returns a RawFileStream,
> > //optional arguments win again
> > Sam
>
> I don't think this is one of Java's highlights, though. A File class shouldn't include directories. Users have a pretty good understanding of what the word "File" means and it doesn't include directories. I think the general consensus is that Java's File class should have been called something like Path.

I would tend to agree with Ben, but Path should be for working with file paths only. Given the subject title, I think it's cool to point out that mango.io places such "attribute" methods in a related class called FileProxy. That way they don't interfere with or pollute mango.io.FilePath. FileProxy takes a FilePath as a constructor argument and, if/when you're ready to actually use the file, FileConduit accepts either FilePath or FileProxy in its constructor.

Decent names are always the hardest thing to come up with though. I usually fail miserably.

In article <cbiarm$hhj$1@digitaldaemon.com>, Andy Friesen says...

>Why parameterize Stream?  It seems much simpler to me if all Streams do is read and write arrays of bytes.  Readers and Writers convert bytes to or frome other formats. (floats, strings, xml, whatever)  Filters could merely be decorator objects which wrap a Stream, Reader, or Writer and do some useful permutation.

Fair enough. That's another way of doing it. The only difference really is that
a dchar-based stream MAY NOT read/write in anything other than multiples of four
bytes. ALL reads/writes, for ALL functions, would have to guarantee this.
Functions like read(byte), read(char), read(short), etc., would all have to
either throw an exception, or (less safe) call read(dchar) and truncate the
result.

But yeah - it could be done that way too, just as easily. Maybe more easily.

Jill

In article <opr959bek45a2sq9@digitalmars.com>, Regan Heath says...

>>> ahh.. ok this would be part of the BufferedStream class. And it would simply return the # in the buffer. Simple, easy, fast, efficient. :)
>>
>> Actually, it should return the number in the buffer PLUS the value
>> returned by
>> the underlying stream's available() function - because the buffered
>> stream can
>> get at least that many bytes from the underlying stream without
>> blocking, and
>> use those bytes to refill its own buffer. (That's assuming that the
>> buffered
>> stream is happy to get less than a bufferful at a time from the
>> underlying
>> stream).
>
>Unless the underlying stream (new word/concept) i.e. the file cannot give it the data without possibly blocking i.e. disk IO right?

Still works even then, because if the underlying stream cannot give it the data without possibly blocking then its available() function will return zero. In this case, adding zero to the return value won't exactly hurt.




>If you like you write RawFile and BufferedStream then typedef BufferedStream!(RawFile) File
>
>Yes?

Fair enough, but the style guide says "meaningless type aliases should be avoided".

Anyway, I /did/ say that many of my file/stream gripes were individually trivial.

Jill