In article <cact25$2406$1@digitaldaemon.com>, Ben Hinkle says...
>
>Eof in the middle of reading an int (for example) shouldn't be expected,
>though.

Agreed.  But what about Eof as a result of reading a char?

> If one is trying to read an int and you only get half an int and a
>return value of false then every call to read(out int) will have to be wrapped in "if" tests instead of try-catches. Reading multiple values would look like
> if (!s.read(x1) || !s.read(x2) || !s.read(x3)) {evasive action}
>instead of
> try{
>  s.read(x1);
>  s.read(x2);
>  s.read(x3);
> } catch (ReadError re) { evasive action }

It doesn't have to be quite so bad.  I would probably do something like this:

if( s.read(x1).read(x2).read(x3).fail() ) {evasive action}

I admit this is a matter of preference, and there are certainly times when I want to throw an exception on read failure, but the fact that I don't *always* want an exception thrown makes me leery of building it into Stream.  An alternative would be to allow the user to specify that he wants exceptions to be thrown by setting a flag.

>Either way I think eof should be the "usual" way of checking if any data remains.

I agree.  Part of my problem was that I didn't investigate how eof() worked until just now.  Here's the call:

bit eof() { return position() == size(); }

This does allow for foreknowledge of eof() location (a feature I wasn't aware
of), but it does this by scanning to the end of the stream and then resetting
the read position.  In some cases this operation can be quite expensive or just
plain impossible.  An alternative might be to specialize such classes to
read-ahead by one byte on every read to check for eof(), but this will wreak
havoc with streams that may block.  I'm personally used to the method of the C
read() function where it will return 0 on eof(), in which case the stream class
would set an eof() flag and return a failure (or throw an exception I suppose).
But this obviously eliminates any foreknowledge of when eof() has been reached,
moving detection back into the read method.

But again this is all my personal style and there may be a better way to do it (not to mention Mango, which follows yet another approach).  The other consequence of my approach is that it isn't compatible with code written against the existing stream model, which is something I was hoping to avoid.

>Is it possible to just have the mixins be the non-overlapping parts? I'm looking at Stream class and the routines that implement InputStream only call readBlock and eof to do their thing. I can't see any overlap with the implementations of OutputStream.

Definately.  The overlap would only be new code and that could all be placed in a separate mixin.  My question was as much academic as practical.  I'm truly curious whether such a design would result in multiply defined data or not.  In C/C++ I would say yes, but with D's module-level linkage model I'm not so sure.

>That way everything is backwards compatible with the existing stream.d. Users can't instantiate a DefaultInputStream by itself but that's ok.

I'm kind of torn so far as backwards compatibility is concerned.  There are things I don't like about the original design, but at the same time I don't want to propose yet another API format.  However, the differences truly aren't that great so I'm going to go with them for now and see how they turn out.  I admit I'm using this as an excuse to become more familiar with the language as much as to add features to the stream lib, so whatever happens I won't consider it a wasted effort.


Sean

In article <cacidg$1jn3$1@digitaldaemon.com>, Sean Kelly says...

>What bothered me about the exception throwing in Stream was that since all methods are implemented in terms of readExact, an exception will always be throw when eof() is reached.  In my opinion, end of file is an expected rather than an exceptional condition.  If I had to wrap all my stream operations in try blocks I'd go nuts :)

I strongly disagree. Exceptions aren't just for errors, you know. There are all SORTS of non-error uses for exceptions. Just because something is expected, doesn't mean that an exception is not the best mechanism.

Maybe I'm just more comfortable working with exceptions than some. To my mind, best practice behavior is to throw an exception on end-of-file, and a /different/ exception on error.

I like this way of working. It beats the hell out of having to test the return value all the time.

Arcane Jill

On Fri, 11 Jun 2004 19:46:47 +0000 (UTC), Arcane Jill wrote:

> In article <cacidg$1jn3$1@digitaldaemon.com>, Sean Kelly says...
> 
>>What bothered me about the exception throwing in Stream was that since all methods are implemented in terms of readExact, an exception will always be throw when eof() is reached.  In my opinion, end of file is an expected rather than an exceptional condition.  If I had to wrap all my stream operations in try blocks I'd go nuts :)
> 
> I strongly disagree. Exceptions aren't just for errors, you know. There are all SORTS of non-error uses for exceptions. Just because something is expected, doesn't mean that an exception is not the best mechanism.
> 
> Maybe I'm just more comfortable working with exceptions than some. To my mind, best practice behavior is to throw an exception on end-of-file, and a /different/ exception on error.
> 
> I like this way of working. It beats the hell out of having to test the return value all the time.
> 
> Arcane Jill

But isn't this sort of like the 'bit' verses 'bool' thing. They are different animals but a 'bit' can be used to simulate a 'bool'. Likewise, reaching the end of file is not an exceptional event, in fact it is quite normal in most cases involving stream files. So although one can use the D exception mechanism to handle eof, the eof event is not really an exception. I know this just semantics, but ... must we be forced into using exceptions to handle eof, just like we are forced to use 'bit' to handle 'bool'?

-- 
Derek
Melbourne, Australia

Derek wrote:

> But isn't this sort of like the 'bit' verses 'bool' thing. They are
> different animals but a 'bit' can be used to simulate a 'bool'. Likewise,
> reaching the end of file is not an exceptional event, in fact it is quite
> normal in most cases involving stream files. So although one can use the D
> exception mechanism to handle eof, the eof event is not really an
> exception. I know this just semantics, but ... must we be forced into using
> exceptions to handle eof, just like we are forced to use 'bit' to handle
> 'bool'?

If you think about it in terms of how many times the method is called and what fraction of those calls will be a read past the end of the stream, it doesn't seem too much of a stretch to call hitting EOF "exceptional".

Further, this model ensures that, unlike C stdio, forgetting to check EOF causes reliable, reproducable behaviour.

 -- andy

Derek wrote:

> On Fri, 11 Jun 2004 19:46:47 +0000 (UTC), Arcane Jill wrote:
> 
>> In article <cacidg$1jn3$1@digitaldaemon.com>, Sean Kelly says...
>> 
>>>What bothered me about the exception throwing in Stream was that since
>>>all methods are implemented in terms of readExact, an exception will
>>>always be throw
>>>when eof() is reached.  In my opinion, end of file is an expected rather
>>>than an
>>>exceptional condition.  If I had to wrap all my stream operations in try
>>>blocks I'd go nuts :)
>> 
>> I strongly disagree. Exceptions aren't just for errors, you know. There are all SORTS of non-error uses for exceptions. Just because something is expected, doesn't mean that an exception is not the best mechanism.
>> 
>> Maybe I'm just more comfortable working with exceptions than some. To my mind, best practice behavior is to throw an exception on end-of-file, and a /different/ exception on error.
>> 
>> I like this way of working. It beats the hell out of having to test the return value all the time.
>> 
>> Arcane Jill
> 
> But isn't this sort of like the 'bit' verses 'bool' thing. They are different animals but a 'bit' can be used to simulate a 'bool'. Likewise, reaching the end of file is not an exceptional event, in fact it is quite normal in most cases involving stream files. So although one can use the D exception mechanism to handle eof, the eof event is not really an exception. I know this just semantics, but ... must we be forced into using exceptions to handle eof, just like we are forced to use 'bit' to handle 'bool'?

Reaching the end of a file is not exceptional. Reaching the end of a file before expected is exceptional.

In article <1l01dqjpufkr1$.e32a6q1rqkxy.dlg@40tude.net>, Derek says...

>But isn't this sort of like the 'bit' verses 'bool' thing. They are different animals but a 'bit' can be used to simulate a 'bool'.

Not exactly. The difference between bool and bit is whether or not compile-time error checking is possible for a certain class of error. The difference between exceptions and return-value-checking is RUN-TIME error checking (versus crashing, which is what would happen if you forgot to check the return code).


>Likewise,
>reaching the end of file is not an exceptional event, in fact it is quite
>normal in most cases involving stream files.

Let me quote you something from "The C++ Programming Language 3rd Edition" by Bjarne Stroustrup (the creator of C++): "Can an event that happens most times a program is run be considered exceptional? Can an event that is planned for and handled be considered an error? The answer to both questions is yes. 'Exceptional' does not mean 'almost never happens' or 'disastrous'. It is better to think of an exception as meaning 'some part of the system couldn't do what it was asked to do'".

Further on in the book, there is a whole section entitled Exceptions that are not Errors, in which he says: "If an exception is expected and caught so that it has no bad effects on the behavior of the program then how can it be an error? Only because the programmer thinks of it as an error and of the exception-handling mechanisms as tools for handling errors. Alternatively, one might think of the exception-handling mechanisms as simply another control structure".

I have written code in which an exception signals success! And this was not because I'm completely mad (I refrain from comment on that point), but because it was a brilliantly elegant thing to do. So it is here. We've reached the end of the file - yippee - now let's roll up the stack and move onto the next bit of code.

In D, we should be using exceptions even more than in C++, because the exception handling mechanism is much, much simpler and there is way less overhead.

Walter has written some good stuff in the D manual about why exceptions are better than return codes in handling errors. He should probably have mentioned that this is also true for non-errors.


>I know this just semantics, but ... must we be forced into using exceptions to handle eof, just like we are forced to use 'bit' to handle 'bool'?

I think you'll find it's the other way round. On the bit/bool question, non-arithmetic bool would have been the safer option, but we have not been offered them. On the exception versus check-the-return code question, you can code it either ways, but one way is MUCH safer than the other.

If I wrote an API which /required/ that callers check the return value from some funtion, how long do you think it would be before someone forgot? Exceptions are the path of safety.

I think people are just afraid of using them, that's all. Maybe because of unfamilairity, or they have come to associate the exception-mechanism with error-handling. To me, refusing to employ the exception-mechanism because it isn't an error is like refusing to use for-loops because it isn't an error.

I would hate to have to go back to the tedious and error-prone methods of return-value-checking. New tools give us new and powerful techniques. We should use them, not be afraid of them.

Arcane Jill

In article <caddo3$2spa$1@digitaldaemon.com>, Ben Hinkle says...
>
>Reaching the end of a file is not exceptional. Reaching the end of a file before expected is exceptional.

If the user calls a function that expects a 4-byte chunk of data and the function hits eof() after 2 bytes then the stream did not behave in an expected manner.  Whether this is sufficiently bad to be worth an exception is really a matter of opinion, and no one agrees on how exceptions should be used anyway. Between what you and Andy said however, I'm inclined to agree that an exception may be warranted to protect the uncautious user from weird errors, even if I might not like the behavior myself :)

How about this: if the stream expects a fixed-size chunk >1 byte and hits eof() before reading the expected number of bytes then it throws an exception.  If the stream expects only 1 byte and hits eof() (ie. the read failed entirely) then it sets a failure state and returns without throwing.  The only thing about this tactic that worries me is that it seems inconsistent and thus confusing.

Here's an alternate: the unformatted read functions will throw an exception on eof() while the formatted read functions will not.  The assumption being that if a programmer is using the unformatted functions then he's likely reading a data file with a known format and any violations of that format would be exceptional. If the programmer really wants to check for eof() to avoid this problem he could do:

if( istr.peek().eof() )


Sean

Okay, I've been thinking about this and you're slowly winning me over.  I do write a lot of code like this:

if( !istr.read( x ) ) throw new MyError();

and it would make a lot more sense to just expect the stream itself to do it. But I'm still on the fence with the end of file issue.  If eof() can always be evaluated proactively (ie. so I can test for it when I need to and thus avoid an exception) then I'd say let's go ahead and throw excpetions on eof() during a read.  The C functions have the eof() function, and I think in Windows I can call ReadFile with a read length of 0.  Sockets can return closed status.  What about other streams like USB ports and such?  Darn, I think I've just been convinced ;)

Sean

Sean Kelly wrote:

> In article <caddo3$2spa$1@digitaldaemon.com>, Ben Hinkle says...
>>
>>Reaching the end of a file is not exceptional. Reaching the end of a file before expected is exceptional.
> 
> If the user calls a function that expects a 4-byte chunk of data and the
> function hits eof() after 2 bytes then the stream did not behave in an
> expected
> manner.  Whether this is sufficiently bad to be worth an exception is
> really a matter of opinion, and no one agrees on how exceptions should be
> used anyway. Between what you and Andy said however, I'm inclined to agree
> that an exception may be warranted to protect the uncautious user from
> weird errors, even if I might not like the behavior myself :)
> 
> How about this: if the stream expects a fixed-size chunk >1 byte and hits
> eof()
> before reading the expected number of bytes then it throws an exception.
> If the stream expects only 1 byte and hits eof() (ie. the read failed
> entirely) then it
> sets a failure state and returns without throwing.  The only thing about
> this tactic that worries me is that it seems inconsistent and thus
> confusing.

readBlock is an option - though not at convenient.

> Here's an alternate: the unformatted read functions will throw an
> exception on
> eof() while the formatted read functions will not.  The assumption being
> that if a programmer is using the unformatted functions then he's likely
> reading a data file with a known format and any violations of that format
> would be exceptional. If the programmer really wants to check for eof() to
> avoid this problem he could do:
> 
> if( istr.peek().eof() )

Looking at Stream.scanf it assumes one can read one character after the
field being read. For example, the code to parse "%d" is
  while (isdigit(c) && width)
  {
    n = n * 10 + (c - '0');
    width--;
    c = getc();
    count++;
  }
and getc() will throw an exception at eof(). That looks like a bug in scanf.
I haven't tried any example, though, so I could just be reading the code
incorrectly. It would be a little more involved to check for eof but the
code would be more robust. Do you think that is a reasonable set of changes
to make?

> 
> Sean

On Fri, 11 Jun 2004 23:32:32 +0000 (UTC), Arcane Jill wrote:

> In article <1l01dqjpufkr1$.e32a6q1rqkxy.dlg@40tude.net>, Derek says...
> 
>>But isn't this sort of like the 'bit' verses 'bool' thing. They are different animals but a 'bit' can be used to simulate a 'bool'.
> 
> Not exactly. The difference between bool and bit is whether or not compile-time error checking is possible for a certain class of error. The difference between exceptions and return-value-checking is RUN-TIME error checking (versus crashing, which is what would happen if you forgot to check the return code).
> 
> 
>>Likewise,
>>reaching the end of file is not an exceptional event, in fact it is quite
>>normal in most cases involving stream files.
> 
> Let me quote you something from "The C++ Programming Language 3rd Edition" by Bjarne Stroustrup (the creator of C++): "Can an event that happens most times a program is run be considered exceptional? Can an event that is planned for and handled be considered an error? The answer to both questions is yes. 'Exceptional' does not mean 'almost never happens' or 'disastrous'. It is better to think of an exception as meaning 'some part of the system couldn't do what it was asked to do'".
> 
> Further on in the book, there is a whole section entitled Exceptions that are not Errors, in which he says: "If an exception is expected and caught so that it has no bad effects on the behavior of the program then how can it be an error? Only because the programmer thinks of it as an error and of the exception-handling mechanisms as tools for handling errors. Alternatively, one might think of the exception-handling mechanisms as simply another control structure".
> 
> I have written code in which an exception signals success! And this was not because I'm completely mad (I refrain from comment on that point), but because it was a brilliantly elegant thing to do. So it is here. We've reached the end of the file - yippee - now let's roll up the stack and move onto the next bit of code.
> 
> In D, we should be using exceptions even more than in C++, because the exception handling mechanism is much, much simpler and there is way less overhead.
> 
> Walter has written some good stuff in the D manual about why exceptions are better than return codes in handling errors. He should probably have mentioned that this is also true for non-errors.
> 
> 
>>I know this just semantics, but ... must we be forced into using exceptions to handle eof, just like we are forced to use 'bit' to handle 'bool'?
> 
> I think you'll find it's the other way round. On the bit/bool question, non-arithmetic bool would have been the safer option, but we have not been offered them. On the exception versus check-the-return code question, you can code it either ways, but one way is MUCH safer than the other.
> 
> If I wrote an API which /required/ that callers check the return value from some funtion, how long do you think it would be before someone forgot? Exceptions are the path of safety.
> 
> I think people are just afraid of using them, that's all. Maybe because of unfamilairity, or they have come to associate the exception-mechanism with error-handling. To me, refusing to employ the exception-mechanism because it isn't an error is like refusing to use for-loops because it isn't an error.
> 
> I would hate to have to go back to the tedious and error-prone methods of return-value-checking. New tools give us new and powerful techniques. We should use them, not be afraid of them.
> 
> Arcane Jill

Got it. You are using Exceptions as a fancy GOTO.

-- 
Derek
Melbourne, Australia