On Thu, 28 Feb 2008 19:34:45 +0300, Russell Lewis <webmaster@villagersonline.com> wrote:

>>  It is not necessary to catch AssertError. Sometimes it is necessary to catch any error. For example, in a HTTP-server you could start processing of a new request and catch different kinds of errors to make appropriate response:
>
> I agree that a web server needs to post an appropriate response.  But if an assert() has failed, you don't know if the failure is in your main program, in a library, or maybe even in your network code.  In that case, you can't really rely on your program to keep working correctly, and the only sane solution would be to restart it.

Violation of contract is not a sign of unrecoverable failure of the program, expecially for preconditions. Quite the contrary contracts help detect unappropriate conditions at earlier stages.

For example, SMS body in 140 bytes long. 7-bit message could be up to 160 symbols, packed into 140 bytes SMS body. You could have 7-bit message packing functions with precondition:

byte[] pack7bitMessage( byte[] message )
  in { assert( message.length <= 160 ); }
  body { ... }

When pack7bitMessage receives message which is longer than 160 symbols there isn't any sign of unrecoverable error. For example, the too long body of message may be received from SMPP PDU submit_sm from a ESME who simply had made an error in their PDU. It may be a sign of presence of error in your code for submit_sm parsing (length of SMS body not checked) but there nothing fatal for whole application.

And that is a situation where application abort and restart don't solve a problem -- ESME simply repeat the problem submit_sm after server restart and server go down again and so on.

> My argument, then, is that you need a metaprogram or "watchdog" (such as the init process in *NIX, or maybe just a "web server launcher") which manages the server programs and restarts them when they crash.

A rather complex application is build from a several layers. A failure on some layer should abort all current processing on that layer, but parent layer could restart the problem layer. So that model with metaprogram/watchdogs or supervisor processes (from Erlang) could be implemented inside a single application with use of safe languages like D, Java or C#.

-- 
Regards,
Yauheni Akhotnikau

It seems that we have a significant programming style difference.  Let me explain and advocate for my style...

I believe that assert()s are ways to programmatically document things that the program must enforce.  An in() contract on a function is a statement that says "any code which calls me, but violates this contract, is buggy."  Exceptions are the way to programmatically handle unusual conditions in well-ordered ways.

So, in your example below, since pack7bitMessage asserts that the message must be no more than 160 characters, then any code which called it must enforce that contract.  Any code which didn't enforce the contract has a bug.

IMHO, if what you want is to have a function which will take input of any length, but refuse to encode things which are longer than 160 characters, then it should be coded as follows:

  byte[] pack7bitMessage( byte[] message )
  {
    if(message.length > 160)
      throw InputTooLongException;

    ... encode ...
  }

This is why I argued that assert()s are unrecoverable errors and exceptions are reasonably recoverable.


Finally, let me ask you a question:

If an failed assert() doesn't represent an unrecoverable logic error, then what does???


eao197 wrote:
> On Thu, 28 Feb 2008 19:34:45 +0300, Russell Lewis <webmaster@villagersonline.com> wrote:
> 
>>>  It is not necessary to catch AssertError. Sometimes it is necessary to catch any error. For example, in a HTTP-server you could start processing of a new request and catch different kinds of errors to make appropriate response:
>>
>> I agree that a web server needs to post an appropriate response.  But if an assert() has failed, you don't know if the failure is in your main program, in a library, or maybe even in your network code.  In that case, you can't really rely on your program to keep working correctly, and the only sane solution would be to restart it.
> 
> Violation of contract is not a sign of unrecoverable failure of the program, expecially for preconditions. Quite the contrary contracts help detect unappropriate conditions at earlier stages.
> 
> For example, SMS body in 140 bytes long. 7-bit message could be up to 160 symbols, packed into 140 bytes SMS body. You could have 7-bit message packing functions with precondition:
> 
> byte[] pack7bitMessage( byte[] message )
>   in { assert( message.length <= 160 ); }
>   body { ... }
> 
> When pack7bitMessage receives message which is longer than 160 symbols there isn't any sign of unrecoverable error. For example, the too long body of message may be received from SMPP PDU submit_sm from a ESME who simply had made an error in their PDU. It may be a sign of presence of error in your code for submit_sm parsing (length of SMS body not checked) but there nothing fatal for whole application.
> 
> And that is a situation where application abort and restart don't solve a problem -- ESME simply repeat the problem submit_sm after server restart and server go down again and so on.
> 
>> My argument, then, is that you need a metaprogram or "watchdog" (such as the init process in *NIX, or maybe just a "web server launcher") which manages the server programs and restarts them when they crash.
> 
> A rather complex application is build from a several layers. A failure on some layer should abort all current processing on that layer, but parent layer could restart the problem layer. So that model with metaprogram/watchdogs or supervisor processes (from Erlang) could be implemented inside a single application with use of safe languages like D, Java or C#.
> 
> --Regards,
> Yauheni Akhotnikau

On Fri, 29 Feb 2008 18:42:05 +0300, Russell Lewis <webmaster@villagersonline.com> wrote:

> I believe that assert()s are ways to programmatically document things that the program must enforce.  An in() contract on a function is a statement that says "any code which calls me, but violates this contract, is buggy."  Exceptions are the way to programmatically handle unusual conditions in well-ordered ways.

Contract may be considered another way: a function with precodition says "I guarantee correct behaviour only when you keeps my precondition".

> So, in your example below, since pack7bitMessage asserts that the message must be no more than 160 characters, then any code which called it must enforce that contract.  Any code which didn't enforce the contract has a bug.

Yes it is. But a bug in code is not always should lead to abortion of entire application.

> IMHO, if what you want is to have a function which will take input of any length, but refuse to encode things which are longer than 160 characters, then it should be coded as follows:
>
>    byte[] pack7bitMessage( byte[] message )
>    {
>      if(message.length > 160)
>        throw InputTooLongException;
>
>      ... encode ...
>    }

Yes that function could be rewritten in such way. But there could be condition when DbC is preferable than defensive programming:
- contracts are part of function prototype, so conract help understanding function behaviour. For example, try look at EiffelBase reference manual (http://docs.eiffel.com) -- sometimes bodies of 'require' and 'ensure' give more information that description of a methods;
- contracts could be turned off in release mode when you sure that there isn't critical bugs. So, function with contracts:

byte[] pack7bitMessage( byte[] message )
  in {
   assert( message );
   assert( message.length <= 160 );
   assert( !is8bitCharFound( message ) );
  }
  body { ... }

will be much faster in speed-critical applications then the function which is written in defensive-programming style.

> Finally, let me ask you a question:
>
> If an failed assert() doesn't represent an unrecoverable logic error, then what does???

As in the case with ordinal exception -- it depends. When you try to add a member into associative array you and get NoMemory exception you don't know is that exception would be unrecoverable or recoverable. If you write yet another WordCounter example such exception will be fatal for your application. But if you write, for example, some image-transformation program for http://picasa.google.com/ you could simple throw away current problematic image, clear all resouces which was allocated for that image, and go to next image in queue.

> eao197 wrote:
>> On Thu, 28 Feb 2008 19:34:45 +0300, Russell Lewis <webmaster@villagersonline.com> wrote:
>>
>>>>  It is not necessary to catch AssertError. Sometimes it is necessary to catch any error. For example, in a HTTP-server you could start processing of a new request and catch different kinds of errors to make appropriate response:
>>>
>>> I agree that a web server needs to post an appropriate response.  But if an assert() has failed, you don't know if the failure is in your main program, in a library, or maybe even in your network code.  In that case, you can't really rely on your program to keep working correctly, and the only sane solution would be to restart it.
>>  Violation of contract is not a sign of unrecoverable failure of the program, expecially for preconditions. Quite the contrary contracts help detect unappropriate conditions at earlier stages.
>>  For example, SMS body in 140 bytes long. 7-bit message could be up to 160 symbols, packed into 140 bytes SMS body. You could have 7-bit message packing functions with precondition:
>>  byte[] pack7bitMessage( byte[] message )
>>   in { assert( message.length <= 160 ); }
>>   body { ... }
>>  When pack7bitMessage receives message which is longer than 160 symbols there isn't any sign of unrecoverable error. For example, the too long body of message may be received from SMPP PDU submit_sm from a ESME who simply had made an error in their PDU. It may be a sign of presence of error in your code for submit_sm parsing (length of SMS body not checked) but there nothing fatal for whole application.
>>  And that is a situation where application abort and restart don't solve a problem -- ESME simply repeat the problem submit_sm after server restart and server go down again and so on.
>>
>>> My argument, then, is that you need a metaprogram or "watchdog" (such as the init process in *NIX, or maybe just a "web server launcher") which manages the server programs and restarts them when they crash.
>>  A rather complex application is build from a several layers. A failure on some layer should abort all current processing on that layer, but parent layer could restart the problem layer. So that model with metaprogram/watchdogs or supervisor processes (from Erlang) could be implemented inside a single application with use of safe languages like D, Java or C#.
>>  --Regards,
>> Yauheni Akhotnikau



-- 
Regards,
Yauheni Akhotnikau

eao197 wrote:
> On Thu, 28 Feb 2008 19:34:45 +0300, Russell Lewis <webmaster@villagersonline.com> wrote:
> 
>>>  It is not necessary to catch AssertError. Sometimes it is necessary to catch any error. For example, in a HTTP-server you could start processing of a new request and catch different kinds of errors to make appropriate response:
>>
>> I agree that a web server needs to post an appropriate response.  But if an assert() has failed, you don't know if the failure is in your 
...
> 
> For example, SMS body in 140 bytes long. 7-bit message could be up to 160 symbols, packed into 140 bytes SMS body. You could have 7-bit message packing functions with precondition:
> 
> byte[] pack7bitMessage( byte[] message )
>   in { assert( message.length <= 160 ); }
>   body { ... }
> 
...

I think you are misunderstanding the purpose of assert and of other conditions.  Remember, these statements will be stripped out of the code when it is compiled for release.  As such, they should only be used to detect conditions which are genuine errors.  The reason that they might be cause would be so that the actual error could be raised from the location where the error is generated, rather than from where it was detected.

In Other Words:  Don't use contracts as a substitute for conditionals, even if the conditional *would* raise an exception.  They aren't the same concept.

Now I'll grant that in some languages the two concepts are fused, and conditionals are constrained to do double duty as exceptions.  Even in those languages, however, there is often a flag that will cause assertions to become null statements when compiled with optimization (or without debugging turned on).
[N.B.:  The prior paragraph is based on certain dialects of C or C++ that are over a decade old now.  It may not apply to currently extant compilers.]

Walter Bright wrote:
> Graham St Jack wrote:
> 
>> It will be a good trick if you can pull it off. I don't see how it can be done without examining the source code of all the called functions, leaving us back where we started with throw specs.
>>
>> The C++ approach of a runtime check isn't perfect, but at least it can be done, and provides a more definite form of documentation than comments.
> 
> 
> Are you sure it does a runtime check for nothrow?
> 
>> If the static checks aren't done well enough and runtime checks aren't done at all, I would prefer to rely on comments.

IMHO, it would behoove the compiler to keep track of possible/potential exceptions a function can throw. (Just as well as (again, IMHO) it would behoove the compiler to keep track of functions that /potentially/ have side effects.)

The workload for doing either defines a pattern, and doing the other is simply a repetition of the pattern. Therefore, doing both is not much more work than doing only one of these.

In the optimal future, entering a function would give you information (not necessarily runtime, but definitely info when browsing the docs) on whether it potentially generates an exception/changes data. A year after this info is standard in the docs, several new ideas about Functional Programming will emerge, in D. (Ermmmm, new to D, anyway.)

-----

Keeping track of these two things may be considered superfluous drudgery, but the potential benefits well outweigh the invested effort.

Robert Fraser wrote:
> Graham St Jack wrote:
> 
>> It will be a good trick if you can pull it off. I don't see how it can be done without examining the source code of all the called functions, leaving us back where we started with throw specs.
> 
> 
> I think the idea would be that nothrow is part of the function signature and a nothrow function can only call other nothrow functions or wrap any function calls or news in a catch(Exception) (since there's no way to know specifically which exceptions could be thrown).

Yes. Therefore, the compiler has to know which functions are "nothrow", at compile time.

Keeping track of this would be a lot of work -- ONE single time when implementing the bokkeeping of it. Later, the compiler programmer (Walter?) would only have to deal with this when creating totally new basic functions or severely altering the existing ones.

And at that time it would not feel like a lot of work. Probably it would be like filling in a few items on a row in a (sql-like) table of function properties for the compiler.

Leandro Lucarella wrote:
> Walter Bright, el 20 de febrero a las 18:58 me escribiste:
> 
>>Graham St Jack wrote:
>>
>>>It is also very helpful when writing code to know that a function you are calling won't throw an exception - it means you have three kinds of functions: those that don't throw, those with comments telling you what they throw, and those that you need to fix the comments for.
>>
>>Experience with this indicates that it is impractical to specify what exceptions a function may throw. The only practical states to specify are:
>>
>>1) does not throw
>>2) always throws
>>3) might throw
>>
>>What is thrown is not reliably computable. For example:
>>
>>import foo;
>>void test()
>>{
>>    foo.bar();
>>}
>>
>>What exceptions does test() throw? That would require knowledge of the import foo, which is not necessarily knowable and can change over time.
> 
> 
> And what's the difference with nothrow? foo.bar's nothrow-ness "is not
> necessarily knowable and can change over time". It's exactly the same,
> you're just limiting the way you specify exceptions, but you are specifing
> them after all.
> 
> This is like limiting C++ exception specification to nothing or an empty
> throw() qualifier.

Of course, there should be a Nothrow-offense exception. Right?!!

Such an exception would be thrown whenever a function that's declared as not throwing an exception actually does throw one.

Georg Wrede wrote:
> Walter Bright wrote:
>> Graham St Jack wrote:
>>
>>> It will be a good trick if you can pull it off. I don't see how it can be done without examining the source code of all the called functions, leaving us back where we started with throw specs.
>>>
>>> The C++ approach of a runtime check isn't perfect, but at least it can be done, and provides a more definite form of documentation than comments.
>>
>>
>> Are you sure it does a runtime check for nothrow?
>>
>>> If the static checks aren't done well enough and runtime checks aren't done at all, I would prefer to rely on comments.
> 
> IMHO, it would behoove the compiler to keep track of possible/potential exceptions a function can throw. (Just as well as (again, IMHO) it would behoove the compiler to keep track of functions that /potentially/ have side effects.)

There's a few problems with what you're asking for:
- extern(D)
- .di files
- delegates
- dynamic loading

There's also the matter of executable bloat.

   Caveat lector!
   Er, upon proofreading it dawned upon me:

       This post does NOT interest Walter or anybody
       else who is concerned with compiler issues,
       the laguage itself, or optimizing. (What, is that
       like everybody in this newsgroup??)

       BUT, it still is very relevant to the advancement
       of D.

       IDE developers, framework architects, and library
       programmers may want to read this.


Christopher Wright wrote:
> Georg Wrede wrote:
>> IMHO, it would behoove the compiler to keep track of possible/potential exceptions a function can throw. (Just as well as (again, IMHO) it would behoove the compiler to keep track of functions that /potentially/ have side effects.)
> 
> There's a few problems with what you're asking for:
> - extern(D)
> - .di files
> - delegates
> - dynamic loading

We probably should reverse the bookkeeping to what the function does /not/ throw. With that, the above things would all be classified as unknown, which translates to "no exception is guaranteed to not get thrown". (For recursive implications, see below.)

> There's also the matter of executable bloat.

All of this is compile-time-only stuff. Once compiled, the code should "look like" code that's never heard of the issue. But it still does the Right Thing.



To be more precise, we're not [here] interested in what a function throws, only in what it is guaranteed /not/ to throw. For example, an in-place string transformation would be guaranteed to not throw an out-of-memory exception. (Unless the programmer is an idiot, but that doesn't count here.)

By the same token, anytime I write a function in my code, the result of what other functions it calls and which they call in their turn, can be drawn as a tree structure. If we had machine readable documentation on each of the leaves, telling us /what it will not throw/, we might go on to the function that uses the leaf, see if it creates new kinds of exceptions and whether it may throw the new ones or any existing exceptions, and then compute the Set Intersection of the exceptions any of them /can not throw/. This gives us a guaranteed set of exceptions that /will not be thrown/ as a result of calling this (er...) Intermediate Node in the tree. By recursively applying this reasoning we ultimately end up with a hard document about my Brand New function, and what it doesn't throw.

(( And as you pointed out earlier, anytime any of the leaves or the nodes uses any of: extern(D), .di files, delegates, dynamic loading; the entire tree gets tainted. That might actually be a shortcut from traversing the tree since we don't need any more information: NO EXCEPTION is guaranteed not to be thrown. ))

While we're at it, this precisely same kind of analysis could be made about side effects. And even using precisely the same mechanics.

All we need is data on the primary functions. Everything else is automatically computable from that. Including the library functions. (As in Phobos, etc.)


The end result is, by the time we've written our own function, it is KNOWABLE what exceptions OUR function doesn't throw. And it is knowable whether our function DOES NOT create ANY side effects.

The value of these two hardly need advertising. Functional programming, real-time stuff, robust code, secure code, the list goes on and on.