View mode: basic / threaded / horizontal-split · Log in · Help
December 03, 2008
Re: Is "Out of Memory" a recoverable error?
== Quote from Leandro Lucarella (llucax@gmail.com)'s article
> Walter Bright, el  2 de diciembre a las 04:13 me escribiste:
> > I asked this over on stackoverflow.com to see what people using other languages
> > have to say, as well as the D community. The reason I ask is to see if memory
> > allocation can be allowed in functions marked "nothrow".
> >
> > http://stackoverflow.com/questions/333736/is-out-of-memory-a-recoverable-error
> I think all the things said in this thread makes sense (adding callbacks
> to the garbage collector, adding a way to ask for memory that don't throw
> if the memory can't be allocated), but I think this don't cover all the
> possible scenarios.
> For example, I'm working on a softswitch (unfortunately no in D). Lets say
> we have a really bad moment and all the subscribers want to talk at the
> same time and we don't support that workload. Lets say our memory is
> exhausted and a new call arrive. A new allocation is done somewhere deep
> inside the call logic, so the PRE COLLECT callback is called. No memory
> can be reclaimed, so the GC runs a collection. Still no memory. POST
> COLLECT and CRISIS are called too without success. My softswitch is down,
> I lost all the current calls. This is not good for business. What I really
> wanted to do is to catch the memory error as shallow in the call logic as
> possible and drop only that current call, leaving all the current
> established calls intact.

The same could be said of any unexpected error in such an application.  Assuming
that you're doing call processing in a multithreaded app as opposed to dedicating
a process per call (as in Erlang) then you'll have to use try/catch blocks carefully
to perform the necessary cleanup at each stage, re-throwing exceptions you don't
intend to handle.

> So what can I do? Should I manually check each and every allocation in all
> the call logic? I think that's unacceptable.

Agreed.  The point of exceptions is to avoid the need to verify the result of every
operation and to avoid duplication of recovery code.


Sean
December 03, 2008
Re: Is "Out of Memory" a recoverable error?
Robert Jacques wrote:
> 2) One can still catch an error if need be.

Not if the function is nothrow, since the function  never sets up an 
exception handling frame (that's the point of this topic, AFAICT).
December 03, 2008
Re: Is "Out of Memory" a recoverable error?
On Wed, 03 Dec 2008 14:47:14 -0500, Robert Fraser  
<fraserofthenight@gmail.com> wrote:
> Robert Jacques wrote:
>> 2) One can still catch an error if need be.

Yeah, I'm not sure what I was thinking here.

> Not if the function is nothrow, since the function  never sets up an  
> exception handling frame (that's the point of this topic, AFAICT).

Okay, while we've been talking about OutOfMemoryError there's also  
RangeError, AssertError, FinalizeError and SwitchError and not allowing  
array indexing, contract programming or switches inside a nothrow function  
drastically reduces their usefulness (more so than allocation, in my  
opinion). So at least in debug mode, the handling frame (I think) exists  
in order to support RangeError and AssertError errors. However,  
SwitchError and HiddenFuncError occur in release code, so I'm not sure if  
nothrow functions will not have some error handling method. (I think  
FinalizeError is related to allocation and therefore OutOfMemoryError)

PS. I would class removing the exception handling frame as an optimization  
which is separate from the nothrow language contract feature.
December 04, 2008
Re: Is "Out of Memory" a recoverable error?
Robert Fraser wrote:
> Robert Jacques wrote:
>> 2) One can still catch an error if need be.
> 
> Not if the function is nothrow, since the function  never sets up an 
> exception handling frame (that's the point of this topic, AFAICT).

Make a non-recoverable version as standard, and a recoverable version 
that throws an Exception/return null as a last-resort if you really want 
to handle the mess (free other memories).
December 04, 2008
Re: Is "Out of Memory" a recoverable error?
On 2008-12-02 07:13:12 -0500, Walter Bright <newshound1@digitalmars.com> said:

> I asked this over on stackoverflow.com to see what people using other 
> languages have to say, as well as the D community. The reason I ask is 
> to see if memory allocation can be allowed in functions marked 
> "nothrow".
> 
> http://stackoverflow.com/questions/333736/is-out-of-memory-a-recoverable-error

I think you got the question wrong. My answer would be: it depends.

I think you could make this simple reasonable rule: anything that would 
throw an exception in a nothrow function yeilds a fatal error. When you 
want to catch out of memory errors (or any other error for that 
matter), just avoid nothrow.

How does that sound?

It sounds to me like nothrow functions are going to be dangerous in big 
applications since it could make some distant part of an app crash the 
whole thing. At least, you should be allowed to handle those exceptions 
(out of memory, range error, etc) in an error handler function before 
it kills the current thread. But even with that it'll be hard to 
guarenty proper cleanup; that's why I'm saying it makes nothrow 
dangerous.

Note however that you can't guarenty proper cleanup either in case of 
division by zero or invalid pointer dereferencing, so by allowing 
dynamic allocation you're not extending the problem very much. It'd be 
great if those could throw exceptions when not in a nothrow function.

-- 
Michel Fortin
michel.fortin@michelf.com
http://michelf.com/
December 04, 2008
Re: Is "Out of Memory" a recoverable error?
On 2008-12-04 07:34:18 -0500, Michel Fortin <michel.fortin@michelf.com> said:

> I think you got the question wrong. My answer would be: it depends.

I forgot to rephase explicitly the question. What I'm answering in the 
parent post is:

	"When should an out of memory exception be a fatal error?"

And the answer I give in the parent post would be:

	"When you're in a nothrow function!"


-- 
Michel Fortin
michel.fortin@michelf.com
http://michelf.com/
December 04, 2008
Re: Is "Out of Memory" a recoverable error?
Tue, 02 Dec 2008 14:57:49 +0100, Don wrote:

> Walter Bright wrote:
>> I asked this over on stackoverflow.com to see what people using other 
>> languages have to say, as well as the D community. The reason I ask is 
>> to see if memory allocation can be allowed in functions marked "nothrow".
>> 
>> http://stackoverflow.com/questions/333736/is-out-of-memory-a-recoverable-error 
>> 
> I don't think it can be recoverable. Or rather, if it is recoverable, it 
> shouldn't have happened in the first place.
> 
> As far as I can tell, the only thing you could do to recover from an 
> out-of-memory condition is (1) to free some memory, or (2) to switch to 
> an algorithm which doesn't need as much memory.

or (3) to fail a self-contained, user initiated, "pure" operation.  Like
image loading process, or an expensive image editing operation, or a
complex calculation.  You'll find this sort of recovery in any
interactive data processing application.
December 04, 2008
Re: Is "Out of Memory" a recoverable error?
Robert Jacques, el  3 de diciembre a las 09:39 me escribiste:
> That's a good point/use case. However,
> 1) Many client-sever applications seperate each client into a logical thread of 
> some kind. And inside this thread, out of memory is not recoverable.

Unfortunatelly not all designs are multithread. In my for example, we work
with a single thread event loop.

> 2) One can still catch an error if need be.

I don't know what do you mean by that. If you mean you could use
a try/catch block, that's no true if allocation can't throw (to be able
to allocate in nothrow functions).

If you mean you can use an allocation function that don't throw but report
the failure in other way (like returning a null pointer), yes, as I said
one, could always use that function, but is a PITA (specially for string
manipulation, hashes and dynamic arrays).

-- 
Leandro Lucarella (luca) | Blog colectivo: http://www.mazziblog.com.ar/blog/
----------------------------------------------------------------------------
GPG Key: 5F5A8D05 (F8CD F9A7 BF00 5431 4145  104C 949E BFB6 5F5A 8D05)
----------------------------------------------------------------------------
La terapia no sirve: es mucho mejor pagar para hacer las perversiones
que para contarlas.
	-- Alberto Giordano (filósofo estilista)
December 04, 2008
Re: Is "Out of Memory" a recoverable error?
Robert Fraser wrote:
> Robert Jacques wrote:
>> 2) One can still catch an error if need be.
> 
> Not if the function is nothrow, since the function  never sets up an 
> exception handling frame (that's the point of this topic, AFAICT).

Not exactly. If you set up a try-catch, an exception handling frame is 
set up. An exception handling frame is not necessary in order to throw 
an exception, it is only necessary to:

1. catch exceptions
2. unwind exceptions (run destructors)

What happens if a nothrow function does throw is that your destructors 
and finally clauses may not get run in between the throw and the catch. 
For example:

nothrow void foo();

void bar()
{
    try
    {
	foo();
    }
    finally
    {
        will_never_execute(); // even if foo() throws
    }
}

because the compiler will optimize away the finally clause.
December 04, 2008
Re: Is "Out of Memory" a recoverable error?
Robert Jacques wrote:
> Okay, while we've been talking about OutOfMemoryError there's also 
> RangeError, AssertError, FinalizeError and SwitchError and not allowing 
> array indexing, contract programming or switches inside a nothrow 
> function drastically reduces their usefulness (more so than allocation, 
> in my opinion). So at least in debug mode, the handling frame (I think) 
> exists in order to support RangeError and AssertError errors. However, 
> SwitchError and HiddenFuncError occur in release code, so I'm not sure 
> if nothrow functions will not have some error handling method. (I think 
> FinalizeError is related to allocation and therefore OutOfMemoryError)

The idea behind Errors is that, even if you catch the exception, 
unwinding may not occur. Thus, your app should not be relying on 
destructors cleaning up properly. Catching an Error should only be used 
for doing things necessary before shutting down the app.
1 2 3 4 5
Top | Discussion index | About this forum | D home