== Quote from Russell Lewis (webmaster@villagersonline.com)'s article
> Walter Bright wrote:
> > I asked this over on stackoverflow.com to see what people using other languages have to say, as well as the D community. The reason I ask is to see if memory allocation can be allowed in functions marked "nothrow".
> >
> > http://stackoverflow.com/questions/333736/is-out-of-memory-a-recoverable-error
> It seems that D has (or rather, can have) a trivial solution to this problem.  Allow programs to register with the GC when they have memory which can be easily freed (caches and such).  Then you can make "out of memory" a non-recoverable error, since it only hits when we fail to recover enough.

Interesting idea.  It would certainly be easy enough to add to the GC.


Sean

Reply to Russell,

> Walter Bright wrote:
> 
>> I asked this over on stackoverflow.com to see what people using other
>> languages have to say, as well as the D community. The reason I ask
>> is to see if memory allocation can be allowed in functions marked
>> "nothrow".
>> 
>> http://stackoverflow.com/questions/333736/is-out-of-memory-a-recovera
>> ble-error
>> 
> It seems that D has (or rather, can have) a trivial solution to this
> problem.  Allow programs to register with the GC when they have memory
> which can be easily freed (caches and such).  Then you can make "out
> of memory" a non-recoverable error, since it only hits when we fail to
> recover enough.
> 
>

Some sort of higher level of granularity would be nice, mostly an ordering hint re the different stages.

> 
> Seems to me that with this mechanism in place, we can treat
> out-of-memory as an unrecoverable error.
> 
> Thoughts?

vote += 0.5; // nice idea, not sure it would help *me* in any way though.

One can get rid of OOM just by adding a couple of lines to the default malloc, that call custom handler (similar to onOutOfMemoryError handler) wich does any magic wanted, so OOM doesn't break nothrow functions, and if it's thrown, this means that recovery code has failed to recover and application exits. So nothrow protocol remains the same.

== Quote from Kagamin (spam@here.lot)'s article
> One can get rid of OOM just by adding a couple of lines to the default malloc, that call custom handler
(similar to onOutOfMemoryError handler) wich does any magic wanted

Endless loop.


Sean

Kagamin Wrote:

> One can get rid of OOM just by adding a couple of lines to the default malloc, that call custom handler (similar to onOutOfMemoryError handler) wich does any magic wanted, so OOM doesn't break nothrow functions, and if it's thrown, this means that recovery code has failed to recover and application exits. So nothrow protocol remains the same.

I think, preventing OOM is the only possible technique to make sure you don't interfere with the rest of the application.

Don wrote:
> Strategy (1):
> Windows used to have a WM_COMPACTING message (maybe it still does) which was sent when the system was running low on memory. In D, you could imagine a similar sort of system callback, which is called when memory is short -- it means, free some memory now, otherwise you'll get an out of memory error.
> This is much simpler and more powerful than catching an OutOfMemoryException, freeing some memory, and then repeating what you were doing.

I like

"Walter Bright" <newshound1@digitalmars.com> wrote in message news:gh38om$m7r$1@digitalmars.com...
>I asked this over on stackoverflow.com to see what people using other languages have to say, as well as the D community. The reason I ask is to see if memory allocation can be allowed in functions marked "nothrow".
>
> http://stackoverflow.com/questions/333736/is-out-of-memory-a-recoverable-error

I haven't looked at any of the other responses yet, but I'd have to say "sometimes". Clearly, getting an out of memory when trying to instantiate a trivial class is nonrecoverable (unless it's happening within a section of memory-intensive code that's allowed to fail, and the rest of the program does very little allocation), but getting an out of memory when trying to allocate a 1GB buffer for video processing is certainly recoverable.

Russell Lewis, el  2 de diciembre a las 09:57 me escribiste:
> PRE-SCAN
> 
> Before the mark & sweep runs, *every one* of these callbacks is called. These are for things which the program can give up with very little cost, such as emptying free pools in allocators.  Since this happens before the scan, you do *not* have to use explicit "delete"s; you can just drop references as normal.  After all of these callbacks are called, the mark & sweep runs, and we hope that it will find some newly-discarded regions.

I think one could benefit from manually deleting stuff, if possible. This way, space can be found before the collection, and you saved yourself some collecting time. Manually deleting stuff could remain optional, no problem with that, so if you have data that it's not that trivial to decide if it's unused, you can just leave that work to the collector.

> Thoughts?

Looks really interesting. This could be a perfect complement to a way to ask for memory when you expect the memory allocation to fail.

-- 
Leandro Lucarella (luca) | Blog colectivo: http://www.mazziblog.com.ar/blog/
----------------------------------------------------------------------------
GPG Key: 5F5A8D05 (F8CD F9A7 BF00 5431 4145  104C 949E BFB6 5F5A 8D05)
----------------------------------------------------------------------------
People love to judge homeless guys.
Like, your giving him money he's just gonna waste it.
He's just gonna waste the money
Well, he lives in a box, what do you want him to do?
Save up and buy a wall unit?
Take a little run to the store for a throw rug and a CD rack?
He's homeless.

Walter Bright, el  2 de diciembre a las 04:13 me escribiste:
> I asked this over on stackoverflow.com to see what people using other languages have to say, as well as the D community. The reason I ask is to see if memory allocation can be allowed in functions marked "nothrow".
> 
> http://stackoverflow.com/questions/333736/is-out-of-memory-a-recoverable-error

I think all the things said in this thread makes sense (adding callbacks to the garbage collector, adding a way to ask for memory that don't throw if the memory can't be allocated), but I think this don't cover all the possible scenarios.

For example, I'm working on a softswitch (unfortunately no in D). Lets say we have a really bad moment and all the subscribers want to talk at the same time and we don't support that workload. Lets say our memory is exhausted and a new call arrive. A new allocation is done somewhere deep inside the call logic, so the PRE COLLECT callback is called. No memory can be reclaimed, so the GC runs a collection. Still no memory. POST COLLECT and CRISIS are called too without success. My softswitch is down, I lost all the current calls. This is not good for business. What I really wanted to do is to catch the memory error as shallow in the call logic as possible and drop only that current call, leaving all the current established calls intact.

So what can I do? Should I manually check each and every allocation in all the call logic? I think that's unacceptable.

I think this scenario apply to each client-server application that needs to stay alive even with high workloads, as the expense of dropping some connection/client (web servers or web applications for example, as someone said in stackoverflow.com).

-- 
Leandro Lucarella (luca) | Blog colectivo: http://www.mazziblog.com.ar/blog/
----------------------------------------------------------------------------
GPG Key: 5F5A8D05 (F8CD F9A7 BF00 5431 4145  104C 949E BFB6 5F5A 8D05)
----------------------------------------------------------------------------
PADRES DENUNCIAN QUE SU HIJA SE ESCAPO CON UN PARAGUAYITO
	-- Crónica TV

On Wed, 03 Dec 2008 08:34:43 -0500, Leandro Lucarella <llucax@gmail.com> wrote:

> Walter Bright, el  2 de diciembre a las 04:13 me escribiste:
>> I asked this over on stackoverflow.com to see what people using other languages
>> have to say, as well as the D community. The reason I ask is to see if memory
>> allocation can be allowed in functions marked "nothrow".
>>
>> http://stackoverflow.com/questions/333736/is-out-of-memory-a-recoverable-error
>
> I think all the things said in this thread makes sense (adding callbacks
> to the garbage collector, adding a way to ask for memory that don't throw
> if the memory can't be allocated), but I think this don't cover all the
> possible scenarios.
>
> For example, I'm working on a softswitch (unfortunately no in D). Lets say
> we have a really bad moment and all the subscribers want to talk at the
> same time and we don't support that workload. Lets say our memory is
> exhausted and a new call arrive. A new allocation is done somewhere deep
> inside the call logic, so the PRE COLLECT callback is called. No memory
> can be reclaimed, so the GC runs a collection. Still no memory. POST
> COLLECT and CRISIS are called too without success. My softswitch is down,
> I lost all the current calls. This is not good for business. What I really
> wanted to do is to catch the memory error as shallow in the call logic as
> possible and drop only that current call, leaving all the current
> established calls intact.
>
> So what can I do? Should I manually check each and every allocation in all
> the call logic? I think that's unacceptable.
>
> I think this scenario apply to each client-server application that needs
> to stay alive even with high workloads, as the expense of dropping some
> connection/client (web servers or web applications for example, as someone
> said in stackoverflow.com).
>

That's a good point/use case. However,
1) Many client-sever applications seperate each client into a logical thread of some kind. And inside this thread, out of memory is not recoverable.
2) One can still catch an error if need be.