I have a function that will *massively* benefit from having a persistent internal buffer it can re-use (and grow) from call to call, instead of re-allocating on every call.

What I don't want is either of:
1. To set a fixed limitation of size, if the user ends up making repeated calls to something larger to my fixed size.
2. For a single big call which will allocate a HUGE internal buffer that will consume all my memory.

What I need is some sort of lazy buffer. Basically, the allocation holds, but I don't want the to prevent the GC from collecting it if it deems it has gotten too big, or needs more memory.

Any idea on how to do something like that? Or literature?

On Wednesday, 11 September 2013 at 08:06:37 UTC, monarch_dodra wrote:
> I have a function that will *massively* benefit from having a persistent internal buffer it can re-use (and grow) from call to call, instead of re-allocating on every call.
>
> What I don't want is either of:
> 1. To set a fixed limitation of size, if the user ends up making repeated calls to something larger to my fixed size.
> 2. For a single big call which will allocate a HUGE internal buffer that will consume all my memory.
>
> What I need is some sort of lazy buffer. Basically, the allocation holds, but I don't want the to prevent the GC from collecting it if it deems it has gotten too big, or needs more memory.
>
> Any idea on how to do something like that? Or literature?

I've done something similar before and the general rule then was to start with a small buffer and if you need more just double it. So start with something like 4k(?) (depending on what you need) and before each call make sure you have enough, if not double the buffer by reallocating. This way you grow the buffer but only when needed. Also doubling makes sure you are not reallocating for each call.

Take a look in the core.memory runtime file for the GC methods. The ones of interest for you are: GC.alloc(size) and GC.realloc(*buffer, newSize) or GC.extend(*buffer, minSize, desiredSize). You can then let the GC handle it or free it yourself with GC.free(*buffer).

On Wednesday, 11 September 2013 at 10:28:37 UTC, Gary Willoughby wrote:
> You can then let the GC handle it or free it yourself with GC.free(*buffer).

But if the buffer is stored in a static variable, the GC will never collect it. I *could* also free it myself, but why/when would I do that?

Did you just just let your buffer grow, and never let it get collected?

Is there a way to do something like "I'm using this buffer, but if you want to collect it, then go ahead. I'll reallocate a new one *if/when* I need it again"

On 11/09/13 12:34, monarch_dodra wrote:
> But if the buffer is stored in a static variable, the GC will never collect it.
> I *could* also free it myself, but why/when would I do that?
>
> Did you just just let your buffer grow, and never let it get collected?
>
> Is there a way to do something like "I'm using this buffer, but if you want to
> collect it, then go ahead. I'll reallocate a new one *if/when* I need it again"

How about GC.addRoot and GC.removeRoot ... ?

On 11/09/13 13:14, Joseph Rushton Wakeling wrote:
> On 11/09/13 12:34, monarch_dodra wrote:
>> But if the buffer is stored in a static variable, the GC will never collect it.
>> I *could* also free it myself, but why/when would I do that?
>>
>> Did you just just let your buffer grow, and never let it get collected?
>>
>> Is there a way to do something like "I'm using this buffer, but if you want to
>> collect it, then go ahead. I'll reallocate a new one *if/when* I need it again"
>
> How about GC.addRoot and GC.removeRoot ... ?

I should clarify that a bit more.  I mean, from what I understand, you want to be able to do something like this:

    void foo(/* vars */)
    {
        // 1. if buffer not allocated, allocate as necessary

        // 2. send GC a message: "Hey, I'm using this buffer!  Don't free!

        // 3. carry out your calculations

        // 4. send GC a message: "Hey, this buffer can be freed if you need to."
    }

If I understand right, GC.addRoot should take care of (2) and GC.removeRoot can take care of (3).  Then, if there's a collection cycle in-between calls to foo, fine; if not, next time you enter foo(), the new call to GC.addRoot will protect the memory for the lifetime of the calculation.

But this is conjecture, not speaking from experience :-)

On Wednesday, 11 September 2013 at 08:06:37 UTC, monarch_dodra wrote:
> I have a function that will *massively* benefit from having a persistent internal buffer it can re-use (and grow) from call to call, instead of re-allocating on every call.
>
> What I don't want is either of:
> 1. To set a fixed limitation of size, if the user ends up making repeated calls to something larger to my fixed size.
> 2. For a single big call which will allocate a HUGE internal buffer that will consume all my memory.
>
> What I need is some sort of lazy buffer. Basically, the allocation holds, but I don't want the to prevent the GC from collecting it if it deems it has gotten too big, or needs more memory.
>
> Any idea on how to do something like that? Or literature?

I do not know if it fits, but I had a similar problem some time ago:
http://forum.dlang.org/thread/wsxajhlsupnraevowcgd@forum.dlang.org

11-Sep-2013 14:34, monarch_dodra пишет:
> On Wednesday, 11 September 2013 at 10:28:37 UTC, Gary Willoughby wrote:
>> You can then let the GC handle it or free it yourself with
>> GC.free(*buffer).
>
> But if the buffer is stored in a static variable, the GC will never
> collect it. I *could* also free it myself, but why/when would I do that?
>
> Did you just just let your buffer grow, and never let it get collected?
>
> Is there a way to do something like "I'm using this buffer, but if you
> want to collect it, then go ahead. I'll reallocate a new one *if/when* I
> need it again"

You need weak references. With manually registered finalize for your buffer + flag you might pull it off (but be extremely careful).
There is something like this in an upcoming std.signals2 IIRC.

Basically the sequence should be - pin the pointer with strong ref if it's valid, use it, unpin. If it wasn't valid - it got collected, allocate new buffer and repeat. The "was valid" is the ugly part, and prone to race condition (GC works in its own thread, got to disable/enable etc.).

All in all this is the kind of stuff that:
a) Druntime/Phobos should provided
b) Is actually needed for other things as well

I'd file an enhancement if there isn't one already.
-- 
Dmitry Olshansky

On Wednesday, 11 September 2013 at 11:19:27 UTC, Joseph Rushton
Wakeling wrote:
> On 11/09/13 13:14, Joseph Rushton Wakeling wrote:
>> On 11/09/13 12:34, monarch_dodra wrote:
>>> But if the buffer is stored in a static variable, the GC will never collect it.
>>> I *could* also free it myself, but why/when would I do that?
>>>
>>> Did you just just let your buffer grow, and never let it get collected?
>>>
>>> Is there a way to do something like "I'm using this buffer, but if you want to
>>> collect it, then go ahead. I'll reallocate a new one *if/when* I need it again"
>>
>> How about GC.addRoot and GC.removeRoot ... ?
>
> I should clarify that a bit more.  I mean, from what I understand, you want to be able to do something like this:
>
>     void foo(/* vars */)
>     {
>         // 1. if buffer not allocated, allocate as necessary
>
>         // 2. send GC a message: "Hey, I'm using this buffer!  Don't free!
>
>         // 3. carry out your calculations
>
>         // 4. send GC a message: "Hey, this buffer can be freed if you need to."
>     }
>
> If I understand right, GC.addRoot should take care of (2) and GC.removeRoot can take care of (3).  Then, if there's a collection cycle in-between calls to foo, fine; if not, next time you enter foo(), the new call to GC.addRoot will protect the memory for the lifetime of the calculation.
>
> But this is conjecture, not speaking from experience :-)

That's somewhat better, as it would allow the GC to collect my buffer, if it wants to, but I wouldn't actually know about it afterwards which leaves me screwed.

I *think* addRoot and removeRoot is really designed to pass GC memory to functions that aren't GC-scanned...

On 11/09/13 15:13, monarch_dodra wrote:
> That's somewhat better, as it would allow the GC to collect my buffer, if it
> wants to, but I wouldn't actually know about it afterwards which leaves me screwed.

Just to clarify, is this buffer meant only for internal use in your function or is it meant to be externally accessed as well?  I'd kind of assumed the former.

Either way, isn't it sufficient to have some kind of

    if (buf is null)
    {
        // allocate the buffer
    }

check in place?  The basic model seems right -- at the moment when you need the buffer, you check if it's allocated (and if not, allocate it as needed); you indicate to the GC that it shouldn't collect the memory; you use the buffer; and the moment it's no longer needed, you indicate to the GC that it's collectable again.

It means having to be very careful to check the buffer's allocation status whenever you want to use it, but I think that's an unavoidable consequence of wanting a static variable that can be freed if needed.

The alternative I thought of was something like comparing the size difference between the currently-needed buffer and the last-needed buffer (... or if you want to be over-the-top, compare to a running average:-), and if the current one is sufficiently smaller, free the old one and re-alloc a new one; but that's a bit _too_ greedy in the free-up-memory stakes, I think.

11-Sep-2013 17:33, Joseph Rushton Wakeling пишет:
> On 11/09/13 15:13, monarch_dodra wrote:
>> That's somewhat better, as it would allow the GC to collect my buffer,
>> if it
>> wants to, but I wouldn't actually know about it afterwards which
>> leaves me screwed.
>
> Just to clarify, is this buffer meant only for internal use in your
> function or is it meant to be externally accessed as well?  I'd kind of
> assumed the former.
>
> Either way, isn't it sufficient to have some kind of
>
>      if (buf is null)
>      {
>          // allocate the buffer
>      }
>
> check in place?  The basic model seems right -- at the moment when you
> need the buffer, you check if it's allocated (and if not, allocate it as
> needed); you indicate to the GC that it shouldn't collect the memory;
> you use the buffer; and the moment it's no longer needed, you indicate
> to the GC that it's collectable again.
>
> It means having to be very careful to check the buffer's allocation
> status whenever you want to use it, but I think that's an unavoidable
> consequence of wanting a static variable that can be freed if needed.
>

Problem is - said GC-freed memory could be then reused in some way. I can't imagine how you'd test that the block that is allocated is *still your old* block.

-- 
Dmitry Olshansky