Hello,

I have ref-counted type (keeps reference to mallocated memory) which I would like to release as soon as possible (when I have no refs from my code).

The problem is - standard algorithms can use some GC allocations to keep refs to my data until GC is collected, so that I have no control over memory consumption when non-@nogc algorithms applied to RC data.

Right now I have to avoid something like my_data.splitOn(), but is there any better solution?

Thanks

On Wednesday, 8 July 2020 at 11:26:20 UTC, ikod wrote:
> Hello,
>
> I have ref-counted type (keeps reference to mallocated memory) which I would like to release as soon as possible (when I have no refs from my code).
>
> The problem is - standard algorithms can use some GC allocations to keep refs to my data until GC is collected, so that I have no control over memory consumption when non-@nogc

memory consumption is not your first problem (explained later)

> algorithms applied to RC data.
>
> Right now I have to avoid something like my_data.splitOn(), but is there any better solution?
>
> Thanks

you must work with also on a `.dup`ed version, to leave your custom RC space, and later copy back the results in your ref counted space. Mixing two systems of memory is no joke. This can lead to crashes it at some point your own RC stuff are seen by the GC but not owned by a GC root.

On 7/8/20 7:26 AM, ikod wrote:
> Hello,
> 
> I have ref-counted type (keeps reference to mallocated memory) which I would like to release as soon as possible (when I have no refs from my code).
> 
> The problem is - standard algorithms can use some GC allocations to keep refs to my data until GC is collected, so that I have no control over memory consumption when non-@nogc algorithms applied to RC data.
> 
> Right now I have to avoid something like my_data.splitOn(), but is there any better solution?

My solution to this was to use the GC for ref counted data.

See here: http://schveiguy.github.io/iopipe/iopipe/refc/RefCounted.html

This way, the resources represented by the type (generally things like files or malloc'd items) are cleaned up, but the memory is still safe to use.

-Steve

On Wednesday, 8 July 2020 at 12:30:09 UTC, Steven Schveighoffer wrote:
> On 7/8/20 7:26 AM, ikod wrote:
>> Hello,
>> 
>> I have ref-counted type (keeps reference to mallocated memory) which I would like to release as soon as possible (when I have no refs from my code).
>> 
>> The problem is - standard algorithms can use some GC allocations to keep refs to my data until GC is collected, so that I have no control over memory consumption when non-@nogc algorithms applied to RC data.
>> 
>> Right now I have to avoid something like my_data.splitOn(), but is there any better solution?
>
> My solution to this was to use the GC for ref counted data.
>
> See here: http://schveiguy.github.io/iopipe/iopipe/refc/RefCounted.html
>
> This way, the resources represented by the type (generally things like files or malloc'd items) are cleaned up, but the memory is still safe to use.
>
> -Steve

My understanding to what he means is that a copy of RefCounted will be kept in GC'd memory. So the counter won't hit 0 until the GC collects and calls the destructor for that memory, which isn't guarantee'd to ever happen. Your implementation would have the same problem.

On 7/8/20 11:27 AM, Avrina wrote:
> On Wednesday, 8 July 2020 at 12:30:09 UTC, Steven Schveighoffer wrote:
>> On 7/8/20 7:26 AM, ikod wrote:
>>> Hello,
>>>
>>> I have ref-counted type (keeps reference to mallocated memory) which I would like to release as soon as possible (when I have no refs from my code).
>>>
>>> The problem is - standard algorithms can use some GC allocations to keep refs to my data until GC is collected, so that I have no control over memory consumption when non-@nogc algorithms applied to RC data.
>>>
>>> Right now I have to avoid something like my_data.splitOn(), but is there any better solution?
>>
>> My solution to this was to use the GC for ref counted data.
>>
>> See here: http://schveiguy.github.io/iopipe/iopipe/refc/RefCounted.html
>>
>> This way, the resources represented by the type (generally things like files or malloc'd items) are cleaned up, but the memory is still safe to use.
>>
> 
> My understanding to what he means is that a copy of RefCounted will be kept in GC'd memory. So the counter won't hit 0 until the GC collects and calls the destructor for that memory, which isn't guarantee'd to ever happen. Your implementation would have the same problem.

Oh yeah, I misunderstood the problem.

That's going to be the case no matter what. The only true solution would be to use algorithms that use RefCounted instead of the GC. Which may be trading one problem for another.

-Steve

On Wednesday, 8 July 2020 at 16:28:02 UTC, Steven Schveighoffer wrote:
> On 7/8/20 11:27 AM, Avrina wrote:
>> On Wednesday, 8 July 2020 at 12:30:09 UTC, Steven Schveighoffer wrote:
>>> On 7/8/20 7:26 AM, ikod wrote:
>>>> Hello,
>>>>

>> My understanding to what he means is that a copy of RefCounted will be kept in GC'd memory. So the counter won't hit 0 until the GC collects and calls the destructor for that memory, which isn't guarantee'd to ever happen. Your implementation would have the same problem.

Yes, this is the problem, thanks Avrina!

>
> Oh yeah, I misunderstood the problem.
>
> That's going to be the case no matter what. The only true solution would be to use algorithms that use RefCounted instead of the GC. Which may be trading one problem for another.
>

I see... Probably this can be configurable option (use GC or RC for algorithms internals) or can be inferred from the data type properties (like use RC if data methods used for algorithm are @nogc). But I'm not sure if this is possible and whether it makes practical sense.

> -Steve

On Wednesday, 8 July 2020 at 11:26:20 UTC, ikod wrote:
> Right now I have to avoid something like my_data.splitOn(), but is there any better solution?

You can lend a weak reference to GC structure, its lifetime will be shorter than that of remaining strong references.