May 15, 2012
And what if i use an array of structs? Also clear instead of
delete?
May 15, 2012
On 5/15/12, Jonathan M Davis <jmdavisProg@gmx.com> wrote:
> If you're going to use anything, use clear.

He might as well just use this if he wants the GC to take care of things: arr = null;

clear(arr) is a little bit of a misnomer. If you have another slice pointing to the same array clear() won't really release memory at all, it will just make the first slice null. so using "arr = null" is good enough.

Also good to note is that clear/null assign is very different from what delete does. delete won't care that you have other slices pointing to the same array and you'll end up with slices that point to garbage after a call to delete.
May 16, 2012
On Tue, 15 May 2012 05:43:45 -0400, Jonathan M Davis <jmdavisProg@gmx.com> wrote:

> On Tuesday, May 15, 2012 11:26:49 Namespace wrote:
>> On Tuesday, 15 May 2012 at 09:23:51 UTC, Kagamin wrote:
>> > Difference with what?
>> > new is a safe feature: it allocates in the GC heap
>>
>> That's what i mean. So i have to delete it yourself with "delete
>> arr;", or not?
>
> No. _Never_ use delete. It's going to be deprecated. The GC worries about
> freeing memory allocated on the GC heap, and new always allocates on the GC
> heap. If you don't want to allocate on the GC heap, then use malloc and free,
> in which case you _do_ need worry about freeing the memory.
>
> If you need to force destruction before the GC collects an object, you can
> call clear on that object to have its destructor called and its vtbl zeroed
> out, but it's memory still isn't freed. That's the GC's job.
>
> If you really have to, you can use core.memory to manipulate the GC heap
> (including calling GC.free), but you really shouldn't be messing with any of
> that unless you really need to and you know what you're doing.

Well, the D GC is conservative, and will always be at least partly conservative.  So freeing memory manually is not really a crime.

I agree you should avoid delete, due to its possible deprecation.

-Steve
May 16, 2012
On Wednesday, May 16, 2012 11:51:59 Steven Schveighoffer wrote:
> Well, the D GC is conservative, and will always be at least partly conservative. So freeing memory manually is not really a crime.

The idea with a GC is that it's supposed to manage the memory allocated through it. So, if you're worrying about freeing GC memory, you're fighting the GC and how it works. And unless you're keeping very close track of the number of reference to a particular chunk of memory, freeing it manually is dangerous. So, anyone looking to use delete is generally going about things the wrong way.

However, it _is_ true that sometimes you need to intervene in order to get sections of code as performant as they need to be (particularly since D's current GC is not the most performant), which is part of the reason that core.memory provides what it does. But it should generally be something used when it's clear that you need that extra performance, not as a matter of course, because using it is inherently unsafe. If there's something that you know ahead of time needs to have more deterministic deallocation, then it's probably better to be use malloc and free with ref-counting than trying to manually manage GC memory.

- Jonathan M Davis
May 16, 2012
On Wed, 16 May 2012 12:52:33 -0400, Jonathan M Davis <jmdavisProg@gmx.com> wrote:

> On Wednesday, May 16, 2012 11:51:59 Steven Schveighoffer wrote:
>> Well, the D GC is conservative, and will always be at least partly
>> conservative. So freeing memory manually is not really a crime.
>
> The idea with a GC is that it's supposed to manage the memory allocated
> through it. So, if you're worrying about freeing GC memory, you're fighting the
> GC and how it works. And unless you're keeping very close track of the number
> of reference to a particular chunk of memory, freeing it manually is
> dangerous. So, anyone looking to use delete is generally going about things
> the wrong way.
>
> However, it _is_ true that sometimes you need to intervene in order to get
> sections of code as performant as they need to be (particularly since D's
> current GC is not the most performant), which is part of the reason that
> core.memory provides what it does. But it should generally be something used
> when it's clear that you need that extra performance, not as a matter of
> course, because using it is inherently unsafe. If there's something that you
> know ahead of time needs to have more deterministic deallocation, then it's
> probably better to be use malloc and free with ref-counting than trying to
> manually manage GC memory.

Where the GC gets into trouble is with allocation of large chunks of data.

If the data 'contains pointers', it creates a very good chance that the data keeps some unknown number of blocks from being deallocated.

And just by the sheer fact of how big the data is, it's bound to be pointed at by some piece of stack data, global data, or TLS data.

So you can go a long way by manually managing larger chunks of data.  I think it's worth worrying about as you get into large chunks (on the order of 10MB or more).

What really really sucks about this is, as you allocate larger chunks of data, the chances that the GC incorrectly keeps garbage memory get larger.  In other words, the more memory you allocate, the less likely it is that the GC will give you some of it back to you!

Precise GC will help this *tremendously*, but I'm unsure if we'll ever get to the point of having a fully-precise GC.  There will always be a need to do manual freeing of memory.

-Steve
May 16, 2012
On Wednesday, May 16, 2012 13:52:12 Steven Schveighoffer wrote:
> Where the GC gets into trouble is with allocation of large chunks of data.
> 
> If the data 'contains pointers', it creates a very good chance that the data keeps some unknown number of blocks from being deallocated.
> 
> And just by the sheer fact of how big the data is, it's bound to be pointed at by some piece of stack data, global data, or TLS data.
> 
> So you can go a long way by manually managing larger chunks of data. I think it's worth worrying about as you get into large chunks (on the order of 10MB or more).
> 
> What really really sucks about this is, as you allocate larger chunks of data, the chances that the GC incorrectly keeps garbage memory get larger. In other words, the more memory you allocate, the less likely it is that the GC will give you some of it back to you!

Using 64-bit helps with this.

> Precise GC will help this *tremendously*, but I'm unsure if we'll ever get to the point of having a fully-precise GC.

Any improvement to the GC is a welcome improvement.

> There will always be a need to
> do manual freeing of memory.

Yes, but my point is that it's something you do when the need arises due to performance concerns in a particular section of code. It's not something that you do as a matter of course in your code in general. If you're always trying to free memory as soon as you don't need it anymore rather than letting the GC do its job (or try to anyway), then you might as well be using malloc and free.

- Jonathan M Davis
May 16, 2012
On Wed, May 16, 2012 at 02:20:47PM -0400, Jonathan M Davis wrote:
> On Wednesday, May 16, 2012 13:52:12 Steven Schveighoffer wrote:
> > Where the GC gets into trouble is with allocation of large chunks of data.
> > 
> > If the data 'contains pointers', it creates a very good chance that the data keeps some unknown number of blocks from being deallocated.
> > 
> > And just by the sheer fact of how big the data is, it's bound to be pointed at by some piece of stack data, global data, or TLS data.
> > 
> > So you can go a long way by manually managing larger chunks of data. I think it's worth worrying about as you get into large chunks (on the order of 10MB or more).
> > 
> > What really really sucks about this is, as you allocate larger chunks of data, the chances that the GC incorrectly keeps garbage memory get larger. In other words, the more memory you allocate, the less likely it is that the GC will give you some of it back to you!
> 
> Using 64-bit helps with this.

Yeah, IIRC somebody did some tests comparing false pointer incidence in 32-bit vs. 64-bit environments, and found that false pointer incidence is much lower in 64-bit.

But of course, a large-enough chunk of memory will still have a non-zero chance of a false pointer into it, which is bad because it will just keep sticking around.


> > Precise GC will help this *tremendously*, but I'm unsure if we'll ever get to the point of having a fully-precise GC.
> 
> Any improvement to the GC is a welcome improvement.

Didn't Walter say that we're pushing toward precise scanning?

Of course, the fact that we interface directly with C (and have union types) means that there will always be _some_ place that can't be fully precise.

But having more precise info, such as which references are slices and which are free pointers, would help a lot. For example, if the 10MB chunk is referenced by a slice of 10 bytes, the GC could in theory release everything except those 10 bytes. You're still out of luck with free pointers, but in regular D code those should be quite rare.


> > There will always be a need to
> > do manual freeing of memory.
> 
> Yes, but my point is that it's something you do when the need arises due to performance concerns in a particular section of code. It's not something that you do as a matter of course in your code in general. If you're always trying to free memory as soon as you don't need it anymore rather than letting the GC do its job (or try to anyway), then you might as well be using malloc and free.
[...]

It would be nice if there was an easy way of switching allocators, so that you can, for example, indicate that all allocations done inside a particular function should use malloc/free (including, say, AA's, arrays, etc.), and have the runtime automatically switch back to the GC on scope exit.


T

-- 
Laissez-faire is a French term commonly interpreted by Conservatives to mean 'lazy fairy,' which is the belief that if governments are lazy enough, the Good Fairy will come down from heaven and do all their work for them.
May 16, 2012
On Wednesday, May 16, 2012 12:05:02 H. S. Teoh wrote:
> It would be nice if there was an easy way of switching allocators, so that you can, for example, indicate that all allocations done inside a particular function should use malloc/free (including, say, AA's, arrays, etc.), and have the runtime automatically switch back to the GC on scope exit.

Once we have custom allocators, that should be easy enough as long as you passing the allocator around. But new will always use the GC, and malloc will always do what it does in C (it _is_ the C function after all), so anything you want in terms of custom allocation is going to have to go through custom allocator objects. However, since the custom allocators are going to be runtime entities (rather than having classes templated on their allocators like the STL does), it would be easy enough to choose the appropriate allocator at runtime and have each function use a different allocator if it so chooses.

- Jonathan M Davis
1 2
Next ›   Last »