View mode: basic / threaded / horizontal-split · Log in · Help
May 15, 2012
Re: allocate array with new
And what if i use an array of structs? Also clear instead of
delete?
May 15, 2012
Re: allocate array with new
On 5/15/12, Jonathan M Davis <jmdavisProg@gmx.com> wrote:
> If you're going to use anything, use clear.

He might as well just use this if he wants the GC to take care of things:
arr = null;

clear(arr) is a little bit of a misnomer. If you have another slice
pointing to the same array clear() won't really release memory at all,
it will just make the first slice null. so using "arr = null" is good
enough.

Also good to note is that clear/null assign is very different from
what delete does. delete won't care that you have other slices
pointing to the same array and you'll end up with slices that point to
garbage after a call to delete.
May 16, 2012
Re: allocate array with new
On Tue, 15 May 2012 05:43:45 -0400, Jonathan M Davis <jmdavisProg@gmx.com>  
wrote:

> On Tuesday, May 15, 2012 11:26:49 Namespace wrote:
>> On Tuesday, 15 May 2012 at 09:23:51 UTC, Kagamin wrote:
>> > Difference with what?
>> > new is a safe feature: it allocates in the GC heap
>>
>> That's what i mean. So i have to delete it yourself with "delete
>> arr;", or not?
>
> No. _Never_ use delete. It's going to be deprecated. The GC worries about
> freeing memory allocated on the GC heap, and new always allocates on the  
> GC
> heap. If you don't want to allocate on the GC heap, then use malloc and  
> free,
> in which case you _do_ need worry about freeing the memory.
>
> If you need to force destruction before the GC collects an object, you  
> can
> call clear on that object to have its destructor called and its vtbl  
> zeroed
> out, but it's memory still isn't freed. That's the GC's job.
>
> If you really have to, you can use core.memory to manipulate the GC heap
> (including calling GC.free), but you really shouldn't be messing with  
> any of
> that unless you really need to and you know what you're doing.

Well, the D GC is conservative, and will always be at least partly  
conservative.  So freeing memory manually is not really a crime.

I agree you should avoid delete, due to its possible deprecation.

-Steve
May 16, 2012
Re: allocate array with new
On Wednesday, May 16, 2012 11:51:59 Steven Schveighoffer wrote:
> Well, the D GC is conservative, and will always be at least partly
> conservative. So freeing memory manually is not really a crime.

The idea with a GC is that it's supposed to manage the memory allocated 
through it. So, if you're worrying about freeing GC memory, you're fighting the 
GC and how it works. And unless you're keeping very close track of the number 
of reference to a particular chunk of memory, freeing it manually is 
dangerous. So, anyone looking to use delete is generally going about things 
the wrong way.

However, it _is_ true that sometimes you need to intervene in order to get 
sections of code as performant as they need to be (particularly since D's 
current GC is not the most performant), which is part of the reason that 
core.memory provides what it does. But it should generally be something used 
when it's clear that you need that extra performance, not as a matter of 
course, because using it is inherently unsafe. If there's something that you 
know ahead of time needs to have more deterministic deallocation, then it's 
probably better to be use malloc and free with ref-counting than trying to 
manually manage GC memory.

- Jonathan M Davis
May 16, 2012
Re: allocate array with new
On Wed, 16 May 2012 12:52:33 -0400, Jonathan M Davis <jmdavisProg@gmx.com>  
wrote:

> On Wednesday, May 16, 2012 11:51:59 Steven Schveighoffer wrote:
>> Well, the D GC is conservative, and will always be at least partly
>> conservative. So freeing memory manually is not really a crime.
>
> The idea with a GC is that it's supposed to manage the memory allocated
> through it. So, if you're worrying about freeing GC memory, you're  
> fighting the
> GC and how it works. And unless you're keeping very close track of the  
> number
> of reference to a particular chunk of memory, freeing it manually is
> dangerous. So, anyone looking to use delete is generally going about  
> things
> the wrong way.
>
> However, it _is_ true that sometimes you need to intervene in order to  
> get
> sections of code as performant as they need to be (particularly since D's
> current GC is not the most performant), which is part of the reason that
> core.memory provides what it does. But it should generally be something  
> used
> when it's clear that you need that extra performance, not as a matter of
> course, because using it is inherently unsafe. If there's something that  
> you
> know ahead of time needs to have more deterministic deallocation, then  
> it's
> probably better to be use malloc and free with ref-counting than trying  
> to
> manually manage GC memory.

Where the GC gets into trouble is with allocation of large chunks of data.

If the data 'contains pointers', it creates a very good chance that the  
data keeps some unknown number of blocks from being deallocated.

And just by the sheer fact of how big the data is, it's bound to be  
pointed at by some piece of stack data, global data, or TLS data.

So you can go a long way by manually managing larger chunks of data.  I  
think it's worth worrying about as you get into large chunks (on the order  
of 10MB or more).

What really really sucks about this is, as you allocate larger chunks of  
data, the chances that the GC incorrectly keeps garbage memory get  
larger.  In other words, the more memory you allocate, the less likely it  
is that the GC will give you some of it back to you!

Precise GC will help this *tremendously*, but I'm unsure if we'll ever get  
to the point of having a fully-precise GC.  There will always be a need to  
do manual freeing of memory.

-Steve
May 16, 2012
Re: allocate array with new
On Wednesday, May 16, 2012 13:52:12 Steven Schveighoffer wrote:
> Where the GC gets into trouble is with allocation of large chunks of data.
> 
> If the data 'contains pointers', it creates a very good chance that the
> data keeps some unknown number of blocks from being deallocated.
> 
> And just by the sheer fact of how big the data is, it's bound to be
> pointed at by some piece of stack data, global data, or TLS data.
> 
> So you can go a long way by manually managing larger chunks of data. I
> think it's worth worrying about as you get into large chunks (on the order
> of 10MB or more).
> 
> What really really sucks about this is, as you allocate larger chunks of
> data, the chances that the GC incorrectly keeps garbage memory get
> larger. In other words, the more memory you allocate, the less likely it
> is that the GC will give you some of it back to you!

Using 64-bit helps with this.

> Precise GC will help this *tremendously*, but I'm unsure if we'll ever get
> to the point of having a fully-precise GC.

Any improvement to the GC is a welcome improvement.

> There will always be a need to
> do manual freeing of memory.

Yes, but my point is that it's something you do when the need arises due to 
performance concerns in a particular section of code. It's not something that 
you do as a matter of course in your code in general. If you're always trying 
to free memory as soon as you don't need it anymore rather than letting the GC 
do its job (or try to anyway), then you might as well be using malloc and 
free.

- Jonathan M Davis
May 16, 2012
Re: allocate array with new
On Wed, May 16, 2012 at 02:20:47PM -0400, Jonathan M Davis wrote:
> On Wednesday, May 16, 2012 13:52:12 Steven Schveighoffer wrote:
> > Where the GC gets into trouble is with allocation of large chunks of
> > data.
> > 
> > If the data 'contains pointers', it creates a very good chance that
> > the data keeps some unknown number of blocks from being deallocated.
> > 
> > And just by the sheer fact of how big the data is, it's bound to be
> > pointed at by some piece of stack data, global data, or TLS data.
> > 
> > So you can go a long way by manually managing larger chunks of data.
> > I think it's worth worrying about as you get into large chunks (on
> > the order of 10MB or more).
> > 
> > What really really sucks about this is, as you allocate larger
> > chunks of data, the chances that the GC incorrectly keeps garbage
> > memory get larger. In other words, the more memory you allocate, the
> > less likely it is that the GC will give you some of it back to you!
> 
> Using 64-bit helps with this.

Yeah, IIRC somebody did some tests comparing false pointer incidence in
32-bit vs. 64-bit environments, and found that false pointer incidence
is much lower in 64-bit.

But of course, a large-enough chunk of memory will still have a non-zero
chance of a false pointer into it, which is bad because it will just
keep sticking around.


> > Precise GC will help this *tremendously*, but I'm unsure if we'll
> > ever get to the point of having a fully-precise GC.
> 
> Any improvement to the GC is a welcome improvement.

Didn't Walter say that we're pushing toward precise scanning?

Of course, the fact that we interface directly with C (and have union
types) means that there will always be _some_ place that can't be fully
precise.

But having more precise info, such as which references are slices and
which are free pointers, would help a lot. For example, if the 10MB
chunk is referenced by a slice of 10 bytes, the GC could in theory
release everything except those 10 bytes. You're still out of luck with
free pointers, but in regular D code those should be quite rare.


> > There will always be a need to
> > do manual freeing of memory.
> 
> Yes, but my point is that it's something you do when the need arises
> due to performance concerns in a particular section of code. It's not
> something that you do as a matter of course in your code in general.
> If you're always trying to free memory as soon as you don't need it
> anymore rather than letting the GC do its job (or try to anyway), then
> you might as well be using malloc and free.
[...]

It would be nice if there was an easy way of switching allocators, so
that you can, for example, indicate that all allocations done inside a
particular function should use malloc/free (including, say, AA's,
arrays, etc.), and have the runtime automatically switch back to the GC
on scope exit.


T

-- 
Laissez-faire is a French term commonly interpreted by Conservatives to mean 'lazy fairy,' which is the belief that if governments are lazy enough, the Good Fairy will come down from heaven and do all their work for them.
May 16, 2012
Re: allocate array with new
On Wednesday, May 16, 2012 12:05:02 H. S. Teoh wrote:
> It would be nice if there was an easy way of switching allocators, so
> that you can, for example, indicate that all allocations done inside a
> particular function should use malloc/free (including, say, AA's,
> arrays, etc.), and have the runtime automatically switch back to the GC
> on scope exit.

Once we have custom allocators, that should be easy enough as long as you 
passing the allocator around. But new will always use the GC, and malloc will 
always do what it does in C (it _is_ the C function after all), so anything 
you want in terms of custom allocation is going to have to go through custom 
allocator objects. However, since the custom allocators are going to be 
runtime entities (rather than having classes templated on their allocators 
like the STL does), it would be easy enough to choose the appropriate 
allocator at runtime and have each function use a different allocator if it so 
chooses.

- Jonathan M Davis
Next ›   Last »
1 2
Top | Discussion index | About this forum | D home