April 06, 2012 Discussion on Go and D | ||||
---|---|---|---|---|
| ||||
A few more samples of people's perception of the two languages: http://news.ycombinator.com/item?id=3805302 Andrei |
April 06, 2012 Re: Discussion on Go and D | ||||
---|---|---|---|---|
| ||||
Posted in reply to Andrei Alexandrescu | Le 06/04/2012 18:07, Andrei Alexandrescu a écrit : > A few more samples of people's perception of the two languages: > > http://news.ycombinator.com/item?id=3805302 > > > Andrei I did some measurement on that point for D lately : http://www.deadalnix.me/2012/03/05/impact-of-64bits-vs-32bits-when-using-non-precise-gc/ |
April 06, 2012 Re: Discussion on Go and D | ||||
---|---|---|---|---|
| ||||
Posted in reply to deadalnix | On 4/6/12 11:20 AM, deadalnix wrote:
> I did some measurement on that point for D lately :
> http://www.deadalnix.me/2012/03/05/impact-of-64bits-vs-32bits-when-using-non-precise-gc/
Sweet, will take a look when I'll get better. Found a typo when skimming: Mesure.
Andrei
|
April 06, 2012 Re: Discussion on Go and D | ||||
---|---|---|---|---|
| ||||
Posted in reply to deadalnix | On 4/6/12 11:20 AM, deadalnix wrote:
> I did some measurement on that point for D lately :
> http://www.deadalnix.me/2012/03/05/impact-of-64bits-vs-32bits-when-using-non-precise-gc/
Also "Lets". You may want to just pass the text through a spellchecker, the impact of typos on perception is higher than most authors think.
Andrei
|
April 06, 2012 Re: Discussion on Go and D | ||||
---|---|---|---|---|
| ||||
Posted in reply to deadalnix |
On 4/6/2012 6:20 PM, deadalnix wrote:
> Le 06/04/2012 18:07, Andrei Alexandrescu a écrit :
>> A few more samples of people's perception of the two languages:
>>
>> http://news.ycombinator.com/item?id=3805302
>>
>>
>> Andrei
>
> I did some measurement on that point for D lately :
> http://www.deadalnix.me/2012/03/05/impact-of-64bits-vs-32bits-when-using-non-precise-gc/
>
GC issues like this are currently blocking development of Visual D (a Win32 project): when just adding spaces to a file, parsing the new file every other second often needs 10 or more parsings until an equal amount of memory is collected compared to the allocated memory. AFAICT Visual D just keeps a reference to the root of the most recent AST of a source file.
What's even worse: when the allocated memory gets larger (say > 200MB), the garbage collection itself takes more than a second stalling the application, which is a real pain if it happens while you are typing source text (it does happen quite often).
I've benchmarked testgc3.d from the dmd test suite a little: it allocates about 200 MB of memory, never collects anything, still the latest garbage collections takes about a second on an i7 boosted to 3GHz.
Compiling testgc3.d for x64 with GDC, it allocates twice as much memory, still the garbage collection takes about a second.
Most of the time is spent in the mark phase of the collection, which could probably be optimized to some extend, but that would have to be a very good optimization to not running into problems with a little more allocated memory.
So my current feeling is that conservative garbage collection with stop-the-world mechanics is unusable in an interactive application allocating a decent amount of memory (which is rather small in today's computers). False pointers add to the problem, but they are not the worst issue.
I hope there is something wrong with my reasoning, and that you could give me some hints to avoid the memory bloat and the application stalls.
Rainer
|
April 06, 2012 Re: Discussion on Go and D | ||||
---|---|---|---|---|
| ||||
Posted in reply to Rainer Schuetze | On 4/6/2012 10:37 AM, Rainer Schuetze wrote:
> I hope there is something wrong with my reasoning, and that you could give me
> some hints to avoid the memory bloat and the application stalls.
A couple of things you can try (they are workarounds, not solutions):
1. Actively delete memory you no longer need, rather than relying on the gc to catch it. Yes, this is as unsafe as using C's free().
2. Null out pointers & references when you are done with them. This helps reduce unwanted pinning of unused gc memory.
3. Minimize use of global memory, as that is a major source of source of roots.
|
April 06, 2012 Re: Discussion on Go and D | ||||
---|---|---|---|---|
| ||||
Posted in reply to Walter Bright | Le 06/04/2012 20:01, Walter Bright a écrit :
> On 4/6/2012 10:37 AM, Rainer Schuetze wrote:
>> I hope there is something wrong with my reasoning, and that you could
>> give me
>> some hints to avoid the memory bloat and the application stalls.
>
> A couple of things you can try (they are workarounds, not solutions):
>
> 1. Actively delete memory you no longer need, rather than relying on the
> gc to catch it. Yes, this is as unsafe as using C's free().
>
> 2. Null out pointers & references when you are done with them. This
> helps reduce unwanted pinning of unused gc memory.
>
> 3. Minimize use of global memory, as that is a major source of source of
> roots.
Additionally, recycle objects (this is a common practice in java when performance matters).
|
April 06, 2012 Re: Discussion on Go and D | ||||
---|---|---|---|---|
| ||||
Posted in reply to Andrei Alexandrescu | Andrei Alexandrescu: > A few more samples of people's perception of the two languages: > > http://news.ycombinator.com/item?id=3805302 Some of the comments in that ycombinator thread are a bit unnerving. There is a similar thread on Reddit too: http://www.reddit.com/r/programming/comments/rvwj0/go_severe_memory_problems_on_32bit_systems/ Two quotes from the Reddit page: >The Boehm GC attempts to mitigate this by detecting false references to free blocks and blacklisting before they become references to live blocks, restricting their use to low-impact situations. Also Boehm supports typed allocation(gc_typed.h/GC_malloc_explicitly_typed) where you tell GC that pointers are located only at specific offsets and everything other should be ignored.< >tag data as no-pointers and allocate in separate section. The garbage collector can avoid scanning this section, with reduces collection time as well as the number of false positives.< Bye, bearophile |
April 06, 2012 Re: Discussion on Go and D | ||||
---|---|---|---|---|
| ||||
Posted in reply to Walter Bright | On 4/6/2012 8:01 PM, Walter Bright wrote: > On 4/6/2012 10:37 AM, Rainer Schuetze wrote: >> I hope there is something wrong with my reasoning, and that you could >> give me >> some hints to avoid the memory bloat and the application stalls. > > A couple of things you can try (they are workarounds, not solutions): > > 1. Actively delete memory you no longer need, rather than relying on the > gc to catch it. Yes, this is as unsafe as using C's free(). Actually, having to deal with lifetime issues myself takes away the biggest plus of the GC, so I am a bit reluctant to do this. > > 2. Null out pointers & references when you are done with them. This > helps reduce unwanted pinning of unused gc memory. > > 3. Minimize use of global memory, as that is a major source of source of > roots. I don't think there are many places in the code where these hints might apply. Are there known ways of hunting down false references? Still, my main concern are the slow collections that stall the application when a decent amount of memory is used. Removing false pointers won't change that, just make it happen a little later. |
April 06, 2012 Re: Discussion on Go and D | ||||
---|---|---|---|---|
| ||||
Posted in reply to Rainer Schuetze Attachments:
| On 7 April 2012 01:08, Rainer Schuetze <r.sagitario@gmx.de> wrote:
>
>
> On 4/6/2012 8:01 PM, Walter Bright wrote:
>
>> On 4/6/2012 10:37 AM, Rainer Schuetze wrote:
>>
>>> I hope there is something wrong with my reasoning, and that you could
>>> give me
>>> some hints to avoid the memory bloat and the application stalls.
>>>
>>
>> A couple of things you can try (they are workarounds, not solutions):
>>
>> 1. Actively delete memory you no longer need, rather than relying on the gc to catch it. Yes, this is as unsafe as using C's free().
>>
>
> Actually, having to deal with lifetime issues myself takes away the biggest plus of the GC, so I am a bit reluctant to do this.
>
>
>
>> 2. Null out pointers & references when you are done with them. This helps reduce unwanted pinning of unused gc memory.
>>
>> 3. Minimize use of global memory, as that is a major source of source of roots.
>>
>
> I don't think there are many places in the code where these hints might apply. Are there known ways of hunting down false references?
>
> Still, my main concern are the slow collections that stall the application when a decent amount of memory is used. Removing false pointers won't change that, just make it happen a little later.
>
An obvious best-practise is to allocate in fewer-larger blocks. Ie, more
compounds and aggregates where possible.
I also expect you are doing a lot of string processing. Using D strings
directly? I presume you have a stack-string class? Put as many working
strings on the stack as possible...
|
Copyright © 1999-2021 by the D Language Foundation