Jump to page: 1 26  
Page
Thread overview
Discussion on Go and D
Apr 06, 2012
deadalnix
Apr 06, 2012
Rainer Schuetze
Apr 06, 2012
Walter Bright
Apr 06, 2012
deadalnix
Apr 06, 2012
Rainer Schuetze
Apr 06, 2012
Manu
Apr 07, 2012
Rainer Schuetze
Apr 07, 2012
Dmitry Olshansky
Apr 07, 2012
Rainer Schuetze
Apr 07, 2012
Dmitry Olshansky
Apr 07, 2012
Manu
Apr 07, 2012
Dmitry Olshansky
Apr 07, 2012
Sean Kelly
Apr 07, 2012
Walter Bright
Apr 07, 2012
Timon Gehr
Apr 08, 2012
Rainer Schuetze
Apr 07, 2012
Chad J
Apr 07, 2012
Chad J
Apr 08, 2012
Andrej Mitrovic
Apr 08, 2012
H. S. Teoh
Apr 07, 2012
Jacob Carlborg
Apr 07, 2012
Manu
Apr 07, 2012
Jacob Carlborg
Apr 07, 2012
Rainer Schuetze
Apr 07, 2012
Sean Kelly
Apr 07, 2012
Rainer Schuetze
Apr 08, 2012
Sean Kelly
Apr 06, 2012
bearophile
Apr 07, 2012
Sean Kelly
Apr 07, 2012
bearophile
Apr 08, 2012
Walter Bright
Apr 08, 2012
Manu
Apr 08, 2012
Walter Bright
Apr 08, 2012
bearophile
Apr 09, 2012
Manu
Apr 09, 2012
Jacob Carlborg
Apr 09, 2012
deadalnix
Apr 09, 2012
SomeDude
Apr 09, 2012
Andrej Mitrovic
Apr 09, 2012
Manu
Apr 09, 2012
Marco Leise
Apr 09, 2012
Jacob Carlborg
Apr 09, 2012
Manu
Apr 09, 2012
Andrej Mitrovic
Apr 09, 2012
Manu
Apr 09, 2012
Artur Skawina
April 06, 2012
A few more samples of people's perception of the two languages:

http://news.ycombinator.com/item?id=3805302


Andrei
April 06, 2012
Le 06/04/2012 18:07, Andrei Alexandrescu a écrit :
> A few more samples of people's perception of the two languages:
>
> http://news.ycombinator.com/item?id=3805302
>
>
> Andrei

I did some measurement on that point for D lately : http://www.deadalnix.me/2012/03/05/impact-of-64bits-vs-32bits-when-using-non-precise-gc/
April 06, 2012
On 4/6/12 11:20 AM, deadalnix wrote:
> I did some measurement on that point for D lately :
> http://www.deadalnix.me/2012/03/05/impact-of-64bits-vs-32bits-when-using-non-precise-gc/

Sweet, will take a look when I'll get better. Found a typo when skimming: Mesure.

Andrei
April 06, 2012
On 4/6/12 11:20 AM, deadalnix wrote:
> I did some measurement on that point for D lately :
> http://www.deadalnix.me/2012/03/05/impact-of-64bits-vs-32bits-when-using-non-precise-gc/

Also "Lets". You may want to just pass the text through a spellchecker, the impact of typos on perception is higher than most authors think.

Andrei


April 06, 2012
On 4/6/2012 6:20 PM, deadalnix wrote:
> Le 06/04/2012 18:07, Andrei Alexandrescu a écrit :
>> A few more samples of people's perception of the two languages:
>>
>> http://news.ycombinator.com/item?id=3805302
>>
>>
>> Andrei
>
> I did some measurement on that point for D lately :
> http://www.deadalnix.me/2012/03/05/impact-of-64bits-vs-32bits-when-using-non-precise-gc/
>

GC issues like this are currently blocking development of Visual D (a Win32 project): when just adding spaces to a file, parsing the new file every other second often needs 10 or more parsings until an equal amount of memory is collected compared to the allocated memory. AFAICT Visual D just keeps a reference to the root of the most recent AST of a source file.

What's even worse: when the allocated memory gets larger (say > 200MB), the garbage collection itself takes more than a second stalling the application, which is a real pain if it happens while you are typing source text (it does happen quite often).

I've benchmarked testgc3.d from the dmd test suite a little: it allocates about 200 MB of memory, never collects anything, still the latest garbage collections takes about a second on an i7 boosted to 3GHz.

Compiling testgc3.d for x64 with GDC, it allocates twice as much memory, still the garbage collection takes about a second.

Most of the time is spent in the mark phase of the collection, which could probably be optimized to some extend, but that would have to be a very good optimization to not running into problems with a little more allocated memory.

So my current feeling is that conservative garbage collection with stop-the-world mechanics is unusable in an interactive application allocating a decent amount of memory (which is rather small in today's computers). False pointers add to the problem, but they are not the worst issue.

I hope there is something wrong with my reasoning, and that you could give me some hints to avoid the memory bloat and the application stalls.

Rainer

April 06, 2012
On 4/6/2012 10:37 AM, Rainer Schuetze wrote:
> I hope there is something wrong with my reasoning, and that you could give me
> some hints to avoid the memory bloat and the application stalls.

A couple of things you can try (they are workarounds, not solutions):

1. Actively delete memory you no longer need, rather than relying on the gc to catch it. Yes, this is as unsafe as using C's free().

2. Null out pointers & references when you are done with them. This helps reduce unwanted pinning of unused gc memory.

3. Minimize use of global memory, as that is a major source of source of roots.
April 06, 2012
Le 06/04/2012 20:01, Walter Bright a écrit :
> On 4/6/2012 10:37 AM, Rainer Schuetze wrote:
>> I hope there is something wrong with my reasoning, and that you could
>> give me
>> some hints to avoid the memory bloat and the application stalls.
>
> A couple of things you can try (they are workarounds, not solutions):
>
> 1. Actively delete memory you no longer need, rather than relying on the
> gc to catch it. Yes, this is as unsafe as using C's free().
>
> 2. Null out pointers & references when you are done with them. This
> helps reduce unwanted pinning of unused gc memory.
>
> 3. Minimize use of global memory, as that is a major source of source of
> roots.

Additionally, recycle objects (this is a common practice in java when performance matters).
April 06, 2012
Andrei Alexandrescu:

> A few more samples of people's perception of the two languages:
> 
> http://news.ycombinator.com/item?id=3805302

Some of the comments in that ycombinator thread are a bit unnerving.

There is a similar thread on Reddit too: http://www.reddit.com/r/programming/comments/rvwj0/go_severe_memory_problems_on_32bit_systems/

Two quotes from the Reddit page:

>The Boehm GC attempts to mitigate this by detecting false references to free blocks and blacklisting before they become references to live blocks, restricting their use to low-impact situations.
Also Boehm supports typed allocation(gc_typed.h/GC_malloc_explicitly_typed) where you tell GC that pointers are located only at specific offsets and everything other should be ignored.<

>tag data as no-pointers and allocate in separate section. The garbage collector can avoid scanning this section, with reduces collection time as well as the number of false positives.<

Bye,
bearophile
April 06, 2012

On 4/6/2012 8:01 PM, Walter Bright wrote:
> On 4/6/2012 10:37 AM, Rainer Schuetze wrote:
>> I hope there is something wrong with my reasoning, and that you could
>> give me
>> some hints to avoid the memory bloat and the application stalls.
>
> A couple of things you can try (they are workarounds, not solutions):
>
> 1. Actively delete memory you no longer need, rather than relying on the
> gc to catch it. Yes, this is as unsafe as using C's free().

Actually, having to deal with lifetime issues myself takes away the biggest plus of the GC, so I am a bit reluctant to do this.

>
> 2. Null out pointers & references when you are done with them. This
> helps reduce unwanted pinning of unused gc memory.
>
> 3. Minimize use of global memory, as that is a major source of source of
> roots.

I don't think there are many places in the code where these hints might apply. Are there known ways of hunting down false references?

Still, my main concern are the slow collections that stall the application when a decent amount of memory is used. Removing false pointers won't change that, just make it happen a little later.
April 06, 2012
On 7 April 2012 01:08, Rainer Schuetze <r.sagitario@gmx.de> wrote:

>
>
> On 4/6/2012 8:01 PM, Walter Bright wrote:
>
>> On 4/6/2012 10:37 AM, Rainer Schuetze wrote:
>>
>>> I hope there is something wrong with my reasoning, and that you could
>>> give me
>>> some hints to avoid the memory bloat and the application stalls.
>>>
>>
>> A couple of things you can try (they are workarounds, not solutions):
>>
>> 1. Actively delete memory you no longer need, rather than relying on the gc to catch it. Yes, this is as unsafe as using C's free().
>>
>
> Actually, having to deal with lifetime issues myself takes away the biggest plus of the GC, so I am a bit reluctant to do this.
>
>
>
>> 2. Null out pointers & references when you are done with them. This helps reduce unwanted pinning of unused gc memory.
>>
>> 3. Minimize use of global memory, as that is a major source of source of roots.
>>
>
> I don't think there are many places in the code where these hints might apply. Are there known ways of hunting down false references?
>
> Still, my main concern are the slow collections that stall the application when a decent amount of memory is used. Removing false pointers won't change that, just make it happen a little later.
>

An obvious best-practise is to allocate in fewer-larger blocks. Ie, more
compounds and aggregates where possible.
I also expect you are doing a lot of string processing. Using D strings
directly? I presume you have a stack-string class? Put as many working
strings on the stack as possible...


« First   ‹ Prev
1 2 3 4 5 6