February 26, 2012
On 26-02-2012 08:48, Paulo Pinto wrote:
> Am 26.02.2012 00:45, schrieb Andrew Wiley:
>> On Sat, Feb 25, 2012 at 5:01 PM, Paulo Pinto<pjmlp@progtools.org> wrote:
>>> Am 25.02.2012 23:40, schrieb Andrew Wiley:
>>>>
>>>> On Sat, Feb 25, 2012 at 4:29 PM, Paulo Pinto<pjmlp@progtools.org>
>>>> wrote:
>>>>>
>>>>> Am 25.02.2012 23:17, schrieb Peter Alexander:
>>>>>
>>>>>
>>>>>> On Saturday, 25 February 2012 at 22:08:31 UTC, Paulo Pinto wrote:
>>>>>>>
>>>>>>>
>>>>>>> Am 25.02.2012 21:26, schrieb Peter Alexander:
>>>>>>>>
>>>>>>>>
>>>>>>>> On Saturday, 25 February 2012 at 20:13:42 UTC, so wrote:
>>>>>>>>>
>>>>>>>>>
>>>>>>>>> On Saturday, 25 February 2012 at 18:47:12 UTC, Nick Sabalausky
>>>>>>>>> wrote:
>>>>>>>>>
>>>>>>>>>> Interesting. I wish he'd elaborate on why it's not an option
>>>>>>>>>> for his
>>>>>>>>>> daily
>>>>>>>>>> work.
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>> Not the design but the implementation, memory management would
>>>>>>>>> be the
>>>>>>>>> first.
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>> Memory management is not a problem. You can manage memory just as
>>>>>>>> easily
>>>>>>>> in D as you can in C or C++. Just don't use global new, which
>>>>>>>> they'll
>>>>>>>> already be doing.
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>> I couldn't agree more.
>>>>>>>
>>>>>>> The GC issue comes around often, but I personally think that the
>>>>>>> main
>>>>>>> issue is that the GC needs to be optimized, not that manual memory
>>>>>>> management is required.
>>>>>>>
>>>>>>> Most standard compiler malloc()/free() implementations are actually
>>>>>>> slower than most advanced GC algorithms.
>>>>>>
>>>>>>
>>>>>>
>>>>>> If you require realtime performance then you don't use either the
>>>>>> GC or
>>>>>> malloc/free. You allocate blocks up front and use those when you need
>>>>>> consistent high performance.
>>>>>>
>>>>>> It doesn't matter how optimised the GC is. The eventual collection is
>>>>>> inevitable and if it takes anything more than a small fraction of a
>>>>>> second then it will be too slow for realtime use.
>>>>>>
>>>>>
>>>>> There are GC realtime algorithms, which are actually in use, in
>>>>> systems
>>>>> like the French Ground Master 400 missile radar system.
>>>>>
>>>>> There is no more realtime than that. I surely would not like that such
>>>>> systems had a pause the world GC.
>>>>>
>>>>
>>>> Can you give any description of how that is done (or any relevant
>>>> papers), and how it can be made to function reasonably on low end
>>>> consumer hardware and standard operating systems? Without that, your
>>>> example is irrelevant.
>>>> Azul has already shown that realtime non-pause GC is certainly
>>>> possible, but only with massive servers, lots of CPUs, and large
>>>> kernel modifications.
>>>>
>>>> And, as far as I'm aware, that still didn't solve the generally
>>>> memory-hungry behaviors of the JVM.
>>>
>>>
>>> Sure.
>>>
>>> http://www.militaryaerospace.com/articles/2009/03/thales-chooses-aonix-perc-virtual-machine-software-for-ballistic-missile-radar.html
>>>
>>>
>>> http://www.atego.com/products/aonix-perc-raven/
>>
>> Neither of those links have any information on how this actually
>> works. In fact, the docs on Atego's site pretty much state that their
>> JVM is highly specialized and requires programmers to follow very
>> different rules from typical Java, which makes this technology look
>> less and less viable for general usage.
>>
>> I don't see how this example is relevant for D. I can't find any
>> details on the system you're mentioning, but assuming they developed
>> something similar to Azul, the fundamental problem is that D has to
>> target platforms in general use, not highly specialized server
>> environments with modified kernels and highly parallel hardware. Until
>> such environments come into general use (assuming they do at all; Azul
>> seems to be having trouble getting their virtual memory manipulation
>> techniques merged into the Linux kernel), D can't make use of them,
>> and we're right back to saying that GCs have unacceptably long pause
>> times for realtime applications.
>
> In Java's case they are following the Java's specification for real time
> applications.
>
> http://java.sun.com/javase/technologies/realtime/index.jsp
>
> I did not mention any specific algorithm, because like most companies, I
> am sure Atego patents most of it.
>
> Still a quick search in Google reveals a few papers:
>
> http://research.microsoft.com/apps/video/dl.aspx?id=103698&amp;l=i
>
> http://www.cs.cmu.edu/~spoons/gc/vee05.pdf
>
> http://domino.research.ibm.com/comm/research_people.nsf/pages/bacon.presentations.html/$FILE/Bacon05BravelyTalk.ppt
>
>
> http://www.cs.technion.ac.il/~erez/Papers/real-time-pldi.pdf
>
> http://www.cs.purdue.edu/homes/lziarek/pldi10.pdf
>
> I know GC use is a bit of a religious debate but C++ was the very last
> systems programming language without automatic memory management. And
> even C++ has got some form in C++11.
>
> At least in the desktop area, in a decade from now, most likely system
> programming in desktop OS will either make use of reference counting
> (WinRT or ARC), or it will use a GC (similar to Spin, Inferno,
> Singularity, Oberon).
>
> This is how I see the trend going, but hey, I am just a simple person
> and I get to be wrong lots of time.
>
> --
> Paulo

Well, there is Clay, which doesn't use a GC.

-- 
- Alex
February 26, 2012
On 2012-02-25 23:36, Andrew Wiley wrote:
> On Sat, Feb 25, 2012 at 4:08 PM, Paulo Pinto<pjmlp@progtools.org>  wrote:
>> Am 25.02.2012 21:26, schrieb Peter Alexander:
>>
>>> On Saturday, 25 February 2012 at 20:13:42 UTC, so wrote:
>>>>
>>>> On Saturday, 25 February 2012 at 18:47:12 UTC, Nick Sabalausky wrote:
>>>>
>>>>> Interesting. I wish he'd elaborate on why it's not an option for his
>>>>> daily
>>>>> work.
>>>>
>>>>
>>>> Not the design but the implementation, memory management would be the
>>>> first.
>>>
>>>
>>> Memory management is not a problem. You can manage memory just as easily
>>> in D as you can in C or C++. Just don't use global new, which they'll
>>> already be doing.
>>
>>
>> I couldn't agree more.
>>
>> The GC issue comes around often, but I personally think that the main
>> issue is that the GC needs to be optimized, not that manual memory
>> management is required.
>>
>> Most standard compiler malloc()/free() implementations are actually slower
>> than most advanced GC algorithms.
>
> That's not the issue here. The issue is that when your game is
> required to render at 60fps, you've got 16.67ms for each frame and no
> time for 100ms+ GC cycle. In this environment, it's mostly irrelevant
> that you'll spend more time total in malloc than you would have spent
> in the GC because you can only spare the time in small chunks, not
> large ones.
>
> One simple solution is to avoid all dynamic allocation, but as a few
> mostly unanswered NG posts have shown, the compiler is currently
> implicitly generating dynamic allocations in a few places, and there's
> no simple way to track them down or do anything about them.

You can remove the GC and you'll get a linker error when its used. Not the best way to track them down but it works.

-- 
/Jacob Carlborg
February 26, 2012
Am 26.02.2012 12:44, schrieb Alex Rønne Petersen:
> On 26-02-2012 08:48, Paulo Pinto wrote:
>> Am 26.02.2012 00:45, schrieb Andrew Wiley:
>>> On Sat, Feb 25, 2012 at 5:01 PM, Paulo Pinto<pjmlp@progtools.org> wrote:
>>>> Am 25.02.2012 23:40, schrieb Andrew Wiley:
>>>>>
>>>>> On Sat, Feb 25, 2012 at 4:29 PM, Paulo Pinto<pjmlp@progtools.org>
>>>>> wrote:
>>>>>>
>>>>>> Am 25.02.2012 23:17, schrieb Peter Alexander:
>>>>>>
>>>>>>
>>>>>>> On Saturday, 25 February 2012 at 22:08:31 UTC, Paulo Pinto wrote:
>>>>>>>>
>>>>>>>>
>>>>>>>> Am 25.02.2012 21:26, schrieb Peter Alexander:
>>>>>>>>>
>>>>>>>>>
>>>>>>>>> On Saturday, 25 February 2012 at 20:13:42 UTC, so wrote:
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>> On Saturday, 25 February 2012 at 18:47:12 UTC, Nick Sabalausky
>>>>>>>>>> wrote:
>>>>>>>>>>
>>>>>>>>>>> Interesting. I wish he'd elaborate on why it's not an option
>>>>>>>>>>> for his
>>>>>>>>>>> daily
>>>>>>>>>>> work.
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>> Not the design but the implementation, memory management would
>>>>>>>>>> be the
>>>>>>>>>> first.
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>> Memory management is not a problem. You can manage memory just as
>>>>>>>>> easily
>>>>>>>>> in D as you can in C or C++. Just don't use global new, which
>>>>>>>>> they'll
>>>>>>>>> already be doing.
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>> I couldn't agree more.
>>>>>>>>
>>>>>>>> The GC issue comes around often, but I personally think that the
>>>>>>>> main
>>>>>>>> issue is that the GC needs to be optimized, not that manual memory
>>>>>>>> management is required.
>>>>>>>>
>>>>>>>> Most standard compiler malloc()/free() implementations are actually
>>>>>>>> slower than most advanced GC algorithms.
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>> If you require realtime performance then you don't use either the
>>>>>>> GC or
>>>>>>> malloc/free. You allocate blocks up front and use those when you
>>>>>>> need
>>>>>>> consistent high performance.
>>>>>>>
>>>>>>> It doesn't matter how optimised the GC is. The eventual
>>>>>>> collection is
>>>>>>> inevitable and if it takes anything more than a small fraction of a
>>>>>>> second then it will be too slow for realtime use.
>>>>>>>
>>>>>>
>>>>>> There are GC realtime algorithms, which are actually in use, in
>>>>>> systems
>>>>>> like the French Ground Master 400 missile radar system.
>>>>>>
>>>>>> There is no more realtime than that. I surely would not like that
>>>>>> such
>>>>>> systems had a pause the world GC.
>>>>>>
>>>>>
>>>>> Can you give any description of how that is done (or any relevant
>>>>> papers), and how it can be made to function reasonably on low end
>>>>> consumer hardware and standard operating systems? Without that, your
>>>>> example is irrelevant.
>>>>> Azul has already shown that realtime non-pause GC is certainly
>>>>> possible, but only with massive servers, lots of CPUs, and large
>>>>> kernel modifications.
>>>>>
>>>>> And, as far as I'm aware, that still didn't solve the generally
>>>>> memory-hungry behaviors of the JVM.
>>>>
>>>>
>>>> Sure.
>>>>
>>>> http://www.militaryaerospace.com/articles/2009/03/thales-chooses-aonix-perc-virtual-machine-software-for-ballistic-missile-radar.html
>>>>
>>>>
>>>>
>>>> http://www.atego.com/products/aonix-perc-raven/
>>>
>>> Neither of those links have any information on how this actually
>>> works. In fact, the docs on Atego's site pretty much state that their
>>> JVM is highly specialized and requires programmers to follow very
>>> different rules from typical Java, which makes this technology look
>>> less and less viable for general usage.
>>>
>>> I don't see how this example is relevant for D. I can't find any
>>> details on the system you're mentioning, but assuming they developed
>>> something similar to Azul, the fundamental problem is that D has to
>>> target platforms in general use, not highly specialized server
>>> environments with modified kernels and highly parallel hardware. Until
>>> such environments come into general use (assuming they do at all; Azul
>>> seems to be having trouble getting their virtual memory manipulation
>>> techniques merged into the Linux kernel), D can't make use of them,
>>> and we're right back to saying that GCs have unacceptably long pause
>>> times for realtime applications.
>>
>> In Java's case they are following the Java's specification for real time
>> applications.
>>
>> http://java.sun.com/javase/technologies/realtime/index.jsp
>>
>> I did not mention any specific algorithm, because like most companies, I
>> am sure Atego patents most of it.
>>
>> Still a quick search in Google reveals a few papers:
>>
>> http://research.microsoft.com/apps/video/dl.aspx?id=103698&amp;l=i
>>
>> http://www.cs.cmu.edu/~spoons/gc/vee05.pdf
>>
>> http://domino.research.ibm.com/comm/research_people.nsf/pages/bacon.presentations.html/$FILE/Bacon05BravelyTalk.ppt
>>
>>
>>
>> http://www.cs.technion.ac.il/~erez/Papers/real-time-pldi.pdf
>>
>> http://www.cs.purdue.edu/homes/lziarek/pldi10.pdf
>>
>> I know GC use is a bit of a religious debate but C++ was the very last
>> systems programming language without automatic memory management. And
>> even C++ has got some form in C++11.
>>
>> At least in the desktop area, in a decade from now, most likely system
>> programming in desktop OS will either make use of reference counting
>> (WinRT or ARC), or it will use a GC (similar to Spin, Inferno,
>> Singularity, Oberon).
>>
>> This is how I see the trend going, but hey, I am just a simple person
>> and I get to be wrong lots of time.
>>
>> --
>> Paulo
>
> Well, there is Clay, which doesn't use a GC.
>

I was unware of it, thanks for pointing it out.

It is still at 0.2 and the newsgroup only has 13 messages, lets see how far it goes.

--
Paulo


February 26, 2012
Le 26/02/2012 08:48, Paulo Pinto a écrit :
> Am 26.02.2012 00:45, schrieb Andrew Wiley:
>> On Sat, Feb 25, 2012 at 5:01 PM, Paulo Pinto<pjmlp@progtools.org> wrote:
>>> Am 25.02.2012 23:40, schrieb Andrew Wiley:
>>>>
>>>> On Sat, Feb 25, 2012 at 4:29 PM, Paulo Pinto<pjmlp@progtools.org>
>>>> wrote:
>>>>>
>>>>> Am 25.02.2012 23:17, schrieb Peter Alexander:
>>>>>
>>>>>
>>>>>> On Saturday, 25 February 2012 at 22:08:31 UTC, Paulo Pinto wrote:
>>>>>>>
>>>>>>>
>>>>>>> Am 25.02.2012 21:26, schrieb Peter Alexander:
>>>>>>>>
>>>>>>>>
>>>>>>>> On Saturday, 25 February 2012 at 20:13:42 UTC, so wrote:
>>>>>>>>>
>>>>>>>>>
>>>>>>>>> On Saturday, 25 February 2012 at 18:47:12 UTC, Nick Sabalausky
>>>>>>>>> wrote:
>>>>>>>>>
>>>>>>>>>> Interesting. I wish he'd elaborate on why it's not an option
>>>>>>>>>> for his
>>>>>>>>>> daily
>>>>>>>>>> work.
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>> Not the design but the implementation, memory management would
>>>>>>>>> be the
>>>>>>>>> first.
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>> Memory management is not a problem. You can manage memory just as
>>>>>>>> easily
>>>>>>>> in D as you can in C or C++. Just don't use global new, which
>>>>>>>> they'll
>>>>>>>> already be doing.
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>> I couldn't agree more.
>>>>>>>
>>>>>>> The GC issue comes around often, but I personally think that the
>>>>>>> main
>>>>>>> issue is that the GC needs to be optimized, not that manual memory
>>>>>>> management is required.
>>>>>>>
>>>>>>> Most standard compiler malloc()/free() implementations are actually
>>>>>>> slower than most advanced GC algorithms.
>>>>>>
>>>>>>
>>>>>>
>>>>>> If you require realtime performance then you don't use either the
>>>>>> GC or
>>>>>> malloc/free. You allocate blocks up front and use those when you need
>>>>>> consistent high performance.
>>>>>>
>>>>>> It doesn't matter how optimised the GC is. The eventual collection is
>>>>>> inevitable and if it takes anything more than a small fraction of a
>>>>>> second then it will be too slow for realtime use.
>>>>>>
>>>>>
>>>>> There are GC realtime algorithms, which are actually in use, in
>>>>> systems
>>>>> like the French Ground Master 400 missile radar system.
>>>>>
>>>>> There is no more realtime than that. I surely would not like that such
>>>>> systems had a pause the world GC.
>>>>>
>>>>
>>>> Can you give any description of how that is done (or any relevant
>>>> papers), and how it can be made to function reasonably on low end
>>>> consumer hardware and standard operating systems? Without that, your
>>>> example is irrelevant.
>>>> Azul has already shown that realtime non-pause GC is certainly
>>>> possible, but only with massive servers, lots of CPUs, and large
>>>> kernel modifications.
>>>>
>>>> And, as far as I'm aware, that still didn't solve the generally
>>>> memory-hungry behaviors of the JVM.
>>>
>>>
>>> Sure.
>>>
>>> http://www.militaryaerospace.com/articles/2009/03/thales-chooses-aonix-perc-virtual-machine-software-for-ballistic-missile-radar.html
>>>
>>>
>>> http://www.atego.com/products/aonix-perc-raven/
>>
>> Neither of those links have any information on how this actually
>> works. In fact, the docs on Atego's site pretty much state that their
>> JVM is highly specialized and requires programmers to follow very
>> different rules from typical Java, which makes this technology look
>> less and less viable for general usage.
>>
>> I don't see how this example is relevant for D. I can't find any
>> details on the system you're mentioning, but assuming they developed
>> something similar to Azul, the fundamental problem is that D has to
>> target platforms in general use, not highly specialized server
>> environments with modified kernels and highly parallel hardware. Until
>> such environments come into general use (assuming they do at all; Azul
>> seems to be having trouble getting their virtual memory manipulation
>> techniques merged into the Linux kernel), D can't make use of them,
>> and we're right back to saying that GCs have unacceptably long pause
>> times for realtime applications.
>
> In Java's case they are following the Java's specification for real time
> applications.
>
> http://java.sun.com/javase/technologies/realtime/index.jsp
>
> I did not mention any specific algorithm, because like most companies, I
> am sure Atego patents most of it.
>
> Still a quick search in Google reveals a few papers:
>
> http://research.microsoft.com/apps/video/dl.aspx?id=103698&amp;l=i
>
> http://www.cs.cmu.edu/~spoons/gc/vee05.pdf
>
> http://domino.research.ibm.com/comm/research_people.nsf/pages/bacon.presentations.html/$FILE/Bacon05BravelyTalk.ppt
>
>
> http://www.cs.technion.ac.il/~erez/Papers/real-time-pldi.pdf
>
> http://www.cs.purdue.edu/homes/lziarek/pldi10.pdf
>
> I know GC use is a bit of a religious debate but C++ was the very last
> systems programming language without automatic memory management. And
> even C++ has got some form in C++11.
>
> At least in the desktop area, in a decade from now, most likely system
> programming in desktop OS will either make use of reference counting
> (WinRT or ARC), or it will use a GC (similar to Spin, Inferno,
> Singularity, Oberon).
>
> This is how I see the trend going, but hey, I am just a simple person
> and I get to be wrong lots of time.
>
> --
> Paulo

Thank you for the documentation. The more I know about GC, the more I think that user should be able to choose which GC they want.
February 26, 2012
On Sunday, 26 February 2012 at 12:09:21 UTC, Paulo Pinto wrote:

> It is still at 0.2 and the newsgroup only has 13 messages, lets see how far it goes.

We are almost done with gpu devolution.
Once we get unified storage none of this will matter, much like flat to round earth transition. So much wasted for absolutely nothing.
February 26, 2012
On 26/02/2012 03:22, Walter Bright wrote:
> On 2/25/2012 4:01 PM, Simon wrote:
>> On 25/02/2012 22:55, Walter Bright wrote:
>>> Enter C++'s shared_ptr. But that works by, for each object, allocating a
>>> *second* chunk of memory to hold the reference count. Right off the bat,
>>> you've got twice as many allocations & frees with shared_ptr than a GC
>>> would have.
>>
>> http://www.boost.org/doc/libs/1_43_0/libs/smart_ptr/make_shared.html
>>
>> so you don't have to have twice as many allocations.
>
> There are many ways to do shared pointers, including one where the
> reference count is part of the object being allocated. But the C++11
> standard share_ptr does an extra allocation.

The stl one is based on boost, so it has make_shared as well:

http://en.cppreference.com/w/cpp/memory/shared_ptr

and it's in vs 2010

http://msdn.microsoft.com/en-us/library/ee410595.aspx

Not that I'm claiming shared pointers are superior to GC.

-- 
My enormous talent is exceeded only by my outrageous laziness.
http://www.ssTk.co.uk
February 26, 2012
Le 25/02/2012 23:55, Walter Bright a écrit :
> On 2/25/2012 2:08 PM, Paulo Pinto wrote:
>> Most standard compiler malloc()/free() implementations are actually
>> slower than
>> most advanced GC algorithms.
>
> Most straight up GC vs malloc/free benchmarks miss something crucial. A
> GC allows one to do substantially *fewer* allocations. It's a lot faster
> to not allocate than to allocate.
>
> Consider C strings. You need to keep track of ownership of it. That
> often means creating extra copies, rather than sharing a single copy.
>
> Enter C++'s shared_ptr. But that works by, for each object, allocating a
> *second* chunk of memory to hold the reference count. Right off the bat,
> you've got twice as many allocations & frees with shared_ptr than a GC
> would have.

True, but the problem of video game isn't how much computation you do to allocate, but to deliver a frame every few miliseconds. In most cases, it worth spending more in allocating but with a predictable result than let the GC does its job.

I wonder how true this will become with multicore and possibility of a 100% concurrent GC.
February 26, 2012
On Sunday, 26 February 2012 at 15:22:09 UTC, deadalnix wrote:

> True, but the problem of video game isn't how much computation you do to allocate, but to deliver a frame every few miliseconds. In most cases, it worth spending more in allocating but with a predictable result than let the GC does its job.

Absolutely! It cracks me up when i see (in this forum or any other graphics related forums) things like "you can't allocate at runtime!!!" or "you shouldn't use standard libraries!!!". Thing is, you can do both just fine if you just RTFM :)


February 26, 2012
On Sun, Feb 26, 2012 at 04:48:12PM +0100, so wrote:
> On Sunday, 26 February 2012 at 15:22:09 UTC, deadalnix wrote:
> 
> >True, but the problem of video game isn't how much computation you do to allocate, but to deliver a frame every few miliseconds. In most cases, it worth spending more in allocating but with a predictable result than let the GC does its job.
> 
> Absolutely! It cracks me up when i see (in this forum or any other graphics related forums) things like "you can't allocate at runtime!!!" or "you shouldn't use standard libraries!!!". Thing is, you can do both just fine if you just RTFM :)
[...]

Would this even be an issue on multicore systems where the GC can run concurrently? As long as the stop-the-world parts are below some given threshold.


T

-- 
Ruby is essentially Perl minus Wall.
February 26, 2012
On Sunday, 26 February 2012 at 15:58:41 UTC, H. S. Teoh wrote:

> Would this even be an issue on multicore systems where the GC can run
> concurrently? As long as the stop-the-world parts are below some given
> threshold.

If it is possible to guarantee that i don't think anyone would bother with manual MM.