Branch: refs/heads/master
  Home:   https://github.com/D-Programming-Language/dmd

  Commit: 2f7ce3475128142068a491430b038c30dff42bd2
      https://github.com/D-Programming-Language/dmd/commit/2f7ce3475128142068a491430b038c30dff42bd2
  Author: Walter Bright <walter at walterbright.com>
  Date:   2011-09-24 (Sat, 24 Sep 2011)

  Changed paths:
    M src/mars.c
  M src/module.c
  A src/root/dmgcmem.c
  A src/root/gc/bits.c
  A src/root/gc/bits.h
  A src/root/gc/gc.c
  A src/root/gc/gc.h
  A src/root/gc/gccbitops.h
  A src/root/gc/linux.c
  A src/root/gc/mscbitops.h
  A src/root/gc/os.h
  A src/root/gc/testgc.c
  A src/root/gc/win32.c
  M src/root/rmem.c
  M src/root/root.c
  M src/win32.mak

  Log Message:
  -----------
  add gc for Windows dmd

On Sat, Sep 24, 2011 at 4:26 AM,  <noreply at github.com> wrote:
> ?Branch: refs/heads/master
> ?Home: ? https://github.com/D-Programming-Language/dmd
>
> ?Commit: 2f7ce3475128142068a491430b038c30dff42bd2
> ? ? ?https://github.com/D-Programming-Language/dmd/commit/2f7ce3475128142068a491430b038c30dff42bd2
> ?Author: Walter Bright <walter at walterbright.com>
> ?Date: ? 2011-09-24 (Sat, 24 Sep 2011)
>
> ?Changed paths:
> ? ?M src/mars.c
> ?M src/module.c
> ?A src/root/dmgcmem.c
> ?A src/root/gc/bits.c
> ?A src/root/gc/bits.h
> ?A src/root/gc/gc.c
> ?A src/root/gc/gc.h
> ?A src/root/gc/gccbitops.h
> ?A src/root/gc/linux.c
> ?A src/root/gc/mscbitops.h
> ?A src/root/gc/os.h
> ?A src/root/gc/testgc.c
> ?A src/root/gc/win32.c
> ?M src/root/rmem.c
> ?M src/root/root.c
> ?M src/win32.mak
>
> ?Log Message:
> ?-----------
> ?add gc for Windows dmd
>


Literally? IE should I try doing some heavy CTFE string generation and see if the memory usage is significantly better?

Looks like the execution time for dmd and phobos have increased fairly substantially since the gc was checked in.

  http://d.puremagic.com/test-results/platform-history.ghtml?os=Win_32
  http://d.puremagic.com/test-results/platform-history.ghtml?os=Linux_32

The linux link is just for comparison to show that no increase has occurred there.  On windows the phobos test time has increased from approx 8 minutes to approx 10 minutes, or 25%.  The dmd test has increased from approx 27 minutes (much more variation and I didn't compute an average or std dev) to approx 39 minutes (with only one completed data point so far, so, well, grain of salt), almost doubling.

A few more runs will help show the averages better, but still worth being concerned about.  That's a significant performance hit to the compiler.

On Saturday, September 24, 2011 2:26:37 AM, noreply at github.com wrote:
>   Branch: refs/heads/master
>   Home:   https://github.com/D-Programming-Language/dmd
>
>   Commit: 2f7ce3475128142068a491430b038c30dff42bd2
>       https://github.com/D-Programming-Language/dmd/commit/2f7ce3475128142068a491430b038c30dff42bd2
>   Author: Walter Bright <walter at walterbright.com>
>   Date:   2011-09-24 (Sat, 24 Sep 2011)
>
>   Changed paths:
>     M src/mars.c
>   M src/module.c
>   A src/root/dmgcmem.c
>   A src/root/gc/bits.c
>   A src/root/gc/bits.h
>   A src/root/gc/gc.c
>   A src/root/gc/gc.h
>   A src/root/gc/gccbitops.h
>   A src/root/gc/linux.c
>   A src/root/gc/mscbitops.h
>   A src/root/gc/os.h
>   A src/root/gc/testgc.c
>   A src/root/gc/win32.c
>   M src/root/rmem.c
>   M src/root/root.c
>   M src/win32.mak
>
>   Log Message:
>   -----------
>   add gc for Windows dmd
>
>

Sorry, not double.  I meant +50%.

On Sep 24, 2011, at 2:10 PM, Brad Roberts <braddr at puremagic.com> wrote:

> Looks like the execution time for dmd and phobos have increased fairly substantially since the gc was checked in.
> 
>  http://d.puremagic.com/test-results/platform-history.ghtml?os=Win_32
>  http://d.puremagic.com/test-results/platform-history.ghtml?os=Linux_32
> 
> The linux link is just for comparison to show that no increase has occurred there.  On windows the phobos test time has increased from approx 8 minutes to approx 10 minutes, or 25%.  The dmd test has increased from approx 27 minutes (much more variation and I didn't compute an average or std dev) to approx 39 minutes (with only one completed data point so far, so, well, grain of salt), almost doubling.
> 
> A few more runs will help show the averages better, but still worth being concerned about.  That's a significant performance hit to the compiler.
> 
> On Saturday, September 24, 2011 2:26:37 AM, noreply at github.com wrote:
>>  Branch: refs/heads/master
>>  Home:   https://github.com/D-Programming-Language/dmd
>> 
>>  Commit: 2f7ce3475128142068a491430b038c30dff42bd2
>>      https://github.com/D-Programming-Language/dmd/commit/2f7ce3475128142068a491430b038c30dff42bd2
>>  Author: Walter Bright <walter at walterbright.com>
>>  Date:   2011-09-24 (Sat, 24 Sep 2011)
>> 
>>  Changed paths:
>>    M src/mars.c
>>  M src/module.c
>>  A src/root/dmgcmem.c
>>  A src/root/gc/bits.c
>>  A src/root/gc/bits.h
>>  A src/root/gc/gc.c
>>  A src/root/gc/gc.h
>>  A src/root/gc/gccbitops.h
>>  A src/root/gc/linux.c
>>  A src/root/gc/mscbitops.h
>>  A src/root/gc/os.h
>>  A src/root/gc/testgc.c
>>  A src/root/gc/win32.c
>>  M src/root/rmem.c
>>  M src/root/root.c
>>  M src/win32.mak
>> 
>>  Log Message:
>>  -----------
>>  add gc for Windows dmd
>> 
>> 
> 
> _______________________________________________
> dmd-internals mailing list
> dmd-internals at puremagic.com
> http://lists.puremagic.com/mailman/listinfo/dmd-internals

I've gotten similar results. It's worse than I'd thought it would be. The addition of the gc substantially slows things down. Granted, the gc could be tuned, but this is a very inauspicious start.

Of course, the test suite is all small programs. How a large one behaved would be interesting.

Fortunately, the gc can be swapped in and out with a commenting/uncommenting 2 lines in win32.mak.

On 9/24/2011 2:10 PM, Brad Roberts wrote:
> Looks like the execution time for dmd and phobos have increased fairly substantially since the gc was checked in.
>
>    http://d.puremagic.com/test-results/platform-history.ghtml?os=Win_32
>    http://d.puremagic.com/test-results/platform-history.ghtml?os=Linux_32
>
> The linux link is just for comparison to show that no increase has occurred there.  On windows the phobos test time has increased from approx 8 minutes to approx 10 minutes, or 25%.  The dmd test has increased from approx 27 minutes (much more variation and I didn't compute an average or std dev) to approx 39 minutes (with only one completed data point so far, so, well, grain of salt), almost doubling.
>
> A few more runs will help show the averages better, but still worth being concerned about.  That's a significant performance hit to the compiler.

Hi,

Maybe a command line switch to DMD itself to enable/disable the GC would be useful.

Regards,
Alex

On Sun, Sep 25, 2011 at 12:53 AM, Walter Bright <walter at digitalmars.com> wrote:
> I've gotten similar results. It's worse than I'd thought it would be. The addition of the gc substantially slows things down. Granted, the gc could be tuned, but this is a very inauspicious start.
>
> Of course, the test suite is all small programs. How a large one behaved would be interesting.
>
> Fortunately, the gc can be swapped in and out with a commenting/uncommenting 2 lines in win32.mak.
>
> On 9/24/2011 2:10 PM, Brad Roberts wrote:
>>
>> Looks like the execution time for dmd and phobos have increased fairly substantially since the gc was checked in.
>>
>> ? http://d.puremagic.com/test-results/platform-history.ghtml?os=Win_32 ? http://d.puremagic.com/test-results/platform-history.ghtml?os=Linux_32
>>
>> The linux link is just for comparison to show that no increase has occurred there. ?On windows the phobos test time has increased from approx 8 minutes to approx 10 minutes, or 25%. ?The dmd test has increased from approx 27 minutes (much more variation and I didn't compute an average or std dev) to approx 39 minutes (with only one completed data point so far, so, well, grain of salt), almost doubling.
>>
>> A few more runs will help show the averages better, but still worth being concerned about. ?That's a significant performance hit to the compiler.
>
> _______________________________________________
> dmd-internals mailing list
> dmd-internals at puremagic.com
> http://lists.puremagic.com/mailman/listinfo/dmd-internals
>

Perhaps the ideal approach here is to only collect when unable to obtain more memory from the OS?  Or turn on collections when DMD hits some threshold for memory consumption?  You really only want the GC to trigger when compiling a lot of CTFE and template code.

On Sep 24, 2011, at 2:10 PM, Brad Roberts wrote:

> Looks like the execution time for dmd and phobos have increased fairly substantially since the gc was checked in.
> 
>  http://d.puremagic.com/test-results/platform-history.ghtml?os=Win_32
>  http://d.puremagic.com/test-results/platform-history.ghtml?os=Linux_32
> 
> The linux link is just for comparison to show that no increase has occurred there.  On windows the phobos test time has increased from approx 8 minutes to approx 10 minutes, or 25%.  The dmd test has increased from approx 27 minutes (much more variation and I didn't compute an average or std dev) to approx 39 minutes (with only one completed data point so far, so, well, grain of salt), almost doubling.
> 
> A few more runs will help show the averages better, but still worth being concerned about.  That's a significant performance hit to the compiler.
> 
> On Saturday, September 24, 2011 2:26:37 AM, noreply at github.com wrote:
>>  Branch: refs/heads/master
>>  Home:   https://github.com/D-Programming-Language/dmd
>> 
>>  Commit: 2f7ce3475128142068a491430b038c30dff42bd2
>>      https://github.com/D-Programming-Language/dmd/commit/2f7ce3475128142068a491430b038c30dff42bd2
>>  Author: Walter Bright <walter at walterbright.com>
>>  Date:   2011-09-24 (Sat, 24 Sep 2011)
>> 
>>  Changed paths:
>>    M src/mars.c
>>  M src/module.c
>>  A src/root/dmgcmem.c
>>  A src/root/gc/bits.c
>>  A src/root/gc/bits.h
>>  A src/root/gc/gc.c
>>  A src/root/gc/gc.h
>>  A src/root/gc/gccbitops.h
>>  A src/root/gc/linux.c
>>  A src/root/gc/mscbitops.h
>>  A src/root/gc/os.h
>>  A src/root/gc/testgc.c
>>  A src/root/gc/win32.c
>>  M src/root/rmem.c
>>  M src/root/root.c
>>  M src/win32.mak
>> 
>>  Log Message:
>>  -----------
>>  add gc for Windows dmd
>> 
>> 
> 
> _______________________________________________
> dmd-internals mailing list
> dmd-internals at puremagic.com
> http://lists.puremagic.com/mailman/listinfo/dmd-internals

The dmd test suite is all small apps, but the phobos test on win32 is still one app (still not particularly large, but essentially the sum of the posix tests).  On posix the phobos tests are small test apps.

On Saturday, September 24, 2011 3:53:44 PM, Walter Bright wrote:
> I've gotten similar results. It's worse than I'd thought it would be. The addition of the gc substantially slows things down. Granted, the gc could be tuned, but this is a very inauspicious start.
>
> Of course, the test suite is all small programs. How a large one behaved would be interesting.
>
> Fortunately, the gc can be swapped in and out with a commenting/uncommenting 2 lines in win32.mak.
>
> On 9/24/2011 2:10 PM, Brad Roberts wrote:
>> Looks like the execution time for dmd and phobos have increased fairly substantially since the gc was checked in.
>>
>>    http://d.puremagic.com/test-results/platform-history.ghtml?os=Win_32
>>    http://d.puremagic.com/test-results/platform-history.ghtml?os=Linux_32
>>
>> The linux link is just for comparison to show that no increase has occurred there.  On windows the phobos test time has increased from approx 8 minutes to approx 10 minutes, or 25%.  The dmd test has increased from approx 27 minutes (much more variation and I didn't compute an average or std dev) to approx 39 minutes (with only one completed data point so far, so, well, grain of salt), almost doubling.
>>
>> A few more runs will help show the averages better, but still worth being concerned about.  That's a significant performance hit to the compiler.
> _______________________________________________
> dmd-internals mailing list
> dmd-internals at puremagic.com
> http://lists.puremagic.com/mailman/listinfo/dmd-internals

On 9/24/11 5:53 PM, Walter Bright wrote:
> I've gotten similar results. It's worse than I'd thought it would be. The addition of the gc substantially slows things down. Granted, the gc could be tuned, but this is a very inauspicious start.
>
> Of course, the test suite is all small programs. How a large one behaved would be interesting.

I think the default settings of today's GCs are years-old, e.g. they kick in at only a couple of hundred megs. We should tune the GC to kick-in only at 2GB RAM RSS.

Andrei

On Saturday, September 24, 2011 20:11:40 Andrei Alexandrescu wrote:
> On 9/24/11 5:53 PM, Walter Bright wrote:
> > I've gotten similar results. It's worse than I'd thought it would be. The addition of the gc substantially slows things down. Granted, the gc could be tuned, but this is a very inauspicious start.
> > 
> > Of course, the test suite is all small programs. How a large one behaved would be interesting.
> 
> I think the default settings of today's GCs are years-old, e.g. they kick in at only a couple of hundred megs. We should tune the GC to kick-in only at 2GB RAM RSS.

I've seen dmd on Windows run out of memory at 900 something MB (why, I don't know, since there was more memory on the box and that's nowhere near the 3.6GB limit of the address space, but I've seen it happen). Compiling the Phobos unit tests were failing that way before when I had std.datetime's unit tests enabled (though it seems that enough has been fixed that they can now be enabled without the memory going quite that high). So, I don't know that 2 GB is really low enough. If we want to do that sort of thing, we're probably going to need to run tests to find an optimal value. I suspect that something more like 500MB would be better than 2GB, but I don't know.

- Jonathan M Davis