On 3/30/15 3:47 PM, lobo wrote:
> On Monday, 30 March 2015 at 22:39:51 UTC, lobo wrote:
>> On Sunday, 29 March 2015 at 23:14:31 UTC, Martin Krejcirik wrote:
>>> It seems like every DMD release makes compilation slower. This time I
>>> see 10.8s vs 7.8s on my little project. I know this is generally
>>> least of concern, and D1's lighting-fast times are long gone, but
>>> since Walter often claims D's superior compilation speeds, maybe some
>>> profiling is in order ?
>>
>> I'm finding memory usage the biggest problem for me. 3s speed increase
>> is not nice but an increase of 500MB RAM usage with DMD 2.067 over
>> 2.066 means I can no longer build one of my projects.
>>
>> bye,
>> lobo
>
> I should add that I am on a 32-bit machine with 4GB RAM. I just ran some
> tests measuring RAM usage:
>
> DMD 2.067 ~4.2GB (fails here so not sure of the full amount required)
> DMD 2.066 ~3.7GB (maximum)
> DMD 2.065 ~3.1GB (maximum)
>
> It was right on the edge with 2.066 anyway but this trend of more RAM
> usage seems to also be occurring with each DMD release.
>
> bye,
> lobo

Part of our acceptance tests should be peak memory, object file size, executable file size, and run time for building a few test programs (starting with "hello, world"). Any change in these must be investigated, justified, and documented. -- Andrei

On Mon, Mar 30, 2015 at 10:39:50PM +0000, lobo via Digitalmars-d wrote:
> On Sunday, 29 March 2015 at 23:14:31 UTC, Martin Krejcirik wrote:
> >It seems like every DMD release makes compilation slower. This time I see 10.8s vs 7.8s on my little project. I know this is generally least of concern, and D1's lighting-fast times are long gone, but since Walter often claims D's superior compilation speeds, maybe some profiling is in order ?
> 
> I'm finding memory usage the biggest problem for me. 3s speed increase is not nice but an increase of 500MB RAM usage with DMD 2.067 over 2.066 means I can no longer build one of my projects.
[...]

Yeah, dmd memory consumption is way off the charts, because under the pretext of compile speed it never frees allocated memory. Unfortunately, the assumption that not managing memory == faster quickly becomes untrue once dmd runs out of RAM and the OS starts thrashing. Compile times quickly skyrocket exponentially as everything gets stuck on I/O.

This is one of the big reasons I can't use D on my work PC, because it's an older machine with limited RAM, and when DMD is running the whole box slows down to an unusable crawl.

This is not the first time this issue was brought up, but it seems nobody in the compiler team cares enough to do anything about it. :-(


T

-- 
Lottery: tax on the stupid. -- Slashdotter

2015-03-31 0:53 GMT+02:00 H. S. Teoh via Digitalmars-d < digitalmars-d@puremagic.com>:

>
> Yeah, dmd memory consumption is way off the charts, because under the pretext of compile speed it never frees allocated memory. Unfortunately, the assumption that not managing memory == faster quickly becomes untrue once dmd runs out of RAM and the OS starts thrashing. Compile times quickly skyrocket exponentially as everything gets stuck on I/O.
>
> This is one of the big reasons I can't use D on my work PC, because it's an older machine with limited RAM, and when DMD is running the whole box slows down to an unusable crawl.
>
> This is not the first time this issue was brought up, but it seems nobody in the compiler team cares enough to do anything about it. :-(
>
>
> T
>
> --
> Lottery: tax on the stupid. -- Slashdotter
>

I can relate. DMD compilation speed was nothing but a myth to me until I migrated from 4GBs to 8 GBs. And everytime I compiled something, my computer froze for a few seconds (or a few minutes, depending of the project).

On Monday, 30 March 2015 at 22:55:50 UTC, H. S. Teoh wrote:
> On Mon, Mar 30, 2015 at 10:39:50PM +0000, lobo via Digitalmars-d wrote:
>> On Sunday, 29 March 2015 at 23:14:31 UTC, Martin Krejcirik wrote:
>> >It seems like every DMD release makes compilation slower. This time I
>> >see 10.8s vs 7.8s on my little project. I know this is generally
>> >least of concern, and D1's lighting-fast times are long gone, but
>> >since Walter often claims D's superior compilation speeds, maybe some
>> >profiling is in order ?
>> 
>> I'm finding memory usage the biggest problem for me. 3s speed increase
>> is not nice but an increase of 500MB RAM usage with DMD 2.067 over
>> 2.066 means I can no longer build one of my projects.
> [...]
>
> Yeah, dmd memory consumption is way off the charts, because under the
> pretext of compile speed it never frees allocated memory. Unfortunately,
> the assumption that not managing memory == faster quickly becomes untrue
> once dmd runs out of RAM and the OS starts thrashing. Compile times
> quickly skyrocket exponentially as everything gets stuck on I/O.
>
> This is one of the big reasons I can't use D on my work PC, because it's
> an older machine with limited RAM, and when DMD is running the whole box
> slows down to an unusable crawl.
>
> This is not the first time this issue was brought up, but it seems
> nobody in the compiler team cares enough to do anything about it. :-(
>
>
> T

I sometimes think DMD's memory should be... garbage collected. I used the forbidden phrase!

Seriously though, allocating a bunch of memory until you hit some maximum threshold, possibly configured, and freeing unreferenced memory at that point, pausing compilation while that happens? This is GC. I wonder if someone enterprising enough would be willing to try it out with DDMD by swapping malloc calls with calls to D's GC or something.

On Monday, 30 March 2015 at 23:28:50 UTC, w0rp wrote:
> I sometimes think DMD's memory should be... garbage collected. I used the forbidden phrase!
>

Yes, set an initial heap size of 100Mb or something and the GC won't kick in for scripts.

Also, free after CTFE !

On 3/30/15 4:28 PM, w0rp wrote:
> I sometimes think DMD's memory should be... garbage collected. I used
> the forbidden phrase!

Compiler workloads are a good candidate for GC. -- Andrei

On Monday, 30 March 2015 at 23:28:50 UTC, w0rp wrote:
> Seriously though, allocating a bunch of memory until you hit some maximum threshold, possibly configured, and freeing unreferenced memory at that point, pausing compilation while that happens? This is GC. I wonder if someone enterprising enough would be willing to try it out with DDMD by swapping malloc calls with calls to D's GC or something.

has anyone tried using boehm with dmd? I'm pretty sure it has a way of being LD_PRELOADed to override malloc IIRC.

On Monday, 30 March 2015 at 22:51:43 UTC, Andrei Alexandrescu wrote:
> Part of our acceptance tests should be peak memory, object file size, executable file size, and run time for building a few test programs (starting with "hello, world"). Any change in these must be investigated, justified, and documented. -- Andrei

I have filed this issue today:

https://issues.dlang.org/show_bug.cgi?id=14381

On Tuesday, 31 March 2015 at 00:54:08 UTC, Andrei Alexandrescu wrote:
> On 3/30/15 4:28 PM, w0rp wrote:
>> I sometimes think DMD's memory should be... garbage collected. I used
>> the forbidden phrase!
>
> Compiler workloads are a good candidate for GC. -- Andrei

Yes, compiler to perform significantly better with GC than with other memory management strategy. Ironically, I think that weighted a bit too much in favor of GC for language design in the general case.

"w0rp"  wrote in message news:leajtjgremulowqoxqpc@forum.dlang.org...

> I sometimes think DMD's memory should be... garbage collected. I used the forbidden phrase!
>
> Seriously though, allocating a bunch of memory until you hit some maximum threshold, possibly configured, and freeing unreferenced memory at that point, pausing compilation while that happens? This is GC. I wonder if someone enterprising enough would be willing to try it out with DDMD by swapping malloc calls with calls to D's GC or something.

I've used D's GC with DDMD.  It works*, but you're trading better memory usage for worse allocation speed.  It's quite possible we could add a switch to ddmd to enable the GC.

* Well actually it currently segfaults, but not because of anything fundamentally wrong with the approach.

After switching to DDMD we will have a HUGE number of options readily available for reducing memory usage, such as using allocation-free range code and enabling the GC.