On 03/31/2015 08:24 PM, deadalnix wrote:
> I'm going to propose again the same thing as in the past :
>  - before CTFE switch pool.
>  - CTFE in the new pool.
>  - deep copy result from ctfe pool to main pool.
>  - ditch ctfe pool.

No, it's trivial enough to implement a full AST interpreter.
The way it's done currently (using AST nodes as CTFE interpreter values)
makes it very hard to use a distinct allocator, because ownership can
move from CTFE to compiler and vice versa.

On Tuesday, 31 March 2015 at 21:53:29 UTC, Martin Nowak wrote:
> On 03/31/2015 08:24 PM, deadalnix wrote:
>> I'm going to propose again the same thing as in the past :
>>  - before CTFE switch pool.
>>  - CTFE in the new pool.
>>  - deep copy result from ctfe pool to main pool.
>>  - ditch ctfe pool.
>
> No, it's trivial enough to implement a full AST interpreter.
> The way it's done currently (using AST nodes as CTFE interpreter values)
> makes it very hard to use a distinct allocator, because ownership can
> move from CTFE to compiler and vice versa.

This is why I introduced a deep copy step in there.

On Tuesday, 31 March 2015 at 19:20:20 UTC, Jake The Baker wrote:
> On Monday, 30 March 2015 at 22:47:51 UTC, lobo wrote:
>> On Monday, 30 March 2015 at 22:39:51 UTC, lobo wrote:
>>> On Sunday, 29 March 2015 at 23:14:31 UTC, Martin Krejcirik wrote:
>>>> It seems like every DMD release makes compilation slower. This time I see 10.8s vs 7.8s on my little project. I know this is generally least of concern, and D1's lighting-fast times are long gone, but since Walter often claims D's superior compilation speeds, maybe some profiling is in order ?
>>>
>>> I'm finding memory usage the biggest problem for me. 3s speed increase is not nice but an increase of 500MB RAM usage with DMD 2.067 over 2.066 means I can no longer build one of my projects.
>>>
>>> bye,
>>> lobo
>>
>> I should add that I am on a 32-bit machine with 4GB RAM. I just ran some tests measuring RAM usage:
>>
>> DMD 2.067 ~4.2GB (fails here so not sure of the full amount required)
>> DMD 2.066 ~3.7GB (maximum)
>> DMD 2.065 ~3.1GB (maximum)
>>
>> It was right on the edge with 2.066 anyway but this trend of more RAM usage seems to also be occurring with each DMD release.
>>
>> bye,
>> lobo
>
> As far as memory is concerned. How hard would it be to simply have DMD use a swap file? This would fix the out of memory issues and provide some safety(at least you can get your project to compile. Seems like it would be a relatively simple thing to add?

It's incredibly slow and unproductive it's not really an option. My primary reason for using D is that I can be as productive as I am in Python but retain the same raw native power of C++.

Anyway, it sounds D devs have a few good ideas on how to resolve this.

bye,
lobo

> I've used D's GC with DDMD.  It works*, but you're trading better memory usage for worse allocation speed.  It's quite possible we could add a switch to ddmd to enable the GC.

As a random d-user (who cares about perf/speed and just happened to read this) a switch sounds VERY good to me. I don't want to pay the price of GC because of some low-end machines. Memory is really cheap these days and pretty much every machine is 64-bits (even phones are trasitioning fast to 64-bits).

Also, I wanted to add that freeing (at least to the OS (does this apply to GC?)) isn't exactly free either. Infact it can be more costly than mallocing.
Here's enlightening article: https://randomascii.wordpress.com/2014/12/10/hidden-costs-of-memory-allocation/

On Tuesday, 31 March 2015 at 19:27:35 UTC, Adam D. Ruppe wrote:
> On Tuesday, 31 March 2015 at 19:20:20 UTC, Jake The Baker wrote:
>> As far as memory is concerned. How hard would it be to simply have DMD use a swap file?
>
> That'd hit the same walls as the operating system trying to use a swap file at least - running out of address space, and being brutally slow even if it does keep running.

I doubt it. If most modules are sparsely used it would improve
memory usage in proportion to that.

Basically if D would monitor file/module usage and compile areas
that are relatively independent it should minimize disk usage.
Basically page out stuff you know won't be needed. If it was
smart enough it could order the data through module usage and
compile the independent ones first, then only the ones that are
simple dependencies, etc).

The benefits to such a system is that larger projects get the
biggest boost(there are more independent modules floating around.
Hence at some point it becomes a non-issue.

On Wednesday, 1 April 2015 at 02:54:48 UTC, Jake The Baker wrote:
> On Tuesday, 31 March 2015 at 19:27:35 UTC, Adam D. Ruppe wrote:
>> On Tuesday, 31 March 2015 at 19:20:20 UTC, Jake The Baker wrote:
>>> As far as memory is concerned. How hard would it be to simply have DMD use a swap file?
>>
>> That'd hit the same walls as the operating system trying to use a swap file at least - running out of address space, and being brutally slow even if it does keep running.
>
> I doubt it. If most modules are sparsely used it would improve
> memory usage in proportion to that.
>
> Basically if D would monitor file/module usage and compile areas
> that are relatively independent it should minimize disk usage.
> Basically page out stuff you know won't be needed. If it was
> smart enough it could order the data through module usage and
> compile the independent ones first, then only the ones that are
> simple dependencies, etc).
>
> The benefits to such a system is that larger projects get the
> biggest boost(there are more independent modules floating around.
> Hence at some point it becomes a non-issue.

I have no idea what you're talking about here, sorry.

I'm compiling modules separately already to object files. I think that helps reduce memory usage but I could be mistaken.

I think the main culprit now is my attempts to (ab)use CTFE. After switching to DMD 2.066 I started adding `enum val=f()` where I could. After reading the discussions here I went about reverting most of these back to `auto val=<blah>` and I'm building again :-)

DMD 2.067 is now maxing out at ~3.8GB and stable.

bye,
lobo

On Wed, 01 Apr 2015 02:25:43 +0000, Random D-user wrote:

> GC because of some low-end machines. Memory is really cheap these days and pretty much every machine is 64-bits (even phones are trasitioning fast to 64-bits).

this is the essense of "modern computing", btw. "hey, we have this resource! hey, we have the only program user will ever want to run, so assume that all that resource is ours! what? just buy a better box!"

On Wednesday, 1 April 2015 at 04:49:55 UTC, ketmar wrote:
> On Wed, 01 Apr 2015 02:25:43 +0000, Random D-user wrote:
>
>> GC because of some low-end machines. Memory is really cheap these days
>> and pretty much every machine is 64-bits (even phones are trasitioning
>> fast to 64-bits).
>
> this is the essense of "modern computing", btw. "hey, we have this
> resource! hey, we have the only program user will ever want to run, so
> assume that all that resource is ours! what? just buy a better box!"

google/mozilla's developer mantra regarding web browsers.

On Wednesday, 1 April 2015 at 04:51:26 UTC, weaselcat wrote:
> On Wednesday, 1 April 2015 at 04:49:55 UTC, ketmar wrote:
>> On Wed, 01 Apr 2015 02:25:43 +0000, Random D-user wrote:
>>
>>> GC because of some low-end machines. Memory is really cheap these days
>>> and pretty much every machine is 64-bits (even phones are trasitioning
>>> fast to 64-bits).
>>
>> this is the essense of "modern computing", btw. "hey, we have this
>> resource! hey, we have the only program user will ever want to run, so
>> assume that all that resource is ours! what? just buy a better box!"
>
> google/mozilla's developer mantra regarding web browsers.

They must have an agreement with DRAM vendor, I see no other explanation...

"lobo"  wrote in message news:vydmnbzapttzjnnctizm@forum.dlang.org...

> I think the main culprit now is my attempts to (ab)use CTFE. After switching to DMD 2.066 I started adding `enum val=f()` where I could. After reading the discussions here I went about reverting most of these back to `auto val=<blah>` and I'm building again :-)
>
> DMD 2.067 is now maxing out at ~3.8GB and stable.

Yeah, the big problem is that dmd's interpreter sort of evolved out of the constant folder, and wasn't designed for ctfe.  A new interpreter for dmd is one of the projects I hope to get to after DDMD is complete, unless somebody beats me to it.