I have some CTFE's and meta programming that cause dmd to run out of memory ;/

I am generating simple classes, but a lot of them. dmd uses about 2GB before it quites. It also only uses about 12% of cpu.

I have 16 GB total memory and about that free. Surely dmd could do a better job? Any way to get it to do such a thing like set the maximum amount of memory it can use?

On Saturday, 18 March 2017 at 20:39:20 UTC, StarGrazer wrote:
> I have some CTFE's and meta programming that cause dmd to run out of memory ;/
>
> I am generating simple classes, but a lot of them. dmd uses about 2GB before it quites. It also only uses about 12% of cpu.
>
> I have 16 GB total memory and about that free. Surely dmd could do a better job? Any way to get it to do such a thing like set the maximum amount of memory it can use?

DMD32 under win ?

On Saturday, 18 March 2017 at 20:43:29 UTC, Basile B. wrote:
> On Saturday, 18 March 2017 at 20:39:20 UTC, StarGrazer wrote:
>> I have some CTFE's and meta programming that cause dmd to run out of memory ;/
>>
>> I am generating simple classes, but a lot of them. dmd uses about 2GB before it quites. It also only uses about 12% of cpu.
>>
>> I have 16 GB total memory and about that free. Surely dmd could do a better job? Any way to get it to do such a thing like set the maximum amount of memory it can use?
>
> DMD32 under win ?

I guess. Whatever the dmd is from the download page. Yes, under windows.

On Saturday, 18 March 2017 at 20:56:47 UTC, StarGrazer wrote:
> On Saturday, 18 March 2017 at 20:43:29 UTC, Basile B. wrote:
>> On Saturday, 18 March 2017 at 20:39:20 UTC, StarGrazer wrote:
>>> I have some CTFE's and meta programming that cause dmd to run out of memory ;/
>>>
>>> I am generating simple classes, but a lot of them. dmd uses about 2GB before it quites. It also only uses about 12% of cpu.
>>>
>>> I have 16 GB total memory and about that free. Surely dmd could do a better job? Any way to get it to do such a thing like set the maximum amount of memory it can use?
>>
>> DMD32 under win ?
>
> I guess. Whatever the dmd is from the download page. Yes, under windows.

32bit processes have a limited access to the ram. Over your 16Gb only 2^^31 can be used. Add even this amount is likely to be reduced by the memory fragmentation.

In the future you'll be able to do more. Current CTFE engine has a suboptimal memory system, which will be changed in 2.075.
See https://dlang.org/blog/2016/11/18/project-highlight-the-new-ctfe-engine/, which explains more the memory thing.

On Saturday, 18 March 2017 at 20:39:20 UTC, StarGrazer wrote:
>
> about 2GB before it quites. It also only uses about 12% of cpu.
>
> I have 16 GB total memory and about that free. Surely dmd could do a better job? Any way to get it to do such a thing like set the maximum amount of memory it can use?

Any 32-bit process gets 2 GB of memory space, regardless of how much physical memory you have.

https://msdn.microsoft.com/library/aa366778.aspx

If you used a 64-bit version of dmd your problems should go away... If the binary for Windows isn't available from the downloads here, you can try compiling it from source yourself... But I'm sure someone somewhere has done it already. Or you can try another compiler such as GDC, which is available for Windows x64.

Also 12.5% probably means 100% of one of 8 cores in your CPU.

The Win64 LDC releases [https://github.com/ldc-developers/ldc/releases] feature a 64-bit compiler.

On Saturday, 18 March 2017 at 23:24:40 UTC, kinke wrote:
> The Win64 LDC releases [https://github.com/ldc-developers/ldc/releases] feature a 64-bit compiler.

just use DMD with the -m64 parameter ;)

On Sunday, 19 March 2017 at 20:49:02 UTC, Gand Alf wrote:
> On Saturday, 18 March 2017 at 23:24:40 UTC, kinke wrote:
>> The Win64 LDC releases [https://github.com/ldc-developers/ldc/releases] feature a 64-bit compiler.
>
> just use DMD with the -m64 parameter ;)

then you should get a x64 DMD

On Sunday, 19 March 2017 at 20:50:50 UTC, Gand Alf wrote:
>>
>> just use DMD with the -m64 parameter ;)
>
> then you should get a x64 DMD

No, at least afaik, then you tell DMD to make a x64 exe, but DMD itself (this particular Windows version) is still a 32-bit exe.

On Saturday, 18 March 2017 at 20:39:20 UTC, StarGrazer wrote:
> I have some CTFE's and meta programming that cause dmd to run out of memory ;/
>
> I am generating simple classes, but a lot of them. dmd uses about 2GB before it quites. It also only uses about 12% of cpu.
>
> I have 16 GB total memory and about that free. Surely dmd could do a better job? Any way to get it to do such a thing like set the maximum amount of memory it can use?

Can you submit this code for analysis ?