Thread overview
[Issue 23603] ICE out of memory when using -lowmem
Jan 11, 2023
Walter Bright
Jan 12, 2023
Bastiaan Veelo
Jan 13, 2023
Bastiaan Veelo
January 11, 2023
https://issues.dlang.org/show_bug.cgi?id=23603

Walter Bright <bugzilla@digitalmars.com> changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
             Status|NEW                         |RESOLVED
                 CC|                            |bugzilla@digitalmars.com
         Resolution|---                         |WONTFIX

--- Comment #1 from Walter Bright <bugzilla@digitalmars.com> ---
Issues:

1. `import std` imports all of Phobos. This is quite impractical, and import by package should never have been added. It will always make compiling desperately slow and will consume vast quantities of memory.

Recommendation: just import modules that are needed

2. The declaration of `data` has an initialization that must happen at compile time. The CTFE to do it does not store expressions efficiently - the array is stored as (256 * 131,070) Expression AST nodes. This is going to consume a lot of memory, and will also be extremely slow. Then, it will have to generate (256 * 131,070) static initializers.

Recommendation: Replace with a pointer initialized at runtime:

    Array!A(255)* pdata;
    static this() { pdata = new Array!(255); }

which will push that all off to runtime, where it is efficiently handled.

This should resolve the slow compiles, vast compile time memory consumption, and large executable file size problems. I don't see any reasonable way to fix this in the language.

--
January 12, 2023
https://issues.dlang.org/show_bug.cgi?id=23603

--- Comment #2 from Bastiaan Veelo <Bastiaan@Veelo.net> ---
I agree with your recommendations obviously, but it doesn't solve the seemingly unexplainable cases where dmd claims to run out of memory. There is plenty of memory available. This example succeeds when I *remove* the -lowmem option. LDC does not run out of memory compiling the same code.

In other cases that are hard to reduce I sometimes get the above verbose message, sometimes just "Error: out of memory", sometimes only exit code -1073741795 (0xC000001D).

When a large code base reduces to a nice concise handfull of lines like this it is easy to rewrite like you suggest, but this particular reduction took four weeks to prepare and required manual steps. I have another case that is still running, not sure if that will ever converge to reveal the problematic code.

--
January 13, 2023
https://issues.dlang.org/show_bug.cgi?id=23603

Bastiaan Veelo <Bastiaan@Veelo.net> changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
         Resolution|WONTFIX                     |INVALID

--- Comment #3 from Bastiaan Veelo <Bastiaan@Veelo.net> ---
Thanks to @kinke for pointing this out, I have been reducing with the wrong compiler. Adjusting the path to c:\d\dmd.2.101.2.windows\dmd2\windows\bin64\dmd.exe using the 64bit compiler this compiles fine.

(In my defence I started with a legitimate problem, I changed to the latest compiler version halfway the reduction, but I did it wrong.)

This particular issue is invalid.

--