Thread overview
[Issue 20562] [dmd] Memory allocation failed (ERROR: This is a compiler bug)
Feb 05
JR
Feb 05
JR
Feb 05
JR
Feb 05
Seb
Feb 05
JR
5 days ago
Jonathan Marler
5 days ago
JR
February 05
https://issues.dlang.org/show_bug.cgi?id=20562

moonlightsentinel@disroot.org changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
                 CC|                            |moonlightsentinel@disroot.o
                   |                            |rg

--- Comment #1 from moonlightsentinel@disroot.org ---
That linker parameter was already present in win32.mak, the PR just moved it to build.d. That bisection seems like a false positive.

An OutOfMemoryError usually signals a failure when allocating <dynamic memory> - not stack memory. Did you try to compile your project using -lowmem (which enables the GC for DMD)?

--
February 05
https://issues.dlang.org/show_bug.cgi?id=20562

--- Comment #2 from JR <zorael@gmail.com> ---
(In reply to moonlightsentinel from comment #1)
> That linker parameter was already present in win32.mak, the PR just moved it to build.d. That bisection seems like a false positive.
> 
> An OutOfMemoryError usually signals a failure when allocating <dynamic memory> - not stack memory. Did you try to compile your project using -lowmem (which enables the GC for DMD)?

I see. Is there a point in retrying the bisection?

-lowmem works and I want to use it, but it doesn't work with dub, which I rely on for dependencies. https://forum.dlang.org/post/ycfikiktizuifuisxdgg@forum.dlang.org seems promising but it's still some time before it will be in a downloadable release.

--
February 05
https://issues.dlang.org/show_bug.cgi?id=20562

--- Comment #3 from moonlightsentinel@disroot.org ---
> I see. Is there a point in retrying the bisection?

Maybe if you could exactly identify a change that increased the memory consumption *a lot*. Then we could try to improve that specific segement to use less memory. But I would expect it to be a more gradual increase scattered over many small changes.

> -lowmem works and I want to use it, but it doesn't work with dub, which I rely on for dependencies

Not a dub expert but you could add -lowmem using the dflags configuration in
dub.(json|sdl) IIRC.

--
February 05
https://issues.dlang.org/show_bug.cgi?id=20562

--- Comment #4 from JR <zorael@gmail.com> ---
(In reply to moonlightsentinel from comment #3)
> > I see. Is there a point in retrying the bisection?
> 
> Maybe if you could exactly identify a change that increased the memory consumption *a lot*. Then we could try to improve that specific segement to use less memory. But I would expect it to be a more gradual increase scattered over many small changes.
> 
> > -lowmem works and I want to use it, but it doesn't work with dub, which I rely on for dependencies
> 
> Not a dub expert but you could add -lowmem using the dflags configuration in
> dub.(json|sdl) IIRC.

This was one of the topics of https://forum.dlang.org/post/ycfikiktizuifuisxdgg@forum.dlang.org; even if defined in dflags dub doesn't pass -lowmem to dmd. The flag is silently ignored.

--
February 05
https://issues.dlang.org/show_bug.cgi?id=20562

--- Comment #5 from JR <zorael@gmail.com> ---
Ignore that, it does seem to be doing something now. It still runs out of memory though, will experiment.

Apologies for the noise.

--
February 05
https://issues.dlang.org/show_bug.cgi?id=20562

Seb <greeenify@gmail.com> changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
                 CC|                            |greeenify@gmail.com

--- Comment #6 from Seb <greeenify@gmail.com> ---
This probably doesn't help you, but as it's easy to test:
Did you try this with 2.090 as well? There was a regression with the change to
build.d that omitted the -O flag on Windows. This has been addressed since.

--
February 05
https://issues.dlang.org/show_bug.cgi?id=20562

--- Comment #7 from JR <zorael@gmail.com> ---
(In reply to Seb from comment #6)
> This probably doesn't help you, but as it's easy to test:
> Did you try this with 2.090 as well? There was a regression with the change
> to build.d that omitted the -O flag on Windows. This has been addressed
> since.

I hadn't, so I tested it now with current master (v2.090.0-beta.1-118-g8fc81f40d) and I get the same results. Further I tried a 64-bit compiler (with digger build --model=64), and it runs out of memory too.

I've also retried the bisection several times hoping it would land on a different commit, but I get the same one. I even made the tester script test each build three times so as to not miss something due to transient successes.

The only thing that works so far is to copy/paste the build command `dub build -v` prints and run it with -lowmem added. Curiously dub with -lowmem DFLAGS does not seem to work. Manually calling dmd with the same command does.

--
5 days ago
https://issues.dlang.org/show_bug.cgi?id=20562

Jonathan Marler <johnnymarler@gmail.com> changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
                 CC|                            |johnnymarler@gmail.com

--- Comment #8 from Jonathan Marler <johnnymarler@gmail.com> ---
JR how did you do your git bisect?  What command were you using to build dmd?

--
5 days ago
https://issues.dlang.org/show_bug.cgi?id=20562

--- Comment #9 from JR <zorael@gmail.com> ---
(In reply to Jonathan Marler from comment #8)
> JR how did you do your git bisect?  What command were you using to build dmd?

I used digger to automate the bisection. I might add that I started all this with trying to make a minimal test case using dustmite, but after a while it gets reduced to a vague state that sometimes compiles, sometimes doesn't.

The test expression I used to bisect was the build command that `dub build -v` printed, grepped for "Memory allocation failed". I needed to invert the result for digger (a success should be when the string was not detected), so I couldn't just leave it as a normal command and had to make a batch script. I don't know Windows so it may be a roundabout way of doing it, but it works.

With my repo cloned into C:\Temp\kameloso, then with a C:\Temp\bisect.bat test script looking like the following: (necessitates having run `dub build` at least once to fetch and build the dependencies)

```
@echo off
cd \Temp\kameloso

dmd -dip25 -dip1008 -m64 -c -ofk -debug -g -w -d -version=WithAdminPlugin -version=WithAutomodePlugin -version=WithBashQuotesPlugin -version=WithChanQueriesService -version=WithChatbotPlugin -version=WithConnectService -version=WithCTCPService -version=WithHelpPlugin -version=WithNotesPlugin -version=WithOnelinersPlugin -version=WithPersistenceService -version=WithPipelinePlugin -version=WithPrinterPlugin -version=WithQuotesPlugin -version=WithSedReplacePlugin -version=WithSeenPlugin -version=WithWebtitlesPlugin -version=WithPlugins -version=Have_kameloso -version=Have_dialect -version=Have_lu -version=FlagAsUpdated -version=Have_lu_core -version=Have_lu_meld -Isource -IC:\Users\zorael\AppData\Local\dub\packages\dialect-0.3.2\dialect\source -IC:\Users\zorael\AppData\Local\dub\packages\lu-0.1.2\lu\source -IC:\Users\zorael\AppData\Local\dub\packages\lu-0.1.2\lu\source source\kameloso\common.d source\kameloso\constants.d source\kameloso\debugging.d source\kameloso\getopt.d source\kameloso\irccolours.d source\kameloso\kameloso.d source\kameloso\logger.d source\kameloso\main.d source\kameloso\messaging.d source\kameloso\plugins\admin.d source\kameloso\plugins\automode.d source\kameloso\plugins\bashquotes.d source\kameloso\plugins\chanqueries.d source\kameloso\plugins\chatbot.d source\kameloso\plugins\common.d source\kameloso\plugins\connect.d source\kameloso\plugins\ctcp.d source\kameloso\plugins\hello.d source\kameloso\plugins\help.d source\kameloso\plugins\notes.d source\kameloso\plugins\oneliners.d source\kameloso\plugins\package.d source\kameloso\plugins\persistence.d source\kameloso\plugins\pipeline.d source\kameloso\plugins\printer.d source\kameloso\plugins\quotes.d source\kameloso\plugins\sedreplace.d source\kameloso\plugins\seen.d source\kameloso\plugins\twitchbot.d source\kameloso\plugins\webtitles.d source\kameloso\printing.d source\kameloso\terminal.d source\kameloso\thread.d source\kameloso\traits.d -vcolumns 2>&1 | findstr /C:"Memory allocation failed"

if %ERRORLEVEL% EQU 0 (exit /b 1) else (exit /b 0)
```

I set up a digger bisect.ini looking like the following:

```
good = master@v2.087.1
bad = master@v2.089.1
reverse = false
tester = C:\Temp\bisect.bat
```

Then I set it off with `dub run digger -- bisect bisect.ini`.

--