[Issue 24293] ImportC: parallel compilation failure

Dec 24

Walter Bright

Dec 24

kdevel

Dec 30

Walter Bright

Dec 30

kdevel

Dec 30

Walter Bright

https://issues.dlang.org/show_bug.cgi?id=24293 Walter Bright <bugzilla@digitalmars.com> changed: What |Removed |Added ---------------------------------------------------------------------------- CC| |bugzilla@digitalmars.com --- Comment #1 from Walter Bright <bugzilla@digitalmars.com> --- Try it serially. --

https://issues.dlang.org/show_bug.cgi?id=24293 --- Comment #2 from kdevel <kdevel@vogtner.de> --- (In reply to Walter Bright from comment #1) > Try it serially. I did so before and it worked (surprisingly!). But when it came to make -j4 the non-reentrancy of dmd showed up. The cause of the problem is, that for every import statement dmd invokes cpp to generate for each header file $h.h a preprocessed $h.i (pure C file). This can be prevented by creating the .i file with cpp -m64 -dD -Wno-builtin-macro-redefined cmysql.c -include [...]dmd2/linux/bin64/../../src/druntime/import/importc.h cmysql.i once and before starting the compilation of the dependent D source files. I would appreciate if dmd had a switch to generate the .i from the .c. A short note in the docs would be nice, too. --

https://issues.dlang.org/show_bug.cgi?id=24293 --- Comment #3 from Walter Bright <bugzilla@digitalmars.com> --- I'm not understanding the need for dmd to generate the .i file rather than cpp? I'm also not understanding the reentrancy problem - are multiple .i files being generated with the same base name, thus conflicting with each other? --

December 30

[Issue 24293] ImportC: parallel compilation failure

Posted by kdevel

Permalink

kdevel

Permalink

https://issues.dlang.org/show_bug.cgi?id=24293

--- Comment #4 from kdevel <kdevel@vogtner.de> ---
(In reply to Walter Bright from comment #3)
> I'm not understanding the need for dmd to generate the .i file rather than cpp?
dmd invokes cpp in order to generade the .i file:

$ strace -s128 -feexecve dmd cmysql.c
[...]
[pid 24506] execve("/usr/bin/cpp", ["cpp", "-m64", "-dD",
"-Wno-builtin-macro-redefined", "cmysql.c", "-include",
"[...]/dmd2/linux/bin64/../../src/druntime/import/importc.h", "cmysql.i"], [/*
96 vars */]) = 0
[...]

> I'm also not understanding the reentrancy problem - are multiple .i files being generated with the same base name, thus conflicting with each other?
There are two or more processes each writing to a file with the same file name
(cmysql.i):

$ strace -s128 -feexecve dmd xmysqltest1.d
[...]
[pid 24560] execve("/usr/bin/cpp", ["cpp", "-m64", "-dD",
"-Wno-builtin-macro-redefined", "cmysql.c", "-include",
"[...]/dmd2/linux/bin64/../../src/druntime/import/importc.h", "cmysql.i"], [/*
96 vars */]) = 0
[...]

$ strace -s128 -feexecve dmd xmysqltest2.d
[...]
[pid 24566] execve("/usr/bin/cpp", ["cpp", "-m64", "-dD",
"-Wno-builtin-macro-redefined", "cmysql.c", "-include",
"[...]/dmd2/linux/bin64/../../src/druntime/import/importc.h", "cmysql.i"], [/*
96 vars */]) = 0
[...]

If the processes run in parallel cmysql.i refers to the same file. Since cmysql.i is opened with O_TRUNC the content already written is lost.

--

https://issues.dlang.org/show_bug.cgi?id=24293 --- Comment #5 from Walter Bright <bugzilla@digitalmars.com> --- Ok, I think I see the problem. apple.d imports pear.c peach.d imports pear.c The commands: dmd -c apple.d dmd -c peach.d both started concurrently mean both try to write then read pear.i, stepping on each other. A possible solution would be to create a temporary file rather than pear.i. --

https://issues.dlang.org/show_bug.cgi?id=24293 Dlang Bot <dlang-bot@dlang.rocks> changed: What |Removed |Added ---------------------------------------------------------------------------- Keywords| |pull --- Comment #6 from Dlang Bot <dlang-bot@dlang.rocks> --- @WalterBright created dlang/dmd pull request #15981 "fix Issue 24293 - ImportC: C preprocessor output should use temporary…" fixing this issue: - fix Issue 24293 - ImportC: C preprocessor output should use temporary files https://github.com/dlang/dmd/pull/15981 --

https://issues.dlang.org/show_bug.cgi?id=24293 Dlang Bot <dlang-bot@dlang.rocks> changed: What |Removed |Added ---------------------------------------------------------------------------- Status|NEW |RESOLVED Resolution|--- |FIXED --- Comment #7 from Dlang Bot <dlang-bot@dlang.rocks> --- dlang/dmd pull request #15981 "fix Issue 24293 - ImportC: C preprocessor output should use temporary…" was merged into master: - 2abdce7fa11fa08a3f33a40206d31c70e2af6fdd by Walter Bright: fix Issue 24293 - ImportC: C preprocessor output should use temporary files https://github.com/dlang/dmd/pull/15981 --

https://issues.dlang.org/show_bug.cgi?id=24293 RazvanN <razvan.nitu1305@gmail.com> changed: What |Removed |Added ---------------------------------------------------------------------------- CC| |ibuclaw@gdcproject.org --- Comment #8 from RazvanN <razvan.nitu1305@gmail.com> --- *** Issue 24252 has been marked as a duplicate of this issue. *** --

Forums