After I compiled the parser as a library I wanted to test it to see if it successfully parses the phobos code. The code I am using is the following:


// test_parser.d
import std.stdio;
import std.file;

import ddmd.parse;
import ddmd.astbase;

import ddmd.id;
import ddmd.globals;

void main()
{
    string path = "../../phobos/std/";
    string regex = "*.d";

    // parser initializations
    Id.initialize();
    global._init();
    global.params.isLinux = true;
    global.params.is64bit = (size_t.sizeof == 8);
    ASTBase.Type._init();

    // get recursively all the .d files
    auto dFiles = dirEntries(path, regex, SpanMode.depth);
    foreach (d; dFiles)
    {
        writeln("Processing:", d.name);

        // comment the following line and it works
        string content = readText(d.name);

        scope p = new Parser!ASTBase(null, content, false);
        p.nextToken();
        p.parseModule();

        writeln("Finished!");
    }
}

This code fails with segmentation fault. The dFiles iterator gets broken somehow. If the line which reads a file is commented, than the code works. Also, if the parser imports are commented (ddmd.astbase and ddmd.parse) and all the lines of code which use code from those modules are commented, it also works.

I compiled the lexer and the parser with the following targets:


$G/lexer.a: $(LEXER_SRCS) $(LEXER_ROOT)
»   CC=$(HOST_CXX) $(HOST_DMD_RUN) -lib -of$@ $(MODEL_FLAG) -I/tmp/.host_dmd-2.072.2/dmd2/src/druntime/import/ -J$G -L-lstdc++ $(DFLAGS) $(LEXER_SRCS) $(LEXER_ROOT)


$G/parser.a: $(PARSER_SRCS) $G/lexer.a $(ROOT_SRCS)
»   CC=$(HOST_CXX) $(HOST_DMD_RUN) -lib -of$@ $(MODEL_FLAG) -I/tmp/.host_dmd-2.072.2/dmd2/src/druntime/import/ -J$G -L-lstdc++ $(DFLAGS) $(PARSER_SRCS) $G/lexer.a $(ROOT_SRCS) -Iddmd -J../res/

Then, the test file is compiled as follows:

parser_test: $G/parser.a test_parser.d
»   CC=$(HOST_CXX) $(HOST_DMD_RUN) -of$@ $(MODEL_FLAG) -vtls -J$G -L-lstdc++ $(DFLAGS) -I$G $G/lexer.a $G/parser.a test_parser.d -Iddmd -J../res/

I thought it may be a mismatch between the versions of druntime used for the compilation of the lexer and parser libraries and the one used for phobos. So I recompiled druntime (master->head) with the latest stable dmd (2.072.2) and then compiled phobos (master->head) with the aforementioned dmd and druntime, still I get the segfault.

I also tried to compile the lexer and the parser using -conf= -defaultlib= -debuglib= and then I tried to pass the path to druntime with -I when compiling test_parser.d, but running the program also ends with segmentation fault.

Does anyone have any idea what the problem might be?

Thank you,
RazvanN

On Wednesday, 24 May 2017 at 10:40:42 UTC, RazvanN wrote:
> Does anyone have any idea what the problem might be?
>
> Thank you,
> RazvanN

Since my emails don't seem to go through in a timely fashion:
> You should also disable the GC.

On Wednesday, 24 May 2017 at 10:57:40 UTC, Mathias Lang wrote:
> On Wednesday, 24 May 2017 at 10:40:42 UTC, RazvanN wrote:
>> Does anyone have any idea what the problem might be?
>>
>> Thank you,
>> RazvanN
>
> Since my emails don't seem to go through in a timely fashion:
>> You should also disable the GC.

Thank you. It does, indeed, solve the problem. But I am curious, why was it segfaulting?

On Thursday, 25 May 2017 at 16:48:00 UTC, RazvanN wrote:
> On Wednesday, 24 May 2017 at 10:57:40 UTC, Mathias Lang wrote:
>> On Wednesday, 24 May 2017 at 10:40:42 UTC, RazvanN wrote:
>>> Does anyone have any idea what the problem might be?
>>>
>>> Thank you,
>>> RazvanN
>>
>> Since my emails don't seem to go through in a timely fashion:
>>> You should also disable the GC.
>
> Thank you. It does, indeed, solve the problem. But I am curious, why was it segfaulting?

DMD uses a mix of allocations strategy.

My guess is that some references to GC-allocated memory are stored into non GC-allocated memory, and those chunks of memory are not added as GC roots. So when the GC decides to collect memory, it thinks those objects are dead and recycle them. From there, you end up with a dangling pointer.