Thread overview
[dmd-internals] Walking through the template function instantiation
Apr 09, 2019
Stefanos Baziotis
Apr 09, 2019
Mike Franklin
Apr 09, 2019
Mike Franklin
Apr 09, 2019
Stefanos Baziotis
Apr 09, 2019
Jacob Carlborg
April 09, 2019
Hi, my name is Stefanos. As part of this [1] thread, I would like
to solve issue #16486 for GSoC.
A comment from Steven Schveighoffer on the issue:
"The issue is that the compiler doesn't replace the alias until AFTER
instantiation."

I don't know if this is the right place to ask, but could
someone help me with the understanding of semantic analysis in DMD?

Best regards,
Stefanos Baziotis

[1] https://forum.dlang.org/thread/pvhacqomqgnhmqienfpi@forum.dlang.org?page=3

_______________________________________________
dmd-internals mailing list
dmd-internals@puremagic.com
http://lists.puremagic.com/mailman/listinfo/dmd-internals


April 09, 2019
That's an awfully broad question.  I'm no expert in DMD, but I do dabble in the source code often.

I'd start here: https://wiki.dlang.org/DMD_Source_Guide  The information there is old but still relevant.

This is my mental model and understanding

source code --> lexer.d --> tokesn
tokens --> parse.d --> expressions (see expression.d)
expressions --> expressionsem.d --> lowered expressions
lowered expressions --> e2ir --> intermediate representation (I'm awufully vauge on this)
After that it's in the backend, and that's pretty much a black box to me.

That, however, is a gross oversimplification.  If you want to browse the code that implements the semantic phase you'll want to look at any file of the form xxxsem.d (e.g. dsymbolsem.d, expressionsem.d, typesem.d).  For the task at hand, issue 16486, you'll probably find the fix somewhere in there, but don't quote me on that.  Search for functions with `resolve` in the name.  Though the fix may also reside in one of the `visit` methods of one of the visitors in those files.

Tracing with `printf` is your friend, which is why you see so many commented `printf` statements in the source code.  You should be able to print out almost anything with with the `toChars()` method (e.g. `printf("%s\n", whatever.toChars());`)

Mike

    On Tuesday, April 9, 2019, 9:38:25 AM GMT+9, Stefanos Baziotis via dmd-internals <dmd-internals@puremagic.com> wrote:

 Hi, my name is Stefanos. As part of this [1] thread, I would like
to solve issue #16486 for GSoC.
A comment from Steven Schveighoffer on the issue:
"The issue is that the compiler doesn't replace the alias until AFTER
instantiation."

I don't know if this is the right place to ask, but could
someone help me with the understanding of semantic analysis in DMD?

Best regards,
Stefanos Baziotis

[1] https://forum.dlang.org/thread/pvhacqomqgnhmqienfpi@forum.dlang.org?page=3

_______________________________________________
dmd-internals mailing list
dmd-internals@puremagic.com
http://lists.puremagic.com/mailman/listinfo/dmd-internals


April 09, 2019
 Another technique I often employ is to grep for the error message, in the case of #16486 "cannot deduce function from argument types".  Once the location of where the error message is emitted, I work back from there to answer the question "why?".
Here's where the error is emitted from:  https://github.com/dlang/dmd/blob/e58579a73438e62a16c0c43dc33ce8d36b520edd/src/dmd/func.d#L2788-L2790

Unfortunately, that resolution code is very difficult to follow and will likely require some lengthy debugging.

Mike




April 09, 2019

> On 9 Apr 2019, at 03:58, Mike Franklin via dmd-internals <dmd-internals@puremagic.com> wrote:
> 
> That's an awfully broad question.  I'm no expert in DMD, but I do dabble in the source code often.
> 
> I'd start here: https://wiki.dlang.org/DMD_Source_Guide  The information there is old but still relevant.
> 
> This is my mental model and understanding
> 
> source code --> lexer.d --> tokesn
> tokens --> parse.d --> expressions (see expression.d)
> expressions --> expressionsem.d --> lowered expressions
> lowered expressions --> e2ir --> intermediate representation (I'm awufully vauge on this)
> After that it's in the backend, and that's pretty much a black box to me.

I can add to the above that the parser is a subclass of the lexer. The parser will retrieve the next token when it needs to, to continue parsing. The parsing is initiated from `Module.parse` in `dmodule.d` [1] which will then call `Parser.parseModule` in `parse.d`. After that the semantic analyzing phase will begin with the call to `dsymbolSemantic` for each module given to the compiler. This is all available in `mars.d`, which contains the entry point of the compiler.

> Tracing with `printf` is your friend, which is why you see so many commented `printf` statements in the source code.  You should be able to print out almost anything with with the `toChars()` method (e.g. `printf("%s\n", whatever.toChars());`)


For debugging I can also recommend tracing with `std.stdio.writeln` and `std.stdio.writefln`.  Although it's not allowed to use Phobos in the compiler, it’s perfectly fine to use it during development and debugging. Just make sure that all traces of Phobos are gone when making a PR. Most classes will have a `toString` method which is doing exactly the same thing as `toChars`, but it will return a D string instead of a C string. `writeln` knows about this and will call `toString` automatically. When printing an enum value with `writeln`, it will print the name of the enum member instead of it’s value, I like that very much. It’s also much easier to print D string with `writeln` than with `printf`. This is especially useful when we’re trying to get rid of all C strings and replace them with D strings.

[1] https://github.com/dlang/dmd/blob/b4429221e0b0024e5f0b99e084e075f02972e19a/src/dmd/dmodule.d#L654

-- 
/Jacob Carlborg



April 09, 2019
> Here's where the error is emitted from: https://github.com/dlang/dmd/blob/e58579a73438e62a16c0c43dc33ce8d36b520edd/src/dmd/func.d#L>
2788-L2790

Thanks for the help Mike! Didn't see your message till now and so
yesterday, I also stopped here by intuitively doing pretty much the same
steps. So here was my first time in dmd codebase in case any newcomer
finds it helpful:
- The main() function is in mars.d, always good to know. :P
- First, I changed the usage message in usage() function.
- Then, I changed some stuff on the lexer, which is in lexer.d and then on
the parser. Actually, these are 2 things I have wrote many times, so it's
easy for me to understand that code but I think for everyone, the lexer is
the next easiest big thing to experiment with. Also, understanding the
parser (or parts of it), makes you understand the language a little
better.

> expressions --> expressionsem.d --> lowered expressions lowered expressions --> e2ir --> intermediate representation

I did not figure that. I guess that expressions -> lowered expressions has somewhere the 3 semantic passes that you see described in the dmd source code guide. [1]

Searching for "semantic" in mars.d will get you here [2].
in which semantic analysis is done for every module. Now, from there on
the not-so-fun part starts. :P

Following dSymbolSemantic, will probably lead you here [3] in dsymbolsem.d
which will in turn lead you here [4] which led me here [5]
because of the issue. And... I didn't understand a thing.
So then I said, ok, let's do a "search the whole project" search, to find
the error message that Mike found. Then, disappointed I went to sleep. :P

This error message is when we try to resolve a call, as one would expect,
i.e. when we try to instantiate the function. However, I guess one has to
understand how also the function declaration is analysed and possibly a
bunch of
other things that I don't know. From my perspective, it will take more than
extensive debugging to get a good understanding on the problem. Let me point
that I have not written a very complex semantic analyzer, but I do know
the basics of semantic analysis (and the visitor pattern coming with it) and
still, that was very rough.

Any help is welcome!
Btw, if any newcomer sees this and wants help with things I am somewhat
comfortable (i.e. lexing, parsing), please don't hesitate to ask!

- Stefanos


[1] https://wiki.dlang.org/DMD_Source_Guide#Compilation_cycle
[2] https://github.com/dlang/dmd/blob/master/src/dmd/mars.d#L566
[3] https://github.com/dlang/dmd/blob/master/src/dmd/dsymbolsem.d#L554
[4] https://github.com/dlang/dmd/blob/master/src/dmd/dsymbolsem.d#L632
[5] https://github.com/dlang/dmd/blob/master/src/dmd/dsymbolsem.d#L3062

P.S. I guess using the "Reply" in forum is not the right way to reply. I
was trying to do that
for 15 minutes and I thought I should respond to the mail which I hope,
will get the message posted.


_______________________________________________
dmd-internals mailing list
dmd-internals@puremagic.com
http://lists.puremagic.com/mailman/listinfo/dmd-internals