April 14, 2012
Le 13/04/2012 22:31, Trass3r a écrit :
>> I think we've got the lexer and parser completely separate from
>> most of the rest of the codebase (like the codegen), due to
>> repeated requests from people who wanted to use these parts for
>> IDEs and other tools.
>
> Still some things to learn from Clang though.
> e.g. it still directly builds an AST instead of using some kind of
> interface.
>

I'm very interested in what you mean here. Do you have a link or can you write an explaination ?

> btw, why do you use exceptions for compiler errors instead of a cheaper
> and more sophisticated system?
>

Currently it is exception only, but this is a known issue.
April 14, 2012
Le 14/04/2012 12:35, Manu a écrit :
> On 13 April 2012 18:25, Jakob Ovrum <jakobovrum@gmail.com
> <mailto:jakobovrum@gmail.com>> wrote:
>
>     On Friday, 13 April 2012 at 13:08:51 UTC, deadalnix wrote:
>
>         SDC have a lot of theses, and I proposed a similar stuff for its
>         evolution. I think it is easier for SDC than it is for dmd
>         considering the codebase of both.
>
>
>     I think we've got the lexer and parser completely separate from
>     most of the rest of the codebase (like the codegen), due to
>     repeated requests from people who wanted to use these parts for
>     IDEs and other tools.
>
>     I've yet to see anyone actually go through with using it though,
>     possibly because there is no documentation for a lot of it.
>     Documenting these parts fully into something of a public API and
>     then putting it online is definitely on the todo list. Perhaps
>     there would be more motivation to do this rather than work on
>     something else if someone actually tried using SDC in their
>     project instead of just talking about it, so it's kind of a
>     catch-22.
>
>     That said, the parser is currently evolving alongside the
>     codegen. When we want to start implementing new parts of the
>     language, we iteratively add it to the parser, hence it's not
>     complete. It's very easy to work with though and it's mostly a
>     menial task (although it's kind of fun to produce beautiful
>     parser errors :P).
>
>     Anyway, for anyone interested, you can find us on Github and
>     #d.sdc on FreeNode.
>
>
> Just out of curiosity, why would anyone write a code gen these days?
> With projects like LLVM, which can perform great codegen to basically
> any architecture, you'd be crazy not to use that...
> Surely 90% of the value of writing your own D compiler would be the
> unique front end, which may have different design principles (like use
> as a lib, usable on tools and stuff as discussed here) ?
>
> I'm sorry to say, if you write your own codegen, I would never use your
> compiler. If you use LLVM in the back end, I'll definitely give it a
> look, then merit will depend entirely on the front end (speed,
> flexibility, quality of error messages, runtime error detection, etc).
> I know it's a fun exercise to write a codegen, so from an educational
> perspective, sure, it's valuable to you as a programmer, but I don't
> think it helps your project at all.

Codegen is done by LLVM in SDC, but you still need some codegen glue.
April 14, 2012
On 14 April 2012 16:51, deadalnix <deadalnix@gmail.com> wrote:

> Le 14/04/2012 12:35, Manu a écrit :
>
>> On 13 April 2012 18:25, Jakob Ovrum <jakobovrum@gmail.com <mailto:jakobovrum@gmail.com>> wrote:
>>
>>    On Friday, 13 April 2012 at 13:08:51 UTC, deadalnix wrote:
>>
>>        SDC have a lot of theses, and I proposed a similar stuff for its
>>        evolution. I think it is easier for SDC than it is for dmd
>>        considering the codebase of both.
>>
>>
>>    I think we've got the lexer and parser completely separate from
>>    most of the rest of the codebase (like the codegen), due to
>>    repeated requests from people who wanted to use these parts for
>>    IDEs and other tools.
>>
>>    I've yet to see anyone actually go through with using it though,
>>    possibly because there is no documentation for a lot of it.
>>    Documenting these parts fully into something of a public API and
>>    then putting it online is definitely on the todo list. Perhaps
>>    there would be more motivation to do this rather than work on
>>    something else if someone actually tried using SDC in their
>>    project instead of just talking about it, so it's kind of a
>>    catch-22.
>>
>>    That said, the parser is currently evolving alongside the
>>    codegen. When we want to start implementing new parts of the
>>    language, we iteratively add it to the parser, hence it's not
>>    complete. It's very easy to work with though and it's mostly a
>>    menial task (although it's kind of fun to produce beautiful
>>    parser errors :P).
>>
>>    Anyway, for anyone interested, you can find us on Github and
>>    #d.sdc on FreeNode.
>>
>>
>> Just out of curiosity, why would anyone write a code gen these days?
>> With projects like LLVM, which can perform great codegen to basically
>> any architecture, you'd be crazy not to use that...
>> Surely 90% of the value of writing your own D compiler would be the
>> unique front end, which may have different design principles (like use
>> as a lib, usable on tools and stuff as discussed here) ?
>>
>> I'm sorry to say, if you write your own codegen, I would never use your compiler. If you use LLVM in the back end, I'll definitely give it a look, then merit will depend entirely on the front end (speed, flexibility, quality of error messages, runtime error detection, etc). I know it's a fun exercise to write a codegen, so from an educational perspective, sure, it's valuable to you as a programmer, but I don't think it helps your project at all.
>>
>
> Codegen is done by LLVM in SDC, but you still need some codegen glue.
>

Ah okay, cool. NM me then :P


April 14, 2012
>> Still some things to learn from Clang though.
>> e.g. it still directly builds an AST instead of using some kind of
>> interface.
>>
>
> I'm very interested in what you mean here. Do you have a link or can you write an explaination ?

Clang decouples the parser from AST construction by letting the parser take an interface class that has a virtual method for everything that is parsed along the lines of ActOnStartOfFunctionDef().
So a client that doesn't need an AST doesn't have to build one.

Don't know if it's properly implemented in Clang now though.
April 14, 2012
Le 14/04/2012 18:26, Trass3r a écrit :
>>> Still some things to learn from Clang though.
>>> e.g. it still directly builds an AST instead of using some kind of
>>> interface.
>>>
>>
>> I'm very interested in what you mean here. Do you have a link or can
>> you write an explaination ?
>
> Clang decouples the parser from AST construction by letting the parser
> take an interface class that has a virtual method for everything that is
> parsed along the lines of ActOnStartOfFunctionDef().
> So a client that doesn't need an AST doesn't have to build one.
>
> Don't know if it's properly implemented in Clang now though.

Ho I see ! That is brilliant !
April 15, 2012
"Jacob Carlborg" <doob@me.com> wrote in message news:jm9ulg$ui3$1@digitalmars.com...
>
> * Refactoring
>

* Make DustMite even more awesomer by attempting to factor out functions, etc.


April 16, 2012
On 4/13/12 9:10 PM, deadalnix wrote:
> Le 13/04/2012 11:58, Ary Manzana a écrit :
>> Having a D compiler available as a library will (at least) give these
>> benefits:
>>
>> 1. Can be used by an IDE: D is statically typed and so an IDE can
>> benefit a lot from this. The features Descent had, as far as I remember,
>> were:
>> 1.1. Outline
>> 1.2. Autocompletion
>> 1.3. Type Hierarchy
>> 1.4. Syntax and semantic errors, showing not only the line number but
>> also column numbers if it makes sense
>> 1.5. Automatic import inclusion (say, typing writefln and getting a list
>> of modules that provide that symbol)
>> 1.6. Compile-time view: replace auto with the inferred type, insert
>> mixins into scope, rewrite operator overloads and other lowerings (but
>> I'm not sure this point is really useful)
>> 1.7. Determine, given a set of versions and flags, which branches of
>> static ifs are used/unused
>> 1.8. Open declaration
>> 1.9. Show implementations (of an interface, of interface's method or,
>> abstract methods, or method overrides).
>> 1.10. Propose to override a method (you type some letters and then hit
>> some key combination and get a list of methods to override)
>> 1.11. Get the code of a template when instantiated.
>> 2. Can be used to build better doc generators: one that shows known
>> subclasses or interface implementation, shows inherited methods, type
>> hierarchy.
>> 3. Can be used for lints and other such tools.
>>
>> As you can see, a simple lexer/parser built into an IDE, doc generator
>> or lint will just give basic features but will never achieve something
>> exceptionally good if it lacks the full semantic knowledge of the code.
>>
>> I'll write a list of things I'd like this compiler-as-library to have,
>> but please help me make it bigger :-)
>>
>> * Don't use global variables (DMD is just thought to be run once, so
>> when used as a library it can just be used, well, once)
>> * Provide a lexer which gives line numbers and column numbers
>> (beginning, end)
>> * Provide a parser with the same features
>> * The semantic phase should not discard any information found while
>> parsing. For example when DMD resolves a type it recursively resolves
>> aliasing and keeps the last one. An example:
>>
>> alias int foo;
>> alias foo* bar;
>>
>> bar something() { ... }
>>
>> It would be nice if "bar", after semantic analysis is done, carries the
>> information that bar is "foo*" and that "foo" is "int". Also that
>> something's return type is "bar", not "int*".
>> * Provide errors and warnings that have line numbers as well as column
>> numbers.
>> * Allow to parse the top-level definitions of a module. Whit this I mean
>> skipping function bodies. At least Descent first built a the outline of
>> the whole project by doing this. This mode should also allow specifying
>> a location as a target, and if that location falls inside a function
>> body then it's contents are returned (useful when editing a file, so you
>> can get the outline as well as semantic info of the function currently
>> being edited, which will never affect semantic in other parts of the
>> module). This will dramatically speed up the editor.
>> * Don't stop parsing on errors (I think DMD already does this).
>> * Provide a visitor class. If possible, use visitors to implement
>> semantic analysis. The visitor will make it super easy to implement
>> lints and to generate documentation.
>
> SDC have a lot of theses, and I proposed a similar stuff for its
> evolution. I think it is easier for SDC than it is for dmd considering
> the codebase of both.

Cool! SDC is the way to go. Let's focus our efforts on that project. :-)

One thing I saw in the code is that global variables are used in it (specially in the sdc.aglobal module).

Also, the lexer returns a TokenStream which contains the full array of tokens. This is very slow compared to returning them as they are scanned, when requested. But I guess this is easy to fix as it's hidden behind the TokenStream interface.

I tried to compile a simple file:

void main() {}

> ./bin/sdc main.d
core.exception.AssertError@sdc.gen.sdcmodule(560): Assertion failure
----------------
5   sdc                                 0x000000010b87aa2a _d_assertm + 42
6   sdc                                 0x000000010b4e2daa void sdc.gen.sdcmodule.__assert(int) + 26
7   sdc                                 0x000000010b55d19c void sdc.gen.sdcmodule.Store.addFunction(sdc.gen.sdcfunction.Function) + 92
8   sdc                                 0x000000010b55d75b void sdc.gen.sdcmodule.Scope.add(immutable(char)[], sdc.gen.sdcfunction.Function) + 123
9   sdc                                 0x000000010b54ee36 void sdc.gen.declaration.declareFunctionDeclaration(sdc.ast.declaration.FunctionDeclaration, sdc.ast.sdcmodule.DeclarationDefinition, sdc.gen.sdcmodule.Module) + 1278
10  sdc                                 0x000000010b54e4b2 void sdc.gen.declaration.declareDeclaration(sdc.ast.declaration.Declaration, sdc.ast.sdcmodule.DeclarationDefinition, sdc.gen.sdcmodule.Module) + 146
11  sdc                                 0x000000010b54d12e void sdc.gen.base.genDeclarationDefinition(sdc.ast.sdcmodule.DeclarationDefinition, sdc.gen.sdcmodule.Module, ulong) + 374
12  sdc                                 0x000000010b54c8d0 void sdc.gen.base.resolveDeclarationDefinitionList(sdc.ast.sdcmodule.DeclarationDefinition[], sdc.gen.sdcmodule.Module, sdc.gen.type.Type) + 564
13  sdc                                 0x000000010b54c439 sdc.gen.sdcmodule.Module sdc.gen.base.genModule(sdc.ast.sdcmodule.Module, sdc.aglobal.TranslationUnit) + 157
14  sdc                                 0x000000010b53432d int sdc.sdc.realmain(immutable(char)[][]) + 3081
15  sdc                                 0x000000010b4e283e _Dmain + 54
16  sdc                                 0x000000010b87b415 extern (C) int rt.dmain2.main(int, char**).void runMain() + 29
17  sdc                                 0x000000010b87adc5 extern (C) int rt.dmain2.main(int, char**).void tryExec(scope void delegate()) + 45
18  sdc                                 0x000000010b87b467 extern (C) int rt.dmain2.main(int, char**).void runAll() + 63
19  sdc                                 0x000000010b87adc5 extern (C) int rt.dmain2.main(int, char**).void tryExec(scope void delegate()) + 45
20  sdc                                 0x000000010b87ad48 main + 232
21  sdc                                 0x000000010b4e2694 start + 52
22  ???                                 0x0000000000000002 0x0 + 2
----------------

I couldn't find the line of the assertion. What are all those "+146" and "+54" about?
April 16, 2012
On 2012-04-16 04:35, Ary Manzana wrote:
> On 4/13/12 9:10 PM, deadalnix wrote:
>> SDC have a lot of theses, and I proposed a similar stuff for its
>> evolution. I think it is easier for SDC than it is for dmd considering
>> the codebase of both.
>
> Cool! SDC is the way to go. Let's focus our efforts on that project. :-)

I noticed that SDC is licensed under the GPL license :( That is not compatible with, for example, EPL used by Eclipse.

-- 
/Jacob Carlborg
April 16, 2012
On Monday, 16 April 2012 at 06:37:00 UTC, Jacob Carlborg wrote:
> On 2012-04-16 04:35, Ary Manzana wrote:
>> On 4/13/12 9:10 PM, deadalnix wrote:
>>> SDC have a lot of theses, and I proposed a similar stuff for its
>>> evolution. I think it is easier for SDC than it is for dmd considering
>>> the codebase of both.
>>
>> Cool! SDC is the way to go. Let's focus our efforts on that project. :-)
>
> I noticed that SDC is licensed under the GPL license :( That is not compatible with, for example, EPL used by Eclipse.

We are changing the license soon, with BSD/MIT in mind. I am really just waiting for Bernard to make the change, we've gotten permission from all contributors as far as I know.
April 16, 2012
On 2012-04-16 09:00, Jakob Ovrum wrote:

> We are changing the license soon, with BSD/MIT in mind. I am really just
> waiting for Bernard to make the change, we've gotten permission from all
> contributors as far as I know.

Ok, that sounds good. I would prefer Boost but I guess BSD/MIT is also ok.

-- 
/Jacob Carlborg