April 14, 2012 Re: D Compiler as a Library | ||||
---|---|---|---|---|
| ||||
Posted in reply to Trass3r | Le 13/04/2012 22:31, Trass3r a écrit : >> I think we've got the lexer and parser completely separate from >> most of the rest of the codebase (like the codegen), due to >> repeated requests from people who wanted to use these parts for >> IDEs and other tools. > > Still some things to learn from Clang though. > e.g. it still directly builds an AST instead of using some kind of > interface. > I'm very interested in what you mean here. Do you have a link or can you write an explaination ? > btw, why do you use exceptions for compiler errors instead of a cheaper > and more sophisticated system? > Currently it is exception only, but this is a known issue. |
April 14, 2012 Re: D Compiler as a Library | ||||
---|---|---|---|---|
| ||||
Posted in reply to Manu | Le 14/04/2012 12:35, Manu a écrit :
> On 13 April 2012 18:25, Jakob Ovrum <jakobovrum@gmail.com
> <mailto:jakobovrum@gmail.com>> wrote:
>
> On Friday, 13 April 2012 at 13:08:51 UTC, deadalnix wrote:
>
> SDC have a lot of theses, and I proposed a similar stuff for its
> evolution. I think it is easier for SDC than it is for dmd
> considering the codebase of both.
>
>
> I think we've got the lexer and parser completely separate from
> most of the rest of the codebase (like the codegen), due to
> repeated requests from people who wanted to use these parts for
> IDEs and other tools.
>
> I've yet to see anyone actually go through with using it though,
> possibly because there is no documentation for a lot of it.
> Documenting these parts fully into something of a public API and
> then putting it online is definitely on the todo list. Perhaps
> there would be more motivation to do this rather than work on
> something else if someone actually tried using SDC in their
> project instead of just talking about it, so it's kind of a
> catch-22.
>
> That said, the parser is currently evolving alongside the
> codegen. When we want to start implementing new parts of the
> language, we iteratively add it to the parser, hence it's not
> complete. It's very easy to work with though and it's mostly a
> menial task (although it's kind of fun to produce beautiful
> parser errors :P).
>
> Anyway, for anyone interested, you can find us on Github and
> #d.sdc on FreeNode.
>
>
> Just out of curiosity, why would anyone write a code gen these days?
> With projects like LLVM, which can perform great codegen to basically
> any architecture, you'd be crazy not to use that...
> Surely 90% of the value of writing your own D compiler would be the
> unique front end, which may have different design principles (like use
> as a lib, usable on tools and stuff as discussed here) ?
>
> I'm sorry to say, if you write your own codegen, I would never use your
> compiler. If you use LLVM in the back end, I'll definitely give it a
> look, then merit will depend entirely on the front end (speed,
> flexibility, quality of error messages, runtime error detection, etc).
> I know it's a fun exercise to write a codegen, so from an educational
> perspective, sure, it's valuable to you as a programmer, but I don't
> think it helps your project at all.
Codegen is done by LLVM in SDC, but you still need some codegen glue.
|
April 14, 2012 Re: D Compiler as a Library | ||||
---|---|---|---|---|
| ||||
Posted in reply to deadalnix Attachments:
| On 14 April 2012 16:51, deadalnix <deadalnix@gmail.com> wrote:
> Le 14/04/2012 12:35, Manu a écrit :
>
>> On 13 April 2012 18:25, Jakob Ovrum <jakobovrum@gmail.com <mailto:jakobovrum@gmail.com>> wrote:
>>
>> On Friday, 13 April 2012 at 13:08:51 UTC, deadalnix wrote:
>>
>> SDC have a lot of theses, and I proposed a similar stuff for its
>> evolution. I think it is easier for SDC than it is for dmd
>> considering the codebase of both.
>>
>>
>> I think we've got the lexer and parser completely separate from
>> most of the rest of the codebase (like the codegen), due to
>> repeated requests from people who wanted to use these parts for
>> IDEs and other tools.
>>
>> I've yet to see anyone actually go through with using it though,
>> possibly because there is no documentation for a lot of it.
>> Documenting these parts fully into something of a public API and
>> then putting it online is definitely on the todo list. Perhaps
>> there would be more motivation to do this rather than work on
>> something else if someone actually tried using SDC in their
>> project instead of just talking about it, so it's kind of a
>> catch-22.
>>
>> That said, the parser is currently evolving alongside the
>> codegen. When we want to start implementing new parts of the
>> language, we iteratively add it to the parser, hence it's not
>> complete. It's very easy to work with though and it's mostly a
>> menial task (although it's kind of fun to produce beautiful
>> parser errors :P).
>>
>> Anyway, for anyone interested, you can find us on Github and
>> #d.sdc on FreeNode.
>>
>>
>> Just out of curiosity, why would anyone write a code gen these days?
>> With projects like LLVM, which can perform great codegen to basically
>> any architecture, you'd be crazy not to use that...
>> Surely 90% of the value of writing your own D compiler would be the
>> unique front end, which may have different design principles (like use
>> as a lib, usable on tools and stuff as discussed here) ?
>>
>> I'm sorry to say, if you write your own codegen, I would never use your compiler. If you use LLVM in the back end, I'll definitely give it a look, then merit will depend entirely on the front end (speed, flexibility, quality of error messages, runtime error detection, etc). I know it's a fun exercise to write a codegen, so from an educational perspective, sure, it's valuable to you as a programmer, but I don't think it helps your project at all.
>>
>
> Codegen is done by LLVM in SDC, but you still need some codegen glue.
>
Ah okay, cool. NM me then :P
|
April 14, 2012 Re: D Compiler as a Library | ||||
---|---|---|---|---|
| ||||
Posted in reply to deadalnix | >> Still some things to learn from Clang though.
>> e.g. it still directly builds an AST instead of using some kind of
>> interface.
>>
>
> I'm very interested in what you mean here. Do you have a link or can you write an explaination ?
Clang decouples the parser from AST construction by letting the parser take an interface class that has a virtual method for everything that is parsed along the lines of ActOnStartOfFunctionDef().
So a client that doesn't need an AST doesn't have to build one.
Don't know if it's properly implemented in Clang now though.
|
April 14, 2012 Re: D Compiler as a Library | ||||
---|---|---|---|---|
| ||||
Posted in reply to Trass3r | Le 14/04/2012 18:26, Trass3r a écrit :
>>> Still some things to learn from Clang though.
>>> e.g. it still directly builds an AST instead of using some kind of
>>> interface.
>>>
>>
>> I'm very interested in what you mean here. Do you have a link or can
>> you write an explaination ?
>
> Clang decouples the parser from AST construction by letting the parser
> take an interface class that has a virtual method for everything that is
> parsed along the lines of ActOnStartOfFunctionDef().
> So a client that doesn't need an AST doesn't have to build one.
>
> Don't know if it's properly implemented in Clang now though.
Ho I see ! That is brilliant !
|
April 15, 2012 Re: D Compiler as a Library | ||||
---|---|---|---|---|
| ||||
Posted in reply to Jacob Carlborg | "Jacob Carlborg" <doob@me.com> wrote in message news:jm9ulg$ui3$1@digitalmars.com... > > * Refactoring > * Make DustMite even more awesomer by attempting to factor out functions, etc. |
April 16, 2012 Re: D Compiler as a Library | ||||
---|---|---|---|---|
| ||||
Posted in reply to deadalnix | On 4/13/12 9:10 PM, deadalnix wrote: > Le 13/04/2012 11:58, Ary Manzana a écrit : >> Having a D compiler available as a library will (at least) give these >> benefits: >> >> 1. Can be used by an IDE: D is statically typed and so an IDE can >> benefit a lot from this. The features Descent had, as far as I remember, >> were: >> 1.1. Outline >> 1.2. Autocompletion >> 1.3. Type Hierarchy >> 1.4. Syntax and semantic errors, showing not only the line number but >> also column numbers if it makes sense >> 1.5. Automatic import inclusion (say, typing writefln and getting a list >> of modules that provide that symbol) >> 1.6. Compile-time view: replace auto with the inferred type, insert >> mixins into scope, rewrite operator overloads and other lowerings (but >> I'm not sure this point is really useful) >> 1.7. Determine, given a set of versions and flags, which branches of >> static ifs are used/unused >> 1.8. Open declaration >> 1.9. Show implementations (of an interface, of interface's method or, >> abstract methods, or method overrides). >> 1.10. Propose to override a method (you type some letters and then hit >> some key combination and get a list of methods to override) >> 1.11. Get the code of a template when instantiated. >> 2. Can be used to build better doc generators: one that shows known >> subclasses or interface implementation, shows inherited methods, type >> hierarchy. >> 3. Can be used for lints and other such tools. >> >> As you can see, a simple lexer/parser built into an IDE, doc generator >> or lint will just give basic features but will never achieve something >> exceptionally good if it lacks the full semantic knowledge of the code. >> >> I'll write a list of things I'd like this compiler-as-library to have, >> but please help me make it bigger :-) >> >> * Don't use global variables (DMD is just thought to be run once, so >> when used as a library it can just be used, well, once) >> * Provide a lexer which gives line numbers and column numbers >> (beginning, end) >> * Provide a parser with the same features >> * The semantic phase should not discard any information found while >> parsing. For example when DMD resolves a type it recursively resolves >> aliasing and keeps the last one. An example: >> >> alias int foo; >> alias foo* bar; >> >> bar something() { ... } >> >> It would be nice if "bar", after semantic analysis is done, carries the >> information that bar is "foo*" and that "foo" is "int". Also that >> something's return type is "bar", not "int*". >> * Provide errors and warnings that have line numbers as well as column >> numbers. >> * Allow to parse the top-level definitions of a module. Whit this I mean >> skipping function bodies. At least Descent first built a the outline of >> the whole project by doing this. This mode should also allow specifying >> a location as a target, and if that location falls inside a function >> body then it's contents are returned (useful when editing a file, so you >> can get the outline as well as semantic info of the function currently >> being edited, which will never affect semantic in other parts of the >> module). This will dramatically speed up the editor. >> * Don't stop parsing on errors (I think DMD already does this). >> * Provide a visitor class. If possible, use visitors to implement >> semantic analysis. The visitor will make it super easy to implement >> lints and to generate documentation. > > SDC have a lot of theses, and I proposed a similar stuff for its > evolution. I think it is easier for SDC than it is for dmd considering > the codebase of both. Cool! SDC is the way to go. Let's focus our efforts on that project. :-) One thing I saw in the code is that global variables are used in it (specially in the sdc.aglobal module). Also, the lexer returns a TokenStream which contains the full array of tokens. This is very slow compared to returning them as they are scanned, when requested. But I guess this is easy to fix as it's hidden behind the TokenStream interface. I tried to compile a simple file: void main() {} > ./bin/sdc main.d core.exception.AssertError@sdc.gen.sdcmodule(560): Assertion failure ---------------- 5 sdc 0x000000010b87aa2a _d_assertm + 42 6 sdc 0x000000010b4e2daa void sdc.gen.sdcmodule.__assert(int) + 26 7 sdc 0x000000010b55d19c void sdc.gen.sdcmodule.Store.addFunction(sdc.gen.sdcfunction.Function) + 92 8 sdc 0x000000010b55d75b void sdc.gen.sdcmodule.Scope.add(immutable(char)[], sdc.gen.sdcfunction.Function) + 123 9 sdc 0x000000010b54ee36 void sdc.gen.declaration.declareFunctionDeclaration(sdc.ast.declaration.FunctionDeclaration, sdc.ast.sdcmodule.DeclarationDefinition, sdc.gen.sdcmodule.Module) + 1278 10 sdc 0x000000010b54e4b2 void sdc.gen.declaration.declareDeclaration(sdc.ast.declaration.Declaration, sdc.ast.sdcmodule.DeclarationDefinition, sdc.gen.sdcmodule.Module) + 146 11 sdc 0x000000010b54d12e void sdc.gen.base.genDeclarationDefinition(sdc.ast.sdcmodule.DeclarationDefinition, sdc.gen.sdcmodule.Module, ulong) + 374 12 sdc 0x000000010b54c8d0 void sdc.gen.base.resolveDeclarationDefinitionList(sdc.ast.sdcmodule.DeclarationDefinition[], sdc.gen.sdcmodule.Module, sdc.gen.type.Type) + 564 13 sdc 0x000000010b54c439 sdc.gen.sdcmodule.Module sdc.gen.base.genModule(sdc.ast.sdcmodule.Module, sdc.aglobal.TranslationUnit) + 157 14 sdc 0x000000010b53432d int sdc.sdc.realmain(immutable(char)[][]) + 3081 15 sdc 0x000000010b4e283e _Dmain + 54 16 sdc 0x000000010b87b415 extern (C) int rt.dmain2.main(int, char**).void runMain() + 29 17 sdc 0x000000010b87adc5 extern (C) int rt.dmain2.main(int, char**).void tryExec(scope void delegate()) + 45 18 sdc 0x000000010b87b467 extern (C) int rt.dmain2.main(int, char**).void runAll() + 63 19 sdc 0x000000010b87adc5 extern (C) int rt.dmain2.main(int, char**).void tryExec(scope void delegate()) + 45 20 sdc 0x000000010b87ad48 main + 232 21 sdc 0x000000010b4e2694 start + 52 22 ??? 0x0000000000000002 0x0 + 2 ---------------- I couldn't find the line of the assertion. What are all those "+146" and "+54" about? |
April 16, 2012 Re: D Compiler as a Library | ||||
---|---|---|---|---|
| ||||
Posted in reply to Ary Manzana | On 2012-04-16 04:35, Ary Manzana wrote: > On 4/13/12 9:10 PM, deadalnix wrote: >> SDC have a lot of theses, and I proposed a similar stuff for its >> evolution. I think it is easier for SDC than it is for dmd considering >> the codebase of both. > > Cool! SDC is the way to go. Let's focus our efforts on that project. :-) I noticed that SDC is licensed under the GPL license :( That is not compatible with, for example, EPL used by Eclipse. -- /Jacob Carlborg |
April 16, 2012 Re: D Compiler as a Library | ||||
---|---|---|---|---|
| ||||
Posted in reply to Jacob Carlborg | On Monday, 16 April 2012 at 06:37:00 UTC, Jacob Carlborg wrote:
> On 2012-04-16 04:35, Ary Manzana wrote:
>> On 4/13/12 9:10 PM, deadalnix wrote:
>>> SDC have a lot of theses, and I proposed a similar stuff for its
>>> evolution. I think it is easier for SDC than it is for dmd considering
>>> the codebase of both.
>>
>> Cool! SDC is the way to go. Let's focus our efforts on that project. :-)
>
> I noticed that SDC is licensed under the GPL license :( That is not compatible with, for example, EPL used by Eclipse.
We are changing the license soon, with BSD/MIT in mind. I am really just waiting for Bernard to make the change, we've gotten permission from all contributors as far as I know.
|
April 16, 2012 Re: D Compiler as a Library | ||||
---|---|---|---|---|
| ||||
Posted in reply to Jakob Ovrum | On 2012-04-16 09:00, Jakob Ovrum wrote: > We are changing the license soon, with BSD/MIT in mind. I am really just > waiting for Bernard to make the change, we've gotten permission from all > contributors as far as I know. Ok, that sounds good. I would prefer Boost but I guess BSD/MIT is also ok. -- /Jacob Carlborg |
Copyright © 1999-2021 by the D Language Foundation