On Sunday, October 23, 2011 22:27:17 Gor Gyolchanyan wrote:
> cool!
> I really _REALY_ want to get a working D front-end into Phobos.
> preferably during my lifetime.
> Obviously there's some work already done in this area.
> I'd like us to chip in and discuss this in-depth and come up with a
> plan to do this.
> AFAIK everyone was in favor of a front-end as a library solution.

The two proposed approaches at the moment are to port dmd's frontend to D for Phobos and to write a template-based generative, generic solution. The first would obviously be D-specific. The second would be for _any_ language. In either case, any solution needs to be broken up into a lexer, parser, etc. in manner which allows you to use only the pieces that you need for a particular application.

And so unless you're going with the "port dmd frontend to D" approach, any solution really needs to be generic and not D-specific. Naturally, we'll provied whatever's needed to generate the D lexer, parser, etc. as part of Phobos, but the lexer, paresr, etc. need to be generic and generative rather than D-specfiic.

http://www.mail-archive.com/digitalmars-d@puremagic.com/msg40358.html

- Jonathan M Davis

Yes. a general-purpose parsing tool would be great.
I don't think, that porting DMD's front-end is a good idea, because
it's far from being generic or modular.
We're gonna have to start from scratch.

Optionally, it would be really great to make it CTFE-able, so it could be used to provide a complete compile-time reflection.

On Sun, Oct 23, 2011 at 10:40 PM, Jonathan M Davis <jmdavisProg@gmx.com> wrote:
> On Sunday, October 23, 2011 22:27:17 Gor Gyolchanyan wrote:
>> cool!
>> I really _REALY_ want to get a working D front-end into Phobos.
>> preferably during my lifetime.
>> Obviously there's some work already done in this area.
>> I'd like us to chip in and discuss this in-depth and come up with a
>> plan to do this.
>> AFAIK everyone was in favor of a front-end as a library solution.
>
> The two proposed approaches at the moment are to port dmd's frontend to D for Phobos and to write a template-based generative, generic solution. The first would obviously be D-specific. The second would be for _any_ language. In either case, any solution needs to be broken up into a lexer, parser, etc. in manner which allows you to use only the pieces that you need for a particular application.
>
> And so unless you're going with the "port dmd frontend to D" approach, any solution really needs to be generic and not D-specific. Naturally, we'll provied whatever's needed to generate the D lexer, parser, etc. as part of Phobos, but the lexer, paresr, etc. need to be generic and generative rather than D-specfiic.
>
> http://www.mail-archive.com/digitalmars-d@puremagic.com/msg40358.html
>
> - Jonathan M Davis
>

On Sunday, October 23, 2011 23:56:34 Gor Gyolchanyan wrote:
> Yes. a general-purpose parsing tool would be great.
> I don't think, that porting DMD's front-end is a good idea, because
> it's far from being generic or modular.
> We're gonna have to start from scratch.

It's generic enough to lex/parse D, which is it's whole purpose. I don't think that much of _anyone_ is suggesting that dmd's frontend's lexer and parser be ported to D _instead of_ having a more generic solution. It's a solution which has the advantage of creating a lexer and parser which are close to dmd's frontend so that it's easy to port changes and fixes from one to the other. A more generic solution needs to be created regardless.

- Jonathan M Davis

On Sun, 23 Oct 2011 21:56:34 +0200, Gor Gyolchanyan <gor.f.gyolchanyan@gmail.com> wrote:

> Yes. a general-purpose parsing tool would be great.
> I don't think, that porting DMD's front-end is a good idea, because
> it's far from being generic or modular.
> We're gonna have to start from scratch.
>
> Optionally, it would be really great to make it CTFE-able, so it could
> be used to provide a complete compile-time reflection.
>
The issue being, that most approaches to generic parsing
suffer from unspecific error handling, bad integration with client code
and the need for dynamic typing in the parse tree.
Language integrated approaches as boost::spirit mostly fail for
anything but simple and regular parsing (e.g. CSV).

My personal conclusion is that I will waste no more time on trying to use or write
general parsers, it's much more pleasant to write recursive decent parsers.

martin

> On Sun, Oct 23, 2011 at 10:40 PM, Jonathan M Davis <jmdavisProg@gmx.com> wrote:
>> On Sunday, October 23, 2011 22:27:17 Gor Gyolchanyan wrote:
>>> cool!
>>> I really _REALY_ want to get a working D front-end into Phobos.
>>> preferably during my lifetime.
>>> Obviously there's some work already done in this area.
>>> I'd like us to chip in and discuss this in-depth and come up with a
>>> plan to do this.
>>> AFAIK everyone was in favor of a front-end as a library solution.
>>
>> The two proposed approaches at the moment are to port dmd's frontend to D for
>> Phobos and to write a template-based generative, generic solution. The first
>> would obviously be D-specific. The second would be for _any_ language. In
>> either case, any solution needs to be broken up into a lexer, parser, etc. in
>> manner which allows you to use only the pieces that you need for a particular
>> application.
>>
>> And so unless you're going with the "port dmd frontend to D" approach, any
>> solution really needs to be generic and not D-specific. Naturally, we'll
>> provied whatever's needed to generate the D lexer, parser, etc. as part of
>> Phobos, but the lexer, paresr, etc. need to be generic and generative rather
>> than D-specfiic.
>>
>> http://www.mail-archive.com/digitalmars-d@puremagic.com/msg40358.html
>>
>> - Jonathan M Davis

Making a general-purpose parser is the secondary issue. The primary issue is having a D parser.


On Mon, Oct 24, 2011 at 11:47 AM, Martin Nowak <dawg@dawgfoto.de> wrote:
> On Sun, 23 Oct 2011 21:56:34 +0200, Gor Gyolchanyan <gor.f.gyolchanyan@gmail.com> wrote:
>
>> Yes. a general-purpose parsing tool would be great.
>> I don't think, that porting DMD's front-end is a good idea, because
>> it's far from being generic or modular.
>> We're gonna have to start from scratch.
>>
>> Optionally, it would be really great to make it CTFE-able, so it could be used to provide a complete compile-time reflection.
>>
> The issue being, that most approaches to generic parsing
> suffer from unspecific error handling, bad integration with client code
> and the need for dynamic typing in the parse tree.
> Language integrated approaches as boost::spirit mostly fail for
> anything but simple and regular parsing (e.g. CSV).
>
> My personal conclusion is that I will waste no more time on trying to use or
> write
> general parsers, it's much more pleasant to write recursive decent parsers.
>
> martin
>
>> On Sun, Oct 23, 2011 at 10:40 PM, Jonathan M Davis <jmdavisProg@gmx.com> wrote:
>>>
>>> On Sunday, October 23, 2011 22:27:17 Gor Gyolchanyan wrote:
>>>>
>>>> cool!
>>>> I really _REALY_ want to get a working D front-end into Phobos.
>>>> preferably during my lifetime.
>>>> Obviously there's some work already done in this area.
>>>> I'd like us to chip in and discuss this in-depth and come up with a
>>>> plan to do this.
>>>> AFAIK everyone was in favor of a front-end as a library solution.
>>>
>>> The two proposed approaches at the moment are to port dmd's frontend to D
>>> for
>>> Phobos and to write a template-based generative, generic solution. The
>>> first
>>> would obviously be D-specific. The second would be for _any_ language. In
>>> either case, any solution needs to be broken up into a lexer, parser,
>>> etc. in
>>> manner which allows you to use only the pieces that you need for a
>>> particular
>>> application.
>>>
>>> And so unless you're going with the "port dmd frontend to D" approach,
>>> any
>>> solution really needs to be generic and not D-specific. Naturally, we'll
>>> provied whatever's needed to generate the D lexer, parser, etc. as part
>>> of
>>> Phobos, but the lexer, paresr, etc. need to be generic and generative
>>> rather
>>> than D-specfiic.
>>>
>>> http://www.mail-archive.com/digitalmars-d@puremagic.com/msg40358.html
>>>
>>> - Jonathan M Davis
>

Am 24.10.2011, 09:55 Uhr, schrieb Gor Gyolchanyan <gor.f.gyolchanyan@gmail.com>:

> Making a general-purpose parser is the secondary issue. The primary
> issue is having a D parser.

I feel like we should make a list of feature that everyone wants the parser to support and go the better safe than sorry approach. It would be a pity if the frontend could do only one thing well, but not be used as a real compiler frontend for example.

o  generates a representation that can be used by a backend: compiler
o  internal representation can be updated incrementally: IDE
o  understands multiple versions of D: "Gofix"
o  can be queried about inferred stuff: pureness, auto, transitive attributes: IDE
o  allows iteration over code elements or hooks/callbacks (e.g. onEnumDecleration(...)): DDoc, "Gofix"

While there has been work done for a compiler I didn't look at the code to see if it is usable for any other use case.

Right.
A generic parser can be created, but it should be extremely flexible,
because parsing is not a trivial task and every specific parser has
it's own tricks, that should be implemented when using the generic
parser.

We could also make tools, that would greatly aid in development of
parsers: state machines.
it would be great to implement a generic state machine, which could be
used with the generic parser to quickly and easily implement a very
specific parser.
Being able to use it at compile-time will be a tremendous improvement,
because then you could easily make a parser for your favourite
language and interpret it at compile-time to generate D code.

On Tue, Oct 25, 2011 at 4:59 AM, Marco Leise <Marco.Leise@gmx.de> wrote:
> Am 24.10.2011, 09:55 Uhr, schrieb Gor Gyolchanyan <gor.f.gyolchanyan@gmail.com>:
>
>> Making a general-purpose parser is the secondary issue. The primary issue is having a D parser.
>
> I feel like we should make a list of feature that everyone wants the parser to support and go the better safe than sorry approach. It would be a pity if the frontend could do only one thing well, but not be used as a real compiler frontend for example.
>
> o  generates a representation that can be used by a backend: compiler
> o  internal representation can be updated incrementally: IDE
> o  understands multiple versions of D: "Gofix"
> o  can be queried about inferred stuff: pureness, auto, transitive
> attributes: IDE
> o  allows iteration over code elements or hooks/callbacks (e.g.
> onEnumDecleration(...)): DDoc, "Gofix"
>
> While there has been work done for a compiler I didn't look at the code to see if it is usable for any other use case.
>

Am 25.10.2011, 08:20 Uhr, schrieb Gor Gyolchanyan <gor.f.gyolchanyan@gmail.com>:

> [...] then you could easily make a parser for your favourite
> language and interpret it at compile-time to generate D code.

I don't get that one ;)