On 2012-07-05 15:08, Roman D. Boiko wrote:

> Anyway I propose to enumerate major use cases first.

Haven't we already done that a couple of times.

-- 
/Jacob Carlborg

Is the actual grammar available somewhere?  The online language reference is all we got I guess? DMD doesn't seem to be using yacc/bison either.

On Thu, Jul 5, 2012 at 7:11 AM, Denis Shelomovskij <verylonglogin.reg@gmail.com> wrote:
> There are more and more projects requiring parsing D code (IDE plugins, DCT
> and dfmt now).
>
> We definitely need a _standard_ fast D parser (suitable as tokenizer). We
> already have a good one at least in Visual D. Maybe dmd's parser is faster.
> If so, it can be (easily?) rewritten in D. We also already have some other
> parsers (in C# from Mono-D etc.).
>
> What about to get one and call it standard?
>
> --
> Денис В. Шеломовский
> Denis V. Shelomovskij
>

On Thursday, 5 July 2012 at 15:42:22 UTC, Caligo wrote:
> Is the actual grammar available somewhere?  The online language
> reference is all we got I guess? DMD doesn't seem to be using
> yacc/bison either.
In PEG format, yes (not necessarily correct):
https://github.com/roman-d-boiko/Pegged/blob/dct/pegged/examples/dgrammar.d

I don't know about any others, though.

On Thursday, 5 July 2012 at 15:40:53 UTC, Jacob Carlborg wrote:
> On 2012-07-05 15:08, Roman D. Boiko wrote:
>
>> Anyway I propose to enumerate major use cases first.
>
> Haven't we already done that a couple of times.

Well, we did something like that for DCT... but I doubt that it would fit general needs.

If we had, why haven't they been analyzed, classified, discussed, etc.? Or have they?

On 2012-07-05 18:04, Roman D. Boiko wrote:

> Well, we did something like that for DCT... but I doubt that it would
> fit general needs.

Why wouldn't it.

> If we had, why haven't they been analyzed, classified, discussed, etc.?
> Or have they?

I don't know. Here is what I wrote for DCT:

* IDE integration
* Refactoring tool
* Static analysis
* Compiler
* Doc generating
* Build tool
* DI generating

In general, use cases that can span several compile phases, i.e. lexing, parsing, semantic analysis and so on. Some of these use cases can be broken in to several new use cases at a lower level. Some examples:

IDE integration:

* Syntax highlighting
* Code completion
* Showing lex, syntax and semantic errors
* Formatter

Refactoring:

* Cross-referencing symbols
* Renaming of symbols
* Extract local variable to instance variable
* Extract variable to function/method
* Extract a piece of code/method into a new class

Build tool:

* Tracking module dependencies

Doc generating:

* Associate a declaration and its documentation

Original post:

http://www.digitalmars.com/d/archives/digitalmars/D/DCT_use_cases_-_draft_168106.html#N168141

-- 
/Jacob Carlborg

On 05-Jul-12 18:29, Andrei Alexandrescu wrote:
> On 7/5/12 9:05 AM, Dmitry Olshansky wrote:
>> On 05-Jul-12 17:04, Roman D. Boiko wrote:
>>> On Thursday, 5 July 2012 at 12:32:19 UTC, Dmitry Olshansky wrote:
>>>> Then do it. It's all about having something so obviously good, fast
>>>> and flexible that other stuff can be considered only as toys.
>>>>
>>>> I might do one, but I'd rather just help other folks make it faster ;)
>>>
>>> Would you help to make Pegged faster?
>>
>> Well why not.
>> But first I'll need to deliver some stuff on my GSOC project.
>
> Good call :o).
>
> Note that we can discuss tweaking the scope of the project after the
> midterm. Ping me if you have some ideas.
>

It's great that you are not opposed to adjusting scope of project.
I have a ton of ideas, but let's discuss them after midterm.


-- 
Dmitry Olshansky

On Thursday, July 05, 2012 18:04:11 Roman D. Boiko wrote:
> On Thursday, 5 July 2012 at 15:40:53 UTC, Jacob Carlborg wrote:
> > On 2012-07-05 15:08, Roman D. Boiko wrote:
> >> Anyway I propose to enumerate major use cases first.
> > 
> > Haven't we already done that a couple of times.
> 
> Well, we did something like that for DCT... but I doubt that it would fit general needs.
> 
> If we had, why haven't they been analyzed, classified, discussed, etc.? Or have they?

It's been discussed before, but there are some obvious use cases such as syntax highlighting, code formatting, and manipulation of D source files (e.g. to strip out the unit tests).

We need to have a lexer in Phobos which parsers D code and generates a range of tokens, and we need to have a parser in Phobos which operates on a range of tokens. As discussed previously, there is desire to have the lexer and parser ported from dmd's frontend to D for Phobos so that what we get is easily maintainable alongside dmd's frontend and produces the same results (and is fast). It's _also_ desirable that we get a lexer/parser generator into Phobos for generating lexers/parsers for whatever language you might want to generate them for. Pegged is a good example of what can be done, and I think that Andrei was trying to get Philippe to prepare a submission to Phobos from it (though I'm not sure), but regarldess of whether pegged (or anything based on it) makes it into Phobos, we definitely want something similar.

So, we have a fair idea of some of the stuff that we want, but it's a question of time and effort. I keep intending to port dmd's lexer to D for submission to Phobos, but I've been too busy to do it. At least a couple of other people have also expressed interest in doing it, but no one has submitted anything for Phobos. So, it remains undone, and anything which would need to lex or parse D code has to find its own solution. As with most everything around here, it's a question of people being having the time and being willing to put in the effort to do it. It's all too common for someone to suggest that we should do something or implement something without ever attempting to do it themselves, and in general, stuff around here gets done because someone really wants it done, takes the time to do it, and sees it through until its done and in Phobos.

- Jonathan M Davis

On Thursday, 5 July 2012 at 16:14:27 UTC, Jacob Carlborg wrote:
> Original post:
>
> http://www.digitalmars.com/d/archives/digitalmars/D/DCT_use_cases_-_draft_168106.html#N168141

OK, fairly complete. Let it be the starting point. Why not put it on some wiki, then add some more, discuss, vote, etc.? Meanwhile create a matrix of features implemented in each of interesting standardization candidates. And pick up one of them as "standard" or "reference" parser.

My vote would be for Pegged, I guess.

On 2012-07-05 18:32, Roman D. Boiko wrote:

> My vote would be for Pegged, I guess.

Aren't you voting on your own project :)

-- 
/Jacob Carlborg

On Thursday, 5 July 2012 at 16:28:57 UTC, Jonathan M Davis wrote:
> It's all too common for someone to suggest that we should
> do something or implement something without ever attempting to do it themselves, and in general, stuff around here gets done because someone really wants it done, takes the time to do it, and sees it through until its done and in Phobos.
>
> - Jonathan M Davis

Resume: everybody is welcome to join effort of translating DMD front end, and improving Pegged.


Also I would like to invite those interested in DCT project to help me with it. Right now I'm trying to understand whether it is possible to incorporate Pegged inside without losing anything critical (and I think it is very likely possible), and how exactly to do that.

Dmitry proposed to help improve Pegged (or some other compiler's) speed.

Anyone else?