On 6/13/2010 5:15 PM, BCS wrote:
> Hello BCS,
> 
>> I'd be willing to do that first step if I got any (semi)official
>> indication that it would be used.
> 
> does anyone know where the SVN repo with the source for the docs is?
> 

The standard phobos repo at dsource in the docsrc directory:

    http://www.dsource.org/projects/phobos/browser/trunk/docsrc

Arguably part of that could (maybe should) be split to the dmd repo now that it
exists, but... shrug.

Hello Brad,

> On 6/13/2010 5:15 PM, BCS wrote:
> 
>> Hello BCS,
>> 
>>> I'd be willing to do that first step if I got any (semi)official
>>> indication that it would be used.
>>> 
>> does anyone know where the SVN repo with the source for the docs is?
>> 
> The standard phobos repo at dsource in the docsrc directory:
> 
> http://www.dsource.org/projects/phobos/browser/trunk/docsrc
> 

Thanks.

-- 
... <IXOYE><

On 06/13/2010 03:15 PM, BCS wrote:
> Right now we have two semi-official definitions of the D grammar; the
> docs (that are wrong) and the parser source (that is effectively
> unreadable by most people). I would like to propose a solution to this
> problem: eliminate one of them and derive it from the other.


In the past, Walter has expressed a dislike of parser generators. I doubt you can get him to change his mind on that one, especially since most parser generators won't be able to handle D anyways, and I doubt any of them will be able to handle D efficiently (what are they called - GLR parser generators? Don't know about these).

As far as analysing parse.c goes, how much effort is it going to take to figure out what

while(1)
  switch(token.value){
    case x:
      parseThis();
      continue;
    case y:
      parseThat();
      break;
  }
  break;
}

should look like in a grammar?

Counterproposal: why don't we just fix the grammar?

>
> I know this will be hard to do but it can be done incrementally with
> each step making improvements over the last.
>
> The first thing to do is put all of the description of the grammar in
> the docs into one place. If the literal text of each production is
> replace with a macro reference then the definitions of these macros can
> be put into a single file and expanded everywhere.
> The incremental improvement here is that having the grammar in one place
> by it's self will make it easier to check.

I would very much like to see the entire grammar in one place. I can't tell you how obnoxious it is to search through the entire site to find one production

And as always, apologies for being a pessimist.

Hello Ellery,

> On 06/13/2010 03:15 PM, BCS wrote:
> 
>> Right now we have two semi-official definitions of the D grammar; the
>> docs (that are wrong) and the parser source (that is effectively
>> unreadable by most people). I would like to propose a solution to
>> this problem: eliminate one of them and derive it from the other.
>> 
> In the past, Walter has expressed a dislike of parser generators. I
> doubt you can get him to change his mind on that one, especially since
> most parser generators won't be able to handle D anyways, and I doubt
> any of them will be able to handle D efficiently (what are they called
> - GLR parser generators? Don't know about these).

I was very careful to leave both options open in the way I phrased that ;).

> 
> As far as analysing parse.c goes, how much effort is it going to take
> to figure out what
> 

void parseSomething()
{

> while(1)
> switch(token.value){
> case x:
> parseThis();
> continue;
> case y:
> parseThat();
> break;
> }
> break;
> }

}


Something ::= This* That;

The switch is just an optimization. As long as the code is regular enough, it can be analyzed. (I will grant that the resulting grammar might be ambiguous or LL(k) where k is higher than it needs to be)

> should look like in a grammar?
> 
> Counterproposal: why don't we just fix the grammar?
> 

That would be a start as well as a necessary step in getting a extractor working (to test the tools output against), but if the docs and the parser are maintained separately, the docs will be wrong sooner rather than later.

>> I know this will be hard to do but it can be done incrementally with
>> each step making improvements over the last.
>> 
>> The first thing to do is put all of the description of the grammar in
>> the docs into one place. If the literal text of each production is
>> replace with a macro reference then the definitions of these macros
>> can
>> be put into a single file and expanded everywhere.
>> The incremental improvement here is that having the grammar in one
>> place
>> by it's self will make it easier to check.
>
> I would very much like to see the entire grammar in one place. I can't
> tell you how obnoxious it is to search through the entire site to find
> one production
>

A while back I took a pass at pulling the grammar from the HTML via sed so I could stuff in into a parser generator. That it worked is the only good thing I can say about the experience.

> And as always, apologies for being a pessimist.
> 
-- 
... <IXOYE><

Hello BCS,

> The first thing to do is put all of the description of the grammar in
> the docs into one place. If the literal text of each production is
> replace with a macro reference then the definitions of these
> macros can be put into a single file and expanded everywhere.
[...]
> I'd be willing to do that first step if I got any (semi)official
> indication that it would be used.

Done, and because I can't attache the patch (it's to big) I started a issue.

http://d.puremagic.com/issues/show_bug.cgi?id=4308

As a side note, the formatting of the grammar is very irregular, using different macros to get the same thing in different places. That would be the next thing I'd clean up.

-- 
... <IXOYE><

BCS wrote:
> That would be a start as well as a necessary step in getting a extractor working (to test the tools output against), but if the docs and the parser are maintained separately, the docs will be wrong sooner rather than later.

I think that's very unlikely to be true. The existing errors in the grammer have formed during the long development of the language, and the grammer docs never really got much attention.
Now that the language has stabilised, the grammer is not going to change rapidly enough to be major source of errors.
The effort involved in making a tool to do it, probably exceeds the effort involved in just fixing it, by a factor of five or ten. There are many other things which could be done which would FAR more useful for D.