View mode: basic / threaded / horizontal-split · Log in · Help
March 30, 2011
D grammar overhaul
Original discussion forked in the D grammar thread, so I'm opening a new one for this specific issue.
The D grammar is in dire need of an overhaul.

I suggest we discuss changes here and then put everything into github to issue a pull request once everything's done.
I already forked the d-programming-language.org repo to get started by fixing some mistakes. Hereafter I list some issues that come to my mind.

Alongside missing things (like @attributes I think) and smaller mistakes parts of it like the whole declarations subgrammar are overly complex
and allow strange constructs, partly cause C style arrays and function pointers are still in there even though considered harmful and thus deprecated.
(see http://www.digitalmars.com/d/2.0/declaration.html)

Furthermore function declarations are obscurely woven into all those productions.
So, correct me if I'm wrong but the current grammar even allows monsters like
auto ubyte[] ([] bar []) (T)(int i) const if(someCodition)  = 2 , foo = 5 ;


Also, like Rainer proposed, BasicType2 should be somehow pulled out of Declarator to get rid of the DeclaratorInitializer-DeclaratorIdentifierList-DeclaratorIdentifier construct which CMIIW only exists to disallow multiple types: int a, *b;


>> isn't AutoDeclaration already covered by Decl -> StorageClasses Decl and so on?
Ok it's just an misleading name. It covers not only declarations with 'auto' but all Declarations where no type but only a StorageClass is given: shared a = 5;


>> Also there are 3 different instances of mixin: MixinExpression, MixinStatement and MixinDeclaration. Even though they all come down to the same thing.
> I think, the different mixin variants are ok, because you might want to 
> generate different AST for these.
Well I can imagine that MixinStatement is necessary because if you have an ExpStatement with a MixinExpression the result is probably discarded (just like 'a' is discarded in a = 5;).
But why do you need another MixinDeclaration?


>> Modifiers like shared are repeated, e.g. in BasicType, Attribute, TypeSpecialization. 
> Yes the confusion of const/shared/immutable etc in Attribute,StorageClass,Types,etc causes a lot of headaches.
March 30, 2011
Re: D grammar overhaul
On Mar 30, 11 20:18, Trass3r wrote:
> Alongside missing things (like @attributes I think)

Property:
	@ Identifier
March 31, 2011
Re: D grammar overhaul
Trass3r wrote:
> Original discussion forked in the D grammar thread, so I'm opening a new one for this specific issue.
> The D grammar is in dire need of an overhaul.
> 
> I suggest we discuss changes here and then put everything into github to issue a pull request once everything's done.
> I already forked the d-programming-language.org repo to get started by fixing some mistakes. Hereafter I list some issues that come to my mind.

I'm not yet friend with git, but this might be a chance to get used to 
it. How can I contribute patches? I guess this would be with pull 
requests, so I'll need a github repository, too? The discussion page for 
pull requests looks like a nice feature...

I'm not sure though it is a good place to keep an overview of the 
grammar in a readable style. The grammar in the ddoc source is sometimes 
rather polluted with formatting information and links.

As a starter, here are some syntactic problems in the ddoc sources that 
cause the generation of grammar.txt to fail:

- in template.dd, $(GNAME Constraint) is not enclosed by $(GRAMMAR)
- in attribute.dd, $(GRAMMAR $(B const)), $(GRAMMAR $(B override)), etc 
are used for formatting without having to do with the grammar. The boxes 
they generate don't make much sense anyway, so they can just be removed.
- MulExpression is only in $(V1), so it does not show for D2

I've attached the patch for these.

>>> Also there are 3 different instances of mixin: MixinExpression, MixinStatement and MixinDeclaration. Even though they all come down to the same thing.
>> I think, the different mixin variants are ok, because you might want to 
>> generate different AST for these.
> Well I can imagine that MixinStatement is necessary because if you have an ExpStatement with a MixinExpression the result is probably discarded (just like 'a' is discarded in a = 5;).
> But why do you need another MixinDeclaration?

MixinDeclaration expands to declarations, while MixinStatement expands 
to statements, so the difference is not interesting during parsing, but 
might be important in the semantic pass (which restarts parsing for the 
string).
March 31, 2011
Re: D grammar overhaul
Rainer Schuetze Wrote:
> How can I contribute patches? I guess this would be with pull 
> requests, so I'll need a github repository, too? The discussion page for 
> pull requests looks like a nice feature...

Yep, you fork the original repo and apply your changes. Once you think you're somewhat done you can open a pull request.
I think you can even comment on single lines in commits.

The advantage of this approach is that your changes don't appear as a single huge diff but as atomic changesets (as long as the author groups changes that belong together in single commits)

> As a starter, here are some syntactic problems in the ddoc sources that 
> cause the generation of grammar.txt to fail:
> 
> - in template.dd, $(GNAME Constraint) is not enclosed by $(GRAMMAR)
> - in attribute.dd, $(GRAMMAR $(B const)), $(GRAMMAR $(B override)), etc 
> are used for formatting without having to do with the grammar. The boxes 
> they generate don't make much sense anyway, so they can just be removed.
> - MulExpression is only in $(V1), so it does not show for D2
> 
> I've attached the patch for these

Nice, I already incorporated some of your changes from the wiki. Will push soon so you can have a look what's already done.
March 31, 2011
Re: D grammar overhaul
On 2011-03-31 02:32, Trass3r wrote:
> Rainer Schuetze Wrote:
> > How can I contribute patches? I guess this would be with pull
> > requests, so I'll need a github repository, too? The discussion page for
> > pull requests looks like a nice feature...
> 
> Yep, you fork the original repo and apply your changes. Once you think
> you're somewhat done you can open a pull request. I think you can even
> comment on single lines in commits.
> 
> The advantage of this approach is that your changes don't appear as a
> single huge diff but as atomic changesets (as long as the author groups
> changes that belong together in single commits)

I would point out though that pull requests are done by branch, so if you have 
commits that you don't want to be grouped with others or put in the pull 
request, then they need to be on a separate branch. If you go and make several 
commits and then only want some of them pulled, that doesn't work so well. 
It's technically feasible via git, but it's more of a pain, and it's not how 
github itself manages the process. So, typically, you'd make a set of changes 
on a branch created for those changes - be they one commit or several. Then 
you do a pull request on that branch. Unrelated changes would be on other 
branches.

- Jonathan M Davis
March 31, 2011
Re: D grammar overhaul
On 3/31/2011 10:34 AM, Jonathan M Davis wrote:
> On 2011-03-31 02:32, Trass3r wrote:
>> Rainer Schuetze Wrote:
>>> How can I contribute patches? I guess this would be with pull
>>> requests, so I'll need a github repository, too? The discussion page for
>>> pull requests looks like a nice feature...
>>
>> Yep, you fork the original repo and apply your changes. Once you think
>> you're somewhat done you can open a pull request. I think you can even
>> comment on single lines in commits.
>>
>> The advantage of this approach is that your changes don't appear as a
>> single huge diff but as atomic changesets (as long as the author groups
>> changes that belong together in single commits)
> 
> I would point out though that pull requests are done by branch, so if you have 
> commits that you don't want to be grouped with others or put in the pull 
> request, then they need to be on a separate branch. If you go and make several 
> commits and then only want some of them pulled, that doesn't work so well. 
> It's technically feasible via git, but it's more of a pain, and it's not how 
> github itself manages the process. So, typically, you'd make a set of changes 
> on a branch created for those changes - be they one commit or several. Then 
> you do a pull request on that branch. Unrelated changes would be on other 
> branches.
> 
> - Jonathan M Davis

Um.. no.  Github allows you to pick both a subrange of a branch.  Take a look at the help docs:

  http://help.github.com/pull-requests/
March 31, 2011
Re: D grammar overhaul
On 2011-03-31 11:04, Brad Roberts wrote:
> On 3/31/2011 10:34 AM, Jonathan M Davis wrote:
> > On 2011-03-31 02:32, Trass3r wrote:
> >> Rainer Schuetze Wrote:
> >>> How can I contribute patches? I guess this would be with pull
> >>> requests, so I'll need a github repository, too? The discussion page
> >>> for pull requests looks like a nice feature...
> >> 
> >> Yep, you fork the original repo and apply your changes. Once you think
> >> you're somewhat done you can open a pull request. I think you can even
> >> comment on single lines in commits.
> >> 
> >> The advantage of this approach is that your changes don't appear as a
> >> single huge diff but as atomic changesets (as long as the author groups
> >> changes that belong together in single commits)
> > 
> > I would point out though that pull requests are done by branch, so if you
> > have commits that you don't want to be grouped with others or put in the
> > pull request, then they need to be on a separate branch. If you go and
> > make several commits and then only want some of them pulled, that
> > doesn't work so well. It's technically feasible via git, but it's more
> > of a pain, and it's not how github itself manages the process. So,
> > typically, you'd make a set of changes on a branch created for those
> > changes - be they one commit or several. Then you do a pull request on
> > that branch. Unrelated changes would be on other branches.
> > 
> > - Jonathan M Davis
> 
> Um.. no.  Github allows you to pick both a subrange of a branch.  Take a
> look at the help docs:
> 
>    http://help.github.com/pull-requests/

I stand corrected. From everything that I've seen (or remembered - I'm not 
sure if I'd read that page before or not), it always takes the whole branch, 
but apparently, you can tell it to do otherwise. That's certainly not the 
default though. Then again, I wouldn't generally be very comfortable with 
someone grabbing just portions of a branch like that. I'm too paranoid about 
likely breaks or merge conflicts, I guess. Still, good to know.

- Jonathan M Davis
April 02, 2011
Re: D grammar overhaul
Before going into details with the grammar, I'd like to clarify one 
thing regarding the lexer, that also has some influence on the grammar:
Do we want to allow spaces and comments between these tokens?

- ! in
- ! is
- extern ( C ++ )
- @ Property (safe, trusted, disable, etc)

here is some example code which also shows an ambiguity introduced by 
combining these from seperate tokens:

extern(C
//
++) void foo();

@ /* comment inside
property */ safe int bar() { return 0; }

void main()
{
	int[int] a;
	struct S { int x; }
	S s;
	int b;

	if (1 ! /* still an infix operator? */ in a) {}
	if (null ! /* still an infix operator? */ is null) {}
	
//	if (s.x !in a) {} /* does not compile: 
http://d.puremagic.com/issues/show_bug.cgi?id=5785 */
	if (b !in a) {} // but this compiles!?
}



Trass3r wrote:
> Original discussion forked in the D grammar thread, so I'm opening a new one for this specific issue.
> The D grammar is in dire need of an overhaul.
> 
> I suggest we discuss changes here and then put everything into github to issue a pull request once everything's done.
> I already forked the d-programming-language.org repo to get started by fixing some mistakes. Hereafter I list some issues that come to my mind.
>
April 04, 2011
Re: D grammar overhaul
> As a starter, here are some syntactic problems in the ddoc sources that
> cause the generation of grammar.txt to fail:
>
> - in template.dd, $(GNAME Constraint) is not enclosed by $(GRAMMAR)
> - in attribute.dd, $(GRAMMAR $(B const)), $(GRAMMAR $(B override)), etc
> are used for formatting without having to do with the grammar. The boxes
> they generate don't make much sense anyway, so they can just be removed.
> - MulExpression is only in $(V1), so it does not show for D2
>
> I've attached the patch for these.

https://github.com/Trass3r/d-programming-language.org/commit/ae532130f0374079c93a2c8f0609f04705dd8f82
April 07, 2011
Re: D grammar overhaul
Am 02.04.2011, 10:11 Uhr, schrieb Rainer Schuetze <r.sagitario@gmx.de>:
> Do we want to allow spaces and comments between these tokens?
> - extern ( C ++ )
Well we would have to introduce C++ as a keyword just for that purpose.
Not worth it.

> - @ Property (safe, trusted, disable, etc)
safe, trusted etc. aren't keywords.
dmd handles it as ('@' identifier) and that's exactly what the grammar  
should be to support user-defined attributes in the future.

> extern(C
> //
> ++) void foo();
>
> @ /* comment inside
> property */ safe int bar() { return 0; }

Would someone really do bullshit like that?
« First   ‹ Prev
1 2
Top | Discussion index | About this forum | D home