View mode: basic / threaded / horizontal-split · Log in · Help
June 14, 2006
Enki 1.1 Released
Enki 1.1 is now available.

Documentation (still in progress)
http://www.dsource.org/projects/ddl/wiki/Enki

* Download
http://svn.dsource.org/projects/ddl/downloads/enki.1.1.win32.bin.zip
http://svn.dsource.org/projects/ddl/downloads/enki.sdk.1.1.src.zip

Enki is a frontend parser generator that uses an annotated EBNF syntax to
produce .d sourcefiles.  More details can be found at the link above. 

Enki 1.1 Changes:
- enhancement: semantic analysis of binding types by search is now in
declaration order
- bug: repeated binding declarations in generated parser
- bug: binding names were output as 'bind_x' in the comments for some
expressions
- enhancement: added function-call capability to '@' operator
- enhancement: added '&' (custom-terminal) operator
- enhancement: added minimal sourcecode for custom parsers to binary
distribution

- EricAnderton at yahoo
June 14, 2006
Re: Enki 1.1 Released
pragma wrote:
> Enki 1.1 is now available.
> 
> Documentation (still in progress)
> http://www.dsource.org/projects/ddl/wiki/Enki
> 
> * Download
> http://svn.dsource.org/projects/ddl/downloads/enki.1.1.win32.bin.zip
> http://svn.dsource.org/projects/ddl/downloads/enki.sdk.1.1.src.zip
> 
> Enki is a frontend parser generator that uses an annotated EBNF syntax to
> produce .d sourcefiles.  More details can be found at the link above. 
> 
> Enki 1.1 Changes:
> - enhancement: semantic analysis of binding types by search is now in
> declaration order
> - bug: repeated binding declarations in generated parser
> - bug: binding names were output as 'bind_x' in the comments for some
> expressions
> - enhancement: added function-call capability to '@' operator
> - enhancement: added '&' (custom-terminal) operator
> - enhancement: added minimal sourcecode for custom parsers to binary
> distribution
> 
> - EricAnderton at yahoo

Cool, tried it out. Didn't get to far (I haven't used a parser generator 
before, so...). The generated parser still doesn't compile out of the 
box. I dug through the code and added a few imports to the render() 
function to get it to work. Here is a patch of what I did.

*** old\enki\EnkiBackend.d	Wed Jun 14 11:47:30 2006
--- new\sand\EnkiBackend.d	Wed Jun 14 11:49:57 2006
***************
*** 184,189 ****
--- 184,191 ----
  				emit("module " ~ moduleName ~ ";");
  			}
  			emit("debug private import std.stdio;");
+ 			emit("import enki.BaseParser;");
+ 			emit("import enki.types;");
  			foreach(imp; imports){
  				emit("private import " ~ imp ~ ";");
  			}
June 14, 2006
Re: Enki 1.1 Released
pragma wrote:
> Enki 1.1 is now available.

This subproject of yours couldn't have come at a better time... here I was writing a 
byte-compiler/vm for a simple scripted object-database language in another project of mine 
(codename "Lyra"), and you drop this lovely golden egg right in my lap!  :)  Goodbye messy 
amateur object hierarchy, hello much easier to work with grammar file and backend library!

-- Chris Nicholson-Sauls
June 14, 2006
Re: Enki 1.1 Released
In article <e6plq9$1upb$1@digitaldaemon.com>, BCS says...
>
>pragma wrote:
>> Enki 1.1 is now available.
>
>Cool, tried it out. Didn't get to far (I haven't used a parser generator 
>before, so...). The generated parser still doesn't compile out of the 
>box. I dug through the code and added a few imports to the render() 
>function to get it to work. Here is a patch of what I did.
>
>*** old\enki\EnkiBackend.d	Wed Jun 14 11:47:30 2006
>--- new\sand\EnkiBackend.d	Wed Jun 14 11:49:57 2006
>***************
>*** 184,189 ****
>--- 184,191 ----
>   				emit("module " ~ moduleName ~ ";");
>   			}
>   			emit("debug private import std.stdio;");
>+ 			emit("import enki.BaseParser;");
>+ 			emit("import enki.types;");
>   			foreach(imp; imports){
>   				emit("private import " ~ imp ~ ";");
>   			}

BCS, thanks for being patient with all this.  I hope I haven't lost your
interest. ;)

What you've done here is hack the code-generator to emit the imports you need -
I like the cut of your jib.  As the documentation is incomplete, I can't blame
you for trying this, but there is an easier way to do this.  The .import()
directive does this job nicely.

> .import("my.import.here");

A solid example of this and other directives are in Enki's BNF definition:

http://svn.dsource.org/projects/ddl/trunk/enki/enki.bnf

You may find the .baseclass() .classname() and .code{{{ }}} directives helpful
as well.  The .start() directive is now depricated, and there's the .include()
and .alias() directives that aren't used in that particular file.


- EricAnderton at yahoo
June 14, 2006
Re: Enki 1.1 Released
pragma wrote:
> In article <e6plq9$1upb$1@digitaldaemon.com>, BCS says...
>>Cool, tried it out. Didn't get to far (I haven't used a parser generator 
>>before, so...). The generated parser still doesn't compile out of the 
>>box. I dug through the code and added a few imports to the render() 
>>function to get it to work. Here is a patch of what I did.
[...]
> 
> 
> BCS, thanks for being patient with all this.  I hope I haven't lost your
> interest. ;)

Nope, you have only wet it.

> 
> What you've done here is hack the code-generator to emit the imports you need -
> I like the cut of your jib.  As the documentation is incomplete, I can't blame
> you for trying this, but there is an easier way to do this.  The .import()
> directive does this job nicely.
> 
> 
>>.import("my.import.here");
> 

Are these imports not always needed? Won't any parser use things from 
these modules? I would think that the basic imports _should_ be hard 
coded (or at least implicitly added to the imports list).

On further thought only "enki.types" will _always_ be needed. 
"enki.BaseParser" will only be used if the default base class is used. 
But this is detectable, so again, it should be emitted unless the base 
class is changed.

> 
> A solid example of this and other directives are in Enki's BNF definition:
> 
> http://svn.dsource.org/projects/ddl/trunk/enki/enki.bnf
> 

I'll take closer a look at that soon.

========

And on another topic: another hack

It fixes a default case bug. Without it any rule of the form

rule ::= (foo bar)

always matches even if both foo and bar don't match. The case where the 
rule has annotations works as advertised.

<patch>
*** old\enki\Rule.d	Wed Jun 14 13:50:06 2006
--- new\enki\Rule.d	Wed Jun 14 13:50:59 2006
***************
*** 241,251 ****
  	}
  	
  	public void renderPass(CodeGenerator generator){
! 		generator.emit("return ResultT!(bool)(true);");
  	}
  	
  	public void renderFail(CodeGenerator generator){
! 		generator.emit("return ResultT!(bool)(false);");
  	}
  	
  	public void semanticPass(Rule thisRule,BaseEnkiParser root){
--- 242,252 ----
  	}
  	
  	public void renderPass(CodeGenerator generator){
! 		generator.emit("return ResultT!(bool)(true,true);");
  	}
  	
  	public void renderFail(CodeGenerator generator){
! 		generator.emit("return ResultT!(bool)(false,false);");
  	}
  	
  	public void semanticPass(Rule thisRule,BaseEnkiParser root){
</patch>
June 14, 2006
Re: Enki 1.1 Released
In article <e6pqne$2826$1@digitaldaemon.com>, Chris Nicholson-Sauls says...
>
>pragma wrote:
>> Enki 1.1 is now available.
>
>This subproject of yours couldn't have come at a better time... here I was writing a 
>byte-compiler/vm for a simple scripted object-database language in another project of mine 
>(codename "Lyra"), and you drop this lovely golden egg right in my lap!  :)  Goodbye messy 
>amateur object hierarchy, hello much easier to work with grammar file and backend library!
>
>-- Chris Nicholson-Sauls

You're quite welcome!  

If you find any bugs, or have any suggestions on how to improve things, don't be
afraid to check in and make some noise. ;)

- EricAnderton at yahoo
June 14, 2006
Re: Enki 1.1 Released
In article <e6ptut$2c61$1@digitaldaemon.com>, BCS says...
>
>>>.import("my.import.here");
>> 
>
>Are these imports not always needed? Won't any parser use things from 
>these modules? I would think that the basic imports _should_ be hard 
>coded (or at least implicitly added to the imports list).
>
>On further thought only "enki.types" will _always_ be needed. 
>"enki.BaseParser" will only be used if the default base class is used. 
>But this is detectable, so again, it should be emitted unless the base 
>class is changed.

I'm inclined to agree, but its also at cross-purposes with the philosophy I had
in mind with the tool.  I really think that it should try to impose as little as
possible, wherever possible.  In this case, what happens if the user moves or
renames 'types.d' to something else, or merged the templates in with their base
parser?  I'd rather make imports explicit so the developer has this flexibility.

>
>And on another topic: another hack
>
>It fixes a default case bug. Without it any rule of the form
>
>rule ::= (foo bar)
>
>always matches even if both foo and bar don't match. The case where the 
>rule has annotations works as advertised.
>
><patch>
>*** old\enki\Rule.d	Wed Jun 14 13:50:06 2006
>--- new\enki\Rule.d	Wed Jun 14 13:50:59 2006
>***************
>*** 241,251 ****
>   	}
>   	
>   	public void renderPass(CodeGenerator generator){
>! 		generator.emit("return ResultT!(bool)(true);");
>   	}
>   	
>   	public void renderFail(CodeGenerator generator){
>! 		generator.emit("return ResultT!(bool)(false);");
>   	}
>   	
>   	public void semanticPass(Rule thisRule,BaseEnkiParser root){
>--- 242,252 ----
>   	}
>   	
>   	public void renderPass(CodeGenerator generator){
>! 		generator.emit("return ResultT!(bool)(true,true);");
>   	}
>   	
>   	public void renderFail(CodeGenerator generator){
>! 		generator.emit("return ResultT!(bool)(false,false);");
>   	}
>   	
>   	public void semanticPass(Rule thisRule,BaseEnkiParser root){
></patch>

Ah, now that's a serious bug.  Thanks for tracking it down.  I'm glad that
things are written such that you can find this stuff so easily.  Thanks for the
patch!

Another way to tackle this would be tochange the failure rule to this:

>   	public void renderFail(CodeGenerator generator){
>! 		generator.emit("return ResultT!(bool)();");
>   	}

Which is equivalent.  When I get around to expanding the docs to detail how to
write custom rules, this will make more sense.

- EricAnderton at yahoo
June 14, 2006
Re: Enki 1.1 Released
pragma wrote:
> In article <e6ptut$2c61$1@digitaldaemon.com>, BCS says...
> 
>>>>.import("my.import.here");
>>>
>>Are these imports not always needed? Won't any parser use things from 
>>these modules? I would think that the basic imports _should_ be hard 
>>coded (or at least implicitly added to the imports list).
>>
>>On further thought only "enki.types" will _always_ be needed. 
>>"enki.BaseParser" will only be used if the default base class is used. 
>>But this is detectable, so again, it should be emitted unless the base 
>>class is changed.
> 
> 
> I'm inclined to agree, but its also at cross-purposes with the philosophy I had
> in mind with the tool.  I really think that it should try to impose as little as
> possible, wherever possible.  In this case, what happens if the user moves or
> renames 'types.d' to something else, or merged the templates in with their base
> parser?  I'd rather make imports explicit so the developer has this flexibility.
> 
> 

OK I'll grant that things might move, however in the general case those 
imports will be needed by most people. Good defaults make for good 
programs. How about make those imports the default and put in some sort 
of Override directive.


>
> Ah, now that's a serious bug.  Thanks for tracking it down.  I'm glad
> that things are written such that you can find this stuff so easily.
> Thanks for the patch!
>

Find-in-files, grep and kin are wonderful aren't they. <g>


Another thought; as it is written, Enki is purely an ASCII text program. 
  It would be nice if the parsed data had its own type that, is only 
used for the input data. This would allow for the parsing of an array on 
any type (enums or structs come to mind). Of course this would requirer 
the introduction of a new terminal matching syntax.

Just a sketch of what might be done:
<code name="somewher.d">
...
enum bar { i1, i2, i3}

struct baz{
	bar thisType;
}
...
</code>

<code name="syn.bnf">
...

.parsetype(baz)

...

foo ::= @.thisType == bar.i1;

...
</code>
June 15, 2006
Re: Enki 1.1 Released
pragma wrote:
> In article <e6ptut$2c61$1@digitaldaemon.com>, BCS says...
> 
>>>> .import("my.import.here");
>>> 
>> Are these imports not always needed? Won't any parser use things
>> from these modules? I would think that the basic imports _should_
>> be hard coded (or at least implicitly added to the imports list).
>> 
>> On further thought only "enki.types" will _always_ be needed. 
>> "enki.BaseParser" will only be used if the default base class is
>> used. But this is detectable, so again, it should be emitted unless
>> the base class is changed.
> 
> I'm inclined to agree, but its also at cross-purposes with the
> philosophy I had in mind with the tool.  I really think that it
> should try to impose as little as possible, wherever possible.  In
> this case, what happens if the user moves or renames 'types.d' to
> something else, or merged the templates in with their base parser?
> I'd rather make imports explicit so the developer has this
> flexibility.

That's what Defaults are for!
June 15, 2006
Re: Enki 1.1 Released
In article <e6q2pu$2cqo$1@digitaldaemon.com>, BCS says...
>
>pragma wrote:
>> In article <e6ptut$2c61$1@digitaldaemon.com>, BCS says...
>> 
>>>>>.import("my.import.here");
>>>>
>>>Are these imports not always needed? Won't any parser use things from 
>>>these modules? I would think that the basic imports _should_ be hard 
>>>coded (or at least implicitly added to the imports list).
>>>
>>>On further thought only "enki.types" will _always_ be needed. 
>>>"enki.BaseParser" will only be used if the default base class is used. 
>>>But this is detectable, so again, it should be emitted unless the base 
>>>class is changed.
>> 
>> 
>> I'm inclined to agree, but its also at cross-purposes with the philosophy I had
>> in mind with the tool.  I really think that it should try to impose as little as
>> possible, wherever possible.  In this case, what happens if the user moves or
>> renames 'types.d' to something else, or merged the templates in with their base
>> parser?  I'd rather make imports explicit so the developer has this flexibility.
>> 
>> 
>
>OK I'll grant that things might move, however in the general case those 
>imports will be needed by most people. Good defaults make for good 
>programs. How about make those imports the default and put in some sort 
>of Override directive.

That's not a bad idea.  Something like a .supportlib() directive or something.
I'll consider it. :)

>
>Another thought; as it is written, Enki is purely an ASCII text program. 
>   It would be nice if the parsed data had its own type that, is only 
>used for the input data. This would allow for the parsing of an array on 
>any type (enums or structs come to mind). Of course this would requirer 
>the introduction of a new terminal matching syntax.

I'm well ahead of you on this front.  I'm presently composing a lexer for D,
which will then support a secondary parser for the lexed tokens.  Both of these
will be Enki based.  

So far I'm starting to see the limitations of the String/char[] setup, and yearn
for multi-byte char support.  So I'll be working on a way to provide this.  As
far as non-ascii parsing is concerned, Enki can already support this to an
extent.

code{{{
enum MyType: uint{
Foo, Bar, Baz
}
}}}

MyRule ::= &Foo:x &Bar:y &Baz:z;

Under the hood, Enki will generate terminal expressions for each & term, which
are then generated as "terminal(Foo)".  As long as your base parser has a
terminal(MyType) declared, you can handle these expressions.  It then becomes
the responsility of that terminal to determine if the token matches or not.

As far as the internal stream of data is concerned, the IParser position() and
sliceData() methods abstract the data away to the same extent.  When working
with a non-text input, sliceData() can be redefined as returning something other
than string.  The only caveat here is that you must be careful to avoid
expressions cases where Enki will evaluate the default type of a binding to
String:

# Enki has no clue what 'x' should be so it becomes a String
Foobar ::= ("hello world"):x;

>.parsetype(baz)

A wonderful idea.  This way Enki can use this as the default binding type when
it encounters expressions like the one above.


- EricAnderton at yahoo
« First   ‹ Prev
1 2
Top | Discussion index | About this forum | D home