View mode: basic / threaded / horizontal-split · Log in · Help
March 27, 2011
Goldie Parsing System v0.4 Released - Now for D2
Goldie is a series of flexible open-source parsing tools, including a D2 
library called GoldieLib. It's compatible with GOLD Parser Builder and can 
be used together with it, but does not require it. In fact, Goldie can be 
used as a cross-platform, shell-scripting-compatible alternative to GOLD 
Parser Builder.

== Links: ==

Main homepage and documentation:
   http://www.semitwist.com/goldie/

Prepackaged downloads:
   http://www.dsource.org/projects/goldie/browser/downloads

The related GOLD Parser Builder:
   http://www.devincook.com/goldparser/

== New in v0.4: ==

   - Switched from D1/Tango to D2/Phobos.

   - New tool: GRMC: Grammar Compiler. Because of this, Goldie no longer 
requires GOLD Parser Builder.

   - Grammars can be compiled not only from GRMC: Grammar Compiler, but 
also through the D API, GoldieLib.

   - No longer requires xfBuild or Rebuild.

   - Executable filenames are now prefixed with 'goldie-' to minimize 
chance of collisions on the PATH.

   - Many misc changes/improvements to tools, API and documentation.

   - Includes a lexing-only D2 grammar: 
http://www.dsource.org/projects/goldie/browser/trunk/lang/dlex.grm

This D2 grammar does have a few small limitations ATM though, which I've 
already described here:
http://www.mail-archive.com/digitalmars-d-learn@puremagic.com/msg11491.html
http://www.mail-archive.com/digitalmars-d-learn@puremagic.com/msg11493.html


== Some of Goldie's benefits: (most are thanks to Goldie's compatibility 
with GOLD Parser Builder) ==

   - Grammars are fully-reusable: No need to create a new grammar for every 
use and every host language. Many grammars are already available.

   - Grammar-agnostic engine: One lexer/parser engine can be used for all 
grammars.

   - Engines for nearly any language or platform: A cross-platform D v2.x 
engine is included via GoldieLib. Engines for many other platforms are also 
available. New engines are easy to write.

   - Dynamic-Style: Dynamic-style lets you write programs that support 
user-created grammars.

   - Static-Style: Static-style provides compile-time checks and extra 
type-safety.

   - Lexing and parsing: Lexing and parsing are defined in the same file 
and handled by one unified tool.

   - Many tools available.

Goldie is fully-usable and has been tested on both Windows and Linux (it 
should also work on OSX and any other platform supported by DMD, but has not 
been tested), although GoldieLib's API is still subject to change. Goldie is 
licensed under The zlib/libpng License.
March 27, 2011
Re: Goldie Parsing System v0.4 Released - Now for D2
I try use the gold from trunk, not the release version .  It seems
very slow for parse css .  please see this
http://www.dsource.org/projects/goldie/ticket/18 .

Is all lalr slow like this , or it is a gold problem .

On Sun, Mar 27, 2011 at 4:11 PM, Nick Sabalausky <a@a.a> wrote:
> Goldie is a series of flexible open-source parsing tools, including a D2
> library called GoldieLib. It's compatible with GOLD Parser Builder and can
> be used together with it, but does not require it. In fact, Goldie can be
> used as a cross-platform, shell-scripting-compatible alternative to GOLD
> Parser Builder.
>
> == Links: ==
>
> Main homepage and documentation:
>    http://www.semitwist.com/goldie/
>
> Prepackaged downloads:
>    http://www.dsource.org/projects/goldie/browser/downloads
>
> The related GOLD Parser Builder:
>    http://www.devincook.com/goldparser/
>
> == New in v0.4: ==
>
>    - Switched from D1/Tango to D2/Phobos.
>
>    - New tool: GRMC: Grammar Compiler. Because of this, Goldie no longer
> requires GOLD Parser Builder.
>
>    - Grammars can be compiled not only from GRMC: Grammar Compiler, but
> also through the D API, GoldieLib.
>
>    - No longer requires xfBuild or Rebuild.
>
>    - Executable filenames are now prefixed with 'goldie-' to minimize
> chance of collisions on the PATH.
>
>    - Many misc changes/improvements to tools, API and documentation.
>
>    - Includes a lexing-only D2 grammar:
> http://www.dsource.org/projects/goldie/browser/trunk/lang/dlex.grm
>
> This D2 grammar does have a few small limitations ATM though, which I've
> already described here:
> http://www.mail-archive.com/digitalmars-d-learn@puremagic.com/msg11491.html
> http://www.mail-archive.com/digitalmars-d-learn@puremagic.com/msg11493.html
>
>
> == Some of Goldie's benefits: (most are thanks to Goldie's compatibility
> with GOLD Parser Builder) ==
>
>    - Grammars are fully-reusable: No need to create a new grammar for every
> use and every host language. Many grammars are already available.
>
>    - Grammar-agnostic engine: One lexer/parser engine can be used for all
> grammars.
>
>    - Engines for nearly any language or platform: A cross-platform D v2.x
> engine is included via GoldieLib. Engines for many other platforms are also
> available. New engines are easy to write.
>
>    - Dynamic-Style: Dynamic-style lets you write programs that support
> user-created grammars.
>
>    - Static-Style: Static-style provides compile-time checks and extra
> type-safety.
>
>    - Lexing and parsing: Lexing and parsing are defined in the same file
> and handled by one unified tool.
>
>    - Many tools available.
>
> Goldie is fully-usable and has been tested on both Windows and Linux (it
> should also work on OSX and any other platform supported by DMD, but has not
> been tested), although GoldieLib's API is still subject to change. Goldie is
> licensed under The zlib/libpng License.
>
>
>



-- 
.
March 27, 2011
Re: Goldie Parsing System v0.4 Released - Now for D2
just read you replay, very look forward to the Character Set Optimization.

and thank you to done such a useful project .

On 3/27/11, Long Chang <changlong@jkys.info> wrote:
> I try use the gold from trunk, not the release version .  It seems
> very slow for parse css .  please see this
> http://www.dsource.org/projects/goldie/ticket/18 .
>
> Is all lalr slow like this , or it is a gold problem .
>
> On Sun, Mar 27, 2011 at 4:11 PM, Nick Sabalausky <a@a.a> wrote:
>> Goldie is a series of flexible open-source parsing tools, including a D2
>> library called GoldieLib. It's compatible with GOLD Parser Builder and
>> can
>> be used together with it, but does not require it. In fact, Goldie can be
>> used as a cross-platform, shell-scripting-compatible alternative to GOLD
>> Parser Builder.
>>
>> == Links: ==
>>
>> Main homepage and documentation:
>>    http://www.semitwist.com/goldie/
>>
>> Prepackaged downloads:
>>    http://www.dsource.org/projects/goldie/browser/downloads
>>
>> The related GOLD Parser Builder:
>>    http://www.devincook.com/goldparser/
>>
>> == New in v0.4: ==
>>
>>    - Switched from D1/Tango to D2/Phobos.
>>
>>    - New tool: GRMC: Grammar Compiler. Because of this, Goldie no longer
>> requires GOLD Parser Builder.
>>
>>    - Grammars can be compiled not only from GRMC: Grammar Compiler, but
>> also through the D API, GoldieLib.
>>
>>    - No longer requires xfBuild or Rebuild.
>>
>>    - Executable filenames are now prefixed with 'goldie-' to minimize
>> chance of collisions on the PATH.
>>
>>    - Many misc changes/improvements to tools, API and documentation.
>>
>>    - Includes a lexing-only D2 grammar:
>> http://www.dsource.org/projects/goldie/browser/trunk/lang/dlex.grm
>>
>> This D2 grammar does have a few small limitations ATM though, which I've
>> already described here:
>> http://www.mail-archive.com/digitalmars-d-learn@puremagic.com/msg11491.html
>> http://www.mail-archive.com/digitalmars-d-learn@puremagic.com/msg11493.html
>>
>>
>> == Some of Goldie's benefits: (most are thanks to Goldie's compatibility
>> with GOLD Parser Builder) ==
>>
>>    - Grammars are fully-reusable: No need to create a new grammar for
>> every
>> use and every host language. Many grammars are already available.
>>
>>    - Grammar-agnostic engine: One lexer/parser engine can be used for all
>> grammars.
>>
>>    - Engines for nearly any language or platform: A cross-platform D v2.x
>> engine is included via GoldieLib. Engines for many other platforms are
>> also
>> available. New engines are easy to write.
>>
>>    - Dynamic-Style: Dynamic-style lets you write programs that support
>> user-created grammars.
>>
>>    - Static-Style: Static-style provides compile-time checks and extra
>> type-safety.
>>
>>    - Lexing and parsing: Lexing and parsing are defined in the same file
>> and handled by one unified tool.
>>
>>    - Many tools available.
>>
>> Goldie is fully-usable and has been tested on both Windows and Linux (it
>> should also work on OSX and any other platform supported by DMD, but has
>> not
>> been tested), although GoldieLib's API is still subject to change. Goldie
>> is
>> licensed under The zlib/libpng License.
>>
>>
>>
>
>
>
> --
> .
>


-- 
.
March 27, 2011
Re: Goldie Parsing System v0.4 Released - Now for D2
"Long Chang" <changlong@jkys.info> wrote in message 
news:mailman.2806.1301240916.4748.digitalmars-d-announce@puremagic.com...
>just read you replay, very look forward to the Character Set Optimization.
>
>and thank you to done such a useful project .
>

Thanks for your interest :)
March 27, 2011
Re: Goldie Parsing System v0.4 Released - Now for D2
"Long Chang" <changlong@jkys.info> wrote in message 
news:mailman.2805.1301240416.4748.digitalmars-d-announce@puremagic.com...
>I try use the gold from trunk, not the release version .  It seems
>very slow for parse css .  please see this
>http://www.dsource.org/projects/goldie/ticket/18 .
>
>Is all lalr slow like this , or it is a gold problem .

For other people who haven't read my reply to that ticket, this is the top 
priority for Goldie now. I've profiled, and the biggest bottleneck by far is 
the (rather stupid and half-assed) way that I'm handling character sets in 
the lexer. (To directly answer the quesion, it's neither an LALR thing nor a 
GOLD thing, it's just a temporary Goldie thing.)

Additionally, my plain is to switch from large infrequent releases to 
smaller more frequent releases, so it shouldn't be another 6+ month wait for 
"v0.5 with a speed boost". This v0.4 version has a lot of big stuff in it 
(the change from D1->D2 and the grammar compiler), so that's why it took so 
long.
April 15, 2011
Re: Goldie Parsing System v0.4 Released - Now for D2
So could your library be used to implement an alternative for HTOD? Or
more simply put, could I use this to do (simple) transformations of C
code?
April 15, 2011
Re: Goldie Parsing System v0.4 Released - Now for D2
"Andrej Mitrovic" <andrej.mitrovich@gmail.com> wrote in message 
news:mailman.3528.1302836832.4748.digitalmars-d-announce@puremagic.com...
> So could your library be used to implement an alternative for HTOD? Or
> more simply put, could I use this to do (simple) transformations of C
> code?

C code is admittedly a bit tricky because it uses a preprocessor. But in 
general, yes, Goldie can be used transform source. The way it would work is 
like this:

1. Define a grammar for the "input" language.

There's an ANSI C grammar here, but I haven't looked at it, so I don't know 
how good it is:
http://www.devincook.com/goldparser/grammars/index.htm

An introduction to the grammar description langauge is here:
http://www.semitwist.com/goldie/Start/Grammar/

2. Use Goldie to parse the input. Details here: 
http://www.semitwist.com/goldie/Start/HowToUse/

3. Once Goldie's parsed the input, it will give you a parse tree (it'll be 
structured based on the grammar you used). You can then walk the tree and do 
whatever you want with it. I don't recommend actually modifying the parse 
tree that Goldie gives you in-place, since the interface isn't really 
designed for that right now (though you may still be able to make it work). 
But you can walk it and either build up your own tree structure, or convert 
it to text however you want, etc.

Actually, you can even take a look at what the parse tree you'll get back 
will look like before writing any code: Use the included Goldie parse ( 
http://www.semitwist.com/goldie/Tools/Parse/ ) to parse a file according to 
whatever grammar you want to use. It'll save the parse tree to JSON. Then 
you can inspect the parse tree with this: ( 
http://www.semitwist.com/goldie/Tools/JsonViewer/ ). But try to use just a 
small sample file: Parse trees tend to get very big, very fast and 
JsonViewer is written in C# and can't handle large files very well.

Since you're talking about C, you'll probably want to run your original C 
code through the "preprocess-only" option of a real C compiler. (I *think* 
DMC will do that.) Then parse the resulting "preprocessed C" files with 
Goldie. (Although if your goal is an HTOD-like tool, maybe you would need to 
deal with the original un-preprocessed source directly. If Golde's grammar 
langauge doesn't seem quite up to the task, it probably wouldn't bee too 
hard to just manually make a basic C preprocessor.)

Right now, the grammar description format isn't really very good as 
describing preprocessors (a limitation Goldie inherited from GOLD Parser 
Builder). But fixing that limitation is one of the things on my TODO list 
for Goldie.

If you do try this, I'd love to hear how it works out :) Even if you 
encounter any problems, it would be very helpful for me to know. Haven't 
gotten a whole lot of feedback yet.
April 15, 2011
Re: Goldie Parsing System v0.4 Released - Now for D2
Am 15.04.2011 09:50, schrieb Nick Sabalausky:
> Since you're talking about C, you'll probably want to run your original C 
> code through the "preprocess-only" option of a real C compiler. (I *think* 
> DMC will do that.) Then parse the resulting "preprocessed C" files with 
> Goldie. (Although if your goal is an HTOD-like tool, maybe you would need to 
> deal with the original un-preprocessed source directly. 

Why? Just call the preprocessor from your tool or from a wrapping script
and go on with the preprocessed C code. Should be much easier and more
compatible because C compilers ought to know how to preprocess correctly.
For GCC the option you're looking for is "-E", btw.

> If Golde's grammar 
> langauge doesn't seem quite up to the task, it probably wouldn't bee too 
> hard to just manually make a basic C preprocessor.)
> 

Cheers,
- Daniel
April 15, 2011
Re: Goldie Parsing System v0.4 Released - Now for D2
"Daniel Gibson" <metalcaedes@gmail.com> wrote in message 
news:io8u12$132q$1@digitalmars.com...
> Am 15.04.2011 09:50, schrieb Nick Sabalausky:
>> Since you're talking about C, you'll probably want to run your original C
>> code through the "preprocess-only" option of a real C compiler. (I 
>> *think*
>> DMC will do that.) Then parse the resulting "preprocessed C" files with
>> Goldie. (Although if your goal is an HTOD-like tool, maybe you would need 
>> to
>> deal with the original un-preprocessed source directly.
>
> Why? Just call the preprocessor from your tool or from a wrapping script
> and go on with the preprocessed C code. Should be much easier and more
> compatible because C compilers ought to know how to preprocess correctly.
> For GCC the option you're looking for is "-E", btw.
>

If by "your tool" mean a program that uses Goldie to process C code, then 
yea, that's what I meant.

If you meant that Goldie should invoke a C preprocessor directly, that's a 
bit tricky: Goldie is a generalized parsing tool (sort of like ANTLR or 
Spirit), so it doesn't really know "Ok, this is supposed to be C". It just 
parses according to whatever grammar it's given. Of course, it's not 
entirely out of the question to have some sort of system for specifying that 
a source should have XYZ tool (such as "C preprocessor") invoked on it 
first, etc, but it's probably easiest if programs using Goldie just invoke 
whatever other tools they need by themselves.

(Sorry if I've stil misunderstood - it's late over here ;) )
April 15, 2011
Re: Goldie Parsing System v0.4 Released - Now for D2
Am 15.04.2011 10:13, schrieb Nick Sabalausky:
> "Daniel Gibson" <metalcaedes@gmail.com> wrote in message 
> news:io8u12$132q$1@digitalmars.com...
>> Am 15.04.2011 09:50, schrieb Nick Sabalausky:
>>> Since you're talking about C, you'll probably want to run your original C
>>> code through the "preprocess-only" option of a real C compiler. (I 
>>> *think*
>>> DMC will do that.) Then parse the resulting "preprocessed C" files with
>>> Goldie. (Although if your goal is an HTOD-like tool, maybe you would need 
>>> to
>>> deal with the original un-preprocessed source directly.
>>
>> Why? Just call the preprocessor from your tool or from a wrapping script
>> and go on with the preprocessed C code. Should be much easier and more
>> compatible because C compilers ought to know how to preprocess correctly.
>> For GCC the option you're looking for is "-E", btw.
>>
> 
> If by "your tool" mean a program that uses Goldie to process C code, then 
> yea, that's what I meant.
> 

I meant Andrej's hypothetical tool using Goldie to process C code :-)
« First   ‹ Prev
1 2
Top | Discussion index | About this forum | D home