On 05/23/12 17:17, d coder wrote:
> I am trying to use Pegged for compile time parsing. The only issue I am facing is with the compile time memory. I have a rather simple grammar, mostly derived from the arithmetic.d example that comes bundled with Pegged.
> 
> I am wondering if the memory could be optimized by fine-tuning the code in the grammar that I have written. I am pasting the code down here for suggestions. On my 64-bit linux machine my small application is taking as much as 2.2GB of RAM at the time of compiling the code using DMD 2.059. I am using the latest Pegged code from github.

It really appears to be that heavy - i once tried to use it for some compile time parsing, but it turned out that just importing the module was so expensive (added seconds to every compiler invocation, iirc) and it got disqualified before i even had a chance to try it...

artur

> and it got disqualified before i even had a chance to try it...
>

Are you using some alternative? :-)

On 05/23/12 17:50, d coder wrote:
> 
>     and it got disqualified before i even had a chance to try it...
> 
> 
> Are you using some alternative? :-)

I gave up on the idea of parsing at compile time.

I was using GDC, and recompiling any file that imported that module
was so slow that it made no sense to do things like that - I'm pretty
sure that I could run a standalone parser app every time the D file got
recompiled and this would take less time...
Maybe Pegged works nicely when used in a normal (ie non-CTFE) context,
but I never tried that.

artur

On 5/23/12, Artur Skawina <art.08.09@gmail.com> wrote:
> I was using GDC, and recompiling any file that imported that module was so slow that it made no sense to do things like that

Have you guys tried to use asModule? https://github.com/PhilippeSigaud/Pegged/wiki/Grammars-as-D-Modules

> Have you guys tried to use asModule? https://github.com/PhilippeSigaud/Pegged/wiki/Grammars-as-D-Modules
>

Thanks for the idea. I just gave it a try and the memory came down by about 50%. Additionally the compilation became much faster.

But that might still not be enough :-( I will try to simplify my grammar rules for now.

Regards
- Puneet

> But that might still not be enough


Some 50-100MB additional memory is getting consumed by the compiler each time I parse a simple expression like "foo > 32;". In the end, if I parse 20 such expressions, the memory usage tops 2GB even while using asModule.

Regards
- Puneet

On Wed, May 23, 2012 at 7:40 PM, d coder <dlang.coder@gmail.com> wrote:
>
>> Have you guys tried to use asModule? https://github.com/PhilippeSigaud/Pegged/wiki/Grammars-as-D-Modules
>
>
> Thanks for the idea. I just gave it a try and the memory came down by about 50%. Additionally the compilation became much faster.

I never measured memory when I use asModule, since for me the compilation time becomes quite small. Even the D grammar (the biggest I tried with Pegged, about 200 rules, so an order of magnitude bigger than your grammar) becomes a module in a few seconds, and my computer is not top of the line.


> But that might still not be enough :-( I will try to simplify my grammar rules for now.

Maybe you can try to inline it some? See https://github.com/PhilippeSigaud/Pegged/wiki/Optimizations


Philippe

On Wed, May 23, 2012 at 5:38 PM, Artur Skawina <art.08.09@gmail.com> wrote:
> On 05/23/12 17:17, d coder wrote:
>> I am trying to use Pegged for compile time parsing. The only issue I am facing is with the compile time memory. I have a rather simple grammar, mostly derived from the arithmetic.d example that comes bundled with Pegged.
>>
>> I am wondering if the memory could be optimized by fine-tuning the code in the grammar that I have written. I am pasting the code down here for suggestions. On my 64-bit linux machine my small application is taking as much as 2.2GB of RAM at the time of compiling the code using DMD 2.059. I am using the latest Pegged code from github.
>
> It really appears to be that heavy - i once tried to use it for some compile time parsing, but it turned out that just importing the module was so expensive (added seconds to every compiler invocation, iirc) and it got disqualified before i even had a chance to try it...

Yeah, I'm seeing that too. I'm (slooowly) working on another engine using functions instead of inner classes as now, to see if that changes anything.

Else, I guess it's possible to generate a parser somewhat akin to hand-made parsers, with lots of gotos and loops.
> artur