Is there a function in D for evaluating expressions before compile time?.

On Wednesday, 30 October 2013 at 22:39:13 UTC, jgetner wrote:
> Is there a function in D for evaluating expressions before compile time?.

what? how it is supposed to work? O_o

Do you want to evaluate an expression at CT or to parse an expression?

D has a powerful constant-folding step during compilation, where whole functions can be evaluated, provided they contain only a subset of D (almost all of D, except taking adresses/pointers and generating classes, IIRC).

http://en.wikipedia.org/wiki/Compile_time_function_execution http://dlang.org/function.html#interpretation


Now, if you want parsing specifically, what do you want to get? A parse
tree? In that case, you can for example use one of my projects, Pegged,
which generates CT-compatible parsers (that is, functions that can use CTFE
to parse a string at CT).
Parse trees can also be manipulated at compile-time, to modify the
expression, and then collapsed down again to a new expression, if that's
what is needed.

https://github.com/PhilippeSigaud/Pegged

On 10/31/2013 07:10 AM, Philippe Sigaud wrote:
> Now, if you want parsing specifically, what do you want to get? A parse
> tree? In that case, you can for example use one of my projects, Pegged,
> which generates CT-compatible parsers (that is, functions that can use
> CTFE to parse a string at CT).
> Parse trees can also be manipulated at compile-time, to modify the
> expression, and then collapsed down again to a new expression, if that's
> what is needed.
>
> https://github.com/PhilippeSigaud/Pegged

Have any serious grammars been tested with this?
Are there any shortcomings?

The examples directory shows different grammars, from JSON to XML to C to D.

I used it with the D grammar, but the one on the website is woefully inadequate (some mistakes, out of date compared to the compiler and written in a somewhat convoluted style full of left-recursion). The shortcomings are that the generated parser is quite slow compared to other D parsers.

That comes from my coding, of course: Pegged generates a simple recursive descent parser. I guess I could push for a better engine, but I'm waiting for CTFE to get a bit better.

The advantages are nice, though: full parse tree, semantic actions, and so on.

On 11/1/13 1:59 PM, Philippe Sigaud wrote:
> The examples directory shows different grammars, from JSON to XML to C to D.
>
> I used it with the D grammar, but the one on the website is woefully
> inadequate (some mistakes, out of date compared to the compiler and
> written in a somewhat convoluted style full of left-recursion). The
> shortcomings are that the generated parser is quite slow compared to
> other D parsers.
>
> That comes from my coding, of course: Pegged generates a simple
> recursive descent parser. I guess I could push for a better engine, but
> I'm waiting for CTFE to get a bit better.
>
> The advantages are nice, though: full parse tree, semantic actions, and
> so on.

I have long believed, and continue to believe, that this kind of work is strategic for D. There has been evidence, too - the code I pushed at work recently included a generated lexer replacing a hand-written lexer with a generated component, and people did notice. Compile-time and run-time improvements sealed the deal.

To succeed against established languages, D must not only do well where others also do well. It's not even enough to do great where others do well. It must do well (or great!) where others don't even stand a chance. Regex and Pegged are in that category.

Bugs stopping Pegged from going forward should receive high priority. I also encourage others to participate to that and similar work.


Andrei

Andrei Alexandrescu wrote:

> Bugs stopping Pegged from going forward should receive high priority.

One prerequisite for every PEG-Parser is, that the language has to be designed to be without any ambiguity.

This is not the case for D, because of its evolution based on recursive descent parsing including tricks.

It is therefore not sufficient to eliminate bugs in Pegged.

-manfred

On 11/1/13 9:06 PM, Manfred Nowak wrote:
> Andrei Alexandrescu wrote:
>
>> Bugs stopping Pegged from going forward should receive high priority.
>
> One prerequisite for every PEG-Parser is, that the language has to be
> designed to be without any ambiguity.
>
> This is not the case for D, because of its evolution based on recursive
> descent parsing including tricks.
>
> It is therefore not sufficient to eliminate bugs in Pegged.
>
> -manfred

That would mean Pegged should accommodate tricks. Very few languages don't have them.

Andrei

Manfred:
>One prerequisite for every PEG-Parser is, that
>the language has to be designed to be without any ambiguity.
>
>This is not the case for D, because of its evolution
> based on recursive descent parsing including tricks.
>
>It is therefore not sufficient to eliminate bugs in Pegged.

That's right. I should test it with 2.064, though, as soon as it's out. I also have a fully recursive parser that manages ambiguity quite well (gives all possible parse trees), but that's not what I want for a programming language.

What we need is a parser able to deal with left-recursive BNF grammar. That way, we can feed it the D grammar.

Note that Pegged has no problem with the official C grammar or the officiel XML specification...

Andrei:
> That would mean Pegged should accommodate tricks. Very few languages
don't have them.

That was ma conclusion also, even if that means stepping outside of what a
PEG is. I just fear losing some of their proprieties.
I really like the composability of PEGs: you can inject / call another
grammar inside another one: creating a grammar becomes a bit like
structured programming, with functions and recursions. That means you can
'grow' your grammar by steps. That's really really nice to have.

Also, discarding the 'scannerless' part of PEGs (to connect the parser to the token range resulting from DScanner, for example), or dealing with left-recursion means changing the internal engine.

Of course, that can be done : the engine could be changed according to compile-time parameters. I 'just' need to code the parsers themsevels :)

Bah, you're getting me all interested in it again! I wanted to write a D tutorial around a ray-tracer!

On 11/2/13 1:19 AM, Philippe Sigaud wrote:
> Bah, you're getting me all interested in it again! I wanted to write a D
> tutorial around a ray-tracer!

If you want maximum impact, Pegged is it. (Not to say tutorials aren't important, but probably the skill to work on Pegged is narrower.)

Andrei