I know macros are an unpopular subject in D, but I think there's a way to get mostly equivalent capabilities to macros, without changing the language at all, simply by standardizing access to the compiler's own D language parser.

There are already 10 separate parsers for D here:
http://wiki.dlang.org/Lexers_Parsers

All these parsers are pretty much written from scratch (one even forked from dmd itself) and they all do pretty much the same thing: take a string of code, and turn it into an AST. Turning that AST back into valid D code is itself a rather difficult and error prone process, but if you could, then you could simply do something like:

import some.parser;
string foo(string bar) = unparse(IfDeclaration(parse(bar),parse(q{writeln("yay")}),parse(q{writeln("boo")}));

void main() {
  mixin foo!"true";
  mixin foo!"false";
}

D isn't a particularly hard to parse language, so it's not too hard to implement these parsers, but they're definitely reinventing the wheel, and there's no assurance that future language features won't break them. But within each D compiler (I'm pretty sure?) there is already a parser, that produces (and manipulates) an AST. For DMD, the parser is written in D itself! Using it would (in theory) be as simple as copying all the ddmd modules into your source directory and importing ddmd.parse. Eliminating the need to maintain all those third party parsers is as simple as having the compiler itself provide the parser as a module.

There is additionally a huge advantage to code being able to use the compiler's D parser, that third party parsers cannot match. The D compiler actually uses the AST generated by its parser. With the other parsers, you have to somehow figure out how to "unparse" their AST into an opaque string of D code, before you can supply that new code as a mixin, but if you used the compiler's own parser, it could accept an AST as a mixin omitting the "parse" step it normally does on mixins.

So the above would end up more like:

import std.parser;
AST foo(string bar) = IfDeclaration(parse(bar), parse(q{writeln("yay")}), parse(q{writeln("boo")}));


void main() {
  mixin foo!"true";
  mixin foo!"false";
}

omitting the complicated and error prone "unparse()" step entirely, not to mention removing the degradation in compilation speed if you have to unparse, and then re-parse an AST.

I'm not sure if it would be possible to have the parser available at runtime, but it seems like you could compile it into the program just like any other module.

On Tuesday, 9 February 2016 at 22:14:25 UTC, cy wrote:

> There are already 10 separate parsers for D here:
> http://wiki.dlang.org/Lexers_Parsers

A couple of dead links on that page...

* https://code.google.com/archive/p/dil/
* https://github.com/DDT-IDE/DDT/tree/master/org.dsource.ddt.dtool/src/dtool/parser
* https://github.com/Hackerpilot/libdparse/blob/master/src/std/d/parser.d
* https://github.com/Hackerpilot/libdparse/blob/master/src/std/d/lexer.d

Anyone have some updates on those links?

On 2016-02-09 23:14, cy wrote:

> So the above would end up more like:
>
> import std.parser;
> AST foo(string bar) = IfDeclaration(parse(bar),
> parse(q{writeln("yay")}), parse(q{writeln("boo")}));
>
>
> void main() {
>    mixin foo!"true";
>    mixin foo!"false";
> }

That's very close to AST macros. The only difference is the need to parse the input, "bar" in this case.

-- 
/Jacob Carlborg

On Wednesday, 10 February 2016 at 12:24:49 UTC, Jacob Carlborg wrote:
> That's very close to AST macros. The only difference is the need to parse the input, "bar" in this case.

Well yes, that's my point. AST macros don't need any new special syntax or language support, just access to the compiler's syntax parser. Then you can use normal mixins to get the code you want.

I dunno. I don't really care that much. It was just an idea I had that turned into way too many words.

On 2016-02-11 05:22, cy wrote:

> Well yes, that's my point. AST macros don't need any new special syntax
> or language support, just access to the compiler's syntax parser. Then
> you can use normal mixins to get the code you want.

Well, language support _is_ required. Perhaps not new syntax though.

In my macro DIP [1] basically the only difference is that one use the "macro" keyword when declaring a function, then "mixin" is not necessary on the call site. Plus parsing the input is not necessary, that's handled automatically. That's all, the rest is just AST manipulation using CTFE.

[1] http://wiki.dlang.org/DIP50

-- 
/Jacob Carlborg