On 10.11.2013. 22:20, Jacob Carlborg wrote:
> I've been thinking quite long of how AST macros could look like in D. I've been posting my vision of AST macros here in the newsgroup a couple of times already. I've now been asked to create a DIP out of it, so here it is:
> 
> http://wiki.dlang.org/DIP50
> 

I took a look at it as here is my conclusion for now:

Statement and attribute macro examples look great. But I don't like Linq example. I don't think code like the following should be allowed.

query {
  from element in array
  where element > 2
  add element to data
}


From my point of view this whole idea is great as it makes it easier what is already possible. For example, with current behavior if I wanted to write.

foo {
  writeln("foo");
  writeln("foo again");
}

I would have to write:

mixin(foo!(q{
  writeln("foo");
  writeln("foo again");
}));

So the proposed behavior looks much nicer, and I agree with it as the content of foo block is actually written in D and I think whoever is reading it would be comfortable with it.


However, for other, non-D syntax-es I would prefer something like:

query q{
  from element in array
  where element > 2
  add element to data
}

Which can be handled by:

macro query (Context context, string dsl) {
    return domainSpecificLanguageToD(dsl);
}

This in terms is already possible by writing the following, it only allows to be written in a more readable way. And the q{ ... } notation clearly points out that there is something special going on. Also by passing such content as string user can implement custom (or call one of the predefined) tokenizer/lexer/parser.

mixin(query!(q{
  from element in array
  where element > 2
  add element to data
}));


I also don't like the <[ ... ]> syntax because:
1. Like others have said, it looks very foreign.
2. I don't think there is a need to add a new syntax.

I think that string concatenation is enough (at least for now), and if you want another syntax for templates you can write a macro for that.

For example:

macro myAssert (Context context, Ast!(bool) val, Ast!(string) str = null) {
  auto message = str ? "Assertion failure: " ~ str.eval : val.toString();
  auto msgExpr = literal(constant(message));

  return "
    if (!" ~ val ~ ")
      throw new AssertError(" ~ msgExpr ~ ");
  ";

  // or
  return astTemplate q{
    if (!$val)
      throw new AssertError($msgExpr);
  };
}

void main () {
    myAssert(1 + 2 == 4);
}


What do you guys think?

--
Luka

On 2013-11-13 00:26, Walter Bright wrote:

> It is limiting, but I don't know about extremely limiting. Eric
> Anderton's regex engine didn't need them. And, if you really do need
> them, you can always define the templates as regular names:
>
>     lessThan(a,b)
>     equals(a,b)
>
> etc. Not the greatest, but not unworkable.

I don't know what's so special about the regex engine. It takes a regex as a string and matches that against another string. Seems to be just like any other regular expression library I've used.

-- 
/Jacob Carlborg

On 2013-11-12 16:20, Dicebot wrote:

> I don't know much about linq but in macro context I'd expect it to
> generate an actual SQL query to database (and execute it) based on
> in-language filter statement (via reflection)

Exactly, spot on. The interesting thing there, in the context of macros, is converting the AST of the lambda to an string containing SQL code.

-- 
/Jacob Carlborg

On 2013-11-12 15:38, John Colvin wrote:

> for those of us entirely unfamiliar with linq, what is this supposed to
> do? Select people with name "John" from a collection of people, like in
> sql? It seems trivial to do this using filter, or am I missing
> something...?

The idea of that example is, as Dicebot said, to convert the AST of the lambda to a string containing SQL code then query the database and return the result. The interesting part here, in the context of macros, is to convert the lambda to an SQL string.

The filter is not applied on the result, it's converted to SQL and performed in the database.

-- 
/Jacob Carlborg

On 2013-11-12 17:14, John Colvin wrote:

> oh, I see. Would AST macros really be enough to make this work in D?
> "Arbitrary code" is a huge feature space in D, including much that
> doesn't map well to anything outside of a relatively low-level language,
> let alone SQL.
> I can see it quickly becoming a nightmare that would be worse than just
> issuing the predicate as an sql string or some generic equivalent.

Person.where(e => e.name == "John")

I'm thinking that we only need to convert the part that is prefixed with, in this example, "e". Any other code should be executed in the context of the caller. It should be possible to do this as well:

auto foo = "John";
auto result = Person.where(e => e.name == foo);

Which will result in the same SQL query.

I'm using a pluign to Ruby on Rails that does something similar but by overloading operators. The problem with this approach, in Ruby, is that you cannot overload operators like || and &&, so instead they overload | and & resulting in new problems like operator precedence. Example:

Person.where{ |e| (e.name == "John") & (e.address == "Main street") }

-- 
/Jacob Carlborg

On 2013-11-12 21:25, Dicebot wrote:

> I had an impression it was exactly the context in which linq was
> originally mentioned, no idea why discussion has moved from that to
> expressing DSL's (which is not really a problem in D) :)

My example is a DSL:

Person.where(e => e.name == "John");

-- 
/Jacob Carlborg

On 13.11.2013. 9:26, Jacob Carlborg wrote:
> On 2013-11-12 17:14, John Colvin wrote:
> 
>> oh, I see. Would AST macros really be enough to make this work in D?
>> "Arbitrary code" is a huge feature space in D, including much that
>> doesn't map well to anything outside of a relatively low-level language,
>> let alone SQL.
>> I can see it quickly becoming a nightmare that would be worse than just
>> issuing the predicate as an sql string or some generic equivalent.
> 
> Person.where(e => e.name == "John")
> 
> I'm thinking that we only need to convert the part that is prefixed with, in this example, "e". Any other code should be executed in the context of the caller. It should be possible to do this as well:
> 
> auto foo = "John";
> auto result = Person.where(e => e.name == foo);
> 
> Which will result in the same SQL query.
> 
> I'm using a pluign to Ruby on Rails that does something similar but by overloading operators. The problem with this approach, in Ruby, is that you cannot overload operators like || and &&, so instead they overload | and & resulting in new problems like operator precedence. Example:
> 
> Person.where{ |e| (e.name == "John") & (e.address == "Main street") }
> 

What about something like this?

class Person {

  macro where (Context context, Statement statement) {
    // ...
  }

}

auto foo = "John";
auto result = Person.where(e => e.name == foo);

// is replaced by
auto foo = "John";
auto result = Person.query("select * from person where person.name = " ~
sqlQuote(foo) ~ ";");

On 2013-11-12 21:36, deadalnix wrote:

> The thing I'd like to be able to do it to create a async/await/yield
> like mechanism, purely as library.

Please add this as an example to the DIP.

-- 
/Jacob Carlborg

On 11/13/2013 12:17 AM, Jacob Carlborg wrote:
> On 2013-11-13 00:26, Walter Bright wrote:
>
>> It is limiting, but I don't know about extremely limiting. Eric
>> Anderton's regex engine didn't need them. And, if you really do need
>> them, you can always define the templates as regular names:
>>
>>     lessThan(a,b)
>>     equals(a,b)
>>
>> etc. Not the greatest, but not unworkable.
>
> I don't know what's so special about the regex engine. It takes a regex as a
> string and matches that against another string. Seems to be just like any other
> regular expression library I've used.

What's special about it was its use of expression templates. It made for a nice demo of how to do such.

On 2013-11-12 21:55, Dmitry Olshansky wrote:

> Actually I couldn't shake off the feeling that macros are just CTFE
> functions on Ast objects. How objects are created and converted back to
> source code is a separate question.

Yes, and reflection.

> If we just had:
>
> //this would invoke compiler's parser at CTFE
> auto ast = "x = y;".astof
>
> and have it work at CTFE to return sensible AST (a big if btw).
>
> And then (after some manipulations):
> ast.toString() //get back a string of D code
>
> It may help code generation of D --> DSL.
>
> Alternatively one can implement D parser that works at CTFE and watch it
> crawl :)

You would still need to do strign mixin. Which results in a some unnecessary conversion between AST's and strings. See the list at the bottom of:

http://forum.dlang.org/thread/l5otb1$1dhi$1@digitalmars.com?page=13#post-l5vcct:242lit:241:40digitalmars.com

It also looks ugly.

-- 
/Jacob Carlborg