On Monday, 18 November 2013 at 22:09:35 UTC, Dicebot wrote:
> On Monday, 18 November 2013 at 21:09:26 UTC, Jacob Carlborg wrote:
>> On 2013-11-18 16:59, Dicebot wrote:
>>
>>> Anything that allows it routinely should be banned whenever macros are
>>> accepted or not. Modifying existing "normal" symbols = no way.
>>
>> The whole point of macros is to modify symbols. Turning this:
>>
>> auto p = Person.where(q => q.name == "John");
>>
>> To an SQL query.
>
> You don't need to modify actual input lambda for this to become SQL query. I fully support macros as convenient way for AST reflection and related code generation but not a single step further. Changing semantics of existing valid piece of code is never worth it.

And that's exactly the kind of thing Walter doesn't want to see.

On 2013-11-18 23:09, Dicebot wrote:

> You don't need to modify actual input lambda for this to become SQL
> query. I fully support macros as convenient way for AST reflection and
> related code generation but not a single step further. Changing
> semantics of existing valid piece of code is never worth it.

Well, it depends on how you look at it. You don't need to modify the lambda, but you're changing the meaning of it.

-- 
/Jacob Carlborg

On Monday, 18 November 2013 at 22:47:59 UTC, Timon Gehr wrote:
> So you oppose macro attributes?

Yes. For me proper "macro attribute" is just an UDA that can be used by macro to tweak code generation. Actually I think I'd oppose a very large part of this specific DIP when it comes to exact syntax and usage models - I am more interested in "AST" than in "macro" part.

On Tuesday, 19 November 2013 at 07:23:52 UTC, Jacob Carlborg wrote:
> Well, it depends on how you look at it. You don't need to modify the lambda, but you're changing the meaning of it.

Not really. You won't say that this snippet changes the meaning of lambda, do you?

```
foo(a => a*2);
// ...
bool foo(T)(T func)
{
    // does not even call `func` and does something unrelated
    return is(ReturnType!T == int);
}
```

This is pretty much the same. Functions and templates can do arbitrary things unrelated to their input and we do not call it "changing the meaning" of the input. Macros are not much different here. Though I'd still prefer requirement for explicit "macro!" syntax or something similar, just to make clear that code gen is incoming.

On 2013-11-19 14:16, Dicebot wrote:

> Not really. You won't say that this snippet changes the meaning of
> lambda, do you?
>
> ```
> foo(a => a*2);
> // ...
> bool foo(T)(T func)
> {
>      // does not even call `func` and does something unrelated
>      return is(ReturnType!T == int);
> }
> ```

No, I guess not.

-- 
/Jacob Carlborg

On Tuesday, 19 November 2013 at 13:10:44 UTC, Dicebot wrote:
> On Monday, 18 November 2013 at 22:47:59 UTC, Timon Gehr wrote:
>> So you oppose macro attributes?
>
> Yes. For me proper "macro attribute" is just an UDA that can be used by macro to tweak code generation. Actually I think I'd oppose a very large part of this specific DIP when it comes to exact syntax and usage models - I am more interested in "AST" than in "macro" part.

This topic really took off.  I'm afraid to say that I've only gotten through about half of it so far.

I noticed computation expressions were mentioned, but has anyone by chance noted this little-known feature of F#?  It sounds exactly like what is being proposed here.
http://msdn.microsoft.com/en-us/library/vstudio/dd233212.aspx

I don't believe I've ever seen F#'s code quotations being used in real code.  But then again, F# doesn't have CTFE or code mixins.

The LINQ sample in the DIP kinda rubs me the wrong way.  LINQ query syntax (and for that matter, F# computation expressions) does not generate imperative code, but rather a chain of extension methods.  More importantly, it does not eagerly chomp through all the data.  It runs on IEnumerable, and is therefore lazy. (Like D ranges.)  I wouldn't mind, however, if I saw these macros generate state machines specifically made for a given query.

On 13.11.2013. 13:47, Jacob Carlborg wrote:
> On 2013-11-13 09:15, luka8088 wrote:
>> I think that string concatenation is enough (at least for now), and if you want another syntax for templates you can write a macro for that.
> 
> Strings are far from enough. Then you have missed the whole idea. It's not supposed to be syntax sugar for string mixins.
> 

Oh, I see. It seems that I indeed missed the point.

It seems to me that this DIP could to be granulated into: AST reflection, AST manipulation and AST template. The reason for this, as far as I see, is that although they overlap in some cases, in other they could be used independently and it could help with understanding.

Regarding AST reflection and AST manipulation, I have been thinking and found a few situations that bugs me:

In the example of

auto person = Person.where(e => e.name == "John");

what would be the response of:

auto f = e => e.name == "John";
auto person = Person.where(f);

I guess it should be a compiler error because f could be modified at runtime and f's body could be hidden. So basically AST macros are something that look like D but actually are not D. This seems to me like an example that look good as a toy example but fails on the larger scale so I agree with Water in this matter (unless better examples/explanations are provided). Or maybe I am not seeing it clear enough.

But! Regarding AST templates, I think something like the following would be a great syntax sugar:

int i = 5;

Ast a = t{ // t as template
  int j = $i;
  int k = $i + $i;
};

// ... alter ast here if necessary

mixin(a);

Where syntax (and semantic) in template must be strictly D. The only difference would be $ sign which would allow reference to symbols outside the template.

So if you would want to have conditional template you could write:

int i = 5;
int j;
bool b = false;

mixin(t{
  $j = $i;
  static if ($b) {
    $j++;
  }
});

On 18.11.2013. 6:05, Walter Bright wrote:
> On 11/17/2013 7:14 PM, deadalnix wrote:
> 
> Ok, then I'm not seeing what AST macros do that lazy parameters / template overloading / mixin templates do not?


Well, this is just a small piece of the puzzle but I would like to be able to write (as syntax sugar):

query q{
  select data;
}

a compiler to rewrite it to:

mixin(query!(q{
  select data;
}));

if the following exists:

mixin statement query (string code) {
  // ...
}

Maybe I am the only one but I found that string mixins + code generation are used this way even i Phobos and I find the first syntax much more cleaner.

Or this is not in the general interest?

On 2013-11-19 19:32, luka8088 wrote:

> Oh, I see. It seems that I indeed missed the point.
>
> It seems to me that this DIP could to be granulated into: AST
> reflection, AST manipulation and AST template. The reason for this, as
> far as I see, is that although they overlap in some cases, in other they
> could be used independently and it could help with understanding.
>
> Regarding AST reflection and AST manipulation, I have been thinking and
> found a few situations that bugs me:
>
> In the example of
>
> auto person = Person.where(e => e.name == "John");
>
> what would be the response of:
>
> auto f = e => e.name == "John";
> auto person = Person.where(f);
>
> I guess it should be a compiler error because f could be modified at
> runtime and f's body could be hidden.

I haven't thought about that. But AST macros work at compile time

> So basically AST macros are
> something that look like D but actually are not D. This seems to me like
> an example that look good as a toy example but fails on the larger scale
> so I agree with Water in this matter (unless better
> examples/explanations are provided). Or maybe I am not seeing it clear
> enough.
>
> But! Regarding AST templates, I think something like the following would
> be a great syntax sugar:
>
> int i = 5;
>
> Ast a = t{ // t as template
>    int j = $i;
>    int k = $i + $i;
> };
>
> // ... alter ast here if necessary
>
> mixin(a);

So you don't like that it's not a "mixin" there with AST macros?

> Where syntax (and semantic) in template must be strictly D. The only
> difference would be $ sign which would allow reference to symbols
> outside the template.

If the semantics need to be valid this is very limited. Even more limiting than what we have now with string mixins. Note that with AST macros, the input needs to syntactically valid. The resulting AST of the macro needs to both be syntactically and semantically valid D code.

-- 
/Jacob Carlborg

On 19.11.2013. 21:32, Jacob Carlborg wrote:
> On 2013-11-19 19:32, luka8088 wrote:
> 
>> Oh, I see. It seems that I indeed missed the point.
>>
>> It seems to me that this DIP could to be granulated into: AST reflection, AST manipulation and AST template. The reason for this, as far as I see, is that although they overlap in some cases, in other they could be used independently and it could help with understanding.
>>
>> Regarding AST reflection and AST manipulation, I have been thinking and found a few situations that bugs me:
>>
>> In the example of
>>
>> auto person = Person.where(e => e.name == "John");
>>
>> what would be the response of:
>>
>> auto f = e => e.name == "John";
>> auto person = Person.where(f);
>>
>> I guess it should be a compiler error because f could be modified at runtime and f's body could be hidden.
> 
> I haven't thought about that. But AST macros work at compile time

Well, do think about that :)

auto f = e => e.name == "John";
if (true)
  f = e => e.name == "Jack";

auto person = Person.where(f);

I can think of a many use cases where conditional query generation is required. But I don't see how this could be done using AST macros.

> 
>> So basically AST macros are
>> something that look like D but actually are not D. This seems to me like
>> an example that look good as a toy example but fails on the larger scale
>> so I agree with Water in this matter (unless better
>> examples/explanations are provided). Or maybe I am not seeing it clear
>> enough.
>>
>> But! Regarding AST templates, I think something like the following would be a great syntax sugar:
>>
>> int i = 5;
>>
>> Ast a = t{ // t as template
>>    int j = $i;
>>    int k = $i + $i;
>> };
>>
>> // ... alter ast here if necessary
>>
>> mixin(a);
> 
> So you don't like that it's not a "mixin" there with AST macros?
Um, sorry. I don't understand the question.

This example (and suggestion) was suppose to show that we could allow AST mixins as well as string mixins. It should behave like string mixins but the main difference is that AST is structured so it much cleaner to manipulate.

> 
>> Where syntax (and semantic) in template must be strictly D. The only difference would be $ sign which would allow reference to symbols outside the template.
> 
> If the semantics need to be valid this is very limited. Even more limiting than what we have now with string mixins. Note that with AST macros, the input needs to syntactically valid. The resulting AST of the macro needs to both be syntactically and semantically valid D code.
> 

When I first started programing I was introduced to static typing. Then I discovered dynamic typing and dynamic structures (in php mostly). And for me at that time it was much better and easier to use. However, over time I learned that sometimes limitations are good and now like static typing much more :) My point here is that limits are sometimes good, although that don't seem that way.

I think that we should think of a more complex (real world) examples and than real issues will reveal themselves.