On Tuesday, 12 November 2013 at 02:37:03 UTC, Andrei Alexandrescu wrote:
> On 11/10/13 11:46 PM, Jacob Carlborg wrote:
>> On 2013-11-11 02:46, bearophile wrote:
>>
>>> It's also useful to take a look at what F# is doing:
>>> http://tomasp.net/blog/2013/computation-zoo-padl/index.html
>>
>> I'll do that. I've been looking at several languages, mainly Scala. But
>> I have not looked at F#.
>
> There is significant regret about the way macros are defined in Scala. Probably not an example to follow.
>
> Andrei

Knowing what went wrong is important so as not to repeat past mistakes, is there more info somewhere to read on what went wrong?

--rt

On 11/11/13 9:16 PM, Rob T wrote:
> On Tuesday, 12 November 2013 at 02:37:03 UTC, Andrei Alexandrescu wrote:
>> On 11/10/13 11:46 PM, Jacob Carlborg wrote:
>>> On 2013-11-11 02:46, bearophile wrote:
>>>
>>>> It's also useful to take a look at what F# is doing:
>>>> http://tomasp.net/blog/2013/computation-zoo-padl/index.html
>>>
>>> I'll do that. I've been looking at several languages, mainly Scala. But
>>> I have not looked at F#.
>>
>> There is significant regret about the way macros are defined in Scala.
>> Probably not an example to follow.
>>
>> Andrei
>
> Knowing what went wrong is important so as not to repeat past mistakes,
> is there more info somewhere to read on what went wrong?

Not yet. Martin mentioned to me in 2012 he doesn't think they took the right turns with defining macros. I don't know whether more information is available at the moment.

Andrei

On Tuesday, 12 November 2013 at 05:59:46 UTC, Andrei Alexandrescu wrote:
> Not yet. Martin mentioned to me in 2012 he doesn't think they took the right turns with defining macros. I don't know whether more information is available at the moment.
>
> Andrei

Loosely related, but worth watching : http://www.infoq.com/presentations/data-types-issues

On 2013-11-12 03:37, Andrei Alexandrescu wrote:

> There is significant regret about the way macros are defined in Scala.
> Probably not an example to follow.

Unfortunately it's not easy to do anything about it if you cannot give us any more details to why. So far I haven't found anything.

-- 
/Jacob Carlborg

On 11/10/2013 1:20 PM, Jacob Carlborg wrote:
> I've been thinking quite long of how AST macros could look like in D. I've been
> posting my vision of AST macros here in the newsgroup a couple of times already.
> I've now been asked to create a DIP out of it, so here it is:
>
> http://wiki.dlang.org/DIP50


I confess I have some serious reservations about AST macros in general:

1. I've seen very heavy use of such macros in macro assemblers. What happens is people use it to invent their own (very baroque) language on top of the existing assembler one. Anyone trying to read the code has to learn this new unique language, and given the limitations of the macro capability, it often comes with unfixable bizarre corner cases and limitations.

This got so bad that in some cases I know of, the poor sap who is given the job of making improvements resorted to running the assembler to generate an object file, then disassembling the object file back into source code! Something went very wrong for this to be necessary. I know in my own assembler work I have abandoned all use of macros.

(Note that macros in typical assemblers are far more powerful than C's macro language, which stands out for being pathetically underpowered.)

2. Macros are something that should be used very sparingly. However, this is not what happens. I used to be perplexed that even top shelf C/C++ programmers tend to write what I not so humbly consider to be pretty abusive use of macros. Now it just saddens me.

3. Lisp is a language that encourages users to write macros to pretty much invent a custom language for the task at hand. Heavy use of such makes the code unrecognizable as being Lisp code. I believe a language ought to have some "anchors" that the person reading the code can reliably recognize. (GO has taken this idea pretty far.)

4. AST macros, pretty much by definition, manipulate the AST. However, if you examine the semantic routines in DMD, a *lot* more is required to do more significant things. The symbol table must be consulted, etc. I just don't see how one could hope to implement something like function overloading with AST macros. Or template type deduction. Or closure semantics, attribute inference, etc. Sure, some forms of foreach can be rewritten into for statements by manipulating the AST, but not the more interesting forms, such as the ones that deal with opApply().

There are some statements in the DIP about the Context parameter and it'll provide semantic information, but I don't see how this can work without making it an ever-expanding collection of arbitrary methods.

5. We've said "no" in the past to things like user-defined tokens, which are closely related.

6. Note that D has a limited form of AST macros with mixin templates.


To sum up, any AST macro proposal has some tough barriers to get over. Within its necessarily severe semantic limitations, it has to deliver a pretty compelling set of solutions, while avoiding the morass of incomprehensibility that far too many macro systems become in practice.

For example, the Linq example in the DIP is not compelling, as aesthetically nicer code can be written using D's ranges and algorithms:

    auto data = arr.filter!(x => x > 5).array;

I see no compelling advantage in trying to make D code look like C#; to be blunt it's like the old:

    #define BEGIN {
    #define END }

macros used in old C code to make it look like Pascal.

I forgot to mention that "expression templates" can be used in D in an equivalent manner that they are used in C++, and can cover much of the functionality used in AST macros.

Eric Anderton who wrote a regex expression template module a few years back:

http://www.dsource.org/projects/ddl/browser/trunk/meta/regex.d

On 11/12/2013 1:55 AM, Walter Bright wrote:
> I forgot to mention that "expression templates" can be used in D in an
> equivalent manner that they are used in C++, and can cover much of the
> functionality used in AST macros.
>
> Eric Anderton who wrote a regex expression template module a few years back:
>
> http://www.dsource.org/projects/ddl/browser/trunk/meta/regex.d

Here's a project to add Linq support to C++:

http://www.codeproject.com/Articles/17844/CLinq-LINQ-support-for-the-C-CLI-language

There's also:

http://pfultz2.github.io/Linq/

https://github.com/hjiang/linqxx/wiki

On Tuesday, 12 November 2013 at 09:55:20 UTC, Walter Bright wrote:
> I forgot to mention that "expression templates" can be used in D in an equivalent manner that they are used in C++,

They are crippled compared to C++, because you have no control over the return type of opCmp and opEquals, which means that you can't have expression templates involving comparisons. That's extremely limiting.


My feeling with AST macros is:

(1) AST macros are basically syntax sugar for string mixins. The one and only thing that string mixins have in their favour, is that they have almost no syntax. But that is actually a *huge* plus.

Because it's syntax sugar, an AST macro syntax would be most convincing if you took some existing string mixins from various real projects, and show how beautiful they would become with macros.


(2) The big functionality we don't have, is comprehensive reflection. I'd like to see more thought in that area. It seems challenging to define reflection in a way that doesn't expose compiler internals.

The question I find particularly interesting is, "if we had macros, would reflection look different?"

On Tuesday, 12 November 2013 at 09:20:07 UTC, Walter Bright wrote:
>
> 5. We've said "no" in the past to things like user-defined tokens, which are closely related.
>
Regarding this point, I get that arbitrary expansion is out, but would you be amenable to adding a small, _strictly-defined_ set of operators that are compile-time errors unless overloaded?

>     #define BEGIN {
>     #define END }
>
> macros used in old C code to make it look like Pascal.

A plague o' their houses!

-Wyatt

On 2013-11-12 10:20, Walter Bright wrote:

> For example, the Linq example in the DIP is not compelling, as
> aesthetically nicer code can be written using D's ranges and algorithms:
>
>      auto data = arr.filter!(x => x > 5).array;

I was actually not the one that added the Linq examples. I like the above better. I've used a similar example as well:

auto person = Person.where(e => e.name == "John");

Translates to:

select * from person where name = 'John'

But I don't see how that can currently be done in D. Operator overloading in D isn't flexible enough. For example, when implementing opEquals, you don't know if it's "==" or "!=" that's being called. Same with the comparison operators.

> I see no compelling advantage in trying to make D code look like C#; to
> be blunt it's like the old:
>
>      #define BEGIN {
>      #define END }
>
> macros used in old C code to make it look like Pascal.

I agree with this.

-- 
/Jacob Carlborg