Davidl wrote:
> heh, mixin an assert is bad, compile time is even longer :p
?? It's no different to a static assert.
> Though you are kinda sticking with CTFE & mixin , I'm pretty
> sure you would vote my syntax of macro for macro, right? :)
No. It's nowhere near clean enough yet. The code below, works right now.
The challenge is to make it nicer.

//--------------------------
// This is a macro library, defining a 'vectorize' macro.
//--------------------------
module Blade;
import ExpressionParser; //common code for any macros which use D syntax
public import std.stdio;

// CTFE macro: returns the code to be mixed in.
// prints all symbols used in the expression
char [] describe(char [] expr, char [][3][] symbols)
{
    char [] result = "Expression:" ~ expr
	~ "\nSYMBOL \tTYPE\t\tVALUE\n";
    for (int i=0; i<symbols.length; ++i) {
        result ~= symbols[i][0] ~ " \t" ~ symbols[i][1]
	~ "  \t" ~ symbols[i][2] ~ \n;
    }
    return `printf("` ~ result ~ `");`;
}

// TO BE MIXED IN -- inserts the boilerplate code for parsing & type extraction,
// then gets the macro text from the named function.
char [] vectorize(char [] expr)
{
    return expressionMacro("describe", expr);
}

//--------------------------
// And this is how the user sees it.
//--------------------------
import Blade;

void main()
{
    auto a = [1.24, 5.35, 324, 2454];
    double [] big = [3.44, 5565, 1.45, 67.1];
    const yx=-2.354e-68Li;
    alias big c;

    mixin(vectorize("a  +=yx *big -c"));
}

===============================
OUTPUT
===============================

Expression:a  +=yx *big -c
SYMBOL  TYPE            VALUE
a       double[4]       a
yx      ireal           -2.354e-68i
big     double[]        big
c       double[]        big

Note that the constant has been evaluated, and the aliases resolved.

And if you change the last line to:

mixin(vectorize("a  +=yx *big -d"));

you get:

test.d(10): Error: undefined identifier d

That happens automatically. Note that the line number is correct.

That expressionMacro must be really horrible, right?
Actually, no. It just makes use of some trivial identifier and operator parsing routines, and applies the .stringof property.
-----------

// Given an expression string 'expr', make a string to mixin in,
// which will have an array of 3 strings for each symbol in the
// expression. [0]=symbol name, [1]=type, [2]=value
char[] getAllIdentifiers(char [] expr)
{
    char [] result;
    char [] rest;
    char [] a;
    char [] b;
    a = parseQualifiedName(expr, rest);
    result = "[[`" ~ a ~"`,typeof(" ~ a ~ ").stringof, " ~a ~ ".stringof]";
    while (rest!="") {
        b = parseOperator(rest, rest);
        char [] r;
        a = parseQualifiedName(rest, r);
        rest=r;
        result ~= "," ~"[`" ~ a ~"`,typeof(" ~ a ~ ").stringof, " ~a ~ ".stringof]";
    }
    return result ~ "]";
}

// Actually it should process the arguments into a standard, convenient form. For now, just pass them on, together with the type information.
char [] expressionMacro(char [] macroname, char [] args)
{
    return "mixin ("~macroname ~"(`" ~ args ~"`," ~ getAllIdentifiers(args)~ "));";
}

-----------
IMHO, this is already pretty good, despite the fact that there's a couple of workarounds for compiler bugs in there.
I just can't believe how powerful this language is now.

Nice work!
The compiler now is damned powerful :)

Though I still prefer less string operation in compile time, any idea of improving the
macro syntax I posted? You see, D is going to bring AST macro in future releases. Let's
discuss the syntax it will use?

Don Clugston wrote:
> Davidl wrote:
>> heh, mixin an assert is bad, compile time is even longer :p
> ?? It's no different to a static assert.
>> Though you are kinda sticking with CTFE & mixin , I'm pretty
>> sure you would vote my syntax of macro for macro, right? :)
> No. It's nowhere near clean enough yet. The code below, works right now.
> The challenge is to make it nicer.
> 
> //--------------------------
> // This is a macro library, defining a 'vectorize' macro.
> //--------------------------
> module Blade;
> import ExpressionParser; //common code for any macros which use D syntax
> public import std.stdio;
> 
> // CTFE macro: returns the code to be mixed in.
> // prints all symbols used in the expression
> char [] describe(char [] expr, char [][3][] symbols)
> {
>     char [] result = "Expression:" ~ expr
>     ~ "\nSYMBOL \tTYPE\t\tVALUE\n";
>     for (int i=0; i<symbols.length; ++i) {
>         result ~= symbols[i][0] ~ " \t" ~ symbols[i][1]
>     ~ "  \t" ~ symbols[i][2] ~ \n;
>     }
>     return `printf("` ~ result ~ `");`;
> }
> 
> // TO BE MIXED IN -- inserts the boilerplate code for parsing & type extraction,
> // then gets the macro text from the named function.
> char [] vectorize(char [] expr)
> {
>     return expressionMacro("describe", expr);
> }
> 
> //--------------------------
> // And this is how the user sees it.
> //--------------------------
> import Blade;
> 
> void main()
> {
>     auto a = [1.24, 5.35, 324, 2454];
>     double [] big = [3.44, 5565, 1.45, 67.1];
>     const yx=-2.354e-68Li;
>     alias big c;
> 
>     mixin(vectorize("a  +=yx *big -c"));
> }
> 
> ===============================
> OUTPUT
> ===============================
> 
> Expression:a  +=yx *big -c
> SYMBOL  TYPE            VALUE
> a       double[4]       a
> yx      ireal           -2.354e-68i
> big     double[]        big
> c       double[]        big
> 
> Note that the constant has been evaluated, and the aliases resolved.
> 
> And if you change the last line to:
> 
> mixin(vectorize("a  +=yx *big -d"));
> 
> you get:
> 
> test.d(10): Error: undefined identifier d
> 
> That happens automatically. Note that the line number is correct.
> 
> That expressionMacro must be really horrible, right?
> Actually, no. It just makes use of some trivial identifier and operator parsing routines, and applies the .stringof property.
> -----------
> 
> // Given an expression string 'expr', make a string to mixin in,
> // which will have an array of 3 strings for each symbol in the
> // expression. [0]=symbol name, [1]=type, [2]=value
> char[] getAllIdentifiers(char [] expr)
> {
>     char [] result;
>     char [] rest;
>     char [] a;
>     char [] b;
>     a = parseQualifiedName(expr, rest);
>     result = "[[`" ~ a ~"`,typeof(" ~ a ~ ").stringof, " ~a ~ ".stringof]";
>     while (rest!="") {
>         b = parseOperator(rest, rest);
>         char [] r;
>         a = parseQualifiedName(rest, r);
>         rest=r;
>         result ~= "," ~"[`" ~ a ~"`,typeof(" ~ a ~ ").stringof, " ~a ~ ".stringof]";
>     }
>     return result ~ "]";
> }
> 
> // Actually it should process the arguments into a standard, convenient form. For now, just pass them on, together with the type information.
> char [] expressionMacro(char [] macroname, char [] args)
> {
>     return "mixin ("~macroname ~"(`" ~ args ~"`," ~ getAllIdentifiers(args)~ "));";
> }
> 
> -----------
> IMHO, this is already pretty good, despite the fact that there's a couple of workarounds for compiler bugs in there.


> I just can't believe how powerful this language is now.

I agree, the CTFE & mixins have really put D over the top, and this is just the beginning.

If we are talking about improving the current syntax I'd really like to see a symbol for explicitly calling a function at call time, having to guess what the code is doing is a PITA , especially if you aren't the original developer.

I also think compile time string manipulation could be improved, or at least documented.

Do you have this new code up somewhere ?

Thanks,
Charlie

Davidl wrote:
> Nice work!
> The compiler now is damned powerful :)
> 
> Though I still prefer less string operation in compile time,
Yes. I think that's the key improvement to make.

 any idea of
> improving the
> macro syntax I posted? You see, D is going to bring AST macro in future releases. Let's
> discuss the syntax it will use?

To the user, we should eliminate the quotes, and the mixin().
My example was a bit specific. Let's consider lazy evaluation.

void logging(bool bLog, lazy char [] msg);

which shouldn't evaluate msg unless bLog is true.
With my method, it would be:

bool bActive=true;
mixin(logging(`bActive, getFromDatabase("db", "downtime", event)`));

Obviously, we want to get rid of the mixin() and the ``. But that's probably the only thing we need to do, outside of library code.
Once we get rid of the ``, we have the requirement that all macro arguments must be syntactically valid D code. This makes things like your := impossible.

Then we have:
macro logging(char [][] args)
{
  return `if (` ~ args[0].stringof ~ `) printf("%s", ` ~ args[1].stringof ~ `);`;
}

(macros always return a char [], which is mixed in).

But that will look pretty awful if you want to do more than a single function call. OTOH, if we just get rid of the ``, we lose the ability to make decisions at compile time.
What then??

Davidl wrote:
> macro (Token... tokens)
> {
>      static if (tokens[0].ID != Token.LParen)
>     pragma(error, `( expected`);
>      else static if (tokens[1].ID != Token.Identifier)
>         pragma(error, `identifer expected`);
>      else static if (tokens[2].ID != Token.RParen)
>         pragma(error, ) expected`);
>      else static if (tokens[3].toChars != `:=`)
>         pragma(error, `:= expected`);
>      static if (tokens[4].ID != Token.LBracket)
>     pragma(error, `( expected`);
>      else static if (tokens[5].ID != Token.Identifier)
>         pragma(error, `identifer expected`);
>      else static if (tokens[6].ID != Token.RBracket)
>         pragma(error, ) expected`);
>      else static if (tokens[7].ID != Token.Identifier)
>         pragma(error, `identifer expected`);
>      else static if (tokens[8].ID != Token.plus)
>         pragma(error, `+ operator expected`);
>      else static if (tokens[9].ID != Token.Identifier)
>         pragma(error, `identifer expected`);
>      alias token[1] arg0;
>      alias token[5] arg1;
>      alias token[7] arg2;
>      alias token[9] arg3;
>      arg0[arg1] = arg2 + arg3;
> }
> 
> seems the above is more valuable.

Huh, that looks awful. This kind of code surely needs some pattern matching magic. Should we add it to D 2.0 wishlist too.

Don Clugston wrote:
> Obviously, we want to get rid of the mixin() and the ``. But that's probably the only thing we need to do, outside of library code. Once we get rid of the ``, we have the requirement that all macro arguments must be syntactically valid D code. This makes things like your := impossible.

> But that will look pretty awful if you want to do more than a single
> function call. OTOH, if we just get rid of the ``, we lose the ability
> to make decisions at compile time.
> What then??

D being a multiparadigm language, it's a bit funny how many 'native' possibilities there are to implement metaprogramming. All of them have some good and some bad sides. So should these new constructs also cover some meta object protocol or s-expressions or just this shorter syntax for string mixins? Or should meta objects and s-expressions be implemented on the library level?

Don Clugston wrote

> Let's consider lazy evaluation.

Yeah. Are you able to eliminate switch-statements out of some code and generate an equivalent series of if-statements---and vice versa?

-manfred