On 03/22/2012 08:12 PM, F i L wrote:
> Timon Gehr wrote:
>> I think that would necessitate the addition of AST macros.
>
> You mean passing DMD's actual AST tree object? Only a stripped down
> version?

We have the macro keyword. I envision something like:

macro replaceAggregate(macro newAggregate, macro loop : foreach(x; aggr){statements}, macro x, macro aggr, macro statements) {
    foreach(x; newAggregate){statements}
}

void main(){
    int[] a = [1,2,3];
    int[] b = [2,3,4];
    replaceAggregate(b, foreach(x;a){writeln(x);});
}

(The syntax looks horrible, but you get the idea: AST walking by pattern matching)

Ideally, such macros would allow introducing local helper variables inside the macro that are not accessible from outside and introducing variables/enums in the context the macro is used. (prefixing identifiers that should interfere with the enclosing context with $ would be an option). Another idea:

macro someFancyDSL(string dsl){
    mixin(someFancyDSLtoD(dsl);
}

void main(){
    int n = 5;
    someFancyDSL {
        import n;
        let x = 1 : x
        let y = product (take n) x
        export y;
    }
    writeln(y);
}
// this would also enable less fancy but useful stuff:
macro until(macro condition, string str){
    while(!condition){
        mixin(str); // string and mixin not strictly necessary,
                    // maybe enable the feature on macro params too
    }
}

void main(){
    bool done = false;
    int x;
    until(done){
        done = foo(x++);
    }
}

This is just a very rough sketch though, we would need a much more refined design. I think getting the symbol scoping right is most important.

Timon Gehr wrote:
> We have the macro keyword. I envision something like:
>
> macro replaceAggregate(macro newAggregate, macro loop : foreach(x; aggr){statements}, macro x, macro aggr, macro statements) {
>     foreach(x; newAggregate){statements}
> }
>
> void main(){
>     int[] a = [1,2,3];
>     int[] b = [2,3,4];
>     replaceAggregate(b, foreach(x;a){writeln(x);});
> }
>
> (The syntax looks horrible, but you get the idea: AST walking by pattern matching)

This looks substantially more complicated than what I had in mind. I think it's a great idea, but something that could be added after the initial functionality was there.


> macro until(macro condition, string str){
>     while(!condition){
>         mixin(str); // string and mixin not strictly necessary,
>                     // maybe enable the feature on macro params too
>     }
> }
>
> void main(){
>     bool done = false;
>     int x;
>     until(done){
>         done = foo(x++);
>     }
> }
>
> This is just a very rough sketch though, we would need a much more refined design. I think getting the symbol scoping right is most important.

I'm a bit confused about what's actually going on here, but it certainly looks interesting. What exactly is being passed to "string str" in the macro?

On 03/23/2012 12:33 AM, F i L wrote:
> Timon Gehr wrote:
>> We have the macro keyword. I envision something like:
>>
>> macro replaceAggregate(macro newAggregate, macro loop : foreach(x;
>> aggr){statements}, macro x, macro aggr, macro statements) {
>> foreach(x; newAggregate){statements}
>> }
>>
>> void main(){
>> int[] a = [1,2,3];
>> int[] b = [2,3,4];
>> replaceAggregate(b, foreach(x;a){writeln(x);});
>> }
>>
>> (The syntax looks horrible, but you get the idea: AST walking by
>> pattern matching)
>
> This looks substantially more complicated than what I had in mind. I
> think it's a great idea, but something that could be added after the
> initial functionality was there.
>
>
>> macro until(macro condition, string str){
>> while(!condition){
>> mixin(str); // string and mixin not strictly necessary,
>> // maybe enable the feature on macro params too
>> }
>> }
>>
>> void main(){
>> bool done = false;
>> int x;
>> until(done){
>> done = foo(x++);
>> }
>> }
>>
>> This is just a very rough sketch though, we would need a much more
>> refined design. I think getting the symbol scoping right is most
>> important.
>
> I'm a bit confused about what's actually going on here, but it certainly
> looks interesting. What exactly is being passed to "string str" in the
> macro?

The idea is that if you have something of the form:

identifier(arguments) {
    body
}

It would get transformed into:

identifier(arguments, q{
    body
});

Le 22/03/2012 19:06, Felix Hufnagel a écrit :
> in addition to .codeof, let's think about .astof returning an abstract
> syntax tree.

WAY better !

Le 22/03/2012 17:00, F i L a écrit :
> So the discussions about Attributes and Aspect Oriented Programming
> (AOP) got me thinking... Basically AOP requires injecting code fragments
> together in a comprehensible way. Similarly, Attributes that go beyond
> @note (such as @GC.NoScan) need similar ability.
>
> D already has the ability to mixin arbitrary code fragments at compile
> time, and to process those in useful ways through CTFE. Which rocks.
> What it lacks is the ability to reflect upon the actual source code due
> to IO limitations of CTFE. So creating a mixin templates which pieces
> together a unique object is, to my knowledge, currently next to
> impossible (and slow since you'd have to parse and isolate code in .d
> file multiple times in a separate process, then compile again to put it
> all together).
>
> So, to quote Walter, what compelling features would it bring? Here's an
> example of a simple AOP program from the AOP wiki page (probably not the
> best implementation, but the concept is there):
>
> struct BankType
> {
> void transfer() { ... }
> void getMoneyBack() { ... }
> }
>
> struct Logger
> {
> void transfer() {
> log("transferring money...");
> }
> void getMoneyBack() {
> log("User requested money back");
> }
> }
>
> and now some magic...
>
> string bankCode(T...)(T aspects) {
> auto code = "struct Bank {";
> auto members = [__traits(allMembers, Bank)];
> foreach (m; members) {
> code ~= "void "~m~"() {";
> code ~= __traits(getMember, Bank, m).codeof;
> foreach (a; aspects) {
> if (__traits(hasMember, a, m) {
> code ~= __traits(getMember, a, m).codeof;
> }
> }
> code ~= "}"
> }
> return code ~ "}";
> }
>
> mixin template Bank(T...)
> {
> mixin(bankCode(T));
> }
>
> mixin Bank!Logger;
>
> void main() {
> auto b = Bank();
> b.transfer(); // logs
> b.getMoneyBack(); // ditto
> }
>
> So this would allow us to make "Compilers" within the Compiler
> (Codeception), since we could parse/strip/append any existing code
> fragments together in endless combination. Generic "Builders" could
> probably be built and put into a std.builder lib for general use.
>
> One particular use I have in mind is for Behavior Objects (Game
> Scripts). Each behavior would hold Property(T) objects which define
> per-property, per-state "binding" dependencies (eg.
> position.x.bind(other.x, State.Idle)) and execution code. On release,
> the Property(T) object would be stripped away (leaving just T) and it's
> behavior code "compressed" with others into optimized functions.
>
> I don't know much about the internals of DMD, so I'm not sure this is a
> realistic request, but I think the idea is compelling. Also, for
> Attributes I'm not sure this technique is really applicable. But it's
> possible that the compiler could exploit this internally for certain
> Attributes like @GC.whatever

As of discussion about properties/attributes, I wanted to do a proposal along that line :

@ttribute attributeName(AST ast, other template parameters . . .) {
    // Eponymous trick
    immutable AST attributeName = process(ast);

    auto process(AST ast) {
        // Do some CTFEable ast magic here.
        return newAst;
    }
}

And that would be used that way :
@attributeName(other template parameters if it make sense)
void fun() {}

And allow us to do inception within fun.

Additionally, I wanted to propose @mixin that return the AST of a string.

This isn't as easy as you think, it have a lot lot of implication about what you get as string. This is definitively interesting, but require more work for a realistic proposal.

Is this at least similar to what you had in mind ?

http://dpaste.dzfl.pl/a5dc2875

module program;

import std.stdio;

mixin template BankAccount () {
  public int amount;
  void deposit (int value) { this.amount += value; }
  void withdraw (int value) { this.amount -= value; }
  auto currentAmount () { return this.amount; }
}

mixin template Logger () {
  void deposit (int value) { writeln("User deposited money"); }
  void withdraw (int value) { writeln("User requested money back"); }
}

void main () {

  mixin aspect!("bank", BankAccount, Logger);

  bank b1;
  bank b2;

  b1.deposit(10);
  b1.deposit(20);
  b1.withdraw(5);
  writeln("b1.currentAmount: ", b1.currentAmount);

  b2.deposit(50);
  b2.withdraw(40);
  b2.deposit(100);
  writeln("b2.currentAmount: ", b2.currentAmount);

}


// generic code

mixin template aspect (string name, T...) {
  template aspectDispatch (string name, uint n) {
    import convert = std.conv;
    static if (n >= 1)
      enum aspectDispatch = ""
        ~ "import std.traits;\n"
        ~ "static if (__traits(hasMember, data.aspect_" ~ convert.to!string(n) ~ ", `" ~ name ~ "`))\n"
        ~ "  static if (!is(ReturnType!(data.aspect_" ~ convert.to!string(n) ~ "." ~ name ~ ") == void))\n"
        ~ "    return data.aspect_" ~ convert.to!string(n) ~ "." ~ name ~ "(arguments);\n"
        ~ "  else\n"
        ~ "    data.aspect_" ~ convert.to!string(n) ~ "." ~ name ~ "(arguments);\n"
        ~ aspectDispatch!(name, n - 1)
      ;
    else
      enum aspectDispatch = "";
  }
  auto code () {
    import convert = std.conv;
    string ret = ""
      ~ "struct " ~ name ~ " {\n"
      ~ "  struct aspectData {\n"
    ;
    uint i = 0;
    foreach (a; T)
      ret ~= "    mixin " ~ __traits(identifier, a) ~ " aspect_" ~ convert.to!string(++i) ~ ";\n";
    ret ~= ""
      ~ "  }\n"
      ~ "  aspectData data;\n"
      ~ "  auto opDispatch (string fn, args...) (args arguments) {\n"
      ~ "    mixin(aspectDispatch!(fn, " ~ convert.to!string(i) ~ "));\n"
      ~ "  }\n"
      ~ "}\n"
    ;
    return ret;
  }
  mixin(code);
}



On Thursday, 22 March 2012 at 16:00:29 UTC, F i L wrote:
> So the discussions about Attributes and Aspect Oriented Programming (AOP) got me thinking... Basically AOP requires injecting code fragments together in a comprehensible way. Similarly, Attributes that go beyond @note (such as @GC.NoScan) need similar ability.
>
> D already has the ability to mixin arbitrary code fragments at compile time, and to process those in useful ways through CTFE. Which rocks. What it lacks is the ability to reflect upon the actual source code due to IO limitations of CTFE. So creating a mixin templates which pieces together a unique object is, to my knowledge, currently next to impossible (and slow since you'd have to parse and isolate code in .d file multiple times in a separate process, then compile again to put it all together).
>
> So, to quote Walter, what compelling features would it bring? Here's an example of a simple AOP program from the AOP wiki page (probably not the best implementation, but the concept is there):
>
>   struct BankType
>   {
>     void transfer() { ... }
>     void getMoneyBack() { ... }
>   }
>
>   struct Logger
>   {
>     void transfer() {
>       log("transferring money...");
>     }
>     void getMoneyBack() {
>       log("User requested money back");
>     }
>   }
>
> and now some magic...
>
>   string bankCode(T...)(T aspects) {
>     auto code = "struct Bank {";
>     auto members = [__traits(allMembers, Bank)];
>     foreach (m; members) {
>       code ~= "void "~m~"() {";
>       code ~= __traits(getMember, Bank, m).codeof;
>       foreach (a; aspects) {
>         if (__traits(hasMember, a, m) {
>           code ~= __traits(getMember, a, m).codeof;
>         }
>       }
>       code ~= "}"
>     }
>     return code ~ "}";
>   }
>
>   mixin template Bank(T...)
>   {
>     mixin(bankCode(T));
>   }
>
>   mixin Bank!Logger;
>
>   void main() {
>     auto b = Bank();
>     b.transfer(); // logs
>     b.getMoneyBack(); // ditto
>   }
>
> So this would allow us to make "Compilers" within the Compiler (Codeception), since we could parse/strip/append any existing code fragments together in endless combination. Generic "Builders" could probably be built and put into a std.builder lib for general use.
>
> One particular use I have in mind is for Behavior Objects (Game Scripts). Each behavior would hold Property(T) objects which define per-property, per-state "binding" dependencies (eg. position.x.bind(other.x, State.Idle)) and execution code. On release, the Property(T) object would be stripped away (leaving just T) and it's behavior code "compressed" with others into optimized functions.
>
> I don't know much about the internals of DMD, so I'm not sure this is a realistic request, but I think the idea is compelling. Also, for Attributes I'm not sure this technique is really applicable. But it's possible that the compiler could exploit this internally for certain Attributes like @GC.whatever

On Tuesday, 9 October 2012 at 13:28:55 UTC, luka8088 wrote:
> Is this at least similar to what you had in mind ?
>
> [ ..code.. ]

Yes, I realized, a bit after I originally posted that, that my suggestion was already possible if BankType & Logger where mixin-templates instead of struct/classes. Thanks for the code example though.

I still think an built-in .codeof/.astof would be nice, but what D really needs to achieve this in a syntactically pleasing and powerful way, is 'macro' templates (like Nimrod has) which work on the AST directly. I doubt this is a major concern ATM however.

On Tuesday, 9 October 2012 at 19:29:34 UTC, F i L wrote:
> On Tuesday, 9 October 2012 at 13:28:55 UTC, luka8088 wrote:
>> Is this at least similar to what you had in mind ?
>>
>> [ ..code.. ]
>
> Yes, I realized, a bit after I originally posted that, that my suggestion was already possible if BankType & Logger where mixin-templates instead of struct/classes. Thanks for the code example though.
>
> I still think an built-in .codeof/.astof would be nice, but what D really needs to achieve this in a syntactically pleasing and powerful way, is 'macro' templates (like Nimrod has) which work on the AST directly. I doubt this is a major concern ATM however.

My point of making this example was to show that nothing is missing in D itself. You just need to be more creative. If you want to write in a manner more similar to your original example (by that I mean without mixin templates) you can use classes, class methods can be turned to delegates with their context pointer changed before execution, and then you would get the same effect. Also having such syntax could be very confusing because someone could introduce some syntax which is very similar to D but behaves differently and it is embedded in a way that looks just like a D code.

Also the idea is to have "// generic code" part in some library and not visible to the *user* so the rest of the code would be syntactically pleasing. If you check current phobos code, you will see that there are some examples of using mixins this way.

Please also check the comments on https://github.com/D-Programming-Language/dmd/pull/953 (if you haven't done that already).