Hi,

I'm currently making some custom AST parser for D :

http://pastebin.com/1hmnTUbY -- AST Parser

http://pastebin.com/ZRc5be1p -- Sum example

Right now both lexer and syntax parser are just awfull and they don't work in CTFE because of class usage. But example looks rather interesting, with such approach (if it works) one can create full list comprehension like python's one or MS LINQ for C# system as user library.

I tried to change class to struct, but failed :-( Any ideas how to make it work in CTFE ?

Hm... there is some limitations in example code, concerning local variables in nested functions... here is workaround   http://pastebin.com/mQaKXaYY

I'm just having a look at this now, looks rather interesting! I'd love to be able to use some Python-like list comprehension tricks in my D code.

P.S. not sure if it's your fault, but you're code has spaces between "q" and "{", which gives a compiler error. q{} has to start without spaces between q and {.   ;)

On 11/25/10, Alex_Dovhal <alex_dovhal@yahoo.com> wrote:
> Hm... there is some limitations in example code, concerning local variables in nested functions... here is workaround   http://pastebin.com/mQaKXaYY
>
>
>

SORRY.

Disregard that, it was my fault. My reindenter f*ked it up. You're code is fine. :)

On 12/21/10, Andrej Mitrovic <andrej.mitrovich@gmail.com> wrote:
> I'm just having a look at this now, looks rather interesting! I'd love to be able to use some Python-like list comprehension tricks in my D code.
>
> P.S. not sure if it's your fault, but you're code has spaces between "q" and "{", which gives a compiler error. q{} has to start without spaces between q and {.   ;)
>
> On 11/25/10, Alex_Dovhal <alex_dovhal@yahoo.com> wrote:
>> Hm... there is some limitations in example code, concerning local
>> variables
>> in nested functions... here is workaround   http://pastebin.com/mQaKXaYY
>>
>>
>>
>

"Andrej Mitrovic" wrote:
> I'm just having a look at this now, looks rather interesting! I'd love to be able to use some Python-like list comprehension tricks in my D code.

Yes list comprehension is also possible this way. But it is not very eye handy right now even if fully implemented, because one can't use variables external to the scope of sum or listcomp... templates, also those parenthesis and braces..., so must write:

float a = 1 ;
int n = 10 ;
float x = mixin(sumImpl(q{i=0:10; a*i})) ;
float y = mixin(sumImpl(q{i=1:n; i*mixin(prodImpl(q{j=0:i ; i+j}))}));

instead of:

float a = 1 ;
int n = 10 ;
float x = sum!(q{i=0:10; a*i}) ;
float y = sum!(q{i=1:n; i*prod!(q{j=0:i ; i+j})});

I made it as my first attempt to use D, that's why maybe it's not solid or well designed, so any ideas are welcomed.

That's why I think it might be a better idea to simply use "DSL blocks" throughout your code. Instead of having to mixin each template, you would only have one template where you have all the rules for your DSL:

float a = 1 ;
int n = 10 ;
// more D code..

mixin(MyDSL!(q{
    // Here you use the syntax defined by the "MyDSL" language
    float x = sum!(i=0:10; a*i) ;
    float y = sum!(i=1:n; i*prod!(j=0:i ; i+j);
    // More MyDSL code..
}));

// Here you use D syntax again, and D will know about float x and float y.

Not to mention that you could potentially have some control over syntax highlighting (if you have a capable editor :p). For example you could write a little script in VIM which detects the use of "MyDSL!()" template, and highlights the code inside the block differently than the one outside the block.



On 12/21/10, Alex_Dovhal <alex_dovhal@yahoo.com> wrote:
> "Andrej Mitrovic" wrote:
>> I'm just having a look at this now, looks rather interesting! I'd love to be able to use some Python-like list comprehension tricks in my D code.
>
> Yes list comprehension is also possible this way. But it is not very eye handy right now even if fully implemented, because one can't use variables external to the scope of sum or listcomp... templates, also those parenthesis and braces..., so must write:
>
> float a = 1 ;
> int n = 10 ;
> float x = mixin(sumImpl(q{i=0:10; a*i})) ;
> float y = mixin(sumImpl(q{i=1:n; i*mixin(prodImpl(q{j=0:i ; i+j}))}));
>
> instead of:
>
> float a = 1 ;
> int n = 10 ;
> float x = sum!(q{i=0:10; a*i}) ;
> float y = sum!(q{i=1:n; i*prod!(q{j=0:i ; i+j})});
>
> I made it as my first attempt to use D, that's why maybe it's not solid or well designed, so any ideas are welcomed.
>
>
>

"Andrej Mitrovic" <andrej.mitrovich@gmail.com> wrote:
> That's why I think it might be a better idea to simply use "DSL blocks" throughout your code. Instead of having to mixin each template, you would only have one template where you have all the rules for your DSL:
>
> float a = 1 ;
> int n = 10 ;
> // more D code..
>
> mixin(MyDSL!(q{
>    // Here you use the syntax defined by the "MyDSL" language
>    float x = sum!(i=0:10; a*i) ;
>    float y = sum!(i=1:n; i*prod!(j=0:i ; i+j);
>    // More MyDSL code..
> }));
>
> // Here you use D syntax again, and D will know about float x and float y.
>
> Not to mention that you could potentially have some control over syntax highlighting (if you have a capable editor :p). For example you could write a little script in VIM which detects the use of "MyDSL!()" template, and highlights the code inside the block differently than the one outside the block.

Very interesting idea :), one note - sum!(...) in DSL is not template any more, so one can change a design as he wants, say:

mixin(MyDSL_Impl(q{
    float x = sum {i=0:10 ; a * i};
    float y = sum {i=1:n; i*prod{j=0:i; i+j}} ;
}));

this MyDSL_Impl only scans for "name" "{" text "}" and replaces it with mixin(nameImpl(q{text})) ;

User can put entire function body or even almost full module (if don't care about compile time) into that DSL in order not to change from DSL to normal code and back. One more thing to think about - how to show correct error line number in there?

On 12/21/10, Alex_Dovhal <alex_dovhal@yahoo.com> wrote:
> User can put entire function body or even almost full module (if don't care about compile time) into that DSL in order not to change from DSL to normal code and back.

Yep. You could even take it a step further and use some form of escape mechanism, which would leave D code alone but parse the rest of the string as a DSL. E.g.:

mixin(MyDSL!(q{

    $(D)
    float a = 1 ;    // will be mixed in as is
    int n = 10 ;
    void test(int t)
    {
        writeln(t);
    }
    $(/D)  // end of D code

   x = sum(i=0:10; a*i) ;  // will be parsed and reconstructed as D code
   y = sum(i=1:n; i*prod!(j=0:i ; i+j);
}));

So now you wouldn't have to write a lot of MyDSL blocks in-between D code.

> One more thing to think about - how to show correct error line number in there?

See https://docs.google.com/leaf?id=0B-f1J0HiG2ovNmQzNzM1ODYtNDM3ZC00N2I4LWI2YmUtYTIyM2Y2MzcyZWIw&sort=name&layout=list&num=50&pli=1
, there's a unittests.d module in /src in the archive. It uses a
parameter that is default initialized to the line (size_t line =
__LINE__))

I guess from there you could chomp up your string into an array for each new line, and if your DSL parser finds a syntax error it would report the line number as e.g. currentLineNumber + line, where currentLineNumber is the line number you're currently parsing, e.g. DSLStringArray[currentLineNumber] << this line has the error.

I think that's about it (but I haven't had my nighttime coffee yet so don't trust me fully on this one :p).