I like CTFE and the meta programming idea for languages like D.

However, I'm wondering why most (everyone?) is trying to do meta-programming using the same language as the one getting compiled. IMO the use-cases a pretty different and doing CTFE, code-generation etc. needs some other approach. If you look at all the strange template syntax, strange hacks etc. it's all far from being obvious.

Why not have a CTL (compile-time-language) that has access to some compiler internals, that follows a more functional concept? We are evaluating sequences of things to generate code, include / exclude code etc.

From my experience with the different approaches, functional thinking is much better suited and simpler to use for CTFE goals.

IMO that would really be a big step ahead. Because you know a hammer, not everything is a nail...

-- 
Robert M. Münch
http://www.saphirion.com
smarter | better | faster

On Sunday, 31 January 2016 at 13:59:06 UTC, Robert M. Münch wrote:
> I like CTFE and the meta programming idea for languages like D.
>
> However, I'm wondering why most (everyone?) is trying to do meta-programming using the same language as the one getting compiled. IMO the use-cases a pretty different and doing CTFE, code-generation etc. needs some other approach. If you look at all the strange template syntax, strange hacks etc. it's all far from being obvious.

I see things differently. First of all I don't see everyone trying
to do meta-programming with the same language. C++ for example has
a quite specific syntax between its arcane templates and the
preprocessor.

I see having the same language as a *huge* advantage. If a function
doesn't have side effects then it can be used at runtime or at
compile-time and integrated with your logic easily.

D's metaprogramming success is IMHO directly linked to it not having
a separate language for it, because it lowers the cost of learning
and using metaprogramming. It's saying "metaprogramming isn't
different from any other kind of programming, you can use the same
tools".

> Why not have a CTL (compile-time-language) that has access to some compiler internals, that follows a more functional concept? We are evaluating sequences of things to generate code, include / exclude code etc.

Having access to some compiler internals is already what is done,
or I don't understand exactly what you mean by that.

> From my experience with the different approaches, functional thinking is much better suited and simpler to use for CTFE goals.

I think you should put some sort of example of how you'd want it,
because right now I don't understand. D has some nice functional
tools and they already show their strength at compile-time.

For example function purity is central to their CTFE-ability, using
template recursion instead of loops has often proved being a
winning strategy etc... Yes applying functionnal principles works
well at compile-time, but why should we need another language to do
that?

On Sunday, 31 January 2016 at 13:59:06 UTC, Robert M. Münch wrote:
> I like CTFE and the meta programming idea for languages like D.
>
> However, I'm wondering why most (everyone?) is trying to do meta-programming using the same language as the one getting compiled. IMO the use-cases a pretty different and doing CTFE, code-generation etc. needs some other approach. If you look at all the strange template syntax, strange hacks etc. it's all far from being obvious.

You're conflating CTFE with the other meta programming tools here. CTFE is the same language as run-time D, but it doesn't have strange template syntax. Templates, static if, __traits, etc. have strange syntax, but they're sort of a different language already.

Are you maybe wishing for a nicer alternative to templates, etc?

> Why not have a CTL (compile-time-language) that has access to some compiler internals, that follows a more functional concept? We are evaluating sequences of things to generate code, include / exclude code etc.
>
> From my experience with the different approaches, functional thinking is much better suited and simpler to use for CTFE goals.
>
> IMO that would really be a big step ahead. Because you know a hammer, not everything is a nail...

I think this is too vague to lead anywhere. At least you should identify specific problems with D's toolset. And if you have concrete ideas for improvements, you should desribe them in more detail, spelling out how they improve upon the status quo.

If you want to have an entirely different meta programming system, then you should show how it would look like, and how it would be better than the status quo. I don't think anyone can make much of "make it more functional". Also, when it's fundamentally different from what we have now, then I don't see it getting into D at the moment. The language is not in a phase of designing fundamentals.

On Sunday, 31 January 2016 at 13:59:06 UTC, Robert M. Münch wrote:
> However, I'm wondering why most (everyone?) is trying to do meta-programming using the same language as the one getting compiled. IMO the use-cases a pretty different and doing CTFE, code-generation etc. needs some other approach. If you look at all the strange template syntax, strange hacks etc. it's all far from being obvious.

 I have trouble coming up with a proper use-case to use Regex during CTFE; I mean, sure you can do a replacement expansion and encode a few bits to expand for a switch statement or something, but at the same time you can already do that.

On Monday, 1 February 2016 at 00:02:55 UTC, Era Scarecrow wrote:
> On Sunday, 31 January 2016 at 13:59:06 UTC, Robert M. Münch wrote:
>> However, I'm wondering why most (everyone?) is trying to do meta-programming using the same language as the one getting compiled. IMO the use-cases a pretty different and doing CTFE, code-generation etc. needs some other approach. If you look at all the strange template syntax, strange hacks etc. it's all far from being obvious.
>
>  I have trouble coming up with a proper use-case to use Regex during CTFE; I mean, sure you can do a replacement expansion and encode a few bits to expand for a switch statement or something, but at the same time you can already do that.

Wrong thread?

On Sunday, 31 January 2016 at 13:59:06 UTC, Robert M. Münch wrote:
> I like CTFE and the meta programming idea for languages like D.
>
> However, I'm wondering why most (everyone?) is trying to do meta-programming using the same language as the one getting compiled. IMO the use-cases a pretty different and doing CTFE, code-generation etc. needs some other approach. If you look at all the strange template syntax, strange hacks etc. it's all far from being obvious.
>
> Why not have a CTL (compile-time-language) that has access to some compiler internals, that follows a more functional concept? We are evaluating sequences of things to generate code, include / exclude code etc.
>
> From my experience with the different approaches, functional thinking is much better suited and simpler to use for CTFE goals.
>
> IMO that would really be a big step ahead. Because you know a hammer, not everything is a nail...

I'm not sure what is preventing you from doing that already. There is compile time reflection (has access to some compiler internals) and D support functional style.

Unless you have some specific in mind, I don't think there is anything we can do to help here. t seems you already have the pieces you want.

On 2016-01-31 14:35:13 +0000, cym13 said:

> I see things differently. First of all I don't see everyone trying to do meta-programming with the same language. C++ for example has a quite specific syntax between its arcane templates and the preprocessor.

Well, ok, maybe a "using the same concepts as the underlaying language" might hit it better.

> I see having the same language as a *huge* advantage. If a function doesn't have side effects then it can be used at runtime or at compile-time and integrated with your logic easily.

The thing I mean is not that you shouldn't be able to reference or use things from the "target language" but how to write down what you want to do.

Code generation by building strings using the D operators for example is something I think is not very elegant. If I could use a list and build it up without having to care about the code / data difference, that would simplify things a lot.

Imagine we could use Lua during compile time and have access to the AST etc.


> D's metaprogramming success is IMHO directly linked to it not having a separate language for it, because it lowers the cost of learning and using metaprogramming.

But limits you to a subset that doesn't feel very natural for doing a lot of common things in code-generation.

> It's saying "metaprogramming isn't different from any other kind of programming, you can use the same tools".

Yes, and I don't think this statement holds. It's very different because the goal is totally different. I need a tool that allows me to manipulate my underlying code during compilation. The main aspect is: Manipulate D code.

>> Why not have a CTL (compile-time-language) that has access to some compiler internals, that follows a more functional concept? We are evaluating sequences of things to generate code, include / exclude code etc.
> 
> Having access to some compiler internals is already what is done, or I don't understand exactly what you mean by that.

Sure, but the question is how do to deal with it. IMO it's not very straight forward at the moment.

> I think you should put some sort of example of how you'd want it, because right now I don't understand. D has some nice functional tools and they already show their strength at compile-time.

Ok, here is a simple example: I want to ensure that specific switch statements handle all cases for a given enum, list, etc. This is a common pattern and often a source of problems because you change a collection but miss to update all side-effecting places.

This is an example how I have done it (maybe there is a much better way to do it, but I didn't come up with one):

==> BEGIN

import std.conv;
import std.stdio;
import std.string;
import std.traits;

// @@example code should work with classes as well
enum A {afoo, bfoo, cfoo};

enum members1 = __traits(allMembers, A); // returns TypeTuple
auto members2 = __traits(allMembers, A);

pragma(msg, typeof(members1));
// pragma(msg, typeid(members1)); // run-time only
// static assert(is(members1 : enum)); // Error: basic type expected, not enu

pragma(msg, typeof(members2));
// pragma(msg, typeid(members2)); // run-time only


// function that generates a string which is used as a mixin at compile time
// result string must conform to syntax as it was hand-written code
string generateEnums(T...)(string type){
    string code = "enum " ~ type ~ " {";

    // this is a static foreach (compile time)
    foreach(m; T){
      debug pragma(msg, m ~ ","); // check what code we get at compile time
      code ~= m ~ ",";
    }

    return(code ~ "}");
}

int main(){
    A switch_var_a;
    final switch(switch_var_a){
      case A.afoo:
      case A.bfoo:
      case A.cfoo:
    }

    string user_input = readln();

    mixin(generateEnums!members1("B"));
    B switch_var_b = chomp(user_input).to!B; // get rid of terminating chars

    final switch (switch_var_b) {
      case B.afoo:
      {
        writeln("a");
        break;
      }
      case B.bfoo: // if commeted will cause a compiler error
      {
        writeln("b");
        break;
      }
      case B.cfoo: {writeln("c");}
    }

    return(0);
}

<== END

How about being able to write something like "ensure_final_switch B;" and have this call a CTF that generates the necessary code and has access to tool for building D structured code, AST etc.? And has a compile-time state I can later access in a upcoming CTF.


-- 
Robert M. Münch
http://www.saphirion.com
smarter | better | faster

On 2016-01-31 15:31:59 +0000, anonymous said:

> You're conflating CTFE with the other meta programming tools here. CTFE is the same language as run-time D, but it doesn't have strange template syntax. Templates, static if, __traits, etc. have strange syntax, but they're sort of a different language already.

See my example in the other post. IMO what I want to achieve is a very simple example but the implementation I came up with is far from being simple.

> Are you maybe wishing for a nicer alternative to templates, etc?

No, my point is that CTFE and meta-programming seems to be much simpler and powerful if I can use a more functional programming approach for it. Working with powerful lists and data = code and code = data concept would simplify this.

-- 
Robert M. Münch
http://www.saphirion.com
smarter | better | faster

On 2016-02-01 08:15:11 +0000, deadalnix said:

> I'm not sure what is preventing you from doing that already. There is compile time reflection (has access to some compiler internals) and D support functional style. Unless you have some specific in mind, I don't think there is anything we can do to help here. t seems you already have the pieces you want.

See my other example. Yes, maybe I can do everything already, but it's not straightforward. I have to fiddle around a lot to get it to work.

Of course that can be my incompetence with CTFE. However, my experience and intuition is that what I would like to use CTFE for (code-generation depending on some other parts of the code like enums, presence of member functions, etc.) can be done much simpler than at the moment.

-- 
Robert M. Münch
http://www.saphirion.com
smarter | better | faster

On Tuesday, 2 February 2016 at 08:34:38 UTC, Robert M. Münch wrote:
> On 2016-02-01 08:15:11 +0000, deadalnix said:
>
>> I'm not sure what is preventing you from doing that already. There is compile time reflection (has access to some compiler internals) and D support functional style. Unless you have some specific in mind, I don't think there is anything we can do to help here. t seems you already have the pieces you want.
>
> See my other example. Yes, maybe I can do everything already, but it's not straightforward. I have to fiddle around a lot to get it to work.
>
> Of course that can be my incompetence with CTFE. However, my experience and intuition is that what I would like to use CTFE for (code-generation depending on some other parts of the code like enums, presence of member functions, etc.) can be done much simpler than at the moment.

That is definitely true that the compile time API is kind of screwy. That's definitively not you. I think the best path forward at this stage is to provide nice API as a library on top of it.