Are there any good libraries for ctfe/code generation?

I don't know, things like parsing support for compile time strings, string formatting, type <-> string

My project seems to be growing ctfe, and it's all horribly hacky and ugly code.

Ellery Newcomer wrote:
> Are there any good libraries for ctfe/code generation?
> 
> I don't know, things like parsing support for compile time strings, string formatting, type <-> string
> 
> My project seems to be growing ctfe, and it's all horribly hacky and ugly code.

I think all ctfe code is horrible hacky and ugly code. :-P

Ellery Newcomer wrote:
> Are there any good libraries for ctfe/code generation?
> 
> I don't know, things like parsing support for compile time strings, string formatting, type <-> string
> 
> My project seems to be growing ctfe, and it's all horribly hacky and ugly code.

I think all ctfe code is horribly hacky and ugly code. :-P

Ellery Newcomer wrote:
> Are there any good libraries for ctfe/code generation?
> 
> I don't know, things like parsing support for compile time strings, string formatting, type <-> string
> 
> My project seems to be growing ctfe, and it's all horribly hacky and ugly code.

I've found that the function below improves things immensely. I might propose it for std.metastrings in the next Phobos release.

===============

/** Escape any quotes and backslashes inside the given string,
 * prefixing them with the given escape sequence. Use `\` to escape
 * once, `\\\` to escape twice.
 */
string enquote(string instr, string escape = `\`)
{
    // This function is critical for compilation speed.
    // Need to minimise the number of allocations.
    // It's worth using copy-on-write even for CTFE.

    for(int i = 0; i < instr.length; ++i)
    {
        if (instr[i] == '"' || instr[i] == '\\')
        {
            string str = instr[0..i] ~ escape;
            int m = i;
            foreach(int k, char c; instr[i+1..$])
            {
                if (c=='"' || c=='\\')
                {
                    str ~= instr[m..i+1+k] ~ escape;
                    m = i+k+1;
                }
            }
            return str ~ instr[m..$];
        }
    }
    return instr;
}

unittest {
    assert(enquote(`"a\"`)==`\"a\\\"`);
    assert(enquote(`abc`)==`abc`);
}

Ary Borenszweig wrote:
> Ellery Newcomer wrote:
>> Are there any good libraries for ctfe/code generation?
>>
>> I don't know, things like parsing support for compile time strings, string formatting, type <-> string
>>
>> My project seems to be growing ctfe, and it's all horribly hacky and ugly code.
> 
> I think all ctfe code is horribly hacky and ugly code. :-P

I think you're talking about string mixins, not CTFE. They are not the same.

I don't see anything particularly ugly or hacky about CTFE code, like the code below:

int greater(int a, int b) {
  if (a>b) return a;
  else return b;
}

int [ greater(short.max/3, 515) ] foo;

On 04/20/2010 10:17 AM, Don wrote:
>
> I don't see anything particularly ugly or hacky about CTFE code, like
> the code below:
>
> int greater(int a, int b) {
> if (a>b) return a;
> else return b;
> }
>
> int [ greater(short.max/3, 515) ] foo;

It isn't. Things don't get ugly until you hit the relatively low ceiling of what ctfe can handle. I don't pay much attention to where that ceiling is, but I am aware that the situation has been improving mostly due to your efforts, and I appreciate that.

So far, all my ctfe code is code generation for string mixins, and it's fairly modest stuff, to the point where a simple string formatting function could probably take care of a fair portion of the problem. It's just that I don't know of any. (I'm in D1/Tango land)

The other half of the problem is parsing input strings for simple dsls. Having to write out a lexer/parser from scratch just sucks regardless, and writing it in ctfe code, which isn't allowed composite data structures (last I checked) is really not worth doing. I think D fails on its promise here, where it says "with string mixins you can create your own dsls", but then provides no additional support.

As ctfe support matures, I dream of a full-fledged parser generator that can be evaluated at compile time, although that's way more heavy duty than what most people will need.

Another idea is something more on the lines of haskell's parsec, which provides building blocks for common tokens, etc. I don't know much about it, though.

Even compile time regular expressions would be better than nothing..

And whatever it is, it needs to be in the standard library.

Ellery Newcomer wrote:
> 
> As ctfe support matures, I dream of a full-fledged parser generator that can be evaluated at compile time, although that's way more heavy duty than what most people will need.
> 

My spirit port might be an option at some point soon:

http://www.sstk.co.uk/spiritd.php

Haven't tried it with current d2 compilers, but should still work w/ d1.

It uses lots of template code to compose parsers, so in theory it ought to be usable at compile time. I doubt it can be used that way at the moment, but hopefully dmd will get there and then you'll be able to do some seriously funky stuff at compile time.

- --
My enormous talent is exceeded only by my outrageous laziness.
http://www.ssTk.co.uk

On 04/20/2010 01:02 PM, div0 wrote:
> -----BEGIN PGP SIGNED MESSAGE-----
> Hash: SHA1
>
> Ellery Newcomer wrote:
>>
>> As ctfe support matures, I dream of a full-fledged parser generator that
>> can be evaluated at compile time, although that's way more heavy duty
>> than what most people will need.
>>
>
> My spirit port might be an option at some point soon:
>
> http://www.sstk.co.uk/spiritd.php
>

Yeah, I had spiritd in the back of my mind when I wrote that. I've never used spirit, but from a once-over of the docs, it looks like something I should make a point of getting familiar with this summer.

> Haven't tried it with current d2 compilers, but should still work w/ d1.
>
> It uses lots of template code to compose parsers, so in theory it ought
> to be usable at compile time. I doubt it can be used that way at the
> moment, but hopefully dmd will get there and then you'll be able to do
> some seriously funky stuff at compile time.

That would be awesome.

Ellery Newcomer wrote:
> On 04/20/2010 10:17 AM, Don wrote:
>>
>> I don't see anything particularly ugly or hacky about CTFE code, like
>> the code below:
>>
>> int greater(int a, int b) {
>> if (a>b) return a;
>> else return b;
>> }
>>
>> int [ greater(short.max/3, 515) ] foo;
> 
> It isn't. Things don't get ugly until you hit the relatively low ceiling of what ctfe can handle. I don't pay much attention to where that ceiling is, but I am aware that the situation has been improving mostly due to your efforts, and I appreciate that.
> 
> So far, all my ctfe code is code generation for string mixins, and it's fairly modest stuff, to the point where a simple string formatting function could probably take care of a fair portion of the problem. It's just that I don't know of any. (I'm in D1/Tango land)
> 
> The other half of the problem is parsing input strings for simple dsls. Having to write out a lexer/parser from scratch just sucks regardless, and writing it in ctfe code, which isn't allowed composite data structures (last I checked) is really not worth doing. 

Now that it can do struct member functions and delegates, it's possible to do fairly sophisticated things in CTFE. Unfortunately the two killer bugs (bug 1330 and 1382) still aren't fixed, so the frustration level is still pretty bad. We're still a few releases away from getting a fix for those ones.

I think D fails
> on its promise here, where it says "with string mixins you can create your own dsls", but then provides no additional support.

Yes, the two CTFE bugs I mention are blockers. Once they're fixed, it will be fairly simple to add support for classes, exceptions, and nearly everything else.

> 
> As ctfe support matures, I dream of a full-fledged parser generator that can be evaluated at compile time, although that's way more heavy duty than what most people will need.
> 
> Another idea is something more on the lines of haskell's parsec, which provides building blocks for common tokens, etc. I don't know much about it, though.
> 
> Even compile time regular expressions would be better than nothing..
> 
> And whatever it is, it needs to be in the standard library.

Exactly.

Don:
> I've found that the function below improves things immensely. I might propose it for std.metastrings in the next Phobos release.

Can you tell me what this function can be useful for?

Thank you, bye,
bearophile