Kevin Bealer wrote:
> Walter Bright wrote:
>> Sean Kelly wrote:
>>> I think this is a fantastic feature.  It uses plain old D syntax and offers all the computational power of templates.  Will this work for template functions that follow the rules as well?
>>
>> Yes.
>>
>>> Also, is there any way to test whether a function is being executed at run time?
>>
>> Yes, it will be always executed at runtime if it is not in a context where it is required to be folded to a constant.
> 
> This looks awesome, by the way!
> 
> I don't have this version of dmd yet, but it looks like I can convert any expression to be run at compile time or runtime with not too much extra syntax.
> 
> int hack(int a, int b)
> {
>    return sqrt(a) + b*b;
> }
> 
> int heck()
> {
>    // at runtime
>    return hack(10, 20);
> }

I think the above will actually be folded to a constant value at compile time, assuming sqrt is defined properly?


Sean

Robby wrote:
> Walter Bright wrote:
>> Don has based some runtime reflection code on demangling names.
> 
> Considering that, will it be fair to say that future reflection capabilities will be based off of demangling? Or is there any other ideas up the sleeve about how it's going to be implemented. (trying not to go way off subject, but feeding general curiosity)

It might be, too soon to tell.

You know .. I proposed this idea a looooooong time ago
http://www.digitalmars.com/d/archives/digitalmars/D/announce/3244.html

I wonder what was your motivation for ignoring me back then <g>

P.S. My terms were stupid .. this kind of meta-programming is not a special case of constant folding; it's a *general* case of constant folding.

Walter Bright wrote:
> The sqrt() example I gave is cute, but not a very interesting reason for doing compile time function execution. So what's the real reason?
> 
> A while back, Eric Anderton and Don Clugston showed how one could use template metaprogramming techniques to do, for example compile time regular expressions. It was a marvelous technical demonstration, and showed that essentially any computation could be done, using templates, at compile time.
> 
> There were some serious problems, though:
> 
> 1) They are hard and unintuitive to write (for those not comfortable with functional programming, which is most of us).
> 
> 2) The result looks awful - templates are just syntactically unsuited to this, even though they're a lot easier on the eye than C++ template metaprograms.
> 
> 3) This is off-putting enough that some question even having such facilities in the language, as it results in impenetrable code unmaintainable by joe coders.
> 
> 4) While theoretically capable of any computation, such template metaprogramming had severe practical limitations. Every step of every computation required generating a unique template. This naturally is incredibly slow and memory consumptive (C++ metaprogramming is notorious for taking all night to do a build). Even worse, if you're going to use string templates to parse, say, a 1000 character DSL, the template name generated must include its arguments, so that template identifier will be 1000 characters long. Then, it slices off the first character, and generates another template for the rest (999), then 998, then 997, etc., until 1000 templates are generated averaging 500 characters long. It doesn't take much of that before the whole thing collapses.
> 
> 5) In casting about for a solution, the compile time regex came up again. This was still strongly disliked.
> 
> 6) I promised to try and make template metaprogramming less obtuse, and what better way to do that than to obsolete a whole class of templates, replacing them with ordinary functions?

Hasan Aljudy wrote:
> You know .. I proposed this idea a looooooong time ago
> http://www.digitalmars.com/d/archives/digitalmars/D/announce/3244.html
> 
> I wonder what was your motivation for ignoring me back then <g>

You weren't ignored; a synapse was reinforced there, and with time, enough neurons fired. :o)

Andrei

Hasan Aljudy wrote:
> You know .. I proposed this idea a looooooong time ago
> http://www.digitalmars.com/d/archives/digitalmars/D/announce/3244.html
> 
> I wonder what was your motivation for ignoring me back then <g>

Lots of great ideas get posted here, it just takes time to get to them. This one wasn't simple to implement.

Walter Bright wrote:
> Hasan Aljudy wrote:
>> You know .. I proposed this idea a looooooong time ago
>> http://www.digitalmars.com/d/archives/digitalmars/D/announce/3244.html
>>
>> I wonder what was your motivation for ignoring me back then <g>
> 
> Lots of great ideas get posted here, it just takes time to get to them. This one wasn't simple to implement.

Yeah heheh .. I was kinda just joking ..

Walter Bright wrote:
> Miles wrote:
>> Walter Bright wrote:
>>> Miles wrote:
>>>> Is there some rationale for not using an MD5 hash of the string?
>>> It's not reversible - no pretty printing, no demangling, etc.
>>
>> For common usage, does it really need to be reversible?
> 
> Problem is, I can't tell when the uncommon use is required.
> 
>> Pretty printing and demangling are only used for debugging purposes, so,
>> this information should go to debug sections of the object file, that
>> are not loaded into memory unless you are running the debugger. And let
>> the debugger use that information to extract what the symbol means.
> 
> Don has based some runtime reflection code on demangling names.
 I wonder how relevant that still is.... <g>

Walter Bright wrote:
> The sqrt() example I gave is cute, but not a very interesting reason for doing compile time function execution. So what's the real reason?
> 
> A while back, Eric Anderton and Don Clugston showed how one could use template metaprogramming techniques to do, for example compile time regular expressions. It was a marvelous technical demonstration, and showed that essentially any computation could be done, using templates, at compile time.
> 
> There were some serious problems, though:
> 
> 1) They are hard and unintuitive to write (for those not comfortable with functional programming, which is most of us).
> 
> 2) The result looks awful - templates are just syntactically unsuited to this, even though they're a lot easier on the eye than C++ template metaprograms.
> 
> 3) This is off-putting enough that some question even having such facilities in the language, as it results in impenetrable code unmaintainable by joe coders.
> 
> 4) While theoretically capable of any computation, such template metaprogramming had severe practical limitations. Every step of every computation required generating a unique template. This naturally is incredibly slow and memory consumptive (C++ metaprogramming is notorious for taking all night to do a build). Even worse, if you're going to use string templates to parse, say, a 1000 character DSL, the template name generated must include its arguments, so that template identifier will be 1000 characters long. Then, it slices off the first character, and generates another template for the rest (999), then 998, then 997, etc., until 1000 templates are generated averaging 500 characters long. It doesn't take much of that before the whole thing collapses.
> 
> 5) In casting about for a solution, the compile time regex came up again. This was still strongly disliked.
> 
> 6) I promised to try and make template metaprogramming less obtuse, and what better way to do that than to obsolete a whole class of templates, replacing them with ordinary functions?

I mentioned this over in announce, but I'm working on taking this to the next level.

I've found that treating meta programming like lisp (thanks to everyone who drew those parallels this week) is the first step.  Adopting this mindset makes #1,#2 and #3 somewhat moot.  You only wind up in trouble if you try to approach it like normal D code (DMD 1.006 is going to change that quite a bit).

Then, I found that aggressive use of tuples makes a lot of things much easier to handle and grok; thanks BCS!  Again, this kind of list passing is very lisp-ish and makes for some very straightforward templates.

To that end, I went about writing a parsing lib in the same vein as Enki.  I currently have a tokenizer (attached) that will turn an input string into a tuple of token 'structs', complete with line/col information.  By this, I've found that using tuples has addressed point #4 only slightly by avoiding the kludge I added to the regex lib.  Passing such large tuples around still bloats the template "manglespace" horribly, and generating the list is about as bad.

That said, Walter, I think you're on the right track by making functions compile-time capable.  I might just refactor my code to take advantage of this.

This leads me to two questions for DMD 1.006:

Say I convert my tuple-processing code into compile-time functions that use const lists instead. What is the primary tradeoff, increased compile-time for unlimited call-depth due to non-bloated namespaces?  Is there another tradeoff that is more dominant here, and not as obvious?


-- 
- EricAnderton at yahoo

Pragma wrote:
> I've found that treating meta programming like lisp (thanks to everyone who drew those parallels this week) is the first step.  Adopting this mindset makes #1,#2 and #3 somewhat moot.  You only wind up in trouble if you try to approach it like normal D code (DMD 1.006 is going to change that quite a bit).

To someone who is comfortable with Lisp, I agree that 1, 2, and 3 are moot. But most of us aren't, and a very common request I'd get is to make metaprogramming look "like normal D code." The eventual metaprogramming goal is to get the power of Lisp expressible in the "normal D" syntax.


> To that end, I went about writing a parsing lib in the same vein as Enki.  I currently have a tokenizer (attached) that will turn an input string into a tuple of token 'structs', complete with line/col information.

I was thinking of writing one myself, you beat me to it.

> By this, I've found that using tuples has addressed point #4 only slightly by avoiding the kludge I added to the regex lib.  

Right, tuples don't fix the mangling problem at all.

> Passing such large tuples around still bloats the template "manglespace" horribly, and generating the list is about as bad.

Yes, this is the motivation for compile time interpretation.

> That said, Walter, I think you're on the right track by making functions compile-time capable.  I might just refactor my code to take advantage of this.

I think for parsing strings, this will vastly improve things.

> This leads me to two questions for DMD 1.006:
> 
> Say I convert my tuple-processing code into compile-time functions that use const lists instead.

I think the right result would be array literals.

> What is the primary tradeoff, increased compile-time for unlimited call-depth due to non-bloated namespaces?  Is there another tradeoff that is more dominant here, and not as obvious?

The tradeoff is you won't be able to generate tuples with functions, but you will be able to handle a couple orders of magnitude larger datasets to work on.

Walter Bright wrote:
> Pragma wrote:
>> I've found that treating meta programming like lisp (thanks to everyone who drew those parallels this week) is the first step.  Adopting this mindset makes #1,#2 and #3 somewhat moot.  You only wind up in trouble if you try to approach it like normal D code (DMD 1.006 is going to change that quite a bit).
> 
> To someone who is comfortable with Lisp, I agree that 1, 2, and 3 are moot. But most of us aren't, and a very common request I'd get is to make metaprogramming look "like normal D code." The eventual metaprogramming goal is to get the power of Lisp expressible in the "normal D" syntax.

Actually, FWIW, I'm not comfortable with Lisp.  D template coding is just "noisy" enough for me to grok as a list processing grammar. In contrast, my comprehension of Lisp breaks down after about three or four nestings.  Yea, I'm probably a huge weirdo in that respect. ;p

But I'm all for the changes and improvements.  Your rationale for adding to things is dead on.

> 
> 
>> To that end, I went about writing a parsing lib in the same vein as Enki.  I currently have a tokenizer (attached) that will turn an input string into a tuple of token 'structs', complete with line/col information.
> 
> I was thinking of writing one myself, you beat me to it.

Well, consider it public domain then.  Everyone, hack away as you wish.

> 
>> By this, I've found that using tuples has addressed point #4 only slightly by avoiding the kludge I added to the regex lib.  
> 
> Right, tuples don't fix the mangling problem at all.
> 
>> Passing such large tuples around still bloats the template "manglespace" horribly, and generating the list is about as bad.
> 
> Yes, this is the motivation for compile time interpretation.
> 
>> That said, Walter, I think you're on the right track by making functions compile-time capable.  I might just refactor my code to take advantage of this.
> 
> I think for parsing strings, this will vastly improve things.
> 
>> This leads me to two questions for DMD 1.006:
>>
>> Say I convert my tuple-processing code into compile-time functions that use const lists instead.
> 
> I think the right result would be array literals.

You're right.  My fingertips weren't connected to my brain in that sentence.

> 
>> What is the primary tradeoff, increased compile-time for unlimited call-depth due to non-bloated namespaces?  Is there another tradeoff that is more dominant here, and not as obvious?
> 
> The tradeoff is you won't be able to generate tuples with functions, but you will be able to handle a couple orders of magnitude larger datasets to work on.

Fantastic.  D is starting to look like one of those all-in-one Swiss army knives.

-- 
- EricAnderton at yahoo