Don Clugston wrote:
> [...]
> But with the proposed changes to 'const', macros, and previous discussion about whether tuples should be automatically flattened, I have doubts over how stable that part of the language is. I wouldn't want to write something that didn't compile any more, or looked hopelessly obsolete by the time it was published.
> I've been bitten before -- CTFE blasted almost all of my previous metaprogramming code into the stone age <g>. What's the shelf life of this stuff?.

Well, hopefully, if Walter breaks your code again, it will be to make it even more elegant (although, that will be a bit of a trick, I admit). Eventually, the language will be so powerful that you can't make your code any more compact, and then you won't have to worry about it getting broken.  To be honest, I'd like to see an article about this in CUJ.  I don't know if they would publish it or not, but if they did, I guarantee it would make more than a few people take a hard look at D.

Dave

David B. Held wrote:
> Don Clugston wrote:
>> [...]
>> But with the proposed changes to 'const', macros, and previous discussion about whether tuples should be automatically flattened, I have doubts over how stable that part of the language is. I wouldn't want to write something that didn't compile any more, or looked hopelessly obsolete by the time it was published.
>> I've been bitten before -- CTFE blasted almost all of my previous metaprogramming code into the stone age <g>. What's the shelf life of this stuff?.
> 
> Well, hopefully, if Walter breaks your code again, it will be to make it even more elegant (although, that will be a bit of a trick, I admit). Eventually, the language will be so powerful that you can't make your code any more compact, and then you won't have to worry about it getting broken.  To be honest, I'd like to see an article about this in CUJ.  I don't know if they would publish it or not, but if they did, I guarantee it would make more than a few people take a hard look at D.

I'm pretty sure DDJ would publish it, and it would reach a much wider audience than CUJ, which has disappeared <g>.

Anyhow, Don, this kind of stuff needs to reach a much wider audience than a random posting in this n.g. would get.

This is really neat.
I like that this basically is a compiler extension and it's still
a) well readable
b) produces simple error messages during compilation (if any)
c) introduces non-artificial syntax for the task

c) will actually depend more on the versatility of operator overloading
than the CTFE stuff.
a) and b) are so much better than what c++ template magic gave us (as
can be observed in boost).

Now i think there's only one requirement missing to make it a perfect
compiler extension and that is to produce debuggable code.
Actually in this case, as the extension is supposed to generate mostly
single line expression statements, it might not be that important. But
if we consider multi-line DSL statements, a way to mark generated code
as belonging to one of the DSL source lines would be great.

I think, a way for a CTF to recieve the file+line where it was called from would do the trick. In BLADE that information could be added to the expression string, passed through to the postfix notation and finally be inserted into the ASM code with #line directives.

Don Clugston wrote:
> I have been trying to come up with a convincing use case for the new mixins (and for metaprogramming in general). My best effort to date can be found at: http://www.dsource.org/projects/mathextra/browser/trunk/mathextra/Blade.d
> 
> It generates near-optimal x87 asm code for BLAS1-style basic vector operations. 32, 64 and 80 bit vectors are all supported.
> 
> Compile with -version=BladeDebug to see the asm code which is generated.
> 
> Typical usage:
> 
> void main()
> {
>     auto p = Vec([1.0L, 2, 18]);    // a vector of 80-bit reals.
>     auto q = Vec([3.5L, 1.1, 3.8]);  // ditto
>     auto r = Vec([17.0f, 28.25, 1]); // a vector of 32-bit floats
>     auto z = Vec([17.0i, 28.1i, 1i]); // a vector of 64-bit idoubles
>     real d = dot(r, p+r+r);
>     ireal e = dot(r, z);
>     q -= ((r+p)*18.0L*314.1L - (p-r))* 35;
>     d = dot(r, p+r+r);
> }
> 
> Notice that mixed-length operations (real[] + float[] - double[]) are
> supported.
> 
> Like the C++ Blitz++ library, expression templates are used to convert
> vector expressions into efficient element-wise operations. Unlike that
> library, however, there is no reliance on the compiler's optimiser.
> Instead, the expression template is manipulated as text, converted into
> postfix, and then passed to a simple CTFE compile-time assembler, which
> creates highly efficient asm code which is used as a mixin.
> To understand the later parts of the code, you need some knowledge of
> x87 assembler. In fact, you probably need to have read Agner Fog's
> superb Pentium optimisation manual (www.agner.org).
> 
> Some observations:
> * I was amazed at how simple the expression template code is (it is
> somewhat cluttered by the code to check for real/imaginary type mismatch
> errors).
> * I've often read that the x87 floating-point stack is notoriously
> difficult for compilers to write code for, but it works quite well in
> this case.
> * The major workarounds are:
> - inability to use a tuple element directly from asm code (bug #1028);
> - inability to define operators for built-in arrays (hence the use of
> 'Vec' wrappers).
> - inability to index through a tuple in a CTFE function (solved by
> converting types into a string).
> * There have been mutterings about how unhygenic/dangerous the new
> mixins are. In this case, the mixin forms the _entire_ body of the
> function. This is an interesting situation which I think a language
> purist will find more palatable.
> 
> Enjoy.

Current compilers tend to dump IR's to a buffer and then adjust them according to parser rules.  The wrong approach.

I totally want to see this kind of programming get implemented into the compiler itself.  I know it's too much to ask, but if DMD could generate optimal floating point math and cache pipelines, it would rend the gaming industry from C++'s hands REAL quick like.

A C++ programmer can say:
"oh yeah, it's a minor upgrade made for my convenience, but I'm already conveniently comfortable with C++"...

Until a decision maker sees that D performs 20% better, dramatically reduces debugging time, cuts source code volume, and has better version control.

Walter Bright wrote:
> David B. Held wrote:
>> Don Clugston wrote:
>>> [...]
>>> But with the proposed changes to 'const', macros, and previous discussion about whether tuples should be automatically flattened, I have doubts over how stable that part of the language is. I wouldn't want to write something that didn't compile any more, or looked hopelessly obsolete by the time it was published.
>>> I've been bitten before -- CTFE blasted almost all of my previous metaprogramming code into the stone age <g>. What's the shelf life of this stuff?.
>>
>> Well, hopefully, if Walter breaks your code again, it will be to make it even more elegant (although, that will be a bit of a trick, I admit). Eventually, the language will be so powerful that you can't make your code any more compact, and then you won't have to worry about it getting broken.  To be honest, I'd like to see an article about this in CUJ.  I don't know if they would publish it or not, but if they did, I guarantee it would make more than a few people take a hard look at D.
> 
> I'm pretty sure DDJ would publish it, and it would reach a much wider audience than CUJ, which has disappeared <g>.
> 
> Anyhow, Don, this kind of stuff needs to reach a much wider audience than a random posting in this n.g. would get.

Ooooooh....
I just updated the code, adding a code generator for D code, for when x87 is unavailable. I could not believe how easy it was. It was about 30 lines of trivial code. Yet directly generating a 'for' loop is still better than what Blitz++ does.
I reckon it'd take less than a week to implement everything those libraries do.

The language synergy of tuples, mixins, CTFE, builtin strings, the ~ operator, constant folding, array slicing, template string parameters, type deduction, defined inline asm, static if, is() expressions... is astonishing. *All* of them are required in order for this to work.

Don Clugston wrote:
> Ooooooh....
> I just updated the code, adding a code generator for D code, for when x87 is unavailable. I could not believe how easy it was. It was about 30 lines of trivial code. Yet directly generating a 'for' loop is still better than what Blitz++ does.
> I reckon it'd take less than a week to implement everything those libraries do.
> 
> The language synergy of tuples, mixins, CTFE, builtin strings, the ~ operator, constant folding, array slicing, template string parameters, type deduction, defined inline asm, static if, is() expressions... is astonishing. *All* of them are required in order for this to work.

Gees, Don,

the above quote would be the perfect ingress (or at least a "quote box") in a DDJ article!!!!

I'm clenching the chair so I wouldn't buy the tickets to come to your place and write it together with you, right now!

Quoting from the page:

> If you have an article idea, you should first send us an article
> proposal. This is a short statement (about one page) that says what your
> article will be about. You can mail it to our <a href="address.htm">office</a>
> or e-mail it to <a href="mailto:editors@ddj.com">editors@ddj.com</a>.
> Be sure to give us several ways to contact you (such as phone number,
> e-mail address, fax number, and mailing address). We'll look at your
> idea and get back to you, usually within a month.</p>
...
> <p>Although many people simply write their entire article and send it
> to us, there are several *advantages* to sending us a proposal. We can
> look at your idea and possibly suggest a particular slant you hadn't
> considered. We can also help you to organize the article.

Later, when you start writing the article itself:

> Write as if you are giving a brief, informal talk to a group of
> coworkers. You want to be concise and well organized because you don't
> want to waste their time, but you don't want to be too stuffy or
> formal because you know them.

Ah, and to give you more of the "feel" of being an article author, here's another quote from the same page:

> Illustrations and screen captures offer an opportunity to lure
> anyone looking through an issue into reading your article. They
> can also provide succinct summaries of complex ideas. Since we
> redraw most figures anyway, to conform to our size and style
> requirements,
> you should not worry about making your images very high quality.
> Many authors prefer to sketch them by hand and fax or mail them.

Heh, especially the last sentence. The same goes for the text itself. The point being, from the magazine's point of view, most stuff sent to them is by teenagers, idiots, mentals, over-the-hill old-timers -- and only a handful is stuff they'd be interested in. Now, for those few, they'd kill, lick your boots, or even sharpen your pencil for you.

As to whether this would interest them, all you have to send them is the above mentioned Ingress. Let them take over from that. Simply naming Walter as a reference should get you a definite YES/NO from them. And once you get the YES, they'll hold your hand through and through.

---

For you personally, having your article in DDJ is of course "nice". But the real benefit is that then you've done it, been there. So, by the time D advances yet another bit, and your "template jujitsu" turns into "template nuke" (as perceived from the audience -- you'll never notice the change yourself!), it's gonna be a lot easier to jot up something for them again. Or for the next magazine. (Ahem, I'm ashamed to say, but at least as I experienced, my first article was a bit like "doing it for the first time with a woman". Once you get over it, the next one is a piece of cake.)

Thusly, both now and in the future, you're in a position to do D a /lot/ of good -- with effort that's actually less than some of your better D forages to date.

PS, if they get back to you within /less/ than 4 weeks, they're drooling. Unless they've read this post, of course. ;-)

Don Clugston wrote:
> Ooooooh....
> I just updated the code, adding a code generator for D code, for when x87 is unavailable. I could not believe how easy it was. It was about 30 lines of trivial code. Yet directly generating a 'for' loop is still better than what Blitz++ does.
> I reckon it'd take less than a week to implement everything those libraries do.
> 
> The language synergy of tuples, mixins, CTFE, builtin strings, the ~ operator, constant folding, array slicing, template string parameters, type deduction, defined inline asm, static if, is() expressions... is astonishing. *All* of them are required in order for this to work.

Just wait 'till you get AST macros!

But this stuff is *awesome*. Scratch DDJ, I don't want to wait 6 months. This needs to get out now, - I suggest www.artima.com, followed up with dzone, slashdot, and digg.

It would also be a real treat to have you present this at Amazon's D conference in August.

Walter Bright Wrote:
> Just wait 'till you get AST macros!
> 
> But this stuff is *awesome*. Scratch DDJ, I don't want to wait 6 months. This needs to get out now, - I suggest www.artima.com, followed up with dzone, slashdot, and digg.
> 
> It would also be a real treat to have you present this at Amazon's D conference in August.

Wow.  That's the most excited I've seen Walter.  : )
I'm going to have to read and understand your code.  I can tell.

Dan wrote:
> Walter Bright Wrote:
> 
>>Just wait 'till you get AST macros!
>>
>>But this stuff is *awesome*. Scratch DDJ, I don't want to wait 6 months. This needs to get out now, - I suggest www.artima.com, followed up with dzone, slashdot, and digg.
>>
>>It would also be a real treat to have you present this at Amazon's D conference in August.
> 
> 
> Wow.  That's the most excited I've seen Walter.  : )

LOL, have to admit I agree!!!