We really need a partial rewrite of Phobos, at least the low/level always used stuff, in a template-based way.

It would be good to have it in the 1.0, at least because a lot of D newbie will LOOK INTO the standard library sources searching for great D code.

my 2c
---
Paolo Invernizzi

Don Clugston wrote:
> Indeed I have. The tuple stuff makes this all really appealing. In particular, I've played around with a writef() equivalent which checks the parameters for 'easy' cases, and splits it off into:
> -> single string only
> -> strings, chars, and integers
> -> floating point, arrays, and objects.
> 
> This way you can avoid linking in the floating-point conversion code if you're not using it, keeping the code size down. In fact, it ought to be possible to remove the usage of TypeInfo completely, moving it all into compile-time.

Jarrett Billingsley wrote:
> A little off the topic, but not really, as it spins off of the idea of a compiler as an interpreter.  I always thought it would be an interesting exercise to make a language where metaprogramming is not only possible, but nearly as full-featured as the actual code.  That is, templates (or whatever they'd be called, because they'd be far more advanced) would be a script for the compiler, which could even be compiled to bytecode.  The input would be language constructs -- symbols, statements, expressions -- and the output would be code which could then be compiled.  It'd basically be a scriptable compiler.

Poor Lisp.  It just sits there, 50 years old, debugged, optimized, and ready to go, while the imperative languages try to inch closer over the decades.

So OOP comes along.  Lisp adds it (in a superior way, imo)
So AOP is (or will be) hot.  Lisp adds it.
Metaprogramming?  The MetaObject Protocol appears, to complement the already
amazing macro facility.

I'm not knocking the imperative languages, as a lot of people know them and use them successfully.  I am more amazed at how such a valuable toolset is consistently under-used and its functionality is rewritten from scratch.

As this thread talks about metaprogramming and syntax, the elegance of Lisp's code being the same as its data is relevant.  To be the ultimate expression of metaprogramming in this way, you cannot do it in a new language.  You'd end up merely with another flavor/dialect of Lisp.

Greenspun's 10th Rule of Programming:  Any sufficiently complicated C or Fortran program contains an ad hoc, informally-specified, bug-ridden, slow implementation of half of Common Lisp.

BA

P.S.  Please no ignorant replies about Lisp is interpreted or Lisp is slower than the imperative languages.

Just a few corrections, as I was still waking up when I wrote this.

Sean Kelly wrote:
> Frank Benoit (keinfarbton) wrote:
>>
>> 1.) Is metaprogramming really useful or only kind of hype?
> 
> In-language code generation has an advantage over using a standalone preprocessor to perform the same task.  And generated code has the potential to contain fewer bugs than hand-written code, as well as reduce coding time for a project.  Finally, template code in general combined with common optimizer techniques can result in extremely fast code because it is essentially equivalent to hand-optimized code in many cases.

Equivalent to or better than.  Inlining is a huge part of why template code is as fast as it is, and even hand tuning typically results in little manually inlined code--it's too difficult to maintain.

> How fast?  Bjarne did a presentation at SDWest regarding some real-world applications where C++ was shown to outperform hand-tuned C and even FORTRAN by an order of magnitude for numeric calculations, and the reason was entirely attributed to the extensive use of template code.

I believe it actually outperformed hand-tuned C by an order of magnitude and FORTRAN by a smaller margin, but it was still faster.  Bjarne attributed the results to templates, but inlining obviously played a huge part.


Sean

Charles D Hixson wrote:
> 
> Of course, what would be really interesting would be a runtime version of D, with runtime type assignment, etc., but without the kind of bloat that would occur if this were done with templates.
> 
> (OTOH, I must admit that I'm guessing at the amount of bloat that a generic template would add.  They don't add that much in Eiffel or Ada code, but they were impossibly bloated the last time I tried it in C++ [admittedly that's over a decade ago].)

Things have improved greatly, though actual results still vary widely from compiler to compiler.  Here's a link to the C++ performance report compiled a few years ago:

http://www.research.att.com/~bs/performanceTR.pdf

The crucial part of reducing code size is for the compiler to recognize that the code for many specializations is actually the same (pointers to different class types, for example), and to eliminate duplicates.  This can be done manually in library code as well (containers might use a thin wrapper on top of a more traditional class that stores values as void*), if the compiler optimizations are not sufficient.


Sean

Brad Anderson wrote:
> Jarrett Billingsley wrote:
> 
>>A little off the topic, but not really, as it spins off of the idea of a compiler as an interpreter.  I always thought it would be an interesting exercise to make a language where metaprogramming is not only possible, but nearly as full-featured as the actual code.  That is, templates (or whatever they'd be called, because they'd be far more advanced) would be a script for the compiler, which could even be compiled to bytecode.  The input would be language constructs -- symbols, statements, expressions -- and the output would be code which could then be compiled.  It'd basically be a scriptable compiler.

You've just described Lisp. And very precisely, at that.

And today's Common Lisp implementations blend compiler and interpreter so thoroughly that you don't know which is going on when.

> Poor Lisp.  It just sits there, 50 years old, debugged, optimized, and ready
> to go, while the imperative languages try to inch closer over the decades.

Somehow I've always felt that it's mostly not about other languages copy catting features. In Lisp's case it's more like, there's a Right Way to do things, and Lisp has found many of them right from the start.

> So OOP comes along.  Lisp adds it (in a superior way, imo)
> So AOP is (or will be) hot.  Lisp adds it.
> Metaprogramming?  The MetaObject Protocol appears, to complement the already
> amazing macro facility.

The most amazing thing about Lisp is how simply trivial all the hard stuff is. So, someone invents OOP, no problem, let's write a complete OOP framework, in just 200 lines of code. AOP anyone, hey, that'll be ready by tomorrow morning.

And Metaprogramming? Ha, even the trivial (3kloc, in D) Lisp-in-D implementation (dLISP) on dsource has defmacro. (Which actually works!)

> I'm not knocking the imperative languages, as a lot of people know them and
> use them successfully.  I am more amazed at how such a valuable toolset is
> consistently under-used and its functionality is rewritten from scratch.

There's a single reason for it. Lisp is /scary/. It's scary in itself, but even worse, folks get scared that it undresses them. ("Start mucking with Lisp, and they might see that you're not as smart as you've led everyone to believe.") And the parentheses are simply a convenient excuse for abstinence.

> As this thread talks about metaprogramming and syntax, the elegance of Lisp's
> code being the same as its data is relevant.  To be the ultimate expression of
> metaprogramming in this way, you cannot do it in a new language.  You'd end up
> merely with another flavor/dialect of Lisp.

Well, Lisp is the attractor towards which the D meta language is moving, whether we want it, or not. We've even moved quite fast lately. And, we're actually much nearer to Lisp now than most folks here even suspect!

> Greenspun's 10th Rule of Programming:  Any sufficiently complicated C or
> Fortran program contains an ad hoc, informally-specified, bug-ridden, slow
> implementation of half of Common Lisp.

It's not a coincidence that I installed Allegro Common Lisp (from Franz Inc, a demo version of an excellent commercial implementation) on my Fedora last week. And dLISP for comparison.

On Tue, 28 Nov 2006 09:31:23 -0500, Brad Anderson <brad@dsource.org> wrote:

>P.S.  Please no ignorant replies about Lisp is interpreted or Lisp is slower than the imperative languages.

No - I already did that a few years ago on comp.lang.python. I'm less ignorant these days.

I have tried to learn Scheme on many occasions, though, and only ever got so far. Sure, it has that key mechanism there, but learning to use the language is a bit like learning C++ by first learning how to write asm blocks, and finding no teacher who is ever willing to teach you the high level tools you need for real everyday work. Fine, you can do anything, but why do you always have to reinvent all those wheels?

The thing is that in the real world, most programmers need a standard, familiar dialect which has all the everyday high level tools available from the start. If OOP is an immediately useful concept, programmers should be using OOP from day one, not learning how to reinvent OOP from the basic building blocks. So, has anyone created widely-used standard librarys for high level programming?

On that XLR site referenced by Leandro in the "Thesis on metaprogramming in D" thread, a reference is made to the problem of dialects in what they call concept-oriented programming. They seem to be getting at some other dialect-related issue, but one dialect issue is simply that if everyone has the ability to invent their own type of object orientation, their own type of generic, etc etc then there you don't really have a single language. XLR takes the view that a standard implementation is provided for each concept, and you should only roll your own concept if you need it, but has that standardisation been done for Scheme or other Lisp-alikes?

Of course in a sense you have a new dialect every time you switch libraries in any language. The dialect of C++ where you use GTK or QT for GUI stuff is very different from the dialect where you use Win32, for instance. But there is at least common ground - terms like 'class' and 'object' have exactly the same meaning.

Whenever I go looking for this standard set of concepts for Scheme or whatever, I can't find it. There's a very low level core library, in the sense of only providing very basic tools, and that's it. Of course there are plenty of libraries out there, but...

1.  They tend to be higher level building blocks rather than useable
    concepts. You get things like PGG for partial evaluation and
    Essence for LR parsing, which are great if you want to create your
    own extension to Scheme with higher level concepts, but the leap
    to actually creating usable, standardised higher level programming
    tools using these things does not seem to have been made.

2.  They don't seem to be widely used standards. They seem to be
    mainly academic experiments. For instance, you can find academic
    papers about them descibing the theory behind them and conclusions
    resulting from implementing them and testing them out with
    contrived examples, but no user guides or tutorials.

So sure, you can in theory make Lisp work more-or-less like Pascal or C, but with the option to work differently when needed, but that's a lot of work to do from the basic Lisp-alike building blocks. So where is that standard high level programming concepts library?

-- 
Remove 'wants' and 'nospam' from e-mail.

== Extrait de l'article de « Brad Anderson (brad@dsource.org) »
> Poor Lisp.  [cut]  I am more amazed at how such a valuable toolset is consistently under-used and its functionality is rewritten from scratch.

Well, I'm not surprised myself: Lisp syntax is not user-friendly, people wants to use language with easy-to-read syntax, not a language with an abstract-syntax-tree syntax. So in all likelihood Lisp will stay mostly unused in the future.

Back on the subject of metaprogamming, one thing which makes me cautious about metaprogramming is debugging: when there is a problem debugging generated code is a nightmare usually..

renoX

Georg Wrede wrote:
> Brad Anderson wrote:
>> Greenspun's 10th Rule of Programming:  Any sufficiently complicated C or
>> Fortran program contains an ad hoc, informally-specified, bug-ridden, slow
>> implementation of half of Common Lisp.
> 
> It's not a coincidence that I installed Allegro Common Lisp (from Franz Inc, a demo version of an excellent commercial implementation) on my Fedora last week. And dLISP for comparison.

Sorry to ask it here, but how do you evaluate dLISP? Is it complete enough?

I'm learning lisp atm, it sounds attractive to embed a lisp interpreter in D programs and use them together. This should be very easy with dLISP I think.

renoX wrote:
> == Extrait de l'article de « Brad Anderson (brad@dsource.org) »
>> Poor Lisp.  [cut]  I am more amazed at how such a valuable toolset is
>> consistently under-used and its functionality is rewritten from scratch.
> 
> Well, I'm not surprised myself: Lisp syntax is not user-friendly, people wants to
> use language with easy-to-read syntax, not a language with an abstract-syntax-tree
> syntax. So in all likelihood Lisp will stay mostly unused in the future.
> 
> Back on the subject of metaprogamming, one thing which makes me cautious about
> metaprogramming is debugging: when there is a problem debugging generated code is
> a nightmare usually..

It's gotten pretty good in C++ and will get better with concept checking.  And D is already better at reporting compile-time errors, since we have the use of static if, static assert, and pragma(msg). Once we get an instantiation trace when static asserts fail I think we'll be in good shape there.  Run-time template debugging is obviously a bit behind, but that's only a matter of time.


Sean

"Brad Anderson" <brad@dsource.org> wrote in message news:ekhh7s$2e7$1@digitaldaemon.com...

> Poor Lisp.  It just sits there, 50 years old, debugged, optimized, and
> ready
> to go, while the imperative languages try to inch closer over the decades.

In that case.. it'd be another interesting experiment to try to come up with a new syntax for Lisp that appeals to more programmers than it does now ;)

I really can't get past the parentheses.  I know Georg said it's an excuse, but I really, truly cannot understand most Lisp code because I can't tell which right paren out of a group of six is closing which left paren.  I'm sure bracket highlighting in a code editor can help, but why should that be necessary?  I'm sure a good deal of those parens can be stripped out, or replaced by other brackets, or just moved around to get a more algebraic syntax.