Here is an interesting result from a budding Ph.D. in computer science who works with Python, a high-level language that is (a) weakly typed and (b) notoriously slow.  He shows how to execute such code with nearly C levels of performance. This is a working product, not an idea.  It shows at least two things that I have said before, (1) we have much to learn from experts, and (2) high level features do not need to spell "slow."  Read his remarks below.

Walter, the code is MIT licensed on SourceForge.  Plagiarize shamelessly!  -M

http://psyco.sourceforge.net/introduction.html

Think of Psyco as a kind of just-in-time (JIT) compiler, a little bit like Java's, that emit machine code on the fly instead of interpreting your Python program step by step. The result is that your unmodified Python programs run faster.  [D could do something similar to support high-level features. -M.]

The actual performance gains can be very large. For common code, expect at least a 2x speed-up, more typically 4x. But where Psyco shines is when running algorithmical code --- these are the first pieces of code that you would consider rewriting in C for performance. If you are in this situation, consider using Psyco instead! You might get 10x to 100x speed-ups. It is theoretically possible to actually speed up this kind of code up to the performance of C itself.

My goal in programming Psyco is to contribute to reduce the following wide gap between academic computer science and industrial programming tools.

While the former develops a number of programming languages with very cool semantics and features, the latter stick with low-level languages principally for performance reasons, on the ground that the higher the level of a language, the slower it is. Althought clearly justified in practice, this belief is theoretically false, and even completely inverted --- for large, evolving systems like a whole operating system and its applications, high-level programming can deliver much higher performances.

The new class of languages called "dynamic scripting languages", of which Python is an example, is semantically close to long-studied languages like Lisp. The constrains behind their designs are however different: some high-level languages can be relatively well statically compiled, we can do some type inference, and so on, whereas with Python it is much harder --- the design goal was different. We now have powerful machines to stick with interpretation for a number of applications. This, of course, contributes to the common belief that high-level languages are terribly slow.

Psyco is both an academic and an industrial project. It is an academic experiment testing some new techniques in the field of on-line specialization. It develops an industrially useful performance benefit for Python.

Mark Evans says...
>
>Here is an interesting result from a budding Ph.D. in computer science

(correction, mathematics; but his software is still sooo cooooooolllll.....he is an expert in my book)

Similar ideas have been around for a while, but I don't know anyone who's gone very far implementing them.

"Mark Evans" <Mark_member@pathlink.com> wrote in message news:ar4gbu$2kpv$1@digitaldaemon.com...
> Here is an interesting result from a budding Ph.D. in computer science who
works
> with Python, a high-level language that is (a) weakly typed and (b)
notoriously
> slow.  He shows how to execute such code with nearly C levels of
performance.
> This is a working product, not an idea.  It shows at least two things that
I
> have said before, (1) we have much to learn from experts, and (2) high
level
> features do not need to spell "slow."  Read his remarks below.
>
> Walter, the code is MIT licensed on SourceForge.  Plagiarize
hamelessly!  -M
>
> http://psyco.sourceforge.net/introduction.html
>
> Think of Psyco as a kind of just-in-time (JIT) compiler, a little bit like Java's, that emit machine code on the fly instead of interpreting your
Python
> program step by step. The result is that your unmodified Python programs
run
> faster.  [D could do something similar to support high-level
features. -M.]
>
> The actual performance gains can be very large. For common code, expect at
least
> a 2x speed-up, more typically 4x. But where Psyco shines is when running algorithmical code --- these are the first pieces of code that you would consider rewriting in C for performance. If you are in this situation,
consider
> using Psyco instead! You might get 10x to 100x speed-ups. It is
theoretically
> possible to actually speed up this kind of code up to the performance of C itself.
>
> My goal in programming Psyco is to contribute to reduce the following wide
gap
> between academic computer science and industrial programming tools.
>
> While the former develops a number of programming languages with very cool semantics and features, the latter stick with low-level languages
principally
> for performance reasons, on the ground that the higher the level of a
language,
> the slower it is. Althought clearly justified in practice, this belief is theoretically false, and even completely inverted --- for large, evolving systems like a whole operating system and its applications, high-level programming can deliver much higher performances.
>
> The new class of languages called "dynamic scripting languages", of which
Python
> is an example, is semantically close to long-studied languages like Lisp.
The
> constrains behind their designs are however different: some high-level
languages
> can be relatively well statically compiled, we can do some type inference,
and
> so on, whereas with Python it is much harder --- the design goal was
different.
> We now have powerful machines to stick with interpretation for a number of applications. This, of course, contributes to the common belief that
high-level
> languages are terribly slow.
>
> Psyco is both an academic and an industrial project. It is an academic experiment testing some new techniques in the field of on-line
specialization.
> It develops an industrially useful performance benefit for Python.
>
>

Several reviews I've read say Psycho takes way too much memorize to use and in most cases (non-algorithmetic) is not even a factor of 2 times faster.

This article is a pretty glowing review.

In article <ar4qre$2uii$1@digitaldaemon.com>, Mark Evans says...
>
>Mark Evans says...
>>
>>Here is an interesting result from a budding Ph.D. in computer science
>
>(correction, mathematics; but his software is still sooo cooooooolllll.....he is an expert in my book)
>
>

johnwhited@hotmail.com wrote:

> Several reviews I've read say Psycho takes way too much memorize to use and in most cases (non-algorithmetic) is not even a factor of 2 times faster.

I can tell you that I've sucessfully used it to improve about a factor of 4 a not specially algorithmetic program (a mail program), and that only psycompiling a few methods, now the entire code (altought I doubt I could gain much more performance psycompiling all).

>I can tell you that I've sucessfully used it to improve about a factor of 4 a not specially algorithmetic program (a mail program), and that only psycompiling a few methods, now the entire code (altought I doubt I could gain much more performance psycompiling all).

More than likely the amount of string manipulation you're doing benefits from the JITing process. Anyhow, I'm just quoting what I read from a couple of places reviewing the perf of this JITer.

I see you're interested in run-time specialiastion. Please read:

http://www.codeonthefly.com/academic.html

It also falls into my sphere of ineterests.

Much more inetersting for me is the runtime generation of numerical code. Imagine a music file, containing music score + playing algorithms + data used by the algorithms. That would involve generating code on the fly.