Walter:

> http://channel9.msdn.com/Shows/C9-GoingNative/GoingNative-6-The-D-Episode-with-Walter-Bright-and-Andrei-Alexandrescu

Andrei says that some new languages suffer because they have a poor implementation, because creating the base for a language is a lot of work.
Today this is issue is much less of a problem, new languages are implemented on the JavaVM, DotNetVM. System languages are implemented with LLVM.
And maybe PyPy will be a good base to create efficient dynamic languages quickly:
  http://tratt.net/laurie/tech_articles/articles/fast_enough_vms_in_fast_enough_time


Regarding the comparison between dynamic languages like Python or Ruby and D: what Andrei has said is not fully fair. A simple common scripting task: read the lines of a text file and put them in a hash. This is probably faster in Python compared to D. I am willing to write a benchmark too, if asked.


Regarding what Walter has said, that most of the code of D applications will be @safe: if safe code gets (or has to get) so common as he says, then it will be good for the D compiler to learn some tricks to avoid (optimize away) array bound tests in some cases.

Bye,
bearophile

On 2/22/2012 7:51 PM, bearophile wrote:
> Andrei says that some new languages suffer because they have a poor
> implementation, because creating the base for a language is a lot of work.
> Today this is issue is much less of a problem, new languages are implemented
> on the JavaVM,

Using the JVM forces your program into Java semantics. For example, there are no structs in the JVM bytecode. No pointers, either. Nor unsigned types. Your new language is fairly boxed in to being a rehash of Java semantics.


> DotNetVM.

Cristi's D compiler on .NET had large problems because array slicing was not expressible in the .NET intermediate code. It's not nearly as bad as the JVM in that regard, but it's still limited.


> System languages are implemented with LLVM.

That works if your language is expressible as C, because LLVM is a C/C++ back end. If your language has different semantics (like how Go does stacks), using LLVM can be a large problem.

Note that early C++ compilers suffered badly when they were forced into using C back ends, because C++ wanted new features (like COMDATs) which are not expressible in C and so not supported by C back ends. Ditto for any language feature that needs something in the back end that C/C++ doesn't need.


> Regarding the comparison between dynamic languages like Python or Ruby and D:
> what Andrei has said is not fully fair. A simple common scripting task: read
> the lines of a text file and put them in a hash. This is probably faster in
> Python compared to D. I am willing to write a benchmark too, if asked.

That would be comparing library code, not language code. Much of Python's implementation is in C, not Python.


> Regarding what Walter has said, that most of the code of D applications will
> be @safe: if safe code gets (or has to get) so common as he says, then it
> will be good for the D compiler to learn some tricks to avoid (optimize away)
> array bound tests in some cases.

I looked into this years ago. Very little of array bounds checking can be optimized away. I've been working on optimizers for 25 years now, including a native code generating Java compiler, and I do know a few things about how to do arrays.

Clang has some pretty good ideas, like the spell checker on undefined identifiers. But others talked about in the spiel at GoingNative have been in compilers for 30 years.

On 23-02-2012 07:51, Walter Bright wrote:
> On 2/22/2012 7:51 PM, bearophile wrote:
>> Andrei says that some new languages suffer because they have a poor
>> implementation, because creating the base for a language is a lot of
>> work.
>> Today this is issue is much less of a problem, new languages are
>> implemented
>> on the JavaVM,
>
> Using the JVM forces your program into Java semantics. For example,
> there are no structs in the JVM bytecode. No pointers, either. Nor
> unsigned types. Your new language is fairly boxed in to being a rehash
> of Java semantics.

I still cannot fathom how the Scala guys thought using the JVM was a good idea.

>
>
>> DotNetVM.
>
> Cristi's D compiler on .NET had large problems because array slicing was
> not expressible in the .NET intermediate code. It's not nearly as bad as
> the JVM in that regard, but it's still limited.

I assume the problem here was the .ptr property? I can't think of anything else about array slices that would be problematic in CIL.

>
>
>  > System languages are implemented with LLVM.
>
> That works if your language is expressible as C, because LLVM is a C/C++
> back end. If your language has different semantics (like how Go does
> stacks), using LLVM can be a large problem.

I don't think that's true. D, Rust, C# (CIL in general), Cray, ActionScript, Python, Java, Lua, Haskell, and many others have been compiled with LLVM successfully.

LLVM is very much engineered for C and C++, but it has many other features that those languages don't make use of at all (see for example the precise GC support; and this is not even something Apple uses, as their Obj-C GC is conservative). There is also the language-specific support for OCaml's precise stack maps. Also, LLVM has segmented stacks on x86 these days. That said, LLVM is definitely not as easy to use for managed languages as it is for systems/native languages.

So, while the design of LLVM certainly is driven by Clang primarily, it's not as if they don't welcome non-C family features.

>
> Note that early C++ compilers suffered badly when they were forced into
> using C back ends, because C++ wanted new features (like COMDATs) which
> are not expressible in C and so not supported by C back ends. Ditto for
> any language feature that needs something in the back end that C/C++
> doesn't need.
>
>
>> Regarding the comparison between dynamic languages like Python or Ruby
>> and D:
>> what Andrei has said is not fully fair. A simple common scripting
>> task: read
>> the lines of a text file and put them in a hash. This is probably
>> faster in
>> Python compared to D. I am willing to write a benchmark too, if asked.
>
> That would be comparing library code, not language code. Much of
> Python's implementation is in C, not Python.
>
>
>> Regarding what Walter has said, that most of the code of D
>> applications will
>> be @safe: if safe code gets (or has to get) so common as he says, then it
>> will be good for the D compiler to learn some tricks to avoid
>> (optimize away)
>> array bound tests in some cases.
>
> I looked into this years ago. Very little of array bounds checking can
> be optimized away. I've been working on optimizers for 25 years now,
> including a native code generating Java compiler, and I do know a few
> things about how to do arrays.

In Mono, we found that ABC removal was actually beneficial in some code; consider for example allocating small static arrays locally and indexing them with constants (or very simple expressions).

>
> Clang has some pretty good ideas, like the spell checker on undefined
> identifiers. But others talked about in the spiel at GoingNative have
> been in compilers for 30 years.
>

-- 
- Alex

On 2/22/12 9:51 PM, bearophile wrote:
> Regarding the comparison between dynamic languages like Python or
> Ruby and D: what Andrei has said is not fully fair. A simple common
> scripting task: read the lines of a text file and put them in a hash.
> This is probably faster in Python compared to D.

I assume you're referring to the point starting around 16:50. I don't see how fairness is an issue here. Anyway, the point there is that in my experience performance of short scripts in dynamic languages generally decays quickly when things get interesting. Of course, it comes without saying that choosing one particular script that exercises a very small, well-honed area of a toolchain and that is essentially dominated by library calls may show excellent performance of a scripting language.

> I am willing to write a benchmark too, if asked.

What would be better would be investigating what particular aspects of D we could improve (and better yet, contribute the improvements themselves).

If one were to corroborate this post with this other one:

http://forum.dlang.org/post/jhn1ub$2tni$1@digitalmars.com

the conclusion is there's a conspiracy afoot targeted at sabotaging D scripting :o).


Andrei

On 2/23/2012 8:57 AM, Alex Rønne Petersen wrote:
>> > System languages are implemented with LLVM.
>>
>> That works if your language is expressible as C, because LLVM is a C/C++
>> back end. If your language has different semantics (like how Go does
>> stacks), using LLVM can be a large problem.
>
> I don't think that's true. D, Rust, C# (CIL in general), Cray, ActionScript,
> Python, Java, Lua, Haskell, and many others have been compiled with LLVM
> successfully.
>
> LLVM is very much engineered for C and C++, but it has many other features that
> those languages don't make use of at all (see for example the precise GC
> support; and this is not even something Apple uses, as their Obj-C GC is
> conservative). There is also the language-specific support for OCaml's precise
> stack maps. Also, LLVM has segmented stacks on x86 these days. That said, LLVM
> is definitely not as easy to use for managed languages as it is for
> systems/native languages.
>
> So, while the design of LLVM certainly is driven by Clang primarily, it's not as
> if they don't welcome non-C family features.

What you're saying is they add support here and there for non-C languages, meaning someone has to do it if it isn't there already.


> In Mono, we found that ABC removal was actually beneficial in some code;
> consider for example allocating small static arrays locally and indexing them
> with constants (or very simple expressions).

This is done already by standard data flow optimization techniques.

On 02/23/2012 11:57 AM, Alex Rønne Petersen wrote:
>
> I still cannot fathom how the Scala guys thought using the JVM was a
> good idea.

It gave them a good garbage collector (an area that has held D's performance back for years), a large library via Java compatibility, and a population of users to appeal to. Scala probably wouldn't have succeeded if it was just another object-oriented language without ties to the JVM.

On 24-02-2012 05:06, Jeff Nowakowski wrote:
> On 02/23/2012 11:57 AM, Alex Rønne Petersen wrote:
>>
>> I still cannot fathom how the Scala guys thought using the JVM was a
>> good idea.
>
> It gave them a good garbage collector (an area that has held D's
> performance back for years), a large library via Java compatibility, and
> a population of users to appeal to. Scala probably wouldn't have
> succeeded if it was just another object-oriented language without ties
> to the JVM.

They could have used the CLI. Both MS.NET and Mono have good garbage collectors, and Mono in particular has SIMD intrinsics, an LLVM back end, etc.

I've heard a lot of people complain that Scala doesn't run on .NET.

-- 
- Alex

On 2/23/12 10:06 PM, Jeff Nowakowski wrote:
> On 02/23/2012 11:57 AM, Alex Rønne Petersen wrote:
>>
>> I still cannot fathom how the Scala guys thought using the JVM was a
>> good idea.
>
> It gave them a good garbage collector (an area that has held D's
> performance back for years)

And still is, unfortunately.

Andrei

On 2/23/12 11:09 PM, Alex Rønne Petersen wrote:
> On 24-02-2012 05:06, Jeff Nowakowski wrote:
>> On 02/23/2012 11:57 AM, Alex Rønne Petersen wrote:
>>>
>>> I still cannot fathom how the Scala guys thought using the JVM was a
>>> good idea.
>>
>> It gave them a good garbage collector (an area that has held D's
>> performance back for years), a large library via Java compatibility, and
>> a population of users to appeal to. Scala probably wouldn't have
>> succeeded if it was just another object-oriented language without ties
>> to the JVM.
>
> They could have used the CLI. Both MS.NET and Mono have good garbage
> collectors, and Mono in particular has SIMD intrinsics, an LLVM back
> end, etc.
>
> I've heard a lot of people complain that Scala doesn't run on .NET.

I think Scala has positioned itself as a Java replacement with a smooth transition. The JVM works on more platforms than .NET so there was a simple argument by numbers to be made. Probably most importantly, Odersky already was a Java expert motivated by needs and desires on that particular platform.

Andrei

On 02/24/2012 12:09 AM, Alex Rønne Petersen wrote:
>
> They could have used the CLI. Both MS.NET and Mono have good garbage
> collectors, and Mono in particular has SIMD intrinsics, an LLVM back
> end, etc.
>
> I've heard a lot of people complain that Scala doesn't run on .NET.

Scala initially ran on .NET, but there was more traction with the Java crowd, so they stopped supporting it. As I understand it, .NET support has recently been funded by Microsoft and is actively being worked on.

There was, and still is, a much greater need for Scala on the JVM than on .NET, because Java was stagnating while Microsoft's C# language was getting big improvements. As for Mono, there's a lot of mistrust against following Microsoft's lead from the Linux crowd. So while there's some interest in Scala on .NET, it pales in comparison to the JVM.