The author talks about C++ performance but D can match it whilst bringing scripting language style programmer productivity, and arguably higher quality code (because you can understand the code base as a coherent whole).   Integration with C++ libraries is really the last missing piece, and it seems clear enough that is on the way to being solved at which point one of the principal relative advantages of Java (and cython / numpy - although latter less applicable to map reduce) goes away.

I heard a talk from the chap behind commoncrawl (previously at Napster and LinkedIn and seemed a hacker by temperament).  He observed that vast data sets put massive strain on code and would tend to find obscure bugs and strangeness sooner or later.  He talked about the non determinacy (for practical purposes) of JIT creating strange bugs that were very difficult to reproduce since the compiler would generate different machine code on each run.

And beyond slower execution speed  of Java, the memory bloat makes a big difference given how cloud pricing works (its peanuts to get a machine with a gig of ram, but 64 gig is not so cheap, and quickly gets very expensive - and one may need hundreds of machines).


http://www.trendcaller.com/2009/05/hadoop-should-target-cllvm-not-java.html

Over the years, there have been many contentious arguments about the performance of C++ versus Java. Oddly, every one I found addressed only one kind of performance (work/time). I can't find any benchmarking of something at least as important in today's massive-scale-computing environments, work/watt. A dirty little secret about JIT technologies like Java, is that they throw a lot more CPU resources at the problem, trying to get up to par with native C++ code. JITs use more memory, and periodically run background optimizer tasks. These overheads are somewhat offset in work/time performance, by extra optimizations which can be performed with more dynamic information. But it results in a hungrier appetite for watts. Another dirty little secret about Java vs C++ benchmarks is that they compare single-workloads. Try running 100 VMs, each with a Java and C++ benchmark in it and Java's hungrier appetite for resources (MHz, cache, RAM) will show. But of course, Java folks don't mention that.

But let's say for the sake of (non-)argument, that Java can achieve a 1:1 work/time performance relative to C++, for a single program. If Java consumes 15% more power doing it, does it matter on a PC? Most people don't dare. Does it matter for small-scale server environments? Maybe not. Does it matter when you deploy Hadoop on a 10,000 node cluster, and the holistic inefficiency (multiple things running concurrently) goes to 30%? Ask the people who sign the checks for the power bill. Unfortunately, inefficiency scales really well.

On Sunday, 25 January 2015 at 21:50:53 UTC, Laeeth Isharc wrote:
> And beyond slower execution speed  of Java, the memory bloat makes a big difference given how cloud pricing works (its peanuts to get a machine with a gig of ram, but 64 gig is not so cheap, and quickly gets very expensive - and one may need hundreds of machines).

Yes, but memory bloat is not D's strength either until manual memory management is addressed in a satisfactory manner. Rust is way ahead.

On Monday, 26 January 2015 at 18:53:45 UTC, Ola Fosheim Grøstad wrote:
> On Sunday, 25 January 2015 at 21:50:53 UTC, Laeeth Isharc wrote:
>> And beyond slower execution speed  of Java, the memory bloat makes a big difference given how cloud pricing works (its peanuts to get a machine with a gig of ram, but 64 gig is not so cheap, and quickly gets very expensive - and one may need hundreds of machines).
>
> Yes, but memory bloat is not D's strength either until manual memory management is addressed in a satisfactory manner. Rust is way ahead.

It seems to me (as a newcomer) that often with D it is the gap between what the language wants to be, and the present reality that upsets people, whereas pragmatically it remains much better than the alternatives even if you have to do a bit of extra work to allocate manually.  It is like seeing a beautiful woman marred by a flaw that you just can't seem to ignore until you get to know her as a person.  (No apologies for sexism here).  The problems of garbage collection in D seem different to those of GC (and memory bloat) in Java whereas people hear GC and not quite perfect and they instantly slot it into their mental slot of GC collected languages, which means Java.

Does Rust have the productivity of D?  And it doesn't have the maturity, as I understand it.

On Monday, 26 January 2015 at 20:19:09 UTC, Laeeth Isharc wrote:
>
> Does Rust have the productivity of D?  And it doesn't have the maturity, as I understand it.

This brings up something that's been bugging me.  D has a pitch for users of a lot of crappy languages, but what do we say when the competition isn't a total slouch?

This exchange (names changed) is what started this train of thought:
<chum> though i don't understand what the point of D is either because once you've already accepted a gc there are better languages you could use
<chum> and if you refuse to accept one, then, well, you either have c++11 or you wait for rust to be usable
<otherguy> chum: what is better than D once youre willing to have managed mem?
<chum> it's functional, but the complaint all the gamedev folks have about fp langs is that their implementations are usually garbage collected and they can't accept gc pauses
<chum> otherguy: f#, ocaml, haskell

Particularly against F#, I'm not sure what to say (it's a pretty interesting language, even if it's kind of ugly to look at and CIL-ly).  Thoughts?

-Wyatt

On Sunday, 25 January 2015 at 21:50:53 UTC, Laeeth Isharc wrote:
> The author talks about C++ performance but D can match it whilst bringing scripting language style programmer productivity, and arguably higher quality code (because you can understand the code base as a coherent whole).   Integration with C++ libraries is really the last missing piece, and it seems clear enough that is on the way to being solved at which point one of the principal relative advantages of Java (and cython / numpy - although latter less applicable to map reduce) goes away.
>
> I heard a talk from the chap behind commoncrawl (previously at Napster and LinkedIn and seemed a hacker by temperament).  He observed that vast data sets put massive strain on code and would tend to find obscure bugs and strangeness sooner or later.  He talked about the non determinacy (for practical purposes) of JIT creating strange bugs that were very difficult to reproduce since the compiler would generate different machine code on each run.
>
> And beyond slower execution speed  of Java, the memory bloat makes a big difference given how cloud pricing works (its peanuts to get a machine with a gig of ram, but 64 gig is not so cheap, and quickly gets very expensive - and one may need hundreds of machines).
>
>
> http://www.trendcaller.com/2009/05/hadoop-should-target-cllvm-not-java.html
>
> Over the years, there have been many contentious arguments about the performance of C++ versus Java. Oddly, every one I found addressed only one kind of performance (work/time). I can't find any benchmarking of something at least as important in today's massive-scale-computing environments, work/watt. A dirty little secret about JIT technologies like Java, is that they throw a lot more CPU resources at the problem, trying to get up to par with native C++ code. JITs use more memory, and periodically run background optimizer tasks. These overheads are somewhat offset in work/time performance, by extra optimizations which can be performed with more dynamic information. But it results in a hungrier appetite for watts. Another dirty little secret about Java vs C++ benchmarks is that they compare single-workloads. Try running 100 VMs, each with a Java and C++ benchmark in it and Java's hungrier appetite for resources (MHz, cache, RAM) will show. But of course, Java folks don't mention that.
>
> But let's say for the sake of (non-)argument, that Java can achieve a 1:1 work/time performance relative to C++, for a single program. If Java consumes 15% more power doing it, does it matter on a PC? Most people don't dare. Does it matter for small-scale server environments? Maybe not. Does it matter when you deploy Hadoop on a 10,000 node cluster, and the holistic inefficiency (multiple things running concurrently) goes to 30%? Ask the people who sign the checks for the power bill. Unfortunately, inefficiency scales really well.

No, Java does not consume 15% doing it, because there isn't just one implementation of Java compilers.

Most comercial JVMs do offer the capability of ahead of time native code compilation or JIT caches.

So when those 15% really matter, enterprises do shell out the money for such JVMs.

Oracle commercial JVM and the OpenJDK are just the reference implementation.

--
Paulo

On Monday, 26 January 2015 at 20:55:14 UTC, Wyatt wrote:
> On Monday, 26 January 2015 at 20:19:09 UTC, Laeeth Isharc wrote:
>>
>> Does Rust have the productivity of D?  And it doesn't have the maturity, as I understand it.
>
> This brings up something that's been bugging me.  D has a pitch for users of a lot of crappy languages, but what do we say when the competition isn't a total slouch?
>
> This exchange (names changed) is what started this train of thought:
> <chum> though i don't understand what the point of D is either because once you've already accepted a gc there are better languages you could use
> <chum> and if you refuse to accept one, then, well, you either have c++11 or you wait for rust to be usable
> <otherguy> chum: what is better than D once youre willing to have managed mem?
> <chum> it's functional, but the complaint all the gamedev folks have about fp langs is that their implementations are usually garbage collected and they can't accept gc pauses
> <chum> otherguy: f#, ocaml, haskell
>
> Particularly against F#, I'm not sure what to say (it's a pretty interesting language, even if it's kind of ugly to look at and CIL-ly).  Thoughts?
>
> -Wyatt

I don't know F#.  I know what you mean, but I don't think the competition to D consists of crappy languages - there are some very smart and creative people with large resources working on them (putting aside the question of the tone one should adopt in public towards peers).

This kind of categorical thinking is a mistake.  I am not certain, but it strikes me that outside of realtime, the kind of problems one may have with the GC (or with avoiding the use of the gC) in D are really quite different to, say, Java, whereas people lump everything together.  I have no experience with realtime applications, so can't comment there.

It's not for me to say, but D isn't a product like toothpaste where you are trying to elbow aside the competition, but one where it needs to be the best 'D' it can be, and communicate that well to people and make it easy for them to take advantage of what it has to offer.

" If Java consumes 15% more power doing it, does
>> it matter on a PC? Most people don't dare. Does it matter for small-scale server environments? Maybe not. Does it matter when you deploy Hadoop on a 10,000 node cluster, and the holistic inefficiency (multiple things running concurrently) goes to 30%? Ask the people who sign the checks for the power bill. Unfortunately, inefficiency scales really well.
>
> No, Java does not consume 15% doing it, because there isn't just one implementation of Java compilers.
>
> Most comercial JVMs do offer the capability of ahead of time native code compilation or JIT caches.
>
> So when those 15% really matter, enterprises do shell out the money for such JVMs.
>
> Oracle commercial JVM and the OpenJDK are just the reference implementation.

Thanks for the colour.  (For clarity, the content from the link wasn't by me, and I meant the general gist rather than the details).  How do commercial JVMs rate in terms of memory usage against thoughtful native (D) code implementations?  Is the basic point mistaken?

On Monday, 26 January 2015 at 22:12:24 UTC, Laeeth Isharc wrote:
> " If Java consumes 15% more power doing it, does
>>> it matter on a PC? Most people don't dare. Does it matter for small-scale server environments? Maybe not. Does it matter when you deploy Hadoop on a 10,000 node cluster, and the holistic inefficiency (multiple things running concurrently) goes to 30%? Ask the people who sign the checks for the power bill. Unfortunately, inefficiency scales really well.
>>
>> No, Java does not consume 15% doing it, because there isn't just one implementation of Java compilers.
>>
>> Most comercial JVMs do offer the capability of ahead of time native code compilation or JIT caches.
>>
>> So when those 15% really matter, enterprises do shell out the money for such JVMs.
>>
>> Oracle commercial JVM and the OpenJDK are just the reference implementation.
>
> Thanks for the colour.  (For clarity, the content from the link wasn't by me, and I meant the general gist rather than the details).  How do commercial JVMs rate in terms of memory usage against thoughtful native (D) code implementations?  Is the basic point mistaken?


So far I just dabbled in D, because our customers choose the platforms, not we.

However, these are the kind of tools you get to analyse performance in commercial JVMs,

http://www.oracle.com/technetwork/java/javaseproducts/mission-control/java-mission-control-1998576.html

http://www.oracle.com/technetwork/server-storage/solarisstudio/features/performance-analyzer-2292312.html

Just providing the examples from Oracle, other vendors have similar tools.

With them, you can drill down the whole JVM and interactions at the OS level and find performance bottlecks all the way down to generated Assembly code.


As for memory usage, Atego JVMs run in quite memory constrained devices.

Here is the tiniest of them, http://www.atego.com/products/atego-perc-ultra/

--
Paulo

On Monday, 26 January 2015 at 20:55:14 UTC, Wyatt wrote:
> On Monday, 26 January 2015 at 20:19:09 UTC, Laeeth Isharc wrote:
>>
>> Does Rust have the productivity of D?  And it doesn't have the maturity, as I understand it.
>
> This brings up something that's been bugging me.  D has a pitch for users of a lot of crappy languages, but what do we say when the competition isn't a total slouch?
>
> This exchange (names changed) is what started this train of thought:
> <chum> though i don't understand what the point of D is either because once you've already accepted a gc there are better languages you could use
> <chum> and if you refuse to accept one, then, well, you either have c++11 or you wait for rust to be usable
> <otherguy> chum: what is better than D once youre willing to have managed mem?
> <chum> it's functional, but the complaint all the gamedev folks have about fp langs is that their implementations are usually garbage collected and they can't accept gc pauses
> <chum> otherguy: f#, ocaml, haskell
>
> Particularly against F#, I'm not sure what to say (it's a pretty interesting language, even if it's kind of ugly to look at and CIL-ly).  Thoughts?
>
> -Wyatt

D GC collects far too often for games, I believe that the GC is more configurable in 2.067 in this regard. I currently have to disable the GC and run it manually. Single larger(ish) pause >>> many, many small pauses.

Yup, most people like to shit on Java, but quite frankly, the ecosystem is way ahead of what exists on most platform.

It is even fairly common to get a Java program up and running + tweaking of the JVM is less time and with better performance than what you would have in C++.

Obviously, given enough time and resource, you can still optimize the C++ version to death to the point it beat java, but, unless you are willing to spend an order of magnitude more for performances, java is a better choice than C++.