I have followed the recent discussions about D and I can see the usual pattern, to wit GC, Go (or whatever) is so much better, everyone blaming each other for not contributing, not being allowed to contribute blah.

First of all, I am in no position to criticize anyone who is contributing to the language. I don't contribute, because I don't have the time to do so. Indeed I have huge, massive respect for everyone who contributes to D. The only thing I do is to actually use the language and tell everyone about it. I have developed a screen reader plug in in D (using C libraries) that was ridiculously easy to integrate on Windows as a DLL. I used vibe.d to create a lightning fast online version of the screen reader. Believe me, D's supposed sluggishness as regards GC is not so important for most applications. I dare say 90% of all applications are fine with the current GC. I compiled both applications with dmd (testing phase) not with ldc or gdc and they are very fast.

Let's not forget that Go has millions and billions of dollars behind it and that it is inevitable that the whole internet will be full of zealots and professional posters who promote Go as "theeee best thing ever". People. Sheep. Meehhh.

Apart from the necessary discussions about language features / improvements we need to focus on the power of D. vibe.d is one example. I think the problem is that we don't bundle the various efforts that have been made independently well enough. Contribution to D is narrowly defined as "contributing to the library / core of the language". There has been mention of integrating vibe.d's fibers into the library. Even if it won't happen, we should set up an infrastructure that facilitates the use of the various, as of now independent, components and point users to it. I have to say that a lot of good things have escaped me simply because nobody told me about them. It's often by accident that I find out about a great library or framework in D.

Sometimes I have the feeling that we blow things out of proportion, because we walk right into the trap. The GC thing, although it is very important, is a good example. Let's not forget that zeolots and professional posters will always point out the flaws of D, and blow them out of proportion. "D doesn't have xyz, so it's shit!" Divide et impera (divide and rule).

Let's first make a list of things that have been achieved with D and that are on a par with or even bettar than in other languages (C, C++, C#, Go, Rust ...). Then, let's bundle the efforts that have been made independently. We will soon have a powerful and impressive framework. And let's not forget, a language (be it a natural or a computer language) only lives and thrives, if people use it.

My 2 cents. At your service.

On 07/11/2014 05:30 PM, Chris wrote:
(...)
> Believe me, D's supposed sluggishness as regards GC is
> not so important for most applications. I dare say 90% of all
> applications are fine with the current GC.
(...)

I agree with this. The bottlenecks i my applications are MySQL and Microsoft Office (Excel, Powerpoint, or even just plain COM). The same bottlenecks as I get when using C#. Of course, it depends a lot on what you do, but for my use (and yours, and probably many others), the GC performance is something you can probably safely ignore.

A little anecdote.. I once got a 20% speed increase in Python by
"moving" a variable instantiation outside a tight loop.
  i = 0
  # loop here
    i = something

rather than
  # loop here
    i = something

The compiler wasn't smart enough to do this.

On 07/11/2014 05:43 PM, simendsjo wrote:
> On 07/11/2014 05:30 PM, Chris wrote:
> (...)
>> Believe me, D's supposed sluggishness as regards GC is
>> not so important for most applications. I dare say 90% of all
>> applications are fine with the current GC.
> (...)
> 
(...)

Oh, and a little GC.disable()/GC.enable() goes a long way. Only had to
do this a couple of times though.

On Fri, 2014-07-11 at 15:30 +0000, Chris via Digitalmars-d wrote: […]
> Let's not forget that Go has millions and billions of dollars behind it and that it is inevitable that the whole internet will be full of zealots and professional posters who promote Go as "theeee best thing ever". People. Sheep. Meehhh.

(I think I detect unintended irony in this post :-)

Go, via goroutines, promotes CSP as an approach to application parallelism and is therefore a Good Thing™. Don't underestimate the power of single threaded processes communicating using channels and no shared memory. It is true that any language has zealots, look at Fortran, Java, Python, D, but a language should not be judged solely by its zealotry level. Well except for JavaScript (aka ECMAScript) of course.

[…]

-- 
Russel. ============================================================================= Dr Russel Winder      t: +44 20 7585 2200   voip: sip:russel.winder@ekiga.net 41 Buckmaster Road    m: +44 7770 465 077   xmpp: russel@winder.org.uk London SW11 1EN, UK   w: www.russel.org.uk  skype: russel_winder

On Fri, 2014-07-11 at 17:43 +0200, simendsjo via Digitalmars-d wrote: […]
> A little anecdote.. I once got a 20% speed increase in Python by
> "moving" a variable instantiation outside a tight loop.
>   i = 0
>   # loop here
>     i = something
> 
> rather than
>   # loop here
>     i = something

This is interesting. I can believe there is some performance benefit, but I am not sure I believe 20% improvement. If you can send me the code you were using, I would like to do some benchmarking on this.

> The compiler wasn't smart enough to do this.

The Python compiler cannot and will never be able to do any such thing. Indeed if it did any such thing, it would be an error since it significantly changes the semantics of the program. Thus not doing this is not the fault of the compiler.  The fact that you were able to do this and it appeared to give you the same results just means that the change in program semantics did not affect your computation. Which is good, but not something the compiler could determine.

-- 
Russel. ============================================================================= Dr Russel Winder      t: +44 20 7585 2200   voip: sip:russel.winder@ekiga.net 41 Buckmaster Road    m: +44 7770 465 077   xmpp: russel@winder.org.uk London SW11 1EN, UK   w: www.russel.org.uk  skype: russel_winder

On 07/11/2014 06:28 PM, Russel Winder via Digitalmars-d wrote:
> On Fri, 2014-07-11 at 17:43 +0200, simendsjo via Digitalmars-d wrote: […]
>> A little anecdote.. I once got a 20% speed increase in Python by
>> "moving" a variable instantiation outside a tight loop.
>>   i = 0
>>   # loop here
>>     i = something
>>
>> rather than
>>   # loop here
>>     i = something
> 
> This is interesting. I can believe there is some performance benefit, but I am not sure I believe 20% improvement. If you can send me the code you were using, I would like to do some benchmarking on this.

Yes, I was very perplexed when I was profiling and finally found the main offender. Unfortunately I don't have the code - it was a project done for a past employer back in 2006/2007 (Python 2.4 IIRC).

>> The compiler wasn't smart enough to do this.
> 
> The Python compiler cannot and will never be able to do any such thing. Indeed if it did any such thing, it would be an error since it significantly changes the semantics of the program. Thus not doing this is not the fault of the compiler.  The fact that you were able to do this and it appeared to give you the same results just means that the change in program semantics did not affect your computation. Which is good, but not something the compiler could determine.

I think of this as a fault in the compiler. It was quite obvious (to me) that nothing else relied on the value so the value didn't have to be created on each iteration.

On Friday, 11 July 2014 at 16:22:27 UTC, Russel Winder via Digitalmars-d wrote:
> On Fri, 2014-07-11 at 15:30 +0000, Chris via Digitalmars-d wrote:
> […]
>> Let's not forget that Go has millions and billions of dollars behind it and that it is inevitable that the whole internet will be full of zealots and professional posters who promote Go as "theeee best thing ever". People. Sheep. Meehhh.
>
> (I think I detect unintended irony in this post :-)

I get the point :-)

> Go, via goroutines, promotes CSP as an approach to application
> parallelism and is therefore a Good Thing™. Don't underestimate the
> power of single threaded processes communicating using channels and no
> shared memory. It is true that any language has zealots, look at
> Fortran, Java, Python, D, but a language should not be judged solely by
> its zealotry level. Well except for JavaScript (aka ECMAScript) of
> course.
>
> […]

I remember Java used to be "theeee" best thing ever. After years of using it, however, I found out how restricted the language was / is. Still, it's been a success, because people believed all the propaganda. What matters to me is not so much the odd fancy feature, it's how well the language performs in general purpose programming. Go was designed for servers and thus will always have one up on D or any other language at that matter. But could I use Go for what I have used D? Not so sure about that. Also, like Java Go is a closed thing. D isn't. Once I read about D that it shows what can be done "once you take a language out of the hands of a committee". Go, like Java, will finally end up in a cul de sac and will have a hard time trying to get out of it. Not because the language is inherently bad, because it's in the hand of a committee. Ideology kills a language. But it doesn't matter, because people will use Go or whatever anyway, will _have_ to use it.

What I'm taking issue with is that everybody focuses on the flaws of D (every language has flaws), which often gives the impression that it's an unfinished, stay-away business. It's not. D can be used, and I've used it, for production code. It's more mature than D or Rust and it is superior to other languages like Java (no OO-ideology for example). Mind you, D is a hindsight language, which makes it wiser. Does it have flaws? Yes. I come across them sometimes. Is there a language without flaws? If there is, tell me about it. Talking about hindsight, I've tried many different languages, I like D because of what it has to offer for general purpose programming, it compiles natively, interfaces with C at no cost at all, it has strong modelling power, features that users require are added. I may sound like a zealot (see "irony"), but I'm not. I'm very pragmatic, D is a good tool and, being community driven, there is a real chance of making it a fantastic tool. Individual features are not everything.

On Friday, 11 July 2014 at 16:54:40 UTC, Chris wrote:
> On Friday, 11 July 2014 at 16:22:27 UTC, Russel Winder via Digitalmars-d wrote:
>> On Fri, 2014-07-11 at 15:30 +0000, Chris via Digitalmars-d wrote:
>> […]
>>> Let's not forget that Go has millions and billions of dollars behind it and that it is inevitable that the whole internet will be full of zealots and professional posters who promote Go as "theeee best thing ever". People. Sheep. Meehhh.
>>
>> (I think I detect unintended irony in this post :-)
>
> I get the point :-)
>
>> Go, via goroutines, promotes CSP as an approach to application
>> parallelism and is therefore a Good Thing™. Don't underestimate the
>> power of single threaded processes communicating using channels and no
>> shared memory. It is true that any language has zealots, look at
>> Fortran, Java, Python, D, but a language should not be judged solely by
>> its zealotry level. Well except for JavaScript (aka ECMAScript) of
>> course.
>>
>> […]
>
> I remember Java used to be "theeee" best thing ever. After years of using it, however, I found out how restricted the language was / is. Still, it's been a success, because people believed all the propaganda. What matters to me is not so much the odd fancy feature, it's how well the language performs in general purpose programming. Go was designed for servers and thus will always have one up on D or any other language at that matter. But could I use Go for what I have used D? Not so sure about that. Also, like Java Go is a closed thing. D isn't. Once I read about D that it shows what can be done "once you take a language out of the hands of a committee". Go, like Java, will finally end up in a cul de sac and will have a hard time trying to get out of it. Not because the language is inherently bad, because it's in the hand of a committee. Ideology kills a language. But it doesn't matter, because people will use Go or whatever anyway, will _have_ to use it.
>
> What I'm taking issue with is that everybody focuses on the flaws of D (every language has flaws), which often gives the impression that it's an unfinished, stay-away business. It's not. D can be used, and I've used it, for production code. It's more mature than D or Rust and it is superior to other languages like Java (no OO-ideology for example). Mind you, D is a hindsight language, which makes it wiser. Does it have flaws? Yes. I come across them sometimes. Is there a language without flaws? If there is, tell me about it. Talking about hindsight, I've tried many different languages, I like D because of what it has to offer for general purpose programming, it compiles natively, interfaces with C at no cost at all, it has strong modelling power, features that users require are added. I may sound like a zealot (see "irony"), but I'm not. I'm very pragmatic, D is a good tool and, being community driven, there is a real chance of making it a fantastic tool. Individual features are not everything.

It should read It's more mature than _Go_ or Rust, of course.

On Fri, Jul 11, 2014 at 03:30:15PM +0000, Chris via Digitalmars-d wrote:
> I have followed the recent discussions about D and I can see the usual pattern, to wit GC, Go (or whatever) is so much better, everyone blaming each other for not contributing, not being allowed to contribute blah.

Well, this forum *is* for discussing ways of improving D, so it shouldn't be surprising that we constantly find things to nitpick about. :-) It doesn't mean at all that D is lousy or the community is bad, 'cos if it were so, we wouldn't even be here to begin with. We're here 'cos we care, and we complain 'cos we care enough to want things to improve.


> First of all, I am in no position to criticize anyone who is contributing to the language. I don't contribute, because I don't have the time to do so.  Indeed I have huge, massive respect for everyone who contributes to D. The only thing I do is to actually use the language and tell everyone about it.  I have developed a screen reader plug in in D (using C libraries) that was ridiculously easy to integrate on Windows as a DLL. I used vibe.d to create a lightning fast online version of the screen reader. Believe me, D's supposed sluggishness as regards GC is not so important for most applications. I dare say 90% of all applications are fine with the current GC. I compiled both applications with dmd (testing phase) not with ldc or gdc and they are very fast.

I agree. I'm still convinced that GC phobia is blown out of proportion -- I used to be in that camp, so I totally sympathize with where they're coming from -- but as you say, only a small percentage of applications actually need to squeeze every last cycle out of the CPU such that the GC actually starts to make a significant difference in performance. Most applications work just fine with the GC, and in fact, I'd argue that they work *better* with the GC, because manual memory management is *hard* (just look at how many security exploits are caused by memory management mistakes) and tedious (look at how often the same memory bugs are repeated over and over).  GC-supported code is cleaner to read, easier to write, and in many cases, the simpler design of the code reduces the likelihood of bugs and eliminates a whole class of bugs. Sure you pay for that by short pauses every now and then, but seriously, 90% of applications don't even *care* about such pauses.

For applications with slightly higher performance demands, gdc -O3 (or whatever the LDC equivalent is) generally improves performance by about 20% or so above dmd. In my own compute-intensive projects, I have consistently noted about a 20-30% performance improvement when compiling with gdc, compared to dmd. That's pretty significant, because GC pauses are generally nowhere near that percentage, so just by recompiling with gdc already eliminates the perceived GC performance issue for 95% of applications. Besides, avoiding frequent small allocations also reduces most of the workload of the GC, so you can still get pretty far without totally turning it off.

So it's really only the remaining 5% of applications that really, absolutely, *have* to go GC-less (or control it very tightly). They do happen to have supporters of the rather vocal kind, so we tend to hear from them a lot more, but that by no means is representative of the grand scheme of things as far as the GC is concerned!


[...]
> Let's first make a list of things that have been achieved with D and that are on a par with or even bettar than in other languages (C, C++, C#, Go, Rust ...).

I don't know C#, Go, or Rust, so I can't really say much on that front, but at least as far as C/C++ are concerned, D totally beats them flat in the following points IMO:

- Metaprogramming. Templates in C++ scarred many for life. Templates in
  D are actually a pleasure to use.

- CTFE. Coupled with metaprogramming, this is a total killer combination
  that I've yet to see another language beat.

- Slices. Finally, a systems-level language whose string support isn't
  crippled (C), maimed (C++), or otherwise handicapped (Java). And this
  extends to arrays in general. While there *are* other language with
  nice string/array manipulation support, D is the only one I know of
  that does it without sacrificing performance.

- Ranges. It can totally revolutionize the way you approach programming.
  And, with metaprogramming/CTFE, they can still perform as fast as
  non-range-based code. Total win!

- Extended meaning of purity: IMO it's a total stroke of genius to
  define "weak purity" that allows you to implement pure functions (in
  the Haskell sense) using mutating primitives (loops and assignments,
  etc.). While the current compilers don't really do that much with this
  presently, there is a lot of potential here that may turn this into a
  killer feature.

- Built-in unittests. Sounds trivial, but I can testify to its value in
  dramatically improving the quality of my code. I've worked with large
  C/C++ codebases, and most of them don't even bother with any kind of
  unit testing -- it's up to the programmer to test everything, and we
  just take his word for it -- and simply accept the countless stream of
  bugs that come thereafter as a fact of life. Of the rare few that
  actually do have tests, the tests are usually (1) outdated, (2)
  commented out 'cos nobody cares to update them, (3) ignored by the
  coders anyway 'cos they can't be bothered to switch to another
  language in another framework just to write tests that nobody will run
  while having their hands tied behind their back. D's built-in unittest
  blocks is a total game changer in this area, in spite of its
  simplicity (which some people have complained about).

   - Along these lines, static assert totally rawkz. It ensures, at
     *compile-time*, that assumptions in your code haven't been violated
     by a careless code change, forcing the person who made the change
     to fix it (rather than introducing a possibly subtle error that
     will only be uncovered months down the road on the customer's
     production site).

- The fastest regex library known on the planet (thanks to, guess what?
  metaprogramming and CTFE!). I'm a regex aficionado, and this is a
  total big deal in my book.

- Built-in Unicode support. Compiler-level support for Unicode is
  something C/C++ sorely lacks, and that immediately puts them in the
  "legacy" category. LibICU is a nightmare to use. D, however, lets you
  treat Unicode directly in the language. (Full Unicode compliance isn't
  quite there yet, but we're getting pretty close.) Modern languages
  like Java/C# also have built-in Unicode support, so D is at least on
  par with them. C/C++ is definitely behind in this category, though.

These are just language-level cool stuff. At a higher level, we also have:

- rdmd: run your D programs like scripts, yet with native compiled
  performance. Rawkage!

- Dustmite: a totally revolutionary tool IMO, that changes finding
  heisenbugs from an impossible game of chance to something that
  actually has hope of being fixed within reasonable amounts of time.

- vibe.d: I haven't used it myself, but from what I hear, it's extremely
  awesome.

I'm sure there are many other items that can be added, but this should be a good start. :)


T

-- 
If it breaks, you get to keep both pieces. -- Software disclaimer notice

On Friday, 11 July 2014 at 17:15:36 UTC, H. S. Teoh via Digitalmars-d wrote:
> On Fri, Jul 11, 2014 at 03:30:15PM +0000, Chris via Digitalmars-d wrote:
>> I have followed the recent discussions about D and I can see the usual
>> pattern, to wit GC, Go (or whatever) is so much better, everyone
>> blaming each other for not contributing, not being allowed to
>> contribute blah.
>
> Well, this forum *is* for discussing ways of improving D, so it
> shouldn't be surprising that we constantly find things to nitpick about.
> :-) It doesn't mean at all that D is lousy or the community is bad, 'cos
> if it were so, we wouldn't even be here to begin with. We're here 'cos
> we care, and we complain 'cos we care enough to want things to improve.
>
>
>> First of all, I am in no position to criticize anyone who is
>> contributing to the language. I don't contribute, because I don't have
>> the time to do so.  Indeed I have huge, massive respect for everyone
>> who contributes to D. The only thing I do is to actually use the
>> language and tell everyone about it.  I have developed a screen reader
>> plug in in D (using C libraries) that was ridiculously easy to
>> integrate on Windows as a DLL. I used vibe.d to create a lightning
>> fast online version of the screen reader. Believe me, D's supposed
>> sluggishness as regards GC is not so important for most applications.
>> I dare say 90% of all applications are fine with the current GC. I
>> compiled both applications with dmd (testing phase) not with ldc or
>> gdc and they are very fast.
>
> I agree. I'm still convinced that GC phobia is blown out of proportion
> -- I used to be in that camp, so I totally sympathize with where they're
> coming from -- but as you say, only a small percentage of applications
> actually need to squeeze every last cycle out of the CPU such that the
> GC actually starts to make a significant difference in performance. Most
> applications work just fine with the GC, and in fact, I'd argue that
> they work *better* with the GC, because manual memory management is
> *hard* (just look at how many security exploits are caused by memory
> management mistakes) and tedious (look at how often the same memory bugs
> are repeated over and over).  GC-supported code is cleaner to read,
> easier to write, and in many cases, the simpler design of the code
> reduces the likelihood of bugs and eliminates a whole class of bugs.
> Sure you pay for that by short pauses every now and then, but seriously,
> 90% of applications don't even *care* about such pauses.
>
> For applications with slightly higher performance demands, gdc -O3 (or
> whatever the LDC equivalent is) generally improves performance by about
> 20% or so above dmd. In my own compute-intensive projects, I have
> consistently noted about a 20-30% performance improvement when compiling
> with gdc, compared to dmd. That's pretty significant, because GC pauses
> are generally nowhere near that percentage, so just by recompiling with
> gdc already eliminates the perceived GC performance issue for 95% of
> applications. Besides, avoiding frequent small allocations also reduces
> most of the workload of the GC, so you can still get pretty far without
> totally turning it off.
>
> So it's really only the remaining 5% of applications that really,
> absolutely, *have* to go GC-less (or control it very tightly). They do
> happen to have supporters of the rather vocal kind, so we tend to hear
> from them a lot more, but that by no means is representative of the
> grand scheme of things as far as the GC is concerned!
>
>
> [...]
>> Let's first make a list of things that have been achieved with D and
>> that are on a par with or even bettar than in other languages (C, C++,
>> C#, Go, Rust ...).
>
> I don't know C#, Go, or Rust, so I can't really say much on that front,
> but at least as far as C/C++ are concerned, D totally beats them flat in
> the following points IMO:
>
> - Metaprogramming. Templates in C++ scarred many for life. Templates in
>   D are actually a pleasure to use.
>
> - CTFE. Coupled with metaprogramming, this is a total killer combination
>   that I've yet to see another language beat.
>
> - Slices. Finally, a systems-level language whose string support isn't
>   crippled (C), maimed (C++), or otherwise handicapped (Java). And this
>   extends to arrays in general. While there *are* other language with
>   nice string/array manipulation support, D is the only one I know of
>   that does it without sacrificing performance.
>
> - Ranges. It can totally revolutionize the way you approach programming.
>   And, with metaprogramming/CTFE, they can still perform as fast as
>   non-range-based code. Total win!
>
> - Extended meaning of purity: IMO it's a total stroke of genius to
>   define "weak purity" that allows you to implement pure functions (in
>   the Haskell sense) using mutating primitives (loops and assignments,
>   etc.). While the current compilers don't really do that much with this
>   presently, there is a lot of potential here that may turn this into a
>   killer feature.
>
> - Built-in unittests. Sounds trivial, but I can testify to its value in
>   dramatically improving the quality of my code. I've worked with large
>   C/C++ codebases, and most of them don't even bother with any kind of
>   unit testing -- it's up to the programmer to test everything, and we
>   just take his word for it -- and simply accept the countless stream of
>   bugs that come thereafter as a fact of life. Of the rare few that
>   actually do have tests, the tests are usually (1) outdated, (2)
>   commented out 'cos nobody cares to update them, (3) ignored by the
>   coders anyway 'cos they can't be bothered to switch to another
>   language in another framework just to write tests that nobody will run
>   while having their hands tied behind their back. D's built-in unittest
>   blocks is a total game changer in this area, in spite of its
>   simplicity (which some people have complained about).
>
>    - Along these lines, static assert totally rawkz. It ensures, at
>      *compile-time*, that assumptions in your code haven't been violated
>      by a careless code change, forcing the person who made the change
>      to fix it (rather than introducing a possibly subtle error that
>      will only be uncovered months down the road on the customer's
>      production site).
>
> - The fastest regex library known on the planet (thanks to, guess what?
>   metaprogramming and CTFE!). I'm a regex aficionado, and this is a
>   total big deal in my book.
>
> - Built-in Unicode support. Compiler-level support for Unicode is
>   something C/C++ sorely lacks, and that immediately puts them in the
>   "legacy" category. LibICU is a nightmare to use. D, however, lets you
>   treat Unicode directly in the language. (Full Unicode compliance isn't
>   quite there yet, but we're getting pretty close.) Modern languages
>   like Java/C# also have built-in Unicode support, so D is at least on
>   par with them. C/C++ is definitely behind in this category, though.
>
> These are just language-level cool stuff. At a higher level, we also
> have:
>
> - rdmd: run your D programs like scripts, yet with native compiled
>   performance. Rawkage!
>
> - Dustmite: a totally revolutionary tool IMO, that changes finding
>   heisenbugs from an impossible game of chance to something that
>   actually has hope of being fixed within reasonable amounts of time.
>
> - vibe.d: I haven't used it myself, but from what I hear, it's extremely
>   awesome.
>
> I'm sure there are many other items that can be added, but this should
> be a good start. :)
>
>
> T

Thanks. That's a nice list. This is what I was talking about, the experience with D, what you can achieve with it and how it compares with other languages. We need more of this. I have the feeling sometimes that to an outsider D might look like an eternally unfinished business. A nice playground for programmers, but not for production, which is absolutely not true. The GC issue is sometimes presented as "the language will stand or fall with this". As you noted and which is also my experience, the GC issue ain't that big. 90-95% of all applications can live with it.