Some of you may remember Jon Degenhardt's talk from one of the Silicon Valley D meetups, where he described the performance improvements he saw when he rewrote some of eBay's command line tools in D. He has now put the effort into crafting a blog post on the same topic, where he takes D version of a command-line tool written in Python and incrementally improves its performance.

The blog:
https://dlang.org/blog/2017/05/24/faster-command-line-tools-in-d/

Reddit:
https://www.reddit.com/r/programming/comments/6d25mg/faster_command_line_tools_in_d/

On Wednesday, 24 May 2017 at 13:39:57 UTC, Mike Parker wrote:
> Some of you may remember Jon Degenhardt's talk from one of the Silicon Valley D meetups, where he described the performance improvements he saw when he rewrote some of eBay's command line tools in D. He has now put the effort into crafting a blog post on the same topic, where he takes D version of a command-line tool written in Python and incrementally improves its performance.
>
> The blog:
> https://dlang.org/blog/2017/05/24/faster-command-line-tools-in-d/
>
> Reddit:
> https://www.reddit.com/r/programming/comments/6d25mg/faster_command_line_tools_in_d/

A bit off topic but I really like that we still get quality content such as this post on this blog. Sustained quality is hard job and I thank everyone involved for that.

On Wednesday, 24 May 2017 at 17:36:29 UTC, cym13 wrote:
> On Wednesday, 24 May 2017 at 13:39:57 UTC, Mike Parker wrote:
> [...snip...]
>
> A bit off topic but I really like that we still get quality content such as this post on this blog. Sustained quality is hard job and I thank everyone involved for that.

The complement to the community is well deserved, thank you for including this post in the company. In this case, the post benefited from some really excellent review feedback and Mike made the publication side really easy.

--Jon

It's now #4 on the front page of Hacker News:

https://news.ycombinator.com/news

On Wednesday, 24 May 2017 at 21:34:08 UTC, Walter Bright wrote:
> It's now #4 on the front page of Hacker News:
>
> https://news.ycombinator.com/news

The comments on HN are useless though, everybody went for the "D versus Python" thing and seem to complain that it's doing a D/Python benchmark while only talking about D optimization...even though optimizing D is the whole point of the article. In the same way they rant against the fact that many iterations on the D script are shown while it is obviously to give different tricks while being clear on what trick gives what.

I am disappointed because there are so many good things to say about this, so many good questions or remarks to make when not familiar with the language, and yet all we get is "Meh, this benchmark shows nothing of D's speed against Python".

On Wednesday, 24 May 2017 at 21:46:10 UTC, cym13 wrote:
> On Wednesday, 24 May 2017 at 21:34:08 UTC, Walter Bright wrote:
>> It's now #4 on the front page of Hacker News:
>>
>> https://news.ycombinator.com/news
>
> The comments on HN are useless though, everybody went for the "D versus Python" thing and seem to complain that it's doing a D/Python benchmark while only talking about D optimization...even though optimizing D is the whole point of the article. In the same way they rant against the fact that many iterations on the D script are shown while it is obviously to give different tricks while being clear on what trick gives what.
>
> I am disappointed because there are so many good things to say about this, so many good questions or remarks to make when not familiar with the language, and yet all we get is "Meh, this benchmark shows nothing of D's speed against Python".

Its not easy writing an article that doesn't draw some form of criticism. FWIW, the reason I gave a Python example is because it is very commonly used for this type of problem and the language is well suited to it. A second reason is that I've seen several posts where someone has tried to rewrite a Python program like this in D, start with `split`, and wonder how to make it faster. My hope is that this will clarify how to achieve this.

Another goal of the article was to describe how performance in the TSV Utilities had been achieved. The article is not about the TSV Utilities, but discussing both the benchmark results and how they had been achieved would be a very long article.

--Jon

On 5/24/2017 3:56 PM, Jon Degenhardt wrote:
> Its not easy writing an article that doesn't draw some form of criticism. FWIW, the reason I gave a Python example is because it is very commonly used for this type of problem and the language is well suited to it. A second reason is that I've seen several posts where someone has tried to rewrite a Python program like this in D, start with `split`, and wonder how to make it faster. My hope is that this will clarify how to achieve this.
> 
> Another goal of the article was to describe how performance in the TSV Utilities had been achieved. The article is not about the TSV Utilities, but discussing both the benchmark results and how they had been achieved would be a very long article.

Any time one writes an article comparing speed between languages X and Y, someone gets their ox gored and will bitterly complain about how unfair the article is (though I noticed that none of the complainers wrote a faster Python version). Even if you tried to optimize the Python program, you'll be inevitably accused of deliberately not doing it right.

The nadir of this for me was when I compared Digital Mars C++ code with DMD. Both share the same optimizer and back end, yet I was accused of "sabotaging" my own C++ compiler in order to make D look better !! Me, I just don't do public comparison benchmarking anymore. It's a waste of time arguing with people about it.

I thought you wrote a fine article, and the criticism about the Python code was unwarranted (especially since nobody suggested better code), because the article was about optimizing D code, not optimizing Python.

On Wednesday, 24 May 2017 at 21:46:10 UTC, cym13 wrote:
> I am disappointed because there are so many good things to say about this, so many good questions or remarks to make when not familiar with the language, and yet all we get is "Meh, this benchmark shows nothing of D's speed against Python".

Wouldn't be the first time https://news.ycombinator.com/item?id=10828450

On Thursday, 25 May 2017 at 05:17:29 UTC, Walter Bright wrote:
> Any time one writes an article comparing speed between languages X and Y, someone gets their ox gored and will bitterly complain about how unfair the article is (though I noticed that none of the complainers wrote a faster Python version). Even if you tried to optimize the Python program, you'll be inevitably accused of deliberately not doing it right.
>
> The nadir of this for me was when I compared Digital Mars C++ code with DMD. Both share the same optimizer and back end, yet I was accused of "sabotaging" my own C++ compiler in order to make D look better !! Me, I just don't do public comparison benchmarking anymore. It's a waste of time arguing with people about it.
>
> I thought you wrote a fine article, and the criticism about the Python code was unwarranted (especially since nobody suggested better code), because the article was about optimizing D code, not optimizing Python.

Thanks Walter, I appreciate your comments. And correct, as multiple people noted, a speed comparison with other languages not at all a goal of the article.

The real intent was to tell a story of how several of D's features play together to enable optimizations like this, without having to write low-level code or step outside the core language features and standard library.

--Jon

On Thursday, 25 May 2017 at 06:22:28 UTC, Jon Degenhardt wrote:
> Thanks Walter, I appreciate your comments. And correct, as multiple people noted, a speed comparison with other languages not at all a goal of the article.
>
> The real intent was to tell a story of how several of D's features play together to enable optimizations like this, without having to write low-level code or step outside the core language features and standard library.

Maybe as a more casual observer the article did feel more like Python vs D. I have not yet read the ycombinator comments, just from my personal observation after reading the article.

My thinking was:

- Python its PyPy is surprising fast.

- Surprised that D was slower in version 1.

- Kind of surprised again that it took so many versions to figure out the best approach.

- Also wondering why one needed std.algorithm splitter, when you expect string split to be the fasted. Even the fact that you need to import std.array to split a string simply felt  strange.

- So much effort for relative little gain ( after v2 splitter ). The time spend on finding a faster solution is in business sense not worth it. But not finding a faster way is simply wasting performance, just on this simple function.

- Started to wonder if Python its PyPy is so optimized that without any effort, your even faster then D. What other D idiomatic functions are slow?

I am not criticizing your article Jon, just mentioning how i felt when reading it yesterday. It felt like the solution was overly complex to find and required too much deep D knowledge. Going to read the ycombinator comments now.


Off-topic:

Yesterday i was struggling with split but for a whole different reason. Take in account that i am new at D.

Needed to split a string. Simple right? Search Google for "split string dlang". Get on the https://dlang.org/phobos/std_string.html page.

After seeing the splitLines and start experimenting with it. Half a hour later i realize that the wrong function was used and needed to import std.array split function.

Call it a issue with the documentation or my own stupidity. But the fact that Split was only listed as a imported function, in this mass of text, totally send me on the wrong direction.

As stated above, i expected split to be part of the std.string, because i am manipulating a string, not that i needed to import std.array what is the end result.

I simply find the documentation confusing with the wall of text. When i search for string split, you expect to arrive on the string.split page. Not only that, the split example are using split as a separate keyword, when i was looking for variable.split().

Veteran D programmers are probably going to laughing at me for this but one does feel a bit salty after that.