On Thursday, May 16, 2013 12:35:11 Dicebot wrote:
> Want to bring into discussion people that are not on Google+. Samuel recently has posted there some simple experiments with bioinformatics and bad performance of Phobos-based snippet has surprised me.
> 
> I did explore issue a bit and reported results in a blog post (snippets are really small and simple) : http://dicebot.blogspot.com/2013/05/short-performance-tuning-story.html
> 
> One open question remains though - can D/Phobos do better here? Can some changes be done to Phobos functions in question to improve performance or creating bioinformatics-specialized library is only practical solution?

1. In general, if you want to operate on ASCII, and you want your code to be fast, use immutable(ubyte)[], not immutable(char)[]. Obviously, that's not gonig to work in this case, because the function is in std.string, but maybe that's a reason for some std.string functions to have ubyte overloads which are ASCII-specific.

2. We actually discussed removing all of the pattern stuff completely and replacing it with regexes (which is why countchars doesn't follow Phobos' naming scheme correctly - I left the pattern-using functions alone). However, that requires that someone who is appropriately familiar with regexes go and implement new versions of all of these functions which use std.regex. It should definitely be done, but no one has taken the time to do so yet.

3. While some functions in Phobos are well-optimized, there are plenty of them which aren't. They do the job, but no one has taken the time to optimize their implementations. This should be fixed, but again, it requires that someone spends the time to do the optimizations, and while that has been done for some functions, it definitely hasn't been done for all. And if python is faster than D at something, odds are that either the code in question is poorly written or that whatever Phobos functions it's using haven't been properly optimized yet.

- Jonathan M Davis

On 5/16/2013 12:15 PM, Jonathan M Davis wrote:
> And if python is faster than
> D at something, odds are that either the code in question is poorly written or
> that whatever Phobos functions it's using haven't been properly optimized yet.

We should also be aware that while Python code itself is slow, its library functions are heavily optimized C code. So, if the benchmark consists of calling a Python library function, it'll run as fast as any optimized C code.

On 05/16/2013 01:46 PM, Nick Sabalausky wrote:
> On Thu, 16 May 2013 09:03:36 -0500
> 1100110 <0b1100110@gmail.com> wrote:
> 
>>> May I also recommend my tool "avgtime" to make simple benchmarks, instead of "time" (you can see an ascii histogram as the output):
>>>
>>>      https://github.com/jmcabo/avgtime/tree/
>>>
>>> For example:
>>>
>>> $ avgtime -r10 -h -q  ls
>>> ------------------------
>>> Total time (ms): 27.413
>>> Repetitions    : 10
>>> Sample mode    : 2.6 (4 ocurrences)
>>> Median time    : 2.6695
>>> Avg time       : 2.7413
>>> Std dev.       : 0.260515
>>> Minimum        : 2.557
>>> Maximum        : 3.505
>>> 95% conf.int.  : [2.2307, 3.2519]  e = 0.510599
>>> 99% conf.int.  : [2.07026, 3.41234]  e = 0.671041
>>> EstimatedAvg95%: [2.57983, 2.90277]  e = 0.161466
>>> EstimatedAvg99%: [2.5291, 2.9535]  e = 0.212202
>>> Histogram      :
>>>     msecs: count  normalized bar
>>>       2.5:     2  ####################
>>>       2.6:     4  ########################################
>>>       2.7:     3  ##############################
>>>       3.5:     1  ##########
>>>
>>> --jm
>>>
>>
>> Thank you for self-promotion, I miss that tool.
>>
>>
> 
> Indeed. I had totally forgotten about that, and yet it *should* be the first thing I think of when I think "timing a program". IMO, that should be a standard tool in any unixy installation.
> 
> 

+1

That's worth creating a package for.

On Thursday, 16 May 2013 at 22:58:42 UTC, 1100110 wrote:
> On 05/16/2013 01:46 PM, Nick Sabalausky wrote:
>> On Thu, 16 May 2013 09:03:36 -0500
>> 1100110 <0b1100110@gmail.com> wrote:
>> 
>>>> May I also recommend my tool "avgtime" to make simple benchmarks,
>>>> instead of "time" (you can see an ascii histogram as the output):
>>>>
>>>>      https://github.com/jmcabo/avgtime/tree/
>>>>
>>>> For example:
>>>>
>>>> $ avgtime -r10 -h -q  ls
>>>> ------------------------
>>>> Total time (ms): 27.413
>>>> Repetitions    : 10
>>>> Sample mode    : 2.6 (4 ocurrences)
>>>> Median time    : 2.6695
>>>> Avg time       : 2.7413
>>>> Std dev.       : 0.260515
>>>> Minimum        : 2.557
>>>> Maximum        : 3.505
>>>> 95% conf.int.  : [2.2307, 3.2519]  e = 0.510599
>>>> 99% conf.int.  : [2.07026, 3.41234]  e = 0.671041
>>>> EstimatedAvg95%: [2.57983, 2.90277]  e = 0.161466
>>>> EstimatedAvg99%: [2.5291, 2.9535]  e = 0.212202
>>>> Histogram      :
>>>>     msecs: count  normalized bar
>>>>       2.5:     2  ####################
>>>>       2.6:     4  ########################################
>>>>       2.7:     3  ##############################
>>>>       3.5:     1  ##########
>>>>
>>>> --jm
>>>>
>>>
>>> Thank you for self-promotion, I miss that tool.
>>>
>>>
>> 
>> Indeed. I had totally forgotten about that, and yet it *should* be the
>> first thing I think of when I think "timing a program". IMO, that
>> should be a standard tool in any unixy installation.
>> 
>> 
>
> +1
>
> That's worth creating a package for.

Thanks!
I currently don't have much time to make a ubuntu/arch/etc. package, between work and the university. I might in the future.

Keep in mind that it also works in windows. Though the process creation overhead is bigger in windows than in linux (because of the OS). Also, you can open the source up and easily modify it to measure your times directly, inside your programs.

--jm

On Fri, 17 May 2013 03:01:38 +0200
"Juan Manuel Cabo" <juanmanuel.cabo@gmail.com> wrote:

> On Thursday, 16 May 2013 at 22:58:42 UTC, 1100110 wrote:
> > On 05/16/2013 01:46 PM, Nick Sabalausky wrote:
> >> 
> >> Indeed. I had totally forgotten about that, and yet it
> >> *should* be the
> >> first thing I think of when I think "timing a program". IMO,
> >> that
> >> should be a standard tool in any unixy installation.
> >> 
[...]
> 
> Keep in mind that it also works in windows. Though the process creation overhead is bigger in windows than in linux (because of the OS). Also, you can open the source up and easily modify it to measure your times directly, inside your programs.

Yea, I almost said "should be a standard tool in any OS installation", but there's a *lot* of things that should be a standard part of any Windows box (bash, grep, a pre-Vista GUI...) and yet never will be ;)

On 2013-05-16 21:54, Walter Bright wrote:

> We should also be aware that while Python code itself is slow, its
> library functions are heavily optimized C code. So, if the benchmark
> consists of calling a Python library function, it'll run as fast as any
> optimized C code.

But someone using Python won't care about that. Most of them will think they just use Python and have no idea there's optimized C code under the hood.

-- 
/Jacob Carlborg

On Thursday, 16 May 2013 at 11:37:14 UTC, bearophile wrote:
> Dicebot:
> In the first of his posts I don't see -noboundscheck used, and it compares different algorithms from C++ (a switch) and D (two nested ifs, that are not optimal).
>
> From my experience if you have some care you are able to write D code for LDC that is about as fast as equivalent as C code, or better.

Sure. I am not interested in benchmarks. What made me curious was "what made this code so slow if you _don't_ have some care".

On Thursday, 16 May 2013 at 13:48:45 UTC, Juan Manuel Cabo wrote:
> On Thursday, 16 May 2013 at 10:35:12 UTC, Dicebot wrote:
>> Want to bring into discussion people that are not on Google+. Samuel recently has posted there some simple experiments with bioinformatics and bad performance of Phobos-based snippet has surprised me.
>>
>> I did explore issue a bit and reported results in a blog post (snippets are really small and simple) : http://dicebot.blogspot.com/2013/05/short-performance-tuning-story.html
>>
>> One open question remains though - can D/Phobos do better here? Can some changes be done to Phobos functions in question to improve performance or creating bioinformatics-specialized library is only practical solution?
>
> I bet the problem is in readln. Currently, File.byLine() and readln() are extremely slow, because they call fgetc() one char at a time.

Both manual and naive phobos version use same readln approach, but former is more than 10x faster. It was my first guess too, but comparing to snippets have shown that this is not the issue this time.

On Thursday, 16 May 2013 at 13:52:01 UTC, Juan Manuel Cabo wrote:
> ...

Thanks for the tool, it is a good one. But I was not doing benchmarks this time, only cared about 2x difference at least, so "time" was enough :)

On Thursday, 16 May 2013 at 14:23:22 UTC, nazriel wrote:
> Very nice blog post.
>
> Something similar should go into D wiki database so it won't get lost in "In 80s we had..." topics.
>
> For sure there is a space for improvements in Phobos but such articles are good start to prevent wave of "D is slow and sucks" and force people to rethink if they are using right tools (functions in this case ie UTF8 aware vs plain ASCII ones) for their job.

Thank you, I am glad at least someone have noticed it is not a call for a benchmarking contest :) Yes, my interest was exactly in case when newbie comes and tries to write some trivial code. If it behaves too slow, that abstract guy won't benchmark or investigate stuff, he will just say "D sucks" and move to the next language.

It is more of an informational issue, than Phobos one.