On Mon, Aug 05, 2013 at 07:30:36PM +0200, JS wrote:
> On Monday, 5 August 2013 at 13:59:24 UTC, jicman wrote:
> >
> >Greetings!
> >
> >I have this code,
> >
> >foreach (...)
> >{
> >
> >  if (std.string.tolower(fext[0]) == "doc" ||
> >    std.string.tolower(fext[0]) == "docx" ||
> >    std.string.tolower(fext[0]) == "xls" ||
> >    std.string.tolower(fext[0]) == "xlsx" ||
> >    std.string.tolower(fext[0]) == "ppt" ||
> >    std.string.tolower(fext[0]) == "pptx")
> >   continue;
> >}
> >
> >foreach (...)
> >{
> >  if (std.string.tolower(fext[0]) == "doc")
> >    continue;
> >  if (std.string.tolower(fext[0]) == "docx")
> >    continue;
> >  if (std.string.tolower(fext[0]) == "xls")
> >    continue;
> >  if (std.string.tolower(fext[0]) == "xlsx")
> >    continue;
> >  if (std.string.tolower(fext[0]) == "ppt")
> >    continue;
> >  if (std.string.tolower(fext[0]) == "pptx")
> >   continue;
> >  ...
> >  ...
> >}
> >
> >thanks.
> >
> >josé
> 
> 
> Both are slow, those trying to optimize out tolower are missing the boat. A good compiler should optimize that out.
> 
> The issue is that n compare are done in those cases with each successive compare getting slower and slower. if the match is doc then it happens in one compare in both cases. If it's pptx then it is 6. So if there are many pptx matches then the code will perform slower than for doc.
> 
> 
> This can be good or bad behavior depending on if you are guarantee that your order is representative of what actually happens(many doc's few pptx's).
> 
> To get O(1) behavior, you must use some sort of jump list. A hash table can provide such behavior. If there are many comparisons needed then at some point this will be, on average, faster.
[...]

There's no need to reinvent the wheel here. std.regex.ctRegex will create the equivalent of jump tables for you.

Besides, I doubt the OP's real performance bottleneck is in this code; sounds like the program is I/O bound. Compared to I/O, even repeatedly calling std.string.tolower repeatedly is just roundoff error in terms of total execution time. Using a profiler will reveal the *real* bottleneck. Based on past experience, I'm expecting that it will reveal that the bottleneck isn't anywhere we're predicting it is. :)


T

-- 
VI = Visual Irritation

On Monday, August 05, 2013 15:59:23 jicman wrote:
> Greetings!
> 
> I have this code,
> 
> foreach (...)
> {
> 
>    if (std.string.tolower(fext[0]) == "doc" ||
>      std.string.tolower(fext[0]) == "docx" ||
>      std.string.tolower(fext[0]) == "xls" ||
>      std.string.tolower(fext[0]) == "xlsx" ||
>      std.string.tolower(fext[0]) == "ppt" ||
>      std.string.tolower(fext[0]) == "pptx")
>     continue;
> }
> 
> foreach (...)
> {
>    if (std.string.tolower(fext[0]) == "doc")
>      continue;
>    if (std.string.tolower(fext[0]) == "docx")
>      continue;
>    if (std.string.tolower(fext[0]) == "xls")
>      continue;
>    if (std.string.tolower(fext[0]) == "xlsx")
>      continue;
>    if (std.string.tolower(fext[0]) == "ppt")
>      continue;
>    if (std.string.tolower(fext[0]) == "pptx")
>     continue;
>    ...
>    ...
> }

As others have pointed out, the two code snippets are semantically identical, and there's a decent chance that the compiler will generate identical code for both. It wouldn't surprise me if the first snippet resulted in more concise assembly code, but there shouldn't be any performance difference between the two. But the first snippet is cleaner, so between those two, you should use that.

Of course, as others have pointed out, it would be far more efficient to simply calculate std.string.tolower(fext[0]) once and re-use the result, which should be a very large performance gain here (assuming that the code around it doesn't dwarf it in cost). However, another possibility that I don't think anyone has pointed out is std.string.icmp, which will do case-insensitive comparison, meaning that you could do

if(fext[0].icmp("doc") == 0 || fext[0].icmp("docx") == 0 || ...)

and that might be more efficient as it avoids having to allocate a new string when fext[0] isn't all lowercase already. However, you would have to benchmark the code to be sure, since depending on the typical input, the code around this code, and how many times this code gets called, calling toLower once could be more efficient or using icmp could be more efficient, since toLower allocates whereas icmp doesn't, but icmp does more comparisons that a simple ==. But if you really care about efficiency, you should try both ways and see which is faster for your use case.

- Jonathan M Davis

On 8/5/13, H. S. Teoh <hsteoh@quickfur.ath.cx> wrote:
> If you really want optimal performance, use std.regex:

Yes and if you also want to bloat your executable to the point that other parts of the system start slowing down.

On Tue, Aug 06, 2013 at 02:32:11PM +0200, jicman wrote:
> On Tuesday, 6 August 2013 at 04:10:57 UTC, Andre Artus wrote:
[...]
> >What exactly are you trying to do with this? I get the impression that there is an attempt at "local optimization" when broader approach could lead to better results.
> >
> >For instance. Using the OS's facilities to filter (six requests, one each for "*.doc", "*.docx") could actually end up being a lot faster.
> >
> >If you could give more detail about what you are trying to achieve then it could be possible to get better results.
> 
> The files are in a network drive and doing a list foreach *.doc, *.docx, etc. will be more expensive than getting the list of all the files at once and then processing them accordingly.

In this case, the bottleneck most likely is in the network latency, so optimizing string comparisons won't get you very far. You should instead be focusing on how to improve the performance of network accesses.

What kind of processing are you doing with the filtered list of files? If it's a complicated operation, you might want to consider having a separate thread for fetching the list of files while work is being done on them in the main thread.


T

-- 
Computers shouldn't beep through the keyhole.