Hasan Aljudy wrote:
> 
> I just assumed that Window$ haters will try to promote linux at the end of the day, since it's usually the case! I don't think there is anything wrong with that, nor anything to take offence for.

OS/2 was darn nice.  If there had been more apps for it than VisualAge C++ I might never have uninstalled it ;-)


Sean

"Sean Kelly" <sean@f4.ca> wrote in message news:drqpmf$1cj5$1@digitaldaemon.com...
> This sounds like a hardware problem to me.  The freezes I've had in the past have been in games and are typically attributable to the video or audio drivers.  I've also had spontaneous resets, though not with my latest PC setup.

The last time I had big problems with XP randomly failing, freezing, mysterious messages, replacing the memory card fixed it.

Sean Kelly wrote:
> Don Clugston wrote:
> 
>>
>> I'm with you. I find XP to be completely unstable. XP-SP2, anyway (The original XP Pro was OK). It seems less stable than even Win95 (admittedly, the hardware now is much more sophisticated).
>> Particularly annoying about SP2 is the fact that Ctrl-Alt-Delete no longer kills programs instantly.
>>
>> I never get blue screens, though. It just freezes.
> 
> 
> This sounds like a hardware problem to me.  The freezes I've had in the past have been in games and are typically attributable to the video or audio drivers.  I've also had spontaneous resets, though not with my latest PC setup.

No, these seem related to networking. The way it handles wireless networks and VPNs is particularly bad. I think that most of the problems lie in Internet Explorer; the timeouts in that program are just insane. In fact, the shell is still using the same ridiculous algorithm they used in Windows 3.0 in the days of floppy disks, where it insists on confirming that each drive still exists before doing anything (instead, that should be done in a background thread; you only need to wait for the disk to respond if you are actually using that disk).

When I say "it freezes", it's usually not a complete system hang. It does actually recover -- eventually. Oddly, hitting ctrl-alt-delete to bring up the task manager usually seems to reset the timeouts, the 'dead' programs revive about 5 seconds after exiting task manager (even when I haven't done anything).

"Matthew" <matthew@stlsoft.com> wrote in message news:drlua3$r7j$1@digitaldaemon.com...
> Most of my scripts use recls/Ruby, which is as simple as:
>
>
> require 'recls'
>
> fs = Recls::FileSearch::new("h:/stlsoft", "*.hpp|*.h", Recls::FILES | Recls::RECURSIVE)
>
> fs.each \
> { |fe|
>
>    lines = IO::readlines(fe.path)
>
>    lines.each_with_index \
>    { |line, index|
>            line.chomp!
>            if line =~ / blah blah blah /
>                lines[index] = blah2 blah2 blah2
>            end
>    }
>
>    if bChanged
>        f = File::new(fe.path, "w")
>        lines.each { |line| f << line << "\n" }
>        f.close
>        puts "Processed #{fe.path}"
>    end
> }
>
> This is the basic form of my scripts, with which I can effect changes to hundreds/thousands of source files in one go. It's really really useful.
>
> (I'd love to be doing the same in D, but ...)

Ruby, 18 lines.
D, 16 lines, including declarations and bug fix for bChanged.
-----------------------------------------
import std.file;
import std.stdio;
import std.string;

void main()
{
    char[][] fs = listdir("h:/stlsoft", "*.h") ~ listdir("h:/stlsoft",
"*.hpp");

    foreach (char[] fe; fs)
    { bool bChanged;
      char[] buffer = cast(char[]) read(fe);
      char[][] lines = splitlines(buffer);

      foreach (inout char[] line; lines)
     {
        if ("blah blah blah" ~~ chomp(line))
        { line = "blah2 blah2 blah2" ~ newline;
           bChanged = true;    // missing in Ruby version
        }
     }

     if (bChanged)
    {
       write(fe, join(lines, null));
       writefln("Processed %s", fe);
     }
    }
}

"Walter Bright" <newshound@digitalmars.com> wrote in message news:dsmlv7$mh2$1@digitaldaemon.com...
>
> "Matthew" <matthew@stlsoft.com> wrote in message news:drlua3$r7j$1@digitaldaemon.com...
> > Most of my scripts use recls/Ruby, which is as simple as:
> >
> >
> > require 'recls'
> >
> > fs = Recls::FileSearch::new("h:/stlsoft", "*.hpp|*.h", Recls::FILES |
> > Recls::RECURSIVE)
> >
> > fs.each \
> > { |fe|
> >
> >    lines = IO::readlines(fe.path)
> >
> >    lines.each_with_index \
> >    { |line, index|
> >            line.chomp!
> >            if line =~ / blah blah blah /
> >                lines[index] = blah2 blah2 blah2
> >            end
> >    }
> >
> >    if bChanged
> >        f = File::new(fe.path, "w")
> >        lines.each { |line| f << line << "\n" }
> >        f.close
> >        puts "Processed #{fe.path}"
> >    end
> > }
> >
> > This is the basic form of my scripts, with which I can effect changes to hundreds/thousands of source files in one go. It's really really useful.
> >
> > (I'd love to be doing the same in D, but ...)
>
> Ruby, 18 lines.
> D, 16 lines, including declarations and bug fix for bChanged.
> -----------------------------------------
> import std.file;
> import std.stdio;
> import std.string;
>
> void main()
> {
>     char[][] fs = listdir("h:/stlsoft", "*.h") ~ listdir("h:/stlsoft",
> "*.hpp");
>
>     foreach (char[] fe; fs)
>     { bool bChanged;
>       char[] buffer = cast(char[]) read(fe);
>       char[][] lines = splitlines(buffer);
>
>       foreach (inout char[] line; lines)
>      {
>         if ("blah blah blah" ~~ chomp(line))
>         { line = "blah2 blah2 blah2" ~ newline;
>            bChanged = true;    // missing in Ruby version
>         }
>      }
>
>      if (bChanged)
>     {
>        write(fe, join(lines, null));
>        writefln("Processed %s", fe);
>      }
>     }
> }

And the recursion?

I don't want to add to the ill-humour that's been exchanged of late, but seriously, wh-ha-at the?!?

I scratch my head - considering the bChanged "correction" (!) and the reuse
of "blah blah blah" - and wonder why you've bothered. You've spent some
effort here in attempting to prove that D doesn't have a problem which it
most certainly does have, and you will only convince those few people who
are not yet sufficiently knowledgeable/experienced in D. You might have only
spent five minutes, but now you've eaten 5 of mine, and to counter this
you're going to have to spend a lot more than those first 5 to get a correct
equivalent to my example. (The main purpose of recls is, er, recursion, you
know.
<g>)

Surely all that effort would have been far better spent upgrading the standard library with the latest version of recls. That way, when someone on c.l.c.m gives you a similar criticism as my example did, you'll be able to trot out a genuine rejoinder, rather than shooting yourself in the foot.


Matthew

P.S. What's your example doing to do if you find a directory with a H or HPP extension?

"Matthew" <matthew@hat.stlsoft.dot.org> wrote in message news:dsn1fp$14hc$1@digitaldaemon.com...
> And the recursion?

std.file.listdir does recursion.

> I don't want to add to the ill-humour that's been exchanged of late, but seriously, wh-ha-at the?!?
>
> I scratch my head - considering the bChanged "correction" (!) and the
> reuse
> of "blah blah blah" - and wonder why you've bothered. You've spent some
> effort here in attempting to prove that D doesn't have a problem which it
> most certainly does have,
> and you will only convince those few people who
> are not yet sufficiently knowledgeable/experienced in D. You might have
> only
> spent five minutes, but now you've eaten 5 of mine, and to counter this
> you're going to have to spend a lot more than those first 5 to get a
> correct
> equivalent to my example. (The main purpose of recls is, er, recursion,
> you
> know.
> <g>)

I think you'll find that the D version does the same thing.

> Surely all that effort would have been far better spent upgrading the
> standard library with the latest version of recls. That way, when someone
> on
> c.l.c.m gives you a similar criticism as my example did, you'll be able to
> trot out a genuine rejoinder, rather than shooting yourself in the foot.
>
> Matthew
>
> P.S. What's your example doing to do if you find a directory with a H or
> HPP
> extension?

Directories don't get added to the list of files, though they get searched. Only files are affected by the pattern match. There isn't much to listdir's implementation:

char[][] listdir(char[] pathname, char[] pattern)
{   char[][] result;

    bool callback(DirEntry* de)
    {
        if (de.isdir)
            listdir(de.name, &callback);
        else
        {   if (std.path.fnmatch(de.name, pattern))
                result ~= de.name;
        }
        return true; // continue
    }

    listdir(pathname, &callback);
    return result;
}

That version uses the simple fnmatch which does * and ?, but one could as easilly use std.regexp and use full regular expressions on the filenames.

"Walter Bright" <newshound@digitalmars.com> wrote in message news:dsn3ca$165g$1@digitaldaemon.com...
>
> "Matthew" <matthew@hat.stlsoft.dot.org> wrote in message news:dsn1fp$14hc$1@digitaldaemon.com...
> > And the recursion?
>
> std.file.listdir does recursion.

Truly? Well, I didn't know that. It's been a while. I stand corrected.

Of course, that's a ghastly proposition. Read on for why this is so.

> Directories don't get added to the list of files, though they get
searched.
> Only files are affected by the pattern match. There isn't much to
listdir's
> implementation:
>
> char[][] listdir(char[] pathname, char[] pattern)
> {   char[][] result;
>
>     bool callback(DirEntry* de)
>     {
>         if (de.isdir)
>             listdir(de.name, &callback);
>         else
>         {   if (std.path.fnmatch(de.name, pattern))
>                 result ~= de.name;
>         }
>         return true; // continue
>     }
>
>     listdir(pathname, &callback);
>     return result;
> }
>
> That version uses the simple fnmatch which does * and ?, but one could as easilly use std.regexp and use full regular expressions on the filenames.

If this is an example of the quality of D's libraries, then it paints s a sad picture. Perhaps someone needed a shoulder buddy when it was being designed? ;-)

If a user wants to use multi-part pattern matching, your client code will have to execute N calls to listdir(). That's simply bad design. First, it makes client code unnecessarily complicated - your taking my example as literal in this respect to have a two-part pattern was quite disengenuous - as you're going to have to do the pattern splitting and loop on listdir() when the utility passes in a pattern (as mine do, of course, in the real world; I was simplifying to make a point).

Second, and at least as importantly (but less obviously), it fails to reflect the fact that directory enumeration is an I/O bound operation: Code using listdir() is going to have poor performance; very poor in fact. And I *know* this because I've done comparative studies that quantified this for recls a couple of years ago that tested out the difference between handling patterns at the level of the search or the level of the individual directory. (It's also going to wait a _long_ time on a big search doing apparently nothing, though I note listdir appears to support some kind of callbacks, so I assume that that can be avoided.) Further, it seems to have been conceived absent any thought to the complexity implications involved in doing really large searches - continually resizing very large arrays is not a good idea. This will significantly add to the poor performance.

(Anyone that cares to verify this can simply write a couple of simple progs with recls to search multi-patterns. (You'll have to write it in C/C++/.NET/Ruby/Python/Java, though, as you can't use D, since the version of std.recls is too old.) Write one to split in the client code and issue N single-part patterns to recls. Write the second to pass multi-part patterns into a single call to recls. Time the two. (You'll need to run each a few times to get a fair comparison, since the OS will cache things up in response.)

Looking through the docs for listdir() it seems like it tries to do some of the things that recls does, albeit far less well. But it does have fewer lines of code, and that's all that counts, surely?. Except it's wrongheaded. Putting aside the performance issues, those fewer lines of code in the library - to be sure, a boon in and of itself - will be *far* outweighed by the increased number of lines of client code. And since number of errors are proportional to numbers of lines of code, moderated by how many "eyes" / executions have covered that code, again D loses out to Ruby (or any other language that uses recls or any similar recursive file-system searching that supports multi-part pattern matching).

I know you'll continue to argue the toss, and you'll doubtless fashion a way to appear to win this argument in this forum, but D continues to stack up poorly against Ruby/X/Y/Z in reality. I haven't tested it, but I wouldn't be the least bit surprised that a Ruby script using recls/Ruby would outperform an equivalent program using D.

... <some eight hours later ...> ...

I decided that facts would be more powerful than conjecture. Enumerating source files - something I commonly do - using both a D program using listdir and a Ruby script using recls/Ruby, I get the following results for one of my main work directories H:/Dev. The search pattern is "*.h;*.rb;*.hpp;*.java;*.d;*.py;*.vbs;*.c;*.cpp;*.cxx;*.idl;*.odl;*.rc". The programs are timed using ptime (http://synesis.com.au/systools.html), and run directly one after the other, to obviate any OS caching (i.e. you can ignore the first couple, but the rest should be pretty representative). Neither do any output, since that'd skew the times, they merely count the number of entries obtained (and were verified to detect the same numbers, of course).

D:      time: elapsed: 7,484ms, kernel: 3,593ms, user: 3140ms
Ruby: time: elapsed: 3,781ms, kernel: 3,031ms, user: 500ms
D:      time: elapsed: 8,906ms, kernel: 3,187ms, user: 3,671ms
Ruby: time: elapsed: 3,687ms, kernel: 2,734ms, user: 703ms
D:      time: elapsed: 8,953ms, kernel: 3,640ms, user: 3296ms
Ruby: time: elapsed: 3,718ms, kernel: 2,890ms, user: 593ms
D:      time: elapsed: 7,500ms, kernel: 4,093ms, user: 2,734ms
Ruby: time: elapsed: 3,750ms, kernel: 3,031ms, user: 531ms
D:      time: elapsed: 8,828ms, kernel: 3,906ms, user: 2,937ms
Ruby: time: elapsed: 3,687ms, kernel: 3,093ms, user: 390ms

and the following for my entire work drive H:/. Ruby had to come first here, as I kept losing my nerve/patience and pressing Ctrl-C on the D version, so I ran Ruby ver first to get a ballpark for how long I might have to wait for the D version. I was misled. Again, you can ignore the first couple of runs, as they are somewhat affected by the OS scrabbling to eat a last bit of toast and some juice before it has to run out to work.

Ruby: time: elapsed: 475,609ms, kernel: 44,640ms, user: 7,218ms
D:      time: elapsed: 4,224,625ms, kernel: 92,421ms, user: 48,625ms
Ruby: time: elapsed: 80,359ms, kernel: 41,781ms, user: 6,484ms
D:      time: elapsed: 2,540,203ms, kernel: 79,515ms, user: 47,031ms
Ruby: time: elapsed: 83,656ms, kernel: 40,484ms, user: 6,984ms
D:      time: elapsed: 2,733,218ms, kernel: 80,078ms, user: 47,203ms
Ruby: time: elapsed: 88,062ms, kernel: 40,578ms, user: 6,812ms
D:      time: elapsed: 2,630,031ms, kernel: 76,890ms, user: 47,609ms
Ruby: time: elapsed: 86,809ms, kernel: 42,000ms, user: 7,093ms
D:      time: elapsed: 2,864,390ms, kernel: 81,046ms, user: 47,765ms

(btw, during the execution of the full-drive search, the ruby.exe process stayed at a constant 4MB. listdir_test.exe didn't behave quite so well .... when I stopped watching it was at 65MB, and VMM was crying before it completed but I didn't get to peak at it as I had wandered off for a sleep.)

For reference, I ran one of my C++ programs, whereis, with the same
patterns. It is not (yet) written with recls, but rather uses "manual"
recursion. (It ran at a constant 1MB.) It gave:

    "time: elapsed: 275,328ms, kernel: 40,921ms, user: 8,984ms"
    "time: elapsed: 257,343ms, kernel: 41,609ms, user: 8,171ms"
    "time: elapsed: 271,656ms, kernel: 41,546ms, user: 8,875ms"

Somewhat unexpectedly, it appears that Ruby+recls/Ruby is faster than a plain-old-C++ program. <g> (Note that whereis and Ruby+recls/Ruby both spend similar amounts of time in the kernel, whereas D clearly does something quite different.)

The "H:/" result clearly shows what a horrible idea it is to return en bloc elements from what is an "Input Iterator" collection (if you'll pardon my C++ parlance). (btw, these and many other issues are covered in my new book, Extended STL, vol 1. Just thought I'd drop that one in, even though I know most D-people couldn't care less about the crusty old has been C++. <g>)

Before I went back and did these tests, I'd written a hang-dog final paragraph:

    "As for std.recls, maybe the best thing to do would be simply to drop it
from Phobos forthwith, since you're clearly not in the slightest bit
interested in updating it, and it's doing no one any good being 2-3 years
behind the game. That way anyone that does want to use it can do so without
all the conflicts and bother that are currently incurred."

However, in light of the concrete results shown above, I now feel quite justified in urging that you again treat recls as a first-class Phobos library and update it forthwith, since it seems to do its job rather well despite it's unmentionable implementation in C/C++. Further, I strongly suggest you toss out listdir() as the ill-conceived and badly performing mess that I've just demonstrated it to be. (You know, there's a reason there hasn't been much progress in writing recursive search libraries before recls: it's not actually as easy as it sounds.)

<anachronistic-aside>I can't help but smile when I recall the criticisms made of newer languages that make programming too easy, as they often lead to very poor performance despite having had features deliberately designed in to aid performance. </anachronistic-aside>

Matthew

Excuse me if I missed this part, but why dont we just update std.recls?

"Matthew" <matthew@hat.stlsoft.dot.org> wrote in message news:dso4rh$2d2u$1@digitaldaemon.com...
> I decided that facts would be more powerful than conjecture. Enumerating
> source files - something I commonly do - using both a D program using
> listdir and a Ruby script using recls/Ruby, I get the following results
> for
> one of my main work directories H:/Dev. The search pattern is
> "*.h;*.rb;*.hpp;*.java;*.d;*.py;*.vbs;*.c;*.cpp;*.cxx;*.idl;*.odl;*.rc".

Please post the code of the benchmark you're using, as I can't comment on it until I see exactly what it's doing.

"AgentOrange" <AgentOrange_member@pathlink.com> wrote in message news:dso6f5$2ee0$1@digitaldaemon.com...
> Excuse me if I missed this part, but why dont we just update std.recls?

I have no idea. As I've explained on the bugs ng, Walter and I had an agreement that recls 1.6 (released March 05) would be the point at which it was updated, but before we got to that point I'd mentioned that I planned (and still plan) to implement a recls2 (which is more of a recursion engine, supporting recursion of a whole lot of other things including file-system, FTP, XML docs, Win32 registry, etc. etc.), and that it would likely be primarily in C (which would mean it would be smaller). Walter then changed his mind and said he'd rather wait for recls2.

Alas, real life has got in the way - my book, and paying the bills, and other libs - so recls2 has not yet got past the core engine stage. My book is almost ready for review, after which recls2 will be one of the things I'll be doing. But it's still likely to be some months out.

I'm afraid this situation is representative a real problem that many engineers have, especially when it comes to issues of languages and/or libraries.

From Walter's perspective recls is flawed because it is large - in object it's not so bad, e.g. 20KB with VC++, but in source it's quite a lot of files ~200KB IIRC. However, the point is apparently missed that it's got a Good Design. It scales to any conceivable search scope, a la my recent tests, and is mapped to other many languages with ease.

Conversely, D's syntax and apparent power means that listdir() is very succinct. In source size it's probably ~1% that of recls, if that. However, this succinctness is specious, misleading the author(s) of listdir() into a Bad Design. Simply put, if listdir() wants to be recursive, it cannot return en bloc. If it wants to return en bloc, it cannot be recursive. One can equivocate as much as one wishes, but that equation cannot be avoided. (This is the reason why I shot off half-cocked about Walter's counter example not supporting recursion. It never entered my head that anyone would combine these two features.)

It's far easier to re-implement a fat implementation of a good design, than it is to fix a bad design. Since D's many months away, and I've demonstrated just how bad listdir() is, I can't see any sensible justification why std.recls is not updated now. I'm very confident that recls2 will be ready before D 1.0, but even were it not, the current version of std.recls isn't going to put the rest of D to shame.

In the spirit of kicking someone while they're down, I would again, as I have many times over the years in which I've been involved with D, observe that being a good compiler designer/writer doesn't make you a good library designer/writer, and being a good library designer/writer doesn't make you a good application designer/writer, and so on and so forth. What D needs is a combination of talents in all such areas. Since there is hardly an overall lack of all such in this NG, perhaps we might consider organising for the common purpose of making D good and ready, rather than attempting to shoot each other's feet off?

Matthew