I do think that versioning is the way to go if the goal is to expose only a standard, defined API. If I receive a patch where parts were clearly copied from another version, I simply reject it. If the effort is manual though I trust it's correct and wait for bug reports to say otherwise.

Regarding the test suite... I was working with the private one yesterday and it took me a while to figure out how to make it work. Platform-specific scripts is bad enough, but scripts that can only be run via the shell app that ships with DMD?  Really?  I wasn't sure how to override the settings it uses so I even had to edit the script to make it work. Might be fine for a personal test suite, but for a publicly available one it needs work. :-)

Sent from my iPhone

On Sep 2, 2010, at 11:40 PM, Walter Bright <walter at digitalmars.com> wrote:

> 
> 
> Brad Roberts wrote:
>> On 9/2/2010 3:00 PM, Walter Bright wrote:
>> 
>>> Brad Roberts wrote:
>>> 
>>>> The major wish I have is that walter would start using the public test suite rather than the private one.  I'm working on getting the public one to work on windows -- making good progress, it runs most of the tests now.
>>>> 
>>>> 
>>> It already works fine on Windows, and I disagree that it needed reengineering so
>>> that it wouldn't.
>>> 
>> 
>> If you honestly think what I did was engineer it so that it wouldn't work on windows, we need to talk.
> 
> No, I don't mean that at all, and apologize for implying that. I meant the current setup is simple and effective and works on all the supported platforms.
> 
> I should add that Andrei agrees with you about this and disagrees with me strongly. So keep that in mind when you read the following.
> 
>>  It needed reenginering for other reasons (probably
>> incomplete, but this is off the top of my head and quickly written):
>> 
>> 1) to use a single driver for all platforms rather than one for each that were
>> never kept in sync -- an old argument you keep loosing
>> 
> 
> Except that there is much customization from one platform to the next. Switches are different, commands are different, paths are different, extensions are different, generated files are different, etc.
> 
> Not sure I keep losing the argument. For example, Phobos1 used to have std.c.stdio loaded with version declarations for the different _iobuf's for each platform. The trouble was, when one updated one platform one tended to break the others. Even worse, adding a new platform meant someone would just cut & paste the info from some other platform, and assume it worked. This added a bunch of work for me as I had to go through and vet every declaration. Things got better when I separated out each platform into its own import. These problems resurfaced with druntime, which tries to do a lot using version declarations instead of separate platform files.
> 
> So yes, on the one hand you can add files to all platforms simultaneously. On the other, there's constant breakage of the platforms the updater didn't test on.
> 
>> 2) to run tests in parallel -- your attempts at doing that in your custom shell
>> engine didn't work out terribly well
>> 
> 
> I didn't spend that much time on the multithreading, either <g>. One problem was that the text output of the different threads got interleaved, a problem that multithreaded make also has. The other problem is that optlink must always run on core 0, meaning that multithreading the test suite on Windows does not actually speed it up. Until I can fix that problem with optlink, it wasn't worth the effort to continue working on parallelizing the tests.
> 
> I've also got things set up now so that with one command I can fire off 4 windows, and have each window ssh'ing into a different machine and running the build & test suite (Windows, Linux, OSX and FreeBSD). Those obviously all run in parallel (!). I can watch them all run on one screen.
> 
>> 3) to separate test driver output from test output -- spewing all that output to
>> the screen is a mess.  Tests should have a single line of output: name of test
>> and pass or fail
>> 
> 
> I guess I never had any trouble noticing when one failed and which one it was.
> 
>> 4) to make it easy to run subsets of the test suite, specifically easy to run
>> the one that's failing.
>> 
> 
> I'm not sure what the problem is with typing:
> 
>  dmc test23
> 
> 
>> 5) to run _all_ of the test suite -- note the 4 bugs I filed based on the
>> failure tests that hadn't been run or would have caught a couple regressions.
>> 
> 
> That failing was mine in not keeping the suite updated, not a fault in the methodology.
> 
>> Also of potential interest.. the biggest problem I've had with the windows
>> support is switching from / to \ based on os since optlink can't handle it.  It
>> looked like everything else was happy -- though dmd might have had some issues,
>> I didn't try to isolate the issue between the two.  The o vs obj and nothing vs
>> .exe were trivial.  Support for \ is a pain in the ass.. having to \\ in places
>> -- right now something somewhere is retaining both \\'s rather than treating it
>> as one escaped.  I got frustrated with that and your post so I set aside the
>> project for the evening.
>> 
> 
> The \ issue is always a problem. I'm always running into windows software, including Microsoft software, that doesn't recognize /.
> 
>> I'll finish it tomorrow or this weekend, maybe.
>> 
>> 
>> 
> _______________________________________________
> dmd-internals mailing list
> dmd-internals at puremagic.com
> http://lists.puremagic.com/mailman/listinfo/dmd-internals

On Fri, 3 Sep 2010, Sean Kelly wrote:

> Regarding the test suite... I was working with the private one yesterday and it took me a while to figure out how to make it work. Platform-specific scripts is bad enough, but scripts that can only be run via the shell app that ships with DMD?  Really?  I wasn't sure how to override the settings it uses so I even had to edit the script to make it work. Might be fine for a personal test suite, but for a publicly available one it needs work. :-)

Did you try the public one?

I'd love usability feedback.

On Sep 3, 2010, at 4:54 PM, Brad Roberts wrote:

> On Fri, 3 Sep 2010, Sean Kelly wrote:
> 
>> Regarding the test suite... I was working with the private one yesterday and it took me a while to figure out how to make it work. Platform-specific scripts is bad enough, but scripts that can only be run via the shell app that ships with DMD?  Really?  I wasn't sure how to override the settings it uses so I even had to edit the script to make it work. Might be fine for a personal test suite, but for a publicly available one it needs work. :-)
> 
> Did you try the public one?
> 
> I'd love usability feedback.

Not yet.  I'll give it a try next week.

On 9/3/2010 3:27 PM, Sean Kelly wrote:
> I do think that versioning is the way to go if the goal is to expose only a standard, defined API. If I receive a patch where parts were clearly copied from another version, I simply reject it. If the effort is manual though I trust it's correct and wait for bug reports to say otherwise.

I must have missed some context somewhere.. to what was this referring?

Brad Roberts wrote:
> Also of potential interest.. the biggest problem I've had with the windows support is switching from / to \ based on os since optlink can't handle it.  It looked like everything else was happy -- though dmd might have had some issues, I didn't try to isolate the issue between the two.  The o vs obj and nothing vs .exe were trivial.  Support for \ is a pain in the ass.. having to \\ in places -- right now something somewhere is retaining both \\'s rather than treating it as one escaped.  I got frustrated with that and your post so I set aside the project for the evening.
>
> I'll finish it tomorrow or this weekend, maybe.
>
> 
I've also tried to make the test suite run under windows (after several other tries, MSys seems to be the best shot). I patched some of the scripts before I realized that it is way easier to just fix the file name handling of dmd and optlink. With this simple patch:

Index: link.c ===================================================================
--- link.c    (revision 657)
+++ link.c    (working copy)
@@ -42,23 +42,23 @@
 {
     /* Loop and see if we need to quote
      */
+    bool needsQuote = false;
     for (size_t i = 0; i < len; i++)
     {   char c = filename[i];

-        if (isalnum(c) || c == '_')
-            continue;
+        if (!isalnum(c) && c != '_')
+            needsQuote = true;
+#ifdef _WIN32 // keep optlink happy
+        if (c == '/')
+            filename[i] = '\\';
+#endif
+    }

-        /* Need to quote
-         */
+    if(needsQuote)
         buf->writeByte('"');
-        buf->write(filename, len);
+    buf->write(filename, len);
+    if(needsQuote)
         buf->writeByte('"');
-        return;
-    }
-
-    /* No quoting necessary
-     */
-    buf->write(filename, len);
 }

 void writeFilename(OutBuffer *buf, char *filename)
@@ -114,7 +114,7 @@

     cmdbuf.writeByte(',');
     if (global.params.mapfile)
-        cmdbuf.writestring(global.params.mapfile);
+        writeFilename(&cmdbuf, global.params.mapfile);
     else if (global.params.run)
         cmdbuf.writestring("nul");
     cmdbuf.writeByte(',');

--->8--------------------

any file name passed from dmd to optlink will have '/' translated to '\' (this was only done for object files so far). I think this patch will remove a lot of confusion for the normal use of dmd, too.

Reversing the behaviour for the object files with this patch

Index: mars.c ===================================================================
--- mars.c    (revision 657)
+++ mars.c    (working copy)
@@ -935,11 +935,11 @@
         p = (char *) files.data[i];

 #if _WIN32
-        // Convert / to \ so linker will work
+        // Convert \ to / to be consistent with unix output,
translation back for linker will be done later
         for (int i = 0; p[i]; i++)
         {
-            if (p[i] == '/')
-                p[i] = '\\';
+            if (p[i] == '\\')
+                p[i] = '/';
         }
 #endif


will even make the output files consistent with the unix versions (except newlines). This latter patch might break other tools relying on output containing back slashes (JSON, dependencies, etc), though.

Rainer

I think Walter was advocating the use of separate script files, and drew a parallel with C headers.

Sent from my iPhone

On Sep 3, 2010, at 10:29 PM, Brad Roberts <braddr at puremagic.com> wrote:

> On 9/3/2010 3:27 PM, Sean Kelly wrote:
>> I do think that versioning is the way to go if the goal is to expose only a standard, defined API. If I receive a patch where parts were clearly copied from another version, I simply reject it. If the effort is manual though I trust it's correct and wait for bug reports to say otherwise.
> 
> I must have missed some context somewhere.. to what was this referring?
> _______________________________________________
> dmd-internals mailing list
> dmd-internals at puremagic.com
> http://lists.puremagic.com/mailman/listinfo/dmd-internals

Rainer Schuetze wrote:
>
> I've also tried to make the test suite run under windows (after several other tries, MSys seems to be the best shot). I patched some of the scripts before I realized that it is way easier to just fix the file name handling of dmd and optlink. With this simple patch:
>

I generally try to avoid things like this. They always seem to cause trouble one way or another, and then one is stuck with it because too much has grown to depend on the hack.

Walter Bright wrote:
>
>
> Rainer Schuetze wrote:
>>
>> I've also tried to make the test suite run under windows (after several other tries, MSys seems to be the best shot). I patched some of the scripts before I realized that it is way easier to just fix the file name handling of dmd and optlink. With this simple patch:
>>
>
> I generally try to avoid things like this. They always seem to cause trouble one way or another, and then one is stuck with it because too much has grown to depend on the hack.
The trouble is happening now, with dmd (correctly) accepting file names with forward slashes, but only converting the object file names to containing backward slashes for optlink. With the first patch that problem should go away without side effects.

I agree the second patch (using forward slashes for source references) can have a bit more dependencies. But the current output might need some clarification anyway. E.g. filenames in dependency files and JSON files have duplicate back slashes on windows. According to the reference files in the test suite, JSON output on linux contains file names with "\/" in them.

On 9/4/2010 1:19 PM, Rainer Schuetze wrote:
> Walter Bright wrote:
>>
>>
>> Rainer Schuetze wrote:
>>>
>>> I've also tried to make the test suite run under windows (after several other tries, MSys seems to be the best shot). I patched some of the scripts before I realized that it is way easier to just fix the file name handling of dmd and optlink. With this simple patch:
>>>
>>
>> I generally try to avoid things like this. They always seem to cause trouble one way or another, and then one is stuck with it because too much has grown to depend on the hack.
> The trouble is happening now, with dmd (correctly) accepting file names with forward slashes, but only converting the object file names to containing backward slashes for optlink. With the first patch that problem should go away without side effects.
> 
> I agree the second patch (using forward slashes for source references) can have a bit more dependencies. But the current output might need some clarification anyway. E.g. filenames in dependency files and JSON files have duplicate back slashes on windows. According to the reference files in the test suite, JSON output on linux contains file names with "\/" in them.

I have the test suite generally working on windows under cygwin without patching dmd (though, I actually think that's a really clever idea and am a little surprised that I didn't think of it too).

The test script itself swaps over to \ for the dmd command line, but uses / everywhere else.  As you discovered, some tests output paths, specifically the cov and profile related tests.  I changed the post-script's for those to exclude the line that contains the path.

For those of you who regularly develop on windows, is running it under cygwin something that you object to enough to not do it?  I suspect Walter's answer is yes.  How about the rest?

The main reason I haven't checked it in yet is that when I synced up everything
to do a last full test, I hit several test failures that have crept in recently.
 I filed regression bugs for them.  I also haven't taken my changes back to
linux to make sure I didn't break anything there.

Regression bugs:
  http://d.puremagic.com/issues/show_bug.cgi?id=4795
  http://d.puremagic.com/issues/show_bug.cgi?id=4797

There was another, but it's already been addressed.

I'm going to work on the first one there today.  It shouldn't be too hard, but I haven't looked at the new demangling code yet.

The demangle output is formatted a bit differently, hopefully that's all it is.

Sent from my iPhone

On Sep 4, 2010, at 1:38 PM, Brad Roberts <braddr at puremagic.com> wrote:

> On 9/4/2010 1:19 PM, Rainer Schuetze wrote:
>> Walter Bright wrote:
>>> 
>>> 
>>> Rainer Schuetze wrote:
>>>> 
>>>> I've also tried to make the test suite run under windows (after several other tries, MSys seems to be the best shot). I patched some of the scripts before I realized that it is way easier to just fix the file name handling of dmd and optlink. With this simple patch:
>>>> 
>>> 
>>> I generally try to avoid things like this. They always seem to cause trouble one way or another, and then one is stuck with it because too much has grown to depend on the hack.
>> The trouble is happening now, with dmd (correctly) accepting file names with forward slashes, but only converting the object file names to containing backward slashes for optlink. With the first patch that problem should go away without side effects.
>> 
>> I agree the second patch (using forward slashes for source references) can have a bit more dependencies. But the current output might need some clarification anyway. E.g. filenames in dependency files and JSON files have duplicate back slashes on windows. According to the reference files in the test suite, JSON output on linux contains file names with "\/" in them.
> 
> I have the test suite generally working on windows under cygwin without patching dmd (though, I actually think that's a really clever idea and am a little surprised that I didn't think of it too).
> 
> The test script itself swaps over to \ for the dmd command line, but uses / everywhere else.  As you discovered, some tests output paths, specifically the cov and profile related tests.  I changed the post-script's for those to exclude the line that contains the path.
> 
> For those of you who regularly develop on windows, is running it under cygwin something that you object to enough to not do it?  I suspect Walter's answer is yes.  How about the rest?
> 
> The main reason I haven't checked it in yet is that when I synced up everything to do a last full test, I hit several test failures that have crept in recently. I filed regression bugs for them.  I also haven't taken my changes back to linux to make sure I didn't break anything there.
> 
> Regression bugs:
>  http://d.puremagic.com/issues/show_bug.cgi?id=4795
>  http://d.puremagic.com/issues/show_bug.cgi?id=4797
> 
> There was another, but it's already been addressed.
> 
> I'm going to work on the first one there today.  It shouldn't be too hard, but I haven't looked at the new demangling code yet.
> 
> _______________________________________________
> dmd-internals mailing list
> dmd-internals at puremagic.com
> http://lists.puremagic.com/mailman/listinfo/dmd-internals