Thread overview
Weird OSX issue
Apr 24, 2015
Jacob Carlborg
Apr 25, 2015
Dan Olson
May 12, 2015
Chris
Apr 27, 2015
Kagamin
April 24, 2015
OK, so I think I found a bug, but I have no idea how to "reproduce" it. It seems to be dependent on environment.

Here is an annotated (using # for comments) session to show you the weirdness. All versions are 2.067, and I did use dmd -v to make sure rogue dmd.conf or library files are not playing any role, and that the files I compare below are being used. Please help figure this out :)

# This shows where dmd runs from when I use the command 'dmd'.
# this was installed using the dmg from dlang.org for 2.067
Stevens-MacBook-Pro:teststatic steves$ type dmd
dmd is hashed (/usr/bin/dmd)

# inside ~/Downloads/dmd2 is an unzipped copy of dmd 2.067. Note
# the binaries and libraries are identical (no output from cmp)
Stevens-MacBook-Pro:teststatic steves$ cmp /usr/bin/dmd ~/Downloads/dmd2/osx/bin/dmd
Stevens-MacBook-Pro:teststatic steves$ cmp /usr/share/dmd/lib/libphobos2.a ~/Downloads/dmd2/osx/lib/libphobos2.a

# Now, checking concurrency.d text, also identical
Stevens-MacBook-Pro:teststatic steves$ diff /usr/share/dmd/src/phobos/std/concurrency.d ~/Downloads/dmd2/src/phobos/std/concurrency.d

# and finally checking core.thread text, also identical
Stevens-MacBook-Pro:teststatic steves$ diff /usr/share/dmd/src/druntime/import/core/thread.d ~/Downloads/dmd2/src/druntime/import/core/thread.d

# here is the program being compiled (I was testing something for a
# forum post)
Stevens-MacBook-Pro:teststatic steves$ cat mod1.d
module mod1;
import std.concurrency;
import std.stdio;

__gshared static int foo;

void main()
{
    auto tid = spawn((Tid o){foo = 5; o.send(true);}, thisTid);
    receiveOnly!bool();
    writeln(foo);
}

# fails with /usr/bin/dmd
Stevens-MacBook-Pro:teststatic steves$ dmd mod1.d
Undefined symbols for architecture x86_64:
  "_D4core6thread6Thread5startMFZv", referenced from:

_D3std11concurrency61__T6_spawnTPFS3std11concurrency3TidZvTS3std11concurrency3TidZ6_spawnFbPFS3std11concurrency3TidZvS3std11concurrency3TidZS3std11concurrency3Tid in mod1.o
ld: symbol(s) not found for architecture x86_64
clang: error: linker command failed with exit code 1 (use -v to see invocation)
--- errorlevel 1

# so let's take a look at the symbol in mod1.o, see how it's
# specified. Looks like it was the same one identified as
# not being present
Stevens-MacBook-Pro:teststatic steves$ nm mod1.o | grep core6thread6Thread5
                 U _D4core6thread6Thread5startMFZv

# works with identical compiler installed at ~/Donwloads/dmd2
Stevens-MacBook-Pro:teststatic steves$ ~/Downloads/dmd2/osx/bin/dmd mod1.d

# Same symbol? Nope! Note the difference seems to be in the parameter
# mangling.
Stevens-MacBook-Pro:teststatic steves$ nm mod1.o | grep core6thread6Thread5
                 U _D4core6thread6Thread5startMFNbZC4core6thread6Thread

So am I going crazy? Or is dmd doing things differently depending on where its environment is? Any compiler gurus out there understand why the symbol is different?

I don't want to file a bug with this, because it seems dependent on installation location, would possibly not be reproducible.

-Steve
April 24, 2015
On 2015-04-24 20:37, Steven Schveighoffer wrote:

> So am I going crazy? Or is dmd doing things differently depending on
> where its environment is? Any compiler gurus out there understand why
> the symbol is different?
>
> I don't want to file a bug with this, because it seems dependent on
> installation location, would possibly not be reproducible.

I can't reproduce this with DMD from DVM (compiler is installed in the user home directory).

-- 
/Jacob Carlborg
April 25, 2015
Jacob Carlborg <doob@me.com> writes:

> On 2015-04-24 20:37, Steven Schveighoffer wrote:
>
>> So am I going crazy? Or is dmd doing things differently depending on where its environment is? Any compiler gurus out there understand why the symbol is different?
>>
>> I don't want to file a bug with this, because it seems dependent on installation location, would possibly not be reproducible.
>
> I can't reproduce this with DMD from DVM (compiler is installed in the
> user home directory).

I have lots of version laying around and they all worked fine on the posted code.

But maybe a clue here...

$ ~/dmd2.066.0/osx/bin/dmd mod1.d
$ nm mod1.o | grep start
                 U _D4core6thread6Thread5startMFZv
$ dmd mod1.d
$ nm mod1.o | grep start
                 U _D4core6thread6Thread5startMFNbZC4core6thread6Thread

--- a/src/core/thread.d
+++ b/src/core/thread.d
@@ -587,7 +587,7 @@ class Thread
      * Throws:
      *  ThreadException if the thread fails to start.
      */
-    final Thread start()
+    final Thread start() nothrow
     in
     {
         assert( !next && !prev );

I wonder if dmd -v will show where its picking up stuff.
April 27, 2015
On Friday, 24 April 2015 at 18:37:40 UTC, Steven Schveighoffer wrote:
> _D4core6thread6Thread5startMFZv
> _D4core6thread6Thread5startMFNbZC4core6thread6Thread

Maybe it will be better understandable if you demangle these symbols?
April 27, 2015
On 4/25/15 3:07 AM, Dan Olson wrote:
> Jacob Carlborg <doob@me.com> writes:
>
>> On 2015-04-24 20:37, Steven Schveighoffer wrote:
>>
>>> So am I going crazy? Or is dmd doing things differently depending on
>>> where its environment is? Any compiler gurus out there understand why
>>> the symbol is different?
>>>
>>> I don't want to file a bug with this, because it seems dependent on
>>> installation location, would possibly not be reproducible.
>>
>> I can't reproduce this with DMD from DVM (compiler is installed in the
>> user home directory).
>
> I have lots of version laying around and they all worked fine on the
> posted code.
>
> But maybe a clue here...
>
> $ ~/dmd2.066.0/osx/bin/dmd mod1.d
> $ nm mod1.o | grep start
>                   U _D4core6thread6Thread5startMFZv
> $ dmd mod1.d
> $ nm mod1.o | grep start
>                   U _D4core6thread6Thread5startMFNbZC4core6thread6Thread
>
> --- a/src/core/thread.d
> +++ b/src/core/thread.d
> @@ -587,7 +587,7 @@ class Thread
>        * Throws:
>        *  ThreadException if the thread fails to start.
>        */
> -    final Thread start()
> +    final Thread start() nothrow
>       in
>       {
>           assert( !next && !prev );
>
> I wonder if dmd -v will show where its picking up stuff.
>

Thank you. It is something I missed. Lo and behold:

dmd -v mod1.d:

...

import    core.thread	(/usr/share/dmd/src/druntime/import/core/thread.di)

...

~/Downloads/dmd2/osx/bin/dmd -v mod1.d:

...

import    core.thread (/Users/steves/Downloads/dmd2/osx/bin/../../src/druntime/import/core/thread.d)

...

Hm... thread.di vs thread.d (and I didn't notice before because all the other imports were .d, I just glanced over that detail).

And so, let's see here:

ls -l /usr/share/dmd/src/druntime/import/core/thread.*

-rw-rw-r--   1 steves  staff  157781 Mar 24 10:44 thread.d
-rw-rw-r--+  1 steves  staff   31382 Feb 24  2014 thread.di

Hey, looky there :)

So, looking at time machine (love that feature), I found that when I installed 2.066, back in October, we had switched to using thread.d instead of thread.di. But the installer did not erase the old thread.di.

BTW, found this: https://github.com/D-Programming-Language/druntime/pull/865

So I think the issue here is that the pkg installer on OSX does not clean up the target directory if it already exists (or at least purposely remove thread.di). Will look into fixing that. At least now, it works properly, thanks (did a rm -rf /usr/share/dmd and reinstall).

-Steve

May 12, 2015
On 4/27/15 9:54 AM, Steven Schveighoffer wrote:

> So I think the issue here is that the pkg installer on OSX does not
> clean up the target directory if it already exists (or at least
> purposely remove thread.di). Will look into fixing that. At least now,
> it works properly, thanks (did a rm -rf /usr/share/dmd and reinstall).

Looked into this further, I added a line to the post install to remove thread.di. However, when attempting to duplicate by installing 2.065 first, and then 2.067 over the top, it appears that installing over an old version doesn't actually leave thread.di behind. I think possibly this happened because I got a new mac and copied the files from my old mac, but I don't think this would be a normal cause for concern.

I do still think a pre-install script of rm -rf /usr/share/dmd would be better. But I don't know if it's warranted.

-Steve
May 12, 2015
On Monday, 27 April 2015 at 13:54:42 UTC, Steven Schveighoffer
wrote:
> On 4/25/15 3:07 AM, Dan Olson wrote:
>> Jacob Carlborg <doob@me.com> writes:
>>
>>> On 2015-04-24 20:37, Steven Schveighoffer wrote:
>>>
>>>> So am I going crazy? Or is dmd doing things differently depending on
>>>> where its environment is? Any compiler gurus out there understand why
>>>> the symbol is different?
>>>>
>>>> I don't want to file a bug with this, because it seems dependent on
>>>> installation location, would possibly not be reproducible.
>>>
>>> I can't reproduce this with DMD from DVM (compiler is installed in the
>>> user home directory).
>>
>> I have lots of version laying around and they all worked fine on the
>> posted code.
>>
>> But maybe a clue here...
>>
>> $ ~/dmd2.066.0/osx/bin/dmd mod1.d
>> $ nm mod1.o | grep start
>>                  U _D4core6thread6Thread5startMFZv
>> $ dmd mod1.d
>> $ nm mod1.o | grep start
>>                  U _D4core6thread6Thread5startMFNbZC4core6thread6Thread
>>
>> --- a/src/core/thread.d
>> +++ b/src/core/thread.d
>> @@ -587,7 +587,7 @@ class Thread
>>       * Throws:
>>       *  ThreadException if the thread fails to start.
>>       */
>> -    final Thread start()
>> +    final Thread start() nothrow
>>      in
>>      {
>>          assert( !next && !prev );
>>
>> I wonder if dmd -v will show where its picking up stuff.
>>
>
> Thank you. It is something I missed. Lo and behold:
>
> dmd -v mod1.d:
>
> ...
>
> import    core.thread	(/usr/share/dmd/src/druntime/import/core/thread.di)
>
> ...
>
> ~/Downloads/dmd2/osx/bin/dmd -v mod1.d:
>
> ...
>
> import    core.thread (/Users/steves/Downloads/dmd2/osx/bin/../../src/druntime/import/core/thread.d)
>
> ...
>
> Hm... thread.di vs thread.d (and I didn't notice before because all the other imports were .d, I just glanced over that detail).
>
> And so, let's see here:
>
> ls -l /usr/share/dmd/src/druntime/import/core/thread.*
>
> -rw-rw-r--   1 steves  staff  157781 Mar 24 10:44 thread.d
> -rw-rw-r--+  1 steves  staff   31382 Feb 24  2014 thread.di
>
> Hey, looky there :)
>
> So, looking at time machine (love that feature), I found that when I installed 2.066, back in October, we had switched to using thread.d instead of thread.di. But the installer did not erase the old thread.di.
>
> BTW, found this: https://github.com/D-Programming-Language/druntime/pull/865
>
> So I think the issue here is that the pkg installer on OSX does not clean up the target directory if it already exists (or at least purposely remove thread.di). Will look into fixing that. At least now, it works properly, thanks (did a rm -rf /usr/share/dmd and reinstall).
>
> -Steve

Now that you mention it, I had similar issues on my Linux distro
at work (Ubuntu, alas!). The installer for 2.067.0 (or 2.066.0?)
didn't erase old files and I got compiler errors for no apparent
reason. What I did was to use dvm to install and manage my dmd
versions. This is much cleaner. I just didn't have the patience
to go through all the /dmd/ directories and clean them up.

dvm installs its own copy, it's not a system wide installation,
so you never have any issues with old files. I wouldn't want to
miss it now.

dvm install 2.067.1

later in your shell you just type

dvm use 2.067.1

https://github.com/jacob-carlborg/dvm