Hey guys,

as a teaser for the curious, here's the current status with a bleeding edge Win64 environment:

* Visual Studio 2015 CTP
* LLVM master (5de9960)
* LDC: merge-2.067
** *.conf.in files manually modified to include "-L/LARGEADDRESSAWARE:NO" as default option
** druntime: + https://github.com/kinke/druntime/commit/1add4f0d401717acc42d92600f9e85ca7d0efe11 + https://github.com/ldc-developers/druntime/pull/17
** phobos: + https://github.com/ldc-developers/phobos/pull/17

druntime + phobos unittests, debug and release:
92% tests passed, 46 tests failed out of 555

failures:
core.thread (segfaults in release only)
std.csv
std.datetime
std.encoding
std.math
std.parallelism
std.path
std.process
std.socket
std.stream
std.string (fails to compile in debug, fails in release)
std.traits
std.uni
std.uri
std.zip
std.zlib
std.algorithm.sorting (fails in debug only)
std.digest.crc
std.digest.md (fails in debug only)
std.digest.ripemd (fails in debug only)
std.digest.sha
std.net.isemail
std.regex.internal.parser
std.regex.internal.tests
std.regex
std.internal.math.gammafunction

"kinke" <noone@nowhere.com> writes:
> failures:
> core.thread (segfaults in release only)

Where does core.thread fail its test?

On Sunday, 10 May 2015 at 14:13:35 UTC, kinke wrote:
> Hey guys,
>
> as a teaser for the curious, here's the current status with a bleeding edge Win64 environment:

Really cool!

> failures:
> std.digest.sha

The failure may be caused by the assembler code because of non-matching ABI conventions.

Regards,
Kai

On Sunday, 10 May 2015 at 22:25:14 UTC, Dan Olson wrote:
> Where does core.thread fail its test?

Apparently an access violation in fiber_switchContext().

asm:
push rbx
xor  rax,rax
push qword ptr gs:[rax]
push qword ptr gs:[rax+8]
push qword ptr gs:[rax+10h]
mov  qword ptr [rcx],rsp
mov  rsp,rdx
pop  qword ptr gs:[rax+10h] --> access violation with rax=0
pop  qword ptr gs:[rax+8]
pop  qword ptr gs:[rax]
pop  rbx

On Tuesday, 12 May 2015 at 22:46:26 UTC, kinke wrote:
> On Sunday, 10 May 2015 at 22:25:14 UTC, Dan Olson wrote:
>> Where does core.thread fail its test?
>
> Apparently an access violation in fiber_switchContext().
>
> asm:
> push rbx
> xor  rax,rax
> push qword ptr gs:[rax]
> push qword ptr gs:[rax+8]
> push qword ptr gs:[rax+10h]
> mov  qword ptr [rcx],rsp
> mov  rsp,rdx
> pop  qword ptr gs:[rax+10h] --> access violation with rax=0
> pop  qword ptr gs:[rax+8]
> pop  qword ptr gs:[rax]
> pop  rbx

I wonder if it issue #666 [1]?  It looks like the same failure I see on OS X/iOS and only in release builds.  Can you see if it happens in the runShared test?  If so, it may pass if you enable the version(Posix) code for sm_this that uses pthread_get/setspecific [2].

[1] https://github.com/ldc-developers/ldc/issues/666
[2] https://github.com/ldc-developers/druntime/blob/ldc/src/core/thread.d#L1135

On Wednesday, 13 May 2015 at 05:14:57 UTC, Dan Olson wrote:
>> Apparently an access violation in fiber_switchContext().
>>
>> asm:
>> push rbx
>> xor  rax,rax
>> push qword ptr gs:[rax]
>> push qword ptr gs:[rax+8]
>> push qword ptr gs:[rax+10h]
>> mov  qword ptr [rcx],rsp
>> mov  rsp,rdx
>> pop  qword ptr gs:[rax+10h] --> access violation with rax=0
>> pop  qword ptr gs:[rax+8]
>> pop  qword ptr gs:[rax]
>> pop  rbx
>
> I wonder if it issue #666 [1]?  It looks like the same failure I see on OS X/iOS and only in release builds.  Can you see if it happens in the runShared test?  If so, it may pass if you enable the version(Posix) code for sm_this that uses pthread_get/setspecific [2].
>
> [1] https://github.com/ldc-developers/ldc/issues/666
> [2] https://github.com/ldc-developers/druntime/blob/ldc/src/core/thread.d#L1135

Thx - it crashes during the runShared test. pthread isn't supported on Windows. The access violation occurs in https://github.com/ldc-developers/druntime/blob/ldc/src/core/thread.d#L3597.

"kinke" <noone@nowhere.com> writes:

> On Wednesday, 13 May 2015 at 05:14:57 UTC, Dan Olson wrote:
>>> Apparently an access violation in fiber_switchContext().
>>>
>>> asm:
>>> push rbx
>>> xor  rax,rax
>>> push qword ptr gs:[rax]
>>> push qword ptr gs:[rax+8]
>>> push qword ptr gs:[rax+10h]
>>> mov  qword ptr [rcx],rsp
>>> mov  rsp,rdx
>>> pop  qword ptr gs:[rax+10h] --> access violation with rax=0
>>> pop  qword ptr gs:[rax+8]
>>> pop  qword ptr gs:[rax]
>>> pop  rbx
>>
>> I wonder if it issue #666 [1]?  It looks like the same failure I see on OS X/iOS and only in release builds.  Can you see if it happens in the runShared test?  If so, it may pass if you enable the version(Posix) code for sm_this that uses pthread_get/setspecific [2].
>>
>> [1] https://github.com/ldc-developers/ldc/issues/666
>> [2]
>> https://github.com/ldc-developers/druntime/blob/ldc/src/core/thread.d#L1135
>
> Thx - it crashes during the runShared test. pthread isn't supported on Windows. The access violation occurs in https://github.com/ldc-developers/druntime/blob/ldc/src/core/thread.d#L3597.

It does seem to be the same problem because the stack to resume on is wrong.  That is what I see on OS X and iOS.  You could try Windows TlsGetValue API for sm_this and mimic the pthread_getspecific code.

If it works, I don't think it is a real fix but does allow rest of thread unittest to run.  Maybe just disabling the runShared test and documenting is best.

I found this older page where boost decided coroutine migration between threads was unsafe because of TLS:

http://www.crystalclearsoftware.com/soc/coroutine/coroutine/coroutine_thread.html

Thinking out loud: If LDC could provide a switch to disable TLS address caching, would folks think to use it?  Should it be on by default to support the ability to migrate Fibers across threads?  Is it worth the performance loss in other TLS cases?  Maybe it is better to have some targets w/ expensive TLS lookup just disable Fiber migration.  Is FIber migration that common?

Most interesting: many of the failing unittests are caused by a single issue, namely, std.concurrency.unregisterMe() invoked by the static destructor of the std.concurrency module.
The following unittests all pass when isolated, i.e., by compiling via 'ldc2 -g -main -unittest <foo>.d' (debug, '-release' added for release) and then running the resulting <foo>.exe:

std.datetime
std.parallelism
std.path
std.process
std.string
std.uni
std.uri
std.zip
std.zlib
std.algorithm.sorting
std.digest.crc
std.digest.md
std.digest.ripemd
std.digest.sha
std.net.isemail
std.regex.internal.parser
std.regex.internal.tests
std.regex

The following failures are NOT caused by std.concurrency.~this():

core.thread
std.csv *
std.encoding
std.math
std.socket
std.stream
std.traits *
std.internal.math.gammafunction *

[*] fixed or worked around on my system, patches being prepared

So we're not far from LDC on Win64 passing all druntime + phobos unit tests! :)

On Thursday, 14 May 2015 at 07:51:16 UTC, Dan Olson wrote:
> You could try Windows TlsGetValue API for sm_this and mimic
> the pthread_getspecific code.

Yep, that makes the runShared test pass. core.thread still doesn't pass all tests in release though (e.g., non-volatile GP registers are apparently not restored correctly).

"kinke" <noone@nowhere.com> writes:

> The following failures are NOT caused by std.concurrency.~this():
>
> core.thread
> std.csv *
> std.encoding
> std.math
> std.socket
> std.stream
> std.traits *
> std.internal.math.gammafunction *
>
> [*] fixed or worked around on my system, patches being prepared
>
> So we're not far from LDC on Win64 passing all druntime + phobos unit tests! :)

Very cool.  The math unittest failures could just be the ones that aren't written for 64-bit real.  There was some work on it by Kevin and Johan in this pull [1], but I have not been following lately.

[1] https://github.com/ldc-developers/phobos/pull/7