Thread overview
Corruption of shared data with multiple threads
Mar 29, 2015
Johan Engelen
Mar 29, 2015
David Nadlinger
Mar 30, 2015
Kai Nacke
Mar 29, 2015
David Nadlinger
Mar 29, 2015
Johan Engelen
March 29, 2015
Hi all,
  I am working on adding coverage analysis (like DMD's) to LDC.
It is a work in progress, but you can test the current status here: https://github.com/JohanEngelen/ldc/tree/coverage
It already works nicely, but the line count is not increased for all types of statements yet. I've been breaking my head on a problem with multithreaded programs.

This test program works fine (ldc2 -cov test.d)
void main() {
   fun(...);
}
void fun(...) {
    ...
}
and outputs the same line count as dmd does.
Making the program multithreaded, like this:
import std.concurrency;
void main() {
   spawn(&fun,...);
}
void fun(...) {
    ...
}
no longer works. (it actually used to work for me, but for some reason with a completely rebuilt druntime, it crashes). I've traced the problem back to the Cover[] gdata data structure in druntime/cover.d becoming corrupted somewhere during program execution. More specifically, in struct Cover, filename.length becomes corrupted somehow.

Is it a known issue, that we have some bugs with shared data + multiple threads?

Thanks,
  Johan

(LLVM 3.6, MSVC, x64)
March 29, 2015
On 03/29/2015 06:46 PM, Johan Engelen via digitalmars-d-ldc wrote:
> Is it a known issue, that we have some bugs with shared data + multiple
> threads?

I don't know of any general threading issues besides #666 (which only affects migrating fibers between threads). We do optimize some things much more aggressively than DMD does, though, so it is possible that broken code works by accident on DMD but fails on LDC.

> (LLVM 3.6, MSVC, x64)

There might be some Win64-specific issues, though. Kai? You might want to test your changes on Linux to be sure.

 — David
March 29, 2015
Oh, I forgot to ask: Did you look into making use of LLVM's profiling data framework instead? We would likely get better performance this way, and might be able profit of all the tooling built around LLVM's coverage support down the road.

See for example:

http://llvm.org/devmtg/2014-04/PDFs/LightningTalks/EuroLLVM'14%20--%20ASan%20%2B%20Coverage.pdf
http://llvm.org/docs/CommandGuide/llvm-cov.html
http://llvm.org/docs/CoverageMappingFormat.html

But then, offering a DMD-compatible implementation might also be nice for users, and it's rather simple…

 — David


On 03/29/2015 07:33 PM, David Nadlinger wrote:
> On 03/29/2015 06:46 PM, Johan Engelen via digitalmars-d-ldc wrote:
>> Is it a known issue, that we have some bugs with shared data + multiple
>> threads?
>
> I don't know of any general threading issues besides #666 (which only
> affects migrating fibers between threads). We do optimize some things
> much more aggressively than DMD does, though, so it is possible that
> broken code works by accident on DMD but fails on LDC.
>
>> (LLVM 3.6, MSVC, x64)
>
> There might be some Win64-specific issues, though. Kai? You might want
> to test your changes on Linux to be sure.
>
>   — David
March 29, 2015
On Sunday, 29 March 2015 at 18:03:52 UTC, David Nadlinger wrote:
> Oh, I forgot to ask: Did you look into making use of LLVM's profiling data framework instead? We would likely get better performance this way, and might be able profit of all the tooling built around LLVM's coverage support down the road.

I did not look at LLVM's profiling framework at all.
I wanted to copy DMD's coverage analysis, as indeed it seemed a straightforward simple task. It is nice to work on to get to know LLVM and LDC. :)

I hope someone can testdrive it on non-Windows, and give me some feedback.
March 30, 2015
On Sunday, 29 March 2015 at 17:33:47 UTC, David Nadlinger wrote:
> On 03/29/2015 06:46 PM, Johan Engelen via digitalmars-d-ldc wrote:
>> (LLVM 3.6, MSVC, x64)
>
> There might be some Win64-specific issues, though. Kai? You might want to test your changes on Linux to be sure.

I have no idea yet. Maybe you try running with the stub garbage collector? There are many parts in the MSVC build which are not well tested. I always see memory corruption if the garbage collector is not working correctly.

(On Linux/PPC I had some strange errors in std.string because the garbage collector freed memory which was still in use. It turned out that the TLS segment was not set up. Something similar could be the case on Win64. See druntime/src/rt/msvc.c for the current initialization of memory segments.)

Regards,
Kai