I was playing around with dmd`s make file trying to see if I can compile dmd with different compilers and different compilation flags. By playing around I found that some dmd files are compiled with -vtls flag unconditionally and that ldc do not support this flag. First I dont know what -vtls flag does so I looked at documentation.

DMD documentation says:
-vtls
List all variables going into thread local storage

GDC documentation says:
-fd-vtls
Print information about all variables going into thread local storage to stdout.

This doesnt help much. I still have no idea what it actually does. Aren't all variables thread local unless explicitly specified? So I tried to compile simple example and dmd printed nothing. GDC printed this:
gdc: error: unrecognized command line option ‘-fd-vtls’

One gdc wiki(https://wiki.dlang.org/Using_GDC) is out of date and second I still have no clue what is does.

This flag blocks compilation and I want to just remove it but since I dont know what it is used for I hesitate. If my attempts were to be successful in compiling dmd with other compilers I would make pull request but now I have more question than answers. What this flag does? Why some files compiled with it and other dont? Why it is added to all compilation targets unconditionally?

On Wednesday, 3 October 2018 at 20:41:15 UTC, welkam wrote:
> I was playing around with dmd`s make file trying to see if I can compile dmd with different compilers and different compilation flags. By playing around I found that some dmd files are compiled with -vtls flag unconditionally and that ldc do not support this flag. First I dont know what -vtls flag does so I looked at documentation...

> Aren't all variables thread local unless explicitly specified?

No, all *static non-immutable* variables are thread-local by default, not just "all variables".

> What this flag does? Why some files compiled with it and other dont? Why it is added to all compilation targets unconditionally?

It does what it says, just prints some diagnostic info. You can just ignore it, i.e. not use it. As for why it's in the makefile, I can only guess that someone keeps it there for diagnostic purposes :)

On Wednesday, 3 October 2018 at 20:58:01 UTC, Stanislav Blinov wrote:
> No, all *static non-immutable* variables are thread-local by default, not just "all variables".
I misspoke but this should write something
https://run.dlang.io/is/3u1wFp

If you look at asm output there are "call _d_arraycatT@PLT32" instruction so those variables weren't optimized away and yet no output to stdout.

If remove that flag ldc now compiles dmd but lacks optimizations so more work needs to be done.

On Wednesday, 3 October 2018 at 21:18:26 UTC, welkam wrote:
> On Wednesday, 3 October 2018 at 20:58:01 UTC, Stanislav Blinov wrote:
>> No, all *static non-immutable* variables are thread-local by default, not just "all variables".
> I misspoke but this should write something
> https://run.dlang.io/is/3u1wFp

And it does print "testtest" when you run it. But there are no thread-local variables there, so the compiler doesn't print anything extra when compiling. That's what -vtls flag does: during compilation reports which variables are thread-local.

> If you look at asm output there are "call _d_arraycatT@PLT32" instruction so those variables weren't optimized away and yet no output to stdout.
>
> If remove that flag ldc now compiles dmd but lacks optimizations so more work needs to be done.

I'm not sure I understand.

string a = "test";
is a variable that mutates so it should be thread local. Also ASM output shows that these variables are not optimized away so compiler should output something but it doesnt. Or I dont understand thread local storage.

On Wednesday, 3 October 2018 at 21:39:12 UTC, welkam wrote:
> string a = "test";
> is a variable that mutates so it should be thread local. Also ASM output shows that these variables are not optimized away so compiler should output something but it doesnt. Or I dont understand thread local storage.

Thread-local storage is memory allocated for each thread.
Only static non-immutable variables go there. Regular variables on the stack aren't explicitly placed in any TLS, they're, well, on the stack as it is.

For example, add this to the top of your program after "import std.stdio;":

int IAmInTls;

Then when you compile with -vtls, dmd will report that indeed a variable named "IAmInTls" will be placed in thread-local storage. That's because module-level variables are static.
If, however, you declare it as immutable int, it won't go to TLS, it'll become a regular global.

Same will happen if you add a static variable inside the main() function, e.g. a "static string str;".

On Wednesday, 3 October 2018 at 20:41:15 UTC, welkam wrote:
> This flag blocks compilation and I want to just remove it but since I dont know what it is used for I hesitate. If my attempts were to be successful in compiling dmd with other compilers I would make pull request but now I have more question than answers. What this flag does? Why some files compiled with it and other dont? Why it is added to all compilation targets unconditionally?

A long time ago, D global and static variables worked just like in C/C++, and went in the data segment (i.e. there was one per program). At one point, this was changed, so that they would go into TLS by default (i.e. one per thread), thus making a lot of code that was previously not thread-safe, magically thread-safe. To allow developers who wanted to make sure that their code compiled to exactly the same behavior as before to update their programs easily, this flag was added, so that they could find all the places in their program where the new behavior will take into effect.

Today, it is still useful for finding globals in programs that shouldn't have any. I don't know why it's used in DMD, but that might certainly be the reason, considering there is a long-term ongoing effort to make DMD usable as a library.

The flag otherwise should have no effect on the result of the compilation (i.e. produced object files / binaries), so you can remove it locally. You will probably need to investigate why the compiler you're using doesn't support it (perhaps it's there under another name) - the code for its implementation should still be there, as all of DMD/LDC/GDC use the same frontend.

On Wednesday, 3 October 2018 at 21:50:49 UTC, Stanislav Blinov wrote:
> Thread-local storage is memory allocated for each thread.
> Only static non-immutable variables go there. Regular variables on the stack aren't explicitly placed in any TLS, they're, well, on the stack as it is.

Oh so its like thread local globals sort of. My reasoning was that stack is a form of storage and is thread local so...

Thanks for responses it became more clear what this flag does.

Quick tests with different compilers.

DMD from Arch repo compiles hello world (dmd -c main.d) in:
~0.1s
DMD compiled with DMD and stock flags
~0.12s
DMD compiled with ldc with -o2
~0.35s


DMD code gen is not that bad

On Thursday, 4 October 2018 at 13:48:24 UTC, welkam wrote:
> Oh so its like thread local globals sort of. My reasoning was that stack is a form of storage and is thread local so...

Thread local storage IS "global" per thread though, instead of per process which "global" is in other languages.

The stack, is... well the stack and has nothing to do with the thread local storage. Although each thread has their own stack, else threads would have to wait for each other every time something was pushed or popped.

Thread local storage is a bit more complex though and very different than regular heap allocations.

See: https://en.wikipedia.org/wiki/Thread-local_storage

On Wednesday, 3 October 2018 at 20:58:01 UTC, Stanislav Blinov wrote:
> On Wednesday, 3 October 2018 at 20:41:15 UTC, welkam wrote:
>> I was playing around with dmd`s make file trying to see if I can compile dmd with different compilers and different compilation flags. By playing around I found that some dmd files are compiled with -vtls flag unconditionally and that ldc do not support this flag. First I dont know what -vtls flag does so I looked at documentation...
>
>> Aren't all variables thread local unless explicitly specified?
>
> No, all *static non-immutable* variables are thread-local by default, not just "all variables".
>
>> What this flag does? Why some files compiled with it and other dont? Why it is added to all compilation targets unconditionally?
>
> It does what it says, just prints some diagnostic info. You can just ignore it, i.e. not use it. As for why it's in the makefile, I can only guess that someone keeps it there for diagnostic purposes :)

It was added during the transition to a D codebase to avoid regressions. These days its more or less useless as it doesn't fail the build anyhow.
Though LDC and GDC still rely on extern(C++) global variables for their interface with the frontend (not TLS static).

See also: https://github.com/dlang/dmd/pull/8018