Thread overview
immutable, static, __gshared, TLS, and compile time allocation
Apr 20, 2012
Manu
Apr 21, 2012
Michel Fortin
Apr 21, 2012
Manu
Apr 21, 2012
Michel Fortin
Apr 21, 2012
Artur Skawina
Apr 21, 2012
Walter Bright
April 20, 2012
I need to clarify some things that have confused me a few times. I've reconsidered these basic attributes a few times, and I thought I understood them, but I obviously don't.

The case that has confused me is here: http://d.puremagic.com/issues/show_bug.cgi?id=7897

In the global scope:

int x; <- x is TLS

but:

static int x;  <- this is... a 'static' global instance, whatever that means? Is this TLS or not? If so, how is it distinct from 'int x;' above? I presume it must still be TLS, and effectively meaningless at the global scope; dlang.org states "static is ignored when applied to other declarations". It is just used for members of struct/classes? Why not produce a syntax error rather than ignore it?

immutable int x;  <- this can't possibly change, so why would it be TLS? it must be a single static instance... right?

__gshared int x;  <- this should behave exactly like a C global right? ie, no TLS + addressable at compile time.

static immutable x;  <- i've seen this a few times, what does it mean?


I'm concerned with which of these are addressable at compile time. By my
logic, all (well, perhaps not the first) should be allocated in the
datablock, addresses known at compile time, and therefore addressable at
compile time (ie, able to alias in template args).
That doesn't seem to be the case however in at least the __gshared case (in
my big report above), which is very surprising to me.

There's another thing that that's had me scratching my head for a while, and gave me the contradictory feeling that the use of TLS data is fairly relaxed:

struct S
{
   int x;
   static int y;
}

In this case y is static, ie, globally shared by all instances, but I
expect it SHOULD be TLS, since it is mutable, and therefore not thread
safe...
That said, I write code that passes static members of structs into
templates as alias parameters all the time, and generate code that writes
to them.
How is this possible? And given that it is, why is it not possible for at
least __gshared at global scope?


struct S
{
   static x;
   __gshared y;
}

What is the difference between x and y here?


I've obviously missed something rather fundamental, but the dlang descriptions of each of these attributes are a little light, and leaves me quite unclear on the details here.

I'm particularly confused that I CAN alias a static member in a struct, which I suspect should be TLS for thread safety, but I CAN'T alias a __gshared at global scope, which is not TLS by definition?

O_o


April 21, 2012
On 2012-04-20 20:45:11 +0000, Manu <turkeyman@gmail.com> said:

> I need to clarify some things that have confused me a few times. I've
> reconsidered these basic attributes a few times, and I thought I understood
> them, but I obviously don't.
> 
> The case that has confused me is here:
> http://d.puremagic.com/issues/show_bug.cgi?id=7897
> 
> In the global scope:
> 
> int x; <- x is TLS
> 
> but:
> 
> static int x;  <- this is... a 'static' global instance, whatever that
> means? Is this TLS or not?

If you're trying to make a C-like global, write this:

	static __gshared int x;

"static" here is optional if you are at global scope. "__gshared" is a storage class which basically mean "no TLS": the global variable is available to all threads (C-style).

Note that because "__gshared" is a storage class (and not a type constructor), the type of x is "int", not "shared(int)". So, despite the same address being seen by all threads, it'll be treated as if it was a thread-local variable as far as the type system is concerned.

One could say that shared and immutable global variables are implicitly "__ghsared", in the sense that they are not in TLS, because they have a type qualifier that make it safe to be visible in all threads.

-- 
Michel Fortin
michel.fortin@michelf.com
http://michelf.com/

April 21, 2012
On 21 April 2012 03:05, Michel Fortin <michel.fortin@michelf.com> wrote:

> If you're trying to make a C-like global, write this:
>
>        static __gshared int x;
>

dlang.org seems to disagree:
"__gshared may also be applied to member variables and local variables. In
these cases, *__gshared is equivalent to static*, except that the variable
is shared by all threads rather than being thread local."
Suggests to me that you are not required to state both.

So which is correct?
It appears most people are confused about this. I'm thinking more and more
it's worth addressing that confusion with some warnings/errors.


April 21, 2012
On 4/20/2012 1:45 PM, Manu wrote:
> static int x; <- this is... a 'static' global instance, whatever that means?

At global scope, the 'static' attribute is redundant and therefore meaningless.

> Is this TLS or not?

static int x; // TLS

> If so, how is it distinct from 'int x;' above?

At global scope, it is the same.

> I presume it
> must still be TLS, and effectively meaningless at the global scope; dlang.org
> <http://dlang.org> states "static is ignored when applied to other
> declarations". It is just used for members of struct/classes? Why not produce a
> syntax error rather than ignore it?

Originally, it meant 'private' at global scope, but this was dropped.

> immutable int x; <- this can't possibly change, so why would it be TLS?

It wouldn't be TLS, because as you said, it can't change.

> it must be a single static instance... right?

Right. But this is an optimization - semantically, it makes no difference if it is TLS or not.


> __gshared int x; <- this should behave exactly like a C global right? ie, no TLS
> + addressable at compile time.

Yes.


> static immutable x; <- i've seen this a few times, what does it mean?

At global scope, it means: immutable x;

> I'm concerned with which of these are addressable at compile time. By my logic,
> all (well, perhaps not the first) should be allocated in the datablock,
> addresses known at compile time, and therefore addressable at compile time (ie,
> able to alias in template args).

This is a misunderstanding of alias. Alias does not mean "has a compile time pointer to it", it means it is a symbol.

> That doesn't seem to be the case however in at least the __gshared case (in my
> big report above), which is very surprising to me.
>
> There's another thing that that's had me scratching my head for a while, and
> gave me the contradictory feeling that the use of TLS data is fairly relaxed:
>
> struct S
> {
>     int x;
>     static int y;
> }
>
> In this case y is static, ie, globally shared by all instances,

y would be TLS.

> but I expect it SHOULD be TLS,

and it is.

> since it is mutable, and therefore not thread safe...
> That said, I write code that passes static members of structs into templates as
> alias parameters all the time, and generate code that writes to them.
> How is this possible? And given that it is, why is it not possible for at least
> __gshared at global scope?

Because alias is a symbolic alias, not a compile time address.


> struct S
> {
>     static x;
>     __gshared y;
> }
>
> What is the difference between x and y here?

x is TLS, y is in the global data segment.

> I've obviously missed something rather fundamental, but the dlang descriptions
> of each of these attributes are a little light, and leaves me quite unclear on
> the details here.
>
> I'm particularly confused that I CAN alias a static member in a struct, which I
> suspect should be TLS for thread safety, but I CAN'T alias a __gshared at global
> scope, which is not TLS by definition?

You can alias a global __gshared symbol, you just cannot alias an offset into a symbol (i.e. a computation on a symbol).

April 21, 2012
On 2012-04-21 00:22:21 +0000, Manu <turkeyman@gmail.com> said:

> On 21 April 2012 03:05, Michel Fortin <michel.fortin@michelf.com> wrote:
> 
>> If you're trying to make a C-like global, write this:
>> 
>> static __gshared int x;
> 
> dlang.org seems to disagree:
> "__gshared may also be applied to member variables and local variables. In
> these cases, *__gshared is equivalent to static*, except that the variable
> is shared by all threads rather than being thread local."
> Suggests to me that you are not required to state both.

I don't really think this is incompatible. __gshared wouldn't make sense for local and member variables, so it makes sense that it sort of implies static. I just didn't know it was implied (and thus redundant).

> So which is correct?
> It appears most people are confused about this. I'm thinking more and more
> it's worth addressing that confusion with some warnings/errors.

I'm not sure what the problem is. If I see __gshared attached to a variable declaration, it means it behaves like a C global. If you don't need to specify "static" explicitly when inside a scope that can have member variables, then great: it's not like non-static __gshared makes sense anyway.

-- 
Michel Fortin
michel.fortin@michelf.com
http://michelf.com/

April 21, 2012
On 04/21/12 02:22, Manu wrote:
> On 21 April 2012 03:05, Michel Fortin <michel.fortin@michelf.com <mailto:michel.fortin@michelf.com>> wrote:
> 
>     If you're trying to make a C-like global, write this:
> 
>            static __gshared int x;
> 
> 
> dlang.org <http://dlang.org> seems to disagree:
> "__gshared may also be applied to member variables and local variables. In these cases, *__gshared is equivalent to static*, except that the variable is shared by all threads rather than being thread local."
> Suggests to me that you are not required to state both.
> 
> So which is correct?

Both. __gshared implies static for _member variables and local variables_ - it has to, otherwise they wouldn't be shared. Would requiring the redundant "static" keyword for these cases really be an improvement?

> It appears most people are confused about this. I'm thinking more and more it's worth addressing that confusion with some warnings/errors.

"static" *should* be silently accepted at module scope (and everywhere else where it's also not needed) - so that things like mixins work and don't need extra static-less versions.

If you want to take a compiletime address of something then you may just as well do that and work with the pointer (and D's lack of '->' helps here); if you really want to work with aliases... well there's always:

    @property ref x() { return thing.x; }
    AliasTheInt!( x )();

which, for simple cases like this one, will do the right thing (ie compile to one store instruction, assuming a sane compiler).

artur