View mode: basic / threaded / horizontal-split · Log in · Help
January 07, 2011
Threads, shread and TLS
So, I thought I sort of understood "shared" and now I think I don't.

If I have a class:

class foo {
 int x;
 static int y;
 shared static int z;

}

So x is one instance per class and is thread-local?
y is one instance per thread?
z is one instance per application, i.e., global?

If that's true (and I realize it might not be), and I want to initialize these
variables in constructors, how does that work?

I think

class foo {
...(as before)

this() { x = 2; } // ok

static this() { y = 3; } // is this called once per thread?

shared static this() { z = 3;} // also, okay, called before main
}

but I don't understand what happens with threads and the "static this"
constructor.  How/when are the thread-local copies constructed?  How do you
initialize/construct the thread-local static data?

Thanks!

Adam
January 07, 2011
Re: Threads, shread and TLS
On Fri, 07 Jan 2011 09:24:18 -0500, Adam Conner-Sax  
<adam_conner_sax@yahoo.com> wrote:

> So, I thought I sort of understood "shared" and now I think I don't.
>
> If I have a class:
>
> class foo {
>   int x;
>   static int y;
>   shared static int z;
>
> }
>
> So x is one instance per class and is thread-local?
> y is one instance per thread?
> z is one instance per application, i.e., global?

Yes to all

>
> If that's true (and I realize it might not be), and I want to initialize  
> these
> variables in constructors, how does that work?
>
> I think
>
> class foo {
> ...(as before)
>
> this() { x = 2; } // ok
>
> static this() { y = 3; } // is this called once per thread?
>
> shared static this() { z = 3;} // also, okay, called before main
> }
>
> but I don't understand what happens with threads and the "static this"
> constructor.  How/when are the thread-local copies constructed?  How do  
> you
> initialize/construct the thread-local static data?

static this() is run upon thread creation (and once at the beginning of  
the program for the main thread), and static ~this() is run when a thread  
is destroyed.

All static this() and shared static this() functions are assembled by the  
runtime into an array of constructors, built in the correct order based on  
import dependencies.  So each time a thread starts, it simply goes through  
an array and calls each function.

-Steve
January 07, 2011
Re: Threads, shread and TLS
== Quote from Adam Conner-Sax (adam_conner_sax@yahoo.com)'s article
> So, I thought I sort of understood "shared" and now I think I don't.
> If I have a class:
> class foo {
>   int x;
>   static int y;
>   shared static int z;
> }
> So x is one instance per class and is thread-local?
> y is one instance per thread?
> z is one instance per application, i.e., global?
> If that's true (and I realize it might not be), and I want to initialize these
> variables in constructors, how does that work?
> I think
> class foo {
> ...(as before)
> this() { x = 2; } // ok
> static this() { y = 3; } // is this called once per thread?
> shared static this() { z = 3;} // also, okay, called before main
> }
> but I don't understand what happens with threads and the "static this"
> constructor.  How/when are the thread-local copies constructed?  How do you
> initialize/construct the thread-local static data?
> Thanks!
> Adam

Nevermind.  Answered myself with the following:

import core.thread;
import std.c.stdio;

class foo {

 int a;
 shared int b;

 static int x;
 shared static int y;
 shared static int[] arr1;
 shared static int[] arr2;

 this() { a = 1; b=10; }
 static this() { x=100; arr1 ~= x; }
 shared static this() { y=1000; arr2 ~= y;  }

 static void A() { x++; y++; }
 void B() { a++; b++; }
 void report() {
   printf("a=%i; b=%i; x=%i; y=%i; arr1.length=%i;
arr2.length=%i\n",a,b,x,y,arr1.length, arr2.length);
 }
}

void main()
{
 auto f = new foo();
 void call_foo_functions() { f.A(); f.B(); f.report(); }
 auto tg = new ThreadGroup();
 foreach (k; 0..3) {
   auto t = new Thread(&call_foo_functions);
   tg.add(t);
   t.start();
 }
 tg.joinAll();
 printf("back in main: ");
 f.report();
}

which output:

a=2; b=11; x=101; y=1001; arr1.length=2; arr2.length=1
a=3; b=12; x=101; y=1002; arr1.length=2; arr2.length=1
a=4; b=13; x=101; y=1003; arr1.length=3; arr2.length=1
back in main: a=4; b=13; x=100; y=1003; arr1.length=3; arr2.length=1


which all makes sense (to me) except a.  Why is a acting global?  Is it since it
isn't static so it belongs to the class and there is only one copy of the class?
Then what makes a and b different?

Also, though maybe it's obvious to everybody else, I think the docs should explain
someplace that "static this()" gets called once *per thread*.  Maybe I just missed it.

Adam
January 07, 2011
Re: Threads, shread and TLS
On Fri, 07 Jan 2011 09:55:33 -0500, Adam Conner-Sax  
<adam_conner_sax@yahoo.com> wrote:

> == Quote from Adam Conner-Sax (adam_conner_sax@yahoo.com)'s article
>> So, I thought I sort of understood "shared" and now I think I don't.
>> If I have a class:
>> class foo {
>>   int x;
>>   static int y;
>>   shared static int z;
>> }
>> So x is one instance per class and is thread-local?
>> y is one instance per thread?
>> z is one instance per application, i.e., global?
>> If that's true (and I realize it might not be), and I want to  
>> initialize these
>> variables in constructors, how does that work?
>> I think
>> class foo {
>> ...(as before)
>> this() { x = 2; } // ok
>> static this() { y = 3; } // is this called once per thread?
>> shared static this() { z = 3;} // also, okay, called before main
>> }
>> but I don't understand what happens with threads and the "static this"
>> constructor.  How/when are the thread-local copies constructed?  How do  
>> you
>> initialize/construct the thread-local static data?
>> Thanks!
>> Adam
>
> Nevermind.  Answered myself with the following:
>
> import core.thread;
> import std.c.stdio;
>
> class foo {
>
>   int a;
>   shared int b;
>
>   static int x;
>   shared static int y;
>   shared static int[] arr1;
>   shared static int[] arr2;
>
>   this() { a = 1; b=10; }
>   static this() { x=100; arr1 ~= x; }
>   shared static this() { y=1000; arr2 ~= y;  }
>
>   static void A() { x++; y++; }
>   void B() { a++; b++; }
>   void report() {
>     printf("a=%i; b=%i; x=%i; y=%i; arr1.length=%i;
> arr2.length=%i\n",a,b,x,y,arr1.length, arr2.length);
>   }
> }
>
> void main()
> {
>   auto f = new foo();
>   void call_foo_functions() { f.A(); f.B(); f.report(); }
>   auto tg = new ThreadGroup();
>   foreach (k; 0..3) {
>     auto t = new Thread(&call_foo_functions);
>     tg.add(t);
>     t.start();
>   }
>   tg.joinAll();
>   printf("back in main: ");
>   f.report();
>  }
>
> which output:
>
> a=2; b=11; x=101; y=1001; arr1.length=2; arr2.length=1
> a=3; b=12; x=101; y=1002; arr1.length=2; arr2.length=1
> a=4; b=13; x=101; y=1003; arr1.length=3; arr2.length=1
> back in main: a=4; b=13; x=100; y=1003; arr1.length=3; arr2.length=1
>
>
> which all makes sense (to me) except a.  Why is a acting global?  Is it  
> since it
> isn't static so it belongs to the class and there is only one copy of  
> the class?
> Then what makes a and b different?

OK, so here is what is happening :)

call_foo_functions is a delegate, which means it has a frame pointer to  
the main function.  So all three threads' f is the *same* f (the one  
defined in main()).

I would suggest that you move call_foo_functions outside main, and  
instantiate an additional f *inside* the function.  This would be more  
correct.

Incidentally, it appears that this allows untagged sharing (i.e. sharing  
data that isn't tagged with shared)  I wonder if this issue has been  
reported before?  Sean?

>
> Also, though maybe it's obvious to everybody else, I think the docs  
> should explain
> someplace that "static this()" gets called once *per thread*.  Maybe I  
> just missed it.
>

Yes, the documentation is out of date.  Could you file a bugzilla report  
on this?

-Steve
January 07, 2011
Re: Threads, shread and TLS
Thanks!

It's not really about correctness as much as trying to understand how these
different storage classes work.  I understand that there is only one foo object.
I wanted to see which parts are thread-local and which are shared and how the
constructors work.

I'm working (way over my head) on a more complex threading issue and I realized
that I didn't quite understand the storage classes and constructors. Now I get it
a bit more.

By "untagged sharing" do you mean the "a" variable which is shared in the sense
that all threads see the same copy but does not have storage class "shared"?  Yes,
that confuses me too.  Should it be an error?

I have never dealt with bugzilla but I will try to figure out how to do what you
ask :)

Thanks again.

Adam



== Quote from Steven Schveighoffer (schveiguy@yahoo.com)'s article
> On Fri, 07 Jan 2011 09:55:33 -0500, Adam Conner-Sax
> <adam_conner_sax@yahoo.com> wrote:
> > == Quote from Adam Conner-Sax (adam_conner_sax@yahoo.com)'s article
> >> So, I thought I sort of understood "shared" and now I think I don't.
> >> If I have a class:
> >> class foo {
> >>   int x;
> >>   static int y;
> >>   shared static int z;
> >> }
> >> So x is one instance per class and is thread-local?
> >> y is one instance per thread?
> >> z is one instance per application, i.e., global?
> >> If that's true (and I realize it might not be), and I want to
> >> initialize these
> >> variables in constructors, how does that work?
> >> I think
> >> class foo {
> >> ...(as before)
> >> this() { x = 2; } // ok
> >> static this() { y = 3; } // is this called once per thread?
> >> shared static this() { z = 3;} // also, okay, called before main
> >> }
> >> but I don't understand what happens with threads and the "static this"
> >> constructor.  How/when are the thread-local copies constructed?  How do
> >> you
> >> initialize/construct the thread-local static data?
> >> Thanks!
> >> Adam
> >
> > Nevermind.  Answered myself with the following:
> >
> > import core.thread;
> > import std.c.stdio;
> >
> > class foo {
> >
> >   int a;
> >   shared int b;
> >
> >   static int x;
> >   shared static int y;
> >   shared static int[] arr1;
> >   shared static int[] arr2;
> >
> >   this() { a = 1; b=10; }
> >   static this() { x=100; arr1 ~= x; }
> >   shared static this() { y=1000; arr2 ~= y;  }
> >
> >   static void A() { x++; y++; }
> >   void B() { a++; b++; }
> >   void report() {
> >     printf("a=%i; b=%i; x=%i; y=%i; arr1.length=%i;
> > arr2.length=%i\n",a,b,x,y,arr1.length, arr2.length);
> >   }
> > }
> >
> > void main()
> > {
> >   auto f = new foo();
> >   void call_foo_functions() { f.A(); f.B(); f.report(); }
> >   auto tg = new ThreadGroup();
> >   foreach (k; 0..3) {
> >     auto t = new Thread(&call_foo_functions);
> >     tg.add(t);
> >     t.start();
> >   }
> >   tg.joinAll();
> >   printf("back in main: ");
> >   f.report();
> >  }
> >
> > which output:
> >
> > a=2; b=11; x=101; y=1001; arr1.length=2; arr2.length=1
> > a=3; b=12; x=101; y=1002; arr1.length=2; arr2.length=1
> > a=4; b=13; x=101; y=1003; arr1.length=3; arr2.length=1
> > back in main: a=4; b=13; x=100; y=1003; arr1.length=3; arr2.length=1
> >
> >
> > which all makes sense (to me) except a.  Why is a acting global?  Is it
> > since it
> > isn't static so it belongs to the class and there is only one copy of
> > the class?
> > Then what makes a and b different?
> OK, so here is what is happening :)
> call_foo_functions is a delegate, which means it has a frame pointer to
> the main function.  So all three threads' f is the *same* f (the one
> defined in main()).
> I would suggest that you move call_foo_functions outside main, and
> instantiate an additional f *inside* the function.  This would be more
> correct.
> Incidentally, it appears that this allows untagged sharing (i.e. sharing
> data that isn't tagged with shared)  I wonder if this issue has been
> reported before?  Sean?
> >
> > Also, though maybe it's obvious to everybody else, I think the docs
> > should explain
> > someplace that "static this()" gets called once *per thread*.  Maybe I
> > just missed it.
> >
> Yes, the documentation is out of date.  Could you file a bugzilla report
> on this?
> -Steve
January 07, 2011
Re: Threads, shread and TLS
On Fri, 07 Jan 2011 11:30:17 -0500, Adam Conner-Sax  
<adam_conner_sax@yahoo.com> wrote:

> Thanks!
>
> It's not really about correctness as much as trying to understand how  
> these
> different storage classes work.  I understand that there is only one foo  
> object.
> I wanted to see which parts are thread-local and which are shared and  
> how the
> constructors work.

A class instance can be shared or unshared.  Either way, changing the data  
on the same instance updates the same instance, there is not a copy of the  
whole world in each thread, just a copy of the thread local storage block.

So you are sort of conflating 'per instance' with 'per thread.'

int x -- per instance (shared or not)
static int x -- per thread (in the TLS block)
shared static int x -- per process, not in any instance.

>
> I'm working (way over my head) on a more complex threading issue and I  
> realized
> that I didn't quite understand the storage classes and constructors. Now  
> I get it
> a bit more.
>
> By "untagged sharing" do you mean the "a" variable which is shared in  
> the sense
> that all threads see the same copy but does not have storage class  
> "shared"?  Yes,
> that confuses me too.  Should it be an error?

Yes, if you have a piece of data that shared and not marked with __gshared  
or shared, then we have a problem.  The problem is that a lot of code  
assumes that situation cannot happen without casts (you can always cast  
and override the type system), so you can make assumptions based on that.   
For example, a lot of C++/java code is written *just in case* an object is  
shared.  With D, the hope is that you are *guaranteed* that it is shared  
or not, so you can optimize your code that way (i.e. use a lock or not).   
As long as the possibility exists that code not marked as shared can be  
easily shared without a cast, we cannot make those assumptions.

I think it should be filed as a bug, but I'm not sure if someone's already  
reported it.

>
> I have never dealt with bugzilla but I will try to figure out how to do  
> what you
> ask :)

Go to http://d.puremagic.com/issues/enter_bug.cgi

You will have to create a user before filing a bug, but bugzilla is  
relatively straightforward to use.

-Steve
Top | Discussion index | About this forum | D home