I'm still a bit fuzzy on how to create immutable data and when said data is safe to share across threads.

To begin with, there is .idup, string literals and constructors of immutable objects. Those can be safely shared, no problem, right?

But then, the spec mentions casting to immutable is ok if you do not have any mutable aliases left (http://www.digitalmars.com/d/2.0/const3.html):

char[] s = ...;
immutable(char)[] p = cast(immutable)s.dup; // ok, unique reference

I do not understand how that works with sharing. Since immutable data is implicitly shared but the data that p refers to is allocated on the TLS, how can you share this? I always thought that threads do not have access to each others TLS at all?

Finally a practical question: when you have a data structure that is too complex to create in a constructor, want to create it and then make it immutable, what is the current way to go about this? If it is created by one thread, would it be ok to type it as __gshared, cast to immutable and then send a message to other threads? Is __gshared required in this case?

Lutger <lutger.blijdestijn@gmail.com> wrote:

> char[] s = ...;
> immutable(char)[] p = cast(immutable)s.dup; // ok, unique reference
>
> I do not understand how that works with sharing. Since immutable data is
> implicitly shared but the data that p refers to is allocated on the TLS, how can
> you share this? I always thought that threads do not have access to each others
> TLS at all?

Only p itself is in TLS - the pointed-to data is on the heap.


> Finally a practical question: when you have a data structure that is too complex
> to create in a constructor, want to create it and then make it immutable, what
> is the current way to go about this? If it is created by one thread, would it be
> ok to type it as __gshared, cast to immutable and then send a message to other
> threads? Is __gshared required in this case?

Immutable global state may be instantiated from non-immutable data in
module constructors. I believe that is the canonical way.

-- 
Simen

Simen kjaeraas wrote:

> Lutger <lutger.blijdestijn@gmail.com> wrote:
> 
>> char[] s = ...;
>> immutable(char)[] p = cast(immutable)s.dup; // ok, unique reference
>>
>> I do not understand how that works with sharing. Since immutable data is
>> implicitly shared but the data that p refers to is allocated on the TLS,
>> how can
>> you share this? I always thought that threads do not have access to each
>> others
>> TLS at all?
> 
> Only p itself is in TLS - the pointed-to data is on the heap.

Aha, thanks. I have made a conceptual diagram to help understand this, would you care to take a look and confirm whether this is correct or not?

http://picasaweb.google.com/Lutger.blijdestijn/Illustration#5519337139518828386

I hope it explains itself, the edges reflect the types and the nodes the memory
storage.

> 
>> Finally a practical question: when you have a data structure that is too
>> complex
>> to create in a constructor, want to create it and then make it
>> immutable, what
>> is the current way to go about this? If it is created by one thread,
>> would it be
>> ok to type it as __gshared, cast to immutable and then send a message to
>> other
>> threads? Is __gshared required in this case?
> 
> Immutable global state may be instantiated from non-immutable data in module constructors. I believe that is the canonical way.
> 

Sometimes this is not possible. For example if you want to create a data structure from user input once, then use it read-only for the rest of the program.

On Tue, 21 Sep 2010 05:17:10 -0400, Lutger <lutger.blijdestijn@gmail.com> wrote:

> I'm still a bit fuzzy on how to create immutable data and when said data is safe
> to share across threads.
>
> To begin with, there is .idup, string literals and constructors of immutable
> objects. Those can be safely shared, no problem, right?

idup is not safe (along with dup), it is the equivalent of a cast right now.  You must still ensure the data has no aliases, or the data idup'd consists of all value types.

See this bug report: http://d.puremagic.com/issues/show_bug.cgi?id=3550

>
> But then, the spec mentions casting to immutable is ok if you do not have any
> mutable aliases left (http://www.digitalmars.com/d/2.0/const3.html):
>
> char[] s = ...;
> immutable(char)[] p = cast(immutable)s.dup; // ok, unique reference

This used to be the only valid way to get immutable data besides idup.

> I do not understand how that works with sharing. Since immutable data is
> implicitly shared but the data that p refers to is allocated on the TLS, how can
> you share this? I always thought that threads do not have access to each others
> TLS at all?

TLS is only for global and static variables.  Data on the heap is not TLS.

-Steve

Lutger <lutger.blijdestijn@gmail.com> wrote:

> Aha, thanks. I have made a conceptual diagram to help understand this, would you
> care to take a look and confirm whether this is correct or not?
>
> http://picasaweb.google.com/Lutger.blijdestijn/Illustration#5519337139518828386
>
> I hope it explains itself, the edges reflect the types and the nodes the memory
> storage.

That looks very much correct. I'm somewhat confused by the 'static data'
part, as it connected only to one thread, and static data is in TLS, as far
as I know.


>> Immutable global state may be instantiated from non-immutable data in
>> module constructors. I believe that is the canonical way.
>>
>
> Sometimes this is not possible. For example if you want to create a data
> structure from user input once, then use it read-only for the rest of the
> program.

Indeed. In that case, would a shared pointer/array to immutable data work for
you? In such a case, you are free to have local (non-shared) pointers to the
very same data, and thus have no need for locking (which might be needed to
dereference the shared pointer).


-- 
Simen

Simen kjaeraas wrote:

> Lutger <lutger.blijdestijn@gmail.com> wrote:
> 
>> Aha, thanks. I have made a conceptual diagram to help understand this,
>> would you
>> care to take a look and confirm whether this is correct or not?
>>
>> 
http://picasaweb.google.com/Lutger.blijdestijn/Illustration#5519337139518828386
>>
>> I hope it explains itself, the edges reflect the types and the nodes the
>> memory
>> storage.
> 
> That looks very much correct. I'm somewhat confused by the 'static data' part, as it connected only to one thread, and static data is in TLS, as far as I know.

Good, things start falling into places. About the static part, I realized it too
late, I meant the readonly data (like .rodata) part of the executable where
string literals and such are dumped. The missing connection was for
uncluttering.

>>> Immutable global state may be instantiated from non-immutable data in module constructors. I believe that is the canonical way.
>>>
>>
>> Sometimes this is not possible. For example if you want to create a data structure from user input once, then use it read-only for the rest of the program.
> 
> Indeed. In that case, would a shared pointer/array to immutable data work
> for
> you? In such a case, you are free to have local (non-shared) pointers to
> the
> very same data, and thus have no need for locking (which might be needed to
> dereference the shared pointer).

I would love to do something like that:

1. create a thread that makes a complex data structure from user input
2. said thread dies when it is finished, but gives a message back to the parent
consisting of a pointer to the data that is now cast as immutable.

Now all mutable aliases are destroyed and the program can enjoy doing parellel work on the data safely, avoiding any locking. It looks like this is possible.

Steven Schveighoffer wrote:

> On Tue, 21 Sep 2010 05:17:10 -0400, Lutger <lutger.blijdestijn@gmail.com> wrote:
> 
>> I'm still a bit fuzzy on how to create immutable data and when said data
>> is safe
>> to share across threads.
>>
>> To begin with, there is .idup, string literals and constructors of
>> immutable
>> objects. Those can be safely shared, no problem, right?
> 
> idup is not safe (along with dup), it is the equivalent of a cast right now.  You must still ensure the data has no aliases, or the data idup'd consists of all value types.
> 
> See this bug report: http://d.puremagic.com/issues/show_bug.cgi?id=3550

Good to know, I voted for it. There are quite a few bugs related to this topic, looks like I have some more studying to do.