I have been "playing" latetly with std.concurrency and core.thread. I like both, but they are entirely different approaches to concurrency.

std.concurrency: Uses message passing. Does not allow passing of mutable types.
core.thread: Essentially the same as the approach taken by Java.

1) Is one favoured over the other? I guess std.concurrency would be, because it uses messages (and I recently read somewhere that immutability + message passing = good thing). Well, my problem about is, how can send mutable data to another thread? _Can_ I do it? If not, how am I supposed to share data between them? Not everything can be immutable right? For example I would like to have a thread calculate the sum of the first half of an array while another thread would calculate the other half, and I could add the two results at the end.

2) What about core.thread? To do proper sync between threads in core.thread, semaphore, mutexes and stuff like that are needed. Are those "good practice" in D?

3) I have also read a bit about syncronized classes and shared classes/structs, altough I didn't understand it too much (well I didn't try too much to be honest).

For all those appoaches, which is the preffered?
For all those appoaches, which is the preffered for game programming?

Thank you.

Minas Mina wrote:

> I have been "playing" latetly with std.concurrency and core.thread. I like both, but they are entirely different approaches to concurrency.
> 
> std.concurrency: Uses message passing. Does not allow passing of
> mutable types.
> core.thread: Essentially the same as the approach taken by Java.
> 
> 1) Is one favoured over the other? I guess std.concurrency would be, because it uses messages (and I recently read somewhere that immutability + message passing = good thing).
I've not really got around to doing any concurrent programming in anger with D yet but from what I've read message passing is the preferred method. Fortunately when it comes to answering your question the concurrency chapter for The D Programming Language book (TDPL) was made freely available as a preview http://www.informit.com/articles/article.aspx?p=1609144. Hopefully that contains your answers :).

> Well, my problem
> about is, how can send mutable data to another thread? _Can_ I do
> it? If not, how am I supposed to share data between them? Not
> everything can be immutable right?
From what I gather this is where the shared keyword typically comes into play which should be detailed in the url above.

> For example I would like to
> have a thread calculate the sum of the first half of an array
> while another thread would calculate the other half, and I could
> add the two results at the end.
For this task there is another concurrency module called std.parallelism that is designed for this type of thing (afaik). http://dlang.org/phobos/std_parallelism.html

> 
> 2) What about core.thread? To do proper sync between threads in core.thread, semaphore, mutexes and stuff like that are needed. Are those "good practice" in D?
> 
> 3) I have also read a bit about syncronized classes and shared classes/structs, altough I didn't understand it too much (well I didn't try too much to be honest).
> 
> For all those appoaches, which is the preffered?
My observation is std.concurrency is being pushed as the preferred default method.
> For all those appoaches, which is the preffered for game programming?
> 
> Thank you.

On Thu, 28 Jun 2012 00:34:50 +0200, Minas Mina wrote:

> I have been "playing" latetly with std.concurrency and core.thread. I like both, but they are entirely different approaches to concurrency.
> 
> std.concurrency: Uses message passing. Does not allow passing of mutable
> types.
> core.thread: Essentially the same as the approach taken by Java.
> 
> 1) Is one favoured over the other? I guess std.concurrency would be, because it uses messages (and I recently read somewhere that immutability + message passing = good thing). Well, my problem about is, how can send mutable data to another thread? _Can_ I do it? If not, how am I supposed to share data between them? Not everything can be immutable right? For example I would like to have a thread calculate the sum of the first half of an array while another thread would calculate the other half, and I could add the two results at the end.
> 
> 2) What about core.thread? To do proper sync between threads in core.thread, semaphore, mutexes and stuff like that are needed. Are those "good practice" in D?
> 
> 3) I have also read a bit about syncronized classes and shared classes/structs, altough I didn't understand it too much (well I didn't try too much to be honest).
> 
> For all those appoaches, which is the preffered?
> For all those appoaches, which is the preffered for game programming?
> 
> Thank you.

Concurrency and parallelism are two related, but different concepts. Concurrency is when you have different computations conducted simultaneously. An example is a modern game engine: you may have a rendering thread, a physics simulation, and a networking stack all operating at the same time on different cores. Concurrency allows (and generally assumes) communication between these parts as they need to share information. Message-passing is, as far as I know, pretty much the only truly correct way of doing this.

Parallelism is, by contrast, the decomposing of a computation into smaller chunks which do not depend on each other. A trivial example is incrementing all integers in an array by one--the result of the increment computation only relies on the integer being worked on, so the computations may be performed in parallel and the results collected.

The std.concurrency and std.parallelism thus exist to solve two different problems and are more of an apples-oranges situation.

Justin

On Wednesday, 27 June 2012 at 22:34:51 UTC, Minas Mina wrote:
> I have been "playing" latetly with std.concurrency and core.thread. I like both, but they are entirely different approaches to concurrency.
>

Aren't they great? ;)

> std.concurrency: Uses message passing. Does not allow passing of mutable types.
> core.thread: Essentially the same as the approach taken by Java.
>
> 1) Is one favoured over the other? I guess std.concurrency would be, because it uses messages (and I recently read somewhere that immutability + message passing = good thing).

Personally I favor std.concurrency, in many cases I find it faster.

> Well, my problem about is, how can send mutable data to another thread? _Can_ I do it? If not, how am I supposed to share data between them? Not everything can be immutable right?

Mutable value types can be sent easily, because they are copied. You share data with shared(T) types, e.g. shared(Object) or shared(int[]).

However, I try to avoid shared data by keeping it in single threads, and having other threads access parts with message passing.

One paradigm is to isolate data into "thread objects," threads which have non-shared data on the stack, and have other threads send GetData messages and get ReturnedData messages.

> For example I would like to have a thread calculate the sum of the first half of an array while another thread would calculate the other half, and I could add the two results at the end.
>

Check out this:
http://dpaste.dzfl.pl/0f7ccb79

Note that for algorithms like this, you should check out std.parallelism, it's more high-level than std.concurrency.

> 2) What about core.thread? To do proper sync between threads in core.thread, semaphore, mutexes and stuff like that are needed. Are those "good practice" in D?
>

Part of D's philosophy is "you can get low-level and dirty if you want, but the easy/safe way should also be the fast way." core.thread is there for you, but std.concurrency is easier. Note that std.concurrency is written on top of core.thread.

I definitely prefer std.concurrency, but I don't write CPU-intensive stuff anyway, so I'm no expert.

> 3) I have also read a bit about syncronized classes and shared classes/structs, altough I didn't understand it too much (well I didn't try too much to be honest).
>

Objects of a synchronized class automatically lock/unlock every method call.

You don't share stack structs, though maybe shared(StructType)*.

> For all those appoaches, which is the preffered?
> For all those appoaches, which is the preffered for game programming?
>
> Thank you.

Hope I've helped.
NMS

On 06/27/2012 04:05 PM, Adam Burton wrote:

>> For example I would like to
>> have a thread calculate the sum of the first half of an array
>> while another thread would calculate the other half, and I could
>> add the two results at the end.
> For this task there is another concurrency module called std.parallelism
> that is designed for this type of thing (afaik).
> http://dlang.org/phobos/std_parallelism.html

TDPL predates std.parallelism, so it's not in the book. Although the module's documentation is straightforward, I have the following page which may be more accessible for some:

  http://ddili.org/ders/d.en/parallelism.html

Clicking [Next] on that pages goes to Message Passing Concurrency:

  http://ddili.org/ders/d.en/concurrency.html

Ali

P.S. Both of those pages are incomplete in some ways, which I will address soon.

You all helped, thank you :)

I will read the concurrency part tomorrow!

On Wednesday, 27 June 2012 at 23:50:17 UTC, Ali Çehreli wrote:
> On 06/27/2012 04:05 PM, Adam Burton wrote:
>
> >> For example I would like to
> >> have a thread calculate the sum of the first half of an array
> >> while another thread would calculate the other half, and I
> could
> >> add the two results at the end.
> > For this task there is another concurrency module called
> std.parallelism
> > that is designed for this type of thing (afaik).
> > http://dlang.org/phobos/std_parallelism.html
>
> TDPL predates std.parallelism, so it's not in the book. Although the module's documentation is straightforward, I have the following page which may be more accessible for some:
>
>   http://ddili.org/ders/d.en/parallelism.html
>
> Clicking [Next] on that pages goes to Message Passing Concurrency:
>
>   http://ddili.org/ders/d.en/concurrency.html
>
> Ali
>
> P.S. Both of those pages are incomplete in some ways, which I will address soon.

Much better than the library reference :)

yeah, I didn't knew std.parallelism was so easy to use. Speed up was really big for minimal effort.

On Thu, 28 Jun 2012 00:34:50 +0200, Minas Mina wrote:

> I have been "playing" latetly with std.concurrency and core.thread. I like both, but they are entirely different approaches to concurrency.
> 
> std.concurrency: Uses message passing. Does not allow passing of mutable
> types.
> core.thread: Essentially the same as the approach taken by Java.
> 
> 1) Is one favoured over the other? I guess std.concurrency would be, because it uses messages (and I recently read somewhere that immutability + message passing = good thing). Well, my problem about is, how can send mutable data to another thread? _Can_ I do it? If not, how am I supposed to share data between them? Not everything can be immutable right? For example I would like to have a thread calculate the sum of the first half of an array while another thread would calculate the other half, and I could add the two results at the end.
> 
> 2) What about core.thread? To do proper sync between threads in core.thread, semaphore, mutexes and stuff like that are needed. Are those "good practice" in D?
> 
> 3) I have also read a bit about syncronized classes and shared classes/structs, altough I didn't understand it too much (well I didn't try too much to be honest).
> 
> For all those appoaches, which is the preffered?
> For all those appoaches, which is the preffered for game programming?
> 
> Thank you.


Both std.concurrency and std.parallelism need core.thread in order to do what they do...


-- 
Dejan Lekic
  mailto:dejan.lekic(a)gmail.com
  http://dejan.lekic.org