View mode: basic / threaded / horizontal-split · Log in · Help
June 27, 2012
Concurrency in D
I have been "playing" latetly with std.concurrency and 
core.thread. I like both, but they are entirely different 
approaches to concurrency.

std.concurrency: Uses message passing. Does not allow passing of 
mutable types.
core.thread: Essentially the same as the approach taken by Java.

1) Is one favoured over the other? I guess std.concurrency would 
be, because it uses messages (and I recently read somewhere that 
immutability + message passing = good thing). Well, my problem 
about is, how can send mutable data to another thread? _Can_ I do 
it? If not, how am I supposed to share data between them? Not 
everything can be immutable right? For example I would like to 
have a thread calculate the sum of the first half of an array 
while another thread would calculate the other half, and I could 
add the two results at the end.

2) What about core.thread? To do proper sync between threads in 
core.thread, semaphore, mutexes and stuff like that are needed. 
Are those "good practice" in D?

3) I have also read a bit about syncronized classes and shared 
classes/structs, altough I didn't understand it too much (well I 
didn't try too much to be honest).

For all those appoaches, which is the preffered?
For all those appoaches, which is the preffered for game 
programming?

Thank you.
June 27, 2012
Re: Concurrency in D
Minas Mina wrote:

> I have been "playing" latetly with std.concurrency and
> core.thread. I like both, but they are entirely different
> approaches to concurrency.
> 
> std.concurrency: Uses message passing. Does not allow passing of
> mutable types.
> core.thread: Essentially the same as the approach taken by Java.
> 
> 1) Is one favoured over the other? I guess std.concurrency would
> be, because it uses messages (and I recently read somewhere that
> immutability + message passing = good thing).
I've not really got around to doing any concurrent programming in anger with 
D yet but from what I've read message passing is the preferred method. 
Fortunately when it comes to answering your question the concurrency chapter 
for The D Programming Language book (TDPL) was made freely available as a 
preview http://www.informit.com/articles/article.aspx?p=1609144. Hopefully 
that contains your answers :).

> Well, my problem
> about is, how can send mutable data to another thread? _Can_ I do
> it? If not, how am I supposed to share data between them? Not
> everything can be immutable right?
From what I gather this is where the shared keyword typically comes into 
play which should be detailed in the url above.

> For example I would like to
> have a thread calculate the sum of the first half of an array
> while another thread would calculate the other half, and I could
> add the two results at the end.
For this task there is another concurrency module called std.parallelism 
that is designed for this type of thing (afaik). 
http://dlang.org/phobos/std_parallelism.html

> 
> 2) What about core.thread? To do proper sync between threads in
> core.thread, semaphore, mutexes and stuff like that are needed.
> Are those "good practice" in D?
> 
> 3) I have also read a bit about syncronized classes and shared
> classes/structs, altough I didn't understand it too much (well I
> didn't try too much to be honest).
> 
> For all those appoaches, which is the preffered?
My observation is std.concurrency is being pushed as the preferred default 
method.
> For all those appoaches, which is the preffered for game
> programming?
> 
> Thank you.
June 27, 2012
Re: Concurrency in D
On Thu, 28 Jun 2012 00:34:50 +0200, Minas Mina wrote:

> I have been "playing" latetly with std.concurrency and core.thread. I
> like both, but they are entirely different approaches to concurrency.
> 
> std.concurrency: Uses message passing. Does not allow passing of mutable
> types.
> core.thread: Essentially the same as the approach taken by Java.
> 
> 1) Is one favoured over the other? I guess std.concurrency would be,
> because it uses messages (and I recently read somewhere that
> immutability + message passing = good thing). Well, my problem about is,
> how can send mutable data to another thread? _Can_ I do it? If not, how
> am I supposed to share data between them? Not everything can be
> immutable right? For example I would like to have a thread calculate the
> sum of the first half of an array while another thread would calculate
> the other half, and I could add the two results at the end.
> 
> 2) What about core.thread? To do proper sync between threads in
> core.thread, semaphore, mutexes and stuff like that are needed. Are
> those "good practice" in D?
> 
> 3) I have also read a bit about syncronized classes and shared
> classes/structs, altough I didn't understand it too much (well I didn't
> try too much to be honest).
> 
> For all those appoaches, which is the preffered?
> For all those appoaches, which is the preffered for game programming?
> 
> Thank you.

Concurrency and parallelism are two related, but different concepts. 
Concurrency is when you have different computations conducted 
simultaneously. An example is a modern game engine: you may have a 
rendering thread, a physics simulation, and a networking stack all 
operating at the same time on different cores. Concurrency allows (and 
generally assumes) communication between these parts as they need to 
share information. Message-passing is, as far as I know, pretty much the 
only truly correct way of doing this.

Parallelism is, by contrast, the decomposing of a computation into 
smaller chunks which do not depend on each other. A trivial example is 
incrementing all integers in an array by one--the result of the increment 
computation only relies on the integer being worked on, so the 
computations may be performed in parallel and the results collected.

The std.concurrency and std.parallelism thus exist to solve two different 
problems and are more of an apples-oranges situation.

Justin
June 27, 2012
Re: Concurrency in D
On Wednesday, 27 June 2012 at 22:34:51 UTC, Minas Mina wrote:
> I have been "playing" latetly with std.concurrency and 
> core.thread. I like both, but they are entirely different 
> approaches to concurrency.
>

Aren't they great? ;)

> std.concurrency: Uses message passing. Does not allow passing 
> of mutable types.
> core.thread: Essentially the same as the approach taken by Java.
>
> 1) Is one favoured over the other? I guess std.concurrency 
> would be, because it uses messages (and I recently read 
> somewhere that immutability + message passing = good thing).

Personally I favor std.concurrency, in many cases I find it 
faster.

> Well, my problem about is, how can send mutable data to another 
> thread? _Can_ I do it? If not, how am I supposed to share data 
> between them? Not everything can be immutable right?

Mutable value types can be sent easily, because they are copied. 
You share data with shared(T) types, e.g. shared(Object) or 
shared(int[]).

However, I try to avoid shared data by keeping it in single 
threads, and having other threads access parts with message 
passing.

One paradigm is to isolate data into "thread objects," threads 
which have non-shared data on the stack, and have other threads 
send GetData messages and get ReturnedData messages.

> For example I would like to have a thread calculate the sum of 
> the first half of an array while another thread would calculate 
> the other half, and I could add the two results at the end.
>

Check out this:
http://dpaste.dzfl.pl/0f7ccb79

Note that for algorithms like this, you should check out 
std.parallelism, it's more high-level than std.concurrency.

> 2) What about core.thread? To do proper sync between threads in 
> core.thread, semaphore, mutexes and stuff like that are needed. 
> Are those "good practice" in D?
>

Part of D's philosophy is "you can get low-level and dirty if you 
want, but the easy/safe way should also be the fast way." 
core.thread is there for you, but std.concurrency is easier. Note 
that std.concurrency is written on top of core.thread.

I definitely prefer std.concurrency, but I don't write 
CPU-intensive stuff anyway, so I'm no expert.

> 3) I have also read a bit about syncronized classes and shared 
> classes/structs, altough I didn't understand it too much (well 
> I didn't try too much to be honest).
>

Objects of a synchronized class automatically lock/unlock every 
method call.

You don't share stack structs, though maybe shared(StructType)*.

> For all those appoaches, which is the preffered?
> For all those appoaches, which is the preffered for game 
> programming?
>
> Thank you.

Hope I've helped.
NMS
June 27, 2012
Re: Concurrency in D
On 06/27/2012 04:05 PM, Adam Burton wrote:

>> For example I would like to
>> have a thread calculate the sum of the first half of an array
>> while another thread would calculate the other half, and I could
>> add the two results at the end.
> For this task there is another concurrency module called std.parallelism
> that is designed for this type of thing (afaik).
> http://dlang.org/phobos/std_parallelism.html

TDPL predates std.parallelism, so it's not in the book. Although the 
module's documentation is straightforward, I have the following page 
which may be more accessible for some:

  http://ddili.org/ders/d.en/parallelism.html

Clicking [Next] on that pages goes to Message Passing Concurrency:

  http://ddili.org/ders/d.en/concurrency.html

Ali

P.S. Both of those pages are incomplete in some ways, which I will 
address soon.
June 27, 2012
Re: Concurrency in D
You all helped, thank you :)

I will read the concurrency part tomorrow!
June 28, 2012
Re: Concurrency in D
On Wednesday, 27 June 2012 at 23:50:17 UTC, Ali Çehreli wrote:
> On 06/27/2012 04:05 PM, Adam Burton wrote:
>
> >> For example I would like to
> >> have a thread calculate the sum of the first half of an array
> >> while another thread would calculate the other half, and I
> could
> >> add the two results at the end.
> > For this task there is another concurrency module called
> std.parallelism
> > that is designed for this type of thing (afaik).
> > http://dlang.org/phobos/std_parallelism.html
>
> TDPL predates std.parallelism, so it's not in the book. 
> Although the module's documentation is straightforward, I have 
> the following page which may be more accessible for some:
>
>   http://ddili.org/ders/d.en/parallelism.html
>
> Clicking [Next] on that pages goes to Message Passing 
> Concurrency:
>
>   http://ddili.org/ders/d.en/concurrency.html
>
> Ali
>
> P.S. Both of those pages are incomplete in some ways, which I 
> will address soon.

Much better than the library reference :)
June 28, 2012
Re: Concurrency in D
yeah, I didn't knew std.parallelism was so easy to use. Speed up was
really big for minimal effort.
June 29, 2012
Re: Concurrency in D
On Thu, 28 Jun 2012 00:34:50 +0200, Minas Mina wrote:

> I have been "playing" latetly with std.concurrency and core.thread. I
> like both, but they are entirely different approaches to concurrency.
> 
> std.concurrency: Uses message passing. Does not allow passing of mutable
> types.
> core.thread: Essentially the same as the approach taken by Java.
> 
> 1) Is one favoured over the other? I guess std.concurrency would be,
> because it uses messages (and I recently read somewhere that
> immutability + message passing = good thing). Well, my problem about is,
> how can send mutable data to another thread? _Can_ I do it? If not, how
> am I supposed to share data between them? Not everything can be
> immutable right? For example I would like to have a thread calculate the
> sum of the first half of an array while another thread would calculate
> the other half, and I could add the two results at the end.
> 
> 2) What about core.thread? To do proper sync between threads in
> core.thread, semaphore, mutexes and stuff like that are needed. Are
> those "good practice" in D?
> 
> 3) I have also read a bit about syncronized classes and shared
> classes/structs, altough I didn't understand it too much (well I didn't
> try too much to be honest).
> 
> For all those appoaches, which is the preffered?
> For all those appoaches, which is the preffered for game programming?
> 
> Thank you.


Both std.concurrency and std.parallelism need core.thread in order to do 
what they do...


-- 
Dejan Lekic
 mailto:dejan.lekic(a)gmail.com
 http://dejan.lekic.org
Top | Discussion index | About this forum | D home