Here's just one article identifying what many of us already know ~ to more easily take advantage of the growing multicore trend, one needs tools with a bent towards such designs: http://theinq.net/?article=27247

Multicore is happening. Take a look here http://arstechnica.com/news.ars/post/20041106-4377.html, here http://www.realworldtech.com/page.cfm?ArticleID=RWT021005084318, here http://www.realworldtech.com/page.cfm?ArticleID=RWT102405055354 and possibly here http://arstechnica.com/articles/paedia/cpu/xbox360-2.ars/7.

We had a (very) brief discussion about 18 months ago as to what D might do to support such environments, and to perhaps leapfrog other contemporary languages in this regard. I say 'contemporary' since Occam http://en.wikipedia.org/wiki/Occam_programming_language was designed from the ground-up to support a model of distributed parallel computation (it failed to prosper due partly to a tight association with the Transputer)

What do you think D might do in this regard? Are there perhaps some notions from Occam (and/or Ada), beyond simple contention-controls, worth pursuing? Is there something at the language-level that might make notable differences vis-a-vis debugging parallel systems?

- Kris

In article <djp91p$g2r$1@digitaldaemon.com>, Kris says...
>
>We had a (very) brief discussion about 18 months ago as to what D might do to support such environments, and to perhaps leapfrog other contemporary languages in this regard. I say 'contemporary' since Occam http://en.wikipedia.org/wiki/Occam_programming_language was designed from the ground-up to support a model of distributed parallel computation (it failed to prosper due partly to a tight association with the Transputer)
>
>What do you think D might do in this regard? Are there perhaps some notions from Occam (and/or Ada), beyond simple contention-controls, worth pursuing? Is there something at the language-level that might make notable differences vis-a-vis debugging parallel systems?

There was a rather lengthy thread on this in comp.arch a few months ago:

http://groups.google.com/group/comp.arch/browse_frm/thread/3d2139bf82f6298e/92c0dbef8aaac3b4#92c0dbef8aaac3b4

Cilk may be worth looking at as well, as it's a parallel version of C:

http://supertech.lcs.mit.edu/cilk/

I think D is already very well suited for common parallel programming techniques as it supports inline assembler, language-level threads, the volatile statement, and it has garbage collection.  These features allow for safe, language complaint two phase update, lock free, and other populalr techniques... and things will be even better once we have stack-based classes.

Beyond that, I'll have to give it some thought.  I think Cilk may be a good reference for desirable features, but I haven't gotten further with it than skimming the docs :-)


Sean

> Cilk may be worth looking at as well, as it's a parallel version of C:
> 
> http://supertech.lcs.mit.edu/cilk/

and corn: http://corn.telefonia.pl/

i like the

{# // <-- threaded
  func1(); // thread1
  func2(); // thread2
}

syntax very much

In article <djrl39$1l0m$1@digitaldaemon.com>, dennis luehring says...
>
>> Cilk may be worth looking at as well, as it's a parallel version of C:
>> 
>> http://supertech.lcs.mit.edu/cilk/
>
>and corn: http://corn.telefonia.pl/
>
>i like the
>
>{# // <-- threaded
>   func1(); // thread1
>   func2(); // thread2
>}
>
>syntax very much

Ah, that's like "parbegin" and "parend" that I was introduced to in an OS design class, though IIRC those were for process-level concurrency--they certainly beat the heck out of fork() though :-)


Sean

Thanks for the links, Sean. That forum had some interesting comments ~ especially regarding transactable execution.

D certainly does has a good range of primitives for thread-support, though some might conceivably be exposed in a somewhat simpler (and perhaps smarter) fashion than they currently are. But, that aside, I was kinda' thinking more along the lines of debugging assistance ~ a known bugbear of multithreading development.

For example, how about the compiler supporting some kind of read/write coloring, to indicate potential read/write conflicts between threads? That would be incredibly useful, and not hard for the compiler to identify, assuming it had some kind of PAR support to recognize threaded sections ... that is ~ once the compiler has some basic knowledge of what the programmer is attempting, it can provide seriously powerful diagnostics.

I suspect there's relatively few of us who have notable experience with multi-threading, as this article asserts (MS architect says "developers ill-prepared for new CPUs") http://arstechnica.com/news.ars/post/20051027-5492.html.

Is that the case? How about an informal poll of the D community:

===============

1) Do you have reasonable experience with the problems associated with MT, and feel reasonably comfortable with that development model?

2) Assuming it were financially attractive, could you make what you feel to be educated & valid assertions about how to successfully migrate a 'traditional' application to take full advantage of the new breed of CPU's?

3) Do you know anyone else who could handle #2? How many others?





"Sean Kelly" <sean@f4.ca> wrote in message news:djppmg$12om$1@digitaldaemon.com...
> In article <djp91p$g2r$1@digitaldaemon.com>, Kris says...
>>
>>We had a (very) brief discussion about 18 months ago as to what D might do to support such environments, and to perhaps leapfrog other contemporary languages in this regard. I say 'contemporary' since Occam http://en.wikipedia.org/wiki/Occam_programming_language was designed from the ground-up to support a model of distributed parallel computation (it failed to prosper due partly to a tight association with the Transputer)
>>
>>What do you think D might do in this regard? Are there perhaps some
>>notions
>>from Occam (and/or Ada), beyond simple contention-controls, worth
>>pursuing?
>>Is there something at the language-level that might make notable
>>differences
>>vis-a-vis debugging parallel systems?
>
> There was a rather lengthy thread on this in comp.arch a few months ago:
>
> http://groups.google.com/group/comp.arch/browse_frm/thread/3d2139bf82f6298e/92c0dbef8aaac3b4#92c0dbef8aaac3b4
>
> Cilk may be worth looking at as well, as it's a parallel version of C:
>
> http://supertech.lcs.mit.edu/cilk/
>
> I think D is already very well suited for common parallel programming
> techniques
> as it supports inline assembler, language-level threads, the volatile
> statement,
> and it has garbage collection.  These features allow for safe, language
> complaint two phase update, lock free, and other populalr techniques...
> and
> things will be even better once we have stack-based classes.
>
> Beyond that, I'll have to give it some thought.  I think Cilk may be a
> good
> reference for desirable features, but I haven't gotten further with it
> than
> skimming the docs :-)
>
>
> Sean
>
> 

On Sun, 30 Oct 2005 13:57:01 -0800, Kris <fu@bar.com> wrote:
> 1) Do you have reasonable experience with the problems associated with MT, and feel reasonably comfortable with that development model?

Yes. 4+ years developing a multi-threaded email server for windows, linux, freebsd, solaris (x86 and sparc), mac osx, etc. 7+ years in that area (web/email server) of development i.e. (single-threaded) web cgi support products and modules for use with the email server. 2+ years doing GUI style work (inherently multi-threaded), also support products for the email server.

> 2) Assuming it were financially attractive, could you make what you feel to be educated & valid assertions about how to successfully migrate a
> 'traditional' application to take full advantage of the new breed of CPU's?

I believe so.

> 3) Do you know anyone else who could handle #2? How many others?

My boss and 4 or 5 (ex-)workmates.

Regan


> "Sean Kelly" <sean@f4.ca> wrote in message
> news:djppmg$12om$1@digitaldaemon.com...
>> In article <djp91p$g2r$1@digitaldaemon.com>, Kris says...
>>>
>>> We had a (very) brief discussion about 18 months ago as to what D might do
>>> to support such environments, and to perhaps leapfrog other contemporary
>>> languages in this regard. I say 'contemporary' since Occam
>>> http://en.wikipedia.org/wiki/Occam_programming_language was designed from
>>> the ground-up to support a model of distributed parallel computation (it
>>> failed to prosper due partly to a tight association with the Transputer)
>>>
>>> What do you think D might do in this regard? Are there perhaps some
>>> notions
>>> from Occam (and/or Ada), beyond simple contention-controls, worth
>>> pursuing?
>>> Is there something at the language-level that might make notable
>>> differences
>>> vis-a-vis debugging parallel systems?
>>
>> There was a rather lengthy thread on this in comp.arch a few months ago:
>>
>> http://groups.google.com/group/comp.arch/browse_frm/thread/3d2139bf82f6298e/92c0dbef8aaac3b4#92c0dbef8aaac3b4
>>
>> Cilk may be worth looking at as well, as it's a parallel version of C:
>>
>> http://supertech.lcs.mit.edu/cilk/
>>
>> I think D is already very well suited for common parallel programming
>> techniques
>> as it supports inline assembler, language-level threads, the volatile
>> statement,
>> and it has garbage collection.  These features allow for safe, language
>> complaint two phase update, lock free, and other populalr techniques...
>> and
>> things will be even better once we have stack-based classes.
>>
>> Beyond that, I'll have to give it some thought.  I think Cilk may be a
>> good
>> reference for desirable features, but I haven't gotten further with it
>> than
>> skimming the docs :-)
>>
>>
>> Sean
>>
>>
>
>

(PS: the tone here might appear aggressive, but wasn't meant as such.)

Kris wrote:
> 1) Do you have reasonable experience with the problems associated with MT, and feel reasonably comfortable with that development model?
> 
Yes. Apart from critical sections, race conditions, and such, MT really
is not hard. As long as you manage to stay on top of what you yourself
are doing. Case in point: somewhere in the '90s, I organized a seminar
for top-level Java programmers, in Lapland. In my spare time, I attended
some of the more interesting lectures.

One of the points I never will forget was, that an object instance, with its current state, may have several simultaneous threads executing its code -- within its current state.

I didn't sleep the next night. But ever since, I've felt comfortable
with MT.

> 2) Assuming it were financially attractive, could you make what you feel to be educated & valid assertions about how to successfully migrate a 'traditional' application to take full advantage of the new
>  breed of CPU's?

No. But that's mainly because I have a hard time understanding what one
vs. two (or more) processors have to do with this.

On the conceptual level, that is. Of course (if this were a job
interview) I could start about spreading the work load between a "ui
process" and a "controller process" or some such BS.

But from a programmer's point of view, you either have MT or you don't.
D has it right in the language, and even if it weren't, you could rather
easily implement cooperative MT yourself, in D. Of course, this isn't
preemptive multitasking, but that was not asked for.

Then again, preemptive multitasking, as presented by the operating
system, is heavy compared with (self-made) cooperative multitasking.
Therefore, they should be used in dissimilar contexts.

---

As to "migrate a 'traditional' application to take full advantage of the
new breed of CPUs", I'd like to be as arrogant as to say, the issue is
way broader than can be asked in one sentence. It's not like there's a
known (or even unknown) receipe on how to do it. The 'traditional'
applications are diverse enough to preclude any single method of
conversion.

(Examples: A traditional app to count vehicles on the Bay Bridge, in
real time, getting input only from an under-road inductance loop, and
differentiating between regular cars and utility vehicles. A bank
back-room application monitoring input and output of cashiers' bought
and sold notes in a currency exchange office. Any old mainframe app
designed to talk with hundreds of VT-100 terminals, converted to be used
with Internet browsers instead. (Or was it a new in-between server,
serving the browsers and appearing to the mainframe as hundreds of VT
terminals?))

> 3) Do you know anyone else who could handle #2? How many others?

Disregarding my own answer to #2, and assuming I guess what you were
asking, my answer would be 3-6 people I know. But then, they aren't
exactly average programmers either.

----

So where does this leave us?

I think you might rephrase the question. First of all, to more specific,
and secondly, more to what you're after. For example, early in your
post, you mention "transactable execution". This naturally biases the
answers towards database related thinking -- right?

Which again raises the question: are we charting things that should be
in the language, or are we charting what should "sort of" be included in
"the eventual standard distribution of D"?

Georg Wrede wrote:
> (PS: the tone here might appear aggressive, but wasn't meant as such.)
> 
> Kris wrote:
> 
>> 1) Do you have reasonable experience with the problems associated with MT, and feel reasonably comfortable with that development model?
>>
> Yes. Apart from critical sections, race conditions, and such, MT really
> is not hard. As long as you manage to stay on top of what you yourself
> are doing.

I'd add deadlock resistance to the list of difficult MT problems.  This by itself is one of the strongest arguments for lock-free concurrency IMO.

>> 2) Assuming it were financially attractive, could you make what you feel to be educated & valid assertions about how to successfully migrate a 'traditional' application to take full advantage of the new
>>  breed of CPU's?
> 
> No. But that's mainly because I have a hard time understanding what one
> vs. two (or more) processors have to do with this.

The hardware layer doesn't matter as such.  The primary reason this is a concern with respect to new CPUs is that clock speed is beginning to plateau, so performance scaling is moving away from clock speed towards parallelization.  This has been a growing concern among the C++ and MT communities for a while now.

> But from a programmer's point of view, you either have MT or you don't.
> D has it right in the language, and even if it weren't, you could rather
> easily implement cooperative MT yourself, in D. Of course, this isn't
> preemptive multitasking, but that was not asked for.

I agree with Kris that it's worth considering other language features to support or facilitate concurrent programming.  While concurrency is a fairly well understood problem by some, the support for it in popular procedural languages is pretty weak.  Java is better off than most, and it doesn't even go much further than "synchronized" and a memory model definition.  D may do quite well to have some additional concurrent programming features available (similar to those in Cilk, for example).

> As to "migrate a 'traditional' application to take full advantage of the
> new breed of CPUs", I'd like to be as arrogant as to say, the issue is
> way broader than can be asked in one sentence. It's not like there's a
> known (or even unknown) receipe on how to do it. The 'traditional'
> applications are diverse enough to preclude any single method of
> conversion.

It would be nice if there were a fairly systematic way for doing this, but I agree that there may not be.  The Intel compiler, for example, can parallelize some code blocks automatically, but this amounts to a very small portion of the code (only loops IIRC).

> Which again raises the question: are we charting things that should be
> in the language, or are we charting what should "sort of" be included in
> "the eventual standard distribution of D"?

A bit of both, I think.


Sean

Good points. Some comments inline:

"Georg Wrede" <georg.wrede@nospam.org> ...
> (PS: the tone here might appear aggressive, but wasn't meant as such.)
>
> Kris wrote:
>> 1) Do you have reasonable experience with the problems associated with MT, and feel reasonably comfortable with that development model?
>>
> Yes. Apart from critical sections, race conditions, and such, MT really is not hard. As long as you manage to stay on top of what you yourself are doing. <snip>

That's a large part of what the linked article was getting at. If a compiler (or other tool) could help more in this regard it could be a big win. The Concur project at MS is apparently hoping to do just that. The write-conflict coloring I mentioned earlier would be an example of such a win: it would identify all possible read/write & write/write conflicts, thereby supporting a developer in their quest to "stay on top of things" (as you put it so well).


>> 2) Assuming it were financially attractive, could you make what you feel
>> to be educated & valid assertions about how to successfully migrate a
>> 'traditional' application to take full advantage of the new
>>  breed of CPU's?
>
> No. But that's mainly because I have a hard time understanding what one vs. two (or more) processors have to do with this.

Only that to take full advantage of the lastest CPUs, one likely has to become familiar with MT. Whereas before, that was rarely a concern for a large swathe of application types (and programmers).

<snip>
> As to "migrate a 'traditional' application to take full advantage of the new breed of CPUs", I'd like to be as arrogant as to say, the issue is way broader than can be asked in one sentence. It's not like there's a known (or even unknown) receipe on how to do it. The 'traditional' applications are diverse enough to preclude any single method of conversion.

Yes; you are quite right. I should have been more specific, and narrow the scope to those applications that stood to gain most from an ever-increasing single-core CPU throughput (MHz is one vector). Such apps rarely deal with MT issues directly (except for the occasional "execute later" API present in certain GUIs). Now that the performance gains are slanted more towards multi-core, developers will perhaps /have/ to face the MT hurdle. Exactly what that linked article was getting at.

On the other hand, I wanted to leave the question open to interpretation. There's many ways to skin the MT cat, as you note; but only those with prior experience would likely be familiar with the concern vectors.

>> 3) Do you know anyone else who could handle #2? How many others?
>
> Disregarding my own answer to #2, and assuming I guess what you were asking, my answer would be 3-6 people I know. But then, they aren't exactly average programmers either.

I think that last sentence identifies an important point. If D could help those 'average' programmers succeed by, for example, pointing out in the code where there might be trouble, then it would stand head-and-shoulders above any other language in this respect (and I'm guessing that this kind of thing is an aspect of the Concur project?).

<snip>
> I think you might rephrase the question. First of all, to more specific, and secondly, more to what you're after. For example, early in your post, you mention "transactable execution". This naturally biases the answers towards database related thinking -- right?

By "transactable execution" I was reflecting the ability to roll-back the effect of execution ~ any execution ~ within certain contraints. There's been a lot of thought in that area over the last 30 years (or whatever) since it would allow parallel /speculative/ execution at a much higher/large-grained level than is currently possible at the die/branch level. Traditionally, that kind of thing has been built into DBs to gain ACID properties (atomic, consistent, isolated, durable), but it has other advantages too. Still, that's all rather speculative at this point, and doesn't really have much bearing on the diagnostic type of tools I'd like to see. I mentioned it only as reference to the links posted by Sean.

> Which again raises the question: are we charting things that should be in the language, or are we charting what should "sort of" be included in "the eventual standard distribution of D"?

Specifically, I was hoping to concentrate on what language attributes (as opposed to library attributes) could help developers in the MT world. Being able to extract features from the code (such as parallel execution) can lead to the coloring example I mentioned. Again, this is just speculation.

I have several other half-baked notions in mind, but they don't really belong in this "poll" thread so I'll just briefly mention a couple of them:

a) A little more than the simple (yet powerful) 'synchronized' keyword; perhaps some of the CS constructs that Doug Lea implements & documents so well? Having some related extractable features at the language level would likely aid diagnostics in a similar manner to a PAR type construct, since they tend to go hand-in-hand.

b) A multi-instance GC. Single-pool allocators, and a GC in particular, can be a serious bottleneck for multi-threaded apps. More so than single-threaded since there's now contention and stalling in the allocator, and often severe contention in the collector (i.e. halts all running threads). I tend to avoid allocations at all cost within a server; precisely for this reason (e.g. the Mango HTTP and Servlet servers do not allocate any per-request memory, once they have warmed up).

Thanks for the feedback, Georg.

Kris wrote:
> Thanks for the links, Sean. That forum had some interesting comments ~ especially regarding transactable execution.

I think so too.  For anyone interested, the post that made me think of the thread in the first place is here (154. Andy Glew):

http://groups.google.com/group/comp.arch/browse_frm/thread/3d2139bf82f6298e/92c0dbef8aaac3b4#92c0dbef8aaac3b4

It does a rather good job of summing up most of the issues.  Now on to the poll!

> 1) Do you have reasonable experience with the problems associated with MT, and feel reasonably comfortable with that development model?

Yes.

> 2) Assuming it were financially attractive, could you make what you feel to be educated & valid assertions about how to successfully migrate a 'traditional' application to take full advantage of the new breed of CPU's?

I believe so, though I'd have to think about whether I could suggest a mechanical approach for doing so without being overly general.

> 3) Do you know anyone else who could handle #2? How many others?

Of my current coworkers, I can think of of one person.  I haven't worked with a terribly large number of people who have much MT programming experience or who know architecture issues well enough to compensate.


Sean