On 3/28/2015 3:20 AM, Jonathan M Davis via Digitalmars-d-announce wrote:
> Personally, I'm not sure that much is gained in pitting Go against D
> precisely because they're so different that they're likely to appeal to
> completely different sets of people.

I also do not regard Go as a competitor to D. It's more of a competitor to Java and Ruby.

On Saturday, 28 March 2015 at 18:47:04 UTC, Walter Bright wrote:
> On 3/28/2015 3:20 AM, Jonathan M Davis via Digitalmars-d-announce wrote:
>> Personally, I'm not sure that much is gained in pitting Go against D
>> precisely because they're so different that they're likely to appeal to
>> completely different sets of people.
>
> I also do not regard Go as a competitor to D. It's more of a competitor to Java and Ruby.

I find most people I know using Go are from the python camp and either wanted static typing or faster runtime execution.

On 3/28/2015 8:41 AM, Sönke Ludwig wrote:
> Am 28.03.2015 um 15:33 schrieb Russel Winder via Digitalmars-d-announce:
>> TLS is the evil here. Anyone working with TLS is either writing an
>> operating system or doing it wrong.
>>
>
> As long as we are talking about a closed system that works exclusively on this
> fiber based concurrency model, I completely agree with you (fiber local storage
> would be fine, though).
>
> But we have the "unfortunate" situation that the language is not an isolated
> ecosystem. There are many C libraries that do thread-specific things in one way
> or another, or worse, make use of ordinary global variables without any
> protection against data races, and we simply cannot ignore that.

One solution (that seems entirely reasonable to me) is to make the droutines (i.e. "goroutines") pure. Then the TLS problem goes away. Of course, then I/O isn't possible either, but perhaps a solution can be found for that.

On Saturday, 28 March 2015 at 14:33:14 UTC, Russel Winder wrote:
> On Sat, 2015-03-28 at 12:52 +0100, Sönke Ludwig via Digitalmars-d-announce wrote:
>> […]
>> 
>> You can access TLS from an event callback just as easy as from a fiber.
> […]
>
> TLS is the evil here. Anyone working with TLS is either writing an
> operating system or doing it wrong.

I don't think it is that simple. From the purist academical parallelism POV - most likely. In practice it often can be quite the contrary, TLS is your best friend (depending on how efficient platfrom implementation is).

To get best high-load performance best strategy is to actually design applications with specific hardware traits in mind, including pre-defined amount of CPU cores and their task affinity. Computation parallelism is relatively easy, it is memory parallelism that remains a challenging task as long as you try to get rid of locking overhead, costly synchronizations and optimize cache loads. Something like moving fibers between threads is absolute disaster in this sense even if it looks like a tempting abstraction on paper. But if you prohibit that by design and maintain strict affinity between fibers and threads, using TLS allows for very nice optimizations (as it is effectively limited sharing without locking / atomics). It can be complicated to design (which is why Go choice makes sense for their target audience) but benefits are also very good.

On Sat, 2015-03-28 at 18:55 +0000, Dicebot via Digitalmars-d-announce wrote:
> 
[…]
> I don't think it is that simple. From the purist academical parallelism POV - most likely. In practice it often can be quite the contrary, TLS is your best friend (depending on how efficient platfrom implementation is).

I suggest it really is that simple even for non-academic applications…

> To get best high-load performance best strategy is to actually design applications with specific hardware traits in mind, including pre-defined amount of CPU cores and their task affinity. Computation parallelism is relatively easy, it is memory parallelism that remains a challenging task as long as you try to get rid of locking overhead, costly synchronizations and optimize cache loads. Something like moving fibers between threads is absolute disaster in this sense even if it looks like a tempting abstraction on paper. But if you prohibit that by design and maintain strict affinity between fibers and threads, using TLS allows for very nice optimizations (as it is effectively limited sharing without locking / atomics). It can be complicated to design (which is why Go choice makes sense for their target audience) but benefits are also very good.

If you write your software to fit a particular platform, including hardware features, then you are writing an operating system dedicated to that one specific platform and no other. If the idea is to write portable applications then:

a. you must abstract away from a specific platform;
b. you must accept some overhead.

The very fact that people are writing in D (or C++, Java, C#,…) means you have accepted some abstraction – otherwise you would be writing in assembly language. Having made the jump to high-level languages why baulk at a small overhead to abstract concurrency and parallelism? Making tasks lightweight processes rather than maintaining shared memory, and using channels to move data around rather than using shared memory and locks, makes applications' concurrency and parallelism easier to construct and manage (*). If we are prepared to accept overhead for stack and heap management, we must accept the overhead of processor management via thread pooling with work stealing.


(*) Andrei, this is something of a claim really just now, since although there is anecdotal evidence, I haven't managed to get the psychology of programming people to do statistically significant experiments. I will be trying again at PPIG 2015 to motivate the experiments.

-- 
Russel. ============================================================================= Dr Russel Winder      t: +44 20 7585 2200   voip: sip:russel.winder@ekiga.net 41 Buckmaster Road    m: +44 7770 465 077   xmpp: russel@winder.org.uk London SW11 1EN, UK   w: www.russel.org.uk  skype: russel_winder

On Sat, 2015-03-28 at 18:51 +0000, weaselcat via Digitalmars-d-announce wrote:
> On Saturday, 28 March 2015 at 18:47:04 UTC, Walter Bright wrote:
> > On 3/28/2015 3:20 AM, Jonathan M Davis via Digitalmars-d-announce wrote:
> > > Personally, I'm not sure that much is gained in pitting Go
> > > against D
> > > precisely because they're so different that they're likely to
> > > appeal to
> > > completely different sets of people.
> > 
> > I also do not regard Go as a competitor to D. It's more of a competitor to Java and Ruby.
> 
> I find most people I know using Go are from the python camp and either wanted static typing or faster runtime execution.

It is a pity that D is not pitching as a Python replacement.

-- 
Russel. ============================================================================= Dr Russel Winder      t: +44 20 7585 2200   voip: sip:russel.winder@ekiga.net 41 Buckmaster Road    m: +44 7770 465 077   xmpp: russel@winder.org.uk London SW11 1EN, UK   w: www.russel.org.uk  skype: russel_winder

On Saturday, 28 March 2015 at 19:16:32 UTC, Russel Winder wrote:
> If you write your software to fit a particular platform, including
> hardware features, then you are writing an operating system dedicated
> to that one specific platform and no other.

Yes and I believe writing dedicated specialized operating systems for certain network services ("OS as a library") is the future of high load server programming - at least in domains where you can afford the investment.

> If the idea is to write
> portable applications then:

"portable" and "fast network service" aren't really best friends :( I have to encounter a single project that even tries to achieve portability of server code..

> The very fact that people are writing in D (or C++, Java, C#,…) means
> you have accepted some abstraction – otherwise you would be writing in
> assembly language. Having made the jump to high-level languages why
> baulk at a small overhead to abstract concurrency and parallelism?

1) "some abstractions" != "any abstractions". One of reasons to use D as opposed to Java or C# is exactly because latter force you into overly expensive abstractions. D in its current state is closer to C++ in this context and this is huge selling point.

2) So far my experience has shown that overhead is not small at all. It depends on type of application of course.

> Making tasks lightweight processes rather than maintaining shared
> memory, and using channels to move data around rather than using
> shared memory and locks, makes applications' concurrency and
> parallelism easier to construct and manage (*).

This comment makes me wonder if we really speak about the same things. Concurrency model based on pinned fibers/threads is not the same thing as getting back to 90s shared memory multi-threading madness.

"lightweight processes" - yes, pinned fibers are very lightweight
"channels to move data around" - message passing between worker threads

At no point I have proposed to use shared memory and locks, there is no objection here.

> If we are prepared to
> accept overhead for stack and heap management, we must accept the
> overhead of processor management via thread pooling with work stealing.

Custom allocators exist pretty much for the very reason that in certain cases heap management overhead cannot be accepted. For concurrency primitives stakes are much higher.

Am 28.03.2015 um 19:51 schrieb Walter Bright:
> On 3/28/2015 8:41 AM, Sönke Ludwig wrote:
>> Am 28.03.2015 um 15:33 schrieb Russel Winder via Digitalmars-d-announce:
>>> TLS is the evil here. Anyone working with TLS is either writing an
>>> operating system or doing it wrong.
>>>
>>
>> As long as we are talking about a closed system that works exclusively
>> on this
>> fiber based concurrency model, I completely agree with you (fiber
>> local storage
>> would be fine, though).
>>
>> But we have the "unfortunate" situation that the language is not an
>> isolated
>> ecosystem. There are many C libraries that do thread-specific things
>> in one way
>> or another, or worse, make use of ordinary global variables without any
>> protection against data races, and we simply cannot ignore that.
>
> One solution (that seems entirely reasonable to me) is to make the
> droutines (i.e. "goroutines") pure. Then the TLS problem goes away. Of
> course, then I/O isn't possible either, but perhaps a solution can be
> found for that.

I/O is crucial of course, but there are also a lot of other important and inherently impure things such as message passing. I think such a restriction would go way too far. Both fiber and task local storage can also be very useful at times, so it would be a pity to rule them out completely.

You'd also usually have the whole application running on "droutines" and not simply use them as a local tool for occasional parallelism needs. This is especially true for any kind of server application. So effectively such a limitation may in practice end up as a limitation of the entire language.

On 3/27/2015 12:34 PM, w0rp wrote:
> Sean Parent's advice for "no raw loops" comes to mind.
> https://channel9.msdn.com/Events/GoingNative/2013/Cpp-Seasoning With that rule,
> basically a one-line body for foreach becomes acceptable.

This really is a great video. Which leads me to wonder why std.algorithm doesn't have a 'rotate'.

On 3/28/2015 1:32 PM, Sönke Ludwig wrote:
> I/O is crucial of course, but there are also a lot of other important and
> inherently impure things such as message passing.

If the message channel is passed as a parameter to the droutine, then the droutine can still be pure.


> I think such a restriction
> would go way too far. Both fiber and task local storage can also be very useful
> at times, so it would be a pity to rule them out completely.
>
> You'd also usually have the whole application running on "droutines" and not
> simply use them as a local tool for occasional parallelism needs. This is
> especially true for any kind of server application. So effectively such a
> limitation may in practice end up as a limitation of the entire language.

On the other hand, if purity can make droutines much more practical, the tradeoff might be worth it.