On Friday, 15 November 2013 at 18:16:17 UTC, Jacek Furmankiewicz wrote:
> taskPool looks like the closest equivalent in D that I could find.

Yes, that's sad truth: if you want to use D, be ready make something yourself.

BTW, why did you decide to migrate to D? Any problems with Java?

No, we didn't decide to migrate to D. Java is working out fine for us.

I am however always interested in what is out there, 'cause you never know if there may not be a better solution.

And from what I've seen so far I really like D in terms of pure language features.

Go is cool too, but it has made some choices which to me are questionable (error codes instead of exceptions, lack of templates/generics).

Coupled with vibe.d, dub, etc. I see some really interesting stuff going on in the D community that seems to have been greatly under the radar.

Definitely plan to spend more time with D on my own, even if I cannot use it at work.

On Fri, 2013-11-15 at 19:05 +0100, ilya-stromberg wrote:
> On Friday, 15 November 2013 at 17:46:41 UTC, Russel Winder wrote:
> > If D programmers are being told to use locks in applications
> > code, then
> > the D programming model and library are failing. Or the advice
> > is
> > wrong ;-)
> 
> It's possible to implement lock-free data structures in D, you
> can use core.atomic
> http://dlang.org/phobos/core_atomic.html
> 
> But it's REALLY difficult to implement and it can be SLOWER than Mutex version (not only in D, it depends from usage situation).

I didn't intend to imply that core data structures had to be lock free,
it is clear that creators of thread and process safe data structures
should be free to use locks if it makes things faster and more
efficient. My point was about applications built on the language
platform: the platform should provide all the things needed so that
applications code never mention locks.

-- 
Russel. ============================================================================= Dr Russel Winder      t: +44 20 7585 2200   voip: sip:russel.winder@ekiga.net 41 Buckmaster Road    m: +44 7770 465 077   xmpp: russel@winder.org.uk London SW11 1EN, UK   w: www.russel.org.uk  skype: russel_winder

On Friday, 15 November 2013 at 17:46:41 UTC, Russel Winder wrote:
> If D programmers are being told to use locks in applications code, then
> the D programming model and library are failing. Or the advice is
> wrong ;-)

I don't really buy it. It is good from simplicity/safety point of view (just use library stuff and your code is thread-safe) but not performance. Back in C++ days we have almost always resorted to writing own concurrent data structures to abuse domain specifics and application architecture as much as possible and thus minimize actual concurrent locking frequency. And most of those solutions were completely unsuitable as generic ones.

On Fri, 2013-11-15 at 18:55 +0100, Jacek Furmankiewicz wrote:
> Yes, that is what they say in Go...but it doesn't scale either. :-)

I don't follow. CSP scales very well and Go implements CSP. (Well an updated version from Hoare's 1978 CSP.)

> I had the exact same discussion on the Go forums a while back and the conclusion was basically the same...roll your own maps with RW locks:
> 
> https://groups.google.com/forum/?fromgroups#!searchin/golang-nuts/furmankiewicz/golang-nuts/jjjvXG4HdUw/ffWytKQ7X9YJ
> 
> But...at the end someone actually built lock-free data structures in Go out of this:
> 
> https://github.com/zond/gotomic

This is, of course, how ConcurrentHashMap arrived in Java, Java didn't have a shared access, thread safe map so someone created it.  Go has no shared access, thread safe map and no-one has created one that is in the standard library.

Of course Java is a shared-memory multithreading language whereas Go is a CSP one, so the idea of a shared access memory safe data structure is actually anathema.

-- 
Russel. ============================================================================= Dr Russel Winder      t: +44 20 7585 2200   voip: sip:russel.winder@ekiga.net 41 Buckmaster Road    m: +44 7770 465 077   xmpp: russel@winder.org.uk London SW11 1EN, UK   w: www.russel.org.uk  skype: russel_winder

if I recall from the initial Go discussion, the Go folks were saying that for close to realtime SLAs the goroutine/channel approach may have some scalability limits...which is why they started recommending the RW mutex approach in the end.

Now, that was a few months ago, since then Go 1.1 (and soon 1.2) came out, so that may be a false statement at this time.

On Fri, 2013-11-15 at 20:10 +0100, Jacek Furmankiewicz wrote:
> if I recall from the initial Go discussion, the Go folks were saying that for close to realtime SLAs the goroutine/channel approach may have some scalability limits...which is why they started recommending the RW mutex approach in the end.
> 
> Now, that was a few months ago, since then Go 1.1 (and soon 1.2) came out, so that may be a false statement at this time.

I guess they were hinting at scheduling issues where there are many more goroutines than kernel threads available. Not a problem for Web services and applications but a potential problem for hard real-time. I don't think 12.0, 1.1, 1.2,… will change the core issue – though it will change the code generation, which is getting better. Though gccgo already produces very efficient code, much faster execution than the main Go system.

-- 
Russel. ============================================================================= Dr Russel Winder      t: +44 20 7585 2200   voip: sip:russel.winder@ekiga.net 41 Buckmaster Road    m: +44 7770 465 077   xmpp: russel@winder.org.uk London SW11 1EN, UK   w: www.russel.org.uk  skype: russel_winder

Thank you Russell for the explanation.

Always a chance to learn something new.

On Thursday, 14 November 2013 at 17:36:09 UTC, Jacek
Furmankiewicz wrote:
> Could anyone point me to what would be the closest D equivalents (maybe in an external library if not part of Phobos) so we can playing around with them?
>
> Much appreciated
> Jacek

In such cases the easiest route is to find some C/C++ library for
such tasks, make a C interface in the latter case, and link with
it. That would require a bit of extra work but much less than
writing your own performant implementation from scratch.
E.g. I once wrote a simple wrapper for the Kyoto Cabinet
key-value store:
https://github.com/lomereiter/kyoto-d/blob/master/kyotocabinet.d

On Thursday, 14 November 2013 at 23:10:58 UTC, Jacek Furmankiewicz wrote:
> While looking a D, I am just trying to focus on the parts which I know would be a showstopper for us on day one...and this particular issue is it.

Yes, I also think for long-running memory-hungry server-stuff the current conservative GC is a show stopper. Some people are working on a concurrent and a  precise GC. Then parallel and concurrent and incremental and generational and whatnot GCs will be explored. The point where it gets interesting for these server-apps is not clear though.

> Regarding the GC, I've seen some slides on DConf about other garbage collectors available. Is there any resource/tutorial that shows how you can swap out the default GC for those alternative implementations?

As far as I know those other GCs are not ready for prime time yet. For sure, there is no other GC shipped with the current D 2.064.2.

Oh, and keep nagging! We need to hear about showstoppers, so we can fix them! ;)