I've read up on Pony [1] and realized that it currently has a superior implementation of the actor model when it comes to combining safety, efficiency and memory management determinism (thread-local reference-counting GC with consensus guarantees)

What libraries do we have at our disposal in D (including code.dlang.org) for implementing task-based parallelism that is close to Pony's solution with regards to

1. @safely sending isolated (transitively unique reference to) messages between actors (tasks) without the need for copying. Vibe.d has, for instance, `makeIsolated` [2] that serves this purpose.

2. a task-scheduler that can move blocked tasks between threads. Yes, I know, this has been discussed many times before...I'm checking to see if there are any updates.

3. could we make such a solution GC-free by requiring immutable data inside isolated messages to be unique references (not currently implicitly shared) aswell using, for instance, https://dlang.org/library/std/typecons/unique.html. I'm thinking of a trait named something like `makeIsolatedUnshared` that checks these restrictions.

[1] https://www.ponylang.org/
[2] http://vibed.org/api/vibe.core.concurrency/makeIsolated

What assistance can/could we currently/in-the-future get from D's type-system to verify correctness of these paradigms?

On Wednesday, 13 June 2018 at 09:45:04 UTC, Per Nordlöw wrote:
> I've read up on Pony [1] and realized that it currently has a superior implementation of the actor model when it comes to combining safety, efficiency and memory management determinism (thread-local reference-counting GC with consensus guarantees)

AFAIK, vibe has thread-local GC.

> 1. @safely sending isolated (transitively unique reference to) messages between actors (tasks) without the need for copying. Vibe.d has, for instance, `makeIsolated` [2] that serves this purpose.

The sender can just retain the root pointer so that it's not collected.

> 2. a task-scheduler that can move blocked tasks between threads. Yes, I know, this has been discussed many times before...I'm checking to see if there are any updates.

The compiler got -vtls switch, with it you can eliminate usage of TLS and use moving scheduler just fine.

> 3. could we make such a solution GC-free by requiring immutable data inside isolated messages to be unique references (not currently implicitly shared) aswell using, for instance, https://dlang.org/library/std/typecons/unique.html. I'm thinking of a trait named something like `makeIsolatedUnshared` that checks these restrictions.

RegionAllocator looks good for this.

On 14/06/2018 12:13 AM, Kagamin wrote:
> AFAIK, vibe has thread-local GC.

No.

On Wed, 2018-06-13 at 09:45 +0000, Per Nordlöw via Digitalmars-d wrote:
> I've read up on Pony [1] and realized that it currently has a superior implementation of the actor model when it comes to combining safety, efficiency and memory management determinism (thread-local reference-counting GC with consensus guarantees)

Worth noting that Pony is not only an actor based system (there are only actors, there is no other top-level structuring), it is a capability based system, with strong compile time checking. My experience is that it is fundamentally a single threaded, asynchronous, actor system. There are ways of doing actual parallelism,but it is very awkward and the scheduler does actually cope well. Of course this is from a while back – my example code behaves the same on version 0.23 but I have no idea if there are new features that would make my code wrong. I actually hope there are new features to support parallelism.

In the following it must be noted that I must consider myself a D newbie. I make the comments anyway in case I can learn something.

> What libraries do we have at our disposal in D (including code.dlang.org) for implementing task-based parallelism that is close to Pony's solution with regards to

So which sort of task-based parallelism: there is std.parallelism which is about data parallelism, but can be coerced to do interesting things using futures; there is thread-based parallelism with std.concurrency which is effectively message passing between actors with actual parallelism if the operating system allows it; there is vibe.d which I believe is a single threaded asynchronous task system – which seems very close to what Pony really is.

> 1. @safely sending isolated (transitively unique reference to) messages between actors (tasks) without the need for copying. Vibe.d has, for instance, `makeIsolated` [2] that serves this purpose.

Does D have move semantics at the program level or does the use of a garbage collector abrogate the ability of a programmer to have unique references to heap objects. Rust does this by default and Pony allows this and other options with iso, val, ref, etc. Of course the vibe.d Isolated type is similar to the Pony iso.

> 2. a task-scheduler that can move blocked tasks between threads. Yes, I know, this has been discussed many times before...I'm checking to see if there are any updates.

std.parallelism has a task pool with work stealing. It does need a review and maintenance but it all still works fine. As far as I am aware the only alternative in D is std.concurrency and write a scheduler and work stealer over tasks,i.e. do something very akin to std.parallelism. std.concurrency could do with a review and maintenance as well. Mostly std.concurrency and std.parallelism would benefit from a big overhaul.

Isn't vibe.d single-threaded?

> 3. could we make such a solution GC-free by requiring immutable data inside isolated messages to be unique references (not currently implicitly shared) aswell using, for instance, https://dlang.org/library/std/typecons/unique.html. I'm thinking of a trait named something like `makeIsolatedUnshared` that checks these restrictions.

I have always assumed D as a garbage collected system and not worried about std.tyecons.Unique. C++'s, std::unique_ptr is a right royal paid in the proverbials, so I assume D's std.typecons.Unique has the exact same properties.

Rust's move semantics approach is actually quite good once you get used to the borrow checker. Certainly memory management is an awful lot easier in Rust that in C++. I don't have Me TV-based data on D as yet since there is a bug in Phobos that Steven has fixed but it hasn't made it into the mainline yet. My prejudice of the moment is that Rusts approach may in fact be better than D's just as it is better than C++'s.

> [1] https://www.ponylang.org/
> [2] http://vibed.org/api/vibe.core.concurrency/makeIsolated
> 
> What assistance can/could we currently/in-the-future get from D's type-system to verify correctness of these paradigms?

Without a D equivalent of Rusts borrow checker, it is not clear that types alone can solve the problem. But the garbage collector would seem to make a borrow checker equivalent really quite hard.

-- 
Russel.
===========================================
Dr Russel Winder      t: +44 20 7585 2200
41 Buckmaster Road    m: +44 7770 465 077
London SW11 1EN, UK   w: www.russel.org.uk

And yes,I do know that Pony uses a garbage collector. One with quite interesting properties because at the top level there are only actors.

-- 
Russel.
===========================================
Dr Russel Winder      t: +44 20 7585 2200
41 Buckmaster Road    m: +44 7770 465 077
London SW11 1EN, UK   w: www.russel.org.uk

On Wednesday, 13 June 2018 at 13:50:54 UTC, Russel Winder wrote:
> [snip]
>
> Does D have move semantics at the program level or does the use of a garbage collector abrogate the ability of a programmer to have unique references to heap objects. Rust does this by default and Pony allows this and other options with iso, val, ref, etc. Of course the vibe.d Isolated type is similar to the Pony iso.

I did not know about vibe.d Isolated.

On Wednesday, 13 June 2018 at 09:45:04 UTC, Per Nordlöw wrote:
> I've read up on Pony [1] and realized that it currently has a superior implementation of the actor model when it comes to combining safety, efficiency and memory management determinism (thread-local reference-counting GC with consensus guarantees)
>
> What libraries do we have at our disposal in D (including code.dlang.org) for implementing task-based parallelism that is close to Pony's solution with regards to
>
> 1. @safely sending isolated (transitively unique reference to) messages between actors (tasks) without the need for copying. Vibe.d has, for instance, `makeIsolated` [2] that serves this purpose.
>
> 2. a task-scheduler that can move blocked tasks between threads. Yes, I know, this has been discussed many times before...I'm checking to see if there are any updates.
>
> 3. could we make such a solution GC-free by requiring immutable data inside isolated messages to be unique references (not currently implicitly shared) aswell using, for instance, https://dlang.org/library/std/typecons/unique.html. I'm thinking of a trait named something like `makeIsolatedUnshared` that checks these restrictions.
>
> [1] https://www.ponylang.org/
> [2] http://vibed.org/api/vibe.core.concurrency/makeIsolated
>
> What assistance can/could we currently/in-the-future get from D's type-system to verify correctness of these paradigms?

I'm working on a library to allow @safe sharing between threads. Originally I was just trying to write a D version of Rust's std::sync::Mutex. It's so alpha it's not even in the dub registry yet:

https://github.com/atilaneves/fearless

The idea is to have a @safe wrapper so the user doesn't have to work with `shared` directly (which I call "Bring Your Own Mutex"). I've been giving some thought to isolated data - it really annoys in me in D that I can't send mutable data to another threads unless I make it `shared`, even though I just created it and will never see it again from this or any other thread!

I need to think about how to do isolated properly. I'll look at vibe.d for inspiration.

Atila

On Thursday, 14 June 2018 at 13:24:06 UTC, Atila Neves wrote:
> [snip]
> I need to think about how to do isolated properly. I'll look at vibe.d for inspiration.
>

I took a look at it yesterday, but the class version depended on a long mixin that I didn't feel like fully examining...

I did notice that vibe.d's Isolated can be used both safely and unsafely though.

On Thursday, 14 June 2018 at 13:24:06 UTC, Atila Neves wrote:
> I need to think about how to do isolated properly. I'll look at vibe.d for inspiration.

Thanks. I'll have a look when you have something working.