First the original post I made on this forum : http://forum.dlang.org/thread/kluaojijixhwigoujeip@forum.dlang.org#post-kluaojijixhwigoujeip:40forum.dlang.org

Now some sample code/explanation to get it better.

Isolated in a proposal adapted from an experiment made in C# for D. It introduces a new qualifier. The qualifier is necessary on function signatures (inference is possible to some extent, like for pure functions), object/struct fields and globals. It is inferred on local variables (but can be explicited if one wish).

An isolated is in an 'island'. The 'island' is implicit and tracked by the compiler. We have one immutable and one shared island. We have one TL island per thread. And we can have an infinity of isolated island.

An isolated is consumed when:
 - it is returned
 - it is passed as argument
 - it is assigned to another island

When an isolated goes out of scope without being consumed, the compiler can free the whole island:

void foo() {
  A a = new A(); // a is isolated if A's ctor allows.

  // a is not consumed. The compiler can insert memory freeing.
}

As we can see, it allows the compiler to do some freeing for us to reduce GC pressure. Manu why aren't you already here cheering ?

When an isolated is consumed, the island it is in is merged into the island that consumes it. All reference to the island become write only until next write.

void foo() {
  A a = new A(); // a is isolated
  immutable b = a; // a's island is merged into immutable

  // a's island has been consumed. a is not readable at this point.
  // a.foo() // Error
  a = new A(); // OK
  a.foo(); // OK, we have written in a.
}

So far, we saw that isolated helps to construct const/immutable/shared objects and can be used by the compiler to insert free.

isolated also help to bridge the RC world and the GC world.

struct RC(T) is(isReferenceType!T) {
  private T mystuff;

  this(isolated T stuff) {
    mystuff = stuff;
  }

  // All code to do ref counting goes here...
}

Here, the RC struct must be constructed with an isolated. To do so, the isolated have to be passed as argument: it is consumed. As a result, the RC struct can be sure that it has the only usable reference to stuff that is around.

Now some concurrency goodies:

void foo() {
  auto tid = spawn(&spawnedFunc, thisTid);

  A a = new A();
  send(tid, a); // OK, a is an isolated.

  // a can't be used here anymore as it is consumed.
  // You can simply pass isolated around across thread safely.
}

Now that is pretty sweet. First we don't have to do the crazy and unsafe dance of "cast to shared and cast back to me" and this is actually safe. Go's type system do is not safe across channel, that puts us ahead in one of the things go does best.

std.parallelism can also benefit from this. There is also several benefit when the optimizer knows about isolated (different island do not alias each other).

I hope the idea get across better with some sample code and will be considered. As sample code shows, isolated do not need to be specified explicitly often. User not annotating can get a lot of benefit out of the concept right away.

On Friday, 2 May 2014 at 06:51:49 UTC, deadalnix wrote:
> First the original post I made on this forum : http://forum.dlang.org/thread/kluaojijixhwigoujeip@forum.dlang.org#post-kluaojijixhwigoujeip:40forum.dlang.org
>
> Now some sample code/explanation to get it better.
>
> Isolated in a proposal adapted from an experiment made in C# for D. It introduces a new qualifier. The qualifier is necessary on function signatures (inference is possible to some extent, like for pure functions), object/struct fields and globals. It is inferred on local variables (but can be explicited if one wish).
>
> An isolated is in an 'island'. The 'island' is implicit and tracked by the compiler. We have one immutable and one shared island. We have one TL island per thread. And we can have an infinity of isolated island.
>
> An isolated is consumed when:
>  - it is returned
>  - it is passed as argument
>  - it is assigned to another island
>
> When an isolated goes out of scope without being consumed, the compiler can free the whole island:
>
> void foo() {
>   A a = new A(); // a is isolated if A's ctor allows.
>
>   // a is not consumed. The compiler can insert memory freeing.
> }
>
> As we can see, it allows the compiler to do some freeing for us to reduce GC pressure. Manu why aren't you already here cheering ?
>
> When an isolated is consumed, the island it is in is merged into the island that consumes it. All reference to the island become write only until next write.
>
> void foo() {
>   A a = new A(); // a is isolated
>   immutable b = a; // a's island is merged into immutable
>
>   // a's island has been consumed. a is not readable at this point.
>   // a.foo() // Error
>   a = new A(); // OK
>   a.foo(); // OK, we have written in a.
> }
>
> So far, we saw that isolated helps to construct const/immutable/shared objects and can be used by the compiler to insert free.
>
> isolated also help to bridge the RC world and the GC world.
>
> struct RC(T) is(isReferenceType!T) {
>   private T mystuff;
>
>   this(isolated T stuff) {
>     mystuff = stuff;
>   }
>
>   // All code to do ref counting goes here...
> }
>
> Here, the RC struct must be constructed with an isolated. To do so, the isolated have to be passed as argument: it is consumed. As a result, the RC struct can be sure that it has the only usable reference to stuff that is around.
>
> Now some concurrency goodies:
>
> void foo() {
>   auto tid = spawn(&spawnedFunc, thisTid);
>
>   A a = new A();
>   send(tid, a); // OK, a is an isolated.
>
>   // a can't be used here anymore as it is consumed.
>   // You can simply pass isolated around across thread safely.
> }
>
> Now that is pretty sweet. First we don't have to do the crazy and unsafe dance of "cast to shared and cast back to me" and this is actually safe. Go's type system do is not safe across channel, that puts us ahead in one of the things go does best.
>
> std.parallelism can also benefit from this. There is also several benefit when the optimizer knows about isolated (different island do not alias each other).
>
> I hope the idea get across better with some sample code and will be considered. As sample code shows, isolated do not need to be specified explicitly often. User not annotating can get a lot of benefit out of the concept right away.

I've been thinking about something similar as being an 'obvious' idea.
It to me seems rather a good idea.

On 5/1/2014 11:51 PM, deadalnix wrote:
> First the original post I made on this forum :
> http://forum.dlang.org/thread/kluaojijixhwigoujeip@forum.dlang.org#post-kluaojijixhwigoujeip:40forum.dlang.org

It's nearly the same as http://wiki.dlang.org/DIP29 except that DIP29 tries to use a library type to take the role of 'isolated' in your proposal.

On Friday, 2 May 2014 at 09:05:13 UTC, Walter Bright wrote:
> On 5/1/2014 11:51 PM, deadalnix wrote:
>> First the original post I made on this forum :
>> http://forum.dlang.org/thread/kluaojijixhwigoujeip@forum.dlang.org#post-kluaojijixhwigoujeip:40forum.dlang.org
>
> It's nearly the same as http://wiki.dlang.org/DIP29 except that DIP29 tries to use a library type to take the role of 'isolated' in your proposal.

DIP29 is not safe, does not help to construct immutables, is not inferred, do not provide aliasing infos for the optimizer, etc...

On 5/2/2014 2:24 AM, deadalnix wrote:
> DIP29 is not safe,

How so?


> does not help to construct immutables,

    immutable p = new int;

works.


> is not inferred,

I'm surprised you'd say that, most of it is about inferring uniqueness.


> do not provide aliasing infos for the optimizer,

That's right.


> etc...

?

I like it a lot, as it solves several problems elegantly! Some comments inline...

On Friday, 2 May 2014 at 06:51:49 UTC, deadalnix wrote:
> An isolated is consumed when:
>  - it is returned
>  - it is passed as argument
>  - it is assigned to another island

Assignment and passing to a scope variable can be exempt from this rule.

>
> When an isolated goes out of scope without being consumed, the compiler can free the whole island:
>
> void foo() {
>   A a = new A(); // a is isolated if A's ctor allows.

I guess the condition is that assignment to isolated is allowed only from a unique expression. Thanks to Walter's recent work, this is now inferred in many cases. But I guess in cases where it cannot be inferred (.di files come to mind), it needs to be annotated explicitly:

    class A {
        this() isolated;
    }

>
>   // a is not consumed. The compiler can insert memory freeing.
> }
>
> As we can see, it allows the compiler to do some freeing for us to reduce GC pressure. Manu why aren't you already here cheering ?

To make this more useful, turn it into a requirement. It gets us deterministic destruction for reference types. Example:

        ...
        isolated tmp = new Tempfile();
        // use tmp
        ...
        // tmp is guaranteed to get cleaned up
    }

>
> When an isolated is consumed, the island it is in is merged into the island that consumes it. All reference to the island become write only until next write.
>
> void foo() {
>   A a = new A(); // a is isolated
>   immutable b = a; // a's island is merged into immutable
>
>   // a's island has been consumed. a is not readable at this point.
>   // a.foo() // Error
>   a = new A(); // OK
>   a.foo(); // OK, we have written in a.
> }

This needs more elaboration. The problem is control flow:

    isolated a = new A();
    if(...) {
        immutable b = a;
        ...
    }
    a.foo(); // <-- ???

(Similar for loops and gotos.)

There are several possibilities:

1) isolateds must be consumed either in every branch or in no branch, and this is statically enforced by the compiler.

2) It's just "forbidden", but the compiler doesn't guarantee it except where it can.

3) The compiler inserts a hidden variable to track the status of the isolated, and asserts if it is used while it's in an invalid state. This can be elided if it can be proven to be unnecessary.

I would prefer 3), as it is the most flexible. I also believe a similar runtime check is done in other situations (to guard against return of locals in @safe code, IIRC).

> I hope the idea get across better with some sample code and will be considered. As sample code shows, isolated do not need to be specified explicitly often. User not annotating can get a lot of benefit out of the concept right away.

That's true, but it is also a breaking change, because then suddenly some variables aren't writable anymore (or alternatively, the compiler would have to analyse all future uses of the variable first to see whether it can be inferred isolated, if that's even possible in the general case). I believe it's fine if explicit annotation is required.

On Fri, 02 May 2014 02:51:47 -0400, deadalnix <deadalnix@gmail.com> wrote:

> First the original post I made on this forum : http://forum.dlang.org/thread/kluaojijixhwigoujeip@forum.dlang.org#post-kluaojijixhwigoujeip:40forum.dlang.org
>
> Now some sample code/explanation to get it better.
>
> Isolated in a proposal adapted from an experiment made in C# for D. It introduces a new qualifier. The qualifier is necessary on function signatures (inference is possible to some extent, like for pure functions), object/struct fields and globals. It is inferred on local variables (but can be explicited if one wish).
>
> An isolated is in an 'island'. The 'island' is implicit and tracked by the compiler. We have one immutable and one shared island. We have one TL island per thread. And we can have an infinity of isolated island.
>
> An isolated is consumed when:
>   - it is returned
>   - it is passed as argument
>   - it is assigned to another island
>
> When an isolated goes out of scope without being consumed, the compiler can free the whole island:
>
> void foo() {
>    A a = new A(); // a is isolated if A's ctor allows.
>
>    // a is not consumed. The compiler can insert memory freeing.
> }
>
> As we can see, it allows the compiler to do some freeing for us to reduce GC pressure. Manu why aren't you already here cheering ?
>
> When an isolated is consumed, the island it is in is merged into the island that consumes it. All reference to the island become write only until next write.
>
> void foo() {
>    A a = new A(); // a is isolated
>    immutable b = a; // a's island is merged into immutable
>
>    // a's island has been consumed. a is not readable at this point.

When you say "consumed", you mean it statically cannot be used, not that it was set to null or something, right?

>    // a.foo() // Error
>    a = new A(); // OK
>    a.foo(); // OK, we have written in a.

OK, but what happens if I do this?

     auto c = a;
     a.foo();

At this point, a was inferred typed as isolated, but in this section, it doesn't have to be. Will the type change in different parts of the function? Or will this simply be statically disallowed?

> }

>
> So far, we saw that isolated helps to construct const/immutable/shared objects and can be used by the compiler to insert free.
>
> isolated also help to bridge the RC world and the GC world.
>
> struct RC(T) is(isReferenceType!T) {
>    private T mystuff;
>
>    this(isolated T stuff) {
>      mystuff = stuff;
>    }
>
>    // All code to do ref counting goes here...
> }
>
> Here, the RC struct must be constructed with an isolated. To do so, the isolated have to be passed as argument: it is consumed. As a result, the RC struct can be sure that it has the only usable reference to stuff that is around.

mystuff should be marked isolated too, no?

> Now some concurrency goodies:
>
> void foo() {
>    auto tid = spawn(&spawnedFunc, thisTid);
>
>    A a = new A();
>    send(tid, a); // OK, a is an isolated.
>
>    // a can't be used here anymore as it is consumed.
>    // You can simply pass isolated around across thread safely.
> }
>
> Now that is pretty sweet. First we don't have to do the crazy and unsafe dance of "cast to shared and cast back to me" and this is actually safe. Go's type system do is not safe across channel, that puts us ahead in one of the things go does best.

Yes, I really like this idea. This is what is missing from the type system.

I read in your previous post that shared can benefit from using isolated, by not having to lock all sub-objects. But I'm confused as to how that would work. If a variable is shared, by default, it can be passed around. But wouldn't the passing of the shared variable mean you have to disallow access to the isolated member? The compiler would have to be aware of the locking protection and enforce it.

Essentially, you could access a locked isolated variable, but not a shared isolated variable. These kinds of requirements need to be specified somehow.

Also, when you call a method, how does the island get handled?

abstract class A
{
   void foo(ref A other) { other = this;}
   void bar(int n);
}

class B
{
   private isolated int x;
   void bar(int n) { x = n; } // I'm assuming this is ok, right?
}

auto b = new B;
A a;
b.foo(a);

At this point, both b and a refer to the same object. How does the compiler know what to prevent here? At what point does the island become inaccessible?

Another issue is with delegates. They have no reliable type for the context pointer.

-Steve

On Friday, 2 May 2014 at 06:51:49 UTC, deadalnix wrote:
> An isolated is consumed when:
>  - it is returned
>  - it is passed as argument
>  - it is assigned to another island

This will not work well with UFCS.

isolated int[] ints = new int[](10);
//put looks like a member function, but
//this desugars to put(ints, 3), so ints
//is consumed
ints.put(3);

On Fri, 02 May 2014 09:50:07 -0400, Meta <jared771@gmail.com> wrote:

> On Friday, 2 May 2014 at 06:51:49 UTC, deadalnix wrote:
>> An isolated is consumed when:
>>  - it is returned
>>  - it is passed as argument
>>  - it is assigned to another island
>
> This will not work well with UFCS.
>
> isolated int[] ints = new int[](10);
> //put looks like a member function, but
> //this desugars to put(ints, 3), so ints
> //is consumed
> ints.put(3);

Some interaction with pure would be in order. I don't think ints.put(3) should consume the ints. However, you have just lost the data you put, since ints cannot have any other reference to it!

-Steve

On 5/2/14, 2:24 AM, deadalnix wrote:
> On Friday, 2 May 2014 at 09:05:13 UTC, Walter Bright wrote:
>> On 5/1/2014 11:51 PM, deadalnix wrote:
>>> First the original post I made on this forum :
>>> http://forum.dlang.org/thread/kluaojijixhwigoujeip@forum.dlang.org#post-kluaojijixhwigoujeip:40forum.dlang.org
>>>
>>
>> It's nearly the same as http://wiki.dlang.org/DIP29 except that DIP29
>> tries to use a library type to take the role of 'isolated' in your
>> proposal.
>
> DIP29 is not safe, does not help to construct immutables, is not
> inferred, do not provide aliasing infos for the optimizer, etc...

I think a more detailed comparison would do well here. My understanding of DIP29 is quite at odds with these claims.

Andrei