On 2012-02-08 12:33, Manu wrote:
> On 8 February 2012 01:53, Timon Gehr <timon.gehr@gmx.ch
> <mailto:timon.gehr@gmx.ch>> wrote:
>
>     On 02/08/2012 12:09 AM, Manu wrote:
>
>         On 8 February 2012 00:33, Sean Kelly <sean@invisibleduck.org
>         <mailto:sean@invisibleduck.org>
>         <mailto:sean@invisibleduck.org
>         <mailto:sean@invisibleduck.org>__>> wrote:
>
>             On Feb 6, 2012, at 1:38 PM, Oliver Puerto wrote:
>
>          > Hello,
>          >
>          > I'm very new to D. Just started reading "The D programming
>             language". I should read it from beginning to end before posting
>             questions here. I know ... But I'm just too impatient. The issue
>             seems not to be that simple, nevertheless. The code below
>         compiles
>             with Visual Studio.
>          >
>          > I want to have something like my actor class that I can start
>             running in it's own thread like in Scala or other languages that
>             support actors. So at best, I would like to do something
>         like this:
>          >
>          >    MyActor myActor = new MyActor();
>          >    auto tid = spawn(&start, &myActor.run());
>
>             This should work:
>
>             void runActor(shared MyActor a) { (cast(MyActor)a)).run(); }
>             MyActor myActor = new MyActor();
>             auto tid = spawn(cast(shared MyActor) myActor, &runActor);
>
>
>         See, my conclusion is, whenever using this API, you inevitably
>         have dog
>         ugly code.
>
>
>     If it is combined with OO.
>
>
> The OO changes nothing, the casting is what I'm getting at.
>
>         That code is barely readable through the casts... I can only
>         draw this up to faulty API design.
>         I understand the premise of 'shared'-ness that the API is trying to
>         assert/guarantee, but the concept is basically broken in the
>         language.
>
>
>     The concept is not broken at all. There are just too few type system
>     features to conveniently support the concept.
>
>
> ... I think you contradicted yourself one sentence after the other :)
>
> You can't use this API at all with out these blind casts, which is,
>
>         basically, a hack, and I am yet to see an example of using this API
>         'properly'.
>
>
>     Passing value type and/or immutable messages works well.
>
>
> I guess so, but I think this would be relatively rare. Many immutable
> things may just be addressed directly/globally.
> Maybe it's just my experience of threading, but I basically never spawn
> a thread where I don't intend to pass some sort of 'workload' to it... I
> shouldn't have to jump through hoops to achieve that basic task.
>
>         The casts are totally self defeating.
>
>
>     They indicate a potentially unsafe operation.
>
>
> Sure, but what's the point of 'indicating' such a thing, when the ONLY
> way to deal with it, is to add ugly blind casts? You are forced to
> ignore the warning and just cast the 'problem' away.
>
>             std.concurrency really should allow unique references to a
>             non-shared type to be passed as well, using something similar to
>             assumeUnique.
>
>
>         Something like that should exist in the language (... or shared
>         should
>         just not be broken).
>
>
>     How would you improve usability of the shared qualifier without some
>     kind of ownership type system?
>
>
> I've thought about that... but I've got nothing. The shared concept
> seems basically broken to me. It's nothing more than a
> self-documentation keyword, with the added bonus that it breaks your
> code and forces casts everywhere you touch it.
> It creates a boundary between 2 worlds of objects. Nothing shared can be
> used in an un-shared environment (ie. all your code), and nothing
> unshared can be passed to a shared environment... and there's no
> implicit(/safe) transfer between the 2 worlds.
> The ONLY way to interact is explicit cast, which doesn't guarantee any
> safety, you just have to type it wherever the compile errors pop up. So
> what's the point? Without a way for the language to assert what
> transfers between worlds are safe or not, the whole premise is self
> defeating.
>
> Passing args by value seems to be the only solution, but that doesn't
> work in an OO environment.

You would basically need to serialize an object or hierarchy of objects to be able to pass it to another thread.

> Do you have any ideas how to make it work?
> I get the feeling, as I say, shared is a nice idea, but it's basically
> broken, and the language doesn't have any mechanism to really support
> it. As is, it's nothing more than an info-keyword that makes your code
> ugly :/
>
>
>         Using a template like assumeUnique is no better
>         than the ugly cast. What does it offer over a cast?
>
>
>     Nothing. I prefer cast(immutable).
>
>
> I'd prefer to not have shared at all in it's current state. It adds zero
> value that I can see, and has clear negative side effects.

I agree with you in general.

-- 
/Jacob Carlborg

On 02/08/2012 12:33 PM, Manu wrote:
> On 8 February 2012 01:53, Timon Gehr <timon.gehr@gmx.ch
> <mailto:timon.gehr@gmx.ch>> wrote:
>
>     On 02/08/2012 12:09 AM, Manu wrote:
>
>         On 8 February 2012 00:33, Sean Kelly <sean@invisibleduck.org
>         <mailto:sean@invisibleduck.org>
>         <mailto:sean@invisibleduck.org
>         <mailto:sean@invisibleduck.org>__>> wrote:
>
>             On Feb 6, 2012, at 1:38 PM, Oliver Puerto wrote:
>
>          > Hello,
>          >
>          > I'm very new to D. Just started reading "The D programming
>             language". I should read it from beginning to end before posting
>             questions here. I know ... But I'm just too impatient. The issue
>             seems not to be that simple, nevertheless. The code below
>         compiles
>             with Visual Studio.
>          >
>          > I want to have something like my actor class that I can start
>             running in it's own thread like in Scala or other languages that
>             support actors. So at best, I would like to do something
>         like this:
>          >
>          >    MyActor myActor = new MyActor();
>          >    auto tid = spawn(&start, &myActor.run());
>
>             This should work:
>
>             void runActor(shared MyActor a) { (cast(MyActor)a)).run(); }
>             MyActor myActor = new MyActor();
>             auto tid = spawn(cast(shared MyActor) myActor, &runActor);
>
>
>         See, my conclusion is, whenever using this API, you inevitably
>         have dog
>         ugly code.
>
>
>     If it is combined with OO.
>
>
> The OO changes nothing, the casting is what I'm getting at.

OO makes heavy use of reference types. Casting to shared is only necessary for data that contains references.


>
>         That code is barely readable through the casts... I can only
>         draw this up to faulty API design.
>         I understand the premise of 'shared'-ness that the API is trying to
>         assert/guarantee, but the concept is basically broken in the
>         language.
>
>
>     The concept is not broken at all. There are just too few type system
>     features to conveniently support the concept.
>
>
> ... I think you contradicted yourself one sentence after the other :)

The first sentence talks about the concept, the second sentence talks about its implementation. It is important to keep the two separate.

>
> You can't use this API at all with out these blind casts, which is,
>
>         basically, a hack, and I am yet to see an example of using this API
>         'properly'.
>
>
>     Passing value type and/or immutable messages works well.
>
>
> I guess so, but I think this would be relatively rare. Many immutable
> things may just be addressed directly/globally.
> Maybe it's just my experience of threading, but I basically never spawn
> a thread where I don't intend to pass some sort of 'workload' to it... I
> shouldn't have to jump through hoops to achieve that basic task.
>
>         The casts are totally self defeating.
>
>
>     They indicate a potentially unsafe operation.
>
>
> Sure, but what's the point of 'indicating' such a thing, when the ONLY
> way to deal with it, is to add ugly blind casts? You are forced to
> ignore the warning and just cast the 'problem' away.
>

You use the cast to work around type system limitations, you are not allowed to break the type system. I think it is nice that the absence of the shared keyword means that nothing is shared between threads.

>             std.concurrency really should allow unique references to a
>             non-shared type to be passed as well, using something similar to
>             assumeUnique.
>
>
>         Something like that should exist in the language (... or shared
>         should
>         just not be broken).
>
>
>     How would you improve usability of the shared qualifier without some
>     kind of ownership type system?
>
>
> I've thought about that... but I've got nothing. The shared concept
> seems basically broken to me. It's nothing more than a
> self-documentation keyword, with the added bonus that it breaks your
> code and forces casts everywhere you touch it.
> It creates a boundary between 2 worlds of objects. Nothing shared can be
> used in an un-shared environment (ie. all your code), and nothing
> unshared can be passed to a shared environment... and there's no
> implicit(/safe) transfer between the 2 worlds.
> The ONLY way to interact is explicit cast, which doesn't guarantee any
> safety, you just have to type it wherever the compile errors pop up. So
> what's the point?

Do you see the bug in the following Java code?

class Code {
    Code other;
    void foo(int x) {
        if(other != null && x>0) {
            other.foo(x-1);
        }else other = null;
    }
}

Hint: There is no bug if the code is interpreted as D code instead.


> Without a way for the language to assert what
> transfers between worlds are safe or not, the whole premise is self
> defeating.
>

Not necessarily.

> Passing args by value seems to be the only solution, but that doesn't
> work in an OO environment.
>
> Do you have any ideas how to make it work?

http://xkcd.com/356/


I think we'd indeed need ownership / an 'unique' type qualifier. This could also be helpful:
http://d.puremagic.com/issues/show_bug.cgi?id=7316

I think type system controlled aliasing is very important to make it work, so another important step would be to enforce the 'scope' storage class through flow-analysis.

As far as the 'unique' type qualifier goes, we need to find a suitable design:


void main(){
    auto x = new C; // if constructor is pure or scope, x can be unique
    pragma(msg, typeof(x)); // ?
    auto y = x; // ?
    pragma(msg, typeof(x)," ",typeof(y)); // ?
    x.foo(); // ?
    spawn(&bar, x) // ?
}

So maybe it should look more like this:

void main(){
    auto x = new C; // pure/scope constructor
    static assert(typeof(x) == C);
    auto y = x;
    y.foo(); // foo is pure or scope
    spawn(&bar, unique x); // fine
}

This would add unique as an unary operator to perform a safe conversion to unique. Similar operators could be added for immutable/shared/const/inout. Such an approach would be backwards-compatible.

This solution uses flow-analysis to make sure that no alias to x is alive after the safe conversion to unique.
Because of the guarantees that unshared gives, the flow analysis can even include unshared fields.

We could even do:

void spawn(F,T...)(F fun, T args) if(!mayHaveMutableNonUniqueAliasing!T) { ... }
void spawn(F,T...)(F fun, unique T args) if(mayHaveMutableNonUniqueAliasing!T) { spawn(fun, args); }

Then the example would look like:

void main(){
    auto x = new C; // pure or scope
    auto y = x;
    y.foo(); // pure or scope
    spawn(&bar, x); // fine!
}

x converts to unique if flow analysis can prove no alias to it is alive after the conversion. The first spawn overload does not match, therefore the second one is chosen. x is implicitly converted to unique and then the other overload is used to finish up.


Open issue:

void main(){
    unique x = new C;
    pragma(msg, typeof(x.z)); // ???
    auto y = x.z; // error?
    immutable z = x;
}

x.z is not necessarily unique, but assigning it to y breaks the guarantees of the unique qualifier of x.

Ideally there would also be a way to say 'this is the only mutable reference, it is implicitly convertible to immutable, but not to shared'. const(unique(T)) could be used (because const(unique) does not have any other sane meaning), but it would probably be a misnomer.


TDPL states that 'shared' enforces sequential consistency. This is not implemented yet, and if it won't be, 'scope shared' could be used to link the shared and unshared worlds:

class C{
    void foo()scope shared{ ... }
}

void main(){
    auto a = new shared(C);
    auto b = new C;
    a.foo(); // fine
    b.foo(); // fine
}


> I get the feeling, as I say, shared is a nice idea, but it's basically
> broken, and the language doesn't have any mechanism to really support
> it. As is, it's nothing more than an info-keyword that makes your code
> ugly :/

I think we should fix it, because such information could turn out to be quite important on NUMA. Furthermore, it could be possible that on such architectures, the best way to do message passing is to serialize the whole object graph and re-build it in the other core's local memory space anyway.

>
>
>         Using a template like assumeUnique is no better
>         than the ugly cast. What does it offer over a cast?
>
>
>     Nothing. I prefer cast(immutable).
>
>
> I'd prefer to not have shared at all in it's current state. It adds zero
> value that I can see, and has clear negative side effects.

I see your point.

On 02/08/12 18:24, Timon Gehr wrote:
> I think we'd indeed need ownership / an 'unique' type qualifier. This could also be helpful: http://d.puremagic.com/issues/show_bug.cgi?id=7316
> 
> I think type system controlled aliasing is very important to make it work, so another important step would be to enforce the 'scope' storage class through flow-analysis.
> 
> As far as the 'unique' type qualifier goes, we need to find a suitable design:
> 
> 
> void main(){
>     auto x = new C; // if constructor is pure or scope, x can be unique
>     pragma(msg, typeof(x)); // ?
>     auto y = x; // ?
>     pragma(msg, typeof(x)," ",typeof(y)); // ?
>     x.foo(); // ?
>     spawn(&bar, x) // ?
> }
> 
> So maybe it should look more like this:
> 
> void main(){
>     auto x = new C; // pure/scope constructor
>     static assert(typeof(x) == C);
>     auto y = x;
>     y.foo(); // foo is pure or scope
>     spawn(&bar, unique x); // fine
> }
> 
> This would add unique as an unary operator to perform a safe conversion to unique. Similar operators could be added for immutable/shared/const/inout. Such an approach would be backwards-compatible.

"cast(uniq)x" would be much more intuitive. But, for the code above, what advantage would an explicit conversion have, given that the compiler has to perform the checks anyway? If spawn() has an overload that does not require conversion (and eg copies the object internally) it's important that the right thing happens automatically.

> This solution uses flow-analysis to make sure that no alias to x is alive after the safe conversion to unique. Because of the guarantees that unshared gives, the flow analysis can even include unshared fields.
> 
> We could even do:
> 
> void spawn(F,T...)(F fun, T args) if(!mayHaveMutableNonUniqueAliasing!T) { ... }
> void spawn(F,T...)(F fun, unique T args) if(mayHaveMutableNonUniqueAliasing!T) { spawn(fun, args); }
> 
> Then the example would look like:
> 
> void main(){
>     auto x = new C; // pure or scope
>     auto y = x;
>     y.foo(); // pure or scope
>     spawn(&bar, x); // fine!
> }
> 
> x converts to unique if flow analysis can prove no alias to it is alive after the conversion. The first spawn overload does not match, therefore the second one is chosen. x is implicitly converted to unique and then the other overload is used to finish up.
> 
> 
> Open issue:
> 
> void main(){
>     unique x = new C;
>     pragma(msg, typeof(x.z)); // ???
>     auto y = x.z; // error?
>     immutable z = x;
> }
> 
> x.z is not necessarily unique, but assigning it to y breaks the guarantees of the unique qualifier of x.

This is why treating "unique" as a static storage class doesn't really work. One
of the two last lines above would have to be disallowed, and that reduces its
usefulness dramatically. The "auto" assignment either has to be illegal, or
trigger a mutation, eg to immutable. I don't think having the compiler keep
track of all possible relations would be a practical solution, even if theoretically
possible.

For those not reading D.learn - unique helps in other cases too: http://lists.puremagic.com/pipermail/digitalmars-d-learn/2012-February/029547.html

artur

On 02/08/2012 07:30 PM, Artur Skawina wrote:
> On 02/08/12 18:24, Timon Gehr wrote:
>>
>> Open issue:
>>
>> void main(){
>>      unique x = new C;
>>      pragma(msg, typeof(x.z)); // ???
>>      auto y = x.z; // error?
>>      immutable z = x;
>> }
>>
>> x.z is not necessarily unique, but assigning it to y breaks the guarantees of the unique qualifier of x.
>
> This is why treating "unique" as a static storage class doesn't really work. One
> of the two last lines above would have to be disallowed,


Actually that is not true. It could type check, because y is not alive after x has been cast to immutable.

The problem shows here:

void main(){
    unique x = new C;
    auto y = x.z;
    immutable z = x;
    foo(y,z);
}


> and that reduces its usefulness dramatically.

Can you show me an example of its use?

> The "auto" assignment either has to be illegal, or
> trigger a mutation, eg to immutable. I don't think having the compiler keep
> track of all possible relations would be a practical solution, even if theoretically
> possible.

I think it would be practical enough.

typeof(C.z) global;

void main(){
    unique x = new C;
    auto y = x.z;
    static assert(!is(typeof(y)==immutable));
    static assert(!is(typeof({global = y;}));
    immutable z = x;
    static assert(is(typeof(y)==immutable));
    static assert(is(typeof(x)==immutable);
    foo(y,z);
}

>
> For those not reading D.learn - unique helps in other cases too:
> http://lists.puremagic.com/pipermail/digitalmars-d-learn/2012-February/029547.html
>
> artur

On 02/08/12 19:51, Timon Gehr wrote:
> On 02/08/2012 07:30 PM, Artur Skawina wrote:
>> On 02/08/12 18:24, Timon Gehr wrote:
>>>
>>> Open issue:
>>>
>>> void main(){
>>>      unique x = new C;
>>>      pragma(msg, typeof(x.z)); // ???
>>>      auto y = x.z; // error?
>>>      immutable z = x;
>>> }
>>>
>>> x.z is not necessarily unique, but assigning it to y breaks the guarantees of the unique qualifier of x.
>>
>> This is why treating "unique" as a static storage class doesn't really work. One of the two last lines above would have to be disallowed,
> 
> 
> Actually that is not true. It could type check, because y is not alive after x has been cast to immutable.

I assumed this was a simplified example and both 'y' and 'z' would be accessed later.

> The problem shows here:
> 
> void main(){
>     unique x = new C;
>     auto y = x.z;
>     immutable z = x;
>     foo(y,z);
> }

But in real code the situation will be more complex, and like i said, while i
think it can be done in theory, i don't think it should. Consider your two above
examples - you want to allow something that's not really useful (dead assignment,
that will be eliminated by an optimizing compiler), while not really solving
the real case (for which there's no ideal solution, if it were to be legal).

>> and that reduces its usefulness dramatically.
> 
> Can you show me an example of its use?

I think that introducing a new type modifier/class might not be necessary. What if in all these examples we were to use "auto" instead of "unique" when declaring the variables _and_ have the compiler internally keep a 'unique' flag? It can then do the safe conversions implicitly, and this is still backwards compatible. The problematic cases are a) accessing the type directly (ie typeof etc) and b) multiple conflicting assignments.

Also, consider your original example from this thread:

> void main(){
>     auto x = new C; // pure or scope
>     auto y = x;
>     y.foo(); // pure or scope
>     spawn(&bar, x); // fine!
> }

What if 'x' is used after the call to spawn()? If we effectively passed ownership of our unique instance to another context, 'x' can no longer be "unique". If it were to mutate to the target type, then leaving it accessible from the current context should be reasonably safe.

(Unless the new type is "unique" too - but allowing this only when there are no further accesses to 'x' should be enough)


>> The "auto" assignment either has to be illegal, or
>> trigger a mutation, eg to immutable. I don't think having the compiler keep
>> track of all possible relations would be a practical solution, even if theoretically
>> possible.
> 
> I think it would be practical enough.
> 
> typeof(C.z) global;
> 
> void main(){
>     unique x = new C;
>     auto y = x.z;
>     static assert(!is(typeof(y)==immutable));
>     static assert(!is(typeof({global = y;}));
>     immutable z = x;
>     static assert(is(typeof(y)==immutable));
>     static assert(is(typeof(x)==immutable);
>     foo(y,z);
> }

'y's type changed here - this wouldn't have worked for a "static" unique type.
And if it can work for "auto" - is an explicit "unique" for 'x' needed?
This example *could* work, but the interesting cases are the ones when the
assignments/mutations are non-trivial and done because the data is actually
used - i'm worried that doing the analysis *then* would be too expensive.

artur

On 02/08/2012 10:26 PM, Artur Skawina wrote:
> On 02/08/12 19:51, Timon Gehr wrote:
>> On 02/08/2012 07:30 PM, Artur Skawina wrote:
>>> On 02/08/12 18:24, Timon Gehr wrote:
>>>>
>>>> Open issue:
>>>>
>>>> void main(){
>>>>       unique x = new C;
>>>>       pragma(msg, typeof(x.z)); // ???
>>>>       auto y = x.z; // error?
>>>>       immutable z = x;
>>>> }
>>>>
>>>> x.z is not necessarily unique, but assigning it to y breaks the guarantees of the unique qualifier of x.
>>>
>>> This is why treating "unique" as a static storage class doesn't really work. One
>>> of the two last lines above would have to be disallowed,
>>
>>
>> Actually that is not true. It could type check, because y is not alive after x has been cast to immutable.
>
> I assumed this was a simplified example and both 'y' and 'z' would be accessed later.
>

Ok, sorry about that.

>> The problem shows here:
>>
>> void main(){
>>      unique x = new C;
>>      auto y = x.z;
>>      immutable z = x;
>>      foo(y,z);
>> }
>
> But in real code the situation will be more complex, and like i said, while i
> think it can be done in theory, i don't think it should. Consider your two above
> examples - you want to allow something that's not really useful (dead assignment,
> that will be eliminated by an optimizing compiler), while not really solving
> the real case (for which there's no ideal solution, if it were to be legal).
>
>>> and that reduces its usefulness dramatically.
>>
>> Can you show me an example of its use?
>
> I think that introducing a new type modifier/class might not be necessary. What if
> in all these examples we were to use "auto" instead of "unique" when declaring the
> variables _and_ have the compiler internally keep a 'unique' flag? It can then do
> the safe conversions implicitly, and this is still backwards compatible.

How do you pass ownership of an object graph between threads if there is no explicit representation of ownership in the type system?

> The problematic cases are a) accessing the type directly (ie typeof etc) and b) multiple
> conflicting assignments.
>
> Also, consider your original example from this thread:
>
>> void main(){
>>      auto x = new C; // pure or scope
>>      auto y = x;
>>      y.foo(); // pure or scope
>>      spawn(&bar, x); // fine!
>> }
>
> What if 'x' is used after the call to spawn()?

Then it does not implicitly convert to unique. This example relied on a modified definition of spawn whose sketch you did not quote.

> If we effectively passed ownership of our unique instance to another context, 'x' can no longer
> be "unique". If it were to mutate to the target type, then leaving it
> accessible from the current context should be reasonably safe.
>

The idea was that spawn could take unique class references and pass ownership to a different thread -- eliminating the need to cast to and from shared.

> (Unless the new type is "unique" too - but allowing this only when there
> are no further accesses to 'x' should be enough)
>
>
>>> The "auto" assignment either has to be illegal, or
>>> trigger a mutation, eg to immutable. I don't think having the compiler keep
>>> track of all possible relations would be a practical solution, even if theoretically
>>> possible.
>>
>> I think it would be practical enough.
>>
>> typeof(C.z) global;
>>
>> void main(){
>>      unique x = new C;
>>      auto y = x.z;
>>      static assert(!is(typeof(y)==immutable));
>>      static assert(!is(typeof({global = y;}));
>>      immutable z = x;
>>      static assert(is(typeof(y)==immutable));
>>      static assert(is(typeof(x)==immutable);
>>      foo(y,z);
>> }
>
> 'y's type changed here - this wouldn't have worked for a "static" unique type.

I know. A unique field / static variable cannot be copied, only moved:

unique int[] a = [1,2,3,4];

void main(){
    unique b = a;
    a = null;
}

> And if it can work for "auto" - is an explicit "unique" for 'x' needed?
> This example *could* work, but the interesting cases are the ones when the
> assignments/mutations are non-trivial and done because the data is actually
> used - i'm worried that doing the analysis *then* would be too expensive.
>

I don't think so.

On Tuesday, 7 February 2012 at 23:10:02 UTC, Manu wrote:
> On 8 February 2012 00:33, Sean Kelly <sean@invisibleduck.org> wrote:
>
>> On Feb 6, 2012, at 1:38 PM, Oliver Puerto wrote:
>>
>> > Hello,
>> >
>> > I'm very new to D. Just started reading "The D programming language". I
>> should read it from beginning to end before posting questions here. I know
>> ... But I'm just too impatient. The issue seems not to be that simple,
>> nevertheless. The code below compiles with Visual Studio.
>> >
>> > I want to have something like my actor class that I can start running in
>> it's own thread like in Scala or other languages that support actors. So at
>> best, I would like to do something like this:
>> >
>> >  MyActor myActor = new MyActor();
>> >  auto tid = spawn(&start, &myActor.run());
>>
>> This should work:
>>
>> void runActor(shared MyActor a) { (cast(MyActor)a)).run(); }
>> MyActor myActor = new MyActor();
>> auto tid = spawn(cast(shared MyActor) myActor, &runActor);
>>
>
> See, my conclusion is, whenever using this API, you inevitably have dog
> ugly code. That code is barely readable through the casts... I can only
> draw this up to faulty API design.
> I understand the premise of 'shared'-ness that the API is trying to
> assert/guarantee, but the concept is basically broken in the language. You
> can't use this API at all with out these blind casts, which is, basically,
> a hack, and I am yet to see an example of using this API 'properly'. The
> casts are totally self defeating.
>
>
>> std.concurrency really should allow unique references to a non-shared type
>> to be passed as well, using something similar to assumeUnique.
>
>
> Something like that should exist in the language (... or shared should just
> not be broken). Using a template like assumeUnique is no better than the
> ugly cast. What does it offer over a cast?

IMO the shared *concept* itself is broken. I'd suggest to take a look at languages such as chapel and x10 to see their designs. E.g http://chapel.cray.com/tutorials/SC11/SC11-5-Locales.pdf discusses the concept of locales and the "on" construct. This allows to specify where each statement is performed and the runtime automatically handles for you the implementation details of migrating data.

foobar:

> I'd suggest to take a look at languages such as chapel and x10 to see their designs. E.g http://chapel.cray.com/tutorials/SC11/SC11-5-Locales.pdf discusses the concept of locales and the "on" construct.

I am saying something similar since like three years :-)

Bye,
bearophile

On 02/08/12 22:47, Timon Gehr wrote:
> On 02/08/2012 10:26 PM, Artur Skawina wrote:
>> On 02/08/12 19:51, Timon Gehr wrote:
>>> On 02/08/2012 07:30 PM, Artur Skawina wrote:
>>>> On 02/08/12 18:24, Timon Gehr wrote:
>>>>>
>>>>> Open issue:
>>>>>
>>>>> void main(){
>>>>>       unique x = new C;
>>>>>       pragma(msg, typeof(x.z)); // ???
>>>>>       auto y = x.z; // error?
>>>>>       immutable z = x;
>>>>> }
>>>>>
>>>>> x.z is not necessarily unique, but assigning it to y breaks the guarantees of the unique qualifier of x.
>>>>
>>>> This is why treating "unique" as a static storage class doesn't really work. One of the two last lines above would have to be disallowed,
>>>
>>>
>>> Actually that is not true. It could type check, because y is not alive after x has been cast to immutable.
>>
>> I assumed this was a simplified example and both 'y' and 'z' would be accessed later.
>>
> 
> Ok, sorry about that.
> 
>>> The problem shows here:
>>>
>>> void main(){
>>>      unique x = new C;
>>>      auto y = x.z;
>>>      immutable z = x;
>>>      foo(y,z);
>>> }
>>
>> But in real code the situation will be more complex, and like i said, while i
>> think it can be done in theory, i don't think it should. Consider your two above
>> examples - you want to allow something that's not really useful (dead assignment,
>> that will be eliminated by an optimizing compiler), while not really solving
>> the real case (for which there's no ideal solution, if it were to be legal).
>>
>>>> and that reduces its usefulness dramatically.
>>>
>>> Can you show me an example of its use?
>>
>> I think that introducing a new type modifier/class might not be necessary. What if in all these examples we were to use "auto" instead of "unique" when declaring the variables _and_ have the compiler internally keep a 'unique' flag? It can then do the safe conversions implicitly, and this is still backwards compatible.
> 
> How do you pass ownership of an object graph between threads if there is no explicit representation of ownership in the type system?
> 
>> The problematic cases are a) accessing the type directly (ie typeof etc) and b) multiple
>> conflicting assignments.
>>
>> Also, consider your original example from this thread:
>>
>>> void main(){
>>>      auto x = new C; // pure or scope
>>>      auto y = x;
>>>      y.foo(); // pure or scope
>>>      spawn(&bar, x); // fine!
>>> }
>>
>> What if 'x' is used after the call to spawn()?
> 
> Then it does not implicitly convert to unique. This example relied on a modified definition of spawn whose sketch you did not quote.
> 
>> If we effectively passed ownership of our unique instance to another context, 'x' can no longer
>> be "unique". If it were to mutate to the target type, then leaving it
>> accessible from the current context should be reasonably safe.
> 
> The idea was that spawn could take unique class references and pass ownership to a different thread -- eliminating the need to cast to and from shared.

I'll rephrase what i said in that d.learn post; *all* I'm suggesting is this:

a) Any result of an expression that the compiler can determine is unique is
   internally flagged as such. This means eg array concatenation or new-expressions.
   Just a simple bitflag set, in addition to the the stored "real" type.
b) Any access to the data clears this flag (with just a few exceptions, below).
c) If the expression needs to be implicitly converted to another type *and*
   no implicit cast is possible *and* the "unique" flag is set - then additional
   safe conversions are tried, and if one succeeds, the "unique" flag gets cleared
   and the type gets modified to the that of the target.

This allows for things which are 100% safe, but currently prohibited by the compiler and require explicit casts.

If I understood you right, you'd like (b) to be much less restrictive, which i think complicates things too much. Some (b)-restrictions for cases that always are cheap to discover /can/ be removed, but this needs to be determined on a case- -by-case basis. Eg. I think any leaked refs to the data don't qualify (IOW any assignment, even if only indirectly via this expression, needs to clear the flag). One thing the (b) probably /has/ to allow is storing the result in an "auto" variable. But making another copy should clear the flag.

While i originally needed this for immutable/const/mutable, it would also work for shared. If spawn() takes a "shared" argument, passing it a "unique" one will work too. And i'm not even convinced the ref needs to disappear from the current context (obviously accessing the now shared data has to treat it as such - but this is not different from what we had before, when using explicit casts; in fact now it's marked as "shared" so it should be safer.)

So the question is: does having an explicit "unique" storage class improve things further?

Other than using it [1] to mark things as unique that the compiler can't figure out by itself.

[1] I'm using "unique", but if it were to become a keyword it should be
    "uniq" or "@uniq", for the same reasons as "int", "auto" or "ref".

>> (Unless the new type is "unique" too - but allowing this only when there are no further accesses to 'x' should be enough)
>>
>>
>>>> The "auto" assignment either has to be illegal, or
>>>> trigger a mutation, eg to immutable. I don't think having the compiler keep
>>>> track of all possible relations would be a practical solution, even if theoretically
>>>> possible.
>>>
>>> I think it would be practical enough.
>>>
>>> typeof(C.z) global;
>>>
>>> void main(){
>>>      unique x = new C;
>>>      auto y = x.z;
>>>      static assert(!is(typeof(y)==immutable));
>>>      static assert(!is(typeof({global = y;}));
>>>      immutable z = x;
>>>      static assert(is(typeof(y)==immutable));
>>>      static assert(is(typeof(x)==immutable);
>>>      foo(y,z);
>>> }
>>
>> 'y's type changed here - this wouldn't have worked for a "static" unique type.
> 
> I know. A unique field / static variable cannot be copied, only moved:
> 
> unique int[] a = [1,2,3,4];
> 
> void main(){
>     unique b = a;
>     a = null;
> }
> 
>> And if it can work for "auto" - is an explicit "unique" for 'x' needed? This example *could* work, but the interesting cases are the ones when the assignments/mutations are non-trivial and done because the data is actually used - i'm worried that doing the analysis *then* would be too expensive.
>>
> 
> I don't think so.

Consider a container, that you need to verify and/or extract some data from,
before passing it on to somewhere else. All the accesses before this handover
happens may indeed be safe and no reference be live anymore. But the compiler
needs to *prove* this before allowing the conversion to happen.
This could be very expensive and because it is not an optimization, but a
correctness issue, the check can *not* be disabled. Every D compiler now has
to do it, every time.

artur

Artur Skawina:

> So the question is: does having an explicit "unique" storage class improve things further?

This was discussed, here and in Bartosz Milewski's blog too. Some different annotations here are surely able to make things more clear for the type system and the compiler. It was not introduced in D because making the type system more powerful has some costs too, you need to add the annotations, and you have to learn to use them. Like many design choices, that probably wasn't a final decision.

Bye,
bearophile