On Saturday, May 11, 2013 13:12:00 Diggory wrote:
> Just listened to this talk and it made me think about the various type qualifiers. Has there ever been any thought of introducing a new type qualifier/attribute, "unique"?

It's been brought up quite a few times (it might have even have been brought up by Bartoz ages ago - I don't recall for sure). The problem is that it complicates the type system yet further. We'd have to find a way to introduce it without impacting the rest of the type system much or complicating it much further. And that's a tall order. Not to mention, we already have a lot of attributes, and every new one adds to the complexity and cognitive load of the language. So, we have to be very careful about what we do and don't add at this point.

- Jonathan M Davis

On 2013-05-12, 00:31, Manu wrote:

> This is a very interesting idea.
> It would also be a massive advantage when passing ownership between
> threads, which is a long-standing problem that's not solves at all.
> There currently exists no good way to say "I now give ownership to you",
> which is what you basically always do when putting a job on a queue to be
> picked up by some foreign thread.
> Using shared is cumbersome, and feels very inelegant, casts everywhere, and
> once the casts appear, any safety is immediately lost.
>
> Can you detail the process involved in assignment from one unique to
> another unique? Would the original unique be destroyed? Leaving only the
> 'copy' remaining?

Not speaking for Diggory, but that's generally the idea, yes. In code:

class A { /* ... */ }

void foo(A a) { /* ... */ }

void fun( ) {
    unique A a = new A();
    unique A b = a;
    assert(a is null);
    foo(b);
    assert(b is null);
}

And with my suggestion for 'lent':

void bar(lent A a) { /* Assigning a (or anything reachable from a) to a global in here is verboten. */ }

void gun( ) {
    unique A a = new A();
    bar(a);
    assert(a !is null);
}

-- 
Simen

On Saturday, 11 May 2013 at 22:24:38 UTC, Simen Kjaeraas wrote:
> In which case we end up with duplicates of all functions to support both
> unique and non-unique parameters. Hence we'd need scope or lent.

OK, I see that we need a way to accept unique values without enforcing them.

> Temporary copies are fine. Only those that escape the function are
> problematic.

The problem I had with this is that it temporarily breaks the invariant on the calling code's "unique" variable. For the duration of the function it is no longer unique.

One way to not break the invariant would be to imagine it as first moving the value into the function when it is called, and then moving it back out when it returns.

Obviously it may not be good to actually implement it this way, but all that is necessary is to make accessing the original variable before the function has returned, undefined. In most cases the only limit this imposes is not passing a "unique" variable for more than one argument to a function.

The problem is when there is a global "unique" variable being passed to a function. Perhaps in this case the compiler could actually emulate the move operation in and out of the function by temporarily clearing the global for the duration of the call so that it can't then be accessed by the function, thus maintaining the unique invariant.

> Can you detail the process involved in assignment from one unique to
> another unique? Would the original unique be destroyed? Leaving only the
> 'copy' remaining?
Yep, it would just be standard "move" semantics, same as if you initialise a variable with an rvalue.

> It's been brought up quite a few times (it might have even have been brought
> up by Bartoz ages ago - I don't recall for sure). The problem is that it
> complicates the type system yet further. We'd have to find a way to introduce
> it without impacting the rest of the type system much or complicating it much
> further. And that's a tall order. Not to mention, we already have a lot of
> attributes, and every new one adds to the complexity and cognitive load of the
> language. So, we have to be very careful about what we do and don't add at
> this point.
>
> - Jonathan M Davis

The main change to existing code would seem to be adding "scope" or "lent" to parameters where relevant so that unique values can be accepted. It only makes sense to use it for ref parameters, classes and slices, so it's not like it would need to be added everywhere. Interestingly that's yet another optimisation that can be done - if a slice is unique it can freely be appended to without some of the usual checks.

With regard to lots of attributes, I think a language which tries to be as much as D tries to be is going to end up with a lot of attributes in the end anyway. With a combination of better IDE and compiler support for inferring attributes, it shouldn't cause too much of a problem - attributes are generally very simple to understand, the hard part is always knowing which one to apply when writing library code.

It's not just for the benefit of the compiler either - attributes help get across the intent of the code rather than just what it does and can be very powerful in ensuring correct code.

On Sunday, May 12, 2013 02:36:28 Diggory wrote:
> It's not just for the benefit of the compiler either - attributes help get across the intent of the code rather than just what it does and can be very powerful in ensuring correct code.

Yes, but the more you have, the more the programmer has to understand and keep track of. There's cost in cognitive load. So, you want to add enough that you can do what you need to do and get some solid benefits from the attributes that you have, but at some point, you have to stop adding them, or the language becomes unwieldy. Whether it would ultimately be good or bad with regards to unique specifically is still an open question, but it means that any attributes you add really need to pull their weight, especially when we already have so many of them.

- Jonathan M Davis

On 2013-05-12, 02:36, Diggory wrote:

>> Temporary copies are fine. Only those that escape the function are
>> problematic.
>
> The problem I had with this is that it temporarily breaks the invariant on the calling code's "unique" variable. For the duration of the function it is no longer unique.

This is basically the same as with D's @pure. You may break purity as
much as you like, as long as no side effects are visible *outside* the
function.


> One way to not break the invariant would be to imagine it as first moving the value into the function when it is called, and then moving it back out when it returns.
>
> Obviously it may not be good to actually implement it this way, but all that is necessary is to make accessing the original variable before the function has returned, undefined. In most cases the only limit this imposes is not passing a "unique" variable for more than one argument to a function.

Hm. I didn't think of that. I believe passing the same unique value twice
should be fine, as long as both are lent. Obviously, in the case of
non-lent, passing it twice should be an error (probably not possible to
statically determine in all cases, so a runtime error occasionally, or
just go conservative and outlaw some valid uses).


> The problem is when there is a global "unique" variable being passed to a function. Perhaps in this case the compiler could actually emulate the move operation in and out of the function by temporarily clearing the global for the duration of the call so that it can't then be accessed by the function, thus maintaining the unique invariant.

I believe you're overthinking this. First, what is global unique variable?
A unique value will per definition have to be thread-local (if not, other
threads have a reference to it, and it's not unique). Thus, when passed to
a function, the value is inaccessible outside of that function until it
returns. (I guess fibers might be an exception here?)

-- 
Simen

On Sunday, 12 May 2013 at 01:16:43 UTC, Simen Kjaeraas wrote:
> I believe you're overthinking this. First, what is global unique variable?
> A unique value will per definition have to be thread-local (if not, other
> threads have a reference to it, and it's not unique). Thus, when passed to
> a function, the value is inaccessible outside of that function until it
> returns. (I guess fibers might be an exception here?)

While in the function, that function can access a value both through the global variable which is supposed to be "unique" and through the lent parameter. This could cause problems because "unique" no longer means "unique", although it's difficult to see how serious that might be.

On Saturday, 11 May 2013 at 22:24:38 UTC, Simen Kjaeraas wrote:
> I'm not convinced. unique, like const or immutable, is transitive. If foo
> is unique, then so is foo.bar.
>

That isn't true. Please read microsoft's paper.

On 2013-05-12, 08:12, deadalnix wrote:

> On Saturday, 11 May 2013 at 22:24:38 UTC, Simen Kjaeraas wrote:
>> I'm not convinced. unique, like const or immutable, is transitive. If foo
>> is unique, then so is foo.bar.
>>
>
> That isn't true. Please read microsoft's paper.

Done. *Mostly* transitive, then. Anything reachable through a unique
reference is either unique or immutable.

-- 
Simen

On 2013-05-12, 11:10, Simen Kjaeraas wrote:

> On 2013-05-12, 08:12, deadalnix wrote:
>
>> On Saturday, 11 May 2013 at 22:24:38 UTC, Simen Kjaeraas wrote:
>>> I'm not convinced. unique, like const or immutable, is transitive. If foo
>>> is unique, then so is foo.bar.
>>>
>>
>> That isn't true. Please read microsoft's paper.
>
> Done. *Mostly* transitive, then. Anything reachable through a unique
> reference is either unique or immutable.

btw, we're free to define unique to be completely transitive, or even
not transitive at all. We need not end up with a carbon copy of
Microsoft's system.

-- 
Simen

On 2013-05-12, 05:16, Diggory wrote:

> On Sunday, 12 May 2013 at 01:16:43 UTC, Simen Kjaeraas wrote:
>> I believe you're overthinking this. First, what is global unique variable?
>> A unique value will per definition have to be thread-local (if not, other
>> threads have a reference to it, and it's not unique). Thus, when passed to
>> a function, the value is inaccessible outside of that function until it
>> returns. (I guess fibers might be an exception here?)
>
> While in the function, that function can access a value both through the global variable which is supposed to be "unique" and through the lent parameter. This could cause problems because "unique" no longer means "unique", although it's difficult to see how serious that might be.

Ah, like that. That's the same problem as passing the same unique value
twice as a parameter to a function, I believe. Let's try:

class A {}

unique A globalA;
A globalNonUnique;

void foo() {
    bar( globalA );
}

void bar(lent A localA) {
    globalNonUnique = globalA;
    // Where the good stuff happens.
}

At the point of the comment, globalA is null, because of unique's rules.
localA is still what it was, because it's lent, which gives no uniqueness
guarantees, only that no new, escaping references be created.

If the parameter had been marked unique instead of lent, globalA would
have been cleared upon its value being passed to the function.


I believe the elephant in the room is lent return values:

lent A baz(lent A a) {
    return a; // Obviously illegal - a new alias is created.
}

lent A qux(ref lent A a) {
    return a; // Sets a to null and returns its old value?
}

-- 
Simen