Okay, so I've been thinking on this for a while... I think I have a pretty good feel for how shared is meant to be.

1. shared should behave exactly like const, except in addition to inhibiting write access, it also inhibits read access.

I think this is the foundation for a useful definition for shared, and it's REALLY easy to understand and explain.

Current situation where you can arbitrarily access shared members
undermines any value it has. Shared must assure you don't access
members unsafely, and the only way to do that with respect to data
members, is to inhibit access completely.
I think shared is just const without read access.

Assuming this world... how do you use shared?

1. traditional; assert that the object become thread-local by
acquiring a lock, cast shared away
2. object may have shared methods; such methods CAN be called on
shared instances. such methods may internally implement
synchronisation to perform their function. perhaps methods of a
lock-free queue structure for instance, or operator overloads on
`Atomic!int`, etc.

In practise, there is no functional change in usage from the current implementation, except we disallow unsafe accesses (which will make the thing useful).

>From there, it opens up another critical opportunity; T* -> shared(T)*
promotion.
Const would be useless without T* -> const(T)* promotion. Shared
suffers a similar problem.
If you write a lock-free queue for instance, and all the methods are
`shared` (ie, threadsafe), then under the current rules, you can't
interact with the object when it's not shared, and that's fairly
useless.

Assuming the rules above: "can't read or write to members", and the understanding that `shared` methods are expected to have threadsafe implementations (because that's the whole point), what are the risks from allowing T* -> shared(T)* conversion?

All the risks that I think have been identified previously assume that you can arbitrarily modify the data. That's insanity... assume we fix that... I think the promotion actually becomes safe now...?

Destroy...

On Monday, 15 October 2018 at 18:46:45 UTC, Manu wrote:
> 2. object may have shared methods; such methods CAN be called on
> shared instances. such methods may internally implement
> synchronisation to perform their function. perhaps methods of a
> lock-free queue structure for instance, or operator overloads on
> `Atomic!int`, etc.

Just checking my understanding: are you saying here that shared methods can effectively do anything and the burden of correctness is on the author? Or do you still have to cast the shared away first?

On Mon, Oct 15, 2018 at 12:15 PM Peter Alexander via Digitalmars-d <digitalmars-d@puremagic.com> wrote:
>
> On Monday, 15 October 2018 at 18:46:45 UTC, Manu wrote:
> > 2. object may have shared methods; such methods CAN be called on shared instances. such methods may internally implement synchronisation to perform their function. perhaps methods of a lock-free queue structure for instance, or operator overloads on `Atomic!int`, etc.
>
> Just checking my understanding: are you saying here that shared methods can effectively do anything and the burden of correctness is on the author? Or do you still have to cast the shared away first?

Burden of correctness that a `shared` method is indeed threadsafe is absolutely on the author; there's nothing we can do to guarantee this (multithreading is hard!), but in this new world, a user of an API will be able to see shared methods and assume that they're threadsafe, since that would be (and should be!) the whole point.

Since `shared` has no read or write access, at the bottom of the
stack, it may be necessary that the function cast shared away to
implement its magic. If you lock a mutex for instance, then you
naturally need to cast it away; but that's solving the problem with a
sledge-hammer.
In my experience, most such functions are implemented with atomics;
and the atomic API already receives shared args, ie:
https://github.com/dlang/druntime/blob/master/src/core/atomic.d#L379
So, if you do your work with atomics, then you don't need any casts.
Also, if the object is a composite, then a `shared` method is able to
call `shared` methods on its members, and in that case, you are able
to do useful aggregate work without casting.

I think the interactions I describe above are all correct.

On Monday, 15 October 2018 at 18:46:45 UTC, Manu wrote:
> Okay, so I've been thinking on this for a while... I think I have a pretty good feel for how shared is meant to be.
>
> 1. shared should behave exactly like const, except in addition to inhibiting write access, it also inhibits read access.
>

Are you familiar with reference capabilities[1] in the pony language? They describe many of them in terms of read/write uniqueness. Another way they describe them [2] is in denying aliases, like deny global read alias.


[1] https://tutorial.ponylang.io/capabilities/reference-capabilities.html
[2] See page 12-14: http://www.doc.ic.ac.uk/~scd/Pony-WG2.16.pdf

On Monday, 15 October 2018 at 19:14:58 UTC, Peter Alexander wrote:
> On Monday, 15 October 2018 at 18:46:45 UTC, Manu wrote:
>> 2. object may have shared methods; such methods CAN be called on
>> shared instances. such methods may internally implement
>> synchronisation to perform their function. perhaps methods of a
>> lock-free queue structure for instance, or operator overloads on
>> `Atomic!int`, etc.
>
> Just checking my understanding: are you saying here that shared methods can effectively do anything and the burden of correctness is on the author? Or do you still have to cast the shared away first?

Well `this` is still shared so you would be prevented from doing anything non-atomically or lock cast away shared, but that will be enforced by the type system. So the burden of  correctness is on the author, but the complier enforces correct behaviour.

On Mon, Oct 15, 2018 at 12:15 PM Peter Alexander via Digitalmars-d <digitalmars-d@puremagic.com> wrote:
>
> On Monday, 15 October 2018 at 18:46:45 UTC, Manu wrote:
> > 2. object may have shared methods; such methods CAN be called on shared instances. such methods may internally implement synchronisation to perform their function. perhaps methods of a lock-free queue structure for instance, or operator overloads on `Atomic!int`, etc.
>
> Just checking my understanding: are you saying here that shared methods can effectively do anything and the burden of correctness is on the author? Or do you still have to cast the shared away first?

Just to be clear:

> are you saying here that shared methods can effectively do anything

No, quite the opposite. I am saying that shared objects have neither read nor write access to members. Shared means "no read or write access". You can only call shared methods.

On Monday, 15 October 2018 at 18:46:45 UTC, Manu wrote:
> Destroy...

Keep in mind immutable is implicitly shared (i.e. not in tls) because nobody can change it. It should stay readable for this reason.

On Monday, 15 October 2018 at 18:46:45 UTC, Manu wrote:
> Destroy...

What you describe sounds better than what we currently have.

I have at least two concerns:

1. A single producer, single consumer (SPSC) queue is necessarily shared, but is only safe if there is one writing thread and one reading thread. Is it ok if shared also requires user discipline and/or runtime checks to ensure correct usage?

2. In your scheme (as I understand), a struct composed entirely of atomics would be able to implement shared methods without any casts, but also be completely thread *unsafe*. Is this okay?

Example of #2:

struct TwoInts {
  Atomic!int x, y;

  void swap() shared {
    int z = x.load;
    x.store(y.load);
    y.store(z);
  }
}

On Monday, 15 October 2018 at 18:46:45 UTC, Manu wrote:

> Assuming the rules above: "can't read or write to members", and the understanding that `shared` methods are expected to have threadsafe implementations (because that's the whole point), what are the risks from allowing T* -> shared(T)* conversion?
>
> All the risks that I think have been identified previously assume that you can arbitrarily modify the data. That's insanity... assume we fix that... I think the promotion actually becomes safe now...?

You're still talking about implicit promotion? No, it does not become safe no matter what restrictions you put on `shared` instances, because the caller of any function that takes `shared` arguments remains blissfully unaware of this promotion.

On Monday, 15 October 2018 at 18:46:45 UTC, Manu wrote:
> If you write a lock-free queue for instance, and all the methods are
> `shared` (ie, threadsafe), then under the current rules, you can't
> interact with the object when it's not shared, and that's fairly
> useless.
Unless the compiler can show that it is ok to implicit/explicity convert the object to share without any unintended consequences. It should reject it.
It seems that the better solution would to implement a implicit/explict covertion system similar to c# conversion Operators
https://docs.microsoft.com/en-us/dotnet/csharp/programming-guide/statements-expressions-operators/using-conversion-operators

But that itself requires an DIP itself.

-Alex