On Mon, Oct 15, 2018 at 12:45 PM Nicholas Wilson via Digitalmars-d <digitalmars-d@puremagic.com> wrote:
>
> On Monday, 15 October 2018 at 18:46:45 UTC, Manu wrote:
> > Destroy...
>
> Keep in mind immutable is implicitly shared (i.e. not in tls) because nobody can change it. It should stay readable for this reason.

Are you saying `is(immutable(int) == shared) == true)` ??

On 10/15/2018 08:46 PM, Manu wrote:
> 1. traditional; assert that the object become thread-local by
> acquiring a lock, cast shared away
> 2. object may have shared methods; such methods CAN be called on
> shared instances. such methods may internally implement
> synchronisation to perform their function. perhaps methods of a
> lock-free queue structure for instance, or operator overloads on
> `Atomic!int`, etc.
[...]
> Assuming the rules above: "can't read or write to members", and the
> understanding that `shared` methods are expected to have threadsafe
> implementations (because that's the whole point), what are the risks
> from allowing T* -> shared(T)* conversion?

As far as I understand, the rule "can't read or write to members" is for the compiler, right? I can still read and write members, but I have to cast `shared` away and ensure thread-safety myself?

If that's so, then my example from the last thread might still apply:

----
struct Bob
{
  int* p;
  void doThing() shared
  {
    p = &s; /* Might need a cast or two here, and a lock or an atomic store or whatever. */
  }
}
shared int s;
----

When the needed casts etc. are added, is `doThing` allowed? If not, I think you have to specify more precisely what a method can and can't do.

If `doThing` is ok, you can't allow T* -> shared(T)*. You'd be allowing aliasing an unqualified int* with a shared(int*):

----
void main()
{
  Bob* b = new Bob;
  shared(Bob)* sb = b; /* You'd allow this line. */
  sb.doThing();
  /* Now the unqualified int* b.p points to the shared int s. */
}
----

On Mon, Oct 15, 2018 at 1:05 PM Peter Alexander via Digitalmars-d <digitalmars-d@puremagic.com> wrote:
>
> On Monday, 15 October 2018 at 18:46:45 UTC, Manu wrote:
> > Destroy...
>
> What you describe sounds better than what we currently have.
>
> I have at least two concerns:
>
> 1. A single producer, single consumer (SPSC) queue is necessarily shared, but is only safe if there is one writing thread and one reading thread. Is it ok if shared also requires user discipline and/or runtime checks to ensure correct usage?

I think you can model this differently... perhaps rather than a single
object, it's a coupled pair.
For instance, a thread-local enque object, and a separate thread-local
deque object. Those would express the 2 thread-local points of access
to queue and dequeue, and the implementation shunts between them in a
threadsafe way. I mean, it's not *really* a threadsafe primitive, so
for the whole object to be 'shared', I think that might be a
design-fail.

> 2. In your scheme (as I understand), a struct composed entirely of atomics would be able to implement shared methods without any casts, but also be completely thread *unsafe*. Is this okay?

It would be as safe as the design intends.
A struct with a bunch of public atomics effectively presents a set of
distinct atomics, and each one is thread-safe relative to eachother.
If the members are actually coupled into aggregate state, then you
make then private and methods implement the state transitions in such
a way guaranteeing atomicity.
Like I say before, the language can't "make it threadsafe" for you...
be producing a shared method, you have a responsibility to make sure
it works right.

> Example of #2:
>
> struct TwoInts {
>    Atomic!int x, y;
>
>    void swap() shared {
>      int z = x.load;
>      x.store(y.load);
>      y.store(z);
>    }
> }

Your swap function is plain broken; it doesn't do what the API promises.
You can write all sorts of broken code, and this is a good example of
just plain broken code.
Also, `x` and `y` probably shouldn't be public, or it effectively
communicates that they're de-coupled state.

On Monday, 15 October 2018 at 20:44:35 UTC, Manu wrote:
> snip
>
> Are you saying `is(immutable(int) == shared) == true)` ??

From the spec:
"Applying any qualifier to immutable T results in immutable T. This makes immutable a fixed point of qualifier combinations and makes types such as const(immutable(shared T)) impossible to create."

Example:

import std.stdio : writeln;

void main()
{
    writeln(is(immutable(int) == shared immutable(int)) == true); //prints true
}

On Mon, Oct 15, 2018 at 1:15 PM Stanislav Blinov via Digitalmars-d <digitalmars-d@puremagic.com> wrote:
>
> On Monday, 15 October 2018 at 18:46:45 UTC, Manu wrote:
>
> > Assuming the rules above: "can't read or write to members", and the understanding that `shared` methods are expected to have threadsafe implementations (because that's the whole point), what are the risks from allowing T* -> shared(T)* conversion?
> >
> > All the risks that I think have been identified previously assume that you can arbitrarily modify the data. That's insanity... assume we fix that... I think the promotion actually becomes safe now...?
>
> You're still talking about implicit promotion?

Absolutely. This is critical to make shared useful, and I think there's a path to make it work.

> No, it does not
> become safe no matter what restrictions you put on `shared`
> instances, because the caller of any function that takes `shared`
> arguments remains blissfully unaware of this promotion.

It doesn't matter... what danger is there of distributing a shared pointer? Are you concerned that someone with a shared reference might call a threadsafe method? If that's an invalid operation, then the method has broken it's basic agreed contract.

And I agree that it's conceivable that you could contrive a bad program, but you can contrive a bad program with literally any language feature!

On Monday, 15 October 2018 at 20:54:12 UTC, jmh530 wrote:
> On Monday, 15 October 2018 at 20:44:35 UTC, Manu wrote:
>> snip
>>
>> Are you saying `is(immutable(int) == shared) == true)` ??
>
> From the spec:
> "Applying any qualifier to immutable T results in immutable T. This makes immutable a fixed point of qualifier combinations and makes types such as const(immutable(shared T)) impossible to create."
>
> Example:
>
> import std.stdio : writeln;
>
> void main()
> {
>     writeln(is(immutable(int) == shared immutable(int)) == true); //prints true
> }

The philosophy of this is that: the value never changes, therefore only one copy of the variable needs to exist (i.e. immutable variables declared at module scope are _not_ thread local) and can be shared between threads with no race conditions.

I'm saying that while what you propose sounds very reasonable, we should make sure that reading immutable variables is still good ;)

On Mon, Oct 15, 2018 at 2:05 PM Nicholas Wilson via Digitalmars-d <digitalmars-d@puremagic.com> wrote:
>
> On Monday, 15 October 2018 at 20:54:12 UTC, jmh530 wrote:
> > On Monday, 15 October 2018 at 20:44:35 UTC, Manu wrote:
> >> snip
> >>
> >> Are you saying `is(immutable(int) == shared) == true)` ??
> >
> > From the spec:
> > "Applying any qualifier to immutable T results in immutable T.
> > This makes immutable a fixed point of qualifier combinations
> > and makes types such as const(immutable(shared T)) impossible
> > to create."
> >
> > Example:
> >
> > import std.stdio : writeln;
> >
> > void main()
> > {
> >     writeln(is(immutable(int) == shared immutable(int)) ==
> > true); //prints true
> > }
>
> The philosophy of this is that: the value never changes, therefore only one copy of the variable needs to exist (i.e. immutable variables declared at module scope are _not_ thread local) and can be shared between threads with no race conditions.
>
> I'm saying that while what you propose sounds very reasonable, we should make sure that reading immutable variables is still good ;)

I don't think what I describe affects immutable in any way;
`is(immutable(int) == shared) == false`, as it should be. Immutable
wouldn't have its rules affected by any change to shared.

On Monday, 15 October 2018 at 21:08:38 UTC, Manu wrote:
> On Mon, Oct 15, 2018 at 2:05 PM Nicholas Wilson via Digitalmars-d <digitalmars-d@puremagic.com> wrote:
>> I'm saying that while what you propose sounds very reasonable, we should make sure that reading immutable variables is still good ;)
>
> I don't think what I describe affects immutable in any way;
> `is(immutable(int) == shared) == false`, as it should be. Immutable
> wouldn't have its rules affected by any change to shared.

OK, just making sure you've got this covered.

On Monday, 15 October 2018 at 20:57:46 UTC, Manu wrote:
> On Mon, Oct 15, 2018 at 1:15 PM Stanislav Blinov via Digitalmars-d <digitalmars-d@puremagic.com> wrote:
>>
>> On Monday, 15 October 2018 at 18:46:45 UTC, Manu wrote:
>>
>> > Assuming the rules above: "can't read or write to members", and the understanding that `shared` methods are expected to have threadsafe implementations (because that's the whole point), what are the risks from allowing T* -> shared(T)* conversion?
>> >
>> > All the risks that I think have been identified previously assume that you can arbitrarily modify the data. That's insanity... assume we fix that... I think the promotion actually becomes safe now...?
>>
>> You're still talking about implicit promotion?
>
> Absolutely. This is critical to make shared useful, and I think there's a path to make it work.
>
>> No, it does not
>> become safe no matter what restrictions you put on `shared`
>> instances, because the caller of any function that takes `shared`
>> arguments remains blissfully unaware of this promotion.
>
> It doesn't matter... what danger is there of distributing a shared pointer? Are you concerned that someone with a shared reference might call a threadsafe method? If that's an invalid operation, then the method has broken it's basic agreed contract.

No, on the contrary. Someone with an unshared pointer may call unshared method or read/write data while someone else accesses it via `shared` interface precisely because you allow T to escape to shared(T). You *need* an explicit cast for this.

> And I agree that it's conceivable that you could contrive a bad program, but you can contrive a bad program with literally any language feature!

On Mon, Oct 15, 2018 at 2:25 PM Stanislav Blinov via Digitalmars-d <digitalmars-d@puremagic.com> wrote:
>
> On Monday, 15 October 2018 at 20:57:46 UTC, Manu wrote:
> > On Mon, Oct 15, 2018 at 1:15 PM Stanislav Blinov via Digitalmars-d <digitalmars-d@puremagic.com> wrote:
> >>
> >> On Monday, 15 October 2018 at 18:46:45 UTC, Manu wrote:
> >>
> >> > Assuming the rules above: "can't read or write to members", and the understanding that `shared` methods are expected to have threadsafe implementations (because that's the whole point), what are the risks from allowing T* -> shared(T)* conversion?
> >> >
> >> > All the risks that I think have been identified previously assume that you can arbitrarily modify the data. That's insanity... assume we fix that... I think the promotion actually becomes safe now...?
> >>
> >> You're still talking about implicit promotion?
> >
> > Absolutely. This is critical to make shared useful, and I think there's a path to make it work.
> >
> >> No, it does not
> >> become safe no matter what restrictions you put on `shared`
> >> instances, because the caller of any function that takes
> >> `shared`
> >> arguments remains blissfully unaware of this promotion.
> >
> > It doesn't matter... what danger is there of distributing a shared pointer? Are you concerned that someone with a shared reference might call a threadsafe method? If that's an invalid operation, then the method has broken it's basic agreed contract.
>
> No, on the contrary. Someone with an unshared pointer may call unshared method or read/write data while someone else accesses it via `shared` interface precisely because you allow T to escape to shared(T). You *need* an explicit cast for this.

If a shared method is incompatible with an unshared method, your class
is broken.
Explicit casting doesn't magically implement thread-safety, it
basically just guarantees failure. What I suggest are rules that lead
to proper behaviour with respect to writing a thread-safe API.
You can write bad code with any feature in any number of ways.

I see it this way:
If your object has shared methods, then it is distinctly and
*deliberately* involved in thread-safety. You have deliberately
opted-in to writing a thread-safe object, and you must deliver on your
promise.
The un-shared API of an object that supports `shared` are not exempt
from the thread-safety commitment, they are simply the subset of the
API that may not be called from a shared context.
If your shared method is incompatible with other methods, your class
is broken, and you violate your promise.

Nobody writes methods of an object such that they don't work with each
other... methods are part of a deliberately crafted and packaged
entity. If you write a shared object, you do so deliberately, and you
buy responsibility of making sure your objects API is thread-safe.
If your object is not thread-safe, don't write shared methods.