On Monday, 15 October 2018 at 20:53:32 UTC, Manu wrote:
> On Mon, Oct 15, 2018 at 1:05 PM Peter Alexander via Digitalmars-d <digitalmars-d@puremagic.com> wrote:
>>
>> On Monday, 15 October 2018 at 18:46:45 UTC, Manu wrote:
>> 1. A single producer, single consumer (SPSC) queue is necessarily shared, but is only safe if there is one writing thread and one reading thread. Is it ok if shared also requires user discipline and/or runtime checks to ensure correct usage?
>
> I think you can model this differently... perhaps rather than a single
> object, it's a coupled pair.

That's a nice design.


> Your swap function is plain broken; it doesn't do what the API promises.
> You can write all sorts of broken code, and this is a good example of
> just plain broken code.

If it is broken then why allow it? Why do we need to cast shared away if they weren't atomic and why do we allow it if they are atomic?

I understand that shared can't magically tell you when code is thread safe or not. It does make sense to disallow almost everything and require casts. I'm just not seeing the value of allowing shared methods to access shared members if it isn't thread safe. Make it require casts.

On Monday, 15 October 2018 at 21:51:43 UTC, Manu wrote:

> If a shared method is incompatible with an unshared method, your class is broken.

What?!? So... my unshared methods should also perform all that's necessary for `shared` methods?

> Explicit casting doesn't magically implement thread-safety, it
> basically just guarantees failure.

It doesn't indeed. It does, however, at least help prevent silent bugs. Via that same guaranteed failure. That failure is about all the help we can get from the compiler anyway.

> What I suggest are rules that lead to proper behaviour with respect to writing a thread-safe API.
> You can write bad code with any feature in any number of ways.

Yup. For example, passing an int* to a function expecting shared int*.

> I see it this way:
> If your object has shared methods, then it is distinctly and
> *deliberately* involved in thread-safety. You have deliberately
> opted-in to writing a thread-safe object, and you must deliver on your promise.

> The un-shared API of an object that supports `shared` are not exempt from the thread-safety commitment, they are simply the subset of the API that may not be called from a shared context.

And therefore they lack any synchronization. So I don't see how they *can* be "compatible" with `shared` methods.

> If your shared method is incompatible with other methods, your class is broken, and you violate your promise.

Nope.

class BigCounter {

    this() { /* don't even need the mutex if I'm not sharing this */ }

    this(Mutex m = null) shared {
        this.m = m ? m : new Mutex;
    }

    void increment() { value += 1; }
    void increment() shared { synchronized(m) *value.assumeUnshared += 1; }

private:
    Mutex m;
    BigInt value;
}

They're not "compatible" in any shape or form. Or would you have the unshared ctor also create the mutex and unshared increment also take the lock? What's the point of having them  then? Better disallow mixed implementations altogether (which is actually not that bad of an idea).

> Nobody writes methods of an object such that they don't work with each other... methods are part of a deliberately crafted and packaged
> entity. If you write a shared object, you do so deliberately, and you buy responsibility of making sure your objects API is thread-safe.
> If your object is not thread-safe, don't write shared methods.

Ahem... Okay...

import std.concurrency;
import core.atomic;

void thread(shared int* x) {
    (*x).atomicOp!"+="(1);
}

shared int c;

void main() {
    int x;
    auto tid = spawn(&thread, &x); // "just" a typo
}

You're saying that's ok, it should "just" compile. It shouldn't. It should produce an error and a mild electric discharge into the developer's chair.

On Mon, Oct 15, 2018 at 4:25 PM Peter Alexander via Digitalmars-d <digitalmars-d@puremagic.com> wrote:
>
> On Monday, 15 October 2018 at 20:53:32 UTC, Manu wrote:
> > On Mon, Oct 15, 2018 at 1:05 PM Peter Alexander via Digitalmars-d <digitalmars-d@puremagic.com> wrote:
> >>
> >> On Monday, 15 October 2018 at 18:46:45 UTC, Manu wrote:
> >> 1. A single producer, single consumer (SPSC) queue is
> >> necessarily shared, but is only safe if there is one writing
> >> thread and one reading thread. Is it ok if shared also
> >> requires user discipline and/or runtime checks to ensure
> >> correct usage?
> >
> > I think you can model this differently... perhaps rather than a
> > single
> > object, it's a coupled pair.
>
> That's a nice design.
>
>
> > Your swap function is plain broken; it doesn't do what the API
> > promises.
> > You can write all sorts of broken code, and this is a good
> > example of
> > just plain broken code.
>
> If it is broken then why allow it? Why do we need to cast shared away if they weren't atomic and why do we allow it if they are atomic?

It's not that it's 'allowed' other than, yes, access to atomic int's is allowed in a threadsafe way because atomic access to individual int's is threadsafe, so the primitive operation is perfectly acceptable.

Your function models a higher-level concept; which is an atomic swap. You need to make sure that a threadsafe API you're authoring does actually deliver on the promise it makes.

> I understand that shared can't magically tell you when code is thread safe or not. It does make sense to disallow almost everything and require casts. I'm just not seeing the value of allowing shared methods to access shared members if it isn't thread safe. Make it require casts.

Because in an awful lot of cases, it is threadsafe. Implementing low-level machinery, and consuming such machinery should have a 1:many relationship. You're talking about adding friction to the 'many' such that the '1' knows that they need to implement their function right? I mean, they already know that, because they wrote 'shared' after their function declaration.

In the common case, perhaps my object might aggregate a threadsafe
queue, and my shared method prepares an item and then adds it to the
queue. I think the vast majority case will be making use of utility
functionality and that shouldn't present undue friction.
It's not like atomic int's that are class members are going to be
accidentally twiddled; you added Atomic!int's to your class, and that
wasn't an accident... and if you're making use of the lowest-level
atomic primitives, it's fair to presume you know how to use them.
You're writing a shared method, which means you already encapsulate
the promise that you are implementing a function that deals with
thread-safety. I don't know what the cast in this case would add.

On Monday, 15 October 2018 at 23:30:43 UTC, Stanislav Blinov wrote:
> On Monday, 15 October 2018 at 21:51:43 UTC, Manu wrote:
>
>> If a shared method is incompatible with an unshared method, your class is broken.
>
> What?!? So... my unshared methods should also perform all that's necessary for `shared` methods?

No, its the other way around: a shared method that does extra synchronisation should work irrespective of wether or not the object needs that synchronisation. e.g. atomic loading a TLS variable is fine.

>> Explicit casting doesn't magically implement thread-safety, it
>> basically just guarantees failure.
>
> It doesn't indeed. It does, however, at least help prevent silent bugs. Via that same guaranteed failure. That failure is about all the help we can get from the compiler anyway.
>
>> What I suggest are rules that lead to proper behaviour with respect to writing a thread-safe API.
>> You can write bad code with any feature in any number of ways.
>
> Yup. For example, passing an int* to a function expecting shared int*.

That is a reasonable thing to do if shared is const + no unsynched reads.

>> I see it this way:
>> If your object has shared methods, then it is distinctly and
>> *deliberately* involved in thread-safety. You have deliberately
>> opted-in to writing a thread-safe object, and you must deliver on your promise.
>>
>> The un-shared API of an object that supports `shared` are not exempt from the thread-safety commitment, they are simply the subset of the API that may not be called from a shared context.
>
> And therefore they lack any synchronization. So I don't see how they *can* be "compatible" with `shared` methods.
>

I think Manu means you have a shared object with some shared methods and some unshared methods. The shared methods deal with synchronisation and can therefore be call from anywhere by anyone, whereas the unshared methods must be called on a locked object.

> snip
>> Nobody writes methods of an object such that they don't work with each other... methods are part of a deliberately crafted and packaged
>> entity. If you write a shared object, you do so deliberately, and you buy responsibility of making sure your objects API is thread-safe.
>> If your object is not thread-safe, don't write shared methods.
>
> Ahem... Okay...
>
> import std.concurrency;
> import core.atomic;
>
> void thread(shared int* x) {
>     (*x).atomicOp!"+="(1);
> }
>
> shared int c;
>
> void main() {
>     int x;
>     auto tid = spawn(&thread, &x); // "just" a typo
> }
>
> You're saying that's ok, it should "just" compile. It shouldn't. It should produce an error and a mild electric discharge into the developer's chair.

Indeed that is just a typo, just as that is a contrived example. You'd notice that pretty quick in a debugger.

On Mon, Oct 15, 2018 at 4:35 PM Stanislav Blinov via Digitalmars-d <digitalmars-d@puremagic.com> wrote:
>
> On Monday, 15 October 2018 at 21:51:43 UTC, Manu wrote:
>
> > If a shared method is incompatible with an unshared method, your class is broken.
>
> What?!? So... my unshared methods should also perform all that's necessary for `shared` methods?

Of course! You're writing a threadsafe object... how could you expect otherwise?

> > Explicit casting doesn't magically implement thread-safety, it basically just guarantees failure.
>
> It doesn't indeed. It does, however, at least help prevent silent bugs. Via that same guaranteed failure. That failure is about all the help we can get from the compiler anyway.

Just to be clear, what I'm suggesting is a significant *restriction*
to what shared already does... there will be a whole lot more safety
under my proposal.
The cast gives exactly nothing that attributing a method as shared
doesn't give you, except that attributing a method shared is so much
more sanitary and clearly communicates intent at the API level.

> > What I suggest are rules that lead to proper behaviour with
> > respect to writing a thread-safe API.
> > You can write bad code with any feature in any number of ways.
>
> Yup. For example, passing an int* to a function expecting shared int*.

I don't understand your example. What's the problem you're suggesting?

> > I see it this way:
> > If your object has shared methods, then it is distinctly and
> > *deliberately* involved in thread-safety. You have deliberately
> > opted-in to writing a thread-safe object, and you must deliver
> > on your promise.
>
> > The un-shared API of an object that supports `shared` are not exempt from the thread-safety commitment, they are simply the subset of the API that may not be called from a shared context.
>
> And therefore they lack any synchronization. So I don't see how they *can* be "compatible" with `shared` methods.

I don't understand this statement either. Who said they lack
synchronisation? If they need it, they will have it.
There's a good chance they don't need it though, they might not
interact with a thread-unsafe portion of the class.

> > If your shared method is incompatible with other methods, your class is broken, and you violate your promise.
>
> Nope.

So certain...

> class BigCounter {
>
>      this() { /* don't even need the mutex if I'm not sharing this
> */ }
>
>      this(Mutex m = null) shared {
>          this.m = m ? m : new Mutex;
>      }
>
>      void increment() { value += 1; }
>      void increment() shared { synchronized(m)
> *value.assumeUnshared += 1; }
>
> private:
>      Mutex m;
>      BigInt value;
> }

You've just conflated 2 classes into one. One is a threadlocal
counter, the other is a threadsafe counter. Which is it?
Like I said before: "you can contrive a bad program with literally any
language feature!"

> They're not "compatible" in any shape or form.

Correct, you wrote 2 different things and mashed them together.

> Or would you have
> the unshared ctor also create the mutex and unshared increment
> also take the lock? What's the point of having them  then? Better
> disallow mixed implementations altogether (which is actually not
> that bad of an idea).

Right. This is key to my whole suggestion. If you write a shared
thing, you accept that it's shared! You don't just accept it, you jam
the stake in the ground.
There's a relatively small number of things that need to be
threadsafe, you won't see `shared` methods appearing at random. If you
use shared, you promise threadsafety OR the members of the thing are
inaccessible without some sort of lock-&-cast-away treatment.

> > Nobody writes methods of an object such that they don't work
> > with each other... methods are part of a deliberately crafted
> > and packaged
> > entity. If you write a shared object, you do so deliberately,
> > and you buy responsibility of making sure your objects API is
> > thread-safe.
> > If your object is not thread-safe, don't write shared methods.
>
> Ahem... Okay...
>
> import std.concurrency;
> import core.atomic;
>
> void thread(shared int* x) {
>      (*x).atomicOp!"+="(1);
> }
>
> shared int c;
>
> void main() {
>      int x;
>      auto tid = spawn(&thread, &x); // "just" a typo
> }
>
> You're saying that's ok, it should "just" compile. It shouldn't. It should produce an error and a mild electric discharge into the developer's chair.

Yup. It's a typo. You passed a stack pointer to a scope that outlives
the caller.
That class of issue is not on trial here. There's DIP1000, and all
sorts of things to try and improve safety in terms of lifetimes.
You only managed to contrive this by spawning a thread. If it were
just a normal function, this would be perfectly legitimate, and again,
that's my whole point.

On Mon, Oct 15, 2018 at 5:10 PM Nicholas Wilson via Digitalmars-d <digitalmars-d@puremagic.com> wrote:
>
> On Monday, 15 October 2018 at 23:30:43 UTC, Stanislav Blinov wrote:
> > On Monday, 15 October 2018 at 21:51:43 UTC, Manu wrote:
> >
> >> I see it this way:
> >> If your object has shared methods, then it is distinctly and
> >> *deliberately* involved in thread-safety. You have deliberately
> >> opted-in to writing a thread-safe object, and you must deliver
> >> on your promise.
> >>
> >> The un-shared API of an object that supports `shared` are not exempt from the thread-safety commitment, they are simply the subset of the API that may not be called from a shared context.
> >
> > And therefore they lack any synchronization. So I don't see how they *can* be "compatible" with `shared` methods.
> >
>
> I think Manu means you have a shared object with some shared methods and some unshared methods. The shared methods deal with synchronisation and can therefore be call from anywhere by anyone, whereas the unshared methods must be called on a locked object.

Yes, except maybe I didn't make it clear that I DO expect the
un-shared methods to be aware that a sibling shared method does exist
(you wrote it!), and that it may manipulate some state, so *if* the
un-shared method does interact with the same data that the shared
method may manipulate (in many cases, it won't; it's likely only a
small subset of an object's functionality that may have thread-safe
access), then the un-shared method does need to acknowledge that
functional overlap.
So even though a method is un-shared, it still needs to be aware that
it may have sibling methods that are shared. If they don't access an
overlapping data-set, no special handling is required. If they do
overlap, they may need to coordinate appropriately.

On Tuesday, 16 October 2018 at 00:36:12 UTC, Manu wrote:
> *snip*
>
> Yes, except maybe I didn't make it clear that I DO expect the
> un-shared methods to be aware that a sibling shared method does exist
> (you wrote it!), and that it may manipulate some state, so *if* the
> un-shared method does interact with the same data that the shared
> method may manipulate (in many cases, it won't; it's likely only a
> small subset of an object's functionality that may have thread-safe
> access), then the un-shared method does need to acknowledge that
> functional overlap.
> So even though a method is un-shared, it still needs to be aware that
> it may have sibling methods that are shared. If they don't access an
> overlapping data-set, no special handling is required. If they do
> overlap, they may need to coordinate appropriately.

I understand your point but I think the current shared (no implicit conversion) has its uses. It can be quite useful to have one interface for when an object is shared and one for when it is not (one with and without the synchronization cost). Sure, something as trivial as a counter can be re-implemented in both ways but more complex objects would easily result in extreme code duplication.

It's also easy to acknowledge that implicit conversion to shared has its uses.

Instead of forcing one way or another, how about we leave the decision up to the programmer? Say something like "alias this shared;" enables implicit conversion to shared.

On Tuesday, 16 October 2018 at 00:15:54 UTC, Manu wrote:
> On Mon, Oct 15, 2018 at 4:35 PM Stanislav Blinov via Digitalmars-d <digitalmars-d@puremagic.com> wrote:

>> What?!? So... my unshared methods should also perform all that's necessary for `shared` methods?

> Of course! You're writing a threadsafe object... how could you expect otherwise?

See below.

> Just to be clear, what I'm suggesting is a significant *restriction*
> to what shared already does... there will be a whole lot more safety under my proposal.

I don't see how an *implicit* cast can be a restriction. At all.

> The cast gives exactly nothing that attributing a method as shared
> doesn't give you, except that attributing a method shared is so much
> more sanitary and clearly communicates intent at the API level.

It's like we're talking about wholly different things here. Casting should be done by the caller, i.e. a programmer that uses some API. If that API expects shared arguments, the caller better make sure they pass shared values. Implicit conversion destroys any obligations between the caller and the API.

>> > You can write bad code with any feature in any number of ways.
>>
>> Yup. For example, passing an int* to a function expecting shared int*.
>
> I don't understand your example. What's the problem you're suggesting?

The problem that I'm suggesting is exactly that: an `int*` is not, and can not, be a `shared int*` at the same time. Substitute int for any type. But D is not Rust and it can't statically prevent that, except for disallowing trivial programming mistakes, which, with implicit conversion introduced, would also go away.

>> ...And therefore they lack any synchronization. So I don't see how they *can* be "compatible" with `shared` methods.
>
> I don't understand this statement either. Who said they lack synchronisation? If they need it, they will have it. There's a good chance they don't need it though, they might not  interact with a thread-unsafe portion of the class.

Or they might.

>> > If your shared method is incompatible with other methods, your class is broken, and you violate your promise.
>>
>> Nope.
>
> So certain...
>
>> class BigCounter {
>>
>>      this() { /* don't even need the mutex if I'm not sharing this
>> */ }
>>
>>      this(Mutex m = null) shared {
>>          this.m = m ? m : new Mutex;
>>      }
>>
>>      void increment() { value += 1; }
>>      void increment() shared { synchronized(m)
>> *value.assumeUnshared += 1; }
>>
>> private:
>>      Mutex m;
>>      BigInt value;
>> }
>
> You've just conflated 2 classes into one. One is a threadlocal
> counter, the other is a threadsafe counter. Which is it?
> Like I said before: "you can contrive a bad program with literally any language feature!"

Because that is exactly the code that a good amount of "developers" will write. Especially those of the "don't think about it" variety. Don't be mistaken for a second: if the language allows it, they'll write it.

>> They're not "compatible" in any shape or form.
>
> Correct, you wrote 2 different things and mashed them together.

Can you actually provide an example of a mixed shared/unshared class that even makes sense then? As I said, at this point I'd rather see such definitions prohibited entirely.

>> Or would you have
>> the unshared ctor also create the mutex and unshared increment also take the lock? What's the point of having them  then? Better disallow mixed implementations altogether (which is actually not that bad of an idea).

> Right. This is key to my whole suggestion. If you write a shared thing, you accept that it's shared! You don't just accept it, you jam the stake in the ground.

Then, once more, `shared` should then just be a type qualifier exclusively, and mixing shared/unshared methods should just not be allowed.

> There's a relatively small number of things that need to be
> threadsafe, you won't see `shared` methods appearing at random. If you use shared, you promise threadsafety OR the members of the thing are inaccessible without some sort of lock-&-cast-away treatment.

As above.

>> import std.concurrency;
>> import core.atomic;
>>
>> void thread(shared int* x) {
>>      (*x).atomicOp!"+="(1);
>> }
>>
>> shared int c;
>>
>> void main() {
>>      int x;
>>      auto tid = spawn(&thread, &x); // "just" a typo
>> }
>>
>> You're saying that's ok, it should "just" compile. It shouldn't. It should produce an error and a mild electric discharge into the developer's chair.
>
> Yup. It's a typo. You passed a stack pointer to a scope that outlives the caller.
> That class of issue is not on trial here. There's DIP1000, and all sorts of things to try and improve safety in terms of lifetimes.

I'm sorry, I'm not very good at writing "real" examples for things that don't exist or don't compile. End of sarcasm.

Let's come back to DIP1000 when it's actually implemented in it's entirety, ok? Anyway, you're nitpicking while actually missing the point altogether. The way `shared` is "implemented" today, the API (`thread` function) *requires* the caller to pass a `shared int*`. Implicit conversion breaks that contract.

At the highest level, the only reason for taking a `shared` argument is to pass that argument to another thread. That is the *only* way to communicate that intent via the type system for the time being. You're suggesting to ignore that fact. `shared` was supposed to protect from unshared aliasing, not silently allow it.
If you allow implicit conversion, there would literally be no way of knowing whether some API will access your data concurrently, other than plain old documentation (or sifting through it's code, which may not be available). This makes `shared` useless as a type qualifier.

> You only managed to contrive this by spawning a thread. If it were just a normal function, this would be perfectly legitimate, and again, that's my whole point.

I think you will agree that passing a pointer to a thread-local variable to another thread is not always a safe thing to do. Conditions do apply, which are on you (the programmer) to uphold, and the compiler can't help you with that. The only way the compiler *can* help you here is make sure you don't do that unintentionally. Which it won't be able to do if you allow such implicit conversion.

On Mon, Oct 15, 2018 at 7:15 PM Isaac S. via Digitalmars-d <digitalmars-d@puremagic.com> wrote:
>
> On Tuesday, 16 October 2018 at 00:36:12 UTC, Manu wrote:
> > *snip*
> >
> > Yes, except maybe I didn't make it clear that I DO expect the
> > un-shared methods to be aware that a sibling shared method does
> > exist
> > (you wrote it!), and that it may manipulate some state, so *if*
> > the
> > un-shared method does interact with the same data that the
> > shared
> > method may manipulate (in many cases, it won't; it's likely
> > only a
> > small subset of an object's functionality that may have
> > thread-safe
> > access), then the un-shared method does need to acknowledge that
> > functional overlap.
> > So even though a method is un-shared, it still needs to be
> > aware that
> > it may have sibling methods that are shared. If they don't
> > access an
> > overlapping data-set, no special handling is required. If they
> > do
> > overlap, they may need to coordinate appropriately.
>
> I understand your point but I think the current shared (no implicit conversion) has its uses. It can be quite useful to have one interface for when an object is shared and one for when it is not (one with and without the synchronization cost). Sure, something as trivial as a counter can be re-implemented in both ways but more complex objects would easily result in extreme code duplication.

If you can give a single 'use', I'm all ears ;)

> It's also easy to acknowledge that implicit conversion to shared has its uses.

I actually know of many real uses for this case.

> Instead of forcing one way or another, how about we leave the decision up to the programmer? Say something like "alias this shared;" enables implicit conversion to shared.

That might be fine.
Like I said in OP, the first point that I think needs to be agreed on,
is that shared can not read or write members. I think that's a
pre-requisite for any interesting development.

On Mon, Oct 15, 2018 at 7:25 PM Stanislav Blinov via Digitalmars-d <digitalmars-d@puremagic.com> wrote:
>
> On Tuesday, 16 October 2018 at 00:15:54 UTC, Manu wrote:
> > On Mon, Oct 15, 2018 at 4:35 PM Stanislav Blinov via Digitalmars-d <digitalmars-d@puremagic.com> wrote:
>
> >> What?!? So... my unshared methods should also perform all that's necessary for `shared` methods?
>
> > Of course! You're writing a threadsafe object... how could you expect otherwise?
>
> See below.
>
> > Just to be clear, what I'm suggesting is a significant
> > *restriction*
> > to what shared already does... there will be a whole lot more
> > safety under my proposal.
>
> I don't see how an *implicit* cast can be a restriction. At all.

Because a shared pointer can't access anything.
You can't do anything with a shared instance, so the can be no harm done.

Only if there are shared methods (that promise thread-safety) is it
that shared gets interesting.
Without that, it's just a market for the existing recommended use of
shared; which is lock and cast away.

> > The cast gives exactly nothing that attributing a method as
> > shared
> > doesn't give you, except that attributing a method shared is so
> > much
> > more sanitary and clearly communicates intent at the API level.
>
> It's like we're talking about wholly different things here. Casting should be done by the caller, i.e. a programmer that uses some API. If that API expects shared arguments, the caller better make sure they pass shared values. Implicit conversion destroys any obligations between the caller and the API.

Why? What could a function do with shared arguments?

> >> > You can write bad code with any feature in any number of ways.
> >>
> >> Yup. For example, passing an int* to a function expecting shared int*.
> >
> > I don't understand your example. What's the problem you're suggesting?
>
> The problem that I'm suggesting is exactly that: an `int*` is not, and can not, be a `shared int*` at the same time. Substitute int for any type. But D is not Rust and it can't statically prevent that, except for disallowing trivial programming mistakes, which, with implicit conversion introduced, would also go away.

Why not? The guy who receives the argument receives an argument that
*may be shared*, and as such, he's restricted access to it
appropriately.
Just like if you receive a const thing, you can't write to it, even if
the caller's thing isn't const.
If you receive a shared thing, you can't read or write to it.

> >> ...And therefore they lack any synchronization. So I don't see how they *can* be "compatible" with `shared` methods.
> >
> > I don't understand this statement either. Who said they lack synchronisation? If they need it, they will have it. There's a good chance they don't need it though, they might not  interact with a thread-unsafe portion of the class.
>
> Or they might.

Then you will implement synchronisation, or have violated your thread-safety promise.

> >> > If your shared method is incompatible with other methods, your class is broken, and you violate your promise.
> >>
> >> Nope.
> >
> > So certain...
> >
> >> class BigCounter {
> >>
> >>      this() { /* don't even need the mutex if I'm not sharing
> >> this
> >> */ }
> >>
> >>      this(Mutex m = null) shared {
> >>          this.m = m ? m : new Mutex;
> >>      }
> >>
> >>      void increment() { value += 1; }
> >>      void increment() shared { synchronized(m)
> >> *value.assumeUnshared += 1; }
> >>
> >> private:
> >>      Mutex m;
> >>      BigInt value;
> >> }
> >
> > You've just conflated 2 classes into one. One is a threadlocal
> > counter, the other is a threadsafe counter. Which is it?
> > Like I said before: "you can contrive a bad program with
> > literally any language feature!"
>
> Because that is exactly the code that a good amount of "developers" will write. Especially those of the "don't think about it" variety. Don't be mistaken for a second: if the language allows it, they'll write it.

This is not even an argument.
Atomic!int must be used with care. Any threading of ANY KIND must be
handled with care.
Saying we shouldn't make shared useful because someone can do
something wrong is like saying we shouldn't have atomic int's and we
shouldn't have spawn(). They're simply too dangerous to give to
users...

> >> They're not "compatible" in any shape or form.
> >
> > Correct, you wrote 2 different things and mashed them together.
>
> Can you actually provide an example of a mixed shared/unshared class that even makes sense then? As I said, at this point I'd rather see such definitions prohibited entirely.

I think this is a typical sort of construction:

struct ThreadsafeQueue(T)
{
  void QueueItem(T*) shared;
  T* UnqueueItem() shared;
}

struct SpecialWorkList
{
  struct Job { ... }

  void MakeJob(int x, float y, string z) shared  // <- any thread may
produce a job
  {
    Job* job = new Job; // <- this is thread-local
    PopulateJob(job, x, y, z); // <- preparation of a job might be
complex, and worthy of the SpecialWorkList implementation

    jobList.QueueItem(job);  // <- QueueItem encapsulates
thread-safety, no need for blunt casts
  }

  void Flush() // <- not shared, thread-local consumer
  {
    Job* job;
    while (job = jobList.UnqueueItem()) // <- it's obviously safe for
a thread-local to call UnqueueItem even though the implementation is
threadsafe
    {
      // thread-local dispatch of work...
      // perhaps rendering, perhaps deferred destruction, perhaps
deferred resource creation... whatever!
    }
  }

  void GetSpecialSystemState() // <- this has NOTHING to do with the
threadsafe part of SpecialWorkList
  {
    return os.functionThatChecksSystemState();
  }

  // there may be any number of utility functions that don't interact
with jobList.

private:
  void PopulateJob(ref Job job, ...)
  {
    // expensive function; not thread-safe, and doesn't have any
interaction with threading.
  }

  ThreadsafeQueue!Job jobList;
}


This isn't an amazing example, but it's typical of a thing that's
mostly thread-local, and only a small controlled part of it's
functionality is thread-safe.
The thread-local method Flush() also deals with thread-safety
internally... because it flushes a thread-safe queue.

All thread-safety concerns are composed by a utility object, so there's no need for locks, magic, or casts here.

> >> Or would you have
> >> the unshared ctor also create the mutex and unshared increment
> >> also take the lock? What's the point of having them  then?
> >> Better disallow mixed implementations altogether (which is
> >> actually not that bad of an idea).
>
> > Right. This is key to my whole suggestion. If you write a shared thing, you accept that it's shared! You don't just accept it, you jam the stake in the ground.
>
> Then, once more, `shared` should then just be a type qualifier exclusively, and mixing shared/unshared methods should just not be allowed.

1. No.
2. I would have to repeat literally everything I've ever said on this
topic to respond to this comment.

> > There's a relatively small number of things that need to be threadsafe, you won't see `shared` methods appearing at random. If you use shared, you promise threadsafety OR the members of the thing are inaccessible without some sort of lock-&-cast-away treatment.
>
> As above.

I don't understand.

> >> import std.concurrency;
> >> import core.atomic;
> >>
> >> void thread(shared int* x) {
> >>      (*x).atomicOp!"+="(1);
> >> }
> >>
> >> shared int c;
> >>
> >> void main() {
> >>      int x;
> >>      auto tid = spawn(&thread, &x); // "just" a typo
> >> }
> >>
> >> You're saying that's ok, it should "just" compile. It shouldn't. It should produce an error and a mild electric discharge into the developer's chair.
> >
> > Yup. It's a typo. You passed a stack pointer to a scope that
> > outlives the caller.
> > That class of issue is not on trial here. There's DIP1000, and
> > all sorts of things to try and improve safety in terms of
> > lifetimes.
>
> I'm sorry, I'm not very good at writing "real" examples for things that don't exist or don't compile. End of sarcasm.
>
> Let's come back to DIP1000 when it's actually implemented in it's entirety, ok? Anyway, you're nitpicking while actually missing the point altogether. The way `shared` is "implemented" today, the API (`thread` function) *requires* the caller to pass a `shared int*`. Implicit conversion breaks that contract.

So?
What does it mean to pass a `shared int*`?

> At the highest level, the only reason for taking a `shared` argument is to pass that argument to another thread.

Not even. This is the most un-useful application I can think of. It
doesn't really model the problem at all.
Transfer of ownership is a job for move semantics.
shared is for interacting with objects that are *already* owned by many threads.
shared needs to model mechanics to do a limited set of thread-safe
interactions with shared objects that are shared. That would make
shared a useful thing, rather than a giant stain.

> That is the
> *only* way to communicate that intent via the type system for the
> time being.

...but that's shit. And it doesn't communicate that intent at all. Bluntly casting attributes on things is a terrible solution to that proposed problem.

> You're suggesting to ignore that fact.

Yes; everything we think about shared today is completely worthless.
Under my proposal, some existing applications might remain untouched
(they do), but they're not worth worrying about from a design point of
view, because they're not really 'designs'.
Focus on making shared a useful thing, and then see where we're at.

> `shared` was
> supposed to protect from unshared aliasing, not silently allow it.

Inhibiting all access satisfies that protection. It doesn't matter if
a pointer is distributed if you can't access the contents.
Now from there, we need a way to make interacting with guaranteed
thread-safe API's interesting and useful, and I'm describing how to do
that.

> If you allow implicit conversion, there would literally be no way of knowing whether some API will access your data concurrently, other than plain old documentation (or sifting through it's code, which may not be available). This makes `shared` useless as a type qualifier.

That's the whole point though.
A thread-safe think couldn't care less if the data is shared or not,
because it's threadsafe.
Now we're able to describe what's thread-safe, and what's not. This
makes shared *useful* as a type qualifier.

> > You only managed to contrive this by spawning a thread. If it were just a normal function, this would be perfectly legitimate, and again, that's my whole point.
>
> I think you will agree that passing a pointer to a thread-local variable to another thread is not always a safe thing to do.

That's the problem I'm trying to resolve by removing all access.
I'm trying to make that interaction safe, and that's the key to moving
forward as I see it.
If the object is thread-local, then no other thread can access the
object in any way, and it's just a fancy int.

> Conditions do apply, which are on you (the programmer) to uphold, and the compiler can't help you with that. The only way the compiler *can* help you here is make sure you don't do that unintentionally. Which it won't be able to do if you allow such implicit conversion.

You need to demonstrate how the implicit conversion may lead to chaos. The conversion is immensely useful, and I haven't thought how it's a problem yet.