On Friday, 19 October 2018 at 00:29:01 UTC, Timon Gehr wrote:
> On 18.10.18 23:34, Erik van Velzen wrote:
>> If you have an object which can be used in both a thread-safe and a thread-unsafe way that's a bug or code smell.
>
> Then why do you not just make all members shared? Because with Manu's proposal, as soon as you have a shared method, all members effectively become shared. It just seems pointless to type them as unshared anyway and then rely on convention within @safe code to prevent unsafe accesses. Because, why? It just makes no sense.

Let's assume you have something like this:

module foo;

private shared int sharedState;

struct Accessor {

    int flags;

    void addUser(this T)()   {
       static if (is(T == shared))
           sharedState.atomicInc(); // unconditionally increment when it's a shared reference
       else {
           // owner may optimize shared access based on it's own state
           if (!(flags & SKIP_LOCKS)) sharedState.atomicInc();
       }
    }

    void removeUser(this T)() {
        static if (is(T == shared))
            sharedState.atomicDec();
        else {
            if (!(flags & SKIP_LOCKS)) sharedState.atomicDec();
        }
    }

    void setFlags(int f) { flags = f; }
}

The 'Accessor' doesn't really hold any shared state, but it accesses a shared module global. Now, the *owner* (e.g. code that instantiated a local Accessor) may use non-`shared` interface to track additional state and make decisions whether or not access the global. A concrete example would be e.g. an I/O lock, where if you know you don't have any threads other than main, you can skip syscalls locking/unlocking the handle.

> With the proposal I posted in the beginning, you would then not only get implicit conversion of class references to shared, but also back to unshared.

Is it in your first posts in this thread? I must've skipped that.

> I think the conflation of shared member functions and thread safe member functions is confusing. shared on a member function just means that the `this` reference is shared.

Nod.

> The only use case for this is overloading on shared. The D approach to multithreading is that /all/ functions should be thread safe, but it is easier for some of them because they don't even need to access any shared state. It is therefore helpful if the type system cleanly separates shared from unshared state.

Nod nod.

On Thu, Oct 18, 2018 at 5:30 PM Timon Gehr via Digitalmars-d <digitalmars-d@puremagic.com> wrote:
>
> On 18.10.18 23:34, Erik van Velzen wrote:
> > If you have an object which can be used in both a thread-safe and a thread-unsafe way that's a bug or code smell.
>
> Then why do you not just make all members shared? Because with Manu's proposal, as soon as you have a shared method, all members effectively become shared.

No they don't, only facets that overlap with the shared method. I tried to present an example before:

struct Threadsafe
{
  int x;
  Atomic!int y;
  void foo() shared { ++y; } // <- shared interaction only affects 'y'
  void bar() { ++x; ++y; } // <- not threadsafe, but does not violate
foo's commitment; only interaction with 'y' has any commitment
associated with it
  void unrelated() { ++x; } // <- no responsibilities are transposed
here, you can continue to do whatever you like throughout the class
where 'y' is not concerned
}

In practise, and in my direct experience, classes tend to have exactly one 'y', and either zero (pure utility), or many such 'x' members. Threadsafe API interacts with 'y', and the rest is just normal thread-local methods which interact with all members thread-locally, and may also interact with 'y' while not violating any threadsafety commitments.

> It just seems pointless to type them as unshared anyway
> and then rely on convention within @safe code to prevent unsafe
> accesses. Because, why? It just makes no sense.

The pattern, is almost 100% of cses is: owner does all sorts of
stuff... threadsafe interactions with things happen in high-frequency
worker things. Parallel-for type workloads happen in bursts, the rest
of the program works like normal.
Parallel for workloads are restricted to the threadsafe API.

> With the proposal I posted in the beginning, you would then not only get implicit conversion of class references to shared, but also back to unshared.

I'm not sure how your proposal (which felt way more complicated to me)
improved on my design.
Can you clarify how my very simple design is ineffective?

> I think the conflation of shared member functions and thread safe member functions is confusing.

What else can it possibly mean to be a shared method? If it's not threadsafe, it's an insta-crash...

> shared on a member function just means that the
> `this` reference is shared.

And by extension, you can only interact with other threadsafe methods.

> The only use case for this is overloading on
> shared.

No, it's for separating threadsafe methods from those that aren't.
Just like const methods separate mutating methods from those that don't.

> The D approach to multithreading is that /all/ functions should
> be thread safe, but it is easier for some of them because they don't
> even need to access any shared state. It is therefore helpful if the
> type system cleanly separates shared from unshared state.

This is patently untrue. I can't implement any interesting shared
architecture with the design as is today.
I am also completely free to violate any sense of threadsafety with no
restrictions of any kind.
Write a shared method, access any members, call it from shared
instances, watch the fire burn.

I don't understand how you can claim that D's design says that all functions are threadsafe, it's so plainly far from the truth...?

On Thu, Oct 18, 2018 at 3:10 PM Simen Kjærås via Digitalmars-d <digitalmars-d@puremagic.com> wrote:
>
>
> Now, Two very good points came up in this post, and I think it's worth stating them again, because they do present possible issues with MP:

It is easy to respond to these.

> 1) How does MP deal with reorderings in non-shared methods?
>
> I don't know. I'd hide behind 'that's for the type implementor to handle', but it's a subtle enough problem that I'm not happy with that answer.

This is a red-herring. Opaque function calls are never reordered.
If they are inlined, the compiler has full visibility to the internal
machinery present.

If you call one function that performs an atomic op, then another that
performs an atomic op, it is impossible for the CPU to reorder atomic
op's around eachother, that would defeat the entire point of hardware
atomic operations.
If the functions used mutexes, or semaphores, exactly the same
applies; they are either function calls (not reordered), or they use
hardware primitives, which have scheduling fences.

If none of those things are present in the threadsafe functions, then the functions aren't threadsafe in the first place!

In short, he made up this issue, it doesn't exist.

> 2) What about default members like opAssign and postblit?
>
> The obvious solution is for the compiler to not generate these when a type has a shared method or is taken as shared by a free function in the same module. I don't like the latter part of that, but it should work.

These aren't issues either. There's nothing wrong with atomic
assignment; you just have to implement an atomic assignment.
Postblit is being replaced with copy-ctor's and `shared` is one of the
explicit reasons why! Copy-ctor's are also fine, it would express an
atomic assignment.


This is just hot air, and only strengthen my conviction.

On Friday, 19 October 2018 at 01:22:53 UTC, Manu wrote:
> On Thu, Oct 18, 2018 at 3:10 PM Simen Kjærås via Digitalmars-d <digitalmars-d@puremagic.com> wrote:
>>
>>
>> Now, Two very good points came up in this post, and I think it's worth stating them again, because they do present possible issues with MP:
>
> It is easy to respond to these.
>
>> 1) How does MP deal with reorderings in non-shared methods?
>>
>> I don't know. I'd hide behind 'that's for the type implementor to handle', but it's a subtle enough problem that I'm not happy with that answer.
>
> This is a red-herring. Opaque function calls are never reordered.
> If they are inlined, the compiler has full visibility to the internal machinery present.

You don't say?.. And what, exactly, stops the optimizer from removing "unnecessary" reads or rearranging them with stores, given the original code, which, if you freaking read it, you'd see there's no indication that it's not allowed to do so.

> If you call one function that performs an atomic op, then another that performs an atomic op, it is impossible for the CPU to reorder atomic op's around eachother, that would defeat the entire point of hardware atomic operations.

I'm not talking about CPU reordering at all. I'm talking about the optimizer.

> In short, he made up this issue, it doesn't exist.

Yeeees, of course I have. What else have I made up, can you tell? You know what doesn't exist though? Even one example of a useful implicit conversion form mutable to shared from you. Not even one.

>> 2) What about default members like opAssign and postblit?
>>
>> The obvious solution is for the compiler to not generate these when a type has a shared method or is taken as shared by a free function in the same module. I don't like the latter part of that, but it should work.
>
> These aren't issues either. There's nothing wrong with atomic
> assignment; you just have to implement an atomic assignment.

You just haven't read the code. Those members aren't even `shared`. The *least* you can do is disable them *if* you're going to cast your variable to `shared`. Otherwise your "interface" remains non-threadsafe.

> Postblit is being replaced with copy-ctor's and `shared` is one of the explicit reasons why! Copy-ctor's are also fine, it would express an atomic assignment.

And this strengthens *my* belief that you haven't at all thought about this. There is literally *no* purpose for any `shared` types to have any copy-ctors. The only feasible copy primitives are from shared to local and from local to shared. Not to mention that again, to even talk about your "implicit" conversions, you must first think about what can happen to the *owned* (non-`shared`) reference after the conversion. Hint: you can't copy it. You can't assign *to it*. Not via default-generated postblits and opAssigns, which are not, and can not, be "atomic".
I'm fully aware about postblits being "replaced" by copy-ctors, I'm also fully aware how "much" thought was put into that wrt. `shared`.

> This is just hot air, and only strengthen my conviction.

You know what, I'm fed up with you too. Just show me one, *one* non-contrived example of useful implicit conversion from mutable to shared. So far you haven't produced *any at all*. Then we can talk about what is hot air here. Produce, or drop this presumptious crap.

On Thu, Oct 18, 2018 at 6:50 PM Stanislav Blinov via Digitalmars-d <digitalmars-d@puremagic.com> wrote:
>
> On Friday, 19 October 2018 at 01:22:53 UTC, Manu wrote:
> > On Thu, Oct 18, 2018 at 3:10 PM Simen Kjærås via Digitalmars-d <digitalmars-d@puremagic.com> wrote:
> >>
> >>
> >> Now, Two very good points came up in this post, and I think it's worth stating them again, because they do present possible issues with MP:
> >
> > It is easy to respond to these.
> >
> >> 1) How does MP deal with reorderings in non-shared methods?
> >>
> >> I don't know. I'd hide behind 'that's for the type implementor to handle', but it's a subtle enough problem that I'm not happy with that answer.
> >
> > This is a red-herring. Opaque function calls are never
> > reordered.
> > If they are inlined, the compiler has full visibility to the
> > internal machinery present.
>
> You don't say?.. And what, exactly, stops the optimizer from removing "unnecessary" reads or rearranging them with stores, given the original code, which, if you freaking read it, you'd see there's no indication that it's not allowed to do so.
>
> > If you call one function that performs an atomic op, then another that performs an atomic op, it is impossible for the CPU to reorder atomic op's around eachother, that would defeat the entire point of hardware atomic operations.
>
> I'm not talking about CPU reordering at all. I'm talking about the optimizer.
>
> > In short, he made up this issue, it doesn't exist.
>
> Yeeees, of course I have. What else have I made up, can you tell? You know what doesn't exist though? Even one example of a useful implicit conversion form mutable to shared from you. Not even one.
>
> >> 2) What about default members like opAssign and postblit?
> >>
> >> The obvious solution is for the compiler to not generate these when a type has a shared method or is taken as shared by a free function in the same module. I don't like the latter part of that, but it should work.
> >
> > These aren't issues either. There's nothing wrong with atomic assignment; you just have to implement an atomic assignment.
>
> You just haven't read the code. Those members aren't even `shared`. The *least* you can do is disable them *if* you're going to cast your variable to `shared`. Otherwise your "interface" remains non-threadsafe.
>
> > Postblit is being replaced with copy-ctor's and `shared` is one of the explicit reasons why! Copy-ctor's are also fine, it would express an atomic assignment.
>
> And this strengthens *my* belief that you haven't at all thought about this. There is literally *no* purpose for any `shared` types to have any copy-ctors. The only feasible copy primitives are from shared to local and from local to shared. Not to mention that again, to even talk about your "implicit" conversions, you must first think about what can happen to the *owned* (non-`shared`) reference after the conversion. Hint: you can't copy it. You can't assign *to it*. Not via default-generated postblits and opAssigns, which are not, and can not, be "atomic". I'm fully aware about postblits being "replaced" by copy-ctors, I'm also fully aware how "much" thought was put into that wrt. `shared`.
>
> > This is just hot air, and only strengthen my conviction.
>
> You know what, I'm fed up with you too. Just show me one, *one* non-contrived example of useful implicit conversion from mutable to shared. So far you haven't produced *any at all*. Then we can talk about what is hot air here. Produce, or drop this presumptious crap.

You are an obscene person. I'm out.
You win.

On Friday, 19 October 2018 at 01:53:00 UTC, Manu wrote:

> This is a red-herring.
> In short, he made up this issue, it doesn't exist.
> This is just hot air, and only strengthen my conviction.

>> Produce, or drop this presumptious crap.

> You are an obscene person. I'm out.

Oooh, I'm soooorry, come baack!
Really though, what is it that you wanted to achieve here? You ask for counter-arguments, are given them on *17 pages already*, are asked numerous times to actually demonstrate the value of a small contained portion of your proposal, and all you do is shrug this all off just because you presume to "know better", and on top of that have the audacity to call someone else *obscene*? Wow... just... wow!

> You win.

I didn't know it was a contest.

On Thu., 18 Oct. 2018, 7:10 pm Stanislav Blinov via Digitalmars-d, < digitalmars-d@puremagic.com> wrote:

> On Friday, 19 October 2018 at 01:53:00 UTC, Manu wrote:
>
> > This is a red-herring.
> > In short, he made up this issue, it doesn't exist.
> > This is just hot air, and only strengthen my conviction.
>
> >> Produce, or drop this presumptious crap.
>
> > You are an obscene person. I'm out.
>
> Oooh, I'm soooorry, come baack!
> Really though, what is it that you wanted to achieve here? You
> ask for counter-arguments, are given them on *17 pages already*,
> are asked numerous times to actually demonstrate the value of a
> small contained portion of your proposal, and all you do is shrug
> this all off just because you presume to "know better", and on
> top of that have the audacity to call someone else *obscene*?
> Wow... just... wow!
>
> > You win.
>
> I didn't know it was a contest.
>

I've given use cases constantly, about taking object ownership, promotions, and distribution for periods (think parallel for), I can achieve all my goals with full @safety, absolutely no casts in user code, and I have infrastructure in production that applies these patterns successfully. It's worth pursuing.

I've spent years thinking on this, I'm trying to move the needle on this issue for the first time in over a decade at least, and you have violently opposed, in principle, from the very first post, and make no effort to actually understand the proposition.

It's clearly a contest from your insurance that my proposal in worthless in every single post you've made. You want me to admit defeat and desist. Fuck you. You win. I don't have the time or energy to argue against a wall.

You are obscene, you're complete unproductive, and destructive from no
apparent reason.
I hope you continue to love shared, just the way it is... useless.

>

On Friday, 19 October 2018 at 02:20:22 UTC, Manu wrote:

> I've given use cases constantly, about taking object ownership, promotions, and distribution for periods (think parallel for),

Manu, you haven't shown *any* code in which conversion from mutable to shared, an *implicit* one at that, was even present, let alone useful. While at the same time blatantly dismissing *all* examples to the contrary as "bad code" or any other reason, especially the "you just don't understand". How do you expect us to understand if all you do is talk about how it's useful, but don't explain why?
You can talk and brag all you want, but until you do show at least one example, I'm sorry but you have no case. And what's this all about? A *small*, localized portion of your proposal, that isn't event the source of the problems you're talking about in the current state of affairs.
In all examples flying about in this thread, all of *required* casts were *the opposite*, from shared to mutable, and you yourself acknowledge that those are indeed required on the lowest level.
As I've said numerous times, from all that's been said and implied in this thread, the *only* use of implicit casting that comes to mind is avoiding writing some forwarding methods, that's about it. I honestly can't see how this is valuable to warrant such a drastic change in the language. Now maybe it's that I'm dumb, then just say so already and we'll all move on. Just don't act like you're the only one who knows it all and everyone else is an unwashed pleb clinging to worshiping the faces in the trees.

> I can achieve all my goals with full @safety, absolutely no casts in user code, and I have infrastructure in production that applies these patterns successfully. It's worth pursuing.

Okay... Then please do show *one* example of useful implicit conversion from mutable to shared.

> I've spent years thinking on this, I'm trying to move the needle on this issue for the first time in over a decade at least,

And you're behaving in this thread as if everyone else, myself included, were sitting on our thumbs counting flies for this same decade.

> and you have violently opposed, in principle, from the very first post, and make no effort to actually understand the proposition.

I haven't done such a thing. I have asked you, numerous times, one, and only one question, and you never so much as replied to *that question*. What is the big value of this implicit conversion that it would warrant changing current language rules regarding type conversions?

> It's clearly a contest from your insurance that my proposal in worthless in every single post you've made. You want me to admit defeat and desist.

And here you are, continuing to presume. Now you suddenly know what I want. Marvelous.

I *don't* want you to admit any defeat, *or* desist. I *want* you to succeed. I *want* to help make `shared` useful so that I and everyone else can actually start writing code with it, not in spite of it. I've agreed with you on pretty much everything in your proposal, *except one thing*. I want you to demonstrate the practical value of *that thing*, and it's benefits over the current state of affairs, and I asked you several times to explain to us mere unwashed mortals exactly how it's useful. What have we been doing for 17 pages? Discussing the benefits of disabling reads/writes on shared? No. Estimating how much existing code could go to trash, what parts of DRuntime/Phobos would need a rewrite? No. We were in constant back-and-forth of "But... nope... but... nope" about this implicit conversion, which you value so much yet for some reason fail to defend. Saying "I had good time with it" is not a very practical defense, not for a language construct anyway.

> Fuck you. You win. I don't have the time or energy to argue against a wall.

If you ask to destroy, be prepared for a fight. Or don't ask. Just stop appealing to your own authority.

> You are obscene, you're complete unproductive, and destructive from no apparent reason.

Give it all you've got, please. Let it all out all at once.

> I hope you continue to love shared, just the way it is... useless.

Yet another presumption. Good on you.

On Friday, 19 October 2018 at 02:20:22 UTC, Manu wrote:
> On Thu., 18 Oct. 2018, 7:10 pm Stanislav Blinov via Digitalmars-d, < digitalmars-d@puremagic.com> wrote:
>
>> On Friday, 19 October 2018 at 01:53:00 UTC, Manu wrote:
>>
>> > This is a red-herring.
>> > In short, he made up this issue, it doesn't exist.
>> > This is just hot air, and only strengthen my conviction.
>>
>> >> Produce, or drop this presumptious crap.
>>
>> > You are an obscene person. I'm out.
>>
>> Oooh, I'm soooorry, come baack!
>> Really though, what is it that you wanted to achieve here? You
>> ask for counter-arguments, are given them on *17 pages already*,
>> are asked numerous times to actually demonstrate the value of a
>> small contained portion of your proposal, and all you do is shrug
>> this all off just because you presume to "know better", and on
>> top of that have the audacity to call someone else *obscene*?
>> Wow... just... wow!
>>
>> > You win.
>>
>> I didn't know it was a contest.
>>
>
> I've given use cases constantly, about taking object ownership, promotions, and distribution for periods (think parallel for), I can achieve all my goals with full @safety, absolutely no casts in user code, and I have infrastructure in production that applies these patterns successfully. It's worth pursuing.
>
> I've spent years thinking on this, I'm trying to move the needle on this issue for the first time in over a decade at least, and you have violently opposed, in principle, from the very first post, and make no effort to actually understand the proposition.
>
> It's clearly a contest from your insurance that my proposal in worthless in every single post you've made. You want me to admit defeat and desist. Fuck you. You win. I don't have the time or energy to argue against a wall.
>
> You are obscene, you're complete unproductive, and destructive from no
> apparent reason.
> I hope you continue to love shared, just the way it is... useless.

There's another way; Stanislav isn't one you need to convince so if that particular discussion is unproductive and disruptive just ignore it. I.e technical discussions should be robust but once they become personal just ignore that input and move on. Isn't always possible I know but in this case I reckon you can.

Convincing Walter, Andrei and the rest of the core dev team of course will require a DIP.

Keep going on this, it is the first hint of movement with shared since like foreva!

bye,
norm

bye,
Norm

On 19/10/2018 7:09 PM, Norm wrote:
> There's another way; Stanislav isn't one you need to convince so if that particular discussion is unproductive and disruptive just ignore it. I.e technical discussions should be robust but once they become personal just ignore that input and move on. Isn't always possible I know but in this case I reckon you can.

Trust me, I think a few of us at least have already figured that out.

> Convincing Walter, Andrei and the rest of the core dev team of course will require a DIP.
> 
> Keep going on this, it is the first hint of movement with shared since like foreva!

As long as it doesn't look like my idea[0] (Andrei doesn't like it, I may have asked) it should have some sort of legs.

[0] https://github.com/rikkimax/DIPs/blob/shared/DIPs/DIP1xxx-RC2.md