Let me start by saying I'm willing to admit that I was factually wrong.

Also keep in mind that "me having an impression" is something that is can't be independently verified and you'll have to take my at my word. Just that the exact reason for that impression was lost to the sands of time.

On Thursday, 18 October 2018 at 20:13:49 UTC, Stanislav Blinov wrote:
> On Thursday, 18 October 2018 at 20:10:18 UTC, Erik van Velzen wrote:
>
>> When shared stood up in its current form,  expectation was made "this will be threadsafe automatically - we'll figure out how in the future".
>
> It never was like that. At all. I don't think either Walter or Andrei are idiots, do you?
>

Obviously not. But they are not infallible and can also change their mind. And features can be used beyond their initially envisioned purpose.

>> Because it works for global variables. But it doesn't seem like an expectation we can deliver on.
>>
>> (I have no direct reference to this but that was certainly my impression)
>
> Your impression was wrong. Open e.g. TDPL and read up on `shared` how it was envisioned back then.

I don't think the book really supports your argument. The first paragraph about shared sound to me like "the compiler will automagically fix it".

Only tangentially it is mentioned that you're actually supposed to write special code yourself. You would have to be a compiler expert to draw the correct conclusion.

Also the last paragraph the quote below is interesting in light of our other discussion about casting to shared.

From  the book:

""""
13.11  The shared Type Qualifier

(...) To the type system, shared indicates that several threads have access to a piece of data. The compiler acknowledges that reality by restricting operations on shared data and by generating special code for the accepted operations.

(...)

Because all shared data is accounted for and protected under the aegis of the language, passing shared data via send and receive is allowed.

(...)

13.12 Operations with shared Data and Their Effects

Working with shared data is peculiar because multiple threads may read and write it at
any moment. Therefore, the compiler makes sure that all operations preserve integrity
of data and also causality of operations.
    Reads and writes of shared values are allowed and guaranteed to be atomic: nu-
meric types (save for real), pointers, arrays, function pointers, delegates, and class ref-
erences. struct types containing exactly one of the mentioned types are also readable
and writable atomically. (...)
    For all numeric types and function pointers, shared-qualified values are convertible implicitly to and from unqualified values. Pointer conversions between shared(T*) and shared(T)* are allowed in both directions. Primitives in std.concurrency allow you to
do arithmetic on shared numeric types.
""""

On Thursday, 18 October 2018 at 20:59:59 UTC, Erik van Velzen wrote:
>
> Let me start by saying I'm willing to admit that I was factually wrong.
>
> Also keep in mind that "me having an impression" is something that is can't be independently verified and you'll have to take my at my word. Just that the exact reason for that impression was lost to the sands of time.

Quite a simple reason: it was years ago, however old you are now you were younger and less experienced, and probably didn't understand something back then.

>> Your impression was wrong. Open e.g. TDPL and read up on `shared` how it was envisioned back then.
>
> I don't think the book really supports your argument. The first paragraph about shared sound to me like "the compiler will automagically fix it".

Then I don't know what to tell you. It literally talks about compiler forbidding unsafe operations and *requiring* you to go the extra mile, by just rejecting invalid code (something that Manu is proposing to forego!). But that's *code*, not logic.

> Only tangentially it is mentioned that you're actually supposed to write special code yourself. You would have to be a compiler expert to draw the correct conclusion.

Tangetially?! There's a whole section on writing `shared`-aware code (none of which would even compile today, I don't know if it's addressed in his errata).

> Also the last paragraph the quote below is interesting in light of our other discussion about casting to shared.
>
> From  the book:
>
> [snip]

Yeah, some of that never happened and never will. But that aside, none of it says "threading will be safe by default". It says "threading will be a lot less unsafe by default". And *that* is what we must achieve.

On Thursday, 18 October 2018 at 21:14:54 UTC, Stanislav Blinov wrote:
> On Thursday, 18 October 2018 at 20:59:59 UTC, Erik van Velzen wrote:
>> [...]
>
> Quite a simple reason: it was years ago, however old you are now you were younger and less experienced, and probably didn't understand something back then.
>
>> [...]
>
> Then I don't know what to tell you. It literally talks about compiler forbidding unsafe operations and *requiring* you to go the extra mile, by just rejecting invalid code (something that Manu is proposing to forego!). But that's *code*, not logic.
>
>> [...]
>
> Tangetially?! There's a whole section on writing `shared`-aware code (none of which would even compile today, I don't know if it's addressed in his errata).
>
>> [...]
>
> Yeah, some of that never happened and never will. But that aside, none of it says "threading will be safe by default". It says "threading will be a lot less unsafe by default". And *that* is what we must achieve.

The "threading will be a lot less unsafe by default" is related to the default TLS usage.

I remember like Erik, maybe wrongly, that the ambitions on shared were more directed  towards the "threading will be safe by default" goal.

I've to read again some post from Bartosz Milewski...

/Paolo

On Thursday, 18 October 2018 at 16:24:39 UTC, Manu wrote:
> On Wednesday, 17 October 2018 at 22:56:26 UTC, H. S. Teoh wrote:
>> What cracks me up with Manu's proposal is that it is its simplicity and lack of ambition that is criticized the most. shared is a clusterfuck, according to what I gathered from the forum, I never had yet to use it in my code. Manu's idea makes it a little less of a clusterfuck, and people attack the idea because it doesn't solve all and everything that's wrong with shared. Funny.
>>
>
> Elaborate on this... It's clearly over-ambitious if anything.
> What issues am I failing to address?

First of all, you called it "shared", but what your concept describes is "theadsave".
If you had called it the later, it would have been clear to everybody that thread local data is indeed automatically threadsave, because only one thread has access to it (that "implicit conversion"). But if something is "shared" (in the common-world sense), it is of course no more "threadsave" - you have to implement special methods to treat it.

Conflating "shared" and "threadsave" in that manner was, I think, the biggest mistake of your proposal.

Another point is the part of "how can the compiler support the expert in writing threadsave methods" - which you answered with "not a little bit at the moment, but we may improve this in the future" - and that is not at all satisfying. Are there really no ideas? No check that the proper atomic funtions are used or the cast to "unshared" is ok at where it is used? Even the expert needs a little help to find the upcomming and well hidden bugs in their oh so threadsave API...

On Friday, 19 October 2018 at 13:40:54 UTC, Dominikus Dittes Scherkl wrote:
> On Thursday, 18 October 2018 at 16:24:39 UTC, Manu wrote:
>> On Wednesday, 17 October 2018 at 22:56:26 UTC, H. S. Teoh wrote:
>>> What cracks me up with Manu's proposal is that it is its simplicity and lack of ambition that is criticized the most. shared is a clusterfuck, according to what I gathered from the forum, I never had yet to use it in my code. Manu's idea makes it a little less of a clusterfuck, and people attack the idea because it doesn't solve all and everything that's wrong with shared. Funny.
>>>
>>
>> Elaborate on this... It's clearly over-ambitious if anything.
>> What issues am I failing to address?
>
> First of all, you called it "shared", but what your concept describes is "theadsave".
> If you had called it the later, it would have been clear to everybody that thread local data is indeed automatically threadsave, because only one thread has access to it (that "implicit conversion"). But if something is "shared" (in the common-world sense), it is of course no more "threadsave" - you have to implement special methods to treat it.
>
> Conflating "shared" and "threadsave" in that manner was, I think, the biggest mistake of your proposal.

He talked about it in a previous thread, and generally I would agree with him that such conflation is indeed beneficial provided that some concessions are made for `shared`. Moreover, yet another attribute? Please no...

struct X {
    void foo(threadsafe const shared Bar* bar) @nogc @trusted notrhow pure const shared threadsafe;
}

Attribute explosion is bad enough already.

On Fri, Oct 19, 2018 at 6:45 AM Dominikus Dittes Scherkl via Digitalmars-d <digitalmars-d@puremagic.com> wrote:
>
> On Thursday, 18 October 2018 at 16:24:39 UTC, Manu wrote:
> > On Wednesday, 17 October 2018 at 22:56:26 UTC, H. S. Teoh wrote:
> >> What cracks me up with Manu's proposal is that it is its simplicity and lack of ambition that is criticized the most. shared is a clusterfuck, according to what I gathered from the forum, I never had yet to use it in my code. Manu's idea makes it a little less of a clusterfuck, and people attack the idea because it doesn't solve all and everything that's wrong with shared. Funny.
> >>
> >
> > Elaborate on this... It's clearly over-ambitious if anything. What issues am I failing to address?
>
> First of all, you called it "shared", but what your concept
> describes is "theadsave".
> If you had called it the later, it would have been clear to
> everybody that thread local data is indeed automatically
> threadsave, because only one thread has access to it (that
> "implicit conversion"). But if something is "shared" (in the
> common-world sense), it is of course no more "threadsave" - you
> have to implement special methods to treat it.
>
> Conflating "shared" and "threadsave" in that manner was, I think, the biggest mistake of your proposal.
>
> Another point is the part of "how can the compiler support the expert in writing threadsave methods" - which you answered with "not a little bit at the moment, but we may improve this in the future" - and that is not at all satisfying.

I think you've misunderstood.
My proposal is @safe... if you stay @safe, you will receive guarantee
that your code is threadsafe.
If you want to implement a low-level device, you must implement a
@trusted function, and I don't know what the compiler can do to help
you.
You will have to produce unsafe casts, so at least it will be
completely clear every operation you perform that is unsafe.
Users of your @trusted library though will experience @safe code
upward through the stack.

So saying that my response that "there is @trusted code at the bottom of the stack" is not satisfying is really just a comment on your opinion about @trusted code in general.

My proposal is designed to be useful and @safe for *users* as primary
goal. Authors that are composing low-level tools/libs will enjoy @safe
guarantees. Only authors implementing ground-level machinery would
need to write @trusted code... and that's the nature of the whole
@safe stack.
@safe can't practically exist without @trusted at the bottom of the stack.

> Are there really no
> ideas?

I have some ideas, but I don't think they're practical to implement.
Some cross-referencing of access would be required for the compiler to
suspect foul-play.
Functions would need to be analysed in conjunction; that sounds hard
to implement.

> No check that the proper atomic funtions are used or the cast to "unshared" is ok at where it is used?

The user has manually cast to unshared inside their unsafe/(@trusted?) function, what more signal do they need that they've engaged in an unsafe operation?

> Even the expert
> needs a little help to find the upcomming and well hidden bugs in
> their oh so threadsave API...

I think a lot of people probably over-estimate the number of tooling
libs that live at the bottom of the stack.
The list is fairly short: Atomic, Mutex/Semaphore, and a couple of
lock-free-queue/list type structures.
I don't know what other useful constructs exist... they will all be in
libraries. Users would almost never come in contact with unsafe code,
and again, that's the whole point of my design!

On Thursday, 18 October 2018 at 19:37:24 UTC, H. S. Teoh wrote:
> On Thu, Oct 18, 2018 at 07:09:42PM +0000, Patrick Schluter via Digitalmars-d wrote: [...]
>> I often have the impression that a lot of things are going slower than necessary because a mentality where the perfect is in the way of good.
>
> That is indeed an all-too-frequent malady around these parts, sad to say. Which has the sad consequence that despite all efforts, there are still unfinished areas in D, and promises that haven't materialized in years (like multiple alias this).
>
> Still, the parts of D that are working well form a very powerful and comfortable-to-use language.  Not quite the ideal we wish it to be, but IMO much closer than any other language I've seen yet.  Recently I began dabbling in Android programming, and the one thing that keeps sticking out to me is how painful writing Java is.  Almost every day of writing Java code has me wishing for this or that feature in D.  Slices. Closures.  Meta-programming.  I found most of my time spent fighting with language limitations rather than make progress with the problem domain.

Yes, this is why I began the Android port: I couldn't imagine writing Java.

> Eventually I resorted to generating Java code from D for some fo the most painful repetitive parts, and the way things are looking, I'm likely to be doing a lot more of that.  I fear the way things are going will have be essentially writing a D to Java compiler at some point!

Why not just use the Android port of D?

On Fri, Oct 19, 2018 at 06:34:50PM +0000, Joakim via Digitalmars-d wrote:
> On Thursday, 18 October 2018 at 19:37:24 UTC, H. S. Teoh wrote:
[...]
> > Eventually I resorted to generating Java code from D for some fo the most painful repetitive parts, and the way things are looking, I'm likely to be doing a lot more of that.  I fear the way things are going will have be essentially writing a D to Java compiler at some point!
> 
> Why not just use the Android port of D?

I want to.  But I couldn't get the cross-compiler setup properly:

	https://forum.dlang.org/post/mailman.4361.1539811552.29801.digitalmars-d@puremagic.com

If I can get past that hurdle, Java is going out the window pronto. :-D


T

-- 
Everybody talks about it, but nobody does anything about it!  -- Mark Twain

On Friday, 19 October 2018 at 18:11:50 UTC, Manu wrote:
> On Fri, Oct 19, 2018 at 6:45 AM Dominikus Dittes Scherkl via Digitalmars-d <digitalmars-d@puremagic.com> wrote:
>>
>> On Thursday, 18 October 2018 at 16:24:39 UTC, Manu wrote:
>> > [...] What issues am I failing to address?
>> [...] Another point is the part of "how can the compiler support the expert in writing threadsave methods" - which you answered with "not a little bit at the moment, but we may improve this in the future" - and that is not at all satisfying.
>
> I think you've misunderstood.
> My proposal is @safe... if you stay @safe, you will receive guarantee that your code is threadsafe.
On user side, yes.
> If you want to implement a low-level device, you must implement a @trusted function, and I don't know what the compiler can do to help you.
Yes, but that's seldom. More often the "expert" will write new shared types using the low level trusted functions like anybody else. But that
still requires special care - he has to consider tread-safety in every
method of a new type, even the non-shared ones. And he has to fill any
possible gap like construction and assignment so that the end-user is really sure to not accidentally misusing the type!
And I think a serious proposal need to address this - I think the compiler could really help here (e.g. prescribe what operators need to be overloaded and check that all methods use the proper mechanisms to lock the shared members before operating on them etc.)

> So saying that my response that "there is @trusted code at the bottom of the stack" is not satisfying is really just a comment on your opinion about @trusted code in general.
That just comes on top of it.

> My proposal is designed to be useful and @safe for *users* as primary goal.
I agree with you, but others seem not so convinced (yet?).
[...]
> The user has manually cast to unshared inside their unsafe/(@trusted?) function, what more signal do they need that they've engaged in an unsafe operation?
Some hints what to do to be able to trust them?
You asked what issues you were failing to address. That was just some ideas of mine what you may address in addition.

On Friday, 19 October 2018 at 15:46:20 UTC, Stanislav Blinov wrote:
> On Friday, 19 October 2018 at 13:40:54 UTC, Dominikus Dittes Scherkl wrote:
>> Conflating "shared" and "threadsave" in that manner was, I think, the biggest mistake of your proposal.
>
> He talked about it in a previous thread, and generally I would agree with him that such conflation is indeed beneficial provided that some concessions are made for `shared`. Moreover, yet another attribute? Please no...

Hmm.
mutable, immutable and const form a triple, the second is a declaration attribute, the last an parameter attribute, indicating that you don't want to modify the parameter, may it be because you can't (as it is immutable) or you only don't need to despite it would be possible (if it was mutable). The later is your responsibility to guarantee (with the help from the compiler).
Therefore it is possible to implicitly cast from mutable or immutable to const but not in any other direction.

I think for unshared, shared and threadsave it should be the same:
The second is a declaration attribute, the third a parameter attribute. The first two can implicitly be cast to threadsave, may be because it is thread-local and therefore no race condition is possible, or may be because you take special care in your type to guarantee the thread safety by using atomic operations or locking or whatever.
That make it possible, that the implicit cast from shared to unshared can be avoided while still providing functions that can take both kinds of arguments.

Yes, that would add a little to the attribute bloat (new keyword) but not to the number of attributes per type or parameter.