I'd like to add that if the compiler can prove that a T* points to a unique T, then it can be implicitly cast to shared(T)*. And it does so, like the result of .dup can be so converted.

On Sun, Oct 21, 2018 at 3:00 AM Walter Bright via Digitalmars-d <digitalmars-d@puremagic.com> wrote:
>
> On 10/20/2018 11:08 AM, Nicholas Wilson wrote:
> > You can if no-one else writes to it, which is the whole point of Manu's proposal. Perhaps it should be const shared instead of shared but still.
>
> There is no purpose whatsoever to data that can be neither read nor written.

There is no primitive type with implicit threadsafety... nor need there be.
Shared data simply can not be read or written in any threadsafe
manner. This is a rock-solid reality, and the type system needs to
reflect that reality as a fundamental premise.
Only from there can we start to define meaningful threadsafety.

All threadsafe interactions with anything involve calling functions. It's completely reasonable to make `shared` inhibit all read and write access to data. We can only call shared methods, because only real-code can implement threadsafety.

> Shared data is only useful if, at some point, it is read/written, presumably by casting it to unshared in @trusted code. As soon as that is done, you've got a data race with the other existing unshared aliases.

If such a race is possible, then the @trusted function is not
threadsafe, so it is not @trusted by definition.
You wrote a bad @trusted function, and you should feel bad.

The simplest way to guarantee that no unsafe access is possible is to use encapsulation to assure no unregulated access exists.

On 21.10.18 17:54, Nicholas Wilson wrote:
> 
>> As soon as that is done, you've got a data race with the other existing unshared aliases.
> 
> You're in @trusted code, that is the whole point. The onus is on the programmer to make that correct, same with regular @safe/@trusted@system code.

Not all of the parties that participate in the data race are in @trusted code. The point of @trusted is modularity: you manually check @trusted code according to some set of restrictions and then you are sure that there is no memory corruption.

Note that you are not allowed to look at any of the @safe code while checking your @trusted code. You will only see an opaque interface to the @safe code that you call and all you know is that all the @safe code type checks according to @safe rules. Note that there might be an arbitrary number of @safe functions and methods that you do not see.

Think about it this way: you first write all the @trusted and @system code, and some evil guy who does not like you comes in after you looks at your code and writes all the @safe code. If there is any memory corruption, it will be your fault and you will face harsh consequences. Now, design the @safe type checking rules. It won't be MP!

Note that there may well be a good way to get the good properties of MP without breaking the type system, but MP itself is not good because it breaks @safe.

On Sun, Oct 21, 2018 at 12:00 PM Timon Gehr via Digitalmars-d <digitalmars-d@puremagic.com> wrote:
>
> On 21.10.18 17:54, Nicholas Wilson wrote:
> >
> >> As soon as that is done, you've got a data race with the other existing unshared aliases.
> >
> > You're in @trusted code, that is the whole point. The onus is on the programmer to make that correct, same with regular @safe/@trusted@system code.
>
> Not all of the parties that participate in the data race are in @trusted code. The point of @trusted is modularity: you manually check @trusted code according to some set of restrictions and then you are sure that there is no memory corruption.
>
> Note that you are not allowed to look at any of the @safe code while checking your @trusted code. You will only see an opaque interface to the @safe code that you call and all you know is that all the @safe code type checks according to @safe rules. Note that there might be an arbitrary number of @safe functions and methods that you do not see.
>
> Think about it this way: you first write all the @trusted and @system code, and some evil guy who does not like you comes in after you looks at your code and writes all the @safe code. If there is any memory corruption, it will be your fault and you will face harsh consequences. Now, design the @safe type checking rules. It won't be MP!
>
> Note that there may well be a good way to get the good properties of MP without breaking the type system, but MP itself is not good because it breaks @safe.

Show me. Nobody has been able to show that yet. I'd really like to know this.

On Sunday, 21 October 2018 at 09:50:09 UTC, Walter Bright wrote:
> ---- Manu's Proposal ---
> @safe:
> int i;
> int* a = &i;
> StartNewThread(a); // Compiles! Coder has no idea!
>
> ... in the new thread ...
> void StartOfNewThread(shared(int)* b) {
>
>     ... we have two threads accessing 'i',
>     one thinks it is shared, the other unshared,
>     and StartOfNewThread() has no idea and anyone
>     writing code for StartOfNewThread() has no way
>     to know anything is wrong ...
>
>     lockedIncrement(b);  // Data Race!

No, does not compile, lockedIncrement takes an int*
Error cannot convert shared(int)* to int*

> Your proposal means that the person writing the lockedIncrement(), which is a perfectly reasonable thing to do,

Indeed.

> simply cannot write it in a way that has a @safe interface, because the person writing the lockedIncrement() library function has no way to know that the data it receives is actually unshared data.

It does, it takes an int* which is not implicitly convertible to given an shared(int)*

> I.e. @trusted code is obliged to proved a safe interface.

Yes.

> Your proposal makes that impossible because the compiler would allow unshared data to be implicitly typed as shared.

Yes, but not the other way around.

On Sunday, 21 October 2018 at 18:45:15 UTC, Walter Bright wrote:
> I'd like to add that if the compiler can prove that a T* points to a unique T, then it can be implicitly cast to shared(T)*. And it does so, like the result of .dup can be so converted.

This can be achieved by using the unique struct and enforce the uniqueness at compile time.

https://github.com/dlang/phobos/blob/master/std/typecons.d#L130

On Sunday, 21 October 2018 at 19:07:37 UTC, Nicholas Wilson wrote:
> On Sunday, 21 October 2018 at 09:50:09 UTC, Walter Bright wrote:
>> Your proposal makes that impossible because the compiler would allow unshared data to be implicitly typed as shared.
>
> Yes, but not the other way around.

Whoops that should read

>> Your proposal makes that impossible

No

>> because the compiler would allow unshared data to be implicitly typed as shared.

Yes, but the problem you describe is arises from implicit conversion in the other direction, which is not part of the proposal.

On Sun, Oct 21, 2018 at 5:50 AM Stanislav Blinov via Digitalmars-d <digitalmars-d@puremagic.com> wrote:
>
> On Sunday, 21 October 2018 at 05:47:14 UTC, Manu wrote:
>
> >> And yet, if we're lucky, we get
> >> a consistent instacrash. If we're unlucky, we get memory
> >> corruption, or an unsolicited write to another currently open
> >> file, either of which can go unnoticed for some time.
>
> > Woah! Now this is way off-piste..
> > Why would get a crash? Why would get memory corruption? None of
> > those things make sense.
>
> Because the whole reason to have `shared` is to avoid the extraneous checks that you mentioned above,

No, it is to assure that you write correct not-broken code.

> and only write actual
> useful code (i.e. lock-write-unlock, or read-put-to-queue-repeat,
> or whatever), not busy-work (testing if the file is open on every
> call).

`shared` is no comment on performance. You have written a slow locking API.
If you care about perf, you would write a completely different API
that favours perf.
This not impossible, nor even particularly hard.

> If you have a `shared` reference, it better be to existing data.

I mean, if I dereference a pointer, it had better not be null!

> If it isn't, the program is invalid already: you've shared something that doesn't "exist" (good for marketing, not so good for multithreading).

I mean, if I dereference a pointer, it had better not be null!

> That's why having `shared` and un-`shared`
> references to the same data simultaneously is not safe: you can't
> guarantee in any way that the owning thread doesn't invalidate
> the data through it's non-`shared` reference while you're doing
> your threadsafe `shared` work; you can only "promise" that by
> convention (documentation).

The owning thread is not a special actor. Your reasoning is wonky here.
Lets say there is no owning instance, only a collection of shared
instances... any one of them can theoretically do an operation that
interferes with the others.
That's issues for general threading, and no design for `shared` can
(or should) interact with that problem. That's a problem for
architecture.

> > So, you call closeFile immediately and read/write start returning null.
>
> And I have partially-read or partially-written data.

I expect you flushed before killing the file.

> Or Maybe I
> call closeFile(), main thread continues and opens another file,
> which gives the same file descriptor, `shared` references to
> FileHandle which the user forgot to wait on continue to work
> oblivious to the fact that it's a different file now.

It's wild to suggest that ANY design for `shared` should somehow deal
with the OS recycling a file handle...
And it's still not an un-@safe crash! It's just a program with a bug.

> It's a
> horrible, but still @safe, implementation of FileHandle, yes, but
> the caller (user) doesn't know that, and can't know that just
> from the interface. The only advice against that is "don't do
> that", but that's irrespective of your proposal.

No proposal can (or should) address these issues. You're concerned
with an issue that is sooooo far left-field at this stage.
Programming languages don't validate that you wrote a working bug-free program.

> > I'm going to assume that `shareWithThreads()` was implemented
> > by an
> > 'expert' who checked the function results for errors. It was
> > detected that the reads/write failed, and an error "failed to
> > read file" was emit, then the function returned promptly.
> > The uncertainty of what happens in this program is however
> > `shareWithThreads()` handles read/write emitting an error.
>
> But you can only find out about these errors in `waitForThreads`, the very call that the user "forgot" to make!

...what?

> [ ... snip ... ]

You have to concede defeat at this point.

Destroy my proposal with another legitimately faulty program.


> I understand that. So... it would seem that your proposal focuses more on @safe than on threadsafety?

I am trying to achieve @safe-ty _with respect to threadsafety_.

'threadsafety' with respect to "proper program operation" is a job for
programmers, and program architecture.
No language attribute can make your program right.

On Sunday, 21 October 2018 at 19:22:45 UTC, Manu wrote:
> On Sun, Oct 21, 2018 at 5:50 AM Stanislav Blinov via Digitalmars-d <digitalmars-d@puremagic.com> wrote:

>> Because the whole reason to have `shared` is to avoid the extraneous checks that you mentioned above,
>
> No, it is to assure that you write correct not-broken code.

You can do that without `shared`.

>> and only write actual useful code (i.e. lock-write-unlock, or read-put-to-queue-repeat, or whatever), not busy-work (testing if the file is open on every call).
>
> `shared` is no comment on performance. You have written a slow locking API.
> If you care about perf, you would write a completely different API that favours perf.
> This not impossible, nor even particularly hard.

You're conflating your assumptions about the code with the topic of this discussion. I can write a million examples and you'll still find a million reasons to talk about how they're incorrectly implemented, instead of focusing on merits or disadvantages of your proposal with the code given as is.

>> If you have a `shared` reference, it better be to existing data.
>
> I mean, if I dereference a pointer, it had better not be null!

Why would you share a null pointer?

>> That's why having `shared` and un-`shared`
>> references to the same data simultaneously is not safe: you can't guarantee in any way that the owning thread doesn't invalidate
>> the data through it's non-`shared` reference while you're doing
>> your threadsafe `shared` work; you can only "promise" that by
>> convention (documentation).
>
> The owning thread is not a special actor. Your reasoning is wonky here.

Why have it then at all? If it's not a "special" actor, just make all shared data `shared`. But your proposal specifically targets the conversion, suggesting you *do* need a special actor.

>> And I have partially-read or partially-written data.
>
> I expect you flushed before killing the file.

So? The threads still weren't done yet.

>> Or Maybe I call closeFile(), main thread continues and opens another file,
>> which gives the same file descriptor, `shared` references to
>> FileHandle which the user forgot to wait on continue to work
>> oblivious to the fact that it's a different file now.

> It's wild to suggest that ANY design for `shared` should somehow deal with the OS recycling a file handle...

I'm not suggesting that at all, you've completely misrepresenting what I'm saying by splitting a quote.

> And it's still not an un-@safe crash! It's just a program with a bug.

Ok, if you say that sticking @safe on code that can partially piece together data from unrelated sources is fine, then sure.

>> > I'm going to assume that `shareWithThreads()` was implemented
>> > by an 'expert' who checked the function results for errors...

>> But you can only find out about these errors in `waitForThreads`, the very call that the user "forgot" to make!
>
> ...what?

Please suggest another way of handling errors reported by other threads. More `shared` state?

>> [ ... snip ... ]
>
> You have to concede defeat at this point.

I agree. No matter how hard I try or how many times I ask you to demonstrate, I still fail to see the value in assuming @safe implicitly conversion of mutable data to shared. Instead of defending your proposal, you chose to attack the opposition. You've defeated me, flawless victory.

> Destroy my proposal with another legitimately faulty program.

Is there a point? I post code, you: "nah, that's wrong". Steven posts code, you: "nah, that's wrong". Timon posts code, you: "nah, that's wrong". Walter posts code, you: "nah, that's wrong"... What's right then?

On 10/21/2018 12:20 PM, Nicholas Wilson wrote:
> Yes, but the problem you describe is arises from implicit conversion in the other direction, which is not part of the proposal.

It's Manu's example.