On Friday, August 12, 2016 16:38:23 Kagamin via Digitalmars-d wrote:
> On Friday, 12 August 2016 at 15:43:54 UTC, Steven Schveighoffer
>
> wrote:
> > All kidding aside, shared is pretty broken currently. If you want to know how it was *supposed* to work, you can read TDPL, though I'm not sure that's still valid.
>
> Ah, was it Andrei, who tried to sell the idea that shared is a silver bullet for concurrency problems? That can explain why we have two different views on shared. Looks like a repetition of the story with autodecoding (with similar rationale).

Honestly, I don't think that shared is broken. It's just that once you've locked the appropriate mutex, you have to cast a way shared and make sure that no thread-local references to that data are still around when you release the lock. shared is doing its job of indicating when a variable isn't thread-local as well as preventing a number of incorrect operations on data that's shared across threads without being protected by a lock. What we're missing is a way safely have shared cast away for us so that we don't have to do the work of verifying that we're not screwing it up. synchronized classes as described in TDPL partially solve the problem in that they provide a safe, automatic way to remove one layer of shared - but it's only one layer, and it's never been implemented. It's the equivalent of having a set of operations that we'd like to be @safe but are stuck as @system, and we're forced to use @trusted to deal with it.

But folks get annoyed with shared, because they have to cast it away at the appropriate time. They just want it to magically work in a safe manner, and we simply haven't figured out how to do that. Instead, we have a way to segregate shared stuff and thus minimize how much code has to deal with the problems that come with sharing data across threads but which requires that you handle it carefully rather than having a way to just handle it safely, automatically.

shared isn't perfect, but having thread-local by default is dowright fantastic, and when you have to deal with shared, you mostly doing the same sorts of things that you're supposed to be doing in C++. It's just that you have to cast away shared to operate on the object while the mutex is locked.

IMHO, the biggest problem with shared is that the related types in druntime (like the mutexes and condition variables) aren't marked with it properly rather than shared itself being fundamentally flawed.

- Jonathan M Davis

On 08/12/2016 12:31 PM, Wyatt wrote:
> On Friday, 12 August 2016 at 12:27:50 UTC, Andrei Alexandrescu wrote:
>> I recall I had a similar reaction as Edward back in the day. No hurt
>> feelings or anything, but the arguments made were so specious I'd roll
>> my eyes whenever I saw them in the C++ forums. -- Andrei
>
> So what changed?  What gave you the initial kick to start moving from
> sideline scoffer to benevolent diarch?

Walter and I talked and he said he'd be interested in working together on the kind of things I was interested in: generic programming, static introspection, and such. -- Andrei

On 08/12/2016 10:51 AM, Shachar Shemesh wrote:
> To me, this is not so much the question of alienating C++ programmers,
> as it is that this arrogance is actively hindering D from becoming better.

I agree it would behoove us to be more humble. -- Andrei

On 8/12/16 1:04 PM, Jonathan M Davis via Digitalmars-d wrote:
>
> Honestly, I don't think that shared is broken.

Yes. It is broken.

shared int x;
++x; // error, must use atomicOp.
x = x + 1; // OK(!)

The thing that is NOT broken is unshared.

There's literally not much you can do with shared, and what you can do isn't consistent or useful.

It needs lots of attention.

-Steve

On Fri, Aug 12, 2016 at 02:04:53PM -0400, Andrei Alexandrescu via Digitalmars-d wrote:
> On 08/12/2016 01:21 PM, Steven Schveighoffer wrote:
> > On 8/12/16 1:04 PM, Jonathan M Davis via Digitalmars-d wrote:
> > > 
> > > Honestly, I don't think that shared is broken.
> > 
> > Yes. It is broken.
> > 
> > shared int x;
> > ++x; // error, must use atomicOp.
> > x = x + 1; // OK(!)
> 
> How is this broken and how should it behave? -- Andrei

?!

Isn't it obvious that assigning to a shared int must require atomicOp or a cast? `x = x + 1;` clearly has a race condition otherwise.


T

-- 
A program should be written to model the concepts of the task it performs rather than the physical world or a process because this maximizes the potential for it to be applied to tasks that are conceptually similar and, more important, to tasks that have not yet been conceived. -- Michael B. Allen

On 08/12/2016 01:21 PM, Steven Schveighoffer wrote:
> On 8/12/16 1:04 PM, Jonathan M Davis via Digitalmars-d wrote:
>>
>> Honestly, I don't think that shared is broken.
>
> Yes. It is broken.
>
> shared int x;
> ++x; // error, must use atomicOp.
> x = x + 1; // OK(!)

How is this broken and how should it behave? -- Andrei

On 08/12/2016 02:01 PM, H. S. Teoh via Digitalmars-d wrote:
> On Fri, Aug 12, 2016 at 02:04:53PM -0400, Andrei Alexandrescu via Digitalmars-d wrote:
>> On 08/12/2016 01:21 PM, Steven Schveighoffer wrote:
>>> On 8/12/16 1:04 PM, Jonathan M Davis via Digitalmars-d wrote:
>>>>
>>>> Honestly, I don't think that shared is broken.
>>>
>>> Yes. It is broken.
>>>
>>> shared int x;
>>> ++x; // error, must use atomicOp.
>>> x = x + 1; // OK(!)
>>
>> How is this broken and how should it behave? -- Andrei
>
> ?!
>
> Isn't it obvious that assigning to a shared int must require atomicOp or
> a cast? `x = x + 1;` clearly has a race condition otherwise.

It is not to me, and it does not seem like a low-level race condition to me (whereas ++x is).

Currently the compiler must ensure that an atomic read and an atomic write are generated for x. Other than that, it is the responsibility of the user. The use of "shared" does not automatically relieve the user from certain responsibilities.

I agree that it would be nice to have stronger protection against higher-level bugs, but those are outside the charter of "shared". Consider:

x = *p + 1;

How would the compiler reject the right uses but not the case when p == &x?


Andrei

On Fri, Aug 12, 2016 at 02:21:04PM -0400, Andrei Alexandrescu via Digitalmars-d wrote:
> On 08/12/2016 02:01 PM, H. S. Teoh via Digitalmars-d wrote:
> > On Fri, Aug 12, 2016 at 02:04:53PM -0400, Andrei Alexandrescu via Digitalmars-d wrote:
> > > On 08/12/2016 01:21 PM, Steven Schveighoffer wrote:
> > > > On 8/12/16 1:04 PM, Jonathan M Davis via Digitalmars-d wrote:
> > > > > 
> > > > > Honestly, I don't think that shared is broken.
> > > > 
> > > > Yes. It is broken.
> > > > 
> > > > shared int x;
> > > > ++x; // error, must use atomicOp.
> > > > x = x + 1; // OK(!)
> > > 
> > > How is this broken and how should it behave? -- Andrei
> > 
> > ?!
> > 
> > Isn't it obvious that assigning to a shared int must require atomicOp or a cast? `x = x + 1;` clearly has a race condition otherwise.
> 
> It is not to me, and it does not seem like a low-level race condition
> to me (whereas ++x is).

The problem is that the line between "low-level" and "high-level" is unclear and arbitrary. Doesn't ++x lower to x = x + 1 on some CPUs anyway (or vice versa, if the optimizer decides to translate it the other way)? Why should the user have to worry about such details? Wouldn't that make shared kinda useless to begin with?


> Currently the compiler must ensure that an atomic read and an atomic write are generated for x. Other than that, it is the responsibility of the user.  The use of "shared" does not automatically relieve the user from certain responsibilities.
>
> I agree that it would be nice to have stronger protection against higher-level bugs, but those are outside the charter of "shared". Consider:
> 
> x = *p + 1;
> 
> How would the compiler reject the right uses but not the case when p == &x?
[...]

The compiler should reject it (without the appropriate casts) if p has a shared type, and the aliasing situation with x is unknown / unclear. The programmer ought to explicitly state the assumption that x isn't aliased by p by specifying a cast (or equivalent), rather than be able to write such code, possibly not being aware of the implications, and have it compile without any warning.


T

-- 
If lightning were to ever strike an orchestra, it'd always hit the conductor first.

On Friday, August 12, 2016 13:21:10 Steven Schveighoffer via Digitalmars-d wrote:
> On 8/12/16 1:04 PM, Jonathan M Davis via Digitalmars-d wrote:
> > Honestly, I don't think that shared is broken.
>
> Yes. It is broken.
>
> shared int x;
> ++x; // error, must use atomicOp.
> x = x + 1; // OK(!)
>
> The thing that is NOT broken is unshared.
>
> There's literally not much you can do with shared, and what you can do isn't consistent or useful.
>
> It needs lots of attention.

It does not surprise me in the least if there are bugs related to shared in the compiler, and we definitely don't deal with it properly in druntime with regards to stuff like Mutex and Condition. But I don't agree with the idea that shared is fundamentally broken.

- Jonathan M Davis

On 8/12/2016 7:41 AM, Shachar Shemesh wrote:
> That table was not expensive to compute, and its constantness wasn't crucial
> enough even for me to put a wrapper pointer and only access it through it. Had
> that not been the case, and had that table been more expensive to computer, I'd
> probably compute at compile time with an external tool.

What I do (and is done in building DMD) is write a program (optabgen.c) to generate the tables and write a C++ source file, then compile the generated file into DMD.