On Thursday, May 16, 2019 4:17:15 AM MDT Dominikus Dittes Scherkl via Digitalmars-d wrote:
> On Wednesday, 15 May 2019 at 09:33:19 UTC, Jonathan M Davis wrote:
> > You can't forbid references to the same data. All that would be required would be something like
> >
> > shared foo = new shared(Foo)(42);
> > auto bar = foo;
>
> Meep. This is either a compile error outside locked block (cannot
> read shared) or bar would be scope (it's lifetime will end after
> lock block is left).

If you can't pass shared class references or pointers around like this, then it wouldn't even be possible to call shared functions on shared objects, because that's what happens when you call a function. And that really would make shared ridiculously hard to use. It would make it impossible to encapsulate thread synchronization primitives and force the programmer to handle it directly. Things like a shared class or struct that handled its own locking would be impossible, because you couldn't even call any of its methods. Code like

shared foo = new shared(Foo)(42);
auto bar = foo;

needs to work as long as the type being passed around is a pointer or class reference, or shared simply won't be useable.

> > and you have two references to the same object without doing anything unsafe. You could also have stuff like
> >
> > shared baz = foo.getBaz();
>
> This will be forced to a copy of the returned value, not a reference.

No matter what the return type is, a line like that would either copy or move the return value (which one would depend on the implementation of getBaz and what exactly it was returning), but for a pointer or class reference, that just means that you have a pointer or class reference referring to the same thing as something that was in getBaz. So, copying is the normal semantics. And if you're trying to claim that it should be a deep copy, D doesn't even have that capability for classes - not without there being user-defind functions which do it, and the compiler won't know that that's what they do and couldn't use them automatically. So, that wouldn't work even if it were desirable - and it wouldn't be, because you really don't want to be deep copying classe normally. And you _really_ don't want to be doing it if shared is involved; that would require locking in a way that copying a simple pointer or reference shouldn't (at least not if we want to be able to actually do stuff like call shared member functions).

Also, just in general, trying to prevent other references to the same data with shared actually makes no sense. If you don't have multiple references to the same data, then there's no point in it being shared in the first place, because you won't have multiple threads accessing the data. Even with TDPL synchronized classes, which prevent references to the member variables from escaping, you still have multiple references to the class itself - and the class itself is shared. Multiple references to the same data is a key part of data being shared. At best, you can avoid it by having only a single reference to some shared data with multiple references to a shared object which has the single reference to the other shared data (i.e. what you get with TDPL synchronized classes). And if that's what you're doing, then you might as well just have TDPL synchronized classes rather than trying to do something more free-from.

- Jonathan M Davis

On Thursday, May 16, 2019 4:35:52 AM MDT Dominikus Dittes Scherkl via Digitalmars-d wrote:
> On Wednesday, 15 May 2019 at 09:33:19 UTC, Jonathan M Davis wrote:
> > On Wednesday, May 15, 2019 12:59:00 AM MDT Dominikus Dittes
> >
> > Scherkl via Digitalmars-d wrote:
> >> No, it can't. Disjunct means: It cannot be called unless all of the given variables are free (not locked by any other mutex).
> >
> > So, you're proposing that something in the runtime keeps track of which variables are currently associated with a locked mutex in order to guarantee that no other lock block is able to access any of those variables at the same time? That would probably require adding a global lock used by the runtime when any code enters or exists a lock block. I'd be _very_ surprised if anything like that were deemed acceptable for D.
>
> Ok, that can be a problem. But yes, this is what I suggest.
> But why the resistence? Nobody is forced to use mutexes, and if
> you don't that part of the runtime could (should!) be eliminated
> from the program.

For the locking mechanism you're proposing to work, you basically have to disallow a lot of stuff that is required for other types of thread synchronization (e.g. casting away shared is needed), and having a global lock that's used whenever a mutex is used would be terrible for performance. In general, we want to avoid global locks, and it seems particularly bad to have a global mutex that needs to be used just to use a normal mutex.

> > And it still doesn't solve the problem of other references referring to any part of the objects referred to by those variables existing and potentially being used elsewhere. If you had
> >
> > lock(mutex, var1)
> > {
> > }
> >
> > and elsewhere
> >
> > lock(mutex, var2)
> > {
> > }
> >
> > when var1 and var2 were references to the same object
>
> Forbidden. See last post.
>
> > or when var2 referred to a piece of data inside of var1,
>
> This would also be impossible. shared vars are no references.

I discussed this in another post, but you _have_ to be able to have shared references or shared is basically useless.

> > If you only had one mutex for the entire program, and casting away shared were illegal, then something like this could probably work,
>
> Thank you, but why this restriction? There can be as many mutexes as you like. The runtime has only to ensure that any running locked block doesn't modify any of the vars in the other running locked blocks.

If it's at all possible to access any of the shared data that your lock blocks protect without using the lock block, then your lock block would not be enough for the compiler to know that it was safe to remove shared from the data within the lock block. And without a global mutex that's used to check that different locks aren't being used for the same variable, then different mutexes could be used for the same varible, defeating the protection - and even that is assuming that it's not possible to have any other references to the same data, which would be too restrictive to be useful. Having multiple references to the same data is a core concept of sharing data across threads.

- Jonathan M Davis

On Thursday, 16 May 2019 at 11:15:13 UTC, Jonathan M Davis wrote:
> [...] that is assuming that it's not possible to have any other references to the same data, which would be too restrictive to be useful. Having multiple references to the same data is a core concept of sharing data across threads.
Hm. I don't understand the necessity of this. If I have a shared symbol x,
this same symbol can be used in different threads, no? I can't see why multiple threads need to have different references to the same data. They just can use the same reference.

On Thursday, 16 May 2019 at 15:47:37 UTC, Dominikus Dittes Scherkl wrote:
> On Thursday, 16 May 2019 at 11:15:13 UTC, Jonathan M Davis wrote:
>> [...] that is assuming that it's not possible to have any other references to the same data, which would be too restrictive to be useful. Having multiple references to the same data is a core concept of sharing data across threads.
> Hm. I don't understand the necessity of this. If I have a shared symbol x,
> this same symbol can be used in different threads, no? I can't see why multiple threads need to have different references to the same data. They just can use the same reference.

Well, no. The reference is the part that gets copied around. It's the address of the shared object. When you do 'auto ref2 = ref1;', you're creating a new reference to an already existing object, but leaving the object itself unchanged.

If more than one thread can access the same object, that means they each have a copy of the reference*. In the same way, when you call a function with a class argument, that function receives a copy of the reference (but not the object). Clearly then, the ability to create multiple references to the same object must be possible without breaking shared.

--
  Simen

* barring __gshared tricks

On Thursday, May 16, 2019 11:11:19 AM MDT Simen Kjærås via Digitalmars-d wrote:
> On Thursday, 16 May 2019 at 15:47:37 UTC, Dominikus Dittes
>
> Scherkl wrote:
> > On Thursday, 16 May 2019 at 11:15:13 UTC, Jonathan M Davis
> >
> > wrote:
> >> [...] that is assuming that it's not possible to have any other references to the same data, which would be too restrictive to be useful. Having multiple references to the same data is a core concept of sharing data across threads.
> >
> > Hm. I don't understand the necessity of this. If I have a
> > shared symbol x,
> > this same symbol can be used in different threads, no? I can't
> > see why multiple threads need to have different references to
> > the same data. They just can use the same reference.
>
> Well, no. The reference is the part that gets copied around. It's the address of the shared object. When you do 'auto ref2 = ref1;', you're creating a new reference to an already existing object, but leaving the object itself unchanged.
>
> If more than one thread can access the same object, that means they each have a copy of the reference*. In the same way, when you call a function with a class argument, that function receives a copy of the reference (but not the object). Clearly then, the ability to create multiple references to the same object must be possible without breaking shared.

Exactly. And since everything in D is thread-local by default, it's usually the case that a variable only exists on one thread. shared changes that if the variable is static, but otherwise, you're just passing around a pointer or reference to shared objects within thread-local objects. So, while  you might have a pointer to shared data contained by a variable with the same name on multiple threads, because that variable is thread-local, it's different on different threads. e.g.

struct S
{
    int i;
    shared Foo* foo;
}

// Every thread will have it's own variable for this
S s;

- Jonathan M Davis

On Thursday, 16 May 2019 at 17:29:25 UTC, Jonathan M Davis wrote:
> On Thursday, May 16, 2019 11:11:19 AM MDT Simen Kjærås via Digitalmars-d wrote:
>> On Thursday, 16 May 2019 at 15:47:37 UTC, Dominikus Dittes
>>
>> Scherkl wrote:
>> > On Thursday, 16 May 2019 at 11:15:13 UTC, Jonathan M Davis
>> >
>> > wrote:
>> >> [...] that is assuming that it's not possible to have any other references to the same data, which would be too restrictive to be useful. Having multiple references to the same data is a core concept of sharing data across threads.
>> >
>> > Hm. I don't understand the necessity of this. If I have a
>> > shared symbol x,
>> > this same symbol can be used in different threads, no? I can't
>> > see why multiple threads need to have different references to
>> > the same data. They just can use the same reference.
>>
>> Well, no. The reference is the part that gets copied around. It's the address of the shared object. When you do 'auto ref2 = ref1;', you're creating a new reference to an already existing object, but leaving the object itself unchanged.
>>
>> If more than one thread can access the same object, that means they each have a copy of the reference*. In the same way, when you call a function with a class argument, that function receives a copy of the reference (but not the object). Clearly then, the ability to create multiple references to the same object must be possible without breaking shared.
>
> Exactly. And since everything in D is thread-local by default, it's usually the case that a variable only exists on one thread. shared changes that if the variable is static, but otherwise, you're just passing around a pointer or reference to shared objects within thread-local objects. So, while  you might have a pointer to shared data contained by a variable with the same name on multiple threads, because that variable is thread-local, it's different on different threads. e.g.

Ok, so then a shared object is in fact a specific memory segment. So something the GC is already aware of. This makes implementing the locking mechanism just more easy. We only need to prevent writing to that address, no matter how many references are pointing there. And that mechanism is well known in GC's - a write barrier. Only new thing is that no other locked block pointing in that address area can be executed. I just can't see what fundamental, all blocking problem you see with that approach. Seems pretty strait forward to me.

On Thu, May 16, 2019 at 07:53:56PM +0000, Dominikus Dittes Scherkl via Digitalmars-d wrote:
> On Thursday, 16 May 2019 at 17:29:25 UTC, Jonathan M Davis wrote:
[...]
> Ok, so then a shared object is in fact a specific memory segment. So something the GC is already aware of. This makes implementing the locking mechanism just more easy. We only need to prevent writing to that address, no matter how many references are pointing there. And that mechanism is well known in GC's - a write barrier.
[...]

D does not have write barriers, however. And it's unlikely to get one in the foreseeable future.


T

-- 
What do you get if you drop a piano down a mineshaft? A flat minor.

On Thursday, May 16, 2019 2:04:13 PM MDT H. S. Teoh via Digitalmars-d wrote:
> On Thu, May 16, 2019 at 07:53:56PM +0000, Dominikus Dittes Scherkl via
Digitalmars-d wrote:
> > On Thursday, 16 May 2019 at 17:29:25 UTC, Jonathan M Davis wrote:
> [...]
>
> > Ok, so then a shared object is in fact a specific memory segment. So something the GC is already aware of. This makes implementing the locking mechanism just more easy. We only need to prevent writing to that address, no matter how many references are pointing there. And that mechanism is well known in GC's - a write barrier.
>
> [...]
>
> D does not have write barriers, however. And it's unlikely to get one in the foreseeable future.

That and not all memory comes from the GC. It's perfectly legitimate to use malloced memory with shared. The type system doesn't care where the memory came from.

- Jonathan M Davis

On Thursday, 16 May 2019 at 20:04:13 UTC, H. S. Teoh wrote:
> What do you get if you drop a piano down a mineshaft? A flat minor.

I love you're signatures. You don't choose them at random, do you?

At least I accept now, it's not as easy as one might think.

Arimasen.

On Fri, May 17, 2019 at 10:20:32AM +0000, Dominikus Dittes Scherkl via Digitalmars-d wrote:
> On Thursday, 16 May 2019 at 20:04:13 UTC, H. S. Teoh wrote:
> > What do you get if you drop a piano down a mineshaft? A flat minor.
> 
> I love you're signatures. You don't choose them at random, do you?
[...]

It's chosen by a Perl script.  It's supposed to be random, but the Perl RNG seems to have a mind of its own sometimes. :-D


T

-- 
Study gravitation, it's a field with a lot of potential.