Thread overview
Problem with clear on shared associative array?
May 27
Serg Gini
May 27
mw
May 26
The following code fails to compile; it appears from the error message that the library's clear() function is not ready to act on a shared AA?

synchronized class F {

private:
    string[int] mydict;

public:
    void clear() {
        this.mydict.clear();
    }
}

void
main()
{
    auto f = new shared F();
    f.clear();
}
May 26
On Sunday, May 26, 2024 8:39:53 AM MDT Andy Valencia via Digitalmars-d-learn wrote:
> The following code fails to compile; it appears from the error message that the library's clear() function is not ready to act on a shared AA?
>
> synchronized class F {
>
> private:
>      string[int] mydict;
>
> public:
>      void clear() {
>          this.mydict.clear();
>      }
> }
>
> void
> main()
> {
>      auto f = new shared F();
>      f.clear();
> }

No operation on an associative array is thread-safe. As such, you should not be doing _any_ operation on a shared AA without first locking a mutex to protect it. Then you need to cast away shared to access or mutate it or do whatever it is you want to do with it other than pass it around. And then when you're done, you make sure that no thread-local references to the AA exist, and you release the mutex. This is true for any type which is not specifically designed to be shared, and none of the built-in types are designed to be shared. They will work as shared so long as you protect them appropriately and then temporarily cast away shared to operate on them as thread-local, but they should never be mutated as shared, and it's on you to make sure that the object is actually protected appropriately when you cast away shared, since the language has no way of knowing when that's actually safe to do. It can just prevent you from accidentally accessing an object when it's shared, since when it's shared, the type system marks it as shared across threads and thus not thread-safe to access.

The language is supposed to prevent you from doing any operations on a shared object which are not guaranteed to be thread-safe (which is pretty much anything other than passing it around by reference or pointer), but unfortunately, since that wasn't entirely implemented up front, some of those checks are currently behind the -preview=nosharedaccess flag and will be made the default at some point in the future (but not now). So, some shared operations may work when they really shouldn't, but the compiler will prevent you from calling a function like clear, because it operates on thread-local variables, not shared ones.

And with regards to AAs, you will need to be particularly careful with the in operator, because it returns a pointer a value in the AA. So, if you cast away shared to operate on it and then use in, you'll get a thread-local pointer to the AA, meaning that the lock needs to stay in place as long as that pointer is around, whereas a function like clear is done as soon as its been called, so unless you're doing more with the AA at that point, you could release the lock then.

Using synchronized is one way to get a mutex, and a synchronized function is implicitly shared, so your clear function is both synchronized and shared, meaning that you can call it on a shared object, and it will lock a mutex when it's called, but you still need to cast away shared from mydict to be able to call anything on it. The compiler is not smart enough to know that removing shared is thread-safe within your function, so it will not do it for you, and the clear function for AAs can't work on shared AAs, because it has no way of guaranteeing that that's thread-safe. So, the solution is that you must explicitly removed shared temporarily to call clear when you know that it's thread-safe to do so.

- Jonathan M Davis



May 26
On Sunday, 26 May 2024 at 20:00:50 UTC, Jonathan M Davis wrote:
> No operation on an associative array is thread-safe. As such, you should not be doing _any_ operation on a shared AA without first locking a mutex to protect it. Then you need to cast away shared to access or mutate it or do whatever it is you want to do with it other than pass it around. And then when you're done, you make sure that no thread-local references to the AA exist, and you release the mutex.
> ...

Thank you, that's exactly the big picture explanation I was hoping for.

For others wrestling with this issue, I found out how to cast to unshared at this article:

https://forum.dlang.org/thread/jwasqvrvkpqzimlutgrm@forum.dlang.org

Andy

May 27
On Sunday, 26 May 2024 at 20:15:54 UTC, Andy Valencia wrote:
> For others wrestling with this issue, I found out how to cast to unshared at this article:
>

You can check also this solution https://github.com/DmitryOlshansky/sharded-map
May 27
On Monday, 27 May 2024 at 00:43:47 UTC, Serg Gini wrote:
> On Sunday, 26 May 2024 at 20:15:54 UTC, Andy Valencia wrote:
>> For others wrestling with this issue, I found out how to cast to unshared at this article:
>>
>
> You can check also this solution https://github.com/DmitryOlshansky/sharded-map


Pls NOTE: it is
a   `sharded` (meaning trunk-ed) NON-concurrent map,
not `shared` concurrent map.

These two words looks similar, but the meaning is very different.


May 28
On Monday, 27 May 2024 at 04:04:03 UTC, mw wrote:
> Pls NOTE: it is
> a   `sharded` (meaning trunk-ed) NON-concurrent map,
> not `shared` concurrent map.

Assuming I put it in shared memory, in what way is it not able to be used concurrently?  It seems to have the needed lock operations?

Thanks,
Andy

May 28
On 28/05/2024 12:36 PM, Andy Valencia wrote:
> On Monday, 27 May 2024 at 04:04:03 UTC, mw wrote:
>> Pls NOTE: it is
>> a   `sharded` (meaning trunk-ed) NON-concurrent map,
>> not `shared` concurrent map.
> 
> Assuming I put it in shared memory, in what way is it not able to be used concurrently?  It seems to have the needed lock operations?
> 
> Thanks,
> Andy

A concurrent data structure handles all of this for you.

AA's are not concurrent because it doesn't offer any protection.

Protecting a data structure with a mutex doesn't make it concurrent, but it may allow you to use it concurrently safely. Subtle difference!