I'm working with mysql-native for a project, and have been using a single, shared Connection (http://semitwist.com/mysql-native-docs/v2.2.0/mysql/connection/Connection.html) among multiple threads. The issue here is that since it's shared, I can't use certain functions such as exec() or close() since they're not shared.

I've been doing the following as a workaround. It's not a huge deal to do, but is this the best way to accomplish this? Short of going through the library and making the functions shared myself of course.

shared Connection con = ...;

void closeDatabase()
{
    synchronized
    {
        auto _con = cast(Connection) con;

        if (!_con.closed)
            _con.close();
    }
}

On Thursday, March 29, 2018 17:41:15 Chris M. via Digitalmars-d-learn wrote:
> I'm working with mysql-native for a project, and have been using
> a single, shared Connection
> (http://semitwist.com/mysql-native-docs/v2.2.0/mysql/connection/Connection
> .html) among multiple threads. The issue here is that since it's shared, I
> can't use certain functions such as exec() or close() since they're not
> shared.
>
> I've been doing the following as a workaround. It's not a huge deal to do, but is this the best way to accomplish this? Short of going through the library and making the functions shared myself of course.
>
> shared Connection con = ...;
>
> void closeDatabase()
> {
>      synchronized
>      {
>          auto _con = cast(Connection) con;
>
>          if (!_con.closed)
>              _con.close();
>      }
> }

In general, the correct way to deal with a shared object is to protect access to it with a mutex and then within that protected section, you cast away shared so that it's treated as thread-local so that it can be operated on. Then when you release the mutex, you make sure that no thread-local references to the shared object persist. Most operations really aren't supposed to work while the object is treated as shared, because then you have a threading problem if the operation is non-atomic and doesn't protect itself with a mutex (though a member function on an object certainly could use a mutex internally, allowing it to be shared and work on a shared object).

So, assuming that access to the Connection is properly protected by the code in general, what you're doing is correct, but if you're not using the same mutex for that operation as for other operations on Connection, and other code could be operating on the Connection at the same time, then you have a problem. At the moment, I don't remember where a synchronized block with no argument gets its mutex, so I'm not sure which mutex you're using. However, since Connection appears to be a class, it has a built-in monitor object, meaning that it should work to just do

synchronized(con)
{
}

and use the Connection object's own internal mutex to lock it. So, if you do that whenever you have to operate on the Connection object, then you know that you're using the same mutex in all cases.

And, of course, if it helps, you can create a wrapper struct or class around the Connection object and have the wrapper type handle all of the locking and casting for you.

- Jonathan M Davis

On Thursday, 29 March 2018 at 18:07:50 UTC, Jonathan M Davis wrote:
> On Thursday, March 29, 2018 17:41:15 Chris M. via Digitalmars-d-learn wrote:
>> [...]
>
> In general, the correct way to deal with a shared object is to protect access to it with a mutex and then within that protected section, you cast away shared so that it's treated as thread-local so that it can be operated on. Then when you release the mutex, you make sure that no thread-local references to the shared object persist. Most operations really aren't supposed to work while the object is treated as shared, because then you have a threading problem if the operation is non-atomic and doesn't protect itself with a mutex (though a member function on an object certainly could use a mutex internally, allowing it to be shared and work on a shared object).
>
> [...]

Awesome, thanks for the explanation. Didn't realize synchronized had that extra syntax and what it did, that'll save me a few headaches. Now what would really be nice is if that syntax also cast away the shared implicitly while inside the critical section, but I can deal with it for now.

On Thursday, March 29, 2018 18:50:39 Chris M. via Digitalmars-d-learn wrote:
> On Thursday, 29 March 2018 at 18:07:50 UTC, Jonathan M Davis
>
> wrote:
> > On Thursday, March 29, 2018 17:41:15 Chris M. via
> >
> > Digitalmars-d-learn wrote:
> >> [...]
> >
> > In general, the correct way to deal with a shared object is to protect access to it with a mutex and then within that protected section, you cast away shared so that it's treated as thread-local so that it can be operated on. Then when you release the mutex, you make sure that no thread-local references to the shared object persist. Most operations really aren't supposed to work while the object is treated as shared, because then you have a threading problem if the operation is non-atomic and doesn't protect itself with a mutex (though a member function on an object certainly could use a mutex internally, allowing it to be shared and work on a shared object).
> >
> > [...]
>
> Awesome, thanks for the explanation. Didn't realize synchronized had that extra syntax and what it did, that'll save me a few headaches. Now what would really be nice is if that syntax also cast away the shared implicitly while inside the critical section, but I can deal with it for now.

Well, there's no guarantee that you've actually properly protected access to the object everywhere, so it wouldn't be safe to automatically cast away shared for you. The language feature that is supposed to act similarly to that is synchronized classes, but they've never been fully implemented. Right now, marking a class as synchronized just makes all of its functions synchronized and disallows public member variables. It doesn't actually provide any benefits with regards to removing shared like it's supposed to (and even the fact that it disallows public member variables is fairly recent).

Fully implemented synchronized classes would remove the outer layer of shared on the member variables, because the compiler could guarantee that the outer layer had not escaped, because it would be impossible to access the member variables except through the member functions which would all be synchronized. So, you could then have a shared object protected by a synchronized class and sort of use it as thread-local within the member functions of the synchronized class. The problem is that it can only safely remove the outer layer of shared, so anything referred to by the shared object would still be shared - e.g. shared(T*) would become shared(T)*. So, the pointer would then be manipulated as thread-local but not what it refers to. Value types would then be fully thread-local, so they'd be fine, but anything involving any kind of pointer reference would be in trouble. I don't think that it would even be possible to strip shared from a member variable that's a class reference, because the only part that could safely have shared stripped away would be the reference itself, and the type system doesn't treat that as separate from the class. Regardless, the class itself couldn't be treated as thread-local any more than the data pointed to by a pointered could be.

Basically, for value types, you'd get to treat them as thread-local, but anything referred to by the member (be it via pointer or reference or a dynamic array) would then still be shared. So, while _some_ operations could then be done as if the member were thread-local, many operations still couldn't be. It would work great for value types, but for reference types, you're pretty much just as screwed as you are now.

So, synchronized classes would help, but they wouldn't really fix the problem. Unfortunately, they're also the best solution that has been proposed. We'd very much like to have a way to safely remove shared automatically while the mutex is locked, but we simply don't have a way to guarantee that everything has been properly protected by the same mutex everywhere, so we can't automatically strip away shared while the mutex is locked. synchronized classes have to restrict things a fair bit in order to be able to remove as much of shared as they do, and even then, they can't remove much. Anything which wanted to fully strip away shared would probably have to somehow guarantee that _nothing_ that it referred to was accessible elsewhere, and the type system doesn't support anything like that. The programmer can often see that that's the case, but the compiler can't prove it. So, ultimately, it's a bit like @trusted. The programmer has to examine the code in question and verify that it's doing the right thing, which includes managing the casting away of shared. It's annoying to be sure, but no one has come up with a better solution yet, and honestly, I doubt that they will without adding something pretty major to the type system.

Ultimately, the way that shared is used is pretty much the same as what you get in C++, except in D, you're prevented from manipulating the object in a manner that isn't thread-safe (assuming shared has no bugs anyway), and you're forced to cast away shared while the mutex protecting the object is locked, whereas in C++, you get no proctections against using the object in a manner which is not thread-safe, and you don't have to cast. But the C++ code would be pretty much the same except that it wouldn't include a cast, and the way that the mutex would be locked wouldn't involve synchronized. However, all of the compiler errors that folks get when trying to use shared seems to confuse and anger most folks. It's frequently assumed that you should be able to use the object while it's shared, but unfortunately, we really don't have a way to do that, much as we'd like to - at least, not while having shared do its job of preventing you from using it in a way that isn't thread-safe.

- Jonathan M Davis