I was working on an I/O library that I plan to use in development, and possibly submit to phobos, and I thought of this case.

A standard file can be shared or unshared.  In C, since there is no notion of shared/unshared, everything is shared.  So any writes/reads from a FILE * lock the object.

But in D, we can avoid those locks when the file is unshared.  However, this means I have to write two pretty much identical functions for each call.

Is there an expected way to do this?  I've really avoided doing anything with shared or threads since the new concurrency model came out, but with I/O, I'll have to deal with it.

I think a logical thing to do would be to have the shared version of the function call the unshared version after locking the object.  Is that a good idea?  Is the correct way to do this to mark the shared function synchronized, and then cast 'this' to unshared to call the other function?  Does this automatically happen with shared functions?

Actually, this whole question is really general, because I'd expect to have to do this with any object that was shared -- putting memory barriers around accessing the object's members isn't enough to guarantee concurrency, and it makes no sense to do the memory barriers when the accesses are protected by a lock, right?

-Steve

On Saturday 19 February 2011 18:26:25 Steven Schveighoffer wrote:
> I was working on an I/O library that I plan to use in development, and possibly submit to phobos, and I thought of this case.
> 
> A standard file can be shared or unshared.  In C, since there is no notion of shared/unshared, everything is shared.  So any writes/reads from a FILE * lock the object.
> 
> But in D, we can avoid those locks when the file is unshared.  However, this means I have to write two pretty much identical functions for each call.
> 
> Is there an expected way to do this?  I've really avoided doing anything with shared or threads since the new concurrency model came out, but with I/O, I'll have to deal with it.
> 
> I think a logical thing to do would be to have the shared version of the function call the unshared version after locking the object.  Is that a good idea?  Is the correct way to do this to mark the shared function synchronized, and then cast 'this' to unshared to call the other function?  Does this automatically happen with shared functions?

I would point out that per TDPL, either an entire class is synchronized or none of it is. You don't synchronize individual functions. Now, I don' think that that's the way that it's implemented at the moment, but that's the eventual situation as I understand it. So, your class shouldn't have a mixture of synchronized or unsynchronized. According to TDPL, it's illegal.

- Jonathan M Davis

On Sat, 19 Feb 2011 21:55:53 -0500, Jonathan M Davis <jmdavisProg@gmx.com> wrote:

> On Saturday 19 February 2011 18:26:25 Steven Schveighoffer wrote:
>> I was working on an I/O library that I plan to use in development, and
>> possibly submit to phobos, and I thought of this case.
>>
>> A standard file can be shared or unshared.  In C, since there is no notion
>> of shared/unshared, everything is shared.  So any writes/reads from a FILE
>> * lock the object.
>>
>> But in D, we can avoid those locks when the file is unshared.  However,
>> this means I have to write two pretty much identical functions for each
>> call.
>>
>> Is there an expected way to do this?  I've really avoided doing anything
>> with shared or threads since the new concurrency model came out, but with
>> I/O, I'll have to deal with it.
>>
>> I think a logical thing to do would be to have the shared version of the
>> function call the unshared version after locking the object.  Is that a
>> good idea?  Is the correct way to do this to mark the shared function
>> synchronized, and then cast 'this' to unshared to call the other
>> function?  Does this automatically happen with shared functions?
>
> I would point out that per TDPL, either an entire class is synchronized or none
> of it is. You don't synchronize individual functions. Now, I don' think that
> that's the way that it's implemented at the moment, but that's the eventual
> situation as I understand it. So, your class shouldn't have a mixture of
> synchronized or unsynchronized. According to TDPL, it's illegal.
>

OK, I kind of remember that now.  So that means, I need to create two identical hierarchies, a synchronized one and a non-synchronized one?

I think what I'll do for now is write the non-synchronized versions, and then see about maybe automating the synchronized parts.

-Steve

On Saturday 19 February 2011 19:01:16 Steven Schveighoffer wrote:
> On Sat, 19 Feb 2011 21:55:53 -0500, Jonathan M Davis <jmdavisProg@gmx.com>
> 
> wrote:
> > On Saturday 19 February 2011 18:26:25 Steven Schveighoffer wrote:
> >> I was working on an I/O library that I plan to use in development, and possibly submit to phobos, and I thought of this case.
> >> 
> >> A standard file can be shared or unshared.  In C, since there is no
> >> notion
> >> of shared/unshared, everything is shared.  So any writes/reads from a
> >> FILE
> >> * lock the object.
> >> 
> >> But in D, we can avoid those locks when the file is unshared.  However, this means I have to write two pretty much identical functions for each call.
> >> 
> >> Is there an expected way to do this?  I've really avoided doing anything
> >> with shared or threads since the new concurrency model came out, but
> >> with
> >> I/O, I'll have to deal with it.
> >> 
> >> I think a logical thing to do would be to have the shared version of the function call the unshared version after locking the object.  Is that a good idea?  Is the correct way to do this to mark the shared function synchronized, and then cast 'this' to unshared to call the other function?  Does this automatically happen with shared functions?
> > 
> > I would point out that per TDPL, either an entire class is synchronized
> > or none
> > of it is. You don't synchronize individual functions. Now, I don' think
> > that
> > that's the way that it's implemented at the moment, but that's the
> > eventual
> > situation as I understand it. So, your class shouldn't have a mixture of
> > synchronized or unsynchronized. According to TDPL, it's illegal.
> 
> OK, I kind of remember that now.  So that means, I need to create two identical hierarchies, a synchronized one and a non-synchronized one?
> 
> I think what I'll do for now is write the non-synchronized versions, and then see about maybe automating the synchronized parts.

I'd have to study up on it more to know what the best way to handle it is (I've generally avoided dealing with shared), but what you'd probably end up doing is creating a wrapper class which was synchronized. However, you might be able to get it to just automatically use the right one (unsynchronized for unshared and synchronized for shared) if you use templates.

- Jonathan M Davis

On Saturday 19 February 2011 19:12:17 Jonathan M Davis wrote:
> On Saturday 19 February 2011 19:01:16 Steven Schveighoffer wrote:
> > On Sat, 19 Feb 2011 21:55:53 -0500, Jonathan M Davis <jmdavisProg@gmx.com>
> > 
> > wrote:
> > > On Saturday 19 February 2011 18:26:25 Steven Schveighoffer wrote:
> > >> I was working on an I/O library that I plan to use in development, and possibly submit to phobos, and I thought of this case.
> > >> 
> > >> A standard file can be shared or unshared.  In C, since there is no
> > >> notion
> > >> of shared/unshared, everything is shared.  So any writes/reads from a
> > >> FILE
> > >> * lock the object.
> > >> 
> > >> But in D, we can avoid those locks when the file is unshared. However, this means I have to write two pretty much identical functions for each call.
> > >> 
> > >> Is there an expected way to do this?  I've really avoided doing
> > >> anything with shared or threads since the new concurrency model came
> > >> out, but with
> > >> I/O, I'll have to deal with it.
> > >> 
> > >> I think a logical thing to do would be to have the shared version of the function call the unshared version after locking the object.  Is that a good idea?  Is the correct way to do this to mark the shared function synchronized, and then cast 'this' to unshared to call the other function?  Does this automatically happen with shared functions?
> > > 
> > > I would point out that per TDPL, either an entire class is synchronized
> > > or none
> > > of it is. You don't synchronize individual functions. Now, I don' think
> > > that
> > > that's the way that it's implemented at the moment, but that's the
> > > eventual
> > > situation as I understand it. So, your class shouldn't have a mixture
> > > of synchronized or unsynchronized. According to TDPL, it's illegal.
> > 
> > OK, I kind of remember that now.  So that means, I need to create two identical hierarchies, a synchronized one and a non-synchronized one?
> > 
> > I think what I'll do for now is write the non-synchronized versions, and then see about maybe automating the synchronized parts.
> 
> I'd have to study up on it more to know what the best way to handle it is (I've generally avoided dealing with shared), but what you'd probably end up doing is creating a wrapper class which was synchronized. However, you might be able to get it to just automatically use the right one (unsynchronized for unshared and synchronized for shared) if you use templates.

Actually, now that I think about it, we're probably going to want a template of some kind in Phobos which wraps an unsynchronized class with a synchronized one. There are going to be plenty of cases where it would be nice to have an unsynchronized class made synchronized (a prime example would be a container class), and it would be annoying to have to keep creating your own wrappers every time that you want to do that.

- Jonathan M Davis

Steven Schveighoffer wrote:
>On Sat, 19 Feb 2011 21:55:53 -0500, Jonathan M Davis <jmdavisProg@gmx.com> wrote:
>
>> On Saturday 19 February 2011 18:26:25 Steven Schveighoffer wrote:
>>> I was working on an I/O library that I plan to use in development, and possibly submit to phobos, and I thought of this case.
>>>
>>> A standard file can be shared or unshared.  In C, since there is
>>> no notion
>>> of shared/unshared, everything is shared.  So any writes/reads from
>>> a FILE
>>> * lock the object.
>>>
>>> But in D, we can avoid those locks when the file is unshared. However, this means I have to write two pretty much identical functions for each call.
>>>
>>> Is there an expected way to do this?  I've really avoided doing
>>> anything with shared or threads since the new concurrency model
>>> came out, but with
>>> I/O, I'll have to deal with it.
>>>
>>> I think a logical thing to do would be to have the shared version of the function call the unshared version after locking the object.  Is that a good idea?  Is the correct way to do this to mark the shared function synchronized, and then cast 'this' to unshared to call the other function?  Does this automatically happen with shared functions?
>>
>> I would point out that per TDPL, either an entire class is
>> synchronized or none
>> of it is. You don't synchronize individual functions. Now, I don'
>> think that
>> that's the way that it's implemented at the moment, but that's the
>> eventual
>> situation as I understand it. So, your class shouldn't have a
>> mixture of synchronized or unsynchronized. According to TDPL, it's
>> illegal.
>>
>
>OK, I kind of remember that now.  So that means, I need to create two identical hierarchies, a synchronized one and a non-synchronized one?
>
>I think what I'll do for now is write the non-synchronized versions, and then see about maybe automating the synchronized parts.
>
>-Steve

Creating two types, one synchronized, one not, doesn't sound like a good solution. If you need fine grained locking you can still do that manually (mark the function as shared, use the synchronized statement in the function). When I tried to do that for my std.signals replacement (which reminds me that I finally have to propose it for review) I found quite some bugs in the shared system. There's no shared signal because of those:

http://d.puremagic.com/issues/show_bug.cgi?id=3733
This one is very annoying and I think I hit one or two cases where
even the workaround didn't work.

IIRC operators couldn't be overloaded with shared functions.

Then I hit this issue: http://lists.puremagic.com/pipermail/digitalmars-d-learn/2010-November/019858.html And I gave up ;-)
-- 
Johannes Pfau

On 02/20/2011 04:41 AM, Jonathan M Davis wrote:
> On Saturday 19 February 2011 19:12:17 Jonathan M Davis wrote:
>> On Saturday 19 February 2011 19:01:16 Steven Schveighoffer wrote:
>>> On Sat, 19 Feb 2011 21:55:53 -0500, Jonathan M Davis
>>> <jmdavisProg@gmx.com>
>>>
>>> wrote:
>>>> On Saturday 19 February 2011 18:26:25 Steven Schveighoffer wrote:
>>>>> I was working on an I/O library that I plan to use in development, and
>>>>> possibly submit to phobos, and I thought of this case.
>>>>>
>>>>> A standard file can be shared or unshared.  In C, since there is no
>>>>> notion
>>>>> of shared/unshared, everything is shared.  So any writes/reads from a
>>>>> FILE
>>>>> * lock the object.
>>>>>
>>>>> But in D, we can avoid those locks when the file is unshared.
>>>>> However, this means I have to write two pretty much identical
>>>>> functions for each call.
>>>>>
>>>>> Is there an expected way to do this?  I've really avoided doing
>>>>> anything with shared or threads since the new concurrency model came
>>>>> out, but with
>>>>> I/O, I'll have to deal with it.
>>>>>
>>>>> I think a logical thing to do would be to have the shared version of
>>>>> the function call the unshared version after locking the object.  Is
>>>>> that a good idea?  Is the correct way to do this to mark the shared
>>>>> function synchronized, and then cast 'this' to unshared to call the
>>>>> other function?  Does this automatically happen with shared
>>>>> functions?
>>>>
>>>> I would point out that per TDPL, either an entire class is synchronized
>>>> or none
>>>> of it is. You don't synchronize individual functions. Now, I don' think
>>>> that
>>>> that's the way that it's implemented at the moment, but that's the
>>>> eventual
>>>> situation as I understand it. So, your class shouldn't have a mixture
>>>> of synchronized or unsynchronized. According to TDPL, it's illegal.
>>>
>>> OK, I kind of remember that now.  So that means, I need to create two
>>> identical hierarchies, a synchronized one and a non-synchronized one?
>>>
>>> I think what I'll do for now is write the non-synchronized versions, and
>>> then see about maybe automating the synchronized parts.
>>
>> I'd have to study up on it more to know what the best way to handle it is
>> (I've generally avoided dealing with shared), but what you'd probably end
>> up doing is creating a wrapper class which was synchronized. However, you
>> might be able to get it to just automatically use the right one
>> (unsynchronized for unshared and synchronized for shared) if you use
>> templates.
>
> Actually, now that I think about it, we're probably going to want a template of
> some kind in Phobos which wraps an unsynchronized class with a synchronized one.
> There are going to be plenty of cases where it would be nice to have an
> unsynchronized class made synchronized (a prime example would be a container
> class), and it would be annoying to have to keep creating your own wrappers
> every time that you want to do that.

I have not had a look at D's threading implementation, but from what I read I thought what you describe precisely was the whole point of unshared-by-default and the 'shared' keyword. Eg I thought something like:
	class Foo {...}
	shared foo = new Foo();
would do what you write above. Else, what does 'shared' mean?

Denis
-- 
_________________
vita es estrany
spir.wikidot.com

On Sunday 20 February 2011 05:36:25 spir wrote:
> On 02/20/2011 04:41 AM, Jonathan M Davis wrote:
> > On Saturday 19 February 2011 19:12:17 Jonathan M Davis wrote:
> >> On Saturday 19 February 2011 19:01:16 Steven Schveighoffer wrote:
> >>> On Sat, 19 Feb 2011 21:55:53 -0500, Jonathan M Davis <jmdavisProg@gmx.com>
> >>> 
> >>> wrote:
> >>>> On Saturday 19 February 2011 18:26:25 Steven Schveighoffer wrote:
> >>>>> I was working on an I/O library that I plan to use in development, and possibly submit to phobos, and I thought of this case.
> >>>>> 
> >>>>> A standard file can be shared or unshared.  In C, since there is no
> >>>>> notion
> >>>>> of shared/unshared, everything is shared.  So any writes/reads from a
> >>>>> FILE
> >>>>> * lock the object.
> >>>>> 
> >>>>> But in D, we can avoid those locks when the file is unshared. However, this means I have to write two pretty much identical functions for each call.
> >>>>> 
> >>>>> Is there an expected way to do this?  I've really avoided doing
> >>>>> anything with shared or threads since the new concurrency model came
> >>>>> out, but with
> >>>>> I/O, I'll have to deal with it.
> >>>>> 
> >>>>> I think a logical thing to do would be to have the shared version of the function call the unshared version after locking the object.  Is that a good idea?  Is the correct way to do this to mark the shared function synchronized, and then cast 'this' to unshared to call the other function?  Does this automatically happen with shared functions?
> >>>> 
> >>>> I would point out that per TDPL, either an entire class is
> >>>> synchronized or none
> >>>> of it is. You don't synchronize individual functions. Now, I don'
> >>>> think that
> >>>> that's the way that it's implemented at the moment, but that's the
> >>>> eventual
> >>>> situation as I understand it. So, your class shouldn't have a mixture
> >>>> of synchronized or unsynchronized. According to TDPL, it's illegal.
> >>> 
> >>> OK, I kind of remember that now.  So that means, I need to create two identical hierarchies, a synchronized one and a non-synchronized one?
> >>> 
> >>> I think what I'll do for now is write the non-synchronized versions, and then see about maybe automating the synchronized parts.
> >> 
> >> I'd have to study up on it more to know what the best way to handle it is (I've generally avoided dealing with shared), but what you'd probably end up doing is creating a wrapper class which was synchronized. However, you might be able to get it to just automatically use the right one (unsynchronized for unshared and synchronized for shared) if you use templates.
> > 
> > Actually, now that I think about it, we're probably going to want a template of some kind in Phobos which wraps an unsynchronized class with a synchronized one. There are going to be plenty of cases where it would be nice to have an unsynchronized class made synchronized (a prime example would be a container class), and it would be annoying to have to keep creating your own wrappers every time that you want to do that.
> 
> I have not had a look at D's threading implementation, but from what I read
> I thought what you describe precisely was the whole point of
> unshared-by-default and the 'shared' keyword. Eg I thought something like:
> 	class Foo {...}
> 	shared foo = new Foo();
> would do what you write above. Else, what does 'shared' mean?

shared means that it's not thread local, and that means that you need to worry about mutexes and synchronized and the like. So, Steve's concern is that if you use the class and it's shared, it needs to be synchronized. But if it's not shared, you don't want the overhead of synchronization. So, you make a version which is not synchronized and then whoever uses it has to worry about using mutexes to protect it. Creating a wrapper class which is synchronized is a good way to do so. Having a template which produced such a wrapper for you would be particularly useful so that you don't have to write a new wrapper class by hand every time that you need have a synchronized version of an unsynchronized class.

- Jonathan M Davis

- Jonathan M Davis