View mode: basic / threaded / horizontal-split · Log in · Help
June 07, 2008
Implementing multithreading policy templates in D?
Greetings,

While implementing a design I ran into a need for multiple implementations with different threading policies.  It looked to me as if porting Loki's Threading Model policies over to D would be just the ticket.  Unfortunately there's no mutex-like class in Phobos that I can find and Object's monitor is not exposed so no way to acquire/release on it other than through synchronized.  

Without a mutex like object having aquire/release semantics, I'd have to litter my code with a bunch of compile time conditionals instead of using a RAII Lock object.  Such a Lock object could be provided via template argument allowing different threading policies for different instantiations.  In the 'non-thread-safe' scenario the Lock object would just be an empty object that (I assume) the compiler would optimize away.

Since I couldn't figure out how to make such a Lock object using the synchronized statement, I tried rolling my own mutex on top of D 2.014's standard library (Phobos) and language primitives.   Since volatile seems to be deprecated, my design uses two synchronization objects through synchronized statements and requires a waitable object that can be signaled by the releasing thread to avoid busy-wait.

At this point I hit brick wall #2 (#1 being the absence of a mutex).  The only waitable object I can find in the standard library is Thread and the only event you can wait on is its death.  Up to this point the pure D standard mutex implementation was doable (though undoubtedly less efficient than native implementations).  Creating a std.Thread derived class to be used as a one shot wait object pushed the design into the yes it will work but it's completely absurd camp.

Having used about every 'mainstream' language over the past twenty odd years, I figure either I'm missing something huge and need to learn an entirely new approach or there's something missing from the standard library.  So I'm left with three questions:

Did I miss something in the docs/std lib code?

Is there a way to implement flexible threading policies using synchronized statements?

What are the chances we'll see Object sporting wait/notify methods or lock/unlock methods in a future release?

Thanks,
Brian
June 07, 2008
Re: Implementing multithreading policy templates in D?
Brian Price wrote:

> Greetings,
> 
> While implementing a design I ran into a need for multiple implementations
> with different threading policies.  It looked to me as if porting Loki's
> Threading Model policies over to D would be just the ticket. 
> Unfortunately there's no mutex-like class in Phobos that I can find and
> Object's monitor is not exposed so no way to acquire/release on it other
> than through synchronized.
> 
> Without a mutex like object having aquire/release semantics, I'd have to
> litter my code with a bunch of compile time conditionals instead of using
> a RAII Lock object.  Such a Lock object could be provided via template
> argument allowing different threading policies for different
> instantiations.  In the 'non-thread-safe' scenario the Lock object would
> just be an empty object that (I assume) the compiler would optimize away.
> 
> Since I couldn't figure out how to make such a Lock object using the
> synchronized statement, I tried rolling my own mutex on top of D 2.014's
> standard library (Phobos) and language primitives.   Since volatile seems
> to be deprecated, my design uses two synchronization objects through
> synchronized statements and requires a waitable object that can be
> signaled by the releasing thread to avoid busy-wait.
> 
> At this point I hit brick wall #2 (#1 being the absence of a mutex).  The
> only waitable object I can find in the standard library is Thread and the
> only event you can wait on is its death.  Up to this point the pure D
> standard mutex implementation was doable (though undoubtedly less
> efficient than native implementations).  Creating a std.Thread derived
> class to be used as a one shot wait object pushed the design into the yes
> it will work but it's completely absurd camp.
> 
> Having used about every 'mainstream' language over the past twenty odd
> years, I figure either I'm missing something huge and need to learn an
> entirely new approach or there's something missing from the standard
> library.  So I'm left with three questions:
> 
> Did I miss something in the docs/std lib code?
> 
> Is there a way to implement flexible threading policies using synchronized
> statements?
> 
> What are the chances we'll see Object sporting wait/notify methods or
> lock/unlock methods in a future release?
> 
> Thanks,
> Brian

Tango should have most (if not all) the threading primitives you need - to
be frank, for any serious threading code, Phobos doesn't cut it.

-- 
Lars Ivar Igesund
blog at http://larsivi.net
DSource, #d.tango & #D: larsivi
Dancing the Tango
June 08, 2008
Re: Implementing multithreading policy templates in D?
Brian Price wrote:
> Greetings,
> 
> While implementing a design I ran into a need for multiple implementations with different threading policies.  It looked to me as if porting Loki's Threading Model policies over to D would be just the ticket.  Unfortunately there's no mutex-like class in Phobos that I can find and Object's monitor is not exposed so no way to acquire/release on it other than through synchronized.  
> 
> Without a mutex like object having aquire/release semantics, I'd have to litter my code with a bunch of compile time conditionals instead of using a RAII Lock object.  Such a Lock object could be provided via template argument allowing different threading policies for different instantiations.  In the 'non-thread-safe' scenario the Lock object would just be an empty object that (I assume) the compiler would optimize away.
> 
> Since I couldn't figure out how to make such a Lock object using the synchronized statement, I tried rolling my own mutex on top of D 2.014's standard library (Phobos) and language primitives.   Since volatile seems to be deprecated, my design uses two synchronization objects through synchronized statements and requires a waitable object that can be signaled by the releasing thread to avoid busy-wait.
> 
> At this point I hit brick wall #2 (#1 being the absence of a mutex).  The only waitable object I can find in the standard library is Thread and the only event you can wait on is its death.  Up to this point the pure D standard mutex implementation was doable (though undoubtedly less efficient than native implementations).  Creating a std.Thread derived class to be used as a one shot wait object pushed the design into the yes it will work but it's completely absurd camp.
> 
> Having used about every 'mainstream' language over the past twenty odd years, I figure either I'm missing something huge and need to learn an entirely new approach or there's something missing from the standard library.  So I'm left with three questions:
> 
> Did I miss something in the docs/std lib code?
> 
> Is there a way to implement flexible threading policies using synchronized statements?
> 
> What are the chances we'll see Object sporting wait/notify methods or lock/unlock methods in a future release?
> 
> Thanks,
> Brian

Scrapple.Tools.Threads implements the most important threading primitives on Win32 and Posix.

http://dsource.org/projects/scrapple/browser/trunk/tools/tools/threads.d

--downs
June 08, 2008
Re: Implementing multithreading policy templates in D?
Lars Ivar Igesund Wrote:

> Brian Price wrote:
> 
> > Greetings,
> > 
> > While implementing a design I ran into a need for multiple implementations
> > with different threading policies.  It looked to me as if porting Loki's
> > Threading Model policies over to D would be just the ticket. 
> > Unfortunately there's no mutex-like class in Phobos that I can find and
> > Object's monitor is not exposed so no way to acquire/release on it other
> > than through synchronized.
<< -- snip --- >>
> > Having used about every 'mainstream' language over the past twenty odd
> > years, I figure either I'm missing something huge and need to learn an
> > entirely new approach or there's something missing from the standard
> > library.  So I'm left with three questions:
> > 
> > Did I miss something in the docs/std lib code?
> > 
> > Is there a way to implement flexible threading policies using synchronized
> > statements?
> > 
> > What are the chances we'll see Object sporting wait/notify methods or
> > lock/unlock methods in a future release?
> > 
> > Thanks,
> > Brian
> 
> Tango should have most (if not all) the threading primitives you need - to
> be frank, for any serious threading code, Phobos doesn't cut it.
> 
> -- 
> Lars Ivar Igesund
> blog at http://larsivi.net
> DSource, #d.tango & #D: larsivi
> Dancing the Tango

Thank you for your quick reply.  I've looked at Tango and it does seem to have an abundance, any idea when it will be ported to D 2.x?

Brian
June 08, 2008
Re: Implementing multithreading policy templates in D?
downs Wrote:

> Brian Price wrote:
> > Greetings,
> > 
> > While implementing a design I ran into a need for multiple implementations with different threading policies.  It looked to me as if porting Loki's Threading Model policies over to D would be just the ticket.  Unfortunately there's no mutex-like class in Phobos that I can find and Object's monitor is not exposed so no way to acquire/release on it other than through synchronized.  
> > 
<<---- snip ---- >>
> > 
> > Having used about every 'mainstream' language over the past twenty odd years, I figure either I'm missing something huge and need to learn an entirely new approach or there's something missing from the standard library.  So I'm left with three questions:
> > 
> > Did I miss something in the docs/std lib code?
> > 
> > Is there a way to implement flexible threading policies using synchronized statements?
> > 
> > What are the chances we'll see Object sporting wait/notify methods or lock/unlock methods in a future release?
> > 
> > Thanks,
> > Brian
> 
> Scrapple.Tools.Threads implements the most important threading primitives on Win32 and Posix.
> 
> http://dsource.org/projects/scrapple/browser/trunk/tools/tools/threads.d
> 
>  --downs

Thanks, it looks good, does it compile & run under D 2.014?  

For the moment I'm using the simple hack: 	
extern (C) void _d_monitorenter(Object obj);
extern (C) void _d_monitorexit(Object obj);
for a basic acquire/release mutex, but as my needs expand I want to avoid reinventing the wheel.

I'd be a lot more comfortable though if the powers that be would expose those two functions as methods on Object.

Brian
June 08, 2008
Re: Implementing multithreading policy templates in D?
Brian Price wrote:
> downs Wrote:
> 
>> Brian Price wrote:
>>> Greetings,
>>>
>>> While implementing a design I ran into a need for multiple implementations with different threading policies.  It looked to me as if porting Loki's Threading Model policies over to D would be just the ticket.  Unfortunately there's no mutex-like class in Phobos that I can find and Object's monitor is not exposed so no way to acquire/release on it other than through synchronized.  
>>>
> <<---- snip ---- >>
>>> Having used about every 'mainstream' language over the past twenty odd years, I figure either I'm missing something huge and need to learn an entirely new approach or there's something missing from the standard library.  So I'm left with three questions:
>>>
>>> Did I miss something in the docs/std lib code?
>>>
>>> Is there a way to implement flexible threading policies using synchronized statements?
>>>
>>> What are the chances we'll see Object sporting wait/notify methods or lock/unlock methods in a future release?
>>>
>>> Thanks,
>>> Brian
>> Scrapple.Tools.Threads implements the most important threading primitives on Win32 and Posix.
>>
>> http://dsource.org/projects/scrapple/browser/trunk/tools/tools/threads.d
>>
>>  --downs
> 
> Thanks, it looks good, does it compile & run under D 2.014?  
> 
No idea, I never tried - GDC 2.0 support is spotty.

> For the moment I'm using the simple hack: 	
> extern (C) void _d_monitorenter(Object obj);
> extern (C) void _d_monitorexit(Object obj);
> for a basic acquire/release mutex, but as my needs expand I want to avoid reinventing the wheel.
> 
> I'd be a lot more comfortable though if the powers that be would expose those two functions as methods on Object.
> 

Very much agreed.

> Brian
>
June 08, 2008
Re: Implementing multithreading policy templates in D?
== Quote from Brian Price (blprice61@yahoo.com)'s article
> downs Wrote:
> > Brian Price wrote:
> > > Greetings,
> > >
> > > While implementing a design I ran into a need for multiple implementations with different threading policies.  It looked to me as if
porting Loki's Threading Model policies over to D would be just the ticket.  Unfortunately there's no mutex-like class in Phobos that I can
find and Object's monitor is not exposed so no way to acquire/release on it other than through synchronized.
> > >
> <<---- snip ---- >>
> > >
> > > Having used about every 'mainstream' language over the past twenty odd years, I figure either I'm missing something huge and need
to learn an entirely new approach or there's something missing from the standard library.  So I'm left with three questions:
> > >
> > > Did I miss something in the docs/std lib code?
> > >
> > > Is there a way to implement flexible threading policies using synchronized statements?
> > >
> > > What are the chances we'll see Object sporting wait/notify methods or lock/unlock methods in a future release?
> > >
> > > Thanks,
> > > Brian
> >
> > Scrapple.Tools.Threads implements the most important threading primitives on Win32 and Posix.
> >
> > http://dsource.org/projects/scrapple/browser/trunk/tools/tools/threads.d
> >
> >  --downs
> Thanks, it looks good, does it compile & run under D 2.014?
> For the moment I'm using the simple hack:
> extern (C) void _d_monitorenter(Object obj);
> extern (C) void _d_monitorexit(Object obj);
> for a basic acquire/release mutex, but as my needs expand I want to avoid reinventing the wheel.
> I'd be a lot more comfortable though if the powers that be would expose those two functions as methods on Object.

I gave this a lot of thought, and decided that building the functionality into Object is a bad idea.  However,
the Mutex classes in Tango are integrated with the "synchronized" statement, which produces a similar
result:

auto m = new Mutex;
auto c = new Condition( m );

synchronized( m )
{
   c.wait;
}


Sean
June 08, 2008
Re: Implementing multithreading policy templates in D?
== Quote from Brian Price (blprice61@yahoo.com)'s article
> Lars Ivar Igesund Wrote:
> > Brian Price wrote:
> >
> > > Greetings,
> > >
> > > While implementing a design I ran into a need for multiple implementations
> > > with different threading policies.  It looked to me as if porting Loki's
> > > Threading Model policies over to D would be just the ticket.
> > > Unfortunately there's no mutex-like class in Phobos that I can find and
> > > Object's monitor is not exposed so no way to acquire/release on it other
> > > than through synchronized.
> << -- snip --- >>
> > > Having used about every 'mainstream' language over the past twenty odd
> > > years, I figure either I'm missing something huge and need to learn an
> > > entirely new approach or there's something missing from the standard
> > > library.  So I'm left with three questions:
> > >
> > > Did I miss something in the docs/std lib code?
> > >
> > > Is there a way to implement flexible threading policies using synchronized
> > > statements?
> > >
> > > What are the chances we'll see Object sporting wait/notify methods or
> > > lock/unlock methods in a future release?
> > >
> > > Thanks,
> > > Brian
> >
> > Tango should have most (if not all) the threading primitives you need - to
> > be frank, for any serious threading code, Phobos doesn't cut it.
> >
> > --
> > Lars Ivar Igesund
> > blog at http://larsivi.net
> > DSource, #d.tango & #D: larsivi
> > Dancing the Tango
> Thank you for your quick reply.  I've looked at Tango and it does seem to have an abundance, any idea
when it will be ported to D 2.x?

"Someday."  Doing so will take time, and I don't even have enough of that to
fix the bugs already on my plate.  That and, to be frank, supporting D 2.0
isn't a terribly high priority.  Perhaps if GDC had better 2.0 support.


Sean
June 08, 2008
Re: Implementing multithreading policy templates in D?
Sean Kelly Wrote:

> == Quote from Brian Price (blprice61@yahoo.com)'s article
> > downs Wrote:
> > > Brian Price wrote:
> > > > Greetings,
> > > >
> > > > While implementing a design I ran into a need for multiple implementations with different threading policies.  It looked to me as if
> porting Loki's Threading Model policies over to D would be just the ticket.  Unfortunately there's no mutex-like class in Phobos that I can
> find and Object's monitor is not exposed so no way to acquire/release on it other than through synchronized.
> > > >
> > <<---- snip ---- >>
> > > >
> > > > Having used about every 'mainstream' language over the past twenty odd years, I figure either I'm missing something huge and need
> to learn an entirely new approach or there's something missing from the standard library.  So I'm left with three questions:
> > > >
> > > > Did I miss something in the docs/std lib code?
> > > >
> > > > Is there a way to implement flexible threading policies using synchronized statements?
> > > >
> > > > What are the chances we'll see Object sporting wait/notify methods or lock/unlock methods in a future release?
> > > >
> > > > Thanks,
> > > > Brian
> > >
> > > Scrapple.Tools.Threads implements the most important threading primitives on Win32 and Posix.
> > >
> > > http://dsource.org/projects/scrapple/browser/trunk/tools/tools/threads.d
> > >
> > >  --downs
> > Thanks, it looks good, does it compile & run under D 2.014?
> > For the moment I'm using the simple hack:
> > extern (C) void _d_monitorenter(Object obj);
> > extern (C) void _d_monitorexit(Object obj);
> > for a basic acquire/release mutex, but as my needs expand I want to avoid reinventing the wheel.
> > I'd be a lot more comfortable though if the powers that be would expose those two functions as methods on Object.
> 
> I gave this a lot of thought, and decided that building the functionality into Object is a bad idea.  However,
> the Mutex classes in Tango are integrated with the "synchronized" statement, which produces a similar
> result:
> 
> auto m = new Mutex;
> auto c = new Condition( m );
> 
> synchronized( m )
> {
>     c.wait;
> }
> 
> 
> Sean

Problem is that the synchronized keyword does not solve the problem, no matter how it's used.  Why should different template class specializations be forced to pay the synchronization penalty when some may be used in an inherently thread safe manner, while others may need class level locking, and still others object level locking?  Usage example follows:

template ObjectLockablePolicy( MutexType = DefaultMutexPolicy )
{
	private MutexType mutex_;

	private void initSynchPolicy() 
	{
		mutex_ = new MutexType();
	}

	scope class Lock
	{
		this() 
		{
			mutex_.acquire();
		}

		~this()
		{
			mutex_.release();
		}
	}
}

class SomeClass
{
	int a,b;
	
	mixin ClassLockablePolicy;

       this() 
       {
             initSynchPolicy();
       }

	void somefunc() 
      {
		scope Lock lock = new Lock();
		writefln("inside scoped lock");
	}
}

This creates the equivalent of a synchronized(mutex_) block around writefln, but consider the ugliness necessary with initSynchPolicy().  If the methods were exposed on Object (instead of hidden away as they are now) that initialization call would be unnecessary.  Lock's constructor/destructor could simply call acquire/release on the outer this pointer in this particular case.

It'd be different if every Object wasn't built on top of a monitor anyhow, but not only is it true of the current implementation, but must be true for any implementation as long as sychronized(someObjectInstance) is allowed.

Sincerely,
Brian Price
June 08, 2008
Re: Implementing multithreading policy templates in D?
== Quote from Brian Price (blprice61@yahoo.com)'s article
> Sean Kelly Wrote:
> > == Quote from Brian Price (blprice61@yahoo.com)'s article
> > > downs Wrote:
> > > > Brian Price wrote:
> > > > > Greetings,
> > > > >
> > > > > While implementing a design I ran into a need for multiple implementations with different threading policies.  It looked to me as if
> > porting Loki's Threading Model policies over to D would be just the ticket.  Unfortunately there's no mutex-like class in Phobos that I can
> > find and Object's monitor is not exposed so no way to acquire/release on it other than through synchronized.
> > > > >
> > > <<---- snip ---- >>
> > > > >
> > > > > Having used about every 'mainstream' language over the past twenty odd years, I figure either I'm missing something huge and need
> > to learn an entirely new approach or there's something missing from the standard library.  So I'm left with three questions:
> > > > >
> > > > > Did I miss something in the docs/std lib code?
> > > > >
> > > > > Is there a way to implement flexible threading policies using synchronized statements?
> > > > >
> > > > > What are the chances we'll see Object sporting wait/notify methods or lock/unlock methods in a future release?
> > > > >
> > > > > Thanks,
> > > > > Brian
> > > >
> > > > Scrapple.Tools.Threads implements the most important threading primitives on Win32 and Posix.
> > > >
> > > > http://dsource.org/projects/scrapple/browser/trunk/tools/tools/threads.d
> > > >
> > > >  --downs
> > > Thanks, it looks good, does it compile & run under D 2.014?
> > > For the moment I'm using the simple hack:
> > > extern (C) void _d_monitorenter(Object obj);
> > > extern (C) void _d_monitorexit(Object obj);
> > > for a basic acquire/release mutex, but as my needs expand I want to avoid reinventing the wheel.
> > > I'd be a lot more comfortable though if the powers that be would expose those two functions as methods on Object.
> >
> > I gave this a lot of thought, and decided that building the functionality into Object is a bad idea.  However,
> > the Mutex classes in Tango are integrated with the "synchronized" statement, which produces a similar
> > result:
> >
> > auto m = new Mutex;
> > auto c = new Condition( m );
> >
> > synchronized( m )
> > {
> >     c.wait;
> > }
> >
> >
> > Sean
> Problem is that the synchronized keyword does not solve the problem, no matter how it's used.  Why should different template class specializations be forced to pay the synchronization
penalty when some may be used in an inherently thread safe manner, while others may need class level locking, and still others object level locking?  Usage example follows:
> template ObjectLockablePolicy( MutexType = DefaultMutexPolicy )
> {
> 	private MutexType mutex_;
> 	private void initSynchPolicy()
> 	{
> 		mutex_ = new MutexType();
> 	}
> 	scope class Lock
> 	{
> 		this()
> 		{
> 			mutex_.acquire();
> 		}
> 		~this()
> 		{
> 			mutex_.release();
> 		}
> 	}
> }
> class SomeClass
> {
> 	int a,b;
> 	mixin ClassLockablePolicy;
>         this()
>         {
>               initSynchPolicy();
>         }
> 	void somefunc()
>        {
> 		scope Lock lock = new Lock();
> 		writefln("inside scoped lock");
> 	}
> }
> This creates the equivalent of a synchronized(mutex_) block around writefln, but consider the ugliness necessary with initSynchPolicy().  If the methods were exposed on Object (instead of
hidden away as they are now) that initialization call would be unnecessary.  Lock's constructor/destructor could simply call acquire/release on the outer this pointer in this particular case.
> It'd be different if every Object wasn't built on top of a monitor anyhow, but not only is it true of the current implementation, but must be true for any implementation as long as
sychronized(someObjectInstance) is allowed.

One interesting aspect of the Tango approach is that you can provide the monitor object for any object in D.  The motivation
for this was similar to what you're saying above.  By building the functionality into Object directly (and therefore into the
runtime), the user is constrained by the assumptions made by the runtime implementation.  By contrast, the Tango approach
allows the user to provide any monitor implementation they wish, as need dictates.  My original motivation for this was to
allow objects to be allocated in shared memory and have "synchronized" still work with them, but the same approach allows
for the use of futexes, etc.  Interestingly, the Tango Mutex is actually also its own monitor because of this approach.  However,
in general I was thinking along these lines:

class C
{
   void foo()
   {
       synchronized
       {
           // A
       }
   }
}

auto c = new C;
injectMonitor( c, new Mutex );
c.foo();

The injectMonitor call is just a one-line function.  So at point A, the user-provided mutex is locked.  In fact, the default runtime-
provided monitor is never constructed at all.  I grant that requiring injectMonitor to be called may be a bit more complicated than
some other approaches, but I feel it's also more flexible.


Sean
« First   ‹ Prev
1 2
Top | Discussion index | About this forum | D home