View mode: basic / threaded / horizontal-split · Log in · Help
June 26, 2004
volatile
What is the intended purpose of the D keyword volatile?

I understand that in a D statement such as:

#    volatile a[i++] = x;

(which would be nonsense in C++, of course) there is merely a guarantee that all
memory writes will be flushed before and after the statement. There is NOT a
guarantee that the statement will be executed atomically. Forgive me, but ... I
don't understand the point of this. Could someone explain? What's it FOR?

On the other hand, the C++ use of volatile is horribly missing from D. In C++, a
declaration such as:

#    volatile int x; // C++

informs that compiler that agents unknown to the compiler will be reading and
writing to this variable without warning. This affects the optimization which
the compiler is allowed to perform. For instance, given the function:

#    void f()
#    {
#        volatile int x = 0; // C++
#    }

The compiler may NOT optimize away either the existence of the variable nor its
initialization. Without the keyword volatile, the compiler could ditch x
entirely.

In D, it is (confusingly) possible to declare:

#    volatile int x;

However, this does NOT make x volatile in the C++ sense - it merely guarantees
that all memory writes will be flushed before and after the variable x is
initialized to zero. There may be a point to this, but I don't know what it is,
and in any case the use of the keyword volatile for this strange purpose is most
definitely counterintuitive if you're used to using the word in C++.

Can anyone help me out here, or is it just a mad quirk?

Arcane Jill
June 26, 2004
Re: volatile
"volatile" in C++ means a read or a write to a location cannot be "optimized
away", and the value cannot be cached in a register, but the practical uses
of that are nearly nil in modern optimizing compilers. If you need that kind
of fine control over read/write cycles, you're coding for a very specific
piece of hardware, and the inline assembler addresses that nicely.

"volatile" in Java means that reads and writes to a variable are atomic.
This can only be accomplished by wrapping the reads/writes in a synchronized
block. D has synchronized blocks, so this capability is there. (On the x86
cpu's, atomic reads/writes happen anyway for 32 bit or less sized values.
The ones that are not atomic are values bigger than 32 bits, like longs and
doubles. How often do you need an atomic write of a double?)

As optimizers have gotten better, and multi-CPU memory access along with
write caching has become commonplace, the real problem emerged - that of
guaranteeing that the *order* of reads and writes to some shared resource
(like memory) can be specified. The C++ spec specifies order only in the
presence of an assumed single threaded process, and operations can be
reordered as long as they follow the "as if" rule, i.e. the end result is
the same "as if" if were done in the same sequence as in the source. This
all falls apart under aggressive optimization. (Scott Meyers presented a
paper at SDWest giving some pretty damning evidence of this problem.)
Neither C++ nor Java volatile semantics gives any help with this.

The answer is to be able to figuratively draw a horizontal line between two
statements in the code, and say all writes before that line are completed
before the line is crossed, and all reads after that line do not happen
until after all the writes are done. This is called a write barrier, and a
read barrier. The D volatile statement is a way to specify where those
barriers go in the code. The optimizer is prevented from moving reads and
writes across that line, and to deal with memory caching problems in
multi-cpu memory, any cpu instructions to 'flush' writes will be inserted.

"Arcane Jill" <Arcane_member@pathlink.com> wrote in message
news:cbjgmm$27ia$1@digitaldaemon.com...
> What is the intended purpose of the D keyword volatile?
>
> I understand that in a D statement such as:
>
> #    volatile a[i++] = x;
>
> (which would be nonsense in C++, of course) there is merely a guarantee
that all
> memory writes will be flushed before and after the statement. There is NOT
a
> guarantee that the statement will be executed atomically. Forgive me, but
... I
> don't understand the point of this. Could someone explain? What's it FOR?
>
> On the other hand, the C++ use of volatile is horribly missing from D. In
C++, a
> declaration such as:
>
> #    volatile int x; // C++
>
> informs that compiler that agents unknown to the compiler will be reading
and
> writing to this variable without warning. This affects the optimization
which
> the compiler is allowed to perform. For instance, given the function:
>
> #    void f()
> #    {
> #        volatile int x = 0; // C++
> #    }
>
> The compiler may NOT optimize away either the existence of the variable
nor its
> initialization. Without the keyword volatile, the compiler could ditch x
> entirely.
>
> In D, it is (confusingly) possible to declare:
>
> #    volatile int x;
>
> However, this does NOT make x volatile in the C++ sense - it merely
guarantees
> that all memory writes will be flushed before and after the variable x is
> initialized to zero. There may be a point to this, but I don't know what
it is,
> and in any case the use of the keyword volatile for this strange purpose
is most
> definitely counterintuitive if you're used to using the word in C++.
>
> Can anyone help me out here, or is it just a mad quirk?
>
> Arcane Jill
>
>
June 26, 2004
Re: volatile
On Sat, 26 Jun 2004 09:47:02 +0000, Arcane Jill wrote:
> However, this does NOT make x volatile in the C++ sense - it merely guarantees
> that all memory writes will be flushed before and after the variable x is
> initialized to zero. There may be a point to this, but I don't know what it is,
> and in any case the use of the keyword volatile for this strange purpose is most
> definitely counterintuitive if you're used to using the word in C++.
> 
> Can anyone help me out here, or is it just a mad quirk?
> 
> Arcane Jill

I asked this very question before ;) If I remember today I'll see about
wikiing it somewhere.

In C++, if I say a variable is volatile, I do it as below, because volatile is
a storage class in C++.

volatile int x;
x += 1; // this statement executed as volatile

In D, I can't do that, because volatile is a statement modifier. You'd do
something like this:

int x;
volatile {
	x += 1;
}

This allows you to access x in a volatile fashion only some of the time,
allowing you to incur that overhead only where it's needed. If you want C++'s
behavior, you might try something like this, using properties:

int x() { volatile return _x; }
int x(int i) { volatile return _x += i; }

A real-world example where this is useful: In the concurrent library, there's
a part where a worker thread reads a volatile flag to determine when to exit.
However, the worker does not write to it. All writes to it happen under a
lock, so I have it accessed volatily (that's a word now, dammit!) only in the
worker thread.

Of course, if you want to do that, you need to be *damn* careful. But then
again, you'd be using Java if you wanted the language to baby-sit you.

Mike Swieton
__
Brutes find out where their talents lie; a bear will not attempt to fly.
	- Jonathan Swift
June 26, 2004
Re: volatile
"Mike Swieton" <mike@swieton.net> wrote in message
news:pan.2004.06.26.16.52.38.466542@swieton.net...
> In C++, if I say a variable is volatile, I do it as below, because
volatile is
> a storage class in C++.
>
> volatile int x;
> x += 1; // this statement executed as volatile
>
> In D, I can't do that, because volatile is a statement modifier. You'd do
> something like this:
>
> int x;
> volatile {
> x += 1;
> }
>
> This allows you to access x in a volatile fashion only some of the time,
> allowing you to incur that overhead only where it's needed. If you want
C++'s
> behavior, you might try something like this, using properties:
>
> int x() { volatile return _x; }
> int x(int i) { volatile return _x += i; }

C++'s volatile doesn't prevent write caching in memory in multi-CPU systems,
nor does it guarantee an order of reads/writes in multithreading. D's does.

> A real-world example where this is useful: In the concurrent library,
there's
> a part where a worker thread reads a volatile flag to determine when to
exit.
> However, the worker does not write to it. All writes to it happen under a
> lock, so I have it accessed volatily (that's a word now, dammit!) only in
the
> worker thread.

If you think you have it working right, be sure and read "C++ and the Perils
of Double-Checked Locking, Part I" (Scott Meyers, Andrei Alexandrescu), July
2004 Dr. Dobb's. Here's a taste of it:
http://weblogs.asp.net/brada/archive/2004/05/12/130935.aspx

> Of course, if you want to do that, you need to be *damn* careful. But then
> again, you'd be using Java if you wanted the language to baby-sit you.

Java's volatile won't help you, since it doesn't prevent reordering of
operations.
June 26, 2004
Re: volatile
Walter wrote:
>
> The answer is to be able to figuratively draw a horizontal line between two
> statements in the code, and say all writes before that line are completed
> before the line is crossed, and all reads after that line do not happen
> until after all the writes are done. This is called a write barrier, and a
> read barrier. The D volatile statement is a way to specify where those
> barriers go in the code. The optimizer is prevented from moving reads and
> writes across that line, and to deal with memory caching problems in
> multi-cpu memory, any cpu instructions to 'flush' writes will be inserted.

This is fantastic.  So you're saying I don't have to write inline ASM to 
do (multi-cpu aware) atomic operations in D?  This may be the best 
feature I've seen yet :)


Sean
June 26, 2004
Re: volatile
"Sean Kelly" <sean@f4.ca> wrote in message
news:cbkhqp$i5u$1@digitaldaemon.com...
> Walter wrote:
> >
> > The answer is to be able to figuratively draw a horizontal line between
two
> > statements in the code, and say all writes before that line are
completed
> > before the line is crossed, and all reads after that line do not happen
> > until after all the writes are done. This is called a write barrier, and
a
> > read barrier. The D volatile statement is a way to specify where those
> > barriers go in the code. The optimizer is prevented from moving reads
and
> > writes across that line, and to deal with memory caching problems in
> > multi-cpu memory, any cpu instructions to 'flush' writes will be
inserted.
>
> This is fantastic.  So you're saying I don't have to write inline ASM to
> do (multi-cpu aware) atomic operations in D?  This may be the best
> feature I've seen yet :)

You'll still have to if you want to use a LOCK prefix :-(
June 26, 2004
Re: volatile
Walter wrote:
> "volatile" in C++ means a read or a write to a location cannot be "optimized
> away", and the value cannot be cached in a register, but the practical uses
> of that are nearly nil in modern optimizing compilers. If you need that kind
> of fine control over read/write cycles, you're coding for a very specific
> piece of hardware, and the inline assembler addresses that nicely.
> 
> "volatile" in Java means that reads and writes to a variable are atomic.
> This can only be accomplished by wrapping the reads/writes in a synchronized
> block. D has synchronized blocks, so this capability is there. (On the x86
> cpu's, atomic reads/writes happen anyway for 32 bit or less sized values.
> The ones that are not atomic are values bigger than 32 bits, like longs and
> doubles. How often do you need an atomic write of a double?)
> 
> As optimizers have gotten better, and multi-CPU memory access along with
> write caching has become commonplace, the real problem emerged - that of
> guaranteeing that the *order* of reads and writes to some shared resource
> (like memory) can be specified. The C++ spec specifies order only in the
> presence of an assumed single threaded process, and operations can be
> reordered as long as they follow the "as if" rule, i.e. the end result is
> the same "as if" if were done in the same sequence as in the source. This
> all falls apart under aggressive optimization. (Scott Meyers presented a
> paper at SDWest giving some pretty damning evidence of this problem.)
> Neither C++ nor Java volatile semantics gives any help with this.
> 
> The answer is to be able to figuratively draw a horizontal line between two
> statements in the code, and say all writes before that line are completed
> before the line is crossed, and all reads after that line do not happen
> until after all the writes are done. This is called a write barrier, and a
> read barrier. The D volatile statement is a way to specify where those
> barriers go in the code. The optimizer is prevented from moving reads and
> writes across that line, and to deal with memory caching problems in
> multi-cpu memory, any cpu instructions to 'flush' writes will be inserted.

I wasn't aware of this, but I would tend to agree with Arcane Jill here, 
that the use of 'volatile' in D doesn't make immediate sense. While I 
see what you're saying, I don't think the word 'volatile' semantically 
indicates an 'optimization barrier', and that possibly a different 
keyword would better indicate this, and diffuse confusion from coming 
from other languages.

As to specific hardware, I think you're wrong. I've seen it used in 
timer code for the Allegro game programming library, and frankly it was 
exceptionally convenient. I wouldn't want to delve into inline assembler 
for the purpose it had there, and would never expect a highish level 
programming language to require me to.

Cheers,
Sigbjørn Lund Olsen
June 26, 2004
Re: volatile
"Sigbjørn Lund Olsen" <sigbjorn@lundolsen.net> wrote in message
news:cbkov1$sqt$1@digitaldaemon.com...
> I wasn't aware of this, but I would tend to agree with Arcane Jill here,
> that the use of 'volatile' in D doesn't make immediate sense. While I
> see what you're saying, I don't think the word 'volatile' semantically
> indicates an 'optimization barrier', and that possibly a different
> keyword would better indicate this, and diffuse confusion from coming
> from other languages.

There is no consistency anyway between languages for the meaning of
volatile, as one will find moving between C++, C# and Java where they mean
*very* different things. The use of volatile in C# is more akin to the D
usage, and as I recall, Java will be moving that direction as well. All I
can say is writing multithread safe code is very demanding, and one will
have to read the documentation very carefully regardless.

> As to specific hardware, I think you're wrong. I've seen it used in
> timer code for the Allegro game programming library, and frankly it was
> exceptionally convenient. I wouldn't want to delve into inline assembler
> for the purpose it had there, and would never expect a highish level
> programming language to require me to.

The C++ semantics don't even make much sense in a hardware environment with
multilevel write caching. If you need to do a read cycle on a specific
memory location, may I suggest:

   void read_cycle(void *location)
   {
       asm { mov EAX, location[ESP]; mov EAX,[EAX]; }
   }

which does what you're asking for rather neatly, and an analogous one could
be done for a write cycle. The volatile semantics in C++ ripple throughout
everything, adding much complication it its wake, all to avoid having a
single, simple function as above.
June 27, 2004
Re: volatile
> The D volatile statement is a way to specify
> where those barriers go in the code. The optimizer is prevented from
> moving reads and writes across that line, and to deal with memory caching
> problems in multi-cpu memory, any cpu instructions to 'flush' writes will
> be inserted.

Can you add that part about "volatile flushes writes" to the doc? It just
mentions the code movement and doesn't say anything about caching. Also I
assume volatile will force reads to be from main memory, right?
June 27, 2004
Re: volatile
"Ben Hinkle" <bhinkle4@juno.com> wrote in message
news:cbmn17$cvn$1@digitaldaemon.com...
> > The D volatile statement is a way to specify
> > where those barriers go in the code. The optimizer is prevented from
> > moving reads and writes across that line, and to deal with memory
caching
> > problems in multi-cpu memory, any cpu instructions to 'flush' writes
will
> > be inserted.
>
> Can you add that part about "volatile flushes writes" to the doc? It just
> mentions the code movement and doesn't say anything about caching. Also I
> assume volatile will force reads to be from main memory, right?

Across a memory barrier, yes.
Top | Discussion index | About this forum | D home