Am Thu, 24 Oct 2013 14:04:44 +0100
schrieb Iain Buclaw <ibuclaw@ubuntu.com>:

> On 24 October 2013 12:10, John Colvin <john.loughran.colvin@gmail.com> wrote:
> > On Thursday, 24 October 2013 at 09:43:51 UTC, Iain Buclaw wrote:
> >>>>
> >>>> 'shared' guarantees that all reads and writes specified in source code happen in the exact order specified with no omissions

Does this include writes to non-shared data? For example:
------------------------------------
shared int x;
int y;

void func()
{
    x = 0;
    y = 3;  //Can the compiler move this assignment?
    x = 1;
}
------------------------------------

So there's no additional overhead (in code / instructions emitted) when using shared instead of volatile in code like this? And this is valid code with shared (assuming reading/assigning to x is atomic)?
------------------------------------
volatile bool x = false;

void waitForX()
{
    while(!x){}
}

__interrupt(X) void x_interrupt()
{
    x = true;
}
------------------------------------

On Thursday, 24 October 2013 at 17:02:51 UTC, Walter Bright wrote:
> On 10/24/2013 4:18 AM, eles wrote:
>> On Thursday, 24 October 2013 at 06:48:07 UTC, Walter Bright wrote:
>>> On 10/23/2013 11:19 PM, Mike wrote:
> Like I said, nobody (on the standards committees) could agree on exactly what that meant.

The standard committees might not agree, but there is somebody out there that really knows very accurately what that should mean: that somebody is the hardware itself.

Just imagine the best hardware example that you have at hand: the microprocessor that you are programming for.

It writes on the bus, there is a short delay before the signals are guaranteed to reach the correct levels, then reads the memory data and so on.

You cannot read the data before the delay passes. You cannot say "well, I could postpone the writing on the address on the bus, let's read the memory location first" -- or you would read garbage. Or you cannot say: well, first I will execute the program without a processor then, when the user is already pissed off, I would finally execute all those instructions at once. Too bad that the computer is already flying through the window at that time.

You command that processor from the compiler. Now, the thing that's needed is to give a way to do the same (ie commanding a hardware) from the program compiled by the compiler.

On 10/24/2013 11:33 AM, eles wrote:
> On Thursday, 24 October 2013 at 17:02:51 UTC, Walter Bright wrote:
>> On 10/24/2013 4:18 AM, eles wrote:
>>> On Thursday, 24 October 2013 at 06:48:07 UTC, Walter Bright wrote:
>>>> On 10/23/2013 11:19 PM, Mike wrote:
>> Like I said, nobody (on the standards committees) could agree on exactly what
>> that meant.
>
> The standard committees might not agree, but there is somebody out there that
> really knows very accurately what that should mean: that somebody is the
> hardware itself.
>
> Just imagine the best hardware example that you have at hand: the microprocessor
> that you are programming for.
>
> It writes on the bus, there is a short delay before the signals are guaranteed
> to reach the correct levels, then reads the memory data and so on.
>
> You cannot read the data before the delay passes. You cannot say "well, I could
> postpone the writing on the address on the bus, let's read the memory location
> first" -- or you would read garbage. Or you cannot say: well, first I will
> execute the program without a processor then, when the user is already pissed
> off, I would finally execute all those instructions at once. Too bad that the
> computer is already flying through the window at that time.
>
> You command that processor from the compiler. Now, the thing that's needed is to
> give a way to do the same (ie commanding a hardware) from the program compiled
> by the compiler.

The trouble with that is since the standards people cannot agree on what volatile means, you're working with a compiler that has non-standard behavior. This is not portable and not reliable.

On 24 October 2013 18:49, Johannes Pfau <nospam@example.com> wrote:
> Am Thu, 24 Oct 2013 14:04:44 +0100
> schrieb Iain Buclaw <ibuclaw@ubuntu.com>:
>
>> On 24 October 2013 12:10, John Colvin <john.loughran.colvin@gmail.com> wrote:
>> > On Thursday, 24 October 2013 at 09:43:51 UTC, Iain Buclaw wrote:
>> >>>>
>> >>>> 'shared' guarantees that all reads and writes specified in source code happen in the exact order specified with no omissions
>
> Does this include writes to non-shared data? For example:
> ------------------------------------
> shared int x;
> int y;
>
> void func()
> {
>     x = 0;
>     y = 3;  //Can the compiler move this assignment?
>     x = 1;
> }
> ------------------------------------
>

Yes, reordering may occur so long as the compiler does not change behaviour in respect to the programs sequential points in the application.  (Your example, for instance, can not possibly be re-ordered).

It is also worth noting while you may have guarantee of this, it does not mean that you can go using __thread data without memory barriers. (For instance, if you have an asynchronous signal handler, it may alter the __thread'ed data at any point in the sequential program).



> So there's no additional overhead (in code / instructions emitted) when using shared instead of volatile in code like this? And this is valid code with shared (assuming reading/assigning to x is atomic)?
> ------------------------------------
> volatile bool x = false;
>
> void waitForX()
> {
>     while(!x){}
> }
>
> __interrupt(X) void x_interrupt()
> {
>     x = true;
> }
> ------------------------------------

That is correct. :o)


Regards
-- 
Iain Buclaw

*(p < e ? p++ : p) = (c & 0x0f) + '0';

On 24 October 2013 07:36, Iain Buclaw <ibuclaw@ubuntu.com> wrote:
> On 24 October 2013 06:37, Walter Bright <newshound2@digitalmars.com> wrote:
>> On 10/23/2013 5:43 PM, Mike wrote:
>>>
>>> I'm interested in ARM bare-metal programming with D, and I'm trying to get
>>> my
>>> head wrapped around how to approach this.  I'm making progress, but I
>>> found
>>> something that was surprising to me: deprecation of the volatile keyword.
>>>
>>> In the bare-metal/hardware/driver world, this keyword is important to
>>> ensure the
>>> optimizer doesn't cache reads to memory-mapped IO, as some hardware
>>> peripheral
>>> may modify the value without involving the processor.
>>>
>>> I've read a few discussions on the D forums about the volatile keyword
>>> debate,
>>> but noone seemed to reconcile the need for volatile in memory-mapped IO.
>>> Was
>>> this an oversight?
>>>
>>> What's D's answer to this?  If one were to use D to read from
>>> memory-mapped IO,
>>> how would one ensure the compiler doesn't cache the value?
>>
>>
>> volatile was never a reliable method for dealing with memory mapped I/O.
>
> Are you talking dmd or in general (it's hard to tell).  In gdc, volatile is the same as in gcc/g++ in behaviour.  Although in one aspect, when the default storage model was switched to thread-local, that made volatile on it's own pointless.
>
> As a side note, 'shared' is considered a volatile type in gdc, which differs from the deprecated keyword which set volatile at a decl/expression level.  There is a difference in semantics, but it escapes this author at 6.30am in the morning.  :o)
>

To elaborate (now I am a little more awake :) - 'volatile' on the type means that it's volatile-qualified.  volatile on the decl means it's treated as volatile in the 'C' sense.

What's the difference?  Well for the backend a volatile type only really has an effect on function returns  (eg: a function that returns a shared int may not be subject for use in, say, tail-call optimisations).  GDC propagates the volatile flag of the type to the decl, so that there is effectively no difference between shared and volatile, except in a semantic sense in the D frontend language implementation.


Regards
-- 
Iain Buclaw

*(p < e ? p++ : p) = (c & 0x0f) + '0';

On Thursday, 24 October 2013 at 19:11:03 UTC, Walter Bright wrote:
> On 10/24/2013 11:33 AM, eles wrote:
>> On Thursday, 24 October 2013 at 17:02:51 UTC, Walter Bright wrote:
>>> On 10/24/2013 4:18 AM, eles wrote:
>>>> On Thursday, 24 October 2013 at 06:48:07 UTC, Walter Bright wrote:
>>>>> On 10/23/2013 11:19 PM, Mike wrote:
>>> Like I said, nobody (on the standards committees) could agree on exactly what
>>> that meant.
>>
>> The standard committees might not agree, but there is somebody out there that
>> really knows very accurately what that should mean: that somebody is the
>> hardware itself.
>>
>> Just imagine the best hardware example that you have at hand: the microprocessor
>> that you are programming for.
>>
>> It writes on the bus, there is a short delay before the signals are guaranteed
>> to reach the correct levels, then reads the memory data and so on.
>>
>> You cannot read the data before the delay passes. You cannot say "well, I could
>> postpone the writing on the address on the bus, let's read the memory location
>> first" -- or you would read garbage. Or you cannot say: well, first I will
>> execute the program without a processor then, when the user is already pissed
>> off, I would finally execute all those instructions at once. Too bad that the
>> computer is already flying through the window at that time.
>>
>> You command that processor from the compiler. Now, the thing that's needed is to
>> give a way to do the same (ie commanding a hardware) from the program compiled
>> by the compiler.
>
> The trouble with that is since the standards people cannot agree on what volatile means, you're working with a compiler that has non-standard behavior. This is not portable and not reliable.

There should be some way, in the D language, to tell the compiler  "Do exactly what I say here, and don't try to be clever about it" without introducing unnecessary (and unfortunate) overhead.  It doesn't have to be /volatile/.

/shared/ may be the solution here, but based on a comment by Iain Buclaw (http://forum.dlang.org/post/mailman.2454.1382619958.1719.digitalmars-d@puremagic.com) it seems there could be some disagreement on what this means to compiler implementers.  I don't see why "shared" could not only mean "shared by more than one thread/cpu", but also "shared by external hardware peripherals".

Maybe /shared/'s definition needs to be further defined to ensure all compilers implement it the same way, and be unambiguous enough to provide a solution to this /volatile/ debate.

Using peek and poke functions is, well, nah... Better methods exist.  Using inline assembly is a reasonable alternative, as is linking to an external C library, but why use D then?  Is low-level/embedded software development a design goal of the D language?

"Timo Sintonen" <t.sintonen@luukku.com> wrote in message news:qdvhyrzshckafkiekvnw@forum.dlang.org...
> On Thursday, 24 October 2013 at 13:22:50 UTC, Iain Buclaw wrote:
>
>> In gdc:
>> ---
>> asm {"" ::: "memory";}
>>
>> An asm instruction without any output operands will be treated identically to a volatile asm instruction in gcc, which indicates that the instruction has important side effects.  So it creates a point in the code which may not be deleted (unless it is proved to be unreachable).
>>
>> The "memory" clobber will tell the backend to not keep memory values cached in registers across the assembler instruction and not optimize stores or loads to that memory.  (That does not prevent a CPU from reordering loads and stores with respect to another CPU, though; you need real memory barrier instructions for that.)
>
> I have not (yet) had any problems when writing io registers but more with read access. Any operation after write should read the register back from real memory and not in processor registers.  Any repetitive read should always read the real io register in memory. The hardware may change the register value at any time.
>
> Now a very common task like
> while (regs.status==0) ...
> may be optimized to an endless loop because the memory is read only once
> before the loop starts.
>
> I understood from earlier posts that variables should not be volatile but the operation should. It seems it is possible to guide the compiler like above. So would the right solution be to have a volatile block, similar to synchronized? Inside that block no memory access is optimized.  This way no information of volatility is needed outside the block or in variables used there.
>

Volatile blocks are already in the language, but they suck.  You don't want to have to mark every access as volatile, because all accesses to that hardware register are going to be volatile.  You want it to be automatic.

I'm really starting to think intrinsics are the way to go.  They are safe, clear, and can be inlined.  The semantics I imagine would be along the lines of llvm's volatile memory accesses (http://llvm.org/docs/LangRef.html#volatile-memory-accesses)

On Friday, 25 October 2013 at 04:30:37 UTC, Mike wrote:
> On Thursday, 24 October 2013 at 19:11:03 UTC, Walter Bright wrote:
>> On 10/24/2013 11:33 AM, eles wrote:
>>> On Thursday, 24 October 2013 at 17:02:51 UTC, Walter Bright wrote:
>>>> On 10/24/2013 4:18 AM, eles wrote:
>>>>> On Thursday, 24 October 2013 at 06:48:07 UTC, Walter Bright wrote:
>>>>>> On 10/23/2013 11:19 PM, Mike wrote:
> Maybe /shared/'s definition needs to be further defined to ensure all compilers implement it the same way, and be unambiguous enough to provide a solution to this /volatile/ debate.

The problem with shared alone variable is that it can be simply placed by the optimizer at another memory location than the intended one, even if all threads are seeing it "as if" at the intended location.

On Friday, 25 October 2013 at 13:07:56 UTC, Daniel Murphy wrote:
> "Timo Sintonen" <t.sintonen@luukku.com> wrote:

>> I have not (yet) had any problems when writing io registers but more with read access. Any operation after write should read the register back from real memory and not in processor registers.  Any repetitive read should always read the real io register in memory. The hardware may change the register value at any time.
>>
>> Now a very common task like
>> while (regs.status==0) ...
>> may be optimized to an endless loop because the memory is read only once before the loop starts.
>>
>> I understood from earlier posts that variables should not be volatile but the operation should. It seems it is possible to guide the compiler like above. So would the right solution be to have a volatile block, similar to synchronized? Inside that block no memory access is optimized.  This way no information of volatility is needed outside the block or in variables used there.
>>
>
> Volatile blocks are already in the language, but they suck.  You don't want
> to have to mark every access as volatile, because all accesses to that
> hardware register are going to be volatile.  You want it to be automatic.
>
> I'm really starting to think intrinsics are the way to go.  They are safe,
> clear, and can be inlined.  The semantics I imagine would be along the lines
> of llvm's volatile memory accesses
> (http://llvm.org/docs/LangRef.html#volatile-memory-accesses)

It seems that it is two different things here. As far as I understand, sharing means something like 'somebody may change my data' and volatility is something like 'I have to know immediately if the data is changed'. It has become obvious that these two are not easy to fit together and make a working model.

The original question in this thread was to have a proper way to access hardware registers. So far, even the top people have offered only workarounds. I wonder how long D can be marketed as system language if it does not have a defined and reliable way to access system hardware.

Register access occurs often in time critical places like interrupt routines. A library routine or external function is not a choice. Whatever the feature is, it has to be built in the language. I don't care if it is related to variables, blocks or files as long as I do not have to put these files in a separate  directory like I do now.

I would like to hear more what would be the options. Then we could make a decision what is the right way to go.

Am Thu, 24 Oct 2013 21:28:45 +0100
schrieb Iain Buclaw <ibuclaw@ubuntu.com>:

> On 24 October 2013 18:49, Johannes Pfau <nospam@example.com> wrote:
> > Am Thu, 24 Oct 2013 14:04:44 +0100
> > schrieb Iain Buclaw <ibuclaw@ubuntu.com>:
> >
> >> On 24 October 2013 12:10, John Colvin <john.loughran.colvin@gmail.com> wrote:
> >> > On Thursday, 24 October 2013 at 09:43:51 UTC, Iain Buclaw wrote:
> >> >>>>
> >> >>>> 'shared' guarantees that all reads and writes specified in source code happen in the exact order specified with no omissions
> >
> > Does this include writes to non-shared data? For example:
> > ------------------------------------
> > shared int x;
> > int y;
> >
> > void func()
> > {
> >     x = 0;
> >     y = 3;  //Can the compiler move this assignment?
> >     x = 1;
> > }
> > ------------------------------------
> >
> 
> Yes, reordering may occur so long as the compiler does not change behaviour in respect to the programs sequential points in the application.  (Your example, for instance, can not possibly be re-ordered).
> 
> It is also worth noting while you may have guarantee of this, it does not mean that you can go using __thread data without memory barriers. (For instance, if you have an asynchronous signal handler, it may alter the __thread'ed data at any point in the sequential program).
> 
> 
> 
> > So there's no additional overhead (in code / instructions emitted)
> > when using shared instead of volatile in code like this? And this
> > is valid code with shared (assuming reading/assigning to x is
> > atomic)? ------------------------------------
> > volatile bool x = false;
> >
> > void waitForX()
> > {
> >     while(!x){}
> > }
> >
> > __interrupt(X) void x_interrupt()
> > {
> >     x = true;
> > }
> > ------------------------------------
> 
> That is correct. :o)
> 
> 
> Regards

Sounds good. Now this should be the standard defined behaviour for all compilers. But I guess it'll take some more time till the shared design is really finalized.