http://bugzilla.gdcproject.org/show_bug.cgi?id=126

--- Comment #1 from Iain Buclaw <ibuclaw@gdcproject.org> ---
peek() and poke() intrinsics that are guaranteed to be volatile accessors are
also on the table.

-- 
You are receiving this mail because:
You are watching all bug changes.

http://bugzilla.gdcproject.org/show_bug.cgi?id=126

Johannes Pfau <johannespfau@gmail.com> changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
                 CC|                            |johannespfau@gmail.com

--- Comment #2 from Johannes Pfau <johannespfau@gmail.com> ---
Has this been discussed at dconf? I actually began writing a DIP for volatile but it's not ready yet. However, I feared that Walter/Andrei would oppose it, but I think this is a big problem for embedded D, probably big enough so embedded D will never take off.

1) we should have a portable volatile attribute not only gdc specific solutions
or we're worse than C
2) Peek/poke are quite complicated to use on simple embedded systems where you
only access registers. Nobody's going to use D if you have to do this to access
registers:
--------------
auto x = peek!uint(address);
x |= 0b1;
poke(address, x);
--------------

if you can just do
--------------
x |= 0b1;
--------------
in C.

(We have to remember that most embedded programming on small (e.g. 8bit) processors works just fine with C. D can only provide very few benifits (for example, ctfe is great, but there's no big usecase on simple programs which only read some input (e.g temparature) and do some IO communication. Templates are out cause executable size is very important,...). But what you do on these systems all the time is accessing volatile memory/registers. If that's more complicated in D than in C, or not portable, then C is a much better language for embedded programming)

@volatile in GDC only is also a problem, because it requires significant changes to the frontend and this is bad for a shared frontend. Two examples which need extended frontend support:

------------------
struct A
{
    int x;
    int readX() {return x;}
    int correctRead() {return x;} volatile
}

volatile(A)* a;
auto x = a.readX(); //Should not compile! Compiler doesn't know that reading x
in readX needs to be volatile. We need rules as for const: Only methods marked
as volatile should work on volatile structs

auto x = a.correctRead(); //Should work. Also works on non volatile instances
(just like const methods / mutable instances)
------------------

------------------
volatile int x;

void func()
{
   auto x2 = x; //x2 is not volatile anymore cause we made and independent copy
   //volatile should be transitive, so we have to be careful if a type has
references. Again see the anology to const.
}
------------------

transitivity of volatile is used in few cases only, but it's necessary. One example is DMA:
------------------
volatile uint** DMASRC, DMADEST;
*DMASRC = address;
doDMA(); //Overwrites *DMADEST
------------------

or, more often in real world, interrupts;
------------------
interrupt()
{
    if(dice)
        data = null;
    else
        *data = 42;
}

volatile(uint*) data;

func doSomething()
{
    uint data1;
    data = &data1;
    //interrupt can modify the pointer itself or the value
}
------------------
(for example you can google how often newbies struggle in C if they have to do
some volatile(volatile uint)* ... stuff)

-- 
You are receiving this mail because:
You are watching all bug changes.

http://bugzilla.gdcproject.org/show_bug.cgi?id=126

--- Comment #3 from Iain Buclaw <ibuclaw@gdcproject.org> ---
Yes this has been raised at DConf.

Michael gave a good talk on Bare Metal programming, and mentioned that he's relying on the use of shared currently meaning volatile to do a lot of low level work in D, but wasn't sure if it was the correct approach.

I am quite open to solutions, other than I don't think 'volatile' as a keyword would be something reintroduced to the language am afaid.

I can certainly point Walter and Michael in the direction of your DIP if that helps. :)

-- 
You are receiving this mail because:
You are watching all bug changes.

http://bugzilla.gdcproject.org/show_bug.cgi?id=126

--- Comment #4 from Johannes Pfau <johannespfau@gmail.com> ---
Yes, I was eagerly waiting for that talk. Unfortunately the most interesting talks are the last ones on each day and I couldn't watch these live because of timezone stuff. I'll watch these as soon as recordings are available though, especially your debugging talk.


Unfortunately I won't have the DIP finished during dconf, some difficult cases are not discussed yet and the rationale is missing completely and the DIP is not very useful without a rationale. But if you read this during dconf there's one thing you could ask Andrei (or everybody):

Quoting Bartosz Milewski from http://stackoverflow.com/a/8833218/471401
> I's still struggling with the use of volatile with atomics: "volatile is completely unnecessary when used with std::atomic". What about the loop optimization of while(x.load(memory_order_relaxed)) ; => bool tmp = x.load(memory_order_relaxed); while(tmp) ; The standard is wishy washy about this and Hans turns into a diplomat when asked this question directly ;-)

So translating this to D:
------------
import core.atomic;

shared bool x;

void main()
{
    while(atomicLoad(x))
    {
    }
}

void main_optimized()
{
    bool tmp = atomicLoad(x);
    while(tmp)
    {
    }
}
------------
http://dpaste.dzfl.pl/b7ecc943ccab

Is optimizing main into main_optimized legal or not? The background of this question is that merging reads is usually legal for atomic loads from non-volatile memory (as-if rule). But in this case with a loop this is not useful behavior. The question then is where to draw the line.

-- 
You are receiving this mail because:
You are watching all bug changes.

http://bugzilla.gdcproject.org/show_bug.cgi?id=126

--- Comment #5 from Johannes Pfau <johannespfau@gmail.com> ---
Oh, and back to topic:

> I am quite open to solutions, other than I don't think 'volatile' as a keyword > would be something reintroduced to the language am afaid.

Yes, that's what I fear as well. But thinking about it volatility is a property of the memory location and that best maps to a type qualifier. (It's not exactly the same thing, but we also have this conflation for immutable type / read only memory and in practice it should work fine).

That's one reason why peek/poke are dangerous, just like the old D1 volatile statements: If a volatile memory area is typed as a normal pointer you can still pass it to functions which access it in 'non-volatile' ways.


OTOH one of the biggest problems with volatile in C is that it's not properly standardized. If we invent GDC/LDC only solutions these will likely be slightly different causing the same mess as in C/C++ (or even worse, if we don't have a standard at all). So this is something which would really benefit from being part of the D standard. Maybe there's some chance we can introduce a new type qualifier, if not we'll have to do the next best thing, whatever that may be.

-- 
You are receiving this mail because:
You are watching all bug changes.

http://bugzilla.gdcproject.org/show_bug.cgi?id=126

--- Comment #6 from Iain Buclaw <ibuclaw@gdcproject.org> ---
(In reply to Johannes Pfau from comment #5)
> Oh, and back to topic:
> 
> > I am quite open to solutions, other than I don't think 'volatile' as a keyword > would be something reintroduced to the language am afaid.
> 
> Yes, that's what I fear as well. But thinking about it volatility is a property of the memory location and that best maps to a type qualifier. (It's not exactly the same thing, but we also have this conflation for immutable type / read only memory and in practice it should work fine).
> 

If we are going for property, maybe @volatile might end up standing. ;)

-- 
You are receiving this mail because:
You are watching all bug changes.

http://bugzilla.gdcproject.org/show_bug.cgi?id=126

Mike <slavo5150@yahoo.com> changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
                 CC|                            |slavo5150@yahoo.com

--- Comment #7 from Mike <slavo5150@yahoo.com> ---
As I understand it, there are 2 issues to be addressed by 'volatile' or some other solution.

1.  Ensure a memory location is always read/written and not cached in registes 2.  Ensure instructions that access 'volatile' memory are not reordered by the compiler.

I believe 'shared' satisfies (1), but not (2).  A 'volatile' memory location is one which the executing thread does not have complete control of, and can therefore be changed by some other entity external to the executing thread. Therefore is it's contents are volatile.

I think of 'shared' similarly.  The memory location is shared by the executing thread with other external entities:  other threads, other processors, a sensor, or other peripheral that may read or write to that memory location outside of the context of the executing thread.  What 'shared' doesn't satisfy is (2).

peek() and poke() intrinsics would satisfy both (1) and (2).

I see at least 3 solutions:
1.  Add volatile semantics with either a keyword or an attribute requiring
compilers to address both (1) and (2)
2.  Use 'shared' to address requirement (1) and add some other "don't reorder"
feature to the front-end
3.  Add peek() and poke() intrinsics.

I prefer (2), (3), then (1).  I prefer (2) simply because it is more granular
and both 'shared' and a "don't reorder" feature could be used for other
purposes outside of my immediate need which is peripheral memory mapped IO.

-- 
You are receiving this mail because:
You are watching all bug changes.

http://bugzilla.gdcproject.org/show_bug.cgi?id=126

--- Comment #8 from Mike <slavo5150@yahoo.com> ---
But regardless of the solution, it would be nice if it were a front-end feature implemented by all compilers.

-- 
You are receiving this mail because:
You are watching all bug changes.

http://bugzilla.gdcproject.org/show_bug.cgi?id=126

--- Comment #9 from Johannes Pfau <johannespfau@gmail.com> ---
@Mike: AFAIK shared has some additional requirements which are not necessary for volatile:

1.) Atomic access: Accessing shared variables must always use atomic access
(This is not yet enforced, but see
https://github.com/D-Programming-Language/dmd/pull/3070)
2.) Accessing shared variables should also prevent reordering by the CPU (I'm
not 100% sure about this though), not only by the compiler, as volatile does.

I guess in the end shared variables will always be accessed via some function (atomicOp, synchronized statement + casting, ...).

Also shared variables could ignore requirement 1. At least in C++/11 multiple accesses to atomics can be merged (http://stackoverflow.com/a/6681505/471401).

Regarding peek/poke functions: Don't you think that's too cumbersome?
I also think shared + peek/poke has the drawback that you can still
accidentally access it like a normal shared variable, without the peek/poke
guarantees.

I'd prefer introducing a volatile type qualifier that enforces only one property: The compiler does not optimize access but we don't guarantee anything about the actual execution on hardware.

Volatile memory also sometimes has properties which don't match traditional memory: I think I've seen architectures where reading a register might return a completely different value than the value written in a previous write. Most functions dealing with 'standard' memory will produce incorrect results if they operate on such memory. By using an additional 'volatile' type qualifier we can make sure that only functions which explicitly accept volatile memory can be used with volatile variables.


BTW: I finally finished the volatile DIP, see http://wiki.dlang.org/DIP62 . It'd be great to get some early feedback from you and Iain, and feel free to edit the DIP :-)

-- 
You are receiving this mail because:
You are watching all bug changes.

http://bugzilla.gdcproject.org/show_bug.cgi?id=126

--- Comment #10 from Iain Buclaw <ibuclaw@gdcproject.org> ---
(In reply to Johannes Pfau from comment #9)
> @Mike: AFAIK shared has some additional requirements which are not necessary for volatile:
> 
> 1.) Atomic access: Accessing shared variables must always use atomic access
> (This is not yet enforced, but see
> https://github.com/D-Programming-Language/dmd/pull/3070)
> 2.) Accessing shared variables should also prevent reordering by the CPU
> (I'm not 100% sure about this though), not only by the compiler, as volatile
> does.
> 
> I guess in the end shared variables will always be accessed via some function (atomicOp, synchronized statement + casting, ...).
> 
> Also shared variables could ignore requirement 1. At least in C++/11 multiple accesses to atomics can be merged (http://stackoverflow.com/a/6681505/471401).
> 

That is a re-order by the compiler, which is (2) - there is still a guarantee that the var isn't cached in some way.

Mike, this is the most common example of reordering I was mentioning about at the conference.

---
shared int myshared;

myshared = 0;
foreach (i; 0 .. 12)
  myshared += 1;

---

Under volatile semantics, the loop would be unrolled and ordering kept.

mov    $0, _D4test4globOi(%rip)    ; myshared = 0

mov    _D4test4globOi(%rip), %eax  ; myshared += 1
add    $1, %eax
mov    %eax, _D4test4globOi(%rip)
; And so on...

---

In future, the compiler would memoize the loop and go straight for the assignment.

mov    $12, _D4test4globi(%rip)

---


Pull #3070 is interesting, as it would disallow most cases where the compiler may get away with this kind of reordering behaviour.

The current implementation of core.atomic does mean that all atomicOps are fully sequenced.  As soon as we switch over from the old-style __sync intrinsics to C++x11 __atomic intrinsics, then this may change (MemoryOrder.raw will be adhered to).

mov    $0, _D4test4globOi    ; glob = 0
mov    $12, %eax             ; count = 12
xor    %edx, %edx
jmp    .L2
.L1
mov    _D4test4globOi, %edx  ; atomicOp!"+="(glob, 1)
.L2
lock add $1, (%edx)          ;
subl   $1, %eax              ; count--
jne    .L1

---

-- 
You are receiving this mail because:
You are watching all bug changes.