On 7/15/2014 9:25 AM, Johannes Pfau wrote:
> DIP62 describes how to solve this problem and make embedded programming
> a first-class citizen in D:
> http://wiki.dlang.org/DIP62

This is a particularly well-written DIP, thank you and the other contributors to it.


> * Although it's explained in the DIP in detail, I must mention this
>    here again: We need _first-class_ support for embedded programming in
>    D. Workarounds might be available (peek/poke, inline asm) but it will
>    be obvious to most C programmers that C has got the better solution
>    with the volatile qualifier. D enthusiasts might be happy if they can
>    use workarounds to make D work on microcontrollers. But if we want to
>    appeal to outsiders we can't confront them with obvious workarounds
>    and hacks as the first thing they see in D. And if they think embedded
>    programming is a second class citizen in D, why should they use it
>    at all?

1. Volatile has caused a major increase in the complexity of the C++ type system - a complexity far out of proportion to the value it provides. It would have a similar complex and pervasive effect on the D type system.

2. You mention peek/poke here, but not in the DIP. [Background: I've designed and built embedded systems, and written the software for them.] There aren't that many memory-mapped I/O registers. They just aren't worth bending the entire language around them. Peek/poke work just fine:

1. they can be implemented as compiler intrinsics so there is no efficiency cost for using them

2. they make it clear in user code when memory-mapped I/O is accessed

3. if you really hate peek/poke calls appearing in the code, you can use UFCS to make them look like variables

4. they are simple to explain and understand, they do not introduce a numbing complexity to the type system

5. peek/poke will not add to the confusion about the difference between volatile and shared. No sane person would be tempted to use peek/poke to implement concurrency. The semantics of volatile in other languages won't confuse things for D peek/poke.


The one thing peek/poke doesn't offer is transitivity. C/C++ don't offer volatile transitivity either. I'm not at all sure that anyone builds a data structure in memory-mapped registers, so I'm not convinced this is a need.

Am Tue, 15 Jul 2014 12:48:21 -0700
schrieb Walter Bright <newshound2@digitalmars.com>:

> On 7/15/2014 9:25 AM, Johannes Pfau wrote:
> > DIP62 describes how to solve this problem and make embedded programming a first-class citizen in D: http://wiki.dlang.org/DIP62
> 
> This is a particularly well-written DIP, thank you and the other contributors to it.
> 
> 
> > * Although it's explained in the DIP in detail, I must mention this
> >    here again: We need _first-class_ support for embedded
> > programming in D. Workarounds might be available (peek/poke, inline
> > asm) but it will be obvious to most C programmers that C has got
> > the better solution with the volatile qualifier. D enthusiasts
> > might be happy if they can use workarounds to make D work on
> > microcontrollers. But if we want to appeal to outsiders we can't
> > confront them with obvious workarounds and hacks as the first thing
> > they see in D. And if they think embedded programming is a second
> > class citizen in D, why should they use it at all?
> 
> 1. Volatile has caused a major increase in the complexity of the C++
> type system
> - a complexity far out of proportion to the value it provides. It
> would have a similar complex and pervasive effect on the D type
> system.

Well as described in the DIP it works just like shared from an implementation point of view, so I doesn't add much complexity in the compiler / type system.

> 
> 2. You mention peek/poke here, but not in the DIP. [Background: I've designed and built embedded systems, and written the software for them.] There aren't that many memory-mapped I/O registers. They just aren't worth bending the entire language around them. Peek/poke work just fine:

Indeed there's nothing about peek/poke in DIP62, I thought I added that. The main point is that we can not leak pointers to volatile memory to users and pretend it's normal memory. This will lead to many problems:
-------------------------------------------------------------
//In runtime:
enum int* SOME_REG = 0xFFFF;

//In user code
peek(SOME_REG);
poke(SOME_REG);
*SOME_REG = 1; //Oops, forgot poke! This is not acceptable
-------------------------------------------------------------
This is like saying we don't need the shared qualifier as people should use atomicOp to access the data.

This is actually the crucial point about this DIP. If we can make this work without a type qualifier than that's fine as well.


The only other solution to this is a wrapper as described by Artur Skawina. (Can basically wrap around peek/poke): http://dpaste.dzfl.pl/6f8ca6d7e7b8

But the problem there is that it produces quite some overhead right
now. We'd need to enforce at least these points:
* @property value must be inlined, there should not be a 'normal'
  function at all
* value must always be treated as in release mode. For example DMD adds
  an assert(this) into that function which is not acceptable
* There should be no TypeInfo for Volatile!T
* There should be no initializer data for Volatile!T
* WE should be careful with debug info for Volatile!T
* We should not emit postblits or similar stuff

In the end the struct basically has to 'vanish' from the object file and '@property value' needs to be substituted into the correct places.

> 
> 1. they can be implemented as compiler intrinsics so there is no efficiency cost for using them
> 
> 2. they make it clear in user code when memory-mapped I/O is accessed

Way more dangerous and important is if you forget peek/poke.

> 
> 3. if you really hate peek/poke calls appearing in the code, you can use UFCS to make them look like variables

But you cant do REGISTER.peek() |= 0b1;

> 
> 4. they are simple to explain and understand, they do not introduce a numbing complexity to the type system
> 

Sorry, but I don't buy any of these complexity arguments. A few months
ago you suggested namespaces for D - in addition to modules - which
would have lead to a complexity disaster.
This just shows the priorities of the project leads:
Desktop apps matter, we add @nogc, c++ namespaces, shared, immutable,
const, -cov, ...
but for embedded systems we won't even add one qualifier.

> 5. peek/poke will not add to the confusion about the difference between volatile and shared. No sane person would be tempted to use peek/poke to implement concurrency. The semantics of volatile in other languages won't confuse things for D peek/poke.

actually peek/poke are not that different from atomicOp from a syntax perspective ;-) You'll still have to educate people about the difference.

> 
> The one thing peek/poke doesn't offer is transitivity. C/C++ don't offer volatile transitivity either. I'm not at all sure that anyone builds a data structure in memory-mapped registers, so I'm not convinced this is a need.

Well such data structures are sometimes used in interrupts, but not that often.

Johannes Pfau:

> Well as described in the DIP it works just like shared from an
> implementation point of view, so I doesn't add much complexity in the compiler / type system.

I am reading blogs about compiler bugs, and I see that the implementation of volatile is the buggiest part of GCC/Clang (and the Intel and Microsoft compilers). Despite numerous bug fixes, it's still a stubbornly buggy part. So it can't be as simple to implement correctly as you say. volatile fights against the optimization stages all the time. And I recognize that Walter has a significant experience on this topic, perhaps higher than yours.


> This just shows the priorities of the project leads:
> Desktop apps matter, we add @nogc, c++ namespaces, shared, immutable,
> const, -cov, ...
> but for embedded systems we won't even add one qualifier.

Having priorities is not something to be ashamed of. D has a GC, works only on 32+ bits, and several of its features are not meant for embedded systems. Even if you can restrict it for such restricted memory and CPU usages, D is not designed primarily for them.

Bye,
bearophile

Johannes Pfau:

> //In runtime:
> enum int* SOME_REG = 0xFFFF;
>
> //In user code
> peek(SOME_REG);
> poke(SOME_REG);
> *SOME_REG = 1; //Oops, forgot poke! This is not acceptable

Perhaps this syntax:

volatile enum int* SOME_REG = 0xFFFF;

Could turn this in a syntax error (only peek/poke are allowed to write and read from this address):

*SOME_REG = 1;

Bye,
bearophile

On 15 July 2014 22:07, bearophile via Digitalmars-d <digitalmars-d@puremagic.com> wrote:
> Johannes Pfau:
>
>
>> Well as described in the DIP it works just like shared from an implementation point of view, so I doesn't add much complexity in the compiler / type system.
>
>
> I am reading blogs about compiler bugs, and I see that the implementation of volatile is the buggiest part of GCC/Clang (and the Intel and Microsoft compilers). Despite numerous bug fixes, it's still a stubbornly buggy part. So it can't be as simple to implement correctly as you say. volatile fights against the optimization stages all the time. And I recognize that Walter has a significant experience on this topic, perhaps higher than yours.
>

The use of volatile can be buggy in C because there is no other safe way to do inter-thread communication.


>
>> This just shows the priorities of the project leads:
>> Desktop apps matter, we add @nogc, c++ namespaces, shared, immutable,
>> const, -cov, ...
>> but for embedded systems we won't even add one qualifier.
>
>
> Having priorities is not something to be ashamed of. D has a GC, works only on 32+ bits, and several of its features are not meant for embedded systems. Even if you can restrict it for such restricted memory and CPU usages, D is not designed primarily for them.
>

So your saying that embedded systems don't matter? Shame on you. :o)

On 15 July 2014 22:17, Iain Buclaw <ibuclaw@gdcproject.org> wrote:
> On 15 July 2014 22:07, bearophile via Digitalmars-d <digitalmars-d@puremagic.com> wrote:
>> Johannes Pfau:
>>
>>
>>> Well as described in the DIP it works just like shared from an implementation point of view, so I doesn't add much complexity in the compiler / type system.
>>
>>
>> I am reading blogs about compiler bugs, and I see that the implementation of volatile is the buggiest part of GCC/Clang (and the Intel and Microsoft compilers). Despite numerous bug fixes, it's still a stubbornly buggy part. So it can't be as simple to implement correctly as you say. volatile fights against the optimization stages all the time. And I recognize that Walter has a significant experience on this topic, perhaps higher than yours.
>>
>
> The use of volatile can be buggy in C because there is no other safe way to do inter-thread communication.
>

I say safe in the lightest possible terms.  Volatile is in no way @safe.

Am Tue, 15 Jul 2014 21:07:21 +0000
schrieb "bearophile" <bearophileHUGS@lycos.com>:

> Johannes Pfau:
> 
> > Well as described in the DIP it works just like shared from an implementation point of view, so I doesn't add much complexity in the compiler / type system.
> 
> I am reading blogs about compiler bugs, and I see that the implementation of volatile is the buggiest part of GCC/Clang (and the Intel and Microsoft compilers). Despite numerous bug fixes, it's still a stubbornly buggy part. So it can't be as simple to implement correctly as you say. volatile fights against the optimization stages all the time. And I recognize that Walter has a significant experience on this topic, perhaps higher than yours.
> 

And how do you think peek/poke are easier to implement in this regard? You still have to be very careful with inlining, instruction scheduling, etc.

The main benefit of a volatile type qualifier is that it's similar to
the C volatile qualifier, so all these bugs have already been fixed.
GDC/LDC could probably just map to the volatile C qualifier.
If we invent some homebrew solution we can start at zero with bug
fixing.

Also what does buggy mean if C doesn't have a clear specification about volatile?

> 
> > This just shows the priorities of the project leads:
> > Desktop apps matter, we add @nogc, c++ namespaces, shared,
> > immutable,
> > const, -cov, ...
> > but for embedded systems we won't even add one qualifier.
> 
> Having priorities is not something to be ashamed of. D has a GC, works only on 32+ bits, and several of its features are not meant for embedded systems. Even if you can restrict it for such restricted memory and CPU usages, D is not designed primarily for them.

Well as long as those priorities are clearly communicated. If you tell
me 'we don't care about embedded programming' then I'll shut up and move
back to C.
But 'D is a systems programming language for low level tasks' and 'we
don't want to introduce a type qualifier for low level programming, but
@nogc is just fine' don't go together. This leaves all contributors and
devs hoping to see D on embedded systems in uncertainty.

Am Tue, 15 Jul 2014 21:16:20 +0000
schrieb "bearophile" <bearophileHUGS@lycos.com>:

> Johannes Pfau:
> 
> > //In runtime:
> > enum int* SOME_REG = 0xFFFF;
> >
> > //In user code
> > peek(SOME_REG);
> > poke(SOME_REG);
> > *SOME_REG = 1; //Oops, forgot poke! This is not acceptable
> 
> Perhaps this syntax:
> 
> volatile enum int* SOME_REG = 0xFFFF;
> 
> Could turn this in a syntax error (only peek/poke are allowed to write and read from this address):
> 
> *SOME_REG = 1;
> 
> Bye,
> bearophile

You just reinvented this DIP ;-)

Once you add a volatile type qualifier, there's no need to use peek/poke. The compiler can then easily detect reads/writes to these variables and do the right thing.

The DIP may seem complicated cause it considers overloading, transitivity and similar stuff. In practice almost nobody will use these features. But if I had not specified them Walters first question would have been "How does it interact with overloading" ;-)

Iain Buclaw:

> The use of volatile can be buggy in C because there is no other safe way to do inter-thread communication.

C11 offers means much better than volatile for multi thread programming:
http://en.wikipedia.org/wiki/C11_%28C_standard_revision%29#Changes_from_C99


> So your saying that embedded systems don't matter?

I was explaining that D design has priorities different from the ones for embedded systems.

And I'd like D to be fitter for low level coding, two hours ago I have written this:
http://forum.dlang.org/post/ftvdiqvjqgmsardoyksp@forum.dlang.org

Bye,
bearophile

On 7/15/2014 1:43 PM, Johannes Pfau wrote:
>> 1. Volatile has caused a major increase in the complexity of the C++
>> type system
>> - a complexity far out of proportion to the value it provides. It
>> would have a similar complex and pervasive effect on the D type
>> system.
>
> Well as described in the DIP it works just like shared from an
> implementation point of view, so I doesn't add much complexity in the
> compiler / type system.

It's not so easy. All the interactions between volatile and other modifiers have to be carefully designed, documented, and implemented.


> Indeed there's nothing about peek/poke in DIP62, I thought I added that.
> The main point is that we can not leak pointers to volatile memory to
> users and pretend it's normal memory. This will lead to many problems:

I have a hard time seeing that MMIO is used so much in an embedded system that this will be a significant problem. And if it is, you can build a wrapper type and control access.


> This is like saying we don't need the shared qualifier as people
> should use atomicOp to access the data.

Shared use, however, is pervasive in concurrent code.


> But the problem there is that it produces quite some overhead right
> now.

Are you sure? The compiler is pretty good at inlining trivial functions.


>> 3. if you really hate peek/poke calls appearing in the code, you can
>> use UFCS to make them look like variables
> But you cant do REGISTER.peek() |= 0b1;

As you mentioned in the DIP, read-modify-write operations should be split up, as their semantics are murky.


>> 4. they are simple to explain and understand, they do not introduce a
>> numbing complexity to the type system
> Sorry, but I don't buy any of these complexity arguments.

No need to be sorry. Your opinion is not invalid.


> A few months
> ago you suggested namespaces for D - in addition to modules - which
> would have lead to a complexity disaster.

C++ namespaces are now implemented in D, and will they lead to complexity disaster? Certainly not in the implementation. For users? I don't think so, but we'll see.

> This just shows the priorities of the project leads:
> Desktop apps matter, we add @nogc, c++ namespaces, shared, immutable,
> const, -cov, ...
> but for embedded systems we won't even add one qualifier.

I don't think this is the right argument. It isn't about embedded systems should be acknowledged with a qualifier. I see it as about how often does MMIO logic appear in an actual embedded program? In the embedded systems I've written, it only appears in a handful of places. It is not pervasive.

BTW, immutable data in D is designed to be placed in ROM for embedded systems. So there's already a qualifier :-)


>> 5. peek/poke will not add to the confusion about the difference
>> between volatile and shared. No sane person would be tempted to use
>> peek/poke to implement concurrency. The semantics of volatile in
>> other languages won't confuse things for D peek/poke.
>
> actually peek/poke are not that different from atomicOp from a
> syntax perspective ;-)

That's correct.


> You'll still have to educate people about the difference.

Peek/poke has a 40 year history of being used for MMIO, and I've never heard of it being used for concurrency. I don't think there's endemic confusion, quite unlike volatile.