On Tuesday, 15 July 2014 at 16:27:53 UTC, Johannes Pfau wrote:
> You may have seen Mike's talk about D on embedded systems and were
> surprised about Iain's comment that using 'shared' as a replacement for
> 'volatile' is actually only possible because of a bug in gdc.
>
> DIP62 describes how to solve this problem and make embedded programming
> a first-class citizen in D:
> http://wiki.dlang.org/DIP62

Well, this was posted just as I went to sleep for the night (I'm in Asia), so I wasn't able to get my word in until the end.  Sorry!

DIP62 is the exemplary DIP:  well researched and thoroughly specified.  I was actually in favor of peek/poke intrinsics until the DIP62 and its citations took me to school.

I still believe that the type qualifier is the right tool for the job, because despite having peek/poke intrinsics, users will still navigate towards making a volatile type to wrap peek/poke, as has already been proposed [1] Volatile!(T), [2] struct VolatilePointerToUint, etc...  It actually justifies DIP62, IMO.

After reading through this thread, the only argument against DIP62 that really made much sense was the cost/benefit tradeoff in the implementation.  I can't speak to the complexity of the implementation, but I do concede that the use of volatile will only be used in the lowest layer of hardware abstraction (although it will be used quite often there), and the workarounds are not burdensome.  The type qualifier would avoid such nonsense.

Anyway, I can see where this is going[3], so I guess I'll have to patch my airplane[4] with a template to make it fly :(

[1] http://forum.dlang.org/post/eznqslwpnxfnwlgzlsll@forum.dlang.org
[2] http://forum.dlang.org/post/lq4hqi$1j77$1@digitalmars.com
[3] https://github.com/D-Programming-Language/dmd/pull/3773
[4] http://www.youtube.com/watch?feature=player_detailpage&v=WKRRgcEk0wg#t=65

Mike

On 7/15/14, 6:25 PM, Walter Bright via Digitalmars-d wrote:
> I've been jawboning about peek/poke for years - finally decided to implement it.
>
>    https://issues.dlang.org/show_bug.cgi?id=13138

I don't have any skin in this particular discussion, but it's worth pointing out here that while it's cute that builtin's can be added, when they're not part of the language definition, they're not really part of a portable solution.

On Tuesday, 15 July 2014 at 19:48:23 UTC, Walter Bright wrote:

> The one thing peek/poke doesn't offer is transitivity. C/C++ don't offer volatile transitivity either. I'm not at all sure that anyone builds a data structure in memory-mapped registers, so I'm not convinced this is a need.

Here's an example from the STM32F4 ARM Cortex-M peripheral library:

#define __IO volatile

/**
  * @brief DMA Controller
  */

typedef struct
{
  __IO uint32_t CR;     /*!< DMA stream x configuration register      */
  __IO uint32_t NDTR;   /*!< DMA stream x number of data register     */
  __IO uint32_t PAR;    /*!< DMA stream x peripheral address register */
  __IO uint32_t M0AR;   /*!< DMA stream x memory 0 address register   */
  __IO uint32_t M1AR;   /*!< DMA stream x memory 1 address register   */
  __IO uint32_t FCR;    /*!< DMA stream x FIFO control register       */
} DMA_Stream_TypeDef;

typedef struct
{
  __IO uint32_t LISR;   /*!< DMA low interrupt status register,      Address offset: 0x00 */
  __IO uint32_t HISR;   /*!< DMA high interrupt status register,     Address offset: 0x04 */
  __IO uint32_t LIFCR;  /*!< DMA low interrupt flag clear register,  Address offset: 0x08 */
  __IO uint32_t HIFCR;  /*!< DMA high interrupt flag clear register, Address offset: 0x0C */
} DMA_TypeDef;

There are several hundreds lines of this stuff in the library.  It is used to model register banks.  Each register bank roughly corresponds to one of the embedded peripherals.

Mike

On Wednesday, 16 July 2014 at 01:25:05 UTC, Walter Bright wrote:
> I've been jawboning about peek/poke for years - finally decided to implement it.
>
>   https://issues.dlang.org/show_bug.cgi?id=13138

If this is indeed the last straw, please mark DIP20 as accepted/implemented and DIP62 as rejected.  I have neither the authority nor the stomach for it.

Mike

DIP20:  http://wiki.dlang.org/DIP20
DIP62:  http://wiki.dlang.org/DIP62

Am Tue, 15 Jul 2014 14:28:23 -0700
schrieb Walter Bright <newshound2@digitalmars.com>:

> On 7/15/2014 1:43 PM, Johannes Pfau wrote:
> > But the problem there is that it produces quite some overhead right now.
> 
> Are you sure? The compiler is pretty good at inlining trivial functions.

Yes, I actually tried tested that simple implementation with gdc. With aggressive optimizations the code inlines just fine and it'll generate the same asm as an access to a volatile variable in C. But it's not space-effecient: Even if it inlines, the function is still in the object file as a distinct function. There's TypeInfo and there's the assert(this), there's an initializer symbol and I guess there could be copy constructors as well. I expect many bug reports/fixes till we get a Volatile!T wrapper to be really space-efficient.


I think if we decide that Volatile!T is the solution I'd probably hack an attribute @TypeWrapper for gdc which simply recognized Volatile!T and does the right thing. The 'standard D' option requires @forceinline @noTypeinfo @noInitializer @noDebug and lots of boilerplate (all opAssign methods have to be implemented as we don't rewrite RMW operations into property get/set which also affects alias this to a property).

> 
> >> 3. if you really hate peek/poke calls appearing in the code, you can use UFCS to make them look like variables
> > But you cant do REGISTER.peek() |= 0b1;
> 
> As you mentioned in the DIP, read-modify-write operations should be split up, as their semantics are murky.
> 
> 

But not necessarily in user code. REGISTER |= is quite common in C and it's perfectly safe. If people will have to split this up into peek |= poke that's a step backwards from C.


> > A few months
> > ago you suggested namespaces for D - in addition to modules - which
> > would have lead to a complexity disaster.
> 
> C++ namespaces are now implemented in D, and will they lead to complexity disaster? Certainly not in the implementation. For users? I don't think so, but we'll see.

You were initially in favor of an option which would have allowed to use namespaces in D code as well (namespace A {}). The current implementation is mostly limited to C++ so that's better, but my point was that nobody cared about complexity back then.

> 
> > This just shows the priorities of the project leads:
> > Desktop apps matter, we add @nogc, c++ namespaces, shared,
> > immutable, const, -cov, ...
> > but for embedded systems we won't even add one qualifier.
> 
> I don't think this is the right argument. It isn't about embedded systems should be acknowledged with a qualifier. I see it as about how often does MMIO logic appear in an actual embedded program? In the embedded systems I've written, it only appears in a handful of places. It is not pervasive.

But it's not only about frequency, it's also about convenience and safety. How often is opDispatch actually used? The synchronized statement? Nevertheless they are there, because someone thought these use cases are important enough.

The perceived importance of a special qualifier for MMIO code is highly
subjective, but I think that most embedded programmers would say it's
necessary. In Andrei's Leverage talk he said (5:57): "Consider this:
Convenience, modeling power, efficiency: [...] You gonna have to have a
large language. This is a fact of life. [...]". We have
weak pure/ strong pure & immutable for functional programming, A
complete set of OO primitives, builtin unit tests, code coverage, ... I
think it's a correct argument to say that you think embedded system
programming is not important enough for the additional complexity
introduced by a new qualifier.

"Johannes Pfau"  wrote in message news:lq51u4$21qb$1@digitalmars.com...

> But it's not only about frequency, it's also about convenience and
> safety. How often is opDispatch actually used? The synchronized
> statement? Nevertheless they are there, because someone thought these
> use cases are important enough.

None of those affect the type system, this does.  If it could be added as a storage class or even a new statement (!) it would be less intrusive.

> The perceived importance of a special qualifier for MMIO code is highly
> subjective, but I think that most embedded programmers would say it's
> necessary. In Andrei's Leverage talk he said (5:57): "Consider this:
> Convenience, modeling power, efficiency: [...] You gonna have to have a
> large language. This is a fact of life. [...]". We have
> weak pure/ strong pure & immutable for functional programming, A
> complete set of OO primitives, builtin unit tests, code coverage, ... I
> think it's a correct argument to say that you think embedded system
> programming is not important enough for the additional complexity
> introduced by a new qualifier.

The problem is not 'this feature is too complex', the problems is that it's more complex than necessary. 

On 7/15/2014 9:29 PM, Brad Roberts via Digitalmars-d wrote:
> I don't have any skin in this particular discussion, but it's worth pointing out
> here that while it's cute that builtin's can be added, when they're not part of
> the language definition, they're not really part of a portable solution.

They're as portable as we want them to be.

On 15 July 2014 22:57, Walter Bright via Digitalmars-d <digitalmars-d@puremagic.com> wrote:
> On 7/15/2014 2:17 PM, Johannes Pfau wrote:
>>
>> Well as long as those priorities are clearly communicated. If you tell me 'we don't care about embedded programming' then I'll shut up and move back to C.
>
>
> Rest assured we definitely do care about embedded systems programming.
>
>
>
>> But 'D is a systems programming language for low level tasks' and 'we don't want to introduce a type qualifier for low level programming, but @nogc is just fine' don't go together. This leaves all contributors and devs hoping to see D on embedded systems in uncertainty.
>
>
> It isn't about whether we want to introduce a type qualifier for low level programming or not. It is about what is the best solution for MMIO for low level programming.
>
> Note that D already supports inp() and outp() as compiler intrinsics, adding
> peek() and poke() will complement them nicely:
>
>   http://dlang.org/phobos/core_bitop.html#.inp

s/D/DMD/

GDC doesn't support it as compiler intrinsics.  Also, isn't inp/outp all x86-specific?

I've never come across a similar thing in other architectures (at
least, of the same name)

On 16 July 2014 07:12, Walter Bright via Digitalmars-d <digitalmars-d@puremagic.com> wrote:
> On 7/15/2014 9:29 PM, Brad Roberts via Digitalmars-d wrote:
>>
>> I don't have any skin in this particular discussion, but it's worth
>> pointing out
>> here that while it's cute that builtin's can be added, when they're not
>> part of
>> the language definition, they're not really part of a portable solution.
>
>
> They're as portable as we want them to be.


I read that as, not portable at all then. :o)

On 7/15/2014 10:15 PM, Johannes Pfau wrote:
> Yes, I actually tried tested that simple implementation with gdc. With
> aggressive optimizations the code inlines just fine and it'll generate
> the same asm as an access to a volatile variable in C. But it's not
> space-effecient: Even if it inlines, the function is still in the
> object file as a distinct function. There's TypeInfo and there's the
> assert(this), there's an initializer symbol and I guess there could be
> copy constructors as well. I expect many bug reports/fixes till we get
> a Volatile!T wrapper to be really space-efficient.

D is intended to have a robust enough feature set in order to implement types as library types rather than builtin, and have them operate as good as a builtin type would. These are more general issues, and are not specific to volatile.


>>>> 3. if you really hate peek/poke calls appearing in the code, you
>>>> can use UFCS to make them look like variables
>>> But you cant do REGISTER.peek() |= 0b1;
>> As you mentioned in the DIP, read-modify-write operations should be
>> split up, as their semantics are murky.
> But not necessarily in user code. REGISTER |= is quite common in C and
> it's perfectly safe.

The point is it is not perfectly safe. There is no definition, and certainly no portable definition, as to how many read/write cycles such an operation entails. That's what I meant, and DIP62 says pretty much said the same thing. Please allow me to quote:

"CISC machines may allow other operations to memory than simple load/store operations. The question is then if a read-modify-write operation like 'x++' should translate to 'r = load x; r++; x = store r' or to a single add instruction 'add x;'. This DIP does not dictate any specific behaviour, but this is recommended: If it is known that instructions operating on memory do not work for memory mapped IO as expected, compilers should default to generating a load/modify/store sequence. Otherwise compilers should generate the same code sequence as for regular variables. It is recommended to provide a compiler switch on CISC architectures to allow the user to choose between these two different behaviours."


> If people will have to split this up into peek |=
> poke that's a step backwards from C.

Support for op= is fairly simple for VolatilePointerToUint to implement, and it can implement them in terms of peek/poke for precise and portable semantics.


>>> A few months
>>> ago you suggested namespaces for D - in addition to modules - which
>>> would have lead to a complexity disaster.
>> C++ namespaces are now implemented in D, and will they lead to
>> complexity disaster? Certainly not in the implementation. For users?
>> I don't think so, but we'll see.
> You were initially in favor of an option which would have allowed to
> use namespaces in D code as well (namespace A {}). The current
> implementation is mostly limited to C++ so that's better, but my point
> was that nobody cared about complexity back then.

I objected to several proposals for namespace support that I regarded as significantly more complex.


>>> This just shows the priorities of the project leads:
>>> Desktop apps matter, we add @nogc, c++ namespaces, shared,
>>> immutable, const, -cov, ...
>>> but for embedded systems we won't even add one qualifier.
>>
>> I don't think this is the right argument. It isn't about embedded
>> systems should be acknowledged with a qualifier. I see it as about
>> how often does MMIO logic appear in an actual embedded program? In
>> the embedded systems I've written, it only appears in a handful of
>> places. It is not pervasive.
>
> But it's not only about frequency, it's also about convenience and
> safety. How often is opDispatch actually used? The synchronized
> statement? Nevertheless they are there, because someone thought these
> use cases are important enough.

The synchronized statement was something I thought would be widely used. It turns out to be largely a failure. opDispatch is actually pretty simple, and I believe it is an "enabling" feature that hasn't, but will, come into its own.

Nevertheless, we should not follow the trap of "we made a mistake on A, so that justifies making another mistake B". Nor should we say "feature X has reasons A,B,C,D, therefore reason D is justification enough for other things." Changes to D must stand/fall on their own merits.


> The perceived importance of a special qualifier for MMIO code is highly
> subjective,

Of course. There is no doubt that both yours and my opinions here are subjective.


> but I think that most embedded programmers would say it's
> necessary.

I do believe that a wrapper type around peek/poke satisfies essentially all your necessary requirements, except for transitivity.


> In Andrei's Leverage talk he said (5:57): "Consider this:
> Convenience, modeling power, efficiency: [...] You gonna have to have a
> large language. This is a fact of life. [...]". We have
> weak pure/ strong pure & immutable for functional programming, A
> complete set of OO primitives, builtin unit tests, code coverage, ... I
> think it's a correct argument to say that you think embedded system
> programming is not important enough for the additional complexity
> introduced by a new qualifier.

That is not what I'm saying at all. Please do not conflate disagreement about the best way to achieve a goal with disagreement about the goal.

I understand that you and others put a lot of work into DIP62. As I said before, it's a very nice example of a well-done DIP.