http://bugzilla.gdcproject.org/show_bug.cgi?id=126

--- Comment #17 from Mike <slavo5150@yahoo.com> ---
I hope we can all agree that volatile semantics are necessary for this kind of programming regardless of whether it is implemented as an asm workaround, compiler intrinsic functions, or a type qualifier.

I also hope we can all agree that neither shared nor shared+atomic, when properly implemented, provides volatile semantics (See Iain's last comment [http://bugzilla.gdcproject.org/show_bug.cgi?id=126#c10] and DIP 4.2.4 - [http://wiki.dlang.org/DIP62#Why_volatile_and_shared_can.27t_be_merged]). Please correct me if I'm wrong.

I think DIP62 4.2.2 justifies why it should be a type qualifier [http://wiki.dlang.org/DIP62#Why_a_type_qualifier]:  because one would never want to access a volatile memory location without volatile semantics.  Using a type qualifier would give the programmer this guarantee but volatile get/set (i.e. peek/poke) intrinsic functions would not.

I have yet to hear an equally strong argument in favor of get/set intrinsics. Until such time, I support DIP62.

-- 
You are receiving this mail because:
You are watching all bug changes.

http://bugzilla.gdcproject.org/show_bug.cgi?id=126

--- Comment #18 from Mike <slavo5150@yahoo.com> ---
I should also add that the inline asm volatileGet/volatileSet workaround is just that:  a workaround.  For the programmer to have to resort to these techniques is an indication that the language is lacking.  If I wanted to use my mmio.d for other architectures (which I actually intend to do), I would have to resort to writing a custom volatileGet/volatileSet pair for each architecture, and that doesn't scale well.

-- 
You are receiving this mail because:
You are watching all bug changes.

http://bugzilla.gdcproject.org/show_bug.cgi?id=126

--- Comment #19 from Mike <slavo5150@yahoo.com> ---
Oh, and about atomic access... I agree that memory-mapped IO is shared, global state, but I also agree that it is wasteful to wrap each and every access in atomic accessors.  A large amount of memory mapped I/O is done during core, peripheral, and board initialization, before any threads or interrupts are even possible.  There is absolutely no need to wrap these in atomic accessors.  I believe the only one capable of making the best decision about when and where to add atomic access is the programmer, not the compiler.

The arguments that favor avoiding atomic access for the sake of performance and codesize are not particularly strong, but nor are they weak.  I agree that memory-mapped IO takes up a small percentage of most programs, but I don't think that's an excuse to be careless and wasteful about efficiency.  These micro-optimizations may not be significant on their own, but when taken in aggregate they can give you a small edge in performance, codesize, battery-life, cost, etc...  I don't want to compromise.  If I have to compromise, D loses its appeal to me and I have less faith in the merits of my own project.  That's very disheartening.  I want to be able to brag about my work and about the usage of D in this domain, not make excuses.

-- 
You are receiving this mail because:
You are watching all bug changes.

http://bugzilla.gdcproject.org/show_bug.cgi?id=126

--- Comment #20 from Iain Buclaw <ibuclaw@gdcproject.org> ---
(In reply to Mike from comment #19)
> Oh, and about atomic access... I agree that memory-mapped IO is shared, global state, but I also agree that it is wasteful to wrap each and every access in atomic accessors.  A large amount of memory mapped I/O is done during core, peripheral, and board initialization, before any threads or interrupts are even possible.  There is absolutely no need to wrap these in atomic accessors.  I believe the only one capable of making the best decision about when and where to add atomic access is the programmer, not the compiler.
>

FYI: This change is in 2.066 out next month.

https://github.com/D-Programming-Language/dmd/commit/7535f3dc86e9e88c4ee609dd98b5305224c3b88a

Having such a restriction in place makes a need for volatile types even more so crucial for any future in embedded development.

It's worth noting that shared has been in a persistent state of slow definition for the last two years, for instance there was a time when it was thought that shared should imply volatile ( https://issues.dlang.org/show_bug.cgi?id=9163#c1 ).  Infact in retrospect, part of the reasoning behind removing volatile was that shared was expected to replace volatile, but come with safer guarantees.

Yet two years on, this viewpoint has changed.  Just from reading discussions of this even in the last months, people are questioning the usefulness of shared as a suitable replacement for the now removed volatile.

The way I see it, we have two camps.  On the one side, there's a real need to have a specialised type for use of hardware access.  On the other, it's trying to bend a type that has been tailored for safe inter-thread communication.  In other words, shared is not just a property of memory, but a property its actions, to which a volatile memory access may or may not uphold.

I certainly do understand where you are coming from when you talk about scaling volatileSet/volatileGet - however it is hard to get that across sometimes when it seems the core developer only see it as "porting a 2 line function between systems".


Recently, I stumbled upon a previously rejected DIP which proposed to keep volatile and why (written by Zor) http://wiki.dlang.org/DIP17

He had some interesting points that I thought be worth examining two years on.

In summary, he explains that the shared type qualifier has been suggested as a solution to the problems volatile tries to solve.  He notes however the following drawbacks:

> It is not implemented in any compiler, so practically using it now is not possible at all.

This is true, atomic types are a relatively new concept in a compiler, but this is has changed over time since C++x11 atomic support has been introduced. Compilers are much better quipped today to handle at least some aspects of what shared tries to solve.

> It does not have any well-defined semantics yet.

I think the change in the upcoming 2.066 release changes this, in that we are now saying shared is clearly defined in both its transitivity (which will improve with library support), and it's thread @safety by disallowing operations that perform more than one operation without the safety of core.atomic memory barriers/fences.

> It will most likely not be portable because it's designed for the x86 memory model.

Again with C++x11 atomics, this has been disproved.  Infact it's probably of no coincidence that the way core.atomic's memory model is designed, it maps pretty much 1:1 to C++x11 atomics.

> If ever implemented, it will result in memory fences and/or atomic operations, which is **not** what volatile memory operations are about. This will severely affect pipelining and performance in general.

This is the number one reason why volatile is needed. And why shared is wholly unsuitable.


Further to his arguments, he also tries to disprove the use of "two-liner" inline assembly to solve the problem, I won't bother reviewing these, the whole point of the matter why volatile is needed is that I am asking you (politely) to not use the inline assembler as an excuse to *not* implement a feature that is rather essential for a systems language.

-- 
You are receiving this mail because:
You are watching all bug changes.

To keep this thread going, I had a quick look at the reference material of the dip and picked some thoughts.

In some languages volatile has a stronger meaning, like guaranteeing an atomic access. In some languages it may not guarantee anything.

In this proposal volatile is only for optimization, not for protection. It does not add any code, it just prevents the optimizer removing some code.

Read-modify-write and multi-word access are not guaranteed. The user should be aware of possible failures and for example avoid longer data types than the word size of the processor.

Volatile is never recommended to use for communicatin between threads. It has been mentioned to be a reasonably good method to exchange data with an interrupt program. It is the only reasonable way to access hardware registers. The only others I have seen are library functions and inline assembly and they are not acceptable.


Walter has been against this and now also Martin. I think there is no use to bring this to the main forum. I understand the point that it is not very good to have something in the language specs that can not be guaranteed.

We just need this to access registers. Dmd is more for desktop and servers. It will never support all the targets gcc can. Could we make this a gdc extension?

While writing this  it just popped to my mind: if volatile is not good, could we reuse the 'system' word? Then it would be clear that this is for accessing system resources and not for application level.

There seems not to be much documentation about system. Tdpl says it may omit some checks and the website says it is quite the same than not having any other attributes. So:
What system means in general?
What it currently means in gdc?
Could we use it instead of volatile?

On 24 June 2014 11:46, Timo Sintonen via D.gnu <d.gnu@puremagic.com> wrote:
>
> While writing this  it just popped to my mind: if volatile is not good, could we reuse the 'system' word? Then it would be clear that this is for accessing system resources and not for application level.
>
> There seems not to be much documentation about system. Tdpl says it may omit
> some checks and the website says it is quite the same than not having any
> other attributes. So:
> What system means in general?

Not @safe.

> What it currently means in gdc?

Not @safe.

> Could we use it instead of volatile?

No.

Am Tue, 24 Jun 2014 10:46:11 +0000
schrieb "Timo Sintonen" <t.sintonen@luukku.com>:

> To keep this thread going, I had a quick look at the reference material of the dip and picked some thoughts.
> 
> In some languages volatile has a stronger meaning, like guaranteeing an atomic access. In some languages it may not guarantee anything.
> 
> In this proposal volatile is only for optimization, not for protection. It does not add any code, it just prevents the optimizer removing some code.
> 
> Read-modify-write and multi-word access are not guaranteed. The user should be aware of possible failures and for example avoid longer data types than the word size of the processor.
> 
> Volatile is never recommended to use for communicatin between threads. It has been mentioned to be a reasonably good method to exchange data with an interrupt program. It is the only reasonable way to access hardware registers. The only others I have seen are library functions and inline assembly and they are not acceptable.
> 
Yes, that's exactly how I envision 'volatile'.

> 
> Walter has been against this and now also Martin. I think there is no use to bring this to the main forum. I understand the point that it is not very good to have something in the language specs that can not be guaranteed.
I think we should at least try to bring this to the main newsgroup,
however I got distracted with other things and I want to extend the DIP
a little (Only rationale stuff, no technical changes). The
newsgroups have been quite busy lately and there've been discussions
about two new DIPs. Neither Andrei nor Walter even posted a response
in these threads so now is probably not a good time to start the
discussion on this DIP.

Volatile loses much of it's use when it's compiler specific. Although we could 'ignore' dmd there's also ldc. (However, as we already have incompatible inline asm compared to dmd and ldc it's probably not that bad)

> 
> We just need this to access registers. Dmd is more for desktop and servers. It will never support all the targets gcc can. Could we make this a gdc extension?

Probably, however I'd make it an UDA then in core.gcc, similar to
@attribute. Main drawback here is that we add gdc specific code to the
frontend. But as we need that code for @attribute anyway we could add
some generic code to the frontend to allow backends to use UDAs.

> While writing this  it just popped to my mind: if volatile is not good, could we reuse the 'system' word? Then it would be clear that this is for accessing system resources and not for application level.

volatile is still a (deprecated) keyword, so we don't break code by reintroducing it. However, if we have to make it a gdc only extension I wouldn't use a keyword with such a prominent name. An @volatile UDA is much better then.

Of course there are also non-technical issues regarding the name 'volatile'. C/C++ coders will already know what it means, but Java/C# coder could be confused.

On Tuesday, 24 June 2014 at 14:14:18 UTC, Johannes Pfau wrote:
> Am Tue, 24 Jun 2014 10:46:11 +0000
> schrieb "Timo Sintonen" <t.sintonen@luukku.com>:
>
>> To keep this thread going, I had a quick look at the reference material of the dip and picked some thoughts.
>> 
>> In some languages volatile has a stronger meaning, like guaranteeing an atomic access. In some languages it may not guarantee anything.
>> 
>> In this proposal volatile is only for optimization, not for protection. It does not add any code, it just prevents the optimizer removing some code.

Agreed.  In fact if this proposal reaches the main forum, we may hear proposals to introduce a "don't optimize" pragma or some other workaround.  I considered it myself :(

>> 
>> Walter has been against this and now also Martin. I think there is no use to bring this to the main forum. I understand the point that it is not very good to have something in the language specs that can not be guaranteed.
> I think we should at least try to bring this to the main newsgroup,
> however I got distracted with other things and I want to extend the DIP
> a little (Only rationale stuff, no technical changes). The
> newsgroups have been quite busy lately and there've been discussions
> about two new DIPs. Neither Andrei nor Walter even posted a response
> in these threads so now is probably not a good time to start the
> discussion on this DIP.

Agreed, it would be a shame to not try. And looking for a low in controversial topics on the main forum would probably be a good idea.  It would be bad for the passions of one discussion to spill over into this one.

Walter told me at DConf that he was in favor of compiler intrinsic peek/poke functions as proposed in DIP20 and he said, in the meantime, I could get by with Martin's volatileGet/volatileSet assembly workaround.

This tells me two things that are working against DIP62:
1) Walter believes volatile is a property of the load/store operation.
2) He doesn't consider volatile semantics a big priority.

DIP20 has Walter's support, but has been collecting mold on the DIP list for a year and a half. Getting DIP62 approved will be a challenge, to put it mildly, but it will still need to be implemented and accepted into DMD.

I was in favor of Walter's suggestion (DIP20) until DIP62. What sold me on DIP62 was the fact that one would never want to access volatile memory with non-volatile semantics. This is an irrefutable truth, and, as I can tell, only a type qualifier can enforce provide this enforcement.

DIP62 will also be a difficult sell given the simple assembly workaround.  I concede, the workaround will solve the problem and is a trivial implementation, but as Iain said, it is "...an excuse to *not* implement a feature that is rather essential for a systems language".

I define a systems programming language as one that is generally accepted and used to implement operating systems (kernels and hardware drivers), and I think this is the traditional definition before the language marketers redefined it.  D currently requires the help of C and/or other techniques to do this and lacks a runtime suitable for such development, so it is not a systems programming language IMO.  But, out of all the other languages I have encountered, D has the most potential of any to be the systems programming language of choice in the future, and DIP62 is a step in that direction.  The work of all those currently participating on this thread also shows some encouraging momentum in that direction.

Unfortunately, there just doesn't seem to be very many people using D for systems programming, likely because of the reasons I gave above, and this poses another hurdle for DIP62:  volatile has very little use outside of systems programming.  DIP62 has my support, but I know that doesn't mean much.  It will likely need the support of those with some clout in the D community.

Mike

Iain Buclaw:

> http://bugzilla.gdcproject.org/show_bug.cgi?id=126

The D1 deprecation warning to replace volatile with synchronized:

https://github.com/D-Programming-Language/dmd/commit/c8d580aea687f16b56ff4ce935f618b41a74c6e7

Bye,
bearophile

On 30 Jun 2014 23:20, "bearophile via D.gnu" <d.gnu@puremagic.com> wrote:
>
> Iain Buclaw:
>
>> http://bugzilla.gdcproject.org/show_bug.cgi?id=126
>
>
> The D1 deprecation warning to replace volatile with synchronized:
>
>
https://github.com/D-Programming-Language/dmd/commit/c8d580aea687f16b56ff4ce935f618b41a74c6e7
>
> Bye,
> bearophile

That's Volatile statements. Not volatile types.

Regards
Iain