I'm confused by shared and how to use it.

import std.stdio;
class Foo {
    File logFile;
    void log(in string line) shared {
      synchronized(this){
        logFile.writeln(line);
      }
    }
}

This (or the equivalent code in my full size program) won't
compile:

source/log.d(256): Error: template std.stdio.File.writeln does
not match any function template declaration. Candidates are:
/usr/include/dmd/phobos/std/stdio.d(781):
std.stdio.File.writeln(S...)(S args)
source/log.d(256): Error: template std.stdio.File.writeln(S...)(S
args) cannot deduce template function from argument types
!()(const(immutable(char)[])) shared

Why is logFile suddenly being treated as shared? This does not
make sense to me. As a programmer I have acknowledged that log()
is a shared function in the declaration 'void log(in string line)
shared', so isn't it up to me to ensure that I do proper
synchronization within the function? Why is the compiler trying
to ensure that all functions called within log() are also marked
shared?
What is the right thing to do here?

Note that I can't just mark log() as not shared; log() must be
shared because elsewhere I have:

shared Foo sfoo = ...
sfoo.log(...)

Am 22.01.2014 06:16, schrieb unknown soldier:
> I'm confused by shared and how to use it.
>
> import std.stdio;
> class Foo {
>      File logFile;
>      void log(in string line) shared {
>        synchronized(this){
>          logFile.writeln(line);
>        }
>      }
> }
>
> This (or the equivalent code in my full size program) won't
> compile:
>
> source/log.d(256): Error: template std.stdio.File.writeln does
> not match any function template declaration. Candidates are:
> /usr/include/dmd/phobos/std/stdio.d(781):
> std.stdio.File.writeln(S...)(S args)
> source/log.d(256): Error: template std.stdio.File.writeln(S...)(S
> args) cannot deduce template function from argument types
> !()(const(immutable(char)[])) shared
>
> Why is logFile suddenly being treated as shared? This does not
> make sense to me. As a programmer I have acknowledged that log()
> is a shared function in the declaration 'void log(in string line)
> shared', so isn't it up to me to ensure that I do proper
> synchronization within the function? Why is the compiler trying
> to ensure that all functions called within log() are also marked
> shared?
> What is the right thing to do here?
>
> Note that I can't just mark log() as not shared; log() must be
> shared because elsewhere I have:
>
> shared Foo sfoo = ...
> sfoo.log(...)

For a shared method the this pointer is also shared. That means that you have to tell the compiler (manually) that it is now safe to use non-shared code. You usually do this by casting the this pointer to a unshared type.

class Foo {
     File logFile;
     void log(in string line) shared {
       synchronized(this){
         // safe from here on because synchronized
         auto self = cast(Foo)this;
         self.logFile.writeln(line);
       }
     }
}

The compiler does not do this for you, because it can not know if accessing any member is truly thread-safe. You might share the logFile with other instances of Foo. This would be a case where the compiler would wrongly remove the shared from this, if it would do so automatically inside synchronized blocks.

> The compiler does not do this for you, because it can not know if accessing any member is truly thread-safe. You might share the logFile with other instances of Foo. This would be a case where the compiler would wrongly remove the shared from this, if it would do so automatically inside synchronized blocks.

Makes sense; thank you!

A minor caveat is GDC treats shared as volatile, which can be not optimal. Even worse, on some platforms like itanium volatile generates fences.

Am Thu, 23 Jan 2014 19:20:34 +0000
schrieb "Kagamin" <spam@here.lot>:

> A minor caveat is GDC treats shared as volatile, which can be not optimal. Even worse, on some platforms like itanium volatile generates fences.

Do you have more information on that issue?
There are claims that Itanium "automatically generates fences", however
that statement is not precise and I can't find a detailed description.

Itanium has special instructions which refresh the caches before reading/writing. But the interesting question is this: Does GCC use these instructions if you access a variable marked as volatile?

If it does, that's really a performance problem. AFAICS we need a 'volatile' in D for embedded programming which just does the following:

* Avoid removing any stores / or loads to that variable (includes
  moving accesses out of loops)
* Avoid instruction reordering between accesses to volatile variables:
  volatile int a, c;
  int b;
  a = 0;
  b = 0;
  c = 0;
  b can be moved anywhere, but c must always come after a

Here volatile should only affect the ASM generated by the compiler, it shouldn't care at all what the final CPU does, that's the programmers task.

In some cases the compiler can't even know: For example some
memory areas in ARM are special and the processor won't reorder
accesses to those areas anyway, so a 'smart compiler' adding memory
barriers would actually only harm performance.
http://infocenter.arm.com/help/topic/com.arm.doc.dui0552a/CIHGEIID.html

Such a 'volatile' could be integrated with shared as it should not cause any huger performance problems. However, if you think of this code:

shared int X;
synchronized(x)
{
    for(int i=0; i< 100; i++)
        X = i;
}

it's actually valid to optimize the loop, so 'volatile' behavior is not wanted. But what if we used atomic ops?

shared int X;

for(int i=0; i< 100; i++)
    atomicSet(X, i);

Now the loop can't be optimized away. Is the compiler smart enough to figure out that an atomic op is used and it can't optimize this? Or would we have to mark this volatile?

for example http://software.intel.com/en-us/blogs/2007/11/30/volatile-almost-useless-for-multi-threaded-programming

http://igoro.com/archive/volatile-keyword-in-c-memory-model-explained/
As I understand, because Itanium doesn't have cache coherency, a memory fence is needed to implement volatile load and store. On x86 load and store are already volatile because of cache coherency, so fences are not needed.

Am Fri, 24 Jan 2014 22:30:13 +0000
schrieb "Kagamin" <spam@here.lot>:

> http://igoro.com/archive/volatile-keyword-in-c-memory-model-explained/
> As I understand, because Itanium doesn't have cache coherency, a
> memory fence is needed to implement volatile load and store. On
> x86 load and store are already volatile because of cache
> coherency, so fences are not needed.

Seems like C#s volatile cares about "compiler optimizations and processor optimizations.". I'd rather want a volatile for D (or as a base for shared) which only cares about compiler optimizations, leaving the processor part to the programmer.

(For example it isn't valid in D to access a shared variable with normal operations anyway, AFAIK. You need to use the atomicOp things and these will the worry about the hardware part, using the correct instructions and so on)

See also: http://www.drdobbs.com/parallel/volatile-vs-volatile/212701484

So volatile is a completely different thing in C#/JAVA and C/C++. This is why I'm confused, C compilers are not supposed to guarantee that another thread can see all changes to a volatile variable, volatile must just prevent compiler optimizations. So if Itanium C compilers automatically use the barrier/fence instructions for volatile that's IMHO a compiler performance bug.

I also can't find any information the GCC adds barriers on Itanium. Maybe this is just true for the Intel compiler, as the main source for these claims are Intel pages.

On Saturday, 25 January 2014 at 10:00:58 UTC, Johannes Pfau wrote:
> (For example it isn't valid in D to access a shared variable with
> normal operations anyway, AFAIK. You need to use the atomicOp things
> and these will the worry about the hardware part, using the correct
> instructions and so on)

Is it invalid to access shared data on stack too? How about closures?

> See also:
> http://www.drdobbs.com/parallel/volatile-vs-volatile/212701484
>
> So volatile is a completely different thing in C#/JAVA and C/C++. This
> is why I'm confused, C compilers are not supposed to guarantee that
> another thread can see all changes to a volatile variable, volatile must
> just prevent compiler optimizations. So if Itanium C compilers
> automatically use the barrier/fence instructions for volatile that's
> IMHO a compiler performance bug.

The standard says "memory" and doesn't address cache and processor optimizations, so it probably allows different interpretations. Even if there's no fence, disallowed compiler optimizations are still a performance hit - on all platforms.

Also if you read a shared value with atomicLoad every time, this disallows caching in registers or on stack, which is also performance hit. The shared value should be read once and cached if possible.