On Friday, 15 November 2013 at 07:42:22 UTC, ilya-stromberg wrote:
> On Thursday, 14 November 2013 at 22:12:10 UTC, Jacek Furmankiewicz wrote:
>> On Thursday, 14 November 2013 at 21:36:46 UTC, ilya-stromberg wrote:
>>> On Thursday, 14 November 2013 at 21:31:52 UTC, Jacek Furmankiewicz wrote:
>>> How often do you change the data? Probably, you should use `immutable` variables.
>>
>> Customer specific. It may change once a year. It may change multiple times per second for a while, then nothing again for weeks.
>>
>> Others may do mass loads of business rules, hence do mass changes every few hours.
>>
>> Next to impossible to predict.
>
> You can use `immutable` variables. It allows you to share the data without any synchronization. Like this:
>
> class MyData
> {
>    int data1;
>    string data2;
>
>    //creates new object
>    this(int data1, string data2)
>    {
>       this.data1 = data1;
>       this.data2 = data2;
>    }
>
>    //modify the data
>    immutable(MyData) editData(int i) const
>    {
>       //copy this object - we can't change immutable variables
>       MyData dataCopy = new MyData(this.data1, this.data2)
>
>       //modify the data copy
>       dataCopy.data1 += i;
>
>       //assume that `dataCopy` is immutable
>       return cast(immutable(MyData)) dataCopy;
>    }
> }
>
> shared myMap;
>
> //map implementation
> synchronized class MyMap
> {
>    HashMap!(int, immutable(MyData)) map;
>
>    void foo()
>    {
>       map[1] = new immutable MyData(1, "data");
>    }
>
>    void bar()
>    {
>       map[1] = map[1].editData(5);
>    }
> }
>
> //init map
> shared static this()
> {
>    myMap = new MyMap();
> }
>
> void main()
> {
>    myMap.foo();
>    myMap.bar();
> }


So what happens when the "write" operation is doing

 map[1] = map[1].editData(5);

and at the same time 50 threads are simultaneously reading the value in map[1]?.

Is that reassignment operation thread safe?
Or would I get corrupted reads with potentially a partially overriden value?

Jacek

On Friday, 15 November 2013 at 15:21:59 UTC, Jacek Furmankiewicz wrote:
> So what happens when the "write" operation is doing
>
>  map[1] = map[1].editData(5);
>
> and at the same time 50 threads are simultaneously reading the value in map[1]?.
>
> Is that reassignment operation thread safe?
> Or would I get corrupted reads with potentially a partially overriden value?
>
> Jacek

Yes, this is thread safe.
Put attention to the `MyMap` class definition, it's marked as `synchronized`. It means that all class functions use the same Mutex. So, "write" operation will block map and all 50 threads will wait.

So, if you add a read() method to MyMap for those threads, would that be synchronized as well?

That is what we would not want due performance impact.

How can you achieve lock-free reads with the synchronized MyMap approach?

On Friday, 15 November 2013 at 16:36:56 UTC, Jacek Furmankiewicz wrote:
> How can you achieve lock-free reads with the synchronized MyMap approach?

In this case you can use Readers-writer lock
http://en.wikipedia.org/wiki/Readers%E2%80%93writer_lock

It allows multiple reads and single write. I think that the easiest way is use OS spesific function, for example `pthread_rwlock_t` for POSX. Note that D supports C ABI, so you can call any C function from D.

I don't know any D implementation of Readers-writer lock, but you can ask this question - maybe it already exist.

On Friday, 15 November 2013 at 17:03:15 UTC, ilya-stromberg wrote:
> I don't know any D implementation of Readers-writer lock, but you can ask this question - maybe it already exist.

http://dlang.org/phobos/core_sync_rwmutex.html

On Friday, 15 November 2013 at 17:09:54 UTC, Dicebot wrote:
> On Friday, 15 November 2013 at 17:03:15 UTC, ilya-stromberg wrote:
>> I don't know any D implementation of Readers-writer lock, but you can ask this question - maybe it already exist.
>
> http://dlang.org/phobos/core_sync_rwmutex.html

Thank you. I just never use it.

On Fri, 2013-11-15 at 18:03 +0100, ilya-stromberg wrote:
> On Friday, 15 November 2013 at 16:36:56 UTC, Jacek Furmankiewicz wrote:
> > How can you achieve lock-free reads with the synchronized MyMap approach?
> 
> In this case you can use Readers-writer lock http://en.wikipedia.org/wiki/Readers%E2%80%93writer_lock
> 
> It allows multiple reads and single write. I think that the easiest way is use OS spesific function, for example `pthread_rwlock_t` for POSX. Note that D supports C ABI, so you can call any C function from D.
> 
> I don't know any D implementation of Readers-writer lock, but you can ask this question - maybe it already exist.

Sorry to come in late on this one (and miss everything that comes
before).

The trend in the JVM-verse is very much "if you use synchronized or an explicit lock, and you are not creating a core library data structure, you are doing it wrong". The background is that the whole purpose of a lock it to control concurrency and thus stop parallelism. Applications programmers should never have to use a lock. ConcurrentHashMap, and thread safe queues are two consequences of all this.

In the Go-verse the attitude is basically the same, you should use channels and communications – the synchronization is managed by the data structure.

If D programmers are being told to use locks in applications code, then the D programming model and library are failing. Or the advice is wrong ;-)

-- 
Russel. ============================================================================= Dr Russel Winder      t: +44 20 7585 2200   voip: sip:russel.winder@ekiga.net 41 Buckmaster Road    m: +44 7770 465 077   xmpp: russel@winder.org.uk London SW11 1EN, UK   w: www.russel.org.uk  skype: russel_winder

Yes, that is what they say in Go...but it doesn't scale either. :-)

I had the exact same discussion on the Go forums a while back and the conclusion was basically the same...roll your own maps with RW locks:

https://groups.google.com/forum/?fromgroups#!searchin/golang-nuts/furmankiewicz/golang-nuts/jjjvXG4HdUw/ffWytKQ7X9YJ

But...at the end someone actually built lock-free data structures in Go out of this:

https://github.com/zond/gotomic

On Friday, 15 November 2013 at 17:46:41 UTC, Russel Winder wrote:
> If D programmers are being told to use locks in applications code, then
> the D programming model and library are failing. Or the advice is
> wrong ;-)

It's possible to implement lock-free data structures in D, you can use core.atomic
http://dlang.org/phobos/core_atomic.html

But it's REALLY difficult to implement and it can be SLOWER than Mutex version (not only in D, it depends from usage situation).

On Friday, 15 November 2013 at 17:46:41 UTC, Russel Winder wrote:
> The trend in the JVM-verse is very much "if you use synchronized or an
> explicit lock, and you are not creating a core library data structure,
> you are doing it wrong". The background is that the whole purpose of a
> lock it to control concurrency and thus stop parallelism. Applications
> programmers should never have to use a lock. ConcurrentHashMap, and
> thread safe queues are two consequences of all this.

True, concurrency in Java is really simple these days (especially with the Executors framework that Python 3 pretty much copies verbatim).

taskPool looks like the closest equivalent in D that I could find.