In our Java code, we make heavy use of ConcurrentHashMap for in-memory caches:

http://docs.oracle.com/javase/7/docs/api/java/util/concurrent/ConcurrentHashMap.html

We use it mostly for in-memory caches. A background thread wakes up every X seconds and resyncs the cache with whatever changes occurred in the DB. In the meantime, all the incoming requests can read from the same cache without any concurrency issues and excellent performance.

All the major Java Map implementations also support the NavigableMap interface:

http://docs.oracle.com/javase/7/docs/api/java/util/NavigableMap.html

which allows you to get the key closest to the one you are looking for (either up or down). This is very useful when you have hierarchies of business rules that have effective date ranges and you are trying to find which rules is active on a particular day.

I read up the chapter on associative arrays in D, but I do not see anything similar to this functionality in there.

Could anyone point me to what would be the closest D equivalents (maybe in an external library if not part of Phobos) so we can playing around with them?

Much appreciated
Jacek

On Thursday, 14 November 2013 at 17:36:09 UTC, Jacek Furmankiewicz wrote:
> In our Java code, we make heavy use of ConcurrentHashMap for in-memory caches:
>
> http://docs.oracle.com/javase/7/docs/api/java/util/concurrent/ConcurrentHashMap.html
>
> We use it mostly for in-memory caches. A background thread wakes up every X seconds and resyncs the cache with whatever changes occurred in the DB. In the meantime, all the incoming requests can read from the same cache without any concurrency issues and excellent performance.
>
> All the major Java Map implementations also support the NavigableMap interface:
>
> http://docs.oracle.com/javase/7/docs/api/java/util/NavigableMap.html
>
> which allows you to get the key closest to the one you are looking for (either up or down). This is very useful when you have hierarchies of business rules that have effective date ranges and you are trying to find which rules is active on a particular day.
>
> I read up the chapter on associative arrays in D, but I do not see anything similar to this functionality in there.
>
> Could anyone point me to what would be the closest D equivalents (maybe in an external library if not part of Phobos) so we can playing around with them?
>
> Much appreciated
> Jacek

D does not have alot of diffrent containers atm. This work has been postponed until a working version of std.allocators is implemented. So atleast in the phobos library right now you will not find what you are looking for.

So how do existing D applications (especially the high perf ones in let's say the financial sector) deal with having some part of the data in memory and keeping it in sync with the DB source?

This must be a very common requirement for any app with realtime or close-to-realtime SLAs.

is there a different idiom or approach in D?

On Thursday, 14 November 2013 at 18:08:22 UTC, Jacek Furmankiewicz wrote:
> So how do existing D applications (especially the high perf ones in let's say the financial sector) deal with having some part of the data in memory and keeping it in sync with the DB source?

Good question. I have no idea.

> is there a different idiom or approach in D?

Well in D the prefered way (or recommended according to TDPL) to do concurrent sharing of resources is to share resources via message passing.

So in this case a single thread would be responsible for the caching of data in memory and other threads would ask this thread for data through message passing.

If this way is faster/better then javas ConcurrentHashMap i am not sure.

On Thursday, 14 November 2013 at 18:08:22 UTC, Jacek Furmankiewicz wrote:
> So how do existing D applications (especially the high perf ones in let's say the financial sector) deal with having some part of the data in memory and keeping it in sync with the DB source?

They don't, because there aren't really such apps in the wild yet.

On Thursday, 14 November 2013 at 17:36:09 UTC, Jacek Furmankiewicz wrote:
> In our Java code, we make heavy use of ConcurrentHashMap for in-memory caches:

Try to look dcollections:
http://www.dsource.org/projects/dcollections

Also, Vibe.d has own hashmap:
https://github.com/rejectedsoftware/vibe.d/blob/master/source/vibe/utils/hashmap.d

Thanks for the links.

I looked at the dcollections docs, but none of their collections seem thread safe. The vibe.d I guess is because it is meant to be used from async I/O in a single thread...but once you add multi-threading to an app I am guessing it would not be usable.

On Thursday, 14 November 2013 at 20:00:10 UTC, Jacek Furmankiewicz wrote:
> I looked at the dcollections docs, but none of their collections seem thread safe. The vibe.d I guess is because it is meant to be used from async I/O in a single thread...but once you add multi-threading to an app I am guessing it would not be usable.

No, you can:
1) Use different hashmap per tread. I don't know your situation, but it can be possible fo read-only cache like this:

import vibe.utils.hashmap;

HashMap!(int, int) map;

void foo()
{
   //use map
   map[1] = 1;
}

2) Use `shared` storage class and mutex like this:

import vibe.utils.hashmap;

shared HashMap!(int, int) map;

void foo()
{
   synchronized
   {
      //use map
      map[1] = 1;
   }
}

> 2) Use `shared` storage class and mutex like this:
>
> import vibe.utils.hashmap;
>
> shared HashMap!(int, int) map;
>
> void foo()
> {
>    synchronized
>    {
>       //use map
>       map[1] = 1;
>    }
> }

Locking every time you use the map dosn't rly seem reasonable. It's not particulary fast and you might forget to lock the map at some point (or does the shared modifier not allow you to do this in D?)

I'm not that fammiliar with the synchronzed statement but shouldn't it be locked on some object?

void bar()
{
   //Can one thread be in this block...
   synchronized
   {
     map[1] = 1;
   }

   //... while another thread is in this block?
   synchronized
   {
     map[2] = 2;
   }
}

If that is the case are you not limited in the way you can update the map eg only in a single block?

On Thursday, 14 November 2013 at 21:16:15 UTC, TheFlyingFiddle wrote:

> If that is the case are you not limited in the way you can update the map eg only in a single block?

Yes, it's probably not the best example. It's valid if you have only 1 synchronized block for map. But you can use something like this:

void bar()
{
   synchronized(map)
   {
     map[1] = 1;
   }

   synchronized(map)
   {
     map[2] = 2;
   }
}

Or this:

//Note: valid only if you have 1 function that use map
synchronized void bar()
{
     map[1] = 1;

     map[2] = 2;
}

Or this:

shared myMap;

synchronized class MyMap
{
   HashMap!(int, int) map;

   void foo()
   {
      map[1] = 1;
   }

   void bar()
   {
      map[2] = 2;
   }
}

//init map
shared static this()
{
   myMap = new MyMap();
}

Probably, it's the best example.