View mode: basic / threaded / horizontal-split · Log in · Help
December 10, 2012
Re: the Disruptor framework vs The Complexities of Concurrency
On Monday, 10 December 2012 at 20:08:32 UTC, Andrei Alexandrescu 
wrote:
> On 12/9/12 10:58 PM, Nick B wrote:
> [about the Disruptor framework]
>> Would Andrei like to comment on any of the comments so far ??
>
> Sorry, I'd need to acquire expertise in Disruptor before 
> discussing it. I found 
> http://disruptor.googlecode.com/files/Disruptor-1.0.pdf quite 
> difficult to get into because it spends the first page stating 
> how good Disruptor is, then the next 3 pages discussing 
> unrelated generalities, to then start discussing implementation 
> details still without really defining the pattern. I had to 
> stop there.
>
> Is there a more concise description of the pattern?
>
>
> Andrei

http://stackoverflow.com/questions/6559308/how-does-lmaxs-disruptor-pattern-work

A couple of posts in there attempt to summarize what it is. One 
of them compares it against a few other strategies.

--rt
December 10, 2012
Re: the Disruptor framework vs The Complexities of Concurrency
On Monday, 10 December 2012 at 20:08:32 UTC, Andrei Alexandrescu 
wrote:
> On 12/9/12 10:58 PM, Nick B wrote:
> [about the Disruptor framework]
>> Would Andrei like to comment on any of the comments so far ??
>
> Sorry, I'd need to acquire expertise in Disruptor before 
> discussing it. I found 
> http://disruptor.googlecode.com/files/Disruptor-1.0.pdf quite 
> difficult to get into because it spends the first page stating 
> how good Disruptor is, then the next 3 pages discussing 
> unrelated generalities, to then start discussing implementation 
> details still without really defining the pattern. I had to 
> stop there.
>
> Is there a more concise description of the pattern?
>
>
> Andrei

I have looked, but I think the answer is to this question is no.

Nick
December 12, 2012
Re: the Disruptor framework vs The Complexities of Concurrency
On Monday, 10 December 2012 at 23:04:56 UTC, Nick B wrote:
> On Monday, 10 December 2012 at 20:08:32 UTC, Andrei 
> Alexandrescu wrote:
>> On 12/9/12 10:58 PM, Nick B wrote:
>> [about the Disruptor framework]
>>> Would Andrei like to comment on any of the comments so far ??
>>
>> Sorry, I'd need to acquire expertise in Disruptor before 
>> discussing it. I found 
>> http://disruptor.googlecode.com/files/Disruptor-1.0.pdf quite 
>> difficult to get into because it spends the first page stating 
>> how good Disruptor is, then the next 3 pages discussing 
>> unrelated generalities, to then start discussing 
>> implementation details still without really defining the 
>> pattern. I had to stop there.
>>
>> Is there a more concise description of the pattern?
>>
>>

Ok, does anyone consider that this pattern, though not well 
described, has any value ?

Nick
December 12, 2012
Re: the Disruptor framework vs The Complexities of Concurrency
On Wednesday, 12 December 2012 at 02:00:26 UTC, Nick B wrote:
> Ok, does anyone consider that this pattern, though not well 
> described, has any value ?
>

I do.
December 12, 2012
Re: the Disruptor framework vs The Complexities of Concurrency
12/12/2012 6:00 AM, Nick B пишет:
> On Monday, 10 December 2012 at 23:04:56 UTC, Nick B wrote:
>> On Monday, 10 December 2012 at 20:08:32 UTC, Andrei Alexandrescu wrote:
>>> On 12/9/12 10:58 PM, Nick B wrote:
>>> [about the Disruptor framework]
>>>> Would Andrei like to comment on any of the comments so far ??
>>>
>>> Sorry, I'd need to acquire expertise in Disruptor before discussing
>>> it. I found http://disruptor.googlecode.com/files/Disruptor-1.0.pdf
>>> quite difficult to get into because it spends the first page stating
>>> how good Disruptor is, then the next 3 pages discussing unrelated
>>> generalities, to then start discussing implementation details still
>>> without really defining the pattern. I had to stop there.
>>>
>>> Is there a more concise description of the pattern?
>>>
>>>
>
> Ok, does anyone consider that this pattern, though not well described,
> has any value ?
>
> Nick

It surely has.

I'd consider http://martinfowler.com/articles/lmax.html to be quite nice 
description as it helped me to decipher the most of details.
The paper itself lacks illustrative material, focuses too much on 
overcoming Java limitations and generally too terse with important (to 
me) details of the framework.

-- 
Dmitry Olshansky
December 12, 2012
Re: the Disruptor framework vs The Complexities of Concurrency
On Wednesday, 12 December 2012 at 08:09:40 UTC, Dmitry Olshansky 
wrote:
> 12/12/2012 6:00 AM, Nick B пишет:
>> On Monday, 10 December 2012 at 23:04:56 UTC, Nick B wrote:
[cut]
>> Ok, does anyone consider that this pattern, though not well 
>> described,
>> has any value ?
>>
>> Nick
>
> It surely has.

Maybe, but I'm still not clear what are the differences between a 
normal ring buffer (not a new concept) and this "disruptor" 
pattern..

BR,
renoX
December 13, 2012
Re: the Disruptor framework vs The Complexities of Concurrency
> Maybe, but I'm still not clear what are the differences between 
> a normal ring buffer (not a new concept) and this "disruptor" 
> pattern..

Key differences with a typical lock-free queue:
- Lightning fast when used correctly. It observes that not only 
is locking expensive, even CAS (compare and swap) is not cheap, 
so it avoids CAS in favor of memory barriers (unless multiple 
writers are required.) Memory allocation is avoided too, by 
preallocating everything.
- Multicast and multisource: multiple readers can view the same 
entries.
- Separation of concerns: disruptors are a whole library instead 
of a single class, so disruptors support several configurations 
of producers and consumers, as opposed to a normal queue that is 
limited to one or two arrangements. To me, one particularly 
interesting feature is that a reader can modify an entry and then 
another reader can flag itself as "dependent" on the output of 
the first reader. So really it supports not just readers and 
writers but "annotators" that both read an write. And the set of 
readers and writers can be arranged as a graph.

See also
http://stackoverflow.com/questions/6559308/how-does-lmaxs-disruptor-pattern-work
December 13, 2012
Re: the Disruptor framework vs The Complexities of Concurrency
12/13/2012 4:59 AM, David Piepgrass пишет:
>> Maybe, but I'm still not clear what are the differences between a
>> normal ring buffer (not a new concept) and this "disruptor" pattern..
>
> Key differences with a typical lock-free queue:

Nice summary. I wasn't sure where should I describing that it's not 
"just a ring buffer". But for start I'd define it is a framework for 
concurrent processing of a stream of tasks/requests/items on a well 
structured multi-staged pipeline.

> - Lightning fast when used correctly. It observes that not only is
> locking expensive, even CAS (compare and swap) is not cheap, so it
> avoids CAS in favor of memory barriers (unless multiple writers are
> required.) Memory allocation is avoided too, by preallocating everything.
> - Multicast and multisource: multiple readers can view the same entries.
> - Separation of concerns: disruptors are a whole library instead of a
> single class, so disruptors support several configurations of producers
> and consumers, as opposed to a normal queue that is limited to one or
> two arrangements. To me, one particularly interesting feature is that a
> reader can modify an entry and then another reader can flag itself as
> "dependent" on the output of the first reader.

And the producer in the end depends on the last consumers in this graph.

There is also highly flexible (policy-based design) selection of how 
consumers wait on data:
- either busy _spin_ on it thus getting the highest responsiveness at 
the cost of wasted CPU cycles
- lazy spin (that yields) no outright burning of CPU resources but 
higher latency
- and even locking with wait-notify that saves greatly on CPU but kills 
responsiveness and throughput (but gives freedom to spend CPU elsewhere)

Plus there are different strategies for multi-prioducer or 
single-producer setup.

>So really it supports not
> just readers and writers but "annotators" that both read an write. And
> the set of readers and writers can be arranged as a graph.
>
Yeah, to me it indicates multi-thread-friendly multi-pass processing.

Another important IMHO observation is that the order of processed items 
is preserved* and this is interesting property if you consider doing the 
same stages as lock-free queues with a pool of consumers at each stage. 
Things will get O-o-O very quickly.

*That was stressed in the article about LMAX i.e. a trade depends on the 
order of other trades. So if your trade happens later then expected (or 
earlier) it going to be a different trade.

> See also
> http://stackoverflow.com/questions/6559308/how-does-lmaxs-disruptor-pattern-work
>


-- 
Dmitry Olshansky
December 13, 2012
Re: the Disruptor framework vs The Complexities of Concurrency
On Thursday, 13 December 2012 at 16:07:13 UTC, Dmitry Olshansky 
wrote:
> 12/13/2012 4:59 AM, David Piepgrass пишет:
>>> Maybe, but I'm still not clear what are the differences 
>>> between a
>>> normal ring buffer (not a new concept) and this "disruptor" 
>>> pattern..
>>
>> Key differences with a typical lock-free queue:
>
> But for start I'd define it is a framework for concurrent 
> processing of a stream of tasks/requests/items on a well 
> structured multi-staged pipeline.
> 
  An excellent one sentence description!

>> [snip]

> There is also highly flexible (policy-based design) selection 
> of how consumers wait on data:
> - either busy _spin_ on it thus getting the highest 
> responsiveness at the cost of wasted CPU cycles
> - lazy spin (that yields) no outright burning of CPU resources 
> but higher latency
> - and even locking with wait-notify that saves greatly on CPU 
> but kills responsiveness and throughput (but gives freedom to 
> spend CPU elsewhere)
>
> Thanks for pointing this out.

[snip]
>
> Another important IMHO observation is that the order of 
> processed items is preserved* and this is interesting property 
> if you consider doing the same stages as lock-free queues with 
> a pool of consumers at each stage. Things will get O-o-O very 
> quickly.


what does O-o-O mean ? >

Nick
December 13, 2012
Re: the Disruptor framework vs The Complexities of Concurrency
On Thursday, 13 December 2012 at 23:32:44 UTC, Nick B wrote:
> what does O-o-O mean ? >

Out-of-order.

David
1 2 3
Top | Discussion index | About this forum | D home