December 10, 2012
On Monday, 10 December 2012 at 20:08:32 UTC, Andrei Alexandrescu wrote:
> On 12/9/12 10:58 PM, Nick B wrote:
> [about the Disruptor framework]
>> Would Andrei like to comment on any of the comments so far ??
>
> Sorry, I'd need to acquire expertise in Disruptor before discussing it. I found http://disruptor.googlecode.com/files/Disruptor-1.0.pdf quite difficult to get into because it spends the first page stating how good Disruptor is, then the next 3 pages discussing unrelated generalities, to then start discussing implementation details still without really defining the pattern. I had to stop there.
>
> Is there a more concise description of the pattern?
>
>
> Andrei

http://stackoverflow.com/questions/6559308/how-does-lmaxs-disruptor-pattern-work

A couple of posts in there attempt to summarize what it is. One of them compares it against a few other strategies.

--rt
December 10, 2012
On Monday, 10 December 2012 at 20:08:32 UTC, Andrei Alexandrescu wrote:
> On 12/9/12 10:58 PM, Nick B wrote:
> [about the Disruptor framework]
>> Would Andrei like to comment on any of the comments so far ??
>
> Sorry, I'd need to acquire expertise in Disruptor before discussing it. I found http://disruptor.googlecode.com/files/Disruptor-1.0.pdf quite difficult to get into because it spends the first page stating how good Disruptor is, then the next 3 pages discussing unrelated generalities, to then start discussing implementation details still without really defining the pattern. I had to stop there.
>
> Is there a more concise description of the pattern?
>
>
> Andrei

I have looked, but I think the answer is to this question is no.

Nick
December 12, 2012
On Monday, 10 December 2012 at 23:04:56 UTC, Nick B wrote:
> On Monday, 10 December 2012 at 20:08:32 UTC, Andrei Alexandrescu wrote:
>> On 12/9/12 10:58 PM, Nick B wrote:
>> [about the Disruptor framework]
>>> Would Andrei like to comment on any of the comments so far ??
>>
>> Sorry, I'd need to acquire expertise in Disruptor before discussing it. I found http://disruptor.googlecode.com/files/Disruptor-1.0.pdf quite difficult to get into because it spends the first page stating how good Disruptor is, then the next 3 pages discussing unrelated generalities, to then start discussing implementation details still without really defining the pattern. I had to stop there.
>>
>> Is there a more concise description of the pattern?
>>
>>

Ok, does anyone consider that this pattern, though not well described, has any value ?

Nick
December 12, 2012
On Wednesday, 12 December 2012 at 02:00:26 UTC, Nick B wrote:
> Ok, does anyone consider that this pattern, though not well described, has any value ?
>

I do.
December 12, 2012
12/12/2012 6:00 AM, Nick B пишет:
> On Monday, 10 December 2012 at 23:04:56 UTC, Nick B wrote:
>> On Monday, 10 December 2012 at 20:08:32 UTC, Andrei Alexandrescu wrote:
>>> On 12/9/12 10:58 PM, Nick B wrote:
>>> [about the Disruptor framework]
>>>> Would Andrei like to comment on any of the comments so far ??
>>>
>>> Sorry, I'd need to acquire expertise in Disruptor before discussing
>>> it. I found http://disruptor.googlecode.com/files/Disruptor-1.0.pdf
>>> quite difficult to get into because it spends the first page stating
>>> how good Disruptor is, then the next 3 pages discussing unrelated
>>> generalities, to then start discussing implementation details still
>>> without really defining the pattern. I had to stop there.
>>>
>>> Is there a more concise description of the pattern?
>>>
>>>
>
> Ok, does anyone consider that this pattern, though not well described,
> has any value ?
>
> Nick

It surely has.

I'd consider http://martinfowler.com/articles/lmax.html to be quite nice description as it helped me to decipher the most of details.
The paper itself lacks illustrative material, focuses too much on overcoming Java limitations and generally too terse with important (to me) details of the framework.

-- 
Dmitry Olshansky
December 12, 2012
On Wednesday, 12 December 2012 at 08:09:40 UTC, Dmitry Olshansky wrote:
> 12/12/2012 6:00 AM, Nick B пишет:
>> On Monday, 10 December 2012 at 23:04:56 UTC, Nick B wrote:
[cut]
>> Ok, does anyone consider that this pattern, though not well described,
>> has any value ?
>>
>> Nick
>
> It surely has.

Maybe, but I'm still not clear what are the differences between a normal ring buffer (not a new concept) and this "disruptor" pattern..

BR,
renoX

December 13, 2012
> Maybe, but I'm still not clear what are the differences between a normal ring buffer (not a new concept) and this "disruptor" pattern..

Key differences with a typical lock-free queue:
- Lightning fast when used correctly. It observes that not only is locking expensive, even CAS (compare and swap) is not cheap, so it avoids CAS in favor of memory barriers (unless multiple writers are required.) Memory allocation is avoided too, by preallocating everything.
- Multicast and multisource: multiple readers can view the same entries.
- Separation of concerns: disruptors are a whole library instead of a single class, so disruptors support several configurations of producers and consumers, as opposed to a normal queue that is limited to one or two arrangements. To me, one particularly interesting feature is that a reader can modify an entry and then another reader can flag itself as "dependent" on the output of the first reader. So really it supports not just readers and writers but "annotators" that both read an write. And the set of readers and writers can be arranged as a graph.

See also
http://stackoverflow.com/questions/6559308/how-does-lmaxs-disruptor-pattern-work
December 13, 2012
12/13/2012 4:59 AM, David Piepgrass пишет:
>> Maybe, but I'm still not clear what are the differences between a
>> normal ring buffer (not a new concept) and this "disruptor" pattern..
>
> Key differences with a typical lock-free queue:

Nice summary. I wasn't sure where should I describing that it's not "just a ring buffer". But for start I'd define it is a framework for concurrent processing of a stream of tasks/requests/items on a well structured multi-staged pipeline.

> - Lightning fast when used correctly. It observes that not only is
> locking expensive, even CAS (compare and swap) is not cheap, so it
> avoids CAS in favor of memory barriers (unless multiple writers are
> required.) Memory allocation is avoided too, by preallocating everything.
> - Multicast and multisource: multiple readers can view the same entries.
> - Separation of concerns: disruptors are a whole library instead of a
> single class, so disruptors support several configurations of producers
> and consumers, as opposed to a normal queue that is limited to one or
> two arrangements. To me, one particularly interesting feature is that a
> reader can modify an entry and then another reader can flag itself as
> "dependent" on the output of the first reader.

And the producer in the end depends on the last consumers in this graph.

There is also highly flexible (policy-based design) selection of how consumers wait on data:
- either busy _spin_ on it thus getting the highest responsiveness at the cost of wasted CPU cycles
- lazy spin (that yields) no outright burning of CPU resources but higher latency
- and even locking with wait-notify that saves greatly on CPU but kills responsiveness and throughput (but gives freedom to spend CPU elsewhere)

Plus there are different strategies for multi-prioducer or single-producer setup.

>So really it supports not
> just readers and writers but "annotators" that both read an write. And
> the set of readers and writers can be arranged as a graph.
>
Yeah, to me it indicates multi-thread-friendly multi-pass processing.

Another important IMHO observation is that the order of processed items is preserved* and this is interesting property if you consider doing the same stages as lock-free queues with a pool of consumers at each stage. Things will get O-o-O very quickly.

*That was stressed in the article about LMAX i.e. a trade depends on the order of other trades. So if your trade happens later then expected (or earlier) it going to be a different trade.

> See also
> http://stackoverflow.com/questions/6559308/how-does-lmaxs-disruptor-pattern-work
>


-- 
Dmitry Olshansky
December 13, 2012
On Thursday, 13 December 2012 at 16:07:13 UTC, Dmitry Olshansky wrote:
> 12/13/2012 4:59 AM, David Piepgrass пишет:
>>> Maybe, but I'm still not clear what are the differences between a
>>> normal ring buffer (not a new concept) and this "disruptor" pattern..
>>
>> Key differences with a typical lock-free queue:
>
> But for start I'd define it is a framework for concurrent processing of a stream of tasks/requests/items on a well structured multi-staged pipeline.
> 
  An excellent one sentence description!

>> [snip]

> There is also highly flexible (policy-based design) selection of how consumers wait on data:
> - either busy _spin_ on it thus getting the highest responsiveness at the cost of wasted CPU cycles
> - lazy spin (that yields) no outright burning of CPU resources but higher latency
> - and even locking with wait-notify that saves greatly on CPU but kills responsiveness and throughput (but gives freedom to spend CPU elsewhere)
>
> Thanks for pointing this out.

[snip]
>
> Another important IMHO observation is that the order of processed items is preserved* and this is interesting property if you consider doing the same stages as lock-free queues with a pool of consumers at each stage. Things will get O-o-O very quickly.


what does O-o-O mean ? >

Nick
December 13, 2012
On Thursday, 13 December 2012 at 23:32:44 UTC, Nick B wrote:
> what does O-o-O mean ? >

Out-of-order.

David