TJB:

> Why a tough sell?  Please explain.

That code must always be hard-real time. So a GC is allowed only during startup time (unless it's a quite special GC), hidden heap allocations are forbidden, data access patterns need to be carefully chosen, you even have to use most of the hot part of the stack, etc.

Bye,
bearophile

On 3/21/2014 5:39 PM, bearophile wrote:
> That code must always be hard-real time. So a GC is allowed only during startup
> time (unless it's a quite special GC), hidden heap allocations are forbidden,
> data access patterns need to be carefully chosen, you even have to use most of
> the hot part of the stack, etc.

These are all very doable in D, and I've written apps that do so. The "fear of GC" is completely overblown.

However, I do recognize that Phobos can be made a lot more non-GC friendly, and I am working on PR's to do that. For example,

https://github.com/D-Programming-Language/phobos/pull/2014

On Saturday, 22 March 2014 at 00:34:22 UTC, TJB wrote:
> On Saturday, 22 March 2014 at 00:14:11 UTC, Daniel Davidson wrote:
>> On Friday, 21 March 2014 at 21:14:15 UTC, TJB wrote:
>>> Walter,
>>>
>>> I see that you will be discussing "High Performance Code Using D" at the 2014 DConf. This will be a very welcomed topic for many of us.  I am a Finance Professor.  I currently teach and do research in computational finance.  Might I suggest that you include some finance (say Monte Carlo options pricing) examples?  If you can get the finance industry interested in D you might see a massive adoption of the language.  Many are desperate for an alternative to C++ in that space.
>>>
>>> Just a thought.
>>>
>>> Best,
>>>
>>> TJB
>>
>> Maybe a good starting point would be to port some of QuantLib and see how the performance compares. In High Frequency Trading I think D would be a tough sell, unfortunately.
>>
>> Thanks
>> Dan
>
> Dan,
>
> Why a tough sell?  Please explain.
>
> TJB

Well, I am a fan of D. That said, there is a ton of money that goes into the HFT low latency arms race. I felt they were always willing to spend whatever it takes - but they had to make choices. First, I know for several firms there was a history the C++ vs Java back and forth until it was clear that the lowest latency systems were C/C++. The difference was attributed to many causes, but garbage collection always came up as a big one. I think just that association of C++ beating Java because of garbage collection will be enough to prevent many HFT shops from considering D for a long while, even if you could get rid of the gc entirely. Additionally, many of the advantages firms get come from dealing intelligently with the kernel and lowest level networking techniques which implies expertise in C and that often comes with a well-deserved pride in knowing how to make things run fast in specific ways. Further they are pushing even lower into custom hardware which is expensive. With the money to throw at it and the various alternatives to choose from there is a go big or go home mentality. They are not as worried about how fun it is to code in, how ugly your templates are, or how it might take longer to do something or to build something.

I agree if you could get the finance industry interested in D that would be great. These are just my opinions and I left HFT a couple years back now. But, in this field, to expect a firm to get behind something like D you would need demonstrable improvements in performance irrespective of cost to develop. If they can do equivalent or better performance in C++ with more time to develop - then C++ still wins. Now if D can actually beat C++ significantly in quant programming performance things might change. Do you have ideas on how that could be done?

If you consider automated, black box trading, most firms roll their own options infrastructure and these days QuantLib would not make the cut. But if you could demonstrate same functionality with improved performance by using D you could make a case.

Thanks
Dan

On Saturday, 22 March 2014 at 01:24:38 UTC, Walter Bright wrote:
> On 3/21/2014 5:39 PM, bearophile wrote:
>> That code must always be hard-real time. So a GC is allowed only during startup
>> time (unless it's a quite special GC), hidden heap allocations are forbidden,
>> data access patterns need to be carefully chosen, you even have to use most of
>> the hot part of the stack, etc.
>
> These are all very doable in D, and I've written apps that do so. The "fear of GC" is completely overblown.
>

Fear of GC is overblown. Fear of D's GC isn't.

Am 22.03.2014 06:58, schrieb deadalnix:
> On Saturday, 22 March 2014 at 01:24:38 UTC, Walter Bright wrote:
>> On 3/21/2014 5:39 PM, bearophile wrote:
>>> That code must always be hard-real time. So a GC is allowed only
>>> during startup
>>> time (unless it's a quite special GC), hidden heap allocations are
>>> forbidden,
>>> data access patterns need to be carefully chosen, you even have to
>>> use most of
>>> the hot part of the stack, etc.
>>
>> These are all very doable in D, and I've written apps that do so. The
>> "fear of GC" is completely overblown.
>>
>
> Fear of GC is overblown. Fear of D's GC isn't.

Yes, as there are a few  high performance trading systems done with JVM/.NET languages.

--
Paulo

On Sat, Mar 22, 2014 at 12:30 AM, Paulo Pinto <pjmlp@progtools.org> wrote:

> Yes, as there are a few  high performance trading systems done with JVM/.NET languages.
>
> --
> Paulo
>

What about AAA games? :) Even though I agree with the pro-GC arguments you put forth, but I really have a hard time imagining something like Battlefield 4 written in Java and running on the JVM (though I would love to be proven wrong).

Am 22.03.2014 09:42, schrieb Ziad Hatahet:
> On Sat, Mar 22, 2014 at 12:30 AM, Paulo Pinto <pjmlp@progtools.org
> <mailto:pjmlp@progtools.org>> wrote:
>
>     Yes, as there are a few  high performance trading systems done with
>     JVM/.NET languages.
>
>     --
>     Paulo
>
>
> What about AAA games? :) Even though I agree with the pro-GC arguments
> you put forth, but I really have a hard time imagining something like
> Battlefield 4 written in Java and running on the JVM (though I would
> love to be proven wrong).

What about ballistic missile tracking radar control systems?

http://www.bloomberg.com/apps/news?pid=newsarchive&sid=aHLFBJZrqaoM

You don't want those babies out of control.

--
Paulo

On Saturday, 22 March 2014 at 08:54:46 UTC, Paulo Pinto wrote:
> Am 22.03.2014 09:42, schrieb Ziad Hatahet:
>> On Sat, Mar 22, 2014 at 12:30 AM, Paulo Pinto <pjmlp@progtools.org
>> <mailto:pjmlp@progtools.org>> wrote:
>>
>>    Yes, as there are a few  high performance trading systems done with
>>    JVM/.NET languages.
>>
>>    --
>>    Paulo
>>
>>
>> What about AAA games? :) Even though I agree with the pro-GC arguments
>> you put forth, but I really have a hard time imagining something like
>> Battlefield 4 written in Java and running on the JVM (though I would
>> love to be proven wrong).
>
> What about ballistic missile tracking radar control systems?
>
> http://www.bloomberg.com/apps/news?pid=newsarchive&sid=aHLFBJZrqaoM
>
> You don't want those babies out of control.
>
> --
> Paulo

Ah Thales... I hope they are better at writing software for missiles than they are at tram management systems.

On Fri, 2014-03-21 at 22:44 +0000, Chris Williams wrote:
> On Friday, 21 March 2014 at 22:28:36 UTC, Walter Bright wrote:
> > It's a good thought, but I have zero knowledge of how C++ is used for high frequency trading.
> 
> Reading through the Wikipedia article on Computational Finance, it looks like it's basically performing simulations where some data is known but other is not. Random numbers are generated for the unknown data and the simulations are run several times to find the range of possible outcomes given the known values.

http://en.wikipedia.org/wiki/Computational_finance

Is seriously lacking in actual content but at least it isn't entirely wrong.

There are many different things happening in what many would label as computational finance from ultra-high-frequency trading systems to modelling macroeconomics for fund management. In the former case of course the most crucial thing is the length of the cable of you computer to the router :-) In the latter case, and indeed quantitative analysis (quant), the mathematical models can be seriously weird (most likely because they are based on phenomenology rather than science-based modelling).

What you are alluding to is the use of Monte Carlo approach to solve some of the models given boundary conditions. This is a "bog standard" approach to numerical modelling.

Many quants use Excel, many are C++ folk. Those using Excel really need to stop it.

Many of the hedge funds are following in the HEP, bioinformatics direction and using Python (PyPy, NumPy, SciPy, etc.) for writing the initial models and then, if greater speed is needed, using Cython or rewriting in C++. Mathematica, R and Julia are increasingly also players in this game.

D could be a very interesting player in this game, but it would have to have some early adopters get on board and show in the domain itself that it can beat Python, C++, Mathematics, R and Julia at their own game.

Whilst I am not a finance person myself: I train a lot of finance
people, and signal processing people, in Python, Java, Scala, and
Groovy. In a number of the major international finance houses a
Python/Scala/C++ stack has taken hold. However this is not fixed
forever, since whenever the CTO changes the stack tends to as well.

-- 
Russel. ============================================================================= Dr Russel Winder      t: +44 20 7585 2200   voip: sip:russel.winder@ekiga.net 41 Buckmaster Road    m: +44 7770 465 077   xmpp: russel@winder.org.uk London SW11 1EN, UK   w: www.russel.org.uk  skype: russel_winder

On Sat, 2014-03-22 at 00:39 +0000, bearophile wrote:
> TJB:
> 
> > Why a tough sell?  Please explain.
> 
> That code must always be hard-real time. So a GC is allowed only during startup time (unless it's a quite special GC), hidden heap allocations are forbidden, data access patterns need to be carefully chosen, you even have to use most of the hot part of the stack, etc.

Not entirely the case. Yes the ultra-high-frequency traders tend to be C ++ and hard real time with no operating system scheduler active. However there are a number of high frequency trading systems using the JVM with the G1 garbage collector since actually only soft real time is needed for the trading they are doing.

It is also worth pointing out the LMAX Disruptor which is a lock-free
ring buffer based framework used to create dealing platforms on the JVM.
They outperform any other trading platform still.

-- 
Russel. ============================================================================= Dr Russel Winder      t: +44 20 7585 2200   voip: sip:russel.winder@ekiga.net 41 Buckmaster Road    m: +44 7770 465 077   xmpp: russel@winder.org.uk London SW11 1EN, UK   w: www.russel.org.uk  skype: russel_winder