> Dr Russel Winder
> 41 Buckmaster Road
> London SW11 1EN, UK

Are there any D users groups/meetups in London?  I see you are not far away (I am in Barnes).


Laeeth

On Monday, 18 August 2014 at 23:12:28 UTC, Laeeth Isharc wrote:
>> For me, NumPy has some serious problems despite being the accepted norm for computational work.
>
> If not too offtopic, do you have a link describing, or would you briefly summarize these problems?  I am intrigued.  And what would you suggest in its place?  Fortran?

I'm not sure which computational work he is referring to, but for statistical analysis, R dominates by a wide margin (although statistical analysis done in Silicon Valley, the type you read about on Hacker News, is often done using Python).

I write functions in D, compile as a shared library, and call from R. The more statically typed code I write, the more I like it. I'm finishing up a blog post describing my usage.

On Mon, 2014-08-18 at 23:16 +0000, Laeeth Isharc via Digitalmars-d-learn wrote:

> Are there any D users groups/meetups in London?  I see you are not far away (I am in Barnes).

Crickey, that is just round the corner . :-)

Yes there is a London D user group, it just hasn't met as yet. I investigated creating the London D User Group on meetup.com, but they want $30.00 per 6 months, which is sort of reasonable for them to ask, but doesn't make sense for me unless we get members, actually meet reasonably regularly and I can defray the cost somehow.

-- 
Russel. ============================================================================= Dr Russel Winder      t: +44 20 7585 2200   voip: sip:russel.winder@ekiga.net 41 Buckmaster Road    m: +44 7770 465 077   xmpp: russel@winder.org.uk London SW11 1EN, UK   w: www.russel.org.uk  skype: russel_winder

On Mon, 2014-08-18 at 23:12 +0000, Laeeth Isharc via Digitalmars-d-learn wrote:
> > Whilst the hardcore Pythonistas remain Pythonistas, some of the periphery has jumped ship to Go. Sadly D did not capture these folk, it perhaps should have done. It would be easy to blame fadism, but I think the actual reasons are far less superficial.
> 
> So I gather that you agree that "what everyone is doing" may not be the best in this case (python vs D) if there are no direct network effects beyond libraries and getting help and you have the freedom to determine your own platform choices?

Go arrived with high marketing as something new, which it is. It is a stripped down, strongly typed C with memory management, objects and extension methods, and (most importantly) goroutines as a lightweight built-in concurrency and parallelism framework based on thread pools. The language itself is very simple (apart from the consequences of semi-colon ellision/re-adding), and so captured the imagination of many. Not unreasonably. I really quite like Go.

However it has some, for me, serious irritants, most especially an obsessive hatred of exceptions, enforcing return codes and error handling at the point of call – though it does support this in a neat way.

One effect of the "fanboi" element of using go has been very rapid evolution of the eco-system, especially strong because of the package approach and the use of DVCS as a tool for accessing packages.

D is not a new language so doesn't have the "new shiny toy" feature, but it remains an rapidly evolving language. It is definitely a better C++ but C++ is evolving fast enough that C++ folks stay with C++. So in a sense D has failed to capture the market it aimed for when it started. Also it is still playing "catch up" in the eco-system stakes.

D does however have a very neat model of abstraction that allows for a very functional programming approach. It isn't functional programming per se, but it gets very declarative. C++ is a long way behind in this, but there is little C++ → D transfer.

The problem for Python folks looking for a native code language is that D is a big language and Go is a small language.

And then there is Rust…

> > For me, NumPy has some serious problems despite being the accepted norm for computational work.
> 
> If not too offtopic, do you have a link describing, or would you briefly summarize these problems?  I am intrigued.  And what would you suggest in its place?  Fortran?
>

I'll start a separate thread for this one.

[…]
> Would you consider D stable enough/suitable for general financial market work with development initially by a small underresourced team?  Not ultra high frequency execution - at most legging in and managing longer term positions.  But I am more interested in sentiment analysis, producing technical analysis indicators that summarize market activity across many different securities, some bond arb stuff.  C++ just seems so ugly, and I feel uncomfortable only having python in the toolbox.  D seems so far to be quite suitable...

The most important thing about any project using a programming language is that the development team, testing team and deployment team all approve of the language being used. It sounds like you approve of D so go for it. The language is not totally stable, so expect incompatibilities on upgrade. I am not a fan of all this "backward compatibility" obsession that has gripped Fortran, C++ and Java (though I can see why it is necessary), but with D you will need to budget for upgrade-related refactoring. Personally I was happy to take that hit.

-- 
Russel. ============================================================================= Dr Russel Winder      t: +44 20 7585 2200   voip: sip:russel.winder@ekiga.net 41 Buckmaster Road    m: +44 7770 465 077   xmpp: russel@winder.org.uk London SW11 1EN, UK   w: www.russel.org.uk  skype: russel_winder

-- 
Russel. ============================================================================= Dr Russel Winder      t: +44 20 7585 2200   voip: sip:russel.winder@ekiga.net 41 Buckmaster Road    m: +44 7770 465 077   xmpp: russel@winder.org.uk London SW11 1EN, UK   w: www.russel.org.uk  skype: russel_winder

On Mon, 2014-08-18 at 23:12 +0000, Laeeth Isharc via Digitalmars-d-learn
wrote:
[…]
> > For me, NumPy has some serious problems despite being the accepted norm for computational work.
> 
> If not too offtopic, do you have a link describing, or would you briefly summarize these problems?  I am intrigued.  And what would you suggest in its place?  Fortran?
[…]

I have no benchmark experiment data as proof yet, just anecdotal evidence forming an hypothesis, but it seems that the underlying data parallelism model of NumPy has some serious overhead problems: speed-ups are not as high as they should be, and scaling is not as good as it should be.

The finance people using Python in London, and indeed the general data analysis using Python folk (cf. PyData meetings around the world) all take NumPy as a given, and that it works well enough for them. I guess those for whom NumPy is not good enough are using Cython, C++ or Fortran (or even C) for the computationally intensive stuff. Or more likely they already had the native code in place and so are not using NumPy other than for data visualization and replacement of Matlab.

I think Numba is a disruptive technology here.

However the danger is that the opaque type approach of NumPy (which is good) is forgotten as a good abstraction in the face of Numba speeds, with people reverting to explicit rather than implicit iteration just because things go faster.

Or maybe this won't happen because all the needed computationally intensive libraries already exists. I guess the PyData meetings are the place where al this will be played out.

-- 
Russel. ============================================================================= Dr Russel Winder      t: +44 20 7585 2200   voip: sip:russel.winder@ekiga.net 41 Buckmaster Road    m: +44 7770 465 077   xmpp: russel@winder.org.uk London SW11 1EN, UK   w: www.russel.org.uk  skype: russel_winder

On Tue, 2014-08-19 at 00:53 +0000, bachmeier via Digitalmars-d-learn
wrote:
[…]
> I'm not sure which computational work he is referring to, but for statistical analysis, R dominates by a wide margin (although statistical analysis done in Silicon Valley, the type you read about on Hacker News, is often done using Python).

My own :-)

I think Python is not making inroads in the world-wide R community, but it is in the Matlab and Mathematica ones. Whether Julia acts as a disruptive technology here we will see.

> I write functions in D, compile as a shared library, and call from R. The more statically typed code I write, the more I like it. I'm finishing up a blog post describing my usage.

This model also works for Python and D and does not require PyD (*). The issue here is that D, like C++, can provide shared objects (aka DLLs) with C linkage entry points and so Python extensions can be created.


(*) Though arguably it is easier using PyD.

-- 
Russel. ============================================================================= Dr Russel Winder      t: +44 20 7585 2200   voip: sip:russel.winder@ekiga.net 41 Buckmaster Road    m: +44 7770 465 077   xmpp: russel@winder.org.uk London SW11 1EN, UK   w: www.russel.org.uk  skype: russel_winder