I have been running some MCMC simulations in Python and it's hard to cope with how unbelievably slow it is.
Takes me almost a minute to run a few hundred thousand samples on my laptop whereas I can run the same simulation with a million samples in under 100ms, on my phone with JavaScript on a browser.

Then, you spend a minute waiting for the simulation to finish, to find out you had an error in your report code that would have been easily caught with static typing. So annoying...

Is anyone doing similar stuff with D? Unfortunately, I couldn't find any plotting libraries nor MATLAB-like numerical/stats libs in dub.

This seems like another area where D could easily pick up momentum with RDMD and perhaps an integration with Jupyter which is becoming very very popular.

On Monday, 9 November 2015 at 19:31:14 UTC, Márcio Martins wrote:
> I have been running some MCMC simulations in Python and it's hard to cope with how unbelievably slow it is.
> Takes me almost a minute to run a few hundred thousand samples on my laptop whereas I can run the same simulation with a million samples in under 100ms, on my phone with JavaScript on a browser.
>
> Then, you spend a minute waiting for the simulation to finish, to find out you had an error in your report code that would have been easily caught with static typing. So annoying...
>
> Is anyone doing similar stuff with D? Unfortunately, I couldn't find any plotting libraries nor MATLAB-like numerical/stats libs in dub.
>
> This seems like another area where D could easily pick up momentum with RDMD and perhaps an integration with Jupyter which is becoming very very popular.

I think you may like to have a look at https://github.com/dscience-developers/dscience that is a collective effort to give D tools for bioinformatics (mainly). Also note that if you want you can just plug some python code in using PyD https://github.com/ariovistus/pyd. A concrete example can be found here: http://d.readthedocs.org/en/latest/examples.html#plotting-with-matplotlib-python

Although, this video could be of some interest to you: http://dconf.org/2015/talks/colvin.html

On Monday, 9 November 2015 at 19:31:14 UTC, Márcio Martins wrote:
> I have been running some MCMC simulations in Python ...
>
> Is anyone doing similar stuff with D? Unfortunately, I couldn't find any plotting libraries nor MATLAB-like numerical/stats libs in dub.
>
> This seems like another area where D could easily pick up momentum with RDMD and perhaps an integration with Jupyter which is becoming very very popular.

see http://dlangscience.github.io/

On Monday, 9 November 2015 at 20:30:49 UTC, Gerald Jansen wrote:
> On Monday, 9 November 2015 at 19:31:14 UTC, Márcio Martins wrote:
>> I have been running some MCMC simulations in Python ...
>>
>> Is anyone doing similar stuff with D? Unfortunately, I couldn't find any plotting libraries nor MATLAB-like numerical/stats libs in dub.
>>
>> This seems like another area where D could easily pick up momentum with RDMD and perhaps an integration with Jupyter which is becoming very very popular.
>
> see http://dlangscience.github.io/

And here is the gitter discussion site:

https://gitter.im/DlangScience/public

I've got this project

https://bitbucket.org/bachmeil/dmdinline2

to embed D inside R on Linux. Unfortunately the documentation isn't good. I'm currently working on going in the other direction, embedding R inside D. There are, of course, many good MCMC options in R that you could call from your D code.

On Monday, 9 November 2015 at 21:05:35 UTC, bachmeier wrote:
> On Monday, 9 November 2015 at 20:30:49 UTC, Gerald Jansen wrote:
>> On Monday, 9 November 2015 at 19:31:14 UTC, Márcio Martins wrote:
>>> I have been running some MCMC simulations in Python ...
>>>
>>> Is anyone doing similar stuff with D? Unfortunately, I couldn't find any plotting libraries nor MATLAB-like numerical/stats libs in dub.
>>>
>>> This seems like another area where D could easily pick up momentum with RDMD and perhaps an integration with Jupyter which is becoming very very popular.
>>
>> see http://dlangscience.github.io/
>
> And here is the gitter discussion site:
>
> https://gitter.im/DlangScience/public
>
> I've got this project
>
> https://bitbucket.org/bachmeil/dmdinline2
>
> to embed D inside R on Linux. Unfortunately the documentation isn't good. I'm currently working on going in the other direction, embedding R inside D. There are, of course, many good MCMC options in R that you could call from your D code.

Weird approach. Usually, when one wants to use an interpreted language as an host to a compiled language, the strategy is precompile the compiled language's code and load it as extensions in the interpreted code.

On Tuesday, 10 November 2015 at 00:01:19 UTC, Idan Arye wrote:
> Weird approach. Usually, when one wants to use an interpreted language as an host to a compiled language, the strategy is precompile the compiled language's code and load it as extensions in the interpreted code.

That's what dmdinline does.

On Mon, 2015-11-09 at 19:31 +0000, Márcio Martins via Digitalmars-d wrote:
> I have been running some MCMC simulations in Python and it's hard
> to cope with how unbelievably slow it is.
> Takes me almost a minute to run a few hundred thousand samples on
> my laptop whereas I can run the same simulation with a million
> samples in under 100ms, on my phone with JavaScript on a browser.

Are you using NumPy with Python, or pure Python? In either case you will be better served by profiling you code to find out whoch is actually the performance bottleneck and then doing one of:

1. Use Numba to JIT the Python and achieve native code speed. 2. Replace the Python code with D code using either CFFI or PyD. 3. Replace the Python code with Chapel code using either CFFI or PyChapel.

You could also use C++ or Rust via CFFI, but I have little experience using Rust with Python and am trying to avoid C++.

> Then, you spend a minute waiting for the simulation to finish, to find out you had an error in your report code that would have been easily caught with static typing. So annoying...
> 
> Is anyone doing similar stuff with D? Unfortunately, I couldn't find any plotting libraries nor MATLAB-like numerical/stats libs in dub.

No need if you leave the coordination code in Python and then use Matplotlib, etc. Just put the computational expensive code into Chapel or D, leave the rest of your code in Python.

> This seems like another area where D could easily pick up momentum with RDMD and perhaps an integration with Jupyter which is becoming very very popular.

Jupyter (née IPython) is only popular in one workflow, creting papers for people to read and play with the executable code fragments. This is a big area but only one of many.

-- 
Russel. ============================================================================= Dr Russel Winder      t: +44 20 7585 2200   voip: sip:russel.winder@ekiga.net 41 Buckmaster Road    m: +44 7770 465 077   xmpp: russel@winder.org.uk London SW11 1EN, UK   w: www.russel.org.uk  skype: russel_winder

On Tuesday, 10 November 2015 at 01:53:25 UTC, bachmeier wrote:
> On Tuesday, 10 November 2015 at 00:01:19 UTC, Idan Arye wrote:
>> Weird approach. Usually, when one wants to use an interpreted language as an host to a compiled language, the strategy is precompile the compiled language's code and load it as extensions in the interpreted code.
>
> That's what dmdinline does.

From the examples, it seems like it doesn't. It seems like it's compiling D code on the fly, rather than loading pre-compiled libraries as R extensions.

On Monday, 9 November 2015 at 19:31:14 UTC, Márcio Martins wrote:
> I have been running some MCMC simulations in Python and it's hard to cope with how unbelievably slow it is.
> Takes me almost a minute to run a few hundred thousand samples on my laptop whereas I can run the same simulation with a million samples in under 100ms, on my phone with JavaScript on a browser.
>
> Then, you spend a minute waiting for the simulation to finish, to find out you had an error in your report code that would have been easily caught with static typing. So annoying...
>
> Is anyone doing similar stuff with D? Unfortunately, I couldn't find any plotting libraries nor MATLAB-like numerical/stats libs in dub.
>
> This seems like another area where D could easily pick up momentum with RDMD and perhaps an integration with Jupyter which is becoming very very popular.

To give D a try in this field is definitely a good idea. It will pay in the long run, I believe. Where D lacks libraries and the like, there's always the option to interface to C (and C++ to a certain extent).

On Tuesday, 10 November 2015 at 10:13:46 UTC, Russel Winder wrote:
> On Mon, 2015-11-09 at 19:31 +0000, Márcio Martins via Digitalmars-d wrote:
>> I have been running some MCMC simulations in Python and it's hard
>> to cope with how unbelievably slow it is.
>> Takes me almost a minute to run a few hundred thousand samples on
>> my laptop whereas I can run the same simulation with a million
>> samples in under 100ms, on my phone with JavaScript on a browser.
>
> Are you using NumPy with Python, or pure Python? In either case you will be better served by profiling you code to find out whoch is actually the performance bottleneck and then doing one of:
>
> 1. Use Numba to JIT the Python and achieve native code speed. 2. Replace the Python code with D code using either CFFI or PyD. 3. Replace the Python code with Chapel code using either CFFI or PyChapel.
>
> You could also use C++ or Rust via CFFI, but I have little experience using Rust with Python and am trying to avoid C++.
>
>> Then, you spend a minute waiting for the simulation to finish, to find out you had an error in your report code that would have been easily caught with static typing. So annoying...
>> 
>> Is anyone doing similar stuff with D? Unfortunately, I couldn't find any plotting libraries nor MATLAB-like numerical/stats libs in dub.
>
> No need if you leave the coordination code in Python and then use Matplotlib, etc. Just put the computational expensive code into Chapel or D, leave the rest of your code in Python.
>
>> This seems like another area where D could easily pick up momentum with RDMD and perhaps an integration with Jupyter which is becoming very very popular.
>
> Jupyter (née IPython) is only popular in one workflow, creting papers for people to read and play with the executable code fragments. This is a big area but only one of many.

Numba worked great! I had never heard of it before, but seems like an elegant enough and practical solution. Still slightly slower than JavaScript but fast enough for prototyping.

No wonder Python is so popular despite being so slow - everything just works and feels elegant with no frustration. From not even having Python installed to having everything setup with Miniconda and having my code running took under 10 minutes and the only thing that wasn't smooth was really the speed, which was really frustrating a the time, but in a way was to be expected, and now was also elegantly fixed with Numba. No hacks, no platform issues, just silky smoothness :)

Thanks!