On Monday, 9 November 2015 at 19:31:14 UTC, Márcio Martins wrote:
> I have been running some MCMC simulations in Python and it's hard to cope with how unbelievably slow it is.
> Takes me almost a minute to run a few hundred thousand samples on my laptop whereas I can run the same simulation with a million samples in under 100ms, on my phone with JavaScript on a browser.
>
> Then, you spend a minute waiting for the simulation to finish, to find out you had an error in your report code that would have been easily caught with static typing. So annoying...

No matter how interested I am in Bayesian statistics, I would think that an MCMC library is relatively lower in importance than a number of other libraries.

I've written some Gibbs samplers in Matlab and Python. They are quite slow in those languages, but I didn't notice that the Python code was more than 2X or so slower than the Matlab code, and I believe that was almost entirely due to Matlab using Intel MKL and Numpy using a slightly less efficient implementation.

While I learned a lot about MCMC by writing my own Gibbs samplers, I don't write them much anymore. I make more use of MC Stan, which can be called from Python with PyStan (maybe easier if you're on Linux than Windows, I tend to use rstan more which works easily with Windows). PyMC is another option that I've heard good things about, but I haven't tried it.

I think the simplest way forward would be something like wrappers to functionality in other languages. With PyD, it shouldn't be inconceivable to have a wrapper to PyMC. Alternately, MC Stan is written in C++ and has interfaces to a number of languages. Being able to call Stan from D would be cool, especially since I don't even think there's a C++ interface yet (you have to use the command line or R or whatever). I have essentially no idea how to do that.

> Is anyone doing similar stuff with D? Unfortunately, I couldn't find any plotting libraries nor MATLAB-like numerical/stats libs in dub.

My attitude is the more the better.

> This seems like another area where D could easily pick up momentum with RDMD and perhaps an integration with Jupyter which is becoming very very popular.

That would be interesting, but I'm not sure how high a priority it is.

On Tuesday, 10 November 2015 at 10:18:14 UTC, Idan Arye wrote:
>> That's what dmdinline does.
>
> From the examples, it seems like it doesn't. It seems like it's compiling D code on the fly, rather than loading pre-compiled libraries as R extensions.

Okay, I see what you're saying. You are correct that the examples look like that. That's because it is common in the R community to write your C/C++ code inline. It becomes an interactive process, and works good for debugging.

You can take the same D functions, insert them in a D file inside an R package, include a Makefile, and your D functions become part of that package. They'll be compiled when you install the package. That way you only compile once.

On Monday, 9 November 2015 at 19:31:14 UTC, Márcio Martins wrote:
> I have been running some MCMC simulations in Python and it's hard to cope with how unbelievably slow it is.
> Takes me almost a minute to run a few hundred thousand samples on my laptop whereas I can run the same simulation with a million samples in under 100ms, on my phone with JavaScript on a browser.
>
> Then, you spend a minute waiting for the simulation to finish, to find out you had an error in your report code that would have been easily caught with static typing. So annoying...
>
> Is anyone doing similar stuff with D? Unfortunately, I couldn't find any plotting libraries nor MATLAB-like numerical/stats libs in dub.

dlangscience has some energy and people behind it.  John Colvin is heavily involved with it, but it's a joint project, and there are others too.  He wrote a draft white paper really thinking through the best design approach, and that will pay dividends over time, but in the meantime it's a central point for different scientific computing libraries.

plotting is a work in progress, I think.  there are some options.  for my stuff, it's not particularly clever so I use the D bindings to mathgl (a nice and simple C library), but depending on what you want to do, other choices may be more suitable.

it's pretty easy to call python libraries from D, and I have done that initially for plotting using bokeh.  the only problem was for callbacks that then meant potentially dealing with javascript as well, and I drew the line at three languages for such a simple thing.

> This seems like another area where D could easily pick up momentum with RDMD and perhaps an integration with Jupyter which is becoming very very popular.

Actually John Colvin has written an extension called pydmagic that allows you to write D code within a Jupyter notebook - it integrates with PyD so you can call D from python and call python from D (even embedded as a string if you like).  It's not yet highly-polished, but it works, and I have used it to get work done.

On Wednesday, 11 November 2015 at 03:29:56 UTC, Laeeth Isharc wrote:
>
> plotting is a work in progress, I think.  there are some options.
>  for my stuff, it's not particularly clever so I use the D bindings to mathgl (a nice and simple C library), but depending on what you want to do, other choices may be more suitable.
>

I just came across
https://github.com/BlackEdder/ggplotd
which looks promising.

On Monday, 9 November 2015 at 21:05:35 UTC, bachmeier wrote:
> On Monday, 9 November 2015 at 20:30:49 UTC, Gerald Jansen wrote:
>> On Monday, 9 November 2015 at 19:31:14 UTC, Márcio Martins wrote:
>>> I have been running some MCMC simulations in Python ...
>>>
>>> Is anyone doing similar stuff with D? Unfortunately, I couldn't find any plotting libraries nor MATLAB-like numerical/stats libs in dub.
>>>
>>> This seems like another area where D could easily pick up momentum with RDMD and perhaps an integration with Jupyter which is becoming very very popular.
>>
>> see http://dlangscience.github.io/
>
> And here is the gitter discussion site:
>
> https://gitter.im/DlangScience/public
>
> I've got this project
>
> https://bitbucket.org/bachmeil/dmdinline2
>
> to embed D inside R on Linux. Unfortunately the documentation isn't good. I'm currently working on going in the other direction, embedding R inside D. There are, of course, many good MCMC options in R that you could call from your D code.

Hi bachmeier.

Hope you're well.

What's the current status of calling R from D and D from R?  A friend who is global head of derivatives research in London for a bank was asking me as he is receptive to exploring alternatives.  It's for research not production so rough around the edges is acceptable - provided one knows what one is dealing with beforehand.


Thanks.


Laeeth.

On Friday, 5 February 2016 at 20:13:42 UTC, Laeeth Isharc wrote:
> On Monday, 9 November 2015 at 21:05:35 UTC, bachmeier wrote:
>> On Monday, 9 November 2015 at 20:30:49 UTC, Gerald Jansen wrote:
>>> On Monday, 9 November 2015 at 19:31:14 UTC, Márcio Martins wrote:
>>>> I have been running some MCMC simulations in Python ...
>>>>
>>>> Is anyone doing similar stuff with D? Unfortunately, I couldn't find any plotting libraries nor MATLAB-like numerical/stats libs in dub.
>>>>
>>>> This seems like another area where D could easily pick up momentum with RDMD and perhaps an integration with Jupyter which is becoming very very popular.
>>>
>>> see http://dlangscience.github.io/
>>
>> And here is the gitter discussion site:
>>
>> https://gitter.im/DlangScience/public
>>
>> I've got this project
>>
>> https://bitbucket.org/bachmeil/dmdinline2
>>
>> to embed D inside R on Linux. Unfortunately the documentation isn't good. I'm currently working on going in the other direction, embedding R inside D. There are, of course, many good MCMC options in R that you could call from your D code.
>
> Hi bachmeier.
>
> Hope you're well.
>
> What's the current status of calling R from D and D from R?  A friend who is global head of derivatives research in London for a bank was asking me as he is receptive to exploring alternatives.  It's for research not production so rough around the edges is acceptable - provided one knows what one is dealing with beforehand.
>
>
> Thanks.
>
>
> Laeeth.

On Linux, works fine in either direction, though my recent efforts have been embedding R inside D, so the underlying library for that is much advanced over the older library for calling D from R. I've set up a simple html page here:

http://lancebachmeier.com/rdlang/

I've gotten most of it working on Windows, but it's been slow, since I don't use or understand that OS for development. I expect to have it working fully in the next couple weeks. It would probably be easy for someone with Windows experience to get it working now.

Send an email to the address at this page http://www.k-state.edu/economics/staff/bios/bachmeier.html if you have any questions.

On Friday, 5 February 2016 at 20:13:42 UTC, Laeeth Isharc wrote:

> Hi bachmeier.
>
> Hope you're well.
>
> What's the current status of calling R from D and D from R?  A friend who is global head of derivatives research in London for a bank was asking me as he is receptive to exploring alternatives.  It's for research not production so rough around the edges is acceptable - provided one knows what one is dealing with beforehand.
>
>
> Thanks.
>
>
> Laeeth.

Something I didn't emphasize before is that embedding R inside D does more than open the door to mixing R and D code together in one program. We now have an interface available in D to any compiled C, C++, or Fortran library for which there is an R package. You can call those libraries from D, no glue code required and using the existing documentation for the R package, yet it's fully efficient because you're passing a pointer to the data from D into the underlying library. R code only gets in the way (i.e., slows your program down) if you want to run functions written in R.