Bare-bones at the moment, but hopefully of interest to some people

https://github.com/DlangScience/PydMagic

This allows you to write D extensions to python, as inline cells in an ipython/jupyter notebook.

On Sun, 2015-06-14 at 15:51 +0000, John Colvin via Digitalmars-d -announce wrote:
> Bare-bones at the moment, but hopefully of interest to some people
> 
> https://github.com/DlangScience/PydMagic
> 
> This allows you to write D extensions to python, as inline cells in an ipython/jupyter notebook.

I am doing a talk on using C++, D and Chapel in Python to the PyData London 2015 people, many I am sure will want to use IPython/Jupyter. If you can guide me to a short demo that has maximum impact for the least risk [*], I can certainly give it an airing.

[*] As ever with my live demos, the demo gods always want a say ;-)

-- 
Russel. ============================================================================= Dr Russel Winder      t: +44 20 7585 2200   voip: sip:russel.winder@ekiga.net 41 Buckmaster Road    m: +44 7770 465 077   xmpp: russel@winder.org.uk London SW11 1EN, UK   w: www.russel.org.uk  skype: russel_winder

On Sunday, 14 June 2015 at 16:14:58 UTC, Russel Winder wrote:
> On Sun, 2015-06-14 at 15:51 +0000, John Colvin via Digitalmars-d -announce wrote:
>> Bare-bones at the moment, but hopefully of interest to some people
>> 
>> https://github.com/DlangScience/PydMagic
>> 
>> This allows you to write D extensions to python, as inline cells in an ipython/jupyter notebook.
>
> I am doing a talk on using C++, D and Chapel in Python to the PyData London 2015 people, many I am sure will want to use IPython/Jupyter. If you can guide me to a short demo that has maximum impact for the least risk [*], I can certainly give it an airing.
>
> [*] As ever with my live demos, the demo gods always want a say ;-)

Ok, see examples/test.ipynb it's not amazing but it's a start.

On Sunday, 14 June 2015 at 15:51:37 UTC, John Colvin wrote:
> Bare-bones at the moment, but hopefully of interest to some people
>
> https://github.com/DlangScience/PydMagic
>
> This allows you to write D extensions to python, as inline cells in an ipython/jupyter notebook.

This was just what I was looking for - great work, and I think this will be very useful.



Laeeth

On Monday, 15 June 2015 at 05:11:17 UTC, Laeeth Isharc wrote:
> On Sunday, 14 June 2015 at 15:51:37 UTC, John Colvin wrote:
>> Bare-bones at the moment, but hopefully of interest to some people
>>
>> https://github.com/DlangScience/PydMagic
>>
>> This allows you to write D extensions to python, as inline cells in an ipython/jupyter notebook.
>
> This was just what I was looking for - great work, and I think this will be very useful.
>
>
>
> Laeeth

It's really only a beginning. Pyd's API is pretty clunky feeling by modern D standards, I hope to extend @pdef to automatically work for most language constructs, amongst other sugar.

I also aim to factor it out in to a library that doesn't depend on ipython, so people can just get a nicer Pyd. Ultimately maybe it could replace the upstream API.

Any contributions are very much welcome, it's a fun thing to work on but I'm crazily busy at the moment. Anyone who's interested, get in touch :)

On Monday, 15 June 2015 at 06:51:44 UTC, John Colvin wrote:
> It's really only a beginning. Pyd's API is pretty clunky feeling by modern D standards, I hope to extend @pdef to automatically work for most language constructs, amongst other sugar.

Yes - I had noticed same, but don't yet have the experience (and don't have available time for now) to do much about it.  Was looking at Facebook torch to see how that fitted with bokeh and inotebook, but glad you actually did something to make it happen.

> Any contributions are very much welcome, it's a fun thing to work on but I'm crazily busy at the moment. Anyone who's interested, get in touch :)

Have some other constraints but will get in touch later if I can.

I hope this wasn't premature to submit:
http://www.reddit.com/r/programming/comments/39xmpj/run_inline_d_dlang_code_for_python_extensions_in/

On Monday, 15 June 2015 at 17:11:26 UTC, Laeeth Isharc wrote:
> On Monday, 15 June 2015 at 06:51:44 UTC, John Colvin wrote:
>> [...]
>
> Yes - I had noticed same, but don't yet have the experience (and don't have available time for now) to do much about it.  Was looking at Facebook torch to see how that fitted with bokeh and inotebook, but glad you actually did something to make it happen.

I haven't looked at torch much, is it up to the hype?

>> [...]
>
> Have some other constraints but will get in touch later if I can.
>
> I hope this wasn't premature to submit:
> http://www.reddit.com/r/programming/comments/39xmpj/run_inline_d_dlang_code_for_python_extensions_in/

Perhaps a little early but I'm sure it won't do any harm.

On Monday, 15 June 2015 at 17:28:52 UTC, John Colvin wrote:
> On Monday, 15 June 2015 at 17:11:26 UTC, Laeeth Isharc wrote:
>> On Monday, 15 June 2015 at 06:51:44 UTC, John Colvin wrote:
>>> [...]
>>
>> Yes - I had noticed same, but don't yet have the experience (and don't have available time for now) to do much about it.  Was looking at Facebook torch to see how that fitted with bokeh and inotebook, but glad you actually did something to make it happen.
>
> I haven't looked at torch much, is it up to the hype?

To be direct: I don't know, as I haven't used it.  I was just interested in seeing how it fitted with ipython and bokeh in particular.

I liked your talk at dconf.  Looks like progress is being made on planting the seeds of a matrix implementation and computation library on top.  Having a way to use D code to explore data within an approach of rapid iteration will be useful too, and I think your work is a great start on this (seems like its value as it is is very much greater than you modestly suggest, and also I am amazed by how short the code is).

At the moment I am using a combination of the python interface to Bokeh and D code to do the work.  It would be great to port the bokeh server side stuff to D - I have made a start, but I haven't the space for a concerted effort.  The funny thing is there is nothing to the server side - all you do is have a shallow layer that translates the cumulative state from API calls to JSON; the hard work of drawing stuff is done in Javascript on the client.

Whether it's Bokeh or not, having some kind of visualisation solution that fits well with iteratively exploring data in D will be nice.  (What you have done means it's already enough to be very usable, but there may be benefits to moving the charting itself to D).

Perhaps not for scientific applications, but for finance: reporting is also something that is an important and rising theme in use cases.  Not just for accounting, but for example to understand portfolio risk in a richer way.  You have for a long time been able to hook up reportlab with D code using PyD, but this element of rapid iteration was not so easy - small frictions have large consequences when you don't have a big team.

So in any case, I really appreciate your work.

One more thing - any thoughts on best spreadsheet like interface either purely on browser or within ipython to display results and allow user to enter parameters?  My friend, Giles Thomas, wrote Resolver but that's too big and aimed at something different, and I don't know if it is being maintained properly.



Laeeth

On 6/15/15 10:28 AM, John Colvin wrote:
> On Monday, 15 June 2015 at 17:11:26 UTC, Laeeth Isharc wrote:
>> On Monday, 15 June 2015 at 06:51:44 UTC, John Colvin wrote:
>>> [...]
>>
>> Yes - I had noticed same, but don't yet have the experience (and don't
>> have available time for now) to do much about it. Was looking at
>> Facebook torch to see how that fitted with bokeh and inotebook, but
>> glad you actually did something to make it happen.
>
> I haven't looked at torch much, is it up to the hype?

Yann LeCun recommended it; I think that's a good endorsement. -- Andrei

On Monday, 15 June 2015 at 19:35:16 UTC, Laeeth Isharc wrote:
> On Monday, 15 June 2015 at 17:28:52 UTC, John Colvin wrote:
>> On Monday, 15 June 2015 at 17:11:26 UTC, Laeeth Isharc wrote:
>>> On Monday, 15 June 2015 at 06:51:44 UTC, John Colvin wrote:
>>>> [...]
>>>
>>> Yes - I had noticed same, but don't yet have the experience (and don't have available time for now) to do much about it.  Was looking at Facebook torch to see how that fitted with bokeh and inotebook, but glad you actually did something to make it happen.
>>
>> I haven't looked at torch much, is it up to the hype?
>
> To be direct: I don't know, as I haven't used it.  I was just interested in seeing how it fitted with ipython and bokeh in particular.
>
> I liked your talk at dconf.

Thank you :)

> Looks like progress is being made on planting the seeds of a matrix implementation and computation library on top.

Ilya Yaroshenko is doing great work here.

> Having a way to use D code to explore data within an approach of rapid iteration will be useful too, and I think your work is a great start on this (seems like its value as it is is very much greater than you modestly suggest, and also I am amazed by how short the code is).

Well, as i briefly mentioned in my talk, it's really very easy to do, Jupyter/IPython makes it simple. Of course, behind the scenes Pyd is doing the hard work.

> At the moment I am using a combination of the python interface to Bokeh and D code to do the work.  It would be great to port the bokeh server side stuff to D - I have made a start, but I haven't the space for a concerted effort.  The funny thing is there is nothing to the server side - all you do is have a shallow layer that translates the cumulative state from API calls to JSON; the hard work of drawing stuff is done in Javascript on the client.
>
> Whether it's Bokeh or not, having some kind of visualisation solution that fits well with iteratively exploring data in D will be nice.  (What you have done means it's already enough to be very usable, but there may be benefits to moving the charting itself to D).

Visualisation is one of those things that no-one seems to have got right, despite the huge efforts that have gone in to it. I have ideas about D-based visualisation and already my own basic 3-D plotting library for huge datasets, but for now the link to python is key to get quick and easy access to the latest tools that the scientific and finance world are creating.

> Perhaps not for scientific applications, but for finance: reporting is also something that is an important and rising theme in use cases.  Not just for accounting, but for example to understand portfolio risk in a richer way.  You have for a long time been able to hook up reportlab with D code using PyD, but this element of rapid iteration was not so easy - small frictions have large consequences when you don't have a big team.
>
> So in any case, I really appreciate your work.
>
> One more thing - any thoughts on best spreadsheet like interface either purely on browser or within ipython to display results and allow user to enter parameters?  My friend, Giles Thomas, wrote Resolver but that's too big and aimed at something different, and I don't know if it is being maintained properly.

There is http://nbviewer.ipython.org/gist/rossant/9463955 (see https://www.youtube.com/watch?v=_JZFSFR6Yeo for demo) which looks like a good starting point, but it's very basic.

If https://github.com/quantopian/qgrid supported editing (perhaps it does already, but I can't see any mention of it) then that would probably be better.