Hi all,

I'm needing to do a bit of ML soon for a project I'm involved in and I can't find any ML libs for D. Most of the libs available are for python (sci-kit learn, TensorFlow, etc.). I originally looked at using TensorFlow's C API but it doesn't include the actual ML stuff, just the tensor things. Could anybody help me out here?

Any recommendations? I was thinking of using RPC to talk to TensorFlow in python, could there be any bad sides to this? I haven't managed to get RPC to talk to python form D in the past

Thanks in advance

On Tuesday, 25 May 2021 at 00:20:26 UTC, Tim wrote:

I haven’t tried it, but you might want to take a look at the Netflix Vector Flow library, which is written in D. https://github.com/Netflix/vectorflow

—Jon

On Tuesday, 25 May 2021 at 00:44:15 UTC, Jon Degenhardt wrote:

That looks really interesting, thanks! Not sure yet whether a NN would be what I need, but I'll definitely look into this more

On Tuesday, 25 May 2021 at 00:20:26 UTC, Tim wrote:

Grain is a deep learning framework for D.

https://github.com/ShigekiKarita/grain

On Tuesday, 25 May 2021 at 01:04:19 UTC, jmh530 wrote:

Also, there is grain21, and tfd: tensorflow for D2.

But seriously, all these ML libraries in Dlang do not have many needed features you can easily find in Python ML libraries. If we step back: when those big companies like Google (TensorFlow etc.), Facebook (PyTorch etc.), Amazon (MxNet etc.), M$ (CNTK etc.) have already invested billions of dollars in either building these libraries directly or sponsoring them, which have been used by millions of users worldwide, why we want to build & use yet another one just in D?

Few D users also means you have high probability to hit bugs in some very basic stuff, e.g. bug in grain's Adam optimizer was only fixed 24 days ago 3.

D is already late for the ML game, maybe the best thing to do is concentrate the limited resource of the D communality on wrapping other mature libraries, e.g. tfd.

Given the current situation, on the practical side I'm using D to pre-processing data, and feed them into Python's ML libraries.

To communicate between D & Python:

-- grpc-dlang cannot talk to Python 4, as you have noted, and this issue has been around for half year now. (As a workaround, I added a Python pyd client talk to D server 8)

-- apache-thrift has build failure 5, which I just send a PR 6 to fix it today.

(again, I'm not sure how many real users these 2 libraries have, they are broken at some very basic stuff, not sure how other people can use them)

-- pyd: this is what I'm using right now. There are some rough edges 7, but at least I can get the basic stuff working finally, i.e. D & Python can talk.

Anyway, it's not easy, good luck!

On Tuesday, 25 May 2021 at 23:37:24 UTC, mw wrote:

links:

1: https://github.com/ShigekiKarita/grain2
2: https://github.com/ShigekiKarita/tfd
3: https://github.com/ShigekiKarita/grain/pull/15
4: https://github.com/huntlabs/grpc-dlang/issues/15
5: https://github.com/D-Programming-Deimos/openssl/issues/63
6: https://github.com/apache/thrift/pull/2397
7: https://github.com/ariovistus/pyd/issues
8: https://github.com/huntlabs/grpc-dlang/pull/19