Hi all,

I have written an article about writing Julia style multiple dispatch code in D (https://github.com/dataPulverizer/dispatch-it-like-julia). I am hoping that it is appropriate for the D blog.

Reviews please.

Many Thanks!

On 8/24/17 9:10 AM, data pulverizer wrote:
> Hi all,
> 
> I have written an article about writing Julia style multiple dispatch code in D (https://github.com/dataPulverizer/dispatch-it-like-julia). I am hoping that it is appropriate for the D blog.
> 
> Reviews please.
> 
> Many Thanks!

Very interesting!

I have a couple suggestions: for newbies like me, it would be nice to include a short explanation of multiple dispatch, and maybe a link to a longer description. Also it wouldn't hurt to include a short excerpt of how this code would look in Julia, along with the D examples.

Finally, I wonder if Discrete and Continuous couldn't be user-defined attributes.

On 08/24/2017 09:10 AM, data pulverizer wrote:
> Hi all,
>
> I have written an article about writing Julia style multiple dispatch
> code in D (https://github.com/dataPulverizer/dispatch-it-like-julia). I
> am hoping that it is appropriate for the D blog.
>
> Reviews please.
>
> Many Thanks!

This works only with compile-time dispatch, right? Does Julia support dynamic multiple dispatch?

In any case, Jean-Louis Leroy did some magic recently to support multiple dynamic dispatch in D. :)

  http://forum.dlang.org/post/cigbfrgipbokyetskypk@forum.dlang.org

Ali

On Thursday, 24 August 2017 at 16:41:54 UTC, David Gileadi wrote:
>
> Very interesting!

Thank you

>
> I have a couple suggestions: for newbies like me, it would be nice to include a short explanation of multiple dispatch, and maybe a link to a longer description.

Wikipedia's description of multiple dispatch is pretty good, I'll include it and add a short description.

> ... Also it wouldn't hurt to include a short excerpt of how this code would look in Julia, along with the D examples.

I included a link to Julia package as an example but I could add a short snippet as illustration.

> Finally, I wonder if Discrete and Continuous couldn't be user-defined attributes.

They could but the code closely models Julia's typing system, but UDAs could add good usability to this style of programming as well.

On Thursday, 24 August 2017 at 17:01:38 UTC, Ali Çehreli wrote:
> This works only with compile-time dispatch, right?

Yes

> ... Does Julia support dynamic multiple dispatch?

Okay Julia is my second favourite language next to D and one of it's cool features is that even though it is a dynamic programming language, methods are compiled for specific call signatures on first use of the function. So officially Julia does dynamic dispatch, but it pre-compiles function signatures.

>
> In any case, Jean-Louis Leroy did some magic recently to support multiple dynamic dispatch in D. :)
>
> http://forum.dlang.org/post/cigbfrgipbokyetskypk@forum.dlang.org

I haven't seen this. I'll have to get back to you when I have read it.

Thanks

On Thursday, 24 August 2017 at 16:10:32 UTC, data pulverizer wrote:
> Hi all,
>
> I have written an article about writing Julia style multiple dispatch code in D (https://github.com/dataPulverizer/dispatch-it-like-julia). I am hoping that it is appropriate for the D blog.
>
> Reviews please.
>
> Many Thanks!

I think at one point I had actually suggested that dstats or something be re-written in a Julia-like way (before I realized how much work that would be!). It looks very pretty. Nevertheless, you might be re-inventing the wheel a bit if you want to build a whole library in this style. My recommendation would be to write a front-end for the dstats.distrib and dstats.random submodules in this style. That way you won't need to re-write all the functions, you can just call ones from dstats that have already been tested.

More generally, I prefer the structs because they don't rely on the garbage collector, but the class/interface version is prettier. Atila's concepts library has implements, which you might find helpful. I have gently nudged him to work on something that also can tell if a type is a subtype of another type more generally (like a struct being a subtype of another struct). I think this would really be a good use case for that functionality.

On Thursday, 24 August 2017 at 17:20:20 UTC, data pulverizer wrote:
>>
>> In any case, Jean-Louis Leroy did some magic recently to support multiple dynamic dispatch in D. :)
>>
>> http://forum.dlang.org/post/cigbfrgipbokyetskypk@forum.dlang.org
>
> I haven't seen this. I'll have to get back to you when I have read it.
>

I wouldn't expect it to be that useful for univariate distributions as you wouldn't have much reason to have a different implmentation at run-time. However, it might be useful with multivariate distributions for the same reason that you might want to specialize matrix math by the size of the matrix.

On Thursday, 24 August 2017 at 18:16:21 UTC, jmh530 wrote:
> I think at one point I had actually suggested that dstats or something be re-written in a Julia-like way (before I realized how much work that would be!). It looks very pretty.

Thanks. I think most of that is down to D's nice syntax and how it easily and clearly emulates Julia. I think that there is certainly more to say on this especially around strategies of how represent concrete types. David Gileadi's point about UDAs could add an interesting spin on things, and Ali's point on dynamic dispatching.

> Nevertheless, you might be re-inventing the wheel a bit if you want to build a whole library in this style.

True. I have found a couple of projects that fresh targets that I am working on and writing code in this style.

> My recommendation would be to write a front-end for the dstats.distrib and dstats.random submodules in this style. That way you won't need to re-write all the functions, you can just call ones from dstats that have already been tested.

True code reuse is important, especially when you want to get something working quickly.

> More generally, I prefer the structs because they don't rely on the garbage collector, but the class/interface version is prettier.

Aha! I was wandering why I see people avoid classes even when using them is clearly the best way to represent their objects. For some reason it never occurred to me that they where just trying to avoid the GC. I just thought they didn't want to use reference objects.

> Atila's concepts library has implements, which you might find helpful. I have gently nudged him to work on something that also can tell if a type is a subtype of another type more generally (like a struct being a subtype of another struct). I think this would really be a good use case for that functionality.

I was thinking about this article for some time, it was Atila's article (https://atilanevesoncode.wordpress.com/2017/08/23/on-the-novelty-factor-of-compile-time-duck-typing/) that was a trigger for me writing it.

On Thursday, 24 August 2017 at 20:11:32 UTC, data pulverizer wrote:
>
> Thanks. I think most of that is down to D's nice syntax and how it easily and clearly emulates Julia. I think that there is certainly more to say on this especially around strategies of how represent concrete types. David Gileadi's point about UDAs could add an interesting spin on things, and Ali's point on dynamic dispatching.
>

On UDAs, at least in the current implementation, I think that the actual issue you are trying to address is to force the type in the distribution to be convertible to double in the continuous case and convertible to long in the discrete case. All things considered, that can be implemented with template constraints, as in

class Gamma(T):
    if(isFloatingPoint!T)
{
    immutable(T) shape;
    immutable(T) scale;
    this(T shape, T scale)
    {
        this.shape = shape;
        this.scale = scale;
    }
}

though you could probably take a more abstract approach. (I'm also not 100% on having immutable member variables). Also, density's signature could then avoid the template constraint.

auto density(D: Gamma!T, U : T, T)(D d, U x)

Even better, if you're calling the dstats functions, you could re-write density as something like

auto pdf(D: Dist!T, U : T, Dist, T)(U x, D d) {
    mixin("return x." ~ lookupdstatdensity!Dist ~ "(" ~ stringmembersofd ~ ")";
}

and create a lookupdstatdensity function that returns a string of the relevant dstats function at compile-time (and a function returning a string of the members of d) (I also would re-name density to pdf and switch the order of x and d). This would probably be the most DRY approach.

On Ali's point on dynamic dispatching, Julia is a scripting language with a JIT compiler. So if you call a function with some types known at compile time and the overload exists, it will compile the correct version of the function for the relevant types. It will then cache that so that if you need it again you don't pay any additional cost. So it's similar to what you're doing on that respect. However, there is a runtime dispatch component that would take something like openmethods to implement, I think.

On Thursday, 24 August 2017 at 21:13:10 UTC, jmh530 wrote:
> On UDAs, at least in the current implementation, I think that the actual issue you are trying to address is to force the type in the distribution to be convertible to double in the continuous case and convertible to long in the discrete case. All things considered, that can be implemented with template constraints, as in
>
> class Gamma(T):
>     if(isFloatingPoint!T)
> {
>     immutable(T) shape;
>     immutable(T) scale;
>     this(T shape, T scale)
>     {
>         this.shape = shape;
>         this.scale = scale;
>     }
> }
>

Okay, I admit that I try to avoid using template constraints whenever I can - i.e. unless the code breaks! It's a habit I am trying to break. In reality I will have to have them everywhere and the code will end up looking much less pretty, in reality, I know I'll probably need one or two catchall functions that look something like this

```
double density(D: UnivariateDistribution!Discrete, U = getVariateType!D, T = GetDistributionParameterType!D)(D d, U x)
if(!is(D == Poisson!T))
{
    assert(false, "density function unimplemented for this distribution: " ~ D.stringof);
}

double density(D: UnivariateDistribution!Continuous, U = getVariateType!D, T = GetDistributionParameterType!D)(D d, U x)
if(!is(D == Gamma!T) && !is(D == Gaussian!T) && !is(D == Uniform!T) && !is(D == Exponential!T))
{
    assert(false, "density function unimplemented for this distribution: " ~ D.stringof);
}

```

She's not so pretty anymore captain! This is why some time ago I suggested introducing the Union keyword that Julia has https://forum.dlang.org/post/lkcmqlpsdcopfebwgikj@forum.dlang.org

> though you could probably take a more abstract approach. (I'm also not 100% on having immutable member variables).

I am 100% sure that I want either the instantiated distribution object to be immutable, or the parameters to be immutable once instantiated. Its a safety feature that I don't see a need for mutable distribution objects. Once the parameters change, its not the same distribution. Ideally I want to be able to say

immutable class(T...){...}

that this class can only create immutable objects without having to write immutable everywhere and or a UDA, but making every member immutable accomplishes the same thing.

> Also, density's signature could then avoid the template constraint.
>
> auto density(D: Gamma!T, U : T, T)(D d, U x)

Sorry U is not T, T is the type of the parameters, U is the type of the variate.

> Even better, if you're calling the dstats functions, you could re-write density as something like
>
> auto pdf(D: Dist!T, U : T, Dist, T)(U x, D d) {
>     mixin("return x." ~ lookupdstatdensity!Dist ~ "(" ~ stringmembersofd ~ ")";
> }
>
> and create a lookupdstatdensity function that returns a string of the relevant dstats function at compile-time (and a function returning a string of the members of d) (I also would re-name density to pdf and switch the order of x and d). This would probably be the most DRY approach.

Sounds like a reasonable approach, though I haven't looked at the dstats package in great detail.

> Julia is a scripting language with a JIT compiler. So if you call a function with some types known at compile time and the overload exists, it will compile the correct version of the function for the relevant types.

Yes, I guess you could say that Julia is an interactive compiler, where you can create new compiled types and methods in the same session.

> So it's similar to what you're doing on that respect. However, there is a runtime dispatch component that would take something like openmethods to implement, I think.

I find OOP-polymorphic types ultimately unsatisfying, but I don't know of anyway to write, compile and load a D script with new types and methods on the fly into the same session.