On Monday, 26 October 2020 at 01:42:42 UTC, jmh530 wrote:
>
> I have no issue with calling libraries from other languages (particularly C) if it's something that is too much work or whatever to do myself. But I think that it's helpful to have a base level of functionality, akin to Numpy/Scipy, that a new person could come in to accomplish a lot.

Aren't numpy and scipy themselves largely built on "calling libraries from other languages"? Specifically, C and Fortran.

On Monday, 26 October 2020 at 01:55:46 UTC, Paul Backus wrote:
> On Monday, 26 October 2020 at 01:42:42 UTC, jmh530 wrote:
>>
>> I have no issue with calling libraries from other languages (particularly C) if it's something that is too much work or whatever to do myself. But I think that it's helpful to have a base level of functionality, akin to Numpy/Scipy, that a new person could come in to accomplish a lot.
>
> Aren't numpy and scipy themselves largely built on "calling libraries from other languages"? Specifically, C and Fortran.

Exactly. And R (actually S at that time) started as a glue language for libraries in those languages.

On Sunday, 25 October 2020 at 21:30:59 UTC, jmh530 wrote:
> [snip]
>
> I think, unfortunately, it is not always easy to communicate why these changes are important or valuable. But, while you weren't able to convince Atila of the value of the proposed features, I also don't think you were ignored either, at least in that case.
>
> That being said, I never understood Atila's argument about this feature as being a light version of Rust traits (I'm not aware of how Haskell's typeclasses work). A Rust trait is a list of functions that must be implemented by a type. However, you can also statically dispatch based on the trait (you can dynamically as well, but I imagine you would prefer not to be able to do that). You conceivably would be more interested in the static dispatch part of it. It's not really about a PackedUpperTriangularMatrix requiring specific functions to be a PackedUpperTriangularMatrix rather than a Slice. All it takes is the right iterator type. So it's more about how the specific type is specialized (giving a specific iterator to PackedUpperTriangularMatrix).
>
> There might be a way to create a feature that's not Rust traits that does what you want and is a more general feature than this type of template alias deduction.

Adding a little more...

The situation that this issue is trying to address is something like a template
T!(U!V)
that you want to be able to use like
W!V.

I can't help but think that concepts could help with this situation. Adapting the C++20 syntax, consider this simplest implementation:

template PackedUpperTriangularMatrix(T)
{
    concept PackedUpperTriangularMatrix = is(T: Slice!(StairsIterator!(U*, "-")), U);
}

Assuming the same functionality as in C++20, you could use this in a function as in
void foo(PackedUpperTriangularMatrix x) {}

However, if you then want to place any constraints on the U above, then you're a bit SOL.

To really get the functionality working, you would need a generic kind of concept, where the concept you are defining is itself generic.
As far as I can tell, you can't do this with C++20, but I would imagine the syntax adapted from above might be something like

template PackedUpperTriangularMatrix(T)
{
    concept PackedUpperTriangularMatrix(U) = is(T: Slice!(StairsIterator!(U*, "-")));
}

and would enable you to write the function as
void foo(T)(PackedUpperTriangularMatrix!T x) {}

In other words, if D had the ability to define concepts that are also generic themselves, then it would enable the functionality you want also.

On Saturday, 24 October 2020 at 16:43:45 UTC, James Blachly wrote:
> Aside / self-promotion:
> We use D extensively in our bioinformatics / computational biology program.
>
> Check out https://github.com/blachlylab/dhtslib/
>
> Also just published a HTS/NGS tool written in D:
>
> https://academic.oup.com/nargab/article/2/4/lqaa070/5917298
>
> I should probably do an `announce` forum post.
>
> Currently trying to decide whether to extend Magpie or roll our own (adding only features that are needed). I've also enjoyed using Mir ndslice in a couple of test projects, but as you know that is not really a dataframe.

Self-promoting as well :-)
I also started to use D in the domain of biophysics. Big computations are still done with our C++ code, but I'm translating old python pre- and posttreatment scripts into D, getting a nice speedup:

https://github.com/glis-glis/biophysics

Please note that all I know about D is from tour.dlang.org and I'm often rather lazy concerning comments (which obviously comes back biting me rather often when I don't understand my own code 2 months later...).

What is sometimes lacking is information and documentation of what exists for D. Here it states that D can work with GPUs:
https://dlang.org/areas-of-d-usage.html#gpu
but all you get is a link to a presentation of 2016.

I needed a way to calculate the principal component analysis. Mir can't do it, but I found out Lubeck can. So, I tried the Lubeck-example of the dlang tour, which didn't work. I was able to correct it, and did a pull-request so the example is now working, but most newcomer woud probably just say "Ok, it's broken, let's get back to Numpy".

On Saturday, 24 October 2020 at 11:05:48 UTC, bioinfornatics wrote:
> On Saturday, 24 October 2020 at 09:29:46 UTC, Russel Winder wrote:
>> On Fri, 2020-10-23 at 23:00 +0000, bioinfornatics via Digitalmars-d wrote: […]
>>> To me a scientific library need to be HPC oriented, able
>>> - to perform // computation on CPU or GPU
>>> - to use divide and conquer strategy in order to compute over
>>> multinode
>>> - to have dataframe features
>>> - to have scipy features
>>> A such library would be awesome as at these time python slowness
>>> become more and more important as data grow exponentially year
>>> after year
>>
>> Acting somewhat as "Devil's Advocate"…
>>
>> Why not just use Chapel https://chapel-lang.org/ – it is a programming language designed to run in parallel contexts and has an awful lot of the stuff other (invariable sequential, cf. C++, D, Rust) programming language have trouble providing.
>>
>> I am not sure Chapel has pandas style data frames explicitly but I'll bet something equivalent is already in there.
>
> Maybe, anyway since years D search the killer app. Really I thanks thisr area it is perfect for D.
> Data Business analysis is so important in this day in science, economy and other D could be a good choice.

D is already quite late for the party.

Besides Chapel, there is Julia, and even Java, F# and C# are getting better in this domain.

The ranges being discussed here have been made available in C# 8.

https://docs.microsoft.com/en-us/dotnet/csharp/tutorials/ranges-indexes

While support for working with Spark just went 1.0 this week,

https://dotnet.microsoft.com/apps/data/spark

D would do better to see how to interoperate with existing stuff.

On Tuesday, 27 October 2020 at 10:23:52 UTC, Paulo Pinto wrote:
> D is already quite late for the party.

Right, but I don't think being CPU centric will work out in that domain anyway. You need to use Vulkan and Metal, aim for future hardware and do it really well. The market is in the future, not here, right now.

On Friday, 23 October 2020 at 19:31:08 UTC, bioinfornatics wrote:
> As a researcher in BioInformatics I use a lot python numpy pandas and scipy. But I am bored by the slowness of python even with cpython code thanks to the GIL and un-optimized tail recursion.
>
> So I thinks really that D could play a big role in this field with MIR and dcompute.
>
> 1/ what is the state of Magpie which was a GSoC 2019:
>  - Mir Data Analysis and Processing Library
>
> 2/ does the scientific computing field is something that D language want to grow ?
>
> Thanks
>
> Best regards

Just ran across a Hacker News thread about pyston that relates to some of the discussion here [1]. It seems there's still a demand for alternatives to python for data science.

[1] https://news.ycombinator.com/item?id=24921790

On Thu, 2020-10-29 at 10:23 +0000, jmh530 via Digitalmars-d wrote:
> 
[…]
> Just ran across a Hacker News thread about pyston that relates to some of the discussion here [1]. It seems there's still a demand for alternatives to python for data science.
> 
> [1] https://news.ycombinator.com/item?id=24921790

I only quickly skimmed the blog page, so this is a first reaction. I shall read the material more carefully tomorrow and send an update.

1. People have been trying to make Python execute faster for 30 years. In the end everyone ends up just using CPython with any and all optimisations it can get in.

2. Python is slow, and fundamentally single threaded. Attempts to make Python multi-threaded seem to fall by the wayside. The micro-benchmarks seem to indicate Pyston is just a slightly faster Python and thus nothing really to write home about – yes even a headline figure of 20% is nothing to write home about!

3. If you want computational performance from Python code, you use C, C++(, or D) extensions. In particular you use NumPy. I would guess that almost all bioinformatics, astronomy, machine learning, AI, data science stuff uses NumPy. Python execution performance is irrelevant compared to NumPy code performance.

I am happy to be shown to be wrong, but I suspect not.

-- 
Russel.
===========================================
Dr Russel Winder      t: +44 20 7585 2200
41 Buckmaster Road    m: +44 7770 465 077
London SW11 1EN, UK   w: www.russel.org.uk

On Tuesday, 27 October 2020 at 10:23:52 UTC, Paulo Pinto wrote:
>
> D is already quite late for the party.


Is there a saying: better late than never :-)


I still think D has chance, if we really want to catch-up, to overthrown Python

On Friday, 30 October 2020 at 02:03:10 UTC, mw wrote:
> On Tuesday, 27 October 2020 at 10:23:52 UTC, Paulo Pinto wrote:
>>
>> D is already quite late for the party.
>
>
> Is there a saying: better late than never :-)
>
>
> I still think D has chance, if we really want to catch-up, to overthrown Python

Of course, this community have to make the effort to make it happen.