On Sat, 2020-10-24 at 11:05 +0000, bioinfornatics via Digitalmars-d wrote:

[…]

> Maybe, anyway since years D search the killer app. Really I
> thanks thisr area it is perfect for D.
> Data Business analysis is so important in this day in science,
> economy and other D could be a good choice.

I agree that D could be the replacement for Python for many scientific milieu: bioinformatics, astronomy, to name but two obvious ones. The issue though is that the Python language over NumPy and associated communities captured the moment years ago and many people contributed many extensions to a few libraries and packages.

Traditionally (as it were) bioinformatics and astronomy have emphasised exploration over computation, and often offloaded computation to C or C++ realised frameworks. This has reinforced prioritising code comprehension and evolution over computation speed, thus militating in favour of Python since the packages were there.

Whilst D could replace Python, the question is will it and the answer is determined by who would write the code. Sadly history tells us this will lead to a (very) long (divergent) thread and result in no-one actually doing anything. I would like to be proved wrong.

The possible upside is that all the major Python packages started as one or two people creating something that others then joined in with and turned into the de facto standard. Might this finally happen in the D community?

-- 
Russel.
===========================================
Dr Russel Winder      t: +44 20 7585 2200
41 Buckmaster Road    m: +44 7770 465 077
London SW11 1EN, UK   w: www.russel.org.uk

On Friday, 23 October 2020 at 19:31:08 UTC, bioinfornatics wrote:
> As a researcher in BioInformatics I use a lot python numpy pandas and scipy. But I am bored by the slowness of python even with cpython code thanks to the GIL and un-optimized tail recursion.
>
> So I thinks really that D could play a big role in this field with MIR and dcompute.
>
> 1/ what is the state of Magpie which was a GSoC 2019:
>  - Mir Data Analysis and Processing Library
>
> 2/ does the scientific computing field is something that D language want to grow ?
>
> Thanks
>
> Best regards

Magpie was another attempt to create Data Frame in D using the architecture patterns from Python/R. It has never been part of Mir infrastructure. DataFrame in D should be a little different from what people have in scripting languages. Otherwise, it is will be not good enough.

> 2/ does the scientific computing field is something that D language want to grow ?

I would like to say Yes. But the reality is that the answer is that D isn't going to grow. The sci related answer is that the members of DLF either ignore my requests or even reject related work for the compiler because they "don't see much of a difference"

The following work is what really will be good for Sci D and especially for DataFrame.

https://github.com/dlang/DIPs/blob/master/DIPs/other/DIP1023.md
https://github.com/dlang/dmd/pull/9778

On Saturday, 24 October 2020 at 12:08:00 UTC, Russel Winder wrote:
> On Sat, 2020-10-24 at 11:05 +0000, bioinfornatics via Digitalmars-d wrote:
>
> […]
>
>> Maybe, anyway since years D search the killer app. Really I
>> thanks thisr area it is perfect for D.
>> Data Business analysis is so important in this day in science,
>> economy and other D could be a good choice.
>
> I agree that D could be the replacement for Python for many scientific milieu: bioinformatics, astronomy, to name but two obvious ones. The issue though is that the Python language over NumPy and associated communities captured the moment years ago and many people contributed many extensions to a few libraries and packages.
>
> Traditionally (as it were) bioinformatics and astronomy have emphasised exploration over computation, and often offloaded computation to C or C++ realised frameworks. This has reinforced prioritising code comprehension and evolution over computation speed, thus militating in favour of Python since the packages were there.
>
> Whilst D could replace Python, the question is will it and the answer is determined by who would write the code. Sadly history tells us this will lead to a (very) long (divergent) thread and result in no-one actually doing anything. I would like to be proved wrong.
>
> The possible upside is that all the major Python packages started as one or two people creating something that others then joined in with and turned into the de facto standard. Might this finally happen in the D community?

Just expecting someone else is doing the work will very likely not happen at this point in time. But you can actually increase the chances it will happen in future.
My opinion: Neither a language feature X nor a specific library Y is missing at the moment but the community needs to do massive advertisements for the D Programming Language. The bigger the community will become, the more libraries will be created.

Therefore you can start here by advising D and its strengths at every channel which make sense.

Kind regards
Andre

On Friday, 23 October 2020 at 19:31:08 UTC, bioinfornatics wrote:
> As a researcher in BioInformatics I use a lot python numpy pandas and scipy. But I am bored by the slowness of python even with cpython code thanks to the GIL and un-optimized tail recursion.
>
> So I thinks really that D could play a big role in this field with MIR and dcompute.
>
> 1/ what is the state of Magpie which was a GSoC 2019:
>  - Mir Data Analysis and Processing Library
>
> 2/ does the scientific computing field is something that D language want to grow ?
>
> Thanks
>
> Best regards

I think the answer to questions like this is often money. You need to hire someone to write the code that does this. In the world of research this means a grant to fund it. If you think there is real merit in the D programming language in your field, the best thing to do is to make the case in form of a grant application to pay a researcher to write the necessary code. This is what languages like R, Python, and Julia do. They are flush with cash because people write grant applications for PhD students and researchers to build libraries for those languages.

On 10/23/20 3:31 PM, bioinfornatics wrote:
> As a researcher in BioInformatics I use a lot python numpy pandas and scipy. But I am bored by the slowness of python even with cpython code thanks to the GIL and un-optimized tail recursion.
> 
> So I thinks really that D could play a big role in this field with MIR and dcompute.
> 
> 1/ what is the state of Magpie which was a GSoC 2019:
>   - Mir Data Analysis and Processing Library
> 
> 2/ does the scientific computing field is something that D language want to grow ?
> 
> Thanks
> 
> Best regards

Aside / self-promotion:
We use D extensively in our bioinformatics / computational biology program.

Check out https://github.com/blachlylab/dhtslib/

Also just published a HTS/NGS tool written in D:

https://academic.oup.com/nargab/article/2/4/lqaa070/5917298

I should probably do an `announce` forum post.

Currently trying to decide whether to extend Magpie or roll our own (adding only features that are needed). I've also enjoyed using Mir ndslice in a couple of test projects, but as you know that is not really a dataframe.

On Friday, 23 October 2020 at 22:38:39 UTC, mw wrote:
> [snip] Build or wrap something like tensorflow, I think will need much more resource than the D community current have, also I'm not sure if it worth the effort.
>

https://github.com/ShigekiKarita/tfd

The author of that has some other useful libraries.

On Friday, 23 October 2020 at 19:31:08 UTC, bioinfornatics wrote:
> [snip]
>
> 2/ does the scientific computing field is something that D language want to grow ?
>
> Thanks
>
> Best regards

I'm certainly interested in it, but doing it well takes time.

On Saturday, 24 October 2020 at 12:26:21 UTC, 9il wrote:
> [snip]
>
> I would like to say Yes. But the reality is that the answer is that D isn't going to grow. The sci related answer is that the members of DLF either ignore my requests or even reject related work for the compiler because they "don't see much of a difference"
>
> The following work is what really will be good for Sci D and especially for DataFrame.
>
> https://github.com/dlang/DIPs/blob/master/DIPs/other/DIP1023.md
> https://github.com/dlang/dmd/pull/9778

I think, unfortunately, it is not always easy to communicate why these changes are important or valuable. But, while you weren't able to convince Atila of the value of the proposed features, I also don't think you were ignored either, at least in that case.

That being said, I never understood Atila's argument about this feature as being a light version of Rust traits (I'm not aware of how Haskell's typeclasses work). A Rust trait is a list of functions that must be implemented by a type. However, you can also statically dispatch based on the trait (you can dynamically as well, but I imagine you would prefer not to be able to do that). You conceivably would be more interested in the static dispatch part of it. It's not really about a PackedUpperTriangularMatrix requiring specific functions to be a PackedUpperTriangularMatrix rather than a Slice. All it takes is the right iterator type. So it's more about how the specific type is specialized (giving a specific iterator to PackedUpperTriangularMatrix).

There might be a way to create a feature that's not Rust traits that does what you want and is a more general feature than this type of template alias deduction.

On Sunday, 25 October 2020 at 20:30:58 UTC, jmh530 wrote:
> On Friday, 23 October 2020 at 19:31:08 UTC, bioinfornatics wrote:
>> [snip]
>>
>> 2/ does the scientific computing field is something that D language want to grow ?
>>
>> Thanks
>>
>> Best regards
>
> I'm certainly interested in it, but doing it well takes time.

Over time, I've come to three conclusions on this topic. I don't know that time is the issue.

1. This community seems to have NIH syndrome. A lot of users are averse to reuising the functionality provided by other languages. I find that downright weird given that one of the selling points of D is its ability to easily interoperate with other languages. It makes sense to *extend* existing projects in other languages using D. The idea of rewriting millions of lines of code for no benefit other than just saying it's written in D is obviously pointless, so there's no motivation to do it.

2. Scientific computing is a big field. In terms of things you'd need to be "complete", you'd have to write maybe ten times as much code as you would to have a complete web development offering. It also requires incredible amounts of expertise. Statistics, economics, physics, math, chemistry, biology, and on and on are all areas that individually require a great deal of specialized knowledge in addition to time. For some things, performance is the most important property, including use of the GPU. That's not simple.

3. D's syntax is okay, but it's not flexible enough to express eveything you need to work comfortably. A DSL or similar might be necessary.

On Sunday, 25 October 2020 at 23:30:10 UTC, bachmeier wrote:
> [snip]
>
> 1. This community seems to have NIH syndrome. A lot of users are averse to reuising the functionality provided by other languages. I find that downright weird given that one of the selling points of D is its ability to easily interoperate with other languages. It makes sense to *extend* existing projects in other languages using D. The idea of rewriting millions of lines of code for no benefit other than just saying it's written in D is obviously pointless, so there's no motivation to do it.
>
> 2. Scientific computing is a big field. In terms of things you'd need to be "complete", you'd have to write maybe ten times as much code as you would to have a complete web development offering. It also requires incredible amounts of expertise. Statistics, economics, physics, math, chemistry, biology, and on and on are all areas that individually require a great deal of specialized knowledge in addition to time. For some things, performance is the most important property, including use of the GPU. That's not simple.
>
> 3. D's syntax is okay, but it's not flexible enough to express eveything you need to work comfortably. A DSL or similar might be necessary.

I have no issue with calling libraries from other languages (particularly C) if it's something that is too much work or whatever to do myself. But I think that it's helpful to have a base level of functionality, akin to Numpy/Scipy, that a new person could come in to accomplish a lot.