On Wednesday, 7 September 2016 at 15:04:38 UTC, jmh530 wrote:
> On Wednesday, 7 September 2016 at 11:37:44 UTC, Russel Winder wrote:
>>
>>> I really don't see what's not working in this.
>>
>> Trying to get new D users from Python users is the main problem.
>>
>
> I came to D from Python/R/Matlab. The biggest issue for me wasn't error messages so much as the lack of good libraries for a lot of things.
>
> Nevertheless, the longer I've been using D, the more I agree that there could be some improvements in D's error messages. Andre had posted about the Sparrow language a while back
> https://forum.dlang.org/thread/ne3265$uef$1@digitalmars.com?page=1
> He liked their use of concepts. I think at a minimum it would enable better error messages.

I too come from the R world and I have been playing the game of flitting between R and C++; using C++ (through RCpp) to speed up slow things in R for some time and I have been looking for a better solution.

For some time I have been considering a problem to do with creating tables with unbounded types, one of the failed attempts is here: https://forum.dlang.org/thread/gdjaoxypicsxlfvzwbvt@forum.dlang.org?page=1
I then exchanged emails with Lucian, Sparrows creator and he very quickly and simply outlined the solution to the problem. Thereafter I read his PhD thesis - one of the most informative texts in computer science I have read and very well written.

At the moment, there are lots of languages attempting to solve the dynamic-static loop, being able to have features inherent in dynamic programming languages, while keeping the safety and performance that comes with a static compiled programming language, and then doing so in a language that doesn't cause your brain to bleed. The "One language to rule them all" motif of Julia has hit the rocks; one reason is because they now realize that their language is being held back because the compiler cannot infer certain types for example: http://www.johnmyleswhite.com/notebook/2015/11/28/why-julias-dataframes-are-still-slow/

A language that can create arbitrary complex programs is the kind of thing that changes the world. I don't think D should be left out and should take Sparrow very seriously indeed.

On Wednesday, 7 September 2016 at 18:55:41 UTC, pineapple wrote:
>
> So the first difference you're likely to notice is that it's not as well documented. (Sorry. I'm a busy woman. I'll get around to it.) I try to make up for it with copious unit tests, which should provide a good example for how to use any given module.
>
> In terms of functionality, the single-biggest difference is that unlike phobos I don't treat arrays or any other collection directly as ranges; instead types may provide an `asrange` property returning a range that enumerates their contents. This architecture allows you to express HOFs as shown in that prior post, not having to worry about whether it's safe to treat the array itself as a range or whether you have to slice it.
>
> Other significant differences include not requiring bidirectional, slicing, random-access ranges to also be saving ("forward") ranges; (for the most part) supporting immutable elements in ranges; and a more clearly defined interface for what insertion and removal operations you may perform upon a range and how they are expected to behave. There are a few things phobos provides that I don't yet, but there's also a handful of things implemented in mach.range that aren't in phobos. (My personal favorite example of such is its small suite of PRNG implementations.)
>

Thanks for the reply. It looks like an interesting idea. You might consider adding this (or a modified version) to a read me in the range subfolder.

Are you familiar with Chapel at all? The language allows the user to specify a domain with an array, facilitating sparsity or arrays distributed across different machines. For some reason I was reminded of that when you say that asrange returns a range that enumerates the contents.

On Wednesday, 7 September 2016 at 19:19:23 UTC, data pulverizer wrote:
> The "One language to rule them all" motif of Julia has hit the rocks; one reason is because they now realize that their language is being held back because the compiler cannot infer certain types for example: http://www.johnmyleswhite.com/notebook/2015/11/28/why-julias-dataframes-are-still-slow/

As an avid user of Julia, I'm going to have to disagree very strongly with this statement. The language is progressing very nicely and while it doesn't aim to be the best choice for every programming task imaginable, it already does an excellent job of letting a scientific programmer such as myself do most of my workflow in a single language with remarkable performance. Furthermore, the article you linked pertains to a simple type inference issue, exposed by the design constraints of a particular library. While certain design patterns can and often do lead to Python-style Julia code with optimal performance, you can always get there by manually enforcing type stability at the cost of less pretty code.

More to the general point of the discussion, I find that most scientifically minded users of Python already appreciate some of the inherent advantages of lower level statically typed languages and often rather write C/C++ code than descend into the likes of Cython. D has considerable advantages over C++ in conciseness and template facilities for achieving zero cost static polymorphism without descending into utter unreadability. Personally, I find myself still forced to write most of my non-Julia high performance code in C++ due to the available libraries and GPGPU support (especially CUDA), but in terms of language properties I'd much rather be writing D.

On Wednesday, 7 September 2016 at 19:19:23 UTC, data pulverizer wrote:
>
> For some time I have been considering a problem to do with creating tables with unbounded types, one of the failed attempts is here: https://forum.dlang.org/thread/gdjaoxypicsxlfvzwbvt@forum.dlang.org?page=1
> I then exchanged emails with Lucian, Sparrows creator and he very quickly and simply outlined the solution to the problem. Thereafter I read his PhD thesis - one of the most informative texts in computer science I have read and very well written.
>
> At the moment, there are lots of languages attempting to solve the dynamic-static loop, being able to have features inherent in dynamic programming languages, while keeping the safety and performance that comes with a static compiled programming language, and then doing so in a language that doesn't cause your brain to bleed. The "One language to rule them all" motif of Julia has hit the rocks; one reason is because they now realize that their language is being held back because the compiler cannot infer certain types for example: http://www.johnmyleswhite.com/notebook/2015/11/28/why-julias-dataframes-are-still-slow/

I don't see any reason why D can't implement pandas DataFrames without needing to change the language at all
http://pandas.pydata.org/pandas-docs/stable/generated/pandas.DataFrame.html
It's just a lot of work.

The simplest I can think of is a struct containing a tuple that contains slices of equal length and an array of strings containing column names. You could have a specialization with a two-dimensional array (or ndslice).

On Wednesday, 7 September 2016 at 20:37:50 UTC, jmh530 wrote:
> On Wednesday, 7 September 2016 at 19:19:23 UTC, data pulverizer wrote:
>>
>> For some time I have been considering a problem to do with creating tables with unbounded types, one of the failed attempts is here: https://forum.dlang.org/thread/gdjaoxypicsxlfvzwbvt@forum.dlang.org?page=1
>> I then exchanged emails with Lucian, Sparrows creator and he very quickly and simply outlined the solution to the problem. Thereafter I read his PhD thesis - one of the most informative texts in computer science I have read and very well written.
>>
>> At the moment, there are lots of languages attempting to solve the dynamic-static loop, being able to have features inherent in dynamic programming languages, while keeping the safety and performance that comes with a static compiled programming language, and then doing so in a language that doesn't cause your brain to bleed. The "One language to rule them all" motif of Julia has hit the rocks; one reason is because they now realize that their language is being held back because the compiler cannot infer certain types for example: http://www.johnmyleswhite.com/notebook/2015/11/28/why-julias-dataframes-are-still-slow/
>
> I don't see any reason why D can't implement pandas DataFrames without needing to change the language at all
> http://pandas.pydata.org/pandas-docs/stable/generated/pandas.DataFrame.html
> It's just a lot of work.
>
> The simplest I can think of is a struct containing a tuple that contains slices of equal length and an array of strings containing column names. You could have a specialization with a two-dimensional array (or ndslice).

You're quite right that D doesn't need to change at all to implement something like pandas or dataframes in R, but I am thinking of how to got further. Very often in data science applications types will turn up that are required but are not currently configured for your table. The choice you have is to have to modify the code or as scala does give programmers the ability to write their own interface to the type so that the it can be stored in their DataFrame. The best solution is that the data table is able to cope with arbitrary number of types which can be done in Sparrow.

On Wednesday, 7 September 2016 at 20:37:50 UTC, jmh530 wrote:
> On Wednesday, 7 September 2016 at 19:19:23 UTC, data pulverizer I don't see any reason why D can't implement pandas DataFrames without needing to change the language at all
> http://pandas.pydata.org/pandas-docs/stable/generated/pandas.DataFrame.html
> It's just a lot of work.
>
> The simplest I can think of is a struct containing a tuple that contains slices of equal length and an array of strings containing column names. You could have a specialization with a two-dimensional array (or ndslice).

p.s. it goes beyond just tables, ... having dynamic capability in a static compiled language really does take computing to a different place indeed.

On Wednesday, 7 September 2016 at 20:29:51 UTC, deXtoRious wrote:
> On Wednesday, 7 September 2016 at 19:19:23 UTC, data pulverizer wrote:
>> The "One language to rule them all" motif of Julia has hit the rocks; one reason is because they now realize that their language is being held back because the compiler cannot infer certain types for example: http://www.johnmyleswhite.com/notebook/2015/11/28/why-julias-dataframes-are-still-slow/
>
> As an avid user of Julia, I'm going to have to disagree very strongly with this statement. The language is progressing very nicely and while it doesn't aim to be the best choice for every programming task imaginable...

Ahem (http://www.wired.com/2014/02/julia/), I'm not saying that the Julia founders approved that title, we all know how the press can inflate things, but there was a certain rhetoric that Julia was creating something super-special that would change everything.

On Wednesday, 7 September 2016 at 19:19:23 UTC, data pulverizer wrote:
> On Wednesday, 7 September 2016 at 15:04:38 UTC, jmh530 wrote:
>> On Wednesday, 7 September 2016 at 11:37:44 UTC, Russel Winder wrote:
>>>
>>>> I really don't see what's not working in this.
>>>
>>> Trying to get new D users from Python users is the main problem.
>>>
>>
>> I came to D from Python/R/Matlab. The biggest issue for me wasn't error messages so much as the lack of good libraries for a lot of things.
>>
>> Nevertheless, the longer I've been using D, the more I agree that there could be some improvements in D's error messages. Andre had posted about the Sparrow language a while back
>> https://forum.dlang.org/thread/ne3265$uef$1@digitalmars.com?page=1
>> He liked their use of concepts. I think at a minimum it would enable better error messages.
>
> I too come from the R world and I have been playing the game of flitting between R and C++; using C++ (through RCpp) to speed up slow things in R for some time and I have been looking for a better solution.

What are you doing with Rcpp that you can't do with D?

On Wednesday, 7 September 2016 at 20:57:03 UTC, data pulverizer wrote:
> On Wednesday, 7 September 2016 at 20:29:51 UTC, deXtoRious wrote:
>> On Wednesday, 7 September 2016 at 19:19:23 UTC, data pulverizer wrote:
>>> The "One language to rule them all" motif of Julia has hit the rocks; one reason is because they now realize that their language is being held back because the compiler cannot infer certain types for example: http://www.johnmyleswhite.com/notebook/2015/11/28/why-julias-dataframes-are-still-slow/
>>
>> As an avid user of Julia, I'm going to have to disagree very strongly with this statement. The language is progressing very nicely and while it doesn't aim to be the best choice for every programming task imaginable...
>
> Ahem (http://www.wired.com/2014/02/julia/), I'm not saying that the Julia founders approved that title, we all know how the press can inflate things, but there was a certain rhetoric that Julia was creating something super-special that would change everything.

That's just typical press nonsense, and even they quote Bezanson saying how Julia isn't at all suited to a whole host of applications. Julia certainly has (justifiable, imho, though only time will tell) aspirations of being useful in certain areas of general computing, not just scientific code, but they are far from universal applicability, let alone optimality. If nothing else, it's an interesting example of thinking rather far outside the usual box of language design, one with demonstrable real world applications.

On Wednesday, 7 September 2016 at 20:57:15 UTC, bachmeier wrote:
> What are you doing with Rcpp that you can't do with D?

That's a very good point, there's nothing that R + C++ can do that is out of D's reach. But, I wander if we can go further ....