August 12, 2012
On 8/11/2012 7:30 AM, Jakob Ovrum wrote:
> On Saturday, 11 August 2012 at 09:40:39 UTC, Walter Bright wrote:
>> Of course it is doing what the language requires, but it is an incorrect
>> diagnostic because a dead assignment is required.
>>
>> And being a dead assignment, it can lead to errors when the code is later
>> modified, as I explained. I also dislike on aesthetic grounds meaningless code
>> being required.
>
> It is not meaningless, it's declarative. The same resulting code as now would be
> generated, but it's easier for the maintainer to understand what's being meant.

No, it is not easier to understand, because there's no way to determine if the intent is to:

1. initialize to a valid value -or-
2. initialize to get the compiler to stop complaining


> I do, however, believe that D programmers are perfectly capable of doing the
> right thing if informed.

Of course they are capable of it. But experience shows they simply don't.


> Consider `pure` member functions - turns out most of them are actually pure
> because the implicit `this` parameter is allowed to be mutated and it's rare for
> a member function to mutate global state, yet we all strive to correctly
> decorate our methods `pure` when applicable.

A better design would be to have pure be the default and impure would require annotation. The same for const/immutable. Unfortunately, it's too late for that now. My fault.


> Java exception specifications have widespread implications for the entire
> codebase, while the difference between '0' and 'float.nan' is constant and
> entirely a local improvement.

I believe there's a lot more potential for success when you have a design where the easiest way is the correct way, and you've got to make some effort to do it wrong. Much of my attitude on that goes back to my experience at Boeing on designing things (yes, my boring Boeing anecdotes again), and Boeing's long experience with pilots and mechanics and what they actually do vs what they're trained to do. (And not only are these people professionals, not fools, but their lives depend on doing it right.)

Over and over and over again, the easy way had better be the correct way. I could bore you even more with the aviation horror stories I heard that justified that attitude.
August 12, 2012
On Sun, 12 Aug 2012 12:38:47 +0200, Walter Bright <newshound2@digitalmars.com> wrote:
> On 8/11/2012 7:30 AM, Jakob Ovrum wrote:
>> On Saturday, 11 August 2012 at 09:40:39 UTC, Walter Bright wrote:
>> Consider `pure` member functions - turns out most of them are actually pure
>> because the implicit `this` parameter is allowed to be mutated and it's rare for
>> a member function to mutate global state, yet we all strive to correctly
>> decorate our methods `pure` when applicable.
>
> A better design would be to have pure be the default and impure would require annotation. The same for const/immutable. Unfortunately, it's too late for that now. My fault.
>

I have thought that many times. The same with default non-null class references. I keep adding assert(someClass) everywhere.
August 12, 2012
Am 12.08.2012 12:38, schrieb Walter Bright:
> On 8/11/2012 7:30 AM, Jakob Ovrum wrote:
>> Consider `pure` member functions - turns out most of them are actually pure
>> because the implicit `this` parameter is allowed to be mutated and it's rare for
>> a member function to mutate global state, yet we all strive to correctly
>> decorate our methods `pure` when applicable.
>
> A better design would be to have pure be the default and impure would require
> annotation. The same for const/immutable. Unfortunately, it's too late for that
> now. My fault.

its never to late - put it back on the list for D 3 - please
(and local variables are immuteable by default - or seomthing like that)

August 12, 2012
On Sunday, 12 August 2012 at 11:34:20 UTC, dennis luehring wrote:
> Am 12.08.2012 12:38, schrieb Walter Bright:
>> A better design would be to have pure be the default and impure would require annotation. The same for const/immutable. Unfortunately, it's too late for that now. My fault.
>
> its never to late - put it back on the list for D 3 - please (and local variables are immutable by default - or something like that)

 Agreed. If it is only a signature change then it might have been possible to accept such a change; as I'm sure it would simplify quite a bit of signatures and only complicate a few. Probably default signatures to try and include are: pure, and @safe (Others off hand I can't think of).

 Make a list of all the issues/mistakes that can be done in D3 (be it ten or fifteen years from now); who knows, maybe the future is just around the corner if there's a big enough reason for it. The largest reason not to make big changes is so people don't get fed up and quit (especially while still trying to write library code); That and this is suppose to be the 'stable' D2 language right now with language changes having to be weighted heavily on.
August 12, 2012
On 8/12/12 12:52 AM, TJB wrote:
> Thanks for bringing this back to the original topic and for your thoughts.
>
> Indeed, a lot of econometricians are using MATLAB, R, Guass, Ox and the
> like. But there are a number of econometricians who need the raw power
> of a natively compiled language (especially financial econometricians
> whose data are huge) who typically program in either Fortran or C/C++.
> It is really this group that I am trying to reach. I think D has a lot
> to offer this group in terms of programmer productivity and reliability
> of code. I think this applies to statisticians as well, as I see a lot
> of them in this latter group too.
>
> I also want to reach the MATLABers because I think they can get a lot
> more modeling power (I like how you put that) without too much more
> difficulty (see Ox - nearly as complicated as C++ but without the
> power). Many MATLAB and R programmers end up recoding a good part of
> their algorithms in C++ and calling that code from the interpreted
> language. I have always found this kind of mixed language programming to
> be messy, time consuming, and error prone. Special tools are cropping up
> to handle this (see Rcpp). This just proves to me the usefulness of a
> productive AND powerful language like D for econometricians!

I think this is a great angle. In our lab when I was a grad student in NLP/ML there was also a very annoying trend going on: people would start with Perl for text preprocessing and Matlab for math, and then, after the proof of concept, would need to recode most parts in C++. (I recall hearing complaints about large overheads in Matlab caused by eager copy semantics, is that true?)

> I am sensitive to the drawbacks you mention (especially lack of numeric
> libraries). I am so sick of wasting my time in C++ though that I have
> almost decided to just start writing my own econometric library in D.
> Earlier in this thread there was a discussion of extended precision in D
> and I mentioned the need to recode things like BLAS and LAPACK in D.
> Templates in D seem perfect for this problem. As an expert in template
> meta-programming what are your thoughts? How is this different than what
> is being done in SciD? It seems they are mostly concerned about wrapping
> the old CBLAS and CLAPACK libraries.

There's a large body of experience and many optimizations accumulated in these libraries, which are worth exploiting. The remaining matter is offering a convenient shell. I think Cristi's work on SciD goes that direction.


Andrei
August 12, 2012
On Sunday, 12 August 2012 at 10:39:01 UTC, Walter Bright wrote:
> No, it is not easier to understand, because there's no way to determine if the intent is to:
>
> 1. initialize to a valid value -or-
> 2. initialize to get the compiler to stop complaining
>

If there is an explicit initializer, it means that the intent is either of those two. The latter case is probably quite rare, and might suggest a problem with the code - if the compiler can't prove your variable to be initialized, then the programmer probably has to spend some time figuring out the real answer. Legitimate cases of the compiler being too conservative can be annotated with a comment to eliminate the ambiguity.

The interesting part is that you can be sure that variables *without* initializers are guaranteed to be initialized at a later point, or the program won't compile. Without the guarantee, the default value could be intended as a valid initializer or there could be a bug in the program.

The current situation is not bad, I just think the one that allows for catching more errors at compile-time is much, much better.

> Of course they are capable of it. But experience shows they simply don't.

If they do it for contagious attributes like const, immutable and pure, I'm sure they'll do it for a simple fix like using explicit 'float.nan' in the rare case the compiler can't prove initialization before use.

> A better design would be to have pure be the default and impure would require annotation. The same for const/immutable. Unfortunately, it's too late for that now. My fault.

I agree, but on the flip side it was easier to port D1 code to D2 this way, and that might have saved D2 from even further alienation by some D1 users during its early stages. The most common complaints I remember from the IRC channel were complaints about const and immutable which was now forced on D programs to some degree due to string literals. This made some people really apprehensive about moving their code to D2, and I can imagine the fallout would be a lot worse if they had to annotate all their impure functions etc.

> I believe there's a lot more potential for success when you have a design where the easiest way is the correct way, and you've got to make some effort to do it wrong. Much of my attitude on that goes back to my experience at Boeing on designing things (yes, my boring Boeing anecdotes again), and Boeing's long experience with pilots and mechanics and what they actually do vs what they're trained to do. (And not only are these people professionals, not fools, but their lives depend on doing it right.)
>
> Over and over and over again, the easy way had better be the correct way. I could bore you even more with the aviation horror stories I heard that justified that attitude.

Problem is, we've pointed out the easy way has issues and is not necessarily correct.

August 12, 2012
Andrei Alexandrescu:

> (I recall hearing complaints about large overheads in Matlab caused by eager copy semantics, is that true?)

In Matlab there is COW:
http://www.matlabtips.com/copy-on-write-in-subfunctions/

Bye,
bearophile
August 12, 2012
On Sunday, 12 August 2012 at 03:30:24 UTC, bearophile wrote:
> Andrei Alexandrescu:
>
>> - The language's superior modeling power and level of control comes at an increase in complexity compared to languages such as e.g. Python. So the statistician would need a larger upfront investment in order to reap the associated benefits.
>
> Statistician often use the R language (http://en.wikipedia.org/wiki/R_language ).
> Python contains much more "computer science" and CS complexity compared to R. Not just advanced stuff like coroutines, metaclasses, decorators, Abstract Base Classes, operator overloading, and so on, but even simpler things, like generators, standard library collections like heaps and deques, and so on.
> For some statisticians I've seen, even several parts of Python are too much hard to use or understand. I have rewritten several of their Python scripts.
>
> Bye,
> bearophile


For people with more advanced CS/programming knowledge, though, this is an advantage of D.  I find Matlab and R incredibly frustrating to use for anything but very standard matrix/statistics computations on data that's already structured the way I like it.  This is mostly because the standard CS concepts you mention are at best awkward and at worst impossible to express and, being aware of them, I naturally want to take advantage of them.

Using Matlab or R feels like being forced to program with half the tools in my toolbox either missing or awkwardly misshapen, so I avoid it whenever practical.  (Actually, languages like C and Java that don't have much modeling power feel the same way to me now that I've primarily used D and to a lesser extent Python for the past few years.  Ironically, these are the languages that are easy to integrate with R and Matlab respectively.  Do most serious programmers who work in problem domains relevant to Matlab and R feel this way or is it just me?).  This was my motivation for writing Dstats and mentoring Cristi's fork of SciD.  D's modeling power is so outstanding that I was able to replace R and Matlab for a lot of use cases with plain old libraries written in D.
August 12, 2012
On Sunday, 12 August 2012 at 17:22:21 UTC, dsimcha wrote:

> ...  I find Matlab and R incredibly frustrating to use for anything but very standard matrix/statistics computations on data that's already structured the way I like it.

This is exactly how I feel, and why I am turning to D. My data sets are huge (64 TB for just a few years of data) and my econometric methods computationally intensive and the limitations of Matlab and R are always almost instantly constraining.

> Using Matlab or R feels like being forced to program with half the tools in my toolbox either missing or awkwardly misshapen, so I avoid it whenever practical.  Actually, languages like C and Java that don't have much modeling power feel the same way to me ...

Very well put - it expresses my feeling precisely.  And C++ is such a complicated beast that I feel caught in between.  I'd been dreaming of a language that offers modeling power as well as efficiency.

> ...  Do most serious programmers who work in problem domains relevant to Matlab and R feel this way or is it just me?.

I certainly feel the same. I only use them when I have to or for very simple prototyping.

> This was my motivation for writing Dstats and mentoring Cristi's fork of SciD.  D's modeling power is so outstanding that I was able to replace R and Matlab for a lot of use cases with plain old libraries written in D.

Thanks for your work on these packages! I will for sure be including them in my write up. I think they offer great possibilities for econometrics in D.

TJB


August 12, 2012
On Sun, 12 Aug 2012 03:38:47 -0700, Walter Bright <newshound2@digitalmars.com> wrote:

> On 8/11/2012 7:30 AM, Jakob Ovrum wrote:
>> On Saturday, 11 August 2012 at 09:40:39 UTC, Walter Bright wrote:
>>> Of course it is doing what the language requires, but it is an incorrect
>>> diagnostic because a dead assignment is required.
>>>
>>> And being a dead assignment, it can lead to errors when the code is later
>>> modified, as I explained. I also dislike on aesthetic grounds meaningless code
>>> being required.
>>
>> It is not meaningless, it's declarative. The same resulting code as now would be
>> generated, but it's easier for the maintainer to understand what's being meant.
>
> No, it is not easier to understand, because there's no way to determine if the intent is to:
>
> 1. initialize to a valid value -or-
> 2. initialize to get the compiler to stop complaining
>
>
>> I do, however, believe that D programmers are perfectly capable of doing the
>> right thing if informed.
>
> Of course they are capable of it. But experience shows they simply don't.
>
>
>> Consider `pure` member functions - turns out most of them are actually pure
>> because the implicit `this` parameter is allowed to be mutated and it's rare for
>> a member function to mutate global state, yet we all strive to correctly
>> decorate our methods `pure` when applicable.
>
> A better design would be to have pure be the default and impure would require annotation. The same for const/immutable. Unfortunately, it's too late for that now. My fault.
>
>
>> Java exception specifications have widespread implications for the entire
>> codebase, while the difference between '0' and 'float.nan' is constant and
>> entirely a local improvement.
>
> I believe there's a lot more potential for success when you have a design where the easiest way is the correct way, and you've got to make some effort to do it wrong. Much of my attitude on that goes back to my experience at Boeing on designing things (yes, my boring Boeing anecdotes again), and Boeing's long experience with pilots and mechanics and what they actually do vs what they're trained to do. (And not only are these people professionals, not fools, but their lives depend on doing it right.)
>
> Over and over and over again, the easy way had better be the correct way. I could bore you even more with the aviation horror stories I heard that justified that attitude.

As a pilot, I completely agree!

-- 
Adam Wilson
IRC: LightBender
Project Coordinator
The Horizon Project
http://www.thehorizonproject.org/