On Tuesday, 29 September 2020 at 13:07:35 UTC, claptrap wrote:
> its a great quote and you can see the wisdom in it but there's a lot lurking under the surface of the words "essential" and "necessary"

Indeed. If you only have goto and if, would programmers then write code where common patterns occur or would the compiler writer just optimize for structures that have the properties of noise as programmers might end up writing very different code structures?

I strongly suspect that one advantage of having a standard library and high level features is that programmers then tend to write code in similar patterns that compiler writers can take advantage of those patterns.

On 9/29/20 8:55 AM, Steven Schveighoffer wrote:
> On 9/29/20 8:02 AM, Patrick Schluter wrote:
>> Your AA example is a good one. If it wasn't in the language, it would be the same issue with AA as in any other language that defines it in libraries: several different incompatible restricted implementations.
> 
> AA is not that far from being a library type.
> 
> If you discount the magic casting that AAs can do (which isn't all that useful), there is one (yes ONE) feature that AAs do that a library type cannot. And that is an implicit adding of an element in certain cases. The reason AAs can do it is because the compiler calls a different opIndex (it's not called opIndex, but some extern(C) hook) when it's in this mode, but only for AAs.
> 
> In fact, AAs are mostly already a library type, thanks to a lot of incremental effort over the years to lower the implementation to templates. Except for this one thing.

Thanks, good to know. I now recall I helped a little with that. (Was it the length function? empty? some range-related addition?) This supports the original point - it took much effort over years to disentangle and undo all this magic.

On Tuesday, 29 September 2020 at 12:02:21 UTC, Patrick Schluter wrote:

> Your AA example is a good one. If it wasn't in the language, it would be the same issue with AA as in any other language that defines it in libraries: several different incompatible restricted implementations.

That's a good example. IMO there's a difference between a language and bits and pieces that can be put together to call random libraries.

On Tuesday, 29 September 2020 at 13:38:25 UTC, bachmeier wrote:
> That's a good example. IMO there's a difference between a language and bits and pieces that can be put together to call random libraries.

Heh, not really. It is not unusual for languages to have a minimal core and default include the most commonly used parts of their standard library. The programmer can often not tell the difference.

On 9/29/20 8:02 AM, Patrick Schluter wrote:
> On Tuesday, 29 September 2020 at 03:14:34 UTC, Andrei Alexandrescu wrote:
>> On 9/28/20 6:46 PM, Bruce Carneal wrote:
> 
> [..]
>>
>> Years ago, I was in a panel with Simon Peyton-Jones, the creator of Haskell. Nicest guy around, not to mention an amazing researcher. This question was asked: "What is the most important principle of programming language design?"
>>
>> His answer was so powerful, it made me literally forget everybody else's answer, including my own. (And there were no slouches in that panel, save for me: Martin Odersky, Guido van Rossum, Bertrand Meyer.) He said the following:
>>
>> "The most important principle in a language design is to define a small core of essential primitives. All necessary syntactic sugar lowers to core constructs. Do everything else in libraries."
>>
> Sorry to not be completely convinced on the pertinence of that statement. For the language designer it might well be true, for the users of the language I'm not completely sold. There are two languages that imho fit that description very well, they are both very powerfull and extremely elegant, but boy do they suck when you just want to get shit done: Lisp and Forth. And (I know, one should not start a sentence with and) it had been mentioned often in this forum, it is this shear flexibility that are in the way of wider adoption as there are so many ways of doing things, but none of them completely right.
> Your AA example is a good one. If it wasn't in the language, it would be the same issue with AA as in any other language that defines it in libraries: several different incompatible restricted implementations.
> 
> Just my uninformed 2 cents.

Completely agree with the sentiment! Also, being informed in language internals paraphernalia is not at stake here.

I've studied classical guitar for many years. Got a couple of prizes, too, but never broke out of the amateur ranks. Nevertheless, I got to meet a few professional players and also played to a variety of audiences.

One thing that's obvious to a music performer that is not at all clear to listeners is that deficiencies in performance show up as second-order effects.

What do I mean by that? I mean that the audience doesn't sit there with the sheet music in hand following the notes and the directions written. They won't put the finger on paper and say, hey, hey, what happened to the mezzoforte in here? No, they just sit back and listen to the music. If they like it, they don't know why. If they don't, they don't know why. Music critics write all about some performer's technique, touché, style, rubato - but listeners are like, yeah, that's beautiful or isn't. An exegete knows that that came out poorly because the performer moved their wrist too much.

No bug report will come as "D uses too many tricks in the compiler that are inaccessible to programs, and that's very problematic." No, they'll all come as problems with surprising limitations and inconsistencies, frustration in use, platform dependencies, deficiencies in implementation, lack of orthogonality, very concrete and very many bugs, bugs, and bugs.

Deficiencies in programming language design show up as second-order effects.

There's also something about limits that needs clarification. Small language kernel being good does not mean the smallest language kernel is the best. (In guitar technique economy of motion is a key ideal, i.e. one should minimize movement of both left and right hand. That of course doesn't mean zero movement is best, because that would produce no sound.) So keeping the core small and reducing sugar to it is an ideal to live into, not a rigid requirement.

On 9/29/20 9:38 AM, bachmeier wrote:
> On Tuesday, 29 September 2020 at 12:02:21 UTC, Patrick Schluter wrote:
> 
>> Your AA example is a good one. If it wasn't in the language, it would be the same issue with AA as in any other language that defines it in libraries: several different incompatible restricted implementations.
> 
> That's a good example. IMO there's a difference between a language and bits and pieces that can be put together to call random libraries.

There is no doubt to Walter or myself that the way built-in AAs were done in D is a resounding failure.

The way they should have been done was a templated library type AssociativeArray!(K, V) with the V[K] type syntax sugar on top. That would have allowed the implementation to improve along with the rest of the language and its library.

(Of course there's historical context to this, i.e. AAs predate templates and then they just continued being there, increasingly standing out like the proverbial sore thumb as everything else evolved around them.)

As things are now, AAs don't quite behave like any other things. They are a poor cousin to ranges. No container infrastructure grew around them. They are dramatically less efficient than library hashtables.

Worst of all, these problems have been well-known for years and years and the progress on fixing them has moved at a glacial pace.

On 9/29/20 12:56 AM, Paul Backus wrote:
> The best we can hope for is to nail down a precise description of all the fiddly corner cases, so that we know what D's semantics *actually* are, rather than what the current spec pretends they are.

Definitely. I'd go even further and call that "survival". C++ has had tremendous success with meticulously documenting all of the odd things they baked in pre-standardization. And then it was a matter of updating things, and it all got better. Most importantly, there was and is a group of people who understood the vital importance of that.

> But even that is likely to be infeasible in practice--who wants to volunteer to be the one to go over, say, funcDeclarationSemantic with a fine-toothed comb, especially after what happened to the last guy? [1]

Actually Walter made a lot of efforts to do that, too, as he very well understands the importance of correct specification. But nobody cared about his related PRs, which probably are flapping in the wind to this day. He has some work on defining the object model, too, which has also been ignored. A kernel of 3-5 strong souls to commit to this would be very necessary. Walter can't do all of this alone. Now he can't even do it formally because of the "no single person" rule.

I should add I've been unfairly harsh about Walter's inclination to do tricks in the compiler. That applies a lot more to the early days than now. As of now our views are aligned to a large proportion on core language vs. library features.

> The fact is, there's no incentive for anyone to do this kind of work other than masochism and sheer stubbornness. The amount of effort it would take, and the amount of time that would pass before you saw any benefit, are simply too high. We can talk about principles and good engineering all we want, but we can't expect people to voluntarily act against their own interest.
> 
> Maybe we can make some improvements at the margins, but my prediction is that the overwhelming majority of D's complexity is here to stay, whether we like it or not. The best we can do is try to offer enough positive value to D programmers to offset its negative effects. And if it doesn't work out, well, there's always Rust. ;)

I think things can be managed, but documentation and control are a must. FWIW I don't think Rust fares that well in that area - it's a very complex language, with the complexity mostly in the wrong places leaving mediocre handling of all that's interesting, and its rigorous documentation is greatly outmatched by its marketing brochures.

On Tuesday, 29 September 2020 at 12:54:53 UTC, Stefan Koch wrote:
> So we had this:
>
>> int mul(
>>   int x,
>>   int y
>> ) {
>>   int result = 0;
>>   for(int i = 0; i < x; i++)
>>   {
>>     result += y;
>>   }
>>   return result;
>> }
>
>
> And the code generated was
>
> 	imull	%esi, %edi // y *= x
> 	xorl	%eax, %eax // result = 0
> 	testl	%esi, %esi // if (y)
> 	cmovgl	%edi, %eax // if (x > 0) result = x;
>         retq               // return result;
>
> On first glance these instructions seem superfluous
> But we have to understand that the compiler had no way of knowing that we actually wanted to generate a mere multilply.
>
> For example there is the loop entrance check.
> Why is it there?
>
> We can proof it's necessity quite easily.
> [snip]

I don't know assembly, but I can see an even simpler reason why the compiler can't generate a multiply instruction. It's `mul(-5, 5)*`.

On Tuesday, 29 September 2020 at 14:42:42 UTC, Dukc wrote:
> On Tuesday, 29 September 2020 at 12:54:53 UTC, Stefan Koch wrote:
>> So we had this:
>>
>>> [...]
>>
>
> I don't know assembly, but I can see an even simpler reason why the compiler can't generate a multiply instruction. It's `mul(-5, 5)*`.

Which is another reason why it's harder to state the problem in a lower form.
I wrote the example wanting to show two semantically equivalent things and I did not.

On Tuesday, 29 September 2020 at 14:07:14 UTC, Andrei Alexandrescu wrote:
> On 9/29/20 9:38 AM, bachmeier wrote:
>> On Tuesday, 29 September 2020 at 12:02:21 UTC, Patrick Schluter wrote:
>> 
>>> Your AA example is a good one. If it wasn't in the language, it would be the same issue with AA as in any other language that defines it in libraries: several different incompatible restricted implementations.
>> 
>> That's a good example. IMO there's a difference between a language and bits and pieces that can be put together to call random libraries.
>
> There is no doubt to Walter or myself that the way built-in AAs were done in D is a resounding failure.

If so, it's a pretty good resounding failure, because I like them and they would most likely not be available to me now if they hadn't been built into the language. Long ago when I used Common Lisp, there was the incredible FSet library that provided functional collections kind of like Clojure. Unfortunately few people knew about them or used them, and it doesn't look like anything has been done with them in a long time: https://common-lisp.net/project/fset/ Sometimes a library solution works, but it's in general a risky proposition to rely on libraries for such a fundamental feature.

If AAs had been introduced as a library in 2012, the repo would probably have a couple hundred open issues and the last development would have been in 2017.

> The way they should have been done was a templated library type AssociativeArray!(K, V) with the V[K] type syntax sugar on top. That would have allowed the implementation to improve along with the rest of the language and its library.

Does anything prevent the introduction of a new library solution? If it's good enough, it can eventually be added to the language.

I'm not going to pretend to be a language designer. My experience as a language user is that a big set of core functionality works better than a small set plus libraries. It's possible I don't understand your proposal.