What is D?
Where is the gap in the language market?
What is the future of D?

It's a quiet Sunday, Andrei and Walter have got me thinking, time for some thoughts to go down an interpipe.

What is D?

D is, or should be, well I'd like it to be:

 (1) A systems language
 (2) A safe language
 (3) A simple language (yes, really) that also has powerful constructs
 (4) A fast (at runtime) language

Where is the gap?

The above language, if it really existed, fills a gap in the current language market. The new boys on the block make claims to some of the above but I don't believe any of them really fill it. Could D?

What is the future of D?

The answers to my first question also answer this question:

  (1) D will be the go-to systems language
  (2) D needs to get _safer_. It needs to be obviously safe to someone reading  D code, and it needs to fulfil the safety promises it makes
  (3) D needs to get _simpler_. Making it simple is the hard part
  (4) D is already fast. Keep it like that

Andrei's thrown the deck of cards in the air. Everyone is watching where they land. Which cards to grab? What is the direction to head in?

  (1) The ones that make D safer
  (2) The ones that make D simpler

Walter is right when he says that more and more we (homo sapiens) need a safe language. We all know why. Will that be D? D+/D3? or D-? Safety and simplicity go hand in hand. We must be able to reason about the code we write. We don't want 90% of our cognitive load to be if this template and these @s and these refs/ins/consts/inouts and these overloads and these memory allocators and these imports and these mixins, when blended together, will do what we expect them to do. Orthogonal features really should be orthogonal.

One judgement. I'm just a bystander, I could be totally wrong but I think my impressions probably reflect the impressions of many other well-wishing bystanders:

D is too complicated. Too many @this and @that. Too many leaky abstractions. Proof: sometimes clever people ask questions why something doesn't work and Jonathan D, Steve S or Adam R are the only people capable of answering them. That doesn't scale and it's counter to having a safe language. Old D hands love D's power because it's 'easy' to achieve an algorithm in a few lines of code. However, it is often not simple to reason about if the code will work and if it will be performant. Ref simple vs easy by Rich Hickey.

On Sunday, 7 April 2019 at 11:39:01 UTC, Abdulhaq wrote:
> What is D?

I think D started not as a better C++, but as a better Java.  I think Walter saw the appeal of Java and wanted to explore making it compile to native binaries.  There is evidence of this with things like `super`, single inheritance with multiple interface implementation, the garbage collector, etc.  When D started it didn't have templates, and I even heard Walter forbade templates at first.

I also think D is more evolution than intelligent design.  It has evolved so much over the years, moving away from a "native java" to to more of a "better C++".  It think that has contributed much to its oddities, warts, and technical debt.

A few years ago D's slogan was changed to "Fast this, fast that, yada yada".  I honestly don't know what it is because I read it once, immediately hated it, and forgot it.  Prior to that change, D's slogan was "Modeling power, modern convenience, and native efficiency" See https://www.youtube.com/watch?v=Kn88228KSpQ for a presentation by Walter elaborating on this. <-- THIS IS D!

I think over the past few years it has become apparent that D has also become a case study in design by introspection:  See https://www.youtube.com/watch?v=Kn88228KSpQ for a presentation by Andrei elaborating on this.

To a lesser degree, I'd say D also has safety, though I hope that some day it will be able to compete with the likes of Rust in that department.  @safe by default, please!

Contrasting D with a few languages that I have some experience with:

I don't like C.  It's a simple language, but it doesn't have modeling power or modern convenience.

I don't like C++.  It has modeling power, but the models quickly become monstrosities.  It also lacks modern convenience; e.g. having to forward declare things is primitive.

I don't like Rust. It lacks modeling power.  It achieves much its safety and robustness by restricting the user.  D, by contrast, empowers the user.

I don't like C#. It lacks modeling power (no metaprogramming capabilities).  Composition is not DRY.  It lacks native efficiency.  I'll probably never be able to use C# to program an ARM Cortex-M microcontroller with only 16KB of flash memory.  It also lacks portability, though .Net core is helping to improve that situation.

I want to be able to use one programming language for everything from cloud services to microcontrollers the size of a grain of rice.

I want to be able to model my ideas the way I envision them; I don't want to have to distort my vision to make it fit my programming language's limitations.

I want to have some of the modern conveniences of languages like C#, but without all of the metaprogramming, platform and problem domain restrictions and limitations.

I don't want to have to repeat myself in header, source files, and forward declarations, or anywhere else for that matter.

I want my programming language to help me get things right (e.g. with regard to memory safety).

D's the only language I've found that has the potential for all of that.  It's not there yet, but it can become that.

There are plenty of things I don't like about D, but of all the languages I've used D seems to have the most potential.  I do not think D is managed well, and I think that is holding it back.  I fear that in 5 years, D will be exactly where it is now; an obscure language with a cult following.  However, Andrei's recent post (https://forum.dlang.org/post/q7j3s0$15n7$1@digitalmars.com) "We've been worrying too much about changing things" has reignited my hope for D.  D's a good language, but it has to clean up its warts and shed some technical debt if it wants to compete in the big leagues.

I'm giving D another try hoping Walter and Andrei are willing to embrace a little change.

Mike

On Sunday, 7 April 2019 at 13:05:14 UTC, Mike Franklin wrote:
> I want to be able to use one programming language for everything from cloud services to microcontrollers the size of a grain of rice.
>
> I want to be able to model my ideas the way I envision them; I don't want to have to distort my vision to make it fit my programming language's limitations.
>
> I want to have some of the modern conveniences of languages like C#, but without all of the metaprogramming, platform and problem domain restrictions and limitations.
>
> I don't want to have to repeat myself in header, source files, and forward declarations, or anywhere else for that matter.
>
> I want my programming language to help me get things right (e.g. with regard to memory safety).
>
> D's the only language I've found that has the potential for all of that.  It's not there yet, but it can become that.

I very much agree. D has the potential to be the universal language I dream of, covering all aspects from low-level/high-performance stuff (with the option to dive into inline assembly and, for LDC, IR level) to high-level stuff (e.g., GUI, with GC convenience *and* metaprogramming etc.), replacing a mixed stack of C++/C#/Javascript I would otherwise use. All of that with a very pleasant and expressive syntax (especially for metaprogramming!) with low hurdles of entry for anyone familiar with C-like languages.

On Sunday, 7 April 2019 at 13:05:14 UTC, Mike Franklin wrote:
> On Sunday, 7 April 2019 at 11:39:01 UTC, Abdulhaq wrote:
>> What is D?
>
> I think D started not as a better C++, but as a better Java.  I think Walter saw the appeal of Java and wanted to explore making it compile to native binaries.  There is evidence of this with things like `super`, single inheritance with multiple interface implementation, the garbage collector, etc.  When D started it didn't have templates, and I even heard Walter forbade templates at first.

It may appear that way, but given what I've learned (and as I've come to understand) in working on the "Origins of the D Programming Language", it really was intended as a better C++. The relevant paragraph from the paper:

'In his years maintaining a C++ compiler, Walter had often reflected upon the language’s
strengths and weaknesses and the lessons that one could derive from them in the design of
a new language. He decided it was time to put those ideas into practice. In October, 1999,
he created a new company, Digital Mars. In November of that year, he began work on the
design and implementation of a new programming language that he called “Mars”.'

Single inheritance multiple interfaces was a response to the problems with multiple inheritance in C++, inspired by Java, but not in service of being a better Java. Ditto the GC. Walter was not a fan of GC initially, but in implementing the Symantec Java compiler, he came to see some benefits that he felt would enhance even a systems language. Since he opted for the Java model of classes as reference types, the GC was a natural fit. Java-like inner classes were an evolutionary feature, implemented only to make it easier to port a large Java codebase like SWT to D.

As for templates, he was jaded by the complexity of their implementation and the horribly complex code they could lead to in general use. It wasn't long after the early alpha releases of DMD that he had the epiphany that a templated function is just a function with an extra parameter list. That led to the simpler syntax and less complex implementation, overcoming his initial objections.

So yes, his experience implementing Java (in C++!) for Symantec had an influence on the design and implementation of D, but several other languages did as well. So did his experience as a Boeing engineer. But it was a desire to improve upon C++ that was the impetus, and most of the influences were applied toward eliminating what he thought as C++ missed or got wrong.

On Sunday, 7 April 2019 at 13:05:14 UTC, Mike Franklin wrote:
> On Sunday, 7 April 2019 at 11:39:01 UTC, Abdulhaq wrote:
>> What is D?
>
> I think D started not as a better C++, but as a better Java.  I think Walter saw the appeal of Java and wanted to explore making it compile to native binaries.  There is evidence of this with things like `super`, single inheritance with multiple interface implementation, the garbage collector, etc.  When D started it didn't have templates, and I even heard Walter forbade templates at first.
>

Mike's already given a good answer to this but yes Walter wrote a compiler for Java a long time ago, and also gcc was able to compile Java too (https://en.wikipedia.org/wiki/GNU_Compiler_for_Java) which was also unsuccessful - there doesn't seem to be a market for that.

> I also think D is more evolution than intelligent design.  It has evolved so much over the years, moving away from a "native java" to to more of a "better C++".  It think that has contributed much to its oddities, warts, and technical debt.
>

Yes it's evolved but was any other way practical?

> A few years ago D's slogan was changed to "Fast this, fast that, yada yada".  I honestly don't know what it is because I read it once, immediately hated it, and forgot it.  Prior to that change, D's slogan was "Modeling power, modern convenience, and native efficiency" See https://www.youtube.com/watch?v=Kn88228KSpQ for a presentation by Walter elaborating on this. <-- THIS IS D!
>

I do get the feeling that speed takes precedence over most other features.

> I think over the past few years it has become apparent that D has also become a case study in design by introspection:  See https://www.youtube.com/watch?v=Kn88228KSpQ for a presentation by Andrei elaborating on this.
>

I haven't watched this yet but keep meaning too. I have a suspicion that it's a bit like duck typing so I should learn more and find out if that's true or not. I can't help thinking that that is what happened to ranges and now Andrei is trying to tighten them up. No doubt I am totally wrong though!

You're points about other languages are well taken and interesting. D does seem to do better here. I think what you value about D is very representative of the body of D users - a very powerful and expressive systems language. Safety comes below expressiveness.

My feelings though are that the future all-conquering C++ replacement will be the language that gets safety right, while providing properly isolated systems style code to do the necessary dirty work, providing as many guarantees as possible given that requirement. D has @system and @safe which are the right direction but don't seem to provide sufficient safety. ATM @safe seems to be subtractive meaning that Walter is slowly picking away with new code that disallows constructs where (it turns out) it doesn't actually provide safety. I'd rather have a solution which was additive in the sense that it starts 100% safe with less functionality (e.g. no raw pointers / systems style code?) and then slowly more power can be opened up. My gut finger in the air feeling is that D is 95% - 98% safe where it says it is safe. The all conquering language that drives cars, flies aeroplanes, controls robots for neurosurgery and governs financial transactions, will be at least 99.99% safe and it won't take great analysis for a reader of the code to have that confidence.

I also agree that W&A have not managed the project in an optimal way conducive to growing the numbers. Instead it seems they've focused on what they enjoy doing - coding, not managing and reviewing - that or, in Andrei's case it seems, doing the washing up and hoovering in in preference to managing. I've all been there!

>
> I'm giving D another try hoping Walter and Andrei are willing to embrace a little change.
>

In broad terms, what would you change?



> Mike

On Monday, 8 April 2019 at 18:36:02 UTC, Abdulhaq wrote:

>> I also think D is more evolution than intelligent design.  It has evolved so much over the years, moving away from a "native java" to to more of a "better C++".  It think that has contributed much to its oddities, warts, and technical debt.
>>
>
> Yes it's evolved but was any other way practical?

No, evolution is great.  But you have to be willing to deprecate and break with the past to move forward.  You can't continue to carry all of your historical baggage with you as you progress.

> I do get the feeling that speed takes precedence over most other features.

I don't see why it has to be one or the other.  This was a brilliant observation of Rust (https://youtu.be/d1uraoHM8Gg?t=68)  We don't have to always trade-off one thing for another.  We can have speed, expressiveness, and speed; they're not always mutally exclusive.

>> I think over the past few years it has become apparent that D has also become a case study in design by introspection:  See https://www.youtube.com/watch?v=Kn88228KSpQ for a presentation by Andrei elaborating on this.
>>
>
> I haven't watched this yet but keep meaning too. I have a suspicion that it's a bit like duck typing so I should learn more and find out if that's true or not. I can't help thinking that that is what happened to ranges and now Andrei is trying to tighten them up. No doubt I am totally wrong though!
>
> You're points about other languages are well taken and interesting. D does seem to do better here. I think what you value about D is very representative of the body of D users - a very powerful and expressive systems language. Safety comes below expressiveness.

Definitely not.  I very much want D to be statically-checked safe language.  I don't think you always have to make tradeoffs to get what you want.  I've once said that my ideal language would be Dust, basically D with a borrow checker.

IMO Rust's borrow checker is the greatest language innovation since C++. Unfortunately, it's only available in Rust.  D's trying with DIP1000 and DIP25, but this blog post (https://atilaoncode.blog/2019/03/13/issues-dip1000-cant-yet-catch/) illustrates it has come up short.  I would love to hear if anyone has any ideas to close that hole.  Walter?


> In broad terms, what would you change?

1.  Change the attitude towards change.  Bottom line is you don't have to be stuck with your mistakes.  With a well-managed deprecation process, or by creating a parallel, new, alternative implementations, D can move forward and leave its mistakes in the past; even the little ones.

With the coming copy constructors and the proposed ProtoObject, I'm cautiously optimistic that the attitude is changing.

I hope Andrei means what he says here: https://forum.dlang.org/post/q7j3s0$15n7$1@digitalmars.com

2.  Embrace what D does best (i.e. design by introspection). Utilize it in the compiler and the druntime to create a an opt-in continuum, again proposed by Andrei here: https://forum.dlang.org/post/q7j4sl$17pe$1@digitalmars.com  We have a couple of GSoC projects that, if chosen and are successful, could bring this closer to reality.

3.  D MUST be @safe by default to be taken seriously.  I realize it's superficial, but it matters.

4.  D MUST have an equally effective alternative to Rust's statically-checked memory safety and fearless concurrency to remain relevant.  DIP1000 and DIP25 give me hope, but D's missing something (that I can't quite put my finger on) as demonstrated in Atila's post above.

IMO, Rust's only feature is the borrow checker; the rest of the language stinks.  But, statically-checked memory safety is so fantastic Rust is destined for success.  If D had an equally effective mechanism for memory safety and fearless concurrency, there would be no reason for Rust to exist, and there would be no competition for D.

Mike

On Tuesday, 9 April 2019 at 00:07:33 UTC, Mike Franklin wrote:
> 3.  D MUST be @safe by default to be taken seriously.  I realize it's superficial, but it matters.

I agree 100% here. Whatever breaking changes are necessary to make this a reality, I say we do it and yell about it ad nauseam. Say it with me:

SafeD is Safety

At work I've recently changed to a position in which I have more leverage to influence which language certain key parts of our product are written in. Currently that's primarily Java, with a collection of small services written in Go. If I had my way, it'd all be rewritten in D, but I need to convince the other members of my team that it's feasible to use D (as an aside, GC was actually a big plus for them - I work in cybersecurity, so memory safety is a core requirement for most parts of our product. Memory safety *without* a GC would be even better, as there are security concerns around data lifetime with a GC, but baby steps).

The first question I was asked was "is it memory safe by default?", to which the answer was painfully "no, only in code annotated with @safe". The inevitable follow-up is "so you have to mark every function with @safe?!"... not a great showing for D in that respect.

It's not a deal-breaker, but it already puts D on an uneven footing in regards to Go and Rust, which both claim full memory safety by default. My plan of attack is to devote a few evenings and weekends to rewriting one of our smaller Go services in D to showcase the obvious (obvious to me, anyway, and hopefully to others shortly after completion) benefits of the language and the feasibility of using D for more future work.

On Tuesday, 9 April 2019 at 00:07:33 UTC, Mike Franklin wrote:
> 
>
> IMO Rust's borrow checker is the greatest language innovation since C++. Unfortunately, it's only available in Rust.  D's trying with DIP1000 and DIP25, but this blog post (https://atilaoncode.blog/2019/03/13/issues-dip1000-cant-yet-catch/) illustrates it has come up short.  I would love to hear if anyone has any ideas to close that hole.  Walter?

This actually was addressed on another thread.

https://forum.dlang.org/post/q6qu9g$30bj$1@digitalmars.com

TLDR: if you track down the offending code there is a @trusted block.

On Tuesday, 9 April 2019 at 01:40:24 UTC, jmh530 wrote:
> On Tuesday, 9 April 2019 at 00:07:33 UTC, Mike Franklin wrote:
>> 
>>
>> IMO Rust's borrow checker is the greatest language innovation since C++. Unfortunately, it's only available in Rust.  D's trying with DIP1000 and DIP25, but this blog post (https://atilaoncode.blog/2019/03/13/issues-dip1000-cant-yet-catch/) illustrates it has come up short.  I would love to hear if anyone has any ideas to close that hole.  Walter?
>
> This actually was addressed on another thread.
>
> https://forum.dlang.org/post/q6qu9g$30bj$1@digitalmars.com
>
> TLDR: if you track down the offending code there is a @trusted block.

Aha!  Thank you for taking the time to point me to that.

Mike

On 09/04/2019 01:07, Mike Franklin via Digitalmars-d wrote:
> IMO, Rust's only feature is the borrow checker; the rest of the language stinks.  But, statically-checked memory safety is so fantastic Rust is destined for success.  If D had an equally effective mechanism for memory safety and fearless concurrency, there would be no reason for Rust to exist, and there would be no competition for D.

It seems to me that Rust's real killer feature is that it is backed by Mozilla. How many major modern languages are really successful without major corporate backing? C# has Microsoft, Java had Sun/has Oracle[1], Javascript had Netscape (when Netscape mattered) as well as browser support from Microsoft (when that mattered), Go has Google, Swift has Apple. In comparison, D does not have big name backing (sorry, no offence intended!).

(I should add that I think that C and C++ are exceptions to this since they have, in comparison, been around for much longer and were able to get up a head of steam in their own standards-driven right before corporate backing came to matter in the same way that it seems to now).

And so even if D had the same (or better) features than Rust or Go, it
would still face competitors.

Thus it seems to me that technical improvements such as adding features to D to improve its concurrency and memory safety are certainly important, as I understand it, but *marketing* is important too. Is it possible that some major project with the profile of Firefox could be (partly) moved to D from C++?




( A bit about me: This is my first post here and I am not currently a D programmer. I joined this mail list to learn a bit more about D. I am currently working on a cross platform thick client application project in C# (yes, thick client app projects still exist). It is possible that future versions of this project could be written in one of Rust, D or Go, depending on how each language develops including issues like readability, wasm support, asynch/concurrency support, and cross platform GUI support ).




Footnote:-
1: Luckily for Java, it had already built up a head of steam in its own
right before Oracle came along.

-- 
Mark Rousell