In the vein of keeping our eyes open to the world around us, I found this article fascinating:

https://blogs.unity3d.com/2019/02/26/on-dots-c-c/

Being a high-performance game engine, Unity3D is right smack in the middle of D's strengths and potential.

Traditionally, Unity's core engine is written in C++ (just like pretty much every other high-performance game engine), whereas the game code is written in C# (which is first compiled down to CLIR and then, depending on the target platform, either run directly on Mono or further compiled down to C++/native.)

They have the usual group of complaints about C++ (long compilation, headers, and concurrency difficulties like races, sharing etc). And they also add another C++ gripe we don't hear about quite as often: Lack of guarantees and control over the generated code. (VERY important for game engine developers, as I'm sure Manu can attest.)

So they have a rather interesting solution: They're switching from C++ to a subset of C#, combined with some custom libs and IL-based tooling. Sounds slightly odd, but they make a very convincing case for it.

I think it's very much in D's best interest to be aware of these problems/solutions/rationales Unity presents here, as they would easily apply to other C++ users and thus a key potential audience for D.

On Wednesday, 27 February 2019 at 19:13:11 UTC, Nick Sabalausky (Abscissa) wrote:
> So they have a rather interesting solution: They're switching from C++ to a subset of C#, combined with some custom libs and IL-based tooling. Sounds slightly odd, but they make a very convincing case for it.

This is exactly where D has the largest advantage over C#. I find that C# is generally good to code with, as long as you don't care about performance beyond avoiding big O crimes. But when you try to do stuff like using structs when you don't need polymorhism or trying to minimize allocation when working with an array, it gets inpractical. With D, of course you still need to put a bit more thought in, but it doesn't feel like the language is fighting your optimization efforts.

I had seen it, it looks really nice. It reminds me of what Weka do using D, not as is, but heavily modified by themselves (specially Phobos). It's an extremely restricted subset of C# and .NET that they have implemented.

I guess Unity's choice for C# was obvious, since it's the same language they were already using for user scripts. Otherwise they might have considered D or others.

On Thursday, 28 February 2019 at 12:05:54 UTC, Dukc wrote:
> when you try to do stuff like using structs when you don't need polymorphism or trying to minimize allocation when working with an array, it gets impractical.

How so?

Precisely the split between GC classes and stack structs without polymorphism, is one thing C# and D have in exactly the same way (and  unlike C++, Java or other languages I know).

On 2/28/19 7:05 AM, Dukc wrote:
> 
> This is exactly where D has the largest advantage over C#. I find that C# is generally good to code with, as long as you don't care about performance beyond avoiding big O crimes. But when you try to do stuff like using structs when you don't need polymorhism or trying to minimize allocation when working with an array, it gets inpractical. With D, of course you still need to put a bit more thought in, but it doesn't feel like the language is fighting your optimization efforts.

Yea, at one point, I used to be just as big a fan of C# as D, but eventually C#'s difficulty of low-level control and lack of expressiveness (compared to D) pushed me away. Although I do still consider it one of the few languages I don't actually hate.

Nonetheless, Unity seems to have very much addressed the "low-level control" problem with plain-C#. As as for the expressiveness, well, like any major game engine developer, they're coming from C++, so as far as a D-like level of expressiveness, they wouldn't have had that before, so they're not really loosing much in that area.

What I think is important for us to look at is: How does D's offering stack up to HPC#, and what could we do to improve?

To be honest, I'm not entirely happy with how D compares:

+ Expressiveness: The only aspects where I think D has a clear advantage.

+ Memory Layout: We're no worse, but HPC# seems to have caught up to us.

- "Performance is correctness": I could be wrong, and I hope I am, but I'm not sure D does much better than C++ in this area. Certainly not as well as HPC# appears to do.

- Cross-architecture: D has improved here from the past, but it's still nowhere near as mature on as wide a range of platforms as Unity is.

- Machine code viewer: We have disassembly and such, but judging by their screenshot, no tool that's quite that nice.

+ Memory Safety: D's probably about on par here.

- Concurrency Safety: We're way beyond many languages, but we have nothing that goes nearly as far as the job scheduling system they describe.

Being basically a better C++, this is exactly the sort of software that D should excel at, so it's rather disappointing how D seems to stack up to HPC#. We *should* be absolutely demolishing it.

On Thu, Feb 28, 2019 at 9:10 AM Nick Sabalausky (Abscissa) via Digitalmars-d <digitalmars-d@puremagic.com> wrote:
>
> On 2/28/19 7:05 AM, Dukc wrote:
> >
> > This is exactly where D has the largest advantage over C#. I find that C# is generally good to code with, as long as you don't care about performance beyond avoiding big O crimes. But when you try to do stuff like using structs when you don't need polymorhism or trying to minimize allocation when working with an array, it gets inpractical. With D, of course you still need to put a bit more thought in, but it doesn't feel like the language is fighting your optimization efforts.
>
> Yea, at one point, I used to be just as big a fan of C# as D, but eventually C#'s difficulty of low-level control and lack of expressiveness (compared to D) pushed me away. Although I do still consider it one of the few languages I don't actually hate.
>
> Nonetheless, Unity seems to have very much addressed the "low-level control" problem with plain-C#. As as for the expressiveness, well, like any major game engine developer, they're coming from C++, so as far as a D-like level of expressiveness, they wouldn't have had that before, so they're not really loosing much in that area.
>
> What I think is important for us to look at is: How does D's offering stack up to HPC#, and what could we do to improve?

Their work you describe is almost exactly the work I've been doing for
the last however long.
This is the best opportunity I've ever had for D to demonstrate
motivating value before me; there are *so many* ways we can make
compelling arguments in favour of C++, but somehow we're still just
not quite there.

I've raised a bunch of issues I've been facing from this angle recently; the `shared` stuff, dll stuff, inline stuff.

> To be honest, I'm not entirely happy with how D compares:
>
> + Expressiveness: The only aspects where I think D has a clear advantage.

Yes, and no. DIP1016 failed, and for all my colleagues, that is the
most annoying thing that appears, every single time, within minutes of
opening their code editors.
It's proven hard to keep people excited and motivated, when they're on
the back foot over something so trivial right off the mark.

Implicit construction is also a major deficiency we have, and people tend to seriously struggle with that. `alias this` as a solution often feels awkward, and only goes so far.

> + Memory Layout: We're no worse, but HPC# seems to have caught up to us.

/agree, there's no meaningful distinction here. D has expressions which can make working with buffers a little but nicer (slices in particular).

> - "Performance is correctness": I could be wrong, and I hope I am, but I'm not sure D does much better than C++ in this area. Certainly not as well as HPC# appears to do.

No linear metric exists, but in practise, I think D often does worse
than C++ actually.
I have to work to write performant D code, but when I do work for it,
it's usually very easy to beat C++.

HPC# has a razor focus on this problem set, we can only compete by developing frameworks which facilitate appropriate patterns for the problem space.

...and that's hard, because my experience so far, is that we really
need to lean on `shared` to do a good job here, but `shared` is
broken.
I started a big thread on shared a while back, nothing came out of it.
If someone wants to fix shared (remove read/write access), that would
be absolutely mega!

> - Cross-architecture: D has improved here from the past, but it's still nowhere near as mature on as wide a range of platforms as Unity is.

I think this is actually okay in a practical sense. I don't think D has an answer for PS4, but all other targets I'm aware of are okay.

> - Machine code viewer: We have disassembly and such, but judging by their screenshot, no tool that's quite that nice.

This is a function of VisualD having low man-power. VisualD has a nice
feature which will compile and disassemble a single source file and
focus on the code at your cursor location, so you can get a quick
disassembly of the code as you're editing it.
The implementation is a little big janky, it needs a little more
polish, but it could get there.

People need to care about VisualStudio to nail this checkmark. The
entire industry uses VS, 100's of thousands of native code developers,
there are practically no exceptions.
VS is not popular in this forum, but people NEED TO CARE about it if
they want to succeed generally, even if they don't use it themselves.
It's the most important piece of tooling by lightyears.

> + Memory Safety: D's probably about on par here.

Not really. It's almost all talk at the moment with little substance. `shared` is a gaping safety violation, escape analysis isn't baked, we still don't have ref-counting.

> - Concurrency Safety: We're way beyond many languages, but we have
> nothing that goes nearly as far as the job scheduling system they describe.

Are we? What do we have?
I argue that we have literally nothing. Shared is straight-up broken,
and `const` is often such that you can't use it... what else is there
to say on this story?

I write job scheduling systems of the kind they describe, that's my job. I tried to work one in D. I started that thread about `shared`, it went no where... I write it in C++ instead with no meaningful loss of safety (ie, there is none either way, and we depend on convention). Infact, in C++ we have const, which helps us a lot. In D we don't even have `const`, because it falls over so often... so we have lost a key typesafety tool to assist with concurrent work. We can use `const` to statically enforce read-only access to shared data. In D, that often doesn't work out.

> Being basically a better C++, this is exactly the sort of software that D should excel at, so it's rather disappointing how D seems to stack up to HPC#. We *should* be absolutely demolishing it.

I agree, we should we killing this. I've asked for help over the last
6 months, we've made no progress.
`shared`s broken, inline's broken.
The thread about making -H (.di output) also able to output .h files
is a big part of it.
We have a lot of fundamental issues too, some are in progress like
copy construction, maybe move construction?

On Thursday, 28 February 2019 at 20:31:34 UTC, Manu wrote:
> 
> If someone wants to fix shared (remove read/write access), that would
> be absolutely mega!
>

I don't really use shared, so I'm confused by the "remove read/write access". I thought the point of shared was to only allow atomic read/write access.

On Thu, Feb 28, 2019 at 12:55 PM jmh530 via Digitalmars-d <digitalmars-d@puremagic.com> wrote:
>
> On Thursday, 28 February 2019 at 20:31:34 UTC, Manu wrote:
> >
> > If someone wants to fix shared (remove read/write access), that
> > would
> > be absolutely mega!
> >
>
> I don't really use shared, so I'm confused by the "remove read/write access". I thought the point of shared was to only allow atomic read/write access.

I'm not sure what the 'point' of shared is exactly in it's current incarnation... but it's definitely not that! ;)

On Thursday, 28 February 2019 at 20:53:01 UTC, jmh530 wrote:
> On Thursday, 28 February 2019 at 20:31:34 UTC, Manu wrote:
>> 
>> If someone wants to fix shared (remove read/write access), that would
>> be absolutely mega!
>>
>
> I don't really use shared, so I'm confused by the "remove read/write access". I thought the point of shared was to only allow atomic read/write access.

From what I gather it's to prevent using functions in the class that aren't marked with shared.

You can modify members that are shared for a struct:

struct A {
   int a;
   void setA(int v) shared {
       synchronized {
          a = v;
       }
   }
}

shared A a;
a.a = 10; // shouldn't be ok

This kind of defeats the purpose of shared, it can't guarantee that this is safe. It should prevent modifying the struct, except through functions marked as shared.

Seems Walter is against this though. I don't think anyone is going to waste their time on something that Walter already has a biased opinion against.

On 2/28/19 3:31 PM, Manu wrote:
> 
> I've raised a bunch of issues I've been facing from this angle
> recently; the `shared` stuff, dll stuff, inline stuff.

My sympathies, I feel your pain :/

But on a side note, this makes me think: We could *really* use a different format for all this argumentative stuff. Forums are too transient. What we need is something like a Wiki, but organized like this:

- Ideas
   - Arguments For/Against, and/or possible Cautions
      - Rebuttals
         - Counter-Rebuttals

...For everything, all in one canonical place. Wouldn't that be nice? I get so tired of everyone's arguments about everything having no viewable structure whatsoever...it's no wonder nobody's arguments ever get anywhere!!! This would be so much more practical as a standard base-of-operations for (hopefully) a meritocracy, don't you think?

>> To be honest, I'm not entirely happy with how D compares:
>>
>> + Expressiveness: The only aspects where I think D has a clear advantage.
> 
> Yes, and no. DIP1016 failed, and for all my colleagues, that is the
> most annoying thing that appears, every single time, within minutes of
> opening their code editors.
> It's proven hard to keep people excited and motivated, when they're on
> the back foot over something so trivial right off the mark.
> 

Ugh, I had to look up "DIP1016"...It's rvalue references, and...just...ugh...

See, this is something we should've had ten freaking years ago. It's so freaking basic, it's so freaking obvious. I mean, I'm all for the *idea* of some sort of DIP system, as long as it isn't too red-tape-y. Heck, I started working on a DIP rough draft recently and the need for technical details has really *helped* it, but...rvalue references?? That should never have even needed a DIP: You make a freaking temporary as necessary. Done! Worried about start/end lifetime of the temporary? Interactions? It's pretty freaking obvious what choices do/don't work, and if we *still* get it wrong, the test suite will tell us. That's the *point* of the test suite - to tell us when/how we've broken something. But NO! We've gotta define a manual process to soak up as much time as it takes to catch things BEFORE the test suite has a chance to!

>> - Cross-architecture: D has improved here from the past, but it's still
>> nowhere near as mature on as wide a range of platforms as Unity is.
> 
> I think this is actually okay in a practical sense. I don't think D
> has an answer for PS4, but all other targets I'm aware of are okay.

This surprises me actually. My perception was that the PS4/XB1 were very on-par with each other in terms of architecture: Both are x64-based with PC-like GPUs (and without all the hardware variation within a single platform, like on desktop/laptop). Wouldn't have expected any significant differences regarding a compiler's ability to generate machine code. Or is this more an OS issue than a hardware one? (Or not allowed to say?)

> 
>> - Machine code viewer: We have disassembly and such, but judging by
>> their screenshot, no tool that's quite that nice.
> 
> This is a function of VisualD having low man-power. VisualD has a nice
> feature which will compile and disassemble a single source file and
> focus on the code at your cursor location, so you can get a quick
> disassembly of the code as you're editing it.
> The implementation is a little big janky, it needs a little more
> polish, but it could get there.

I guess that sounds promising at least, y'know, considering. 'Least it's not something fundamental that requires changing any higher-ups minds ;)

> People need to care about VisualStudio to nail this checkmark. The
> entire industry uses VS, 100's of thousands of native code developers,
> there are practically no exceptions.
> VS is not popular in this forum, but people NEED TO CARE about it if
> they want to succeed generally, even if they don't use it themselves.
> It's the most important piece of tooling by lightyears.

A good point. I guess at some point D has to decide where its priorities are: Increased adoption of D vs meeting the needs of existing D users, or a more equitable balance of both. Or perhaps D's already made it's decision here...*shudder*...

>> - Concurrency Safety: We're way beyond many languages, but we have
>> nothing that goes nearly as far as the job scheduling system they describe.
> 
> Are we? What do we have?
> I argue that we have literally nothing. Shared is straight-up broken,
> and `const` is often such that you can't use it... what else is there
> to say on this story?

I guess I was just thinking of TLS-by-default. Perhaps I overstated D's case.

> I write job scheduling systems of the kind they describe, that's my
> job. I tried to work one in D. I started that thread about `shared`,
> it went no where... I write it in C++ instead with no meaningful loss
> of safety (ie, there is none either way, and we depend on convention).

That's sad to hear :(

> Infact, in C++ we have const, which helps us a lot. In D we don't even
> have `const`, because it falls over so often...

I admit, I don't really use it much myself. Partly because of bad experiences attempting to do so.

Do you think the lack of both tailconst and logical-const are the ultimate issues here, or does it go deeper than that?

>> Being basically a better C++, this is exactly the sort of software that
>> D should excel at, so it's rather disappointing how D seems to stack up
>> to HPC#. We *should* be absolutely demolishing it.
> 
> I agree, we should we killing this. I've asked for help over the last
> 6 months, we've made no progress.
> `shared`s broken, inline's broken.
> The thread about making -H (.di output) also able to output .h files
> is a big part of it.
> We have a lot of fundamental issues too, some are in progress like
> copy construction, maybe move construction?
> 

I honesty find D very frustrating these days. On one hand, it's still by far my favorite language, due to expressiveness, ranges, the low-level abilities it does have (unlike most languages these days), and all of the basic common-sense pragmatism that went into its early design about ten or so years ago. But OTOH, I feel the focus on pragmatism and common sense has been thrown straight out the window for the past many years, and its already accelerating down the same paths that led C++ to become the mess that it is today. I feel like D's obsessively doing *exactly* what Scott Meyers warned us against, in stark contrast to the early principles that made D worthwhile in the first place.

On Thursday, 28 February 2019 at 12:05:54 UTC, Dukc wrote:
> On Wednesday, 27 February 2019 at 19:13:11 UTC, Nick Sabalausky (Abscissa) wrote:
>> So they have a rather interesting solution: They're switching from C++ to a subset of C#, combined with some custom libs and IL-based tooling. Sounds slightly odd, but they make a very convincing case for it.
>
> This is exactly where D has the largest advantage over C#. I find that C# is generally good to code with, as long as you don't care about performance beyond avoiding big O crimes. But when you try to do stuff like using structs when you don't need polymorhism or trying to minimize allocation when working with an array, it gets inpractical. With D, of course you still need to put a bit more thought in, but it doesn't feel like the language is fighting your optimization efforts.

Only if talking about C# up to version 6.

Versions 7 and 8 incorporate many of the Midori lessons, some of which even landend on C++.