January 05, 2012
>
> D is not a compiler, it is a language. Furthermore it is not true that DMDs backend is rubbish and there are already more backends than just the DMC backend.
>

Sorry, I was generalising a little general in that claim. And where I say
'rubbish', I was drawing comparison to the maturity of C compilers for x86,
which STILL have trouble and make lots of mistakes with centuries of man
hours of work.
DMD has inferior code gen (there was a post last night comparing some
disassemblies of trivial programs with GDC), will probably never rival GCC,
that's fine, it's a reference, I get that.
But to say that using GDC will magically fix code gen is also false. I'm
not familiar with the GCC code, so I may be wrong, but my understanding is
that there is frontend work, and frontend-GCC glue work that will allow for
back end optimisation (which GCC can do quite well) to work properly. This
is still a lot of work for a small OSS team.
I also wonder if the D language provides some opportunities for
optimisation that aren't expressible in other languages, and therefore may
not already have an expression in the GCC back end... so I can imagine some
of future optimisations frequently discussed in this forum won't just
magically appear with GCC/LLVM maturity. I can't imagine Iain and co
extending the GCC back end to support some obscure D optimisations
happening any time soon.

The point I was making (which you seem to have missed thanks to my inflamatory comment ;), was that I don't have faith that compiler maturity will solve all these problems in prompt time, and even if they do for x86, what about for less common architectures that receive a lot less love (as is also the case in C)?

I make this argument in support of the language expressing optimal
constructs with ease and by default, rather than expressing some concept
that feels nice to programmers, but puts a burden on the
whole-program-optimiser to fix.
For example, virtual-by-default RELIES on whole-program-optimisation to
fix, whereas final by default has no performance implications, and will
produce the best code automatically.

I think it would be nice if you stopped spreading FUD. You seem to have
> reasonable requests.
>

Perhaps a fair request, but I only do this because after a couple of months
now, I have a good measure of FUD, and I receive very little response to
anything I've raised that would make me feel otherwise.
The most distressing thing to me is the pattern I see where most of my more
trivial (but still significant) points are outright dismissed, and the hard
ones are ignored, rather than reasonably argued and putting my FUD to rest.
:)

I also accept that I produce very little evidence to support any of my claims, so I'm easy to ignore, but this is because all my work and experience is commercial, private, and I can't easily extract anything without wasting a lot of work time to present it... not to mention breaking NDA's. Most problem cases aren't trivial, require a large context to prove with benchmarks. I can't easily write a few lines and say "here you go". At some level I'd like to think people would accept the word of a seasoned game engine dev who's genuinely interested in adopting the language for that sort of work, but I completely understand those who are skeptical. ;)


January 05, 2012
On 1/5/12 10:49 AM, Manu wrote:
> I make this argument in support of the language expressing optimal
> constructs with ease and by default, rather than expressing some concept
> that feels nice to programmers, but puts a burden on the
> whole-program-optimiser to fix.
> For example, virtual-by-default RELIES on whole-program-optimisation to
> fix, whereas final by default has no performance implications, and will
> produce the best code automatically.

Your point is well meaning. I trust you understood and internalized your options outside a language change: using final or private in interfaces and classes, using struct instead of class, switching design to static polymorphism etc. Our assessment is that these work very well, promote good class hierarchy design, require reasonably little work from the programmer, and do not need advanced compiler optimizations.

The D programming language is stabilizing. Making a change of such a magnitude is not negotiable, and moreover we believe the current design is very good in that regard so we are twice as motivated to keep it.

At this point you need to evaluate whether you can live with this annoyance or forgo use of the language.


Thanks,

Andrei

January 05, 2012
On 5 January 2012 19:35, Andrei Alexandrescu <SeeWebsiteForEmail@erdani.org>wrote:

> On 1/5/12 10:49 AM, Manu wrote:
>
>> I make this argument in support of the language expressing optimal
>> constructs with ease and by default, rather than expressing some concept
>> that feels nice to programmers, but puts a burden on the
>> whole-program-optimiser to fix.
>> For example, virtual-by-default RELIES on whole-program-optimisation to
>> fix, whereas final by default has no performance implications, and will
>> produce the best code automatically.
>>
>
> Your point is well meaning. I trust you understood and internalized your options outside a language change: using final or private in interfaces and classes, using struct instead of class, switching design to static polymorphism etc. Our assessment is that these work very well, promote good class hierarchy design, require reasonably little work from the programmer, and do not need advanced compiler optimizations.
>
> The D programming language is stabilizing. Making a change of such a magnitude is not negotiable, and moreover we believe the current design is very good in that regard so we are twice as motivated to keep it.
>
> At this point you need to evaluate whether you can live with this annoyance or forgo use of the language.
>

I do realise all the implementation details you suggest. My core point is
that this is dangerous. A new and/or junior programmer is not likely to
know all that.. they will most probably type 'class', and then start typing
methods. Why would they do anything else? Every other language I can think
of trains them to do that.
The D catch phrase 'the right thing is the easiest thing to do' doesn't
seem to hold up here... although that does depend on your point of view on
'right', for which I've made my argument numerous times, I'll desist from
here on ;)

I also realise this issue is non-negotiable. I said that in a previous email, and I'm prepared to live with it (although I wonder if a compiler option would be possible?)... As said, I still felt it was important to raise this one for conversations sake, and to make sure this point of view towards issues like this are more seriously considered in future.

That said, this is just one of numerous issues myself and the OP raised. I
don't know why this one became the most popular for discussion... my
suspicion is that is because this is the easiest of my complaints to
dismiss and shut down ;)
This is also the least interesting to me personally of the issues I and the
OP raised (knowing it can't be changed)... I'd rather be discussing the
others ;)


January 05, 2012
On Jan 5, 2012, at 10:02 AM, Manu wrote:
> 
> That said, this is just one of numerous issues myself and the OP raised. I don't know why this one became the most popular for discussion... my suspicion is that is because this is the easiest of my complaints to dismiss and shut down ;)

It's also about the only language change among the issues you mentioned.  Most of the others are QOI issues for compiler vendors.  What I've been curious about is if you really have a need for the performance that would be granted by these features, or if this is more of an idealistic issue.
January 05, 2012
Manu <turkeyman@gmail.com> writes:
> It's rare to step through optimised code. You tend to debug and step in debug/ unoptimised builds, where inline functions are usually not even inlined, and code flow still looks natural, and easy to follow.. This saves lots of time. C/C++ macros present the same problem of not being able to step and inspect values. Most industry programmers I work with tend to avoid macros for this reason above all others.

I do it all the time.  Normally I with optimized builds because our code takes a long time to run, so if I can find the problem without going to the debug build, I save a fair bit of time.

I find that you get used to the weirdnesses that show up when stepping through optimized code.  I'm probably able to find problems I'm looking for 60-70% of the time without resorting to using the debug build.

Jerry
January 05, 2012
> A language defined 128bit SIMD type would be fine for basically all architectures. Even though they support different operations on these registers, the size and allocation patterns are always the same across all architectures; 128 bits, 16byte aligned, etc. This allows at minimum platform independent expression of structures containing simd data, and calling of functions passing these types as args.

You forgot about AVX. It uses 256 bit registers and is supported in new Intel and AMD processors.

January 05, 2012
On 5 January 2012 21:41, Sean Kelly <sean@invisibleduck.org> wrote:

> On Jan 5, 2012, at 10:02 AM, Manu wrote:
> >
> > That said, this is just one of numerous issues myself and the OP raised.
> I don't know why this one became the most popular for discussion... my suspicion is that is because this is the easiest of my complaints to dismiss and shut down ;)
>
> It's also about the only language change among the issues you mentioned.
>  Most of the others are QOI issues for compiler vendors.  What I've been
> curious about is if you really have a need for the performance that would
> be granted by these features, or if this is more of an idealistic issue.
>

I think they are *all* language requests. They could be implemented by 3rd party compilers, but these things are certainly nice to standardise in the language, or you end up with C, which is a mess I'm trying to escape.

Of the topics been discussed:
  * vector type - I can't be expected to write a game without using the
vector hardware... I'd rather use a portable standardised type than a GDC
extension. Why the resistance? I haven't heard any good arguments against,
other than some murmurings about float[4] as the official syntax, which I
gave detailed arguments against.

  * inline asm supporting pseudo regs - As an extension of using the vector
hardware, I'll need inline asm, and inline assembly without pseudo regs is
pretty useless... it's mandatory that the compiler shedule the register
allocation otherwise inline asm will most likely be an un-optimisation. If
D had pseudo regs in its inline assembler, it would make it REALLY
attractive for embedded systems programmers.
    In lieu of that, I need to use opcode intrinsics instead, which I
believe GDC exposes, but again, I'm done with C and versioning (#ifdef-ing)
every compiler I intend to use. Why not standardise these things? At least
put the intrinsics in the standard lib...

  * __restrict - Not a deal breaker, but console game dev industry uses
this all the time. There are countless articles on the topic (many are
private or on console vendor forums). If this is not standardised in the
language, GDC will still expose it I'm sure, fragmenting the language.

  * __forceinline - I'd rather have a proper keyword than using tricks like
mixins and stuff as have been discussed. The reason for this is debugging.
Code is not-inlined in debug builds, looks&feels like normal code, can
still evaluate, and step like regular code... and guarantee that it inlines
properly when optimised. This just saves time; no frills debugging.

  * multiple return values - This is a purely idealistic feature request,
but would lead to some really nice optimisations while retaining very tidy
code if implemented properly. Other languages support this, it's a nice
modern feature... why not have it in D?

I couldn't release the games I do without efficient use of vector hardware
and __restrict used in appropriate places. I use them every day, and I
wouldn't begin a project in D without knowing that these features are
supported, or are definitely coming to the language.
On fixed hardware systems, the bar is high and it's very competitive... you
can't waste processor time.
Trust me when I say that using __restrict appropriately might lead to twice
as many particles on screen, or allowing more physics bodies, or more
accurate simulation. SIMD hardware is mandatory, and will usually increase
performance 2-5 times in my experience. The type of code that usually
benefits the most ranges from particle simulation, collision/physics,
procedural geometry/texturing, and funnily enough, memcopy ;) .. stock
memcopy doesn't take advantage of 16byte simd registers for copying memory,
Ill bet D doesn't either.
A little while back I rewrote a bitplane compositor (raw binary munging,
not a typical vector hardware job) in VMX. Very tricky, but it was around
10 times faster... which is good, because our game had to hold rock solid
60fps, and it saved the build and even allowed us to add some more nice
features ;)

If D can't compete with, or beat C, it won't be used in this market on high
end products, though perhaps still viable on smaller/not-cutting-edge
projects if productivity is considered more important.
Engine programmers are thoroughly aware of code generation, and in C,
tricks/techniques to coerce the compiler to generate the code you want to
see are common place... and often very, very ugly. I think the industry
would be very impressed and enthusiastic if D were able to generate the
best possible code with conventional and elegant language semantics,
without annoying tricks or proprietary compiler extensions to do so.


January 05, 2012
On 1/5/2012 1:16 AM, Manu wrote:
> On 5 January 2012 03:06, Walter Bright <newshound2@digitalmars.com
> <mailto:newshound2@digitalmars.com>> wrote:
>
>     On 1/4/2012 4:30 PM, Sean Kelly wrote:
>
>         If a library is written without consideration to what is virtual and what is
>         not, its performance will be the least of your problems.
>
>
>     I agree. Such is a massive failure in designing a polymorphic type, and the
>     language can't help with that.
>
>
> I don't follow.. how is someone failing (or forgetting) to type 'final' a
> "massive design failure"? It's not a design failure, it's not even 'wrong'...
> it's INEVITABLE.
> And the language CAN help with that, by making expensive operations require
> explicit declaration.


In any class design, one must decide which functions are overrideable and which are not. The language cannot do it for you; certainly not by switching around the default behavior.


> At least make a compiler flag so I can disable virtual-by-default for my project...?

I'm afraid that such a switch would have disastrous results, because it fundamentally alters the meaning of existing code.
January 05, 2012
On 5 January 2012 21:41, Sean Kelly <sean@invisibleduck.org> wrote:

> On Jan 5, 2012, at 10:02 AM, Manu wrote:
> >
> > That said, this is just one of numerous issues myself and the OP raised.
> I don't know why this one became the most popular for discussion... my suspicion is that is because this is the easiest of my complaints to dismiss and shut down ;)
>
> It's also about the only language change among the issues you mentioned.
>  Most of the others are QOI issues for compiler vendors.  What I've been
> curious about is if you really have a need for the performance that would
> be granted by these features, or if this is more of an idealistic issue.
>

I think they are *all* language requests. They could be implemented by 3rd party compilers, but these things are certainly nice to standardise in the language, or you end up with C, which is a mess I'm trying to escape.

Of the topics been discussed:
  * vector type - I can't be expected to write a game without using the
vector hardware... I'd rather use a portable standardised type than a GDC
extension. Why the resistance? I haven't heard any good arguments against,
other than some murmurings about float[4] as the official syntax, which I
gave detailed arguments against.

  * inline asm supporting pseudo regs - As an extension of using the vector
hardware, I'll need inline asm, and inline assembly without pseudo regs is
pretty useless... it's mandatory that the compiler shedule the register
allocation otherwise inline asm will most likely be an un-optimisation. If
D had pseudo regs in its inline assembler, it would make it REALLY
attractive for embedded systems programmers.
    In lieu of that, I need to use opcode intrinsics instead, which I
believe GDC exposes, but again, I'm done with C and versioning (#ifdef-ing)
every compiler I intend to use. Why not standardise these things? At least
put the intrinsics in the standard lib...

  * __restrict - Not a deal breaker, but console game dev industry uses
this all the time. There are countless articles on the topic (many are
private or on console vendor forums). If this is not standardised in the
language, GDC will still expose it I'm sure, fragmenting the language.

  * __forceinline - I'd rather have a proper keyword than using tricks like
mixins and stuff as have been discussed. The reason for this is debugging.
Code is not-inlined in debug builds, looks&feels like normal code, can
still evaluate, and step like regular code... and guarantee that it inlines
properly when optimised. This just saves time; no frills debugging.

  * multiple return values - This is a purely idealistic feature request,
but would lead to some really nice optimisations while retaining very tidy
code if implemented properly. Other languages support this, it's a nice
modern feature... why not have it in D?

I couldn't release the games I do without efficient use of vector hardware
and __restrict used in appropriate places. I use them every day, and I
wouldn't begin a project in D without knowing that these features are
supported, or are definitely coming to the language.
On fixed hardware systems, the bar is high and it's very competitive... you
can't waste processor time.
Trust me when I say that using __restrict appropriately might lead to twice
as many particles on screen, or allowing more physics bodies, or more
accurate simulation. SIMD hardware is mandatory, and will usually increase
performance 2-5 times in my experience. The type of code that usually
benefits the most ranges from particle simulation, collision/physics,
procedural geometry/texturing, and funnily enough, memcopy ;) .. stock
memcopy doesn't take advantage of 16byte simd registers for copying memory,
Ill bet D doesn't either.
A little while back I rewrote a bitplane compositor (raw binary munging,
not a typical vector hardware job) in VMX. Very tricky, but it was around
10 times faster... which is good, because our game had to hold rock solid
60fps, and it saved the build and even allowed us to add some more nice
features ;)

If D can't compete with, or beat C, it won't be used in this market on high
end products, though perhaps still viable on smaller/not-cutting-edge
projects if productivity is considered more important.
Engine programmers are thoroughly aware of code generation, and in C,
tricks/techniques to coerce the compiler to generate the code you want to
see are common place... and often very, very ugly. I think the industry
would be very impressed and enthusiastic if D were able to generate the
best possible code with conventional and elegant language semantics,
without annoying tricks or proprietary compiler extensions to do so.


January 05, 2012
On 5 January 2012 22:33, a <a@a.com> wrote:

>
> > A language defined 128bit SIMD type would be fine for basically all architectures. Even though they support different operations on these registers, the size and allocation patterns are always the same across
> all
> > architectures; 128 bits, 16byte aligned, etc. This allows at minimum platform independent expression of structures containing simd data, and calling of functions passing these types as args.
>
> You forgot about AVX. It uses 256 bit registers and is supported in new Intel and AMD processors.
>

AVX is another type, a new 256bit type, as double is to float, and should also have a keyword ;)