On 2012-11-02 18:22, Walter Bright wrote:

> It's a very fair question.
>
> Manu works for Remedy Games, a developer of hit PC games, such as Max
> Payne and Alan Wake. He champions a team that is investigating
> committing to D for a major new game. This would be a huge design win
> for D, and an enormous boost for D. I view the most effective use of my
> time at the moment is to ensure that they can bet on D and win. Remedy's
> use of D for a high profile product will prove that D is ready and able
> for the big time, and that others can develop using D with confidence.
>
> Nearly all of what he needs the D community needs anyway, such as the
> big push for Win64 support and the SIMD support (more on that in another
> post).

I can absolutely see the point in this. Thanks for taking the time and explaining this.

-- 
/Jacob Carlborg

On 2012-11-02 18:46, Andrei Alexandrescu wrote:

> I should add that I'm also totally behind this. When Walter jumped into
> implementing SIMD support on a hunch, I completely disagreed, but that
> was a great decision.

I can absolutely understand why he did it but it would be really nice if you (Walter, Andrei and probably others as well) could be more transparent about things like these. I think this would really help the community.

-- 
/Jacob Carlborg

Le 02/11/2012 19:19, Jacob Carlborg a écrit :
> On 2012-11-02 18:46, Andrei Alexandrescu wrote:
>
>> I should add that I'm also totally behind this. When Walter jumped into
>> implementing SIMD support on a hunch, I completely disagreed, but that
>> was a great decision.
>
> I can absolutely understand why he did it but it would be really nice if
> you (Walter, Andrei and probably others as well) could be more
> transparent about things like these. I think this would really help the
> community.
>

I couldn't agree more.

On Friday, 2 November 2012 at 18:19:54 UTC, Jacob Carlborg wrote:
> On 2012-11-02 18:46, Andrei Alexandrescu wrote:
>
>> I should add that I'm also totally behind this. When Walter jumped into
>> implementing SIMD support on a hunch, I completely disagreed, but that
>> was a great decision.
>
> I can absolutely understand why he did it but it would be really nice if you (Walter, Andrei and probably others as well) could be more transparent about things like these. I think this would really help the community.

So true. Strong mid-term vision of rationales of main language developers really helps to reason about it.

On 11/2/2012 11:19 AM, Jacob Carlborg wrote:
> I can absolutely understand why he did it but it would be really nice if you
> (Walter, Andrei and probably others as well) could be more transparent about
> things like these. I think this would really help the community.

I apologize for being circumspect about this, but I have to respect peoples' privacy and I cleared what I posted about Remedy with them before posting it.

Le 02/11/2012 19:25, mist a écrit :
> On Friday, 2 November 2012 at 18:19:54 UTC, Jacob Carlborg wrote:
>> On 2012-11-02 18:46, Andrei Alexandrescu wrote:
>>
>>> I should add that I'm also totally behind this. When Walter jumped into
>>> implementing SIMD support on a hunch, I completely disagreed, but that
>>> was a great decision.
>>
>> I can absolutely understand why he did it but it would be really nice
>> if you (Walter, Andrei and probably others as well) could be more
>> transparent about things like these. I think this would really help
>> the community.
>
> So true. Strong mid-term vision of rationales of main language
> developers really helps to reason about it.

Yeah, and avoid noise and help people concentrate on main points.

Uh oh, I just caught wind of this thread! ;)

If there's a single argument I'd like to make in defense of the changes that have gone through as a result of my motivating, it's that they're either big tickets and mutually beneficial for the whole community anyway (DMD-win64), or relatively trivial (function prototypes + definitions) details that enable us to realistically consider proceeding with D commercially.

The SIMD types support may be considered a little grey, but I've encountered loads of general D users who appreciate the SIMD work already, and while it's certainly not trivial, it's nowhere the mammoth scale of task auto-vectorisation would be. I can't see it as a loss for the language or the community, and again, it has further enabled our consideration of D commercially; which I'd like to think is a general goal for the language.

I know plenty of you don't care about me or my industry, but I maintain that it's an entire industry in desperate need of salvation from C++, there's a lot of potential in the games industry to get fantastic value from using D, and I'm interested in proving that it's a realistic consideration.

I'd also like to note that we are also very conscious of the time Walter and other contributors have kindly offered to our support, and we are hopeful to be able to give reasonable value back to the community should it all go well for us, in whatever way that manifests...

On 2 November 2012 11:01, Jens Mueller <jens.k.mueller@gmx.de> wrote:

> Jacob Carlborg wrote:
> > On 2012-11-01 23:51, Walter Bright wrote:
> >
> > >What about all your feature requests? I think you've made more than anyone, by a factor of 10 at least!
> > >
> > >:-)
> > >
> > >As for Manu's request
> > >
> > >http://d.puremagic.com/issues/show_bug.cgi?id=8108
> > >
> > >I've gone over with him why he needs it, and there's no other reasonable way. He needs it for real code in a real application.
> >
> > This is quite interesting. Manu comes in from basically nowhere and fairly quickly manage to convince Walter to implement at least two feature requests, this one and the SIMD support. I'm not saying that they shouldn't have been implemented. Although I think something like AST macros could possible solve issue 8108 and a whole bunch of other features, a few already present in the language.
>
> I had the same thought when reading this. Very disappointing. An issue
> with zero votes is fixed instead of more important ones. Why do I vote
> anyway?
> Regarding SIMD I have the feeling that because it is built into the
> compiler static vectors have actually failed what they promised. I
> thought D proposed a portable way of vector operations such that you
> write
> float[4] = a[] + b[]
> and the compiler generates SIMD code for you.
>

As I mentioned above, I think that is a MUCH larger task, and probably
fairly unrealistic in the near-term regardless.
But secondly, it's just not that simple. (I can see Walter has already
addressed it in a previous post)

It's worth considering that a very significant portion of the silicon on
modern processors (well in excess of 50% on some chips) is dedicated to
hardware acceleration of SIMD/media functions. Until recently, D simply
offered no mechanism at all to interact with half of the silicone in your
box. That's a massive language hole.
At least as a starting point, low level access to this hardware is vital.
Portable libraries can be built using this technology, which are
immediately useful.

The definition of hardware SIMD types doesn't rule out possible future auto-vectorisation either. And you can probably access auto-vectorising backends right now if you use GDC or LDC.

On 2 November 2012 19:32, Andrej Mitrovic <andrej.mitrovich@gmail.com>wrote:

> On 11/2/12, Walter Bright <newshound2@digitalmars.com> wrote:
> > Manu works for Remedy Games, a developer of hit PC games, such as Max
> Payne
> > and
> > Alan Wake. He champions a team that is investigating committing to D for
> a
> > major
> > new game.
>
> Wow that's really cool!
>
> Manu if you know of any trivial-type bugs you'd like to get fixed (for example error messages) let me know so I can prioritize them. There's a ton of open issues and I usually select the ones for fixing by random. I've managed to fix a few so far. :)
>

Hey cheers man! :)
Actually, one thing that does consistently bite me are nonsensical error
messages. I'll start keeping tabs on them.

On 2 November 2012 20:02, Walter Bright <newshound2@digitalmars.com> wrote:

> On 11/2/2012 3:50 AM, Jens Mueller wrote:
> > Okay. For me they look the same. Can you elaborate, please? Assume I want to add two float vectors which is common in both games and scientific computing. The only difference is in games their length is usually 3 or 4 whereas in scientific computing they are of arbitrary length. Why do I need instrinsics to support the game setting?
>
> Another excellent question.
>
> Most languages have taken the "auto-vectorization" approach of reverse engineering loops to turn them into high level constructs, and then compiling the code into special SIMD instructions.
>
> How to do this is explained in detail in the (rare) book "The Software Vectorization Handbook" by Bik, which I fortunately was able to obtain a copy of.
>
> This struck me as a terrible approach, however. It just seemed stupid to try to teach the compiler to reverse engineer low level code into high level code. A better design would be to start with high level code. Hence, the appearance of D vector operations.
>
> The trouble with D vector operations, however, is that they are too general purpose. The SIMD instructions are very quirky, and it's easy to unwittingly and silently cause the compiler to generate absolutely terribly slow code. The reasons for that are the alignment requirements, coupled with the SIMD instructions not being orthogonal - some operations work for some types and not for others, in a way that is unintuitive unless you're carefully reading the SIMD specs.
>
> Just saying align(16) isn't good enough, as the vector ops work on slices and those slices aren't always aligned. So each one has to check alignment at runtime, which is murder on performance.
>
> If a particular vector op for a particular type has no SIMD support, then the compiler has to generate workaround code. This can also have terrible performance consequences.
>
> So the user writes vector code, benchmarks it, finds zero improvement, and the reasons why will be elusive to anyone but an expert SIMD programmer.
>
> (Auto-vectorizing technology has similar issues, pretty much meaning you won't get fast code out of it unless you've got a habit of examining the assembler output and tweaking as necessary.)
>
> Enter Manu, who has a lot of experience making SIMD work for games. His proposal was:
>
> 1. Have native SIMD types. This will guarantee alignment, and will guarantee a compile time error for SIMD types that are not supported by the CPU.
>
> 2. Have the compiler issue an error for SIMD operations that are not supported by the CPU, rather than silently generating inefficient workaround code.
>
> 3. There are all kinds of weird but highly useful SIMD instructions that don't have a straightforward representation in high level code, such as saturated arithmetic. Manu's answer was to expose these instructions via intrinsics, so the user can string them together, be sure that they will generate real SIMD instructions, while the compiler can deal with register allocation.
>

Well, I wouldn't claim any credit for the approach ;) .. I think this is
the standard for maximum performance, and also very well understood.
But the thing that excites me most is the potential quality of libraries
that can be built on top. D has so much potential to extend on this SIMD
foundation with it's templates being able to intelligently handle far more
context specific situations.
What we do already in other languages will be far more convenient, more
portable, and possibly even produce better code in D. And the biggest
bonus, it will be readable! :)

I think it's a low risk investment, and it doesn't prohibit higher level support in the future.


This approach works, is inlineable, generates code as good as hand-built
> assembler, and is useable by regular programmers.
>
> I won't say there aren't better approaches, but this one we know works.
>

Aye, and it's relatively un-intrusive too. Some new types and a few intrinsics, build useful libraries on top. It shouldn't have complex side effects, and if offers something that was sorely missing from the language today.

On 2 November 2012 20:10, deadalnix <deadalnix@gmail.com> wrote:

> Le 02/11/2012 18:46, Andrei Alexandrescu a écrit :
>
>  On 11/2/12 1:22 PM, Walter Bright wrote:
>>
>>> On 11/2/2012 2:01 AM, Jens Mueller wrote:
>>> > I had the same thought when reading this. Very disappointing. An issue with zero votes is fixed instead of more important ones.
>>>
>>> It's a very fair question.
>>>
>>> Manu works for Remedy Games, a developer of hit PC games, such as Max Payne and Alan Wake. He champions a team that is investigating committing to D for a major new game. This would be a huge design win for D, and an enormous boost for D. I view the most effective use of my time at the moment is to ensure that they can bet on D and win. Remedy's use of D for a high profile product will prove that D is ready and able for the big time, and that others can develop using D with confidence.
>>>
>>> Nearly all of what he needs the D community needs anyway, such as the big push for Win64 support and the SIMD support (more on that in another post).
>>>
>>
>> I should add that I'm also totally behind this. When Walter jumped into implementing SIMD support on a hunch, I completely disagreed, but that was a great decision.
>>
>> Andrei
>>
>
> I still don't understand the benefice over align(16) float[4] .
>

It's a mechanism to insist on register placement and usage semantics. On the vast majority of architectures, floats and simd are absolutely incompatible. It's awfully dangerous to describe them both with a single type when they are mutually exclusive...