On Saturday, January 14, 2012 14:58:19 Andrei Alexandrescu wrote:
> IMHO, fixing const postblit (and the last mile of const) an immediate issue, followed by shared and threads. In light of that, working on simd now appears even more like a waste of time.

I completely agree that const and shared issues are more important than SIMD, given that they affect far more people and have to do with some of the most frustrating bugs. So, I'm not sure that this work on SIMD is particularly well-timed. But I very much doubt that it's a waste of time. It could really help make D acceptable to some of the folks who would otherwise insist on using C or C++ for efficiency. And it's definitely addressing the complaints of some of D's users. Still, I'd very much like to see the remaining core language issues resolved sooner rather than implement enhancements like this right now.

- Jonathan M Davis

Jonathan M Davis:

> I completely agree that const and shared issues are more important than SIMD, given that they affect far more people and have to do with some of the most frustrating bugs. So, I'm not sure that this work on SIMD is particularly well-timed. But I very much doubt that it's a waste of time.

A lot of time ago Walter said that bugs are not all equal, some of them are much more important than others. At that time I didn't fully understand his words.

Now I am compiling DMD every time some bugs get fixed. And I have seen that only a small percentage of those bug fixes has a big influence on my D2 code. When one of such bugs gets fixed, I am able to remove work-arounds, make my code more readable, introduce simpler or shorter idioms, they increase my pleasure of using D2.

So I hope to see fixed every month some of such special bugs :-)

Bye and thank you,
bearophile

On 2012-01-15 01:06:23 +0000, Jonathan M Davis <jmdavisProg@gmx.com> said:

> On Saturday, January 14, 2012 14:58:19 Andrei Alexandrescu wrote:
>> IMHO, fixing const postblit (and the last mile of const) an immediate
>> issue, followed by shared and threads. In light of that, working on simd
>> now appears even more like a waste of time.
> 
> I completely agree that const and shared issues are more important than SIMD,
> given that they affect far more people and have to do with some of the most
> frustrating bugs. So, I'm not sure that this work on SIMD is particularly
> well-timed. But I very much doubt that it's a waste of time. It could really
> help make D acceptable to some of the folks who would otherwise insist on
> using C or C++ for efficiency. And it's definitely addressing the complaints of
> some of D's users. Still, I'd very much like to see the remaining core
> language issues resolved sooner rather than implement enhancements like this
> right now.

Still, Walter perfectly has the right to decide on what he wants to work. I understand that he saw implementing SIMD as an interesting challenge, and if working on SIMD keeps things interesting for him, that can only be great.

Fixing long standing bugs is boring stuff compared to that. Yes it's important, but does it needs to be Walter doing it? Even for an issue requiring a language change, couldn't someone design the language change, implements it, and create a pull request for it? Then it'd be up to Walter to look at it, ask for amendments, then accept it or reject it…

Looks good in theory, but in practice this approach hasn't worked very well for pull request number 3.


-- 
Michel Fortin
michel.fortin@michelf.com
http://michelf.com/

On Saturday, January 14, 2012 20:54:55 Michel Fortin wrote:
> Still, Walter perfectly has the right to decide on what he wants to work. I understand that he saw implementing SIMD as an interesting challenge, and if working on SIMD keeps things interesting for him, that can only be great.

Sure. I can understand why the most important thing to work  on might not be the most interesting thing to work on. And Walter certainly has the right to work on whatever he wants to work on. It could even be that if the more important stuff is tedious enough, working on something new and interesting could make it so that Walter is able to make progress on the more important stuff more quickly when he does work on it. But from the perspective of the language and community at large, SIMD support is great to have but not at all the thing that we need the most.

Regardless, I don't wish to make a big deal about it. I was just echoing Andrei's sentiment that there are more important things to be working on than SIMD support right now.

> Fixing long standing bugs is boring stuff compared to that. Yes it's important, but does it needs to be Walter doing it? Even for an issue requiring a language change, couldn't someone design the language change, implements it, and create a pull request for it? Then it'd be up to Walter to look at it, ask for amendments, then accept it or reject it…
> 
> Looks good in theory, but in practice this approach hasn't worked very well for pull request number 3.

It seems that the problem is that while Walter can be saved effort by having someone else do that kind of work, it doesn't necessarily save him all that much effort, since he still must ultimately take the time to fully understand what's being changed and what the implications are before he merges it in. And the ramifications of pull request #3 are larger than most, so it makes sense if it takes longer for it to be sorted out. Still, given the importance of pull request #3, I would have thought that it would have been handled much sooner than this. It's getting very close to being a year old.

- Jonathan M Davis

On 1/14/12 9:06 PM, Jonathan M Davis wrote:
> On Saturday, January 14, 2012 20:54:55 Michel Fortin wrote:
>> Still, Walter perfectly has the right to decide on what he wants to
>> work. I understand that he saw implementing SIMD as an interesting
>> challenge, and if working on SIMD keeps things interesting for him,
>> that can only be great.
>
> Sure. I can understand why the most important thing to work  on might not be
> the most interesting thing to work on. And Walter certainly has the right to
> work on whatever he wants to work on.

Exactly and perfectly right. To add to that, with my limited time, I can hardly afford to work on stuff that isn't fun (although lately I've done some of it - e.g. I'm not a web designer). There comes a point, however, when we need to decide whether our main priority is having fun or making D successful.

In the latter case:

1. SIMD is not the top of the list. Two weeks ago it wasn't _on_ the list. Now it's like the last 'copter outta Saigon.

2. We haven't identified game designers as a core market, and one that's more important than e.g. general purpose programmers who need the like of working qualifiers, multithreading, and shared libraries.

3. There was never a promise or even a mention that we'll deliver SIMD. We virtually promise we deliver threads and expressive qualifiers, and there's still work to do on that.

4. There was broad agreement that the main foci going forward would be quality, expressive qualifiers, shared libraries, Phobos work, and publicizing the language. We can't work with and publicize D's awesome concurrency design if parts of it aren't implemented.

5. The SIMD work has _zero_ acceleration on existing code; it only allows experts to write non-portable code that uses SIMD instructions. Updating to the next release of dmd has zero SIMD-related benefit to statistically our entire user base.

Walter and I spend hours on the phone discussing strategies and tactics to make D more successful. And then comes this binge. Doing anything on SIMD now is a mistake that I am sorry I was unable to stop. About the only thing that's good about it all is that it'll be over soon.


Andrei

On 2012-01-15 03:06:22 +0000, Jonathan M Davis <jmdavisProg@gmx.com> said:

> It seems that the problem is that while Walter can be saved effort by having someone else do that kind of work, it doesn't necessarily save him all that much effort, since he still must ultimately take the time to fully understand what's being changed and what the implications are before he merges it in.

It still saves the effort of coming up with a solution, implementing that solution, updating the test suite, etc. I think this is significant. But you're right, it's more effort to review than most other pull requests, and it might not be something he likes to do.


> And the ramifications of pull request #3 are larger than most, so it makes sense if it takes longer for it to be sorted out. Still, given the importance of pull request #3, I would have thought that it would have been handled much sooner than this. It's getting very close to being a year old.

A year old and outdated: it won't merge cleanly anymore (despite the update I made this summer). I should probably keep maintaining it, but as it is now I don't think I'll bother until Walter is ready to look at it.

But I'm a little surprised he hasn't even looked at it yet given this is a problem he tried to fix himself before. In his place, I'd be very curious to understand how it works.

And also, #3 is pretty small compared to something else I could come with one day (D/Objective-C).

-- 
Michel Fortin
michel.fortin@michelf.com
http://michelf.com/

On Sat, 14 Jan 2012 21:31:14 -0600
Andrei Alexandrescu <SeeWebsiteForEmail@erdani.org> wrote:

> In the latter case:
> 
> 1. SIMD is not the top of the list. Two weeks ago it wasn't _on_ the list. Now it's like the last 'copter outta Saigon.
> 
> 2. We haven't identified game designers as a core market, and one that's more important than e.g. general purpose programmers who need the like of working qualifiers, multithreading, and shared libraries.
> 
> 3. There was never a promise or even a mention that we'll deliver SIMD. We virtually promise we deliver threads and expressive qualifiers, and there's still work to do on that.
> 
> 4. There was broad agreement that the main foci going forward would be quality, expressive qualifiers, shared libraries, Phobos work, and publicizing the language. We can't work with and publicize D's awesome concurrency design if parts of it aren't implemented.
> 
> 5. The SIMD work has _zero_ acceleration on existing code; it only allows experts to write non-portable code that uses SIMD instructions. Updating to the next release of dmd has zero SIMD-related benefit to statistically our entire user base.

Very nicely put together. Thank you for that.

> Walter and I spend hours on the phone discussing strategies and tactics to make D more successful. And then comes this binge. Doing anything on SIMD now is a mistake that I am sorry I was unable to stop. About the only thing that's good about it all is that it'll be over soon.

It looks as some of the GTD wisdom to choose the rigt NextActions would be beneficial in D community. ;)


Sincerely,
Gour


-- 
Everyone is forced to act helplessly according to the qualities he has acquired from the modes of material nature; therefore no one can refrain from doing something, not even for a moment.

http://atmarama.net | Hlapicina (Croatia) | GPG: 52B5C810

On Sun, 15 Jan 2012 04:31:14 +0100, Andrei Alexandrescu <SeeWebsiteForEmail@erdani.org> wrote:

> On 1/14/12 9:06 PM, Jonathan M Davis wrote:
>> On Saturday, January 14, 2012 20:54:55 Michel Fortin wrote:
>>> Still, Walter perfectly has the right to decide on what he wants to
>>> work. I understand that he saw implementing SIMD as an interesting
>>> challenge, and if working on SIMD keeps things interesting for him,
>>> that can only be great.
>>
>> Sure. I can understand why the most important thing to work  on might not be
>> the most interesting thing to work on. And Walter certainly has the right to
>> work on whatever he wants to work on.
>
> Exactly and perfectly right. To add to that, with my limited time, I can hardly afford to work on stuff that isn't fun (although lately I've done some of it - e.g. I'm not a web designer). There comes a point, however, when we need to decide whether our main priority is having fun or making D successful.
>
> In the latter case:
>
> 1. SIMD is not the top of the list. Two weeks ago it wasn't _on_ the list. Now it's like the last 'copter outta Saigon.
>
> 2. We haven't identified game designers as a core market, and one that's more important than e.g. general purpose programmers who need the like of working qualifiers, multithreading, and shared libraries.
>
> 3. There was never a promise or even a mention that we'll deliver SIMD. We virtually promise we deliver threads and expressive qualifiers, and there's still work to do on that.
>
> 4. There was broad agreement that the main foci going forward would be quality, expressive qualifiers, shared libraries, Phobos work, and publicizing the language. We can't work with and publicize D's awesome concurrency design if parts of it aren't implemented.
>
> 5. The SIMD work has _zero_ acceleration on existing code; it only allows experts to write non-portable code that uses SIMD instructions. Updating to the next release of dmd has zero SIMD-related benefit to statistically our entire user base.
>
> Walter and I spend hours on the phone discussing strategies and tactics to make D more successful. And then comes this binge. Doing anything on SIMD now is a mistake that I am sorry I was unable to stop. About the only thing that's good about it all is that it'll be over soon.
>
>
> Andrei

This is a huge selling point.
    float4 v1,v2,v3;
    v1 = v2;
    v1 = v2 + v3;
    v1 = v2 - v3;
    v1 = v2 * v3;
    v1 = v2 / v3;

Considering the recent switch to XMM it was just opportune to do
this now rather than later. Context switches are expensive.

martin

On Sunday, January 15, 2012 10:05:34 Martin Nowak wrote:
> This is a huge selling point.
>      float4 v1,v2,v3;
>      v1 = v2;
>      v1 = v2 + v3;
>      v1 = v2 - v3;
>      v1 = v2 * v3;
>      v1 = v2 / v3;

I think that the main point is that it is _not_ a big selling point for the majority of users, whereas things that the language has already promised to do are not currently working properly. So, for the vast majority of users, what's currently being worked on does not match what the highest priorities are.

It would be one thing to add SIMD support if there weren't major issues which are causing problems for the majority of users. It's quite another when there _are_ such issues.

> Considering the recent switch to XMM it was just opportune to do this now rather than later. Context switches are expensive.

That may be true. And maybe SIMD will be taken care of more efficiently this way, but it doesn't match what's supposed to be the main priorities right now.

- Jonathan M Davis

On 15/01/12 3:31 AM, Andrei Alexandrescu wrote:
> 1. SIMD is not the top of the list. Two weeks ago it wasn't _on_ the
> list. Now it's like the last 'copter outta Saigon.

That's not true. SIMD intrinsics has always been on the list. I've only been in this newsgroup for a year or so, but I've definitely seen D's lack of SIMD support mentioned in several discussions so far. It's a frequent complaint about the language.


> 2. We haven't identified game designers as a core market, and one that's
> more important than e.g. general purpose programmers who need the like
> of working qualifiers, multithreading, and shared libraries.

Game programming is quite clearly a very large market for D. The game industry is perhaps one of the largest software industries that still relies on having C/C++ level of efficiency and low-level access. It's also an industry that is absolutely sick of having to use C++. D is quite close to the perfect language for game developers.

I think it's also telling that perhaps the most prolific (ex-)D coder, Tomasz, was a game programmer, and that one of the largest D libraries is Derelict, which is also aimed at games.

Finally, if you look at the dsource projects, http://dsource.org/projects/ you'll see that Games make up quite a large chunk of what people are using D for.

What do you think our core market is?


> 3. There was never a promise or even a mention that we'll deliver SIMD.
> We virtually promise we deliver threads and expressive qualifiers, and
> there's still work to do on that.

Fair point.


> 4. There was broad agreement that the main foci going forward would be
> quality, expressive qualifiers, shared libraries, Phobos work, and
> publicizing the language. We can't work with and publicize D's awesome
> concurrency design if parts of it aren't implemented.

I suspect that SIMD support will greatly help to publicize the language.


> 5. The SIMD work has _zero_ acceleration on existing code; it only
> allows experts to write non-portable code that uses SIMD instructions.
> Updating to the next release of dmd has zero SIMD-related benefit to
> statistically our entire user base.

Where are you getting the figures for the % of people that will benefit from SIMD support? The SIMD support thread is rather large, so that suggests to me that a significant number of people are quite interested in the SIMD work.


> Walter and I spend hours on the phone discussing strategies and tactics
> to make D more successful. And then comes this binge. Doing anything on
> SIMD now is a mistake that I am sorry I was unable to stop. About the
> only thing that's good about it all is that it'll be over soon.

I can't speak on your private conversations with Walter, but I think you're underestimating how important SIMD support is for D.

Also, it appears (from his rate of progress) that Walter is quite enjoying the SIMD work. I see no harm in a short-lived "binge" if it reinvigorates Walter's interest in compiler work -- especially if you please a large part of the community in the process.