Don wrote:
> Ary Borenszweig wrote:
>> Don wrote:
>>> Lars T. Kyllingstad wrote:
>>>> Don wrote:
>>>>> bearophile wrote:
>>>>>> So far I've just given a light reading of the code. Notes:
>>>>>
>>>>>> - pow(x, 2) and sqrt(y) can be written as x ^^ 2 and y ^^ 0.5 (but you have to import std.math anyway, because of a bug).
>>>>>
>>>>> That's not a bug. It's intentional. x ^^ y will probably always require import std.math, if y is a floating point number.
>>>>
>>>> Really?  Why is that?  I find that kind of disappointing, I always believed it to be a temporary solution.
>>>>
>>>> I think the inconsistency with the other operators will make this a major WTF for people new to the language.  Why should a^^b require an explicit import while a*b doesn't?
>>>
>>> Because pow() for floating point, when implemented properly, is a HUGE function, that ends up dragging almost all of std.math into the executable. And I think it's deceptive to do that silently.
>>> To make it completely built-in, basically all of std.math would need to be moved into druntime. Feel free to try to change my mind, of course.
>>
>> Is there a better way to do pow() for floating point numbers without importing std.math?
>>
>> I see this:
>>
>> 1. You want to do x ^^ fp.
>> 2. The compiler complains saying "if you want to do that, import std.math. I'm telling you this just to let you know you'll be importing the whole module".
>>
>> Alternative 1 for user:
>> User says "Ok, I import std.math"
>>
>> Alternative 2 for user:
>> User says "No way I'm importing std.math just to make a pow. But... I still *need* to make that pow... what is your advice, mr. compiler?"
> 
> That's a good point, it should be possible to use a static import as well. I do think it's pretty odd to be doing floating point without importing std.math, though. I mean, abs() is absolutely fundamental.

But if you do a static import the whole module gets linked in, right?

My point is, if you are going to pow, you will need std.math, so it'll always be a burden to import it by hand when using it. ^^

Ary Borenszweig:
> My point is, if you are going to pow, you will need std.math, so it'll always be a burden to import it by hand when using it. ^^

Can the automatic import happen only iff a module uses the ^^fp?

Bye,
bearophile

Ary Borenszweig wrote:
> Don wrote:
>> Ary Borenszweig wrote:
>>> Don wrote:
>>>> Lars T. Kyllingstad wrote:
>>>>> Don wrote:
>>>>>> bearophile wrote:
>>>>>>> So far I've just given a light reading of the code. Notes:
>>>>>>
>>>>>>> - pow(x, 2) and sqrt(y) can be written as x ^^ 2 and y ^^ 0.5 (but you have to import std.math anyway, because of a bug).
>>>>>>
>>>>>> That's not a bug. It's intentional. x ^^ y will probably always require import std.math, if y is a floating point number.
>>>>>
>>>>> Really?  Why is that?  I find that kind of disappointing, I always believed it to be a temporary solution.
>>>>>
>>>>> I think the inconsistency with the other operators will make this a major WTF for people new to the language.  Why should a^^b require an explicit import while a*b doesn't?
>>>>
>>>> Because pow() for floating point, when implemented properly, is a HUGE function, that ends up dragging almost all of std.math into the executable. And I think it's deceptive to do that silently.
>>>> To make it completely built-in, basically all of std.math would need to be moved into druntime. Feel free to try to change my mind, of course.
>>>
>>> Is there a better way to do pow() for floating point numbers without importing std.math?
>>>
>>> I see this:
>>>
>>> 1. You want to do x ^^ fp.
>>> 2. The compiler complains saying "if you want to do that, import std.math. I'm telling you this just to let you know you'll be importing the whole module".
>>>
>>> Alternative 1 for user:
>>> User says "Ok, I import std.math"
>>>
>>> Alternative 2 for user:
>>> User says "No way I'm importing std.math just to make a pow. But... I still *need* to make that pow... what is your advice, mr. compiler?"
>>
>> That's a good point, it should be possible to use a static import as well. I do think it's pretty odd to be doing floating point without importing std.math, though. I mean, abs() is absolutely fundamental.
> 
> But if you do a static import the whole module gets linked in, right?
> 
> My point is, if you are going to pow, you will need std.math, so it'll always be a burden to import it by hand when using it. ^^

But you're assuming that you're using ^^ without using anything else from std.math. I think that's a very obscure case.
For example, any code which is ported from C or C++, or D1, that uses pow, will already be importing std.math. Cases where you see that you could use ^^ that isn't already using pow() (eg, where you see z = x*x + y*y), you will need to add an import.

bearophile wrote:
> You are right, sorry.

No need to apologise!  You helped me significantly improve my code, helped me understand D a lot better and left me feeling generally very positive about developing further in D.  I'd call that a result. :-)

>> So I think it's probably just compiler difference that's to blame for speed differences<
> 
> This can be true, but such differences are not magic, they can be found, and eventually they can even become precise enhancement requests for llvm developers, they are open to such requests.

I hope my questions here have been useful in this respect ...

>> After all, D in general and DMD in particular is in development.<
> 
> DMD has a quite old back-end that is not in active development. Maybe it will become 64 bit.

The DMD backend seems a generally problematic piece of code given the proprietary restrictions that apply to it.  I have the impression that it's being used simply because it's the backend that is known and tested and that over the longer term there may be a switch ... ?

The one compiler-related concern I have is whether there will be an effective free-as-in-freedom compiler for D2 in the near future.  Work is going on GDC but it's moving slowly -- and I don't see a clear LDC roadmap to D2 support.

> In this program I have seen that a good percentage of the speed difference between ldc/dmd is caused by foreach loops. When you iterate over structs longer than size_t use ref (or in D2 better ref const(...)).
> foreach (Rating r; ratings)
> ==>
> foreach (ref const(Rating) r; ratings)
> 
> There is nothing algorithmically strange going on here, but to be sure you can take a look at the produced asm.

Interesting ... !  I'm afraid the asm is over my head, but I will read up and try to understand it.

> To find why the C++ code is faster you can show me the equivalent C++ code that I can compile myself, or you can compile your C++ code and show me the asm of the two functions that are hot spots.

Sure -- attached.  The key functions are the ones in tref.cpp (the smalltref class) and trefsim.cpp, and the treftest.cpp file that runs a roughly-equivalent simulation to test.d.

It's a little embarrassing to share because it's fairly badly-designed code -- my first serious C++ attempt -- it served its purpose some time ago and was done with.  It started to become vaguely relevant again so I thought I'd try and redo it with a better design, and rather than do it in C++ (which I could do much better by now) this seemed like a nice excuse to build my first D project ... :-)

I already made a couple of tweaks based on your improvements to the D code, but I think g++ probably already has such optimizations during the compile process.

I also realised that I wasn't comparing like with like -- the code was using the RanLux random number generator, and with mt19937 it shaves quite a bit of time off.  So C++ still has a bit of an edge right now.

There's surely a lot of other stuff that can be done to improve this C++, but please don't put lots of effort in unless it helps with improving D (the language and/or frontend, not my code, which is not so important:-).

Thanks & best wishes,

    -- Joe

Don wrote:
> Ary Borenszweig wrote:
>> Don wrote:
>>> Ary Borenszweig wrote:
>>>> Don wrote:
>>>>> Lars T. Kyllingstad wrote:
>>>>>> Don wrote:
>>>>>>> bearophile wrote:
>>>>>>>> So far I've just given a light reading of the code. Notes:
>>>>>>>
>>>>>>>> - pow(x, 2) and sqrt(y) can be written as x ^^ 2 and y ^^ 0.5 (but you have to import std.math anyway, because of a bug).
>>>>>>>
>>>>>>> That's not a bug. It's intentional. x ^^ y will probably always require import std.math, if y is a floating point number.
>>>>>>
>>>>>> Really?  Why is that?  I find that kind of disappointing, I always believed it to be a temporary solution.
>>>>>>
>>>>>> I think the inconsistency with the other operators will make this a major WTF for people new to the language.  Why should a^^b require an explicit import while a*b doesn't?
>>>>>
>>>>> Because pow() for floating point, when implemented properly, is a HUGE function, that ends up dragging almost all of std.math into the executable. And I think it's deceptive to do that silently.
>>>>> To make it completely built-in, basically all of std.math would need to be moved into druntime. Feel free to try to change my mind, of course.
>>>>
>>>> Is there a better way to do pow() for floating point numbers without importing std.math?
>>>>
>>>> I see this:
>>>>
>>>> 1. You want to do x ^^ fp.
>>>> 2. The compiler complains saying "if you want to do that, import std.math. I'm telling you this just to let you know you'll be importing the whole module".
>>>>
>>>> Alternative 1 for user:
>>>> User says "Ok, I import std.math"
>>>>
>>>> Alternative 2 for user:
>>>> User says "No way I'm importing std.math just to make a pow. But... I still *need* to make that pow... what is your advice, mr. compiler?"
>>>
>>> That's a good point, it should be possible to use a static import as well. I do think it's pretty odd to be doing floating point without importing std.math, though. I mean, abs() is absolutely fundamental.
>>
>> But if you do a static import the whole module gets linked in, right?
>>
>> My point is, if you are going to pow, you will need std.math, so it'll always be a burden to import it by hand when using it. ^^
> 
> But you're assuming that you're using ^^ without using anything else from std.math. I think that's a very obscure case.
> For example, any code which is ported from C or C++, or D1, that uses pow, will already be importing std.math. Cases where you see that you could use ^^ that isn't already using pow() (eg, where you see z = x*x + y*y), you will need to add an import.
> 

- Oh, hi Foo, I want to show you this great language D. You can doo blah and bleh.
- Cool, I will code something little to see how it is.
  ...
  Hey, WTF, I use the pow operand and it tells me I need to add an import? WTF??

You know, people program from scratch, they don't spend their life porting C and C++ code... ^^

bearophile wrote:
> Ary Borenszweig:
>> My point is, if you are going to pow, you will need std.math, so it'll always be a burden to import it by hand when using it. ^^
> 
> Can the automatic import happen only iff a module uses the ^^fp?
> 
> Bye,
> bearophile

That's what I'm asking for.

bearophile wrote:
>> This is the middle parts (the two loops) of Yzlm.objectReputationUpdate compiled with ldc, this is one of the two main hot spots of the program, the you can compared this asm with the asm of the same part from the C++ version:
> 
> To find why the C++ code is faster you can show me the equivalent C++ code that I can compile myself, or you can compile your C++ code and show me the asm of the two functions that are hot spots.

I took a little time last night and wrote up a C++ version that is much closer in design to the D code (as well as smaller, simpler, less hell to read...).  It has libboost as a dependency for random number generation and for the BOOST_FOREACH macro.

Following your observation with D, it looks like the C++ also benefits from using references in the FOREACH loops for elements larger than size_t -- this shaves about 20s off running time on my machine.  Either way, like its uglier predecessor it's quite a bit faster than DMD or LDC.  Unfortunately I can't get llvm-g++ running on my machine, or I would compare performance there too.

Hope this makes an interesting point of comparison. :-)

Best wishes,

    -- Joe