Thread overview
wrong rounding
Jun 01, 2015
Oleg B
Jun 01, 2015
John Colvin
Jun 01, 2015
Manfred Nowak
Jun 02, 2015
Marc Schütz
Jun 02, 2015
Matthias Bentrup
June 01, 2015
Hello. I found unexpected (for me) behavior of rounding double values at casting to ulong:

$ cat roundtest.d
import std.stdio;

void main()
{
    double a = 10, b = 0.01;

    writeln( "int:   ", cast(int)(a/b) );
    writeln( "uint:  ", cast(uint)(a/b) );
    writeln( "long:  ", cast(long)(a/b) );
    writeln( "ulong: ", cast(ulong)(a/b) );
}

$ rdmd roundtest.d
int:   1000
uint:  1000
long:  1000
ulong: 999   <----- WTF?? -------

$ dmd --version
DMD64 D Compiler v2.067.1
Copyright (c) 1999-2014 by Digital Mars written by Walter Bright

$ ldc2 roundtest.d && ./roundtest
int:   1000
uint:  1000
long:  1000
ulong: 1000  <------- Ok --------

$ ldc2 --version
LDC - the LLVM D compiler (0.15.2-beta1):
  based on DMD v2.066.1 and LLVM 3.6.0
  Default target: x86_64-unknown-linux-gnu
  Host CPU: core-avx-i
  http://dlang.org - http://wiki.dlang.org/LDC

  Registered Targets:
    x86    - 32-bit X86: Pentium-Pro and above
    x86-64 - 64-bit X86: EM64T and AMD64

$ uname -a
Linux lenovo 4.0.4-201.fc21.x86_64 #1 SMP Thu May 21 15:58:47 UTC 2015 x86_64 x86_64 x86_64 GNU/Linux

have it behavior explain? can I use casting in dmd for rounding values or it not safe?
June 01, 2015
On 6/1/15 5:29 PM, Oleg B wrote:
> Hello. I found unexpected (for me) behavior of rounding double values at
> casting to ulong:
>
> $ cat roundtest.d
> import std.stdio;
>
> void main()
> {
>      double a = 10, b = 0.01;
>
>      writeln( "int:   ", cast(int)(a/b) );
>      writeln( "uint:  ", cast(uint)(a/b) );
>      writeln( "long:  ", cast(long)(a/b) );
>      writeln( "ulong: ", cast(ulong)(a/b) );
> }
>
> $ rdmd roundtest.d
> int:   1000
> uint:  1000
> long:  1000
> ulong: 999   <----- WTF?? -------

These are NOT roundings. They are truncations.

Note that for floating point 0.01 is not representable exactly. This is the reason you get the error. On other systems, you may not get errors for this one case, but you could get errors for other. Please read about floating point error: http://en.wikipedia.org/wiki/Floating_point#Accuracy_problems

-Steve
June 01, 2015
On Monday, 1 June 2015 at 21:44:57 UTC, Steven Schveighoffer wrote:
> On 6/1/15 5:29 PM, Oleg B wrote:
>> Hello. I found unexpected (for me) behavior of rounding double values at
>> casting to ulong:
>>
>> $ cat roundtest.d
>> import std.stdio;
>>
>> void main()
>> {
>>     double a = 10, b = 0.01;
>>
>>     writeln( "int:   ", cast(int)(a/b) );
>>     writeln( "uint:  ", cast(uint)(a/b) );
>>     writeln( "long:  ", cast(long)(a/b) );
>>     writeln( "ulong: ", cast(ulong)(a/b) );
>> }
>>
>> $ rdmd roundtest.d
>> int:   1000
>> uint:  1000
>> long:  1000
>> ulong: 999   <----- WTF?? -------
>
> These are NOT roundings. They are truncations.
>
> Note that for floating point 0.01 is not representable exactly. This is the reason you get the error. On other systems, you may not get errors for this one case, but you could get errors for other. Please read about floating point error: http://en.wikipedia.org/wiki/Floating_point#Accuracy_problems
>
> -Steve

Nonetheless, surely in this case the values should be the same regardless of the integer target type, no?
June 01, 2015
John Colvin wrote:

> regardless of the integer target type
on win64 dmd 2.067.1 gives correct results

int:   999
uint:  999
long:  999
ulong: 999

-manfred
June 01, 2015
On 6/1/15 6:24 PM, John Colvin wrote:
> Nonetheless, surely in this case the values should be the same
> regardless of the integer target type, no?

Hm... yeah probably :) I didn't grok the example thoroughly.

Makes me curious actually, and that does seem like a bug.

Doing some testing, if you assign a/b to a double and cast, it works. If you use a function to truncate taking real as a parameter, it returns 999 in both ulong and long cases.

The assembly is too much over my head to figure out where the difference is.

I think you should file a codegen bug.

-Steve
June 02, 2015
On Monday, 1 June 2015 at 22:24:31 UTC, John Colvin wrote:
> Nonetheless, surely in this case the values should be the same regardless of the integer target type, no?

Not sure... the spec says that the compiler is allowed to do intermediate computations at higher precision, but it doesn't say it is required to. So, strictly speaking, its at the compiler's discretion to use higher precision when rounding to `int` than when rounding to `ulong`...
June 02, 2015
Interestingly dmd computes the a/b with SSE instructions if the result is cast to int, uint or long, but uses x87 instructions for the division if the result is cast to ulong.
June 02, 2015
On 6/2/15 5:53 AM, "Marc =?UTF-8?B?U2Now7x0eiI=?= <schuetzm@gmx.net>" wrote:
> On Monday, 1 June 2015 at 22:24:31 UTC, John Colvin wrote:
>> Nonetheless, surely in this case the values should be the same
>> regardless of the integer target type, no?
>
> Not sure... the spec says that the compiler is allowed to do
> intermediate computations at higher precision, but it doesn't say it is
> required to. So, strictly speaking, its at the compiler's discretion to
> use higher precision when rounding to `int` than when rounding to
> `ulong`...

Right, but there is a difference between ulong and long. These should be the same exact code.

Something definitely isn't right.

-Steve