Jump to page: 1 2
Thread overview
cast(int) getting an unexpected number
Nov 04, 2009
Joel Christensen
Nov 04, 2009
rmcguire
Nov 04, 2009
Michal Minich
Nov 04, 2009
Charles Hixson
Nov 05, 2009
rmcguire
Nov 05, 2009
rmcguire
Nov 04, 2009
Ali Cehreli
Nov 04, 2009
Ali Cehreli
Nov 05, 2009
Joel Christensen
Nov 05, 2009
Charles Hixson
Nov 06, 2009
Joel Christensen
November 04, 2009
Input:
import std.stdio;
void main() {
	float f=0.01;
	writefln("%0.2f->%d",f,cast(int)(f*100f));
}

Output:
0.01->0

Comment:
What!?
November 04, 2009
On Wed, 04 Nov 2009 06:14:36 -0500, Joel Christensen <joelcnz@gmail.com> wrote:

> Input:
> import std.stdio;
> void main() {
> 	float f=0.01;
> 	writefln("%0.2f->%d",f,cast(int)(f*100f));
> }
>
> Output:
> 0.01->0
>
> Comment:
> What!?

hehe.  .01 is not exactly represented by float, because it's stored as a binary value, not a decimal value.  Think about how there is no way to represent 1/3 in decimal.

If you imagined that the computer stored things in decimal, see how the 1/3 example would work

a = 1.0/3; // truncated to 0.3333333
a *= 3; // 0.9999999
auto i = cast(int)a; // 0

This is analagous to what you are asking the compiler to do.

To be safe, whenever converting to int, always add a small epsilon.  I think you can use float.epsilon, but I don't have any experience with whether that is always reasonable.

-Steve
November 04, 2009
"Steven Schveighoffer" <schveiguy@yahoo.com> wrote:

> On Wed, 04 Nov 2009 06:14:36 -0500, Joel Christensen <joelcnz@gmail.com> wrote:
> 
>> Input:
>> import std.stdio;
>> void main() {
>>  float f=0.01;
>>  writefln("%0.2f->%d",f,cast(int)(f*100f));
>> }
>>
>> Output:
>> 0.01->0
>>
>> Comment:
>> What!?
> 
> hehe.  .01 is not exactly represented by float, because it's stored as a binary value, not a decimal value.  Think about how there is no way to represent 1/3 in decimal.
> 
> If you imagined that the computer stored things in decimal, see how the 1/3 example would work
> 
> a = 1.0/3; // truncated to 0.3333333
> a *= 3; // 0.9999999
> auto i = cast(int)a; // 0
> 
> This is analagous to what you are asking the compiler to do.
> 
> To be safe, whenever converting to int, always add a small epsilon.  I think you can use float.epsilon, but I don't have any experience with whether that is always reasonable.
> 
> -Steve
> 

why is this not a compiler bug?
because:
import std.stdio;

void main() {
        float f=0.01;
        writefln("%0.2f->%d",f,cast(int)(f*100f));
        writefln("%0.2f->%d",f,cast(int)(.01*100f));
        writefln("%0.2f->%f",f,(f*100f));
}

results in:
0.01->0
0.01->1
0.01->1.000000

I would say something is dodgy.

-Rory

November 04, 2009
Hello rmcguire,

> why is this not a compiler bug?
> because:
> import std.stdio;
> void main() {
> float f=0.01;
> writefln("%0.2f->%d",f,cast(int)(f*100f));
> writefln("%0.2f->%d",f,cast(int)(.01*100f));
> writefln("%0.2f->%f",f,(f*100f));
> }
> results in:
> 0.01->0
> 0.01->1
> 0.01->1.000000
> I would say something is dodgy.
> 
> -Rory
> 

I think this may be case of:
At comple time floating point computations may be done at a higher precision than run time.

http://www.digitalmars.com/d/2.0/function.html#interpretation

It seems that .01*100f is computed at compile time.
try declarting float f as const or invariant and see what happens (I'm not sure..)


November 04, 2009
Michal Minich wrote:
> Hello rmcguire,
> 
>> why is this not a compiler bug?
>> because:
>> import std.stdio;
>> void main() {
>> float f=0.01;
>> writefln("%0.2f->%d",f,cast(int)(f*100f));
>> writefln("%0.2f->%d",f,cast(int)(.01*100f));
>> writefln("%0.2f->%f",f,(f*100f));
>> }
>> results in:
>> 0.01->0
>> 0.01->1
>> 0.01->1.000000
>> I would say something is dodgy.
>>
>> -Rory
>>
> 
> I think this may be case of:
> At comple time floating point computations may be done at a higher precision than run time.


Yes, if you do this:

  float f = 0.01;
  float g = f * 100f;
  real r  = f * 100f;
  writeln("%s, %s, %s", f, cast(int) g, cast(int) r);

you get:

  0.01, 0, 1

I believe just writing cast(int)(f*100f) is more or less the same as the 'real' case above.

-Lars
November 04, 2009
Lars T. Kyllingstad wrote:
> Michal Minich wrote:
>> Hello rmcguire,
>>
>>> why is this not a compiler bug?
>>> because:
>>> import std.stdio;
>>> void main() {
>>> float f=0.01;
>>> writefln("%0.2f->%d",f,cast(int)(f*100f));
>>> writefln("%0.2f->%d",f,cast(int)(.01*100f));
>>> writefln("%0.2f->%f",f,(f*100f));
>>> }
>>> results in:
>>> 0.01->0
>>> 0.01->1
>>> 0.01->1.000000
>>> I would say something is dodgy.
>>>
>>> -Rory
>>>
>>
>> I think this may be case of:
>> At comple time floating point computations may be done at a higher
>> precision than run time.
>
>
> Yes, if you do this:
>
> float f = 0.01;
> float g = f * 100f;
> real r = f * 100f;
> writeln("%s, %s, %s", f, cast(int) g, cast(int) r);
>
> you get:
>
> 0.01, 0, 1
>
> I believe just writing cast(int)(f*100f) is more or less the same as the
> 'real' case above.
>
> -Lars
Can that *really* be the explanation??  I know that float doesn't have all that much precision, but I thought it was more than 5 or 6 places...and this is, essentially, two places.

November 04, 2009
Charles Hixson wrote:
> Lars T. Kyllingstad wrote:
>> Michal Minich wrote:
>>> Hello rmcguire,
>>>
>>>> why is this not a compiler bug?
>>>> because:
>>>> import std.stdio;
>>>> void main() {
>>>> float f=0.01;
>>>> writefln("%0.2f->%d",f,cast(int)(f*100f));
>>>> writefln("%0.2f->%d",f,cast(int)(.01*100f));
>>>> writefln("%0.2f->%f",f,(f*100f));
>>>> }
>>>> results in:
>>>> 0.01->0
>>>> 0.01->1
>>>> 0.01->1.000000
>>>> I would say something is dodgy.
>>>>
>>>> -Rory
>>>>
>>>
>>> I think this may be case of:
>>> At comple time floating point computations may be done at a higher
>>> precision than run time.
>>
>>
>> Yes, if you do this:
>>
>> float f = 0.01;
>> float g = f * 100f;
>> real r = f * 100f;
>> writeln("%s, %s, %s", f, cast(int) g, cast(int) r);
>>
>> you get:
>>
>> 0.01, 0, 1
>>
>> I believe just writing cast(int)(f*100f) is more or less the same as the
>> 'real' case above.
>>
>> -Lars
> Can that *really* be the explanation??  I know that float doesn't have all that much precision, but I thought it was more than 5 or 6 places...and this is, essentially, two places.


Yes it does, but that's not what matters.

Say that for floats, the representable number closest to 0.01 is 0.01000000000001. (This is just an example, I don't know the true number.) Then you have a lot more precision than the two digits you mention, and multiplying with 100 gives 1.000000000001. Round this towards zero (which is what cast(int) does) and you get 1. This is the 'float g = f*100f;' case in my example.

Now, say that for reals, which (I think) is what the compiler uses internally, the number closest to 0.01 is
    0.0099999999999999999999999999999999999999.
Again, just an example, the point is that the precision is higher than the above, but the closest number is now smaller than 0.01. Multiply this by 100, and you get
    0.99999999999999999999999999999999999999.
This number will be cast to the integer 0, which happens in the OP's case.

I admit I'm no expert in these things, but I suspect this is how it goes. By the way, I recommend Don's excellent article on floating-point numbers. It has really cleared things up for me:

  http://www.digitalmars.com/d/2.0/d-floating-point.html

-Lars
November 04, 2009
rmcguire Wrote:

> why is this not a compiler bug?
> because:
> import std.stdio;
> 
> void main() {
>         float f=0.01;

> results in:
> 0.01->0

0.01 is double, there is type conversion there.

>         writefln("%0.2f->%d",f,cast(int)(f*100f));

> results in:
> 0.01->1

f*100f is represented to be less than 0.

>         writefln("%0.2f->%d",f,cast(int)(.01*100f));

.01 * 100f is double and apparently represented as something more than 1.

>         writefln("%0.2f->%f",f,(f*100f));

> results in:
> 0.01->1.000000

Someting that is less than 0 is being "printed" as 1 at that precision. Printing with a higher precision should print something less than 0.

Ali

November 04, 2009
Ok, I messed up my previous comment while moving rmcguire's lines around. Trying again... Also, I am not sure about the last bit now. :)

rmcguire Wrote:

> why is this not a compiler bug?
> because:
> import std.stdio;
> 
> void main() {
>         float f=0.01;

Just an information: 0.01 is double, there is type conversion there.

>         writefln("%0.2f->%d",f,cast(int)(f*100f));
> results in:
> 0.01->0

f*100f is represented to be less than 0.

>         writefln("%0.2f->%d",f,cast(int)(.01*100f));
> results in:
> 0.01->1

.01 * 100f is double and apparently represented as something more than 1.

>         writefln("%0.2f->%f",f,(f*100f));
> results in:
> 0.01->1.000000

If it's like in C, floating point values must be passed as double arguments to functions. So there is float-to-double conversion before the function call, effectively representing the argument as double.

Ali

November 05, 2009
> To be safe, whenever converting to int, always add a small epsilon.  I think you can use float.epsilon, but I don't have any experience with whether that is always reasonable.
> 
> -Steve

Thanks every one for the replies.

I still have problems. How do I use epsilon? 'real' helps in my example program but not in my money program. I think there's a function out there that does dollars and cents (eg. .89 -> 89c and 1.00 -> $1), but I'm interested how to solve this problem.
« First   ‹ Prev
1 2