Thread overview | |||||||||||||||||||
---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
|
November 04, 2009 cast(int) getting an unexpected number | ||||
---|---|---|---|---|
| ||||
Input: import std.stdio; void main() { float f=0.01; writefln("%0.2f->%d",f,cast(int)(f*100f)); } Output: 0.01->0 Comment: What!? |
November 04, 2009 Re: cast(int) getting an unexpected number | ||||
---|---|---|---|---|
| ||||
Posted in reply to Joel Christensen | On Wed, 04 Nov 2009 06:14:36 -0500, Joel Christensen <joelcnz@gmail.com> wrote:
> Input:
> import std.stdio;
> void main() {
> float f=0.01;
> writefln("%0.2f->%d",f,cast(int)(f*100f));
> }
>
> Output:
> 0.01->0
>
> Comment:
> What!?
hehe. .01 is not exactly represented by float, because it's stored as a binary value, not a decimal value. Think about how there is no way to represent 1/3 in decimal.
If you imagined that the computer stored things in decimal, see how the 1/3 example would work
a = 1.0/3; // truncated to 0.3333333
a *= 3; // 0.9999999
auto i = cast(int)a; // 0
This is analagous to what you are asking the compiler to do.
To be safe, whenever converting to int, always add a small epsilon. I think you can use float.epsilon, but I don't have any experience with whether that is always reasonable.
-Steve
|
November 04, 2009 Re: cast(int) getting an unexpected number | ||||
---|---|---|---|---|
| ||||
Posted in reply to Steven Schveighoffer | "Steven Schveighoffer" <schveiguy@yahoo.com> wrote: > On Wed, 04 Nov 2009 06:14:36 -0500, Joel Christensen <joelcnz@gmail.com> wrote: > >> Input: >> import std.stdio; >> void main() { >> float f=0.01; >> writefln("%0.2f->%d",f,cast(int)(f*100f)); >> } >> >> Output: >> 0.01->0 >> >> Comment: >> What!? > > hehe. .01 is not exactly represented by float, because it's stored as a binary value, not a decimal value. Think about how there is no way to represent 1/3 in decimal. > > If you imagined that the computer stored things in decimal, see how the 1/3 example would work > > a = 1.0/3; // truncated to 0.3333333 > a *= 3; // 0.9999999 > auto i = cast(int)a; // 0 > > This is analagous to what you are asking the compiler to do. > > To be safe, whenever converting to int, always add a small epsilon. I think you can use float.epsilon, but I don't have any experience with whether that is always reasonable. > > -Steve > why is this not a compiler bug? because: import std.stdio; void main() { float f=0.01; writefln("%0.2f->%d",f,cast(int)(f*100f)); writefln("%0.2f->%d",f,cast(int)(.01*100f)); writefln("%0.2f->%f",f,(f*100f)); } results in: 0.01->0 0.01->1 0.01->1.000000 I would say something is dodgy. -Rory |
November 04, 2009 Re: cast(int) getting an unexpected number | ||||
---|---|---|---|---|
| ||||
Posted in reply to rmcguire | Hello rmcguire, > why is this not a compiler bug? > because: > import std.stdio; > void main() { > float f=0.01; > writefln("%0.2f->%d",f,cast(int)(f*100f)); > writefln("%0.2f->%d",f,cast(int)(.01*100f)); > writefln("%0.2f->%f",f,(f*100f)); > } > results in: > 0.01->0 > 0.01->1 > 0.01->1.000000 > I would say something is dodgy. > > -Rory > I think this may be case of: At comple time floating point computations may be done at a higher precision than run time. http://www.digitalmars.com/d/2.0/function.html#interpretation It seems that .01*100f is computed at compile time. try declarting float f as const or invariant and see what happens (I'm not sure..) |
November 04, 2009 Re: cast(int) getting an unexpected number | ||||
---|---|---|---|---|
| ||||
Posted in reply to Michal Minich | Michal Minich wrote:
> Hello rmcguire,
>
>> why is this not a compiler bug?
>> because:
>> import std.stdio;
>> void main() {
>> float f=0.01;
>> writefln("%0.2f->%d",f,cast(int)(f*100f));
>> writefln("%0.2f->%d",f,cast(int)(.01*100f));
>> writefln("%0.2f->%f",f,(f*100f));
>> }
>> results in:
>> 0.01->0
>> 0.01->1
>> 0.01->1.000000
>> I would say something is dodgy.
>>
>> -Rory
>>
>
> I think this may be case of:
> At comple time floating point computations may be done at a higher precision than run time.
Yes, if you do this:
float f = 0.01;
float g = f * 100f;
real r = f * 100f;
writeln("%s, %s, %s", f, cast(int) g, cast(int) r);
you get:
0.01, 0, 1
I believe just writing cast(int)(f*100f) is more or less the same as the 'real' case above.
-Lars
|
November 04, 2009 Re: cast(int) getting an unexpected number | ||||
---|---|---|---|---|
| ||||
Posted in reply to Lars T. Kyllingstad | Lars T. Kyllingstad wrote:
> Michal Minich wrote:
>> Hello rmcguire,
>>
>>> why is this not a compiler bug?
>>> because:
>>> import std.stdio;
>>> void main() {
>>> float f=0.01;
>>> writefln("%0.2f->%d",f,cast(int)(f*100f));
>>> writefln("%0.2f->%d",f,cast(int)(.01*100f));
>>> writefln("%0.2f->%f",f,(f*100f));
>>> }
>>> results in:
>>> 0.01->0
>>> 0.01->1
>>> 0.01->1.000000
>>> I would say something is dodgy.
>>>
>>> -Rory
>>>
>>
>> I think this may be case of:
>> At comple time floating point computations may be done at a higher
>> precision than run time.
>
>
> Yes, if you do this:
>
> float f = 0.01;
> float g = f * 100f;
> real r = f * 100f;
> writeln("%s, %s, %s", f, cast(int) g, cast(int) r);
>
> you get:
>
> 0.01, 0, 1
>
> I believe just writing cast(int)(f*100f) is more or less the same as the
> 'real' case above.
>
> -Lars
Can that *really* be the explanation?? I know that float doesn't have all that much precision, but I thought it was more than 5 or 6 places...and this is, essentially, two places.
|
November 04, 2009 Re: cast(int) getting an unexpected number | ||||
---|---|---|---|---|
| ||||
Posted in reply to Charles Hixson | Charles Hixson wrote: > Lars T. Kyllingstad wrote: >> Michal Minich wrote: >>> Hello rmcguire, >>> >>>> why is this not a compiler bug? >>>> because: >>>> import std.stdio; >>>> void main() { >>>> float f=0.01; >>>> writefln("%0.2f->%d",f,cast(int)(f*100f)); >>>> writefln("%0.2f->%d",f,cast(int)(.01*100f)); >>>> writefln("%0.2f->%f",f,(f*100f)); >>>> } >>>> results in: >>>> 0.01->0 >>>> 0.01->1 >>>> 0.01->1.000000 >>>> I would say something is dodgy. >>>> >>>> -Rory >>>> >>> >>> I think this may be case of: >>> At comple time floating point computations may be done at a higher >>> precision than run time. >> >> >> Yes, if you do this: >> >> float f = 0.01; >> float g = f * 100f; >> real r = f * 100f; >> writeln("%s, %s, %s", f, cast(int) g, cast(int) r); >> >> you get: >> >> 0.01, 0, 1 >> >> I believe just writing cast(int)(f*100f) is more or less the same as the >> 'real' case above. >> >> -Lars > Can that *really* be the explanation?? I know that float doesn't have all that much precision, but I thought it was more than 5 or 6 places...and this is, essentially, two places. Yes it does, but that's not what matters. Say that for floats, the representable number closest to 0.01 is 0.01000000000001. (This is just an example, I don't know the true number.) Then you have a lot more precision than the two digits you mention, and multiplying with 100 gives 1.000000000001. Round this towards zero (which is what cast(int) does) and you get 1. This is the 'float g = f*100f;' case in my example. Now, say that for reals, which (I think) is what the compiler uses internally, the number closest to 0.01 is 0.0099999999999999999999999999999999999999. Again, just an example, the point is that the precision is higher than the above, but the closest number is now smaller than 0.01. Multiply this by 100, and you get 0.99999999999999999999999999999999999999. This number will be cast to the integer 0, which happens in the OP's case. I admit I'm no expert in these things, but I suspect this is how it goes. By the way, I recommend Don's excellent article on floating-point numbers. It has really cleared things up for me: http://www.digitalmars.com/d/2.0/d-floating-point.html -Lars |
November 04, 2009 Re: cast(int) getting an unexpected number | ||||
---|---|---|---|---|
| ||||
Posted in reply to rmcguire | rmcguire Wrote: > why is this not a compiler bug? > because: > import std.stdio; > > void main() { > float f=0.01; > results in: > 0.01->0 0.01 is double, there is type conversion there. > writefln("%0.2f->%d",f,cast(int)(f*100f)); > results in: > 0.01->1 f*100f is represented to be less than 0. > writefln("%0.2f->%d",f,cast(int)(.01*100f)); .01 * 100f is double and apparently represented as something more than 1. > writefln("%0.2f->%f",f,(f*100f)); > results in: > 0.01->1.000000 Someting that is less than 0 is being "printed" as 1 at that precision. Printing with a higher precision should print something less than 0. Ali |
November 04, 2009 Re: cast(int) getting an unexpected number | ||||
---|---|---|---|---|
| ||||
Posted in reply to rmcguire | Ok, I messed up my previous comment while moving rmcguire's lines around. Trying again... Also, I am not sure about the last bit now. :) rmcguire Wrote: > why is this not a compiler bug? > because: > import std.stdio; > > void main() { > float f=0.01; Just an information: 0.01 is double, there is type conversion there. > writefln("%0.2f->%d",f,cast(int)(f*100f)); > results in: > 0.01->0 f*100f is represented to be less than 0. > writefln("%0.2f->%d",f,cast(int)(.01*100f)); > results in: > 0.01->1 .01 * 100f is double and apparently represented as something more than 1. > writefln("%0.2f->%f",f,(f*100f)); > results in: > 0.01->1.000000 If it's like in C, floating point values must be passed as double arguments to functions. So there is float-to-double conversion before the function call, effectively representing the argument as double. Ali |
November 05, 2009 Re: cast(int) getting an unexpected number | ||||
---|---|---|---|---|
| ||||
Posted in reply to Steven Schveighoffer | > To be safe, whenever converting to int, always add a small epsilon. I think you can use float.epsilon, but I don't have any experience with whether that is always reasonable.
>
> -Steve
Thanks every one for the replies.
I still have problems. How do I use epsilon? 'real' helps in my example program but not in my money program. I think there's a function out there that does dollars and cents (eg. .89 -> 89c and 1.00 -> $1), but I'm interested how to solve this problem.
|
Copyright © 1999-2021 by the D Language Foundation