Thread overview | |||||||
---|---|---|---|---|---|---|---|
|
November 17, 2004 cast problem | ||||
---|---|---|---|---|
| ||||
void main(){ double fVal = 66.6f; int rate = 10; int intValue = cast(int)(fVal*rate); printf("Value = %d\n", intValue); } // Value = 665 // What is the matter? |
November 17, 2004 Re: cast problem | ||||
---|---|---|---|---|
| ||||
Posted in reply to LiuXuHong | On Wed, 17 Nov 2004 09:35:52 +0000 (UTC), LiuXuHong <LiuXuHong_member@pathlink.com> wrote: > void main(){ > double fVal = 66.6f; > int rate = 10; > > int intValue = cast(int)(fVal*rate); > printf("Value = %d\n", intValue); > } > > // Value = 665 > // What is the matter? > > This is what is most likely happing: fVal : 66.6 x10 : ~666.0 (ie 665.93853...) (int): 665 [ .93853 cut off] This is due to the fact that floating points are represented in binary, and therefore only values like xxx/1024 can be represented exactly, and when a floating point number is cast to an int, everything after the decemal place is simply truncated. If you must round the number properly, add .5 to it before casting. -- Using Opera's revolutionary e-mail client: http://www.opera.com/m2/ |
November 17, 2004 Re: cast problem | ||||
---|---|---|---|---|
| ||||
Posted in reply to Simon Buchan | On Thu, 18 Nov 2004 01:09:13 +1300, Simon Buchan <currently@no.where> wrote: > On Wed, 17 Nov 2004 09:35:52 +0000 (UTC), LiuXuHong <LiuXuHong_member@pathlink.com> wrote: > >> void main(){ >> double fVal = 66.6f; >> int rate = 10; >> >> int intValue = cast(int)(fVal*rate); >> printf("Value = %d\n", intValue); >> } >> >> // Value = 665 >> // What is the matter? >> >> > > This is what is most likely happing: > fVal : 66.6 > x10 : ~666.0 (ie 665.93853...) > (int): 665 [ .93853 cut off] > > This is due to the fact that floating points are represented in binary, > and therefore only values like xxx/1024 can be represented exactly, > and when a floating point number is cast to an int, everything after the > decemal place is simply truncated. > If you must round the number properly, add .5 to it before casting. Depending on the result you're after you can also move the cast i.e. to get 666: intValue = (cast(int)fVal*rate); (this truncates the 66.6 to 66, then does x10) to get 670: intValue = (cast(int)(fVal+0.5)*rate); (this does what Simon mentioned, then truncates, then does x10) Regan -- Using M2, Opera's revolutionary e-mail client: http://www.opera.com/m2/ |
November 18, 2004 Re: cast problem | ||||
---|---|---|---|---|
| ||||
Posted in reply to Regan Heath | On Thu, 18 Nov 2004 11:39:19 +1300, Regan Heath <regan@netwin.co.nz> wrote: <snip> > Depending on the result you're after you can also move the cast i.e. > > to get 666: intValue = (cast(int)fVal*rate); > (this truncates the 66.6 to 66, then does x10) > > to get 670: intValue = (cast(int)(fVal+0.5)*rate); > (this does what Simon mentioned, then truncates, then does x10) > > Regan > Actually the top one will give you 660 (since when does 66 x 10 give 666 :D), and the bottom will give you 671. (67.1 x 10) You only multiply by .5 just before the cast, of course. (cast(int)(fVal*rate + 0.5)) This gives you your 666. -- Using Opera's revolutionary e-mail client: http://www.opera.com/m2/ |
November 18, 2004 Re: cast problem | ||||
---|---|---|---|---|
| ||||
Posted in reply to Simon Buchan | On Thu, 18 Nov 2004 15:13:59 +1300, Simon Buchan <currently@no.where> wrote: > On Thu, 18 Nov 2004 11:39:19 +1300, Regan Heath <regan@netwin.co.nz> wrote: > > <snip> >> Depending on the result you're after you can also move the cast i.e. >> >> to get 666: intValue = (cast(int)fVal*rate); >> (this truncates the 66.6 to 66, then does x10) >> >> to get 670: intValue = (cast(int)(fVal+0.5)*rate); >> (this does what Simon mentioned, then truncates, then does x10) >> >> Regan >> > > Actually the top one will give you 660 (since when does 66 x 10 give 666 :D), Doh! I mean't to type that ;o) > and the bottom will give you 671. (67.1 x 10) No, it gives 670. The cast to int truncates 67.1 to 67 before the x10 occurs. (cast must have higher precedence than *?) > You only multiply by .5 just before the cast, of course. (cast(int)(fVal*rate + 0.5)) > This gives you your 666. Ahh, so that is what you meant. Multiply, add 0.5 then cast to int truncating 666.x to 666. Regan -- Using M2, Opera's revolutionary e-mail client: http://www.opera.com/m2/ |
Copyright © 1999-2021 by the D Language Foundation