Thread overview
cast problem
Nov 17, 2004
LiuXuHong
Nov 17, 2004
Simon Buchan
Nov 17, 2004
Regan Heath
Nov 18, 2004
Simon Buchan
Nov 18, 2004
Regan Heath
November 17, 2004
void main(){
double fVal = 66.6f;
int rate = 10;

int intValue = cast(int)(fVal*rate);
printf("Value = %d\n", intValue);
}

// Value = 665
// What is the matter?


November 17, 2004
On Wed, 17 Nov 2004 09:35:52 +0000 (UTC), LiuXuHong <LiuXuHong_member@pathlink.com> wrote:

> void main(){
> double fVal = 66.6f;
> int rate = 10;
>
> int intValue = cast(int)(fVal*rate);
> printf("Value = %d\n", intValue);	
> }
>
> // Value = 665
> // What is the matter?
>
>

This is what is most likely happing:
fVal : 66.6
x10  : ~666.0 (ie 665.93853...)
(int): 665 [ .93853 cut off]

This is due to the fact that floating points are represented in binary,
and therefore only values like xxx/1024 can be represented exactly,
and when a floating point number is cast to an int, everything after the
decemal place is simply truncated.
If you must round the number properly, add .5 to it before casting.

-- 
Using Opera's revolutionary e-mail client: http://www.opera.com/m2/
November 17, 2004
On Thu, 18 Nov 2004 01:09:13 +1300, Simon Buchan <currently@no.where> wrote:
> On Wed, 17 Nov 2004 09:35:52 +0000 (UTC), LiuXuHong  <LiuXuHong_member@pathlink.com> wrote:
>
>> void main(){
>> double fVal = 66.6f;
>> int rate = 10;
>>
>> int intValue = cast(int)(fVal*rate);
>> printf("Value = %d\n", intValue);	
>> }
>>
>> // Value = 665
>> // What is the matter?
>>
>>
>
> This is what is most likely happing:
> fVal : 66.6
> x10  : ~666.0 (ie 665.93853...)
> (int): 665 [ .93853 cut off]
>
> This is due to the fact that floating points are represented in binary,
> and therefore only values like xxx/1024 can be represented exactly,
> and when a floating point number is cast to an int, everything after the
> decemal place is simply truncated.
> If you must round the number properly, add .5 to it before casting.

Depending on the result you're after you can also move the cast i.e.

to get 666: intValue = (cast(int)fVal*rate);
(this truncates the 66.6 to 66, then does x10)

to get 670: intValue = (cast(int)(fVal+0.5)*rate);
(this does what Simon mentioned, then truncates, then does x10)

Regan

-- 
Using M2, Opera's revolutionary e-mail client: http://www.opera.com/m2/
November 18, 2004
On Thu, 18 Nov 2004 11:39:19 +1300, Regan Heath <regan@netwin.co.nz> wrote:

 <snip>
> Depending on the result you're after you can also move the cast i.e.
>
> to get 666: intValue = (cast(int)fVal*rate);
> (this truncates the 66.6 to 66, then does x10)
>
> to get 670: intValue = (cast(int)(fVal+0.5)*rate);
> (this does what Simon mentioned, then truncates, then does x10)
>
> Regan
>

Actually the top one will give you 660 (since when does 66 x 10 give 666 :D),
and the bottom will give you 671. (67.1 x 10)

You only multiply by .5 just before the cast, of course. (cast(int)(fVal*rate + 0.5))
This gives you your 666.

-- 
Using Opera's revolutionary e-mail client: http://www.opera.com/m2/
November 18, 2004
On Thu, 18 Nov 2004 15:13:59 +1300, Simon Buchan <currently@no.where> wrote:
> On Thu, 18 Nov 2004 11:39:19 +1300, Regan Heath <regan@netwin.co.nz> wrote:
>
>   <snip>
>> Depending on the result you're after you can also move the cast i.e.
>>
>> to get 666: intValue = (cast(int)fVal*rate);
>> (this truncates the 66.6 to 66, then does x10)
>>
>> to get 670: intValue = (cast(int)(fVal+0.5)*rate);
>> (this does what Simon mentioned, then truncates, then does x10)
>>
>> Regan
>>
>
> Actually the top one will give you 660 (since when does 66 x 10 give 666  :D),

Doh! I mean't to type that ;o)

> and the bottom will give you 671. (67.1 x 10)

No, it gives 670. The cast to int truncates 67.1 to 67 before the x10 occurs.
(cast must have higher precedence than *?)

> You only multiply by .5 just before the cast, of course.  (cast(int)(fVal*rate + 0.5))
> This gives you your 666.

Ahh, so that is what you meant. Multiply, add 0.5 then cast to int truncating 666.x to 666.

Regan

-- 
Using M2, Opera's revolutionary e-mail client: http://www.opera.com/m2/