February 07, 2007 signed / unsigned oddity | ||||
---|---|---|---|---|
| ||||
Is this a bug or did I misunderstood arithmetic or cast()? cast(int)(-5 * 1 * 0.41)); // result: -2 cast(int)(-5 * 1u); // result: -5 cast(int)(-5 * 1u * 0.41)); // result: 1760936589 The last result seems strange to me. I use dmd 1.005. Henning |
February 07, 2007 Re: signed / unsigned oddity | ||||
---|---|---|---|---|
| ||||
Posted in reply to Henning Hasemann | Henning Hasemann wrote:
> Is this a bug or did I misunderstood arithmetic or cast()?
>
> cast(int)(-5 * 1 * 0.41)); // result: -2
> cast(int)(-5 * 1u); // result: -5
> cast(int)(-5 * 1u * 0.41)); // result: 1760936589
>
> The last result seems strange to me. I use dmd 1.005.
This is caused by the type conversion rules: typeof(-5 * 1u) == uint, so -5 * 1u * 0.41 == cast(uint)-5 * 0.41 == 0xfffffffb * 0.41 == 1760936589.31 which rounds to 1760936589 when cast to an int.
|
Copyright © 1999-2021 by the D Language Foundation