Thread overview | ||||||
---|---|---|---|---|---|---|
|
October 13, 2004 / operator for float values | ||||
---|---|---|---|---|
| ||||
code: # float f = 1.2e-3; # float a = 12.0; # float b = 10000.0; # float c = a/b; # assert(f == c); # assert(f == a/b); The first assertion passes but the second fails. test case: svn://svn.kuehne.cn/dstress/run/float_literal_dec_02.d |
October 13, 2004 Re: / operator for float values | ||||
---|---|---|---|---|
| ||||
Posted in reply to Thomas Kuehne | Thomas Kuehne wrote:
> code:
> # float f = 1.2e-3;
> # float a = 12.0;
> # float b = 10000.0;
> # float c = a/b;
> # assert(f == c);
> # assert(f == a/b);
>
> The first assertion passes but the second fails.
>
> test case: svn://svn.kuehne.cn/dstress/run/float_literal_dec_02.d
This is because the first comparison uses float while the second uses real; the float truncates some rounding error in the division that is preserved for the comparison in the second assert. This is a natural effect of type promotion.
The real bug is that this doesn't work properly:
assert(f == cast(float) (a/b));
This should cause the value to be truncated, but it doesn't.
You can cause your code to succeed if you eliminate the rounding error. Divide by a power of 2 instead:
float f = 0.01171875;
float a = 12.0;
float b = 1024.0;
This will then work. That could probably be considered a correct test of the language, as D depends on IEEE being present.
|
October 13, 2004 Re: / operator for float values | ||||
---|---|---|---|---|
| ||||
Posted in reply to Burton Radons | Burton Radons schrieb:
> > code:
> > # float f = 1.2e-3;
> > # float a = 12.0;
> > # float b = 10000.0;
> > # float c = a/b;
> > # assert(f == c);
> > # assert(f == a/b);
> >
> > The first assertion passes but the second fails.
> >
> > test case: svn://svn.kuehne.cn/dstress/run/float_literal_dec_02.d
>
> This is because the first comparison uses float while the second uses real; the float truncates some rounding error in the division that is preserved for the comparison in the second assert. This is a natural effect of type promotion.
>
> The real bug is that this doesn't work properly:
>
> assert(f == cast(float) (a/b));
>
> This should cause the value to be truncated, but it doesn't.
Thanks for the fix - guess I have to read a little about IEEE :)
Thomas
|
October 13, 2004 Re: / operator for float values | ||||
---|---|---|---|---|
| ||||
Posted in reply to Burton Radons | Thanks for the time and effort the two of you are putting into making 'D' an even better tool. "Burton Radons" <burton-radons@smocky.com> wrote in message news:ckj8ps$2te8$1@digitaldaemon.com... > Thomas Kuehne wrote: > > > code: > > # float f = 1.2e-3; |
Copyright © 1999-2021 by the D Language Foundation