Menu
Search

Forums

Forum Index

Thread overview
Weird floating point rounding - Bug or how to control it correctly
Sep 14, 2023
An Pham
Sep 14, 2023
Jonathan M Davis
Sep 14, 2023
Basile B.
September 14, 2023
Gravatar of An PhamPosted by An PhamReply
An Pham
Gravatar of An Pham


Reply

import std.stdio;

void main()
{
    float f = 6394763.345f;
	
    import std.format : sformat;

    char[80] vBuffer = void;
    writeln("6394763.345 = ", sformat(vBuffer[], "%.4f", f));
	
}

Output
6394763.345 = 6394763.5000

September 13, 2023
Gravatar of Jonathan M DavisPosted by Jonathan M Davis
in reply to An Pham		Reply
Jonathan M Davis
Gravatar of Jonathan M Davis
Posted in reply to An Pham


Reply
On Wednesday, September 13, 2023 9:23:48 PM MDT An Pham via Digitalmars-d- learn wrote:
> import std.stdio;
>
>      void main()
>      {
>          float f = 6394763.345f;
>
>          import std.format : sformat;
>
>          char[80] vBuffer = void;
>          writeln("6394763.345 = ", sformat(vBuffer[], "%.4f", f));
>
>      }
>
> Output
> 6394763.345 = 6394763.5000

The nature of floating point numbers is such that there a bunch of values that they can't actually represent, and they get rounded pretty easily depending on the exact numbers involved. So, in general, you shouldn't expect floating point numbers to be exact. You will generally do better with increased precision though, and in this case, it looks like your number will stay the same if you use double or real instead of float.

I would suggest that you watch this video from dconf 2016 which discusses floating point values:

https://www.youtube.com/watch?v=YEUAUnamQiA

- Jonathan M Davis
September 14, 2023
Gravatar of Basile B.Posted by Basile B.
in reply to An Pham		Reply
Basile B.
Gravatar of Basile B.
Posted in reply to An Pham


Reply

On Thursday, 14 September 2023 at 03:23:48 UTC, An Pham wrote:

>

import std.stdio;

void main()
{
    float f = 6394763.345f;
	
    import std.format : sformat;

    char[80] vBuffer = void;
    writeln("6394763.345 = ", sformat(vBuffer[], "%.4f", f));
	
}

Output
6394763.345 = 6394763.5000

Classic question. The float literal 6394763.345f is not representable as IEEE-754 floating point number.

Try https://www.h-schmidt.net/FloatConverter/IEEE754.html for a short introduction to the issue.