Thread overview
bug? floating point precision with std.format
Jun 05, 2017
Seb
Jun 05, 2017
H. S. Teoh
Jun 05, 2017
Jonathan M Davis
June 05, 2017
It appears that the precision parameter in std.format differs from its meaning in printf. Is that expected behavior?

Example:

import std.stdio;
import core.stdc.stdio;

void main()
{
    auto f = 20.66666;
    writeln(f);
    writefln("%0.3s", f);
    printf("%0.3f\n", f);
}

prints:
20.6667
20.7
20.667

It appears that the precision specifier is dictating the total number of digits on *both sides* of the decimal place. Whereas, in C, it's only the number of digits *after* the decimal place.

I'm trying to specify 3 places of precision after the decimal. How do I do this easily?

I'm having a hard time believing this behavior has never been reported, but I can't find anything about it in bugzilla. Tested all the way back to 2.040.

-Steve
June 05, 2017
On Monday, 5 June 2017 at 15:37:42 UTC, Steven Schveighoffer wrote:
> It appears that the precision parameter in std.format differs from its meaning in printf. Is that expected behavior?
>
> Example:
>
> import std.stdio;
> import core.stdc.stdio;
>
> void main()
> {
>     auto f = 20.66666;
>     writeln(f);
>     writefln("%0.3s", f);
>     printf("%0.3f\n", f);
> }
>
> prints:
> 20.6667
> 20.7
> 20.667
>
> It appears that the precision specifier is dictating the total number of digits on *both sides* of the decimal place. Whereas, in C, it's only the number of digits *after* the decimal place.
>
> I'm trying to specify 3 places of precision after the decimal. How do I do this easily?
>
> I'm having a hard time believing this behavior has never been reported, but I can't find anything about it in bugzilla. Tested all the way back to 2.040.
>
> -Steve

You do realize that you have used "s" in the D version?
This works as expected:

writefln("%0.3f", f); // 20.667
printf("%0.3f\n", f); // 20.667

This is a bit more interesting:

writefln("%0.3s", f); // 20.7
printf("%0.3s\n", f); // 20.
June 05, 2017
On Mon, Jun 05, 2017 at 04:29:06PM +0000, Seb via Digitalmars-d wrote:
> On Monday, 5 June 2017 at 15:37:42 UTC, Steven Schveighoffer wrote:
> > It appears that the precision parameter in std.format differs from its meaning in printf. Is that expected behavior?
> > 
> > Example:
> > 
> > import std.stdio;
> > import core.stdc.stdio;
> > 
> > void main()
> > {
> >     auto f = 20.66666;
> >     writeln(f);
> >     writefln("%0.3s", f);
[...]

That should be "%0.3f", not "%0.3s".

If you use the "%s" specifier, precision is interpreted differently, i.e., as "maximum number of characters", as per "%s" in C's printf.


T

-- 
If it tastes good, it's probably bad for you.
June 05, 2017
On 6/5/17 12:53 PM, H. S. Teoh via Digitalmars-d wrote:
> On Mon, Jun 05, 2017 at 04:29:06PM +0000, Seb via Digitalmars-d wrote:
>> On Monday, 5 June 2017 at 15:37:42 UTC, Steven Schveighoffer wrote:
>>> It appears that the precision parameter in std.format differs from its
>>> meaning in printf. Is that expected behavior?
>>>
>>> Example:
>>>
>>> import std.stdio;
>>> import core.stdc.stdio;
>>>
>>> void main()
>>> {
>>>     auto f = 20.66666;
>>>     writeln(f);
>>>     writefln("%0.3s", f);
> [...]
>
> That should be "%0.3f", not "%0.3s".
>
> If you use the "%s" specifier, precision is interpreted differently,
> i.e., as "maximum number of characters", as per "%s" in C's printf.

Interesting. I thought s just stood for "interpret based on the type", and would automatically switch to floating point 'f'. I see in the docs now, it uses 'g', something I've never used.

Curious that 'f' isn't used, I thought it would have been the default.

In any case, I have a fix for my code, move along :)

-Steve
June 05, 2017
On 6/5/17 12:29 PM, Seb wrote:
> You do realize that you have used "s" in the D version?

Yes, I thought it was a stand in for "use the type to determine the specifier", and I mistakenly assumed that would be 'f', since that's what I've always used for floating point. Apparently it is 'g', which behaves as I have shown.

> This is a bit more interesting:
>
> writefln("%0.3s", f); // 20.7
> printf("%0.3s\n", f); // 20.

That is really bad, because %s means interpret the parameter as a char * string. So the memory pointed at by the bit pattern of 20.66666 cast to a pointer, has the first 3 bytes '2', '0', and '.', and then a null character (or a bunch of unprintable characters, followed by a null).

Bizarre...

-Steve
June 05, 2017
On Monday, June 05, 2017 13:23:38 Steven Schveighoffer via Digitalmars-d wrote:
> On 6/5/17 12:29 PM, Seb wrote:
> > You do realize that you have used "s" in the D version?
>
> Yes, I thought it was a stand in for "use the type to determine the specifier", and I mistakenly assumed that would be 'f', since that's what I've always used for floating point. Apparently it is 'g', which behaves as I have shown.

I always assumed that it just meant "convert to string" and that it did basically the same thing that to!string would do, in which case, doing something like passing a number to it like you did would not be legal. Clearly, I never read the docs. :)

- Jonathan M Davis