I have almost identical (I believe it at least) implementation (D and C++) of the same algorithm that uses Kalman filtering. These implementations though show different results (least significant digits). Before I start investigating I would like to ask if this issue (different results of floating points calculation for D and C++) is well known? May be I can read something about that in web? Does D implementation of floating point types is different than the one of C++?

Most of all I'm interesting in equal results to ease comparing outputs of both implementations between each other. The accuracy itself is enough in my case, but this difference is annoying in some cases.

On 23/07/2019 12:49 AM, drug wrote:
> I have almost identical (I believe it at least) implementation (D and C++) of the same algorithm that uses Kalman filtering. These implementations though show different results (least significant digits). Before I start investigating I would like to ask if this issue (different results of floating points calculation for D and C++) is well known? May be I can read something about that in web? Does D implementation of floating point types is different than the one of C++?
> 
> Most of all I'm interesting in equal results to ease comparing outputs of both implementations between each other. The accuracy itself is enough in my case, but this difference is annoying in some cases.

https://godbolt.org/z/EtZLG0

22.07.2019 15:53, rikki cattermole пишет:
> 
> https://godbolt.org/z/EtZLG0

hmm, in short - this is my local problem?

On 23/07/2019 12:58 AM, drug wrote:
> 22.07.2019 15:53, rikki cattermole пишет:
>>
>> https://godbolt.org/z/EtZLG0
> 
> hmm, in short - this is my local problem?

That is not how I would describe it.

I would describe it as IEEE-754 doing what IEEE-754 is good at.
But my point is, you can get the results to match up, if you care about it.

On Monday, 22 July 2019 at 12:49:24 UTC, drug wrote:
> I have almost identical (I believe it at least) implementation (D and C++) of the same algorithm that uses Kalman filtering. These implementations though show different results (least significant digits). Before I start investigating I would like to ask if this issue (different results of floating points calculation for D and C++) is well known? May be I can read something about that in web? Does D implementation of floating point types is different than the one of C++?
>
> Most of all I'm interesting in equal results to ease comparing outputs of both implementations between each other. The accuracy itself is enough in my case, but this difference is annoying in some cases.

Typical floating point operations in single-precision like a simple (a * b) + c will provide a -140dB difference if order is changed. It's likely the order of operations is not the same in your program, so the least significant digit should be different.

On Monday, 22 July 2019 at 13:23:26 UTC, Guillaume Piolat wrote:
> On Monday, 22 July 2019 at 12:49:24 UTC, drug wrote:
>> I have almost identical (I believe it at least) implementation (D and C++) of the same algorithm that uses Kalman filtering. These implementations though show different results (least significant digits). Before I start investigating I would like to ask if this issue (different results of floating points calculation for D and C++) is well known? May be I can read something about that in web? Does D implementation of floating point types is different than the one of C++?
>>
>> Most of all I'm interesting in equal results to ease comparing outputs of both implementations between each other. The accuracy itself is enough in my case, but this difference is annoying in some cases.
>
> Typical floating point operations in single-precision like a simple (a * b) + c will provide a -140dB difference if order is changed. It's likely the order of operations is not the same in your program, so the least significant digit should be different.

What I would recommend is compute the mean relative error, in double, and if it's below -200 dB, not bother. This is an incredibly low relative error of 0.00000001%.
You will have no difficulty making your D program deterministic, but knowing exactly where the C++ and D differ will be long and serve no purpose.

On Monday, 22 July 2019 at 12:49:24 UTC, drug wrote:
> Before I start investigating I would like to ask if this issue (different results of floating points calculation for D and C++) is well known?

This likely has little to do with the language, and more with the implementation. Basic floating point operations at the same precision should give the same results. There can be differences in float printing (see [1]) and math functions (sqrt, cos, pow etc.) however.

Tips for getting consistent results between C/C++ and D:
- Use the same backend, so compare DMD with DMC, LDC with CLANG and GDC with GCC.
- Use the same C runtime library. On Unix glibc will likely be the default, on Windows you likely use snn.lib, libcmt.lib or msvcrt.dll.
- On the D side, use core.stdc.math instead of std.math
- Use the same optimizations. (Don't use -ffast-math for C)

[1] https://forum.dlang.org/post/fndyoiawueefqoeobdur@forum.dlang.org

22.07.2019 16:26, Guillaume Piolat пишет:
>>
>> Typical floating point operations in single-precision like a simple (a * b) + c will provide a -140dB difference if order is changed. It's likely the order of operations is not the same in your program, so the least significant digit should be different.
> 
> What I would recommend is compute the mean relative error, in double, and if it's below -200 dB, not bother. This is an incredibly low relative error of 0.00000001%.
> You will have no difficulty making your D program deterministic, but knowing exactly where the C++ and D differ will be long and serve no purpose.
Unfortunately error has been turned out to be much bigger than I guessed before. So obviously there is a problem either on D side or on C++ side. Error is too huge to ignore it.

22.07.2019 17:19, drug пишет:
> 22.07.2019 16:26, Guillaume Piolat пишет:
>>>
>>> Typical floating point operations in single-precision like a simple (a * b) + c will provide a -140dB difference if order is changed. It's likely the order of operations is not the same in your program, so the least significant digit should be different.
>>
>> What I would recommend is compute the mean relative error, in double, and if it's below -200 dB, not bother. This is an incredibly low relative error of 0.00000001%.
>> You will have no difficulty making your D program deterministic, but knowing exactly where the C++ and D differ will be long and serve no purpose.
> Unfortunately error has been turned out to be much bigger than I guessed before. So obviously there is a problem either on D side or on C++ side. Error is too huge to ignore it.

There was a typo in C++ implementation. I did simple-n-dirt Python version and after the typo fixed all three implementations show the same result if one filter update occurs. But if several updates happen a subtle difference exists nevertheless, error accumulates somewhere else - time for numerical methods using.

On 22.07.19 14:49, drug wrote:
> I have almost identical (I believe it at least) implementation (D and C++) of the same algorithm that uses Kalman filtering. These implementations though show different results (least significant digits). Before I start investigating I would like to ask if this issue (different results of floating points calculation for D and C++) is well known? May be I can read something about that in web? Does D implementation of floating point types is different than the one of C++?
> 
> Most of all I'm interesting in equal results to ease comparing outputs of both implementations between each other. The accuracy itself is enough in my case, but this difference is annoying in some cases.

This is probably not your problem, but it may be good to know anyway: D allows compilers to perform arbitrary "enhancement" of floating-point precision for parts of the computation, including those performed at compile time. I think this is stupid, but I haven't been able to convince Walter.