June 13
Quirin Schroll kirjoitti 12.6.2024 klo 16.25:
> On Sunday, 9 June 2024 at 17:54:38 UTC, Walter Bright wrote:
>> On 5/26/2024 1:05 AM, Daniel N wrote:
>>> I was aware of `-ffast-math` but not that the seemingly harmless enabling of gnu dialect extensions would change excess-precision!
>>
>> Supporting various kinds of fp math with a switch is just a bad idea, mainly because so few people understand fp math and its tradeoffs. It will also bork up code that is carefully tuned to the default settings.
> 
> Every once in a while, I believe floating-point numbers are such delicate tools, they should be disabled by default and require a compiler switch to enable. Something like `--enable-floating-point-types--yes-i-know-what-i-am-doing-with-them--i-swear-i-read-the-docs`. And yeah, this is tongue in cheek, but I taught applied numerics and programming for mathematicians for 4 years, and even I get it wrong sometimes. In my work, we use arbitrary precision rational numbers because they’re fool-proof and we don’t need any transcendental functions.

My rule of thumb - if I don't have a better idea - is to treat FP numbers like I'd treat readings from a physical instrument: there's always going to be a slight "random" factor in my calculations. I can't trust `==` operator with FP expression result just like I can't trust I'll get exactly the same result if I measure the length of the same metal rod twice.

Would you agree this being a good basic rule?
June 13

On Wednesday, 12 June 2024 at 21:05:39 UTC, Dukc wrote:

>

Quirin Schroll kirjoitti 12.6.2024 klo 16.25:

>

On Sunday, 9 June 2024 at 17:54:38 UTC, Walter Bright wrote:

>

On 5/26/2024 1:05 AM, Daniel N wrote:

>

I was aware of -ffast-math but not that the seemingly harmless enabling of gnu dialect extensions would change excess-precision!

Supporting various kinds of fp math with a switch is just a bad idea, mainly because so few people understand fp math and its tradeoffs. It will also bork up code that is carefully tuned to the default settings.

Every once in a while, I believe floating-point numbers are such delicate tools, they should be disabled by default and require a compiler switch to enable. Something like --enable-floating-point-types--yes-i-know-what-i-am-doing-with-them--i-swear-i-read-the-docs. And yeah, this is tongue in cheek, but I taught applied numerics and programming for mathematicians for 4 years, and even I get it wrong sometimes. In my work, we use arbitrary precision rational numbers because they’re fool-proof and we don’t need any transcendental functions.

My rule of thumb - if I don't have a better idea - is to treat FP numbers like I'd treat readings from a physical instrument: there's always going to be a slight "random" factor in my calculations. I can't trust == operator with FP expression result just like I can't trust I'll get exactly the same result if I measure the length of the same metal rod twice.

Would you agree this being a good basic rule?

That’s so fundamental, it’s not even a good first step. It’s a good first half-step. The problem isn’t simple rules such as “don’t use ==.” The problem isn’t knowing there are rounding errors. If you know that, you might say: Obviously, that means I can’t trust every single digit printed. True, but that’s all just the beginning. If you implement an algorithm, you have to take into account how rounding errors propagate through the calculations. The issue is that you can’t do that intuitively. You just can’t. You can intuit some obvious problems. Generally speaking, if you implement a formula, you must extract from the algorithm what exactly you are doing and then calculate the so-called condition which tells you if errors add up. While that sounds easy, it can be next to impossible for non-linear problems. (IIRC, for linear ones, it’s always possible, it may just be a lot of work in practice.)

Not to mention other quirks such as == not being an equivalence relation, == equal values not being substitutable, and lexically ordering a bunch of arrays of float values is huge fun.

I haven’t touched FPs in years, and I’m not planning to do so in any professional form maybe ever. If my employer needed something like FPs from me, I suggest to use rationals unless those are a proven bottleneck.

June 14

On Thursday, 13 June 2024 at 19:50:34 UTC, Quirin Schroll wrote:

>

[...] If you implement an algorithm, you have to take into account how rounding errors propagate through the calculations. The issue is that you can’t do that intuitively. You just can’t.

Best is to use an interval-type, that says the result is between the lower and the upper bound. This contains both the error and the accuracy information.
e.g. sqrt(2) may be ]1.414213, 1.414214[, so you know the exact result is in this interval and you can't trust digits after the 6th. If now you square this, the result is ]1.9999984, 2.0000012[, which for sure contains the correct answer.
The comparison operators on these types are ⊂, ⊃, ⊆ and ⊇ - which of course can not only be true or false, but also ND (not defined) if the two intervals intersect.
so checks would ask something like "if(sqrt(2)^^2 ⊂ ]1.999, 2.001[)", so they don't ask only for the value but also for the allowed error.

June 14

On Friday, 14 June 2024 at 06:39:59 UTC, Dom DiSc wrote:

>

On Thursday, 13 June 2024 at 19:50:34 UTC, Quirin Schroll wrote:

>

[...] If you implement an algorithm, you have to take into account how rounding errors propagate through the calculations. The issue is that you can’t do that intuitively. You just can’t.

Best is to use an interval-type, that says the result is between the lower and the upper bound. This contains both the error and the accuracy information.
e.g. sqrt(2) may be ]1.414213, 1.414214[, so you know the exact result is in this interval and you can't trust digits after the 6th. If now you square this, the result is ]1.9999984, 2.0000012[, which for sure contains the correct answer.
The comparison operators on these types are ⊂, ⊃, ⊆ and ⊇ - which of course can not only be true or false, but also ND (not defined) if the two intervals intersect.
so checks would ask something like "if(sqrt(2)^^2 ⊂ ]1.999, 2.001[)", so they don't ask only for the value but also for the allowed error.

[Note: The following contain fractions which only render nice on some fonts, excluding the Dlang forum font.]

Interval arithmetic (even on precise numbers such as rationals) fails when the operations blow up the bound to infinity. This can happen because of two reasons: The actual bounds blow up (i.e. the algorithm isn’t well-conditioned, evidenced by analysis), or analysis shows the bounds are actually bounded, but interval arithmetic doesn’t see it. An example for the latter case:

Let x ∈ [1⁄4, 3⁄4]. Set
y = 1⁄2 − x ⋅ (1 − x) and
z = 1 / y

Then, standard interval arithmetic gives
y ∈ [1⁄2, 1⁄2] − [1⁄16, 9⁄16] = [−1⁄16, 7⁄16]
z ∈ [−∞, −16] ∪ [16⁄7, +∞]

However, rather simple analysis shows:
y ∈ [1⁄2, 1⁄2] − [3⁄16, 1⁄4] = [1⁄4, 5⁄16]
z ∈ [16⁄5, 4]

Of course, interval arithmetic isn’t mathematically wrong, after all
 [16⁄5, 4] ⊂ [−∞, −16] ∪ [16⁄7, +∞],
but the left-hand side interval is a useful bound (it’s optimal, in fact), but the right-hand side one is near-useless as it contains infinity.

The big-picture reason is that interval arithmetic doesn’t work well when a value interacts with itself: It counts all occurrences as independent values. In the example above, in x ⋅ (1 − x), the two factors aren’t independent. And as you can see, for the upper bound of y, that’s not even an issue: If interval arithmetic determined that y is at most 7⁄16 (instead of 5⁄16) and therefore z is at least 16⁄7 (instead of 16⁄5), that’s not too bad. The issue is the lower bound.

Essentially, what one has to do is make every expression containing a variable more than once into its own (primitive) operation, i.e. define f(x) = x ⋅ (1 − x), then determine how f acts on intervals, and use that. An interval type (implemented in D or any other programming language) probably can’t do easily automate that. I could maybe see a way using expression templates or something like that where it could determine that two variables are in fact the same variable, but that is a lot of work.

June 14
Quirin Schroll kirjoitti 14.6.2024 klo 15.15:
> Interval arithmetic (even on precise numbers such as rationals) fails when the operations blow up the bound to infinity. This can happen because of two reasons: The actual bounds blow up (i.e. the algorithm isn’t well-conditioned, evidenced by analysis), or analysis shows the bounds are actually bounded, but interval arithmetic doesn’t see it. An example for the latter case:


We're drifting off-topic as this isn't related to bitfields anymore.  It was largely my fault since I kinda started it by asking you the unrelated guestion about floats - sorry. Anyway, let's move this discussion to another thread or stop it here.
July 03
On Sunday, 5 May 2024 at 23:08:29 UTC, Walter Bright wrote:
> https://github.com/WalterBright/documents/blob/2ec9c5966dccc423a2c4c736a6783d77c255403a/bitfields.md
>
> Adds introspection capabilities.
>
> https://github.com/dlang/dmd/pull/16444

Wouldn’t a more D-like syntax be to have the number of bits following the type instead of following the variable name? This would be more consistent with how D arrays are different than C arrays.

Example:

```
struct Foo {
    uint:4 x;
    uint:5 y;
}
```
July 04

On Wednesday, 3 July 2024 at 19:02:59 UTC, Dave P. wrote:

>

Wouldn’t a more D-like syntax be to have the number of bits following the type instead of following the variable name? This would be more consistent with how D arrays are different than C arrays.

struct Foo {
    uint:4 x;
    uint:5 y;
}

I like this. As for arrays it's much more readable than the C syntax.

July 04

On Thursday, 4 July 2024 at 12:20:52 UTC, Dom DiSc wrote:

>

On Wednesday, 3 July 2024 at 19:02:59 UTC, Dave P. wrote:

>

Wouldn’t a more D-like syntax be to have the number of bits following the type instead of following the variable name? This would be more consistent with how D arrays are different than C arrays.

struct Foo {
    uint:4 x;
    uint:5 y;
}

I like this. As for arrays it's much more readable than the C syntax.

Yes, good idea.

July 04

On Wednesday, 3 July 2024 at 19:02:59 UTC, Dave P. wrote:

>

On Sunday, 5 May 2024 at 23:08:29 UTC, Walter Bright wrote:

>

https://github.com/WalterBright/documents/blob/2ec9c5966dccc423a2c4c736a6783d77c255403a/bitfields.md

Adds introspection capabilities.

https://github.com/dlang/dmd/pull/16444

Wouldn’t a more D-like syntax be to have the number of bits following the type instead of following the variable name? This would be more consistent with how D arrays are different than C arrays.

Example:

struct Foo {
    uint:4 x;
    uint:5 y;
}

This is a great idea. And then you can extract this as a type as well!

I see potential for some nice things here. For instance VRP:

uint:4 value(){
   // return 200; // error
   return 7; // ok
}

int[16] arr;
arr[value()] == 5; // VRP makes this valid

Walter, is this possible to do? This would make a lot of the traits parts unnecessary -- you could introspect the bits just from the type. e.g.

enum bitsForType(T : V:N, V, size_t N) = N;

-Steve

July 05

On Thursday, 4 July 2024 at 18:34:00 UTC, Steven Schveighoffer wrote:

>

This is a great idea. And then you can extract this as a type as well!

I see potential for some nice things here. For instance VRP:

uint:4 value(){
   // return 200; // error
   return 7; // ok
}

int[16] arr;
arr[value()] == 5; // VRP makes this valid

Walter, is this possible to do? This would make a lot of the traits parts unnecessary -- you could introspect the bits just from the type. e.g.

enum bitsForType(T : V:N, V, size_t N) = N;

-Steve

Fits nicely with C23 _BitInt.
https://www.open-std.org/jtc1/sc22/wg14/www/docs/n2709.pdf