On Wednesday, 18 May 2016 at 11:11:08 UTC, Walter Bright wrote:
> On 5/18/2016 3:15 AM, deadalnix wrote:
>> On Wednesday, 18 May 2016 at 08:21:18 UTC, Walter Bright wrote:
>>> Trying to make D behave exactly like various C++ compilers do, with all their
>>> semi-documented behavior and semi-documented switches that affect constant
>>> folding behavior, is a hopeless task.
>>>
>>> I doubt various C++ compilers are this compatible, even if they follow the
>>> same ABI.
>>>
>>
>> They aren't. For instance, GCC use arbitrary precision FB, and LLVM uses 128
>> bits soft floats in their innards.
>
> Looks like LLVM had the same idea as myself.
>
> Anyhow, this pretty much destroys the idea that I have proposed some sort of cowboy FP that's going to wreck FP programs.
>
> (What is arbitrary precision FB?)

Typo: arbitrary precision FP. Meaning some soft float that grows as big as necessary to not lose precision à la BitInt but for floats.

On Wednesday, 18 May 2016 at 11:38:23 UTC, Manu wrote:
> That's precisely the suggestion; that compile time execution of a
> given type mirror the runtime, that is, matching precisions in this
> case.
> ...within reason; as Walter has pointed out consistently, it's very
> difficult to be PERFECT for all the reasons he's been repeating, but
> there's still a massive difference between the runtime executing a
> bunch of float code, and the compile time executing it all promoted to
> 80bits. Results will drift apart very quickly.

What do you think can be productively done to improve the situation? I am beginning to think a Wiki-like structure would be better for these discussions where each major debater has their thoughts on a specific issue in a relevantly headed section so there is more clarity.

On 18 May 2016 at 21:53, ixid via Digitalmars-d <digitalmars-d@puremagic.com> wrote:
> On Wednesday, 18 May 2016 at 11:38:23 UTC, Manu wrote:
>>
>> That's precisely the suggestion; that compile time execution of a
>> given type mirror the runtime, that is, matching precisions in this
>> case.
>> ...within reason; as Walter has pointed out consistently, it's very
>> difficult to be PERFECT for all the reasons he's been repeating, but
>> there's still a massive difference between the runtime executing a
>> bunch of float code, and the compile time executing it all promoted to
>> 80bits. Results will drift apart very quickly.
>
>
> What do you think can be productively done to improve the situation? I am beginning to think a Wiki-like structure would be better for these discussions where each major debater has their thoughts on a specific issue in a relevantly headed section so there is more clarity.

I've said so many times; I think it would be more useful if CTFE
operated on the *specified type*. That is all. That should produce
results as close as reasonably possible to the runtime.
If I want CTFE to operate at real precision (as is often the case),
float functions just take an 'isFloatingPoint!T' and users can execute
them at whatever precision they like. Most float functions already
have this form.

On Wednesday, 18 May 2016 at 11:16:44 UTC, Joakim wrote:
> Welcome to the wonderful world of C++! :D
>
> More seriously, it is well-defined for that implementation, you did not raise the issue of the spec till now.  In fact, you seemed not to care what the specs say.

Eh? All C/C++ compilers I have ever used coerce to single precision upon binding.

I care about what the spec says, why it says it and how it is used when targetting the specific hardware.

Or more generally, when writing libraries I target the language spec, when writing applications I target the platform (language + compiler + hardware).

> No, it has nothing to do with language semantics and everything to do with bad numerical programming.

No. You can rant all you want about bad numerical programming. But by your definition all DSP programmers are bad at numerics. Which _obviously_ is not true.

That is grasping for straws.

There is NO excuse for preventing programmers from writing reliable performant code that solves the problem at hand to the accuracy that is required by the application.

> Because floating-point is itself fuzzy, in so many different ways.  You are depending on exactly repeatable results with a numerical type that wasn't meant for it.

Floating point is not fuzzy. It is basically a random sample on an interval of potential solutions with a range that increase with each computation. Unfortunately, you have to use interval arithmetics to get that interval, as in regular floating point you loose the information about the interval. It is an approximation. IEEE 754-2008 makes it possible to get that interval, btw.

Fuzzy is different. Fuzzy means that the range itself is the value. It does not represent an approximation. ;-)


> You keep saying this: where did anyone mention unit tests not running with the same precision till you just brought it up out of nowhere?  The only prior mention was that compile-time calculation of constants that are then checked for bit-exact equality in the tests might have problems, but that's certainly not all tests and I've repeatedly pointed out you should never be checking for bit-exact equality.

I have stated time and time again that it is completely unacceptable if there is even a remote chance for anything in the unit test to evaluate at a higher precision than in the production code.

That is not a unit test, it is a sham.

> The point is that what you consider reliable will be less accurate, sometimes much less.

Define accurate. Accurate has no meaning without a desired outcome to reference.

I care about 4 oscillators having the same phase. THAT MEANS: they all have to use the exact same constants.

If not, they _will_ drift and phase cancellation _will_ occur.

I care about adding and removing the same constant. If not, (more) DC offsets will build up.

It is all about getting below the right tolerance threshold while staying real time. You can compensate by increasing the control rate (reduce the number of interpolated values).


> The point is that there are _always_ bounds, so you can never check for the same value.  Almost any guessed bounds will be better than incorrectly checking for the bit-exact value.

No. Guessed bounds are not better. I have never said that anyone should _incorrectly_ do anything. I have said that they should _correctly_ understand what they are doing and that guessing bounds just leads to a worse problem:

Programs that intermittently fail in spectacular ways that are very difficult to debug.

You cannot even compute the bounds if the compiler can use higher precision. IEEE754-2008 makes it possible to accurately compute bounds.

Not supporting that is _very_ bad.


> should always be thought about.  In the latter case, ie your f(x) example, it has nothing to do with error bounds, but that your f(x) is not only invalid at 2, but in a range around 2.

It isn't. For what typed number besides 2 is it invalid?

> Zero is not the only number that screws up that calculation.

It is. Not only can it screw up the calculation. It can screw up the real time properties of the algorithm on a specific FPU. Which is even worse.

> f(x) and what isn't, that has nothing to do with D.  You want D to provide you a way to only check for 0.0, whereas my point is that there are many numbers in the neighborhood of 0.0 which will screw up your calculation, so really you should be using approxEqual.

What IEEE32 number besides 2 can cause problems?

> It isn't changing your model, you can always use a very small

But it is!

> If your point is that you're modeling artificial worlds that have nothing to do with reality, you can always change your threshold around 0.0 to be much smaller, and who cares if it can't go all the way to zero, it's all artificial, right? :)

Why would I have a threshold around 2, when only 2 is causing problems at the hardware level?


> If you're modeling the real world, any function that blows up and gives you bad data, blows up over a range, never a single point, because that's how measurement works.

If I am analyzing real world data I _absolutely_ want all code paths to use the specified precision. I absolutely don't want to wonder whether some computations have a different pattern than others because of the compiler.

Now, providing 64, 128 or 256 bits mantissa and software emulation is _perfectly_ sound. Computing 10 and 24 bits mantissas as if they are 256 bits without the programmer specifying it is bad.

And yes, half-precision is only 10 bits.

> They will give you large numbers that can be represented in the computer, but do not work out to describe the real world, because such formulas are really invalid in a neighborhood of 2, not just at 2.

I don't understand what you mean. Even Inf as an outcome tells me something. Or in the case of simulation Inf might be completely valid input for the next stage.

> I have not measured this speed myself so I can't say.

www.agner.org

It is of course even worse for 32 bit floats. Then we are at 10x-20x faster than 80bit.

> A lot of hand-waving about how more precision is worse, with no real example, which is what Walter keeps asking for.

I have provided plenty of examples.

You guys just don't want to listen. Because it is against the sects religious beliefs that D falls short of expectations both on integers and floating point.

This keeps D at the hobby level as a language. It is a very deep-rooted cultural problem.

But that is ok. Just don't complain about people saying that D is not fit for production. Because they have several good reasons to say that.

Am Wed, 18 May 2016 11:48:49 +0000
schrieb deadalnix <deadalnix@gmail.com>:

> On Wednesday, 18 May 2016 at 11:11:08 UTC, Walter Bright wrote:
> > On 5/18/2016 3:15 AM, deadalnix wrote:
> >> On Wednesday, 18 May 2016 at 08:21:18 UTC, Walter Bright wrote:
> >>> Trying to make D behave exactly like various C++ compilers
> >>> do, with all their
> >>> semi-documented behavior and semi-documented switches that
> >>> affect constant
> >>> folding behavior, is a hopeless task.
> >>>
> >>> I doubt various C++ compilers are this compatible, even if
> >>> they follow the
> >>> same ABI.
> >>> 
> >>
> >> They aren't. For instance, GCC use arbitrary precision FB, and
> >> LLVM uses 128
> >> bits soft floats in their innards.
> >
> > Looks like LLVM had the same idea as myself.
> >
> > Anyhow, this pretty much destroys the idea that I have proposed some sort of cowboy FP that's going to wreck FP programs.
> >
> > (What is arbitrary precision FB?)
> 
> Typo: arbitrary precision FP. Meaning some soft float that grows as big as necessary to not lose precision à la BitInt but for floats.
> 

Do you have a link explaining GCC actually uses such a soft float? For example https://github.com/gcc-mirror/gcc/blob/master/gcc/fold-const.c#L20 still says "This file should be rewritten to use an arbitrary precision..."

On Wednesday, 18 May 2016 at 12:39:21 UTC, Johannes Pfau wrote:
> Do you have a link explaining GCC actually uses such a soft float? For example https://github.com/gcc-mirror/gcc/blob/master/gcc/fold-const.c#L20 still says "This file should be rewritten to use an arbitrary precision..."

Alright, it seems that GCC doesn't use arbitrary precision float everywhere :)

On Wednesday, 18 May 2016 at 12:27:38 UTC, Ola Fosheim Grøstad wrote:
> And yes, half-precision is only 10 bits.

Actually, it turns out that the mantissa is 11 bits. So it clearly plays louder than other floats. ;-)

On Wednesday, 18 May 2016 at 14:29:42 UTC, Ola Fosheim Grøstad wrote:
> On Wednesday, 18 May 2016 at 12:27:38 UTC, Ola Fosheim Grøstad wrote:
>> And yes, half-precision is only 10 bits.
>
> Actually, it turns out that the mantissa is 11 bits. So it clearly plays louder than other floats. ;-)

The mantissa is 10 bits, but it has 11 bit precision, just as the float type has a 23 bit mantissa and 24 bit precision. AFAIK the only float format that stores the always-one-bit of the mantissa is the x87 80 bit format.

On Wednesday, 18 May 2016 at 12:27:38 UTC, Ola Fosheim Grøstad wrote:
> On Wednesday, 18 May 2016 at 11:16:44 UTC, Joakim wrote:
>> Welcome to the wonderful world of C++! :D
>>
>> More seriously, it is well-defined for that implementation, you did not raise the issue of the spec till now.  In fact, you seemed not to care what the specs say.
>
> Eh? All C/C++ compilers I have ever used coerce to single precision upon binding.
>
> I care about what the spec says, why it says it and how it is used when targetting the specific hardware.
>
> Or more generally, when writing libraries I target the language spec, when writing applications I target the platform (language + compiler + hardware).

I see, so the fact that both the C++ and D specs say the same thing doesn't matter, and the fact that D also has the const float in your example as single-precision at runtime, contrary to your claims, none of that matters.

>> No, it has nothing to do with language semantics and everything to do with bad numerical programming.
>
> No. You can rant all you want about bad numerical programming. But by your definition all DSP programmers are bad at numerics. Which _obviously_ is not true.
>
> That is grasping for straws.
>
> There is NO excuse for preventing programmers from writing reliable performant code that solves the problem at hand to the accuracy that is required by the application.

No sane DSP programmer would write that like you did.

>> Because floating-point is itself fuzzy, in so many different ways.  You are depending on exactly repeatable results with a numerical type that wasn't meant for it.
>
> Floating point is not fuzzy. It is basically a random sample on an interval of potential solutions with a range that increase with each computation. Unfortunately, you have to use interval arithmetics to get that interval, as in regular floating point you loose the information about the interval. It is an approximation. IEEE 754-2008 makes it possible to get that interval, btw.

Or in the vernacular, fuzzy. :) Of course, this is true of all non-integer calculation.

> Fuzzy is different. Fuzzy means that the range itself is the value. It does not represent an approximation. ;-)

It's all approximations.

>> You keep saying this: where did anyone mention unit tests not running with the same precision till you just brought it up out of nowhere?  The only prior mention was that compile-time calculation of constants that are then checked for bit-exact equality in the tests might have problems, but that's certainly not all tests and I've repeatedly pointed out you should never be checking for bit-exact equality.
>
> I have stated time and time again that it is completely unacceptable if there is even a remote chance for anything in the unit test to evaluate at a higher precision than in the production code.
>
> That is not a unit test, it is a sham.

Since the vast majority of tests will never use such compile-test constants, your opinion is not only wrong but irrelevant.

>> The point is that what you consider reliable will be less accurate, sometimes much less.
>
> Define accurate. Accurate has no meaning without a desired outcome to reference.
>
> I care about 4 oscillators having the same phase. THAT MEANS: they all have to use the exact same constants.
>
> If not, they _will_ drift and phase cancellation _will_ occur.
>
> I care about adding and removing the same constant. If not, (more) DC offsets will build up.
>
> It is all about getting below the right tolerance threshold while staying real time. You can compensate by increasing the control rate (reduce the number of interpolated values).

Then don't use differently defined constants in different places and don't use floating point, simple.

>> The point is that there are _always_ bounds, so you can never check for the same value.  Almost any guessed bounds will be better than incorrectly checking for the bit-exact value.
>
> No. Guessed bounds are not better. I have never said that anyone should _incorrectly_ do anything. I have said that they should _correctly_ understand what they are doing and that guessing bounds just leads to a worse problem:
>
> Programs that intermittently fail in spectacular ways that are very difficult to debug.
>
> You cannot even compute the bounds if the compiler can use higher precision. IEEE754-2008 makes it possible to accurately compute bounds.
>
> Not supporting that is _very_ bad.

If you're comparing bit-exact equality, you're not using _any_ error bounds: that's the worst possible choice.

>> should always be thought about.  In the latter case, ie your f(x) example, it has nothing to do with error bounds, but that your f(x) is not only invalid at 2, but in a range around 2.
>
> It isn't. For what typed number besides 2 is it invalid?
>
>> Zero is not the only number that screws up that calculation.
>
> It is. Not only can it screw up the calculation. It can screw up the real time properties of the algorithm on a specific FPU. Which is even worse.
>
>> f(x) and what isn't, that has nothing to do with D.  You want D to provide you a way to only check for 0.0, whereas my point is that there are many numbers in the neighborhood of 0.0 which will screw up your calculation, so really you should be using approxEqual.
>
> What IEEE32 number besides 2 can cause problems?

Sigh, any formula f(x) like the one you present is an approximation for the real world.  The fact that it blows up at 2 means they couldn't measure any values there, so they just stuck that singularity in the formula.  That means it's not merely undefined at 2, but that they couldn't take measurements _around_ 2, which is why any number close to 2 will give you unreasonably large output.

Simply blindly plugging that formula in and claiming that it's only invalid at 2 and nowhere else means you don't really understand the math, nor the science that underlies it.

>> It isn't changing your model, you can always use a very small
>
> But it is!

Not in any meaningful way.

>> If your point is that you're modeling artificial worlds that have nothing to do with reality, you can always change your threshold around 0.0 to be much smaller, and who cares if it can't go all the way to zero, it's all artificial, right? :)
>
> Why would I have a threshold around 2, when only 2 is causing problems at the hardware level?

Because it's not only the hardware that matters, the validity of your formula in the neighborhood of 2 also matters.

>> If you're modeling the real world, any function that blows up and gives you bad data, blows up over a range, never a single point, because that's how measurement works.
>
> If I am analyzing real world data I _absolutely_ want all code paths to use the specified precision. I absolutely don't want to wonder whether some computations have a different pattern than others because of the compiler.
>
> Now, providing 64, 128 or 256 bits mantissa and software emulation is _perfectly_ sound. Computing 10 and 24 bits mantissas as if they are 256 bits without the programmer specifying it is bad.
>
> And yes, half-precision is only 10 bits.

None of this has anything to do with the point you responded to.

>> They will give you large numbers that can be represented in the computer, but do not work out to describe the real world, because such formulas are really invalid in a neighborhood of 2, not just at 2.
>
> I don't understand what you mean. Even Inf as an outcome tells me something. Or in the case of simulation Inf might be completely valid input for the next stage.

Then why are you checking for it?  This entire example was predicated on your desire to avoid Inf at precisely 2.  My point is that it's not only Inf that has to be avoided, other output can be finite and still worthless, because the formula doesn't really apply there. So you should really be using approxEqual, as anywhere else with floating point math, and your example is wrong.

>> A lot of hand-waving about how more precision is worse, with no real example, which is what Walter keeps asking for.
>
> I have provided plenty of examples.

No, you only provided one, the one about const float, that had little relevance.  The rest was a lot of hand-waving about vague scenarios, with no actual example.

> You guys just don't want to listen. Because it is against the sects religious beliefs that D falls short of expectations both on integers and floating point.
>
> This keeps D at the hobby level as a language. It is a very deep-rooted cultural problem.
>
> But that is ok. Just don't complain about people saying that D is not fit for production. Because they have several good reasons to say that.

Funny, considering this entire thread has had many making countervailing claims about the choices Walter has made for D.

On Wednesday, 18 May 2016 at 15:30:42 UTC, Matthias Bentrup wrote:
> On Wednesday, 18 May 2016 at 14:29:42 UTC, Ola Fosheim Grøstad wrote:
>> On Wednesday, 18 May 2016 at 12:27:38 UTC, Ola Fosheim Grøstad wrote:
>>> And yes, half-precision is only 10 bits.
>>
>> Actually, it turns out that the mantissa is 11 bits. So it clearly plays louder than other floats. ;-)
>
> The mantissa is 10 bits, but it has 11 bit precision, just as the float type has a 23 bit mantissa and 24 bit precision. AFAIK the only float format that stores the always-one-bit of the mantissa is the x87 80 bit format.

It turns out both 10bits and 11bits are "right" and that one should qualify it with «with/without the hidden bit», except that it shouldn't be called the «mantissa», but the «significand»:

https://en.wikipedia.org/wiki/Significand

Thanks, always fun with learning  ;-)