fix formats for 20230208.3

This commit is contained in:
Edward Liu 2023-04-08 12:27:49 +08:00
parent 940da0da94
commit 1f4aa672ca

View File

@ -10,26 +10,16 @@
Why does 0.1 + 0.2 = 0.30000000000000004?
======
Hello! I was trying to write about floating point yesterday,
and I found myself wondering about this calculation, with 64-bit floats:
Hello! I was trying to write about floating point yesterday, and I found myself wondering about this calculation, with 64-bit floats:
```
>>> 0.1 + 0.2
0.30000000000000004
```
I realized that I didnt understand exactly how it worked. I mean, I know
floating point calculations are inexact, and I know that you cant exactly
represent `0.1` in binary, but: theres a floating point number thats closer to
0.3 than `0.30000000000000004`! So why do we get the answer
`0.30000000000000004`?
I realized that I didnt understand exactly how it worked. I mean, I know floating point calculations are inexact, and I know that you cant exactly represent `0.1` in binary, but: theres a floating point number thats closer to 0.3 than `0.30000000000000004`! So why do we get the answer `0.30000000000000004`?
If you dont feel like reading this whole post with a bunch of calculations, the short answer is that
`0.1000000000000000055511151231257827021181583404541015625 + 0.200000000000000011102230246251565404236316680908203125` lies exactly between
2 floating point numbers,
`0.299999999999999988897769753748434595763683319091796875` (usually printed as `0.3`) and
`0.3000000000000000444089209850062616169452667236328125` (usually printed as `0.30000000000000004`). The answer is
`0.30000000000000004` (the second one) because its significand is even.
If you dont feel like reading this whole post with a bunch of calculations, the short answer is that `0.1000000000000000055511151231257827021181583404541015625 + 0.200000000000000011102230246251565404236316680908203125` lies exactly between 2 floating point numbers, `0.299999999999999988897769753748434595763683319091796875` (usually printed as `0.3`) and `0.3000000000000000444089209850062616169452667236328125` (usually printed as `0.30000000000000004`). The answer is `0.30000000000000004` (the second one) because its significand is even.
#### how floating point addition works
@ -38,9 +28,7 @@ This is roughly how floating point addition works:
- Add together the numbers (with extra precision)
- Round the result to the nearest floating point number
So lets use these rules to calculate 0.1 + 0.2. I just learned how floating
point addition works yesterday so its possible Ive made some mistakes in this
post, but I did get the answers I expected at the end.
So lets use these rules to calculate 0.1 + 0.2. I just learned how floating point addition works yesterday so its possible Ive made some mistakes in this post, but I did get the answers I expected at the end.
#### step 1: find out what 0.1 and 0.2 are
@ -53,9 +41,7 @@ First, lets use Python to figure out what the exact values of `0.1` and `0.2`
'0.20000000000000001110223024625156540423631668090820312500000000000000000000000000'
```
These really are the exact values: because floating point numbers are in base
2, you can represent them all exactly in base 10. You just need a lot of digits
sometimes :)
These really are the exact values: because floating point numbers are in base 2, you can represent them all exactly in base 10. You just need a lot of digits sometimes :)
#### step 2: add the numbers together
@ -79,8 +65,7 @@ Now, lets look at the floating point numbers around `0.3`. Heres the close
'0.29999999999999998889776975374843459576368331909179687500000000000000000000000000'
```
We can figure out the next floating point number after `0.3` by serializing
`0.3` to 8 bytes with `struct.pack`, adding 1, and then using `struct.unpack`:
We can figure out the next floating point number after `0.3` by serializing `0.3` to 8 bytes with `struct.pack`, adding 1, and then using `struct.unpack`:
```
>>> struct.pack("!d", 0.3)
@ -100,17 +85,13 @@ Apparently you can also do this with `math.nextafter`:
0.30000000000000004
```
So the two 64-bit floats around
`0.3` are
`0.299999999999999988897769753748434595763683319091796875` and
So the two 64-bit floats around `0.3` are `0.299999999999999988897769753748434595763683319091796875` and
`0.3000000000000000444089209850062616169452667236328125`
#### step 4: find out which one is closest to our result
It turns out that `0.3000000000000000166533453693773481063544750213623046875`
is exactly in the middle of
`0.299999999999999988897769753748434595763683319091796875` and
`0.3000000000000000444089209850062616169452667236328125`.
It turns out that `0.3000000000000000166533453693773481063544750213623046875` is exactly in the middle of
`0.299999999999999988897769753748434595763683319091796875` and `0.3000000000000000444089209850062616169452667236328125`.
You can see that with this calculation:
@ -123,10 +104,7 @@ So neither of them is closest.
#### how does it know which one to round to?
In the binary representation of a floating point number, theres a number
called the “significand”. In cases like this (where the result is exactly in
between 2 successive floating point number, itll round to the one with the
even significand.
In the binary representation of a floating point number, theres a number called the “significand”. In cases like this (where the result is exactly in between 2 successive floating point number, itll round to the one with the even significand.
In this case thats `0.300000000000000044408920985006261616945266723632812500`
@ -135,20 +113,13 @@ We actually saw the significand of this number a bit earlier:
- 0.30000000000000004 is `struct.unpack('!d', b'?\xd3333334')`
- 0.3 is `struct.unpack('!d', b'?\xd3333333')`
The last digit of the big endian hex representation of `0.30000000000000004` is
`4`, so thats the one with the even significand (because the significand is at
the end).
The last digit of the big endian hex representation of `0.30000000000000004` is `4`, so thats the one with the even significand (because the significand is at the end).
#### lets also work out the whole calculation in binary
Above we did the calculation in decimal, because thats a little more intuitive
to read. But of course computers dont do these calculations in decimal
theyre done in a base 2 representation. So I wanted to get an idea of how that
worked too.
Above we did the calculation in decimal, because thats a little more intuitive to read. But of course computers dont do these calculations in decimal theyre done in a base 2 representation. So I wanted to get an idea of how that worked too.
I dont think this binary calculation part of the post is particularly clear
but it was helpful for me to write out. There are a really a lot of numbers and
it might be terrible to read.
I dont think this binary calculation part of the post is particularly clear but it was helpful for me to write out. There are a really a lot of numbers and it might be terrible to read.
#### how 64-bit floats numbers work: exponent and significand
@ -181,11 +152,9 @@ def get_significand(f):
return x ^ (exponent << 52)
```
Im ignoring the sign bit (the first bit) because we only need these functions
to work on two numbers (0.1 and 0.2) and those two numbers are both positive.
Im ignoring the sign bit (the first bit) because we only need these functions to work on two numbers (0.1 and 0.2) and those two numbers are both positive.
First, lets get the exponent and significand of 0.1. We need to subtract 1023
to get the actual exponent because thats how floating point works.
First, lets get the exponent and significand of 0.1. We need to subtract 1023 to get the actual exponent because thats how floating point works.
```
>>> get_exponent(0.1) - 1023
@ -203,9 +172,7 @@ Heres that calculation in Python:
0.1
```
(you might legitimately be worried about floating point accuracy issues with
this calculation, but in this case Im pretty sure its fine because these
numbers by definition dont have accuracy issues the floating point numbers starting at `2**-4` go up in steps of `1/2**(52 + 4)`)
(you might legitimately be worried about floating point accuracy issues with this calculation, but in this case Im pretty sure its fine because these numbers by definition dont have accuracy issues the floating point numbers starting at `2**-4` go up in steps of `1/2**(52 + 4)`)
We can do the same thing for `0.2`:
@ -309,10 +276,7 @@ Thats the answer we expected:
#### this probably isnt exactly how it works in hardware
The way Ive described the operations here isnt literally exactly
what happens when you do floating point addition (its not “solving for X” for
example), Im sure there are a lot of efficient tricks. But I think its about
the same idea.
The way Ive described the operations here isnt literally exactly what happens when you do floating point addition (its not “solving for X” for example), Im sure there are a lot of efficient tricks. But I think its about the same idea.
#### printing out floating point numbers is pretty weird
@ -325,48 +289,31 @@ We said earlier that the floating point number 0.3 isnt equal to 0.3. Its
So when you print out that number, why does it display `0.3`?
The computer isnt actually printing out the exact value of the number, instead
its printing out the _shortest_ decimal number `d` which has the property that
our floating point number `f` is the closest floating point number to `d`.
The computer isnt actually printing out the exact value of the number, instead its printing out the _shortest_ decimal number `d` which has the property that our floating point number `f` is the closest floating point number to `d`.
It turns out that doing this efficiently isnt trivial at all, and there are a bunch of academic papers about it like [Printing Floating-Point Numbers Quickly and Accurately][1]. or [How to print floating point numbers accurately][2].
#### would it be more intuitive if computers printed out the exact value of a float?
Rounding to a nice clean decimal value is nice, but in a way I feel like it
might be more intuitive if computers just printed out the exact value of a
floating point number it might make it seem a lot less surprising when you
get weird results.
Rounding to a nice clean decimal value is nice, but in a way I feel like it might be more intuitive if computers just printed out the exact value of a floating point number it might make it seem a lot less surprising when you get weird results.
To me,
0.1000000000000000055511151231257827021181583404541015625 +
0.200000000000000011102230246251565404236316680908203125
= 0.3000000000000000444089209850062616169452667236328125 feels less surprising than 0.1 + 0.2 = 0.30000000000000004.
To me, 0.1000000000000000055511151231257827021181583404541015625 + 0.200000000000000011102230246251565404236316680908203125 = 0.3000000000000000444089209850062616169452667236328125 feels less surprising than 0.1 + 0.2 = 0.30000000000000004.
Probably this is a bad idea, it would definitely use a lot of screen space.
#### a quick note on PHP
Someone in the comments somewhere pointed out that `<?php echo (0.1 + 0.2 );?>`
prints out `0.3`. Does that mean that floating point math is different in PHP?
Someone in the comments somewhere pointed out that `<?php echo (0.1 + 0.2 );?>` prints out `0.3`. Does that mean that floating point math is different in PHP?
I think the answer is no if I run:
`<?php echo (0.1 + 0.2 )- 0.3);?>` on [this
page][3], I get the exact same answer as in
Python 5.5511151231258E-17. So it seems like the underlying floating point
math is the same.
`<?php echo (0.1 + 0.2 )- 0.3);?>` on [this page][3], I get the exact same answer as in Python 5.5511151231258E-17. So it seems like the underlying floating point math is the same.
I think the reason that `0.1 + 0.2` prints out `0.3` in PHP is that PHPs
algorithm for displaying floating point numbers is less precise than Pythons
itll display `0.3` even if that number isnt the closest floating point
number to 0.3.
I think the reason that `0.1 + 0.2` prints out `0.3` in PHP is that PHPs algorithm for displaying floating point numbers is less precise than Pythons itll display `0.3` even if that number isnt the closest floating point number to 0.3.
#### thats all!
I kind of doubt that anyone had the patience to follow all of that arithmetic,
but it was helpful for me to write down, so Im publishing this post anyway.
Hopefully some of this makes sense.
I kind of doubt that anyone had the patience to follow all of that arithmetic, but it was helpful for me to write down, so Im publishing this post anyway. Hopefully some of this makes sense.
--------------------------------------------------------------------------------
@ -383,4 +330,4 @@ via: https://jvns.ca/blog/2023/02/08/why-does-0-1-plus-0-2-equal-0-3000000000000
[b]: https://github.com/lkxed/
[1]: https://legacy.cs.indiana.edu/~dyb/pubs/FP-Printing-PLDI96.pdf
[2]: https://lists.nongnu.org/archive/html/gcl-devel/2012-10/pdfkieTlklRzN.pdf
[3]: https://replit.com/languages/php_cli
[3]: https://replit.com/languages/php_cli