From 1f4aa672cab101bd928b5970597b9d7b6ce86a41 Mon Sep 17 00:00:00 2001 From: Edward Liu Date: Sat, 8 Apr 2023 12:27:49 +0800 Subject: [PATCH] fix formats for 20230208.3 --- ...ļøā­ļø Why does 0.1 + 0.2 = 0.30000000000000004.md | 103 +++++------------- 1 file changed, 25 insertions(+), 78 deletions(-) diff --git a/sources/tech/20230208.3 ā­ļøā­ļøā­ļø Why does 0.1 + 0.2 = 0.30000000000000004.md b/sources/tech/20230208.3 ā­ļøā­ļøā­ļø Why does 0.1 + 0.2 = 0.30000000000000004.md index 595ac37fb4..6dcac0ae73 100644 --- a/sources/tech/20230208.3 ā­ļøā­ļøā­ļø Why does 0.1 + 0.2 = 0.30000000000000004.md +++ b/sources/tech/20230208.3 ā­ļøā­ļøā­ļø Why does 0.1 + 0.2 = 0.30000000000000004.md @@ -10,26 +10,16 @@ Why does 0.1 + 0.2 = 0.30000000000000004? ====== -Hello! I was trying to write about floating point yesterday, -and I found myself wondering about this calculation, with 64-bit floats: +Hello! I was trying to write about floating point yesterday, and I found myself wondering about this calculation, with 64-bit floats: ``` >>> 0.1 + 0.2 0.30000000000000004 ``` -I realized that I didnā€™t understand exactly how it worked. I mean, I know -floating point calculations are inexact, and I know that you canā€™t exactly -represent `0.1` in binary, but: thereā€™s a floating point number thatā€™s closer to -0.3 than `0.30000000000000004`! So why do we get the answer -`0.30000000000000004`? +I realized that I didnā€™t understand exactly how it worked. I mean, I know floating point calculations are inexact, and I know that you canā€™t exactly represent `0.1` in binary, but: thereā€™s a floating point number thatā€™s closer to 0.3 than `0.30000000000000004`! So why do we get the answer `0.30000000000000004`? -If you donā€™t feel like reading this whole post with a bunch of calculations, the short answer is that -`0.1000000000000000055511151231257827021181583404541015625 + 0.200000000000000011102230246251565404236316680908203125` lies exactly between -2 floating point numbers, -`0.299999999999999988897769753748434595763683319091796875` (usually printed as `0.3`) and -`0.3000000000000000444089209850062616169452667236328125` (usually printed as `0.30000000000000004`). The answer is -`0.30000000000000004` (the second one) because its significand is even. +If you donā€™t feel like reading this whole post with a bunch of calculations, the short answer is that `0.1000000000000000055511151231257827021181583404541015625 + 0.200000000000000011102230246251565404236316680908203125` lies exactly between 2 floating point numbers, `0.299999999999999988897769753748434595763683319091796875` (usually printed as `0.3`) and `0.3000000000000000444089209850062616169452667236328125` (usually printed as `0.30000000000000004`). The answer is `0.30000000000000004` (the second one) because its significand is even. #### how floating point addition works @@ -38,9 +28,7 @@ This is roughly how floating point addition works: - Add together the numbers (with extra precision) - Round the result to the nearest floating point number -So letā€™s use these rules to calculate 0.1 + 0.2. I just learned how floating -point addition works yesterday so itā€™s possible Iā€™ve made some mistakes in this -post, but I did get the answers I expected at the end. +So letā€™s use these rules to calculate 0.1 + 0.2. I just learned how floating point addition works yesterday so itā€™s possible Iā€™ve made some mistakes in this post, but I did get the answers I expected at the end. #### step 1: find out what 0.1 and 0.2 are @@ -53,9 +41,7 @@ First, letā€™s use Python to figure out what the exact values of `0.1` and `0.2` '0.20000000000000001110223024625156540423631668090820312500000000000000000000000000' ``` -These really are the exact values: because floating point numbers are in base -2, you can represent them all exactly in base 10. You just need a lot of digits -sometimes :) +These really are the exact values: because floating point numbers are in base 2, you can represent them all exactly in base 10. You just need a lot of digits sometimes :) #### step 2: add the numbers together @@ -79,8 +65,7 @@ Now, letā€™s look at the floating point numbers around `0.3`. Hereā€™s the close '0.29999999999999998889776975374843459576368331909179687500000000000000000000000000' ``` -We can figure out the next floating point number after `0.3` by serializing -`0.3` to 8 bytes with `struct.pack`, adding 1, and then using `struct.unpack`: +We can figure out the next floating point number after `0.3` by serializing `0.3` to 8 bytes with `struct.pack`, adding 1, and then using `struct.unpack`: ``` >>> struct.pack("!d", 0.3) @@ -100,17 +85,13 @@ Apparently you can also do this with `math.nextafter`: 0.30000000000000004 ``` -So the two 64-bit floats around -`0.3` are -`0.299999999999999988897769753748434595763683319091796875` and +So the two 64-bit floats around `0.3` are `0.299999999999999988897769753748434595763683319091796875` and `0.3000000000000000444089209850062616169452667236328125` #### step 4: find out which one is closest to our result -It turns out that `0.3000000000000000166533453693773481063544750213623046875` -is exactly in the middle of -`0.299999999999999988897769753748434595763683319091796875` and -`0.3000000000000000444089209850062616169452667236328125`. +It turns out that `0.3000000000000000166533453693773481063544750213623046875` is exactly in the middle of +`0.299999999999999988897769753748434595763683319091796875` and `0.3000000000000000444089209850062616169452667236328125`. You can see that with this calculation: @@ -123,10 +104,7 @@ So neither of them is closest. #### how does it know which one to round to? -In the binary representation of a floating point number, thereā€™s a number -called the ā€œsignificandā€. In cases like this (where the result is exactly in -between 2 successive floating point number, itā€™ll round to the one with the -even significand. +In the binary representation of a floating point number, thereā€™s a number called the ā€œsignificandā€. In cases like this (where the result is exactly in between 2 successive floating point number, itā€™ll round to the one with the even significand. In this case thatā€™s `0.300000000000000044408920985006261616945266723632812500` @@ -135,20 +113,13 @@ We actually saw the significand of this number a bit earlier: - 0.30000000000000004 is `struct.unpack('!d', b'?\xd3333334')` - 0.3 is `struct.unpack('!d', b'?\xd3333333')` -The last digit of the big endian hex representation of `0.30000000000000004` is -`4`, so thatā€™s the one with the even significand (because the significand is at -the end). +The last digit of the big endian hex representation of `0.30000000000000004` is `4`, so thatā€™s the one with the even significand (because the significand is at the end). #### letā€™s also work out the whole calculation in binary -Above we did the calculation in decimal, because thatā€™s a little more intuitive -to read. But of course computers donā€™t do these calculations in decimal ā€“ -theyā€™re done in a base 2 representation. So I wanted to get an idea of how that -worked too. +Above we did the calculation in decimal, because thatā€™s a little more intuitive to read. But of course computers donā€™t do these calculations in decimal ā€“ theyā€™re done in a base 2 representation. So I wanted to get an idea of how that worked too. -I donā€™t think this binary calculation part of the post is particularly clear -but it was helpful for me to write out. There are a really a lot of numbers and -it might be terrible to read. +I donā€™t think this binary calculation part of the post is particularly clear but it was helpful for me to write out. There are a really a lot of numbers and it might be terrible to read. #### how 64-bit floats numbers work: exponent and significand @@ -181,11 +152,9 @@ def get_significand(f): return x ^ (exponent << 52) ``` -Iā€™m ignoring the sign bit (the first bit) because we only need these functions -to work on two numbers (0.1 and 0.2) and those two numbers are both positive. +Iā€™m ignoring the sign bit (the first bit) because we only need these functions to work on two numbers (0.1 and 0.2) and those two numbers are both positive. -First, letā€™s get the exponent and significand of 0.1. We need to subtract 1023 -to get the actual exponent because thatā€™s how floating point works. +First, letā€™s get the exponent and significand of 0.1. We need to subtract 1023 to get the actual exponent because thatā€™s how floating point works. ``` >>> get_exponent(0.1) - 1023 @@ -203,9 +172,7 @@ Hereā€™s that calculation in Python: 0.1 ``` -(you might legitimately be worried about floating point accuracy issues with -this calculation, but in this case Iā€™m pretty sure itā€™s fine because these -numbers by definition donā€™t have accuracy issues ā€“ the floating point numbers starting at `2**-4` go up in steps of `1/2**(52 + 4)`) +(you might legitimately be worried about floating point accuracy issues with this calculation, but in this case Iā€™m pretty sure itā€™s fine because these numbers by definition donā€™t have accuracy issues ā€“ the floating point numbers starting at `2**-4` go up in steps of `1/2**(52 + 4)`) We can do the same thing for `0.2`: @@ -309,10 +276,7 @@ Thatā€™s the answer we expected: #### this probably isnā€™t exactly how it works in hardware -The way Iā€™ve described the operations here isnā€™t literally exactly -what happens when you do floating point addition (itā€™s not ā€œsolving for Xā€ for -example), Iā€™m sure there are a lot of efficient tricks. But I think itā€™s about -the same idea. +The way Iā€™ve described the operations here isnā€™t literally exactly what happens when you do floating point addition (itā€™s not ā€œsolving for Xā€ for example), Iā€™m sure there are a lot of efficient tricks. But I think itā€™s about the same idea. #### printing out floating point numbers is pretty weird @@ -325,48 +289,31 @@ We said earlier that the floating point number 0.3 isnā€™t equal to 0.3. Itā€™s So when you print out that number, why does it display `0.3`? -The computer isnā€™t actually printing out the exact value of the number, instead -itā€™s printing out the _shortest_ decimal number `d` which has the property that -our floating point number `f` is the closest floating point number to `d`. +The computer isnā€™t actually printing out the exact value of the number, instead itā€™s printing out the _shortest_ decimal number `d` which has the property that our floating point number `f` is the closest floating point number to `d`. It turns out that doing this efficiently isnā€™t trivial at all, and there are a bunch of academic papers about it like [Printing Floating-Point Numbers Quickly and Accurately][1]. or [How to print floating point numbers accurately][2]. #### would it be more intuitive if computers printed out the exact value of a float? -Rounding to a nice clean decimal value is nice, but in a way I feel like it -might be more intuitive if computers just printed out the exact value of a -floating point number ā€“ it might make it seem a lot less surprising when you -get weird results. +Rounding to a nice clean decimal value is nice, but in a way I feel like it might be more intuitive if computers just printed out the exact value of a floating point number ā€“ it might make it seem a lot less surprising when you get weird results. -To me, -0.1000000000000000055511151231257827021181583404541015625 + -0.200000000000000011102230246251565404236316680908203125 -= 0.3000000000000000444089209850062616169452667236328125 feels less surprising than 0.1 + 0.2 = 0.30000000000000004. +To me, 0.1000000000000000055511151231257827021181583404541015625 + 0.200000000000000011102230246251565404236316680908203125 = 0.3000000000000000444089209850062616169452667236328125 feels less surprising than 0.1 + 0.2 = 0.30000000000000004. Probably this is a bad idea, it would definitely use a lot of screen space. #### a quick note on PHP -Someone in the comments somewhere pointed out that `` -prints out `0.3`. Does that mean that floating point math is different in PHP? +Someone in the comments somewhere pointed out that `` prints out `0.3`. Does that mean that floating point math is different in PHP? I think the answer is no ā€“ if I run: -`` on [this - page][3], I get the exact same answer as in - Python 5.5511151231258E-17. So it seems like the underlying floating point - math is the same. +`` on [this page][3], I get the exact same answer as in Python 5.5511151231258E-17. So it seems like the underlying floating point math is the same. -I think the reason that `0.1 + 0.2` prints out `0.3` in PHP is that PHPā€™s - algorithm for displaying floating point numbers is less precise than Pythonā€™s - ā€“ itā€™ll display `0.3` even if that number isnā€™t the closest floating point - number to 0.3. +I think the reason that `0.1 + 0.2` prints out `0.3` in PHP is that PHPā€™s algorithm for displaying floating point numbers is less precise than Pythonā€™s ā€“ itā€™ll display `0.3` even if that number isnā€™t the closest floating point number to 0.3. #### thatā€™s all! -I kind of doubt that anyone had the patience to follow all of that arithmetic, -but it was helpful for me to write down, so Iā€™m publishing this post anyway. -Hopefully some of this makes sense. +I kind of doubt that anyone had the patience to follow all of that arithmetic, but it was helpful for me to write down, so Iā€™m publishing this post anyway. Hopefully some of this makes sense. -------------------------------------------------------------------------------- @@ -383,4 +330,4 @@ via: https://jvns.ca/blog/2023/02/08/why-does-0-1-plus-0-2-equal-0-3000000000000 [b]: https://github.com/lkxed/ [1]: https://legacy.cs.indiana.edu/~dyb/pubs/FP-Printing-PLDI96.pdf [2]: https://lists.nongnu.org/archive/html/gcl-devel/2012-10/pdfkieTlklRzN.pdf -[3]: https://replit.com/languages/php_cli +[3]: https://replit.com/languages/php_cli