Content deleted Content added
moved footnotes out of references and into footnote § Tag: references removed |
Phil Boswell (talk | contribs) m →Accuracy and binary storage: Spelling/grammar/punctuation/typographical correction |
||
Line 28:
}}
</ref>
To illustrate, the lower figure tabulates the simple addition {{nobr|{{math| 1 + ''x'' − 1}} }} for several values of {{mvar|x}}. All the values of {{mvar|x}} begin at the 15 th decimal, so Excel must take them into account. Before calculating the sum {{nobr|{{math| 1 + ''x''}} ,}} Excel first approximates {{mvar|x}} as a binary number. If this binary version of {{mvar|x}} is a simple power of 2, the 15 digit decimal approximation to {{mvar|x}} is stored in the sum, and the top two examples of the figure indicate recovery of {{mvar|x}} without error. In the third example, {{mvar|x}} is a more complicated binary number, {{nobr|{{math| ''x'' {{=}} 1.110111⋯111 × 2{{sup|−49}} }} }} (15 bits altogether). Here the 'IEEE 754 double value' resulting of the 15 bit figure is 3.330560653658221E-15, which is rounded by Excel for the 'user interface' to 15 digits 3.33056065365822E-15, and then displayed with 30 decimals digits gets one 'fake zero' added, thus the 'binary' and 'decimal' values in the sample are identical only in display, the values associated with the cells are different (1.1101111111111100000000000000000000000000000000000000 × 2<sup>−49</sup> vs. 1.1101111111111011111111111111111111111111111111111101 × 2<sup>−49</sup>). Similar is done by other spreadsheets, the handling of the different amount of decimal digits which can be exactly stored in the 53 bit mantissa of a 'double' (e.g. 16 digits between 1 and 8, but only 15 between {{sfrac|1|2}} and 1 and between 8 and 10) is somewhat difficult and solved 'suboptimal'. In the fourth example, {{mvar|x}} is a ''decimal'' number not equivalent to a simple binary (although it agrees with the binary of the third example to the precision displayed). The decimal input is approximated by a binary and then ''that'' decimal is used. These two middle examples in the figure show that some error is introduced.
The last two examples illustrate what happens if {{mvar|x}} is a rather small number. In the second from last example, {{nobr|{{math|''x'' {{=}} 1.110111⋯111 × 2{{sup|−50}} }} ;}} 15 bits altogether. The binary is replaced very crudely by a single power of 2 (in this example, 2{{sup|−49}}) and its decimal equivalent is used. In the bottom example, a decimal identical with the binary above to the precision shown, is nonetheless approximated differently from the binary, and is eliminated by truncation to 15 significant figures, making no contribution to {{nobr|{{math|1 + ''x'' − 1}} ,}} leading to {{nobr|{{math|''x'' {{=}} 0}} .}}{{efn|
| |||