Content deleted Content added
m clean up spacing around commas and other punctuation fixes, replaced: , → , , ; → ; (2) |
Adding short description: "Details of data storage in a spreadsheet application" |
||
| (3 intermediate revisions by 3 users not shown) | |||
Line 1:
{{Short description|Details of data storage in a spreadsheet application}}
As with other spreadsheets, [[Microsoft Excel]] works only to limited accuracy because it retains only a certain number of figures to describe numbers (it has limited [[Arithmetic precision|precision]]). With some exceptions regarding erroneous values, infinities, and denormalized numbers, Excel calculates in [[double-precision floating-point format]] from the [[IEEE 754-2008|IEEE 754 specification]]<ref name=microsoft_spec>
Line 70 ⟶ 71:
|id=KB 78113
|website=Microsoft support
|date=6 June 2024
|url=http://support.microsoft.com/kb/78113/en-us
}} — A detailed explanation with examples of the binary/15 sig fig storage consequences.
Line 87 ⟶ 89:
|title=What every computer scientist should know about floating point
|journal=Computing Surveys
|doi=10.1145/103162.103163
|type=edited reprint
|id=E19957-01 / 806-3568
|via=Sun Microsystems
|url=http://docs.sun.com/source/806-3568/ncg_goldberg.html
|url-access=subscription
}} — Focuses upon examples of floating point representations of numbers.
* {{cite web
|title=Visual Basic and arithmetic precision
Line 112 ⟶ 116:
}}
1. The shortcomings in the {{code|{{=}} 1 + x - 1}} tasks are a combination of 'fp-math weaknesses' and 'how Excel handles it', especially Excel's rounding. Excel does some rounding and / or 'snap to zero' for most of its results, in average chopping the last 3 bits of the IEEE double representation. This behavior can be switched
2. It is not only clean powers of two surviving, but any combination of values constructed of bits which will be within the 53 bits once the decimal 1 is added. As most decimal values do not have a clean finite representation in binary they will suffer from 'round off' and 'cancellation' in tasks like the above.
Line 173 ⟶ 177:
|title=What every computer scientist should know about floating point
|journal=Computing Surveys
|doi=10.1145/103162.103163
|type=edited reprint
|id=E19957-01 / 806-3568
|via=Sun Microsystems
|url=https://docs.oracle.com/cd/E19957-01/806-3568/ncg_goldberg.html
|url-access=subscription
}} — more or less 'the holy book' of fp-math
</ref>
who states:
Line 260 ⟶ 266:
|series=Department of Physics
|publisher=[[Lund University]]
|url=https://www.researchgate.net/publication/242226580
}}
</ref>
| |||