Numeric precision in Microsoft Excel: Difference between revisions

Revision as of 04:56, 28 December 2010

Revision as of 05:01, 3 March 2011

Line 1:

As with other spreadsheets, [[Microsoft Excel|Excel]] works only to limited accuracy because it retains only a certain number of figures to describe numbers (it has limited [[Arithmetic precision|precision]]). Excel nominally works with [[byte|8-byte]] numbers by default, a modified 1985 version of the [[IEEE 754-2008|IEEE 754 specification]]<ref name=microsoft_spec>

{{cite web |url=http://support.microsoft.com/kb/78113/en-us |title=Floating-point arithmetic may give inaccurate results in Excel |publisher=Microsoft support |work=Revision 8.2 ; article ID: 78113 |date=June 30, 2010 |accessdate=2010-07-02}}

Line 12:

Round-off is the loss of accuracy when numbers that differ by small amounts are subtracted. Because each number has only fifteen significant digits, their difference is inaccurate when there aren't enough significant digits to express the difference.

</ref> [[truncation]], and [[Binary numeral system|binary storage]].

==Accuracy and binary storage==