Content deleted Content added
m ISBNs (Build KE) |
|||
Line 6:
{{cite book |title=Financial Applications Using Excel Add-in Development in C/C++
|author=Steve Dalton |chapter=Table 2.3: Worksheet data types and limits |pages=13–14 |isbn=
</ref>) Although Excel can display 30 decimal points, its precision for a specified number is confined to 15 [[significant figures]], and calculations may have an accuracy that is even less due to three issues: [[round-off error|round off]],<ref name=roundoff>
Line 21:
The inaccuracy in Excel calculations is more complicated than errors due to a precision of 15 significant figures. Excel's storage of numbers in binary format also affects its accuracy.<ref name=deLevie>
{{cite book |title=Advanced Excel for scientific data analysis |publisher=Oxford University Press |author=Robert de Levie |year=2004 |isbn=
</ref> To illustrate, the lower figure tabulates the simple addition {{nowrap|1 + ''x'' − 1}} for several values of ''x''. All the values of ''x'' begin at the 15-th decimal, so Excel must take them into account. Before calculating the sum 1 + ''x'', Excel first approximates ''x'' as a binary number. If this binary version of ''x'' is a simple power of 2, the 15-digit decimal approximation to ''x'' is stored in the sum, and the top two examples of the figure indicate recovery of ''x'' without error. In the third example, ''x'' is a more complicated binary number, ''x'' = 1.110111⋯111 × 2<sup>−49</sup> (15 bits altogether). Here ''x'' is approximated by the 4-bit binary 1.111 × 2<sup>−49</sup> (some insight into this approximation can be found using [[geometric progression]]: ''x'' = 1.11 × 2<sup>−49</sup> + 2<sup>−52</sup> × (1 − 2<sup>−11</sup>) ≈ 1.11 × 2<sup>−49</sup> + 2<sup>−52</sup> = 1.111 × 2<sup>−49</sup> ) and the decimal equivalent of this crude 4-bit approximation is used. In the fourth example, ''x'' is a ''decimal'' number not equivalent to a simple binary (although it agrees with the binary of the third example to the precision displayed). The decimal input is approximated by a binary and then ''that'' decimal is used. These two middle examples in the figure show that some error is introduced.
Line 41:
In short, a variety of accuracy behavior is introduced by the combination of representing a number with a limited number of binary digits, along with [[Truncation error|truncating]] numbers beyond the fifteenth significant figure.<ref name=deLevie3>
{{cite book |title=cited work |author=Robert de Levie |year=2004 |isbn=
</ref><!-- These figures are simply screen shots of the listed arithmetic using Excel 2007 --> Excel's treatment of numbers beyond 15 significant figures sometimes contributes better accuracy to the final few significant figures of a computation than working directly with only 15 significant figures, and sometimes not.
Line 51:
*[http://docs.sun.com/source/806-3568/ncg_goldberg.html What every computer scientist should know about floating point] Focuses upon examples of floating point representations of numbers.
*[http://support.microsoft.com/default.aspx?scid=http://support.microsoft.com:80/support/kb/articles/Q279/7/55.ASP&NoWebContent=1 Visual basic and arithmetic precision]: Oriented toward VBA, which does things a bit differently.
*{{cite book |title=A guide to Microsoft Excel 2007 for scientists and engineers |author=Bernard V. Liengme |chapter=Mathematical limitations of Excel |page=31 ''ff'' |url=http://books.google.com/books?id=0qDm7uuDmv0C&pg=PA31 |isbn=
</ref>
Line 62:
Accuracy in Excel-provided functions can be an issue. [[Micah Altman]] ''et al.'' provide this example:<ref name=Altman>
{{cite book |title=Numerical issues in statistical computing for the social scientist |author= Micah Altman, Jeff Gill, Michael McDonald |year=2004 |publisher=Wiley-IEEE |isbn=
</ref> The population standard deviation given by:
Line 74:
However, the first form keeps better numerical accuracy for large values of ''x'', because squares of differences between ''x'' and ''x''<sub>av</sub> leads to less round-off than the differences between the much larger numbers Σx<sup>2</sup> and (Σx)<sup>2</sup>. The built-in Excel function STDEVP(), however, uses the less accurate formulation because it is faster computationally.<ref name=Levie>
{{cite book |title=Advanced Excel for scientific data analysis |author=Robert de Levie |publisher=Oxford University Press |year=2004 |isbn=
Line 90:
It is possible to determine the round-off error by using the [[Taylor series]] formula for the square root:<ref name=Ryzhik>
{{cite book |chapter=§1.112 Power series |title=Table of integrals, series and products |author=IS Gradshteyn & IM Ryzhik |edition =7th |publisher=Academic Press |year=2007 |isbn=
</ref>
Line 135:
</ref> Using Visual Basic for Applications, any of these methods can be implemented in Excel. Numerical methods use a grid where functions are evaluated. The functions may be interpolated between grid points or extrapolated to locate adjacent grid points. These formulas involve comparisons of adjacent values. If the grid is spaced very finely, round-off error will occur, and the less the precision used, the worse the round-off error. If spaced widely, accuracy will suffer. If the numerical procedure is thought of as a [[Negative feedback amplifier|feedback system]], this calculation noise may be viewed as a signal that is applied to the system, which will lead to instability unless the system is carefully designed.<ref name=Hamming>
{{cite book |author=[[Richard Hamming|R. W. Hamming]] |title=Numerical Methods for Scientists and Engineers |year= 1986 |isbn=
</ref>
Line 142:
Although Excel nominally works with [[byte|8-byte]] numbers by default, [[Visual basic for applications|VBA]] has a variety of data types. The ''Double'' data type is 8 bytes, the ''Integer'' data type is 2 bytes, and the general purpose 16 byte ''Variant'' data type can be converted to a 12 byte ''Decimal'' data type using the VBA conversion function ''CDec''.<ref name=John_Walkenbach>
{{cite book |title=Excel 2010 Power Programming with VBA |chapter=Defining data types |pages=198 ''ff'' and Table 8-1|isbn=
</ref> Choice of variable types in a VBA calculation involves consideration of storage requirements, accuracy and speed.
| |||