Content deleted Content added
m MOS:FRAC / MOS:UNITNAMES for inches, feet, and arc, with metric conversion (via WP:JWB) |
GreenC bot (talk | contribs) Rescued 1 archive link; reformat 1 link. Wayback Medic 2.5 per WP:USURPURL and JUDI batch #21aa |
||
Line 288:
The representation has a limited precision. For example, only 15 decimal digits can be represented with a 64-bit real. If a very small floating-point number is added to a large one, the result is just the large one. The small number was too small to even show up in 15 or 16 digits of resolution, and the computer effectively discards it. Analyzing the effect of limited precision is a well-studied problem. Estimates of the magnitude of round-off errors and methods to limit their effect on large calculations are part of any large computation project. The precision limit is different from the range limit, as it affects the significand, not the exponent.
The significand is a binary fraction that doesn't necessarily perfectly match a decimal fraction. In many cases a sum of reciprocal powers of 2 does not match a specific decimal fraction, and the results of computations will be slightly off. For example, the decimal fraction "0.1" is equivalent to an infinitely repeating binary fraction: 0.000110011 ...<ref>{{cite web|last=Goebel|first=Greg|title=Computer Numbering Format|url=http://www.vectorsite.net/tsfloat.html|archive-url=https://archive.today/20130222091425/http://www.vectorsite.net/tsfloat.html|url-status=usurped|archive-date=February 22, 2013|access-date=10 September 2012}}</ref>
==Numbers in programming languages==
| |||