Content deleted Content added
Additional material on random testing. |
|||
Line 6:
In software, Duran and Ntafos had examined random testing in 1984.<ref>{{cite journal|title=An Evaluation of Random Testing|first1=J. W.|last1=Duran|first2=S. C.|last2=Ntafos|date=1 July 1984|journal=IEEE Transactions on Software Engineering|volume=SE-10|issue=4|pages=438–444|doi=10.1109/TSE.1984.5010257}}</ref>
The use of hypothesis testing as a theoretical basis for random testing was described by Howden in ''Functional Testing and Analysis''. The book also contained the development of a simple formula for estimating the number of tests ''n'' that are needed to have confidence at least 1-1/''n'' in a failure rate of no larger than 1/n. The formula is the lower bound ''n''log''n'', which indicates the large number of failure-free tests needed to have even modest confidence in a modest failure rate bound.<ref name=":0">{{Cite book|last=Howden|first=William|title=Functional Program Testing and Analysis|publisher=McGraw Hill|year=1987|isbn=0-07-030550-1|___location=New York|pages=51-53}}</ref>
==Overview==
Line 60 ⟶ 59:
==Strengths and weaknesses==
{{Unreferenced section|date=August 2014}}
Random testing is
*It is cheap to use: it does not need to be smart about the program under test.
*It does not have any bias: unlike manual testing, it does not overlook bugs because there is misplaced trust in some code.
Line 66 ⟶ 65:
*If software is properly specified: it finds real bugs.
The following weaknesses
*It only finds basic bugs (f.ex. [[null pointer]] dereferencing).
*It is only as precise as the specification and specifications are typically imprecise.
Line 72 ⟶ 71:
*If different inputs are randomly selected on each test run, this can create problems for [[continuous integration]] because the same tests will pass or fail randomly.<ref name="so">{{cite web|url=https://stackoverflow.com/q/636353 |title=Is it a bad practice to randomly-generate test data?|website=stackoverflow.com|accessdate=15 November 2017}}</ref>
*Some argue that it would be better to thoughtfully cover all relevant cases with manually constructed tests in a white-box fashion, than to rely on randomness.<ref name="so" />
*It may require a very large number of tests for modest levels of confidence in modest failure rates. For example, it will require 459 failure-free tests to have at least 99% confidence that the probability of failure is less than 1/100.<ref name=":0" />
==Types of random testing==
Line 110:
==References==
{{Reflist}}<ref name=":0" />
==External links==
| |||