Content deleted Content added
I have added a one feature of random testing |
Citation bot (talk | contribs) Alter: title, template type. Add: chapter-url, chapter. Removed or converted URL. Removed parameters. Some additions/deletions were parameter name changes. | Use this bot. Report bugs. | Suggested by Headbomb | Linked from Wikipedia:WikiProject_Academic_Journals/Journals_cited_by_Wikipedia/Sandbox | #UCB_webform_linked 686/1156 |
||
| (26 intermediate revisions by 16 users not shown) | |||
Line 1:
{{Refimprove|date=September 2014}}
'''Random testing''' is a black-box [[software testing]] technique where programs are tested by [[random number generation|generating]] random, independent inputs. Results of the output are compared against software specifications to verify that the test output is pass or fail.<ref name="Hamlet94"/> In case of absence of specifications the exceptions of the language are used which means if an exception arises during test execution then it means there is a fault in the program, it is also used as a way to avoid biased testing
==History of random testing==
Random testing for hardware was first examined by [[Melvin Breuer]] in 1971 and initial effort to evaluate its effectiveness was done by Pratima and [[Vishwani Agrawal]] in 1975.<ref>{{cite journal
In software, Duran and Ntafos had examined random testing in 1984.<ref>{{cite journal
The use of hypothesis testing as a theoretical basis for random testing was described by Howden in ''Functional Testing and Analysis''. The book also contained the development of a simple formula for estimating the number of tests ''n'' that are needed to have confidence at least 1-1/''n'' in a failure rate of no larger than 1/n. The formula is the lower bound ''n''log''n'', which indicates the large number of failure-free tests needed to have even modest confidence in a modest failure rate bound.<ref name=":0">{{Cite book|last=Howden|first=William|title=Functional Program Testing and Analysis|publisher=McGraw Hill|year=1987|isbn=0-07-030550-1|___location=New York|pages=51–53}}</ref>
==Overview==
Line 12 ⟶ 14:
<syntaxhighlight lang="cpp">
int myAbs(int x) {
if (x > 0) {
return x;
}
Line 28 ⟶ 30:
int x = getRandomInput();
int result = myAbs(x);
assert(result >= 0);
}
}
Line 36 ⟶ 38:
<syntaxhighlight lang="cpp">
int getRandomInput() {
}
Line 44 ⟶ 46:
long y = getRandomInput();
long result = fastMultiplication(x, y);
assert(x * y == result);
}
}
Line 53 ⟶ 55:
==On randomness==
According to the seminal paper on random testing by D. Hamlet
<blockquote>[..] the technical, mathematical meaning of "random testing" refers to an explicit lack of "system" in the choice of test data, so that there is no correlation among different tests.<ref name=Hamlet94>{{cite book|title=Encyclopedia of Software Engineering|year=1994|publisher=John Wiley and Sons|isbn=978-0471540021|author=Richard Hamlet|edition=1st|editor=John J. Marciniak|chapter=Random Testing}}</ref></blockquote>
==Strengths and weaknesses==
{{Unreferenced section|date=August 2014}}
Random testing is
*It is cheap to use: it does not need to be smart about the program under test.
*It does not have any bias: unlike manual testing, it does not overlook bugs because there is misplaced trust in some code.
Line 63 ⟶ 65:
*If software is properly specified: it finds real bugs.
The following weaknesses
*It only finds basic bugs (
*It is only as precise as the specification and specifications are typically imprecise.
*It compares poorly with other techniques to find bugs (
*If different inputs are randomly selected on each test run, this can create problems for [[continuous integration]] because the same tests will pass or fail randomly.<ref name="so">{{cite web|url=
*Some argue that it would be better to thoughtfully cover all relevant cases with manually constructed tests in a white-box fashion, than to rely on randomness.<ref name="so" />
*It may require a very large number of tests for modest levels of confidence in modest failure rates. For example, it will require 459 failure-free tests to have at least 99% confidence that the probability of failure is less than 1/100.<ref name=":0" />
==Types of random testing==
Line 74 ⟶ 77:
===With respect to the input===
*Random input sequence generation (i.e. a sequence of method calls)
*Random sequence of data inputs (sometimes called stochastic testing) -
*Random data selection from existing database
===Guided vs. unguided===
*undirected random test generation - with no heuristics to guide its search
*directed random test generation -
== Implementations ==
Some tools implementing random testing:
* [[QuickCheck]] - a famous test tool, originally developed for [[Haskell (programming language)|Haskell]] but ported to many other languages, that generates random sequences of API calls based on a model and verifies system properties that should hold true after each run
* Randoop - generates sequences of methods and constructor invocations for the classes under test and creates [[JUnit]] tests from these
* Simulant - a [[Clojure]] tool that runs simulations of various agents (
* AutoTest - a tool integrated to EiffelStudio testing automatically Eiffel code with contracts based on the eponymous research prototype.<ref name="AutoTest"/>·
* York Extensible Testing Infrastructure (YETI) - a language agnostic tool which targets various programming languages (Java, JML, CoFoJa, .NET, C, Kermeta).
Line 94 ⟶ 97:
<blockquote>Random testing has only a specialized niche in practice, mostly because an effective oracle is seldom available, but also because of difficulties with the operational profile and with generation of pseudorandom input values.<ref name="Hamlet94"/></blockquote>
For programming languages and platforms which have contracts (
==See also==
*[[Fuzz testing]] - a kind of random testing which provides invalid input to the tested program
*[[Lazy systematic unit testing#Systematic Testing]] - a systematic way of exploring "all" method calls, as implemented e.g. by NASA's [https://web.archive.org/web/20110107010258/http://babelfish.arc.nasa.gov/trac/jpf/wiki/intro/what_is_jpf Java Path Finder] (which blends [https://web.archive.org/web/20110106040136/http://babelfish.arc.nasa.gov/trac/jpf/wiki/intro/testing_vs_model_checking testing with model checking] by limiting the state space to a reasonable size by various means)
*[[SystemVerilog#Constrained random generation|Constrained random generation in SystemVerilog]]
*[[Corner case]]
Line 107 ⟶ 110:
==References==
{{Reflist}}<!--<ref name=":0" />-->
==External links==
Line 114 ⟶ 117:
*[http://c2.com/cgi/wiki?RandomTesting Random Testing wiki] at Cunningham & Cunningham, Inc.
{{software testing}}
[[Category:Software testing]]
| |||