Random testing: Difference between revisions

Revision as of 15:17, 15 November 2017

Revision as of 15:18, 15 November 2017

Line 3:

==History of random testing==

Random testing for hardware was first examined by [[Melvin Breuer]] in 1971 and initial effort to evaluate its effectiveness was done by Pratima and [[Vishwani Agrawal]] in 1975.<ref>[{{cite journal|url=http://ieeexplore.ieee.org/xpls/abs_all.jsp?arnumber=1672882&tag=1 Agrawal, P.; Agrawal, V.D., "|title=Probabilistic Analysis of Random Test Generation Method for Irredundant Combinational Logic Networks,"|first1=P.|last1=Agrawal|first2=V. Computers,D.|last2=Agrawal|date=1 July 1975|publisher=|journal=IEEE Transactions on , vol.Computers|volume=C-24, no.|issue=7,|pages=691–695|accessdate=15 pp.691,695,November July2017|via=IEEE Xplore|doi=10.1109/T-C.1975].224289}}</ref>

In software, Duran and Ntafos had examined random testing in 1984.<ref>[{{cite journal|url=http://ieeexplore.ieee.org/stamp/stamp.jsp?tp=&arnumber=5010257|title=An Duran,Evaluation of Random Testing|first1=J. and W.|last1=Duran|first2=S. C.|last2=Ntafos,|date=1 An evaluation of random testing,July 1984|publisher=|journal=IEEE Trans.Transactions on Software Eng. Engineering|volume=SE-10|issue=4|pages=438–444|accessdate=15 (July,November 1984),2017|via=IEEE 438-444]Xplore|doi=10.1109/TSE.1984.5010257}}</ref> Earlier Howden had termed it functional testing in 1980.{{Citation needed|date=January 2015}}

==Overview==

Line 67:

*It is only as precise as the specification and specifications are typically imprecise.

*It compares poorly with other techniques to find bugs (f.ex. [[static program analysis]]).

*If different inputs are randomly selected on each test run, this can create problems for [[continuous integration]] because the same tests will pass or fail randomly.<ref name="so">{{cite web|url=http://stackoverflow.com/questions/636353/is-it-a-bad-practice-to-randomly-generate-test-data|title=Is it a bad practice to randomly-generate test data?|website=stackoverflow.com|accessdate=15 November 2017}}</ref>

*Some argue that it would be better to thoughtfully cover all relevant cases with manually constructed tests in a white-box fashion, than to rely on randomness.<ref name="so" />

Line 96:

An [[oracle (software testing)|oracle]] is an instrument for verifying whether the outcomes match the program specification or not. An operation profile is knowledge about usage patterns of the program and thus which parts are more important.

For programming languages and platforms which have contracts (for example Eiffel. .NET or various extensions of Java like JML, CoFoJa...) contracts act as natural oracles and the approach has been applied successfully.<ref name="AutoTest">{{cite web|url=http://se.inf.ethz.ch/research/autotest/|title=AutoTest - Chair of Software Engineering|website=se.inf.ethz.ch|accessdate=15 November 2017}}</ref> In particular, random testing finds more bugs than manual inspections or user reports (albeit different ones).<ref name="ManualvsRandom">{{cite journal|title=On the number and nature of faults found by random testing|year=2009|publisher=John Wiley and Sons|url=http://onlinelibrary.wiley.com/doi/10.1002/stvr.415/abstract|author=Ilinca Ciupa|author2=Alexander Pretschner|author3=Manuel Oriol|author4=Andreas Leitner|author5=Bertrand Meyer|journal=Software Testing, Verification and Reliability|doi=10.1002/stvr.415|volume=21|pages=3–28}}</ref>

==See also==