Test oracle: Difference between revisions

Revision as of 22:30, 4 February 2019

Revision as of 12:41, 5 February 2019

Line 1:

{{other uses2|oracle}}

In [[computing]], [[software engineering]] and [[software testing]] a '''test oracle''', or just '''oracle''', is a mechanism for determining whether a test has passed or failed.<ref>Kaner, Cem; [http://www.testingeducation.org/k04/OracleExamples.htm ''A Course in Black Box Software Testing''], 2004</ref> The use of oracles involves comparing the output(s) of the system under test, for a given [[test case|test-case]] input, to the output(s) that the oracle determines that product should have. The term "test oracle" was first introduced in a paper by William E. Howden.<ref>{{cite journal |last1=Howden |first1=W.E. |date=July 1978 |title=Theoretical and Empirical Studies of Program Testing |url=http://ieeexplore.ieee.org/document/1702537/ |journal=IEEE Transactions on Software Engineering |volume=4 |issue=4 |pages=293–298 |doi=10.1109/TSE.1978.231514 |access-date=27 December 2017 }}</ref> Additional work on different kinds of oracles was explored by [[Elaine Weyuker]].<ref>Weyuker, Elaine J.; "The Oracle Assumption of Program Testing", in ''Proceedings of the 13th International Conference on System Sciences (ICSS), Honolulu, HI, January 1980'', pp. 44-49</ref>

Oracles often operate separately from the system under test.<ref name="038720881X">Jalote, Pankaj; ''An Integrated Approach to Software Engineering'', Springer/Birkhäuser, 2005, {{ISBN|0-387-20881-X}}</ref> However, [[Method (computer programming)|method]] postconditions are part of the system under test, as automated oracles in [[design by contract]] models.<ref>{{cite journal |last1=Meyer |first1=Bertrand |last2=Fiva |first2=Arno |last3=Ciupa |first3=Ilinca |last4=Leitner |first4=Andreas |last5=Wei |first5=Yi |last6=Stapf |first6=Emmanuel |date=September 2009 |title=Programs That Test Themselves |url=http://ieeexplore.ieee.org/document/5233506/ |journal=Computer |volume=42 |issue=9 |pages=46–55 |doi= 10.1109/MC.2009.296 |access-date=29 December 2017 }}</ref> Determining the correct output for a given input (and a set of program/system states) is known as the '''oracle problem''' or '''test oracle problem'''<ref name="Oracle survey"/>{{rp|507}} which is a much harder problem than it seems, and involves working with problems related to controllability and observability.<ref name="ammann-intro">Ammann, Paul; and Offutt, Jeff; "Introduction to Software Testing", ''Cambridge University Press'', 2008, {{ISBN|978-0-521-88038-1}}</ref>