Test oracle: Difference between revisions

Revision as of 22:01, 16 March 2024

Revision as of 10:37, 19 March 2024

Line 2:

{{other uses|Oracle (disambiguation)}}

In [[computing]], [[software engineering]], and [[software testing]],

given an input for a system a '''test oracle''' (or just '''oracle''') is a procedure that distinguishes between the correct and incorrect behaviors of the System Under Test (SUT).<ref>Earl T. Barr et al; ''[https://ieeexplore.ieee.org/stamp/stamp.jsp?arnumber=6963470 The Oracle Problem in Software Testing: A Survey]'', 2015</ref>. The use of oracles involves comparing the output(s) of the system under test, for a given [[test case|test-case]] input, to the output(s) that the oracle determines that product should have. The term "test oracle" was first introduced in a paper by William E. Howden.<ref>{{cite journal |last1=Howden |first1=W.E. |date=July 1978 |title=Theoretical and Empirical Studies of Program Testing |journal=IEEE Transactions on Software Engineering |volume=4 |issue=4 |pages=293–298 |doi=10.1109/TSE.1978.231514 }}</ref> Additional work on different kinds of oracles was explored by [[Elaine Weyuker]].<ref>Weyuker, Elaine J.; "The Oracle Assumption of Program Testing", in ''Proceedings of the 13th International Conference on System Sciences (ICSS), Honolulu, HI, January 1980'', pp. 44-49</ref>

Oracles often operate separately from the system under test.<ref name="038720881X">Jalote, Pankaj; ''An Integrated Approach to Software Engineering'', Springer/Birkhäuser, 2005, {{ISBN|0-387-20881-X}}</ref> However, [[Method (computer programming)|method]] postconditions are part of the system under test, as automated oracles in [[design by contract]] models.<ref>{{cite journal |last1=Meyer |first1=Bertrand |last2=Fiva |first2=Arno |last3=Ciupa |first3=Ilinca |last4=Leitner |first4=Andreas |last5=Wei |first5=Yi |last6=Stapf |first6=Emmanuel |date=September 2009 |title=Programs That Test Themselves |journal=Computer |volume=42 |issue=9 |pages=46–55 |doi= 10.1109/MC.2009.296 }}</ref> Determining the correct output for a given input (and a set of program or system states) is known as the '''oracle problem''' or test oracle problem,<ref name="Oracle survey"/>{{rp|507}} which is a much harder problem than it seems, and involves working with problems related to controllability and observability.<ref name="ammann-intro">Ammann, Paul; and Offutt, Jeff; "Introduction to Software Testing, 2nd edition", ''Cambridge University Press'', 2016, {{ISBN|978-1107172012}}</ref>