|
== Categories ==
A research literature survey covering 1978 to 2012<ref name="Oracle survey">{{cite journal |last1=Barr |first1=Earl T. |last2=Harman |first2=Mark |last3=McMinn |first3=Phil |last4=Shahbaz |first4=Muzammil |last5=Yoo |first5=Shin |date=November 2014 |title=The Oracle Problem in Software Testing: A Survey |journal=IEEE Transactions on Software Engineering |volume=41 |issue=5 |pages=507–525 |doi=10.1109/TSE.2014.2372785 |url=http://discovery.ucl.ac.uk/1471263/1/06963470.pdf }}</ref> found several potential categorisationscategories forof test oracles.
=== Specified ===
== Examples ==
Examples of common applications of the categories of oracles:
Common oracles include:
* [[specificationSpecification (technical standard)|specificationsSpecifications]] and [[software documentation|documentation]].<ref>{{cite thesis | last = Peters | first = Dennis K. | citeseerx = 10.1.1.69.4331 | title = Generating a Test Oracle from Program Documentation | publisher = McMaster University | degree = M. Eng. | year = 1995 }}</ref><ref>{{cite conference | last1 = Peters | first1 = Dennis K. | last2 = Parnas | first2 = David L. | url = http://www.engr.mun.ca/~dpeters/papers/issta.pdf | title = Generating a Test Oracle from Program Documentation | book-title = Proceedings of the 1994 International Symposium on Software Testing and Analysis | conference = ISSTA | publisher = ACM Press | pages = 58–65 }}</ref> A formal specification used as input to [[model-based design]] and [[model-based testing]] would be an example of a ''specified test oracle''. Documentation that was not a [[formal specification]] of the product would typically be a ''derived test oracle'', e.g. a usage or installation guide, or a record of performance characteristics or minimum machine requirements for the software.
* other products (for instance, anAn oracle for a software program might be a second program that uses a different [[algorithm]] to evaluate the same mathematical expression as the product under test). This is an example of a ''derived test oracle'', a pseudo-oracle.<ref name="pseudo-oracle"/>{{rp|466}}
* During [[Google]] search, we do not have a complete oracle to verify whether the number of returned results is correct. We may define a metamorphic relation<ref>{{cite journal |first1=Z.Q. |last1=Zhou |first2=S. |last2=Zhang |first3=M. |last3=Hagenbuchner |first4=T.H. |last4=Tse |first5=F.-C. | last5=Kuo |first6=T.Y. |last6=Chen |date=2012 |title=Automated functional testing of online search services |journal=Software Testing, Verification and Reliability |volume=22 |issue=4 |pages=221–243 }}</ref> such that a follow-up narrowed-down search will produce fewer results. This is an example of a ''partial oracle''.
* aA ''heuristic oracle'' that provides approximate results or exact results for a set of a few test inputs.<ref>
Hoffman, Douglas; [http://www.softwarequalitymethods.com/Papers/STQE%20Heuristic.pdf Heuristic Test Oracles], Software Testing & Quality Engineering Magazine, 1999</ref>
* aA ''statistical oracle'' that uses statistical characteristics,<ref>{{cite conference |url=http://www.mathematik.uni-ulm.de/sai/mayer/publications/oracles.pdf |title=Test Oracles Using Statistical Methods |last1=Mayer |first1=Johannes |last2=Guderlei |first2=Ralph |date=2004 |publisher=Springer |book-title=Proceedings of the First International Workshop on Software Quality, Lecture Notes in Informatics |pages=179–189 |conference=First International Workshop on Software Quality}}</ref> for example with image analysis where a range of certainty/uncertainty is defined for the test oracle to pronounce a match or not. This would be an example of a ''human test oracle''.
* aA ''consistency oracle'' that compares the results of one test execution to another for similarity.<ref>
Hoffman, Douglas; [http://www.softwarequalitymethods.com/Papers/OracleTax.pdf ''Analysis of a Taxonomy for Test Oracles''], Quality Week, 1998
</ref> This is an example of a ''derived test oracle''.
* aA ''model-based oracle'' that uses the same model to generate and verify system behavior,<ref>Robinson, Harry; [http://www.harryrobinson.net/MBT-on-a-shoestring.pdf ''Finite State Model-Based Testing on a Shoestring''], STAR West 1999</ref> an example of a ''specified test oracle''.
* aA ''human oracle'' (i.e. the correctness of the system under test is determined by manual analysis).<ref name="ammann-intro" />
==References==
| 