Test oracle: Difference between revisions

Revision as of 17:25, 13 January 2021

Revision as of 15:56, 17 January 2021

Line 16:

A derived test oracle differentiates correct and incorrect behaviour by using information derived from artefacts of the system. These may include documentation, system execution results and characteristics of versions of the system under test.<ref name="Oracle survey"/>{{rp|514}} Regression test suites (or reports) are an example of a derived test oracle - they are built on the assumption that the result from a previous system version can be used as aid (oracle) for a future system version. Previously measured performance characteristics may be used as an oracle for future system versions, for example, to trigger a question about observed potential performance degradation. Textual documentation from previous system versions may be used as a basis to guide expectations in future system versions.

A '''pseudo-oracle''' falls into the category<ref name="Oracle survey"/>{{rp|515}} falls into the category of derived test oracle. A pseudo-oracle, as defined by Weyuker,<ref name="pseudo-oracle">{{cite journal |last1=Weyuker |first1=E.J. |date=November 1982 |title=On Testing Non-Testable Programs |journal=The Computer Journal |volume=25 |issue=4 |pages=465–470 |doi=10.1093/comjnl/25.4.465 |doi-access=free }}</ref> is a separately written program which can take the same input as the program/system under test so that their outputs may be compared to understand if there might be a problem to investigate.

A '''partial oracle'''<ref name="Oracle survey"/>{{rp|515}} is a hybrid between specified test oracle and derived test oracle. It specifies important (but not complete) properties of the system under test. For example, [[metamorphic testing]] exploits such properties, called metamorphic relations, across multiple executions of the system.

=== Implicit ===