Software testability: Difference between revisions

Revision as of 13:31, 23 September 2014

Latest revision as of 21:23, 4 December 2024

(29 intermediate revisions by 22 users not shown)

Line 1:

{{Short description|Extent to which software can be tested}}

{{more footnotes|date=September 2014}}

'''Software testability''' is the degree to which a software artifact (i.e.g. a software system, software module, requirements-requirement, or design document) supports [[Software testing|testing]] in a given test context. If the testability of the softwarean artifact is high, then finding faults in the system (if it has any) by means of testing is easier.

Formally, some systems are testable, and some are not. This classification can be achieved by noticing that, to be testable, for a functionality of the system under test "S", which takes input "I", a computable [[functional predicate]] "V" must exists such that <math>V(S,I)</math> is true when S, given input I, produce a valid output, false otherwise. This function "V" is known as the verification function for the system with input I.

Testability is not an [[intrinsic]] property of a software artifact and can not be measured directly (such as software size). Instead testability is an [[extrinsic]] property which results from interdependency of the software to be tested and the test goals, test methods used, and test resources (i.e., the test context).

Many software systems are untestable, or not immediately testable. For example, Google's [[ReCAPTCHA]], without having any metadata about the images is not a testable system. Recaptcha, however, can be immediately tested if for each image shown, there is a tag stored elsewhere. Given this meta information, one can test the system.

A lower degree of testability results in increased [[test effort]]. In extreme cases a lack of testability may hinder testing parts of the software or [[software requirements]] <u>at all</u>.

Therefore, testability is often thought of as an [[extrinsic]] property which results from interdependency of the software to be tested and the test goals, test methods used, and test resources (i.e., the test context). Even though testability can not be measured directly (such as software size) it should be considered an [[intrinsic]] property of a software artifact because it is highly correlated with other key software qualities such as encapsulation, coupling, cohesion, and redundancy.

The correlation of 'testability' to good design can be observed by seeing that code that has weak cohesion, tight coupling, redundancy and lack of encapsulation is difficult to test.<ref name="DesignPatternsExplained2ndEd">{{cite book | last1=Shalloway | first1=Alan | last2=Trott | first2=Jim | title=Design Patterns Explained, 2nd Ed | page=[https://archive.org/details/isbn_9780321247148/page/133 133] | year=2004 | isbn=978-0321247148 | url-access=registration | url=https://archive.org/details/isbn_9780321247148/page/133 }}</ref>

In order to link the testability with the difficulty to find potential faults in a system (if they exist) by testing it, a relevant measure to assess the testability is how many test cases are needed in each case to form a complete test suite (i.e. a test suite such that, after applying all test cases to the system, collected outputs will let us unambiguously determine whether the system is correct or not according to some specification). If this size is small, then the testability is high. Based on this measure, a [[#Testability hierarchy|testability hierarchy]] has been proposed.

A lower degree of testability results in increased [[test effort]]. In extreme cases a lack of testability may hinder testing parts of the software or [[software requirements]] <u>at all</u>.

== Background ==

Line 30 ⟶ 35:

The testability of software components can be improved by:

* [[Test-driven development]]

* [[Design for testing|Design for testability]] (similar to [[design for test]] in the hardware ___domain)

== Testability hierarchy ==

Based on the amount of test cases required to construct a complete test suite in each context (i.e. a test suite such that, if it is applied to the implementation under test, then we collect enough information to precisely determine whether the system is correct or incorrect according to some specification), a testability hierarchy with the following testability classes has been proposed:<ref>I. Rodríguez, L. Llana, P. Rabanal (2014) “A General Testability Theory: Classes, properties, complexity, and testing reductions”. IEEE Transactions on Software Engineering. doi link: http://doi.ieeecomputersociety.org/10.1109/TSE.2014.2331690</ref><ref>I. Rodríguez (2009) "A General Testability Theory". CONCUR 2009, Springer, LNCS 5710, pp.572–586. doi: 10.1007/978-3-642-04081-8_38</ref>

* Class I: there exists a finite complete test suite.

* Class II: any partial distinguishing rate (i.e. any incomplete capability to distinguish correct systems from incorrect systems) can be reached with a finite test suite.

* Class III: there exists a countable complete test suite.

* Class IV: there exists a complete test suite.

* Class V: all cases.

It has been proved that each class is strictly included into the next. For instance, testing when we assume that the behavior of the implementation under test can be denoted by a deterministic [[finite-state machine]] for some known finite sets of inputs and outputs and with some known number of states belongs to Class I (and all subsequent classes). However, if the number of states is not known, then it only belongs to all classes from Class II on. If the implementation under test must be a deterministic finite-state machine failing the specification for a single trace (and its continuations), and its number of states is unknown, then it only belongs to classes from Class III on. Testing temporal machines where transitions are triggered if inputs are produced within some real-bounded interval only belongs to classes from Class IV on, whereas testing many non-deterministic systems only belongs to Class V (but not all, and some even belong to Class I). The inclusion into Class I does not require the simplicity of the assumed computation model, as some testing cases involving implementations written in any programming language, and testing implementations defined as machines depending on continuous magnitudes, have been proved to be in Class I. Other elaborated cases, such as the testing framework by [[Matthew Hennessy]] under must semantics, and temporal machines with rational timeouts, belong to Class II.

== Testability of requirements ==

Line 56 ⟶ 50:

standing for specification.

Now, take a test input <math> I_t </math>, which generates the output <math>I_tO_t</math>, that is the test tuple <math>\tau = (I_t,O_t) </math>. Now, the question is whether or not <math> \tau \in \Sigma </math> or <math> \tau \not \in \Sigma </math>. If it is in the set, the test tuple <math> \tau </math> passes, else the system fails the test input. Therefore, it is of imperative importance to figure out : can we or can we not create a function that effectively translates into the notion of the set [[indicator function]] for the specification set <math> \Sigma </math>.

By the notion, <math>1_{\Sigma}</math> is the testability function for the specification <math>\Sigma</math>.

Line 62 ⟶ 56:

== See also ==

{{Portal|Software Testing}}

* [[testabilityTestability]]

== References ==

{{Reflist}}

* Robert V. Binder: Testing Object-Oriented Systems: Models, Patterns, and Tools, {{ISBN |0-201-80938-9}}

* Stefan Jungmayr: [https://web.archive.org/web/20071009021801/http://www.dissertation.de/index.php3?active_document=/FDP/sj929%2FFDP%2Fsj929.pdf Improving testability of object-oriented systems], {{ISBN |3-89825-781-9}}

* Wanderlei Souza: [http://patternsgrace-wg.fuka.info.waseda.accenter.jp/SPAQUwp-content/proceedings2009uploads/32012/06/GRACE-TR-P22009-AbstractTestabilityPatterns07.pdf Abstract Testability Patterns], ISSN 1884-0760

* Boris Beizer: [httphttps://books.google.co.incom/books?id=Ixf97h356zcC], Software Testing Techniques

{{Portal|Software Testingquality}}

[[Category:Software testing]]

[[Category:Software quality]]