Software testability: Difference between revisions

Revision as of 13:26, 23 September 2014

Revision as of 13:29, 23 September 2014

Line 43:

It has been proved that each class is strictly included into the next. For instance, testing when we assume that the behavior of the implementation under test can be denoted by a deterministic [[finite-state machine]] for some known finite sets of inputs and outputs and with some known number of states belongs to Class I (and all subsequent classes). However, if the number of states is not known, then it only belongs to all classes from Class II on. If the implementation under test must be a deterministic finite-state machine failing the specification for a single trace (and its continuations), and its number of states is unknown, then it only belongs to classes from Class III on. Testing temporal machines where transitions are triggered if inputs are produced within some real-bounded interval only belongs to classes from Class IV on, whereas testing many non-deterministic systems only belongs to Class V (but not all, and some even belong to Class I). The inclusion into Class I does not require the simplicity of the assumed computation model, as some testing cases involving implementations written in any programming language, and testing implementations defined as machines depending on continuous magnitudes, have been proved to be in Class I. Other elaborated cases, such as the testing framework by [[Matthew Hennessy]] under must semantics, and temporal machines with rational timeouts, belong to Class II.

== Testability of Requirementsrequirements ==

Requirements need to fulfill the following criteria in order to be testable:

* '''consistent'''

* '''complete'''

* '''unambiguous'''

* '''quantitative''' (a requirement like "fast response time" can not be [[Verification and Validation (software)|Verificationverification/verified]])

* '''[[Verification and Validation (software)|Verificationverification/verifiable]] in practice''' (a test is feasible not only in theory but also in practice with limited resources)

Treating the requirement as axioms, testability can be treated via asserting existence of a function <math> F_S</math> (software)

such that input <math> I_k </math> generates output <math> O_k </math>, therefore <math> F_S : I \to O </math>. Therefore, the ideal software generates the tuple <math>(I_k,O_k)</math> which is the input-output set <math>\Sigma</math>,

Therefore, the ideal software generates the tuple <math> (I_k,O_k) </math> which is the input-output set <math> \Sigma </math>,

standing for specification.

Now, take a test input <math> I_t </math>, which generates the output <math>I_t</math>, that is the test tuple <math>\tau = (I_t,O_t) </math>. Now, the question is whether or not <math> \tau \in \Sigma </math> or <math> \tau \not \in \Sigma </math>. If it is in the set, the test tuple <math> \tau </math> passes, else the system fails the test input. Therefore, it is of imperative importance to figure out : can we or can we not create a function that effectively translates into the notion of the set [[indicator function]] for the specification set <math> \Sigma </math>.

Now, take a test input <math> I_t </math>, which generates the output <math> I_t </math>, that is the test tuple

<math> \tau = (I_t,O_t) </math>. Now, the question is whether or not <math> \tau \in \Sigma </math> or <math> \tau \not \in \Sigma </math>. If it is in the set, the test tuple <math> \tau </math> passes, else the system fails the test input.

Therefore, it is of imperative importance to figure out : can we or can we not create a function that effectively translates into the notion of the set [[Indicator function]] for the specification set <math> \Sigma </math>.

By the notion, <math> 1_{\Sigma} </math> is the testability function for the specification <math> \Sigma </math>.

The existence should not merely be asserted, should be proven rigorously. Therefore, obviously without algebraic consistency, no such function can be found, and therefore, the specification cease to be termed as testable.

Therefore, obviously without algebraic consistency, no such function can be found, and therefore, the specification cease to be termed as testable.

== See also ==