Content deleted Content added
m nicer refs III |
Jnestorius (talk | contribs) |
||
| (47 intermediate revisions by 28 users not shown) | |||
Line 1:
{{About|a software testing methodology|decision trees and prediction modeling using them (known as Classification tree)|Decision tree|and|Classification tree}}
The '''Classification Tree Method''' is a method for test design,<ref>{{cite book|last=Bath|first=Graham|author2=McKay, Judy|title=The software test engineer's handbook : a study guide for the ISTQB test analyst and technical test analyst advanced level certificates|year=2008|publisher=Rocky Nook|___location=Santa Barbara, CA|isbn=9781933952246|edition=1st}}</ref> as it is used in different areas of [[software development]].<ref name=advStest>{{cite book|last=Hass|first=Anne Mette Jonassen|title=Guide to advanced software testing|year=2008|publisher=Artech House|___location=Boston|isbn=
It
Classification Trees in terms of the Classification Tree Method must not be confused with [[decision tree]]s.
The classification tree method consists of two major steps:<ref name=lkk>{{cite book|last=Kuhn|first=D. Richard|author2=Kacker, Raghu N. |author3=Lei, Yu |title=Introduction to combinatorial testing|year=2013|publisher=Crc Pr Inc|isbn=
# Identification of test relevant aspects (so called ''classifications'') and their corresponding values (called ''classes'') as well as
# Combination of different classes from all classifications into [[test case (software)|test case]]s.
The identification of test relevant aspects usually follows the (functional) [[specification]] (e.g. [[requirements]], [[use case]]s …) of the [[system under test]].
Line 19:
Prerequisites for applying the classification tree method (CTM) is the selection (or definition) of a [[system under test]].
The CTM is a [[black-box testing]] method and supports any type of system under test. This includes (but is not limited to) [[Computer hardware|hardware
With a selected system under test, the first step of the classification tree method is the identification of test relevant aspects.<ref name=lkk
Any system under test can be described by a set of classifications, holding both input and output parameters.
(Input parameters can also include [[Environment (systems)|environment]]s states, [[pre-condition]]s and other, rather uncommon parameters).<ref name=advStest
Each classification can have any number of disjoint classes, describing the occurrence of the parameter.
The selection of classes typically follows the principle of [[equivalence partitioning]] for abstract test cases and [[boundary-value analysis]] for concrete test cases.<ref name=pierre
Together, all classifications form the classification tree.
For semantic purpose, classifications can be grouped into ''compositions''.
Line 32:
The minimum number of test cases is the number of classes in the classification with the most containing classes.
In the
The selection of test cases originally<ref name=GG
=== Example ===
Line 59:
=== Background ===
The CTM introduced the following advantages<ref name=advStest /> over the Category Partition Method<ref>{{cite journal|last=Ostrand|first=T. J.|author2=Balcer, M. J.|title=The category-partition method for specifying and generating functional tests|journal=Communications of the ACM|year=1988|volume=31|issue=6|pages=676–686|doi=10.1145/62959.62964|s2cid=207647895|doi-access=free}}</ref> (CPM) by
* '''Notation''': CPM only had a textual notation, whereas CTM uses a graphical, tree-shaped representation.
* '''Refinements''' Selecting one representative might have an influence on the occurrence of other representatives.
Line 65:
: CTM allows modeling of hierarchical refinements in the classification tree, also called ''implicit dependencies''.
* '''Tool support''': The tool presented by Ostrand and Balcer only supported test case generation, but not the partitioning itself.
: Grochtmann and Wegener presented their tool, the [[Classification Tree Method#Classification Tree Editor|Classification Tree Editor]] (CTE) which supports both partitioning as well as test case generation.<ref name="GW">{{cite journal|last=Grochtmann|first=Matthias|author2=Wegener, Joachim|title=Test Case Design Using Classification Trees and the Classification-Tree Editor CTE|journal=Proceedings of the 8th International Software Quality Week(QW '95), San Francisco, USA|year=1995|url=http://www.systematic-testing.com/documents/qualityweek1995_1.pdf|access-date=2013-08-12|archive-url=https://web.archive.org/web/20150924112623/http://www.systematic-testing.com/documents/qualityweek1995_1.pdf|archive-date=2015-09-24|url-status=dead}}</ref>
[[File:Classification Tree for Embedded System.png|thumb|Classification Tree for Embedded System Example containing concrete values, concrete timing, (different) transitions and distinguish between States and Actions]]
=== Classification Tree Method for Embedded Systems ===
The classification tree method first was intended for the design and specification of abstract test cases. With the classification tree method for embedded systems,<ref name="conrad">{{cite journal|last=Conrad|first=Mirko|author2=Krupp, Alexander|title=An Extension of the Classification-Tree Method for Embedded Systems for the Description of Events|journal=Electronic Notes in Theoretical Computer Science|date=1 October 2006|volume=164|issue=4|pages=3–11|doi=10.1016/j.entcs.2006.09.002|doi-access=free}}</ref> test implementation can also be performed. Several additional features are integrated with the method:
# In addition to atomic test cases, test sequences containing several test steps can be specified.
# A concrete timing (e.g. in Seconds, Minutes ...) can be specified for each test step.
Line 79:
=== Dependency Rules and Automated Test Case Generation ===
One way of modelling constraints is using the refinement mechanism in the classification tree method. This, however, does not allow for modelling [[Constraint (information theory)|constraint]]s between classes of different classifications. Lehmann and Wegener introduced Dependency Rules based on [[Boolean expression]]s with their incarnation of the CTE.<ref name="LW">{{cite journal|last=Lehmann|first=Eckard|author2=Wegener, Joachim|title=Test Case Design by Means of the CTE XL|journal=Proceedings of the 8th European International Conference on Software Testing, Analysis & Review (EuroSTAR 2000)|year=2000|url=http://www.systematic-testing.com/documents/eurostar2000.pdf|access-date=2013-08-12|archive-url=https://web.archive.org/web/20160304191237/http://www.systematic-testing.com/documents/eurostar2000.pdf|archive-date=2016-03-04|url-status=dead}}</ref> Further features include the automated generation of [[test suite]]s using combinatorial test design (e.g. [[all-pairs testing]]).
=== Prioritized Test Case Generation ===
Recent enhancements to the classification tree method include the [[Risk-based testing|prioritized test case generation]]: It is possible to assign weights to the elements of the classification tree in terms of occurrence and error [[probability]] or risk. These weights are then used during test case generation to prioritize test cases.<ref name="luniak">{{cite journal|last=Kruse|first=Peter M.|author2=Luniak, Magdalena|title=Automated Test Case Generation Using Classification Trees|journal=Software Quality Professional|date=December 2010|volume=13|issue=1|pages=4–12}}</ref><ref>Franke M, Gerke D, Hans C. und andere. Method-Driven Test Case Generation for Functional System Verification. Proceedings ATOS. Delft. 2012. P.36-44.</ref> [[Statistics|Statistical testing]] is also available (e.g. for [[wear]] and [[Fatigue (material)|fatigue]] tests) by interpreting the element weights as a [[discrete probability distribution]].
=== Test Sequence Generation ===
With the addition of valid transitions between individual classes of a classification, classifications can be interpreted as a [[Finite-state machine|state machine]], and therefore the whole classification tree as a [[Statechart#Harel statechart|Statechart]]. This defines an allowed order of class usages in test steps and allows to automatically create test sequences.<ref name="seqgen">{{cite
=== Numerical Constraints ===
In addition to Boolean dependency rules referring to classes of the classification tree, Numerical Constraints allow to specify [[formula]]s with classifications as variables, which will evaluate to the selected class in a test case.<ref name="numdeb">{{cite
== Classification Tree Editor ==
The '''Classification Tree Editor''' (CTE) is a software tool for test design that implements the classification tree method.<ref name=vehicleElectronics>{{cite book|last=International|first=SAE|title=Vehicle electronics to digital mobility : the next generation of convergence
Over the time, several editions of the CTE tool have appeared, written in several (by that time popular) [[programming languages]] and developed by several companies.
Line 107:
In 1997 a major re-implementation was performed, leading to CTE 2. Development again was at Daimler-Benz Industrial Research. It was written in [[C (programming language)|C]] and available for [[win32]] systems.
The CTE 2 was
The classification tree editor for embedded systems<ref name="conrad" /><ref name=behave /> also based upon this edition.
[[Razorcat]] has been developing the CTE since 2001 and has CTE registered a brand name in 2003.
The last version CTE 3.2 was published with the tool [[TESSY]] 4.0 in 2016. Note the Versions table below.
=== CTE 4 ===
The CTE 4 was implemented in [[TESSY]] 4.1.7 as an [[Eclipse (software)|Eclipse]] plug-in in 2018. The latest CTE 4 version is still being developed as part of [[TESSY]] 4.3 in 2021.
=== CTE XL ===
In 2000, Lehmann and Wegener introduced Dependency Rules with their incarnation of the CTE, the CTE XL (eXtended Logics).<ref name=LW /><ref name=vehicleElectronics /><ref name=model2 /><ref name=rob>{{cite book|last=Olejniczak|first=Robert|title=Systematisierung des funktionalen Tests eingebetteter Software|year=2008|publisher=Technical University Munich|___location=Doctoral dissertation|pages=61–63|url=http://d-nb.info/988858436/34.pdf|accessdate=10 October 2013|archive-url=https://web.archive.org/web/20160306062722/http://d-nb.info/988858436/34.pdf|archive-date=6 March 2016|url-status=dead}}</ref> Further features include the automated generation of [[test suite]]s using combinatorial test design (e.g. [[all-pairs testing]]).<ref>{{cite
Development was performed by [[DaimlerChrysler]]. CTE XL was written in [[Java (programming language)|Java]] and was supported on win32 systems.
Line 121 ⟶ 128:
=== CTE XL Professional ===
Starting in 2010, CTE XL Professional was developed by Berner&Mattner.<ref name="luniak" /> A complete re-
New developments included:
* Prioritized test case generation: It is possible to assign weights to the elements of the classification tree in terms of occurrence and error [[probability]] or risk. These weights are then used during test case generation to prioritize test cases.<ref name="luniak" /><ref>Franke, M.; Gerke, D.; Hans, C; and others: Method-Driven Test Case Genera-tion for Functional System Verification, Air Transport and Operations Sym-posium 2012; p.354-365. Proceedings ATOS. Delft 2012.</ref> [[Risk-based testing|Risk-based]] and statistical testing is also available.
* Test Sequence Generation<ref name="seqgen" /> using [[Multi-Agent System]]s
* Numerical Constraints<ref name="numdeb" />
Line 142 ⟶ 149:
| CTE 1.0 ||1993|| Original Version,<ref name=GW /><ref name=justyna /> limited to 1000 test cases (fix!)|| [[Pascal (programming language)|Pascal]] || [[Unix]]
|-
| CTE 2.0 ||1998|| Windows Version,<ref name=behave /> unlimited number of test cases|| [[C (programming language)|C]]++ ||
|-
|CTE 2.1
|2003
|Embedded system version of [[Razorcat]]s part of the [[TESSY]] tool.
|C++
|win32
|-
| CTE XL 1.0 ||2000|| Dependency Rules, Test Case Generation<ref name=LW /><ref name=vehicleElectronics /><ref name=model2 /> || [[Java (programming language)|Java]] || win32
Line 152 ⟶ 165:
| CTE XL 1.9 ||2009|| Last Java-only Version|| Java || win32
|-
| CTE XL Professional 2.1 || 2011-02-21 || First [[Eclipse (software)|Eclipse]]-based Version, Prioritized Test Case Generation,<ref name=luniak /> [[Deterministic]] Test Case Generation, [[Requirements traceability|Requirements-Tracing]] with [[IBM Rational DOORS|DOORS]]|| [[Java version history#Java SE 6|Java 6]], [[Eclipse (software)|Eclipse]] 3.5 || win32
|-
| CTE XL Professional 2.3 || 2011-08-02 || [[Quality Center|QualityCenter]] integration, Requirements Coverage Analysis and [[Traceability matrix|Traceability Matrix]], [[API]]|| Java 6, Eclipse 3.6 || win32
|-
| CTE XL Professional 2.5 || 2011-11-11 || Test
|-
| CTE XL Professional 2.7 || 2012-01-30 || Bug fix release|| Java 6, Eclipse 3.6 || win32, win64
|-
| CTE XL Professional 2.9 || 2012-06-08 || Implicit Mark Mode, Default classes, [[
|-
| CTE XL Professional 3.1 || 2012-10-19 || Test Post-Evaluation (e.g. for [[Root cause analysis|Root Cause Analysis]]), Test Sequence Generation,<ref name=seqgen /> Numerical Constraints<ref name=numdeb />|| Java 6, Eclipse 3.7 || win32, win64
|-
| CTE XL Professional 3.3 || 2013-05-28 || Test Coverage Analysis, Variant Management (e.g. as part of [[Product family engineering|Product Family Engineering]]), Equivalence Class Testing|| Java 6, Eclipse 3.7 || win32, win64
|-
| CTE XL Professional 3.5 || 2013-12-18 ||
|-
| TESTONA 4.1 || 2014-09-22 || Bug fix release || Java 7, Eclipse 3.8 || win32, win64
|-
| TESTONA 4.3 || 2015-07-08 || Generation of Executable [[Test script|Test Scripts]] ([[Code generation (compiler)|Code Generation]]), Import of Test Results<ref>{{cite web|last1=Berner&Mattner|title=Press Release: Test Case Implementation with TESTONA 4.3|url=http://www.testona.net/en/media/press-releases/press-release-testona-4.3.html}}</ref> || Java 7, Eclipse 3.8 || win32, win64
|-
| TESTONA 4.5 || 2016-01-21 || Enhanced Export Facilities, [[GUI]] Improvements || Java 7, Eclipse 3.8 || win32, win64
|-
| TESTONA 5.1 || 2016-07-19 || Bug fix release, Switch to [[Java version history#Java SE 8|Java 8]], Eclipse 4.5 || [[Java version history#Java SE 8|Java 8]], Eclipse 4.5 || win32, win64
|-
|CTE 4.0
|2018-08-01
|New implementation of [[Razorcat]] as a plug-in for the [[TESSY]] 4.1 tool based on [[Eclipse (software)|Eclipse]]. Support in creating (model-based) test cases.
|Java
|win32
win64
|}
Line 180 ⟶ 206:
* When test design with the classification tree method is performed without proper test decomposition, classification trees can get large and cumbersome.
* New users tend to include too many (esp. irrelevant) test aspects resulting in too many test cases.
* There is no algorithm or strict guidance for selection of test relevant aspects.<ref>{{cite
== References ==
{{Reflist}}
== External links ==
* [http://www.systematic-testing.com/functional_testing/cte_main.php?cte=1 Systematic Testing]
[[Category:Software development process]]
| |||