Classification tree method

from Wikipedia, the free encyclopedia

The classification tree method (English classification tree method is) non in the area of embedded software method for the determination of functional black-box tests. It was originally developed by Grimm and Grochtmann in 1993. These are not classification trees in the sense of decision trees .

The classification tree method essentially consists of two steps:

  1. Determination of the test-relevant aspects (so-called classifications ) and possible characteristics (called classes ), such as
  2. Combination of different classes for all classifications to test cases .

The starting point is the functional specification (e.g. requirements , application , ...) of the test object . The second step of the method follows the principles of combinatorial test design.

The result is a set of test case specifications that should contain test case specifications that are as error-sensitive as possible, but not redundant. The methodical approach is intended to ensure that the resulting set of test case specifications contains all relevant test cases.

Although the pen and paper method can be used, the Classification Tree Editor is usually used for this.

The application of the classification tree method

A test object must be specified in order to use the method . The classification tree method is a method for the black box test and supports very different types of test items , such as hardware systems, integrated hardware-software systems, pure software systems , in particular also embedded systems , user interfaces , Operating systems , parsers , as well as subsystems of these.

After selecting the test item, the first step is to determine the test-relevant aspects. Any system can be described as a collection of classifications for all input and output parameters . (Whereby input parameters can also contain environment variables , preconditions and other, rather untypical parameters.) Each classification consists of any number of disjoint classes which describe possible characteristics of the parameters. The choice of classes typically follows the principle of equivalence class formation for abstract test cases and limit value analysis for specific test cases. Together, all classifications and classes form the classification tree. In addition, classifications can also be semantically grouped into compositions.

This division of the problem into different aspects, which can be refined separately below, reduces the complexity of the original test problem.

An equivalence class can be further classified according to additional criteria. The classification tree ultimately results from the repeated formation of sub- classifications with the associated classes. Its root is the problem; it grows from top to bottom; Classifications are framed by rectangles; Classes have no framework.

In the second essential step of the method, test cases are then created by choosing exactly one class for each classification in the tree. The selection of the test cases was originally a manual activity of the tester.

The leaves of the classification tree (i.e. classes without further classification) form the head of the combination table. A line in the combination table specifies a test case by marking the leaf classes of the tree from which a value is used for this test case .

example

Classification tree for a database system

A test design is to be carried out for a database system . When using the classification tree method, the determination of the test-relevant aspects results in the classifications in the first step: user role , database query and type of access . Two classes have been identified for user role n: normal user and administrator . There are three types of database query : add , edit, and delete . Three classes were found for the type of access : native tool , web browser , API . The web browser class is further refined with the brand test aspect ; there are three possible classes here: Internet Explorer , Mozilla Firefox , and Apple Safari .

The first step of the method is now complete. Of course there are other possible test aspects, such as the speed of the network connection or the number of existing database entries. Thanks to the graphical representation of the test aspects in the tree, the selected aspects and the intended values ​​can be viewed quickly.

In the second step, three test cases were selected manually:

  1. A normal user inserts a new record using a native tool.
  2. An administrator edits an existing data record with the Firefox browser.
  3. A normal user deletes a record through the API.

A little more statistics: There are 30 possible test cases in total (2 user roles * 3 database queries * 5 types of access). 5 test cases are sufficient for minimum coverage, since there are 5 types of access (and the type of access is the classification with the largest number of classes).

Extensions

origin

The classification tree method was developed in the Daimler-Benz software research laboratory in Berlin and is a further development of the Category Partition Method (CPM) by Ostrand and Balcer. It offers the following advantages over CPM:

  • Notation : With the CPM there was only a textual description in contrast to the graphical tree representation.
  • Refinements allow the specification of conditional test aspects.
CPM only offered restrictions. The classification tree method offers hierarchical refinements directly in the tree for implicit dependencies .
  • Tool support : The Ostrand and Balcer tool only supported test case generation without determining the test aspects.
The classification tree editor Testona (formerly Classification Tree Editor (CTE) ) by Grochtmann and Wegener, on the other hand, supports both partitioning and test case generation.
Classification Tree for Embedded System Example containing concrete values, concrete timing, (different) transitions and distinguish between States and Actions

Classification tree method for embedded systems

The classification tree method was originally intended for the design and specification of abstract test cases. Tests can also be implemented using the classification tree method for embedded systems. For this purpose, some additional information has been provided:

  1. In addition to atomic test cases, test sequences can be defined with several individual test steps.
  2. Concrete time behavior (e.g. in seconds, minutes ...) can be specified for each individual test step.
  3. Signal curves (e.g. linear , spline , sine ...) can be specified between selected classes of consecutive test steps.
  4. A distinction between action and state can be modeled. It is represented by various graphic markings in the test.

The module test tool TESSY also uses some of these extensions.

Concrete test data and time aspects

Approaches to generate test data from classification trees were also pursued.

The Time Partition Testing (TPT) method is an extension of the classification tree method to include temporal aspects that play a major role when testing control and regulation systems. E. Lehmann describes in his dissertation how classifications can be mapped to states and state transitions in the TPT machines, and how temporal aspects can also be mapped.

Dependency rules and automatic test case generation

The refinement mechanism in the classification tree method can be used for modeling dependencies. However, no dependencies between classes of different classifications can be modeled in this way. Lehmann and Wegener introduced dependency rules based on Boolean algebra for the CTE. They also enabled the automatic generation of test case sets based on combinatorial test design (e.g. the Pairwise method ).

Prioritizing test case generation

More recent expansions to the classification tree method concern the prioritizing test case generation: With them it is possible to assign weights in the form of probability of occurrence, probability of error and risk to elements of the classification tree. These weights can then be used in the test case generation to generate test cases in the order of their priority (the covered weights). Statistical testing (e.g. for testing material fatigue and wear ) is also possible in that the assigned weights are interpreted as a discrete probability distribution .

Test sequence generation

By adding valid transitions between the individual classes of a classification, these classifications can be interpreted as a state machine . The totality of all classifications in the tree becomes the statechart . This procedure defines a permitted sequence of class uses in test steps and thus also the automatic generation of valid test sequences. Different coverage rates are available, such as state coverage, transition coverage (path coverage), and coverage of state pairs and transition pairs.

Numerical dependencies

In addition to Boolean dependencies, which make statements about the common occurrence of classes in a test case, numerical dependencies can also be defined, which specify a calculation rule in which classifications are interpreted as variables and whose assignment results from the classes used in the test case.

Classification Tree Editor Testona

The classification tree editor Testona (formerly CTE) supports the design of the classification tree and the specification of the test cases. Testona has a graphics editor specially designed for drawing classification trees. For example, Testona takes into account the prescribed sequence of classification or composition and class and suggests the appropriate tree element when drawing. The test case specifications are created in the combination table and administered by Testona. They can be summarized in test sequences that are required to describe dynamic processes. For documentation purposes, Testona reports can be generated in Excel, Word or as a text file. The test cases created can be exported directly to the TESSY module test tool .

advantages

Visualization of the test ideas

A major advantage of the classification tree method is the visualization of the test ideas, which are easy to convey, for example in reviews . Reviews are also an effective means of ensuring that no test case specifications are missing.

Builds trust

With a carefully crafted tree and well-considered test case specifications in place, there is a good chance that you will find most, or perhaps almost all, of the bugs.

Reduces complexity

When you first think about the necessary test cases for a particular problem, you usually have numerous ideas and ideas. The challenge is rather to structure the ideas and identify redundant test cases. The classification tree method reduces the complexity involved in this task , as it allows individual aspects ( classifications ) to be viewed in isolation and targeted thoughts about the necessary (and unnecessary!) Equivalence classes.

Checks the specification

The first step in the classification tree method is to analyze the problem specification. This automatically forces the specification to be checked for omissions, contradictions and ambiguities.

Estimation of the test effort

Certain dimensions can already be determined from the classification tree, e.g. B. the minimum number of test cases for a given tree. These and other dimensions are determined by the Testona tool.

Cons and criticism

Creating good classification trees is made more difficult by mixing different aspects. While with the equivalence class method and the limit value analysis each variable is initially considered individually for the breakdown into relevant areas and only in a second step dependencies between variables are considered, both steps must be carried out together in the sense of the classification tree method. This is necessary because the tree structure is significantly influenced by dependencies between sizes.

Classification trees are clear with a limited number of classifications, equivalence classes and test cases. It is difficult to keep track of large classification trees.

See also

Individual evidence

  1. ^ A b c Anne Mette Jonassen Hass: Guide to advanced software testing . Artech House, Boston 2008, ISBN 1-59693-286-4 , pp. 179-186.
  2. ^ Graham Bath, Judy McKay: The software test engineer's handbook: a study guide for the ISTQB test analyst and technical test analyst advanced level certificates , 1st. Edition, Rocky Nook, Santa Barbara, CA 2008, ISBN 978-1-933952-24-6 .
  3. ^ A b Matthias Grochtmann, Klaus Grimm: Classification Trees for Partition Testing . In: Software Testing, Verification & Reliability . 3, No. 2, 1993, pp. 63-82. doi : 10.1002 / stvr . 4370030203 .
  4. ^ A b c D. Richard Kuhn, Raghu N. Kacker, Yu Lei: Introduction to combinatorial testing . Crc Pr Inc, 2013, ISBN 1-4665-5229-8 , pp. 76-81.
  5. ↑ a b Pierre Henry: The testing network an integral approach to test activities in large software projects . Springer, Berlin 2008, ISBN 978-3-540-78504-0 , p. 87.
  6. ↑ a b Matthias Grochtmann, Wegener, Joachim: Test Case Design Using Classification Trees and the Classification Tree Editor CTE Archived from the original on September 24, 2015. Info: The archive link was inserted automatically and has not yet been checked. Please check the original and archive link according to the instructions and then remove this notice. In: Proceedings of the 8th International Software Quality Week (QW '95), San Francisco, USA . 1995. Retrieved October 14, 2013. @1@ 2Template: Webachiv / IABot / www.systematic-testing.com
  7. ^ TJ Ostrand, MJ Balcer: The category-partition method for specifying and generating functional tests . In: Communications of the ACM . 31, No. 6, 1988, pp. 676-686. doi : 10.1145 / 62959.62964 .
  8. ^ Mirko Conrad, Alexander Krupp: An Extension of the Classification Tree Method for Embedded Systems for the Description of Events . In: Electronic Notes in Theoretical Computer Science . 164, No. 4, October 1, 2006, pp. 3-11. doi : 10.1016 / j.entcs.2006.09.002 .
  9. ↑ Stefan Lützkendorf, Klaus Bothe: Attributed classification trees for test data determination . In: Software technology trends . 23, No. 1, 2003.
  10. ↑ Zhen Ru Dai, Peter H. Deussen, Maik Busch, Laurette Pianta Lacmene, Titus Ngwangwen, Jens Herrmann, Michael Schmidt: Automatic Test Data Generation for TTCN-3 using CTE Archived from the original on March 4, 2016. Info: The archive link was inserted automatically and not yet tested. Please check the original and archive link according to the instructions and then remove this notice. In: Proceedings of the 18th International Conference Software & Systems Engineering and their Applications (ICSSEA) . 2005. Retrieved October 14, 2013. @1@ 2Template: Webachiv / IABot / webmail.testingtech.de
  11. ↑ Eckard Lehmann: Time Partition Testing - Systematic Test of the Continuous Behavior of Embedded Systems 2003.
  12. ↑ Eckard Lehmann, Joachim Wegener: Test Case Design by Means of the CTE XL Archived from the original on March 4, 2016. Info: The archive link was inserted automatically and has not yet been checked. Please check the original and archive link according to the instructions and then remove this notice. In: Proceedings of the 8th European International Conference on Software Testing, Analysis & Review (EuroSTAR 2000) . 2000. Retrieved November 14, 2013. @1@ 2Template: Webachiv / IABot / www.systematic-testing.com
  13. ^ Peter M. Kruse, Magdalena Luniak: Automated Test Case Generation Using Classification Trees . In: Software Quality Professional . 13, No. 1, December 2010, pp. 4–12.
  14. ↑ Peter M. Kruse, Joachim Wegener: Test Sequence Generation from Classification Trees . In: Software Testing, Verification and Validation (ICST), 2012 IEEE Fifth International Conference on . April 2012, pp. 539-548. ISBN 978-0-7695-4670-4 . doi : 10.1109 / ICST.2012.139 .
  15. ^ Peter M. Kruse, Jürgen Bauer, Joachim Wegener: Numerical Constraints for Combinatorial Interaction Testing . In: Software Testing, Verification and Validation (ICST), 2012 IEEE Fifth International Conference on . April 2012, pp. 758-763. ISBN 978-0-7695-4670-4 . doi : 10.1109 / ICST.2012.170 .