Exploratory testing

from Wikipedia, the free encyclopedia

Exploratory testing is an approach to the testing of software , which is described as the simultaneous learning, test design and test execution. Cem Kaner, who coined the term in 1993, defines exploratory testing as "a style of software testing that emphasizes the personal freedom and responsibility of the individual tester to continuously optimize the quality of his or her work through test-related learning, test design, test execution and interpretation of the Test results are used as mutually supportive activities that run in parallel throughout the project. "

While the software is being tested, the tester learns things that, together with experience and creativity, produce new tests. Exploratory testing is often mistakenly viewed as a black box testing technique . Correctly, it is a testing approach that can be applied to any testing technique at any stage of the development process. The key is neither the test technique nor the object to be tested or verified. The key is the tester's cognitive engagement and the tester's personal responsibility for the meaningful use of his time for testing.

history

Experienced testers conducted exploratory tests before. In the early 1990s, ad hoc was often synonymous with sloppy and careless work. As a result, a group of testing methodologists (now calling themselves the contextual school) began to use the term "exploratory" to highlight the predominant thought process in testing without script and to develop practice into a teachable discipline. This new terminology was first published by Cem Kaner in his book Testing Computer Software and in Lessons Learned in Software Testing . expanded. Exploratory testing can be learned just like any other intellectual activity.

description

Exploratory testing tries to find out how the software actually works and to ask questions about how it can handle difficult and simple cases. The quality of the test depends on the ability of the tester, test cases to invent and bugs to be found. The more the tester knows about the product and the various test methods, the better the test will be.

Exploratory tests can be compared with their antithesis, the scripted tests. With scripted tests, test cases are designed in advance. This includes both the individual steps and the expected results. These tests are later performed by a tester who compares the actual result with the expected. When conducting exploratory tests, expectations are open. Some results can be predicted and expected; others may not. The tester configures, operates, observes and evaluates the product and its behavior. He examines the result critically and reports information that is likely to be a bug (which could reduce the value of the product) or a problem (which jeopardizes the quality of testing).

In practice, testing is almost always a combination of exploratory and script-based testing, but there is a tendency towards both depending on the context.

According to Cem Kaner and James Marcus Bach, exploratory testing is more of a mindset, or "... a way of thinking about testing," than a methodology.

The documentation of exploratory tests ranges from the sole documentation of the errors found to the documentation of all tests carried out.

In pair testing, two people test together; the couple often consists of a professional tester and a software developer or a business analyst. As a side effect, in the first case the software developer is likely to gain insights into risks, while the tester gets an insight into the architecture. When a tester and business analyst work as a pair, the tester is likely to learn more about the business and expectations of the software.

Session-based testing is a method that was specially developed to make exploratory tests verifiable and measurable on a larger scale.

Exploratory testers often use tools, including screen recording or video tools to record or document the tests, or tools to quickly generate interesting situations for the test in question, e.g. B. James Bach's pearl clip.

Advantages and disadvantages

The main advantage of exploratory testing over traditional testing is that it requires less preparation, finds critical bugs quickly, and is more intellectually stimulating to conduct than scripted tests.

Another great advantage is that testers can react to the results of previous tests and change their current tests accordingly. Instead of taking a series of scripting tests, you can focus on or explore new requirements. This also speeds up error detection.

Another advantage is that most errors in a program or part of a program are discovered after a short time with exploratory testing. Programs that pass certain tests tend to continue to pass the same tests and are more likely to pass other tests or scenarios that have not yet been investigated. "

A disadvantage of exploratory testing is that the tests that are invented and carried out during operation cannot be checked in advance, thus preventing misunderstandings regarding the requirements or errors in test cases. It can also be difficult to show exactly which tests were performed. The traceability of the errors found may suffer as a result.

Exploratory testing is unlikely to run in exactly the same way if you re-run it. This can be beneficial when it is important to find new bugs, but also a disadvantage when it is more important to repeat certain details of previous tests. This can be controlled with specific techniques (e.g. recording the tests) by preparing automated tests.

Investigations

Replicated experiments have shown that scripted and exploratory tests lead to similar error detection effectiveness (the total number of errors found). Exploratory results lead to a higher efficiency (number of errors per unit of time), since no effort is expended for the pre-planning of the test cases.

Observations on exploratory testers have shown that the use of knowledge about the domain, the system under test, and customers is an important factor in the effectiveness of exploratory tests.

A case study at three companies found that the ability to provide quick feedback was viewed as a benefit of exploratory testing, while management of test coverage was viewed as a deficiency.

A survey found that exploratory tests are also used in critical areas and that exploratory tests place high demands on the person performing the tests.

use

Exploratory testing is particularly useful when requirements and specifications are incomplete or when there is not enough time to test. The approach can also be used to verify that previous tests have identified the main flaws.

Individual evidence

  1. ^ Cem Kaner: A Tutorial in Exploratory Testing. (PDF) April 2008, p. 2 , accessed on March 1, 2020 (English).
  2. ^ Cem Kaner: A Tutorial in Exploratory Testing. (PDF) April 2008, p. 36 , accessed on March 1, 2020 (English).
  3. ^ Cem Kaner: A Tutorial in Exploratory Testing. (PDF) April 2008, pp. 37–39 , accessed on March 1, 2020 (English).
  4. ^ Cem Kaner: Testing Computer Software . 2nd Edition. John Wiley & Sons , 1999, ISBN 978-0-471-35846-6 , pp. 6, 7-11 (English, 496 pages).
  5. ^ Cem Kaner, James Bach, Bret Pettichord: Lessons Learned in Software Testing . A context-driven approach. 1st edition. John Wiley & Sons , 2004, ISBN 978-0-471-08112-8 (English, 320 pages).
  6. ^ Cem Kaner, James Bach, Exploratory & Risk Based Testing , www.testingeducation.org , 2004, p. 10 Template: Citation broken
  7. ↑ Hendrickson, Elisabeth .: Explore It! : How software developers and testers use exploratory testing to reduce risks and uncover errors . 1. Aufl. Dpunkt-Verl, Heidelberg 2014, ISBN 3-86490-093-X , 13.2 Researching in pairs: “One possibility to involve all team members in research is to form pairs for research. Couples made up of a professional tester and someone else are especially effective. When a tester and a business analyst work as a pair, the tester is likely to learn more about the business and expectations of the software ... When a tester and programmer team up for research as a couple, the programmer is likely to gain insight into risk, while the tester gets an insight into the architecture. "
  8. ↑ Juha Itkonen, Mika Mäntylä: Are test cases needed? Replicated comparison between exploratory and test-case-based software testing . In: Empirical Software Engineering . tape 19 , no. 2 , July 11, 2013, ISSN  1382-3256 , p. 303–342 , doi : 10.1007 / s10664-013-9266-8 (English).
  9. ↑ Juha Itkonen, Mika Mäntylä, C. Lassenius: The Role of the Tester's Knowledge in Exploratory Software Testing . In: IEEE (Ed.): IEEE Transactions on Software Engineering . tape 39 , no. 5 , September 11, 2012, ISSN  0098-5589 , p. 707-724 , doi : 10.1109 / TSE.2012.55 (English, ieee.org [accessed March 1, 2020]).
  10. ↑ Juha Itkonen, K. Rautiainen: Exploratory testing: a multiple case study . In: 2005 International Symposium on Empirical Software Engineering . 2005, ISBN 978-0-7803-9507-7 , pp. 84–93 , doi : 10.1109 / ISESE.2005.1541817 (English, researchgate.net [PDF; accessed March 1, 2020]).
  11. ^ Dietmar Pfahl, Huishi Yin, Mika V. Mäntylä, Jürgen Münch: How is Exploratory Testing Used? A state-of-the-practice survey . In: ACM (Ed.): Proceedings of the 8th ACM / IEEE International Symposium on Empirical Software Engineering and Measurement . ESEM '14. New York, NY, USA 2014, ISBN 978-1-4503-2774-9 , pp. 5: 1–5: 10 , doi : 10.1145 / 2652524.2652531 (English, researchgate.net [PDF; accessed March 1, 2020]).
  12. ^ Cem Kaner: A Tutorial in Exploratory Testing. (PDF) April 2008, pp. 37,118 , accessed on March 1, 2020 (English).

Web links