Agile testing

Under agile testing means testing of software as part of an agile development project. Testing in agile development projects primarily requires a focus on supporting the development team. Many characteristics of agile testers also help a traditional tester, but the associated problems are not so clear and therefore not solved by every project team. Agile testing follows the principles of the agile manifesto and applies the agile principles to testing.

overview

Agile testing must be designed in such a way that it optimally supports the goals of agile software development , namely

high customer satisfaction, which results from the close cooperation between the customer or customer representative (in Scrum, for example, the product owner ) and the development team,
high product quality, which results from a consistent focus of development activities on error prevention and early error detection,
high development speed, which is brought about by reducing blind work (unnecessary documentation, unnecessary error correction work, etc.),
quick response to changes made possible by short iterations,
high team motivation, which is supported by self-organization and a permanent work pace .

Under certain circumstances, values from testing in the “classic”, mostly phase-oriented or iterative-incremental procedure must be taken into account, e. B. in a regulated environment:

good predictability, traceability and verifiability of the test activities
high systematics of the test design, resulting in a high and reproducible test coverage

Basic principles

In order to meet the requirements for speed and avoidance of unnecessary work on the one hand and systematics and trustworthiness on the other, an agile test procedure should be based on the following basic principles:

Fast feedback

Newly developed or changed program code must be tested before delivery. In iterative methods such as Scrum, this means that an agile tester has to provide the team with information about the functionality that has been created and changed, usually within two weeks, through quick feedback. In order for this quick feedback cycle to work, functions must be available in a first testable version after just a few days. The tester is also involved in defining the function before the sprint.

In order to be able to provide the fastest possible feedback on the current status of the team, many teams rely on a balance between a high level of automation and lightweight manual exploratory tests . Programmers support the work of testers in automation, while testers with their critical skills provide the starting point for test automation. In addition, agile teams are aware that not everything can be automated. Corresponding risks such as poorly designed user interfaces and hidden operating elements are therefore addressed using exploratory tests.

High degree of automation

In order to support quick responsiveness to changing requirements and the permanent refactoring of program code, as many systematic test cases as possible must be designed and automated . This includes both structure-based tests ( unit tests ) and subject- specific system and acceptance tests .

Low overhead

The effort for test activities that are not directly operational (e.g. test management , error management , documentation work) must be kept as low as possible.

Dissolution of test roles

Following the agile principles, the responsibility for all test activities is shared among the entire team. As a result, the boundaries between the classic test roles ( test manager , test analyst , tester ) and sometimes the boundaries between software developer and tester disappear . Due to the high qualification required for both development and test work, however, there is a narrow limit to the latter.

Resolution of test levels

There is no time for a sequential sequence of test levels (e.g. unit , integration and system test ) within the typical agile iteration length of 2–3 weeks. However, the different test objectives covered by the different test levels can still be achieved and must be covered by test activities with different content in "micro test cycles" (typically a sequence of unit, integration and system tests in less than 24 hours). In an agile context, the test levels should be understood as content-related test types rather than temporal test phases.

Close teamwork

The desire for quick feedback and the assumption of responsibility for testing by the entire team require close cooperation between the "testers" (ie the team members who are mostly involved in testing) and the "developers" as well as the "testers" and the customer representatives ( the Product Owner in Scrum). This cooperation is achieved through direct communication (and thus largely dispensing with documentation that has no value for the customer and the team) as well as cooperation in pairs (so-called pairing ).

Scrum framework

By “anchoring” systematic testing in some Scrum artifacts as well as in some key points of the process, the goals of agile testing can be realized.

"Definition of READY" (DoR)

The DoR serves to ensure testability at an early stage. It is a checklist that is used when the user stories are created by the product owner as well as in their quality assurance and at the latest when the stories are transferred from the product to the sprint backlog . Stories that are not "ready" in the sense of the definition may be rejected by the team at the sprint planning meeting. The DoR ensures the professional and technical testability of the stories and calls for the definition of operational acceptance criteria as possible . Good acceptance criteria result from the "specification by example" approach. A good starting point for a DoR are the so-called INVEST criteria for requirements. When creating the stories, it makes sense to pair tester and product owner in order to combine methodical knowledge of testing and quality assurance with the specialist knowledge of the product owner.

"Definition of DONE" (DoD)

The DoD anchors the quality goals. It is another checklist and describes which goals must be achieved by the team in the implementation of the story before it is considered “ready for submission in the sprint review”. In the DoD, test objectives such as B. the necessary test types, the test coverage to be achieved and the test end criteria (generally the removal of all errors found) are specified. The DoD thus serves directly to ensure product quality and customer satisfaction.

"Definition of TEST" (DoT)

The DoT anchor the test strategy in an agile team. They formulate the “rules of the game” that apply to the team in a central document, the so-called “team charter”. This charter is also an ideal place for anchoring common approaches, e.g. B. for test case design and tool usage for the testers.

A DoT describes the information on the basis of which information (e.g. risk and value classification of a story and its description) and how test cases are to be designed, implemented and carried out. It thus replaces the artifact of the risk or value-based test strategy known from the "classic" test. The “agile testing quadrants” are a good starting point for defining an agile test strategy.

Tests accompanying sprint

The principle of implementation and testing non-stop applies . During the sprint, it must be ensured that the "testers" are integrated into the team from day one and work in parallel with (or together with) the "developers" on the realization of the stories. Key skills for this are:

The participation of the testers in the sprint planning and the daily scrum meetings
The timing of development and test activities. Principles such as “Test Driven Development” or “Test First” and pairing between developers and testers are suitable for this
The transparent representation of tests in the planning and status tracking through suitable representation on the task board, the central planning tool of the agile team
The implementation of systematic unit tests, ie those that result from the application of systematic test methods such as equivalence class analysis, state analysis or decision tables
The combination of systematic tests and exploratory tests. The latter provide feedback particularly quickly and find errors that systematic tests may not be able to find
The operation of a CI (continuous integration, continuous integration are selected) environment in the automated unit, integration and system testing upon introduction of changes (check-in) for an appropriate risk-based strategy, and carried out as a regression tests

Mature test automation

Not only unit and integration tests, but also system and acceptance tests must be designed and automated early in the sprint (if possible at the same time as the implementation of a story or even before that in the sense of “test first”). During the sprint, they must be permanently operational in order to serve as a "safety net" for changes and refactorings . At the level of black box tests, various frameworks prove to be particularly useful for setting up such test automation that can be used at an early stage and is robust against changes and incorrect behavior of the software, for example

" Behavior-driven " frameworks such as Cucumber (Ruby, Java etc.), SpecFlow (.NET) or Jasmine ( ECMAScript )
" Keyword-driven testing " frameworks
" Model-driven testing " frameworks

For example, the business requirement can be written in Gherkin in the form of user stories and automatically tested using a BDD framework , while the incorrect behavior of integration points and the performance criteria are checked using a test harness . The testers' task here is to identify additional test cases and to deliberately induce the application to behave incorrectly, in particular through incorrect entries.

Individual evidence

^ Gojko Adzic: Specification by Example. How Successful Teams Deliver the Right Software. Manning Publications, Shelter Island NY 2011, ISBN 978-1-61729-008-4 .
^ Mike Cohn: User Stories Applied. For agile software development. Addison-Wesley, Boston MA et al. 2004, ISBN 0-321-20568-5 .
^ Lisa Crispin, Janet Gregory: Agile Testing. A Practical Guide for Testers and Agile Teams. Addison-Wesley, Upper Saddle River NJ et al. 2009, ISBN 978-0-321-53446-0 .
↑ Andreas Spillner, Tilo Linz: Basic knowledge software test. Training and further education to become a Certified Tester. Foundation level according to the ISTQB standard. 4th, revised and updated edition. dpunkt.verlag, Heidelberg 2010, ISBN 978-3-89864-642-0 .
↑ FitNesse acceptance testing framework
^ Robot Framework
↑ imbus TestBench
^ Helmut Götz, Markus Nickolaus, Thomas Roßner, Knut Salomon: Model-based testing. Modeling and generation of tests. Basics, criteria for tool use, tools in an overview (= IX Study. 01, March 2009). Heise Zeitschriften Verlag, Hanover 2009.

[1] Gojko Adzic: Specification by Example. How Successful Teams Deliver the Right Software. Manning Publications, Shelter Island NY 2011, ISBN 978-1-61729-008-4 .

[2] Mike Cohn: User Stories Applied. For agile software development. Addison-Wesley, Boston MA et al. 2004, ISBN 0-321-20568-5 .

[3] Lisa Crispin, Janet Gregory: Agile Testing. A Practical Guide for Testers and Agile Teams. Addison-Wesley, Upper Saddle River NJ et al. 2009, ISBN 978-0-321-53446-0 .

[4] Andreas Spillner, Tilo Linz: Basic knowledge software test. Training and further education to become a Certified Tester. Foundation level according to the ISTQB standard. 4th, revised and updated edition. dpunkt.verlag, Heidelberg 2010, ISBN 978-3-89864-642-0 .

[FITnesse-5] FitNesse acceptance testing framework

[Robot-6] Robot Framework

[iTB-7] us TestBench

[8] Helmut Götz, Markus Nickolaus, Thomas Roßner, Knut Salomon: Model-based testing. Modeling and generation of tests. Basics, criteria for tool use, tools in an overview (= IX Study. 01, March 2009). Heise Zeitschriften Verlag, Hanover 2009.