Managed Testing Services

Managed Testing Services (MTS), operator test models and operator test solutions are among the managed services (operator models) and describe a comparatively new form of outsourcing software tests to service providers outside the company. However, these are not simply commissioned on a case-by-case basis, but rather become strategic, long-term partners, in which the company contractually assigns the provider the responsibility for very specific test tasks for a period of around three to five years. This enables, among other things, a more flexible, more transparent and better calculable billing of costs, which is less effort-oriented and more result-oriented. With the MTS business and distribution model , it is usually possible to achieve a significantly higher degree of automation during testing than is possible within the company. The potential savings that can be achieved here are greater, the more it is about recurring, standardized test efforts, such as those that arise from regular updates when operating large SAP systems . In addition, the testing effort in many companies is increasing due to the increasing use of standard software with ever shorter release cycles. If a separate test center is maintained for this, the resources there are usually only used in phases, whereas MTS gives hope for a more efficient use of resources.

Definition of terms

MTS are explicitly not to be equated with the (mostly primarily technical) outsourcing of software tests, but rather represent an alternative that makes the service provider much more responsible for the long term. Even at MTS, the provider's services are usually defined in service level agreements ; However, control over the IT strategy, the entire system architecture and governance remain largely with the customer, i.e. with the contracting company. There MTS falls into the areas of responsibility of IT management , quality management and product development .

With regard to the basic approach of providing , maintaining and operating software and services not in the company itself, but as a service based on Internet technologies, MTS at first glance resembles the paradigm of Software as a Service (SaaS). The usage-oriented billing of test services in the service model, which may also include the necessary license costs for tools, is an essential connection. In this context, the term Testing as a Service (TaaS) is becoming increasingly popular.

Compared to the providers of managed services, Application Service Providers (ASP) i. d. Usually no range of services is available, but rather specific applications (e.g. an ERP system). However, the arguments for ASP and MTS are similar, for example standardization, flexibility, scalability, cost reduction and, last but not least, the customer's focus on their core competencies. Due to the fact that MTS standardize and automate software tests as far as possible, there is often talk of increasing "industrialization" of IT and software development .

A fundamental difficulty in understanding arises from the ambiguity of the term "service". Service is used synonymously for any service regardless of the method of payment or duration of use. Managed Testing Services are aimed at the long-term provision of clearly defined test services at fixed prices. Both parties to the contract are bound by each other for a predetermined period of time (usually at least three years). The contractual relationship enables the service provider to amortize investments over the term of the contract. For short-term individual use without binding usage quotas, such models are economically risky.

Market development

Consulting firms and analysts (Frost & Sullivan, Gartner, Steria Mummert and others) are unanimously expecting significant growth in the area of ​​IT services in general and managed services in particular. Software testing is expected to increase at an above-average rate among managed services. The British consulting firm Ovum is forecasting a market volume of 56 billion US dollars worldwide by 2013, which would correspond to an average annual growth rate of 9.5 percent. The French consulting firm Pierre Audoin Consultants (PAC) is even predicting 100 billion euros for 2014. The area of ​​external test services will grow almost twice as fast as the number of software testers working directly in companies. According to PAC, as the market develops, clients will increasingly shift from purely effort-based pricing models towards higher-quality MTS contracts, which actually give service providers an incentive to continuously improve and standardize their customers' test processes. In addition to pure application tests , infrastructure tests are also becoming increasingly important due to the establishment of cloud computing . At the end of 2010, around 27 percent of small and medium-sized companies and 41 percent of large companies were using managed testing services, according to a market analysis by Steria Mummert Consulting. Accordingly, more than half of the implemented outsourcing measures are carried out in the test environment at the customer's site (onsite).

Process of an MTS project

In a first step, the need for action and savings potential are analyzed and often verified in a pilot project. On this basis, the provider submits an individual price model to the customer and thus also assumes part of the project risk, so to speak. The MTS provider only accepts test elements from its customer in exceptional cases; rather, he tries to rely as far as possible on his own, proven standards and tools. If you combine the automation of software tests with a relocation of activities to more cost-effective regions ( onshoring , e.g. in East Germany, nearshoring , e.g. in Romania or Egypt, offshoring , e.g. in India or Russia etc.) , according to the various providers, savings of between 30 and 50 percent can be achieved. If you consider not only the pure test costs in isolation, but also the entire life cycle of the software (for example, errors that occur due to insufficient tests in production or even during application), even higher savings are possible. In Managed Testing Services, billing is less in the form of the usual basic and fixed costs, but more result-oriented. It is therefore of particular importance that results and costs are made transparent for the customer through professional reporting. Compared to previous models of outsourcing software tests, MTS offers more options for customer-specific pricing. This means that billing can also be based on the number of lines of code or test cases, or a conceptual review en bloc for the earliest possible detection of software errors. Because MTS often go far beyond the purely technical outsourcing of processes, not only methodical knowledge is required of the provider or his employees, but also considerable industry and company knowledge.

criticism

A large number of very different services are now offered as "MTS", which makes it difficult for companies to make a quick decision. The definition of the term, the scope of the services and the billing models differ greatly. Critics therefore advise investing time in the first step in seeking a clear common (also linguistic) understanding between all internal and external parties involved. Conflicts of objectives can arise from the fact that the same service providers who have implemented or even developed the system in question are commissioned to test a system. There is a risk here that the service provider may test the functionality of its software, but not whether it also meets the overall business requirements. There are still MTS providers on the market whose employees are not certified according to international standards such as ISTQB (International Software Testing Qualifications Board). For companies with particularly security-critical products, such as banks and insurance companies, security issues such as experience with synthesizing data are of crucial importance. In principle, there are also companies for which MTS alone are out of the question, for example for a medium-sized company in the manufacturing industry that only carries out software tests twice a year.

providers

The market for MTS is still comparatively fragmented. Even market leaders like IBM have less than five percent global share. The consulting company PAC divides the providers of MTS into five groups:

• Big generalists like IBM, HP or Accenture, which, according to PAC, are often perceived as too expensive and too unspecific to the customer
• Offshoring specialists like Wipro, TCS or Cognizant, who focus on low wages in Asia
• Regional generalists such as the German T-Systems or providers such as Sopra and Steria with particular experience in vertical markets
• Test specialists such as the German SQS or providers such as NCC and Tescom, who are more active in Europe than in the USA, but now also offer all shoring models
• Small, very specialized niche providers

literature

• Elfriede Dustin, Thom Garrett, Bernie Gauf: Implementing Automated Software Testing. How to Save Time and Lower Costs While Raising Quality. Addison-Wesley Longman, Upper Saddle River NJ et al. 2009, ISBN 978-0-321-58051-1 .
• Simon Hülsbömer: German companies have IT tested by professionals . In: Computerwoche or CIO . December 10, 2010.
• Andreas Spillner, Tilo Linz: Basic knowledge of software testing. Training and further education to become a Certified Tester. Foundation level according to the ISTQB standard. 3rd revised and updated edition. dpunkt-Verlag, Heidelberg 2005, ISBN 3-89864-358-1 .

Individual evidence

1. ↑ cf. Frost & Sullivan: Global Managed Security Service Providers Rollup - PAID
2. ↑ cf. Linda R. Cohen / Bob Igou (Gartner): Steering Your Business Through the IT Services and Outsourcing Revolution (August 26, 2010)
3. ↑ cf. Jason Harris / Kathryn Hale (Gartner): IT Services Forecast Overview, 2009-2014. (July 30, 2010) - REGISTRATION REQUIRED
4. ↑ cf. Frances Karamouzis (Gartner): IT Services: One of the Highest Opportunities for Savings Given It's One of the Largest Sources of IT Spending. (February 11, 2009) - REGISTRATION REQUIRED
5. ↑ cf. William Maurer et al. (Gartner): Predicts 2011: Sourcing IT Services. (November 19, 2010) - REGISTRATION REQUIRED
6. ↑ cf. AFP News Agency (via Google News): Software testing market resilient despite crisis (March 10, 2009)
7. ↑ cf. Nick Mayes (Pierre Audoin Consultants, PAC): Worldwide Testing Services Market 2010-2014: Key Growth Opportunities & Sector Trends (Aug 2010) ( Memento of the original from December 21, 2010 in the Internet Archive ) Info: The archive link was automatically inserted and still Not checked. Please check the original and archive link according to the instructions and then remove this notice. 
8. ↑ cf. Lars Hinrichsen et al. (Steria Mummert Consulting): Success model outsourcing trend study 2010  ( page no longer available , search in web archives )  Info: The link was automatically marked as defective. Please check the link according to the instructions and then remove this notice. , Summary of the study  ( page no longer available , search in web archives )  Info: The link was automatically marked as defective. Please check the link according to the instructions and then remove this notice. (PDF; 1.9 MB)    
9. ↑ PDF at www.applabs.com ( Memento of the original dated December 21, 2010 in the Internet Archive ) Info: The archive link was inserted automatically and has not yet been checked. Please check the original and archive link according to the instructions and then remove this notice.