Delphi method

from Wikipedia, the free encyclopedia

The Delphi method (also known as the Delphi study, Delphi procedure or Delphi survey ) is a systematic, multi-stage survey procedure with feedback and is an estimation method that is used to assess future events, trends , technical developments and the like as well as possible .

The method is named after the ancient oracle of Delphi , which gave its listeners advice for the future.

history

The Delphi method was developed - after preliminary work at the end of the 1950s - by the American RAND Corporation in 1963 and has been used since then, albeit in a varied form, for the determination of forecasts / trends as well as for other opinion-forming within the framework of system tasks. The process has increasingly developed into an evaluation process for topics, in which it can be determined whether there is a consensus on the topic (or whether this can be reached) or not. In Germany in the 1990s, it was the then Federal Ministry for Research and Technology (BMFT) that commissioned the first Delphi studies on the development of science and technology. The studies German Delphi report on the development of science and technology (1993) and Delphi '98 survey. Demand for the future. Studies on the global development of science and technology (1998) were carried out by the Fraunhofer Institute for Systems and Innovation Research (ISI).

method

In a Delphi survey, a group of experts is presented with a catalog of questions or theses in the relevant subject. The respondents have the opportunity to assess the theses in two or more so-called rounds . From the second round onwards, feedback is given on how other experts have responded, usually anonymously. In this way an attempt is made to counteract the usual group dynamics with very dominant people.

The answers, estimates, results, etc. received in writing in the first round are therefore listed and, for example, summarized with the help of a special averaging , percentiles or average value calculations and anonymously presented to the experts again for further discussion, clarification and refinement of the estimates. This controlled process of opinion formation usually takes place over several stages. The end result is a prepared group opinion that contains the statements themselves and information about the range of existing opinions.

The opinion-forming process contains the elements: generation, correction or partial adaptation or refinement, averaging or limit value formation , often also open fields for explanations. Disturbing influences are eliminated through anonymization , the requirement to use the written form and individualization. The strategy of the Delphi method consists of: Concentration on the essentials, a multi-stage, partially fed back editing process and more reliable, more comprehensive statements by permitting statistical fuzzy-like results. One problem can be that the experts do not change their opinion once expressed in the following rounds despite anonymity. The additional benefit of further rounds would be limited in this case.

In addition to the Delphi method, e.g. B. the cross-impact matrix method can be used. Elements of the Delphi method can also be found in the D2 method. Combinations with scenarios are now also carried out more frequently. Simple roadmaps can be derived from Delphi results , so that this combination is also enjoying increasing popularity.

There are various forms of the Delphi method which vary the method of estimation somewhat: the standard and broadband methods are some of the variants. Most of the procedures are now carried out electronically. Realtime Delphi processes (with immediate feedback of the results) are a variant that is only possible electronically.

Standard Delphi Method

In the standard Delphi method, several experts are used to estimate a project - or to make a forecast - who are not allowed to coordinate with one another. The process looks like this:

  • A project manager prepares a project description in which the individual sub-products are listed and prepares them in a work form.
  • The project manager presents the goals of the overall project and distributes a copy of the work form to each expert. There is no discussion of the estimates.
  • Every expert values ​​the work packages contained in the work form. None of the experts work with another expert.
  • All work forms are collected and evaluated by the project manager.
  • If there are serious discrepancies, the project manager will uniformly comment on these on all work forms with regard to the deviation upwards or downwards. Each work form then goes back to its original editor.
  • The experts reconsider their estimates depending on the comments. This process can be seen as a kind of indirect discussion, as the experts are not in contact with one another and yet influence one another.
  • The loop described is repeated until a consensus is reached in the estimates independently of one another (within a tolerance range).
  • The mean values ​​of all estimates are calculated and presented as a final estimate.

The lack of any discussion has two aspects that a project manager must assess: On the one hand, it prevents currents and tendencies from developing in the opinions due to unwanted group dynamics that may prevent good estimates. On the other hand, group discussions could help avoid deficits in the know-how of individual experts and the associated misjudgments.

Delphi surveys are often conducted in writing and separately, so the questionnaires are sent to the experts by letter or email. The individual experts never see each other and only know the names of the other respondents after all the survey rounds have been completed. This approach is more reliable than bringing all experts together in one room. Once the final report is available, all experts and other interested parties are usually invited to a symposium.

Broadband Delphi Method

With the broadband Delphi method, several experts are used to estimate a project, who are allowed to coordinate with one another. The process looks like this:

  • A project manager prepares a project description in which the individual sub-products are listed and prepares them in a work form.
  • The project manager presents the goals of the overall project and distributes a copy of the work form to each expert. There is a discussion of the work packages among the experts, in which the view of the individual experts is conveyed to the other participants in relation to the overall project and the subtasks.
  • Each expert then estimates the work packages contained in their work form. None of the experts is working with another expert.
  • The project manager summarizes the individual estimates, but does not justify the information or differences. The results are distributed to all experts.
  • The project manager calls a new meeting with the experts and addresses the largest discrepancies in the estimates. Each work form then goes back to its original editor.
  • The experts reconsider their estimates depending on the specified deviations.
  • The loop described is repeated until a consensus is reached in the estimates independently of one another (within a tolerance range).
  • The mean values ​​of all estimates are calculated and presented as a final estimate.

The interactions between the experts convey different views, which accelerates the formation of consensus. The advantage of this method is, on the one hand, the anonymity of the estimates: the experts are not confronted with serious deviations in the estimates and can thus influence the estimation effort in their favor. Significant deviations from mean values ​​are disclosed. The disadvantage of this method is the risk of opinion-forming through the group dynamics, in which a serious deviation in estimates that may be necessary is subject to peer pressure. Another disadvantage is that, due to several iteration loops for forming opinions, the entire estimation effort can be quite extensive. The broadband Delphi method is a useful technique for estimating large projects in which complex architectures can lead to realistic values ​​through a large panel of experts with the help of the interaction between the experts.

criticism

The Delphi method tries to reduce the experts' misjudgments through the multi-level design, sometimes based on consensus. However, not all problems of the expert survey can be avoided; further restrictions arise from interviewing several people.

  • The main point of criticism relates to the basic assumption that experts have knowledge that goes beyond the normal and that the combination can generate future knowledge. However, this assumption cannot be proven.
  • There are no criteria for selecting experts ; the term “expert” is purely subjective and arbitrary; the selection of the participants already influences the result.
  • Topics and theses must first be formulated before they can go through the two-stage process. In some cases, the theses are developed in-house, but other methods are usually necessary.
  • The theses must be short, concise, but clearly formulated. This can be an advantage as it forces the participants to concentrate on the essentials. Methodologically, however, complex topics can only be assessed to a limited extent.
  • By definition, experts essentially concentrate on their specialist area. The interdependencies with other developments, which are particularly important in large-scale studies, are often neglected or have to be reworked.
  • If relevant framework conditions (e.g. social developments when forecasting the technical development of mobile communication) are taken into account, one cannot rely on the respondents having the same reliable expertise as in their actual specialist area.
  • Experts tend to overestimate the speed of developments. Above all, the speed of diffusion of an innovation in society is quickly overestimated.
  • When interviewing a group , a social situation arises. This can lead to distortions due to authority or personal trench warfare. It is not always clear whether a consensus (or a dissent) is actually only based on the intensive questioning of one's own opinion. An anonymization in the feedback round can i. A. Do not avoid completely. When using questionnaires (e-mail or post), the Delphi method is explicitly used to circumvent this dominance. This is only possible to a limited extent for face-to-face meetings.

See also

literature

  • Marlen Niederberger, Ortwin Renn: Delphi method in the social and health sciences: concept, variants and application examples . Springer VS, 2019, ISBN 978-3-658-21656-6 .
  • Marlen Niederberger, Ortwin Renn: The group Delphi process: from concept to application . Springer VS, 2018, ISBN 978-3-658-18754-5 .
  • Michael Häder (Ed.): Delphi surveys. A work book . West German Verlag, Wiesbaden 2002, ISBN 3-531-13748-4 .
  • Michael Häder, Sabine Häder (Hrsg.): The Delphi technique in the social sciences: Methodical research and innovative applications (ZUMA publications) . Springer VS, 2000, ISBN 978-3-531-13523-6 .
  • Ursula Ammon: Delphi survey. Quantitative organizational research. Qualitative-Research.net, 2005. Online portal for qualitative social research, Free University of Berlin. Available online
  • Kerstin Cuhls, Knut Blind, Hariolf Grupp (eds.): Delphi '98 survey. Demand for the future. Study on the global development of science and technology. Karlsruhe 1998.
  • Hariolf Grupp (ed.): The Delphi Report. with the collaboration of Breiner, Sibylle and Cuhls, Kerstin. dva-Verlag, Stuttgart 1995.
  • HA von der Gracht: The Delphi Technique for Futures Research. In: The Future of Logistics , Springer / Gabler, 2008, Chapter 3, pp. 21-68, ISBN 978-3-8349-9764-7 .
  • HA von der Gracht: Consensus Measurement in Delphi Studies - Review and Implications for Future Quality Assurance. In: Technological Forecasting & Social Change. 79, No. 8, 2012, pp. 1525-1536, doi: 10.1016 / j.techfore.2012.04.013 . - A summary of methods and criteria for panel consensus and dissent measurement
  • A. Spickermann et al. a .: Surface- and deep-level diversity in panel selection - Exploring diversity effects on response behavior in foresight. In: Technological Forecasting & Social Change. Vol. 85, 2014, pp. 105-120, doi: 10.1016 / j.techfore.2013.04.009 . - An example of scientifically sound expert criteria and selection
  • Stefan Aengenheyster et al: Real-Time Delphi in practice - A comparative analysis of existing software-based tools. In: Technological Forecasting & Social Change. Vol. 118, 2017, pp. 15-27, doi: 10.1016 / j.techfore.2017.01.023 . - Comparison of different realtime Delphi applications
  • T. Gnatzy et al .: Validating an innovative real-time Delphi approach - A methodological comparison between real-time and conventional Delphi studies. In: Technological Forecasting & Social Change. Vol. 78, No. 9, 2011, pp. 1681-1694, doi: 10.1016 / j.techfore.2011.04.006 . - Comparison of methods of real-time and classic Delphi studies
  • USAF Project RAND Report Delphi Assessment: Expert Opinion, Forecasting and Group Process (pdf; 6.0 MB)

Secondary literature on the Delphi method:

Delphi studies:

  • Kerstin Cuhls, Jürgen von Oertzen, Simone Kimpeler: Future Information Technology for the Health Sector. FAZIT series, Stuttgart 2007, www.fazit-forschung.de
  • Kerstin Cuhls, Jürgen von Oertzen, Simone Kimpeler: Future information technology for the health sector. Results of a Delphi survey. (= FAZIT series of publications ). Stuttgart 2007, www.fazit-forschung.de
  • HA Linstone: The Delphi-Method - Techniques & Applications. Massachusetts 1975.
  • HA von der Gracht, I.-L. Darkow: Scenarios for the Logistics Service Industry: A Delphi-based analysis for 2025. In: International Journal of Production Economics. Vol. 127, No. 1, 2010, pp. 46-59, doi: 10.1016 / j.ijpe.2010.04.013 . - An example of scientifically sound Delphi-based scenario development in the logistics / mobility industry
  • O. Quasdorff: The Lean Factory, taking the digital factory into account. In: Uwe Bracht (Ed.): Innovations in factory planning and organization. Volume 36, Shaker Verlag, Aachen 2016, ISBN 978-3-8440-4934-3 .
  • K. Ullrich and C. Wenger: Vision 2017 - What will move people tomorrow. Heidelberg 2008, ISBN 978-3-636-01582-2 .
  • D. Knowledge: Bibliography of the future - future of bibliography - an expert survey using Delphi technology in archives and libraries in Germany, Austria and Switzerland. Berlin 2007.

Web links

Individual evidence

  1. ↑ Manfred Hüttner : Market and sales forecasts. Kohlhammer, 1982, ISBN 3-17-007325-7 , p. 29.
  2. ↑ Kevin Tappe: Cooke Method. In: Journal for Controlling. Volume 24, Issue 4/5, 2012, pp. 278–279; for an exemplary application of the Cooke method, see Nature. Issue 463, pp. 294-295.