Meta-analysis

from Wikipedia, the free encyclopedia
Forest plot to graphically illustrate the results of a meta-analysis. The vertical line denotes the opportunity ratio or risk ratio that does not correspond to any relationship between the influencing factor examined and the dependent variable. The squares represent the degree of risk of the individual studies, the horizontal lines the respective confidence intervals . Similarly, the rhombus and the associated line show the values ​​of the combined data.

A meta-analysis is a summary of primary studies on metadata , which works with quantitative and statistical means. It tries to summarize and present earlier research work quantitatively or statistically. It differs from the systematic review article (also called “review”) in that a review critically evaluates previous research data and publications, while the meta-analysis only covers the quantitative and statistical processing of previous results.

Meta-analyzes are carried out in all research areas in which empirical data arise. This includes social sciences, medicine and many natural sciences.

Creation of the term

The term was introduced in 1976 by the psychologist Gene V. Glass (* 1940) in his article "Primary, Secondary and Meta-Analysis of Research" . He defines meta-analysis as […] analysis of analyzes. I use it to refer to the statistical analysis of a large collection on analysis results from individual studies for the purpose of integrating the findings . (German: "[...] Analysis of analyzes. By that I mean the statistical analysis of a large collection of analysis results from several individual studies, which are to be merged."). However, the first meta-analysis was carried out as early as 1904 by Karl Pearson , who wanted to increase the test strength of studies with a few subjects by summarizing them.

Areas of application and reasons

Meta-analyzes make it possible to combine various studies into one scientific research area. The individual empirical results of primary studies with homogeneous content are summarized. The aim is to estimate the effect size . It is to be examined whether there is an effect and how big it is.

Reasons for performing meta-analyzes include:

  • For various reasons, the samples of the primary studies are far too small to produce reliable results that can be transferred to similar cases. If many examinations are grouped together appropriately, the result obtained can be more reliable due to the larger total number of samples.
  • Often the primary studies use different methods, definitions or do not draw their samples from the same population . There are also factors that can influence individual results. A meta-analysis can determine which influences exist and how strong they are and whether a valid overall picture is still possible.
  • It should not be underestimated that carrying out meta-analyzes (and also reviews) is relatively inexpensive and still delivers valuable results. It provides the basis for an entire research area from which to outline future research activities.
  • The publication bias , ie the relationship between positive results and “negative”, ie non-significant results, can be determined.

method

A meta-analysis encompasses all elements of the social science research process, as they are also carried out in primary research (Cooper 1982; Schnell et al. 1995).

  1. At the beginning there is the selection of a suitable research problem as well as the exact definition of the research subject.
  2. In the case of publication-based meta-analyzes, data collection consists of a systematic and as exhaustive as possible a literature search.
  3. The information in the collected publications is then coded and processed electronically.
  4. The actual (statistical) data analysis usually consists of two steps: the integration of the findings and the subsequent heterogeneity analysis .
  5. Finally, the results are to be appropriately prepared and interpreted with reference to the research problem.

The process of meta-analysis is similar to the narrative review , which presents the relevant literature on a scientific topic in a structured manner, compares it and provides critical comments. The subjectivity of the selection of examinations is criticized. Here the meta-analysis achieves greater objectivity by defining criteria for the selection of the primary studies for the meta-analysis. However, this reduces the possible number of examinations that can be included in a meta-analysis.

The combination of different studies into one scientific research area only makes sense if the effect sizes of the individual studies are estimates of a common population effect size. A homogeneity check is necessary.

Homogeneity tests assume a uniform effect size Δ (read: delta). Δ is a universal effect size measure and corresponds to the bivariate product-moment correlation . It is preferred because various statistical parameters (e.g. r , t , F ) can be transformed into Δ.

The study-specific effect sizes are then checked for homogeneity using a significance test. If the effect sizes are homogeneous, an average Δ value can be calculated; it corresponds to the estimate of the population effect size and can be checked for significance and classified with regard to its size.

If the effect sizes are heterogeneous, strategies can be used that divide the studies with heterogeneous effect sizes into homogeneous subgroups. Then the influence of moderator variables on the heterogeneity should be determined; this is done using variance or cluster analysis . If there are no direct assumptions about the effect of moderator variables, a correlation between the moderator variables and the study-specific effect sizes can be calculated. The level of correlation describes the influence of the moderator variables on the heterogeneity of the individual effect sizes.

Since examination reports are often incomplete and sometimes only significant or insignificant results are reported, there are methods that make it possible to use these examinations meta-analytically (e.g. counting, sign test , binomial test and calculation of the exact probability of error , es results in the Stouffer test variable ). With the help of the ' fail-safe N ' (according to Rosenthal, 1979), if a significant overall test is available, it can be calculated how many studies with a mean effect of size zero would have to be available for the overall test to be insignificant. By calculating the Fail-Save N, the problem is to be countered that gray literature, i.e. unpublished studies, is not recorded (see below).

The meta-analysis can be rounded off at the end with a so-called meta-regression . With regression method is to determine which properties of the individual studies (eg. As diagnostic criteria, origin, number of subjects, ...) lead to which effect sizes.

Discussion of the method

Garbage-in-garbage-out problem

It is criticized that the results of a meta-analysis are not very valid because any investigation, regardless of its methodological quality, is included in the meta-analysis. However, the influence of the methodological quality of the investigations on the result of the meta-analysis can be controlled by either giving more weight to quality criteria of qualitatively outstanding studies or by excluding exclusion criteria for inadequate studies. The studies can be grouped according to methodological quality and evaluated separately.

It should be emphasized here that studies with a small number of test subjects or samples are not unsatisfactory per se.

The apple and pear problem

It is criticized that meta-analyzes summarize studies with different operationalization variants. It is required that there must be homogeneous operationalizations, especially with regard to the dependent variable, since they should all be indicators for the same construct . Otherwise the examinations relate to different criteria, a summary would then not make sense.

This problem can also be eliminated or mitigated by grouping the studies according to publication period or different definitions / operationalizations and evaluating them separately.

Drawer problem

( File Drawer Problem ): Often only results are published that confirm assumed hypotheses or show significant results, while studies with non-significant results are not published ( publication bias ). This leads to a distortion of the meta-analytical results, since they prove the existence of an effect too often. Unpublished literature is also known as gray literature . "Interestingly, the gray literature in the former GDR as non-censored literature usually had a higher scientific value than the official, state-controlled literature." "A list of the latest diploma theses and dissertations in the subject of psychology is attached to the Psychological Review every six months." access to gray literature is dealt with by Marylu C. Rosenthal (1994).

The publication bias is estimated in every careful meta-analysis, for example using the funnel plot . At the beginning of a meta-analysis, however, it should be assessed to what extent and with what effort the gray, unpublished literature should be obtained and taken into account. In the published literature, however, it is not uncommon to find references to unpublished research results, and nowadays, thanks to the Internet, it is easy to ask researchers for unpublished data. However, this approach in turn creates a methodological weakness that can hardly be remedied.

Problem of dependent measurements

This problem occurs when different (dependent) partial results have been collected on the same sample. This can occur, for example, if a scientist has investigated the same health problem several times in a row in the same geographical area and published the results in several publications. However, since the units of meta-analysis are individual studies and not partial samples, only one result of an investigation may be included in the meta-analysis, otherwise this investigation would be given greater weight than an investigation that is only included in the meta-analysis with one result. Either you limit yourself to the most important or most meaningful result among the partial results, or you form the arithmetic mean as an estimate of the total result. However, if the raw data from all partial results are available, an overall result can be reconstructed.

literature

  • Harris M. Cooper: Scientific guidelines for conducting integrative research reviews . In: Review of Educational Research 52, 1982, ISSN  0034-6543 , pp. 291-302.
  • Harris Cooper, Larry V. Hedges (Eds.): The Handbook of Research Synthesis . Russell Sage Foundation, New York NY 1994, ISBN 0-87154-226-9 .
  • Gene V. Glass: Primary, Secondary and Meta-Analysis of Research . In: Educational Researcher 5, 1976, ISSN  0013-189X , pp. 3-8.
  • Joachim Hartung , Guido Knapp, Bimal K. Sinha: Statistical meta-analysis with applications . J. Wiley & Sons, Inc., Hoboken NJ 2008, ISBN 978-0-470-29089-7 , (Wiley Series in Probability and Statistics) .
  • John E. Hunter, Frank L. Schmidt, Gregg B. Jackson: Meta-Analysis. Cumulating research findings across studies . Sage Publications, Beverly Hills CA 1982, ISBN 0-8039-1864-X , ( Studying organizations 4), (Recommended introduction).
  • Rainer Schnell , Paul B. Hill , Elke Esser: Methods of empirical social research . 5th completely revised and enlarged edition. Oldenbourg Verlag, Munich et al. 1995, ISBN 3-486-23489-7 .

Web links

Individual evidence

  1. ^ Gene V. Glass: Primary, Secondary and Meta-Analysis of Research . In: Educational Researcher 5, 1976, pp. 3-8, doi : 10.3102 / 0013189X005010003 JSTOR 1174772 .
  2. ↑ a b Performance and trust in business partnerships: A meta-analysis on the determining factors for the relationship between performance and trust in business partnerships . kassel university press GmbH, 2011, ISBN 978-3-86219-215-1 , p. 135 ( limited preview in Google Book search).
  3. ^ Jürgen Bortz , Nicola Döring: Research methods and evaluation: for human and social scientists . Springer-Verlag, 2013, ISBN 978-3-662-07299-8 , pp. 646 ( limited preview in Google Book Search).
  4. ↑ meta-analysis . Rainer Hampp Verlag, 2014, ISBN 978-3-86618-975-1 , p. 77 ( limited preview in Google Book search).
  5. ^ A b Jürgen Bortz , Nicola Döring: Research methods and evaluation for human and social scientists. 4th, revised edition. Springer Medizin-Verlag, Heidelberg 2006, ISBN 3-540-33305-3 , p. 674, (online) .
  6. ^ A b Jürgen Bortz, Nicola Döring: Research methods and evaluation for human and social scientists. 4th, revised edition. Springer Medizin-Verlag, Heidelberg 2006, ISBN 3-540-33305-3 , p. 360, (online) .
  7. ^ Marylu C. Rosenthal: The Fugitive Literature. In: Harris Cooper, Larry V. Hedges (Eds.): The Handbook of Research Synthesis. Russell Sage Foundation, New York NY 1994, ISBN 0-87154-226-9 , pp. 85-94.