Evaluation

from Wikipedia, the free encyclopedia
This article or section is not worded in a generally understandable manner. The shortcomings are described under Discussion: Evaluation . If you remove this module, please explain this on the article discussion page and add the automatically created project page section Wikipedia: Incomprehensible articles #Evaluation{{ Done | 1 = ~~~~}}.

Evaluation or evaluation (from the Latin valere "strong, worth being") means appropriate and professional (investigation and) assessment .

overview

Evaluation is usually understood to mean the assessment or assessment of projects , processes and functional units (e.g. devices, objects) and: organizational units . Context, structure, process, effort and result can be included. In general, an evaluation can also be understood as the fundamental investigation into whether and to what extent something appears suitable to fulfill an intended purpose. In linguistic usage, evaluation, investigation and analysis are also used synonymously in the sense of an inventory without any special purpose orientation.

Areas of application for evaluation are, for example, education , social work , administration , economics or political advice . For an evaluation, data are methodically collected and systematically documented in order to make the investigation, the procedure and the results comprehensible and verifiable. Standard procedures for internal and external data collection are questioning , observation, monitoring , test , questionnaire and material analysis. The evaluation is carried out by comparing the determined actual values ​​with previously explicitly defined, operationalized and justified target values ​​on the basis of defined indicators. Evaluation must meet certain “quality criteria”: in addition to the basic requirements of usefulness and objectivity , these are reliability , validity , economy and standardization .

Evaluation is used for retrospective control of effectiveness, foresighted control and an understanding of situations and processes. Based on the evaluation data, examined processes can be adapted and optimized. Depending on the time of deployment, the following types of evaluation are distinguished:

Anticipatory or prospective evaluation
This takes place before a measure.
Formative evaluation
The formative or process-accompanying evaluation is carried out accompanying a measure. The measure is examined at regular intervals and interim results are collected. This is also done with the aim of adapting and optimizing the current measure. These new measures can in turn be evaluated. In addition, within the scope of the formative evaluation there is the possibility of receiving subjective impressions from those affected (without retrospective distortions).
Summative evaluation
As a summative evaluation, a result-evaluating, i. H. an evaluation that takes place after the completion of a measure. This enables the effectiveness of a measure to be assessed in summary and can relate to conception, implementation, effectiveness and efficiency.

With the research evaluation combines the quest for empirical verifiability, but it examines specific individual cases and practical.

Word origin

The word evaluation was first used in French in the 19th century , and in English at the beginning of the 20th century . In the USA it has been used in modern terms since the 1930s (since the Eight Years Study 1933–1941 by Ralph W. Tyler et al.). Tyler's behavioral concept was criticized and changed after the Sputnik shock in 1957 in the educational reform that followed. In Germany, the English word evaluation was adopted from this American tradition at the end of the 1960s; A collection of translations of English-language publications by Christoph Wulf was important for this . a. in the context of curriculum research . The word meaning "evaluation" was initially narrowed to the evaluation of the work of educational institutions in particular (the curriculum development since the 1930s). In the meantime, as in the USA for a long time, other areas of social activity are also being “evaluated” in Germany.

Demarcation

According to Balzer, professionally carried out evaluations differ from everyday assessments in the following criteria:

  • Related to a clearly defined subject
  • Carried out by experts
  • Evaluation based on precisely defined and disclosed criteria (evaluation / assessment criteria)
  • Obtaining information through empirical data collection
  • Systematic evaluation of information based on certain rules

Balzer distinguishes: “Evaluation is a process ... in which an object of evaluation is evaluated according to previously defined goals and criteria that are explicitly related to the facts and are justified. This is done with the help of sociological methods by people who are particularly qualified for this. The product of an evaluation process consists in the feedback of usable results in the form of descriptions, well-founded interpretations and recommendations to as many people involved and affected as possible in order to optimize the object of the evaluation and support future action. "

The assessment of persons or institutions associated with (often negative) sanctions must be distinguished from the assessment of measures that are examined for their effectiveness and which can be referred to as evaluation in the true sense. An initial survey is carried out on the basis of a target agreement and measures are then planned with which the targets are to be achieved. Then measuring instruments / assessment criteria have to be developed with which one can check whether the measures have led to success . Possibly after interim surveys during the implementation, the success of the measure is checked in a final survey in order to make new target agreements and to re-enter the cycle. Those affected should become participants, so that the process runs consensually and is not determined by foreign interests and unclear criteria.

Models

There are essentially three different directions in the evaluation:

Method-oriented evaluation:

  • Objectives-Oriented Evaluation (R. Tyler)
  • The methodological view ( Donald T. Campbell )
  • Evaluative Research (EA Suchman)
  • Causal Generalization (T. Cook)
  • Theory Driven Evaluation (H.-T. Chen)
  • Evaluation as Art (LJ Cronbach)

Benefit-oriented evaluation

  • CIPP Model (DL Stufflebeam)
  • Evaluation Research, Political Context & Enlightment (CH Weiss)
  • Utilization-focused Evaluation (MQ Patton)
  • The Four Levels ( Donald Kirkpatrick )
  • Empowerment Evaluation (D. Fettermann)

Assessment-oriented evaluation

  • Valuing (M. Scriven)
  • Responsive Evaluation (RE Stake)
  • 4th Generation - Constructivist Evaluation (Guba & Lincoln)
  • Adversery Evaluation (Owens & Wolf)
  • Connoisseurship Model (EW Eisner)

Methods

Bortz & Döring differentiate between three types of methods:

  • Exploratory methods. They “serve to explore intervention processes and their effects. They aim at the formulation or specification of effect hypotheses and help to identify and operationalize the relevant variables. "
  • Population descriptive methods. They "enable an assessment of the distribution and the background of a situation and facilitate the definition of the target population."
  • Hypothesis-testing methods. They "test the influence of the examined intervention on meaningfully operationalized impact criteria."

In 1976, on the occasion of a college level trial in North Rhine-Westphalia, Andreas Gruschka formulated what has since found its way into the manuals: "Quality criteria for the evaluation are no longer primarily validity, reliability and objectivity, but communication, intervention, transparency and relevance."

Methods for data acquisition and data processing are:

Areas

Important areas of application are e.g. B.

Quality management

Evaluation has long been an integral part of quality management ( TQM , ISO 9000 , 2Q, Q2E). In the EFQM model , evaluation is e.g. B. required in order to record the results (of the work) with customers, employees and society. Evaluation functions here as a feedback element for process control in the context of organizational and quality development. It is checked whether the interventions actually produce the desired results or effects (cause-effect relationship). Both subjective data about the perception of those affected, i.e. employees and customers, as well as objective performance indicators that can be recorded internally in the company are recorded.

In addition to evaluating the results, the EFQM model also requires the direct evaluation of processes, interventions and measures in the areas of leadership , strategy and personnel. This is where the self-evaluation method is used to evaluate the processes and ultimately to improve them. The company's executives and employees evaluate themselves, their teams and their organization based on given criteria. To do this, you use the most objective, reliable and valid instruments and methods that have been specially developed for quality management (QM). Recommendations for action and change measures are derived on the basis of the data, assessments and evidence obtained.

Both the self and third party evaluations can be supplemented by benchmarking and thus made more meaningful. Data from other comparable organizations serve as benchmarks. They provide standards for evaluating and interpreting one's own values ​​and thus one's own situation. Ideally, concrete contact with the best in the industry can also enable mutual learning.

In addition to these purely internal evaluations are to certification and external evaluations required purposes: external independent auditors evaluate the organization using the same criteria and give their vote from. The traditional approaches of EFQM or ISO9000 are mostly based on very linear cause and effect models. Examples of other approaches are the cybernetic approaches of Fredmund Malik .

education

As a pedagogical or andragogical technical term, evaluation means “the methodical recording and justified evaluation of processes and results for a better understanding and design of a practical measure in the educational sector through effectiveness control, control and reflection.” The subject of evaluation can include processes and results from the field of Microdidactics as well as macrodidactics . In addition, entire programs can be evaluated.

environment

So far, environmental protection has not been one of the central application areas of the evaluation either in the USA or in Europe. With a few exceptions, social science studies to evaluate environmental problems are rather rare, especially in the areas of energy consumption, noise protection and waste production, even within the framework of larger political programs.

One reason is certainly to be seen in the comparatively low recognition of the social sciences as a competent partner in environmental issues, which only changed fundamentally in the nineties, given the lack of success in measures to increase environmental awareness. After the formula environmental education creates environmental awareness and environmental awareness leads to appropriate behavior was refuted by a large number of social science studies and the importance of social factors for environmentally relevant individual behavior was pointed out, the environmental discussion began to open up to social science issues and increasingly abandoned the view, that environmental problems can be overcome purely through technical solutions.

At present, the demand for environmental evaluations is very much determined by public clients. The contracts relate primarily to the evaluation of political programs or individual projects. In international organizations in particular, tendencies towards the development of quality requirements can be identified, which are intended to prepare the introduction of environmental evaluations as a standard procedure in the implementation of political procedures. In Germany, at the federal and state level, the political authorities at best have made the first steps in this direction. Due to the great importance of evaluations in urban and spatial planning, the establishment of social-scientific environmental evaluations through integration into existing, more technically oriented (planning) processes such as B. the environmental impact assessment is even more likely to be expected at regional and municipal level. The private demand for environmental evaluations by companies as well as associations and civil society organizations is still very low. Despite the development of environmental management systems (e.g. EMAS), it has not yet been possible to integrate the findings of the social science evaluation into the quality management systems of the companies.

On the supply side, at least some state-funded research institutes dominate the market. However, there are no institutes that specialize exclusively in carrying out environmental evaluations. The development at the Center for Evaluation at Saarland University and the Center for Sustainable Management (CSM) at Lüneburg University goes furthest .

Even in private consultations, only a few have focused their activities on social-scientific environmental evaluations. Examples to be mentioned here are the Institute for Social-Ecological Research (ISOE) and the Institute for Ecological Economic Research (IÖW). In addition, established environmental science institutes such as the Wuppertal Institute , the Öko-Institut in Freiburg or the IFEU Institute in Heidelberg have expanded their offerings in the direction of social science research, but without seeing this as a primary business area and without carrying out corresponding development work. In view of the increasing requirements that result from the concept of "sustainable development" for the introduction of corresponding measures, an expansive development of the evaluation market and a progressive professionalization of the offer can be expected in the environmental sector as well, despite the difficult starting position.

International development cooperation

Evaluations in development cooperation have a long tradition. The focus is on evaluating projects and programs.

The pronounced culture of evaluation within development cooperation is primarily due to the fact that the funds used in projects and programs always had to and must be justified to the donors.

Evaluation approaches in development cooperation are also strongly shaped by international agreements, in particular the Millennium Declaration and the Paris Declaration. The so-called "harmonization" as a general principle of development cooperation is reflected, for example, in the demand for joint evaluations of the actors. In Germany, state organizations have committed themselves to “unified development cooperation” and also strive for a uniform approach to evaluations. Partner countries are increasingly assuming the roles of the client and the executor of the evaluation.

As far as the evaluation objects are concerned, the focus is shifting more and more from output evaluation as proof of performance to evaluating the results (outcomes and impacts) of projects and programs in the field of development cooperation. You work with chains of effects and sophisticated evaluation designs, such as B. the Logical Framework Matrix ("Logframe").

Traditional evaluation concepts and methods based on linear causality models are increasingly being questioned and supplemented or even displaced by cybernetic approaches. A development similar to that currently taking place in the area of ​​organizational development and evaluation. In addition to the search for innovative (self-) evaluation methods (e.g. learning helix), the (further) development of scientifically founded concepts appears as the basis for evaluation - such as B. the service quality or the quality of life concept - as particularly important. Compared to other industries, such as In development cooperation , for example, education or health and social services , comparatively little scientific know-how (evidence base) is included in the evaluation.

Organizations

The German DeGEval - Society for Evaluation - has numerous methodical and sectoral working groups. It has published standards, recommendations and guidelines for planning and carrying out evaluations. Most of its members come from Germany and Austria.

In the international area, organizations - such as the UN , EU , OECD , the IMF , GIZ , etc. - have their own evaluation departments which, in addition to dealing with specific questions, also collect, develop and process general standards and methods.

In addition, research organizations in German-speaking countries, such as the Center for Evaluation in Saarland or proEval in Austria, are involved in professionalizing evaluation.

See also

literature

  • Lars Balzer: How do evaluation projects become successful? - An integrating theoretical approach and an empirical study on the evaluation process. Verlag Empirische Pädagogik, Landau 2005, ISBN 3-937333-24-X .
  • Wolfgang Beywl: On the further development of the evaluation methodology. Foundation, conception and application of a model of responsive evaluation. Peter Lang, Frankfurt am Main 1984.
  • Jürgen Bortz, Nicola Döring: Research methods and evaluation for human and social scientists. 4th edition. Springer, Berlin 2006.
  • H. Kromrey: Evaluation - a complex concept. Concept and methodology of evaluation and evaluation research. Recommendations for practice. Social Sciences and Professional Practice 24 (2001), 105–129.
  • M. Ruep, G. Keller: School evaluation. Frankfurt am Main 2007.
  • James R. Sanders: Handbook of Evaluation Standards. The standards of the "Joint Committee on Standards for Educational Evaluation", translated by W. Beywl and T. Widmer, 3rd edition, Wiesbaden 2006.
  • Reinhard Stockmann (Ed.): Handbook for Evaluation. A practical guide to action. Waxmann, Münster a. a. 2007.
  • H. Wottawa, H. Thierau: Textbook Evaluation. 2nd Edition. Hans Huber, Bern 1998.
  • H. Wottawa: Evaluation. In: A. Krapp, B. Weidenmann (Ed.): Pedagogical Psychology. Beltz, Weinheim 2001, pp. 649-674.

Web links

Commons : Evaluation  - collection of images, videos and audio files
Wiktionary: Evaluation  - explanations of meanings, word origins, synonyms, translations

Individual evidence

  1. ^ Evaluation - Duden , Bibliographisches Institut ; 2017
  2. ↑ K. Götz: For the evaluation of professional training. Deutscher Studienverlag, Weinheim; 1993
  3. ↑ Markus Pospeschill: Empirical Methods in Psychology . tape 4010 . UTB, Munich 2013, ISBN 978-3-8252-4010-3 .
  4. ↑ See Craig Kridel, Robert V. Bullough: Stories of the Eight-Year Study. State University of New York Press, New York 2007; Peter Hlebowitsh: Ralph W. Tyler (1902-1994) . Contribution to Testing and Curriculum Development, Advisory Role, in: Education Encyclopedia, StateUniversity.com.
  5. ↑ Christoph Wulf (Ed.): Evaluation . Description and evaluation of lessons, curricula and school trials. Education in Science and Practice 18, Piper, Munich 1972 ( table of contents ).
  6. ↑ a b L. Balzer: How do evaluation projects become successful? - An integrating theoretical approach and an empirical study on the evaluation process. Verlag Empirische Pädagogik, Landau 2005.
  7. ^ Marvin C. Alkin, Christina A. Christie: An Evaluation Theory Tree. In: Marvin C. Alkin (Ed.): Evaluation Roots. Tracing Theorists' Views and Influences . Thousand Oaks: Sage 2004, pp. 12-65.
  8. ↑ Bortz & Döring 2005, p. 118.
  9. ↑ Andreas Gruschka (Ed.): A school experiment is checked . The evaluation design for college level NW as a concept of action-oriented accompanying research, Kronberg 1976, pp. 142–151; often quoted or reformulated, e.g. B. Bortz & Döring 2006 or Reinhard Stockmann: Evaluation in Germany. In the S. (Ed.): Evaluation research : Fundamentals and selected research fields, Waxmann, Münster 3rd A. 2006, pp. 15–46, here p. 23.
  10. ↑ Provide a comprehensive overview of the German-speaking area: Widmer, Thomas / Beywl, Wolfgang / Fabian, Carlo (eds.) (2009): Evaluation. A systematic manual. Wiesbaden: VS Verlag.
  11. ↑ Jost Reischmann: Further education evaluation . Making learning success measurable, basics of further training work aids, Luchterhand, Neuwied 2003, p. 18; Augsburg 2nd August 2006.