Quality assurance in psychological diagnostics

from Wikipedia, the free encyclopedia

In the field of psychological diagnostics, there are various concepts and approaches to ensure or increase quality . The aim is the use of scientifically founded instruments by appropriately trained persons for issues for which the instruments are actually developed / suitable. Appropriate quality initiatives are also intended to distinguish themselves from the unqualified use of test procedures by unskilled persons (which can even go as far as charlatanism ).

Need for the concept of quality

The quality of psychodiagnostic procedures is not always easy to see for clients of psychodiagnostics and those diagnosed, because often only the results of the procedures are disclosed and the functionality of the instruments is not adequately presented. This makes it difficult for those involved to distinguish professionally developed instruments from those that do not or only partially meet the quality criteria of psychodiagnostic procedures . According to several estimates by the professional associations, only about 20% of all psychodiagnostic examinations are carried out by qualified persons with suitable instruments (i.e. with sufficient quality).

Because such instruments are often and increasingly used for decisions of high importance (admission decisions for educational paths, decisions about filling positions and career planning, suitability decisions for certain activities; determining illnesses and disorders and deriving the correct treatment measures), and wrong decisions cause considerable additional costs, begins with to develop a quality awareness for everyone involved. For the legality of these decisions, it is necessary that the underlying rules are not only accessible to the experts, but are also generally understandable and comprehensible.

This is contradicted by the fact that such binding guidelines at the legal level are missing and are not in sight. This is also due to the fact that there are no universally recognized criteria for quality - the details of the necessary minimum standards are still being discussed in the subject itself. In recent years there have been several attempts to define criteria and present them in a generally understandable manner. The process of reaching a consensus is still ongoing. It should be noted that there has long been a consensus on the basic requirements within psychology. These are part of basic psychological knowledge and compliance with them is part of the ethical principles of psychological activity. However, the diagnostics provider market is not limited to psychological specialists and, above all, resistance outside of psychology (cf. DIN 33430 ) delays the finding of a consensus.

It has been recognized within psychology that it is not enough for these guidelines to be understandable only to experts. Knowledge of these is also important for clients and those diagnosed in order to recognize misuse of psychological procedures or to evaluate suitable providers. After all, in the end it is primarily the clients who decide in many areas which processes will prevail on the market (e.g. personnel diagnostics).

Elements that require quality assurance

Quality assurance measures must encompass the entire diagnostic process , which includes three aspects:

  • The quality of the procedures used, in particular the fulfillment of the quality criteria of psychodiagnostic procedures
  • The qualifications of the people involved
  • Compliance with procedures and rules (data protection, preservation of personal integrity, type of feedback on the results) from planning to the diagnostic decision.

International initiative

The International Test Commission (ITC) has adopted international guidelines for the use of tests , which are recognized by many professional associations (including the Professional Association of German Psychologists ).

Content:

  • 1 RESPONSIBILITY FOR ETHICALLY CORRECT USE OF THE TEST
    • 1.1 Act in a professional and ethically correct manner
    • 1.2 Make sure they are competent in the test application
    • 1.3 Take responsibility for your application of tests
    • 1.4 Ensure the safe storage of test material
    • 1.5 Guarantee the confidential treatment of test results
  • 2 TECHNICAL COMPETENT PRACTICE IN TEST APPLICATION
    • 2.1 Evaluation of the possible usefulness of tests in a diagnostic situation
    • 2.2 Selection of technically flawless tests that are appropriate for the situation
    • 2.3 Consideration of questions of fairness in the test application
    • 2.4 Necessary preparations for carrying out the test
    • 2.5 Professionally competent test specification
    • 2.6 Accurate test evaluation and analysis of test results
    • 2.7 Appropriate interpretation of test results
    • 2.8 Clear and precise communication of test results
    • 2.9 Review of the adequacy of a test and its application
  • APPENDIX A: GUIDELINES FOR DESIGNING PRINCIPLES IN TEST APPLICATION
  • APPENDIX B: GUIDELINES FOR AGREEMENTS BETWEEN PARTICIPANTS IN A TESTING PROCESS
  • APPENDIX C: FACTS THAT SHOULD BE CONSIDERED WHEN TESTING WITH THE DISABLED OR OTHERALLY IMPAIRED PERSONS

Further guidelines (guidelines) of the International Test Commission concern

  • the adaptation of tests (transferring them to other languages ​​or cultures)
  • computer-based and internet-based testing
  • the quality control for test evaluation, test analysis and result report
  • the security of tests, examinations and other measurements ("test protection")
  • Practitioner Use of Test Revisions, Obsolete Tests, and Test Disposal (German test revision, obsolete tests and test disposal)

A guideline for testing not in the mother tongue is in preparation

Germany: DIN 33430 and test board

For suitability diagnostics, there is DIN standard 33430 , which relates to the entire diagnostic process (procedure, people involved and processes). In Austria, this standard was adopted with essentially the same content as Ö-NORM D4000. In Germany, the Diagnostic and Test Board of Trustees of the Federation of German Psychological Associations is a facility for monitoring the quality of psychological procedures. Quality guidelines were developed as checklists (TBS-TK), according to which the quality of tests is assessed and published in the form of reviews. In 2016 a new version of the standard was published, which was adapted to the current conditions.

ISO standard 10667

At the initiative of DIN , an international ISO working group was set up in March 2007 to develop DIN 33430 and other international initiatives into an ISO standard “Psychological assessment” (Procedures and methods to assess people in work and organizational settings). A final form of the standard was published in October 2011. The ISO 10667 standard Assessment service delivery - Procedures and methods to assess people in work and organizational settings is divided into two parts (Part 1: Requirements for the client , Part 2: Requirements for service providers ). First critical analyzes show that it calls for a lower level of requirements compared to DIN 33430 and is more tailored to the American market (which actually led to the drafting in two parts).

Quality standards for special aspects

Adaptation of tests in other languages

Many tests are highly language and culture dependent. This applies to formulations (which can be idiomatic) or to skills and abilities that are differently promoted in different educational systems or through different educational pathways. This becomes relevant for the selection of the adequate comparative norms that apply to the evaluation. For example, extraversion is different in America than it is in Europe. It is not enough to translate tests. It must also be checked whether the same is still measured. In addition, it is almost always necessary to re-standardize the tests in the other languages.

The ITC's "Guidelines on Test Adaptation" contain answers to important questions that arise during test adaptations . A forerunner was the Technical Standards for Translating and Adapting Tests and Establishing Test Score Equivalence , which were published in 1994 by six leading international psychological societies.

Computer and Internet Testing Guidelines

The use of information technology in psychological diagnostics is becoming more and more widespread. The International Test Commission ITC has therefore drawn up guidelines for computer and internet-based testing which take into account the possibilities and limits of this access.

Procedures for occupational medicine and occupational safety

The Federal Institute for Occupational Safety and Health (BAuA) offers an overview of test procedures in its "Toolbox". Under the heading "Quality criteria" you will find information that is important for quality assurance. Under the heading “design reference” it is indicated whether the procedure is used for relationship prevention or for behavioral prevention. For individual psychological diagnostics, the methods of behavior prevention are more likely to be considered, while methods of relative prevention in occupational safety are required to assess working conditions.

Other countries

USA: Standards for Educational and Psychological Testing

The American Psychological Association (APA), together with the American Educational Research Association (AERA) and the National Council on Measurement in Education (NCME), has published an internationally recognized quality guideline, the "Standards for Educational and Psychological Testing". The current fourth version was adopted in the USA in 1999 and revised in 2014.

An earlier, but similar version was published in German as Supplementum 1/1998 of the journals Diagnostica and the Zeitschrift für Differielle und Diagnostische Psychologie. It includes:

  • Methodological standards for test design and evaluation
  • Standards for a professionally competent test application
  • Standards for special applications (linguistic minorities, people with disabilities)
  • Standards for procedures (implementation, evaluation, interpretation; protection of the rights of the test persons)

USA: Code of Fair Testing Practices in Education

These standards (as of 2004) are primarily aimed at diagnostics in the education sector. They are compatible with the above, very much geared towards comprehensibility and practicality.

  • A. Developing and Selecting Appropriate Tests (test development and test selection are discussed in connection with suitability for a question)
  • B. Administering and Scoring Tests (application recommendations and the correct evaluation)
  • C. Reporting and Interpreting Test Results (how should test results be interpreted correctly)
  • D. Informing Test Takers (how to communicate test results and diagnostic decisions).

Information about tests and diagnostic procedures

As before, it remains difficult for laypeople to assess the quality of psychodiagnostic procedures and approaches. For this purpose, detailed information about the procedures is necessary, which usually has to be provided by the test developers in the form of manuals. The minimum standard of documentation is statements about

  • the theoretical foundations on which the procedure is based;
  • the individual development steps;
  • the review of the quality criteria;
  • detailed guidelines for implementation, evaluation and interpretation.

These manuals do not always exist, they are not always made freely available (although this is legitimate if the process has to be protected).

However, clients, especially those involved in larger projects, should insist on viewing these manuals. It may still be difficult for laypeople to assess the quality of the procedure. You can either call in experts to evaluate the process or look for reviews of these processes in the specialist literature . A number of scientific journals regularly publish such reviews, which are usually carried out independently of the test developers.

26 European publishers who publish tests have organized themselves into the European Test Publishers Group. The main objective is to promote high quality psychometric testing practice in Europe.

Web links

Individual evidence

  1. ↑ The ITC Guidelines on Test Use (PDF; 154 kB)
  2. ↑ German version of the International Guidelines for Test Application
  3. ↑ Overview page of the ITC guidelines
  4. ↑ Further explanations on the test protection of proficiency tests. ZTD Univ. Freiburg
  5. ↑ Test Board Homepage
  6. ↑ test assessment system TBS TK
  7. ↑ Test reviews according to TBS-TK
  8. ↑ DIN 33430 portal of the German Psychological Academy ( Memento of the original from August 14, 2015 in the Internet Archive ) Info: The archive link was inserted automatically and has not yet been checked. Please check the original and archive link according to the instructions and then remove this notice. @1@ 2Template: Webachiv / IABot / www.din33430portal.de
  9. ↑ ISO standard 10667.1 and 10667.2
  10. ↑ Critique of ISO 10667 at haufe.de
  11. ^ ITC Guidelines on Test Adaptation
  12. ↑ Guidelines for Computer and Internet Based Testing
  13. ↑ BAuA Toolbox
  14. ^ A b Nicola Döring, Jürgen Bortz: Research methods and evaluation in the social and human sciences . Springer-Verlag, 2015, ISBN 978-3-642-41089-5 , pp. 450 ( limited preview in Google Book Search).
  15. ^ European Test Publishers Group