Online survey

An online survey is an Internet-based survey method in empirical social , educational , market and opinion research . In contrast to a printed questionnaire , an online questionnaire is filled out in the web browser .

Technical implementation

The questionnaire is stored on a web server , either as static HTML or as part of a survey software that runs on the server. In order to fill out the questionnaire, the person to be interviewed must go to the relevant Internet address . Questions are displayed there that can usually be answered in an HTML form .

Today online surveys are almost exclusively carried out with the help of survey servers, so-called CAWI systems (Computer Assisted Web Interviewing). The technology for controlling such surveys was developed in the 1980s for computer-aided telephone surveys ( CATI systems ). The survey software presents the test person with the questionnaire page by page and immediately saves the answers entered on the server. Sophisticated programs also have a large number of control options. This includes:

Filter guidance (example: the respondent only sees question Y if he has entered 3 in question X)
Adjustment of questionnaire content during runtime (e.g. display of individualized texts in questions)
Question rotation or randomization
Rotation of question blocks
Quota control
Random assignment to experimental groups (in online experiments)

In addition, most programs have a number of other administrative functions, such as:

Online reporting: This component generates the project statistics and makes them available to the project manager or the client via the Internet. Under certain circumstances, essential results can already be viewed during the survey via the online reporting.
Access management: Access control based on the IP address, a password or an individual access key. In the latter case, each participant receives an individual access key and can only use this to access the questionnaire. Abandoned questionnaires can be resumed with the help of the access key.
Support of multilingual surveys: The possibility of realizing questionnaires in different languages.

The collection of data via the World Wide Web with the help of input forms has been technically possible since the publication of HTML 2.0 in July 1994. HTML is a markup language in which WWW documents are written. Version 2.0 contained the <form> element for the first time, with which input forms can be integrated into WWW documents. When exactly the first surveys took place in Germany via the WWW cannot be determined with any accuracy. However, it can be assumed that no data was collected via the WWW in Germany before 1995.

Methodological considerations

Recruitment of respondents

The respondents can be asked to participate, for example by e-mail, post or telephone. If a broad distribution is required or if only a special group of people is of interest, advertisements (banners, PopUp, PopIn, Layer) are placed or calls for participation are published on websites or in thematically relevant forums. In cases in which people are not contacted individually, they come across the request to participate more or less by chance and then decide for themselves whether they want to go through the questionnaire or not.

In particular, the surveys often found on websites, in which anyone can participate without special request, are often criticized for their scientific use. The respondents are not selected in a controlled manner as a random sample from a population, but decide themselves to participate ( self-selection ). Especially when it comes to controversial topics, there are also mass calls for participation in certain surveys in Internet forums and other websites in order to influence them in the desired way. The lack of representativeness of such selection procedures is generally criticized. A distinction must be made here, however, as to whether the population of those who read the call is identical to the population of interest (for example in a questionnaire on the quality of an individual website) or not (for example if the entire population of a country is to be examined). There is no question that self-selection is linked to certain personal characteristics about which little is known so far.

Multiple votes by individual participants cannot be prevented in principle. Protective measures (storage of the IP address , use of cookies , captchas , etc.) can be bypassed relatively easily, so that automated mass voting by individuals is possible. Multiple votes can only be reliably prevented if the respondents are contacted individually in advance and receive an individual access key. Nevertheless, the anonymity of the respondents can be guaranteed through suitable procedures.

In practice, multiple votes are only problematic to a limited extent, namely with votes. In most scientific surveys, people cannot achieve their own goals by answering questions. Therefore, hardly anyone will go to the trouble of answering questionnaires multiple times or even developing a technical solution for multiple answers.

New options for online surveys

The Internet as a survey medium offers the interviewer interesting options for developing questionnaires, some of which are reserved for personal interviews or which cannot be used in other forms of survey. A bonus is often offered to acquire participants via the Internet. This is usually paid out to the voluntary participant in the form of cash or vouchers for each completed online survey. As a result, websites on the Internet have specialized in the placement of participants and panels, creating a new branch in the area of online market research .

New types of questions

The online survey enables the use of question types that are not possible in a face-to-face survey or on paper or are associated with a lot of effort

Slider for stepless entry of the answer value (visual-analog scale)
Drag and drop to prioritize elements
Films, images and audio elements

Adaptive questionnaire management

In addition to adaptive questioning based on already recorded answers (filters), answers can also form new questions. For example, specifying a vehicle “ Auto XY ” enables it to be used in a further question: “How satisfied are you with your Auto XY ?” Using personal data, an individual approach is possible which, if used sensibly, can increase acceptance by the participants becomes. Unnecessary questions can be avoided.
In addition, in extensive rating studies in which the individual participants are only shown a subset of the stimuli to be evaluated, the sequence of stimuli can be balanced so that each stimulus is presented with the same frequency.

Automatic controls

Automatic controls that are easy to implement can help increase validity :

Avoidance of row position effects through item rotation
Plausibility check for open entries and answers
Completion control
Recording of the processing time (together with control questions enables the formation of a validity index)

However, the use of automated plausibility checks can also have disadvantages if, for example, people only want to view the questionnaire and enter irrelevant data in order to see the next pages. In this case, implausible or missing answers can make data cleansing much easier. Motivated respondents usually give plausible answers in well-designed questionnaires without automatic control. Often, it only makes sense to be compelled to fill in individual questions, for example if the respondents might overlook an item.

Advantages of the online survey

Online surveys can be carried out much faster than face-to-face surveys. With small samples (up to around 200 cases), telephone surveys and online surveys are about the same speed. With large samples, however, the results of online surveys are usually available more quickly.
Online surveys are significantly cheaper than face-to-face and telephone surveys. Exception: samples with a low incidence in online panels.
The interviewer influence and thus also the social influence on answering questions is eliminated.
There is no need to manually enter paper questionnaires, and entry errors cannot occur.
The data collected are immediately available on the server. As a rule, clear interim reports with the most important results can be generated at any time.
Multimedia surveys are possible: Pictures and films can be shown and e.g. B. Jingles are played.
The survey is computer-aided, which means that certain methods (such as conjoint measurement ) can be integrated relatively easily.
The effort for the internationalization of questionnaires is low, the participant can choose his own survey language
The interviewee decides when he can spare the time for the interview.
Special interest groups or geographically dispersed target groups are easy to address, offline this would mean enormous effort.
The questionnaire can be implemented as a program (technical variant 2). If you choose this variant, a logically consistent questionnaire run is ensured.
The data quality of data collected online is usually high.
The standardized data collected can also be compared in the long term.

Disadvantages of the online survey

Representativeness
- Especially in the pioneering phase of online research, people were simply recruited in large numbers via advertising banners in surveys. This procedure also leads to unrepresentative random sampling, as the content of the sites on which the advertising is advertised results in a preselection. In addition, the test subjects select themselves ( self-selection ). This problem has been solved by setting up large pools of respondents, so-called panels . Here only the institute carrying out the survey decides which panel participant will take part in a survey. The online panels cannot avoid a certain self-selectivity either, but panel research is now accepted as sufficiently representative in practice.

Another methodological problem with Internet surveys can arise from too high a proportion of interview dropouts. This reduces the use of the sample, which in turn reduces the representativeness of the results. That is why financial incentives are often created to fill out the questionnaire completely. The design of the questionnaire also has an impact on the dropout rate.

There is also the risk that individual respondents will go through the questionnaire several times . However, there are technical possibilities to counter this problem, the effect sometimes distorts the results only slightly or not at all.

The possible link between the address database and the access link (see section Technical implementation ) endangers the anonymity of the participants if this link is actually established by the interviewer. Knowledgeable target persons can be prevented from participating in the survey (see the section on Discussion of the methodological disadvantages )

There is a risk of the effect of social decontextualization : Due to the reduced orientation of the test subjects to social norms, which can be traced back to the online environment, social aspects are considered to be less important in online surveys. For this, individual characteristics, such as goals or wishes, are emphasized more strongly here.

Discussion of the methodological disadvantages

Regarding the representativeness of the sample: The accessibility of all elements of the sample is given or can be established in the following cases:
- If the subject of the survey is the Internet itself or requires Internet access.
- If the respondents are recruited by telephone or some other way that in principle can completely reach the population, but if these people do not have online access, they have to be interviewed using another survey method.
To draw the sample (avoidance of auto-selection): The drawing of the sample from the elements of the population must be actively carried out by the person carrying out the project. Only the drawn elements then have to be actively asked to participate. In order to avoid that other respondents show up by chance or by self-selection, the selected elements of the population must be privileged via a password or via a link that is only given to them individually.
Too few elements of the sample begin to fill out the questionnaire and / or too many break off the questionnaire): This problem can be alleviated by monetary and non-monetary incentives, by reminder e-mails or reminder calls. This is only possible if you have chosen a methodical and technical approach that allows access to the person to be interviewed. It must be taken into account here that the incentives can lead to a distortion.
For the same person to go through the questionnaire multiple times: This can be avoided by assigning a unique password to each element of the sample or a link to the questionnaire that can only be used once. In the event that the questionnaire is interrupted, there must then be a procedure that the respondent can access again to continue - but only to the questions that have not yet been answered. If the respondents are completely anonymous, multiple runs cannot be avoided. Some multiple answers can be sorted out in exceptional cases, for example by comparing IP addresses, or better by using cookies . However, IP addresses can provide incorrect results, as participants in a company, for example, can have an identical IP address, but are different subjects. When identifying through the use of cookies, it should be noted that a cookie only has a limited lifespan. However, the methodological weaknesses of this approach discussed above are no longer relevant to the problem of multiple passes.
Regarding the mistrust of the respondent because the questioning institution has access to the personal data: This can only be resolved through the reputable image of the respondent. However, this problem is not limited to online surveys: Even with telephone surveys and, of course, face-to-face surveys, the institution or person carrying out the interview is generally aware of the interviewed individual with essential personal data (name, address, telephone number).
Regarding the effect of social decontextualization: Research on this is still in its infancy, especially in Germany. Since this is a method effect, it cannot be prevented by taking precautionary measures in the research process. Depending on the question, however, it does not have too much of an effect or can even be helpful (for example, if you want to research your very personal wishes).

Two examples of online surveys

Employee surveys within companies and (global) corporations

Many of the discussed disadvantages of this type of survey are less pronounced or easier to control. The population can be clearly identified; Personnel lists can clearly indicate who belongs to the population and who does not. In most companies nowadays, all employees have the option of Internet or intranet access. The target population and the survey population (i.e. the number of objects in the population from which the sample is actually drawn) show a high degree of agreement. However, it also applies to online surveys in companies that the people to be interviewed are selected by means of appropriate sampling procedures. Surveys in which the participant himself decides whether to take part or not (self-selection) do not meet the requirements for a representative survey (see, for example, the standards for conducting online surveys of the German Market and Social Research Institute Association - ADM).

After all, by communicating and addressing employees, the company can assign a high priority to a survey and thus reduce the risk that selected employees will not take part or that the survey will be terminated prematurely. Other advantages come fully into play: The survey can be carried out within a short time all over the world at very low cost. Time of day and place do not matter, the costs of communication are usually already paid for by the already existing networking and internet presence.

Ad tests with CATI support

In this type of interview, a Computer Assisted Telephone Interview (CATI) and an online interview are carried out at the same time. Since the recruitment takes place via the CATI interview and the interviewer ensures that the interviewed person does not drop out prematurely, the main disadvantages of the online survey are eliminated. The CATI interviewer goes through the online questionnaire together with the respondent. Either he asks the questions and enters the answers in his CATI questionnaire or the person interviewed enters the answers in the online questionnaire himself. Mixed forms are also conceivable. This type of survey is often used when advertising motifs or packaging designs are to be assessed. It combines the advantages of direct contact from the telephone interview with the multimedia capacity of the Internet.

Risks of the online survey

The survey carried out online can have legal consequences for the survey creator. This applies particularly when the participants are invited via an external survey tool and when the associated transfer of participant data such as name, email address and other attributes (gender, age). In this case, Section 11 of the Federal Data Protection Act and Section 80 of Book 10 of the Social Code apply . The survey creator is obliged to enter into a contract data processing contract with the survey tool operator. However, this does not apply to operators outside the EU. In this case, other laws apply, such as the Safe Harbor Agreement with the USA up to the Safe Harbor judgment in 2015 . Another risk posed by online surveys concerns anonymity: the internet-based survey can lead to a large amount of data leading to the anonymity of the participants being removed. This can be done either directly (via the IP address) or indirectly (by cross-tabulating the metadata such as browser, operating system, device, start date).

Web links

@1@ 2( Page no longer available , search in web archives: Ethical aspects of online research ) (PDF file; 138 kB)
Practical instructions for creating online surveys (PDF file; 766 kB)
Brief introduction to the methodology of online examinations (PDF file; 118 kB)
Web Survey Methodology Portal (including international software overview)

literature

Don A. Dillman, Jolene D. Smyth, Leah Melani Christian: Internet, mail, and mixed-mode surveys: the tailored design method. Wiley, New York 2009, ISBN 978-0-471-69868-5 .
L. Kaczmirek: Human survey-interaction: Usability and nonresponse in online surveys. Halem, Cologne 2009, ISBN 978-3-938258-57-6 .
Nikolaus Jackob , Harald Schoen, Thomas Zerback (eds.): Social research on the Internet: methodology and practice of the online survey. VS Verlag, Wiesbaden 2009, ISBN 978-3-531-16071-9 .
Monika Taddicken: Method effects in web surveys. Restrictions on data quality due to a ‹reduced communication medium›. Halem Verlag, Cologne 2008, ISBN 978-3-938258-50-7 .
M. Welker, A. Werner, J. Scholz: Online Research. Market and Social Research Using the Internet. dpunkt.verlag, Heidelberg 2005, ISBN 3-89864-308-5 .

Individual evidence

↑ Exemplary: Chinese websites call for manipulation - hundreds of thousands of Chinese click tagesschau.de survey.

↑ F. Funke, U.-D. Reips: Data collection on the net: measurement methods and scales. In: M. Welker, O. Wenzel (Hrsg.): Online research 2007: Basics and case studies. Halem, Cologne 2007, pp. 52-76.

↑ MT Thielsch, S. Weltzin: Online surveys in practice. In: T. Brandenburg, MT Thielsch (Ed.): Practice of business psychology: Topics and case studies for study and practice. Monsenstein and Vannerdat , Münster 2009, pp. 69–85 (PDF)

↑ Birgit van Eimeren, Beate Frees: Results of the ARD / ZDF online study 2009: The Internet user 2009 - multimedia and totally networked? on: ard.de. In: Media Perspektiven . No. 7, 2009, pp. 334-348.

↑ SD Gosling, p Vazire, S. Srivastava, OP John: Should we trust web-based studies? A comparative analysis of six preconceptions about Internet questionnaires. In: American Psychologist . 59, 2, 2004, pp. 93-104.

^ S. Srivastava, OP John, SD Gosling, J. Potter: Development of personality in early and middle adulthood: Set like plaster or persistent change? In: Journal of Personality and Social Psychology. 84, 5, 2003, pp. 1041-1053.

↑ Standards for conducting online surveys (PDF file)

↑ W. Dzeyk: Ethical dimensions of online research. In: Cologne Psychological Studies. Volume VI, Issue, 1, 2001, pp. 1–30.

[1] Exemplary: Chinese websites call for manipulation - hundreds of thousands of Chinese click tagesschau.de survey.

[2] F. Funke, U.-D. Reips: Data collection on the net: measurement methods and scales. In: M. Welker, O. Wenzel (Hrsg.): Online research 2007: Basics and case studies. Halem, Cologne 2007, pp. 52-76.

[3] MT Thielsch, S. Weltzin: Online surveys in practice. In: T. Brandenburg, MT Thielsch (Ed.): Practice of business psychology: Topics and case studies for study and practice. Monsenstein and Vannerdat , Münster 2009, pp. 69–85 (PDF)