Exploratory data analysis

The exploratory data analysis (EDA) or explorative statistics is a branch of statistics . She examines and assesses data of which there is little knowledge about their interrelationships. Many EDA techniques are used in data mining . They are also often taught in courses on statistics as an introduction to statistical thinking.

This designation was introduced by John W. Tukey in the 1970s. Tukey pointed out that too much emphasis in statistics is placed on evaluating and testing given hypotheses . He also suggested using data to obtain possible hypotheses, which are then tested. Indeed, the confusion of terms between the two types of analysis and applying them to the same amount of data can create a bias.

aims

The goals of exploratory statistics are:

• Making assumptions (hypotheses) about the cause and reason of the observed data
• Assess assumptions on what statistical inference can be based
• To support the selection of suitable statistical tools and techniques
• To provide a basis for further data collection through surveys or statistical test planning

Procedure

Basic graphical methods used in exploratory statistics are:

• 

  Box plot

• 

  histogram

• 

  QQ diagram

• 

  Scatter plot

• 

  Mosaic plot

• MultiVari Chart
• Run chart
• Pareto chart
• Stem-leaf diagram

Basic quantitative methods are:

• Median polish
• Letter values
• Resistant line
• Resistant smooth
• Rootogram

Special procedures

• Grand Tour
• Projection pursuit

software

• GeoDa - Free, user-friendly graphical user interface for the application of various methods of exploratory data analysis
• Ggobi - Free interactive multivariate visualization software with link to R .
• LiveGraph - Open Source Framework for Data Visualization and Exploratory Data Analysis (includes a real-time graph plotter).
• MANET - Free Mac Interactive EDA Software.
• Mondrian - Free Interactive Software for EDA.
• Orange ( Memento from January 10, 2011 in the Internet Archive ) - Free component-based software for an interactive EDA.
• PS Explore
• Visalix - Free interactive web application for data visualization and exploratory data analysis.
• openAnalyzer - data analysis software with modules for exploratory data analysis (free for university + students; free version limited to 5000 data sets).
• DataLab - full version as a free research license , free evaluation version is limited in the size of the data matrix
• Fathom 2 (dynamic stochastics and data analysis software) - free evaluation version can be run for one year but includes printing, storage and export restrictions.

literature

• Literature on exploratory data analysis in the catalog of the German National Library
• Hoaglin, DC; Mosteller, F & Tukey, JW (Eds) (1985) Exploring Data Tables, Trends and Shapes ISBN 0-471-09776-4
• Hoaglin, DC; Mosteller, F & Tukey, JW (Eds) (1983) Understanding Robust and Exploratory Data Analysis ISBN 0-471-09777-2
• Tukey, JW (1977) Exploratory Data Analysis ISBN 0-201-07616-0
• Velleman, PF & Hoaglin, DC (1981) Applications, Basics and Computing of Exploratory Data Analysis ISBN 0-87150-409-X
• Wolfgang Polasek: EDA Exploratory Data Analysis. Introduction to descriptive statistics (2nd edition). Springer-Verlag, Berlin 1994, ISBN 978-3-540-58394-3
• Siegfried Heiler, Paul Michels: Descriptive and exploratory data analysis. Oldenbourg, Munich 1994, ISBN 3-486-22786-6