Goal Question Metric

from Wikipedia, the free encyclopedia

Goal Question Metric ( GQM ) is a systematic procedure for creating specific quality models in the field of software development. This can be represented as a tree structure . As a root is the goal ( goal ), which via the node ( Questions ) to the leaves ( Metric is refined). In this way, questions and software measures are derived. The measured values ​​are interpreted on the way from the leaves to the roots.

Suitable metrics can be identified in software metrics by answering the following questions:

  • Which goal should be achieved by the measurement? ( Goal )
  • What should be measured or which questions should the measurement answer? ( Question )
  • Which metric is / which metrics are able to describe the necessary properties? ( Metric )

history

"How can you decide what to measure in order to achieve your goals?"

The inventors of the Goal Question Metric model, Victor R. Basili and David M. Weiss , faced precisely such a problem when they worked on various flight dynamics in the vicinity of the software engineering laboratory, NASA's Goddard Space Flight Center Projects were working. The definition of goals helped them to concentrate on the essentials and important. The writing down of questions made it easier to specify the goals more precisely and at the same time relevant metrics could be derived. GQM was born and quickly established itself as the quality model at the Goddard Space Flight Center.

Goal-Question-Metric-Model

The GQM model describes the procedure for creating a quality model, whereby the model can be divided into six steps:

  1. Characterization of the company and project environment, recording of the mission statement, definition of measurement goals
  2. Formulating questions to define the goals more precisely
  3. Identify measurement targets and derive metrics
  4. Development of mechanisms for data collection
  5. Collect, analyze and interpret data
  6. Summarize and apply experiences

In short: "The GQM process begins with the characterization of the organizational and project environment. Taking into account the environment, information needs are recorded in the second step using goals and corresponding questions. Then, in the third step, measurements are documented that serve to quantify these information needs In the fifth and fifth step, the measurements are carried out and the resulting data interpreted. Finally, in the sixth step, there is follow-up work. For example, the quality planning and the knowledge gained are secured. "

However, the literature does not entirely agree on the division of the six steps. Therefore, when researching GQM you come across a wide variety of classifications and various different names. The only agreement in the literature is that there are six steps and that the procedures on which they are based are always applied in the same order. The first three steps of the GQM model are often referred to as the definition phase. In this phase, the goals, questions and metrics that give the GQM model its name are determined.

Identify goals, what we want to achieve; Questions, if answered, tell us whether we are achieving the goals or help us to understand and interpret them; and the metrics identify the measurements that are necessary to answer the questions and quantify the goals. The goals, questions and metrics with their associated dimensions, diagrams and other expansions are summarized as a GQM plan. Several questions can be assigned to one goal and one question to several metrics. In other words: The relationship between "goals and questions" and "questions and metrics" is both 1 to n. The entire GQM model is defined "top-down", i.e. H. it is a goal-oriented approach that defines questions and metrics downwards from the goals. The analysis and interpretation then takes place "bottom-up", i. H. from bottom to top.

Step 1: Characterization of the company and project environment, recording of the mission statement, definition of measurement goals

This first step is about defining GQM goals. There are two types of GQM goals:

  • Business goals
  • Measurement goals

Business goals, also known as mission statements, are the overarching model of a company and define its long-term goals. A company always defines business goals. These must be identified and used as a focus for creating measurement targets. Without business goals, the entire measurement would have no direction. As a result, business goals drive the identification of measurement goals, whereby business goals and measurement goals do not necessarily have to differ from one another, which is why we will only speak of GQM goals in the following.

Every GQM goal should be expressed in five terms:

  • Object of measurement
  • purpose
  • Quality focus
  • Point of view
  • context

Define the measurement object

Specifying the object of measurement means finding out what you want to find out more about: “What do you want to measure?” GQM allows you to examine a wide variety of objects. In general terms, these could be processes, products, resources, projects, etc., to name just a few examples.

Establishing the purpose

Determining the purpose determines what is to be achieved. Examples of this would be characterization, improvement, monitoring, evaluation, prediction or optimization.

Setting the quality focus

Define quality focus: The quality focus is an element of the measurement objective. The following examples show a wide range of quality attributes:

  • Reliability / availability
  • Usability
  • safety
  • functional safety
  • Scalability
  • Maintainability
  • Adaptability
  • Time to market
  • performance
  • Efficiency
  • portability
  • Interface compatibility

Set the viewing angle

Define perspective: This is the perspective from which the measurement object is viewed, which is usually the customer, developer, tester, project manager or contractor.

Set the context

Define context: Here it is determined in which context the measurement object is. The context should be explained as precisely as possible, whereby it is advisable to provide information about the company, department, project and period.

Memorandum

The definition of the GQM goals can be presented with the following auxiliary sentence: 

Analysiere den Entwicklungsprozess
zum Zwecke der Änderung
in Bezug auf Korrektheit
vom Blickwinkel des Entwicklers
im Kontext des Projekts X.

GQM plan

Collecting the targets can be done in different ways. The most common methods are workshops and interviews. It is important here that goals are raised from the most varied of those involved in the process in order to obtain the broadest possible spectrum. The goals themselves are documented in a so-called abstraction sheet. Often several abstraction sheets are recorded in a GQM plan, which are then given priorities before further processing. The GQM plan is the Goal Question Metric documentation tool. The structure is simple. It begins with a thematic introduction to match the defined goal. Then questions are written down with the associated metrics. The following template can help in creating a GQM plan. 

Q.1 Frage1
M.1.1 Metrik1
M.1.2 Metrik2
Q.2 Frage2
M.2.1 Metrik1
M.2.2 ...
Q.3 ...

Abstraction Sheets

Abstraction sheets are used to assign important information to a GQM goal or to group them with the GQM goal. Several abstraction sheets can exist in parallel. The sheet is divided into five parts:

  • The five aspects (object of measurement, purpose, quality focus, perspective, context) express the GQM goal to which the sheet belongs.
  • Quality factors relate directly to the GQM goal. To find out, ask yourself the question: "What factors need to be considered in this GQM objective and which are influencing it?"
  • The hypothesis is the estimated or expected answer to the defined quality factors.
  • Influencing factors in turn relate directly to the quality factors: "What influences the quality factors?"
  • Influences on the hypotheses are recorded in the last section of the sheet: "How do the influencing factors affect the hypotheses?"

Step 2: Formulating questions to define the goals more precisely

In this phase of the model, questions are asked that are derived from the abstraction sheets. A question can usually also be formulated for each recorded component of an abstraction sheet. Answering these questions helps to get closer to the goal. To define questions, one should first determine what one would like to learn about the properties in the quality focus of the target. The following points should be observed here:

  • Conformity with the goal
    The questions must only reflect the purpose and point of view.
  • Quantifiability
    The questions must be formulated in such a way that they can be answered using measured values. So not “Which test cases would be good in this context?” But “Which test cases found the most errors?”. More complicated questions should be broken down into several simple questions.
  • Meaning of the answer
    It has to be clarified what can be done with the answer to this question and what possible measures are to achieve the corresponding goal. If the defined question does not lead to improved actions, the need for the question should be reconsidered.
  • Understandable charts
    It must be clear what is shown on the axes of the chart, what analysis precedes the chart and what data has been excluded.
  • Use of known diagram
    types When selecting the diagram types, known diagrams (suitable for the question type) should be used whenever possible.
  • Costs / benefits
    It has to be evaluated whether the benefit expected from answering the question outweighs the costs of data collection.
  • Clarity of terms
    It should be ensured that terms such as quality, effectiveness, efficiency or reliability are precisely defined.
  • Reasonable number of questions
    Experience from successfully implemented measurement programs shows that three to seven questions should be defined per goal.
  • Good relationship between quality and influencing factors
    The question "What influences this quality aspect" should be used to check whether there are important influences. However, no more than about three times as many influencing factors as quality factors should be defined, since one is ultimately interested in the latter.

Step 3: Identify measurement goals and derive metrics

This step builds on the previous one and tries to derive metrics with the help of the previously worked out GQM goals. It is particularly important to find the right metric for the question asked, which often turns out to be easier than previously thought. The following measures or metrics and the resulting measures turn out to be particularly suitable:

Step 4: Develop data collection mechanisms

This step describes how the data and measurements are to be recorded that are necessary for the GQM plan. One also speaks of the creation of a measurement plan. The following points should be covered with the help of the measurement plan:

  • Formal definition of measurements
  • Written drafting of the measurements
  • All output values ​​of the measurements
  • The person or role taking the measurements
  • The time when the measurements are taken
  • The medium (tool or questionnaire) used to take the measurements

For the entire data collection and measurement, people who carry out measurements or collect data must be specially trained. This is necessary so that the data collection is valid, uniform and comparable. After this step, raw data, also known as primary data, are available. They are given this name because they have not yet been processed in any way and are not yet representative in this form.

Measurement plan

The measurement plan contains how raw data are collected most efficiently and to whom they are passed on for processing. It is defined by the metrics that were previously selected. It deals with the procedure during the measurement, which raw data must be collected and how.

Step 5: collect, analyze and interpret data

In this step, the raw data previously defined in the measurement plan are collected. This step now describes how the raw data is recorded. The main work is not the collection of the raw data, but the analysis and interpretation of the raw data, so that at the end of this step one receives processed data, so-called secondary data. You can collect raw data using various methods:

Questionnaires

Questionnaires can be queried semi-automatically via a web frontend in the Internet or Intranet or in the traditional way using paper sheets. Ideally, questionnaires can be used to record experiences, perceptions and feelings.

Data Sheets

Data sheets can also be queried semi-automatically or in the classic way. The difference to questionnaires is that they are more static and technical. So you can record technical dimensions much more precisely.

Interviews

Interviews are meetings with one or more people. Interviews are more dynamic than previous methods, as additional input can be collected in addition to the given questions. The effort is greater, but it can be worthwhile.

observation

Observation means that a specially trained person (the observer) takes part in the development process in the company. The observer observes a defined process or an action of a process such as the creation of reports or teamwork. The observation can take from a few weeks to years, depending on the size of the process. The results of this method are recorded in a logbook.

Automated tools

Automated tools can't always be used, but they make the job easier and faster. In particular, the automatic data collection is used in all processes whose output (e.g. source code) is already available in electronic form.

Validation

Regardless of the method used, the recorded raw data must be validated and processed before they can be used for analysis. In this context, validation means not only checking the raw data for correctness, but also checking for completeness and consistency. It happens again and again that there are outliers in the raw data. Outliers are values ​​that differ greatly from the other values ​​surveyed. These must be tracked down, i. d. Usually automated, and removed, as they would otherwise falsify the measurement result.

analysis

The analysis section begins as soon as valid raw data is available. The data analysis is necessary to obtain measurement results. The validated and analyzed measurement results then in turn help to drive the measurement goals (or GQM goals) and initiate improvements. Here you can also see the bottom-up interpretation of the GQM model very well.

Step 6: summarize and apply experiences

In this last step of the GQM model, all previously obtained results are bundled together with a description. Again, there are two types of result bundling:

Measurement results

Measurement results in the form of validated and analyzed data are available when the third step is completed. These are summarized again here and are available for further processing.

Lessons learned

Lessons Learned is a condensed collection of experiences, developments, notes, errors, risks, etc. that were made during a process. They are bundled in lessons learned so that they can be used again at a later point in time in a similar situation or for the same problem. In contrast to the measurement results, lessons learned are more of a qualitative evaluation.

Summary

The Goal Question Metric Model is a quality model for securing and improving the quality of processes, which was developed by Dr. Victor R. Basili was developed. GQM is not specialized in a specific process, but can be used in a variety of ways. It helps to analyze technical as well as personal processes. It is also a goal-oriented model that is structured in six steps. The first three steps are also called the “definition phase”. In the definition phase, the goals that are to be achieved are first defined. Questions help to better specify the goals and metrics are defined to make the process measurable. The last three steps are also known as the “interpretation phase”. In this phase, a plan is drawn up that describes how the necessary measurement data for the metrics defined in the definition phase are obtained. With the help of the plan, the raw data is recorded, then validated and analyzed. Finally, the results are recorded in order to be able to apply them again and to initiate improvement measures. A compilation of the results in the form of “lessons learned” is also a common measure.

literature

  • VR Basili, HD Rombach: The TAME project. Towards improvement-oriented software environments . In: IEEE Transactions on Software Engineering . tape 14 , no. 6 , 1988, pp. 758-773 , doi : 10.1109 / 32.6156 .
  • Reiner Dumke: Software engineering. An introduction for computer scientists and engineers: systems, experiences, methods, tools . tape 4 , 2003, p. 234 .
  • Helmut Balzert: Textbook of software technology: software management . Spectrum Academic Publishing House, 2008, ISBN 978-3-8274-1161-7
  • Rini van Solingen and Egon Berghout: Goal / Question / Metric Method . McGraw-Hill Publishing Company, 1999, ISBN 978-0-07-709553-6
  • Christ Ebert, Reiner Dumke: Software metrics in practice . Springer-Verlag Berlin Heidelberg New York, 1996, ISBN 3-540-60372-7 , pp. 28-34 .

Web links