# Quantization deviation

Above: Original (blue) and quantized signal (red).
Bottom: The quantization deviation.

The quantization deviation or the quantization error is the deviation that arises during the quantization of analog quantities (e.g. during the analog-to-digital conversion ). While analog signals satisfy the range of values for real numbers , only discrete values ​​are used in digital representation . Therefore, there is rounding associated with the quantization, which causes a deviation.

## definition

Above: Borderline cases of a digital representation instead of an analog one. Both linearly graduated characteristics start at with . Below: the corresponding quantization deviation  . This drawing uses the definition used in various other disciplines: error or measurement deviation = output minus true or correct value.${\ displaystyle U = 0}$${\ displaystyle N = 0}$
${\ displaystyle F_ {q}}$

The quantization deviation is the difference between the input value and the quantized value : ${\ displaystyle e_ {q}}$${\ displaystyle x}$${\ displaystyle {\ hat {x}}}$

${\ displaystyle e_ {q} = x - {\ hat {x}}}$.

The "granular error" (from English granular error ) can generally not be larger than the associated quantization interval, otherwise it would lie in the adjacent interval.

${\ displaystyle | e_ {q} | \ leq \ Delta}$

In the case of a uniform quantization curve with an interval width , the quantization deviation with rounds to the nearest quantization level is always between ${\ displaystyle \ Delta}$

${\ displaystyle - {\ frac {\ Delta} {2}} .

In the adjacent picture you can see the deviation clearly from the distance between the stepped characteristic curve and the non-stepped inclined straight line. The step width corresponds to the step height of one digit (digit step on the lowest digit of the whole number ) . The vertical deviation in the picture on the right is in the range 0… –1 digit. If a zero arises as a quantized value, the lateral position of the step function is still undefined by (almost) a step width; the deviation can also be +1 digit… 0, as in the left picture, or anywhere in between, e.g. B. in the range -0.5 ... +0.5 digit. ${\ displaystyle N}$${\ displaystyle U_ {q}}$${\ displaystyle U = 0}$

## Examples

### Analog-to-digital converter

A 10-bit converter with a linear quantization characteristic, which resolves in = 2 10 = 1024 quantization steps, is to convert an electrical voltage in a range from 0 to 10.24 V. This requires a step size of . ${\ displaystyle N _ {\ text {max}}}$ ${\ displaystyle U}$${\ displaystyle U_ {q} = {\ frac {10 {,} 24 \, \ mathrm {V}} {2 ^ {10}}} = 10 \, \ mathrm {mV}}$

The levels are consecutively numbered with a number , where is. Then the value rounded by the graduation is ${\ displaystyle N}$${\ displaystyle 0 \ leq N \ leq N _ {\ text {max}} - 1}$

${\ displaystyle U _ {\ text {stepped}} = N \ cdot U_ {q}}$.

The value deviates from the true value by the quantization deviation ${\ displaystyle N \ cdot U_ {q}}$${\ displaystyle U}$

${\ displaystyle F_ {q} = N \ cdot U_ {q} -U}$.

### Digital multimeter

In measuring devices with a numeric display , the zero point cannot be adjusted within the width of a step of the characteristic curve using the zero display ( zero point deviation ). Summarized with the quantization deviation, a measuring device deviation of up to ± 1 digit step must be taken into account when reading a measured value, even with otherwise error-free operation.

The portion of the absolute error limit of a measuring device caused by the quantization is therefore a constant and amounts to or in the numerical image 1 digit. Occasionally, information relating to the full scale value is also used - ${\ displaystyle U_ {q}}$ ${\ displaystyle U _ {\ text {MB}}}$

Example: 0.05% FS if the measuring range is resolved from 0 to 2000 steps.${\ displaystyle U _ {\ text {MB}}}$

The associated relative error limit is and becomes smaller the larger the number . ${\ displaystyle 1 / N _ {\ text {max}}}$${\ displaystyle N _ {\ text {max}}}$

## Quantization noise

The deviation of the sample values from the original signal can be described as noise and, depending on the application, also has such an influence.

If some assumptions can be made about the input signal, the quantization deviation can also be modeled as a stochastic process - the quantization noise . It is assumed that the deviation is constantly evenly distributed , white , stationary and uncorrelated to the input signal. Furthermore, it is assumed for the input signal that it is free of mean values and stationary. This assumption is made in practice e.g. B. on speech or music and especially when a sufficiently high quantization resolution is used.

This noise signal is added to the input signal (instead of quantization) and then results in the quantized value:

${\ displaystyle {\ hat {x}} (t) = x (t) + e_ {q} (t)}$

In this way, the quantizer can be described and analyzed as an LTI system .

In order to keep the signal-to-noise ratio as high as possible for a signal, signals with smaller amplitudes are resolved more finely and larger amplitudes are more coarsely resolved, which is also referred to as non-linear quantization .

### Signal to (quantization) noise ratio

When specifying the signal-to-noise ratio , a fully modulated (normalized to 1) sinusoidal input signal is usually assumed. Its mean power is

${\ displaystyle P_ {A} = {\ frac {1} {2}} {\ hat {u}} ^ {2} = {\ frac {1} {2}}}$.

The mean power with a uniformly distributed quantization deviation corresponds to the variance of the uniform distribution :

${\ displaystyle \ sigma ^ {2} = {\ frac {q ^ {2}} {12}}}$.

This gives you

${\ displaystyle {\ text {SNR}} = {\ frac {P_ {A}} {\ sigma ^ {2}}} = {\ frac {1/2} {q ^ {2} / 12}} = { \ frac {6} {q ^ {2}}}}$

or as a logarithmic ratio (according to the notation as in)

${\ displaystyle Q _ {\ text {(SNR)}} = 10 \; \ lg {\ frac {P_ {A}} {\ sigma ^ {2}}} \; \ mathrm {dB}}$.

It is the width of quantization intervals for linear quantization curve; when coding with bits per sample, this is because here the quantization is from −1 to 1 (the sinusoidal input signal is normalized to 1). ${\ displaystyle q}$${\ displaystyle n}$${\ displaystyle q = 2 ^ {1-n}}$

When this is used, the most common formula is obtained:

${\ displaystyle Q _ {\ text {(SNR)}} = 10 \, \ lg (1 {,} 5 \ cdot 2 ^ {2 \ cdot n}) \, \ mathrm {dB} = n \ cdot 6 {, } 02 \, \ mathrm {dB + 1 {,} 76 \, dB}}$

A 16-bit AD converter with a sinusoidal input signal and full level would have a signal-to-noise ratio of 98.1 dB. It is essential that this calculation only delivers valid results under the above conditions and that this equation does not represent a generally valid solution for calculating the quantization noise. In the case of AD converters with a non-linear characteristic, as used, for example, in the A-law method in the field of telecommunications, the derived relationship of the quantization noise does not apply due to the non-linear transfer function.

If the rms value of the voltage of the signal in relation to the rms value of the noise is not used as a reference, but the peak-valley value of the voltage of the signal in relation to the rms value of the noise (common in video) applies

${\ displaystyle Q _ {\ text {(SNR)}} = n \ cdot \ mathrm {6 {,} 02 \, dB + 10 {,} 78 \, dB}}$

With real converters, the values ​​are reduced by additional deviations in the converter. Another aspect is that in practice the noise is often evaluated (e.g. DIN-A or CCIR-468) or band-limited (e.g. 0 Hz… 20 kHz).

### Increase in the SNR through oversampling

The SNR can be further increased by a combination of oversampling and low-pass filtering after the quantization - possibly additional noise shaping .

${\ displaystyle Q _ {\ text {(SNR)}} = n \ cdot 6 {,} 02 \, \ mathrm {dB} +1 {,} 76 \, \ mathrm {dB} +10 \, \ lg \ left ({\ frac {f _ {\ text {sample}}} {2 \ cdot B}} \ right)}$

( : Number of bits ,: sampling frequency,: bandwidth of the input signal) ${\ displaystyle n}$${\ displaystyle f _ {\ text {sample}}}$${\ displaystyle B}$

### Quantization noise for non-sinusoidal signals

If the quantization noise is to be determined not only for sinusoidal signals, the following generalized calculation for the quantization noise at full level can also be determined for any stationary signals and with a linear A / D converter:

${\ displaystyle Q _ {\ text {(SNR)}} = n \ cdot 6 {,} 02 \, \ mathrm {dB} +4 {,} 77 \, \ mathrm {dB} -20 \ cdot \ lg \ left ({\ frac {A _ {\ text {peak}}} {A _ {\ text {eff}}}} \ right)}$

It represents the peak value of the useful signal and the rms value . With a sinusoidal signal, the relationship between the peak value and the rms value is what leads to the above equation. ${\ displaystyle A _ {\ text {peak}}}$${\ displaystyle A _ {\ text {eff}}}$${\ displaystyle A _ {\ text {peak}} / A _ {\ text {eff}} = {\ sqrt {2}}}$

For typical audio signals such as music and speech, a factor of around 4 can be calculated as a good approximation as the relation between peak value and rms value. With otherwise the same parameters, the signal-to-noise ratio due to the quantization noise in a speech signal is around 9 dB worse than in a purely sinusoidal signal.

### Examples

Voltage curve for 1-bit (above) and 4-bit quantization.

The diagram shows the voltage curve for two signals. The lower one was quantized with 4 bits, corresponding to 16 different values. 1 bit with 2 different voltage values ​​was available for the upper signal.

The difference between the useful signal power and the noise power with 1-bit quantization is almost 8 dB in this example. It is above the noise threshold required for speech intelligibility. Even when scanning with only two different voltage values, language remains understandable. Even volume modulations remain recognizable.

Sound samples:

• Signal with 8 bit sampling, approx. 50 dB signal-to-noise ratio:
(Occam's razor)
• Signal with 4 bits, approx. 26 dB:
• Signal with 1 bit, approx. 8 dB:

## literature

• Alan V. Oppenheim, Ronald W. Schafer: Discrete-time signal processing. 3rd revised edition. Oldenbourg Verlag, Munich et al. 1999, ISBN 3-486-24145-1 .

## Individual evidence

1. HR. Tränkler: Pocket book of measurement technology. 2nd Edition. 1990, p. 127.
2. K. Bergmann: Electrical measurement technology. 6th edition. 2000, p. 24, p. 30.
3. ^ Karl Dirk Kammeyer, Kristian Kroschel: Digital signal processing: filtering and spectral analysis with MATLAB exercises. , 7th edition. 2009, p. 126.
4. ^ Bronstein-Semendjajew: Taschenbuch der Mathematik. 19th edition. 1979, p. 151.
5. DIN 1319-1 Basics of measurement technology; Basic concepts. 1995, No. 5.11.
6. John G. Proakis, Dimitris G. Manolakis: Digital Signal Processing . 3. Edition. Prentice Hall, 1996, ISBN 0-13-394289-9 , Chapter 9.2, pp. 751 ff .
7. ^ Roman Kuc: Introduction to Digital Signal Processing . BSP, Wiley, 1982, ISBN 81-7800-168-3 , pp. 395 f .
8. K.-D. Kammeyer: Communication. 3. Edition. Teubner, 2004, ISBN 3-519-26142-1 .
9. DIN EN 60027-3: 2007 Symbols for electrical engineering - logarithmic and related quantities in their units
10. Walt Kester: Taking the Mystery out of the Infamous Formula, "SNR = 6.02N + 1.76dB," and Why You Should Care. (Pdf) Analog Devices, 2009, p. 7 , accessed on March 10, 2014 (English) .