Tone mapping

Tone mapping , tone reproduction or dynamic compression are synonymous terms that denote the compression of the dynamic range of high-contrast images (high dynamic range images), i.e. digital images with a high range of brightness. With tone mapping, the contrast range of a high-contrast image is reduced so that it can be displayed on conventional output devices.

Physiological background

In nature there is a dynamic range (ratio of the highest and lowest luminance ) of over 10 ⁹ : 1 when comparing sunlight with starlight. The dynamic range typically observed at a given point in time is on the order of 1: 10,000. Human visual perception solves the tone mapping problem because it is able to adapt to the prevailing brightness conditions . The eye reacts non-linearly to different absolute brightness conditions ( photopic , mesopic, scotopic).

Many tone mapping methods are based on the knowledge about human visual perception, since their aim is to calculate an image that appears as natural as possible. The most important role is played by the photoreceptors , the adaptation of which can be described according to the Naka-Rushton equation as follows:

{\ displaystyle {\ frac {R} {R _ {\ mathrm {max}}}} = {\ frac {I ^ {n}} {I ^ {n} + \ sigma _ {\ mathrm {b}} ^ { n}}}}

Here is the photoreceptor stimulus intensity, is the maximum stimulus intensity, is the light intensity and the light intensity that causes half the stimulus intensity at the prevailing background intensity. Several tone mapping techniques are based on an equation that is similar to this. ${\ displaystyle R}$ ${\ displaystyle R _ {\ mathrm {max}}}$ ${\ displaystyle I}$ ${\ displaystyle \ sigma _ {\ mathrm {b}}}$

Procedure

There are numerous tone mapping operators, but they can only be divided into a few fundamentally different classes. So-called global operators use a function that assigns a dynamically compressed value to each HDR value and which is applied to each pixel . In contrast to this, with local operators this function is varied for each pixel depending on the local adaptation level. Frequency-based operators use a fundamentally different technique in which the dynamic range of image regions is reduced depending on the spatial frequency . Finally, there are also gradient-based operators that weaken the brightness gradients of the output image for each pixel in order to generate the LDR image (image with a small range of brightness).

Many operators expect that the values of the output image are calibrated as luminance in a certain unit (cd / m²). This is because the non-linear perception of absolute brightness is taken into account; a daylight scene is therefore displayed differently than a night scene. However, it is often possible to reconstruct the original lighting conditions directly using the HDR image by evaluating the histogram . Most tone mapping processes largely ignore color perception and apply the new brightness value equally to all channels.

An HDR image after applying four different tone mapping operators (clockwise: Reinhard, Fattal, Mantiuk, Reinhard / Devlin)

Global operators

Global operators process the pixels of the original image independently of one another. They are faster than other methods and can often be carried out in real time. However, they are less suitable for scenes with a very large dynamic range, as they tend to lose details in very light and very dark areas.

Many global operators are based on adaptation models for which the background intensity must be known. This intensity can be estimated by calculating the arithmetic mean of the pixel values, but the geometric mean is the preferred method.

The simplest global operator calculates the values of the output image linearly down to the dynamic range of the LDR image. However, this method is inadequate because detail and contrast are lost.

Miller: The first global tone mapping operator was introduced by Miller and Hoffmann in 1984. The goal was to present rendered HDR images for architectural lighting design . For this purpose, the output image is first converted into perceived brightness values. The brightness ratios of different image areas should remain the same. This is achieved by normalizing the brightnesses and then converting them into luminance values. The brightness-luminance relationship is based on psychophysical data that are only valid up to approx. 1000 cd / m², so they are only suitable for interior scenes.
Tumblin-Rushmeier: Tumblin and Rushmeier's operators use the same psychophysical data as Miller's method. However, a slightly different brightness function is used than with Miller. In contrast to Miller, the Tumblin-Rushmeier operator does not try to obtain the brightness ratios, but rather the brightness values themselves.
Ward: Wards Operator is one of the tone mapping processes that do not focus on the perception of brightness, but rather try to maintain contrasts. The JND values perceived when viewing the initial image are related to the corresponding JNDs in the LDR image using a threshold-versus-intensity function (“threshold value versus intensity”). The output values are scaled with a constant factor determined in this way.
Ferwerda: The operator from Ferwerda et al. tries to relate JNDs. He also works linearly, but in contrast to Ward takes a scotopic component into account . The loss of sharpness in dark scenes is also taken into account by filtering image frequencies above a threshold value.
Drago: Dragos Operator takes advantage of the fact that the human visual system reacts to intensity logarithmically to a first approximation . The base of the logarithm is chosen differently for each pixel, so that lighter areas are compressed more.
Reinhard and Devlin: Outside a certain range, the stimulus strength of the visual system is no longer logarithmic. Reinhard and Devlin's operator is based on the results of electrophysiological experiments which suggest that photoreceptors respond to intensity according to a sigmoid function . The operator is applied to the different color channels individually.
Ward (Histogram Adjustment): Ward's histogram adjustment technique calculates a histogram from the logarithm of the pixel values. The tone mapping is based on the histogram calculated in this way, whereby care is taken that the contrast is maintained and that the visual perception is taken into account. In a post-processing step, aspects such as glare, color sensitivity and sharpness are taken into account. These factors make a big difference, especially with night scenes.
Silt: This method, called uniform rational quantization, uses a sigmoid-like function. The operator can be configured using two user-defined parameters, the effect of which, however, is difficult to estimate intuitively.
Reinhard: The method published by Reinhard and others is based on the tone mapping techniques used in photography, especially the zone system . As in modern photography, light regions in particular are compressed. There is also a local variant for this operator (see below).

Local and frequency based operators

Typical Halo artifacts in tone mapping with an outdated local operator

Local operators can process a large class of HDR images because they can represent a greater dynamic range without losing detail. They assume that the human perception of brightness does not adapt to the entire picture, but only to smaller regions.

To calculate the local brightness value for each pixel, a radial filter can be used which is applied to the neighboring pixels. However, this method leads to halo artifacts and contrast inversions near edges, since there are too great differences in brightness within the filter radius. There are several methods that can be used to work around this problem:

One possibility is to vary the filter radius. Starting from the value 1, the radius of the filter is doubled until the pixels of the edge falsify the result, i.e. when the new mean value deviates from the old one by a certain value.

Another option is bilateral filtering . Here, the radial filter is used not only to filter depending on the distance to the central pixel, but also depending on the absolute difference between the brightness values. Pixels whose values differ greatly from those of the central pixel have little influence on the result. Durand and Dorsey use Gaussian functions for both factors ; Pattanaik and Yee use a cylinder function for the radial factor and an exponential function for the brightness factor.

Bilateral filtering tends to soften abrupt changes in the brightness gradient . On the other hand, curved areas and regions with a high gradient are not sufficiently blurred. With trilateral filtering, Choudhury and Tumblin have presented an extension that also takes brightness gradients into account.

A number of low-pass filtered versions of the output image can be used to determine the optimal filter radius .

Frequency-based operators divide the output image into a filtered HDR image with low spatial frequencies and an unfiltered LDR image with high frequencies, which are then combined. However, the filtered image can also be interpreted in such a way that each pixel supplies a local adaptation value. It is therefore not always possible to clearly distinguish between local and frequency-based operators.

Chiu: Chiu et al. were the first to recognize the advantages of an operator that varies depending on the region of the image. Since their method only uses a simple Gaussian filter, strong halo effects are to be expected.
Rahman: Rahman and Jobson's work is based on the Retinex theory . The operator is applied to the color channels individually. The simple variant is similar to the Chius operator, but works in the logarithmic range. In the more complex variant, the Rahman operator calculates the LDR image from a weighted sum of different, differently blurred versions of the original image. The weighting of the images is controlled by a user-defined parameter.
Fairchild

Gradient Based Operators

This class of tone mapping operators calculates the gradients of the output image and attenuates them.

horn

comparison

Tone mapping operators differ in the speed, presence and strength of artifacts, retention of image details, and the ability to compress HDR images with very large dynamic range. Some studies have looked at the comparison of tone mapping techniques. The International Commission on Illumination has set up Working Committee TC8-08 to develop methods for validating tone mapping operators. When comparing different operators visually, the difficulty arises that changes to parameters can have a major impact on the result.

literature

Erik Reinhard u. a .: High Dynamic Range Imaging. Morgan Kaufman, San Francisco 2006, ISBN 0-12-585263-0

Web links

Evaluation of Tone Mapping Operators (English)
Erik Reinhard comment to unnatural tone-mapping parameter settings (English)

Individual evidence

↑ ^a ^b Reinhard et al: High Dynamic Range Imaging, p. 187

↑ Rafał Mantiuk et al: A Perceptual Framework for Contrast Processing of High Dynamic Range Images. In Proceedings of Second Symposium on Applied Perception in Graphics and Visualization 2005, pp. 87-94. ACM Press, New York 2005, ISBN 1-59593-139-2

↑ Reinhard et al: High Dynamic Range Imaging, p. 212

^ Gene Miller, C. Robert Hoffmann: Illumination and Reflection Maps: Simulated Objects in Simulated and Real Environments. In SIGGRAPH 84 Course Notes for Advanced Computer Graphics Animation. ( http://www.cs.berkeley.edu/~debevec/ReflectionMapping/illumap.pdf ( Memento from August 30, 2000 in the Internet Archive ) )

↑ John Tumblin, Holly Rushmeier: Tone Reproduction for Realistic Computer Generated Images. Technical Report GIT-GVU-91-13, Graphics, Visualization, and Usability Center, Georgia Institute of Technology 1991 ( Online )

↑ John Tumblin, Holly Rushmeier: Tone Reproduction for Computer Generated Images. IEEE Computer Graphics and Applications 13,6 (Nov. 1993): 42-48, ISSN 0272-1716

^ Greg Ward: Real Pixels. In James Avro (Ed.): Graphics Gems II, pp. 80-83. Academic Press, San Diego 1992, ISBN 0-12-286166-3

↑ James Ferwerda et al .: A Model of Visual Adaptation for Realistic Image Synthesis. In SIGGRAPH 96 Conference Proceedings, pp. 249-258. ACM, New York 1996, ISBN 0-89791-746-4 ( PDF, 570 KB ( Memento of June 13, 2007 in the Internet Archive )) archived from the original www.graphics.cornell.edu/~jaf/publications/sig96_paper.pdf

↑ Frédéric Drago et al: Adaptive Logarithmic Mapping for Displaying High Contrast Scenes. Computer Graphics Forum 22, 3 (2003): 419-426, ISSN 0167-7055 ( online )

^ E. Reinhard, K. Devlin: Dynamic Range Reduction Inspired by Photoreceptor Physiology. IEEE Transactions on Visualization and Computer Graphics 11, 1 (Jan./Feb. 2005): 13–24, ISSN 1077-2626 ( PDF, 5.5 MB ( Memento of the original from October 1, 2008 in the Internet Archive ) Info: Der Archive link was automatically inserted and not yet checked. Please check the original and archive link according to the instructions and then remove this note. ) @1@ 2

^ Gregory Ward Larson: A Visibility Matching Tone Reproduction Operator for High Dynamic Range Scenes. IEEE Transactions on Visualization and Computer Graphics 3, 4 (Oct.-Dec. 1997): 291–306 ( PDF, 880 KB )

↑ Christophe Schlick: Quantization Techniques for the Visualization of High Dynamic Range Pictures. In Georgios Sakas et al. (Ed.): Photorealistic Rendering Techniques, pp. 7-20. Springer, New York 1995, ISBN 3-540-58475-7

↑ ^a ^b ^c ^d Erik Reinhard et al: Photographic Tone Reproduction for Digital Images. ACM Transactions on Graphics 21, 3 (Jul. 2002): 267-276, ISSN 0730-0301 ( online )

↑ ^a ^b ^c Michael Ashikhmin: A Tone Mapping Algorithm for High Contrast Images. In Proceedings of 13th Eurographics Workshop on Rendering, pp. 145-155. Eurographics, Aire-la-Ville 2002, ISBN 1-58113-534-3

↑ ^a ^b Frédo Durand, Julie Dorsey: Fast Bilateral Filtering for the Display of High-dynamic-Range Images. ACM Transactions on Graphics, 21, 3 (2002): 257-266 ( online )

↑ ^a ^b Sumanta Pattanaik, Hector Yee: Adaptive Gain Control for High Dynamic Range Image Display. In Proceedings of the Spring Conference on Computer Graphics 2002, pp. 83-87. ACM, New York 2002 ( PDF, 140 KB )

↑ ^a ^b Prasun Choudhury, Jack Tumblin: The Trilateral Filter for High Contrast Images and Meshes. In Proceedings of the Eurographics Symposium on Rendering 2003, pp. 186-196. ( PDF, 2.0 MB )

↑ K. Chiu et al: Spatially Nonuniform Scaling Functions for High Contrast Images. In Proceedings of Graphics Interface '93, pp. 245-253. Toronto 1993 ( PDF, 420 KB )

↑ Zia-ur Rahman et al: A Multiscale Retinex for Color Rendition and Dynamic Range Compression. In SPIE Proceedings: Applications of Digital Image Processing XIX, Vol. 2847. SPIE, Denver 1996 ( Online )

↑ Mark Fairchild, Garrett Johnson: Meet iCAM: An Image Color Appearance Model. In IS & T / SID 11th Color Imaging Conference, pp. 36-41. IS&T, Scottsdale 2003 ( online )

↑ Sumanta Pattanaik et al: A Multiscale Model of Adaptation and Spatial Vision for Realistic Image Display. In SIGGRAPH 98 Conference Proceedings, pp. 287-298. ACM, New York 1998, ISBN 0-89791-999-8 ( PDF, 1.5 MB )

↑ Hector Yee, Sumanta Pattanaik: Segmentation and Adaptive Assimilation for Detail-Preserving Display of High-dynamic Range Images. The Visual Computer 19, 7–8 (2003): 457–466, ISSN 0178-2789 ( PDF, 220 KB ( Memento of the original from June 7, 2010 in the Internet Archive ) Info: The archive link was inserted automatically and has not yet been checked. Please check the original and archive link according to the instructions and then remove this note. ) @1@ 2

^ A. Oppenheim et al.: Nonlinear Filtering of Multiplied and Convolved Signals. IEEE Transactions on Audio and Electroacoustics 16, 3 (Sep. 1968): 437-466, ISSN 0018-9278

↑ Jiangtao Kuang et al: iCAM06: A refined image appearance model for HDR image rendering. Journal of Visual Communication and Image Representation 18, 5 (Oct. 2007): 406–414, ISSN 1047-3203 ( online )

^ Berthold Horn: Determining Lightness from an Image. Computer Graphics and Image Processing 3 (1974): 277-299, ISSN 1530-1834 ( PDF, 1.1 MB )

^ Raanan Fattal et al .: Gradient Domain High Dynamic Range Compression. ACM Transactions on Graphics 21, 3 (Jul. 2002): 249-256 ( online )

↑ Frédéric Drago et al .: Perceptual Evaluation of Tone Mapping Operators with Regard to Similarity and Preference. Technical Report MPI-I-2002-4-002, Max Planck Institute for Computer Science 2002 ( PDF, 2.0 MB )

↑ Jiangtao Kuang et al: Testing HDR Image Rendering Algorithms. In Proceedings of IS & T / SID 12th Color Imaging Conference. IS&T, Scottsdale 2004

↑ Akiko Yoshida et al: Perceptual Evaluation of Tone Mapping Operators with Real-World Scenes. In Human Vision and Electronic Imaging X, IS & T / SPIE's 17th Annual Symposium on Electronic Imaging, pp. 192–203 SPIE, San Jose 2005 ( PDF, 480 KB )

^ Garrett Johnson: Cares and Concerns of CIE TC8-08: Spatial Appearance Modeling and HDR Imaging. In SPIE / IS & T Electronic Imaging Conference. IS&T, San Jose 2005 ( PDF, 2.1 MB )

[reinhard-187-1] Reinhard et al: High Dynamic Range Imaging, p. 187

[2] Rafał Mantiuk et al: A Perceptual Framework for Contrast Processing of High Dynamic Range Images. In Proceedings of Second Symposium on Applied Perception in Graphics and Visualization 2005, pp. 87-94. ACM Press, New York 2005, ISBN 1-59593-139-2

[3] Reinhard et al: High Dynamic Range Imaging, p. 212

[4] Gene Miller, C. Robert Hoffmann: Illumination and Reflection Maps: Simulated Objects in Simulated and Real Environments. In SIGGRAPH 84 Course Notes for Advanced Computer Graphics Animation. ( http://www.cs.berkeley.edu/~debevec/ReflectionMapping/illumap.pdf ( Memento from August 30, 2000 in the Internet Archive ) )

[5] John Tumblin, Holly Rushmeier: Tone Reproduction for Realistic Computer Generated Images. Technical Report GIT-GVU-91-13, Graphics, Visualization, and Usability Center, Georgia Institute of Technology 1991 ( Online )

[6] John Tumblin, Holly Rushmeier: Tone Reproduction for Computer Generated Images. IEEE Computer Graphics and Applications 13,6 (Nov. 1993): 42-48, ISSN 0272-1716

[7] Greg Ward: Real Pixels. In James Avro (Ed.): Graphics Gems II, pp. 80-83. Academic Press, San Diego 1992, ISBN 0-12-286166-3

[8] James Ferwerda et al .: A Model of Visual Adaptation for Realistic Image Synthesis. In SIGGRAPH 96 Conference Proceedings, pp. 249-258. ACM, New York 1996, ISBN 0-89791-746-4 ( PDF, 570 KB ( Memento of June 13, 2007 in the Internet Archive )) archived from the original www.graphics.cornell.edu/~jaf/publications/sig96_paper.pdf

[9] Frédéric Drago et al: Adaptive Logarithmic Mapping for Displaying High Contrast Scenes. Computer Graphics Forum 22, 3 (2003): 419-426, ISSN 0167-7055 ( online )