Demosaicing
Sensor image before demosaicing
Reconstructed image after demosaicing
As demosaicing (also Demosaicking ) is known in the digital photography of the reconstruction of a colored raster graphics from the brightness values of a superimposed with mosaic color filter image sensor .
Basics
The image sensors in most digital cameras are based on Bayer sensors . These are CMOS sensors that are coated with a matrix of regularly arranged red, green and blue filters . Since each pixel can only record the brightness value of one color channel , the color information is incomplete. To generate a colored raster graphic that stores complete RGB values for each pixel, neighboring basic color values must be interpolated .
In addition to Bayer sensors, there are also image sensors with RGBE (red, green, blue, cyan) or CYGM (cyan, yellow, green, magenta) filters, which also require demosaicing. With the Foveon X3 direct image sensors or three-shot sensors , the values of all three basic colors are recorded for each pixel; no demosaicing is necessary here.
The demosaicing can be carried out by the firmware of the camera, whereby a JPEG or TIFF image is usually generated, and can also be used later as software on an image in raw data format.
Procedures and Artifacts
A simple way of demosaicing is interpolation using a reconstruction filter , for example using bilinear filtering (see also scaling ). This can lead to blurring and other image artifacts :
- "Zipper-like" checkerboard patterns are created on edges that do not run along the color filters of a basic color;
- Color shifts arise as aliasing effects when the color filter matrix interferes with regularly arranged image details.
Numerous other demosaicing methods have been developed with the aim of reducing or eliminating these artifacts. This includes:
- Hue interpolation: Instead of interpolating RGB values individually, only the hue values, defined as , are interpolated. This method is one of the first to be used in commercial digital camera systems.
- Median filter: In the first step, an image is generated through bilinear filtering. A median filter is then applied to the differences in the color channels (e.g. R - G and B - G) in this image. The output image is then calculated from the two median-filtered color difference images and the original sensor image. The red value of a green sensor pixel is calculated, for example, by adding the value of the median-filtered R − G image to the green value.
- Edge-based interpolation: In the first step, an interpolated brightness image is generated from the green values of the sensor image. In the second step, an interpolated color difference image is generated from the red and blue values (R - G or B - G). A simple edge detection takes place, so that usually only two horizontally or vertically adjacent values are interpolated. The chrominance channels (R and B) are then reconstructed from these color difference images .
- Pattern matching and recognition: Techniques such as pattern matching or pattern recognition are used to reconstruct the missing color values.
literature
- Richard Szeliski: Computer Vision: Algorithms and Applications, p. 440 f. Springer, London 2011, ISBN 978-1-84882-934-3
Individual evidence
- ↑ Lanlan Chang, Yap-Peng Tan: Hybrid color filter array demosaicking for effective artifact suppression. Journal of Electronic Imaging 15, 1 (Jan.-March 2006): 013003, ISSN 1017-9909
- ^ Rajeev Ramanath et al.: Demosaicking methods for Bayer color arrays. Journal of Electronic Imaging 11, 3 (July 2002): 306–315 ( PDF, 560 kB ( Memento from April 20, 2006 in the Internet Archive ))
- ↑ Patent US4642678 : Signal processing method and apparatus for producing interpolated chrominance values in a sampled color image signal. Inventor: David R. Cok.
- ↑ Patent US4724395 : Median filter for reconstructing missing color samples. Inventor: William T. Freeman.
- ↑ See, inter alia, patent US5373322 : Apparatus and method for adaptively interpolating a full color image utilizing chrominance gradients. Inventors: Claude A. Laroche, Mark A. Prescott.
- ↑ See for example: W. XiaoLin, CK Wai, B. Paul: Color Restoration from Digital Camera Data by Pattern Matching. Proceedings of SPIE 3018 (1997): 12-17