Plenoptic camera
A plenoptic camera , also known as a light field camera , records, in addition to the usual 2 image dimensions, another, namely the direction of incident light rays. Due to the additional dimension, plenoptic recordings contain information about the image depth.
The particular advantage of plenoptic cameras lies in the theoretically infinite depth of field and the possibility of refocusing, i.e. the subsequent shifting of the focus plane in the object space ( focus variation ). Thanks to the additional depth information, a plenoptic camera can also be used as a 3D camera .
It is crucial for the function of plenoptic cameras that the same scene is recorded from several angles. It is irrelevant whether an array of conventional cameras captures the scene or whether the different viewing angles are generated within the camera - e.g. B. by an array of microlenses in front of the image sensor .
The development of plenoptic systems is based on theoretical considerations by the physicist Gabriel Lippmann , who presented the concept in 1908.
Plenoptic function
The distribution of the radiance along light rays in an area of three-dimensional space, which is caused by static light sources that cannot be changed over time, is called the plenoptic function. The plenoptic function is an idealized function that is used in image processing and computer graphics to describe an image at a specific point in time from any position from any angle, i.e. regardless of the position of the viewer and camera parameters such as aperture and distance setting .
In practice, the plenoptic function is not used, but it is useful to understand various other concepts of image processing and computer graphics. A straight line (ray) is described by a point on the ray (three coordinate values) and the direction of the ray (two angles). Since the point can be translated along the ray, its three coordinates effectively represent only two degrees of freedom. Hence the plenoptic function is four-dimensional. Wavelength , angle of polarization and time can possibly be viewed as additional variables, adding additional dimensions.
Plenoptic camera
Each pixel is refracted again by the lens grid and expanded into a cone that meets the sensor surface in a circle. This reveals the direction from which the light beam originally came: A perpendicularly incident light beam lands in the center of the circle, a diagonally incident one further along the edge. Software can be used to recalculate the sharpness and the focal point can be changed like a conventional lens. The information from a scene must be mapped onto several pixels of the camera chip so that the information about the direction of the incident light beam can be used. Therefore, this method is accompanied by a reduction in the effective resolution of the camera sensor.
Implementations
Stanford University
A team at Stanford University uses a 16 megapixel camera with a 90,000 microlens array, which means that each microlens exposes around 175 pixels.
Adobe
A design by the American company Adobe uses 19 lenses in a circular arrangement and exposes a 100 megapixel sensor so that each image has a resolution of around 5 megapixels.
Raytrix
A plenoptic camera for use in industry and research, which has been sold commercially by the German company Raytrix since 2010, uses a lenticular grid made up of three different lenses that differ in their focal length. Each microlens covers three to six pixels on the camera sensor. As a result, the effective resolution of the camera is only three to six times lower than the resolution of the sensor chip.
Lytro
The American company Lytro began selling a light field camera for the consumer sector in April 2012. This model has a sevenfold optical zoom range, a fixed, continuous f-number of 2.0, a touch-sensitive screen and an internal memory of 8 or 16 Giga byte for color still images. The effective image resolution with a file size of around 20 megabytes is 540 by 540 pixels (this corresponds to 0.29 megapixels), which is suitable for demonstration purposes, but not for high-quality reproductions. With around 11 million registered pixels, this results in a ratio of six times six (36) light rays per pixel.
In April 2014, the successor model Illum was presented, which is similar to a classic camera. It should have a higher image resolution of a maximum of 4 MP for 2D image files.
In April 2016, the Lytro company announced that it wanted to withdraw from the end customer business for light field cameras due to a lack of sales opportunities and to concentrate on the professional TV and video market from now on. The new light field camera Lytro Cinema enables a resolution of 755 megapixels, a dynamic range of 16 f-stops, 40k videos and 300 FPS. The company announced on March 28, 2018 that it would cease operations.
Sample recordings
A close up of a keyboard with a plenoptic camera with reproductions at four different distance settings:
A recording of a keyboard with a plenoptic camera with reproductions at five different distance settings:
Pelican Imaging
The company Pelican Imaging announced in May 2013 that it would be launching a flat light field camera integrated in a smartphone with a matrix of four by four lenses together with the mobile phone provider Nokia .
HTC
In addition to the built-in 4 MP image chip, the HTC One M8 uses a second camera chip to record the depth information to simulate the functional principle. With this function called U-focus, it is also possible to change depth of field and focus areas at a later date.
Google's camera app Google Camera , which has been available for Android devices since April 2014, enables the subsequent setting and editing of depth of field in photos in the “Lens Blur” (“focus effect”) photo mode. No additional sensors or optics are required for this. Instead, a 3D positioning of the elements in the image is calculated from the automatic multiple recordings required for this, which then allows subsequent processing.
K | Lens
The Saarbrücken company K | Lens is developing a plenoptic interchangeable lens for standard DSLRs and system cameras. Research from the Max Planck Institute for Computer Science forms the basis for this . In contrast to other plenoptic processes, K | Lens uses the kaleidoscope effect instead of microlenses to scan the light field.
Web links
- Plenoptic cameras , Wikibook digital imaging procedures , chapter image acquisition
- The camera of the future. Adopt all perspectives when filming in the Lichtfeld 3Sat : nano , broadcast on April 30, 2019
- Plenopticam , open source application for image processing of light field recordings (e.g. Lytro images)
Individual evidence
- ↑ G. Lippmann: Épreuves réversibles donnant la sensation du relief . In: Journal de Physique Théorique et Appliquée , vol. 7 (1908) , p. 821
- ^ Edward H. Adelson, James R. Bergen: The plenoptic function and the elements of early vision . In: M. Landy, J. A. Movshon (Eds.): Computation Models of Visual Processing . MIT Press, Cambridge 1991, ISBN 0-262-12155-7 , pp. 3–20 ( PDF [accessed October 23, 2011]).
- ^ Ren Ng, Marc Levoy, Mathieu Brédif, Gene Duval, Mark Horowitz, Pat Hanrahan: Light Field Photography with a Hand-held Plenoptic Camera . In: Stanford Tech Report CTSR 2005-02 . 2005 ( PDF [accessed on October 23, 2011] report on the creation and subsequent processing of plenoptic camera images).
- ↑ Video: Light Field Photography with a Hand-held Plenoptic Camera. ( WMV ; 9.3 MB)
- ↑ Jonathon Keats, Kris Holland, Gary McLeod: PopSci's How It Works - 100 Megapixel Camera. In: Popsci.com. Archived from the original on January 17, 2008 ; accessed on October 23, 2011 (English).
- ↑ Christian Perwaß, Lennart Wietzke: The Next Generation of Photography: An Introduction to Light Field Photography. (PDF) (No longer available online.) Raytrix GmbH, January 2010, archived from the original on October 27, 2011 ; accessed on February 4, 2014 .
- ↑ lytro.com ( Memento from November 4, 2011 in the Internet Archive )
- ↑ Scharfmacher - Lytro Lightfield Camera in the test . heise.de
- ↑ Lytro light field camera , test.de from June 28, 2012, accessed November 19, 2012
- ↑ Lytro light field camera: Harbinger of the Revolution , test.de of June 28, 2012, accessed November 19, 2012
- ↑ Lytro - Camera Specs - Light field resolution ( Memento November 20, 2012 in the Internet Archive ) Lytro, accessed November 20, 2012
- ↑ Lytro Illum camera lets users refocus blurred photos after shooting . theguardian.com, April 22, 2014, accessed April 23, 2014
- ↑ Technical data Lytro Illum ( Memento from April 24, 2014 in the Internet Archive ), accessed on April 23, 2014
- ↑ Christoph Jehle: Lytro gives up end customer business for light field cameras. In: Heise Online. Heise Medien, April 7, 2016, accessed April 7, 2016 .
- ↑ Christoph Jehle: Lytro Cinema - light field camera with 755 megapixels and 40k video. In: Heise Online. Heise Medien, April 13, 2016, accessed on April 13, 2016 .
- ^ To the Cinematic and VR Community, Live Long and Prosper. Accessed March 29, 2018 .
- ↑ Compare also: Close-up of keys on a computer keyboard with a Lytro light field camera
- ↑ Compare also: Lytro light field camera - Piano Player
- ^ Technology Pelican Imaging, accessed May 7, 2013
- ↑ Nokia invests in light field camera technology . heise.de, May 2, 2013, accessed May 7, 2013
- ↑ HTC launches One M8 with new 'Duo Camera' , dpreview March 15, 2014, accessed online April 18, 2014
- ↑ Google camera: depth of field can be adjusted later , heise.de. News from April 17, 2014, accessed online on April 18, 2014
- ↑ Lens Blur in the new Google Camera app Google Research blog entry from April 16, 2014, accessed online April 18, 2014
- ↑ k-lens.de: product
- ↑ lightfield-forum.com: Lens presents first sample photos
- ↑ hal.inria.fr: A Reconfigurable Camera Add-On for High Dynamic Range, Multispectral, Polarization, and Light-Field Imaging