TOF camera

from Wikipedia, the free encyclopedia

TOF cameras are 3D camera systems , with the transit time method (English: time of flight , TOF, and TOF) measure distances. They are also called PMD cameras after the PMD sensor used. For this purpose, the scene is illuminated by means of a light pulse, and the camera measures the time it takes for the light to reach the object and back again for each pixel. The time required is directly proportional to the distance. The camera thus supplies the distance of the object imaged on it for each pixel. The principle corresponds to laser scanning with the advantage that an entire scene is recorded at once and does not have to be scanned.

In contrast to other methods, TOF cameras are a relatively new development. The systems can be used in a distance range from a few decimeters to approx. 40 m. The distance resolution is around 1 cm, the lateral resolutions are around 200 × 200  pixels . One advantage of these cameras is the high frame rate of up to 512 frames per second.

construction

A TOF camera consists of at least the following components:

  • Lighting unit: It illuminates the scene. Either LEDs or laser diodes are used here, which can be modulated sufficiently quickly so that the sensor can correctly measure the transit time. The pulse duration is in the nanosecond range. The lighting mostly sends in the near infrared so that the surroundings are not disturbed by the camera.
  • Optics: Optics collect the light reflected from the environment and depict the scene on the sensor. An optical bandpass filter only lets through the wavelength with which the lighting works. A large part of the disturbing background light is thus eliminated.
  • Sensor: The heart of the TOF camera is the sensor, which measures the transit time for each pixel separately. The image sensor is similar to other chips for digital cameras with the difference that a pixel has a much more complicated structure: It does not simply have to be able to collect the incident light, but rather measure the transit time. Due to the more complicated structure, the pixels are large compared to digital cameras, with side lengths of up to 100 µm. The highest resolution of ToF sensors to date is 204 × 204 pixels with an edge length of 45 µm.
  • Control electronics : The lighting and the sensor must be controlled with complex electronics in order to achieve the highest possible accuracy. If the control signals between the lighting and the sensor are only shifted by 10  ps , the measured distance changes by 1.5 mm.
  • Evaluation / Interface: The distance is usually calculated directly in the camera system from the measured values. For this purpose, calibration values ​​are also saved in the system. USB or Ethernet are used as interfaces .

functionality

Principle of TOF cameras

The simplest form of TOF cameras works with light pulses : the lighting is switched on for a short moment, the light pulse illuminates the scene and is reflected on the objects. The camera's lens collects this light and images the scene on the sensor. Depending on the distance, the light hitting the individual pixels experiences a delay. Since the light propagates at the speed of light (in the air approx. 299,710 kilometers per second), these times are very short: The transit time of the light from the camera to an object 2.5 m away and back to the camera is:

Based on the evaluation principle (see below), the pulse length of the lighting determines the maximum distance that the camera can cover. With a pulse length of 50 ns, distances of up to

be measured. These short times show that lighting is a critical part of the system. It is only possible to generate such short pulses with selected LEDs or with lasers that are more complex to control.

The individual pixels consist of a photosensitive element (e.g. photodiode ) that converts the light into a current. One or more fast locks or switches are attached to the photodiode, which only allow the electrical signal to pass through for a very specific period of time. A downstream storage element sums up the signal.

In the example sketch, the pixel works with 2 switches (G 1 and G 2 ) and storage elements (S 1 and S 2 ). The switches are controlled with a pulse signal with the same length as the light pulse, the control signal for G 2 being shifted by one pulse length. If the reflected light hits the pixel with a delay, only part of the signal reaches the storage element S 1 , the other part is collected in S 2 . Depending on the distance, the ratio of S 1 and S 2 changes , as shown in the second graphic. Since only very little light can be collected within 50 ns, not just one pulse, but several thousand with a repetition rate of (t R-1 ) is transmitted and collected, which increases the signal strength.

After the recording, the pixels are read out and the subsequent stage measures the signals S 1 and S 2 . Since the length of the light pulse is known, the distance can be calculated as follows:

In the example, S 1 = 0.66 and S 2 = 0.33. The distance is thus

Background light results in an additional signal component on the two storage elements. In order to eliminate this, the recording can be carried out again with inactive lighting and these values ​​can be subtracted from the signals with lighting. If the objects are further away than the distance range, the above formula results in incorrect distance values. This can also be suppressed with a second measurement in which the switching signals are shifted again by t 0 . Other systems work with a sinusoidal modulation instead of pulses , in which the requirements for the slope of the lighting are lower .

Advantages and disadvantages

advantages

  • Simple construction: In contrast to laser scanners, the camera does not contain any moving parts. Since the lighting and the lens are close to each other, the space requirement is smaller compared to stereo and triangulation systems , and shading is excluded.
  • Efficient data evaluation: With the distance information from the TOF cameras, it is easy to extract only the areas of interest from an image: A distance threshold is set and only the pixels that provide closer distances are taken into account.
  • Speed: The TOF cameras depict the entire scene in one shot. The frame rates reach up to 160 frames per second and thus enable real-time applications.
  • Pattern independence: In contrast to stereo systems, which can have difficulties with repeating patterns or uniform surfaces, TOF cameras work with all diffuse reflective materials.

disadvantage

  • Background light: Although most of the background light is suppressed by the optical filter, the pixel must still be able to cope with a very high level of dynamics and the resulting charge must also be able to be stored or discharged. For comparison: the illuminance levels that are currently economically feasible are in the range of one watt. The sun still produces 50 watts per square meter in the filtered wavelength range. If the illuminated scene is one square meter, the sun is 50 times as strong as the useful signal.
    The manufacturers have developed different strategies for their sensors in order to be able to suppress this background signal to a large extent (see for example SBI of the PMD sensor , which works up to 150  klx ).
  • Mutual interference: If several systems are in operation, it is possible that the various cameras interfere with one another and thus the distance value is falsified. There are several ways to get around this:
    • Time division multiplex: A higher-level controller starts measuring the individual cameras one after the other so that only one lighting is in operation at a time.
    • Different frequencies: If the cameras work with slightly different modulation frequencies, the light from one camera is only demodulated as a background component in the other and does not falsify the measurement.
  • Multiple reflection : Since, in contrast to laser scanning systems, an entire scene and not just a single point is illuminated, it is possible that light that is reflected multiple times from an object can reach the sensor. In this case, the measured distance can be greater than the actual one.

application areas

Distance image recorded with TOF camera for gesture control

Automobile applications

TOF cameras are used as driver assistance and safety sensors in the automotive sector. This includes applications such as active pedestrian protection and emergency brake assist , but also in the interior, such as checking for the correct driving position.

Human-machine interfaces / gaming

Thanks to the real-time capability of the TOF cameras, human movements can be followed. This opens up new opportunities for interaction with the devices. In addition to controlling z. B. televisions, the application of TOF cameras to game consoles is an interesting topic.

Measurement technology / industrial image processing

Measurement of objects using 3D TOF images

The third dimension makes it easy to carry out measuring tasks such as determining fill levels in silos . In industrial image processing, for example, robots that have to pick up objects from a conveyor belt benefit from the additional height information. Door controls can easily differentiate between animals and people with the help of height.

robotics

Another area of ​​application are mobile robots: with the real-time image of the surroundings, mobile robots can quickly overlook their surroundings, avoid obstacles or, for example, follow a person. With the help of a TOF camera, a map of the surroundings can be created from many individual recordings.

medicine

Patient positioning with the help of 3D TOF images

TOF cameras can also be used as an additional imaging modality in medical technology. Examples of these applications are briefly shown below:

  • Breath detection: With the help of TOF cameras it is possible to calculate a multidimensional breath signal. It is thus possible to measure independent breathing signals in several places (e.g. thorax , abdomen ) without contact and without markers . This is v. a. important for the irradiation of tumors in the upper body area or for the reduction of artifacts, e.g. B. in magnetic resonance (MRI) or positron emission tomography (PET).
  • Patient positioning: TOF cameras also enable the exact positioning of patients in the clinical environment. For this purpose, the 3D point clouds of a reference TOF image are registered with another point cloud. The result of this registration is the translation and rotation that must be applied to the patient so that he is exactly the same as when the reference image was recorded. Applications for this can also be found in radiation therapy. The irradiation is planned here with an initial CT measurement. This plan must be valid for all subsequent radiation sessions. This can be done by registering TOF images.

literature

  • Pia Breuer: Development of a prototypical gesture recognition in real time using an IR depth camera. 2005 (University of Koblenz Landau; diploma thesis; PDF file ; 2.80 MB; gesture recognition with TOF cameras)
  • Martin Profittlich: Minority Report -Futuristic Interface Technologies through 3D Image Processing. In: Inspect. No. 2, 2009 ( HTML version , accessed September 30, 2009).
  • Christian Schaller, Jochen Penne, Joachim Hornegger: Time-of-flight sensor for respiratory motion gating . In: Medical Physics . tape 35 , no. 7 , 2008, p. 3090–3093 , doi : 10.1118 / 1.2938521 ( PDF file; 214 kB - breath detection with TOF cameras, TOF research group at the Chair of Pattern Recognition University Erlangen-Nürnberg).
  • Technological overview of time-of-flight cameras. Description of the technology and comparison to other real-time 3-D acquisition systems, Metrilus.de, accessed July 13, 2011.

Web links

Individual evidence

  1. Christoph Heckenkamp: The magic eye - Basics of image processing: The PMD principle . In: Inspect. No. 1, 2008, pp. 25-28.
  2. TOF camera from ESPROS with up to 512 fps. Retrieved April 27, 2017 .
  3. Bluetechnix brings ToF 3D camera with 160 fps onto the market ( memento of the original from June 17, 2013 in the Internet Archive ) Info: The archive link was inserted automatically and has not yet been checked. Please check the original and archive link according to the instructions and then remove this notice. @1@ 2Template: Webachiv / IABot / www.ptext.at
  4. Michael Paintner: Everything at a glance - driver assistance and safety functions with a 3D PMD camera. In: AutomobilKonstruktion. No. 2, 2007, pp. 66-67 ( PDF file ; 954 kB).
  5. Gerd Kucera: NoAE innovation award of the automotive industry for 3-D camera received . In: Electronics Practice. August 10, 2009. Retrieved August 11, 2009.
  6. See also the first 3D industrial camera that was developed by ifm electronic GmbH in cooperation with PMDTechnologies GmbH.
  7. see also the TOF camera ESPROS / TOF developed in a research project in Switzerland together with the CSEM .