Multimedia technology
The multimedia technologies deal with the technical basics of all multimedia applications. This article deals on the one hand with the basic hardware components as well as with the fundamentals of computer-based graphics and audio technology. The increasing need for new, improved processes and systems is a major driver of developments in the field of multimedia. Society not only shapes demand, but multimedia technologies themselves have led to social changes.
technical basics
Powerful and cross-technology hardware is the basic component of every multimedia system. With the spread of the Internet, the development of network-based multimedia technologies began. The networks must achieve a high level of quality of service in order to be able to meet the requirements of multimedia. The big breakthrough came with the development of optical data storage devices , which made it possible to store large amounts of data easily and on the move.
hardware
Powerful hardware is required for computationally intensive applications such as video editing . Above all, this includes good processors , large RAM and generally strong graphics cards . External hardware such as loudspeakers or cameras are also part of the basic equipment of multimedia systems.
Following these requirements, the industry tried, under the leadership of Microsoft , Creative Labs , Dell , Gateway , Fujitsu and others, to implement a hardware standard for multimedia systems in the 1990s. According to this standard, which bears the name "Multimedia PC" (MPC for short), the PC systems were preconfigured in such a way that they could support all common multimedia applications. There were a total of three specification levels from 1991 to 1996. However, they could not be accepted by all manufacturers. Later, the development of the hardware was driven by new applications such as video editing, image processing and computer games .
Nowadays the computer game industry places the highest demands on hardware systems. The following points in particular play an essential role:
- Broadband internet access
- 3D graphics support # * intuitive input devices
- Multicore support, multithreading
- Surround sound
important components for a multimedia system:
Data storage
In the Von Neumann architecture, there are memory cells in each module. That is why the data memory plays an essential role in the processing of multimedia data. In contrast to the development of the processors, the performance in the form of memory space increased steadily.
Important performance characteristics of storage systems are:
- flexible data access
- fast data read and write rates
- Storage space size in minimal space
Video processing in particular generates large amounts of data that have to be loaded quickly from the hard drive into the main memory. But not only the hardware plays a role, but also the operating system , which creates a file system on the memory and thus takes on the management of the data.
Processors
Multimedia systems are not only based on a main processor (CPU), but rather from several concurrent processors from the field of observation. Other important processors are:
- Graphics processor on the graphics card
- Communication processor on the network card
- the signal processors on the sound card
There are also special multimedia processors, such as AMD Xilleon or AMD Imageon , which have been optimized for image and video processing.
Multimedia software usually does not use the entire word length (i.e. integers), but places more emphasis on short or char values. For this reason, processor architectures such as Single Instruction Multiple Data (SIMD) were developed, which process several small pieces of data in one instruction. Due to the high degree of parallelizability when processing image data, extensions such as MMX were created to meet the new requirements of multimedia applications.
Graphics cards
With the increasing demands of the computer game industry, the need for a computing unit that specialized in graphical aspects increased. Graphics cards contain their own processors (usually called GPUs), which are particularly suitable for fast image processing. This specialization makes them superior to the universal CPUs in the areas of parallel calculations.
Monitors
In order to optimally utilize the performance of graphics cards, the monitor used should support the performance features of the graphics card.
Performance criteria:
- resolution
- Refresh rate
- Colours
- brightness
- contrast
- Response time for liquid crystal screens
- Interfaces ( HDMI , VGA )
- Aspect ratio
- Vertical frequency
Sound cards
-4.jpg)
Sound cards are required to convert digital to analog signals (and vice versa).
Performance criteria:
- dynamics
- Frequency response
- Distortion factor
- Voltage at line-in, line-out
- Signal-to-noise ratio
speaker
Loudspeakers that are connected directly to the sound card are required for the acoustic reproduction of the audio data.
Performance criteria:
- Frequency response
- Impedance
- Distortion factor
- Music resilience
- Sound power
- Sinus load capacity
- Efficiency
Operating systems
Operating systems form the connection between the computer hardware and the software components and thus also serve as an interface for many multimedia applications. The sources of media data are mostly in hardware components such as microphones, cameras or touchpads and their targets are loudspeakers, video windows or the user himself. In order to process this data, the operating system has to perform at least one move, copy or transfer operation. Real-time processing, process management, resource reservation , resource management and memory management play important roles here.
Multimedia frameworks
Frameworks such as DirectX or OpenGL represent a programming interface for multimedia-intensive applications. There is a component for each multimedia module. The programmer should be able to get quick and direct access to the graphics card.
- DirectMusic is the component that functions as a synthesizer for playing MIDI files .
- DirectShow is particularly important for streaming video and audio data.
- With DirectSound / XAudio sound effects can be improved and interfaces for recording and playback of audio signals can be created.
- DirectX provides its DirectX graphics component, which combines DirectDraw for 2D graphics and DirectX for 3D graphics.
Hardware acceleration plays an important role in 3D graphics .
An "open source" and platform-independent alternative to DirectX is OpenGl .
Networks
In order to be able to guarantee a continuous exchange of data between media devices, networks must be created using suitable services and protocols.
communication
Communication in an interactive network requires low delays, support for the transmission of high data rates and a low error rate. To make this possible, network technologies such as FDDI , high-speed Ethernet, Token Ring and ATM were created. Communication between multimedia systems, which form the network layer, is implemented using network and transport protocols.
| Forms of communication / routing schemes |
|---|
|
Unicast
|
|
Broadcast
|
|
Anycast
|
|
Multicast
|
|
Geocast
|
Requirements for services and protocols
- Depending on the application, audio or video data require time-limited processing. The data transmission must accordingly take place during the specified intervals.
- The delay between the end users of an application, for example from interactive mode (e.g. telephone), must be kept low.
- The main area of application of communication-capable multimedia systems is mostly on cooperative work and conference systems using several different media. It should be possible to connect and disconnect conference members without having to completely re-establish the conference connection.
- When transmitting preferred or guaranteed multimedia data, there must be no restriction in the communication of the remaining data.
- In order to protect the fairness principle between different applications, users and stations, no client may be preferred.
In most cases, the underlying network systems, for example Ethernet , are used for communicative multimedia applications and no new separate networks are created. The assumption that at least a bandwidth of at least 100 bit / s is required for multimedia communication systems is therefore not fundamentally true.
An overview of important protocols
IGMP (Internet Group Management Protocol):
- can manage internet multicast groups
- dynamic login and logout of systems possible
- does not have to be a member of the group
IP (Internet Protocol):
- enables data transfer between the sender and the target system
- several transfer systems (gateways or routers) can be run through
RTP (Real-Time Transport Protocol):
- supports multicast sessions
RTSP (Real-Time Streaming Protocol):
- does not control user data, but data streams
- Similar to HTTP, but bidirectional and state-based
SIP (Session Initiation Protocol):
- sets up meetings and then takes over management of the meeting
TCP (Transmission Control Protocol):
- is required for video conferences
TP (Telnet Protocol):
- includes the RMTP (Real-Time Message Transport Protocol) and MTP (Continuous Media Transport Protocol)
- supports the transmission of multimedia data, as it guarantees certain services when delivering data
Quality of service
The quality of service (English QoS (Quality of Service)) is a fundamental concept for multimedia systems. It can be used to record, process, transmit and display quantifiable technical parameters for networks in real time. An evaluation of the multimedia system is especially important in terms of resource management.
Operating resources (quality of service)
- Bandwidth
- Latency / Runtime
- maximum bit error rate
- maximum packet loss
- Processor power
- Storage space
The two approaches of scaling and resource reservation, which are often used at the same time, exist for the management of operating resources.
Scaling refers to the simplest possible adaptation of the data flow to predefined boundary conditions. To do this, the resources are related to the data stream.
The second regards individual components as resources that must be reserved before the evaluation can be carried out. As soon as all resources have been reserved, resource management can be used to allocate resources in a meaningful way.
Difficult or unquantifiable properties
- Usability of the end devices used
- unpredictable external influences
- Disturbances in the network
Optical data storage
The multimedia world was fundamentally revolutionized with the development of optical storage. They are mainly of interest because of their high requirements, since they allow random access in contrast to magnetic mass storage devices. Further advantages over the outdated magnetic memory are their durability and low wear and tear.
In the course of the development of optical storage, the aim has always been to pack the data on the storage media as compactly as possible in order to obtain a high storage capacity. The only fundamental difference between the compact disk (CD) and the digital versatile disc (DVD) is the significantly higher storage capacity due to a higher data density.
The idea is to evaluate the intensity of the light reflected on a preformed surface in order to be able to draw conclusions about the coding of the data. Here are Laser (CDs regard.) To keep as low as possible to focus with a wavelength of about 780 nm used.
A substrate layer lies on the storage disk, which on the one hand contains depressions (pits) and plateaus (lands). By alternately reading pits and lands, the coding can be decoded from 1s and 0s.
| Type |
wavelength Numerical aperture |
Sectional drawing of the beam geometry |
|---|---|---|
| CD | Wavelength: 780 nm Numerical aperture: 0.45 Spot diameter: 2.1 µm Track spacing: 1.6 µm |
|
| DVD | Wavelength: 650 nm Numerical aperture: 0.6 Spot diameter: 1.3 µm Track spacing: 0.74 µm |
|
When the data is to be applied to the storage medium, the necessary bits are burned into the substrate layer as pits (see also burner ). All information is plotted on one track, which makes it easier to store continuous data. The track has the shape of a spiral, the distance between the tracks being 1.6 µm and the track width of the pits being 0.6 µm. Together this results in a data density of up to 1,000,000 bit / mm².
Audio
Audio signals
- noise
- Sound superimposed on several frequencies
- Bang
- score
- Noise
- language
- Tone a frequency
Composition of the audio signals
The following properties are decisive for the technical implementation of audio signals:
frequency
The frequency in the audio signal determines the pitch of the recorded sound. The frequency spectrum extends logarithmically from 10 kHz up to 10 MHz and represents the 8 fundamental tones.
Pressure level
The pressure level is a logarithmic measure to describe the strength of a sound event.
direction
The direction of the sound event is important for spatial hearing. The following points are decisive in order to be able to reproduce a direction and the associated distance.
- reverberation
- Sound absorption
- Sonic shadows
- Signal transit times
The audio quality is greatly improved by the direction in large buildings such as operas and cinemas.
Timbre
The timbre can be used to distinguish voices from instruments. The timbre is represented by the amplitude in the signal curve. Here, shrill tones with high deflections and dull tones with low deflections are displayed.
Audio files
- Parameter:
- Sampling frequency in kHz
- Number of channels
- Compression rate in kbps
The size of the audio file results from these parameters.
compression
Audio compression can be broken down into two large parts. On the one hand there is the lossless audio compression ( data compression ), on the other hand the lossy audio compression ( data reduction ).
Data compression uses more universal compression methods such as LZW or Huffman coding . This type of compression is therefore not a special feature of audio compression.
Data reduction
With the help of data reduction one tries to remove tones from the frequency spectrum that do not contribute to the sound quality. In the preparatory work, a frequency spectrum of the audio file is created using Fast Fourier Transformation (FFT) or Discrete Cosine Transformation (DCT).
The following points are also carried out:
- Hearing threshold masking
This removes the tones that are outside the human hearing threshold in the frequency spectrum of the file.
- Frequency masking
Many tones are covered by other tones and are therefore of no greater importance for the human ear.
- Time masking
Using the time, temporal overlapping of tones can be specifically found and removed.
- Stereo redundancy
Often one and the same signal is transmitted on different channels; this is an attempt to prevent this. Depending on the process, there can be a considerable loss of quality from this point on.
- Coefficient quantization
Finally, the coefficients of the frequency spectrum are quantized, whereby a so-called quantization error becomes noticeable (6 dB noise per omitted bit).
Compressed audio formats
Film / DVD / DF1:
- Dolby AC-3 (Dolby Digital)
- MPEG 2 AAC, up to 48 audio channels
- MPEG 2 Layer 2
Internet:
Radio-TV / ADR / DAB:
- MPEG 2 Layer 2
Stereo system:
- Digital Compact Cassette (DCC): MPEG 1 Layer 2 (192 kbits / s)
- Sony Minidisc (MD)
Graphics
Image formats:
| Image formats | Compression (lossless / afflicted and type) |
supported color models |
Transparency support |
supported bit depths |
Number and type of channels |
Advantages and disadvantages |
Frequent use |
| bitmap | lossless (RLE) | RGB, SW, induced colors | No | 1, 4, 8 and 24 | 4th | + easy generation - size |
Laser printer |
| JPEG | lossy (Huffman, DCT) | RGB, CMYK, grayscale | No | 24 | 3 or 4 | + good for photographic images + size - bad for digitally generated images |
photos |
| GIF | lossless (LZW) | induced colors | Yes | 1-8 | 1 | + Animations + well suited for the Internet because of its small size |
Internet |
| PNG | lossless (LZW variant) | RGB, grayscale | Yes | 8, 16, 24, 48 | 3 + 1 alpha channel | + also 24 bit + interlaced |
Raster graphics |
| RAW | no compression | RGB, grayscale | no | until 14 | 3 | + best quality format for photos - size |
photos |
| SVG | lossless (gzip) | RGB | Yes | 24 | 3 | + lossless scalability - become very large quickly |
Internet |
| TIFF | lossless (LZW variant) | CMYK, SW, RGB | Yes | 1-48 | 1, 3 or 4 | + Platform - independent + a lot of meta-information can be stored -many sub-formats |
photos |
Image size and resolution
While the image size indicates the total number of picture elements ( pixels ), the resolution, like the bit depth, is a quality characteristic for the graphic.
- Image size
- represents the total number of pixels
- Number of columns (width) and rows (height)
- Resolution:
- Image elements per length in ppi (pixels per inch)
- Dot density of a display or image sample
- in addition to the depth of color, a measure of the quality
- In raster graphics no resolution can be specified because it depends on the playback
Bit depth
With the help of the bit or color depth, one would like to unfold a suitable color spectrum which is relevant for the created image. Basically: With n bits 2 ^ n colors can be represented.
With 16 bits you get a color space of 65,000 colors, this color space is called "high color".
32 bits and thus 16,000,000 colors are sufficient to represent the natural color space with close quality. A 32-bit color depth is called "true color".
Color models
Color models are geometrical or mathematical frameworks which are supposed to describe the color perception of humans. For this purpose, color models have one or more dimensions that together represent the visible spectrum of colors.
In order to be able to describe, calculate, compare or classify the colors, the colors are indexed numerically.
With more modern models, such as the HSK color model, the light intensity was added as an additional dimension in order to be able to reproduce shades better.
channels
The color channel contains the color components of each pixel in the digital image. Several color channels can be assigned to each pixel. The red color channel defines how much red is contained in a pixel, the green color channel how much green and so on. The number of channels thus indicates the color model.
Dithering
Dithering represents the diffusion of errors in color gradients in images. This is used to reproduce color gradations in images with few colors. With the help of certain pixel arrangements, an artificial color gradient is created. The human eye perceives dithering as a mixture of individual colors, creating the illusion of greater color depth.
| Dithering based on an example |
|---|
| Color gradient with 300 colors (without dithering) |
|
| Gradient with 16 colors (with dithering) |
|
| Gradient with 4 colors (with dithering) |
|
| Color gradient with 2 colors (with dithering) |
|
transparency
In computer graphics, transparency refers to image elements that allow the content of the elements underneath to be fully or partially visible.
An additional α-channel is used for bitmaps, which indicates its transparency for each individual pixel.
use
Overview of developments
Multimedia technology is firmly anchored in today's world. Hardly any other technology affects as many people as this one.
Screen technologies:
- cathode ray tube
- LCD / LED
- New development (still in the test phase): contact lens as a screen
- Projector / beamer
- TFT
- Touch screen
Developments in the field of image / graphics:
Developments in the audio sector:
- Audio editing programs (cutting and conversion programs)
- speaker
- microphone
- MP3 player
Developments in the video sector:
- Digital camera
- Cutting and converting programs
- Webcam
latest developments:
- Augmented reality
- optical motion detection ( Wii , eyeToy )
Technologies in society
E-learning is likely to be one of the main fields of multimedia development. There are already numerous developments here that are intended to simplify learning.
Exercise programs
Exercise programs serve to consolidate knowledge (mostly acquired beforehand) through repetition and application. There are large numbers of such programs from almost all areas of knowledge and learner types, starting with vocabulary trainers and electronic exercise collections to mathematical exercise programs.
Test software
Test software is used to carry out and evaluate learning controls. Learners do not answer questions verbally or on paper, but on the computer. Multiple-choice tasks or fill-in-the-blank (e.g. driver's license test) are particularly suitable for this. Here the lecturer is relieved, as the performance control takes place electronically. Better tests are made possible by adaptive systems that dynamically determine the type, difficulty and number of test items depending on the previous course of the test.
Tutorial systems
Tutorial systems , often also referred to as computer-based training (CBT), take on all teaching functions. The learning software conveys an area of knowledge to the learner, lets them work on tasks, gives feedback on the level of performance and determines further learning paths depending on previous performance.
swell
- Ralf Steinmetz Multimedia-Technologie Published by Springer Berlin Heidelberg; Edition: 3rd, revised. Edition (August 21, 2000) ISBN 978-3540673323