H.264
H.264 / MPEG-4 AVC is an H. standard for video compression . Successors are H.265 , VP9 and Daala . It was initially developed by the ITU (Study Group 16, Video Coding Experts Group) under the name H.26L. In 2001, the ITU group merged with MPEG- Visual and continued development in the Joint Video Team (JVT). The aim of the project was to design a compression process that, compared to the standards of the time, reduced the required data rate by at least half while maintaining the same quality for mobile applications as well as in the TV and HD area. Members of the working group come from the Fraunhofer Institute for Telecommunications , Microsoft and Cisco Systems , among others . In 2003 the standard was adopted by both organizations with identical wording. The ITU designation is H.264. At ISO / IEC MPEG, the standard runs under the name MPEG-4 / AVC ( Advanced Video Coding ) and is the tenth part of the MPEG-4 standard ( MPEG-4 / Part 10, ISO / IEC 14496-10 ). The standard should not be confused with the MPEG-4 / ASP standard (MPEG-4 / Part 2, ISO / IEC 14496-2).
H.264 typically achieves a coding efficiency about three times as high as H.262 ( MPEG-2 ) and, like H.262, is designed for high-resolution image data (e.g. HDTV ). This means that comparable quality can be achieved with around a third of the MPEG-2 data volume. However, the computational effort is also 2 to 3 times higher.
The FourCCs used for the standard are "AVC1", "DAVC", "H264", x264 and "VSSH". The Matroska Codec ID is "V_MPEG4 / ISO / AVC".
The standardized file format / container format is MP4 .
Areas of application
H.264 was not tailored to a specific purpose, but unfolds its power in a fairly wide range of applications. Therefore, the most promising areas of application at the moment are of very different shapes:
- HDTV
- H.264 is one of the mandatory video compression methods of the Blu-ray standard and is mandatory for high-resolution television transmission using DVB-S2 (e.g. HD-1 and Sky HD , as well as ProSieben HD or RTL HD). The HD-DVD standard, which was discontinued in 2008, also provided for video compression according to the H.264 method.
- Portable video
- The competing mobile television standards DVB-H and DMB both use (among other things) H.264 for video coding for mobile devices such as cell phones or PDAs . The PlayStation Portable , the fifth generation of the Apple iPod , the iPhone and the Zune player, which is only available in the USA, can also play H.264 videos.
- multimedia
- Apple delivers its multimedia framework QuickTime from version 7 with an H.264 codec.
- Video conferencing technology
- Since 2005, video conference end systems with H.264 codecs have been available to users. For example, multiple video conferences are possible for the first time in iChat AV 3.x.
- Video / digicams
- A number of digital cameras and video cameras support H.264 compression for video recording.
Related procedures
During the specification of H.264, several commercial development paths split off, which are more or less directly based on H.264, but ultimately represent a partially incompatible unit:
- Sorenson Video 3 , a codec that is very common in the QuickTime environment and in the context of the * .mov file format, which differs from H.264 only in details.
- Microsoft Windows Media Video 9 (* .wmv, FourCC: WMV9 / wvc1) is a video codec from Microsoft, which also uses loop filtering. The variant VC-1 (FourCC: wvc1) (formerly VC-9 according to WMV9) is also one of the codecs that are used in HD DVD and Blu-ray Disc .
- VP8 is a video codec developed by the Google company On2 Technologies. It is based on the TrueMotion development series .
Further developments
- Multiview Video Coding (MVC) for stereoscopic recordings.
- High Efficiency Video Coding (HEVC or H.265), successor to H.264
- VP9 is a video codec developed by Google. It is based on VP8 and is implemented in the Chrome browser.
- AV1 , is a video codec developed by the Alliance for Open Media . It builds on VP9.
Technical details
H.264 is largely based on its predecessors MPEG-1, MPEG-2, MPEG-4 and the H.261 family, but has significant changes and extensions (see also):
- Instead of a discrete cosine transformation (DCT) with 8 × 8 pixel blocks, an integer transformation to 4 × 4 pixels (in the high profile can also be selected up to 8 × 8 pixels) blocks is used.
- The entropy coding was adapted to the changed transformation. In addition to the conventional Huffman-type VLC codes ( CAVLC , Context Adaptive Variable Length Coding ), H.264 also supports a more powerful arithmetic coding ( Context-Adaptive Binary Arithmetic Coding (CABAC)).
- Macroblock Partitioning : Macroblocks with a size of 16 × 16 pixels can be divided into sub-blocks down to a size of 4 × 4 pixels. Since a separate motion vector is stored for each block , spatially sharply delimited and, above all, complex movements can be coded better.
- The motion compensation is always ¼ pixels (at exactly H.263 , d. H. MPEG-4 / ASP, this was an option). The motion compensation for half pixels is no longer done with a filter with two coefficients (averaging: +1/2, +1/2), but with a more complex filter with six coefficients (+1/32, -5/32, + 20 / 32, +20/32, -5/32, +1/32). Quarter pixels are calculated from the half-pixel data by a further averaging.
- Intra prediction : A form of image prediction is also used within I-frames . The pixel values of a block are estimated from the surrounding, already decoded pixels and then only the difference to the actual image content is coded. This method is generally more efficient than JPEG 2000 for smaller image formats .
- Long-Term Prediction : P- and B-frames can not only contain references to the last I- or P-frame, but also to a maximum of 16 previous frames, theoretically as far back as you want, in order to enable more efficient coding of periodic movements enable.
- Weighted Prediction : If image content from several reference frames is mixed, the sources can be included in the mix with any weighting. In this way, fades and fades can be coded very efficiently.
- Deblocking filter : While the previous MPEG codecs exclusively to an optional external post-processing / filtering (English postprocessing left), is a deblocking filter integral part of H.264 (as it did for H.263 in Appendix J). References to frames that have already been decoded also relate to the images that have already been filtered. This achieves a very high perceived image quality. Particularly in conjunction with the reduced transformation size of 4 × 4, there are great subjective (and to a lesser extent objective) improvements.
- Switching slices : enable the seamless transition between different video streams without having to use "expensive" (because large) I-frames.
- Flexible Macroblock Ordering : The macroblocks within a frame or slice can be specified in a relatively free order. This can be done e.g. B. to hide errors in case of channel losses (mobile video). However, most encoders and decoders do not support FMO, nor is it part of the Main and High profiles .
Profiles
Profiles and levels should support the cooperation of the various implementations of the manufacturers. The profiles summarize certain features that must be supported. A level sets restrictions on the variables of a data stream, such as the maximum resolution or bit rate.
| Baseline | Extended | Main | High | High 10 | High 4: 2: 2 | High 4: 4: 4 | |
|---|---|---|---|---|---|---|---|
| I and P slices | Yes | Yes | Yes | Yes | Yes | Yes | Yes |
| B slices | No | Yes | Yes | Yes | Yes | Yes | Yes |
| SI and SP slices | No | Yes | No | No | No | No | No |
| Multiple reference frames | Yes | Yes | Yes | Yes | Yes | Yes | Yes |
| In-loop deblocking filter | Yes | Yes | Yes | Yes | Yes | Yes | Yes |
| CAVLC Entropy Coding | Yes | Yes | Yes | Yes | Yes | Yes | Yes |
| CABAC Entropy Coding | No | No | Yes | Yes | Yes | Yes | Yes |
| Flexible Macroblock Ordering (FMO) | Yes | Yes | No | No | No | No | No |
| Arbitrary Slice Ordering (ASO) | Yes | Yes | No | No | No | No | No |
| Redundant Slices (RS) | Yes | Yes | No | No | No | No | No |
| Data partitioning | No | Yes | No | No | No | No | No |
| Interlaced coding (PicAFF, MBAFF) | No | Yes | Yes | Yes | Yes | Yes | Yes |
| 4: 2: 0 chroma format | Yes | Yes | Yes | Yes | Yes | Yes | Yes |
| 4: 2: 2 chroma format | No | No | No | No | No | Yes | Yes |
| 4: 4: 4 chroma format | No | No | No | No | No | No | Yes |
| 8 bit sample depth | Yes | Yes | Yes | Yes | Yes | Yes | Yes |
| 9 and 10 bit sample depth | No | No | No | No | Yes | Yes | Yes |
| 11 and 12 bit sample depth | No | No | No | No | No | No | Yes |
| 8x8 vs. 4x4 transform adaptivity | No | No | No | Yes | Yes | Yes | Yes |
| Quantization Scaling Matrices | No | No | No | Yes | Yes | Yes | Yes |
| Separate Cb and Cr QP control | No | No | No | Yes | Yes | Yes | Yes |
| Monochrome video format | No | No | No | Yes | Yes | Yes | Yes |
| Residual Color Transform | No | No | No | No | No | No | Yes |
| Predictive lossless coding | No | No | No | No | No | No | Yes |
Level
As in MPEG-2, H.264 defines different levels. The higher the bit rate of the video, the higher these are.
The following table shows the permissible limit values for the individual profiles:
| Macroblocks pro | examples for | Video bit rate (VCL) for profiles | |||||
|---|---|---|---|---|---|---|---|
| Level | Frame | second | Resolution / frame rate of this level |
Baseline Extended Main |
High | High 10 | High 4: 2: 2 High 4: 4: 4 |
| 1 | 99 | 1 485 | 128 × 96/30 176 × 144/15 |
64 kbit / s | 80 kbit / s | 192 kbit / s | 256 kbit / s |
| 1b | 128 kbit / s | 160 kbit / s | 384 kbit / s | 512 kbit / s | |||
| 1.1 | 396 | 3,000 | 176 × 144/30 320 × 240/10 352 × 288 / 7.5 |
192 kbit / s | 240 kbit / s | 576 kbit / s | 768 kbit / s |
| 1.2 | 6,000 | 176 × 144/60 320 × 240/20 352 × 288/15 |
384 kbit / s | 480 kbit / s | 1152 kbit / s | 1536 kbit / s | |
| 1.3 | 11 880 | 320 × 240/40 352 × 288/30 |
768 kbit / s | 960 kbit / s | 2304 kbit / s | 3072 kbit / s | |
| 2 | 2 Mbit / s | 2.5 Mbit / s | 6 Mbit / s | 8 Mbit / s | |||
| 2.1 | 792 | 19 800 | 352 × 288/50 352 × 576/25 |
4 Mbit / s | 5 Mbit / s | 12 Mbit / s | 16 Mbit / s |
| 2.2 | 1,620 | 20 250 | 352 × 288/50 720 × 480/15 |
4 Mbit / s | 5 Mbit / s | 12 Mbit / s | 16 Mbit / s |
| 3 | 40 500 | 720 × 480/30 720 × 576/25 |
10 Mbit / s | 12.5 Mbit / s | 30 Mbit / s | 40 Mbit / s | |
| 3.1 | 3 600 | 108,000 | 720 × 576/60 1280 × 720/30 |
14 Mbit / s | 17.5 Mbit / s | 42 Mbit / s | 56 Mbit / s |
| 3.2 | 5 120 | 216,000 | 1280 × 720/60 1280 × 1024 / 42.2 |
20 Mbit / s | 25 Mbit / s | 60 Mbit / s | 80 Mbit / s |
| 4th | 8 192 | 245 760 | 1280 × 720 / 68.3 1280 × 1024/48 1920 × 1080/30 |
20 Mbit / s | 25 Mbit / s | 60 Mbit / s | 80 Mbit / s |
| 4.1 | 50 Mbit / s | 62.5 Mbit / s | 150 Mbit / s | 200 Mbit / s | |||
| 4.2 | 8 704 | 522 240 | 1280 × 720/145 1920 × 1080/64 2048 × 1080/60 |
50 Mbit / s | 62.5 Mbit / s | 150 Mbit / s | 200 Mbit / s |
| 5 | 22 080 | 589 824 | 1920 × 1080 / 72.3 2048 × 1080 / 67.8 3672 × 1536 / 26.7 |
135 Mbit / s | 168.75 Mbit / s | 405 Mbit / s | 540 Mbit / s |
| 5.1 | 36 864 | 983 040 | 2048 × 1080 / 112.9 3840 × 2160 / 31.7 4096 × 2160 / 28.5 |
240 Mbit / s | 300 Mbit / s | 720 Mbit / s | 960 Mbit / s |
| 5.2 | 36 864 | 2,073,600 | 2048 × 1080/172 3840 × 2160 / 66.8 4096 × 2160/60 |
240 Mbit / s | 300 Mbit / s | 720 Mbit / s | 960 Mbit / s |
Implementations
- AVCHD multimedia standard for high-resolution, digital consumer and prosumer video cameras.
- AVC-Intra implementation by Panasonic for professional video productions
- JM reference software from the Joint Video Team
- x264 is an H.264 encoder licensed under the GPL .
- The free, LGPL- licensed library libavcodec includes an H.264 decoder.
- Apple has integrated H.264 in macOS from version 10.4 (Tiger) and into QuickTime version 7.
- Nero Digital , contains an H.264 encoder developed by Nero AG and Ateme .
- Sorenson offers an implementation of H.264.
- Main Concept offers an H.264 encoder as software, OpenCL and CUDA encoder.
- Elecard H.264
- Lead Technologies has an H.264 video codec.
- CoreAVC is a software H.264 decoder
- Pixeltools Expert-H264
- Adobe Flash Player 9+
- DivX H.264
- Intel Ivy Bridge QuickSync (GPU encoder)
- DiscretePhoton
- nanocosmos offers H.264 codecs for Windows and MacOS
- Citrix uses the compression method for its HDX protocol (formerly ICA) in the products XenApp 6.5 and XenDesktop 7 for the transmission of moving images.
Testing
In some tests by the MSU Graphics & Media Lab (Video Group), the x264 codec has established itself as the leader. Compared to DivX, it only needs 2/3 of the data stream for the same result. Due to the continuous further development, the codec has been greatly improved in recent years for a lower data stream with the desired image quality. This is very important, especially for example for satellite and internet transmissions with limited capacity.
Compared to H.265 and H.264, the brother codec x265 has a 20% lower data stream with the same image quality and is therefore 25% better.
Patents
Many of the methods used in H.264 are protected by patents . As with MPEG-2 , manufacturers and service providers who want to use this standard also require a fee for MPEG-4 . So that you don't have to ask permission from each patent holder individually, they usually give their patents to a patent pool for administration, which takes care of the breakdown of the individual income. However, since this is a voluntary measure, two patent pools have formed for H.264, which not all patent holders have yet joined. On the one hand there is the MPEG Licensing Administration (" MPEG Licensing Administration ", "MPEG LA" for short), which already manages the patent pools for MPEG-2, and on the other hand there is a patent pool of Via Licensing, a subsidiary of Dolby Laboratories .
The first license period ("initial term of license") - of the video codec "H.264 / AVC" from MPEG LA - ran until December 31, 2010.
According to a communication from MPEG LA, developers can use it free of charge for another five years (in the second license period from January 1, 2011 to December 31, 2015) as long as the video streaming remains free for users. MPEG LA has meanwhile announced that it will permanently not charge any license fees for the use of free streaming offers.
Currently, around 15% of patents have expired by the end of 2019.
Playback on the PC
With the help of ffdshow , videos encoded in H.264 can be played with DirectShow-based video playback programs such as Windows Media Player or Media Player Classic .
An alternative to DirectShow-based players is e.g. B. given with the MPlayer or the VLC media player . Both are proficient in decoding H.264 and can read all relevant containers. Both programs are also freely available. The ffdshow filter, MPlayer and the VLC player use the libavcodec library for decoding H.264 , which supports user-defined quantization matrices.
Furthermore, Apple's QuickTime 7 or later will be able to play H.264. QuickTime is available for recent versions of macOS and Windows . However, QuickTime 7 does not play content produced by the ffdshow H.264 encoder, but shows a black picture. The Perian extension can help .
The Apple iPod plays H.264 in MP4 and .mov containers . An iPod-compatible H.264 video can be created either with the Quicktime H.264 encoder or the x264 . For technical reasons, the qualitative possibilities of the encoder on the iPod are more limited than the H.264 standard would allow. In Generation 6, however, the iPod supports a low-complexity version of the H.264 baseline profile with a resolution of up to 640 × 480 pixels at 30 frames per second and a maximum data rate of 2.5 Mbit / s Level 3.0.
Since H.264 is not tied to a specific container format, the videos can be available as MP4, but also as AVI , Matroska or Ogg Media files. It is even possible to save H.264 videos as raw data (.264). Such raw data can then e.g. B. be multiplexed with MP4Box (GPAC) or mkvmerge in a suitable container. In addition to a decoder, you also need a splitter (demuxer) that supports the container format used. The Haali Media Splitter is suitable for this under Windows , a source filter for DirectShow that can handle almost all relevant container formats.
Playback via software decoding required a lot of CPU power at the time of publication. Relief for the CPU is possible with suitable hardware, which is integrated in many graphics cards, in conjunction with the decoder software based on it, e.g. B. by means of DXVA or VDPAU .
Web links
- x264 , a free H.264 / AVC encoder
- MPEG LA , administrator of the H.264 patent pool
- Codec comparison H.264 with VC-1 on multimedia wiki (English)
- MPEG Industry Forum , promoter of MPEG-4 AVC / H.264 and other MPEG standards (English)
Individual evidence
- ^ Department for press and information: Again price for video coding. Prof. Dr. Thomas Wiegand receives the IEEE Masaru Ibuka Consumer Electronics Award in Las Vegas . Technische Universität Berlin, January 16, 2012, accessed on January 18, 2012 : “Thomas Wiegand will be honored together with Dr. Gisle Bjontegaard from Cisco and Dr. Gary Sullivan from Microsoft for the development and establishment of the international video coding standard H.264 / MPEG4-AVC. "
- ↑ HDTV was introduced with MPEG-2 (1920x1080 @ 30), the station Euro1080 broadcast in MPEG-2 until January 6, 2007. Incidentally, the first HDTV was analog and successful (Japan, MUSE , regular operation 1991-2007 with visible 1035 / total 1125 lines)
- ↑ Strutz: Image data compression. SpringerVieweg, 2009
- ↑ ITU-T publication page: H.264: Advanced video coding for generic audiovisual services ITU-T, Jan 2012.
- ↑ H.264 / AVC JM Reference Software - Page at the HHI ; As of April 22, 2010
- ↑ http://blogs.citrix.com/2012/10/17/reinventing-hdx-adaptive-display-for-mobile-workers/
- ↑ 1st MPEG4-AVC / H.264 Comparison. Retrieved April 22, 2016 .
- ↑ 2-nd MPEG4-AVC / H.264 Comparison. Retrieved April 22, 2016 .
- ↑ 3-rd MPEG4-AVC / H.264 Comparison. Retrieved April 22, 2016 .
- ↑ 4-th MPEG-4 AVC / H.264 Video Codecs Comparison. Retrieved April 22, 2016 .
- ↑ 5-th MPEG-4 AVC / H.264 Video Codecs Comparison. Retrieved April 22, 2016 .
- ↑ 6th MPEG-4 AVC / H.264 Video Codecs Comparison. Retrieved April 22, 2016 .
- ↑ 7th MPEG-4 AVC / H.264 Video Codecs Comparison. Retrieved April 22, 2016 .
- ↑ 8-th MPEG-4 AVC / H.264 Video Codecs Comparison. Retrieved April 22, 2016 .
- ↑ HEVC Codec Comparison - 2015! Retrieved April 22, 2016 .
- ↑ MPEG LA (English) - official website; As of August 28, 2009
- ↑ License conditions for video codec H.264 / AVC published - article at heise online , dated November 18, 2003.
- ↑ Media (English) - Page at MPEG LA ; As of February 5, 2010.
- ↑ MPEG LA's AVC License Will Not Charge Royalties for Internet Video That Is Free to End Users Through Life of License (English) - Article on Business Wire , August 26, 2010.
- ↑ MPEG LA: Permanently no license costs for free internet streaming of MPEG-4 AVC (H.264) - article at heise online , from August 26, 2010.
- ↑ https://www.mpegla.com/wp-content/uploads/avc-att1.pdf
- ↑ Fit for high definition .