Opus (audio format)

from Wikipedia, the free encyclopedia
opus
logo
File extension : .opus
MIME type : audio / ogg, audio / opus
Developed by: IETF Codec Working Group
Initial release: September 11, 2012
Type: Audio
Contained in: Ogg , WebM
Extended by: CELT , SILK
Standard (s) : RFC 6716 (September 2012)

RFC 8251 (October 2017)

Website : opus-codec.org

Opus is a data format for lossy audio data compression that is particularly suitable for interactive real-time transmissions over the Internet.

It is described as an international open standard in RFC 6716 . Frequency transformation and linear predictive coding (LPC) are used as basic methods . It enables particularly high sound quality and particularly low delay in transmissions.

features

Range of possible bit rates and algorithmic delays in comparison

Opus has a particularly low codec latency in order to cause as little delay ( latency ) as possible in real-time applications in processing the signal that is typically generated immediately before the compressed transmission . Opus works with block lengths from 2.5 to 20 (60) ms with dynamic, artifact-free change between different block lengths. Depending on the mode, there are also 2.5 to 5 ms of lookahead. It allows constant and variable bit rates over a very wide range from 6 kbit / s up to 510 kbit / s and the mapping of the entire human hearing range . Only two channels (stereo) can be linked . More channels can be displayed by multiplexing them (independently or possibly coupled in pairs) together in a container file. The EBU recommendation R 128 is supported for loudness adjustments at the receiver end .

The procedure is openly documented as an open standard and a reference implementation is published in the source code. Parts of the process are encumbered with software patents , with the rights holders having consented to unrestricted use of their patents in the context of the use of the codec, including future versions of the standard. However, all of them reserve the right to use their patents to defend against patent claims by third parties.

Opus is a hybrid process of CELT and a heavily modified, incompatible version of SILK . The method knows three kinds of modes, two for pure speech signals and one unspecialized for music, for example. With the speech modes, a mode is available in which the entire human hearing range is mapped, whereby the algorithms of CELT, which are essentially based on frequency transformation (MDCT), for an upper frequency component from 8 kHz, which are essentially based on linear predictive coding (LPC ) based SILK algorithms are responsible for the lower. For low bit rates (meaning below about 30 kbit / s) the frequency range can be restricted and the CELT layer omitted. For other types of signals, the SILK layer, which specializes in voice signals, can be switched off and only the unspecialized CELT can be used. From version 0.9.2 (March 2011) you can switch seamlessly between these modes during operation and the encoder automatically selects the mode by default.

Opus data can be packed in Ogg containers. The content of such Ogg-Opus data streams is then audio/ogg; codecs=opusspecified and the file name extension is .opusrecommended for Ogg-Opus files . Support for the encapsulation of Opus in the Matroska container format is in progress.

Sound quality

Coding efficiency curve in comparison

In comparative listening tests to Opus shows at low bit rates to date through the use of proprietary spectral band replication dominant HE-AAC superior in quality codecs. At bit rates of 12 kbit / s and below, a version of the codec from mid-February 2011 was subject to the AMR codecs from GSM for voice signals , which represent the state of the art in terms of quality. For voice signals, the hybrid mode proved to be superior internally at bit rates between around 20 and 48 kbit / s - above that the purely MDCT-based and below that the purely LPC-based mode. In contrast to other common transformation methods, strongly tonal signals are particularly difficult for Opus and complex passages can be represented relatively sparingly.

technology

Spectrogram of an Opus-encoded music recording at different average bit rates (≈32 to ≈160 kbit / s) clearly shows the low-pass behavior of the encoder and the comparatively good preservation of the band energy by CELT (compare original, Vorbis, MP3, AAC ).

Opus is a hybrid codec that combines two different, originally separate processes. A transformation layer (originally CELT) works on the basis of the modified discrete cosine transformation (MDCT) and approaches from CELP (code book for excitation, but in the frequency domain). A layer specializing in speech signals (originally SILK) is based on Linear Predictive Coding (LPC). The original SILK was modified and, among other things, support for 10 millisecond blocks was added. The common area coding of the two parts of a hybrid data stream was taken over by CELT. The LPC part works internally with a sampling frequency of 16 kHz. The encoder has a built-in sample rate converter . To compensate for the lower lookahead of the CELT signal, it is delayed accordingly. SILK has a greater algorithmic delay in order to minimize the administrative data arising from the transmission protocols in its typical application scenario. For example, lower latencies are possible in pure CELT operation.

Transformation Layer (CELT)

See also article CELT, section technology

Technically, the transformation layer of Opus works largely in the same way as the abandoned independent CELT. However, it was modified and further developed for integration with SILK. Added support for 20 millisecond blocks and signalable deviations from the defined allocation of available bits to the (Bark) bands with a so-called “allocation tilt” and a so-called “band boost”.

Involved persons and organizations

Opus is recommended by the Internet Engineering Task Force (IETF) as Request for Comments (RFC) 6716 as the international open standard for lossy audio data compression on the Internet. It was developed in the standardization process at the IETF by the codec working group with staff from and on the basis of initially separate proposals from the Xiph.Org Foundation and Skype Technologies SA (now Microsoft ). The main developers are Jean-Marc Valin (Xiph.Org, Octasic, Mozilla Corporation ), Koen Vos (Skype) and Timothy B. Terriberry (Mozilla Corporation). Raymond Chen ( Broadcom ), Gregory Maxwell (Xiph.Org) and Christopher Montgomery (Xiph.Org) were also involved.

Mozilla pays lead developer Valin a salary for his development work on Opus as part of a paid job. The browser manufacturer Opera Software also explicitly supports Opus as a new, open standard. Google Inc. is committed to establishing Opus as a license-free standard format on the Internet. Microsoft's Skype department is still actively involved in the standardization process as (co-) initiator. Broadcom's Juin-Hwey (Raymond) Chen contributed a pre- and post-filter for pitch prediction in CELT. Other participants in the standardization process at the IETF were representatives of the Chair for Communication Networks at the University of Tübingen and its commercial spin-off Symonics, Polycom and Cisco Systems .

Broadcom and the Xiph.Org Foundation hold patents relating to CELT, and Skype / Microsoft patents are relevant to the SILK portion. Alleged patent claims by Qualcomm and Huawei turned out to be incorrect.

history

The development of the CELT part goes back to considerations for a successor for Vorbis , as its successor it can also be considered in terms of the characteristics. Furthermore, as the new speech codec of the Xiph Foundation , Opus is replacing its older speech codec Speex , which is also the predecessor project of the main developer Jean-Marc Valin, with whom he gained experience in designing audio codecs. Practical work has been going on at CELT since November 2007. The SILK part, which has been developed by Skype since January 2007, is the follow-up project to the SVOPC , which, as an in-house development, should make the company independent of the external, chargeable, licensed solutions iSAC and iLBC .

See also article CELT , history section and article SILK , history section

In March 2009, Skype encouraged the IETF to develop and standardize a broadband audio codec. Almost a year then passed with long and heated debates about the formation of a working group. Representatives of various companies that were involved in the standardization of patented competing formats expressed objections to the start of standardization work for a codec free of license fees. Representatives of the companies Polycom and Ericsson - the makers and license sellers of G.719 - as well as France Télécom , Huawei and the Orange Labs (division of France Télécom), which were involved in G.718 , appeared. In February 2010 the working group finally came into being, which was even assured of support from the corresponding Study Group 16 of the ITU-T .

In July a first prototype of a hybrid format was published that combined the two codec candidates SILK and CELT submitted - the original version of Opus. The format was submitted to the IETF for standardization in September 2010. In the meantime it was known as "Harmony" for a short time before it got its current name in October 2010. The bit stream format has been provisionally set since the beginning of February 2011, subject to final changes. At the end of July 2011, Jean-Marc Valin got a paid job at Mozilla Corporation to work (more) on Opus. The IETF working group last called for changes to the bitstream format in November 2011. On July 2, 2012, the format was accepted by the IETF. The software had been in the release candidate stage since August 8th . The final specification and version 1.0 and 1.0.1 of the reference software were officially published on September 11, 2012.

Support for the format was built into a variety of software relatively quickly. In the same year, numerous corresponding development versions were available, including from influential and popular projects such as the Firefox web browser , the Debian operating system and the VLC media player .

Version 1.1

In the meantime, an experimental development branch of the reference coder has begun to work towards a version 1.1 with significantly better sound quality. On December 21, 2012, after more than a year of development, the first alpha version of the 1.1 series was released. The beta phase for version 1.1 began on July 11, 2013 and version 1.0.3 was released, which fixes some bugs and takes over the surround sound API of the 1.1 series. The completed version 1.1 was published on December 5, 2013.

Version 1.1 of the reference coder has reportedly achieved significantly better sound quality and efficiency, especially with particularly tonal sequences, by using more options in the format and improved coding decisions. For this purpose, it uses, among other things, a dynamic distribution of the available bit rate between frequency bands ("dynalloc" - "band boost", "allocation tilt") the possibilities of the format for variability of the bit rate (VBR). It has an unrestricted VBR mode and adapts the bit rate more aggressively to the complexity of the source material. With the new VBR mode, an attempt is now made to achieve a constant quality across files and no longer to achieve the specified target bit rate per file. The encoder was calibrated in order to approximate the specified target bit rate on average with a large amount of encoded material with a broad mixture of different, typical useful signals. Several new analysis steps examine signal characteristics and inform coding decisions. Among other things, based on an assessment of the tonality of particularly tonal passages, the bit rate is increased in a targeted manner and the system automatically switches between the integrated LPC-based speech codec, the MDCT-based coding and the hybrid mode by recognizing speech signals. For surround sound formats with more than two channels, there is now a dynamic bit rate allocation to the individual channels using masking effects between the channels and there have been quality improvements for the LFE channel. Due to the initial code optimization, the entire reference codec now has a significantly higher operating speed, especially on ARM devices.

Further innovations are the possibility of predictive detection of signal characteristics for operating scenarios when time delays are not critical, the more efficient display of strongly correlated stereo signals, the discarding of direct current components (3 Hz high pass ), the temporal variation of the bit rate based on the volume and a Level limiter to prevent hard clipping .

Version 1.1.1 with assembler optimizations for x86 (SSE, SSE2, SSE4.1), MIPS and ARM (NEON) was released on November 26, 2015 .

software

libopus

Opus logo2.svg
Libopus screenshot.png
Basic data

developer Xiph.Org Foundation
Publishing year 2010
Current  version 1.3.1
(April 12, 2019)
operating system platform independent
programming language C.
category Audio compression
License BSD license
Github

Opus is openly documented and with libopus a free program library is available. It is written in C and can be compiled for architectures with and without floating point units.

FFmpeg has had its own native Opus encoder and decoder since 2017 .

The supplied diagnostic tool "opusinfo" reports detailed technical information on Opus files, including the correctness of the bit stream format. It is a spin-off of ogginfo from vorbis-tools and is therefore available under version 2 of the GPL, unlike encoders and decoders .

Format support

There is extensive support in software for simple file or stream playback, IP telephony or audio streaming, in web browsers, operating systems, and others. Important examples:

  • Support in playback software can be found in VLC media player, foobar2000 , AIMP, and others.
  • VoIP software includes the popular audio chat programs Discord , Mumble and TeamSpeak , SIP clients such as Phoner / PhonerLite or SFLphone or the chat client Jitsi . The messengers Signal and Threema also use Opus for telephony via the app.
  • The Chrome and Firefox web browsers support Opus. In addition, Opus support for WebRTC compatible browsers is mandatory.
  • Audio streaming in Opus is possible with software such as Icecast and Krad Radio.

Support is available on many platforms:

  • The reference implementation is available for many operating systems and hardware platforms.
  • Corresponding DirectShow filters and are available for Windows
  • In important GNU / Linux operating systems ( Debian and descendants such as Ubuntu , Fedora ) software such as the reference implementations and the multimedia frameworks GStreamer and libavcodec ( FFmpeg / Libav ) can be installed directly from the standard package sources.
  • The mobile platform Android supports version 5 (Lollipop) Opus embedded in the Matroska - container format natively. Opus in Ogg containers, which is the more common form especially for podcasts, was not supported. From version 7, Opus was also supported in the Ogg container, but only files with the extension * .ogg were recognized as such, and not * .opus, which made the support only very limited in practice. From Android 8 onwards, Opus is supported in all common variants. There are also a number of software options for Opus playback.
  • The alternative firmware Rockbox supports a number of portable media playback devices (including products from the iPod range from Apple and iriver and Archos devices).

Web links

Commons : Opus (audio codec)  - album containing pictures, videos and audio files

Individual evidence

  1. ↑ a b Timothy B. Terriberry, Ron Lee, Ralph Giles:  RFC 7845  - Ogg Encapsulation for the Opus Audio Codec . April 2016. Section 9: Content Type. (Updated by RFC 8486  - Internet Engineering Task Force - English).
  2. ↑ Network Working Group: RTP Payload Format and File Storage Format for Opus Speech and Audio Codec . In: Opus codec . IETF. July 4, 2011. Retrieved October 26, 2011.
  3. ↑ a b audio recording ( memento of the original from February 11, 2013 in the Internet Archive ) Info: The archive link was inserted automatically and has not yet been checked. Please check the original and archive link according to the instructions and then remove this notice. from the meeting of the IETF Codec Working Group at the IETF78 conference in Nagasaki, Japan (MP3, ~ 70 MiB)  @1@ 2Template: Webachiv / IABot / nagasaki.bogus.com
  4. ↑ JM. Valin, K. Vos, T. Terriberry:  RFC 6716 . - Definition of the Opus Audio Codec . [Errata: RFC 6716 ]. September 2012. (English).
  5. ↑ wiki.xiph.org
  6. ^ Next-Gen Low-Latency Open Codec Beats HE-AAC , Slashdot report from April 14, 2011
  7. ↑ Anssi Ramo, Henri Toukomaa: Voice Quality Characterization of IETF Opus Codec, Proceedings of Interspeech 2011, Florence, Italy, August 2011.
  8. ^ Christian Hoene (Editor), Jean-Marc Valin, Koen Vos, Jan Skoglund: Summary of Opus listening test results. Internet Engineering Task Force, May 17, 2013, accessed January 17, 2017 .
  9. ↑ Audio codec "Opus" is the new Internet standard. In: heise.de. Retrieved September 12, 2012 .
  10. ↑ lists.xiph.org
  11. ↑ hacks.mozilla.org
  12. ↑ ietf.org
  13. ↑ heise.de
  14. ↑ ietf.org
  15. ↑ ietf.org
  16. ↑ hydrogenaudio.org
  17. ↑ jmspeex.livejournal.com
  18. ↑ Volker Zota: Universal audio codec Opus on the way to the Internet standard . heise.de; 5th July 2012
  19. ↑ lists.xiph.org
  20. ↑ lists.xiph.org
  21. ↑ git.xiph.org
  22. ↑ hydrogenaudio.org
  23. ↑ lists.xiph.org
  24. ↑ Opus Codec (section News )
  25. ↑ jmspeex.livejournal.com
  26. ↑ hydrogenaudio.org
  27. ↑ hydrogenaudio.org
  28. ^ Jean-Marc Valin: The Opus Audio Codec . Ed .: The Xiph.Org Foundation & The Mozilla Corporation. November 2012, Current development, p. 50 (English, jmvalin.ca [PDF]). jmvalin.ca ( Memento of the original from March 5, 2016 in the Internet Archive ) Info: The archive link was inserted automatically and has not yet been checked. Please check the original and archive link according to the instructions and then remove this notice.  @1@ 2Template: Webachiv / IABot / jmvalin.ca
  29. ↑ Christopher Montgomery : Opus update 20130712: 1.1 Beta Release (demo 3). In: Monty's demo pages. Xiph.Org Foundation , July 14, 2013, accessed July 17, 2013 .
  30. ↑ Volker Zota: Universal audio codec Opus accelerated . heise, November 28, 2013
  31. ↑ opus-codec.org
  32. ↑ License
  33. ^ Opus native FFmpeg encoder under development , HydrogenAudio
  34. ↑ FFmpeg Codecs Documentation
  35. ↑ trac.videolan.org
  36. ↑ hydrogenaudio.org
  37. ↑ Decoding support from version 3.20 Build 1125 Beta 1 from September 2nd, 2012, see aimp.ru
  38. ↑ Information on the implementation of the Opus Codec in version 1.2. In: Discord Support. Retrieved January 6, 2019 (American English).
  39. ↑ mumble.sourceforge.net ( memento of the original from July 9, 2014 in the Internet Archive ) Info: The archive link was automatically inserted and not yet checked. Please check the original and archive link according to the instructions and then remove this notice.  @1@ 2Template: Webachiv / IABot / mumble.sourceforge.net
  40. ↑ phonerlite.de ( Memento of the original from September 26, 2012 in the Internet Archive ) Info: The archive link was inserted automatically and has not yet been checked. Please check the original and archive link according to the instructions and then remove this notice.  @1@ 2Template: Webachiv / IABot / www.phonerlite.de
  41. ↑ projects.savoirfairelinux.com ( Memento of the original from December 17th, 2012 in the web archive archive.today ) Info: The archive link was automatically inserted and not yet checked. Please check the original and archive link according to the instructions and then remove this notice.  @1@ 2Template: Webachiv / IABot / projects.savoirfairelinux.com
  42. ↑ jitsi.org ( Memento of the original from May 20, 2012 in the Internet Archive ) Info: The archive link was inserted automatically and has not yet been checked. Please check the original and archive link according to the instructions and then remove this notice. jitsi.org ( Memento of the original from October 27, 2012 in the Internet Archive ) Info: The archive link was inserted automatically and has not yet been checked. Please check the original and archive link according to the instructions and then remove this notice.  @1@ 2Template: Webachiv / IABot / jitsi.org  @1@ 2Template: Webachiv / IABot / jitsi.org
  43. ↑ developer.mozilla.org ( Memento of the original from June 27, 2010 in the Internet Archive ) Info: The archive link was inserted automatically and has not yet been checked. Please check the original and archive link according to the instructions and then remove this notice.  @1@ 2Template: Webachiv / IABot / developer.mozilla.org
  44. ↑ heise online: Two audio codecs for real-time communication in the browser. In: heise online. Retrieved August 29, 2016 .
  45. ↑ lists.xiph.org
  46. ↑ github.com  ( page no longer available , search in web archives )  Info: The link was automatically marked as defective. Please check the link according to the instructions and then remove this notice. @1@ 2Template: Dead Link / github.com  
  47. ↑ reino.degeelebosch.nl ( Memento of the original from September 20, 2012 in the Internet Archive ) Info: The archive link was inserted automatically and has not yet been checked. Please check the original and archive link according to the instructions and then remove this notice.  @1@ 2Template: Webachiv / IABot / reino.degeelebosch.nl
  48. ↑ lavfilters.googlecode.com ( Memento of the original from January 3, 2013 in the web archive archive.today ) Info: The archive link was inserted automatically and has not yet been checked. Please check the original and archive link according to the instructions and then remove this notice.  @1@ 2Template: Webachiv / IABot / lavfilters.googlecode.com
  49. ↑ packages.debian.org
  50. ↑ packages.ubuntu.com
  51. ↑ apps.fedoraproject.org
  52. ↑ gstreamer.freedesktop.org
  53. ↑ trac.ffmpeg.org
  54. ↑ patches.libav.org ( Memento of the original from December 20, 2013 in the Internet Archive ) Info: The archive link has been inserted automatically and has not yet been checked. Please check the original and archive link according to the instructions and then remove this notice.  @1@ 2Template: Webachiv / IABot / patches.libav.org
  55. ↑ Supported Media Formats. In: Android.com. Retrieved January 28, 2016 .
  56. ↑ Android 7 Compatibility Definition. Accessed September 3, 2018 .
  57. ^ VLC
  58. ^ "Rockbox as an Application" (RaaA), rockbox.org
  59. ↑ Firefox
  60. ↑ GoneMAD Music Player (GMMP) for Android, from version 1.4, see gonemadmusicplayer.blogspot.de
  61. ↑ Neutron Music Player for Android, version 1.63 or higher , see neutronmp.com
  62. ↑ forum.powerampapp.com
  63. ↑ BS.Player for Android, see forum.bsplayer.com
  64. ↑ rockbox.org