OpenSMILE
| openSMILE | |
|---|---|
| Basic data
|
|
| developer | audEERING GmbH |
| Publishing year | 2010 |
| Current version | 2.3 (October 28, 2016) |
| operating system | Linux , macOS , Windows |
| programming language | C ++ |
| category | Machine learning |
| License | Open source , proprietary |
| German speaking | No |
| audeering.com | |
openSMILE is an open source software for the automatic extraction of features from audio signals as well as for the classification of speech and music signals. "SMILE" stands for "Speech & Music Interpretation by Large-space Extraction". The software is primarily used in the field of automatic emotion recognition and is widely used in the affective computing research community. The openSMILE project has existed since 2008 and has been continued by the German company audEERING GmbH since 2013. openSMILE is offered free of charge for research purposes and personal use under an open source license. The company audEERING offers individual license options for commercial use of the tool.
application areas
openSMILE is used both in academic research and in commercial applications to automatically analyze speech and music signals in real time . In contrast to automatic speech recognition , which extracts the spoken content from a speech signal, openSMILE recognizes the characteristics of a speech or music segment. Examples of characteristics in human language are emotion , age, gender and personality of the speaker, as well as speaking states such as depression , drunkenness or pathological impairments of the voice. The software also includes music classification technologies to identify mood, chorus segments, key , chords , tempo , time signature , dance style and genre .
The openSMILE toolkit serves as a benchmark for many research competitions such as Interspeech ComParE, AVEC, MediaEval and EmotiW.
history
The openSMILE project was started in 2008 at the Technical University of Munich as part of the EU research project SEMAINE by Florian Eyben, Martin Wöllmer and Björn Schuller . The aim of the SEMAINE project was to develop a virtual agent with emotional and social intelligence . In this system, openSMILE was used for real-time analysis of speech and emotion. The openSMILE version 1.0.1 is used in the final release of SEMAINE.
In 2009 the first open source emotion recognition toolkit (openEAR) was published based on openSMILE. "EAR" stands for "Emotion and Affect Recognition".
In 2010 the openSMILE version 1.0.1 was presented and awarded at the ACM Multimedia Open Source Software Challenge .
Between 2011 and 2013, openSMILE was developed further by Florian Eyben and Felix Weninger as part of their doctoral thesis at the Technical University of Munich. The software was also used in the EU-funded ASC-Inclusion project and was expanded by Erik Marchi for this purpose.
In 2013 the company audEERING acquired the rights to the code base from the Technical University of Munich, and version 2.0 was published under an open source research license.
By 2016, openSMILE had been accessed more than 50,000 times worldwide and has established itself as the standard toolkit for emotion recognition.
Awards
openSMILE was awarded in 2010 in the context of the ACM Multimedia Open Source Competition . The tool is used in many scientific publications on the subject of automatic emotion recognition. openSMILE and the openEAR extension have been cited in over a thousand scientific publications.
Web links
- openSMILE website
- openSMILE book
- Google Scholar page on openSMILE
- Google Scholar page on openEAR
- Article on uepo.de
- Article on artificialintelligenceonline.com
Individual evidence
- ↑ F. Eyben, M. Wöllmer, B. Schuller: “ openSMILE - The Munich Versatile and Fast Open-Source Audio Feature Extractor ”, In Proc. ACM Multimedia (MM), ACM, Florence, Italy, ACM, pp. 1459-1462, October 2010.
- ↑ B. Schuller, B. Vlasenko, F. Eyben, M. Wöllmer, A. Stuhlsatz, A. Wendemuth, G. Rigoll, " Cross-Corpus Acoustic Emotion Recognition: Variances and Strategies (Extended Abstract) ," in Proc. of ACII 2015, Xi'an, China, invited for the Special Session on Most Influential Articles in IEEE Transactions on Affective Computing.
- ↑ B. Schuller, S. Steidl, A. Batliner, J. Hirschberg, JK Burgoon, A. Elkins, Y. Zhang, E. Coutinho: “ The INTERSPEECH 2016 Computational Paralinguistics Challenge: Deception & Sincerity ( Memento of the original from 9. June 2017 in the Internet Archive ) Info: The archive link was inserted automatically and has not yet been checked. Please check the original and archive link according to the instructions and then remove this notice. “, Proceedings INTERSPEECH 2016, ISCA, San Francisco, USA, 2016.
- ↑ F. Ringeval, B. Schuller, M. Valstar, R. Cowie, M. Pantic, “ AVEC 2015 - The 5th International Audio / Visual Emotion Challenge and Workshop ,” in Proceedings of the 23rd ACM International Conference on Multimedia, MM 2015 , (Brisbane, Australia), ACM, October 2015.
- ↑ M. Eskevich, R. Aly, D. Racca, R. Ordelman, S. Chen, GJ Jones, " The search and hyperlinking task at MediaEval 2014 ".
- ↑ F. Ringeval, S. Amiriparian, F. Eyben, K. Scherer, B. Schuller, “ Emotion Recognition in the Wild: Incorporating Voice and Lip Activity in Multimodal Decision-Level Fusion ,” in Proceedings of the ICMI 2014 EmotiW - Emotion Recognition In The Wild Challenge and Workshop (EmotiW 2014), Satellite of the 16th ACM International Conference on Multimodal Interaction (ICMI 2014), (Istanbul, Turkey), pp. 473-480, ACM, November 2014.
- ↑ scholar.google.de
- ↑ scholar.google.de
