MPEG-4 Part 3 or MPEG-4 Audio (formally ISO/IEC 14496-3) is the third part of the ISO/IEC MPEG-4 international standard developed by Moving Picture Experts Group. It specifies audio coding methods. The first version of ISO/IEC 14496-3 was published in 1999.
The MPEG-4 Part 3 consists of a variety of audio coding technologies – from lossy speech coding (HVXC, CELP), general audio coding (AAC, TwinVQ, BSAC), lossless audio compression (MPEG-4 SLS, Audio Lossless Coding, MPEG-4 DST), a Text-To-Speech Interface (TTSI), Structured Audio (using SAOL, SASL, MIDI) and many additional audio synthesis and coding techniques.
MPEG-4 Audio does not target a single application such as real-time telephony or high-quality audio compression. It applies to every application which requires the use of advanced sound compression, synthesis, manipulation, or playback. MPEG-4 Audio is a new type of audio standard that integrates numerous different types of audio coding: natural sound and synthetic sound, low bitrate delivery and high-quality delivery, speech and music, complex soundtracks and simple ones, traditional content and interactive content.
Versions +MPEG-4 Audio versions and editions Edition Release date Latest amendment Standard Description First edition 1999 2001 ISO/IEC 14496-3:1999 also known as "MPEG-4 Audio Version 1" 2000 ISO/IEC 14496-3:1999/Amd 1:2000 also known as "MPEG-4 Audio Version 2", an Amendment to first edition Second edition 2001 2005 ISO/IEC 14496-3:2001 Third edition 2005 2008 ISO/IEC 14496-3:2005 Fourth edition 2009 2015 and under development ISO/IEC 14496-3:2009 Fifth edition 2019 ISO/IEC 14496-3:2019 Current version
Subparts MPEG-4 Part 3 contains following subparts: Subpart 1: Main (list of Audio Object Types, Profiles, Levels, interface to ISO/IEC 14496-1, MPEG-4 Audio transport stream, etc.) Subpart 2: Speech coding – HVXC (Harmonic Vector eXcitation Coding) Subpart 3: Speech coding – CELP (Code Excited Linear Prediction) Subpart 4: General Audio Coding (GA) (Time/Frequency Coding) – AAC, TwinVQ, BSAC Subpart 5: Structured Audio (SA) Subpart 6: Text to Speech Interface (TTSI) Subpart 7: Parametric Audio Coding – HILN (Harmonic and Individual Line plus Noise) Subpart 8: Technical description of parametric coding for high quality audio (SSC, Parametric Stereo) Subpart 9: MPEG-1/MPEG-2 Audio in MPEG-4 Subpart 10: Technical description of lossless coding of oversampled audio (MPEG-4 DST – Direct Stream Transfer) Subpart 11: Audio Lossless Coding (ALS) Subpart 12: Scalable Lossless Coding (SLS)
MPEG-4 Audio Object Types MPEG-4 Audio includes a system for handling a diverse group of audio formats in a uniform manner. Each format is assigned a unique Audio Object Type to represent it. Object Type is used to distinguish between different coding methods. It directly determines the MPEG-4 tool subset required to decode a specific object. The MPEG-4 profiles are based on the object types and each profile supports a different list of object types.
+MPEG-4 Audio Object Types Object Type ID Audio Object Type First public release date Description 1 AAC Main 1999 contains AAC LC 2 AAC LC (Low Complexity) 1999 Used in the "AAC Profile". MPEG-4 AAC LC Audio Object Type is based on the MPEG-2 Part 7 Low Complexity profile (LC) combined with Perceptual Noise Substitution (PNS) (defined in MPEG-4 Part 3 Subpart 4). 3 AAC SSR (Scalable Sample Rate) 1999 MPEG-4 AAC SSR Audio Object Type is based on the MPEG-2 Part 7 Scalable Sampling Rate profile (SSR) combined with Perceptual Noise Substitution (PNS) (defined in MPEG-4 Part 3 Subpart 4). 4 AAC LTP (Long Term Prediction) 1999 contains AAC LC 5 SBR (Spectral Band Replication) 2003 used with AAC LC in the "High Efficiency AAC Profile" (HE-AAC v1) 6 AAC Scalable 1999 7 TwinVQ 1999 audio coding at very low bitrates 8 CELP (Code Excited Linear Prediction) 1999 speech coding 9 HVXC (Harmonic Vector eXcitation Coding) 1999 speech coding 10 (Reserved) 11 (Reserved) 12 TTSI (Text-To-Speech Interface) 1999 13 Main synthesis 1999 contains 'wavetable' sample-based synthesis and Algorithmic Synthesis and Audio Effects 14 'wavetable' sample-based synthesis 1999 based on SoundFont and DownLoadable Sounds, contains General MIDI 15 General MIDI 1999 16 Algorithmic Synthesis and Audio Effects 1999 17 ER AAC LC 2000 Error Resilient 18 (Reserved ) 19 ER AAC LTP 2000 Error Resilient 20 ER AAC Scalable 2000 Error Resilient 21 ER TwinVQ 2000 Error Resilient 22 ER BSAC (Bit-Sliced Arithmetic Coding) 2000 It is also known as "Fine Granule Audio" or fine grain scalability tool. It is used in combination with the AAC coding tools and replaces the noiseless coding and the bitstream formatting of MPEG-4 Version 1 GA coder. Error Resilient 23 ER AAC LD (Low Delay) 2000 Error Resilient, used with CELP, ER CELP, HVXC, ER HVXC and TTSI in the "Low Delay Profile", (commonly used for real-time conversation applications) 24 ER CELP 2000 Error Resilient 25 ER HVXC 2000 Error Resilient 26 ER HILN (Harmonic and Individual Lines plus Noise) 2000 Error Resilient 27 ER Parametric 2000 Error Resilient 28 SSC (SinuSoidal Coding) 2004 29 PS (Parametric Stereo) 2004 and 2006 used with AAC LC and SBR in the "HE-AAC v2 Profile". PS coding tool was defined in 2004 and Object Type defined in 2006. 30 MPEG Surround 2007 also known as MPEG Spatial Audio Coding (SAC), it is a type of spatial audio coding (MPEG Surround was also defined in ISO/IEC 23003-1 in 2007) 31 (ESCAPE) 32 MPEG-1/2 Layer-1 2005 33 MPEG-1/2 Layer-2 2005 34 MPEG-1/2 Layer-3 2005 also known as "MP3onMP4" 35 DST (Direct Stream Transfer) 2005 lossless audio coding, used on Super Audio CD 36 ALS (Audio Lossless Coding) 2006 lossless audio coding 37 SLS (Scalable Lossless Coding) 2006 two-layer audio coding with lossless layer and lossy General Audio core/layer (e.g. AAC) 38 SLS non-core 2006 lossless audio coding without lossy General Audio core/layer (e.g. AAC) 39 ER AAC ELD (Enhanced Low Delay) 2008 Error Resilient 40 SMR (Symbolic Music Representation) Simple 2008 note: Symbolic Music Representation is also the MPEG-4 Part 23 standard (ISO/IEC 14496-23:2008) 41 SMR Main 2008 42 USAC (Unified Speech and Audio Coding) 2012 Unified Speech and audio Coding is defined in MPEG-D Part 3 (ISO/IEC 23003-3:2012) 43 SAOC (Spatial Audio Object Coding) 2010 note: Spatial Audio Object Coding is also the MPEG-D Part 2 standard (ISO/IEC 23003-2:2010) 44 LD MPEG Surround 2010 This object type conveys Low Delay MPEG Surround Coding side information (that was defined in MPEG-D Part 2 – ISO/IEC 23003-2) in the MPEG-4 Audio framework. 45 SAOC-DE 2013 Spatial Audio Object Coding Dialogue Enhancement 46 Audio Sync 2015 The audio synchronization tool provides capability of synchronizing multiple contents in multiple devices.
Audio Profiles
The MPEG-4 Audio standard defines several profiles. These profiles are based on the object types and each profile supports different list of object types. Each profile may also have several levels, which limit some parameters of the tools present in a profile. These parameters usually are the sampling rate and the number of audio channels decoded at the same time.
+MPEG-4 Audio Profiles Audio Profile Audio Object Types First public release date AAC Profile AAC LC 2003 High Efficiency AAC Profile AAC LC, SBR 2003 HE-AAC v2 Profile AAC LC, SBR, PS 2006 Main Audio Profile AAC Main, AAC LC, AAC SSR, AAC LTP, AAC Scalable, TwinVQ, CELP, HVXC, TTSI, Main synthesis 1999 Scalable Audio Profile AAC LC, AAC LTP, AAC Scalable, TwinVQ, CELP, HVXC, TTSI 1999 Speech Audio Profile CELP, HVXC, TTSI 1999 Synthetic Audio Profile TTSI, Main synthesis 1999 High Quality Audio Profile AAC LC, AAC LTP, AAC Scalable, CELP, ER AAC LC, ER AAC LTP, ER AAC Scalable, ER CELP 2000 Low Delay Audio Profile CELP, HVXC, TTSI, ER AAC LD, ER CELP, ER HVXC 2000 Natural Audio Profile AAC Main, AAC LC, AAC SSR, AAC LTP, AAC Scalable, TwinVQ, CELP, HVXC, TTSI, ER AAC LC, ER AAC LTP, ER AAC Scalable, ER TwinVQ, ER BSAC, ER AAC LD, ER CELP, ER HVXC, ER HILN, ER Parametric 2000 Mobile Audio Internetworking Profile ER AAC LC, ER AAC Scalable, ER TwinVQ, ER BSAC, ER AAC LD 2000 HD-AAC Profile AAC LC, SLS 2009 ALS Simple Profile ALS 2010
Audio storage and transport +Multiplex, storage and transmission formats for MPEG-4 Audio Standard Description Multiplex ISO/IEC 14496-1 MPEG-4 Multiplex scheme (M4Mux) Multiplex ISO/IEC 14496-3 Low Overhead Audio Transport Multiplex (LATM) Storage ISO/IEC 14496-3 (informative) Audio Data Interchange Format (ADIF) – only for AAC Storage ISO/IEC 14496-12 MPEG-4 file format (MP4) / ISO base media file format Transmission ISO/IEC 14496-3 (informative) Audio Data Transport Stream (ADTS) – only for AAC Transmission ISO/IEC 14496-3 Low Overhead Audio Stream (LOAS), based on LATM
There is no standard for transport of elementary streams over a channel, because the broad range of MPEG-4 applications have delivery requirements that are too wide to easily characterize with a single solution.
The capabilities of a transport layer and the communication between transport, multiplex, and demultiplex functions are described in the Delivery Multimedia Integration Framework (DMIF) in ISO/IEC 14496-6. A wide variety of delivery mechanisms exist below this interface, e.g., MPEG transport stream, Real-time Transport Protocol (RTP), etc.
Transport in Real-time Transport Protocol is defined in RFC 3016 (RTP Payload Format for MPEG-4 Audio/Visual Streams), RFC 3640 (RTP Payload Format for Transport of MPEG-4 Elementary Streams), RFC 4281 (The Codecs Parameter for "Bucket" Media Types) and RFC 4337 (MIME Type Registration for MPEG-4).
LATM and LOAS were defined for natural audio applications, which do not require sophisticated object-based coding or other functions provided by MPEG-4 Systems.
Bifurcation in the AAC technical standard
The Advanced Audio Coding in MPEG-4 Part 3 (MPEG-4 Audio) Subpart 4 was enhanced relative to the previous standard MPEG-2 Part 7 (Advanced Audio Coding), in order to provide better sound quality for a given encoding bitrate.
It is assumed that any Part 3 and Part 7 differences will be ironed out by the ISO standards body in the near future to avoid the possibility of future bitstream incompatibilities. At present there are no known player or codec incompatibilities due to the newness of the standard.
The MPEG-2 Part 7 standard (Advanced Audio Coding) was first published in 1997 and offers three default profiles: Low Complexity profile (LC), Main profile and Scalable Sampling Rate profile (SSR).
The MPEG-4 Part 3 Subpart 4 (General Audio Coding) combined the profiles from MPEG-2 Part 7 with Perceptual Noise Substitution (PNS) and defined them as Audio Object Types (AAC LC, AAC Main, AAC SSR).
HE-AAC
High-Efficiency Advanced Audio Coding is an extension of AAC LC using spectral band replication (SBR), and Parametric Stereo (PS). It is designed to increase coding efficiency at low bitrates by using partial parametric representation of audio.
AAC-SSR AAC Scalable Sample Rate was introduced by Sony to the MPEG-2 Part 7 and MPEG-4 Part 3 standards. It was first published in ISO/IEC 13818-7, Part 7: Advanced Audio Coding (AAC) in 1997. The audio signal is first split into 4 bands using a 4 band polyphase quadrature filter bank. Then these 4 bands are further split using MDCTs with a size k of 32 or 256 samples. This is similar to normal AAC LC which uses MDCTs with a size k of 128 or 1024 directly on the audio signal.
The advantage of this technique is that short block switching can be done separately for every PQF band. So high frequencies can be encoded using a short block to enhance temporal resolution, low frequencies can be still encoded with high spectral resolution. However, due to aliasing between the 4 PQF bands, coding efficiency around (1,2,3) * fs/8 is worse than with normal MPEG-4 AAC LC.
MPEG-4 AAC-SSR is very similar to ATRAC and ATRAC-3.