NEXT-GENERATION AUDIO NEW OPPORTUNITIES FOR TERRESTRIAL UHD BROADCASTING
What Is Next-Generation Audio? Immersive Sound A viewer becomes part of the audience Delivered to mainstream consumers, not just enthusiast viewers Interactivity Hear your home team or language Turn the announcer or dialogue up or down Hear your pit crew Universal Delivery Home Theater Kitchen TV Phone earbuds on plane VR Headset 3D Soundbar Reference Design MPEG-H Android TV App
How Do We Make Next-Generation Audio? The MPEG-H Audio Elements Audio Channels Signals that correspond to speaker positions just as for stereo and surround Static Audio Objects A channel-like signal that can be adjusted in volume during playback Static Audio Objects Cinema-like signal that can be panned and faded over time by the broadcaster and/or the viewer
How Do We Make Next-Generation Audio? The MPEG-H Audio Elements (II) Ambisonics Set of signals that describe the sound field at a point in space, usually captured with a multi-element ball microphone. Popular for virtual reality due to ease of rotation in viewer Audio Scene Metadata Describes one or more programs ( presets ) that are a combination of the above elements. MPEG-H rendering allows any combination to be played in any viewer s speaker layout or head orientation. Advanced DRC Dynamic Range Control information to tailor presentation to the viewer s device and environment
MPEG-H Flexible Content Formats Up to 16 audio signals in MPEG-H LC Profile Level 3 Every combination of channel signals, objects or HOA Several versions from one audio stream: Presets Channel Based Audio Object Based Audio MPEG- H Audio Combination of these formats + Metadata Scene Based Audio (HOA)
Reproducing Immersive Sound in the Home Display from our 3D interactive object panner For enthusiasts: 5.1 surround + 4 overhead speakers (or even 7.1 + 4H) 3D Soundbar Reference Design For mainstream consumers: 3D Soundbars
Interactivity opens new creative possibilities An audio object is basically a channel that can be positioned or panned and adjusted in volume during playback Objects allow elements of the audio mix to be sent separately to the TV, allowing the viewer to adjust the mix to his preference Presets allow the casual viewer to experience different prepared mixes Current Approach: Compromise mix made by A1 New Approach: Viewer selects preset or makes his own mix (within broadcaster-set limits) Transmit sub-mixes as audio objects MPEG-H
MPEG-H Audio Personalization Features Choose between content versions Change the language of a program Change balance between dialogue and background Interactivity: Select additional tracks (On-Off) Change the position of sound events Gain Control
MPEG-H 3D Audio Production Environment
MPEG-H Production Environment Post Production Scenario 3D Plug-In Composition of 3D sound Metadata (control track) Monitoring & Authoring Monitoring & Authoring
Production Tools for 3D Audio Audio Workstations and Plug-Ins Nuendo DSpatial Pyramix Sound Particles
MPEG-H Production Environment 3D Plug-In Spatial Audio Designer (NAT) Leading Plug-In for surround and 3D content production (movie, music, games) on ProTools Wide range of DAW support and platforms (Win, Mac) ProTools, Nuendo, Cubase Support of MPEG-H metadata Rendering Authoring Read/Write Control Track
MPEG-H Production Tools MPEG-H Authoring Tool
MPEG-H Production Environment Live Broadcast Scenario 3D Plug-In Composition of 3D sound Metadata (control track) Monitoring & Authoring Metadata (control track) Rendering Monitoring & Authoring Monitoring & Authoring
MPEG-H Production Environment Jünger Multichannel Monitoring & Authoring (MMA) MPEG-H Rendering Setup-Box Emulation (speaker setup, downmix, DRC profiles, Interactivity) MPEG-H Authoring Scene, Content, Preset Object position + panning Loudness metering & control Generation of control track
MPEG-H Production Environment Linear Acoustic Authoring and Monitoring System (AMS)
MPEG-H Production Tools Linear Acoustic - Authoring & Monitoring System (AMS) MPEG-H Authoring MPEG-H Rendering Direct Connection to consoles via Audio over IP
MPEG-H 3D Audio Broadcast Implementations
MPEG-H Application Standards and Market Introduction ATSC and DVB have adapted MPEG-H 3D Audio in their standards Korea will launch their terrestrial UHDTV service based on the ATSC 3.0 standard using MPEG-H 3D Audio as the only audio codec. This will be the first regular service using a Next Generation Audio worldwide. The service launch is scheduled for May 31, 2017. Test transmissions are currently already on air.
MPEG-H Audio Broadcast Encoders Announced at NAB 2016: KaiMedia DS Broadcast Announced in Sept. 2016: Media Excel Announced in Jan. 2017 Pixtree
First-Generation ATSC 3.0 TVs Enter Korean Market March 2017: LG TV models on sale in Korean stores April 2017: Samsung TV models on sale in Korean stores
MPEG-H 3D Audio Use Case Virtual Reality
MPEG-H Audio System A New Codec Design Perfect for VR Ideal for Virtual Reality: Supports audio channels, objects and ambisonics audio (FOA/HOA) Audio bit stream can contain all audio elements plus metadata Height information creates immersive experience of being there Excellent coding efficiency enables 3D audio at 2D bit rates MPEG-H Low Complexity (LC) Profile for best quality & efficient implementation Dedicated VR production tools available
Immersive Audio Production Tools for VR Post-production: Composer DAW VST / AAX Plugin Mix channel-beds, ambisonic soundfields and audio objects in virtual spaces Export audio essence plus metadata for MPEG-H encoding Post-production Unity Plugin Immersive Audio Plugin or Unity-based VR app development Binaurally renders audio essence and metadata from Cingo composer plugin
MPEG-H Audio System What do objects do for VR Audio? Personalization / Interactive Features Live VR benefits from same possibilities as Broadcast TV Turn up the announcer or switch to team radio Allow for precise placement of sound events anywhere in space This enables to guide audiences through 360 environments
MPEG-H Audio System Binaural Rendering for VR? Must support head-tracking Must render audio elements from every direction & distance Must have optimized implementations that match VR platform Should allow for selective audio element tracking Should provide predefined virtual environments
Summary MPEG-H 3D Audio is not just a new audio codec it is a complete new audio system Object based audio introduces new interactivity features which allows for the first time the personalization of audio by the user The MPEG-H 3D Audio feature set offers the new possibilities for creatives and users Application areas any delivery of audio content including VR applications MPEG-H 3D Audio is ready now it is time to make use of it.
Obrigado! alfonso.carrera@iis-extern.fraunhofer.de