Ambisonics plug-in suite for production and performance usage

Transcription

1 Ambisonics plug-in suite for production and performance usage Matthias Kronlachner Linux Audio Conference 013 May 9th - 1th, 013 Graz, Austria

2 What? used JUCE framework to create cross-platform audio plug-ins (LV, VST, AU) and Jack standalone apps above 3rd order Ambisonics, ambix convention advanced (remote) control features for the encoder binaural decoder with customizable presets

3 Osaka, Pischelsdorf (AT), 011-??

4 What is Ambisonic(s)? surround recording and playback technique, Michael Gerzon 1970s little commercial success so far, patents expired independent of playback loudspeaker configuration, scaleable D and 3D sound-fields can be synthesized 4

5 What is Ambisonic(s)? not using phantom sources (VBAP, DBAP) trying to recreate original sound-field in sweet spot y [m] 0 0 y [m] x [m] 1 (g) HOA Hz (h) HOA 8 000Hz (i) plane wave from 0 (M. Frank) Abbildung 6: HOA: Schalldruck des reproduziertes Schallfeldes für verschiedene Vector Based Amplitude Panning (V.Pulkki) und Frequenzen (für den Fall einer virtuellen ebenen Welle aus 0 ± Richtung) 5

6 What is Ambisonic(s)? loudspeaker feeds are a linear combination of the B- format signals (decoder matrix) e.g.: p j = 1 L apple W ( 1 p )+X (cos j cos j )+Y (sin j cos j )+Z (sin j ) (3.1 ALL speakers work together to synthesize the sound-field LS8 x LS1 LS LS7 φ LS3 y LS6 LS4 LS5 B-Format (1st order Ambisonics) 6 Regular D loudspeaker placement

7 What is Ambisonic(s)? sound field is sampled using spherical harmonics: D: N+1 or 3D: (N+1)^ 0th order (W) 1st order (Y, Z, X) nd order 3rd order 4th order 5th order Spherical harmonics up to 5th order 3D 7

8 Ambisonics production chain mic. calibration data Ambisonics Microphone MicEnc rotation, zoom Loudspeaker configuration Mono Sound Source Mono Sound Source azimuth, elevation Encoder Encoder L Manipulation L Ambisonics N (optional) Decoder Independent from Loudspeaker configuration L 3D =(M + 1) L D =M +1 M Ambisonics Order L Number of Ambisonics Channels N Number of Loudspeakers N L Loudspeaker signals overview of an Ambisonics production/playback system 8

9 Problems with existing solutions DAWs handling high channel count - Ardour and Reaper operating system compatibility (Win, Mac OS, Linux) and plug-in standards (VST, AU, AAX, RTAS, LV, LADSPA) listening/producing at home? jumps in angular representation between -180 and 180 keeping track of the sound source positions different standards about channel sequence and normalization, restricted orders 9

10 Channel sequence Y m n (', #) n + n + m FuMa n m ACN SID* 0th W Ist X Y 1 0 Z nd R S T U V 8 8 3rd * Single Index Designation

11 Channel sequence Y m n (', #) n + n + m FuMa n m ACN SID 0th W Ist Y Z X nd V T R S U 8 4 3rd... 11

12 0th Ist nd Channel normalization Y m n (', #) N3D 0dB 4.77dB 4.77dB 4.77dB 6.99dB 6.99dB 6.99dB 6.99dB 6.99dB n m SN3D 0dB 0dB 0dB 0dB 0dB 0dB 0dB 0dB 0dB FuMa -3dB 0dB 0dB 0dB 1.5dB 1.5dB 0dB 1.5dB 1.5dB 3rd... 1 converter example...!

13 Channel normalization N3D: clipping of higher order components SN3D: no channel exceeds 0th order (W)

14 Ambisonics standards sequence normalization ambix (I) ACN SN3D Universal Ambisonics -.ua SID N3D.amb (I1) FuMa FuMa iem_ambi Pd externals SID SN3D... (I) Nachbar, Zotter, Deleflie, Sontacchi - AMBIX - A SUGGESTED AMBISONICS FORMAT AMBISONICS SYMPOSIUM 011 (II) AMB plug-ins, Wigware, Ambisonics Studio plug-ins (Daniel Courville) 14

15 First approach... adjust scaling and channel order in every stage mic. calibration data Ambisonics Microphone MicEnc rotation, zoom Loudspeaker configuration never released... Mono Sound Source Mono Sound Source azimuth, elevation Encoder Encoder L Manipulation L Ambisonics N (optional) Decoder Independent from Loudspeaker configuration 15 N L Loudspeaker signals

16 Ambisonics with ambix mic calibration Ambisonics microphone MicProc/ Converter other Ambisonics formats channel order/scaling Mono sound source Converter Encoder L Meter HOAFilter Head Tracking Rotation L Converter (opt.) L' Loudspeaker configuration Decoder N Loudspeaker signals Mono sound source azimuth, elevation, size Encoder ambix standard M Ambisonics Order L Number of Ambisonics Channels N Number of Loudspeakers Binaural Decoder left right Headphone signals 16

17 ambix-converter 17 presets for ambix-converter

18 Encoders (panner) LADSPA AMB Plugins by Fons Adriaensen and Jörn Nettingsmeier (Linux and MacOS - Ardour) Daniel Courville s Ambisonics Suite (Mac OS) nd order 3D, 5th order D 18 mic. calibration data Ambisonics Microphone Mono Sound Source Mono Sound Source azimuth, elevation MicEnc Encoder Encoder rotation, zoom Loudspeaker configuration L Manipulation L Ambisonics N (optional) Decoder Independent from Loudspeaker configuration N L Loudspeaker

19 Remote control and visualization individual OSC/UDP connection from visualization to encoders azimuth, elevation, size Encoder Encoder... Encoder OSC/UDP azimuth, elevation, size, audio level central Visualization and Remote Control digital audio workstation 19...example!

20 Jumping angular representation (visual) jump between -180 and 180 0

21 advanced control parameters 1...example!

22 Listening at home head tracking (Open Sound Control) virtual loudspeaker configuration individual or general HRTFs Ambisonics B-Format L Soundfield L Ambisonics N Rotation Decoder L Number of Ambisonics Channels N Number of Virtual Loudspeakers virtual loudspeaker signals HRTF Convolution Filter left right headphone signals HRTF.. Head Related Transfer Function spk1 spk8 Ambisonics Binaural Decoder F1 F8 spk F F7 spk7 F3 F6 spk3 spk6 F4 F5 spk4 spk5 loudspeakers - left ear transfer path mic. calibration data Ambisonics Microphone Mono Sound Source Mono Sound Source azimuth, elevation MicEnc Encoder Encoder rotation, zoom Loudspeaker configuration L Manipulation L Ambisonics N (optional) Decoder Independent from Loudspeaker configuration N L Loudspeaker signals

23 Binaural decoder L L Stereo using zita-convoler and zita-resampler config files for impulse responses and decoder matrices using identity matrix for directly driving the speakers Headtracking: G. Fanelli - Real Time Head Pose Estimation from Consumer Depth Cameras 3

24 Binaural decoder Mumuth Graz, 9 Speakers in the hemisphere Florian HOLLERWEGER and Martin RUMORI Production and Application of Room Impulse Responses for Multichannel Setups using FLOSS Tools LAC013

25 Binaural decoder IEM Cube Graz, 4 Speakers in the hemisphere Florian HOLLERWEGER and Martin RUMORI Production and Application of Room Impulse Responses for Multichannel Setups using FLOSS Tools LAC013

26 Things to do for me (and maybe helpers?) fix bugs, especially with GUI easy (cross-platform) build system (DISTRHO?) ambisonics rotators (pitch, yaw, roll) above 3rd order... for the community accept a standard concerning channel sequence and normalization above 3rd order (ambix?) DIY Higher Order Microphones 6

27 Questions?

28 Thank you!