Interactive Sound Design

Size: px

Start display at page:

Download "Interactive Sound Design"

Alban Sims
6 years ago
Views:

1 Interactive Sound Design -^-._.-^-._.- Leonard J. Paul Lotus Audio

2 Overview Creative :: How to think about audio for games from an artistic standpoint Process :: How to turn a creative audio idea into an interactive sound design Technical :: How to make the sound design come to life within the limits of the hardware and software info {at} Slide 2 of 26

Creative Top-down or bottom-up approach Top-down means you figure out your big idea and worry later about how to actually do it Bottom-up means you look at what the hardware and software is capable

3 Creative Top-down or bottom-up approach Top-down means you figure out your big idea and worry later about how to actually do it Bottom-up means you look at what the hardware and software is capable of and figure out creative ways to get the most of them Often you'll be going back and forth between the two as technical issues and process limitations will force you to be more creative info {at} Slide 3 of 26

4 Process What do you need to do to get your audio into the game? Over-prepare for the worst What tools and resources are you going to use and what software will support your creative design? Work well with your coder & producer A good process can be found by making all the right mistakes once and taking risks - exciting! Audio is always last, so be prepared for feature drop, no money, no time & no love :) info {at} Slide 4 of 26

Technical Find out the limits of the software and hardware and push them Expand the software capabilities and design better audio tools to suit the game Realize

5 Technical Find out the limits of the software and hardware and push them Expand the software capabilities and design better audio tools to suit the game Realize that often you will be pressed by the team for memory & CPU, so hold your ground Things can work one day and be broken (and useless!) the next info {at} Slide 5 of 26

6 Games Review Some recent games with great audio: info {at} Slide 6 of 26

7 Dragon Age : Forest Ambience info {at} Slide 7 of 26

8 Forest Ambience : Simple Loop Creative challenge is how do we make a realistic sounding ambience that can be heard for a long time? Process might be to use long loops from a sample library or record your own Technical challenges include how to fit it in memory, data compression type, making a seamless loop, mixing it at the right level and making it transition smoothly to the next ambience info {at} Slide 8 of 26

9 Forest Ambience : Simple Loop Looping using Wavelab: Green markers Start/End Zero crossing Live audition Crossfade loop Volume flatten Pitch flatten Spectral edit info {at} Slide 9 of 26

Procedural Sound Design Generate audio in real-time based on rules and behaviours Has been used in games since the beginning Parametric, generative & real-time

10 Procedural Sound Design Generate audio in real-time based on rules and behaviours Has been used in games since the beginning Parametric, generative & real-time (ie. flexible) Can combine samples with synthesis Allow flexibility of layers and real-time effects instead of stereo bounce late binding info {at} Slide 10 of 26

11 Forest Ambience : Procedural Creative challenge is how do we make a realistic sounding ambience that reacts to the player? c Process in Wwise, make a Blend container that has separate streams for insects, birds, frogs, and wind that are mixed together at run-time Technical issues can include: no CPU for synthesis, not enough memory, limited streams & no overlap crossfading between areas info {at} Slide 11 of 26

12 Forest Ambience: Procedural Use blend containers in Wwise and react to RTPC (real-time parameter control) from game: info {at} Slide 12 of 26

13 Dragon Roar Creative challenges include having a hard time finding a real dragon to record! So, how do we construct the sound? Process might include real-time effects processing and techniques such as granulation Technical risks include not having enough DSP to support effects processing and synthesis especially on smaller platforms info {at} Slide 13 of 26

14 Chinatown Wars : Ambience info {at} Slide 14 of 26

15 Chinatown Wars : Ambience Creative goal is to make the city ambience affected by time of day Process is to make separate streams and random one-shots for traffic, pedestrians, cats, and crickets Technical challenges include memory, data compression type, making a seamless loop, mixing it at the right level and making it transition smoothly to the next ambience info {at} Slide 15 of 26

Sound Design Via Tracker Chinatown Wars NDS likely doesn't use synthesis, but it could use noise with envelopes for several things, such as gunshots,

16 Sound Design Via Tracker Chinatown Wars NDS likely doesn't use synthesis, but it could use noise with envelopes for several things, such as gunshots, explosions & footsteps Allows sound artist control over parameters and mix of synthesis and samples but is also tricky to learn info {at} Slide 16 of 26

17 Forza Motorsport 3 : Engines info {at} Slide 17 of 26

Car Engines Creative challenge is to make a reactive and

the car engine elements down into shift sounds, exhaust,

make the content react realistically and keep the memory

18 Car Engines Creative challenge is to make a reactive and realistic-sounding car engine Procedure involves breaking the car engine elements down into shift sounds, exhaust, engine, revs and more Technical difficulty is trying to make the content react realistically and keep the memory size small a car is a complicated instrument info {at} Slide 18 of 26

But... Often there isn't enough time, so people will go with what they know and advanced techniques are overlooked Long time required for development and tuning Leap

19 But... Often there isn't enough time, so people will go with what they know and advanced techniques are overlooked Long time required for development and tuning Leap of knowledge and time to learn for those invested in older techniques Requires more CPU for DSP and synthesis Balance creative & technical? -> info {at} Slide 19 of 26

20 Demos Using Pure Data: Procedural Ambience : Forest Granulation : Dragon Roar Engine : Helicopter Layered Impacts : Crash Granulation : Engine Idle Using Half-Life 2 and OSC: Granulation, layering and more info {at} Slide 20 of 26

21 Game Audio Pipeline Game Code RTPCs OSC Layer OSC Layer Game Audio Code script sound events Game Audio Tool raw audio Commercial Sound Tool xma/ogg/atrac Sound Driver Code info {at} Slide 21 of 26

22 Pure Data (PD) Relatively easy to learn open-source interactive audio implementation environment Can integrate directly into game code (as in EA's Spore), but not optimal as it is interpreted Create compiled code from (simple) patches as shown by pd2j2me info {at} public void met0bang() { double a; a = counter.bang(); a = a * 3; a = a / 4; } Slide 22 of 26

Half-Life 2 Half-life 2 is a PC game/xbox that won over 40 Game of the Year awards in 2004 and allows users to change the maps and other content of the game using

23 Half-Life 2 Half-life 2 is a PC game/xbox that won over 40 Game of the Year awards in 2004 and allows users to change the maps and other content of the game using the Source engine technology in C++ Left 4 Dead and other games based on Source Hammer editor allows for map editing Make your own as well info {at} Slide 23 of 26

24 Half-Life 2 Source Mod A mod is a modification of the game that can be quite simple or be advanced enough to create an entirely new game (ie. total conversion) Change code using Microsoft's free compiler Visual C Express Edition Cost for replicating this system is just the cost of the game - $19.95 info {at} Slide 24 of 26

25 HL2 Demonstration info {at} Slide 25 of 26

26 Questions? Leonard J. Paul info{at} web: info {at} Slide 26 of 26

GAME AUDIO LAB - AN ARCHITECTURAL FRAMEWORK FOR NONLINEAR AUDIO IN GAMES.

GAME AUDIO LAB - AN ARCHITECTURAL FRAMEWORK FOR NONLINEAR AUDIO IN GAMES. SANDER HUIBERTS, RICHARD VAN TOL, KEES WENT Music Design Research Group, Utrecht School of the Arts, Netherlands. adaptms[at]kmt.hku.nl