The Brief History of Virtual Analog Synthesis

Transcription

1 The Brief History of Virtual Analog Synthesis Jussi Pekonen and Vesa Välimäki Department of Signal Processing and Acoustics, Aalto University School of Electrical Engineering, Espoo, Finland. Summary In the 1960s and 1970s, most electrical (or analog) synthesizers were based on a sound synthesis technique called subtractive synthesis, in which a spectrally rich source signal, typically a sawtooth or rectangular pulse wave, is filtered with a time-varying filter. In the 1980s and 1990s this synthesis technique was almost overtaken by other synthesis techniques but in the mid-1990s musicians started again to show interest in the warm sound of subtractive synthesis. To meet this interest, a Swedish company Clavia introduced in 1995 the NordLead synthesizer that used the subtractive sound synthesis but in digital domain. Furthermore, they introduced the term virtual analog to represent the digital simulation of the analog audio devices. Since then research on digital signal processing methods for subtractive sound synthesis has increased and more and more interest has been shown to the topic in the past few years. In fact, the last year (2010) was again record-breaking when the number of publication on this topic is considered. In this paper, the history of the publications of virtual analog synthesis research is reviewed. In addition to the publication count over the years, the viewpoints of this paper include the distribution of the methodologies presented in the publications. PACS no Ww, Zz 1. Introduction The most of the early music synthesizers in the 1960s and 1970s that were based on analog electronics used subtractive synthesis as their sound production principle. Subtractive sound synthesis can be thought to be the opposite of additive synthesis where the (complex) target tone is generated as a sum of primitive waveforms (sine waves). Subtractive synthesis starts by generating a spectrally rich source signal that is then filtered with a time-varying filter. As the source signal one of the classical geometric waveforms, the sawtooth, the rectangular pulse, or the triangle waveform, is typically used. Sometimes a mix of the abovementioned waveforms or a noise signal is used. The filtering is typically performed with a lowpass filter that has a controllable resonance close to its cutoff frequency. In the 1980s and 1990s, subtractive synthesis was overtaken as the most popular sound generation principle mainly by frequency modulation synthesis and sampling synthesis. At the same time, digital signal processing was rapidly increasing its popularity also in music synthesizers. It appeared that subtractive synthesis used by the analog synthesizers would eventually become a rare sound generation principle. How- (c) European Acoustics Association ever, in the mid-1990s musician started to show interest in the warm sound of subtractive synthesis. To meet the musicians increasing interest in subtractive synthesis, a Swedish company called Clavia introduced in 1995 the NordLead synthesizer that generated the sound with subtractive synthesis but using digital signal processing tools. The NordLead synthesizer was, in fact, the very first digital synthesizer that emulated the sound generation principle of analog synthesizers [1 3], though some of the features of analog synthesizers were used earlier in the Roland JD-990 synthesizer. With the help of digital signal processing the downsides of analog electronics could be avoided. Whereas the analog synthesizers were quite bulky and they suffered from temperaturedependent tuning problems, digital synthesizers will always be in tune and they can be constructed to a smaller chassis. Furthermore, together with the introduction of the NordLead Clavia introduced a term virtual analog to represent the digital simulation of the analog audio devices [1 3]. Since then research on digital signal processing methods for subtractive sound synthesis has increased both in academia and in music technology companies such as Yamaha, Korg, Roland, Native Instruments, Access, and Arturia, and more and more interest has been shown to the topic in the past few years. In this paper, the history of the publications of virtual analog synthesis research is reviewed.

2 The remainder of this paper is organized as follows. Section 2 discusses about the historical evolution of virtual analog synthesis research by presenting the publication counts over the years. In addition, the types of publications dealing with virtual analog synthesis are discussed. In Section 3, the methodologies utilized in the virtual analog synthesis publications are presented. The current trends and the future of virtual analog synthesis research are discussed in Section 4, and the paper is concluded in Section Publications dealing with virtual analog synthesis Research on virtual analog synthesis has increased greatly in the past few years. This is illustrated in Figure 1 where the publication counts in different years are plotted. The publications focusing only on the source signal generation are plotted in black whereas publications that discuss only the filter modeling are plotted in white. The publications plotted in gray discuss about both the source signal generation and the filter modeling. The dashed line in Figure 1 indicates the publication year of the NordLead synthesizer. As can be seen in Figure 1, the research on this topic has indeed increased dramatically in the past few years. Last year, 2010, a record-breaking 10 publications dealing with virtual analog synthesis were published, which is a fifth of the total publication count. To analyze the publication shares in different years more closely, they are listed in Table I together with the reversed cumulative shares. The reversed cumulative shares are shown to indicate how the research has mainly done in the past few years, as about half of all publications have been published in the last three years, approximately two thirds after 2005, and about 80% in the past decade. In Figure 1 one can also see that there are six publications that precede the NordLead synthesizer (1995) and that are related to virtual analog synthesis. These publications are in fact some of the pioneering works on digital subtractive synthesis, and they present experimental approaches. In addition, Figure 1 shows that there are more publications that focus on the source signal generation and not on the filter modeling. This is justified by the modular structure of subtractive synthesis and the fact that the traditionally used waveforms suffer from harsh aliasing caused by the discontinuities in the waveform or in the waveform derivative when they are trivially generated. By 2010, there are 34 publications that have focused only on the aliasing problem of the source signals and nine publications that have focused only on the filter modeling. Seven publications have dealt with both source signal generation and the filter modeling. This means that 82% of the virtual analog synthesis pub- Table I. Shares of the different publication year counts. The reversive cumulative share is shown to indicate how the research has focused to past few years. Year Count Share (%) Rev. cumulative (%) Total lications discuss the digital generation of the classical geometric waveforms. The number of different types of the publications dealing with virtual analog synthesis are shown in Figure 2. Four of the publications are published as a book chapter, 12 as a journal article, and 23 as a conference article. In addition, there are five patents, two master s theses, and one doctoral dissertation where this topic is addressed. Moreover, there are three unpublished online publications. 3. Methodologies As mentioned above, the research on the source signal generation and the filter modeling has been split into separate topics. Moreover, due to the dissimilar research objectives, the approaches used in the filter modeling have not been applied in the source signal generation. This can be noted in Sections 3.1 and 3.2 where the methodologies used in the source signal generation and the filter modeling are discussed, respectively Source signal generation The methodologies used in the publications that deal with the source signal generation can be categorized roughly to 1. ideally bandlimited algorithms, 2. quasi-bandlimited algorithms, 3. alias-suppressing algorithms, and 4. ad-hoc approaches.

3 Count Only source signal generation Both source signal generation and filter modeling Only filter modeling Publication of the NordLead synthesizer Figure 1. Count of virtual analog synthesis publications in different years. The publications that discuss only about the source signal generation are plotted in black and the publications focusing only on the filter modeling are plotted in white. The publications that deal with both source signal generation and filter modeling are plotted in gray. The dashed line indicates the publication year of the NordLead synthesizer. Year Count B J C P M D W Type Figure 2. Types of the publications dealing with virtual analog synthesis: B denotes book chapters, J journal articles, C conference articles, P patents, M master s theses, D doctoral dissertations, and W (unpublished) online publications. Next, brief descriptions of these categories are given. The ideally bandlimited algorithms synthesize only a fixed number of harmonic components of the source signal in the usable baseband of the digital signal system. These algorithms utilize additive synthesis [4], discrete summation formulae [5 7], inverse (fast) Fourier transform [8], and wavetable synthesis [9, 10] as the generation technique. The algorithms of the quasi-bandlimited category allow some aliasing that is mainly at high frequencies where the human perception is less sensitive than at low and middle frequencies. These algorithms either synthesize a bandlimited impulse train (BLIT) [11,12] or a sequence of bandlimited step functions (BLEP) [13,14], both of which can be used to generate classical waveforms. Modifications to the original table-based algorithms have been suggested, and these modifications include the use of fractional delay filters [15 20], feedback frequency modulation (FM) [21], and modified FM pulses [22] as the bandlimited basis function generators. In addition, the original table-based approaches have been improved by optimizing the lookup table entries [23]. The alias-suppressing algorithms can be considered to sample a signal that has the same spectral content as the target waveform but with a steeper spectral tilt than the target. Now, the sampled signal contains aliasing in the whole baseband but suppressed. After the sampling the spectral tilt of the desired harmonics is retained by applying a digital post-processing highpass filter to the sampled signal. Algorithms that implement this approach include oversampling, i.e. sampling at higher sample rate [9, 24] (does not need the highpass filtering), filtering of full-wave rectified sine wave [25], and differentiated piece-wise polynomial waveform [26 30]. The fourth category contains miscellaneous algorithms that generate signals that resemble the classical waveforms but that are not necessarily bandlimited. In fact, in many of these algorithms the objective is to generate the waveform with using the readily available simple signal processing tools. These algorithms include post-filtering of an alias-corrupted signal [31] (post-reduction of aliasing), waveshaping of a sinusoid (amplitude distortion) [17,32,33], phaseshaping of a sinusoid (phase distortion) [32 38], feedback amplitude modulation [39], bit-wise logical modulation [40], and nonlinear modification of the trivially sampled sawtooth waveform [41, 42]. In addition to the actual waveform synthesis techniques listed above, there is one publication that explains how the other waveforms can be obtained from a sawtooth signal with a digital comb filter [43]. Also, one publication discusses digital modeling of an astable analog multivibrator circuit that produces the continuous-time classical waveforms [44] Filter modeling Whereas there are several categories for the source signal generation algorithms, the filter modeling publications are based on either a black-box or a whitebox modeling approach. In the signal-based black-box

4 modeling the relationship of the filter input and output signals is investigated. In the circuit-based whitebox modeling the signal flow inside the filter circuit is analyzed. However, the main difference between the publications dealing with filter modeling is usually the filter that is under investigation. However, it should be noted that in addition to the models of real analog synthesizer filters, specially designed filter models for the virtual analog synthesis have also been suggested [9, 44, 45]. Both linear [46,47] and nonlinear [28,48,49] circuitbased models for the popular Moog transistor-based voltage-controlled ladder filter have been suggested. In addition to the circuit-based models of the Moog filter, signal models based on a Volterra series representation [50, 51] have been suggested. Nonlinear circuit-based models have also been proposed for the diode-based EMS VCS3 filter [52, 53] and for the second-order resonant lowpass filter of the Korg MS- 20 analog synthesizer [30]. 4. Future of virtual analog synthesis research At the moment the future of virtual analog synthesis research looks bright. To the knowledge of authors, there is one journal article [54] (in addition to this paper) that has been published in 2011 and that discusses source signal generation. In addition, there are two publications [55, 56] that have been accepted for publication and that are in press at the time of writing of this paper. In spite of the several publications dealing with the source signal generation, the signal generation task is not a completely solved problem. None of the existing sources signal algorithms is an optimal algorithm having the following properties: 1. the produced waveforms are perceptually aliasingfree in the range of musical fundamental frequencies (approximately from 20 Hz to 8 khz), 2. the algorithm is computationally efficient and has low memory consumption, and 3. the algorithm does not require a division by a timevarying model parameter, e.g. the fundamental frequency. With some of the existing source signal generation algorithms the first two properties can be achieved, but the third requirement is not fulfilled (see e.g. [19, 29,56]). On the other hand, there are algorithms that have the third property, but they do not fulfill either the first or the second requirement (see e.g. [33, 40, 54]). Therefore, the search for the optimal algorithm will still remain an active research focus in virtual analog synthesis research. In addition to the search for the optimal source signal algorithm, the source signal research has also started to investigate the detailed modeling of the output waveforms of analog synthesizers [55]. The waveforms of the analog synthesizers differ from the textbook waveforms that have been the target of the source signal algorithms so far, and for realistic virtual analog synthesis modeling these differences should be included in the model. It seems that this research direction will gain interest in the near future as different analog synthesizers will be analyzed. Due to the similarity to the filter modeling, the models of the analog synthesizer source signals will eventually use the same methodologies used in the filter modeling publications, i.e. signal-based and circuit-based techniques. In the filter modeling, the existing filter models will definitely be refined and improved. One possible approach in the refinement could be the optimization of the computational load required by the models by taking into account only the perceptually significant properties of the target filter. By doing so the computation will be focused on the properties that are really relevant. On the other hand, the selection of filters that are modeled will most probably increase, and different modeling approaches will be applied to them. 5. Conclusion In this paper, the historical evolution of the virtual analog synthesis research was reviewed. It was shown that the research on virtual analog synthesis has increased in the past few years quite much when the number of publications is considered. About the half, two thirds, and four fifths of the publications have been published in the past three, five, and ten years, respectively, and last year, 2010, a record-breaking ten publications dealing with the topic were published. It was also shown that over 80% of the virtual analog synthesis publications have dealt with the source signal generation, a subtopic that has justly been investigated due to the inherent aliasing problem in the generation of the traditionally used source signals. In addition to the publication count over the years, the methodologies presented in the publications were briefly reviewed. Whereas there are only two main approaches used in the filter modeling, the algorithms for the source signal generation can be categorized into four major groups. Each of these categories approach the aliasing problem of the source signals from a different perspective. Furthermore, the current trends and the future of virtual analog synthesis research were discussed. It was pointed out that the source signal generation task has not been completely solved yet. Of the three desired properties of the optimal source signal algorithm two have been achieved by the existing approaches, meaning that the search for the optimal algorithm will remain an active objective also in the future. In addition, it was noted that the modeling of the actual output waveforms of the analog synthesizers will be another focus in the source signal research.

5 In the filter modeling, on the other hand, work on the optimizing the computational load of the algorithms to the perceptually relevant properties will be one of the research focuses. Moreover, more synthesizer filters will definitely be modeled in the future. Acknowledgement This work has been partly funded by the Academy of Finland (project no ). The authors are grateful to Julian Parker for valuable comments. References [1] J. O. Smith: Physical modeling synthesis update. Computer Music J. 20 (1996), [2] C. Erkut, V. Välimäki, M. Karjalainen, H. Penttinen: Physics-based sound synthesis. - In: Sound to Sense, Sense to Sound: A State of the Art in Sound and Music Computing, chap. 8. P. Polotti, D. Rocchesso (eds.). Sound and Music Computing Network, Available online org/files/s2s2book1.pdf. [3] Synthesizer history timeline wiki page. Synthesizer_History_Timeline (checked March 31, 2011), [4] A. Chaudhary: Bandlimited simulation of analog synthesizer modules by additive synthesis. Proc. 105th Audio Eng. Soc. Conv., San Francisco, CA, Preprint number [5] G. Winham, K. Steiglitz: Input generators for digital sound synthesis. J. Acoust. Soc. Am. 47 (1970), [6] J. A. Moorer: The synthesis of complex audio spectra by means of discrete summation formulas. J. Audio Eng. Soc. 24 (1976), [7] J. Lazzaro, J. Wawrzynek: Subtractive synthesis without filters. - In: Audio Anecdotes II Tools, Tips, and Techniques for Digital Audio, K. Greenebaum, R. Barzel (eds.). A. K. Peters, Ltd, Wellesley, MA, [8] G. Deslauriers, C. Leider: A bandlimited oscillator by frequency-domain synthesis for virtual analog applications. Proc. 127th Audio Eng. Soc. Conv., New York, Preprint number [9] H. Chamberlin: Musical Applications of Microprocessors, chap. 13, Hayden Book Company, Hasbrouck Heights, NJ, 2nd edn., [10] P. Burk: Band limited oscillators using wave table synthesis. - In: Audio Anecdotes II Tools, Tips, and Techniques for Digital Audio, K. Greenebaum, R. Barzel (eds.). A. K. Peters, Ltd, Wellesley, MA, [11] T. Stilson, J. O. Smith: Alias-free digital synthesis of classic analog waveforms. Proc. Intl. Computer Music Conf., Hong Kong, China, [12] T. Stilson: Efficiently-variable non-oversampling algorithms in virtual-analog music synthesis a rootlocus perspective. Ph.D. thesis, Stanford University, Stanford, CA, Available online stanford.edu/~stilti/papers/. [13] E. Brandt: Hard sync without aliasing. Proc. Intl. Computer Music Conf., Havana, Cuba, 2001, [14] A. B. Leary, C. T. Bright: Bandlimited digital synthesis of analog waveforms. U.S. Patent 7,589,272, [15] B. Frei: Digital sound generation. Available online digital-sound-generation/ (checked March 31, 2011), [16] V. Välimäki, A. Huovilainen: Antialiasing oscillators in subtractive synthesis. IEEE Signal Process. Mag. 24 (2007), [17] J. Pekonen: Computationally efficient music synthesis methods and sound design. Master of Science (Technology) thesis, TKK Helsinki University of Technology, Espoo, Finland, Available online files/theses/pekonen_mst/. [18] J. Nam, V. Välimäki, J. S. Abel, J. O. Smith: Aliasfree oscillators using feedback delay loops. Proc. 12th Intl. Conf. Digital Audio Effects (DAFx-09), Como, Italy, 2009, [19] J. Nam, V. Välimäki, J. S. Abel, J. O. Smith: Efficient antialiasing oscillator algorithms using loworder fractional delay filters. IEEE Trans. Audio, Speech, and Lang. Process. 18 (2010), [20] J. Pekonen, V. Välimäki, J. Nam, J. S. Abel, J. O. Smith: Variable fractional delay filters in bandlimited oscillator algorithms for music synthesis. Proc Intl. Conf. Green Circ. Syst. (ICGCS2010), Shanghai, China, [21] P. Schoffhauzer: Synthesis of quasi-bandlimited analog waveforms using frequency modulation. Available online http//scp.web.elte.hu/papers/ synthesis1.pdf (checked March 31, 2011), [22] J. Timoney, V. Lazzarini, T. Lysaght: A modified FM synthesis approach to bandlimited signal generation. Proc. 11th Intl. Conf. Digital Audio Effects (DAFx- 08), Espoo, Finland, 2008, [23] J. Pekonen, J. Nam, J. O. Smith, J. S. Abel, V. Välimäki: On minimizing the look-up table size in quasi bandlimited classical waveform synthesis. Proc. 13th Intl. Conf. Digital Audio Effects (DAFx- 10), Graz, Austria, 2010, [24] M. Puckette: The Theory and Technique of Electronic Music, World Scientific Publishing Co., Hackensack, NJ, [25] J. Lane, D. Hoory, E. Martinez, P. Wang: Modeling analog synthesis with DSPs. Computer Music J. 21 (1997), [26] V. Välimäki: Discrete-time synthesis of the sawtooth waveform with reduced aliasing. IEEE Signal Process. Lett. 12 (2005), [27] A. Huovilainen, V. Välimäki: New approaches to digital subtractive synthesis. Proc. Intl. Computer Music Conf., Barcelona, Spain, 2005, [28] V. Välimäki, A. Huovilainen: Oscillator and filter algorithms for virtual analog synthesis. Computer Music J. 30 (2006), [29] V. Välimäki, J. Nam, J. O. Smith, J. S. Abel: Aliassuppressed oscillators based on differentiated polynomial waveforms. IEEE Trans. Audio, Speech, and Lang. Process. 18 (2010),

6 [30] A. Huovilainen: Design of a scalable polyphony- MIDI synthesizer for a low cost DSP. Master of Science (Technology) thesis, Aalto University School of Science and Technology, Espoo, Finland, Available online Dipl/2010/urn pdf. [31] J. Pekonen, V. Välimäki: Filter-based alias reduction in classical waveform synthesis. Proc IEEE Intl. Conf. Acoust., Speech, and Signal Process. (ICASSP 08), Las Vegas, NV, 2008, [32] J. Timoney, V. Lazzarini, B. Carty, J. Pekonen: Phase and amplitude distortion methods for digital synthesis of classic analogue waveforms. Proc. 126th Audio Eng. Soc. Conv., Munich, Germany, Preprint number [33] V. Lazzarini, J. Timoney: New perspectives on distortion synthesis for virtual analog oscillators. Computer Music J. 34 (2010), [34] M. Ishibashi: Electronic musical instrument. U.S. Patent 4,658,691, [35] J. Timoney, V. Lazzarini, J. Pekonen, V. Välimäki: Spectrally rich phase distortion sound synthesis using an allpass filter. Proc IEEE Intl. Conf. Acoust., Speech, and Signal Process. (ICASSP 09), Taipei, Taiwan, [36] J. Kleimola, J. Pekonen, H. Penttinen, V. Välimäki, J. S. Abel: Sound synthesis using an allpass filter chain with audio-rate coefficient modulation. Proc. 12th Intl. Conf. Digital Audio Effects (DAFx- 09), Como, Italy, 2009, [37] V. Lazzarini, J. Timoney, J. Pekonen, V. Välimäki: Adaptive phase distortion synthesis. Proc. 12th Intl. Conf. Digital Audio Effects (DAFx-09), Como, Italy, 2009, [38] J. Kleimola, V. Lazzarini, J. Timoney, V. Välimäki: Phaseshaping oscillator algorithms for musical sound synthesis. Proc. 7th Sound and Music Computing Conf., Barcelona, Spain, 2010, [39] V. Lazzarini, J. Timoney, J. Kleimola, V. Välimäki: Five variations on a feedback theme. Proc. 12th Intl. Conf. Digital Audio Effects (DAFx-09), Como, Italy, 2009, [40] J. Kleimola: Audio synthesis by bitwise logical modulation. Proc. 11th Intl. Conf. Digital Audio Effects (DAFx-08), Espoo, Finland, 2008, [41] R. J. Lisle, B. S. McDonald: Method and apparatus for selective reduction of upper harmonic content in digital synthesizer excitation signals. U.S. Patent 5,194,684, [42] R. Chidlaw, R. Muha: Oscillator providing waveform having dynamically continuously variable waveshape. U.S. Patent 6,806,413, [43] D. Lowenfels: Virtual analog synthesis with a time-varying comb filter. Proc. 115th Audio Eng. Soc. Conv., New York, Preprint number [44] G. De Sanctis, A. Sarti: Virtual analog modeling in the wave-digital domain. IEEE Trans. Audio, Speech, and Lang. Process. 18 (2010), [45] R. J. Lisle, B. S. McDonald: Digital filter for a music synthesizer. European Patent EP (A2), [46] T. Stilson, J. O. Smith: Analyzing the Moog VCF with considerations for digital implementation. Proc. Intl. Computer Music Conf., Hong Kong, China, 1996, [47] F. Fontana: Preserving the structure of the Moog VCF in the digital domain. Proc. Intl. Computer Music Conf., Thessaloniki, Greece, 1997, [48] A. Huovilainen: Non-linear digital implementation of the Moog ladder filter. Proc. 7th Intl. Conf. Digital Audio Effects (DAFx-04), Naples, Italy, 2004, [49] T. E. Stinchcombe: Analysis of the Moog transistor ladder and derivative filters. Available online ladder_tf.pdf (checked March 31, 2011), [50] T. Hélie: On the use of Volterra series for realtime simulations of weakly nonlinear analog audio devices: Application to the Moog ladder filter. Proc. 9th Intl. Conf. Digital Audio Effects (DAFx-06), Montréal, QC, Canada, 2006, [51] T. Hélie: Volterra series and state transformation for real-time simulations of audio circuits including saturations: Application to the Moog ladder filter. IEEE Trans. Audio, Speech, and Lang. Process. 18 (2010), [52] M. Civolani, F. Fontana: A nonlinear digital model of the EMS VCS3 voltage-controlled filter. Proc. 11th Intl. Conf. Digital Audio Effects (DAFx-08), Espoo, Finland, 2008, [53] F. Fontana, M. Civolani: Modeling of the EMS VCS3 voltage-controlled filter as a nonlinear filter network. IEEE Trans. Audio, Speech, and Lang. Process. 18 (2010), [54] J. Kleimola, V. Lazzarini, V. Välimäki, J. Timoney: Feedback amplitude modulation synthesis. EURASIP J. Advances in Signal Process (2001). [55] J. Pekonen, V. Lazzarini, J. Timoney, J. Kleimola, V. Välimäki: Discrete-time modelling of the Moog sawtooth oscillator waveform. EURASIP J. Advances in Signal Process (2011). [56] V. Välimäki, J. Pekonen, J. Nam: Perceptually informed synthesis of bandlimited classical waveforms using integrated polynomial interpolation. J. Acoust. Soc. Am. (2011). In press.