CSC475 Music Information Retrieval

Size: px

Start display at page:

Download "CSC475 Music Information Retrieval"

Basil Welch
5 years ago
Views:

1 CSC475 Music Information Retrieval Sinusoids and DSP notation George Tzanetakis University of Victoria 2014 G. Tzanetakis 1 / 38

2 Table of Contents I 1 Time and Frequency 2 Sinusoids and Phasors G. Tzanetakis 2 / 38

3 Motivation Frequently the computer science students who take this course have no background in Digital Signal Processing (DSP) so I always try to do a few lectures introducing some DSP fundamentals. An introduction to DSP typically requires an entire course and learning DSP is a life long pursuit so what one can do in a few lectures is rather limited. My goal is to stress intuition and attempt to demystify the basics of the mathematical notation used. In addition, DSP contains some beautiful mathematical ideas that connect the continuous mathematics of the physical world with the discrete mathematics needed by computers. I hope this material will motivate you to learn more about DSP. G. Tzanetakis 3 / 38

4 Digital Audio Recordings Recordings in analog media (like vinyl or magnetic tape) degrade over time Digital audio representations theoretically can remain accurate without any loss of information through copying of patterns of bits. MIR requires a distilling information from an extremely large amount of data Digitally storing 3 minutes of audio requires approximately 16 million numbers. A tempo extraction program must somehow convert these to a single numerical estimate of the tempo. G. Tzanetakis 4 / 38

5 Production and Perception of Periodic Sounds Animal sound generation and perception The sound generation and perception systems of animals have evolved to help them survive in their environment. From an evolutionary perspective the intentional sounds generated by animals should be distinct from the random sounds of the environment. Repetition Repetition is a key property of sounds that can make them more identifiable as coming from other animals (predators, prey, potential mates) and therefore animal hearing systems have evolved to be good at detecting periodic sounds. G. Tzanetakis 5 / 38

Pitch Perception Pitch When the same sound is repeated more than 10-20 times per second instead of it being

pitch that is related to the underlying period of repetition.

6 Pitch Perception Pitch When the same sound is repeated more than times per second instead of it being perceived as a sequence of individual sound events it is fused into a single sonic event with a property we call pitch that is related to the underlying period of repetition. Note that this fusion is something that our perception does rather than reflect some underlying singal change other than the decrease of the repetition period. G. Tzanetakis 6 / 38

Time-Frequency Representations Music Notation When listening to mixtures of sounds (including music) we are interested in when specific sounds take place (time) and what is their

7 Time-Frequency Representations Music Notation When listening to mixtures of sounds (including music) we are interested in when specific sounds take place (time) and what is their source of origin (pitch, timbre). This is also reflected in music notation which fundamentally represents time from left to right and pitch from bottom to top. G. Tzanetakis 7 / 38

8 Spectrum Informal definition of Spectrum A fundamental concept in DSP is the notion of a spectrum. Informally complex sounds such as the ones produced by musical instruments and their combinations can be modeled as linear combinations of simple elementary sinusoidal signals with different frequencies. A spectrum shows how much each such basis sinusoidal component contributes to the overall mixture. It can be used to extract information about the sound such as its perceived pitch or what instrument(s) are playing. A spectrum corresponds to a short snapshot of the sound in time. G. Tzanetakis 8 / 38

9 Spectrum example Spectrum of a tenor saxophone note G. Tzanetakis 9 / 38

10 Spectrograms Spectrograms Music and sound change over time. A spectrum does not provide any information about the time evolution of different frequencies. It just shows the relative contribution of each frequency to the mixture signal over the duration analyzed. In order to capture the time evolution of sound and music the standard approach is to segment the audio signal into small chunks (called windows or frames) and calculate the spectrum for each of these windows. The assumption is that during the relatively short period of analysis (typically less than a second) there is not much change and therefore the calculated short-time spectrum is an accurate representation of the underlying signal. The resulting sequence of spectra over time is called a spectrogram. G. Tzanetakis 10 / 38

11 Examples of spectrograms Spectrogram of a few tenor saxophone notes G. Tzanetakis 11 / 38

12 Waterfall spectrogram view Waterfall display using sndpeek G. Tzanetakis 12 / 38

13 Table of Contents I 1 Time and Frequency 2 Sinusoids and Phasors G. Tzanetakis 13 / 38

14 Why is DSP important for MIR? A large amount of MIR research deals with audio signals. Audio signals are represented digitally as very long sequences of numbers. Digital Signal Processing techniques are essential in extracting information from audio signals. The mathematical ideas behind DSP are amazing. For example it is through DSP that you can understand how any sound that you can hear can be expressed as a sum of sine waves or represented as a long sequence of 1 s and 0 s. G. Tzanetakis 14 / 38

15 DSP for MIR Digital Signal Processing is a large field and therefore impossible to cover adequately in this course. The main goal of the lectures focusing on DSP will be to provide you with some intuition behind the main concepts and techniques that form the foundation of many MIR algorithms. I hope that they serve as a seed for growing a long term passion and interest for DSP and the textbook provides some pointers for further reading. G. Tzanetakis 15 / 38

16 Sinusoids We start our exposition with discussing sinusoids which are elementary signals that are crucial in understading both DSP concepts and the mathematical notation used to understand them. Our ultimate goal of the DSP lectures is to make equations such as less intimidating and more meaningfull: X (f ) = x(t)e j2πft dt (1) G. Tzanetakis 16 / 38

17 What is a sinusoid? Family of elementary signals that have a particular shape/pattern of repetition. sin(ωt) and cosin(ωt) are particular examples of sinusoids that can be described by the more general equation: x(t) = sin(ωt + φ) (2) where ω is the frequency and φ is the phase. There is an infinite number of continuous periodic signals that belong to the sinusoid family. Each is characterized by three numbers: the amplitude the frequency and the phase. G. Tzanetakis 17 / 38

18 Figure : Simple sinusoids G. Tzanetakis 18 / 38

19 4 motivating viewpoints for sinusoids Solutions to the differential equations that describe simple systems of vibration Family of signals that pass unchanged through LTI systems Phasors (rotating vectors) providing geometric intution about DSP concepts and notation Basis functions of the Fourier Transform G. Tzanetakis 19 / 38

20 Simple vibration I Consider striking the tine of a tuning fork. The tine will deform, the be restored to the original position, the inertia will make it overshoot and deform in the other direction and the pattern will repeat. At any particular displacement x Newton s second law applies: F = ma = kx (3) The accelaration is the second derivative of the displacement x with respect to t so the equation can be rewritten: d 2 x = (k/m)x (4) dt2 G. Tzanetakis 20 / 38

21 Sinusoids satisfy the equation We are looking for a signal x(t) that satisfies the equation describing simple vibrations i.e we are looking for a signal that is proportional to its second derivative. d dt sin(ωt) = ω cos(ωt) d 2 dt 2 sin(ωt) = ω2 sin(ωt) (5) So it turns out that sinusoidal signals arise as the solutions to the physics equations that describe simple systems of vibration that can potentially generate sound. G. Tzanetakis 21 / 38

22 Linear Time Invariant Systems Definition Systems are transformations of signals. They take a input a signal x(t) and produce a corresponding output signal y(t). Example: y(t) = [x(t)] LTI Systems Linearity means that one can calculate the output of the system to the sum of two input signals by summing the system outputs for each input signal individually. Formally if y 1 (t) = S{x 1 (t)} and y 2 (t) = S{x 2 (t)} then S{x 1 (t) + x 2 (t)} = y sum (t) = y 1 (t) + y 2 (t). Time invariance shift in input results in shift in output. G. Tzanetakis 22 / 38

23 Sinusoids and LTI Systems When a sinusoids of frequency ω goes through a LTI system it stays in the family of sinusoids of frequency ω i.e only the amplitude and the phase are changed by the system. Because of linearity this implies that if a complex signal is a sum of sinusoids of different frequencies then the system output will not contain any new frequencies. The behavior of the system can be completely understood by simply analyzing how it responds to elementary sinusoids. Examples of LTI systems in music: guitar boy, vocal tract, outer ear, concert hall. G. Tzanetakis 23 / 38

24 Thinking in circles Key insight Think of sinusoidal signal as a vector rotating at a constant speed in the plane (phasor) rather than a single valued signal that goes up and down. Amplitude = Length Frequency = Speed Phase = Angle at time t G. Tzanetakis 24 / 38

25 Projecting a phasor The projection of the rotating vector or phasor on the x-axis is a cosine wave and on the y-axis a sine wave. G. Tzanetakis 25 / 38

26 Notating a phasor Complex numbers An elegant notation system for describing and manipulating rotating vectors. x + jy where x is called the real part and y is called the imaginary part. If we represent a sinusoid as a rotating vector then using complex number notation we can simply write: cos(ωt) + jsin(ωt) G. Tzanetakis 26 / 38

27 Multiplication by j Multiplication by j is an operation of rotation in the plane. You can think of it as rotate +90 degrees counter-clockwise. Two successive rotations by +90 degrees bring us to the negative real axis, hence j 2 = 1. This geometric viewpoint shows that there is nothing imaginary or strange about complex numbers. G. Tzanetakis 27 / 38

28 Complex number multiplication Complex number addition is the same as vector addition i.e we add the x-coordinates (real parts) and the y-coordinates (imaginary parts). Where complex numbers draw their power is when they are multiplied. Complex number multiplication is can be done by following the rules of algebra blindly, and replacing j 2 with 1 when needed. However complex number multiplication makes more sense when we represent the complex numbers as vectors in polar form. When represented in polar form complex number multiplication has the property that the magnitude of the product is the product of the magnitudes and the angle of the product is the sum of the angles. This is the underlying reason why complex numbers are a great notation for dealing with rotations. G. Tzanetakis 28 / 38

29 Euler s formula Key insight The rotating vector that represents a sinusoid is just a single complex number raised to progressively higher and higher powers. Consider a rotating vector of unit magnitude. Let s use E(θ) the function that represents the vector at some arbitrary angle θ. Then from simple geometry: E(θ) = cos(θ) + j sin(θ) and de(θ) dθ = sin θ + j cos(θ) = je(θ) G. Tzanetakis 29 / 38

30 As can be seen this is a function for which the derivative is proportional to the original function and from calculus we know that that only the exponential function has this property so we can write our function E(θ) as: E(θ) = e jθ (6) So now we can express the fact that a rotating vector arising from simple harmonic motion can be notated as a complex number raised to higher and higher powers using the famous Euler formula named after the Swiss mathematician Leonard Euler ( ): e jθ = cos θ + j sin(θ) (7) G. Tzanetakis 30 / 38

31 Complex Conjugate A bit of notation that will be used later. Given a complex number z = Re jθ, its complex conjugate is defined as z = Re jθ. Geometrically z is the reflection of z in the real axis. G. Tzanetakis 31 / 38

32 Adding sinusoids of the same frequency I G. Tzanetakis 32 / 38

33 Adding sinusoids of the same frequency II Geometric view of the property that sinusoids (phasors) of a particular frequency ω are closed under addition. G. Tzanetakis 33 / 38

34 Negative frequencies and phasors G. Tzanetakis 34 / 38

35 Measuring the amplitude of a sinusoid G. Tzanetakis 35 / 38

36 Other DSP concepts with phasors Many DSP concepts can be visualized and understood nicely using phasors. It is fun to create animations similar to the ones I showed in this lecture to illustrate concepts such as: Sampling, nyquist frequency and aliasing (taking snapshots of the phasor as it goes around the circle) Filtering (effect of simple low-pass filter) Beating (phasors that are close in frequency) G. Tzanetakis 36 / 38

37 Book that inspired this DSP exposition A Digital Signal Processing Primer by Ken Steiglitz G. Tzanetakis 37 / 38

38 Summary Sinusoidal signals are fundamental in understanding DSP Representing them as phasors (i.e vectors rotating at a constant speed) can help understand intuitively several concepts in DSP Complex numbers are an elegant system for expressing rotations and can be used to notate phasors in a way that leverages our knowledge of algebra Thinking this way makes e jωt more intuitive. G. Tzanetakis 38 / 38

Introduction to signals and systems

CHAPTER Introduction to signals and systems Welcome to Introduction to Signals and Systems. This text will focus on the properties of signals and systems, and the relationship between the inputs and outputs