Department of Electrical Engineering IIT Madras
Outline Definition of Power Spectrum Deterministic signal example Power Spectrum of a Random Process The Periodogram Estimator The Averaged Periodogram Blackman-Tukey Method Use of Data Windowing in Spectral Analysis Spectrogram: Speech Signal Example
What is Spectral Analysis? Spectral analysis is the estimation of the frequency content of a random process By frequency content we mean the distribution of power over frequency Also called Power Spectral Density, or simply spectrum What frequency components are present? What is the intensity of each component?
The Earliest Spectral Analyzer All colours are present with equal intensity
What is Frequency? Our notion of frequency comes from sin(2πf t) and cos(2πf t) both are called sinusoids Frequency Sinusoidal Frequency: f cycles/sec (Hz) exp(j2πf t) is the basis function needed for representing a component with frequency f for an arbitrary frequency component, it becomes exp(j2πft) As f varies from to, we get the Fourier basis set! f is sometimes called Fourier frequency
Spectral Analysis Expansion Using Fourier Basis Spectral analysis is nothing but expanding a signal x(t) using the Fourier basis X (f ) = x(t) exp( j2πft) dt inner product! X (f ) is the continuous-time Fourier transform of x(t) X (f 1 ) large dominant frequency component at f = f 1 X (f 1 ) = no frequency component at f = f 1 Plot of X (f ) 2 as a function of f is called the power spectrum
Deterministic Signal Example Gated sinusoid with f = 1 khz Magnitude 1 1 6 4 2 2 2 4 6 8 1 12 14 16 Time (ms) 4 3 2 1 1 2 3 4 Frequency (Hz) Log Magnitude 4 2 2 4 3 2 1 1 2 3 4 Frequency (Hz)
What About PSD of a Random Process? Spectral analysis is the estimation of the frequency content of a random process An ensemble of sample waveforms constitute a random process X (f )? = x(t) exp( j2πft) dt Does it exist? Even if it does, is it meaningful? We ll focus on discrete-time random processes, i.e., ensemble of x[n], where n Z
PSD of a WSS Random Process Let x[n] be a complex wide-sense stationary process Its autocorrelation sequence (ACS) is defined as Wiener-Khinchine Theorem: P xx (f ) = r xx [k] = E{x [n] x[n + k]} r xx [k] exp( j2πfk) 1 2 f 1 2 That is, ACS DTFT PSD
An Alternative Definition for PSD If the ACS decays sufficiently rapidly, P xx (f ) = lim E 1 M 2 x[n] exp( j2πfn) M 2M + 1 M The so-called Direct Method is based on the above formula
Why is the Problem Difficult? ACS is not available Finite number of samples from one realization We are only given x[], x[1],..., x[n 1] No best spectral estimator exists Many practical signals, such as speech, are non-stationary P xx (f ) obtained from given data is a random variable Bias versus Variance trade-off
The Periodogram Estimator Recall P xx (f ) = lim E 1 M 2 x[n] exp( j2πfn) M 2M + 1 In practice we drop because data are finite lim M M the expectation operator E since we have only one realization The Periodogram estimator is defined as ˆP PER (f ) def = 1 N N 1 x[n] exp( j2πfn) n= Direct Method, since it deals with the data directly 2
Example: Two Sine Waves + Noise x[n] = 1 exp(j 2π.15n) + 2 exp(j 2π.2n) + z[n] z[n] complex N (, 1), N = 2 5 4 3 Magnitude (db) 2 1 1 2 3.5.4.3.2.1.1.2.3.4.5 Frequency
Periodogram is a Biased Estimator For Finite Data For finite N, periodogram is a biased estimator Bias is the difference between the true and expected values 5 4 averaged noiseless Magnitude (db) 3 2 1 1 N = 2 2 3.5.4.3.2.1.1.2.3.4.5 Frequency
Periodogram: Bias Decreases With Increasing N If data length is increased, bias decreases: } lim {ˆP E xx (f ) = P xx (f ) N 6 5 4 Magnitude (db) 3 2 1 1 2 N = 1 3.5.4.3.2.1.1.2.3.4.5 Frequency
The More (Samples) the Merrier? For most estimators, bias and variance decrease with increasing N An estimator is said to be consistent if ( ) lim ˆθ Pr θ > ɛ = N where ˆθ is the estimate of θ This implies that, as N, bias variance
Is the Periodogram Consistent? Consider white noise sequence for various N True P xx (f ) = constant If the Periodogram estimator were consistent, ˆP xx (f ) constant as N increases Consider noise sequences of length 32, 64, 128, and 256 N = 32; % white noise sequence x = randn(n,1); % of length 32 Does ˆP xx (f ) tend to a constant as N increases?
White Noise Example 4 PSD of White Noise N=32.5.5 4.5.5 4.5.5 4 N=256.5.4.3.2.1.1.2.3.4.5 Frequency As N increases, variance of the estimate does not decrease Periodogram is an inconsistent estimator
What Went Wrong? In practice we drop lim because data are finite M the expectation operator E since we have only one realization For white noise, increasing the data length did not help What can be done to capture the benefits of E{ }?
Averaging: The Poor Man s Expectation Operator Expectation operator can be approximated by averaging Averaged Periodogram: ˆP AVPER (f ) = 1 M M m=1 ˆP (m) PER (f ) where ˆP (m) PER (f ) is periodogram of m-th segment of length N For independent data records } var {ˆP AVPER (f ) = 1 } {ˆP M var PER (f )
Averaged Periodogram of White Noise Result of averaging 8 periodograms 4 Averaged Periodogram 35 3 25 Magnitude (db) 2 15 1 5 5 1.5.25.25.5 Frequency
Variance Decreases, But Bias Increases! Two Sines + Noise example 2 1 N=256, M=1 N=64, M=4 N=16, M=16 Averaged Periodogram for Two Sines + Noise Magnitude (db) 1 2 3 4 5.5.25.25.5 Frequency
Welch s Method Overlapping blocks by 5% Reduces variance without worsening bias 15 1 5 Block Length = 64 No. of blocks = 7 Overlap = 5% Welch s Method Magnitude (db) 5 1 15 2 25.5.25.25.5 Frequency
Why Did The Periodogram Fail? Periodogram was defined as ˆP PER (f ) = 1 N N 1 n= x[n] exp( j2πfn) 2 Equivalent to where ˆr xx [k] = ˆP PER (f ) = N 1 k 1 N n= ˆr xx[ k] N 1 (N 1) ˆr xx [k] exp( j2πfk) x [n] x[n + k] k =, 1,..., N 1 Note that ˆr xx [N 1] = x []x[n 1]/N No averaging estimate with high variance! k = (N 1),..., 1
Blackman-Tukey Method Recall P xx (f ) = r xx [k] exp( j2πfk) 1 2 f 1 2 In practice: (a) replace r xx [k] by estimate ˆr xx [k], (b) truncate the summation, and (c) apply lag window M ˆP BT f ) = w[k] ˆr xx [k] exp( j2πfk) M where w[k] w[] = 1 w[k] = for k > M w[ k] = w[k] W (f ) Indirect Method, since it does not deal with the data directly
Example: Two Sine Waves + Noise Data length N = 1, Correlation Lag M = 1 6 5 4 Magnitude (db) 3 2 1 1 2 3.5.4.3.2.1.1.2.3.4.5 Frequency
Periodogram Vs. Blackman-Tukey Magnitude (db) Magnitude (db) 6 4 2 2 Periodogram.5.25.25.5 6 4 2 2 Blackman Tukey.5.25.25.5 Frequency Blackman-Tukey method: reduction in variance comes at the expense of increased bias Speech Analysis
Data Windowing in Spectral Analysis Useful for data containing sinusoids + noise Sidelobes of a stronger sinusoid may mask the main lobe of a nearby weak sinusoid We multiply x[n] by data window w[n] before computing periodogram Weaker sinusoid becomes more visible Main lobe of each sinusoid broadens: two close peaks may merge into one
Example: How Many Sine Waves Are There? 6 How Many Sinusoids Are There? 4 Magnitude (db) 2 2 4 6.1.12.14.16.18.2.22 Frequency
Example: Three Sine Waves 6 Three Sinusoids: Rectangular Window 4 Magnitude (db) 2 2 4.15.157 6.1.12.14.16.18.2.22 Frequency
Example: Three Sine Waves 6 Three Sinusoids: Hanning Window 4 Magnitude (db) 2 2 4.15.157 6.1.12.14.16.18.2.22 Frequency
Commonly Used Windows Name w[k] Fourier transform sin πf (2M + 1) Rectangular 1 W R (f ) = sin πf Bartlett 1 k ( ) 1 sin πfm 2 M M sin πf Hanning.5 +.5 cos πk ( ).25 W R f 1 M ( 2M ) +.5 WR (f ) +.25 W R f + 1 Hamming.54 +.46 cos πk M w[k] = for k > M 2M.23 W ( ) R f 1 ( 2M ) +.54 WR (f ) +.23 W R f + 1 2M
Hamming Vs. Hanning Magnitude (db) Magnitude (db) 5 1.5.25.25.5 Frequency 2 4 Fourier Transforms of Hamming and Hanning Windows Hamming Hanning 6.5.25.25.5 Frequency
Three Sine Waves 6 Rectangular Vs. Hamming Vs. Hanning 4 Magnitude (db) 2 2 4.15.157 6.1.12.14.16.18.2.22 Frequency
How Can We Analyze Non-Stationary Signals? Consider a linear chirp, i.e., a signal whose frequency increases linearly from f 1 Hz to f 2 Hz over a time interval T What is its magnitude spectrum? 1 Amplitude Magnitdue (db) 1.1.2.3.4.5 Time 6 4 2 2 4 6 8 1 Frequency
Need a More Useful Representation In Fourier analysis, even if a signal is non-stationary, it is still represented using stationary sinusoids An unsatisfactory approach Power spectrum is identical to x( t), whose frequency decreases from f 2 to f 1 x(t) and x( t) differ only in the phase of the Fourier transform What we really want to know is how frequency varies with time Can it still be called frequency?
Spectrogram Plot of power spectrum of short blocks of a signal as a function of time Over each short block, signal is considered to be stationary Speech is a classic example of a commonly occurring non-stationary signal Voiced sounds: /a/, /e/, /i/, /o/, /u/ (quasi-periodic) Unvoiced sounds: /s/, /sh/, /f/ (noise-like) Plosives: /p/, /t/, /k/ (transient sounds)
Spectrogram of Linear Chirp 6 5 4 Frequency 3 2 1.1.2.3.4 Time
Non-stationarity in Speech Signal.2 /k/.2 1.16 1.17 1.18 1.19 1.2 1.21 1.22 1.23 1.24 1 /ow/ 1 1.24 1.26 1.28 1.3 1.32 1.34 1.36 1.38.5 /s/.5 1.75 1.8 1.85 1.9 Time
Application to Speech Analysis 1 Should We Chase Those Cowboys? Amplitude.5.5 Frequency 1.25.5.75 1 1.25 1.5 1.75 2 5 4 3 2 1.25.5.75 1 1.25 1.5 1.75 2 Time
Application to Speech Analysis 1 Should We Chase Those Cowboys? Amplitude.5.5 Frequency 1.25.5.75 1 1.25 1.5 1.75 2 5 4 3 2 1.25.5.75 1 1.25 1.5 1.75 2 Time
Application to Speech Analysis 1 Should We Chase Those Cowboys? Amplitude.5.5 Frequency 1.25.5.75 1 1.25 1.5 1.75 2 5 4 3 2 1.25.5.75 1 1.25 1.5 1.75 2 Time
Application to Speech Analysis 1 Should We Chase Those Cowboys? Amplitude.5.5 Frequency 1.25.5.75 1 1.25 1.5 1.75 2 5 4 3 2 1.25.5.75 1 1.25 1.5 1.75 Time
Summary Definition of Power Spectrum Deterministic signal example Power Spectrum of a Random Process The Periodogram Estimator The Averaged Periodogram Bias versus Variance Blackman-Tukey Method Use of Data Windowing in Spectral Analysis Spectrogram: Speech Signal Example