Prewhitening. 1. Make the ACF of the time series appear more like a delta function. 2. Make the spectrum appear flat.

Transcription

1 Prewhitening What is Prewhitening? Prewhitening is an operation that processes a time series (or some other data sequence) to make it behave statistically like white noise. The pre means that whitening precedes some other analysis that likely works better if the additive noise is white. These operations can be viewed in either the time domain or the frequency domain: 1. Make the ACF of the time series appear more like a delta function. 2. Make the spectrum appear flat. Example data sets that may require prewhitening: 1. A well behaved noise process with an additive low frequency (or polynomial) trend added to it. 2. A deterministic signal with an additive red-noise process. Viewed in the frequency domain, prewhitening means that the dynamic range of the measured data is reduced. 1

2 Why bother? Recall from our discussions of spectral analysis the issues of leakage and bias. These arise from sidelobes inherent to spectral estimation. We can minimize leakage in two ways: (1) make sidelobes smaller and (2) minimize the power that is prone to leaking into sidelobes. Spectral windows address the former while prewhitening mitigates the latter. Leakage into sidelobes also constitutes bias in spectral estimates. However bias appears in other data analysis procedures. Consider least-squares fitting of a sinusoid to a signal of the form x(t) = A cos(ωt + φ) + r(t) + n(t), where n(t) is WSS white noise and r(t) is red noise with a steep power spectrum. Red noise can strongly bias fitting of a model ˆx(t) = Â cos(ˆωt + ˆφ) because its power can leak across the underlying spectrum causing a least-squares fit to give highly discrepant values of Â, ˆω, and ˆφ. Prewhitening of the time series ideally would yield a transformed time series of the form x (t) = A cos(ωt + φ) + n (t) to which fitting a sinusoidal model will be less biased. 2

3 Procedures: We have already seen one analysis that is related to prewhitening: the matched filter (MF). The MF doesn t whiten the spectrum of the output but it does weight the frequency components of the measured quantity to maximize the S/N of the signal. The signal model in this case is x(t) = a A(t) + n(t). Recall for an arbitrary spectrum S n (f) for additive noise that the frequency-domain MF for a signal A(t) is h(f) Ã(f) S n (f). Taking equality for simplicity, when the filter is applied to the measurements x(t), we have ỹ(f) = x(f) h (f) a Ã(f) 2 S n (f) + ñ(f)ã (f). S n (f) This means that the ensemble-average spectrum of the filter output is ỹ(f) 2 = a2 Ã(f) 4 S 2 n(f) + ñ(f) 2 Ã(f) 2 S 2 n(f) = a2 Ã(f) 4 S 2 n(f) + S n(f) Ã(f) 2 S 2 n(f) = a2 Ã(f) 4 Sn(f) 2 = Ã(f) 2 S n (f) 3 + Ã(f) 2 S n (f) [ a 2 Ã(f) 2 S n (f) + 1 ]

4 Signals with trends: A common situation is where a quantity of the form a A(t) + n(t) is superposed with a strong trend, such as a baseline variation. Similar issues arise in measurements of spectra. Consequences of trends include: 1. Bias in estimating parameters of A(t t ) or its spectral analog A(ν ν ). 2. Erroneous estimates of cross correlations between two time series such as x(t) = s 1 (t) + n 1 (t) and y(t) = s 2 (t) + n 2 (t), where s 1,2 are signals of interest and n 1,2 are measurement errors. I.e. we may be interested in the correlation C = 1 s 1 (t)s 2 (t) or C = 1 [s 1 (t) s 1 ][s 2 (t) s 2 ] N t N t t where s 1,2 = (1/N t ) t s 1,2(t) are the sample means. If there are trends p 1,2 (t) added to x(t) and y(t) the correlation Ĉ of x and y used to estimate C may be dominated completely by the trends and not the signal parts of the measurements. A fix: Trends can often be modeled as a polynomial of some order that can be fitted to the measurements. The order of the polynomial needs to be chosen wisely. For a pulse or spectral line confined to some range of t or ν this is straight forward. But for a detection problem where the signal location is not known, the situation is very tricky. t 4

5 Prewhitening filter: Consider again x(t) = a A(t) + n(t) and let s trivially construct a frequencydomain filter that whitens the measurements. We want a filter h(t) that flattens the noise n(t) in the frequency domain. Let y(t) = x(t) h(t) where means convolution. All we need is h(f) = S n (f). Then the ensemble spectrum of the output ỹ(f) is ỹ(f) 2 = x(f) 2 h(f) 2 = x(f) 2 S n (f) = a2 Ã(f) 2 S n (f) Note how this differs from the result for a matched filter. But the result is that in the mean the spectrum of the additive noise has been flattened. Prewhitening is important in both detection and estimation applications

6 Prewhitening in the least-squares estimation context: Consider our standard linear model y = Xθ + n, which has a least-squares solution for the parameter vector θ = ( X C 1 n X ) 1 X C 1 n y, where the covariance matrix of the noise vector n is C n = nn. This is also the maximum likelihood solution in the right circumstances (which are?). As with any covariance matrix, C n is Hermitian and positive, semi-definite. This means that the quadratic form for an arbitrary vector z satisfies z C n z. Such matrices can always be factored according to the Cholesky decomposition: where L is a lower-diagonal matrix; e.g. L = C n = LL a b c d e f g h i j. 6

7 Utility: we can transform the model as follows using L: y = Ly w X = LX w. Substituting into the solution vector for θ and using yields y = (Ly w ) = y wl, X = (LX w ) = X wl, and C 1 n θ = ( X C 1 n X ) 1 X C 1 n y = (LL ) 1 = L 1 L 1 = (X w L C 1 n L }{{} I = ( X wx w ) 1 X w y. X w ) 1 X w L C 1 n L y }{{} I So what? The solution is identical to the least-squares case where the noise covariance matrix is diagonal; i.e. the noise vector n w = L 1 n has been transformed to white noise. We have whitened the data. When is this useful? An example is the fitting of a sinusoidal function amid red noise where leakage effects are important just as they are for spectral analysis. A specific example is the fitting of astrometric parameters or periodicities in radial velocity data. What s the catch? You need to know the covariance matrix of the noise n to do the Cholesky decomposition. This can be easier said than done! 7

8 Examples of sine wave + red and white noise Examples were generated with a signal y(t) = cos(2πt/p + φ) + r(t)/snr r + w(t)/snr w where r, w have unit variance and are scaled by the signal to noise ratios snr r and snr w, respectively. The covariance matrix for the combined noise n = r + w was calculated by averaging C n = nn over 1 realizations. Note that for some real situations where we have only a single time series, we would need to calculate C n differently, e.g. from first principles, prior knowledge, etc. In practice, realizations of r were generated and the mean subtracted. Then white noise was added to form n and then the Cholesky decomposition was done using the command L = scipy.linalg.cholesky(c n, lower=true) For data vectors of length N, the lower-diagonal matrix L is N N. If the mean had been subtracted from the white noise as well, the rank of the covariance matrix would be N 1 and the decomposition would fail. Results in the following figures indicate that 1. Power-law red noise with spectral indices s i < 2 do not benefit particularly from whitening because leakage is much less. 8

9 2. What matters is the signal to noise ratio of the cosine to the signal contained in one resolution bandwidth f T 1 centered on the frequency of the sinusoid. For a steep power law, only a small fraction of the total power in the red noise is in this band whereas the flatter the spectrum, the larger this fraction is. 9

10 Signal + Noise Noise only Cholesky whitening: N =256 Sine+RN+WN S i = 1. S/N r =.1 S/N w = 1. Time Series Spectra Time (bins) Frequency (bins) Figure 1: Example of whitening using the Cholesky decomposition. The signal consists of a sine wave with period of 1.23 time bins with additive red and white noise. Signal-to-noise ratios of the signal relative to each kind of noise are given. Top left: original time series (red) and whitened time series (black). Bottom left: original noise (red) and whitened noise (black). Top right: power spectra of the original and whitened time series. Bottom right: power spectra of original and whitened noise sequences. 1

11 Cholesky whitening: N =256 Sine+RN+WN S i = 1. S/N r =.5 S/N w = 1. Time Series Spectra 6 1 Signal + Noise Noise only Time (bins) Frequency (bins) Figure 2: Example of whitening using the Cholesky decomposition. The signal consists of a sine wave with period of 1.23 time bins with additive red and white noise. Signal-to-noise ratios of the signal relative to each kind of noise are given. Top left: original time series (red) and whitened time series (black). Bottom left: original noise (red) and whitened noise (black). Top right: power spectra of the original and whitened time series. Bottom right: power spectra of original and whitened noise sequences. 11

12 Signal + Noise Noise only Cholesky whitening: N =256 Sine+RN+WN S i = 2. S/N r =.1 S/N w = 1. Time Series Spectra Time (bins) Frequency (bins) Figure 3: Example of whitening using the Cholesky decomposition. The signal consists of a sine wave with period of 1.23 time bins with additive red and white noise. Signal-to-noise ratios of the signal relative to each kind of noise are given. Top left: original time series (red) and whitened time series (black). Bottom left: original noise (red) and whitened noise (black). Top right: power spectra of the original and whitened time series. Bottom right: power spectra of original and whitened noise sequences. 12

13 Signal + Noise Noise only Cholesky whitening: N =256 Sine+RN+WN S i = 2. S/N r =.1 S/N w = 1. Time Series Spectra Time (bins) Frequency (bins) Figure 4: Example of whitening using the Cholesky decomposition. The signal consists of a sine wave with period of 1.23 time bins with additive red and white noise. Signal-to-noise ratios of the signal relative to each kind of noise are given. Top left: original time series (red) and whitened time series (black). Bottom left: original noise (red) and whitened noise (black). Top right: power spectra of the original and whitened time series. Bottom right: power spectra of original and whitened noise sequences. 13

14 Signal + Noise Noise only Cholesky whitening: N =256 Sine+RN+WN S i = 2. S/N r =.5 S/N w = 1. 5 Time Series 1 Spectra Time (bins) Frequency (bins) Figure 5: Example of whitening using the Cholesky decomposition. The signal consists of a sine wave with period of 1.23 time bins with additive red and white noise. Signal-to-noise ratios of the signal relative to each kind of noise are given. Top left: original time series (red) and whitened time series (black). Bottom left: original noise (red) and whitened noise (black). Top right: power spectra of the original and whitened time series. Bottom right: power spectra of original and whitened noise sequences. 14

15 Signal + Noise Noise only Cholesky whitening: N =256 Sine+RN+WN S i = 3. S/N r =.1 S/N w = 1. 4 Time Series 1 3 Spectra Time (bins) Frequency (bins) Figure 6: Example of whitening using the Cholesky decomposition. The signal consists of a sine wave with period of 1.23 time bins with additive red and white noise. Signal-to-noise ratios of the signal relative to each kind of noise are given. Top left: original time series (red) and whitened time series (black). Bottom left: original noise (red) and whitened noise (black). Top right: power spectra of the original and whitened time series. Bottom right: power spectra of original and whitened noise sequences. 15

16 Signal + Noise Noise only Cholesky whitening: N =256 Sine+RN+WN S i = 5. S/N r =.1 S/N w = 1. Time Series Spectra Time (bins) Frequency (bins) Figure 7: Example of whitening using the Cholesky decomposition. The signal consists of a sine wave with period of 1.23 time bins with additive red and white noise. Signal-to-noise ratios of the signal relative to each kind of noise are given. Top left: original time series (red) and whitened time series (black). Bottom left: original noise (red) and whitened noise (black). Top right: power spectra of the original and whitened time series. Bottom right: power spectra of original and whitened noise sequences. 16

17 Signal + Noise Noise only Cholesky whitening: N =256 Sine+RN+WN S i = 5. S/N r =.1 S/N w = 1. Time Series Spectra Time (bins) Frequency (bins) Figure 8: Example of whitening using the Cholesky decomposition. The signal consists of a sine wave with period of 1.23 time bins with additive red and white noise. Signal-to-noise ratios of the signal relative to each kind of noise are given. Top left: original time series (red) and whitened time series (black). Bottom left: original noise (red) and whitened noise (black). Top right: power spectra of the original and whitened time series. Bottom right: power spectra of original and whitened noise sequences. 17

18 Cholesky whitening: N =256 Sine+RN+WN S i = 5. S/N r =.5 S/N w = 1. Time Series Spectra 6 1 Signal + Noise Noise only Time (bins) Frequency (bins) Figure 9: Example of whitening using the Cholesky decomposition. The signal consists of a sine wave with period of 1.23 time bins with additive red and white noise. Signal-to-noise ratios of the signal relative to each kind of noise are given. Top left: original time series (red) and whitened time series (black). Bottom left: original noise (red) and whitened noise (black). Top right: power spectra of the original and whitened time series. Bottom right: power spectra of original and whitened noise sequences. 18

19 Impulse Response and Spectrum of Whitening Filter We can think of the Cholesky decomposition as a filter that suppresses low frequencies for the purpose of estimating the parameters of a sinusoid. The filter response can be calculated from the impulse response as follows: Construct a data vector i corresponding to i j = for all j except j = j where i j = 1. Then the impulse response is h = L 1 i. Then, expressed as a time function h j, j = 1,, N, the frequency-domain response is the squared magnitude of the DFT of h j : H k = h k 2 19

20 Figure 1: Example of whitening using the Cholesky decomposition along with the impulse response and its spectrum. The signal consists of a sine wave with period of 1.23 time bins with additive red and white noise. Signal-to-noise ratios of the signal relative to each kind of noise are given. Left figure: Top left: original time series (red) and whitened time series (black). Bottom left: original noise (red) and whitened noise (black). Top right: power spectra of the original and whitened time series. Bottom right: power spectra of original and whitened noise sequences. Right figure: Top panel: input impulse (red) and impulse response of the Cholesky filter. Bottom Panel: Spectra of the impulse and impulse response, respectively. The filter shows the suppression of frequencies below about 25 bins; this frequency is signal-to-noise ratio dependent. 2