Square Waves, Sinusoids and Gaussian White Noise: A Matching Pursuit Conundrum? Don Percival

Square Waves, Sinusoids and Gaussian Whie Noise: A Maching Pursui Conundrum? Don Percival Applied Physics Laboraory Deparmen of Saisics Universiy of Washingon Seale, Washingon, USA hp://faculy.washingon.edu/dbp Saniy Checkers: Peer Craigmile and Barry Quinn

Inroducion maching pursui approximaes a vecor of ime series values X = [X 0, X 1,..., X N 1 ] T using a linear combinaion of vecors picked from a (ypically quie large) se of vecors D each vecor in D has some inerpreaion, allowing us o exrac feaures of poenial ineres from X inroduced ino engineering lieraure by Malla & Zhang (1993) alk will focus on an unexpeced finding (he conundrum!) ha appeared when applying maching pursui o a climaology ime series 1

Overline of Remainder of Talk discuss basic ideas behind maching pursui (MP) discuss applicaion of MP o climaology ime series ha led o conundrum discuss enaive bu unsaisfying explanaion of conundrum los of open quesions, including wha (if anyhing!) o do nex 2

Maching Pursui: I given a ime series X of dimension N and a vecor d of similar dimension saisfying kdk 2 = hd, di = N 1 X =0 d 2 = 1, consider approximaing X using d in a linear model: X = βd + e, where β is unknown, and e is he error in he approximaion can minimize kek 2 by seing β equal o hx, di = P N 1 =0 X d approximaion is A = hx, did & residuals are R = X A 3

Maching Pursui: II in addiion o addiive decomposiion X = A + R, also have decomposiion of sum of squares: kxk 2 = kak 2 + krk 2 = hx, di 2 + krk 2 now consider a se of vecors D, each d k D leading o X = A k + R k and kxk 2 = hx, d k i 2 + kr k k 2 declare bes approximaion o be he one for which kr k k 2 is smalles, i.e., for which hx, d k i is larges call his approximaion A (1) = hx, d (1) id (1), and le R (1) be he corresponding vecor of residuals so ha X = A (1) + R (1) and kxk 2 = hx, d (1) i 2 + kr (1) k 2 4

Maching Pursui: III firs sage of MP leads o X = A (1) + R (1) and kxk 2 = hx, d (1) i 2 + kr (1) k 2 second sage reas R (1) as X was reaed, leading o R (1) = A (2) + R (2) and kr (1) k 2 = hr (1), d (2) i 2 + kr (2) k 2 sages j = 3, 4... give us R (j 1) = A (j) +R (j) and kr (j 1) k 2 = hr (j 1), d (j) i 2 +kr (j) k 2 defining R (0) = X, afer J such seps, have JX JX X = A (j) +R (J) and kxk 2 = hr (j 1), d (j) i 2 +kr (J) k 2 j=1 j=1 5

Maching Pursui: IV MP is greedy in ha, a each sage j, approximaing vecor is he one maximizing hr (j 1), d k i amongs all d k D under cerain condiions on conens of D, kr (j) k 2 mus decrease and reach zero as j increases choice of vecors o place in D is obviously criical o qualiy of resuling approximaion and is applicaion dependen 6

millibars Norh Pacific Index (NPI): I area-weighed sea level pressure over 30 N o 65 N & 160 E o 140 W & over November o March for each year from 1900 o 1999 (Trenberh & Paolino, 1980; Trenberh & Hurrell, 1994) 1016 1014 1012 1010 1008 1006 1004 1002 1900 1920 1940 1960 1980 2000 year 7

Norh Pacific Index (NPI): II Minobe (1999) posulaed exisence of pena- and bi-decadal oscillaions in NPI ha... canno be aribued o a single sinusoidal-wavelike variabiliy... ; i.e., ransiions beween values above and below he long erm mean of NPI occur much faser han sinusoidal variaions can easily accoun for can (informally) evaluae Minobe s hypohesis by subjecing NPI o MP (X hus conains all N = 100 values of NPI, bu afer cenering by subracing off he sample mean) D consiss of boh sinusoidal and square wave oscillaions, wih frequencies dicaed by Fourier frequencies j/100, j = 1, 2,..., 50 (periods are 100/j years), along wih all possible phase shifs 8

Examples of Vecors in D period of 100 years, and one of 50 possible phase shifs 9

Examples of Vecors in D period of 50 years, and one of 25 possible phase shifs 10

Examples of Vecors in D period of 100/3 years (oher phase shifs no shown) 11

Examples of Vecors in D period of 25 years (oher phase shifs no shown) 12

Examples of Vecors in D period of 20 years (oher phase shifs no shown) 13

Examples of Vecors in D period of 4 years (oher phase shifs no shown) 14

Maching Pursui of NPI: I j = 1: square wave, 50 years; 17.4% of variance explained 1900 1920 1940 1960 1980 2000 year 15

Maching Pursui of NPI: II j = 2: square wave, 20 years; 24.1% of variance explained 1900 1920 1940 1960 1980 2000 year 16

Maching Pursui of NPI: III j = 3: square wave, 14 years; 30.6% of variance explained 1900 1920 1940 1960 1980 2000 year 17

Maching Pursui of NPI: IV j = 4: sinusoid, 4.3 years; 36.4% of variance explained 1900 1920 1940 1960 1980 2000 year 18

Maching Pursui of NPI: V MP lends credence o Minobe s hypohesis (pena- and bidecadal oscillaions wih faser above/below ransiions han sinusoids can explain) Q: wha (if anyhing) can we say abou saisical significance of paerns picked ou by MP? 19

The Conundrum: I o address quesion of significance, need o consider wha MP does under various null hypoheses simplies such hypohesis is ha X is Gaussian whie noise (i.e., independen and idenically disribued normal random variables) noe ha X should have no discernable srucure will ake X o have zero mean and covariance/correlaion marix I N (Nh order ideniy marix) le K denoe number of vecors d k in se D, and le D = [d 1, d 2,..., d K ] so ha kh elemen of Y D T X is hx, d k i Y is mulivariae Gaussian wih zero mean and wih Σ D T D as is covariance/correlaion marix noe ha (j, k)h elemen of Σ is d T j d k 20

The Conundrum: II firs sep of MP picks elemen of Y wih larges magniude, so disribuion of his pick depends jus on mulivariae Gaussian correlaion marix Σ if D = {d 1, d 2 }, hen 1 d Σ = T 1 d 2 d T 2 d, 1 1 and, by symmery, MP will pick d 1 & d 2 each 50% of he ime, no maer wha hey are (e.g., a sinusoid & a square wave) if D has more hen wo elemens, analysis becomes messy, bu can resor o Mone Carlo experimens using same D as in NPI analysis (50% of vecors are sinsuoids, and 50% are square waves), MP picks sinusoids 15% of he ime and square waves 85% of he ime!?! 21

Slouching Towards an Explanaion: I why does Gaussian whie noise mach up beer wih square waves han sinusoids? consider case N = 8 wih D conaining four sinusoids (d 1, d 2, d 3 and d 4 ) and four square waves (d 5, d 6, d 7 and d 8 ), all wih a period of 8 22

Two of Eigh Vecors in D 0 1 2 3 4 5 6 7 0 1 2 3 4 5 6 7 23

Slouching Towards an Explanaion: II Mone Carlo experimens indicae ha MP picks a sinusoid 29% of he ime and a square wave 71% of he ime correlaion marix Σ in his case looks like he following: d 1 d 2 d 3 d 4 d 5 d 6 d 7 d 8 d 1 1.0 d 2 0.7 1.0 d 3 0.0 0.7 1.0 d 4 0.7 0.0 0.7 1.0 d 5 0.9 0.9 0.4 0.4 1.0 d 6 0.4 0.9 0.9 0.4 0.5 1.0 d 7 0.4 0.4 0.9 0.9 0.0 0.5 1.0 d 8 0.9 0.4 0.4 0.9 0.5 0.0 0.5 1.0 sinusoids have more exreme cross-correlaions han do square waves is his par of he explanaion? 24

Slouching Towards an Explanaion: III consider anoher D, his ime wih wo sinusoids (d 1 and d 2 ) and wo square waves (d 3 and d 4 ), all again wih a period of 8 25

Two of Four Vecors in D 0 1 2 3 4 5 6 7 0 1 2 3 4 5 6 7 26

Slouching Towards an Explanaion: IV Mone Carlo experimens indicae ha MP picks a sinusoid 48.5% of he ime and a square wave 51.5% of he ime correlaion marix Σ in his case looks like he following: d 1 d 2 d 3 d 4 d 1 1.00 d 2 0.00 1.00 d 3 0.35 0.85 1.00 d 4 0.35 0.85 0.50 1.00 sinusoids now have zero cross-correlaion, whereas square waves have a posiive cross-correlaion, ye square waves are sill preferred (bu jus slighly so) canno explain conundrum in erms of jus cross-correlaions 27

Hmmm... 28

References S. G. Malla and Z. Zhang (1993), Maching Pursuis wih Time-Frequency Dicionaries, IEEE Transacions on Signal Processing, 41, pp. 3397 3415 S. Minobe (1999) Resonance in Bidecadal and Penadecadal Climae Oscillaions over he Norh Pacific: Role in Climae Regime Shifs, Geophysical Research Leers, 26, pp. 855 858 D. B. Percival, J. E. Overland and H. O. Mofjeld (2002), Using Maching Pursui o Assess Amospheric Circulaion Changes over he Norh Pacific, unpublished manuscrip available a hp://faculy.washingon.edu/dbp/research.hml K. E. Trenberh and J. W. Hurrell (1994), Decadal Amosphere Ocean Variaions in he Pacifc, Climae Dynamics, 9, pp. 303 19 K. E. Trenberh and D. D. Paolino (1980), The Norhern Hemisphere Sea Level Pressure Daa Se: Trends, Errors, and Disconinuiies. Monhly Weaher Review, 108, pp. 855 72 29