Square Waves, Sinusoids and Gaussian White Noise: A Matching Pursuit Conundrum? Don Percival

Transcription

1 Square Waves, Sinusoids and Gaussian Whie Noise: A Maching Pursui Conundrum? Don Percival Applied Physics Laboraory Deparmen of Saisics Universiy of Washingon Seale, Washingon, USA hp://faculy.washingon.edu/dbp Saniy Checkers: Peer Craigmile and Barry Quinn

2 Inroducion maching pursui approximaes a vecor of ime series values X = [X 0, X 1,..., X N 1 ] T using a linear combinaion of vecors picked from a (ypically quie large) se of vecors D each vecor in D has some inerpreaion, allowing us o exrac feaures of poenial ineres from X inroduced ino engineering lieraure by Malla & Zhang (1993) alk will focus on an unexpeced finding (he conundrum!) ha appeared when applying maching pursui o a climaology ime series 1

3 Overline of Remainder of Talk discuss basic ideas behind maching pursui (MP) discuss applicaion of MP o climaology ime series ha led o conundrum discuss enaive bu unsaisfying explanaion of conundrum los of open quesions, including wha (if anyhing!) o do nex 2

4 Maching Pursui: I given a ime series X of dimension N and a vecor d of similar dimension saisfying kdk 2 = hd, di = N 1 X =0 d 2 = 1, consider approximaing X using d in a linear model: X = βd + e, where β is unknown, and e is he error in he approximaion can minimize kek 2 by seing β equal o hx, di = P N 1 =0 X d approximaion is A = hx, did & residuals are R = X A 3

5 Maching Pursui: II in addiion o addiive decomposiion X = A + R, also have decomposiion of sum of squares: kxk 2 = kak 2 + krk 2 = hx, di 2 + krk 2 now consider a se of vecors D, each d k D leading o X = A k + R k and kxk 2 = hx, d k i 2 + kr k k 2 declare bes approximaion o be he one for which kr k k 2 is smalles, i.e., for which hx, d k i is larges call his approximaion A (1) = hx, d (1) id (1), and le R (1) be he corresponding vecor of residuals so ha X = A (1) + R (1) and kxk 2 = hx, d (1) i 2 + kr (1) k 2 4

6 Maching Pursui: III firs sage of MP leads o X = A (1) + R (1) and kxk 2 = hx, d (1) i 2 + kr (1) k 2 second sage reas R (1) as X was reaed, leading o R (1) = A (2) + R (2) and kr (1) k 2 = hr (1), d (2) i 2 + kr (2) k 2 sages j = 3, 4... give us R (j 1) = A (j) +R (j) and kr (j 1) k 2 = hr (j 1), d (j) i 2 +kr (j) k 2 defining R (0) = X, afer J such seps, have JX JX X = A (j) +R (J) and kxk 2 = hr (j 1), d (j) i 2 +kr (J) k 2 j=1 j=1 5

7 Maching Pursui: IV MP is greedy in ha, a each sage j, approximaing vecor is he one maximizing hr (j 1), d k i amongs all d k D under cerain condiions on conens of D, kr (j) k 2 mus decrease and reach zero as j increases choice of vecors o place in D is obviously criical o qualiy of resuling approximaion and is applicaion dependen 6

8 millibars Norh Pacific Index (NPI): I area-weighed sea level pressure over 30 N o 65 N & 160 E o 140 W & over November o March for each year from 1900 o 1999 (Trenberh & Paolino, 1980; Trenberh & Hurrell, 1994) year 7

9 Norh Pacific Index (NPI): II Minobe (1999) posulaed exisence of pena- and bi-decadal oscillaions in NPI ha... canno be aribued o a single sinusoidal-wavelike variabiliy... ; i.e., ransiions beween values above and below he long erm mean of NPI occur much faser han sinusoidal variaions can easily accoun for can (informally) evaluae Minobe s hypohesis by subjecing NPI o MP (X hus conains all N = 100 values of NPI, bu afer cenering by subracing off he sample mean) D consiss of boh sinusoidal and square wave oscillaions, wih frequencies dicaed by Fourier frequencies j/100, j = 1, 2,..., 50 (periods are 100/j years), along wih all possible phase shifs 8

10 Examples of Vecors in D period of 100 years, and one of 50 possible phase shifs 9

11 Examples of Vecors in D period of 100 years, and one of 50 possible phase shifs 9

12 Examples of Vecors in D period of 100 years, and one of 50 possible phase shifs 9

13 Examples of Vecors in D period of 100 years, and one of 50 possible phase shifs 9

14 Examples of Vecors in D period of 100 years, and one of 50 possible phase shifs 9

15 Examples of Vecors in D period of 100 years, and one of 50 possible phase shifs 9

16 Examples of Vecors in D period of 100 years, and one of 50 possible phase shifs 9

17 Examples of Vecors in D period of 100 years, and one of 50 possible phase shifs 9

18 Examples of Vecors in D period of 100 years, and one of 50 possible phase shifs 9

19 Examples of Vecors in D period of 100 years, and one of 50 possible phase shifs 9

20 Examples of Vecors in D period of 100 years, and one of 50 possible phase shifs 9

21 Examples of Vecors in D period of 100 years, and one of 50 possible phase shifs 9

22 Examples of Vecors in D period of 100 years, and one of 50 possible phase shifs 9

23 Examples of Vecors in D period of 100 years, and one of 50 possible phase shifs 9

24 Examples of Vecors in D period of 100 years, and one of 50 possible phase shifs 9

25 Examples of Vecors in D period of 100 years, and one of 50 possible phase shifs 9

26 Examples of Vecors in D period of 100 years, and one of 50 possible phase shifs 9

27 Examples of Vecors in D period of 100 years, and one of 50 possible phase shifs 9

28 Examples of Vecors in D period of 100 years, and one of 50 possible phase shifs 9

29 Examples of Vecors in D period of 100 years, and one of 50 possible phase shifs 9

30 Examples of Vecors in D period of 100 years, and one of 50 possible phase shifs 9

31 Examples of Vecors in D period of 100 years, and one of 50 possible phase shifs 9

32 Examples of Vecors in D period of 100 years, and one of 50 possible phase shifs 9

33 Examples of Vecors in D period of 100 years, and one of 50 possible phase shifs 9

34 Examples of Vecors in D period of 100 years, and one of 50 possible phase shifs 9

35 Examples of Vecors in D period of 100 years, and one of 50 possible phase shifs 9

36 Examples of Vecors in D period of 100 years, and one of 50 possible phase shifs 9

37 Examples of Vecors in D period of 100 years, and one of 50 possible phase shifs 9

38 Examples of Vecors in D period of 100 years, and one of 50 possible phase shifs 9

39 Examples of Vecors in D period of 100 years, and one of 50 possible phase shifs 9

40 Examples of Vecors in D period of 100 years, and one of 50 possible phase shifs 9

41 Examples of Vecors in D period of 100 years, and one of 50 possible phase shifs 9

42 Examples of Vecors in D period of 100 years, and one of 50 possible phase shifs 9

43 Examples of Vecors in D period of 100 years, and one of 50 possible phase shifs 9

44 Examples of Vecors in D period of 100 years, and one of 50 possible phase shifs 9

45 Examples of Vecors in D period of 100 years, and one of 50 possible phase shifs 9

46 Examples of Vecors in D period of 100 years, and one of 50 possible phase shifs 9

47 Examples of Vecors in D period of 100 years, and one of 50 possible phase shifs 9

48 Examples of Vecors in D period of 100 years, and one of 50 possible phase shifs 9

49 Examples of Vecors in D period of 100 years, and one of 50 possible phase shifs 9

50 Examples of Vecors in D period of 100 years, and one of 50 possible phase shifs 9

51 Examples of Vecors in D period of 100 years, and one of 50 possible phase shifs 9

52 Examples of Vecors in D period of 100 years, and one of 50 possible phase shifs 9

53 Examples of Vecors in D period of 100 years, and one of 50 possible phase shifs 9

54 Examples of Vecors in D period of 100 years, and one of 50 possible phase shifs 9

55 Examples of Vecors in D period of 100 years, and one of 50 possible phase shifs 9

56 Examples of Vecors in D period of 100 years, and one of 50 possible phase shifs 9

57 Examples of Vecors in D period of 100 years, and one of 50 possible phase shifs 9

58 Examples of Vecors in D period of 100 years, and one of 50 possible phase shifs 9

59 Examples of Vecors in D period of 100 years, and one of 50 possible phase shifs 9

60 Examples of Vecors in D period of 50 years, and one of 25 possible phase shifs 10

61 Examples of Vecors in D period of 50 years, and one of 25 possible phase shifs 10

62 Examples of Vecors in D period of 50 years, and one of 25 possible phase shifs 10

63 Examples of Vecors in D period of 50 years, and one of 25 possible phase shifs 10

64 Examples of Vecors in D period of 50 years, and one of 25 possible phase shifs 10

65 Examples of Vecors in D period of 50 years, and one of 25 possible phase shifs 10

66 Examples of Vecors in D period of 50 years, and one of 25 possible phase shifs 10

67 Examples of Vecors in D period of 50 years, and one of 25 possible phase shifs 10

68 Examples of Vecors in D period of 50 years, and one of 25 possible phase shifs 10

69 Examples of Vecors in D period of 50 years, and one of 25 possible phase shifs 10

70 Examples of Vecors in D period of 50 years, and one of 25 possible phase shifs 10

71 Examples of Vecors in D period of 50 years, and one of 25 possible phase shifs 10

72 Examples of Vecors in D period of 50 years, and one of 25 possible phase shifs 10

73 Examples of Vecors in D period of 50 years, and one of 25 possible phase shifs 10

74 Examples of Vecors in D period of 50 years, and one of 25 possible phase shifs 10

75 Examples of Vecors in D period of 50 years, and one of 25 possible phase shifs 10

76 Examples of Vecors in D period of 50 years, and one of 25 possible phase shifs 10

77 Examples of Vecors in D period of 50 years, and one of 25 possible phase shifs 10

78 Examples of Vecors in D period of 50 years, and one of 25 possible phase shifs 10

79 Examples of Vecors in D period of 50 years, and one of 25 possible phase shifs 10

80 Examples of Vecors in D period of 50 years, and one of 25 possible phase shifs 10

81 Examples of Vecors in D period of 50 years, and one of 25 possible phase shifs 10

82 Examples of Vecors in D period of 50 years, and one of 25 possible phase shifs 10

83 Examples of Vecors in D period of 50 years, and one of 25 possible phase shifs 10

84 Examples of Vecors in D period of 50 years, and one of 25 possible phase shifs 10

85 Examples of Vecors in D period of 100/3 years (oher phase shifs no shown) 11

86 Examples of Vecors in D period of 25 years (oher phase shifs no shown) 12

87 Examples of Vecors in D period of 20 years (oher phase shifs no shown) 13

88 Examples of Vecors in D period of 4 years (oher phase shifs no shown) 14

89 Maching Pursui of NPI: I j = 1: square wave, 50 years; 17.4% of variance explained year 15

90 Maching Pursui of NPI: II j = 2: square wave, 20 years; 24.1% of variance explained year 16

91 Maching Pursui of NPI: III j = 3: square wave, 14 years; 30.6% of variance explained year 17

92 Maching Pursui of NPI: IV j = 4: sinusoid, 4.3 years; 36.4% of variance explained year 18

93 Maching Pursui of NPI: V MP lends credence o Minobe s hypohesis (pena- and bidecadal oscillaions wih faser above/below ransiions han sinusoids can explain) Q: wha (if anyhing) can we say abou saisical significance of paerns picked ou by MP? 19

94 The Conundrum: I o address quesion of significance, need o consider wha MP does under various null hypoheses simplies such hypohesis is ha X is Gaussian whie noise (i.e., independen and idenically disribued normal random variables) noe ha X should have no discernable srucure will ake X o have zero mean and covariance/correlaion marix I N (Nh order ideniy marix) le K denoe number of vecors d k in se D, and le D = [d 1, d 2,..., d K ] so ha kh elemen of Y D T X is hx, d k i Y is mulivariae Gaussian wih zero mean and wih Σ D T D as is covariance/correlaion marix noe ha (j, k)h elemen of Σ is d T j d k 20

95 The Conundrum: II firs sep of MP picks elemen of Y wih larges magniude, so disribuion of his pick depends jus on mulivariae Gaussian correlaion marix Σ if D = {d 1, d 2 }, hen 1 d Σ = T 1 d 2 d T 2 d, 1 1 and, by symmery, MP will pick d 1 & d 2 each 50% of he ime, no maer wha hey are (e.g., a sinusoid & a square wave) if D has more hen wo elemens, analysis becomes messy, bu can resor o Mone Carlo experimens using same D as in NPI analysis (50% of vecors are sinsuoids, and 50% are square waves), MP picks sinusoids 15% of he ime and square waves 85% of he ime!?! 21

96 Slouching Towards an Explanaion: I why does Gaussian whie noise mach up beer wih square waves han sinusoids? consider case N = 8 wih D conaining four sinusoids (d 1, d 2, d 3 and d 4 ) and four square waves (d 5, d 6, d 7 and d 8 ), all wih a period of 8 22

97 Two of Eigh Vecors in D

98 Two of Eigh Vecors in D

99 Two of Eigh Vecors in D

100 Two of Eigh Vecors in D

101 Slouching Towards an Explanaion: II Mone Carlo experimens indicae ha MP picks a sinusoid 29% of he ime and a square wave 71% of he ime correlaion marix Σ in his case looks like he following: d 1 d 2 d 3 d 4 d 5 d 6 d 7 d 8 d d d d d d d d sinusoids have more exreme cross-correlaions han do square waves is his par of he explanaion? 24

102 Slouching Towards an Explanaion: III consider anoher D, his ime wih wo sinusoids (d 1 and d 2 ) and wo square waves (d 3 and d 4 ), all again wih a period of 8 25

103 Two of Four Vecors in D

104 Two of Four Vecors in D

105 Slouching Towards an Explanaion: IV Mone Carlo experimens indicae ha MP picks a sinusoid 48.5% of he ime and a square wave 51.5% of he ime correlaion marix Σ in his case looks like he following: d 1 d 2 d 3 d 4 d d d d sinusoids now have zero cross-correlaion, whereas square waves have a posiive cross-correlaion, ye square waves are sill preferred (bu jus slighly so) canno explain conundrum in erms of jus cross-correlaions 27

106 Hmmm... 28

107 References S. G. Malla and Z. Zhang (1993), Maching Pursuis wih Time-Frequency Dicionaries, IEEE Transacions on Signal Processing, 41, pp S. Minobe (1999) Resonance in Bidecadal and Penadecadal Climae Oscillaions over he Norh Pacific: Role in Climae Regime Shifs, Geophysical Research Leers, 26, pp D. B. Percival, J. E. Overland and H. O. Mofjeld (2002), Using Maching Pursui o Assess Amospheric Circulaion Changes over he Norh Pacific, unpublished manuscrip available a hp://faculy.washingon.edu/dbp/research.hml K. E. Trenberh and J. W. Hurrell (1994), Decadal Amosphere Ocean Variaions in he Pacifc, Climae Dynamics, 9, pp K. E. Trenberh and D. D. Paolino (1980), The Norhern Hemisphere Sea Level Pressure Daa Se: Trends, Errors, and Disconinuiies. Monhly Weaher Review, 108, pp