II. Random Processes Review

Transcription

1 II. Random Processes Review - [p. 2] RP Definition - [p. 3] RP stationarity characteristics - [p. 7] Correlation & cross-correlation - [p. 9] Covariance and cross-covariance - [p. 10] WSS property - [p. 13] RP time average and ergodicity - [p. 18] Periodic RP properties - [p. 22] Power Spectral Density - [p. 26] Linear transformations of RPs - [p. 32] Bandpass/lowpass (complex envelope) representations - [p. 46] Linear systems and bandpass/lowpass representations - [p. 58] Noise process: bandpass & lowpass (complex envelope) - [p. 67] Envelope statistics and use in signal detection - [p. 75] Monte Carlo performance evaluation and use in detection 1

2 Random Process (RP): A RP is a mapping function that attributes a function x(t) = x(t,ξ) to each outcome of the random experiment ξ 1 x(t, ξ 1 ) t ξ 2 ξ 3 x(t, ξ 2 ) x(t, ξ 3 ) t t Consider sequence x(t)= x(t,ξ) for a fixed t, x(t) is a Random Variable (RV) x(t) : random signal ( can be infinite dimensional) x(t,ξ) for fixed RV ξ: called realization/trial of the random process Example: x(t,ξ) = ξcos(πt/10), where ξ = U[0,1]. 2

3 Statistical Characterization of Random Processes: Random processes are characterized by joint distribution (or density) of sample values Consider the RP x(t) evaluated at specific points t k s, k=1,,n F x (x 1, x 2,, x k, t 1,, t k ) = Pr [x(t 1 ) x 1, x(t k ) x k ] F(.) is highly complex to compute - difficult or impossible to obtain in practice Stationarity: Definition: a RP is said to be stationary if any joint density or distribution function depends only on the spacing between samples, not where in the sequence these samples occur f x (x 1,, x N ; t 1,, t N ) = f x (x 1,, x N ; t 1+k,, t N+k ) for any k and any joint pdf 3

4 Stationarity con t: Recall: a RP is said to be stationary if any joint density or distribution function depends only on the spacing between samples, not where in the sequence the samples occur f x (x 1,, x N ; t 1,, t N ) = f x (x 1,, x N ; t 1+k,, t N+k ) for any k and any joint pdf If x(t) is stationary for all orders N = 1, 2, x(t) is said to be strict-sense stationary. If x(t) is stationary for order N = 1, f ( xt, ) = f( xt, + T) x Pdf is identical for all times samples x Stationary up to order 2 wide-sense stationary (WSS). 4

5 Stationarity of order N=1 - Physical interpretation for a discrete process x(n, ξ 1 ) x(n, ξ 2 ) x(n, ξ 3 ) x(n, ξ 4 ) x(n, ξ 5 ) x(n, ξ P )... x 1 x... 1 x 1 x 1... x 1... x 1 n n n n n n n Experiment is performed P times leads to P time sequences How to compute F x (x 1 ; n 1 ) = Pr [x(n 1 ) x 1 ] [Probability that the functions x(n,ξ) do not exceed x 1 at time n 1 ] Select values for x 1 and n 1 Count the number of trials K for which x(n 1 ) x 1 F x (x 1 ; n 1 ) = Pr [x(n 1 ) x 1 ] = K/P [1] 5

6 Stationarity of order N=2 - Physical interpretation for a discrete process x(n, ξ 1 ) x(n, ξ 2 ) x(n, ξ 3 ) x(n, ξ 4 ) x(n, ξ 5 ) x(n, ξ P )... x 1 x 2 x... 1 x 2 x 1 x 2 x 1... x 2 x 1... x 2 x 1 x 2 n 1 n n n n n n n Experiment is performed P times leads to P time sequences How to compute F x (x 1, x 2 ; n 1, n 2 ) = Pr [x(n 1 ) x 1, x(n 2 ) x 2 ] [Probability that the functions x(n,ξ) do not exceed x 1 at time n 1 and x 2 at time n 2 ] Select values for x 1, x 2,n 1, n 2 Count the number of trials K for which x(n 1 ) x 1 and x(n 2 ) x 2 F x (x 1,x 2 ; n 1, n 2 ) = K/P [1] 6

7 Random Process autocorrelation function { * } R t, t = R t, t = E x t x t xx ( ) ( ) ( ) ( ) 1 2 x Measures the dependency between values of the process at two different times. Allows to evaluate: 1) How quickly a random process changes with respect to time. 2) Whether the process has a periodic component and what the expected frequency might be, etc 7

8 Random Process cross-correlation function { * } R t, t = E x t y t xy ( ) ( ) ( ) Measures the dependency between values of two process at two different times. Allows to evaluate whether/how two processes are related 8

9 Random Process (auto)covariance function C t, t = C t, t xx ( ) ( ) 1 2 x 1 2 { ( ) ( ) * ( } 1 x( 1))( 2 x( 2)) = E x t m t x t m t Similar to correlation function: measures the dependency between values of the process at two different times, but Removes means impacts. Random Process cross-covariance function { * } C t, t = E ( x t m ( t ))( y t m ( t )) ( ) ( ) ( ) xy x 1 2 y 2 9

10 Wide-Sense Stationarity: Definition: a RP x(t) is called wide-sense stationary (WSS) if (1) the mean is a constant independent of the time { ( )} x ( ) E x t = m t = m (2) the autocorrelation depends only on the time lag distance τ = t 1 t 2 * R ( t, t ) = E x( t ) x ( t ) = R t t x { } x( ) * ( τ ) { ( ) ( τ )} = R = E x t x t x Consequence: the variance is a constant independent of t σ {( ) } () ( 2 ) () () () ( 0) 2 x x x ( 2) x ( ) { 2 } () 2 2 t = E x t m t = E x t m t x x x = R m = σ 10

11 Correlation Function Properties for wss x(t) (1) Conjugate symmetry R t = R t x () *( ) (3) R x (t) max at t = 0 and R x (0)>0 (can we have R x (0)=0?) x 11

12 RP Example: White noise Definition: A random process w(n) is called a white noise process with mean m w and variance σ 2 w iff E{w(t)}=m w R w (τ)= σ 2 ωδ(τ)=2n 0 δ(τ) Notes: Textbook notation 1) all frequencies contribute the same amount (as in the case of white light, therefore the name of white noise ) 2) if the pdf of w(t) is Gaussian: it is called white Gaussian noise 3) White noise is the simplest RP around because it doesn t have any structure and can be used as a building block 4) Physically impossible; in practice restricted to specific bandwidth B leading to power 2N 0 B 12

13 RP (time) Average: T 1 x() t = lim T + 2 T Ergodicity: T () x t dt in many applications only one realization of a RP is available in general, one single member doesn t provide information about the statistics of the process except when process is stationary +ergodic: statistical information cannot be derived from one realization of RP Def: a RP is called ergodic if: all ensemble averages = all corresponding time averages 13

14 Ergodicity cont : Def: a RP is said to be ergodic in the mean if: Def: a RP is said to be ergodic in correlation at time lag τ if: 1 Rx () τ = lim x* t x* t T + 2T T T () ( τ ) Process can be stationary and NOT ergodic 14

15 Ergodicity con t: Example 1: Assume RP x(t) which is a dc voltage waveform where the pdf for the voltage is given by U[0, 10]. 1) Plot several possible trials for the RP 2) Is the process wss? 3) Is the process ergodic in the mean? 15

16 Ergodicity cont : Example 2: (Telegraph Signal) Assume the stationary and ergodic RP x(t) takes values ±1 during every time interval T c with equal probability. Start of the first pulse after t=0, equally likely in the interval [0,T c ] 1) Plot a possible trial for the RP 2) Compute the correlation function 16

17 17

18 Periodic Random Process if x(t) is periodic, xt () = xt ( + T) Mean ( ) = ( ) = ( + ) = ( + ) m x t E x t E x t kt m x t kt Correlation/Covariance for stationary RP x (, ) = ( ) R t t R t t x 1 2 x 1 2 ( ) = x( + ) () = ( + ) R t R t kt C t C t kt x x 18

19 Example 3: x(t) = A exp (j(ωt + θ)), θ ~ U [0,2π] Compute R x (τ) & m x (t) 19

20 20

21 Example 4: y(t)=s(t)+w(t), where s(t)=a exp (j(ωt+ θ)), θ ~ U [0,2π], w(t) zero-mean white wss noise, w(t) & s(t) are independent. Compute R y (τ) and m y (t) 21

22 Frequency Domain Description of Stationary Processes Power spectral density (PSD) ( ) ( ) j2πτ f Sx f = FT Rx = Rx() e d R ( ) ( ) x = IFT Sx f τ τ τ 22

23 Example 5: Compute the PSD for the telegraph signal wave function investigated earlier. 23

24 24

25 25

26 Linear Transformations of RPs Process stationary random processes using LTI systems x( t ) h( t ) y( t) Mean: = m h() τ dτ x ( ) = ( )* ( ) = ( τ) ( τ) τ [ ] = ( ) ( ) = ( ) ( ) yt xt ht x ht d Output random processes properties y n k k x n k x k h n k k E{ y() t } = E h()( τ xt τ) dτ h() τ E[ xt ( τ) ] = dτ k 26

27 Input-output cross-correlation: ( τ ) { } ( ) *( ) * E xt () y*( t τ) = E xt h u x ( t udu ) { τ } ( ) Eytx t = E huxt u xt = * () *( ) ( ) ( ) ( τ) 27

28 Output correlation: { } ( ) * E y() t y*( t τ) = E h( u) x( t u) du y ( t τ) = * hue () xt ( u) y( t τ) du 28

29 Output covariance: same properties as for correlation y xy y y ( τ) = () xy( τ) Cy( τ) = h() t Cxy( t τ) dt * * ( τ) = ( τ) * x ( τ) Cxy ( τ) = h ( τ) * Cx ( τ) ( τ) = xy( τ) * ( τ) Cy( τ) = Cxy( τ) * h( τ) * * ( τ ) = ( τ) * ( τ) * ( τ) C ( τ ) = C ( τ) * h( τ) * h ( τ) R h t R t dt R h R R R h R R h h x y C τ = R τ m x ( ) ( ) 2 y y y 29

30 Example 6: Given a wss zero mean white noise RP x(t) with covariance C x (l)=σ 0 2 δ(l) Compute the mean, correlation function, covariance function, and PSD of the output RP y(t) to the LTI system with impulse response h(t) = e -at u(t), a>0. 30

31 31

32 Bandpass & complex envelope signal representations Communication and radar signals are usually concentrated in a narrow bandwidth around a center frequency Complex lowpass equivalent (i.e., complex envelope) signals derived from the bandpass signal usually simplify analysis Need to define lowpass/baseband and bandpass signals Baseband/lowpass signal Signal s(t) with frequency information is restricted for f B Bandpass signal Signal s(t) with frequency information restricted around ± f c 32

33 Bandpass & complex envelope signal representations Assume s( t) = a( t) cos(2 π f t + θ ( t)) = a( t c ){cos( θ ( t))cos(2 π f t) sin( θ ( t)) sin(2 π f t)} = s ( t) cos( 2 π f t) s ( t ) sin( 2 π f t ) I a( t) cos( θ ( t)) In-phase component (lowpass) Information signal c Q a( t)sin( θ ( t)) c Quadrature component (lowpass) Note: representation shows that 2 signals may be transmitted within the same bandwidth c c 33

34 I and Q signal contributions are orthogonal s ( t) = a( t) cos( θ ( t)) I s ( t) = a( t)sin( θ ( t)) Q Correlation between two signals? T 0 si()cos(2 t fct) sq()sin(2 t π fct) dt = [ π ] 34

35 Bandpass & complex envelope signal representations, cont Note: There is a relationship between bandpass signal s(t) and complex envelope u(t) st () = at ()cos(2 π ft+ θ ()) t = s () t cos(2 π ft) s ( t) sin(2 π ft) I { s () t + js () t )(cos(2 π ft) j sin(2 π f t) )} = Real ( + I Q c c Q c c c ut () Defined as the complex envelope (complex baseband, i.e., lowpass, equivalent signal) s ( t) = a( t)cos( θ ( t)) I s ( t) = a( t)sin( θ ( t)) Q { j 2 π f t u ( t ) } c s( t) = R e exp 35

36 Bandpass & complex envelope signal representations, cont Note: There is a relationship between S(f) and U(f), when s(t) is deterministic { j 2π f t u t } c s() t = Re ()exp 1 S( f) = FT ()exp 2 u t c + ( u()exp t c ) = j2π f t j2 π f t * 36

37 Bandpass & complex envelope signal representations, cont Note: There is a relationship between PSD expressions S s (f) and S u (f) when u(t) is random 37

38 Bandpass & complex envelope signal representations, cont Note: The energy of s(t) may be expressed as ε = s 2 () tdt= 38

39 Communication signal application Amplitude shift keying (ASK) On-Off Keying (OOK) signal Send either a sinusoid for 1 or nothing for 0 Assume T is duration of one bit symbol s () t = 0 0 s () t = Asin( ω t) t T 39

40 Recall: st () = at ()cos(2 π ft+ θ ()) t = s () t cos(2 π ft) s ( t) sin(2 π ft) I { s () t + js () t )(cos(2 π ft) j sin(2 π f t) )} = Real ( + s0() t = 0 s () t = Asin( ω t) 1 0 I Q ut () Compute: In phase component s I (t) Quadrature component s Q (t) Complex envelope u(t) c c Q c c c 40

41 Communication signal application Phase shift keying (PSK) signal Send either a sinusoid with one specific phase for 1 or with different phase for 0 Assume T is duration of one bit symbol s () t = Asin( ω t) 0 0 s () t = Asin( ω t) t T Phase changes 41

42 Recall: st () = at ()cos(2 π ft+ θ ()) t = s () t cos(2 π ft) s ( t) sin(2 π ft) I { s () t + js () t )(cos(2 π ft) j sin(2 π f t) )} = Real ( + s0() t = Asin( ω0t) s () t = Asin( ω t) 1 0 I Q ut () Compute: In phase component s I (t) Quadrature component s Q (t) Complex envelope u(t) c c Q c c c 42

43 Example 7: Consider the real signal s(t) defined as st ( ) = Acos(2 π ft c + π ), 0 t T 4 Compute 1) the complex lowpass equivalent (complex envelope) signal u(t), Fourier transform U(f), and S(f) 2) the signal energy 43

44 44

45 45

46 Linear systems & bandpass signals/complex envelopes LTI filter output of a bandpass signal may be computed using signal and impulse response complex envelope expressions much simpler than by using original bandpass expressions s(t) s 0 (t) u(t) v(t) h B (t) h(t) Bandpass Has a complex envelope expression st () = Re () { 2π jf } c ute t 1 S( f) = [ U( f fc ) 2 + ( )] * U f fc Can be modeled as a bandpass filter Has a complex envelope expression [ { 2π jf } c t h () t = 2Re h() t e B H ( f) = H( f f ) B c + ( )] * H f fc 46

47 Linear systems & bandpass signals/complex envelopes, cont s(t) s 0 (t) h B (t) u(t) v(t) h(t) st () = Re () { 2π jf } c ute t 1 S( f) = [ U( f fc ) 2 + ( )] 0 0 * U f fc s () t = st ()* h () t S ( f) B = S( f) H B ( f) [ 2π c { jf t } h () t = 2Re h() t e B H ( f) = H( f f ) B c + ( )] * H f fc 1 * = [ U( f fc) + U ( f fc)] [ H f f + H f f 2 * ( c) ( c)] 47

48 Linear systems & bandpass signals/complex envelopes, cont s(t) h B (t) s 0 (t) u(t) h(t) v(t) S 0 ( f) = S( f) H B ( f) 1 * = [ U( f f ) + U ( f f )][ [ * * = U ( f fc ) H ( f fc) + U ( f fc ) H ( f fc) 2 + U( f f ) H U ( ) * c c H( f fc) H ( f fc)] c * * ( f fc ) + f fc H( f fc )] 48

49 Linear systems & bandpass signals/complex envelopes, cont s(t) h B (t) s 0 (t) u(t) h(t) v(t) S S 0 0 ( f) ( f ) 1 * * = [ U( f fc) H( f fc) + U ( f fc) H ( f fc) 2 + U f f ( f f ) + U H ( f f )] * * ( c ) H c ( f fc ) 1 [ * * U( f fc) H( f fc ) U ( f fc) H ( f fc ) ] = + 2 c 49

50 Linear systems & bandpass signals/complex envelopes, cont S s(t) h B (t) s 0 (t) 1 = * * 0( f) [ U( f fc) H( f fc) U ( f fc) H ( f fc) ] * = [ V( f fc) V ( c) ] 2 u(t) h(t) + f v(t) f vt ( ) is the complex envelope expression of s ( t) 0 V( f) = U( f) H ( f v() t = ut ( ) h() t ) Complex envelope of filter response Complex envelope of output signal Complex envelope of input signal 50

51 Linear systems & bandpass signals/complex envelopes, cont s(t) h B (t) s 0 (t) u(t) h(t) v(t) V( f) = U( f) H ( f ) v() t = ut ( ) h() t Complex envelope of output signal = Complex envelope of input signal Complex envelope of filter response 51

52 Linear systems & bandpass signals/complex envelopes, cont vt) = ut () ht () = ( s() t + js ( t)) ( h() t + jh ()) t ( I Q I Q = 52

53 Conclusion: Complex baseband (complex envelope) representation of bandpass signals allows for accurate representation and analysis of signals independent of the signal carrier frequency. Leads to simpler evaluation of filter outputs and system performance analysis 53

54 Example 8: Consider the bandpass signal st ( ) = 2cos(2 π ft)cos(2 π ft) 0 sin(2 π f t)sin(2 π f t), f < f 0 c 0 c c Consider the bandpass filter with jf / f, f f f 2, 2 f f 2 f j, f f 2 f HI( f) =, H ( f) = 0, ow, 2 0, ow Q j f0 f f0 1) Compute in-phase and quadrature components of s(t) 2) Compute the complex envelope of s(t) 3) Compute the complex envelope of the filter output 4) Compute the bandpass filter output 54

55 55

56 56

57 57

58 Bandpass noise process Defined as a process which is 1) centered at a non zero frequency 2) does not extend to zero frequency (i.e., has no DC term) S n (f) f 58

59 Bandpass noise process, cont Noise process may be represented as: nt () = a()cos(2 t π ft+ θ ()) t = n ( t) cos(2 π ft) n ( t) sin(2 π ft) { n () t + jn ( t) π ft j π ft } = Real ( )(cos(2 ) + sin(2 )) = Real n c n I I Q { j2 f ( ) } ct zte π c zt ( ) complex envelope Q c c a ( t) cos( θ ( t)) n a ( t)sin( θ ( t)) n c 59

60 Bandpass noise process, cont Correlation properties between bandpass noise components ni ( t) = an( t) cos( θ ( t)) n ( t) = a ( t)sin( θ ( t)) Q R ( ) n τ = n Trig identities cos acos b= (1/ 2)(cos( a+ b) + cos( a b)), sin asin b= (1/ 2)(cos( a+ b) cos( a b)) sin acos b= (1/ 2)(sin( a+ b) + sin( a b)), cos asin b= (1/ 2)(sin( a+ b) sin( a b)) 60

61 61

62 Bandpass noise process, cont R = E z t z t = R + jr n ( t) + jn ( t) * z( τ ) ( ) ( τ) 2 N ( τ) 2 N N ( τ) I S ( f) = 2 S ( f) + 2 jr ( f) z N N N Q I Q I I Q I 1 R ( ) Re [ ( )exp( 2 )] n τ = Rz τ j π fcτ 2 S f = S f f + S f f * n( ) (1/4) z( c) (1/4) z( c) Conclusion: PSD of a random bandpass process can be derived from the PSD of the complex envelope and vice-versa 62

63 63

64 Example: bandpass white noise process n(t) with PSD shown below Compute the PSD and the correlation of the complex envelope S n (f) N 0 /2 B -f c f c 64

65 65

66 66

67 Envelope statistics and their use in signal detection Commonly used in non-coherent signal detection where unknown signal parameters are treated as random variables Look at - noise only - signal + noise Noise-only case: Bandpass noise n(t) is defined as: nt () = Re () { j2π f } ct wte with wt () = w() t + jw () t I Q w I (t) & w Q (t) are zero-mean, statistically independent Gaussian processes, with variance σ 2 67

68 Noise-only case cont nt () = Re () { j2π f } ct wte with wt () = w() t + jw () t I Noise envelope is defined as: ρ = w( t) = Q Pdf of the envelope is 68

69 Noise-only case cont nt () = Re () { j2π f } ct wte with wt () = w() t + jw () t Noise phase is defined as I Q θ n w ( ) 1 Q t ( t ) = tan w I ( t) Pdf of the noise phase: 69

70 Signal + Noise case: Assume the received signal { j2π f } ct y() t = Re v() t e + n() t { j2π f } { 2 } ct j π fct v t e w t e = Re () + Re () { j2π f } ct vt wt e = Re ( ( ) + ( )) { j2π f } ct v t wi t jwq t e = Re ( ( ) + ( ) + ( )) Complex envelope ρ for y(t) is given by (Assume v(t)=a) 70

71 Turns out the pdf for the envelope is Ricean and given by: f ρ ρ A ρ I 2 0 e 2 ( ρ) = σ σ 0, otherwise ( ρ + A )/2σ, ρ 0 71

72 Example Consider the radar application where we want to decide whether a signal x(t) is present or not using envelope statistics only Noise envelope is given by: Signal +noise envelope is given by: 72

73 Define the probability of detection P D = Define the probability of false alarm P fa = More details later on. 73

74 2 A γ = 2 σ [Sch] 74

75 Monte Carlo performance evaluation Deals with computer evaluation of a probability Useful in cases where one cannot determine analytically or numerically expressions of the form P( x K), P( x K), P( K1 x K2) Can be found in detection problems where we may wish to evaluate probability that a given statistic exceeds/falls below a threshold 75

76 Expressions involve computations of an integral expression I b = a h( x) dx Expressions can be rewritten as I = b a hx ( ) f( x) f( x) dx, assume f( x) : (1) is a valid pdf, N 1 I = E [ ( )] x wx = w( xi ) N (2) 0 over integration range i= 1 x i with pdf f(x) 76

77 Example: Compute E x where x exponential RV with λ=1 E x = xexp( x) dx 0 N 1 = xi, N i= 1 where x has exponential pdf i 77

78 MATLAB Implementation % generates data x with exp. pdf u=rand(1000,1); x=-log(u); h= x.^(0.5); %Value obtained using the Monte Carlo method mean(h) % Value obtained using numerical integration F (x) sqrt(x).*exp(-x); quad(f,0,10) % evaluate integration from 0 to

79 Check exponential data fit for x 79

80 Example: Evaluate Px ( S), where x N(0,1) Px ( S) = f( xdx ) S + = hx ( ) f( x) dx, with 1, x S h( x) = 0,ow 80

81 + Px ( S) = hx ( ) f( xdx ), with 1 = N N i= 1 1, x S h( x) = 0,ow # xi S hx ( i ) = N x i ~ N(0,1) Example: Px ( 2) = Q(2) = MC with N=10, MC with N=100, Problem with MC method: Need high number of samples!! 81

82 MC is an estimation process, estimated value will have a certain variance. MC performance evaluation works but. large number of trials needed to insure accurate estimates of P D and P FA estimates in radar/coms applications Evaluating a small P FA requires ~ 100/P FA samples in radar applications P FA =10-5 not unusual Above discussion emphasizes need for reduction of the sampling size and insure probability estimates are still accurate. Reduction may be obtained via importance sampling. Reduction obtained by reframing the problem into one where events of interests are not rare, so fewer evaluations are needed, i.e., use samples where the value of the function to integrate is NOT small. 82

83 83

84 References 92

85 [1] W. Chan, Foundation Course on Probability, Random Variable and Random Processes 93