Transmission Scheduling for Remote State Estimation and Control With an Energy Harvesting Sensor

Transcription

1 Transmission Scheduling for Remote State Estimation and Control With an Energy Harvesting Sensor Daniel E. Quevedo Chair for Automatic Control Institute of Electrical Engineering (EIM-E) Paderborn University, Germany Indian Institute of Technology Bombay, March 2018 Daniel Quevedo Scheduling with Energy Harvesting IITB, March / 38

2 Introduction Wireless Sensor Technologies Due to advances in micro-electro-mechanical systems technology, small and low cost sensors with sensing, computation and wireless communication capabilities have become widely available Key components in wireless sensor networks, networked control systems, cyber-physical systems, Internet of Things, etc. Daniel Quevedo Scheduling with Energy Harvesting IITB, March / 38

3 Introduction Energy Management Communication between sensors often over wireless networks Wireless channels are usually randomly time-varying Transmitted signals can be attenuated, distorted, delayed, or lost Transmission reliability can be improved by increasing transmission energy, but this reduces battery life energy management !"!#$%&$ '"(($%&$ -40 RSSI (dbm) Measurements taken at Holmen s Paper Mill in Iggesund, Sweden (A. Ahlén) -80 Node34.log - Pol 1: mean= , std= Node34.log - Pol 2: mean= , std= Time (h) Daniel Quevedo (dquevedo@ieee.org) Scheduling with Energy Harvesting IITB, March / 38

4 Introduction Energy Harvesting Sensors often run on batteries which are not easily replaced Sensors may need to operate for years without battery change Energy harvesting sensors recharge their batteries by collecting energy from the environment e.g. solar, thermal, mechanical vibrations Potential for self-sustaining systems Daniel Quevedo Scheduling with Energy Harvesting IITB, March / 38

5 Introduction Energy Harvesting Energy Management E k H k delay B k Battery level evolves as B k+1 = min{b k E k + H k+1, B max }, where B k is battery level at time k, E k is energy used at time k, H k+1 is energy harvested between times k and k + 1, B max is maximum battery capacity Key Issue: How much energy E k should be used at time k? Should we use more energy now, or save energy for later? Also try to avoid battery level saturating Daniel Quevedo (dquevedo@ieee.org) Scheduling with Energy Harvesting IITB, March / 38

6 Introduction Energy Harvesting Energy Management E k H k delay B k Energy harvesting has been studied extensively in wireless communications, e.g. maximizing throughput or minimizing transmission delay Has also gained recent attention in state estimation and control, e.g. minimizing estimation error covariance 4 5 or minimizing LQG control cost 6 1 Sharma, Mukherji, Joseph, Gupta, IEEE Trans. Wireless Commun., Ozel, Tutuncuoglu, Yang, Ulukus, Yener, IEEE J. Sel. Areas Commun., Ho, Zhang, IEEE Trans. Signal Process., Nourian, Leong, Dey, IEEE Trans. Automat. Control, Li, Zhang, Quevedo, Lau, Dey, Shi, IEEE Trans. Automat. Control, Knorn, Dey, Automatica, 2017 Daniel Quevedo (dquevedo@ieee.org) Scheduling with Energy Harvesting IITB, March / 38

7 Introduction Event Triggered Estimation and Control Dynamical System Sensor Estimator Dynamical System Sensor Controller Traditionally in estimation and control, measurements and control signals are transmitted periodically Event Triggered View - Transmit only when certain events occur, e.g. if system performance has deteriorated by a large amount Event triggering can achieve energy savings Event triggered estimation and control has been studied by Åström, Başar, Dimarogonas, Heemels, Hespanha, Hirche, Johansson, Lemmon, Shi, Tabuada, Trimpe, Wu,... Daniel Quevedo (dquevedo@ieee.org) Scheduling with Energy Harvesting IITB, March / 38

8 Introduction Event Triggered Estimation and Control Different transmission strategies have been studied Threshold policies often proposed 4 performance loss periodic event triggered time Daniel Quevedo (dquevedo@ieee.org) Scheduling with Energy Harvesting IITB, March / 38

9 Introduction Key Questions What are good transmission policies for remote state estimation using wireless sensors with energy harvesting capabilities? What is the role of event triggered methods? Daniel Quevedo Scheduling with Energy Harvesting IITB, March / 38

10 Remote State Estimation with an Energy Harvesting Sensor Outline 1 Introduction 2 Remote State Estimation with an Energy Harvesting Sensor 3 Optimal Transmission Scheduling 4 Transmission Scheduling for Control 5 Simulation Studies 6 Conclusion Daniel Quevedo (dquevedo@ieee.org) Scheduling with Energy Harvesting IITB, March / 38

11 Remote State Estimation with an Energy Harvesting Sensor Remote State Estimation Energy Harvester Hk ~ Process x k y k Sensor Battery B k Local KF ^x s s k, P k ν k Packet drops γ k Remote Estimator P k ^x k k Feedback, γ k Process x k+1 = Ax k + w k, w k N(0, Q) Sensor measurement y k = Cx k + v k, v k N(0, R) Sensor runs a local Kalman filter to compute (posterior) local estimates ˆx s k Local estimates transmitted over i.i.d. packet dropping link Daniel Quevedo (dquevedo@ieee.org) Scheduling with Energy Harvesting IITB, March / 38

12 Remote State Estimation with an Energy Harvesting Sensor Local Sensor Computations Energy Harvester Hk ~ Process x k y k Sensor Battery B k Local KF ^x s s k, P k ν k Packet drops γ k Remote Estimator P k ^x k k Feedback, γ k (Local) State estimates ˆx k k 1 s E[x k y 0,..., y k 1 ], ˆx k s E[x k y 0,..., y k ] (Local) Estimation error covariances Pk k 1 s E[(x k ˆx k k 1 s )(x k ˆx k k 1 s )T y 0,..., y k 1 ] Pk s E[(x k ˆx k k s )(x k ˆx k k s )T y 0,..., y k ] Daniel Quevedo (dquevedo@ieee.org) Scheduling with Energy Harvesting IITB, March / 38

13 Remote State Estimation with an Energy Harvesting Sensor Local Sensor Computations State estimates and error covariances are computed using the Kalman filter ˆx s k+1 k = Aˆx s k ˆx s k = ˆx s k k 1 +K k(y k Cˆx s k k 1 ) P s k+1 k =APs k AT + Q P s k =Ps k k 1 Ps k k 1 CT (CP s k k 1 CT + R) 1 CP s k k 1 where K k = P s k k 1 CT (CP s k k 1 CT + R) 1 Under standard assumptions 7, P s k P as k 7 (A, C) observable and (A, Q 1/2 ) controllable Daniel Quevedo (dquevedo@ieee.org) Scheduling with Energy Harvesting IITB, March / 38

14 Remote State Estimation with an Energy Harvesting Sensor Sensor Transmissions Energy Harvester Hk ~ Process x k y k Sensor Battery B k Local KF ^x s s k, P k ν k Packet drops γ k Remote Estimator P k ^x k k Feedback, γ k Transmission decisions: Sensor transmits local state estimate to remote estimator if ν k = 1, doesn t transmit if ν k = 0 Transmitting local state estimates gives better performance over packet dropping link than transmitting measurements a, as local estimate captures all relevant information when received a Xu, Hespanha, Proc. CDC, 2005 Packet drop process i.i.d. Bernoulli with γ k = 1 if transmission successful, γ k = 0 otherwise Daniel Quevedo (dquevedo@ieee.org) Scheduling with Energy Harvesting IITB, March / 38

15 Remote State Estimation with an Energy Harvesting Sensor Remote Estimator In the presence of dropouts, the information available to the remote estimator at time k is I k {ν 0,..., ν k, ν 0 γ 0,..., ν k γ k, ν 0 γ 0ˆx s 0,..., ν kγ k ˆx s k } Define remote state estimates and estimation error covariances ˆx k E[x k I k ], P k E[(x k ˆx k )(x k ˆx k ) T I k ]. Remote estimator has the form { Aˆxk 1, ν ˆx k = k γ k = 0 ˆx k s, ν k γ k = 1 { APk 1 A P k = T + Q, ν k γ k = 0 P, ν k γ k = 1 When transmission received, update remote estimate as local estimate. When transmission is not received, use one step ahead prediction Daniel Quevedo (dquevedo@ieee.org) Scheduling with Energy Harvesting IITB, March / 38

16 Remote State Estimation with an Energy Harvesting Sensor Energy Management Energy Management E k H k delay B k Transmission decisions: Sensor transmits local state estimate if ν k = 1, doesn t transmit if ν k = 0 Each transmission uses energy E Battery level evolves as B k+1 = min{b k E k + H k+1, B max } = min{b k ν k E + H k+1, B max } Harvested energy process {H k } is Markov Daniel Quevedo (dquevedo@ieee.org) Scheduling with Energy Harvesting IITB, March / 38

17 Remote State Estimation with an Energy Harvesting Sensor Energy Management Harvested energy process {H k } is Markov, to model time correlations in amount of energy harvested Example 1. For solar energy, very little/no energy can be harvested at night Example 2. Suppose the weather X n on day n is either sunny (state 1) or rainy (state 2), and is modelled as a Markov chain with transition probabilities P = [ with the (i, j)-th entry of P representing P(X n+1 = j X n = i) ], Daniel Quevedo (dquevedo@ieee.org) Scheduling with Energy Harvesting IITB, March / 38

18 Optimal Transmission Scheduling Outline 1 Introduction 2 Remote State Estimation with an Energy Harvesting Sensor 3 Optimal Transmission Scheduling 4 Transmission Scheduling for Control 5 Simulation Studies 6 Conclusion Daniel Quevedo (dquevedo@ieee.org) Scheduling with Energy Harvesting IITB, March / 38

19 Optimal Transmission Scheduling Transmission Scheduling Energy Harvester Hk ~ Process x k y k Sensor Battery B k Local KF ^x s s k, P k ν k Packet drops γ k Remote Estimator P k ^x k k Feedback, γ k Battery level evolves as B k+1 = min{b k ν k E + H k+1, B max } Key Question: Should we transmit now, or save energy for later? Daniel Quevedo (dquevedo@ieee.org) Scheduling with Energy Harvesting IITB, March / 38

20 Optimal Transmission Scheduling Optimal Transmission Scheduling Energy Harvester Hk ~ Process x k y k Sensor Battery B k Local KF ^x s s k, P k ν k Packet drops γ k Remote Estimator P k ^x k k Feedback, γ k Determine the transmission schedule that minimizes the expected error covariance at remote estimator K min E[trP k ] {ν 1,...,ν K } k=1 subject to energy harvesting constraints ν k E B k, k, with battery dynamics B k+1 = min{b k ν k E + H k+1, B max } Decision variables ν k depend on (P k 1, H k, B k ) Daniel Quevedo (dquevedo@ieee.org) Scheduling with Energy Harvesting IITB, March / 38

21 Optimal Transmission Scheduling Optimal Transmission Scheduling subject to min {ν 1,...,ν K } k=1 K E[trP k ] ν k E B k, k, B k+1 = min{b k ν k E + H k+1, B max }, where decision variables ν k depend on (P k 1, H k, B k ) Problem can be solved numerically using dynamic programming However dynamic programming doesn t provide much insight into the form of the optimal solution We will analyze the problem further to derive structural results This leads to insights and computational savings Daniel Quevedo (dquevedo@ieee.org) Scheduling with Energy Harvesting IITB, March / 38

22 Optimal Transmission Scheduling Structural Properties of Optimal Schedule Theorem (i) For fixed B k and H k, the optimal ν k is a threshold policy on P k 1 of the form: ν k (P k 1, B k, H k ) = { 0, Pk 1 P k 1, otherwise where the threshold P k depends on k, B k and H k. For large P k 1, it is better to transmit than not transmit Idea of proof: Show that the difference in expected cost between transmitting and not transmitting is monotonic in P k 1 (when B k and H k are fixed) Use an induction argument to prove this Daniel Quevedo (dquevedo@ieee.org) Scheduling with Energy Harvesting IITB, March / 38

23 Optimal Transmission Scheduling Structural Properties of Optimal Schedule Theorem (ii) For fixed P k 1 and H k, the optimal ν k is a threshold policy on B k of the form: ν k (P k 1, B k, H k ) = { 0, Bk B k 1, otherwise where the threshold B k depends on k, P k 1 and H k. More likely to transmit when battery level is high Idea of proof: Show that the value functions of dynamic programming algorithm, when regarded as a function of B k and ν k, are submodular in (B k, ν k ). This then implies 8 that ν k is non-decreasing with P k 1. 8 Topkis, Operations Research, 1978 Daniel Quevedo (dquevedo@ieee.org) Scheduling with Energy Harvesting IITB, March / 38

24 Optimal Transmission Scheduling Structural Properties of Optimal Schedule Optimal policies are of threshold-type, event based simplifies real-time implementation can also provide computational savings in numerical solution 5 ν k * =1 4 3 B k n Pk 1 = f n ( P), where f ( P) A T PA + Q Daniel Quevedo (dquevedo@ieee.org) Scheduling with Energy Harvesting IITB, March / 38

25 Transmission Scheduling for Control Outline 1 Introduction 2 Remote State Estimation with an Energy Harvesting Sensor 3 Optimal Transmission Scheduling 4 Transmission Scheduling for Control 5 Simulation Studies 6 Conclusion Daniel Quevedo (dquevedo@ieee.org) Scheduling with Energy Harvesting IITB, March / 38

26 Transmission Scheduling for Control Transmission Scheduling for Control Energy Harvester Hk ~ Process x k y k Sensor Battery B k Local KF ^x s s k, P k ν k Packet drops γ k Controller Feedback, γ k u k Can also study the control problem System model similar to estimation problem, except process is now x k+1 = Ax k + Bu k + w k Daniel Quevedo (dquevedo@ieee.org) Scheduling with Energy Harvesting IITB, March / 38

27 Transmission Scheduling for Control Transmission Scheduling for Control Equations for local Kalman filter are now ˆx k+1 k s = Aˆx k s + Bu k ˆx k s = ˆx k k 1 s +K k(y k Cˆx k k 1 s ) Pk+1 k s =APs k AT + Q Pk s =Ps k k 1 Ps k k 1 CT (CPk k 1 s CT + R) 1 CPk k 1 s where K k = P s k k 1 CT (CP s k k 1 CT + R) 1 Note that u k can be reconstructed at sensor from γ k, since ˆx k can be reconstructed from γ k, and optimal u k will be a linear function of ˆx k (see later) Daniel Quevedo (dquevedo@ieee.org) Scheduling with Energy Harvesting IITB, March / 38

28 Transmission Scheduling for Control Transmission Scheduling for Control Want to solve the following problem [ K ] min E (xk T Wx k + uk T Uu k) + xk T +1 Wx K +1 {ν 1,...,ν K, u 1,...,u K } k=1 subject to energy harvesting constraints ν k E B k, k Is a joint control and scheduling problem For transmission decisions ν k dependent on (P k 1, B k, H k ), problem can be shown to be separable, and is equivalent to [ [ min min E K ]] (xk T Wx k +uk T Uu k) +xk T +1 Wx K +1 {ν 1,...,ν K } {u 1,...,u K } k=1 Daniel Quevedo (dquevedo@ieee.org) Scheduling with Energy Harvesting IITB, March / 38

29 Transmission Scheduling for Control [ [ min min E K ]] (xk T Wx k +uk T Uu k) +xk T +1 Wx K +1 {ν 1,...,ν K } {u 1,...,u K } k=1 Inner optimization is LQG-type problem with solution u k = (BT S k+1 B + U) 1 B T S k+1 Aˆx k, S K +1 = W, S k = A T S k+1 A +W A T S k+1 B(B T S k+1 B+U) 1 B T S k+1 A Optimal cost is tr(s 1 P 1 ) + K tr(s k+1 Q) + k=1 K tr ( (A T S k+1 A + W S k )E[P k ] ) k=1 Daniel Quevedo (dquevedo@ieee.org) Scheduling with Energy Harvesting IITB, March / 38

30 Transmission Scheduling for Control [ [ min min E K ]] (xk T Wx k +uk T Uu k) +xk T +1 Wx K +1 {ν 1,...,ν K } {u 1,...,u K } k=1 Substituting optimal cost of inner optimization tr(s 1 P 1 ) + K tr(s k+1 Q) + k=1 K tr ( (A T S k+1 A + W S k )E[P k ] ), k=1 the following transmission scheduling problem remains: [ K min tr ( (A T S k+1 A+W S k )E[P k ] )], {ν 1,...,ν K } k=1 subject to energy harvesting constraint ν k E B k, k Similar to transmission scheduling problem for remote estimation discussed before Daniel Quevedo (dquevedo@ieee.org) Scheduling with Energy Harvesting IITB, March / 38

31 Transmission Scheduling for Control Theorem In the transmission scheduling problem for control: (i) For fixed B k and H k, the optimal ν k is a threshold policy on P k 1 of the form: ν k (P k 1, B k, H k ) = { 0, Pk 1 P k 1, otherwise where the threshold P k depends on k, B k and H k. (ii) For fixed P k 1 and H k, the optimal ν k is a threshold policy on B k of the form: ν k (P k 1, B k, H k ) = { 0, Bk B k 1, otherwise where the threshold B k depends on k, P k 1 and H k. Daniel Quevedo (dquevedo@ieee.org) Scheduling with Energy Harvesting IITB, March / 38

32 Simulation Studies Outline 1 Introduction 2 Remote State Estimation with an Energy Harvesting Sensor 3 Optimal Transmission Scheduling 4 Transmission Scheduling for Control 5 Simulation Studies 6 Conclusion Daniel Quevedo (dquevedo@ieee.org) Scheduling with Energy Harvesting IITB, March / 38

33 Simulation Studies Simulation Studies Parameters A = [ ], C = [ 1 1 ], Q = I, R = 1 Packet reception probability λ = 0.7, transmission energy E = 2 Harvested energy process {H k } is Markov with state space {0, 1, 2} and transition probability matrix 9 Horizon K = 10. P = Energy is scarce in this example Daniel Quevedo (dquevedo@ieee.org) Scheduling with Energy Harvesting IITB, March / 38

34 Simulation Studies Simulation Studies Estimation problem. Comparison with greedy method which always transmits provided there is enough energy in battery Always transmit Optimal solution tr E[P k ] B max The optimal solution outperforms greedy method, without using more energy Daniel Quevedo (dquevedo@ieee.org) Scheduling with Energy Harvesting IITB, March / 38

35 Simulation Studies Simulation Studies Control problem. Same parameters as estimation problem, plus B = [ 1 2 ] T, W = I, U = 1. Comparison with greedy method which always transmits provided there is enough energy, together with optimal LQG controller 155 Always transmit Optimal solution E[control cost] B max Daniel Quevedo (dquevedo@ieee.org) Scheduling with Energy Harvesting IITB, March / 38

36 Conclusion Conclusion Energy harvesting introduces new design issues We have studied transmission scheduling problems for remote state estimation and control with an energy harvesting sensor We showed that threshold policies are optimal Daniel Quevedo Scheduling with Energy Harvesting IITB, March / 38

37 Conclusion Open Questions Derive structural results for Power control instead of transmission scheduling Multiple sensors Wireless power transfer and energy sharing Transfer of electrical energy without wires using electro-magnetic (EM) fields and EM radiation Both near field (e.g. wireless phone chargers) and far field (over km distances) techniques currently under active investigation Energy harvesting from ambient EM waves also being investigated Daniel Quevedo Scheduling with Energy Harvesting IITB, March / 38

38 Conclusion Further Reading The current presentation is based on: Leong, Dey, Quevedo, Transmission Scheduling for Remote State Estimation and Control With an Energy Harvesting Sensor, to be published in Automatica Other related work: Li, Zhang, Quevedo, Lau, Dey, Shi, Power Control of an Energy Harvesting Sensor for Remote State Estimation, IEEE Transactions on Automatic Control, January 2017 Leong, Quevedo, Dey, Optimal Control of Energy Resources for State Estimation Over Wireless Channels, Springer, 2018 SPRINGER BRIEFS IN ELECTRICAL AND COMPUTER ENGINEERING CONTROL, AUTOMATION AND ROBOTICS Alex S. Leong Daniel E. Quevedo Subhrakanti Dey Optimal Control of Energy Resources for State Estimation Over Wireless Channels 123 Daniel Quevedo Scheduling with Energy Harvesting IITB, March / 38