Time-average constraints in stochastic Model Predictive Control

Time-average constraints in stochastic Model Predictive Control James Fleming Mark Cannon ACC, May 2017 James Fleming, Mark Cannon Time-average constraints in stochastic MPC ACC, May 2017 1 / 24

Outline 1. Motivation 2. Problem statement 3. MPC for parametric uncertainty 4. Handling time-average constraints 5. Numerical example James Fleming, Mark Cannon Time-average constraints in stochastic MPC ACC, May 2017 2 / 24

Motivation For linear dynamics + additive disturbances parameter uncertainty state/control constraints: Enforcing constraints for all uncertainty realisations can be conservative Some applications allow non-zero constraint violation rate e.g. engine knock control wind turbine control building climate control hence consider constraint softening methods: Chance constraints Schwarm 99, Oldewurtel 08, Cannon 11 Time-average constraints Oldewurtel 13, Korda 14 James Fleming, Mark Cannon Time-average constraints in stochastic MPC ACC, May 2017 3 / 24

Motivation Chance constraints: computationally hard & possibly non-convex if invoked explicitly Prékopa 95 hold with confidence < 1 if imposed approximately Campi 08, Calafiore 10 Time-average constraints constraints tightened according to observed violation frequency asymptotic limit on average violation rate Oldewurtel 13, Korda 14 This paper s approach: limit on number of violations on any time-interval of a given length James Fleming, Mark Cannon Time-average constraints in stochastic MPC ACC, May 2017 4 / 24

Problem formulation Control objective: regulate state x t to 0 subject to F x t 1 sufficiently frequently System model: x t+1 = A(ω t )x t + B(ω t )u t + w(ω t ) ω t : unknown parameter with polytopic bound, hence x t+1 Conv { A (i) x t + B (i) u t + w (i)}, i = 1,..., p Stabilizability assumption: K, V, g exist such that {x R n : V x g} is: invariant for x t+1 Conv { (A (i) + B (i) K)x t + w (i)} a subset of {x : F x 1} James Fleming, Mark Cannon Time-average constraints in stochastic MPC ACC, May 2017 5 / 24

Problem formulation Consider two types of time-average constraint Type 1: average number of violations in interval T must not exceed ε for all t and given T, t+t 1 1 require M k ε T k=t { 0, F x t 1 where M t = 1, F x t 1 implies 1 t 1 M k ε for all t T t k=0 analogous to chance constraint: P { F x t 1 } ɛ James Fleming, Mark Cannon Time-average constraints in stochastic MPC ACC, May 2017 6 / 24

Problem formulation Consider two types of time-average constraint Type 2: average loss over interval T must not exceed l for all t and given T, require t+t 1 1 l(f x k 1) T l k=t where l : R q R convex loss function allows higher penalties on larger violations analogous to expected value constraint: E { l(f x t 1) } l James Fleming, Mark Cannon Time-average constraints in stochastic MPC ACC, May 2017 6 / 24

Dual Mode Predictions Predicted control input at time t: u k t = Kx k t + c k t, k = 0, 1,..., { decision variable k < N (mode 1) c k t = 0 k N (mode 2) Predicted state trajectories x k+1 t Conv{Φ (i) x k t + B (i) c k t + w (i), i = 1... p} Φ (i) = A (i) + B (i) K state space e.g. N = 4 (1 realisation) x 0 t x x 1 t x2 t x x 3 t x x x 4 t x terminal set 0 1 2 3 4 prediction time James Fleming, Mark Cannon Time-average constraints in stochastic MPC ACC, May 2017 8 / 24

Tube MPC p uncertainty vertices = x k t belongs to a set with p k vertices Bound x k t using fixed-complexity polytopes T k t to avoid exponential growth: x 0 t T 0 t, x k t T k t = x k+1 t T k+1 T T N t : robustly invariant under u = Kx state space T 0 t T 1 t e.g. N = 4 T 2 t T 3 t T 4 t terminal set 0 1 2 3 4 prediction time James Fleming, Mark Cannon Time-average constraints in stochastic MPC ACC, May 2017 9 / 24

Tube MPC Define T k t = {x : V x α k t }, k = 0, 1,... fixed face normals, flexibility due to variable α k t set α k t = g for all k N = T N t robustly invariant optimize c t = (c 0 t,..., c N 1 t ) and α t = (α 0 t,..., α N 1 t ) at each time t James Fleming, Mark Cannon Time-average constraints in stochastic MPC ACC, May 2017 10 / 24

Tube constraints Inclusion constraints: { T k t x : Φ (i) x + B (i) c k t + w (i) T k+1 t, } i = 1,..., p T k t T k+1 t Feasibility constraints: T k t { x : F x 1 } T k t Impose these conditions using F x 1 {x : V 1 x g 1 } {x : V 2 x g 2 } iff H 0 s.t. HV 1 = V 2, Hg 1 g 2 James Fleming, Mark Cannon Time-average constraints in stochastic MPC ACC, May 2017 11 / 24

Tube constraints Proposition T k t { x : Φ (i) x + B (i) c k t + w (i) T k+1 t, i = 1,..., p } if T k t { x : F x 1 } if Γ (i) α k t + V B (i) c k t + V w (i) α k+1 t Λα k t 1 for some Γ (i) 0 satisfying Γ (i) V = V Φ (i) for some Λ 0 satisfying ΛV = F Determine Γ (i), Λ offline by minimizing the row-sums: Γ (i) j = arg min y {1 y : y V = V j Φ (i), y 0} Λ j = arg min y {1 y : y V = F j, y 0} then Γ (i), Λ are sparse with at most n non-zero elements in each row James Fleming, Mark Cannon Time-average constraints in stochastic MPC ACC, May 2017 12 / 24

Cost function Penalise deviation from a target set Ω = {x : W x h}, Ω robustly invariant under u = Kx Quadratic cost: J t = min s 0 t Ω k=0 ( ) E x k t s k t 2 Q + u k t Ks k t 2 R where s k+1 t = Φ(ω k t )s k t + w(ω k t ), k = 0, 1,... Proposition Let x 0 t = x t, then J t = min s t Ω where P E { Ψ (ω)p Ψ(ω) } = Q, with [ ] A(ω) + B(ω)K B(ω)E Ψ(ω) = 0 S and Ec t = c 0 t, Sc t = [c 1 t c N 1 t 0] [ ] xt s t 2 c t P [ ] Q + K Q RK K RE = E RK E RE James Fleming, Mark Cannon Time-average constraints in stochastic MPC ACC, May 2017 13 / 24

Time-average constraints Relaxed constraints: F x k t 1 + r b k t, where r : determines maximum constraint violation b k t : 0 or 1 depending on M t T +1,..., M t Enforce the time-average violation limit via constraints, for k = 1,..., N 1: v k t ɛ where v k t = 1 t M j + 1 k b k t T T j=t (T 1)+k j=1 M t T +2 M t 1 M t b 1 t v1 t M t T +3 M t b 1 t b 2 t v2 t. M t T +N M t b 1 t b N 1 t vn 1 t past future James Fleming, Mark Cannon Time-average constraints in stochastic MPC ACC, May 2017 15 / 24

Time-average constraints Relaxed constraints: F x k t 1 + r b k t, where r : determines maximum constraint violation b k t : 0 or 1 depending on M t T +1,..., M t Enforce the time-average violation limit via constraints, for k = 1,..., N 1: v k t ɛ where v k t = 1 t M j + 1 k b k t T T j=t (T 1)+k j=1 hence v 0 t = v 1 t 1 1 T b 1 t 1 + 1 T M t v k t = v k 1 t 1 T M t T +k + 1 T b k t, for all k 1 so if v k 1 t 1 T M t T +k + 1 T ɛ, otherwise, set b k t = 0 then b k t = 1 is allowed James Fleming, Mark Cannon Time-average constraints in stochastic MPC ACC, May 2017 15 / 24

MPC Algorithm Initialize: M T = = M 1 = 0. At each time t = 0, 1,...: 1. determine M t and v 0 t given x t 2. for each k = 1,..., N 1: { 1 if v k 1 t 1 (a). b k t := T M t T +k + 1 T ε 0 otherwise (b). v k t := v k 1 t 1 T M t T +k + 1 T b k t 3. solve the QP: (c t, α t, s t ) := argmin c t,α t,s t 4. u t := Kx t + c 0 t s.t. [ ] xt s t 2 c t P V x t α 0 t, W s t h Λα k t 1 + b k t r Γ (i) α k t + V B (i) c k t + V w (i) α k+1 t for k = 0,..., N 1, α t+n t = g James Fleming, Mark Cannon Time-average constraints in stochastic MPC ACC, May 2017 16 / 24

Feasibility and constraint satisfaction Lemma If step 3 is feasible at time t, then for k < N 1 we have v k t+1 v k+1 t and b k+1 t = 1 = b k t+1 = 1 M t T +2 M t 1 M t b 1 t v1 t M t T +3 M t b 1 t b 2 t v2 t. v N 1 t Mt T +N M t b 1 t b N 1 t past future James Fleming, Mark Cannon Time-average constraints in stochastic MPC ACC, May 2017 17 / 24

Feasibility and constraint satisfaction Lemma If step 3 is feasible at time t, then for k < N 1 we have v k t+1 v k+1 t and b k+1 t = 1 = b k t+1 = 1 M t T +2 M t 1 M t M t+1 v0 t+1 M t T +3 M t M t+1 b 1 t+1 v1 t+1. v N 2 t+1 Mt T +N M t M t+1 b N 2 t+1 past future v N 1 t+1 Mt T +N+1 M t+1 b N 2 t+1 b N 1 t+1 James Fleming, Mark Cannon Time-average constraints in stochastic MPC ACC, May 2017 17 / 24

Feasibility and constraint satisfaction Lemma If step 3 is feasible at time t, then for k < N 1 we have v k t+1 v k+1 t and b k+1 t = 1 = b k t+1 = 1 for all k < N 1 we have v k t+1 v k+1 t = 1 T (M t+1 b 1 t ) + 1 T and M t+1 = 1 = k (b j t+1 b j+1 t ) j=1 b 1 t = 1 since step 3 is feasible at t = v k t+1 = v k+1 t in step 2 M t+1 = 0 = b 1 t = 1 or 0 = v k t+1 = v k+1 t 1 T or v k+1 t in step 2 therefore step 2 cannot tighten constraints at t + 1 for k < N 1 hence step 3 feasible at t = step 3 feasible at t + 1 James Fleming, Mark Cannon Time-average constraints in stochastic MPC ACC, May 2017 17 / 24

Feasibility and constraint satisfaction Theorem If step 3 is feasible at t = 0 then for all t 0 we have t+t 1 1 M k ɛ T k=t Step 3 enforces the constraint F x 1 t 1 + rb 1 t, so that M t+1 b 1 t Step 2 chooses b 1 t so that v 1 t ɛ Hence 1 T t+1 k=t T 2 M k 1 T t k=t T 2 M k + 1 T b 1 t = v 1 t ɛ James Fleming, Mark Cannon Time-average constraints in stochastic MPC ACC, May 2017 18 / 24

Closed loop stability Theorem Let φ t = (x t s t, c t ), then there exists δ (0, 1) such that, for all t 0, E { φ t 2 P } δ (t/n) 1 φ 0 2 P Hence convergence: x t s t and c t 0 in mean square Result follows from feasibility at time t + 1 of (c 1 t,..., c N 1 t, 0), (α 1 t,..., α N 1 t, g), Φ(ω t)s t + w(ω t ) which implies E { φ t+1 2 } P φt 2 P φ t 2 Q t 0 James Fleming, Mark Cannon Time-average constraints in stochastic MPC ACC, May 2017 19 / 24

Example Model parameters: A(ω) = A 0 + (1) A ω 1 + (2) A ω 2 + (3) A ω 3 B(ω) = B 0 + (1) B ω 4 + (2) B ω 5 w(ω) = w (1) ω 6 + w (2) ω 7 where ω 1,..., ω 7 are independent, uniformly distributed on [0, 1] and A 0 = [ ] [ ] 1.9 1.4 1, B 0.7 0.5 0 = 0.25 ] [ (1) 0.01 0.05 A = 0.05 0.01 ] (1) B = [ 0.03 0.02 [, (2) 0.01 0.05 A = 0 0.01 ], (2) B = [ 0.03 0.02, w (1) = [ 0.2 0.2 ] [ ], (3) 0 0 A = 0.05 0.02 ] [ ], w (2) 0.2 = 0.2 James Fleming, Mark Cannon Time-average constraints in stochastic MPC ACC, May 2017 20 / 24

Example Cost parameters: Q = I, R = 1 Ω = min. RPI set under u = Kx Constraint: [ 0.5 1]x t 1 relaxed version: [ 0.5 1]x k t 1 + 0.8 b k t, r = 0.8 Prediction and constraint horizons: N = T = 3 Time-average constraint: t+2 M t 1, ɛ = 1/3 k=t Compare sample-based MPC with chance constraint Pr { [ 0.5 1]x 1 t > 1 } 1/3 James Fleming, Mark Cannon Time-average constraints in stochastic MPC ACC, May 2017 20 / 24

Example Time-average constraints Chance constraints James Fleming, Mark Cannon Time-average constraints in stochastic MPC ACC, May 2017 21 / 24

Example MPC with chance constraints % constraints satisfied, at t = 2 68 66 average solver time (ms) 132 69 closed-loop cost 198.3 195.2 MPC with time-average constraints Chance constraints imposed via sampling with (greedy) discarding, with # samples limited by requirement for similar computation Fair comparison since both methods satisfy ensemble-average constraints James Fleming, Mark Cannon Time-average constraints in stochastic MPC ACC, May 2017 22 / 24

Conclusions Time-average constraints: MPC algorithm: enforced over an arbitrary time-interval satisfied robustly in transients as well as asymptotically recursively feasible and robustly stable computational requirement similar to robust MPC Impose limits on time-average of loss function using similar approach (details in proceedings) Sequential computation of tightening parameters is suboptimal but same approach allows for simultaneous optimization (requires MIQP) James Fleming, Mark Cannon Time-average constraints in stochastic MPC ACC, May 2017 23 / 24

Thank you for your attention! James Fleming, Mark Cannon Time-average constraints in stochastic MPC ACC, May 2017 24 / 24