Residential Load Control with Communications Delays and Constraints

Transcription

1 power systems eehlaboratory Gregory Stephen Ledva Residential Load Control with Communications Delays and Constraints Master Thesis PSL1330 EEH Power Systems Laboratory Swiss Federal Institute of Technology (ETH) Zurich Expert: Prof. Dr. Göran Andersson Supervisors: Dr. Johanna L. Mathieu, Dipl.-Ing. Evangelos Vrettos, and Dr. Silvia Mastellone (ABB Corporate Research) Zurich, March 19, 2014

2 Abstract Residential thermostatically controlled loads (TCLs), such as refrigerators and air conditioners, can be used for fast time-scale demand response to manage energy imbalances in power systems. Many direct load control strategies rely on information exchange between a central controller and loads in realtime. This project investigates the effects of limited bandwidth, latencies, and lost measurement or input signals on the ability to centrally control load aggregations to follow requests for a desired demand level. It builds on past work within aggregate TCL modeling, estimation, and control by adapting estimation and control strategies from networked control theory that explicitly handle the previously mentioned communications issues. The results of this work show that in the absence of delays, a central controller requires infrequent information to track highly dynamic demand changes. Specifically, it is shown that requests on all frequency control time-scales are fulfilled when receiving TCL measurements every fifteen minutes. The introduction of delays degrades performance, as expected. However, the controller adequately fulfills requests comparable to secondary frequency control with average TCL transmission delays of 60 seconds and average input delays of 20 seconds. Faster reference changes can be tracked adequately, but delays become increasingly important with high frequency reference changes and may cause inadequate tracking in some cases. Methods are suggested for overcoming these shortcomings and allow the reliable fulfillment of high-frequency demand changes. i

3 Contents List of Symbols iv 1 Introduction 1 2 Control System Overview Communication Network Characteristics Measurement Characteristics Central Controller Overview TCL Modeling The Individual TCL Thermal Model The Aggregate TCL Population Model Networked Predictive Control Addressing the Expected Input Delay Addressing a Finite Set of Disturbances Networked State Estimation The Standard Kalman Filter The Networked, Multi-Sensor Kalman Filter State Estimation of a TCL Population Addressing Individual TCL Measurement Delays Sampling Window Method Parallel Filter Method Individual Model Method Case Studies TCL Parameters Case Study Development Reference Signals Communication Cases Delay Cases Results for MPC Case Studies ii

4 CONTENTS iii 7.4 Results for State Estimator Case Studies Parameter Identification Discussion Results for MPC-Estimator Case Studies Ideal Communication Varying Delay Scenarios Conclusions 53 A Parameter Identification Algorithm 55 B Reference Signals 56 C Parameter Estimation Histograms 59 D Tracking Threshold Data 64 E RMS Tracking Error Graphs 70 F Time Series Plots 73 Bibliography 84

5 List of Symbols ˆx i t Ĉ i t K t The bin-based state estimate corresponding to TCL i The identified thermal capacitance of TCL i at time t The Kalman gain at time t n sensors The number of sensors indexed within the set of sensors p max yk e τ u t t x t x,c δ i δ k + δk ɛ i t γk t P γk t x γk t i û t ˆx t A maximum probability threshold The output error at time-step k The probability distribution of the input delays Duration of the discrete time-steps Sampling period of the TCL states The time between transmissions of the TCL measurements The width of the temperature dead-band Variable implementing a soft upper bound Variable implementing a soft lower bound Gaussian disturbance of appliance i at time t Operator that rounds down the contained value The arrival indicator for the aggregate power measurement The arrival indicator for the aggregate state measurement An arrival indicator for a measurement from sensor i, sampled at time-step k, for the computation at time t Estimated effective input within the controller Estimated state of the system iv

6 CONTENTS v ˆx t The a priori state estimate at time t N 0 The set of natural numbers including 0 R M i t N The set of real numbers The set of on/off mode measurements of TCL i transmitted at time t Indicating a Gaussian distribution N sensors The set of sensors N TCL P k T T x T x,c U k The set of TCLs The set of probabilities associated each of the possible inputs within U k The set of discrete time-steps The time-steps corresponding to TCL sampling instants The set of discrete time-steps corresponding to the transmission of TCL measurements The set of possible inputs used at time-step k µ The mean of a probability distribution P on ψ k t σ τ τ agg τ P t τ u,i t τ i t τ P k t The average power demand of a TCL within the population An arbitrary value corresponding to time-step k that depends on the calculation time t The complement of a set The standard deviation of a probability distribution Indicating a value is sampled from a given probability distribution Arbitrary sampled delay The duration of the sampling window Delay on the power measurement and time t Input delay on the communication path to appliance i at time t Measurement delay for appliance i s internal state at time t The delay of the aggregate power measurement

7 CONTENTS vi τ x k t τ i k t COP i θ i set θ i g θ i t θ i a,t Θ i t P TCL i P rms R i t C k t R k t ṽ k t ỹ k t A a i B C C i C P C x c δ K x The delay of the aggregate state measurement Delay of a measurement from sensor i corresponding to time-step k The coefficient of performance for TCL i The temperature set-point of TCL i The thermal gain of TCL i Temperature of appliance i at time t The ambient temperature for TCL i at time t The set of temperature measurements of TCL i transmitted at time t The estimated aggregate power demand of the TCL population The RMS power estimation error of a simulation The identified thermal resistance of TCL i at time t The augmented output matrix with place-holding zeros removed The augmented measurement covariance with place-holding zeros removed The augmented measurement noise with place-holding zeros removed The augmented observation vector with place-holding zeros removed The state transition matrix The thermal parameter of TCL i The input matrix The output matrix The thermal capacitance of TCL i The power observation s output matrix The aggregate state observation s output matrix The penalty for violating the soft constraint within the MPC algorithm Number of discrete time-steps within a TCL sampling period

8 CONTENTS vii K x,c m i t N bin The number of sampling periods between transmission times or the number of samples within the transmission On/off mode of appliance i at time t The number of discrete state bins within the aggregate model N MPC The number of time-steps within the MPC prediction horizon n TCL N kf t p( ) P agg t P noise t P ref t P TCL i P i rat P rms P t P t Q R R i R x T t t x,c t i k u t The number of appliances within the population The recalculation horizon of the networked Kalman filter The probability of the given event Aggregate TCL demand including noise Measurement noise on the aggregate TCL demand Requested demand level The noise-free aggregate power of the TCL population TCL i s rated power The RMS tracking error of a simulation The a priori estimation error covariance at time t Thea posteriori estimation error covariance at time t The tracking error penalty within the MPC algorithm or the covariance of the process noise within the Kalman filter The input penalty within the MPC algorithm or the covariance of the measurement noise within the Kalman filter The thermal resistance of TCL i The covariance of the measurement noise on the aggregate state The transpose of a matrix Index of the discrete time-steps Index for TCL transmission times The indicator for when a measurement taken by sensor i at time k first arrives at the estimator The broadcast input

9 CONTENTS viii u i j u k τ u v P t v x t v t w t x t y ref k y i k y t y P t y x t An input, u, designed to be implemented at time-step i was generated by the MPC calculation at time j An input delayed by τ u The noise on the power measurement The noise on the aggregate state measurement The measurement noise at time t The process noise at time t The system state at time t The requested power demand at time k within the MPC algorithm The measurement taken at time-step k by sensor i The system output at time t The power observation at time t The aggregate state observation at time t

10 Chapter 1 Introduction The liberalization of power networks has facilitated the development of market-based supply of power. Many of these markets rely exclusively on generation assets to provide the nominal power supply, reserves to counteract forecast errors in real-time, and reserves that secure the network against potential disturbances. Furthermore, there is growing interest in incorporating larger shares of uncontrollable generation from renewable sources such as wind and solar. This creates increased variability within the real-time operation of the power network and leads to increased quantities of generation capacity that are used for system stability and security [1]. Including flexible demand within power markets can activate a large reservoir of existing capacity to provide fast-acting power reserves [2], decrease the required security margin for generation assets [1], and reduce network operation costs [1]. Flexible demand can be realized through pricebased methods that provide an incentive for the end-users to adjust their nominal usage. Alternatively, signals can be sent directly to energy consuming devices to manipulate the total power demand of the population. This is often referred to as direct load control, and is the method used within this work. A major focus of the research within direct load control is modeling and manipulating the demand of residential thermostatically controllable loads (TCLs) [3 10]. TCLs are appliances such as water heaters, electric space heaters, air conditioners, and refrigerators that use their energy consumption to alter the temperature of an internal thermal mass within a set of temperature limits. These appliances operate using hysteresis control, and they can be thought of as energy storage devices that charge during periods of power consumption and discharge during standby periods. TCLs can be coercively controlled by either manipulating their temperature setpoint as in [7, 8] or by signaling them to switch their on/off mode 1

11 CHAPTER 1. INTRODUCTION 2 before the internal temperature has reached a temperature limit as in [6]. This thesis focuses on the use of on/off switching to implement direct load control as it allows continued operation within the nominal temperature range, and the resulting impact to the end-users is minimal. Implementing coercive control of TCLs requires communication between the spatially distributed, controllable loads and the central controller. The use of a shared, digital communication network between the various components allows a flexible network architecture while reducing installation and maintenance costs of the required infrastructure [11]. However, these networks are imperfect and introduce challenges to the controller performance such as bandwidth limitations, latencies, lost measurements or inputs, and out-of-order measurement or input arrivals [12]. Several works investigate demand response while including bandwidth limitations [9, 10, 13, 14] or latencies and packet loss [15 17]. Bandwidth limitations are addressed in [9], which investigates a demand response controller s performance when subject to varying levels of sensing capabilities. Only power measurements of the TCL population s total demand are required within [10], which avoids any need for sensors at individual TCLs. [14] develops a controller that relies on infrequent information from the individual TCLs but frequent measurements of the TCL population s aggregate power. Ref [15] characterizes the performance degradation of a demand response system when exposed to lost and delayed signal values. Ref [16] provides demand response control algorithms that converge to near-optimal values on an hourly level when faced with lost inputs or measurements. Finally, [17] investigates frequency control of demand-side batteries while dealing with network delays. It finds that attempting to control too large of a population results in latencies that prohibit frequency control. While the above papers begin addressing communication issues that can be encountered in demand response programs, there are a vast set of state estimation and control algorithms that have been developed to overcome communication issues within the theory of control over networks [11, 12, 18 21]. Some frameworks for addressing these include robust control, optimal stochastic control, hybrid systems, and perturbation methods as summarized in [19], the delayed differential equation method summarized in [11], and active control methods that take advantage of the particular capabilities of networked control system as summarized in [12]. The novelty of this thesis is the development of a demand response control algorithm holistically using concepts and capabilities within the networked control literature. Within this work, a measurement system is tailored to the capabilities of digital communication networks, which enables

12 CHAPTER 1. INTRODUCTION 3 on-line parameter estimation and effective state estimation despite communication delays and losses. A Kalman filter is modified to account for delayed, lost, or out-of-sequence measurements where the delays are known once the measurement arrives at the estimator. In the most advanced state estimation scheme, identified TCL parameters are used to generate predictions regarding the TCL population, and these predictions are supplied to the Kalman filter as measurements to enable more accurate state estimations. A model predictive control (MPC) algorithm is developed that generates and sends a set of inputs, corresponding to the present and future times, to the plant. This is in contrast to the standard MPC scheme where only the first generated input is used and the remaining inputs determined within a given calculation are discarded. The MPC is further modified using stochastic programming principles to proactively account for the probabilities of the communication issues, which are assumed to be known based on prior history. Finally, the estimator-controller system is evaluated in its ability to track a set of references that correspond to frequency control at various timescales, e.g., tertiary and secondary control. The tracking at different timescales is first evaluated at different TCL measurement transmission intervals using ideal communication to set an upper bound for tracking with different communication levels. Following this, the degradation of tracking performance for the various TCL measurement transmission intervals is evaluated by simulating increasingly severe levels of communication disturbances. It is shown that without communication delays the TCL population can track references with time scales comparable to primary control when the central controller receives TCL measurements at an interval of fifteen minutes. When large measurement and input delays are included along with infrequently receiving TCL measurements, the controller is able to track a reference comparable to secondary frequency control, and methods of allowing tracking of more dynamic references are proposed. The remainder of this work is organized as follows: Chapter 2 presents the details of the overall control scheme including the communication network and measurement characteristics. Chapter 3 develops the TCL modeling framework. Chapter 4 develops the MPC controller that addresses the delays introduced by the communication network. Chapter 5 develops a state estimation framework that addresses delayed measurements where it is assumed all TCL measurements are delayed equally at a given time. Chapter 6 then extends this framework to include the possibility of individually delayed measurements from each TCL. Chapter 7 then provides case studies investigating the performance of the system. Finally, Chapter 8 concludes this work.

13 Chapter 2 Control System Overview P ref t Central Controller Model Predictive Controller ˆx t û t u t Input Communication Network. u t τ u,1 t u t τ u,2 t u t τ u,n t ɛ 1 t ɛ 2 t ɛ n t Plant TCL 1 TCL 2. TCL n P n t P 2 t P 1 t θ n t m n t θ 2 t m 2 t θ 1 t m 1 t State Estimator. P agg t τ P t θ 1, m 1 t τt 1 t τt 1 θ 2 t τ 2, m2 t τ 2 t θ n t τ n t, mn t τ n t Power Measurement Communication Network P agg t... P noise t.... TCL Measurement Communication Network Figure 2.1: Block diagram describing the overall control system. The control scenario within this work is similar to that of [13] and [14] and is shown in Figure 2.1. It consists of the individual TCLs representing the plant, models of the various communication networks, and control algorithms of a central controller. The controller operates from the perspective of a demand aggregator that has offered an ancillary service, e.g., secondary frequency control, to the network operator. The objective of the controller is to manipulate the on/off modes of a set of TCLs such that the population s total demand follows a request for the offered ancillary service, P ref t. As in [13] and [14], the information available to the controller includes frequent, noisy measurements of the total TCL demand, shown in red in Figure 2.1, and infrequent, noise-free measurements of the TCL temperatures and on/off modes, shown in green. The power measurements, P agg t, are assumed to be available to the controller at each time-step, which is on the order of seconds. They are obtained by sampling the power supplied through distribution substations that serve the TCLs and then isolating the 4

14 CHAPTER 2. CONTROL SYSTEM OVERVIEW 5 real-time TCL demand. The need to isolate the TCL demand from substation power measurements results in measurements containing a significant amount of noise, P noise t. In addition to the power measurements, the temperatures and on/off modes, θ i t and m i t, for each TCL i are assumed to be available to the controller on the order of ten minutes. These measurements are provided directly from sensors at each TCL, and so they are considered noise-free. The long sampling interval is chosen to reduce the bandwidth required for thousands of TCLs to communicate with a central controller. Each TCL i also experience a disturbance, ɛ i t, arising from usage of the appliance, which influence the evolution of their internal temperatures. While the system shown in Figure 2.1 and that considered in [13] and [14] are very similar, there is also a major addition in the present system: the inclusion of communication network effects on the signals such as latencies, lost samples, and out-of-sequence sample arrival. Latencies mainly arise from the physical travel time and the queuing time in transmitting a measurement or input. Lost samples arise due to excessive transmission latencies or failed transmissions. Finally, out-of-sequence sample arrival arises due to time-varying latencies that can exceed one sampling interval. These three effects can all be modeled using time delays, and these delays are represented in the figure above using τ u,i t for each appliance s input delay, τt P for the power measurement delay, and τt i for each appliance i s measurement delay. The algorithms for the central controller are designed specifically to take into account these communication issues. A state estimation algorithm is used to address the measurement noise and measurement delays while providing an estimate of the underlying system state, ˆx t. This system state is then provided to a control algorithm that proactively accounts for the expected input delays. It broadcasts the input, u t, to all TCLs, which signals coercive on/off switching. The input delays cause uncertainty at the controller regarding the realized input, and an estimate of the realized input, û t, is provided to the state estimator to take this into account. The following sections provide more details on the communication network characteristics, on the measurement characteristics, and on the control algorithms. 2.1 Communication Network Characteristics Digital communication networks utilize packets, which are discrete messages, to communicate between the various computing components, or nodes, connected to the network. The nodes use standardized instructions, called protocols, to define logistic procedures within the network such as packet

15 CHAPTER 2. CONTROL SYSTEM OVERVIEW 6 construction, packet addressing/forwarding, packet interpretation at its destination, etc [18]. This communication methodology creates several valuable capabilities within the networked control system [12]: 1. the ability to transmit multiple values within one packet due to predefined packet sizes; 2. the ability to time-stamp packets indicating when they were constructed; 3. intelligent actuators due to decoders that carry out network access routines, which implies they have some computational abilities. These characteristics can be used to reduce the effects of the time delays, packet loss, and out-of-sequence measurements that are introduced by the networks within Figure 2.1. For example, the time-stamping of packets allows transmission delays to be known instantaneously once the packet arrives based on the discrepancy between the time-stamp and the receiving node s internal clock. Additionally, the transmission of multiple values within one packet allows a series of measurements or inputs to be transmitted simultaneously. Combining this with intelligent actuators allows the selection of an appropriate input from a transmitted horizon to counteract delays in the input network. The communication issues of clock drift and quantization are not address because it is assumed that sampling times are long enough to disregard any loss of synchronization among the various nodes internal clocks and that the data capacity in a packet is large enough to disregard quantization. To create the latencies, packet loss, and out-of-sequence arrival within the simulated control system, a delay is applied as each packet, i.e., each measurement or input set, is sent through its respective communication network. These delays are determined for each packet by independently sampling a discrete, log-normal distribution: τ = e d, d N (µ, σ 2 ) (2.1) where indicates the value is rounded down to the nearest integer and N (µ, σ 2 ) indicates a normal distribution with mean µ and covariance σ 2. Each of the three communication networks are modeled as separate delay distributions that are characterized by a separate mean (µ) and covariance (σ 2 ). The separate distribution are used to account for the possibility that separate infrastructures may be used for communicating inputs, TCL measurements, and power measurements. The use of a log-normal distribution generates delays that are non-negative and unbounded above, which allows the possibility of packet loss. Furthermore, no restrictions are placed on the derivative of the delay, which allows the of out-of-sequence packet arrivals.

16 CHAPTER 2. CONTROL SYSTEM OVERVIEW 7 Based on the above formulation, three stationary probability distributions describe the delays in the three separate communication networks. More advanced delay models could be used that capture correlations in delays at successive time-steps, or by introducing time-varying models that capture the dependence of the delay probabilities on network characteristics such as traffic levels [22]. The drawback of the simpler delay model is that some realism is lost within the system and minor modifications are needed for some networked control algorithms, which typically use correlated delay models. Alternatively, the use of a simple delay model allows for a more approachable initial investigation into the impact of communication issues within demand response systems, and the methods provided can easily be adapted to handle correlated delays or time-varying delay distributions. These extensions are provided throughout this work where necessary. 2.2 Measurement Characteristics Frequent, noisy measurements of the system output, the aggregate TCL demand, are available to the controller, and infrequent, accurate measurements of the system state, the TCL temperatures and on/off modes, are available to the controller. Previously, [13] showed that this measurement framework is feasible for TCL control, and the measurement framework is extended here to include capabilities of digital communication networks. In order to access power measurements for the TCL population, measurements are assumed to be available at the distribution substations serving these appliances. As in [9] and [13], this work assumes that a noisy measurement of the aggregate TCL demand is sampled at the necessary substations and transmitted to the controller at each time-step. The discrete time-steps are described using the set of natural numbers including 0: T N 0. Individual time-steps within this set are indexed using t, and the period of a, dictates the duration of the discrete time-steps and is generally of the order of seconds. time-step is defined as t. The sampling rate of the reference, P ref t The aggregate power measurements are represented as P agg t within Figure 2.1 and are determined by sampling the total power supplied at the distribution substation and isolating the portion of the supplied power that corresponds to the TCL population. Isolating the TCL demand can be done by subtracting the real power measurements from a total demand forecast constructed without any coercive control, which results in a signal containing forecast errors and demand response adjustments. Adding the steady-state consumption of the TCL population leads to a noisy measurement of the TCL population.

17 CHAPTER 2. CONTROL SYSTEM OVERVIEW 8 Measurements of each TCL s state are also available to the controller within a framework that takes advantage of the capabilities of networked control systems. Within [9] the states were sampled on the order of seconds, and each measurement was transmitted to the controller independently. However, using the capabilities of networked control systems, it is assumed that the TCL states are sampled every few seconds, are buffered into a set by the appliance controller or smart meter, and the set of measurements is transmitted to the controller as one packet every few minutes. The TCL sampling rate is dictated by the buffering capability at the appliance and also by the protocol defining the data capacity within a single packet. The transmission rate depends on the bandwidth of the communication network that can be dedicated to this specific task. The sampling period of the TCL states can be described as t x = K x t, where K x defines the number of discrete time-steps between each sampling instant. The resulting set of time indices is defined as T x T. The period between the communication of measurement sets to the central controller is defined as t x,c = K x,c t x, where K x,c defines the number of samples within each data transmission. Finally, the set of discrete time-steps corresponding to instants where the TCL measurement sets are communicated is defined as T x,c T x and is indexed using t x,c when necessary. The timing of the sampling periods and transmission periods can be seen in Figure 2.2 θ t and m t Measurements t x {}}{ Time Inidividual TCL Measurement Buffer Inidividual TCL Measurement Buffer P agg t Measurements t {}}{ t x,c Time Transmit to the Central Controller Figure 2.2: Schematic showing the relationship between the power sampling times, TCL sampling times, and TCL transmission times.

18 CHAPTER 2. CONTROL SYSTEM OVERVIEW Central Controller Overview The central controller attempts to manipulate the on/off mode of individual TCLs within the population such that their aggregate power demand follows a request for an ancillary service by the system operator. The algorithms within the controller use a model that captures the dynamics of the TCL population s aggregate power demand rather than the physical laws governing each appliance s operation. This aggregate model is used within an MPC algorithm to determine an optimal input, u t, which manipulates the total power demand by dictating the portion of TCLs that should switch their on/off mode. In addition to the MPC algorithm, a state estimator incorporates the aggregate model and the received measurements to determine the portion of TCLs that are within a set of discrete states. The state estimator is a Kalman filter-based algorithm that uses the aggregate model to capture the dynamics of the underlying TCL population. It then incorporates measurements of the TCL population s total power demand, measurements of the TCL states, and estimates of the actuated input signal, û t, to determine an estimate of the system state, ˆx t. Modifications are made to a standard Kalman filter to counteract the delayed, lost, and out-of-sequence measurements that may arrive at the controller. Additionally, a parameter identification algorithm is used to simulate individual TCLs within the estimator. These identified TCL models are used to generate state predictions at a higher sampling frequency than the actual TCL measurement arrivals. The Kalman filter algorithms are adapted from [23] and [24], and they are presented in Chapter 5 before modifying them for the current scenario within Chapter 6. The other portion of the control problem, generating inputs, is handled by the MPC algorithm. This algorithm uses the state estimate for a given time-step, provided by the state estimation algorithm, to initialize a model-based control algorithm with state and input constraints. The MPC algorithm uses optimization-based control to determine inputs corresponding to the current time-step and some set of future time-steps, called the prediction horizon. Rather than sending only the first input of the horizon to the actuator as is done in standard MPC, a modification for networked control is used where the entire horizon of inputs is sent [12]. The input sets are broadcast to all TCLs, and their individual response to the input depends on a TCL s local state and the delay in communicating the input to the TCL. The control algorithm is based on the work in [12] and [25]. It is modified slightly to accommodate the delay distribution used within this work as well as the fact that the plant is made up of a large number of distributed, independent resources. The resulting algorithm is presented in more detail within Chapter 4.

19 Chapter 3 TCL Modeling Two methods for TCL modeling are described below. Section 3.1 models the thermal dynamics of the individual appliances, which is used as the plant model. While this model is accurate, incorporating thousands of these models into a control algorithm is too computationally intensive. To overcome this, Section 3.2 models the aggregate behavior of the TCL population using a discrete, linear, time-invariant system that is suitable for use within the control algorithms. 3.1 The Individual TCL Thermal Model The evolution of each appliance i s internal temperature, θ i t, and on/off mode, m i t, are often modeled in literature using the following equations [26]: θ i t = a i θ t 1 + (1 a i )(θ i a,t m i t 1θ i g) + ɛ i t i N TCL (3.1a) 0 θt i < θset i δ i /2 m i t = 1 θt i > θset i + δ i /2 m i t 1 otherwise i N TCL (3.1b) where N TCL is the set indexing the appliances, which consists of natural numbers from 1 to the number of appliances, n TCL. The parameter a i is defined as a i = exp ( t ), where t is the discretization time-step, R i is the C i R i thermal resistance of appliance i, and C i is the thermal capacitance. The on/off mode, m i t, describes whether the appliance is drawing power (m i t = 1) or in standby (m i t = 0). θa,t i is the ambient temperature, ɛ i t is a disturbance, θset i describes the temperature set-point, and δ i describes the width of the temperature dead-band. The temperature gain, θ i g = ±R i P i rat COP i, is defined as positive for cooling appliances. It describes the energy transfer to the conditioned medium while the appliance is drawing power, and it is based 10

20 CHAPTER 3. TCL MODELING 11 on the appliance s rated power, P i rat, its coefficient of performance, COP i, and its thermal resistance. Implementing demand response through coercive on/off switching modifies equation 3.1b such that the appliance can also be forced to switch on or off before reaching its boundary temperature. This creates an additional degree of freedom within the operation of the appliance, and allows the power consumption profile to be manipulated. Additionally, the local controller is assumed to remain active and overrides any coercive switching signals when the temperature is outside of the dead-band. This ensures any appliance outside of its allowable temperature dead-band switches appropriately. 3.2 The Aggregate TCL Population Model The individual model above effectively captures the dynamics of each TCL [27], however each TCL is modeled as a hybrid system containing both discrete (m i t) and continuous (θt) i states. Implementing demand response programs requires the coordinated control of thousands of TCLs, and using the hybrid model for each appliance results in a computationally intensive task. An aggregate model was originally developed in [6] to facilitate the control of a large population of TCLs. It models the aggregate behavior of the population within the following discrete-time, linear, time-invariant system: x t = Ax t 1 + Bu t y t = Cx t. (3.2a) (3.2b) To arrive at the above formulation, the temperature dead-band of each TCL is mapped to a normalized dead-band that admits all appliances. The normalized dead-band is then divided into N bin 2 temperature intervals, and two separate bins are created in each interval one for appliances that are on and one for appliances that are off. This results in N bin total discrete bins that capture a TCL s position within its temperature dead-band and its current on/off mode. The first N bin 2 bins correspond appliances in standby, and the bins from N bin 2 +1 to N bin correspond to appliances that are drawing power. Figure 3.1 provides a schematic of this discrete state system. The state of the aggregate model, x t R N bin 1, is defined as the portion of the TCL population contained within each of the bins described above. Rather than modeling the physical dynamics of the TCLs, the state transition matrix, A R N bin N bin, consists of a transposed Markov Transition Matrix that captures the probability of a given TCL moving from an initial

21 CHAPTER 3. TCL MODELING 12 ON N bin N bin -1 N bin -2 N bin -3!" N bin 2 +4 N bin 2 +3 N bin 2 +2 N bin 2 +1 OFF !" N bin 2-3 N bin 2-2 N bin 2-1 N bin 2 temperature Figure 3.1: Diagram showing the discrete state bins for a cooling appliance. Taken from [9]. bin into another. The resulting A-matrix is: p 1,1 p 1,2 p 1,Nbins p A = 2,1 p 2,2 p 2,Nbins p Nbins,1 p Nbins,2 p Nbins,N bins T (3.3) where p i,j is the probability that a TCL within bin i moves to bin j during the time-step. In practical applications with heterogeneity in multiple parameters, the A-matrix must be estimated using system data as was done within [6]. The input, u t R N bin 2 1, indicates the portion of the TCL population within each bin that are forced to switch on or off. The input matrix, B R N bin N bin 2, is constructed as: 1 0 B = , (3.4) 1 0 and it simply forces portions of the population from one state bin into the equivalent bin of the opposite on/off mode. Based on the construction, the input forces TCLs on if the given element of u is positive and off if the given

22 CHAPTER 3. TCL MODELING 13 element is negative. The input signal is converted into a switching probability before being broadcast to the TCLs by dividing by the state estimate, and the TCLs carry out a random process locally to determine whether each unit responds to the applicable signal. With a large enough population, this reproduces the desired aggregate response in the total population. Finally, the output, y t R 1, is defined as the overall power consumption of the TCL population, P agg. The corresponding C-matrix is: C = P on n TCL [ 0 0 N bin ] (3.5) N bin 2 where P on describes the average power demand of a TCL in the population and n TCL indicates the number of TCLs in the population. The zero entries in the C-matrix correspond to bins where the appliances are off and not drawing power, and the ones within the matrix correspond to bins containing appliances that are on and drawing power. Alternative forms of the C-matrix are needed depending on the application within the controller, however these will be developed as needed in the following chapters. The system of the identified A-matrix and the C matrix described above is observable in all practical scenarios. To ensure observability, the A-matrix must contain at least two probabilities within each column: the probability of remaining in a given state and the probability of transitioning into the next state. Given that these transitions are very likely to be observed within the identification period, the A-matrix using the output matrix defined above is considered observable.

23 Chapter 4 Networked Predictive Control Using the aggregate model developed in the previous chapter, a controller is developed that manipulates the on/off modes of the TCL population such that the total power demand tracks a requested value. The control task is complicated by the fact that the broadcast input signal experiences an independent delay on the communication path between the controller and each TCL. However, it is assumed that the stochastic description of these delays is known to the controller based on past experience. The capabilities of communication networks highlighted within Section 2.1 are utilized along with principles from stochastic programming to counteract these difficulties. The specific network characteristics that are utilized are the ability to time-stamp packets to indicate when they were constructed, the ability to transmit multiple inputs simultaneously, and the ability of the TCLs to select an input from a given set [12]. The ability to transmit multiple inputs within one signal allows an MPC algorithm to send the entire horizon of calculated inputs to the plant. This is in contrast with the standard implementation of MPC where only the first input of the horizon is used. Each TCL is assumed to have the ability to store an input set once it has been received. Using time-stamping, the TCL stores the most recently calculated input set that has been received, which counteracts out-of-sequence arrival of inputs by discarding older sets. Furthermore, the TCL uses its selection capabilities to select an input from the stored horizon corresponding to the current time-step. At time-steps where no new input set arrives due to packet loss or delays, the TCL overcomes this by continuing to use the currently stored input set and using the next appropriate value. This corresponds to using the next applicable open-loop input generated within the currently stored MPC horizon. In the case where the stored input set 14

24 CHAPTER 4. NETWORKED PREDICTIVE CONTROL 15 does not have a valid input for the current time-step, an input of zero is applied. In the remainder of this chapter, an MPC scheme is first developed where the average input delay is accounted for using state augmentation. Following this, an alternate MPC scheme is developed that takes advantage of the fact that the entire probability distribution of delays is assumed to be known, and the set of delays, i.e., disturbances, are finite. This MPC scheme uses stochastic programming to determine an optimal set of inputs based on the probability of each disturbance. 4.1 Addressing the Expected Input Delay Due to the presence of input and state constraints, a finite-horizon MPC controller is implemented instead of a traditional linear-quadratic regulator. The MPC formulation consists of an objective function, equality and inequality constraints, and a state augmentation method of including the average input delay within the dynamics of the system. The objective function assigns a cost to each potential set of state evolutions over the prediction horizon. The constraints are used to dictate the evolution of the dynamics and to limit the set of acceptable input, state, or output values. The solution of the MPC formulation at each time-step is the set of inputs that minimize the cost of the objective function while satisfying the given constraints. Readers that are unfamiliar with MPC are referred to [28] for more details. The objective function, constraints, prediction horizon, and the state augmentation method of incorporating delayed inputs are now described in more detail. The prediction horizon of the MPC problem, N MPC, is determined based on the probability distribution of the input delays within this formulation. In cases where there is an upper bound on the delay, a natural choice for the horizon length is this upper bound. In cases where the delay is unbounded, a horizon can be set where the probability of a delay exceeding the horizon length is less than a threshold, p max. This can be defined as: N MPC = min {N MPC p (N MPC τ u τ u ) < p max } (4.1) where p( ) is the probability of a given event and τ u τ u indicates an input delay sampled from its distribution, τ u. This prediction horizon is used within the MPC formulation to determine the set of time-steps considered within the current optimal control calculation. The constraints within the MPC formulation include a set of soft state constraints, equality constraints that incorporate the dynamics into the

25 CHAPTER 4. NETWORKED PREDICTIVE CONTROL 16 problem, and input constraints. As in the standard linear-quadratic regulator, the state evolution and resulting output of the system are included as equality constraints. An input constraint is included to reflect the physical reality that the entire population of the TCLs can be turned on or off, which corresponds to feasible values between -1 and 1. An inequality constraint is included for the states to reflects the fact that the they must have a value between zero and one based on the state vector s definition as the portion of the population within discrete bins. Soft bounds are used on the state constraint because the initial state value within each calculation is provided by an unconstrained estimator, which may not provide a feasible initial value. The use of soft bounds allows usable solutions to be found in this case, and the violations of these soft bounds are penalized in the objective function. The MPC s objective function assigns some cost to each set of control actions that satisfy the constraints over the given prediction horizon, and the optimal set of inputs is determined such that the cost is minimized over the horizon. The objective function consists of a quadratic cost term penalizing the output s deviation from the reference value and a quadratic cost term penalizing the input effort as in the standard linear-quadratic regulator formulation. A linear cost term penalizing the soft constraint on the system state is added where the cost coefficient is set high enough that the controller will not actively choose to violate it. To include knowledge of the input delays into the control algorithm, it is assumed that the probability distribution of the delays is known at the controller based on historical delays. A typical approach within stochastic programming is to determine an optimal solution while taking the expected value of the random variable into account. This amounts to assuming the input delays are deterministic and equal to the mean of the probability distribution. State augmentation can then be used to include the delayed inputs into the system s dynamics by extending the state vector to store delayed inputs. The A-matrix is extended to progress the delayed inputs through the extra states as time progresses, and it allows the delayed inputs to be actuated after the given delay has expired. This allows optimal inputs to be determined within the formulation while taking the input delays into account.

26 CHAPTER 4. NETWORKED PREDICTIVE CONTROL 17 The MPC formulation can be summarized as: minimize u subject to N MPC k=1 (y e k )T Q (y e k ) + ut k τ u R u k τ u + c δ(δ k + δ+ k ) x k = A x k 1 + B u k τ u yk e = yref k C x k 0 δk x k 1 + δ k + 1 u k τ u 1 0 δ k, δ+ k (4.2) where k = t,..., t+n MPC 1 applies to the constraints above. The A, B, and C matrices correspond to those of the aggregate model developed in Section 3.2. The value yk ref R 1 1 is the requested aggregate power at the time of the calculation, and a persistent value equal to the present request is used for the duration of the horizon. The output, y k, is the aggregate demand of the TCLs, and the tracking error yk e is the state-dependent value that is minimized in the objective function. The δ +, k values are used to impose the soft constraints on the state, and c δ is the penalty on the soft constraints. Finally, the delayed inputs are indexed using u k τ u to indicate that state augmentation is used to include the delay. This method utilizes knowledge of the mean input delay, however, it does not fully account for the known probability distribution of delays. Taking advantage of the logic used to select inputs at the TCLs, there is a finite set of possible inputs corresponding to a given time-step based on inputs that were broadcast at previous time-steps. This can be incorporated into the MPC formulation as a finite set of disturbances with known probabilities at each time-step. This knowledge is incorporated into the control algorithm in the following section. 4.2 Addressing a Finite Set of Disturbances The MPC algorithm outputs a horizon of inputs, and the actual value used at each of the TCLs depends on the realized delay on the specific communication path. Because of this fact, there is uncertainty at the central controller regarding what input was actually used within the plant as was described in [25]. Furthermore, because there are a large number of individual TCLs acting on a single, broadcast input signal, portions of the TCL population will utilize different inputs based on the specific delays that have been realized on each path. This section adapts an MPC extension from [25] that utilizes the logic of the TCLs to limit the set of inputs to a finite set and assign a probability to each of the possible inputs.

27 CHAPTER 4. NETWORKED PREDICTIVE CONTROL 18 U k u t u k t u k+1 t u k+2 t u k+3 t u t 1 u k 1 t 1 u k t 1 u k+1 t 1 u k+2 t 1 u t 2 u k 2 t 2 u k 1 t 2 u k t 2 u k+1 t 2 u t 3 u k 3 t 3 u k 2 t 3 u k 1 t 3 u k t 3 Figure 4.1: Diagram portraying the input selection concept at the actuators. Due to the logic used at the TCLs, there are a finite set of inputs that can be used at a given time-step. Specifically, each TCL only uses inputs that correspond to the present time-step, and there are a limited number of valid inputs due to the finite horizon used within the MPC formulation. This concept can be seen in Figure 4.1. The time indexing within this section uses u i j to indicate that an input, u, designed to be implemented at time-step i was generated by the MPC calculation at time j. The quantity U k defines the set of inputs that could possibly be used by an actuator at time-step k. The set of inputs can be thought of as the designed input plus a set of disturbances, i.e., delayed inputs, that could be utilized by the TCLs. U k consists of N MPC inputs that are generated during MPC calculations with time-step k contained in the calculation horizon. The set of inputs that can be actuated at a given time-step are defined as: U k = [u k k u k k 1 u k k NMPC +1] T k T (4.3) where each element of U k is an N bin 2 1 vector. The element u k k consists of the standard input that would be used if no delays were present, and the remaining inputs are the disturbances that could influence the plant due to delays. Each time-step of the MPC horizon now contains a vector of possible inputs. Figure 4.2 provides a matrix of possible inputs for a horizon length of four where each column corresponds to a set U for the given time-step. The time indices indicating which time-step the input applies to and at which time the input was calculated have been explicitly written out for clarity. The portion of the matrix highlighted in green corresponds to inputs that were previously determined, and the right portion of their time indices is less than t. The gray inputs correspond to the values that will be output by the MPC algorithm from the current calculation, and their calculation times all correspond to the current time-step, t. Finally, the blue elements correspond to future values that are not yet finalized, and their calculation times all correspond to times greater than t.