Adaptive Model-based Control for Cost-aware Household Appliances

As a step towards sustainable energy management systems, energy providers use various demand side management techniques to reduce fluctuations in consumer energy demand. Dynamic pricing schemes encourage consumers to shift their energy usage patterns from peak hours towards off-peak periods, but here usually the active participation of consumers is anticipated. Smart household devices, which can autonomously shift their time of operation, can efficiently support such demand side management techniques. In this paper an adaptive model-based control scheme is proposed to create intelligent cost-aware household appliances, which can change their behaviour to minimize the cost of consumed energy and at the same time provide the required quality of service. The controller utilizes the dynamically changing energy price list, published ahead by the energy provider, thus it cooperatively supports demand side management. The proposed methods are evaluated in a case study, utilizing a household refrigerator. The proposed adaptive model predictive controller can save 5-10% of the energy bill, according to simulation results.


INTRODUCTION
Financial incentives can provide possible means for energy imbalance management, since a sufficiently motivating dynamic pricing may urge consumers to shift their energy consumption pattern towards inexpensive off-peak periods and decrease their energy utilization in more expensive peak periods. Such Demand Side Management (DSM) techniques are beneficial for both the consumer (possibly resulting in lower energy bills) and the provider (resulting in more balanced energy utilization).
Several energy providers experiment with hourly pricing systems published one day ahead [1]. To be efficient, such pricing schemes require reactive intelligent appliances on the consumer side. This paper investigates the possibility of creating intelligent cost-aware household appliances, which can adaptively change their control strategies to reduce energy cost and at the same time maintain the quality of service. As a case study, a refrigerator is utilized, which has stringent functional requirements to provide a fixed temperature range for the stored food, but at the same time allows some flexibility for the control strategy due to its thermal capacity.
The proposed Model Predictive Control (MPC) system is illustrated in Figure 1. The controller utilizes the measured actual state (inner air and back panel temperatures) of the controlled appliance, and also utilizes a behavioural model and the energy price list for the next few hours. The utilized behavioural model can predict the state of the appliance for the near future, depending on the possible control actions, thus the controller can choose the optimal control strategy in order to shift the time of energy consumption towards lower-priced time periods in order to minimize the total price of consumed energy, while maintaining the required quality of service. The hardware requirements of the proposed solution are low: the two temperatures can be measured using inexpensive temperature sensors.
The proposed MPC solution uses two kinds of predictor. The simpler one is a static State Space (SS) model based solution, while the other method uses an adaptive Time-Domain Non-Parametric (TDNP) model.
The paper describes the utilized system model, the predictors and the control algorithm. The efficiency of the method is illustrated through simulations and real measured data.

RELATED WORK
The technical advancement of the world generates an increasing energy demand in both industry and households. This increasing demand is not uniform in time, rather it has peaks and valleys, mainly depending on the time of day: statistically the most energy is used in early morning and in the evening, while during the day and late nights the energy consumption is much smaller. Due to the uneven energy demand, the operation of the energy network is less efficient: the network must be designed to satisfy sporadic high peaks, resulting in unused capacity in most of the time [2]. The approach of DSM attempts to mitigate this ever increasing problem by modifying consumer demand in time so that actual demand and supply be close to each other [3]. The approach was shown to be efficient to mitigate load related congestions [4]. Day ahead markets create a dynamic environment for DSM [5], which will also be used in the proposed technique. Novel home energy systems open the possibility to apply more efficient DSM in household level [6]. An overview of smart-agent based models for home automation can be found in [7]. DSM can be performed using several control means. The most common way is to use different (but permanent) pricing for day and night periods. Unfortunately, users are reluctant to change their consumer habits, given the potential inconveniences and small economic reward. Controlled supply (e.g. for air conditioners or heaters) allows more direct intervention from the provider's side, but users tend to resist external control of energy utilization [8]. Local buffering methods (e.g. using batteries or heat) can help end users to shift their energy utilization from the network without the need of changing the consumer habits and cost savings can be as much as 20%, according to simulation results of Adika and Wang [9]. DSM combined with storage and photovoltaic generators provided 15% decrease of both load and cost in real experiments [10].
A scenario analysis for future German energy market for 2030 [11] showed that automatic domestic DSM systems along dynamic energy pricing has great potential. This approach utilizes cost-aware household appliances, which can react to changes of energy price without noticeable change of the provided services towards the users. Good candidates are appliances which can shift their time of operation: refrigerators have significant heat capacitance thus some shift in time does not cause large change in their performance, washing machines may not need to perform their operation immediately thus time shifting is again possible. In this paper the concept will be applied to refrigerators.
MPC has been successfully utilized to create smart appliances. In [12] large refrigerators utilized in supermarkets are investigated. The proposed solution considers the changing energy prizes, the heat capacity of the stored food and the daily change of external temperature, to perform nonlinear optimization. In [13] MPC was utilized in household refrigerators and various linear and nonlinear system models were evaluated.
In the literature, several models were proposed to capture the dynamic behaviour of refrigerators. Continuous differential equations were used [14] to describe the physical operations. Much faster discrete time models with similar accuracy were proposed in Lin et al. [15]. In [13] simpler stochastic differential equations were used and a third order model was able to capture nearly all the dynamics contained in the measurements.
In this paper, a switched linear low-order SS model and an adaptive non-parametric model will be used to create the predictor. In the MPC controller, instead of computationally intensive optimization methods, a heuristic controller will be utilized.

HYBRID STATE SPACE MODEL
In order to model the operation of the refrigerator a hybrid (or switched) SS model will be utilized. The proposed MIMO system has two inputs: denotes the control signal (on: 1, off: 0) and is the external (i.e., room) temperature. The output vector contains inside air temperature and back panel temperature .
The utilized model is the following: where is the state vector of length , = [ , ] is the input vector, = [ , ] is the output vector, is the state transition matrix of size × , is the input matrix of size × 2, is the output matrix of size 2 × and is the feedforward matrix of size 2 × 2.
The proposed hybrid model has the advantage that the behaviour of the system can be described by low order models, thus the operation of the controller requires smaller computational capacity. In our case a second-order model ( = 2) was enough to adequately model the refrigerator under test. The model parameters were determined using iterative grey-box modelling, resulting in the following parameter sets: and The performance of the proposed model is illustrated in Figure 2. The refrigerator under test was a conventional 800 W model. The operation of the refrigerator was measured during a day: the control signal, the inside air temperature, the back panel temperature, and the room temperature were recorded. The system model of (1-4) was operated from the measured initial state, i.e., the measured inside and back panel temperatures at 0:00 were set as initial values of 0 . The measured operation of the device is shown in Figure 2 by green colour. The measured room temperature along with the measured control signal was provided as input for the system model. The simulated back panel temperature is shown in red, while the simulated inside air temperature is shown in blue in Figure 2. The simulated results correspond well with the measured data. Notice that the current system model does not contain means to describe external disturbances (e.g., opening of the door), thus during the measurement the device was operated without any user intervention. The hybrid SS model is accurate enough for utilization in model MPCs. However, the modelled system may change in time, thus the model should be re-identified from time to time. In case of refrigerators, the model change frequently happens, e.g., when a large amount of food is placed into the refrigerator, thus changing the heat capacitance of the device. Thus an adaptive predictive model is also proposed in the next section.

ADAPTIVE TIME-DOMAIN NON-PARAMETRIC MODEL
For the adaptive modelling of the refrigerator a (TDNP) model with linear interpolation is utilized. The main idea of the model is to approximate the step responses of the back panel and the inside air temperature. The solution handles separately the warming up and cooling down phases, for each phase using a different table. Figure 3 illustrates the operation of the warm-up table. The warm-up table in row ) → ) contains the time necessary for the back panel to warm up from temperature ) to temperature ) (with compressor off). The cool-down table in row ) → ) contains the time necessary for the back panel to cool down from temperature ) to temperature ) (with compressor on).
The adaptive prediction operates as follows: if the next assumed control state is "on" then the cool-down table is selected. If the next assumed control state is "off" then the warm-up table is selected. According to the current measured temperature ) + , the predicted temperature ) , + + 1 in the next time instant can be calculated as follows: where ∆0 is the sampling interval and -. is the slope, read from the table row ) . → ) .1 , such that ) + ∈ [) . , ) .1 ]. To predict the temperature in the successive time instants, earlier predictions are used, as follows: where the slope (-. is selected similarly from the table row for which ) , + ∈ [) . , ) .1 ].
The prediction steps (6) can be repeated to generate predictions for a future time horizon, i.e., ) , + + 1 , ) , + + 2 , … , ) , + + . The tables are continually updated while the refrigerator is operated, thus providing adaptive behaviour. The system stores the time instants when the measured temperature crosses a reference temperature value, e.g., the compressor is off, thus temperature is increasing. Reference temperature ) . was detected at time 0 . and the next reference temperature ) .1 is detected at time 0 .1 . The table maintenance process is activated when the measured temperature fully crosses a reference temperature range. The time spent in the reference region [) . , ) .1 ] is ∆0 . = 0 .1 − 0 . , which is used to update the table entry in row ) . → ) .1 , as follows: Year 2018 Volume 6, Issue 3, pp 547-558 552 -. = ) .1 − ) . /∆0 .
The operation of the model is illustrated by Figure 3. It is supposed that the temperature changes linearly between reference points. The approximation of the exponential-like step responses is fairly good if sufficiently dense reference points are used (e.g., in 1 °C steps). Figure 4 illustrates the reconstructed warm-up step response of a refrigerator back panel, based on a real measured warm-up table, where the reference temperature values were −19 °C, −18 °C, …, +7 °C in 1 °C steps. The measured slopes correspond to the slopes of the short line segments of the interpolated step response. An example for the adaptation process is shown in Figure 5, where the SS mode was used to provide realistic simulation (red reference curve). The TDNP model was used to create a static model (the tables were trained before the experiment and then were left unchanged, shown by black curves) and an adaptive model (the tables were trained continuously, shown by green curves). The TDNP model was used to estimate both the back panel and the inside air temperatures. In the experiment the ambient temperature increased from 22.5 °C to 35 °C at 4:00, and decreased from 35 °C to 15 °C at 13:00. At the beginning both models had small error until the change at 4:00, when both models produced high error. The adaptive model learns the new behaviour soon, but the error of the static model is permanent. A similar effect can be seen at 13:00: the adaptive model learns the new behaviour quickly after a short transient, while the performance of the static model remains poor.

HEURISTIC MODEL PREDICTIVE CONTROL
The proposed controller assumes that the energy price 5 0 is known ahead with a reasonable horizon [1]. Such pricing schemes are utilized, e.g., in Northern European countries, where hourly energy prices are published 24 hours ahead [16]. The controller utilizes the evolution of the prices and the built-in model to optimize the control decisions. The control goal is to minimize the Total Cost (TC) of the operation: ) ) = 6 5 0 0 70 While minimizing the cost, the required quality of service must be provided. The constraints of the control contain the required temperature range for the inside air and the allowed temperature range for the back panel, as follows: Instead of formal optimization of eq. (8) and eq. (9), the proposed MPC operates using heuristic control laws. Since the time constants of a domestic refrigerator are below one hour, the time horizon is set to two hours: the controller makes its decision based on the current price and the price of the next hour. The following cases are considered: • A: The price of the current hour is the same as the price of the next hour; • B: The price of the current hour is higher than the price of the next hour; • C: The price of the current hour is lower than the price of the next hour. Notice that the control has effect on the current hour, the next hour's data is used only to choose the strategy.

Control law A
The control simply applies conventional control techniques: the cooler is operated until either the temperature of the back panel reaches ) FGH,IJKLM or the temperature of the inside air reaches ) FGH,JNO . Then the cooler is switched off until either the temperature of the back panel reaches ) FPQ,IJKLM or the temperature of the inside air reaches ) FPQ,JNO . In this case no prediction is necessary.

Control law B
The controller tries to minimize the operation time in the current (expensive) hour and delay the operation to the next (cheaper) hour, thus the applied heuristics are the following: operate the device using Control law A, but stop the last cooling period so that at the end of the hour the back panel temperature reaches exactly ) FPQ,IJKLM . To reach this control law the controller makes predictions periodically during the cooling phases (in every 2 seconds), using the actual measured system state as initial state for the model, and performing the simulation with = 0 . The cooler is switched off when the predicted ,RST equals ) FPQ,IJKLM , where ,RST is the simulated back panel temperature at the end of the hour.
If the prediction is accurate, the back panel temperature reaches its allowed maximum exactly at the end of the current hour. However, prediction errors may be present. If the back panel does not warm up to the allowed maximum, some cost is wasted, since the cooler could have been switched off earlier, but otherwise no control action is required. If the back panel temperature reaches its allowed maximum too early, then the cooler must be switched on again, causing again some unnecessary cost.

Control law C
The controller tries to minimize the operation time in the next hour, and thus cools down the device by the end of the current (cheaper) hour as much as possible, so that in the next (expensive) hour the operation time be less. The heuristic control law is the following: operate the device according to Control law A, but when the compressor is off, it makes predictions periodically (every 2 seconds), using the actual measured system state as initial state for the model and performing the simulation with = 1. The cooler is switched on when, according to prediction, ,RST = ) FGH,IJKLM , where ,RST is the simulated back panel temperature at the end of the hour.
In the presence of prediction errors, the back panel may reach its allowed minimum either too early (in this case the cooler must be switched off), or it may not reach its allowed minimum by the end of the hour (in this case no control action is required). In both cases some extra cost occurs. Notice that the required temperature range is always provided, regardless of the accuracy of the model.

EVALUATION
The performance of the proposed control system was evaluated using the refrigerator model of eq. (1-4). In the evaluation four control laws were utilized.

Conventional control
In the refrigerator under test a simple control was utilized, based on the temperature of the back panel. If the temperature of the back panel is above ) FPQ,IJKLM , the compressor is switched on. If the temperature of the cooling back panel reaches ) FGH,IJKLM , the compressor is switched off.

Model Predictive Control-State Space
It uses hybrid SS model in the prediction process. In the simulation no model changes were utilized, thus the SS model, although static, provided accurate prediction.

Model Predictive Control-Time-Domain Non-Parametric
This version of the proposed heuristic control uses adaptive TDNP model in the prediction process.

Optimal Model Predictive Control
The optimal control was calculated using a heuristic search method, which allows to find optimal control for piecewise affine systems [17]. The optimal controller used 4 hours look ahead time. Note that the optimal controller allowed switching the compressor on/off in every 5 minutes, which may not be ideal for a real device, but the results are nevertheless useful for comparison.
For the test, real prices were downloaded from [16], for a whole week starting on 21 March 2016. For each day all three algorithms were run and the total cost was calculated. An example (Tuesday) is shown in Figure 6. The inside air temperature is shown by blue, while the back panel temperature is shown by red, for all four algorithms. The conventional, the MPC-SS and the MPC-TDNP algorithms used control strategies with long cooling and warming periods, while the optimal controller applied short periods, thus the resulting inside temperature is smoother. Figure 6. One day of simulated operation of the refrigerator using the optimal, heuristic and conventional controllers, using real price data Notice that the optimal controller also found the heuristic rules. Between 18:00 and 19:00 an expensive period is visible: the optimal controller cooled down the unit before 18:00 (similarly to Control law C) and let it warm up by 19:00 (similarly to Control law B). The proposed heuristic controller utilizes long periods but they are adequately shifted to produce low cost.
The results are summarized in Table 1, showing the energy cost for each day and for the whole week, for the four algorithms. On those days, where the energy prices show significant fluctuations, substantial saving around 10-14% can be reached, using the optimal control. The proposed heuristic controller with hybrid SS model provides somewhat lower saving results, reaching 8-12% on the best days. The MPC-TDNP algorithm is less accurate and can produce fewer savings than the MPC-SS, as expected. It can reach 6-12% on the best days. On days where the prices are more uniform the achieved savings are moderate. During the whole week, the optimal control allowed saving of 10.4%, while the MPC-SS (accurate, but non adaptive) provided 6.9% and the MPC-TDNP (adaptive, but less accurate) gave us 6.8% savings.
Notice that the optimal controller switches the compressor on and off with much higher frequency than the conventional controller (see Figure 6). This may have undesired effect of the lifetime of the device. The proposed controller, however, increases the switching frequency only moderately.
The computational cost of the conventional controller is negligible. The MPC controllers must run the decision-making process periodically, in order to take into consideration disturbances and changes in system dynamics. The computational needs of the MPC algorithms were compared using the same 2-hour horizon. The optimal and heuristic controllers required 2.2 s and 0.01 s of computation for one iteration, respectively, on a 2.3 GHz PC. The computational needs of proposed method are significantly lower, thus its real implementation on embedded hardware is more realistic.

CONCLUSIONS
A heuristic MPC was proposed for domestic refrigerators which allows cost savings and also supports demand side management. Based on measurements on a domestic refrigerator, a static hybrid SS and an adaptive TDNP model were created. The models were then utilized as predictors in the MPC, to allow realizing simple heuristic control laws, which shift the time of operation, according to price changes.
The performance of the proposed heuristic controller with SS and TDNP predictors was compared to that of a conventional controller and an optimal controller. According to simulation results, the achievable cost saving is about 6.8%, as opposed to 10% gain of the optimal controller.
While the optimal controller provided a theoretical optimum, its real application is not realistic, since it has high computational need, and the provided control strategy is not hardware-friendly (frequent switching of the cooler). The proposed controller, however, can be applied to any hardware (it has similar dynamic characteristics to the conventional controllers) and the computational needs are much less.
The proposed MPC-SS solution uses a static prediction method which is accurate in special simulation or very controlled real life situation, but this attribution is questionable in an average household. That is the case why the MPC-TDNP was developed which uses an adaptive predictive solution. Although this solution is less accurate than the SS model in the simulation environment, in the real life situation the adaptation attribution can compensate it.
The proposed methods showed high potential for real implementations, but future work is required. Real implementations also will require further optimization of the controller and in particular the predictor, so that the controller can be embedded in small inexpensive microcontrollers. Also the effect of door openings could be incorporated in the predictor.