Q-learning based control algorithm with dynamic combination of peak shaving and self-consumption optimization for industrial battery storage systems

Abstract:
In the use of battery storage in industrial companies, the associated cost reduction depends strongly on the strategy used. Typically, companies use either self-consumption optimization or peak shaving as operation management strategies. In order to combine these two goals and achieve an optimal cost reduction, a control system must be developed that reacts dynamically depending on the current load peak of the prevailing month. For this reason, a Q-learning based control system was developed which reacts depending on the current peak load. In addition, a method was implemented to optimally design battery storage for this purpose in order to reduce investment costs and to conserve resources by selecting the smallest possible storage size. The reinforcement-learning agent was trained and validated on the basis of a real load profile of a local bakery chain. With a storage capacity of 27 kWh, a reduction of the peak load by 5.7 kW and an optimization of the self-consumption of 79% of the theoretical maximum could be achieved.