Reinforcement Learning-Based False Data Injection Attacks Detector for Modular Multilevel Converters

Abstract

The modular multilevel converter (MMC) is a prominent solution for medium- to high-voltage and high-power conversion applications. Recently, distributed control strategies have been proposed to make this converter modular in terms of software and control hardware. In this control architecture, high-level control tasks are performed by a central controller (CC), whereas low-level control tasks are achieved by local controllers (LCs) placed on the MMC submodules. The CC and LCs use a cyber-physical system (CPS) to share all the necessary information to execute their respective control schemes. In this context, the CPS is vulnerable to cyberattacks, such as the false data injection attack (FDIA), where the data seen by the controllers are corrupted through illegitimate data intrusion. This cyberattack may hinder the MMC performance, producing suboptimal, or even unstable operations. Even more, diverse FDIAs can be generated using artificial intelligence methods to deceive FDIA detectors. This article proposes an FDIA detector based on the reinforcement learning (RL) technique to detect sophisticated FDIAs targeting the MMC control system. The performance of the proposed RL-based FDIA detector is verified via hardware-in-the-loop studies, showing its effectiveness in detecting sophisticated attack sequences affecting the MMC control system.