How to manage GenAI cyber risk in industrial control systems
AI is reshaping cyber warfare. As we face this new reality, understanding the role of AI and GenAI in cyberattacks and defences and the opportunity costs of security decisions is critical
A bustling city awakens to familiar routines: commuters head to work, children trek to school, and businesses open their doors. Unseen, a silent storm brews: malware quietly infiltrates the city's power grid, water supply, and transportation networks. Then, in a coordinated strike, traffic lights fail, blackouts spread, and water supplies falter. Businesses close, hospitals switch to emergency power, and airports fall silent.
How could such a massive disruption happen without warning? AI is reshaping cyber warfare, enabling malware to blend into everyday operations, learn system weaknesses, and strike with precision. With (Gen)AI, even amateurs could orchestrate attacks once reserved for sophisticated actors.
As we face this new reality, understanding the role of (Gen)AI in cyberattacks and defences and the opportunity costs of security decisions is critical. Each choice carries trade-offs that influence the Cyber Kill Chain, and preparing for AI-driven threats is now a societal imperative, essential to protecting the stability of our interconnected world.
Industrial Control Systems are Vulnerable
Industrial Control Systems (ICSs) are the backbone of critical infrastructure, managing operations in sectors like power, water, oil, telecommunications, and nuclear facilities. These systems use sensors, actuators, and PLCs to automate and monitor industrial processes, enabling real-time data collection, efficiency, and scalability. Without ICSs, operations would rely on manual monitoring, leading to slower responses, more downtime, and increased risk of failure. Disruptions to ICSs can have serious implications for public safety, economic stability, and national security.
Despite the critical functions ICSs provide for society, ICSs remain one of the most vulnerable systems in terms of cybersecurity. This vulnerability arises from various factors, all ultimately linked to the opportunity costs involved in managing and securing these systems.
ICSs are littered with legacy systems: Infrastructure upgrades are often delayed due to the high operational costs and the immediate and wide-reaching impacts of downtime, forcing organisations to weigh the immediate opportunity cost of disruptions against the long-term risk of unpatched vulnerabilities.
OT systems have poor visibility: Ensuring full visibility of OT systems' hardware, software, and firmware levels requires significant investment in monitoring tools and integration with IT systems. However, the opportunity cost of achieving this visibility is the potential diversion of resources from other critical areas, such as maintaining or upgrading physical infrastructure, underscoring the trade-off between comprehensive security monitoring and the efficient use of limited resources.
IT/OT convergence leads to attack spillover: While IT/OT convergence boosts efficiency through real-time data sharing and automation, it expands the attack surface, exposing critical systems to IT vulnerabilities. The opportunity cost here involves the decision to integrate these systems for enhanced functionality versus the increased security risks that come with such integration. This leads to a complex trade-off between innovation and security.
Lack of security awareness: Budget constraints often lead to underinvestment in employee training, increasing susceptibility to phishing, ransomware, and breaches—risks whose costs can far exceed initial training expenses.
(Gen)AI amplifies existing ICS security challenges and introduces new ones. It enables low-skill attackers to launch sophisticated, automated, and scalable attacks, quickly exploiting known vulnerabilities—especially in legacy systems with weak passwords, poor visibility, and unpatched software. IT/OT convergence further broadens the attack surface. (Gen)AI also enhances phishing by mimicking writing styles, leaving untrained employees vulnerable. By adapting to defences and refining each stage of the Cyber Kill Chain, (Gen)AI makes threats faster, stealthier, and harder to counter.
Figure 1: How attackers and defenders can leverage (Gen)AI at each step of the CKC.
How (Gen)AI empowers attackers
(Gen)AI empowers the attackers at each step of the Cyber Kill Chain. While each step of the CKC is crucial for understanding and responding to cyber threats, we focus on select steps—reconnaissance, weaponisation, command and control—to emphasise the importance of the interplay between (Gen)AI and the CKC. (Gen)AI enables adversaries to execute more sophisticated and evasive attacks, posing significant challenges to traditional cybersecurity defences. Importantly, the use of (Gen)AI by attackers comes with no significant opportunity cost—rather, it only serves to enhance their capabilities, making their operations more efficient and effective while saving time and without sacrificing other methods or resources.
Reconnaissance: In the first stage of a cyber attack, attackers gather information to identify vulnerabilities. (Gen)AI enhances this process: convolutional neural networks can analyse satellite imagery to map physical security features like cameras or guard towers, while other AI tools can mine forums and documentation for SCADA vulnerabilities, default passwords, configuration tools, etc. GenAI excels at synthesising this data, generating rapid summaries of potential targets using NLP and deep learning.
Weaponisation: In the weaponisation stage, attackers craft malware to exploit vulnerabilities found during reconnaissance. (Gen)AI is increasingly used to generate such malware– examples include AI-generated code for stealing files, installing remote access tools, or encrypting machines [6]. As (Gen)AI advances, we can expect it to become capable of writing code that targets specific ICS components such as ensors, PLCs, actuators, etc. With ICSs' long lifespans, limited visibility, and poor patching, known vulnerabilities remain plentiful and easily weaponised by AI.
Command and Control (C2): In the command-and-control (C2) stage, malware establishes a remote channel for attacker control. To avoid detection, (Gen)AI can generate realistic network traffic that mimics normal operations, such as energy consumption or grid control data, and align C2 activity with high-traffic periods by learning typical timing patterns, making malicious communications harder to detect.
In summary, (Gen)AI enhances attackers' capabilities across the Cyber Kill Chain by automating and improving the efficiency of reconnaissance, malware creation, and targeted attacks, posing significant challenges to traditional cybersecurity defences. Publicly available (Gen)AI platforms, such as ChatGPT or Claude, are easily accessible tools for attackers to formulate malicious behaviours without imposing significant resource requirements.
(Gen)AI empowers defenders
While we have highlighted the various ways attackers can use (Gen)AI, there are also many opportunities for defenders to fight back using (Gen)AI. Understanding how attackers may strike at key stages of the CKC allows for better insights into establishing an effective defence. However, for defenders, there is an opportunity cost in deploying (Gen)AI—resources, time, and focus must be carefully allocated to ensure that AI defences are effective and efficiently integrated into existing security measures.
Reconnaissance: Defenders can utilise AI-powered firewalls and honeypots to prevent attackers from snooping around a system and reaping information about system architecture, software, and more. AI algorithms harness insights from network traffic behaviours, autonomously generating rules and adapting to real-time, evolving threats. Fortinet's FortiGate Rugged Firewall specialises in defending OT. Using AI, the firewall continuously and automatically assesses and responds to threats, with deep packet inspection for various OT applications and protocols. Honeypots, on the other hand, can act as decoys for attackers seeking information about systems. GenAI can generate false data and reports about critical infrastructure architecture that deceive attackers. However, deploying these AI-driven defences involves an opportunity cost—investing resources in reconnaissance can divert attention and funding away from enhancing incident response capabilities. This shift may lead to slower response times during an actual attack, as defenders may lack the necessary tools or skills to effectively respond to threats had they focused on other incident response measures.
Weaponisation: While defenders cannot detect adversaries building malware, they can take steps to prevent exploitation. For example, the Dragos Platform monitors and defends ICS environments by acting as an OT security incident and event management system. Harnessing the power of AI, the Dragos Platform assesses vulnerabilities by identifying weaknesses in software or configurations. The platform also employs risk prioritisation mechanisms that evaluate the potential impact of vulnerabilities on critical processes and help organisations focus on the most critical threats. By actively working to patch vulnerabilities, adversaries may find it challenging to weaponise against the system effectively. The opportunity-cost trade-off for implementing such a proactive defence is that human resources and attention must be allocated to continuously monitor and update the system, which might otherwise be used for other operational needs.
Command and Control (C2): At this stage, network monitoring tools should be used to detect unusual traffic patterns indicative of command and control activities. One such tool is ScadaShield, which uses AI/ML to enhance ICS cybersecurity by monitoring network traffic, amongst other functionalities. ScadaShield thoroughly analyses network packets to detect OT and IT attack vectors. Additionally, ScadaShield uses ML to determine a "normal" state, during which traffic and configurations are compared; if the system deviates from this baseline, alerts are triggered. However, an opportunity cost is associated with deploying and maintaining such a tool, as human hours must be allocated to manage the system and respond to its alerts.
In summary, defenders can leverage (Gen)AI to counter (Gen)AI-powered attacks by anticipating, detecting, and neutralising threats. AI-powered tools hinder reconnaissance, detect advanced threats, and automate incident response, thereby strengthening ICS security. However, investment decisions must be made carefully, as there are costs associated with implementation, maintenance, and the need to train personnel to use the tool effectively.