AI systems intended to be used as safety components in the management and operation of critical digital infrastructure, such as road traffic, or the supply of water, gas, heating or electricity, are considered high-risk as their failure or malfunctioning could put people's lives and health at risk and lead to appreciable disruptions in social and economic activities. For the horizontal conditions on safety components in critical infrastructure, see draft Guidelines.
Use cases classified as high-risk:
2(a) The critical digital infrastructure use case
- An AI system used as a fire alarm controlling system in cloud computing centres, due to its direct safety function (Recital 55 AI Act). None of the exceptions listed in Article 6(3) AI Act apply to such a system due to its role in preventing direct damage to the physical infrastructure of the centre, as well as harm to natural persons.
- AI systems intended to be used to improve the service quality and operations of critical digital infrastructure , e.g.: AI systems used for trouble tickets management; AI systems intended to be used to optimise the network and adapt to tailored customer needs; AI-based interactions with technical user guide for network maintenance; AI systems intended to be used to predict a network load. Such AI systems should not be considered to have a direct safety function.
- An AI system intended to be used as a real-time translation tool. Such a system is necessary for transport service employees to overcome language barriers between the control centre and drivers or between passengers and drivers, which ensures linguistic communication and contributes to the safety of public transport operations.
- An AI system intended to be used to monitor road traffic and to adjust traffic lights accordingly. Such a system controls or limits possible physical harm to people or physical damage to vehicles, road users and surroundings by directly protecting the physical integrity of road traffic management system. It should therefore be considered to have a direct impact on road traffic safety.
- An AI system intended to be used in the recognition of heavy objects on vulnerable bridges and quaysides. Such a system helps prevent the collapse of bridges and quay walls. It prevents physical harm to natural persons and physical damage to objects under a bridge or a quayside by directly protecting the physical integrity of road infrastructure dependent on the road traffic management system. The failure of the AI system could result in potential collapse not being detected on time and natural persons on the bridges and quay walls being at risk when they collapse. The AI system is not needed for managing bridges and quays and is an additional safety component, comparable to an emergency button.
- An AI system intended to be used to link real-time water level data to lock management for road traffic (e.g. in flood-risk scenarios). Such a system monitors and detects situations that may directly lead to physical harm to natural persons or physical damage to property by directly protecting the physical integrity of the road infrastructure dependent on the road traffic management system.
- An AI system intended to be used as a control system capable of communicating with road users via telecommunication networks (Intelligent Traffic Control System - iTCS). Such a system can recognize traffic, perform analyses on received data, and make decisions based on these analyses. This enables the prioritisation of certain road users over others; a key advantage being improved traffic flow on busy roads. Through data exchange with iTCS, road users can also be informed about how long a traffic light will remain red or green. Such an AI system prevents accidents (e.g., head-on collisions if two traffic flows cross paths, chain-reaction accidents in congestion), its malfunction could cause gridlock, the obstruction of emergency services, (e.g., ambulances) or an increased likelihood of accidents at busy junctions, and such failures create direct and immediate risks to both infrastructure integrity (damage to signals, road surfaces, vehicles, etc.) and health and safety of natural persons (fatalities and injuries).
- An AI-enabled traffic flow optimisation system based on real-time data collection. Such a system is a traffic data analytics platform enabling intelligent analysis for flow optimisation functions, but does not directly trigger changes in the traffic management that may impact the safety; it does not directly protect safety or physical integrity. These systems provide insights but do not directly protect physical integrity. Traffic systems can function without it.
- An AI system used for predictive maintenance which allows for more accurate predictions regarding infrastructure maintenance as part of a range of safety measures. Such an AI system merely helps prevent failures; it is not a safety component and malfunctioning of the system would not endanger the safety as the system can function without it as it is only one measure that is precautionary in nature among other safety safeguards and not essential for the safe operation of the system.
- An AI system intended to be used as a pressure sensor in water pressure monitoring systems. Such an AI system fulfils the definition of a ‘safety component’ and it is intended to be used in the distribution of drinking water, which is covered by the supply of water use case (Recital 55 AI Act).
- An AI system intended to be used to predict sewage system overflow that impacts drinking water. Such an AI system fulfils the definition of a ‘safety component’ and it is intended to be used in the distribution of drinking water, which is covered by the supply of water use case.
AI system used as a predictive maintenance tool for gas pipeline monitoring by analysing operational data to predict maintenance needs. Such AI system does not directly control safety functions and existing safety systems remain independently operational.
AI used in autonomously moving robots that perform regular, predefined patrols and checkpoints in heating (power) plants. Such an AI system can only be used for detection purposes. It is not capable of taking preventive measures, actively intervening in operations, rectifying malfunctions, or averting dangers. It therefore lacks a safety function.
An AI system used in a surveillance and protection for physical perimeter protection such as camera systems, radar systems and drone control systems used to directly protect the physical integrity of the infrastructure.
An AI system used for the detection of anomalies in data patterns when operating electricity grids for the purpose of monitoring and supporting decision-making in relation to critical functions, such as power load distribution, grid stability, or shutdown procedures.
- An AI system used for electricity grid optimization by forecasting the energy demand. The core safety functions are handled separately, so that the system cannot be considered to have a direct safety function.
- An AI system used in a cybersecurity monitoring for energy grid networks. Such a system is intended to be used solely for cybersecurity purposes. It continuously monitors network traffic, detects anomalous patterns, and identifies potential cyber threats across electricity grid communication networks. It analyses data flows, identifies suspicious activities, and alerts security personnel to potential intrusions or malware. It operates independently from operational technology systems that directly control physical grid operations.
- An AI system used as a quality assurance of meter installation. Such an AI system analyses the images of installed electricity meters. The output of the system is provided to the human worker with recommendations regarding errors and improvements. It does not have a direct safety function.
- An AI system used as an incident detection for e.g. precautionary outage prevention. The output is the detection of anomalies and the prediction of impending outages or incidents. In order to prevent them from happening the AI system is sending the grid control centre a warning of the detected or foreseen incident as part of a range of safety measures. The grid control centre operator can then take precautionary measures like switching loads to prevent an outage. Such AI system is not considered as a high risk due to lack of a direct safety function as it is only one measure that is precautionary in nature and not essential for the safe operation of the system. It does not act on itself but rather only enriches the decision-making options in the grid control centre. Even if the system wrongly indicates impending outages or incidents, no outage is created.
- An AI system used for detection and analysing anomalies in data provided from operating the power grid, where the output e.g. is the basis for future improvements in operating the power grid. An AI system that is used for detection of anomalies in data patterns when operating electricity grids so that they are checked by the human operator exists to enrich the decision-making process. It lacks a direct safety function.
- An AI system used for providing recommendations as an assistant, predicting energy consumption at transmission, regional or local level. Such an AI system lacks a direct safety function.
- An AI-system, supervised by a human worker, makes forecasts for grid imbalances, e.g. 15-minutes interval. That forecasts support operational planning, but also directly influence market players. A malfunctioning in the AI-system could mislead both external and internal actors, potentially causing suboptimal dispatch or hedging strategies. This can affect grid stability, which can lead to malfunctioning in electricity supply. This will not be classified as a high-risk system due to lack of a direct link between the system and potential harm.