Article 6(1) AI Act sets the classification rules for AI systems intended to be used as a safety component of a product or that are products covered by the Union harmonisation legislation listed in Annex I AI Act (‘regulated products’).
Article 6(1) AI Act can be understood as laying down two cumulative conditions that must be satisfied for an AI system to be classified as high-risk pursuant to that provision.
- The AI system must be intended to be used as a safety component of a product, or the AI system itself must be a product, covered by the Union harmonisation legislation listed in Annex I.
- The product whose safety component is the AI system or the AI system itself is required to undergo a third-party conformity assessment.
Safety components
Article 3(14) AI Act defines a ‘safety component’ as ‘a component of a product or of an AI system which fulfils a safety function for that product or AI system, or the failure or malfunctioning of which endangers the health and safety of persons or property’
Safety function
An AI system fulfils a ‘safety function’ where its intended purpose, as determined by the provider, is to prevent or mitigate risks to health, safety or property.
Preventive functions
- Monitoring and detection of situations which may lead to physical harm or damage to people or property (e.g. AI system detecting abnormal system behaviour);
- monitoring and detection of a need for maintenance or inspections, where failure to perform such actions may lead to physical harm to people or property (e.g. an AI system detecting whether safety-related parts, or parts whose failure may lead to harm to people or property , are worn down and may need replacement or maintenance);
- prevention of physical harm to people or property (e.g. an AI system preventing the system from starting up if abnormal behaviour is detected);
- supervision of another system that performs a safety function (e.g. an AI system that supervises through sensors an operation in real time of a safety component that directly performs the safety function).
Mitigation functions
- Control or limitation of physical harm to people or property (e.g. an AI system controlling specific behaviour or the functioning of a system and adjusting its function accordingly);
- mitigation of consequences of possible physical harm to people or property (e.g. an AI system that triggers action, such as safe stop if dangerous conditions arise);
- control of another system that performs a safety function.
Concrete examples
• Machinery: an AI based computer vision system that detects human presence in a robot cell and triggers a safe stop or speed reduction where the intended purpose is to prevent injury;
• ATEX equipment: an AI system that intends to monitor gas concentrations and command shutdown;
• Pressure equipment: an AI system intended to predict runaway pressure and actuate protective measures, such as triggering shutdown linked to safety accessories;
• Rail interoperability: an AI system in a train intended to monitor speed limits and to prevent collisions and derailments.
All other AI system functions that are not intended by the provider to prevent and mitigate safety risks do not fall within the notion of ‘safety function’. This includes functions related to performance optimisation, service efficiency, automation, comfort, convenience, or quality control operations of non-safety related aspects. By way of example, the following functions are not in themselves safety functions:
- optimisation of product performance (e.g. efficiency or user preference optimisation) where failure would not directly lead to risks to health or safety of persons or property;
- optimisation of service efficiency or service or optimisation of the functioning of a product (e.g. AI systems for billing controls or optimising customers claims processing);
- quality control of non-safety related functions or services (e.g. AI systems for quality-of-service monitoring).
Failure of malfunctioning endangering health and safety of persons or property
Under the second scenario, an AI system constitutes a safety component of a product where its failure or malfunctioning could endanger the health and safety of persons or property.
- Lifts: an AI system managing door closing timing and obstacle detection. Although the provider’s intended purpose for such a system may be efficient lift operation, a malfunctioning of the system could cause injury and thus endanger health and safety of persons;
- Vehicles: an AI system for lane assistance. While a provider’s intended purpose for such a system may be the enhancement of the user experience, a malfunctioning of such a system, if it steers unexpectedly to cause a collision, could cause injury and thus endangers health and safety of persons and property.
- Agriculture: an AI system targeting the areas of agricultural land for spraying chemicals. While a provider’s intended purpose for such a system may be to optimise the use of chemicals, not adequately accounting for the presence of nearby persons could endanger the health and safety of persons.
Not safety components
These systems are not safety components because they neither fulfil a safety function, nor would their failure or malfunctioning endanger health, safety or property.
• Toys: an AI system that recommends music in a connected toy. Such an AI system neither fulfils a safety function, nor would its malfunction endanger health, safety or property.
• Agriculture: an AI system integrated into drone or robot and used for certain agronomic purposes, such as yield forecasting or irrigation optimisation. Such an AI system neither fulfils a safety function, nor would its malfunction endanger health, safety or property.