As biometric data constitutes a special category of personal data, it is appropriate to classify as high-risk several critical-use cases of biometric systems.
Point 1 of Annex III AI Act lists three high-risk use cases covering AI systems intended to be used in the field of biometrics, in so far as such use is permitted under relevant Union or national law.
Use cases classified as high-risk:
1(a) Remote biometric identification (RBI) systems
- Facial/voice recognition for media archives: AI-enabled facial and voice recognition technology intended to be used to identify selected individuals (e.g., public figures) from audiovisual content by comparing that content against a reference database, such as media archives maintained by public broadcasters and national audiovisual repositories, without the individuals’ active involvement.
- (Voice)print matching in databases: AI-enabled solutions intended to be used to analyse recordings or voiceprints to identify individual speakers, by comparing their biometric data against an established database of known individuals and voiceprints without their active involvement.
- CCTV cameras installed in the walls and/or ceilings of a stadium (post remote): If an incident happens during a match, the captured biometric data (i.e. facial images) is compared to a biometric database to identify offenders, without the active involvement of visitors to the stadium.
- An AI system intended to be used to identify, from an image taken from a CCTV camera installed on private property, an individual suspected of carrying out a burglary and operating without their active involvement. Unlike the prohibition in Article 5(1)(h) AI Act, the high-risk use case listed in point 1(a) of Annex III is not limited to RBI in publicly accessible spaces.
- AI systems intended to be used to compare child sexual abuse material (CSAM) found on the Internet to an established national or international database of suspects containing facial images and tattoos to identify an offender without their active involvement.
- An investigation requiring the identification on the internet, via backwards search (reverse image searches), of a terrorist living undercover, without his or her active involvement: A reference database can be a transformation of biometric template(s) taken from information available on the website or the Internet in general.
- An AI system intended to be used to compare attributes in a picture, e.g., taken from the Internet or CCTV, with pictures taken from the Internet and indexed in a database (a reference database can be a transformation of biometric template(s) taken from information available on the website or generally the Internet) to identify the person in the picture without their active involvement.
- A CCTV facial recognition system at the entry to a stadium to detect blacklisted individuals forbidden to enter the stadium via direct access control: such a system would not fulfil the condition of remoteness, because natural persons need to actively participate, i.e., actively step in front of the AI system to get access to the stadium.
- AI systems intended to be used for biometric verification (authentication) or for identification that is not remote, such as:
- AI based solutions to unlock a smartphone.
- AI-based solutions to streamline the onboarding experience for the purposes of identifying traders.
- AI-based solutions for the authentication of online exam proctoring (so-called remote proctoring).
- AI-based solutions to streamline access to a service (e.g., logging in to a bank account online (login authentication) or calling customer service of a bank to receive information about specific transactions).
- AI-based solution to carry out verification of identity during a roadside check by law enforcement. Officer uses a mobile device to compare the person’s live fingerprint solely against the fingerprint stored on the ID card to verify the person’s identity.
- AI systems for smart homes/residential areas: An AI system intended to be used to authenticate or verify the identity of an individual to give access to a home or residential area by scanning facial images, irises, voice (voiceprint identification), or fingerprints at the entrance and which grants access after a successful check against a reference database made up of biometric data which the relevant individual agreed to register, would be considered to be for biometric verification purposes and would not be considered to fulfil the condition of remoteness (the individual agreed to register and consciously presents themselves at the entrance).
- Smart corridor AI biometrics system: an AI system that captures the facial images of individuals from several angles in a short corridor or passage and compares those images against a biometric database of persons that have registered with the system through which those persons can enter a concert, stadium, major public event at a city square, etc., without stopping, whereas unregistered persons are approached and ejected by security staff or by automatic door systems, depending on the application. Such systems are used solely to gain access to a service/premises and persons that registered would be considered to actively participate in the identification.
- AI systems intended to be used to confirm the identity of a natural person for the sole purpose of having access to a service, unlocking a device or having secure access to premises: e.g., an AI-enabled facial recognition system to contol access to hazardous areas (e.g., used in industrial settings to grant or deny access to machinery or areas with elevated safety risks); an AI-based solution to control physical access to secure areas within a corporation’s office buildings; an AI biometric system for secure facility access used in critical electricity infrastructure or seaport/river port facilities (employees/authorized personnel only).
- AI systems that make use of biometric data, but which are not intended to be used for biometric identification: e.g. an AI system that uses a voice sample collected with the individual’s active involvement to carry out language/dialect analysis in order to identify the country and region from where a person originates in the context of migration and asylum processes. The aim of such a tool is not to identify the person. Rather, a voice sample is matched against a pool of accredited voice samples to substantiate country of origin. However, see Section 3.1.3 on Point 1(b) and Section 3.7 on Point 7 of Annex III below.
- AI systems filtering seized material to structure its content: e.g. the AI system filters the seized footage, e.g., 100 000 files, in preparation for the evaluation of the seized evidence, e.g., whether there is a child present in a photo or not.
- AI systems used by law enforcement authorities to analyse crime scenes: All kinds of material are collected from a crime scene including CCTV, witnesses’ material, etc. The objective is to gain investigative insights in what has exactly happened by analysing occurrences of persons and objects (e.g. via clustering, tracking or localisation based on objects such as cars, number plates, clothes, bags, weapons, drugs, and possibly as well for faces, tattoos and birthmarks). Persons are not matched against external databases, i.e. their identity is not established; and only material that has a direct link to the crime place is being used. This is the work-step prior to initial identification, and even if there might be a comparison of faces involved, in view of the closed dataset this would not yet be subject to the high-risk classification under Article 6(2) and Annex III, point (1)(a) AI Act.
- AI systems that use multispectral imaging to capture information from the surface and sub-surface (dermis) of the skin (collecting of biometric data) to capture fingerprints without carrying out biometric matching against a reference database.
- AI systems used solely for cybersecurity and personal data protection purposes: e.g. AI systems with the sole purpose of anonymising personal data, such as biometric systems which are intended to be used solely for the purpose of enabling cybersecurity and personal data protection measures as indicated in Recital 54 AI Act.
- AI systems that use cameras (e.g., satellite, infrared/thermal) to detect the presence of humans, such as:
- AI systems that provide information on seat occupancy in trains or counting people: the AI system uses infrared thermal cameras installed on the ceiling in the train compartment; captures the temperature for each of the seats; and analyses the data and provides updates to the passenger via website/mobile application.
- AI systems that are used to detect people on tracks (trespassers) to prevent accidents and damages to the infrastructure: the AI system uses infrared thermal cameras installed around rail tracks, in front of the train, etc.; alerts driver/security team that there are trespassers.
- AI systems using images or recordings of specific spaces captured by cameras carried by drones/satellites to verify whether there are any humans present in those spaces based on non-biometric data (e.g., unusual motions, radio frequency signals emitted by the electronic device carried by humans) in the search for missing people (e.g., people lost in vast uninhabited spaces, for instance large forests). Although such AI systems can be used to identify individuals, they do not involve biometric data.
- AI systems that use infrared thermal cameras to detect the presence of humans are not classified as high-risk AI systems because their purpose is not to identify individuals, but instead to detect the mere presence of humans. In addition, such systems do not rely on biometric data.
- AI systems that do not track an individual on the basis of biometric data: e.g., law enforcement uses AI video analytics tools to track a fleeing suspect across cameras based on clothing colour and body outline, without extracting or matching biometric features.
- AI-based large data processing systems used by law enforcement to cross-check and identify links between various criminal cases on the basis of facial imagery without using biometric data to identify the common suspects.
- An AI system to categorise patients. The AI system is used to detect early symptoms of diseases that manifest themselves in mobility issues. The AI system captures patients’ gait, infers their health data based on captured gait data, and assigns those individuals to pre-defined categories (e.g., early stages, advanced stages of diseases).
- AI system to categorise passengers. The AI system is used in cameras installed at the airport to analyse passengers’ movement between terminals. The AI system captures travellers’ biometric data (e.g., gait, facial templates), infers biometric identifiers based on these captured data, and uses them to assign travellers into a category. Initial categorisation is based on inferred biometrics, e.g., ethnic origin. These categories are used, for example, to generate aggregated passenger profiles (e.g., solo passenger, passengers travelling as a family) or to support targeted operational measures.
- AI system to categorise foreign persons crossing the EU border. The AI system is used in cameras installed at border crossing points and captures facial images to infer foreigners’ age and gender. The system is not high-risk because attributes such as age and gender are not considered as sensitive or protected under Article 9(1) GDPR. If age and gender are collected for the purposes of assessing the risk posed by foreigners entering into the EU territory, the AI system could be qualified as a high-risk system under point 7 (b) of Annex III, AI Act.
- AI system to categorise customers based on their gender to offer/improve personalised experience for customers. The AI system captures keystrokes, analyses them to assign a gender to the customer to offer/improve personalized experience (e.g., personalised advertising). The AI system deploys biometric categorisation (assigns a customer to a certain category based on their biometric data). However, the AI system infers gender, which is not a sensitive or protected attribute or characteristic under Article 9(1) GDPR.
- AI systems intended to be used for age estimation. The AI system prevents minors from, e.g., (i) accessing online harmful content, (ii) accessing age-restricted services (such as online betting, personal loans, credit cards, investment platforms, mobile payment wallets), (iii) accessing physical premises not appropriate for children, or (iv) using vending machines that provide products intended exclusively for persons over a legally defined age threshold (e.g., tobacco). The AI system captures keystrokes or/and facial features through a camera and assigns individuals to a specific age group on the basis of a comparison of the captured images/keystrokes patterns with characteristics of people in certain age groups and a probability prediction. The AI system infers age, which is not sensitive or protected attribute or characteristic under Article 9(1) GDPR.
- AI system intended to be used as a content moderation tool. The AI system scans text/pictures to identify illegal or inappropriate content. Although the system categorises content based on specific indications (e.g., extreme content that may reveal political opinions and be considered as a sensitive or protected attribute or characteristic), that categorisation is not made on the basis of biometric data.
- The integration of AI with body-worn cameras or remote surveillance systems (so-called police bodycams): e.g. AI-enhanced body-worn cameras used across patrol units that enables automated behavioural pattern detection, flagging ‘aggressive posture’ through gait, body posture, movement and facial analysis to assess whether an individual is likely to engage in fight during police encounters. This qualifies as an emotion recognition system because it interprets physical cues to infer an individual’s emotional or mental state - in this case, anger.
- An AI system for the gaming industry that is intended to measure a gaming experience to further improve the product (real-life gaming experience). The AI system tracks body posture, facial expressions, eye closures and gazes to measure the reaction of the player, including their excitement, anger, frustration, amusement; analyses the above-mentioned recorded data to identify relevant elements of the gaming experience that needs improvement. Such a system involves processing of biometric data (e.g., facial expressions which are a source of biometric data). It also identifies or infers gamers’ emotions based on their biometric data. Excitement, anger, frustrations are examples of emotions.
- An AI-based solution to maintain order during concert events. The AI system is installed in cameras at walls or ceilings; screens the mood of the audience, e.g., voices, faces and movements; when the system finds that the mood is getting aggressive in a certain area of the stadium, more security personnel is sent to that area. The AI system is intended for emotion recognition: it screens emotions of individual participants to assess the mood of the audience.
- An AI system used in call centres to infer emotions of customers. The AI system analyses customers’ voices and evaluates vocal tone, pitch and volume to gauge customer satisfaction level for statistical reasons, to identify the moment to route them to a human agent (e.g., anger), for troubleshooting purposes, or to enhance customer care business unit. These AI systems deduce from the customer’s voice their emotions, e.g., if they are happy or angry. It does not matter whether the biometric system is intended to identify the customer.
- Smart watch mood monitor/wearable device. The AI system is integrated into a smart watch to measure biometric data (e.g., voice, heart rate) and monitor user emotions (e.g., sad, curious, happy, bored) to assist users in recognising their emotional state and provide emotional improvement suggestions. Such AI systems infer user emotions, which is intrusive. It is not relevant whether the emotion recognition results are disclosed solely to the specific user.
- An AI system intended to be used by the automotive industry to help prevent accidents by detecting when a driver loses focus (i.e., drowsiness, falling asleep, experiencing sudden health issues). The AI system tracks body posture, facial expressions, eye closures and gazes via installed cameras and motion detectors in a vehicle; analyses the collected data to alert the driver or assume control of the vehicle, and safely bring it to a stop. Such AI systems are not considered high-risk AI systems because emotions or intentions do not include pain or fatigue, which are physical states.
- The observation of readily apparent expressions. The mere observation that a person is smiling is not emotional recognition. A TV broadcaster using a device that allows to track how many times its news presenters smile at the camera is not emotion recognition.