Law Enforcement

Domestic violence risk assessment systems

• Domestic violence risk assessment systems intended to be used to analyse victim statements, police reports, previous incidents, restraining order data, social services records and other data to determine the level of risk faced by a (potential) victim of domestic violence should be considered to fall in the use case listed in point 6(a) of Annex III. Such AI systems are a classic example of a tool that is used by law enforcement authorities to assess the risk of a natural person becoming the victim of a criminal offence. They influence critical interventions, including police protection measures and legal actions, which can either prevent harm from arising or, if flawed, leave victims vulnerable to further abuse. The use of such AI systems involves profiling, so that they cannot benefit from the filtering mechanism laid down in Article 6(3) AI Act.
• An AI-based risk assessment system intended to be used by judicial authorities in criminal court proceedings to estimate the probability of severe or repeated domestic intimate partner violence, for instance when issuing or maintaining restraining orders. As such risk assessment involves profiling, the exemption under Article 6(3) AI Act cannot be applied. 

Vulnerability exploitation risk assessment systems

• Human trafficking vulnerability detection systems intended to be used to assess a risk of an individual being trafficked, especially in cross-border, labour exploitation or migration contexts, or in the case of young individuals from particularly vulnerable backgrounds, should be considered to fall in the use case listed in point 6(a) of Annex III. Such systems aggregate indicators such as age, lack of valid documentation, traveling with an unrelated adult, signs of control by others, and links to high-risk sectors, to produce a score indicating the likelihood of future human trafficking. Those systems involve individual risk assessment and have potential consequences for fundamental rights. The use of such AI systems involves profiling, so that they cannot benefit from the filtering mechanism laid down in Article 6(3) AI Act.

AI systems predicting location-focused risks

• AI systems intended to be used to analyse the risk that a specific environment/location will see offending and victimisation (i.e., people in certain areas will become victims of crime) should be considered to fall outside the use case listed in point 6(a) of Annex III. Being environment/location-focused, such a system does not explicitly assess risks to specific persons (victims) except by virtue of them being in a risky area. However, if the system is intended to be used to predict the potential risk of a specific person or persons committing a crime in that area, such a system would fall within the use case listed in point 6(d) of Annex III.
• Situational awareness and risk assessment AI systems intended to be used to support law enforcement during the initial stages of responding to a reported crime scene should be considered to fall outside the use case listed in point 6(a) of Annex III. Such a system can be deployed via a robot or drone, and is designed to process visual and contextual inputs to detect critical elements, such as presence of armed individuals, weapons or environmental hazards. Since the aim of the system is to evaluate the immediate environment, rather than assessing the risks related to specific individuals based on personal or contextual data, it should not be classified as high-risk on that basis alone. However, if the system also had biometric capabilities, it could fall within the use case listed in point 1 of Annex III (see Section 3.1. above).

AI systems predicting risks of accidents or administrative breaches

• AI systems intended to be used to identify road segments or times where accidents are likely to occur, thus where people are at risk of becoming victims of road traffic accidents, should be considered to fall outside the use case listed in point 6(a) of Annex III. For such a system to fall within that use case, it should be intended to be used to analyse the risk of becoming the victim of criminal offences and not of accidents. Moreover, such systems are location-focused and do not analyse risks linked to concrete persons.
• AI-based video analytics intended to be used to predict overcrowding, stampedes, or structural hazards at large gatherings should be considered to fall outside the use case listed in point 6(a) of Annex III. 
• AI flood-risk and wildfire-spread models intended to be used to identify populations at risk of becoming victims of accidents/catastrophes should be considered to fall outside the use case listed in point 6(a) of Annex III. See also Section 3.6.5. on the use case listed in point 5(d) of Annex III.

• AI systems intended to be used to analyse facial micro-expressions to assess the credibility of answers during an interrogation. Likewise, AI systems measuring eye-tracking and pupillary responses while a suspect answers questions. The role of such systems in decision-making and profiling excludes them from benefitting from the exceptions listed in Article 6(3) AI Act.

• AI systems intended to be used to monitor police officers’ fatigue or stress levels (via wearable sensors or facial analysis) for occupational health and safety reasons. Such systems should also be considered to fall outside the use case listed in point 1(c) of Annex III, as further clarified in Recital 18 AI Act (see Section 3.1.4); being intended to be used for medical and safety reasons, they are also excluded from the prohibited practice listed in Article 5(1)(f) AI Act.
• AI systems intended to be used by law enforcement authorities that transcribe interviews and flag changes in tone (e.g. higher tone, slower/faster speaking pace), without drawing conclusions about the truthfulness of statements made. It should, however be checked whether a particular use case would not amount to emotion recognition system and thus fall under point 1 of the Annex III of the AI Act.
• AI systems that flag irregularities in how different polygraph examiners interpret similar cases, highlighting inconsistencies for law enforcement authorities. Such system is not a polygraph itself, but is intended to be used to interpret the work of polygraph examiners.

Authenticity verification

• AI-enabled digital forensics solutions for image, audio and video analysis intended to be used by law enforcement authorities to detect whether digital images, audio recordings or videos have been altered or manipulated (e.g. to confirm the authenticity of CCTV footage or smartphone recordings that were presented as evidence; to verify if collected pictures are not deepfakes). 

Data integrity checks and source reliability assessment 

• AI systems intended to be used by law enforcement authorities to determine and evaluate alterations of documents , or forged signatures.
• AI systems intended to be used for evaluating the authenticity of digital evidence retrieved from mobile or other devices, including by analysing digital traces (e.g. metadata such as geolocation records, file creation and modification histories) and by flagging elements requiring closer human examination.
• AI systems intended to be used to assist competent law enforcement authorities in assessing the reliability of evidence, including from human sources, such as the credibility of informants or witnesses, by analysing factors such as the internal consistency of statements, their coherence with verified facts, and the accuracy of information provided by the same source in past cases.

Image enhancement and data recovery

• AI systems intended to be used for forensic reconstruction, image enhancement, or data recovery from devices. Such systems do not evaluate the reliability of evidence, but assist human experts by processing, restoring, or extracting data that can then be examined and relied upon. 

AI systems reconstructing crime scenes

• AI systems intended to be used to map or reconstruct crime scenes or to provide visualisations or models based on data collected (photos, measurements, etc.). Such systems help investigators better understand the spatial relationships and sequence of events at a crime scene without assessing evidence reliability.

Pattern detection

• AI systems intended to be used to detect patterns in investigations and notify law enforcement authorities thereof (e.g. identifying similar break-in methods in recent thefts or notify of similar victim profile in recent homicides). Such systems do not evaluate the reliability of evidence, but find correlations or patterns that might otherwise go unnoticed and help law enforcement focus their investigations or spot potential links between cases.  

Content flagging and searches

• AI systems intended to be used to detect child sexual abuse material online, without evaluating the reliability of it as evidence. The primary function of such a system is to identify, flag and filter content that potentially violates the law. 
• AI-enabled solutions intended to be used by law enforcement authorities for screening child sexual abuse material online and the classifying thereof in the following groups for prioritisation: (i) new material; (ii) material already registered in the database; and (iii) deepfake pictures. Such systems do not evaluate the reliability of evidence, but are designed to support investigative workflows by filtering or preselecting large volumes of incoming material. The application of Article 6(3) of the AI Act in this use-case is without prejudice to the potential qualification of the AI system at issue as high-risk system under other use cases listed in Annex III of the AI Act (e.g. point 6(a) of Annex III).
• AI systems intended to be used by law enforcement authorities to analyse online advertisements in the sex industry and to flag those most likely to be linked to human trafficking. Such systems do not evaluate the reliability of evidence, but are designed to identify, flag and filter content that potentially violates the law. 
• AI systems intended to be used to match the ballistic markings on a bullet to markings registered in a reference database for the purposes of identifying whether a particular weapon was used for committing a crime. Such systems do not evaluate the reliability of evidence, but constitute a means of file handling or of searching a database by evaluating the similarities between evidence material and a reference database.

• AI systems that are used in the course of evaluation of evidence but only classify documents, pictures, videos or other material selected to be analysed for authenticity into categories and thus structure the evaluation of evidence where that categorisation does not impact the assessment of the reliability of those documents as evidence. Such systems fall under the exception for AI systems intended to perform narrow procedural tasks in Article 6(3)(a) AI Act.
• An AI system that is used in the course of evaluation of evidence and is indexing and tagging financial documents collected to be evaluated for reliability of evidence (e.g. to be checked with bank account flows;  or to be checked for existence of names or objects). Such system fall under the exceptions for AI systems intended to perform narrow procedural or preparatory tasks in Article 6(3)(a) and (d) AI Act.

AI systems used when processing detained persons to assess the risk of offending/reoffending

• An AI system intended to be used during the initial processing of detained youth suspects by police to calculate a risk score that indicates the likelihood of reoffending, which is then used to make a decision on measures to be taken. The AI system does not autonomously make decisions; the outcome is considered alongside other information, and final decisions are made by the competent authorities. The AI system is intended to be used in support of law enforcement authorities for assessing the risk of a natural person re-offending and this assessment is done not solely on the basis of the profiling of natural persons as referred to in Article 3(4) LED, or to assess personality traits and characteristics or past criminal behaviour of natural persons or groups, as there are verifiable facts directly linked to a criminal activity (the assessment is performed post-detention).

AI systems used by probation officers

• An AI system intended to be used by probation officers to support assessments informing decisions on parole or conditional release, by evaluating factors such as criminal record, behaviour and progress during the custodial sentence, and other case-relevant supervision-related information. Like the example above, there are verifiable facts directly linking a person to a criminal activity (the person having already been convicted of a criminal offence).

AI systems used by penitentiary institutions 

• An AI system used by a penitentiary institution to predict the likelihood of an offender re-offending based on prior criminal record, violence within the prison facilities, or socio-economic data. Like the examples above, there are verifiable facts directly linking a person to criminal activity (the person having either already been convicted of a criminal offence or being detained on suspicion of having committed such an offence).

AI systems used by criminal courts to assess the risk of offending/reoffending

• An AI-based risk assessment system intended to be used in criminal courts to estimate the likelihood of recidivism, violent reoffending, or the commission of further serious offences. Since such a risk assessment system involves profiling, the exceptions of Article 6(3) AI Act do not apply.

Location focused AI-systems

• AI systems predicting locations and timeframes of likely crimes (e.g., burglary risk in a neighbourhood), without linking the prediction to identifiable individuals. The use case in point 6(d) concerns risk assessments of natural persons, not geospatial predictions.

• An AI system that checks whether all required fields are completed in a probation officer’s risk assessment form or flags inconsistencies in entered data (e.g., mismatched dates of offence or custody period). Such an AI system performs a preparatory task for the risk assessment and does not evaluate personal characteristics . It is also intended for a narrow procedural task (Article 6(3)(a) AI Act), since it only flags inconsistencies and would in any event fall under the exception for AI systems intended to perform a preparatory task listed in Article 6(3)(d) AI Act.
• An AI system that collects and pre-sorts prior convictions, probation reports, and other records to prepare a case file for human officers. Such a system performs a preparatory task for risk assessment and therefore qualifies for an exemption under Article 6(3)(d) AI Act.

AI systems intended to be used in assisting the identification of potential suspects 

• AI systems assisting in identifying potential suspects where a child is reported missing that analyse the neighbourhood, persons living in the neighbourhood that are on the sex offenders’ registry and their movements, as well as types of children they have abused earlier. The purpose of the AI system is to generate individualised risk inferences from behavioural patterns, thereby filtering persons as potential suspects. Such a system would involve profiling, since it performs the automated processing of personal data to evaluate and predict aspects of natural persons’ behaviour. Where the assessment of potential involvement in criminal activities is supported bypast criminal behaviour and other verifiable facts, the system would not be solely based on profiling and assessing personal characteristics. It therefore falls within the use case of point 6(e) of Annex III AI Act, rather than the prohibition on predictive policing laid down in Article 5(1)(d) AI Act.
• AI systems assisting in identifying potential suspects based on descriptions of witnesses and other personal data. Such an AI system processes personal data to infer identity-linked traits and other personal aspects based on patterns, and to single out individuals as potential suspects. 

AI systems intended to be used for targeted online monitoring

• An AI system that monitors online posts to classify users based on sentiment analysis, posting patterns, and networks, and assigning flags for potential extremist affiliation. The system automatically analyses communication patterns, language use, and behavioural indicators to classify persons as potential radicalised subjects. The system performs profiling, since it evaluates and predicts natural persons’ characteristics and likely future behaviour for law enforcement purposes. This does not amount to social scoring as prohibited by Article 5(1)(c) AI Act, since the system is limited to targeted radicalisation monitoring within specific spheres or contexts.

• AI systems analysing neighbourhoods and assigning criminality scores (predictive crime mapping systems) where they do not include profiling of individuals as per points 6(d) and (e) of Annex III.
• AI systems detecting suspicious transaction patterns that indicate money laundering or terrorist financing, even though that allows linking such transaction to personal profiles. However, borderline cases may arise where, in addition to focusing on transactions only (the structural, quantitative, or behavioural transactions data, e.g., transaction amounts, frequencies, intervals, geographic routing), such systems are combined with, or enriched by, personal data, thereby enabling the construction of individual profiles (e.g., names, account numbers).If the system assesses such personal characteristics, it should be classified as high-risk.
• An AI-enabled crime-linkage system analysing police crime reports and case files containing variables, such as time, place, modus operandi, suspect/victim descriptions, physical traits, clothing, vehicles, and socio-demographic indicators, with the purpose to identify patterns and similarities across different incidents, linking them into possible ‘series’ committed by the same offender(s). Its main outputs are alerts, crime link hypotheses, and forecasts of likely future offences (not linked to a particular individual). 

• AI systems used for automating administrative tasks (e.g., administration and accounting, case management, equipment management, AI systems scheduling patrol shifts). Such AI systems are intended for purely ancillary administrative activities and therefore do not fall within the use cases listed in point 6 of Annex III.
• Chatbots on police websites providing general information, guiding users through the website. Such AI systems are intended for purely ancillary administrative activities and therefore do not fall within the use cases listed in.
• Automated number plate readers seeking to identify concrete vehicles. Such AI systems are not considered as biometrics within the meaning of point 1 and do not involve profiling withing the meaning of point 6 of Annex III, since the technology itself is vehicle-focused, not person-focused. However, how the data is used, stored, and interpreted can, at a later stage, create risks of surveillance overreach or privacy violations (e.g. if enabling the reconstruction of individuals’ travel patterns).
• AI systems checking whether road safety rules are broken (e.g. speeding cars, running red lights, motorbike riders failing to wear a helmet, smartphone use while driving). Such AI systems do not fall within the use cases listed in in point 6 of Annex III, nor do they constitute biometrics within the meaning of point 1 of Annex III.
• AI systems intended for the detection of objects (e.g. weapons, suspicious parcels, dangerous objects, stolen goods including cultural objects and fine art). Such AI systems do not fall within the use cases listed in point 6 of Annex III, nor do they constitute biometrics within the meaning of point 1 of Annex III.
• AI systems intended for the detection of unusual behaviour or movements, such as running, raised arms, a fighting stance, a physical position that could indicate someone is holding a weapon, etc., real-time or in video feeds (not identifying a person, but whether a process or behaviour is potentially unlawful).
• AI systems detecting gunshot sounds in real time. Such AI systems do not fall within the use cases listed in in point 6 of Annex III, since they function based on the detection of sounds and not people.
• AI systems used by custom authorities to assess consignments (goods crossing the border). Custom authorities checking consignments normally do not classify as law enforcement authorities within the context of point 6 of Annex III. In addition, those systems evaluate the compliance of goods with EU legislation based on verifiable data (e.g. container number, description of goods, routing, transport, payment method), not personal characteristics or behaviour or persons. In certain cases, such AI systems may also process information about the prior involvement of the importer or exporter in irregularities related to the import of goods, their affiliation to criminal organisations, or a criminal record for drug trafficking. Such systems, as noted in the Guidelines on prohibited artificial intelligence practices (para. 214), fall outside the scope of the prohibition in Article 5(1)(d)AI Act because any prediction of the likelihood of a natural person being involved in the import or export of illicit goods is not solely based on profiling, but on objective and verifiable information related to the goods and the importer or exporter’s prior involvement in criminal activity and subject to a human review to determine whether the situation requires a customs control or risk mitigation action.
• AI systems used by customs and/or law enforcement authorities to analyse data related to the movement of goods and transport operations, such as shipping routes, frequency and patterns of consignments, anomalies in declared cargo, transport modalities, etc., to detect misuse of commercial transport systems to transport trafficked goods. Such AI systems do not fall within the use cases listed in point 6 of Annex III, since they do not seek to predict the likelihood of a natural person committing an offence or to profile a natural person. Rather, they review objective and verifiable data relating to the commercial transportation of goods to determine an anomaly that may require further investigation.