Administration of justice and democratic processes

The AI Act identifies the administration of justice and democratic processes as one of the areas in which certain use cases of AI systems can pose significant risks to health, safety and fundamental rights. This is explained by their potentially significant impact on democracy, the rule of law, individual freedoms as well as the right to an effective remedy and fair trial (recital 61).

Use cases classified as high-risk:

8(a) AI systems intended to be used to assist judicial authorities or in alternative dispute resolution

Judicial decision/judgment drafting systems

An AI system that analyses the factual circumstances of the case, submissions received, identifies applicable law and case law and then can generate drafts of judicial decisions/judgments or their parts, in particular the reasoning part and decision. Such an AI system falls within the use case of point 8(a) of Annex III, since it is intended to assist a judge in researching and interpreting facts and the law and also in applying the law to a concrete set of facts. Fulfilling one condition would be sufficient; however, in this case, both conditions are satisfied.
An AI system that generates (draft) decisions in small claims cases (e.g. dispute values under a certain amount) or orders for payment (when there is no dispute that a certain amount is owned in a civil proceedings), relying on structured case data and legal templates. Such a system falls within the use case of point 8(a) of Annex III, since it is intended to assist a judicial authority in researching and interpreting facts and the law and in applying the law to a concrete set of facts. Due to its proximity to the decision-making and impact on the decision, the system does not qualify for any exception listed in Article 6(3) AI Act.

AI systems intended to be used for repetitive cases

An AI system that is intended to support judges in handling identical or similar cases by (i) extracting relevant facts from the claim and clustering them based on recurring factual constellations across similar cases; (ii) researching a database of previous decisions to find similar cases and identify legal reasoning patterns; (iii) preparing suggestions and corresponding text modules for the final decision, based on outcomes of similar previous cases. Such an AI system would fall under point (8)(a) of Annex III AI Act. This follows because, for the purposes of high-risk classification, the system must be assessed in its combined configuration, the joint outputs of which may materially influence the outcome of the individual decision, rather than by isolating modules that, if considered separately, might either fall outside point 8(a) of Annex III or satisfy one of the conditions set out in Article 6(3). An AI system that solely performs the activity under (i) would be intended to perform a preparatory task within the meaning of Article 6(3)(d) and could thus qualify for an exemption under the filter.

AI systems selecting relevant precedents and law for a concrete set of facts

An AI system that analyses the presented facts of the case (or their summary), then assists judges in identifying relevant laws and legal precedents and suggests how they apply to the facts of a case. Such an AI system directly assists in the core judicial task of interpreting and applying the law to a concrete set of facts and therefore falls within the use case of point 8(a) of Annex III. Due to its influence on decision-making, the system cannot benefit from any exception listed in Article 6(3) AI Act. However, for an AI system to ‘assist in applying the law to a concrete set of facts’, the system should select precedents and applicable law by analysing the presented facts. If the system simply performs AI-enabled searches of a database on the basis of the general area of law involved (e.g. family maintenance cases, financial crime, or attempted homicides) or keywords, such a system should be regarded as a search engine and therefore cannot be seen as intended to assist a judicial authority in researching and interpreting facts and the law, or in applying the law to a concrete set of facts.

Speech-to-text systems

An AI speech-to-text system used by a court to transcribe audio recordings of court proceedings and hearings, which then become part of the file. The system has the following functionalities: (i) it can take various audio file formats (or live audio) and produce textual transcripts, including automated speaker separation (distinguishing different voices) and insertion of punctuation;(ii) it can correct terminology – for example, recognising legal terms or names and adjusting the vocabulary accordingly.
While the transcription of court hearings that become part of the file could be considered to assist a judicial authority to access testimonies of witnesses and parties to a proceeding, such a tool does not assist in researching and interpreting facts and the law. The fact that recordings or transcripts generated by an AI system may later form part of the evidentiary record in judicial proceedings, and that inaccuracies may affect fundamental rights, does not in itself mean that the system is intended to assist a judicial authority in researching and interpreting facts and the law within the meaning of point 8(a) of Annex III. In this case, the system’s intended purpose is limited to technical recording or transcription without analytical or evaluative functionalities.
The same would remain true if such an AI system is later used as technical assistance in information retrieval to help interrogate the court transcript (e.g., requesting to search "when was topic X discussed” in order to find the location in the original audio file). That conclusion could be different where voice-to-text transcripts are further processed by an AI system to summarise or select the key arguments or produce further evaluations of relevance for the judicial decision-making, including when resulting in recommendations to judges.

AI systems facilitating communication with the public

AI-enabled chatbots offered on the websites of courts to assist visitors to the website in finding relevant information. Such systems function as virtual assistants, and are designed to provide information to individuals about procedural steps, required documents, deadlines, and available remedies. They do not assist judicial authorities in researching and interpreting facts and the law or in applying the law to a concrete set of facts.
An AI system designed to assist judicial authorities in drafting press releases or legal summaries on public interest cases. The system translates the technical language of a judgement into accessible summaries. Such a system performs ancillary administrative activities that do not affect the actual administration of justice in individual cases.

Case assignment systems

An AI system assigning cases to judges taking into account their specialisation, workload, or holiday schedule. Such a system is intended to perform ancillary administrative activities that do not affect the actual administration of justice in individual cases. The evaluation could be different if the AI system also performs judicial assessments of a specific case regarding procedural prerequisite, such as whether a case should be heard in public session, whether the court has jurisdiction, or whether the case should be heard by a single judge or a panel.

Processing and management of evidentiary submissions in judicial proceedings

An AI-enabled system intended to assist a judicial authority in managing and searching large volumes of case evidence more efficiently. The functionality of the system includes: (i) providing a chronology of certain events that are documented in case evidence where the AI reconstructs timelines of events as documented across different pieces of evidence; (ii) searching and providing a reference to content within case evidence. Such an AI system is intended to assist a judicial authority in the organisation and access to factual material, without analysing, evaluating or drawing conclusions from that material. Such a system is not intended to assist in researching and interpreting facts and the law, nor in applying the law to concrete set of facts.

Other ancillary administrative activities

AI systems intended for purely ancillary administrative activities that do not affect the adjudication of individual cases, such as the anonymisation or pseudonymisation of judicial decisions, documents, or data; internal communication between personnel; searching for delivery addresses and dispatching documents; verifying the payment of court fees; processing electronic signatures; verifying powers of attorney; and other similar administrative tasks.

AI systems assisting in pre-classification of incoming applications or claims

An AI system that analyses the content of the application or claim received by a court, as well as prior case law patterns to pre-classify the type of the case (e.g. contract law, inheritance case, etc.). In principle, such an AI system could be considered to assist in researching and interpreting facts and the law and thus fall within the use case of point 8(a) of Annex III. However, such an AI system would benefit from the exceptions for AI systems that are intended to perform a narrow procedural task or a preparatory task to an assessment listed in Article 6(3)(a) and (d) AI Act respectively. If the system also makes recommendations regarding the admissibility of the case (unless only technical checks), even if this is later reviewed by court staff, that system cannot benefit from those exceptions since it no longer can be considered to perform a narrow procedural task, due to the substantive assessment involved, nor a mere preparatory task, due to the proximity and impact on the final decision-making.

Metadata data extraction

An AI system performing data analysis for the purpose of metadata extraction based on the facts of the file (e.g. extraction of procedural roles from the claim (parties, legal representatives, witnesses, experts, etc.) to support clerical work such as summons. Such a system would not normally be considered to be intended to assist in researching and interpreting facts and the law. Even if certain functionalities could lead to such a conclusion, the system would in any event benefit from the exception for AI systems intended to perform a narrow procedural task listed in Article 6(3)(a) AI Act.

Advanced search engines

An AI system integrated in a public database of national case law used by public and by court personnel for searching for decisions using classical methods (Boolean operators, search through keywords, relevant legal basis, etc,); the creation of summaries of the published decisions and summaries of search results; and searching based on natural language (NLP). While such an AI system is intended to assist in researching and interpreting the law (inter alia judicial authorities) it is carrying out only narrow procedural tasks.

Language editing assistance

An AI system intended for proofreading or improving the style of judicial decisions or judgments drafted by judges, without changing the content. It is doubtful whether such an AI system would be considered as an AI system intended to assist in applying the law to a concrete set of facts, because the degree of assistance appears negligible. In any event, the system would benefit from the exception for AI systems intended to improve the result of a previously completed human activity listed in Article 6(3)(b) AI Act.

AI systems suggesting factual questions

An AI system is used after the judge has drafted the decision to compare the draft judgment with the case file and identify factual questions or aspects that the draft appears not to address. This may involve an interpretation of factual material. However, where the system merely carries out a compliance check, thus helping the judge to check whether all relevant factual aspects have been considered, and does not assess credibility, determine what happened, suggest how any factual question should be answered, or otherwise materially influence the judicial determination of the facts, it would performs a task intended to improve the result of a previously completed human activity (Article 6(3)(b) AI Act).

8(b) AI systems intended to be used for influencing the outcome of elections or referendum

Optimisation of political campaigns from an administrative or logistical point of view

AI systems that optimise campaign staff (including volunteers) management; AI systems scheduling rallies, assigning party members to them or selecting topics to focus on. Such systems are excluded from high-risk classification by the second sentence of point 8 (b) of Annex III.
AI systems analysing earlier party donor databases to predict likelihood of contributions for the current or future campaign. Such systems are excluded from high-risk classification by the second sentence of point 8(b) of Annex III.
AI systems designed for generating political advertising content (e.g. generative AI systems that could be used to draft political slogans and communications to be reviewed and delivered by a political party), without further disseminating it. Such systems assist in preparation of political campaigning materials and are not ‘intended to influence the outcome of an election’.

AI systems intended for monitoring, research and pattern recognition

AI systems monitoring the activity of elected officials in submitting legislative proposals or amendments based on content provided by a parliament’s web presence. This monitoring activity mainly serves informational purposes; it does not have as its direct purpose to influence the outcome of elections. This system falls outside the scope of point 8(b) of Annex III.
AI systems deployed by sociological agencies for the purpose of collecting, analysing, and presenting information in an objective and transparent manner. Such AI systems do not fall under point 8(b) of Annex III, provided that their sole function is to inform the public or relevant stakeholders without seeking to influence electoral processes.
AI systems used by academic institutions for modelling electoral behaviour for research purposes. Such systems are not intended to influence the outcome of elections. In any event, it is likely that such AI system would be excluded under Article 2(6) AI Act.

Technical assistance in vote counting

AI systems for counting ballots automatically. If an AI system is used only for mechanical recognition and tallying of ballots (e.g. scanning, OCR), then it is functioning as a technical aid and should not be classified as high-risk as its intended purpose does not correspond to the one defined in point 8(b) of Annex III.

Chatbots to provide information on the elections

An AI-enabled chatbot designed to provide information to voters on behalf of election authorities, operating in a politically neutral way (e.g. providing information when the elections take place, where a person can vote, voting times, etc.).