AI systems used in education or vocational training may determine the educational and professional course of a person’s life and therefore may affect that person’s ability to secure a livelihood. When improperly designed and used, AI systems may violate: the right to education and training, the right not to be discriminated against, and perpetuate historical patterns of discrimination.
Use cases classified as high-risk:
3(a) AI systems intended to be used to determine access or admission or to assign natural persons to educational and vocational training institutions at all levels
- Automated admissions systems intended to be used to review and evaluate student applications, transcripts, and test scores to determine eligibility for admission to an educational institution. This includes systems that evaluate applications from prospective students and determine their eligibility for admission, since the system’s output will directly affect the applicant’s right to study, train, or enrol at the institution. That will be the case, for instance, where the system's evaluation and recommendation have the potential to influence the decision-making process, in that it determines the outcome of an admissions decision. In cases where the system does not perform profiling, the filter mechanism in Article 6(3) AI Act could apply, for example, if the system is instead intended only to perform a preparatory task to the assessment of eligibility done by the educational institution (Article 6(3), letter (d)).
- School assignment systems which take their decisions based on location, availability and other personal characteristics of the applicant that is intended to be used by municipalities or regional authorities to assign students - often in primary or secondary education - to public schools in an automated manner. Such systems function by collecting structured data, such as the student’s home address, the geographical boundaries of school catchment areas, and the available capacity at each school. They also factor in sibling attendance and parental status to keep families together or to optimise for logistical efficiency, such as minimising commuting distance. Such systems process such data to assign students to schools, ensuring compliance with zoning rules and balancing enrolment across institutions. Since the system’s assessment of data constitutes profiling, the filter mechanism in Article 6(3) AI Act cannot be applied to it.
- Vocational training assignment AI tools intended to be used by a regional employment agency to process applicants’ prior education records, completed certifications, and standardized aptitude test results to match individuals to available apprenticeship programs should be classified as high-risk. Such systems apply predefined eligibility criteria, such as minimum qualifications required for each program and capacity limits, to assign candidates to appropriate training slots. Since the system’s assessment of data constitutes profiling, the filter mechanism in Article 6(3) AI Act cannot be applied to it.
- Automated scholarship eligibility assessment system intended to be used to automate the process of determining an individual’s eligibility for a scholarship programme based on data about the individual’s financial situation, including their income, expenses, and family size. Such systems assess the eligibility of an individual’s eligibility for financial assistance, thus determining their access to an educational institutional.
- Educational programme matching platform using AI to provide secondary school students with recommendations on tertiary education programmes and the most suitable institutions based on their indicated preferences and previously selected topics of interest The output of such systems does not determine access or admission to educational programs, but rather serves as a tool to inform prospective students to support their own decision-making process on whether to apply to a specific institution or programme.
- Chatbot for admission information provided by a university to answer prospective students’ questions and to provide them with information about its admission requirements, application process, and available programs. While such chatbots are linked to the admission process, they do not make decisions about student admissions or provide personalised recommendations that could influence an admissions decision. Such a chatbot’s output is limited to providing general information and guidance, and it does not have the potential to materially influence the decision-making process.
- AI systems supporting the processing of applications to be used by an education institution for file handling, such as indexing, searching, text and speech processing, as well as for the translation of documents provided along with applications, and for extracting, transforming, and organising the data collected into a meaningful and usable format. Although such systems support the assessment process, they do not have a relevant impact on the decision-making process, since they are only intended to facilitate the organisation and review of applicant files. Such systems therefore fall under the exception for AI systems intended to perform a preparatory task listed in Article 6(3)(d) AI Act.
- AI-enabled admissions decision review tools intended to be used by an educational institution to review and analyse ex post admissions decisions made by an admissions committee. Such systems are not intended to be used to replace or influence human assessment, but to provide a quality control check to ensure that the admissions process is functioning as intended and to support and improve future decision-making processes, rather than to make decisions. Since such systems are intended to detect decision-making patterns and deviations, and are not meant to replace or influence the previously completed human assessment, they can be considered to fall under the exception for AI systems intended to detect decision-making patterns listed in Article 6(3)(c) AI Act.
- AI-enabled admissions data organisers intended to be used by an educational institution to automatically categorise and organise incoming applications prior to the admissions process. Such systems are intended to be used to extract relevant information from unstructured application documents, such as resumes, cover letters, transcripts, and convert it into a structured format, to classify each application into one of several predefined categories, such as undergraduate, graduate, international student, and to identify and flag duplicate applications to prevent unnecessary processing. Provided the system does not take any decision on admission or access to the educational institution, it can be considered to fall within the exception for systems intended to perform a narrow procedural task of data processing and organisation listed in Article 6(3)(a) AI Act.
- AI-enabled grading and feedback systems intended to be used to evaluate students’ assignments, such as tests, quizzes, and exams which count towards a final evaluation. Such AI systems have an impact on the summative evaluation of students by analysing the results of assignments and proposing grades.
- AI-enabled personalised learning assistant intended to be used to provide recommendations and feedback to students on their progress by generating reports at the end of each unit, which include grades, areas of strength and weakness, and recommendations for improvement. Although such AI systems support formative evaluation by providing ongoing feedback and guidance to students to help them improve their understanding of course material, their output is also used by the teacher to inform decisions about student’s grades at the end of the academic period and therefore have an impact on summative evaluation.
- AI-enabled language learning software applications intended to be used by students to support non-formal and informal learning by providing instant feedback and corrections on identified errors, but which does not lead to the attainment of a credential, certification, or form of accredited validation. Such AI systems are used by students on their own volition, without being required to do so by an educational institution.
- AI-enabled neurodiverse learning companions intended to be used to support students with neurodivergence, such as autism, dyslexia, and ADHD, which uses machine learning algorithms to create personalised learning pathways that cater to the individual needs and learning styles of each student by providing real-time feedback and adjustments to the learning materials, pace, and format to help such students stay engaged and motivated. For example, such an AI system may provide a student with dyslexia text-to-speech functionality, font size and colour adjustments, and multisensory learning materials to help that student better understand and retain information. Such a system may also provide accommodations such as extra time to complete assignments, breaks, and stress-reducing exercises to help students with anxiety or sensory processing issues. Such a system’s output is used to inform the teacher’s instruction and provide additional support to students with neurodivergence, but it is not intended to be used to determine grades or to assess student learning.
- AI-enabled pronunciation feedback tool intended to be used to provide feedback to students on their language pronunciation or fluency by analysing the student’s speech and providing suggestions for improvement, such as correcting intonation, rhythm, and accent. The feedback provided by the system is used by the students to improve their pronunciation skills; it is not used by educators to determine grades counting towards a final evaluation or assess language proficiency. Same example can be applied for an AI tool on reading skills providing feedback intended to be used by learners or students only.
- AI-enabled exam quality checker intended to be used to check an exam prepared by a teacher for errors, such as grammatical issues, ambiguous wording, or inconsistencies with the rubric, and to suggests revisions for the teacher’s consideration. However, since the AI system does not alter the core content of the exam, nor does it change the level of difficulty or the educational intent behind the questions, and the teacher remains responsible for any final decisions on the exam’s content, the AI system’s role should be considered strictly to enhance the clarity and accuracy of the exam, ensuring that it is error-free and aligns with the intended learning outcomes. Such a system would therefore fall under the exception for systems intended to improve the result of a previously completed human activity listed in in Article 6(3)(b) AI Act, namely, the preparation of the exam, by refining its quality without altering the substance or impact of the teacher’s original educational decisions.
- AI-enabled grade calculator intended to be used by an educational institution to process data related to assessment and test grades obtained by students during the academic period which determines the final grade of the students’ learning outcomes for the year. Such an AI system is designed to perform a simple calculation of the average grade, taking into account the grades and weights of each assessment, and test the output of the system as a numerical value representing the average grade, which is then used by the teacher to inform their final evaluation of the student’s performance. Such a system would therefore fall under the exception for systems intended to perform a narrow procedural task listed in Article 6(3)(a) AI Act.
- AI-enabled assessment review tool intended to be used to detect decision-making patterns or deviations from prior decision-making patterns, with the goal of identifying potential inconsistencies or anomalies in the assessment decisions made by instructors. Such an AI system analyses the grading patterns of instructors over time, and flags any deviations from the expected pattern. For example, if an instructor has consistently given high grades to students who complete a certain assignment, but suddenly gives low grades to a group of students who complete the same assignment in the same manner, the AI system will flag this deviation as a potential anomaly. The output of the AI system is then reviewed by a human assessor, who investigates the flagged deviations and determines whether they are justified or not. The human assessor may decide to adjust the grades or take other corrective action, but the AI system’s output is not used to automatically change the assessment decisions. Such a system would therefore fall under the exception for systems intended to detect decision-making patterns or deviations from prior decision-making patterns listed in Article 6(3)(c) AI Act, since it is not meant to replace or influence a previously completed human assessment.
- AI-powered adaptive placement tools intended to be used by an educational or vocational training institution to determine the optimal course level for incoming students. Such a system assesses a student’s abilities to formulate recommendations addressed to educators whether the student should be in beginner, intermediate, or advanced classes, or decides if he or she should move to higher education or vocational training based on their skills.
- Vocational training level assessment systems intended to be used by a vocational training institution to assess the level of education that a learner will be able to access in a specific vocational program. Such a system may use a combination of placement tests and skill assessments to evaluate the learner’s knowledge and skills in relevant areas, such as mathematics, physics, and engineering principles, and determine their eligibility for a particular level of study or course.
- AI classifiers for special education intended to be used to classify students with special educational needs into appropriate educational programs and recommend the most suitable educational placement for each student, including the level of support required. Such a system may use a machine learning algorithm to analyse a range of data, including student assessment data, such as results from standardized tests, including IQ tests, achievement tests, and behavioural assessments. It may also consider teacher’s feedback on student’s behaviour, social skills, and academic performance, as well as psychological evaluations, including reports from psychologists on student’s cognitive abilities, emotional intelligence, and learning styles, and the student’s medical history, including information on any diagnosed conditions or disabilities.
- Personalised education recommendations for students: An AI system intended to be used to provide personalised educational recommendations to students for the appropriate level of education, such as courses, programs, or certifications, that they may be interested in applying for the following academic year on the basis of information provided by students themselves, such as interests and career goals. Since the system is not intended to be used by an educational institution, but is only intended to help students make informed decisions about their educational path, of the system does not fall within the use case listed in point 3(c) of Annex III.
- Trend Analysis of Educational Attainment: An AI system intended to be used by educational institutions, policymakers, and researchers to analyse from various sources, including educational institutions and government databases, trends in the levels of education accessed after completion of secondary education by students, and to identify patterns in the types of educational programs and levels of education that students are pursuing. Since the system’s output is not used to make decisions about individual students’ educational placement or opportunities, but rather to provide a broader understanding of the educational landscape, it does not fall within the use case listed in point 3(c) of Annex III.
- AI-enabled pre-assessment tool intended to be used to help educators prepare for the assessment of a student’s readiness for a particular level of education or training, based on data about the student’s prior education and work experience, and generating a set of questions and topics that the educator can use to assess the student’s readiness should not be considered high-risk. Since the system’s output is used by educators to inform their assessment methodology, but does not itself make any decisions about the appropriate level of education that an individual will receive or will be able to access, it should therefore be considered to fall under the exception for systems intended to perform a preparatory task listed in Article 6(3)(d) AI Act.
- Improving recommendations for advanced course placement: An AI system intended to be used by an educational or vocational training institution to analyse the results of tutors’ recommendation to apprentices to pursue certain advanced courses, based on data from the apprentice's performance records and learning outcomes, and providing suggestions for improving the recommendation methodology, such as identifying additional factors that could be considered, or suggesting alternative courses of action that could be taken. Since the system’s output is used by the institution to refine its recommendations and to improve the decision-making process, but it does not take any decisions on the apprentice’s educational placement or opportunities, it should be considered to fall under the exception for systems intended to improve the result of a previously completed human activity listed in Article 6(3)(b) AI Act.
- AI-enabled proctoring system for exam certification intended to be used by an educational or vocational training institution to monitor test-takers for prohibited behaviour during a certification exam, such as accessing unauthorized materials or communicating with someone else during the exam, by using a combination of facial recognition, keystroke analysis, and screen monitoring.
- Real-time behaviour analysis system using machine learning algorithms intended to be used by an educational institution to analyse student behaviour and detect patterns that may indicate cheating during online exams.
- AI-enabled exam monitoring system intended to be used by an educational institution to monitor and detect prohibited behaviour during in-person exams, such as the use of unauthorized materials or devices, by using facial recognition and object detection.
- AI system intended to be used by an educational institution to check homework or assignments for plagiarism against a database of existing content. Such a system is not intended to be used to monitor and detect prohibited behaviour during tests, but to analyse submitted work which has been produced in a non-supervised environment and does not involve live monitoring or real-time behavioural analysis during the testing process.
- AI system for non-academic student behaviour monitoring intended to be used by an educational institution to monitor and detect student behaviour in the cafeteria, hallways, and other non-testing environments and to alert staff to potential incidents of bullying, harassment, or other undesirable behaviour. Such a system is not used to monitor and detect prohibited behaviour during tests, but for non-academic purposes, such as maintaining a safe and orderly school environment, and it does not involve the evaluation or assessment of student academic performance. However, this AI system could be classified as high-risk under point 1 of Annex III, if the system fulfils the definition of a remote biometric identification system.
- AI-enabled confirmation of suspicious behaviour: During an online test, a human proctor observes a student exhibiting suspicious behaviour, such as consistently looking away from the screen or typing in a pattern that suggests they may be copying from a hidden source and uses an AI system to confirm or deny his suspicions. The AI system is trained on a dataset of known cheating behaviours and can identify patterns that are indicative of academic misconduct. The AI system analyses the student’s data and provides a report to the human proctor, indicating that the student's behaviour is consistent with cheating. The report highlights specific patterns and anomalies that the AI system has identified, such as unusual keystroke patterns or excessive mouse movements. The human proctor reviews the report and may decide to investigate further and take appropriate action, such as alerting the student's instructor or initiating a formal investigation. Such a system would fall under the exception for systems intended to improve the result of a previously completed human activity listed in Article 6(3)(b) AI Act.
- Identity verification: An AI system intended to be used during a test to automatically processes students’ identity verification documents, such as IDs or biometric data like facial recognition scans, to confirm that the person taking the test matches the registered candidate. Since the system helps streamline identity verification, but it does not directly decide if the student can proceed with the exam or if any sanctions should be applied in the case of misrepresentation, as these decisions are made by the human proctor, it should fall under the exception for systems intended to perform a procedural task listed in Article 6(3)(a) AI Act.