Technische Universität München
Learning maths with eye tracking and AI
TECHNICAL UNIVERSITY OF MUNICH
NEWS RELEASE
Now in use at first school in North Rhine-Westphalia
AI-based math: individualized support for schoolchildren
Researchers at the Technical University of Munich (TUM) and the University of Cologne have developed an AI-based learning system that recognizes strengths and weaknesses in mathematics by tracking eye movements with a webcam to generate problem-solving hints. This enables teachers to provide significantly more children with individualized support.
An up-to-date PC, a good graphics card and a standard webcam: according to research by Prof. Achim Lilienthal, that's all you need to identify pupils' strengths and weaknesses in mathematics. The principle: a webcam tracks the eye movements. Depending on the task, specific patterns emerge that can be displayed digitally on a heatmap, with red indicating areas where the children look frequently and green the areas where they glance only briefly. This helps the researchers analyze the data. “The AI system classifies the patterns,” says the TUM robotics professor. On this basis, the software selects learning videos and exercises for the pupil.
Identify learning strategies via heat maps
“Tracking eye movements in a single system using a webcam, recognizing learning strategies via patterns and offering individual support, and finally creating automated support reports for teachers is completely new,” says Maike Schindler. The Professor of Mathematics in Inclusive and Special Education Contexts at the University of Cologne has worked with TUM Professor Lilienthal for ten years. She also heads the recently completed KI-ALF research project, which was funded by the German Federal Ministry of Education and Research (BMBF) and in which the webcam-based eye-tracking system was developed. Her research focuses on pupils “who have great difficulties in learning mathematics.” Prof. Lilienthal believes that “individually customized lessons” for high-achieving children are also possible in the future.
Prof. Schindler – who holds a teaching degree – and her team have defined hundreds of tasks in which children add, subtract, multiply and divide numbers, or have to recognize or represent them. “Tasks involving visually presented, digital learning materials are particularly suitable for this approach,” says Schindler. For example, the children are asked to count the dots in a ten-row table with a few dots missing only in the bottom row.
The pupils who catch on quickly jump to the bottom row and only count backwards. Those who count the rows and dots individually are among the ones who need support. The digital system uses a heat map to show where the children look and the AI translates the patterns into individual practice programs.
Simplified, high-precision eye tracking
To develop the simplified eye tracking system, which now registers eye movements, TUM Professor Lilienthal benefits from the fact that he also works with corresponding systems in robotics research. In that work, he currently uses eye trackers with the small humanoid robot Nao. This enables it to communicate better with humans. However, these very precise systems cost many thousands of euros.
To find a more cost-effective solution for schools, the researchers cleverly combined technical expertise with knowledge from mathematical didactics. While advanced systems work with a maximum deviation of one degree, webcams have a lower accuracy of three to four degrees. The solution: “With the AI-ALF math tasks, we know that the students are ultimately looking at the on-screen display of the problems,” says Lilienthal. “We use this to automatically readjust the eye tracking with the webcam.” The system has gradually learned to deal with inaccuracy. “Today it makes no difference to our application whether we work with our webcams or high-end eye trackers,” says the professor. This makes the AI system developed with Prof Maike Schindler affordable and, therefore, increasingly important for school use.
Wulfen Comprehensive School: first school in Germany to use the system
This is one reason why the first school to use the AI-based learning system is the Wulfen Comprehensive School in Dorsten, North Rhine-Westphalia. Here, a standardized math test revealed that one-third of 180 children at the start of Year 5 had “arithmetic difficulties.” “We are delighted that we can now support significantly more children in their basic math skills with the help of the AI-based learning system. This means we can help more learners improve their math performance than in the past due to a lack of teachers.”
In the comprehensive school, five pupils can work with the KI-ALF system simultaneously in individual remedial lessons and are supported and accompanied by a math teacher. Normally teachers can give individual support to only one child at a time. “Especially in times of scarce resources and teacher shortages, our system for promoting basic math skills is simply an excellent support for schools,” says Schindler.
Additional material for media outlets:
Video on Youtube: https://youtu.be/7fzWbT1O4Zs
Foto zum Download: https://mediatum.ub.tum.de/1773230
Homepage of the research project: www.ki-alf.de
Publication:
Introduction to eye tracking in mathematics education: interpretation, potential, and challenges; Maike Schindler, Anna Shvarts & Achim J. Lilienthal; Educational Studies in Mathematics (ESM); 3-2025; https://link.springer.com/article/10.1007/s10649-025-10393-1
Structure Sense in Students’ Quantity Comparison and Repeating Pattern Extension Tasks: An Eye-Tracking Study with First Graders; Demetra Pitta-Pantazi, Eleni Demosthenous, Maike Schindler, Achim J. Lilienthal, and Constantinos Christou; Educational Studies in Mathematics (ESM), 2024, pp. 1 – 19; https://link.springer.com/article/10.1007/s10649-023-10290-5
Subject matter expert:
Prof. Achim Lilienthal
Technical University of Munich
Chair of Perception for Intelligent Systems
TUM Corporate Communications Center contact:
Andreas Schmitz
Media Relations Robotics and AI
0162-27 46 193
The Technical University of Munich (TUM) is one of the world’s leading universities in terms of research, teaching and innovation, with around 700 professorships, 53,000 students and 12,000 staff. TUM’s range of subjects includes engineering, natural and life sciences, medicine, computer sciences, mathematics, economics and social sciences. As an entrepreneurial university, TUM envisages itself as a global hub of knowledge exchange, open to society. Every year, more than 70 start-ups are founded at TUM, which acts as a key player in Munich’s high-tech ecosystem. The university is represented around the world by its TUM Asia campus in Singapore along with offices in Beijing, Brussels, Mumbai, San Francisco and São Paulo. Nobel Prize laureates and inventors such as Rudolf Diesel, Carl von Linde and Rudolf Mößbauer have conducted research at TUM, which was awarded the title of University of Excellence in 2006, 2012 and 2019. International rankings regularly cite TUM as the best university in the European Union.