Technische Universität München
Jack the robot moves like a human
TECHNICAL UNIVERSITY OF MUNICH
NEWS RELEASE
Autonomous vehicle thinks for itself
Jack the robot moves like a human
Researchers at the Technical University of Munich (TUM) have developed a wheeled robot that makes its way through a crowd of people safely and without hesitation. A computer on board predicts the movement of people in the vicinity and how they are likely to react to the robot. From this, it calculates the fastest route. Similar algorithms could also be used for humanoid robots or autonomous driving to enable safe interaction between robots and humans.
The little robot weaves its way through crowds of people on its wheels like a human. To make this possible, researchers from TUM Professor Angela Schoellig's Learning Systems and Robotics Lab have combined computing power, sensors and mathematical skills. ‘Our robot models the way people will react to its movements to plan its paths. This is the big difference to other approaches that typically ignore this interaction,’ explains Prof. Schoellig.
New route calculated ten times per second
A lidar system constantly sends laser beams into the surroundings, measures their reflections, and uses them to build a precise 360-degree map of what the robot sees. A special focus is placed on people walking around nearby. At the same time, sensors in the wheels measure the robot's own speed and the distances traveled. A computer processes this information, calculates the estimated distances people will cover in the next two seconds, and simultaneously plans the optimum route to the destination. ‘Our robot adjusts its route ten times a second while simultaneously recognizing people's paths,’ explains TUM researcher Sepehr Samavi.
System learns from data based on people in crowds
The robot, which Sepehr Samavi has named ‘Jack’, learns behavioral patterns from humans so that it does not constantly stop due to the risk of collisions. ‘Our mathematical model, on which the planning algorithm is based, was derived from human movements and translated into equations,’ explains Professor Schoellig. For Jack's decision, this means that he does not stop immediately as soon as a person approaches him. He takes into account that people will adapt to the situation, react and change their route slightly so that they do not collide with him. If someone remains on a collision course contrary to expectations, the robot changes its plans quickly and takes a different route - but does not stop.
The researchers also incorporate data sets that show people's behavior in crowds. The robot, which has already been used outside of the laboratory, is constantly learning and becoming more human-like: ‘Jack knows his destination, observes people and sees where they are going to constantly optimize his own paths,’ says Prof. Schoellig, ‘almost like a human being.’
The TUM researchers have already reached the third evolutionary stage with the new algorithm. Instead of ‘only’ reacting to a situation (stage 1) or ‘merely’ predicting the movements of oncoming people (stage 2), the TUM robot is interactive (stage 3). ‘On the one hand, it predicts other people's movements, but it also manages to influence these people through its behavior and simultaneously avoid collisions,’ explains researcher Sepehr Samavi.
Development for use in autonomous driving
It is precisely such interactive scenarios that form the bottleneck in autonomous driving, says Prof Schoellig. For example, if a vehicle starts accelerating on a highway on-ramp, many drivers will change lanes or tap the brakes. The new approach makes it possible to consider the reaction of others in such a scenario. However, the researchers are initially looking at applications in delivery robots or with wheelchair users. The advantage: these vehicles can reach their destination independently and reliably. Even humanoid robots could benefit from the new algorithms. However, the intelligent vehicle has one decisive disadvantage: ‘A moving robot can simply stop if necessary - humanoids are still quite wobbly and quickly lose their balance,’ says Prof. Schoellig.
Additional material for media outlets:
Photo: https://mediatum.ub.tum.de/1773744
Researcher Sepehr Samavi introduces to Jack: https://youtu.be/7pxZhRulXVY
Scientific Video: https://www.youtube.com/watch?v=BY23UXwM1sM
Publications:
Sepehr Samavi, James R. Han, Florian Shkurti, and Angela P. Schoellig; SICNav: Safe and Interactive Crowd Navigation Using Model Predictive Control and Bilevel Optimization; IEEE Transaction on Robotics; 2025; https://ieeexplore.ieee.org/document/10726864
Further information:
Prof. Angela Schoellig heads the Chair of Safety, Performance, and Reliability for Learning Systems at the TUM School of Computation, Information and Technology. She is responsible for international relations on the board of the Munich Institute of Robotics and Machine Intelligence (MIRMI) and also coordinates the Robotics Institute Germany, which is funded by the Federal Ministry of Education and Research (BMBF) – https://www.robotics-institute-germany.de/. She is also a member of the Munich Data Science Institute (MDSI) at TUM.
Subject matter expert:
Prof. Angela Schoellig
Technical University of Munich (TUM)
Chair of Safety, Performance and Reliability for Learning Systems
TUM Corporate Communications Center contact:
Andreas Schmitz
Media Relations Robotics and AI
0162-27 46 193
The Technical University of Munich (TUM) is one of the world’s leading universities in terms of research, teaching and innovation, with around 700 professorships, 53,000 students and 12,000 staff. TUM’s range of subjects includes engineering, natural and life sciences, medicine, computer sciences, mathematics, economics and social sciences. As an entrepreneurial university, TUM envisages itself as a global hub of knowledge exchange, open to society. Every year, more than 70 start-ups are founded at TUM, which acts as a key player in Munich’s high-tech ecosystem. The university is represented around the world by its TUM Asia campus in Singapore along with offices in Beijing, Brussels, Mumbai, San Francisco and São Paulo. Nobel Prize laureates and inventors such as Rudolf Diesel, Carl von Linde and Rudolf Mößbauer have conducted research at TUM, which was awarded the title of University of Excellence in 2006, 2012 and 2019. International rankings regularly cite TUM as the best university in the European Union.