Spot the Mistake before it happens: Successful PhD Defense for Santosh Thoduka in Robotics

Intro Slide Defense Thoduka, H-BRS, 2026

From dropped objects to failed handovers, new research shows how robot vision can detect—and eventually prevent—mistakes in real time. Hochschule Bonn-Rhein-Sieg (H-BRS) celebrates the successful doctoral defense of Santosh Thoduka, whose research advances the reliability of robotic systems in everyday tasks. His dissertation explores how robots can use video from their own cameras to recognize when something goes wrong—an essential step toward safer and more intelligent human-robot interaction.

Seeing Failure Through the Robot’s Eyes

At the core of the research is a simple but powerful idea: robots can learn to detect failures by analyzing visual information from their own perspective. These failures range from slipping or dropping objects, unsuccessful grasp attempts, or even subtle task-level issues such as failing to properly hand over an item to a person.

To investigate this, Santosh Thoduka developed new datasets, including the Handover Failure Detection dataset, alongside the use of established benchmarks such as the ARMBench dataset from an Amazon warehouse environment. The results demonstrate that video data alone provides strong signals for identifying failures—but performance improves significantly when combined with additional sensory inputs, such as force-torque and tactile data, as well as contextual task information (e.g., expected human actions during interaction).

Looking ahead, the work highlights the growing importance of large-scale vision-language and vision-language-action models, which are expected to play a key role in enabling robots to better understand and respond to complex real-world situations.

Doctoral Defense and Academic Committee

The PhD defense was conducted before a distinguished committee consisting of Prof. Dr. Maren Bennewitz (University of Bonn, chair), Prof. Dr. Jürgen Gall (University of Bonn, supervisor), Prof. Dr. Paul G. Plöger (H-BRS, supervisor), and Prof. Dr. Christoph Hanhart (University of Bonn / Forschungszentrum Jülich). The event was attended by colleagues from academia and research institutions, including members of the Computer Vision Group at the University of Bonn, Fraunhofer IAIS, and H-BRS.

Continuing Research in Multi-Modal AI

Following the successful completion of his PhD, Santosh Thoduka will continue his work as a Research Engineer at Fraunhofer IAIS, contributing to the development of multi-modal foundation models in the team of Prof. Sebastian Houben.

„If I could give some advice to myself when I started the PhD, it would be: Most of the research and work for the PhD might happen during small periods of high productivity, but it is also important to make incremental progress, however small it might be, during the periods in between.”

Dr. Santosh Thoduka - PhD in Computer Science 2026