All news

MIT teaches robots to find and correct incorrect object descriptions in their own memory

Prepared by Sintelligent with the help of SI. Published in SI: . Updated: .

Robots gain a practical way to notice doubtful memories and clarify them on the move instead of confidently acting on an error.

UQ-DAAAM evaluates both a vision-language model's confidence in an object description and the consistency of descriptions from different viewpoints. When uncertainty is high, the robot selects additional frames expected to reduce it most effectively, and a language model merges the captions into a corrected entry in a three-dimensional memory map. A mathematically derived frame-selection rule avoids rerunning the vision-language model and lets a moving robot update its memory in fractions of a second. In experiments, the system resolved ambiguity for about 92% of objects using an average of only two additional captions per object and improved description accuracy by 5% over a state-of-the-art baseline. The work will be presented at NeurIPS.

Sources 1