Can’t find your phone? There’s a robot for that

Engineers at the University of Waterloo have discovered a new way to program robots to help people with dementia locate medicine, glasses, phones and other objects they need but have lost.

And while the initial focus is on assisting a specific group of people, the technology could someday be used by anyone who has searched high and low for something they’ve misplaced.

“The long-term impact of this is really exciting,” said Dr. Ali Ayub, a post-doctoral fellow in electrical and computer engineering. “A user can be involved not just with a companion robot but a personalized companion robot that can give them more independence.”

Ayub and three colleagues were struck by the rapidly rising number of people coping with dementia, a condition that restricts brain function, causing confusion, memory loss and disability. Many of these individuals repeatedly forget the location of everyday objects, which diminishes their quality of life and places additional burdens on caregivers.

Engineers believed a companion robot with an episodic memory of its own could be a game-changer in such situations. And they succeeded in using artificial intelligence to create a new kind of artificial memory.

The research team began with a Fetch mobile manipulator robot, which has a camera for perceiving the world around it.

Next, using an object-detection algorithm, they programmed the robot to detect, track and keep a memory log of specific objects in its camera view through stored video. With the robot capable of distinguishing one object from another, it can record the time and date objects enter or leave its view.

Researchers then developed a graphical interface to enable users to choose objects they want to be tracked and, after typing the objects’ names, search for them on a smartphone app or computer. Once that happens, the robot can indicate when and where it last observed the specific object.

Tests have shown the system is highly accurate. And while some individuals with dementia might find the technology daunting, Ayub said caregivers could readily use it.

Moving forward, researchers will conduct user studies with people without disabilities, then people with dementia.

A paper on the project,” Where is my phone? Towards developing an episodic memory model for companion robots to track users’ salient objects,” was presented at the recent 2023 ACM/IEEE International Conference on Human-Robot Interaction.