Barkour benchmark measures quadruped robot agility

The accelerated pace of robotics development has given us a veritable zoo filled with creatures sometimes indistinguishable from the real deal.

We’ve got RoboFish, Bionic Kangaroo, Robot Snake, and even RoboTuna, Bionic Ants and RoboSalamander.

Of particular interest to developers, however, are quadruped creations. They provide invaluable opportunities to study locomotion, stability and ruggedness over varying terrains that will help develop robotic solutions for disaster response, medical treatment, environmental monitoring and surveillance.

Spot, from Boston Dynamics, for example, has grabbed headlines as a law enforcement tool in Honolulu, a monitor in Singapore enforcing social distancing during the COVID-19 pandemic, a thermal-camera sentry at Haneda Airport in Tokyo and an aide at construction sites in Denmark.

There are other four-legged beasts in this race: AIbo, ANYmal, WildCat and Hyq.

One challenge facing researchers and potential customers of robotic quadrupeds is how to measure the differences in capabilities of a growing menagerie of AI-based critters.

After all, there are tests for seemingly everything digital: Dhrystone measures performance of integers operations on a CPU; N-Body Simulation checks performance of simulating the motion of particles in a gravitational field; Whetstone is used for floating point performance on a CPU; Pro Agility Shuttle for lateral quickness and change of direction; and T-Test for shuffling, sprinting and backpedaling. Scores of additional tests assess countless other attributes of digital machines.

Research scientists at Google say it’s time to zero in on a sound measurement tool specifically for robotic agility.

“[W]hile researchers have enabled robots to hike or jump over some obstacles, there is still no generally accepted benchmark that comprehensively measures robot agility or mobility,” the Google DeepMind team reported in a paper posted on the pre-print server arXiv last week. “In contrast, benchmarks are driving forces behind the development of machine learning, such as ImageNet for computer vision, and OpenAI Gym for reinforcement learning.”

The Google researchers designed an obstacle course that includes weave poles, A-frames and a jump board. A scoring system rewards tasks completed successfully within a specified period of time, penalizing entrants who fail or skip challenges. The project is called Barkour.

“It is essential to establish metrics that can accurately measure robot agility and to define a standard set of tasks that can serve as a common evaluation framework,” the team said in their paper, “Barkour: Benchmarking Animal-level Agility with Quadruped Robots.”

The course was tested on real dogs first—”dooglers,” in this instance, who were pets owned by Google staff members.

The team then trained robotic quadrupeds to navigate the varying terrains while efficiently accommodating balance and speed demands. The challenges included walking, slope climbing and jumping.

While the team achieved a practical set of measurements to test agility, they reported that live untrained dogs were able to successfully tackle the obstacle course in about half the time it took their custom-built quadruped robot to do so.

“There is still big gap in agility between robots and their animal counterparts,” the team acknowledged. “There is still notable room to push towards dog-level agility by improving speed and robustness.”

But they maintained that “Barkour will serve the robotics community as an important testbed for different learning and control methods and different hardware designs” in the future.

© 2023 Science X Network