An open-source benchmark to evaluate the manipulation and planning skills of assembly robots

Research in the field of robotics has been booming over the past decade with a view to tackle challenges of real value to industry and the public domain. With new robotic systems appearing every other day, developing reliable tools that can be used to evaluate their performance and test algorithms underpinning their functioning is salient.

Researchers at the Oxford Robotics Institute, the Manufacturing Technology Center (MTC) and the University of Birmingham have recently introduced RAMP (Robotic Assembly Manipulation and Planning), a benchmark that can be used to assess the ability of assembly robots (i.e., robots designed to assemble products) to manipulate objects and plan their work. The new benchmark, introduced in a paper published on the arXiv pre-print server, could help to enhance the capabilities of assembly robots, facilitating their use in the manufacturing industry.

“The paper came about through a collaboration with the Manufacturing Technology Center (MTC) in Coventry, U.K. who have a range of partners in industry facing similar problems all within the domain of long-horizon planning for assembly,” Jack Collins, one of the researchers who carried out the study, told Tech Xplore.

“This domain of problems features many challenges and open questions within robotics and so working with the MTC we designed a benchmark that reflects the domain and the challenges whilst also ensuring it was accessible to researchers so they can make contributions towards solving aligned problems.”

The work is being conducted under the remit of an EPSRC Program Grant in Embodied Intelligence directed by Prof. Ingmar Posner at Oxford. “The objective of our benchmark is to be challenge-driven, accessible and open-ended. Our vision is to offer a platform that will further evolve into a community-driven effort to advance robotics research with a specific application focus in manufacturing and assembly.”

Posner, Collins and their colleagues set out to create an open-ended benchmark that could be applied to a broader range of robotic assembly tasks. RAMP utilizes a series of base parts that can be reconfigured and expanded, to test the ability of robots to assemble a broader range of objects.

“To solve for the variation in goal configurations RAMP requires explicit reasoning and planning to decide the order to assemble the beams without which the result would be infeasible assembly sequences,” Collins explained.

Essentially, RAMP allows users to reliably assess the ability of a robotic manipulator to assemble various beams into one of several different target configurations. The benchmark was designed around common challenges in offsite construction, where parts are prefabricated and assembled offsite, for example to form the frames that make up the internal structure of buildings.

“The benchmark features beams that are created from a series of 3D printed joints and aluminum profiles, the goal is to assemble these beams into the given goal configuration as quickly as possible whilst ensuring repeatability,” Collins said. “RAMP is uniquely designed to assess long-horizon assembly tasks that require explicit reasoning as to the order of the assembly. Our benchmark also lowers the barrier to entry by making everything publicly available for people to get started.”

RAMP is publicly available, so researchers and manufacturers worldwide can now use it to test their own robotic systems. Users are provided with everything they need to start using the benchmark, including the base parts, an assembly guide, code for a baseline implementation and code for a highly realistic simulation environment based on the Nvidia Isaac platform.

“RAMP also features a baseline method that is able to solve the easy class of assemblies, we release the baseline publicly with the intention of allowing others to use and build upon our own work,” Collins said. “We believe the implication of this is measurable progress on a problem domain that has direct use in industry whilst also pushing progress on a range of open challenges known within robotics. We see this benchmark evolving into a community-driven effort with future users growing the benchmark to encompass new areas such as bimanual manipulation, 3D construction and deformable manipulation.”

The new benchmark created by this team of researchers could soon prove to be a valuable tool for evaluating the planning and manipulation skills of robots designed to assemble structures in offsite construction sites. Ultimately, RAMP could help to better assess the progress in the field of robotic assembly, while also highlighting potential issues with robots and how to overcome them.

“In our future research, we will look to use the associated simulation environment to apply learning-based approaches to come up with more robust methods to solve the assemblies,” Collins added. “We will also look at how we can solve for the assembly order without requiring an expert to first write an abstraction of the domain that can be solved by a symbolic planner.”

© 2023 Science X Network