New technology gives smart cars ‘X-ray’ vision, detecting hidden pedestrians, cyclists

Australian researchers have developed disruptive technology allowing autonomous vehicles to track running pedestrians hidden behind buildings, and cyclists obscured by larger cars, trucks, and buses.

The autonomous vehicle uses game changing technology that allows it to “see” the world around it, including using X-ray style vision that penetrates through to pedestrians in blind spots and to detect cyclists obscured by fast-moving vehicles.

The iMOVE Cooperative Research Centre-funded project collaborating with the University of Sydney’s Australian Centre for Field Robotics and Australian connected vehicle solutions company Cohda Wireless has just released its new findings in a final report following three years of research and development.

The technology’s applications, which are being commercialized by Cohda, involve an emerging and promising technology for intelligent transportation systems (ITS) called cooperative or collective perception (CP).

Using roadside ITS information-sharing units equipped with additional sensors such as cameras and lidar (“ITS stations”), vehicles can share what they “see” with others using vehicle-to-X (V2X) communication.

This allows autonomous vehicles to tap into various viewpoints. By being hooked up to the one system significantly increases range of perception, allowing connected vehicles to see things they wouldn’t normally.

The engineers and scientists developing the technology said it could benefit all vehicles, not just those connected to the system.

“This is a game changer for both human-operated and autonomous vehicles which we hope will substantially improve the efficiency and safety of road transportation,” Professor Eduardo Nebot from the Australian Centre for Field Robotics said.

“The connected vehicle was able to track a pedestrian visually obstructed by a building with CP information. This was achieved seconds before its local perception sensors or the driver could possibly see the same pedestrian around the corner, providing extra time for the driver or the navigation stack to react to this safety hazard,” he said.

Another experiment demonstrated the CP technology’s ability to safely interact with walking pedestrians, responding based on the perception information provided by the roadside ITS station.

The three-year project also demonstrated the expected behavior of a connected vehicle when interacting with a pedestrian rushing towards a designated crossing area.

“Using the ITS system, the connected autonomous vehicle managed to take preemptive action: braking and stopping before the pedestrian crossing area based on the predicted movement of the pedestrian,” Professor Nebot said.

“The pedestrian tracking, prediction, path planning and decision making were based on the perception information received from the ITS roadside stations.

“CP enables the smart vehicles to break the physical and practical limitations of onboard perception sensors,” he said.

Lead project researcher Dr. Mao Shan said the research confirmed using CP could improve awareness of vulnerable road users and safety in many traffic scenarios.

“Our research has demonstrated that a connected vehicle can ‘see’ a pedestrian around corners. More importantly, we demonstrate how connected autonomous vehicles can autonomously and safely interact with walking and running pedestrians, relying only on information from the ITS roadside station,” he said.

Cohda Wireless Chief Technical Officer Professor Paul Alexander said the new technology “has the potential to increase safety in scenarios with both human operated and autonomous vehicles.”

“CP enables the smart vehicles to break the physical and practical limitations of onboard perception sensors, and embrace improved perception quality and robustness,” Professor Alexander said.

“This could lower per vehicle cost to facilitate the massive deployment of CAV technology.”

Professor Alexander said using CP for manually driven connected vehicles “also brings an attractive advantage of enabling perception capability without retrofitting the vehicle with perception sensors and the associated processing unit.”

iMOVE Managing Director Ian Christensen said the project was a great example of industry collaborating with scientists to unleash new innovations and commercial and public good not just for the benefit of Australians, but road users all over the world, including pedestrians and cyclists.

“When we bring industry and scientists together, we can achieve many great things as a nation. iMOVE CRC is proud to have initiated this exciting project and many others like it—which are all about getting our best and brightest minds working together to develop new technologies and innovations for real-world problems and needs,” Christensen said.