Researchers from Stanford University in California (US) have developed a laser-based imaging technology to see objects hidden around corners. The extremely sensitive non-line-of-sight (NLOS) imaging system could enable self-driving cars to react to obstacles and dangers before human passengers can see them.
“Non-line-of-sight imaging is the idea of building a camera that can see around corners,” says Gordon Wetzstein, assistant professor of electrical engineering and senior author of the paper “Confocal non-line-of-sight imaging based on the light-cone transform,” published in the journal Nature. “That is, it can resolve three-dimensional shapes of objects that are outside the direct line of sight.”
Besides autonomous vehicles, other NLOS imaging applications could include seeing through foliage from aerial vehicles or giving rescue teams the ability to find people blocked from view by walls and rubble.
As Wetzstein puts it: “It can make the invisible visible.”
The algorithm was the crux
The Stanford group´s research advance stands out from other laser-based methods for imaging objects hidden around corners through the extremely effective algorithm the researchers have developed to process the final image. Figuring out an efficient way to recover the 3D structure of the hidden object from the noisy measurements is a substantial challenge in non-line-of-sight imaging, according to the researchers, who regard the computational efficiency of the work as the most important impact of this method.
“This idea [of NLOS imaging] was only introduced a few years ago, and no attempt to realize it in the past seems practical, because either the equipment is prohibitively expensive or the acquisition and reconstruction times are prohibitively long or because the amount of information that can be inferred about the hidden objects is very small,” says Wetzstein, who is also a member of Stanford Bio-X and the Stanford Neurosciences Institute.
“The hardware components we propose for this task have not been described in this ‘confocal´ configuration, and we show that our confocal scanning allows us to derive an extremely compute and memory-efficient reconstruction algorithm.”
Asked about the most challenging aspect of this research work, the scientists replies: “Just coming up with the idea that there is, mathematically speaking, such an efficient solution for the problem was the most critical part of the project. It was really just having a great idea.”
The first practical technology for NLOS imaging
“What we introduce is the first practical technology for NLOS imaging,” Wetzstein emphasizes. The system uses a standard laser and single-photon detector commonly found in lidar systems. The laser is positioned next to a photon detector sensitive enough to record a single particle of light. Laser pulses shot at a wall bounce off objects around the corner and bounce back to the wall and to the detecto, all invisible to the human eye. While this process until recently could take between two minutes and an hour, depending on conditions such as lighting and the reflectivity of the hidden object, the professor can meanwhile report a new paradigm: “By now, we have it working in real-time, and we demonstrated state-of-the-art image reconstruction quality. We achieve this through a clever co-design of hardware and software.”
Furthermore, the team is working on enhancing the system to be able to better handle the variability of the real world, including improvements in daylight and with objects in motion. So far, the system has been successfully tested outside, but only with indirect light. The technology performed particularly well at recognizing retroreflective objects, such as road signs, safety vests or road markers. The researchers do admit, though, that the system might still struggle in the real world with a person wearing non-reflective clothing.
Wetzstein agrees his team´s research breakthrough could have significant impact on the future of self-driving cars, as it could lead to “more robust decision making over longer range and in challenging scenarios.”
The researcher says his team´s next step is to actually put their laser-based NLOS imaging system on a car. However, they are not currently working on commercializing this technology.
Written by Sandra Henderson, Research Editor, Novus Light Technology Today
Back to Light Research
