A new type of camera inspired by the visual system of the mantis shrimp--among the most complex found in nature--could help cameras in self-driving cars to see significantly better. Researchers at the University of Illinois at Urbana-Champaign (UIUC) created the biomimicking camera with never-achieved-before imaging capabilities of polarization information over a wide dynamic range.
Earlier this year, we have reported about another camera developed by the same team at UIUC. That camera, which mimics the intricate visual system of the morpho butterfly, aims to enhance image-guided cancer surgery.
This new camera was inspired by the visual system of the mantis shrimp. “In particular, we mimicked the high-dynamic-range imaging capabilities of the mantis shrimp and the polarization imaging sensing,” explains Viktor Gruev, UIUC associate professor of electrical and computer engineering, who once again led the research team.
Thus, the new bioinspired camera can sense polarization information over a wide dynamic range.
A new camera, inspired by the vision system of the mantis shrimp, features a dynamic range about 10000 times higher than todays commercial cameras and can also image polarization information The dynamic range and polarization capability can be seen in the images above.
“Compared with commercial polarization cameras, such as the ones from Sony that debuted this year, our polarization sensor captures images with a dynamic range that is about one-million times better than Sony’s cameras,” Gruev notes, crediting the success to their approach of mimicking the visual system of the mantis shrimp as closely as possible.
Biomimickry: How the mantis shrimp inspired the new camera design
“The mantis shrimp can sense polarization information with logarithmic photo detectors, which enables them to sense polarization information over a wide dynamic range of illuminations,” Gruev says. So they designed a camera that also has a logarithmic response to light illumination. He says the physics behind the new camera is essentially the same as that of the mantis shrimp visual system. “The chemistry is different,” he points out. “We use different materials to construct this sensor then what you find in nature.”
New imaging capabilities of polarization information over a wide dynamic range
Gruev says the innovation in this camera is having imaging capabilities of polarization information over a wide dynamic range. “This is critical for many applications, such as cars where the dynamic range of the scene is high, and polarization information can help in seeing cars or pedestrians with high contrast, compared with regular cameras.”
The researcher emphasizes that their polarization camera enables sensing cars or pedestrians at further distances then regular cameras when imaging through foggy or rainy conditions. What is more, the new bioinspired camera also helps to detect objects when imaging over a wide dynamic range, such as driving out of a tunnel on a sunny day, which represents a vast improvement for autonomous vehicles.
Testing the camera in real life
The UIUC team took their latest camera innovation out in the field to test its performance under real-world conditions. “We were able to detect cars and pedestrians at distances three times higher then using regular cameras,” the research leader reports. “For example, driving through a foggy day, we could identify a car in front of us at about 10 meters with a regular camera, and with the polarization camera, we were able to detect it at 30 meters.”
Impact on the design of future image sensors
Gruev believes these latest findings can change the imaging sensor field by encouraging alternative ways of designing the next generation of imagers. He hopes that other researchers will particularly look at natural visual systems for inspiration. “Our work challenges the polarization imaging market because this technology improves current state-of-the-art polarization cameras by one-million times in terms of dynamic range.”
Outlook
The next step for Gruev and his colleagues will be to develop the necessary computer vision and machine learning algorithms for this particular type of sensor to enable autonomous object detection in foggy and rainy conditions and integrate it with self-driving cars.
The article “Bioinspired polarization imager with high dynamic range,” published In the research journals Optica, describes the new bioinspired camera in detail.
Written by Sandra Henderson, research editor Novus Light Technologies Today
Back to Features
