In the past, systems integrators were faced with a limited set of choices when considering how to build vision systems to capture three-dimensional images of objects. Either they could deploy a passive triangulation system with two cameras, or an active triangulation system in which a single camera is used to capture an image of a target illuminated by structured light.
Over the past decade, however, a relatively new class of device called the Time of Flight (ToF) camera has emerged that looks set to provide designers of vision systems with a further option. Potentially, the ToF camera could be used in a number of specific industrial applications to capture both a 2D representation of a scene as well as the depth data associated with it.
Initially, the first slew of ToF cameras that were introduced into the market were based on measuring depth using a technique known as continuous-wave (CW) modulation, which involves calculating the phase differences between emitted light from a light source and the light captured by the imager in the camera. Such cameras are now available from companies such as Mesa Imaging, Ifm Electronic and pmd.
Pulsed modulation technique
Recently, however, a new breed of ToF camera has been introduced that employ a somewhat different concept. These cameras, which are available from Odos Imaging and Basler, use what is known as a pulsed modulation technique.
Here, the camera triggers an illumination unit, which then emits intense pulses of light which are reflected from objects within the scene. Pulses reflected from objects that are closer return to the time-of-flight sensor in the camera before objects that are further away. By capturing a number of consecutive frames as each pulse returns to the imager in the camera, it is then possible to extract range from the scene. As such, the frame rate of the camera directly corresponds to the target working range of the camera.
Theoretically, a pulsed ToF camera could use any wavelength of light to illuminate a scene. However, both the cameras currently available from Basler and Odos Imaging illuminate scenes and capture light in the near-IR spectrum, although Basler does plan to release an RGB ToF camera in the future.
According to Phillip Dale, Product Manager of New Business Development at Basler, the most important reason behind Basler’s decision to start development with a near-IR light camera is that the depth image resolution of such NIR cameras are four times higher than their RGB counterparts. But there are other benefits too. NIR light is less intrusive than visible light, since it is invisible to the human eye and hence reduces operator fatigue. Another is that such cameras are impervious to ambient light and can even operate in dark environments.
ToF cameras have certain advantages over both passive and active triangulation systems. First, they do not suffer from the correspondence problem inherent with those other techniques, since there is no need for a vision system to determine which features in the image captured by one camera corresponds to the same feature in another. Secondly, it is unnecessary to measure extrinsic parameters such as the relative position and orientation of two cameras, since only a single camera is used. Lastly, since objects are automatically illuminated by a light source which is synchronized to the camera, there is no need to integrate an additional external light source into the system.
However, ToF cameras also have evident disadvantages that should not be overlooked by systems integrators either. Earlier continuous-wave (CW) modulation entries into the market, for example, sported very low resolution sensors compared with those cameras more typically used in the machine vision industry, rendering them unsuitable for use in many industrial applications. And while new pulsed modulation cameras from both Basler and Odos offer a much improved resolution, both technologies still suffer from the fact that the depth resolution that they offer at frame rates of 30 frames/sec is no more than +/- 1cm over working distances between 0.5 to 5m.
Basler’s Phillip Dale acknowledges the fact that earlier designs would be inappropriate for use in many applications in the vision systems industry where highly accurate depth measurement is required. However, he says that the higher VGA resolution of the prototype that his company has developed will now make it applicable in certain industrial applications where previous 3D solutions such as laser scanning systems have proved too expensive. Furthermore, he claims that the 5m operational range of the camera is more than enough to cover over 80 per cent of all his customer's applications.
According to Ritchie Logan, the VP of business development at Odos Imaging, the new breed of pulsed light ToF cameras naturally lend themselves to applications in logistics automation applications.
Dimensioning packages prior to palletization
Many existing robotics vision systems at work in distribution depots, packaging plants and warehouses, for example, are regularly used to dimension packages prior to palletization.
In February, Odos Imaging demonstrated how its ToF camera and BBS-1000 software could be used to capture and dimension an object such as a box at the LogiMAT 2015 show in Stuttgart. Images courtesy of Odos Imaging.
Having done so, a computer-based system driving a robot can use that data to enable software-based space optimizing routines to determine the most efficient palletizing sequence. From these, the robot can build up the pallet from products of either similar, or dissimilar, sizes.
Such automated palletization systems are now becoming commonplace in many warehouse environments. And it is with the developers of such vision controlled robotics systems that both Odos Imaging’s Ritchie Logan and Basler’s Phillip Dale see a definite market opportunity for ToF cameras.
Basler’s ToF cameras can be used in logistics applications to determine the volume of packages as well as read the bar codes on them. Image courtesy of Basler.
Deploying a ToF camera in such systems, the dimensions of a package to be palletized could be captured while the package was stationary. Furthermore, if more than one package was present in the field of view of the imager at any one instant, then all the packages could be dimensioned simultaneously. More importantly, however, since the cameras can capture high resolution images of the package as well as depth data, a ToF camera could also identify one or two dimensional monochrome or color bar codes on the packages, enabling them to not only be palletized efficiently, but also tracked by a warehouse management system.
Needless to say, the new breed of ToF cameras may also find a plethora of applications beyond the world of logistics too, in identifying parts on robotic pick and place machines, as well as calculating the volumes of packages moving down production lines. And while the ToF cameras may never offer the levels of accuracy of stereo vision or structured light systems, they will undoubtedly complement the range of 3D options open to a systems integrator and find their own market niches where their resolution and depth accuracy is acceptable.
Written by Dave Wilson, Senior Editor, Novus Light Technologies Today
Photo at top: In logistics applications, ToF cameras could be employed in a vision-based robotic palletizing system to determine the volume of products before they are stacked onto a pallet. Image courtesy: Axium, Montreal, Canada.
Back to Features
