The Earth's oceans and seas are a source of enduring fascination and valuable resource, whilst also representing some of the harshest environments on the planet. Increasing our understanding of the subsea world is dependent on access to suitable sensors to measure and capture underwater images. However, obtaining a clear picture within this environment is a complex task, with many optical effects combining to pollute images and reduce contrast.
During 2015 the UTOFIA consortium was created within the European Commission Horizon 2020 framework to break the current paradigm of underwater imaging. Bringing together experts in imaging, illumination, optics, ocean science, and marine hardware, the consortium has assembled several new underwater camera systems which are capable of capturing high-contrast images and 3D information in a single compact camera platform.
Range gating captures time slices
Users of underwater cameras are well aware of how quickly an image can be degraded in less than ideal conditions. Ambient light is absorbed by the water and reduces dramatically with increasing depth, whilst muddy or turbid water contains many fine particles that scatter light, reducing the effectiveness of incorporating artificial light sources into cameras. Range-gating techniques can be used to address both of these problems, by using a pulsed light source synchronised with a short-exposure image sensor to capture sequential 'time-slices' of the scene. The resulting images can be processed to significantly increase contrast, by the removal of high intensity scatter close to the camera. A further benefit is that the 'time-slices' contain range or distance information, opening the route to 3D reconstruction of the underwater scene.
The basic principle of range-gating has been known for many years, although underwater systems incorporating the technology have often been large, bulky and expensive. limiting its use to a small community. The UTOFIA team has taken a holistic approach, addressing each component in the system, with the aim of creating a compact and cost effective camera platform, capable of operation by non-expert users. An ultra-compact laser (shown above) provides high-power pulsed illumination, highly efficient real-time image processing algorithms are implemented directly within camera hardware, and an entirely new 3D capable image sensor has been fabricated with the capability for very short exposure times and range-gating operation. The whole system is controlled from a simple surface control unit with a conventional wired connection to the underwater camera.
Packaged QVGA resolution ToF image sensor from Fraunhofer IMS, Duisburg. The sensor is capable of very fast gating and in-pixel accumulation.
Laser illumination with Q-switch technology
Within the project, laser specialist Bright Solutions of Italy, has miniaturised their proprietary Q-switch technology to produce a dedicated high power illumination source occupying a volume of less than 1.2 litres. The laser is capable of producing greater than 3 mJ optical pulses with a width of less than 2 ns at a wavelength of 532 nm, at kilohertz repetition rates and has a beam quality factor (M$^$) of 3.
Several challenges were addressed in the design of the source, including the development of a new Master Oscillator Power Amplifier (MOPA) based architecture, and fitting the complex internal optical bench and electronics into the tightly constrained volume. An important aspect is the thermal management of the laser achieved by active temperature sensing, together with a dedicated mechanical design where the laser housing mates perfectly to the outer underwater rated camera housing.
New time-of-flight image sensor
For the latest UTOFIA camera platform, an entirely new CMOS time-of-flight (ToF) image sensor has been designed and fabricated by the Fraunhofer Institute for Microelectronic Circuits and Systems (IMS) in Duisburg (Germany). The sensor has a resolution of 320x240 pixels and operates with two independent timing controls and outputs for each pixel. This architecture allows the sensor to operate in several different modes, including a short-shutter range gating mode, as well as an indirect time-of-flight mode where the two outputs are combined to directly calculate range. Very short exposure times in the range of tens of nanoseconds are possible, with fast rise and fall times, and high-repetition rate in-pixel accumulation of multiple exposures are all possible.
The new QVGA ToF image sensor has been designed into the UTOFIA platform by interfacing to existing camera electronics developed by Odos Imaging. This approach enables the UTOFIA camera to have two distinct variants, both built on a common hardware platform, with each incorporating a different image sensor, and providing the complementary features of high sensitivity, fast operation at lower resolution, or high-resolution images at lower frame rates.
Fully integrated UTOFIA underwater 3D camera system with a total volume of less than 7 litres.
To access the dual benefits of high-contrast images and 3D information, image processing experts at SINTEF in Norway have developed an adaptive 3D reconstruction algorithm. Range-gated sweeps are obtained for each pixel in the system, which are used to identify both the object distance as well as the position of optimal contrast on a per-pixel basis. A final image is then reconstructed consisting of the high-contrast grey scale overlaid with 3D information. In order to achieve this output at video rates, very efficient data processing and filtering was carried out on an internal FPGA within the operation.
Real-world applications
Typical use scenarios for the UTOFIA camera systems are as an attachment to an underwater Remote Operating Vehicle (ROV). Generally, the ROV is controlled by a user on the surface with a single robust cable connection, providing the data and control interface. A single surface control unit has been assembled by ATZI Technalia of Spain, which provides power-over-ethernet control of the camera, together with a simple GUI interface displaying a real-time video feed of the underwater scene.
Extensive tests of each camera platform have been completed, both in test tanks with controlled turbidity as well as sea trials. Removing contrast-reducing scatter from images with the on-board processing provides a distinct advantage in side-by-side comparisons with conventional underwater cameras. At the same time, range information can be overlaid on the image as a colour gradient, giving users a quick and readily visible indication of distance within the scene. Under remote operation, the range information can be used to guide an ROV around obstacles, taking into account the relative speed of the ROV, as well as estimate object dimensions visible within the image.
Field trial results of the UTOFIA camera compared to a standard underwater camera. The improvement in contrast achieved through range gating is visible in the upper image.
The latest UTOFIA camera platform is the third hardware iteration within the project, with each new development offering greater imaging power and flexibility in a smaller package for the end user. Key to the success of the project is the ability for each hardware component to be optimised within a unified system concept, informed by experts in the field. From the basic technology validation to the ongoing extensive characterisation and evaluation in real applications, the future of underwater imaging looks brighter and clearer.
Written by Chris Yates, CEO of Odos Imaging Limited
Back to Features
