Recently, however, scientists at the Singapore Polytechnic University developed an AI-based hybrid imaging system merging computer vision with deep learning networks that took this technology in a new direction: construction worker training certification. They aimed to minimize the mundane manual process of assessing apprentices, therefore saving labor hours and potentially improving the accuracy of test results.
In the construction industry, apprentices are required to go through extensive training and certification to ensure they have the necessary skills for a highly specialized trade, such as carpentry or plumbing. In order to be certified, apprentices build intricate assemblies from blueprints that must adhere to modern building requirements and be finished in a set amount of time. Apprentices' work is painstakingly evaluated by senior tradesmen, a task that can take hours.
Corner Detection Algorithm
In the study, the AI-based computer vision system devised at Singapore Polytechnic employs a corner detector algorithm available in Matlab image processing libraries. This algorithm is combined with deep learning techniques, along with a high-resolution camera, to measure the dimensions, orientation, and position of a complex wall assembly consisting of electrical boxes, switches, steel conduit, fittings, raceway, and wiring. Once assembled by an apprentice, each part is analyzed by the algorithm to determine any discrepancies from the blueprint.
Deep learning can detect the positioning of the electrical components and their reference corner points in each acquired image. Using a pixel-based filter, such as the Minimum Eigen Value Algorithm, and further filtering the results with the backing of deep learning output, the Singapore Polytechnic system detected corners of components very accurately.
System performance depends on the identification of individual components and the position of their corner points. Therefore, the imaging capabilities of the camera were critical. After researching several cameras, scientists chose an SVS-Vistek EXO 540CU3 industrial camera featuring a 24.5-megapixel color sensor featuring a global shutter. Equipped with a 50mm lens, the SVS-Vistek camera was set at the midline of the wall assembly with a resolution set at 5320 × 4600 pixels. Images were transferred at 15 frames-per-second (FPS) utilizing the camera's USB3 Vision interface. The camera was calibrated with checkerboard images, along with Matlab's camera calibration tools, for accuracy.
Automated System Processing
Using the acquired image, the Singapore Polytechnic system first detects the type of wall assembly being assembled with its trained deep learning model so it can perform computations accordingly. Next, object corners are detected that act as reference points in the measurement of component dimensions, alignments, and angles.
Before testing, a template of the finished wall assembly is uploaded into the system as a CAD drawing. Detected features are compared against the template so discrepancies can be identified. Finally, the system generates a report detailing the inspection results that sre used in certification.
The scientists acquired a collection of images with drastically varying lighting conditions and camera angles to further assess the accuracy and consistency of the trained network, keeping in mind that the inspection system will operate in dynamic "real world" environments rather than controlled labs. Underscoring the image quality of the system, the majority of these test results were above 93% accuracy.
Future System Development
Certification of apprentices can be an arduous, time-consuming process requiring multiple experts to be on-site to monitor testing. This new computer vision solution can reduce certification time significantly compared to manual inspection. Additionally, it is possible to capture and process images remotely, which eliminates the need for travel.
According to Singapore Polytechnic, the system opens up new avenues to explore other certification processes in the trades besides electricians, such as apprentices in plumbing, HVAC, and carpentry. The scientists intend to create a general-purpose system with multiple configurations that can be customized by instructors to analyze diverse apprentice work.
Back to News
