Using surface orientation, particularly the effect it has on reflected light, photometric stereo intended for industrial applications produces a contrast image that accentuates local 3D surface variations.
This method is attracting a significant amount of attention due to low-cost multi-light solutions, new specialized algorithms, and an increasing awareness of the need for good lighting to ensure machine vision success.
The photometric stereo technique can show surface defects on textured surfaces such as synthetic leather.
Real-world objects have three dimensions—height, width, and depth. For automated systems like robots to operate successfully, they should have the ability to “see” in these three dimensions. Machine vision systems, which include a camera, lighting, and a PC for image processing, give this “sight” to such automated systems.
However, decreasing the amount of data that must be processed to locate and analyze an object correctly is one of the major challenges facing the machine vision industry.
In order to reduce the data, machine vision designers would use black-and-white cameras, filters, and lights that address color machine vision applications. Since the grayscale images that are generated contain fewer amounts of data, they can be processed more quickly.
In a similar way, engineers would develop motion control systems and mechanical fixtures in order to solve a traditional 3D application using a 2D machine vision solution.
While designers gain greater processing power with today’s microprocessors, field programmable gate arrays (FPGAs), and graphic processor units (GPUs), processing power still remains finite. The most cost-effective solution for 3D applications could be provided by a nascent machine vision technique known as “photometric stereo.”
3D Vision at a Glance
The need to reduce the amount of data needed in color and 3D applications has been eased out by affordable processing power.
An example is provided by integrated laser triangulation systems for conveyor-based 3D systems, which have been facilitated by low-cost data processing, lasers, and optics. These systems are capable of producing tens of thousands of 2D profiles every second during the process of creating a 3D object map.
Another option is offered by new time-of-flight cameras. These provide low-resolution 3D maps for a wide range of applications, without the associated safety risks of laser illumination.
For 3D projects with a larger area, multiple pictures of the same object from different locations can be taken by mounting single-camera photogrammetric systems on the end of a robot.
Using these images, the 3D position of every pixel in the image can be established based on a predetermined geometric relationship between the camera and the object. With respect to large-area 3D inspections, two cameras are aligned side-by-side in order to imitate human eyes and capture the 3D information.
Conversely, qualitative data is potentially useful for inspecting objects without a large field of view at high speed, whereas quantitative 3D data isn’t invariably needed for measurement purposes. This is where the photometric stereo technique comes in.
Photometric Stereo Advantages
Determining the height of any given pixel is not the chief concern of photometric stereo; instead, this technique produces a contrast image accentuating local 3D surface variations by using 3D surface orientation and its effect on the reflected light. The variations shown may not be visible if traditional 2D imaging is used.
When using photometric stereo solutions, it is not necessary to know the exact 3D association between the object tested and the camera, nor is it necessary to use two cameras to capture the 3D data. Rather, a single camera with multiple illumination sources is employed.
By observing an object under different lighting conditions, its surface is established during the photometric stereo technique. The principle behind this method is the observation that the amount of light a surface reflects depends on the surface’s orientation with regards to the light source and the observer.
Due to the new specialized algorithms, an increasing awareness of the necessity for good lighting to ensure machine vision success, and low-cost multi-light solutions such as Smart Vision Lights’ LED Light Manager (LLM) (which allows four lights to be controlled via a simple browser-based interface at a cost lower than a frame grabber or smart camera break-out box), the application of photometric stereo in industrial applications is inviting a great deal of attention.
At present, the exclusive benefits of photometric stereo applications are enabling many traditional industrial inspection applications, which were previously difficult or impossible to solve.
Application: Clips and Tires
Machine vision systems have always had problems reading raised letters on parts. This example shows a plastic connector that has numerous functional surface features, as well as a directional symbol and the number 2. No contrast is seen because there are no differences between the raised letter and the material using which the clip is made.
Manufacturers have used laser triangular systems on larger objects, for example, tires, to create a 3D surface map. These laser-scanning systems are often a complex and costly solution for 3D measurements, in spite of becoming much more effective and integrated recently.
In Figures 1 to 4, Smart Vision Lights’ linear miniature (LM) LED lights are positioned at 90°, 180°, 270°, and 360° around the tire’s perimeter to illuminate the black plastic clip. An LLM regulates the LED lights. As the Matrox camera activates each exposure, the LLM triggers a light from a different direction.
Figure 1. (Photo courtesy of Matrox Imaging)
Figure 2. (Photo courtesy of Matrox Imaging)
Figure 3. (Photo courtesy of Matrox Imaging)
Figure 4. (Photo courtesy of Matrox Imaging)
Each image is fed by the camera into a PC that runs an image library photometric stereo registration algorithm, combining all corresponding pixels, establishing local surface properties, and producing one or more types of composite images from these. A contrast image of the local 3D geometries and an albedo image (Figure 5) are some examples.
Figure 5. (Photo courtesy of Matrox Imaging)
More is revealed by these composite images compared to any of the constituent images alone. The edges forming the black-on-black lettering on the surface of the clips are clearly shown in the resulting composition. Also shown are the edges of the various injection-molded parts containing the whole.
Application: Synthetic Leather Perforations
In the next example, four more pictures of a synthetic leather material are shown (Figures 6 to 9). Leatherette, similar to the organic material that it mimics, has significant surface texture. Visualizing the entire surface texture over the full image, let alone a computer, is almost impossible for the human eye.
Figure 6. (Photo courtesy of Matrox Imaging)
Figure 7. (Photo courtesy of Matrox Imaging)
Figure 8. (Photo courtesy of Matrox Imaging)
Figure 9. (Photo courtesy of Matrox Imaging)
The warp of the material creates strong shadows in each constituent image while it lies on a supporting substrate, whereas other parts of the image tend toward saturation because of the strong light reflection.
The photometric stereo registration algorithm (Figure 10) produces a final composition showing a texture that is evenly illuminated over the full field of the camera’s view, with a sharp contrast across each crevice and the highlighting of holes.
Figure 10. (Photo courtesy of Matrox Imaging)
It is also possible to use the photometric stereo technique on pores of metal-machined surfaces, such as engine heads. Other areas which will clearly be at an advantage as a result of these low-cost photometric stereo solutions are cast parts, laser marking, and direct part-marking systems like dot peen.
Photometric Stereo Outlook
The 3D world in which humans live will continue to depend on 3D vision solutions. The global 3D machine vision market is projected to grow at a CAGR of 9.5% between 2017 and 2024, increasing its annual value from $15.4 billion to nearly $32 billion, according to DBMR Research.
However, high installation costs and a lack of technical knowledge constitute the most significant challenges to the growth of 3D machine vision market. As demonstrated by the packaged photometric stereo registration tool and one-click programming of the LLM LED light manager, the machine vision industry is ready for the next major step in 3D machine vision.
This information has been sourced, reviewed and adapted from materials provided by Smart Vision Lights.
For more information on this source, please visit Smart Vision Lights.