Detection and Verification of Missing Components in SMD using AOI Techniques

Transcription

1 , pp Detection and Verification of Missing Components in SMD using AOI Techniques Sharat Chandra Bhardwaj Graphic Era University, India Abstract Recognition of pattern in an image is work of ease for a human eye but it is a big challenge for a machine to detect that. This comes with the trend of autonomous machines with the utilization of image processing techniques. The same problem is faced in automated inspection for SMD (surface mounted device) to find mounted components. This paper is concentrated on image subtraction operation for finding the component, histogram matching for verification and template method is being used for detection of presence of components. The resultant images from these operations are further analyzed for finding the location of missing component. Keywords: Automatic Optical Inspection (AOI), SMD (Surface Mounted Device), Image Subtraction, Histogram Matching, Template Matching 1. Introduction In today s scenario the industries are moving towards the Automated Optical Inspection system (AOI) for PCB inspections as speed and accuracy are important parameters for PCB assembling. PCB inspection includes the most complicated process like detecting a missing component; solder bridging, lack of soldering, misalignment, etc., SMD basically deals with the mounting of components like resistors, transistors, capacitors and IC s. AOI technique is proposed to facilitate easy offline and online inspection of PCB. Teoh et al., [1] identified the missing component and misalignment by using histogram, Loh. [2] used run length encoding technique to predict the wrong integrate circuit component. Pattern matching is used by Peng. [3] to detect missing components. Here AOI method is adopted for the inspection of imminent defects in the PCB assembling. Inspection process flow chart comprises of five phases as shown in Figure 1. Image Acquition Of Source And Missed Component Image Background Substraction Template Matching Estimate The Region Of Interest Histogram Verification Detection Of Missing Component Figure 1. Flow Chart of the Inspection Process ISSN: IJCG Copyright c 2016 SERSC

2 2. Theory and Methodology 2.1 Image Acquition For the processing of missing component, the image is acquired in such an environment where the intensity of light is uniform throughout the PCB. This is done to avoid the patch problem that appears due to the shadow of the mounted component. As humans perceive color through wavelength-sensitive sensory cells called cones. There are three different types of cones, each with a different sensitivity to Electromagnetic Radiation (light) of different wavelength. One type of cone is mainly sensitive to red light, one to green light, and one to blue light. By combination of three basic colors (red, green and blue), three types of cones are obtained. This is the reason why color images are stored as three separate image matrices; known as RGB format as shown in Figure 2. In grayscale images, it is not easy to decide the involvement of different colors. The total amount of emitted light can be estimated for each pixel. Little light intensity gives dark pixels and higher intensity is perceived as bright pixels. When converting an RGB image to grayscale, the RGB values for each pixel reflect the single value brightness of that pixel. a. RGB Image b. Gray Scale Image Figure 2. Conversion of RGB Image(a) to Gray Scale Image(b) 2.2 Image Subtraction To detect the missing components from the SMD image, the image subtraction methodology is adopted as shown in Figure 3. Image Subtraction of one image from another is performed by taking out the absolute difference of the two images. Figure 3. Subtraction Operation of Two Images 2.3 Construction of Region of Interest (ROI) The ROI technique is commonly used in many inspection application areas. For example, in medical imaging, for find the boundaries of a tumor, for the purpose of 14 Copyright c 2016 SERSC

3 measuring its size etc. In geographical information systems (GIS), a region of interest can be taken as an area selection from a 2D map. In machine vision and optical character recognition, ROI locate the borders of an object under consideration. The different parameters of the selected image can be extracted such as x-axis and y-axis coordinates area, centroid and etc. as the ROI coordinates are shown in Figure 5. The x-y coordinates and the area are used to locate the region of the missing component in the SMD. This area can further be utilized to identify the different components mounted on PCB such as IC s, capacitors and resistors. After defining the region of interest the exact location of the specific component is being found out through the process of template matching. Moreover, once the coordinates of each component is determined, it become possible to calculate the boundary parameters of every component. 2.4 Template Matching Figure 5. ROI Offset Diagram For finding small parts of an image template matching techniques are commonly used, furthermore it can also be utilized to detect edges in images. Template matching has two approaches depending upon them their applications diverge: Feature-Based Matching Template-Based Matching. The feature-based approach, use the features of the search image such as edges or corners, as a matching metrics, to find the best matching location in the main image. The template-based approach is applied in this paper with normalized cross-correlation mathematical operation that determines the best location by testing all or a sample of selected/specified locations (ROI) within the main image with which the template image may match. Normalized cross-correlation can be calculated by following certain steps: Cross-correlation is calculated either in spatial or the frequency domain, depending on image size. Calculate intermediate sums by computing the running sums. Intermediate sums are used to normalize the cross-correlation for obtaining correlation coefficients. The basic formula for Normalized cross-correlation: ρ m,n = I,Jj [K(I,J) <K>][S(I m,j n) <S>] I,J[K(I,J) <K>] 2 I,J[S(I m,j n) <S>] 2 Copyright c 2016 SERSC 15

4 Where: < > represents the mean K represents the sub-image S represents the main image I, J represent the coordinates Before applying this approach the images should be in grey scale. Template matching uses a cross-correlation mask (template), applied to a specific region of the search image, which we want to detect. The cross-correlation output will be high where large image values get multiplied by large mask values i.e., image structure perfectly matching the mask structure. When a number of the template image constitutes the matching image, a templatebased approach may be adopted. A template matching operation and the resultant images for alphabet K is shown in Figure Histogram Verification Figure 6. Template Matching of Alphabet K An image histogram is a graphical representation of tone distribution in a digital image. It is a plot of number of pixels for each tone value. The horizontal axis of the graph represents the tone variations, while the vertical axis represents the number of pixels in that tone. Histogram of a very dark image will have its maximum data points on the left side and center of the graph and vice-versa is possible. Here in this paper the histogram change determined for the main SMD image and the inspected SMD image for missing component. The variation of the tone values of the pixel shows a defect in the SMD and by analyzing such differences we can easily identify the specific missing component. 3. Results and Analysis In this paper, the missing component is being detected from an SMD through the process of Automated Optical Inspection (AOI) Techniques. At the initial phase, the acquired images of the Reference SMD and the Inspection SMD are converted to gray scale as shown in Figure 7 and Figure Copyright c 2016 SERSC

5 Figure 7. Reference SMD Image Figure 8. Inspected SMD Image in Gray Scale 3.1 Finding of Missing Component By using image subtraction process, all the missing components of the inspected SMD can be predicted. As in Figure 9, the white pixels in subtracted image, shows that there is a rectangular component i.e., yet to be mounted on the SMD. Hence, the larger number of white pixels on different locations in the subtracted image, the larger will be the possibility of missing components in the inspecting SMD. The pure black image obtained through subtraction indicates that the no component is missing. By analyzing the resultant image the missing components can be found. Figure 9. Image after Image Subtraction Represents the Missing Component in Inspecting SMD Copyright c 2016 SERSC 17

6 3.2 Finding the Location of Missing Component and Verification The construction ROI clearly indicates the respective coordinates of each component mounted on the reference SMD. It is also possible to calculate the total number of components to be mounted and inspected on a particular PCB. The resultant image (Figure 10) is shown after selection of ROI for components i.e., chips. In this image the blue rectangular box shows the region of component that are going to be inspected. These ROI s can be selected either by providing the coordinates of component in SMD reference image area or by drawing the rectangular box around component. For obtaining the reference coordinates rectangular boxes are drawn around component. These coordinated are used to inspect the component presence in the inspected SMD image. Figure 10. Selection of ROI for Components in Reference Image Each component at different location has different intensity profile and these profiles are being used to find the presence of component. For obtaining this intensity profile the ROI of each component is extracted from the rest of image (Figure 11) and later on histogram of components are obtained (Figure 12). These histograms of component act like an intensity profile and matched to the histogram of inspected image component located at same ROI location. In another word these are reference database which are compared to find defect in inspected SMD. Figure 11. Extraction of ROI of Component from Rest of Image 18 Copyright c 2016 SERSC

7 Histogram of GL2000 CHIP Image (I) Histogram of SUNPLUS CHIP Image (II) Figure 12. Histogram of (I) Component in Figure 11(a) and (II) Component in Figure 11(b) For finding the presence of component in the inspected image (Figure 13) histogram profile of components are obtained at different ROI. These histograms are compared one by one in terms of location so that change in histogram profile at certain location can be easily identified. The resultant histograms of component in inspected SMD, after compared with reference histogram are shown in Figure 14 and Figure 15. SMD Image with Missing Component Figure 13. Image of Inspected SMD By analyzing these figures, a resultant change in intensity at first component from reference histogram is observed i.e., intensity peaks in histogram (Figure 14) but in case of second component there is no resultant change from reference histogram. i.e., no intensity peaks (Figure 15). Thus a huge change in resultant histogram at particular location could be considered as missing component at that location. Copyright c 2016 SERSC 19

8 2500 Histogram of Subtracted Image when GL2000 CHIP is not there Figure14. Resultant Histogram of First Component Location after Comparison from Reference Histogram 3500 Histogram of Subtracted Image when SUNPLUS CHIP is there Figure 15. Resultant Histogram of Second Component Location after Comparison from Reference Histogram 3.3 Finding of Missing Component by Template Matching Apart from these two methods, the paper is also dealing with a third method of verification i.e., of Template Matching. The template images of component are obtained from reference SMD image, shown in Figure 16. These templates are being used to find the presence of component in the inspected SMD. Figure 16. Template Image of Components The normalized cross-correlation function is applied on template matching reduce the number of sampling points by reducing the template images by the same factor. As shown in Figure 17, normalized cross-correlation image is obtained which indicates the exact location of the present component with the help of a blinking white pixel. If there is no such blinking pixel as in Figure 18, it clearly indicates that there is no such component in the investigating SMD i.e., the component is missing. 20 Copyright c 2016 SERSC

9 Figure 17. Normalized Cross-Correlation Image for Present Chip Figure 18. Normalized Cross-Correlation for Missing Chip The highest correlation value of the component i.e., present is The row column coordinates of the best match is the cross correlation of second component are The row coordinates of the Best Match in Original Matrix of second component are Once an exact location is achieved through the above process of cross-correlation, the components that are present in the SMD are highlighted with a colored line at its respective co-ordinates as shown in Figure 15. It clearly indicates that there is no such component in the investigating SMD i.e., the component is missing. Once the exact location is achieved through the above process of cross-correlation, the components that are present in the SMD are highlighted with a colored line as shown in Figure 19. Figure 19. Detection of Present Component in the Inspected SMD Copyright c 2016 SERSC 21

10 On the contrary, the missing component are not highlighted in the investigating SMD that yet to be mounted on the respective PCB. 4. Conclusion and Future Scope In this paper, results of different AOI techniques i.e. Image subtraction, Histogram matching, Template matching are analyzed for detection of missing component. By image subtraction operation it can be found that whether any component is missing or not. Histogram matching results are analyzed finding and verification the missing component, and template matching results detection of presence of components. Furthermore the resultant images from these operations are further analyzed for finding the location of missing component. These obtained results are obtained specific lighting condition. Thus effect of lightning condition on results obtained could be further analyzed. Also different type of component which have different geometrical properties could be considered for inspection. References [1] C. A.B. Mello and D. C. Costa, A Complete System for Vehicle License Plate Recognition, IEEE, (2009). [2] B. H. Friemel, L. N. Bohs and G. E. Trahey, Relative Performance of Two-Dimensional Speckle- Tracking Techniques: Normalized Correlation, Non-Normalized Correlation and Sum-Absolute- Difference, IEEE Ultrasonics Symposium, pp [3] H. Lin and J. Si, Region-Of-Interest detection and its application to image Segmentation and compression, IEEE, (2007). [4] S.-C. Lin and C.-H. Su, A visual inspection system for surface mounted devices on printed circuit board, IEEE, (2006). [5] M. Moganti and F. Ercal, Automatic PCB Inspection Algorithms A Survey, Computer Vision and Image Understanding, vol. 63, no. 2, (1996), pp [6] D.-B. Perng and C.-P. Liu, Advanced SMD PCB vision inspection machine development, th IPPR Conference on Computer vision, Graphics and Image Processing. [7] C.-J. Yan, A New Auto-Inspection System for SMD PCB by Vision Inspection Technique, Master Thesis, Department of Industrial Engineering and Management, National Chiao Tung University, Hsin- Chu, Taiwan 30010, ROC, (2000). [8] H.-H. Loh and M.-S. Le, Printed Circuit Board Inspection Using Image Analysis, IEEE Transactions on Industry Applications, vol. 35, no. 2, (1999), pp [9] T. H. Cormen, C. E. Leiserson, R. L. Rivest and C. Stein, Introd. To Algorithms, MIT Press, (2001). 22 Copyright c 2016 SERSC