EE368/CS232 Digital Image Processing Winter 2017-2018 Lecture Review and Quizzes (Due: Wednesday, January 31, 1:30pm) Please review what you have learned in class and then complete the online quiz questions for the following sections on OpenEdX 1 : (a) Morphological Image Processing Homework #3 Released: Monday, January 22 Due: Wednesday, January 31, 1:30pm 1. License Plate Recognition (Total of 13 points) In this problem, we study how to recognize the characters on a license plate using morphological image processing. Please download the two images hw3_license_plate_clean.png and hw3_license_plate_noisy.png from the handouts webpage. Please also download the file hw3_character_templates.zip, which contains 36 separate images of the alphanumeric characters ABC XYZ0123456789 in the same font as the EIBLT8H written on the license plate. hw3_license_plate_clean.png hw3_license_plate_noisy.png (a) Binarize the clean license plate image, the noisy license plate image, and the template images, so that the large characters in the middle of the plate appear as white and the background appears as black. Choose the threshold by applying Otsu s method on the clean license plate image, and use the same threshold for all binarization operations in this problem. Submit the binarized clean and noisy license plate images and the binarized template images. (2 points) 1 https://suclass.stanford.edu/courses/course-v1:engineering+ee368+winter2018 1
(b) Perform character detection by erosion in the binarized clean license (MATLAB function: imerode). To eliminate the effects of slight mismatches along the character boundaries, compute a slightly eroded template which is erode(template, 3x3 square), and then use the eroded template as the SE in the erosion detector. For each template that generates a nonzero detection result, dilate the eroded license plate image (MATLAB function: imdilate) by the template for visualization purposes and submit the dilation result, like shown in the following example for the template B. Which characters are detected by this approach? Are there any false positive detections, and if so, why? (3 points) (c) Repeat Part B, except use a hit-miss filter in place of erosion for the detector (MATLAB function: bwhitmiss). Use the same SE as in Part B for the foreground SE. For the background SE, use the difference dilate(template, 5x5 square) dilate(template, 3x3 square) which extracts a thin outline around the character. Comment on the advantages of the hitmiss detector compared to the erosion detector. (3 points) (d) Repeat Part C, except replace the clean license plate image by the noisy license plate image. Comment on how the noise affects the hit-miss detector s accuracy. (2 points) (e) Repeat Part C, except replace the clean license plate image by the noisy license plate image and replace the hit-miss filter by the minimum of two rank filters: rank-filter(binary image, p1, SE1) and rank-filter(not[binary image], p2, SE2). Here, SE1 and SE2 are the same foreground and background SEs used in Part C, and p1 and p2 indicate the two ranks. Note that if p1 = p2 = 1, then the minimum of the two rank filters is the same as the hit-miss filter. Choose and report the ranks p1 and p2 that enable correct detection of the characters in the noisy license plate without false positives. The MATLAB function for rank filtering is ordfilt2. (3 points) Note: Please attach relevant MATLAB code. 2
2. Plant Leaf Classification (Total of 8 points) Applications such as LeafSnap 2 use plant leaf classification algorithms to identify different plant species in nature. Please download the images hw3_leaf_training_{1,2,3,4,5}.jpg and hw3_leaf_testing_1.jpg from the handouts webpage. hw3_leaf_training_{1,2,3,4,5}.jpg hw3_leaf_testing_1.jpg Design and implement an image processing algorithm to automatically determine which of the 5 training images is most similar to the testing image. Clearly describe the steps of your algorithm, including intermediate results if they help to explain the process. Your algorithm should generate a numerical similarity score, or alternatively a distance score, between a pair of leaf images, e.g., similarity(training image #1, testing image) = 0.8, similarity(training image #2, testing image) = 0.6, etc. Report the similarity score or distance score between every training image and the testing image, i.e., 5 numbers. Does your algorithm assign the highest similarity score, or equivalently lowest distance score, to the correct training leaf? Creative solutions are welcome. (8 points) Note: Please attach relevant MATLAB code. 2 http://www.leafsnap.com 3
3. Sharpness Enhancement by Dilation and Erosion (Total of 7 points) In this problem, we use an iterative grayscale morphological image processing algorithm to enhance the sharpness of structures in a blurry image. (a) Please download the image hw3_road_sign_school_blurry.jpg from the handouts webpage. This image shows a school crossing road sign taken by an out-of-focus camera. Apply 10 iterations of the following algorithm. Design and report your structuring element. Submit the resulting image after 10 iterations. Comment on which features in the image have been made sharper. (MATLAB functions: imdilate, imerode, strel). Im := Input Image For Iteration = 1:NumIterations Im_d = dilate(im, W) % Note that this is grayscale dilation Im_e = erode(im, W) % or erosion with structuring element W Im_h = 0.5(Im_d + Im_e) End % Perform the following test for each pixel If Im > Im_h Im := Im_d Else Im := Im_e End (b) Submit plots of the intensity profile in row 338 after iterations 1, 2,, 6. For example, the following is a plot of the intensity profile in row 338 in the original image. Comment on how the algorithm iteratively changes the intensity profile. (3 points) Intensity Profile in Row 338 for Original Image 1 0.5 0 50 100 150 200 250 300 350 400 450 500 550 x Note: Please attach relevant MATLAB code. 4
4. Noise Reduction by Median Filtering (Total of 8 points) In this problem, we study how to use different types of median filtering to reduce salt-and-pepper noise in vintage photographs. (a) Please download the images hw3_building.jpg and hw3_train.jpg from the handouts webpage. Apply median filtering with a 3x3 window and a 5x5 window (MATLAB function: medfilt2). Display and submit the resulting images. For each window size, comment on how effectively the noise is reduced while sharp edges and features in the image are preserved. (b) Given a set of input values f 1, f 2,!, f and weights w N 1,w 2,,w N, weighted median filtering repeats the value fi by wi times and then computes the median of all the repeated values: Apply your weighted median filter to each image. Display and submit the resulting image. Compare to the results from part (a) in terms of how effectively noise is reduced while sharp edges and features are preserved. Note: Please attach relevant MATLAB code. g = median( w 1 f 1,w 2 f 2,,w N f N ) w f = f, f,, f Implement a weighted median filter with the following 5x5 window of weights, where the brackets around 4 indicate the center of the sliding window: 0 1 1 1 0 1 2 2 2 1 1 2 [ 4] 2 1 1 2 2 2 1 0 1 1 1 0 w times 5
Bonus Problems You can get 100% of the credit on this homework without doing any of the bonus problems. You can get 4 extra points if you solve one of the two bonus problems. If you do so, please submit your solution for only one of them (otherwise only the first will be graded and we will ignore the second). Bonus #1: Android with OpenCV (Total of 4 points) Read Tutorial #2 (OpenCV for Android Setup) on the class Android webpage. Download and install all the necessary software as explained in Tutorial #2. Select one of the examples found on the class GitHub repository (https://github.com/ee368/ee368-android-samples) that uses OpenCV: (i) locally adaptive binarization (Tutorial2/AdaptiveBinarization) (ii) human face detection (Tutorial2/FaceDetection) (iii) local feature keypoint extraction (OtherSamples/FeatureKeypoints) (iv) edges/lines/circle detection (OtherSamples/EdgesLinesCircles) Run the example on an Android device and take a screenshot. The screenshot should clearly demonstrate a result of the underlying method or algorithm (e.g., local feature keypoints overlaid on an object). Submit your screenshot. 6
Bonus #2: OpenCV Library (Total of 4 points) This problem guides you through the process of setting up the OpenCV library on your computer. After the installation, you will be able to apply image processing functions implemented in OpenCV on images/videos you load into MATLAB. (a) Method 1: Mac OS X only Install the latest version of Xcode from the app store. Install Homebrew (https://brew.sh) Install OpenCV using Homebrew package manager using the commands below. brew install opencv3 brew link opencv3 Method 2: Any platform Download and install OpenCV: https://opencv.org/releases.html Supported versions: OpenCV 2.4 through 3.3 For OpenCV versions 3.0 and up make sure to also install opencv_contrib (https://github.com/opencv/opencv_contrib/releases) **Important** opencv_contrib version MUST match OpenCV version! Windows: the libraries have already been pre-built and are available through DLL files. Mac/Linux: the libraries must be generated: http://docs.opencv.org/doc/tutorials/introduction/linux_install/linux_install.html (b) In MATLAB, please type mex setup and select one of the available compilers. For Windows users, you can download Visual C++ Express to obtain a suitable compiler: https://www.visualstudio.com/ For Mac users, you can download XCode to obtain gcc as the compiler: https://developer.apple.com/xcode/ (c) This mexopencv toolkit provides a MATLAB MEX interface to many OpenCV functions. If you are using OpenCV 3.3, please download and install the toolkit from: https://github.com/kyamagu/mexopencv If you are using an older version of OpenCV, please install the version of the toolkit that matches your OpenCV version. For instance, if you have OpenCV 2.4, the appropriate toolkit version can be found at: https://github.com/kyamagu/mexopencv/tree/v2.4 7
**Important** Mexopencv version MUST match OpenCV version Please unzip the mexopencv folder to a convenient location on your hard drive. Then, please read the README file contained inside the folder to get specific installation instructions for your operating system. For Mac users, pkg-config can be obtained either using Homebrew (brew install pkg-config) or as part of the MacPorts package: http://www.macports.org/ For Mac users, the following page may be useful for troubleshooting issues with mex: http://www.mathworks.com/matlabcentral/answers/14222-mex-can-t-find-gcc-in-path (d) Please complete one of the following tasks. (i) Download the images hw3_faces_1.jpg and hw3_faces_2.jpg from the handouts webpage. Detect faces in each image using the Viola-Jones face detector that is already implemented in OpenCV. Reference code exists on the mexopencv webpage and in the samples folder. Submit a version of each image where the bounding boxes of the detected faces are clearly overlaid on top of the image. (ii) Download the images hw3_cd_cover_1.jpg and hw3_cd_cover_2.jpg from the handouts webpage. Detect SURF keypoints in each image. Reference code exists in the mexopencv samples folder. Keep only the 150 SURF keypoints with the largest Hessian response values. Submit a version of each image where the SURF keypoints are clearly overlaid on top of the image, with each keypoints position, scale, and orientation clearly indicated (function: cv.drawkeypoints). (iii) Download the image hw3_paper.png from the handouts webpage. Apply locally adaptive thresholding (function: cv.adaptivethreshold) to binarize the image, 8
so that the text appears as black pixels and the blank space on the page appears as white pixels. Report the block size you select for locally adaptive thresholding. Submit the binarized image. 9