Android Test Apps documentation

Transcription

1 Uncanny Vision Android Test Apps documentation Revised on: 6th Oct 2014

2

3 Contents Introduction Image Recognition Demo Introduction How the App works How to install Setting Reference Image How to test Which images work well UncannyCV Demo Introduction How to install How to test A short note about each demo Canny Edge detection - Pedestrian detection Vehicle detection - Detecting cars from the back side Face detection Optical Flow - Lucas Kanade Optical Flow. Semi-Dense Optical Flow Tamper Detection Kmeans - Sobel Lane Departure Fast Keypoints - FAST9 corner detection algorithm Limitations

4 1. Introduction This document explains in some detail about the following Uncanny Vision Android test Apps and it usage. Image Recognition UncannyCV Demo 2. Image Recognition Demo 1. Introduction This app uses a keypoint detection and matching algorithm to recognize 2D markers, based on a reference image(more details in next section on how to set the reference image) it has in its database. Would be useful in following 1. Augmented Reality applications for recognizing markers 2. In printers it could be used for recognizing special images(like watermarks) and then deciding whether the document should be allowed to be printed Please test this app on your Android phone(it should work on any Android, Ice Cream Sandwich or later, but we have tested only on Kitkat). Please remember it is only a demo app to demonstrate keypoint detection and matching to customers. The app by itself has a few small drawbacks. The algorithm itself can be improved too Both quality(increase the number of scales points etc that need to be detected) and speed(currently its single core; we can make it multi core and do other optimizations specific to your usecase perhaps) can be improved by playing around with the parameters. 2. How the App works The way it functions currently is as follows On opening the app and choosing the "camera demo" option, it will first look for the most recently modified file in your internal memory and tries to open it as an image file. It will then rescale this to VGA size(640x480) and then use this image as a reference image. It will then start comparing the camera preview data(which is usually at VGA again) with the reference image using a keypoint detection and matching algorithm. When it finds sufficient matches and if its in the right order(locations this is done using homography) we declare a successful match and then draw a green rectangle(the position, scale and orientation of which indicates that of the detected window) on the screen.

5 3. How to install Please try one of the options below 1. Mail Send the app as attachment in your . Set the account on your phone. And open the and click on the attachment(imagerecognition.apk). The app will first download and then install. 2. Copy the apk to phones internal memory or external sdcard. Use an Android File explorer to open the app using the phone. Now double click on the app. The app will see your permission to install itself. Other points to note 1. To install on phone, for the very first time, you may have to go to settings~security and check the box for allowing installation of apps from unknown sources. 2. While installing this app for the first time on an Android phone, you may be prompted to install OpenCV apk from the playstore. If yes please install OpenCV from the Google playstore. OpenCV is used for reading jpeg files and decoding them into YUV pixel format. 3. If you were to re install the app for some reason you will have to first uninstall the same app and then install again. 4. Setting Reference Image 1. Using Camera 1. Open your phones camera app. 1. Go to settings >resolution and make sure you choose a resolution whose aspect ratio is 4:3(same as 640x480). Example 2048x Go to settings and change the storage to internal memory(since the image recognition app searches for the latest image in the internal memory). If the app does not support this option, you may have to manually copy the image to the internal folder after taking a photo. 2. Take a picture with your phone in landscape mode. 3. All the above limitations are due to a limitations of the app, which can be fixed and has got nothing to do with the algorithm itself. 2. Setting Image directly Connect phone to PC and transfer the jpeg file(whose resolution is having aspect ratio of 4:3) into the DCIM folder of the internal memory(usually in the path /storage/sdcard0/dcim) of the Android phone. Beware The app always takes the latest file in the internal memory and assumes its an image gile. So if you first transfer an image from PC, for use as a reference image; but then take a picture using your camera app, the UncannyCV Image Recognition app will now start taking the camera image as the reference since thats the latest file in the internal memory. Example image We use the following image cropped to 4:3 ratio for our testing. However any image that has 4:3 ratio should work. Neverending Dreamer

6 5. How to test 1. Open the UncannyCV app. 2. Chose option camera demo(press Start button under option Camera Demo) 3. Point your camera at the 2D marker which you have provided as reference to the app. Its always best to first test with a known usecase, example image given in previous section. So first printout the same image and then point the camera at the printout.

7 The green rectangle in the above picture indicates that the image has been detected and also shows the scale and alignment(if you point the phone at the image from the side, while running our app, the rectangle will be skewed. Similarly when you move the camera away from the screen, the rectangle would grow smaller ) has been detected correctly. 4. To verify if the correct reference image has been chosen, please do the following. On starting the app it will create a rescaled bmp of the camera image and store it as grayimage.bmp in internal memory root folder. Please check if this file has been created, and also open the image to verify if the app has chosen the correct reference image. 6. Which images work well It works best when the reference image is sharp, taken without shake or reflections. Its also ideal if the image is a natural image(contains more unique real world keypoints), and not a graphics images(may contain keypoints that have identical signatures/descriptors). It may not work very well in the following cases

8 1. Image that contains lots of text. The keypoint matching will start to fail because the image will contain many instances of the same letter(or character). Or will contain nearly identical corners even in different characters. Images that contains grid like stuff. Same problem as above. The grid points will be detected as corners, but there will be several that look alike. 2. Blurry images 3. Images that contains reflections from screens, windows etc the accuracy goes down a bit. 3. UncannyCV Demo 1. Introduction UncannyCV demo 1. Edge detection Canny edge detection 2. Pedestrian detection 3. Vehicle detection Detecting cars from the back side 4. Face detection 5. Optical Flow Lucas Kanade Optical Flow 6. Semi Dense Optical Flow 7. Tamper Detection 8. Kmeans 9. Sobel 10. Lane Departure 11. Fast Keypoints FAST9 corner detection algorithm 2. How to install Please try one of the options below 1. Mail Send the app as attachment to your . Set the account on your phone. And open the and click on the attachment(imagerecognition.apk). The app will first download and then install. 2. Copy the apk to phones internal memory or external sdcard. Use an Android File explorer to open the app using the phone. Now double click on the app. The app will see your permission to install itself. Other points to note 1. To install on phone, for the very first time, you may have to go to settings~security and check the box for allowing installation of apps from unknown sources. 2. If you were to re install the app for some reason you will have to first uninstall the same app and then install again.

9 3. How to test 1. Open the UncannyCV app. 2. Chose option camera demo(press Start button under option Camera Demo) 3. Point phone at object of interest in landscape mode. 4. Press the Demo options drop down menu to see other demo options(shown in figure below). 5. The Processing time can be observed on the top left corner(shown in figure below)

10 4. A short note about each demo 1. Canny Edge detection Operates on input frame of size 640x480. Single threaded, hence it uses only a single core Based on Canny edge detection. The time required for processing a frame depends on the amount of edges. For an input frame with very few edges the processing time will be minimal. 2. Pedestrian detection Can detect pedestrians who are completely contained inside the image(head to toe). Minimum sized pedestrian that can be detected is a person whose height is approximately 50% of height of input image. Based on HOG(Histogram of Oriented Gradients) algorithm. The time required for processing a frame is almost independent of the number of pedestrians present in the frame. 3. Vehicle detection Detecting cars from the back side Operates on input frame of size 640x480. But uses 3 ROIs(Region of Interest). The 3 green rectangles indicate the 3 ROIs. The red rectangle indicates detected cars. The smallest ROI can detect small cars. The medium sized ROI can detect the medium sized car and so on. The small cars have to be inside the small green rectangle to be detected. Any car which is larger than the small green rectangle, but smaller than the medium sized rectangle, has to be then present in medium green rectangle. How to demo Run a video sequence of a car moving through a street or on highway(example video provided here). Test image Please test with this image to confirm its working Face detection Detects frontal faces(looking into camera). Based on LBP(Local Binary Pattern) based Adaboost Cascade detector. Minimum sized face that can be detected is a face whose width and height is approximately 10% of height of input image. Point camera at a face. The subject should be looking directly at camera. Perhaps due to lack of images of people wearing spectacles in the original training database, the algorithm has a slightly lower detection rate when the subject is wearing spectacles. Detection rate can be improved significantly by using tracking. 5. Optical Flow Lucas Kanade Optical Flow. Operates on input frame of size 640x480. Good features to track followed by LK Optical flow. Tracks 750 points(15x15 sized kernels) in each frame across 5 scales and 20 iterations(it has early exit condition, so every point may not execute 20

11 iterations). How to demo Move camera lightly from from left to right or top to bottom in front of object of interest. Or move the object of interest in front of camera. The green lines indicate the motion flow direction and magnitude. The good features to track algorithm looks for corner points. If the camera is out of focus(example object is too close to camera), then the image will be blurred and the good features to track algorithm will not detect good points. 6. Semi Dense Optical Flow LK Optical flow for each 16x16 block in the image. 7. Tamper Detection This algorithm is useful in video surveillance applications to detect if someone is trying to tamper with the camera by covering it with a cloth/paper or some other material. How to test Keep phone still on a table, leaning on a stand or something, with the camera pointed at some scene of interest. Switch on the app, and chose the Tamper detection option. The display will turn blue indicating that the algorithm is learning the scene. Once the scene learning is complete the display will turn normal. Now if you try to cover the camera with your hand or with some other material, the display will turn red indicating that the algorithm has detected someone trying to tamper the camera. 8. Kmeans Image segmentation using the 3 RGB channels as features. Uses Kmeans clustering for segmenting the image. Always segments to 3 segments. 9. Sobel Sobel edge detection. 10. Lane Departure How to test similar to the vehicle detection algorithm. There are 2 while horizontal lines on display. The road directly in front of the car should be between these 2 lanes. Please refer to this video on how to test and how to hold camera This algorithm is sensitive to where the camera is kept in the car. The current assumption is that it has been kept inside the car on the dash. Currently the warning flashing on the screen when the car crosses a lane has been disabled, instead we will replace this with a warning based on colour of lanes. However this has not been implemented yet in the app. Test image Please test with this image to confirm its working Fast Keypoints FAST9 corner detection algorithm

12 4. Limitations