Vision Problems Baby Food

Size: px

Start display at page:

Download "Vision Problems Baby Food"

Lester Wilcox
6 years ago
Views:

10 Vision Problems This final chapter presents a series of computer vision application problems which are possible to solve using the theory (and practice) presented in this text.

1 10 Vision Problems This final chapter presents a series of computer vision application problems which are possible to solve using the theory (and practice) presented in this text. Images and videos for these problems are provided in the electronic resources accompanying the text Baby Food On a production line cans of baby food are made by: 1. Fabricating the sides and lid of the can together but omitting the base of the can. 2. Dropping a spoon into the empty upside-down can. 3. Pouring the baby food (powder) into upside-down the can. 4. Sealing the base onto the can. You are asked to develop an inspection system looking at the can between steps 2 and 3 in order to check that a single spoon has been placed into it (see Figure 10.1). Figure 10.1 Sample baby food can images from the production just after the spoon has been dropped into the can. Examples of no spoon, one spoon and two spoons are shown A Practical Introduction to Computer Vision with OpenCV, First Edition. Kenneth Dawson-Howe John Wiley & Sons, Ltd. Published 2014 by John Wiley & Sons, Ltd.

190 A Practical Introduction to Computer Vision with OpenCV 10.2 Labels on Glue On a production line for bottles of glue it is necessary to perform a number of inspection tests (see Figure 10.2): 1.

2 190 A Practical Introduction to Computer Vision with OpenCV 10.2 Labels on Glue On a production line for bottles of glue it is necessary to perform a number of inspection tests (see Figure 10.2): 1. Check that each bottle has a label. 2. Check that the label on the bottle is straight. 3. Check that the label is not torn or folded. Figure 10.2 Sample images of bottles of glue are shown some without labels and some with crooked (or damaged labels). Note that these are not images from a real production line and in many cases the labels have been manually damaged/removed

3 Vision Problems O-rings You are asked to detect defects (notches and breaks) in rubber O-rings as they pass before an inspection camera (see Figure 10.3). Good O-rings: Breaks: Notches: Figure 10.3 Sample images of good O-rings (i.e. with no defects; see top row), broken O-rings (middle row) and O-rings with notches out of them (bottom row)

4 192 A Practical Introduction to Computer Vision with OpenCV 10.4 Staying in Lane In order to assist drivers, you are asked to develop a system to ensure that they stay in lane, and raise an alarm if they start to drift into another lane (without indicating). Given a video feed from a camera mounted on the dashboard of a car, looking forwards, you are asking to automatically detect the lines on each side of the lane of traffic that the car is in (see Figure 10.4). This is an area of significant research interest and there are other datasets available online: 1. PETS (Performance Evaluation of Tracking and Surveillance) 2001 dataset 5: 4 videos (2 looking forwards, 2 looking backwards) from a vehicle on a motorway. rdg.ac.uk/pets2001/dataset5/ 2. CamVid (Cambridge-driving Labeled Video Database): 4 videos looking forwards from a vehicle 3. Daimler Pedestrian Detection Benchmark Dataset: A small number of long video sequences looking forwards from a vehicle. Figure 10.4 Sample images from the dashboard mounted camera

5 Vision Problems Reading Notices In order to facilitate auto-translation, you are asked to locate any text within an image and extract the text as a series of characters (see Figure 10.5). You may assume that the text will take up a reasonable portion of the image and will be more-or-less the right way up! Figure 10.5 Sample images of notices

6 194 A Practical Introduction to Computer Vision with OpenCV 10.6 Mailboxes Given a live feed from a camera monitoring mailboxes, you are asked to determine if there is any post in each mailbox (see Figure 10.6). To make the processing a bit easier a pattern of black and white stripes has been put in each box. As people put mail in and take mail from the mailboxes on a regular basis, your solution must be able to cope with occlusion of the camera by moving people (without generating any false mail notifications). Figure 10.6 Three sample frames from a video of the mailboxes, all empty (left), partly occluded (centre and with mail in three of the boxes (right)

Vision Problems 195 10.7 Abandoned and Removed Object Detection You are asked to locate any abandoned objects and any removed objects (see Figures 10.7 and 10.8).

CAVIAR (Context Aware Vision using Image-based Active Recognition project): 5 videos in a lobby from above of various objects being abandoned and removed. http://homepages. inf.ed.ac.uk/rbf/caviar/.

7 Vision Problems Abandoned and Removed Object Detection You are asked to locate any abandoned objects and any removed objects (see Figures 10.7 and 10.8). This is an area of significant research interest and there are many datasets available online: 1. CAVIAR (Context Aware Vision using Image-based Active Recognition project): 5 videos in a lobby from above of various objects being abandoned and removed. inf.ed.ac.uk/rbf/caviar/. 2. ViSOR (Video Surveillance Online Repository): 19 videos of abandoning and removing objects in an outdoor car-park. and it/visor/ 3. PETS (Performance Evaluation of Tracking and Surveillance) 2006 dataset: 7 left luggage datasets of videos from 4 different cameras in a train station. PETS2006/data.html 4. PETS (Performance Evaluation of Tracking and Surveillance) 2007: 7 left luggage datasets of videos from 4 different cameras in a busy airport terminal. Figure 10.7 Sample object abandonment Figure 10.8 Sample object removal

8 196 A Practical Introduction to Computer Vision with OpenCV 10.8 Surveillance You are asked to find and classify (as people, cars, etc.) all moving objects in a video sequence from a static camera (see Figure 10.9). You must be able to deal with environmental effects (such as changing lighting and the effects of wind, etc.). This is an area of significant research interest and there are large numbers of datasets available online. For example: 1. CAVIAR (Context Aware Vision using Image-based Active Recognition project): homepages.inf.ed.ac.uk/rbf/caviar/. 2. ViSOR (Video Surveillance Online Repository): and imagelab.ing.unimore.it/visor/ 3. PETS (Performance Evaluation of Tracking and Surveillance datasets: ac.uk/ 4. SPEVI (Surveillance Performance EValuation Initiative) datasets ac.uk/staffinfo/andrea/spevi.html 5. ETISEO dataset from INRIA OTCBVS dataset LIMU dataset from Kyushu University. Figure 10.9 Four frames from PETS 2001 (Dataset 1, Camera 1). Reproduced by permission of Dr. James Ferryman, University of Reading

Vision Problems 197 10.9 Traffic Lights You are asked to locate any traffic lights in a camera view and to determine what light is lit (green, amber or red) (see Figure 10.10).

9 Vision Problems Traffic Lights You are asked to locate any traffic lights in a camera view and to determine what light is lit (green, amber or red) (see Figure 10.10). There are videos, available online, taken from moving vehicles which can be used for traffic light detection: 1. CamVid (Cambridge-driving Labeled Video Database): 4 videos looking forwards from a vehicle 2. Daimler Pedestrian Detection Benchmark Dataset: A small number of long video sequences looking forwards from a vehicle. Figure Three frames from a stationary camera at a traffic light controlled road junction

198 A Practical Introduction to Computer Vision with OpenCV 10.

10 198 A Practical Introduction to Computer Vision with OpenCV Real Time Face Tracking Assuming that the cascade of Haar classifiers is too slow, propose a way of tracking a face in (close to) real time in a video (see Figure 10.11). Figure Sample frames from a video showing a face to be tracking

Vision Problems 199 10.11 Playing Pool You are asked to locate the balls on a pool table so that their positions can be analysed for possible shots (see Figure 10.12).

11 Vision Problems Playing Pool You are asked to locate the balls on a pool table so that their positions can be analysed for possible shots (see Figure 10.12). Ball positions should be computed only when there is no motion on the table and no one is occluding the table. Also, ball positions should be reported relative to the table (rather than the image). Figure Sample images from a video of a game of pool

200 A Practical Introduction to Computer Vision with OpenCV 10.

12 200 A Practical Introduction to Computer Vision with OpenCV Open Windows You are asked to develop a system to determine what (top-hung) windows are open in images of a building. This is intended to aid in the efficient heating and ventilation of the building (see Figure 10.13). Figure Three images from a camera monitoring a set of windows

Vision Problems 201 10.13 Modelling Doors Given a video feed from a surveillance camera, you are asked to locate any doors in the view.

13 Vision Problems Modelling Doors Given a video feed from a surveillance camera, you are asked to locate any doors in the view. This can be done during a training phase either from video or still images. This should aid the system when attempting to understand the actions of people within the scene (see Figure 10.14). Figure Three sample frames from a camera monitoring a doorway

202 A Practical Introduction to Computer Vision with OpenCV 10.

14 202 A Practical Introduction to Computer Vision with OpenCV Determining the Time from Analogue Clocks You are asked to determine the time from an image of an analogue clock (see Figure 10.15). Figure Sample clock images

is currently viewing so that augmented content can be added to the page through the use

15 Vision Problems Which Page You are asked to determine which page (if any) out of a given text a reader is currently viewing so that augmented content can be added to the page through the use of a projector (see Figures and 10.17). Figure Some of the provided sample page images Figure Sample views of the book while it is being read

204 A Practical Introduction to Computer Vision with OpenCV 10.

16 204 A Practical Introduction to Computer Vision with OpenCV Nut/Bolt/Washer Classification You are asked to classify parts coming down a conveyor line as nuts, bolts or washers (or something unknown) (see Figure 10.18). Figure Sample views of the parts on the conveyor belt

Vision Problems 205 10.17 Road Sign Recognition Assuming that you have sample (CAD) images of road signs such as those in Figure 10.

17 Vision Problems Road Sign Recognition Assuming that you have sample (CAD) images of road signs such as those in Figure 10.19, you are asked to develop a system to automatically recognise these road signs from single images (taken from the dashboard of a car looking forwards). They will appear at different sizes (as the car moves towards them), but should be facing directly towards the vehicle (see Figure 10.20). Figure Sample CAD road signs Figure Sample images (containing road signs) taken looking forwards from a vehicle

206 A Practical Introduction to Computer Vision with OpenCV 10.18 License Plates You are asked to recognise the number on license plates taken with a handheld camera.

18 206 A Practical Introduction to Computer Vision with OpenCV License Plates You are asked to recognise the number on license plates taken with a handheld camera. You are provided with example images of the digits in two forms (synthetic and real) (see Figures 10.21, and 10.23). Figure Sample synthetics digits Figure Sample digits from license plates Figure Sample images of car license plates

Vision Problems 207 10.19 Counting Bicycles You are asked to locate any bicycles moving along a path (see Figure 10.24).

19 Vision Problems Counting Bicycles You are asked to locate any bicycles moving along a path (see Figure 10.24). Note that there can be other bicycles or people walking, and that you must be able to cope with changes in the lighting (as the sun goes in and out) and with the waves on the sea (which are constantly moving). Figure Some frames from the sample bicycle video

208 A Practical Introduction to Computer Vision with OpenCV 10.

Figure 10.25 Sample images from an observer in a gallery. Photographs taken by Dr. Kenneth Dawson- Howe at the National Gallery of Ireland Figure 10.26 Sample known painting images.

20 208 A Practical Introduction to Computer Vision with OpenCV Recognise Paintings As part of an augmented reality system you are asked to automatically determine the identity of any painting in the view of an observer (camera) (see Figures and 10.26). Figure Sample images from an observer in a gallery. Photographs taken by Dr. Kenneth Dawson- Howe at the National Gallery of Ireland Figure Sample known painting images. The painting are: (top left) The Marriage Feast an Cana (late 1660s) by Jan Havicksz. Steen, (top centre) A Thunderstorm: The Frightened Wagoner (1832) by James Arthur O Connor, (top right) Joseph Selling Corn in Egypt (1612) by Pieter Lastman, (bottom left) A Sick Call (1863) by Matthew James Lawless, (bottom centre) A View of the Rye Water near Leixlip (1850s) by William Davis, and finally (bottom right) The Arrival of the Kattendijk at Texel (1702) by Ludolf Backhuysen I. Photographs taken by Dr. Kenneth Dawson-Howe at the National Gallery of Ireland

Transportation Informatics Group, ALPEN-ADRIA University of Klagenfurt. Transportation Informatics Group University of Klagenfurt 3/10/2009 1

Machine Vision Transportation Informatics Group University of Klagenfurt Alireza Fasih, 2009 3/10/2009 1 Address: L4.2.02, Lakeside Park, Haus B04, Ebene 2, Klagenfurt-Austria Index Driver Fatigue Detection