Barcode Reading Algorithm For Blind Users

Transcription

1 Barcode Reading Algorithm For Blind Users Pooja Gajul 1 Supriya Gawai 2 Shashank Chavan 3 Prof.Archita Dad 4 ABSTRACT 1,2,3 BE-CMPN ( Pursuing ),4 Project Guide,Department of Computer Engineering Atharva College of Engineering, Malad (W), Mumbai (India) Regarding the typical image distortions we present an algorithm for the recognition of barcodes using camera phones, which is highly robust. A database of barcode images have been created, which covers falsification in homogenous illustration, reflections, or unclear image due to movements of camera. We present results from experiments with over 1,000 images from this database using a C# implementation of our algorithm. Most camera-based systems for finding and reading barcodes are designed to be used by sighted users and assume the user carefully centers the barcode in the image before the barcode is read. Unsighted individuals could benefit greatly from such systems to identify packaged goods, but unfortunately in their current form these systems are completely inaccessible because of their reliance on visual feedback from the user. The algorithm issues directional information with audio feedback (e.g. left, right ) and thereby guides a blind user holding a portable camera to locate and home in on a barcode.when the barcode is detected at sufficiently close range, a barcode reading algorithm scans and reads aloud the barcode and the corresponding product information. Keywords:Algorithm,Barcode,falsification 1.INTRODUCTION The 1D barcode which used as a symbol to represent ideas has become a universal system for labeling packaged goods with codes that uniquely identify product information. Blind and visually impaired persons could greatly benefit from the ability to read barcodes, which have been very useful for recognizing products in a supermarket or in a pantry at home.he or she may have no way of distinguishing between two cans with the same shape and weight (e.g. tomato sauce or beans). If the product barcodes could be located and read by a camera phone application, the related product information could immediately be looked up and read aloud to the user. A computer vision algorithm enabling a blind person to locate and read barcodes could be implemented on a phone camera, perhaps in addition to other functionality providing access to printed information (such as the KNFB reader, which is a commercial OCR system that runs on a Nokia phone camera). Such a system would be much more convenient than conventional laser scanner technology that requires the user to carry a dedicated device, such as the i.d. mate Talking Bar Code Reader. Our main contributions are two- 105 Pooja Gajul, Supriya Gawai, Shashank Chavan, Prof.Archita Dad

2 fold. (1)We have developed a fast, novel algorithm for detecting barcodes at a distance based on a cascade of filters that progressively rule out more and more regions of the image; only regions of the image that resemble barcodes are processed intensively, which is the key to the algorithm s speed. (2) We have coupled our algorithm with an algorithm that we developed previously [9] to read barcodes, and have executed the current version of our system with some improvements to the model using a PC and a camera. We added speech feedback to guide a blind user to a detected barcode, and we tested the system while wearing blind folds, demonstrating the feasibility of the system for totally blind users. 2.RELATED WORK More recent work has addressed the problem of reading barcodes from images received by a camera. Most of this work has simplified the problem of locating and segmenting the barcode in the image by assuming certain constraints hold. In [5], the barcode is assumed to be horizontal in the image and viewed close enough for the long width of the barcode to subtend about two-thirds of the image width; al-though the authors point out that the method can be easily extended to the case of undefined orientation, it is unclear how well the algorithm would work with barcodes viewed from farther away. Other work [2] assumes that the barcode can be detected from the morphological structure of binarized regions of the, but the procedure may fail on images that are noisier than the crisp.[7] Assumes that a barcode can be detected by searching for a black bar using a spiral method, which scans in a spiral outward from the center of the image, but it is not clear that an individual black bar can be sufficiently well resolved at a distance. We build on our recent work [9, 8] focusing on reading a barcode (assuming it has already been segmented) using a Bayesian deformable template algorithm that combines a prior model of the geometric shape of the barcode pattern with a likelihood model that evaluates evidence in the image for edges. S uch an approach is an important technique of decoding noisy barcode images that contain spurious (or missing) edges. A related approach [3], also based on impairable templates, can successfully decode barcodes from extremely blurry and noisy pictures. 3.INTERACTIVE CAMERA ALIGNMENT The shopper always takes a picture of a fixed surface after the phone s camera is aligned with the surface in the yaw and pitch planes. The alignment is carried out through an iterative haptic control loop. If the product is a box, the user is instructed to find the top of the box, i.e. the side with an open tab, by touch and then align the camera with the bottom, i.e. the opposite side. This is because on most boxes sold in the U.S. the UPC barcode is on the bottom. Once the bottom is found, the shopper aligns the camera with an edge of the side and clicks a button on the touch screen to start the alignment loop. The current readings of the phone s orientation sensor are taken to define the absolute pitch and yaw planes. The shopper then slowly moves the camera away from the surface. If the camera is misaligned in the pitch or yaw planes, specific haptic signals (vibrations) are issued to re-align it. The signals persist until the camera is aligned. The user is instructed to stop moving the camera away from the surface when she thinks that the camera is approximately 10cm from the surface. Then the picture is taken by a click of a button on the touch screen. If the product is a can, the shopper aligns the camera with the edge of the top or bottom circle. If it is a bottle, the camera is aligned with the edge of the bottom circle. When the shopper reaches an aisle, she aligns the phone 106 Pooja Gajul, Supriya Gawai, Shashank Chavan, Prof.Archita Dad

3 camera with a shelf and captures an image. If a barcode is decoded successfully and happens to be a shelf barcode, the system checks if it is the target barcode. If it is, the system verbally informs the shopper about it. If it is not, the system generates verbal directions to the target product (e.g. Two shelves up, three barcodes left. ). If the barcode cannot be decoded in the image, the system divides the image into sub-images that are likely to contain barcodes. Each sub image is processed with a barcode decoder. If the barcode is decoded, the system goes into the verbal instruction mode. If the barcode is not decoded, the system assumes that the barcode is rotated by 90 degrees. This assumption is enforced by the interactive haptic alignment loop. The image is rotated by 90 degrees, and the entire processes is repeated again. If part of a barcode is detected, the system gives the user haptic feedback to move the camera slightly left, right, up, or down, depending on where the partial barcode is detected. When the system is unable to find the barcode in the image, it notifies the user through audio feedback (currently a ping) to take another image. The overall flow of control of the barcode localization and decoding method is given in Figure. If necessary, user feedback messages can be delivered in a variety of formats: haptic, audio, speech, or a combination thereof 4.WORK FLOW: Fig. 1 Accessible Barcode Localization and Decoding Method. 107 Pooja Gajul, Supriya Gawai, Shashank Chavan, Prof.Archita Dad

4 Fig.2 Barcode System 5. STEPS TO BE UNDERTAKEN 5.1. Detection: (1) Lines in 4 different orientations swept to determine collection of edge points with alternating polarities. (2) Line scores tallied in direction perpendicular to sweep direction to get 2D representation of possible barcode areas. (3) Orientation entropy used to eliminate false positives (e.g. dense text) Direction: (1) A maximal bounding box to enclose the detected barcode is calculated. (2) The user is directed to the barcode by voice commands until enough edges are seen Decoding: (1) Slices with maximum number of edges are found and edges localized with sub-pixel accuracy. (2) Maximum likelihood (ML) estimation of the fundamental width and fixed edges. (3) ML estimation of the barcode digits using the check bit. (4) Detection attempted both right side up and upside down Output: (1) Product information retrieved from database and read out. In the next few sections, the details of these stages are provided. 108 Pooja Gajul, Supriya Gawai, Shashank Chavan, Prof.Archita Dad

5 6.ALGORITHMS: 6.1 Line Scan Algorithm: INITIALIZATION: _G = minimum gradient threshold ne = minimum # of edges required de = maximum distance between consecutive edges SWEEP: for orientation t = 0, 45, 90, 135 do for line l = 1,..., lastlineinthisorientation do count 0 for pixel i = 1,..., lastpixelonthisline do Let j be the last pixel on this line that was counted. if Ii > _G then //Gradient above threshold and angle approx. perpendicular to sweep line if \ Ii orientation then if Ii max{ Ii 1, Ii+1 } then //non-maximum suppression if \ Ii is 180 degrees out of phase with \ Ij, and dij < de then count count + 1 //count this pixel else count count 1 //pixel with strong gradient at wrong orientation else if dij > de then //no edge pixel seen in a while count count 1; if dij > 2 de then //no edges in a long while count 0 //end of candidate segment if count = 0 then //see if end of segment has been reached score maxi2lastsegment count(i) //score is the max count for this segment if score > ne then //if the minimum # edges has been seen Record this segment as a barcode candidate segment for this line else Discard this segment 109 Pooja Gajul, Supriya Gawai, Shashank Chavan, Prof.Archita Dad

6 6.2 Line Tally Algorithm: INITIALIZATION: nl = minimum # of lines required dl = maximum distance between matching lines _S = minimum score required to declare barcode candidate B = {} //list of candidate areas SWEEP: for orientation t = 0, 45, 90, 135 do for line l = 0,..., lastlineinthisorientation do for barcode segment candidate c = 1,..., lastcandidateonthisline do if b B: beginb beginc, endb endc and dlb < dl then //There was a previous barcode area candidate with similar beginning and end a little earlier countb countb + 1 else B B + c //Add this segment as the beginning of a new candidate area countc 1 for b B do //check that the candidates are still valid if dlb > dl then //have not seen a match in a while countb countb 1 if dlb > 2 dl then //have not seen a match in a long while countb 0 if countb = 0 then //end of candidate scoreb maxl2bcount(l) //score is the max count over all lines in this area if scoreb > _S then //if the minimum # lines has been seen Record b as a barcode area segment else B B b //Discard this candidate 110 Pooja Gajul, Supriya Gawai, Shashank Chavan, Prof.Archita Dad

7 6.3 Barcode Decoding Algorithm: INITIALIZATION: Xinitial = lastedge firstedge 95 FIND EDGES AND DIGIT PROBABILITIES: Find the Nslices lines in the barcode with the highest edge count for Slice i = 1,...,Nslices do Estimate NfixedEdgeEstimates fixed edges for Fixed Edge Estimate j = 1,...,NfixedEdgeEstimates do for Barcode digit d = 1,..., 12 do Get digit probabilities for each numeric digit 0,..., 9 Marginalize digit probabilities over all fixed edge estimates Marginalize digit probabilities over all slices BARCODE ESTIMATION: for Barcode digit d = 1,..., 12 do Calculate auxiliary running parity check digit probabilities. ML Estimation of the 2 most likely sequence of auxiliary random variables Convert auxiliary random variables back to barcode digits BARCODE VERIFICATION: if Probability of most likely sequence > K Probability of the second most likely sequence then if Most likely sequence differs from individually most likely digits by at most 1 digit then OUTPUT BARCODE Get new frame 7. CONCLUSION We have described a novel algorithm for finding and reading 1D barcodes, which is use by blind users.the most important feature of this algorithm is the ability to detect barcodes at some distance, allowing the user to scan packages before homing in on a barcode. Experiments with a blind volunteer demonstrate proof of concept of our system, which allows the blind user to locate barcodes which were then translated to product information that was announced to the user. We tested a series of products, as well as the barcodes generated by the user labeling household items that may not come with barcodes. 8. ACKNOWLEDGEMENT We are thankful to our Principal Dr. Shrikant Kallurkar, Project Coordinator Prof. Deepali Maste and other senior faculties of Computer Department for technical assistance and feedback through discussions. Our thanks to some of our colleagues who contributed towards development of the flowchart, leading to a success of this project 9. REFERENCES [1] A. Adelmann, M. Langheinrich, and G. Floerkemeier. A toolkit for bar-code-recognition and resolving on camera phones - jump starting the internet of things. In Workshop on Mobile and Embedded Interactive Systems (MEIS 06) at Informatik 2006, Pooja Gajul, Supriya Gawai, Shashank Chavan, Prof.Archita Dad

8 [2] D. Chai and F. Hock. Locating and decoding EAN-13 barcodes from images captured by digital cameras. In Information, Communications and Signal Processing, 2005 Fifth International Conference on, pages , [3] O. Gallo and R. Manduchi. Reading challenging barcodes with cameras. In IEEE Workshop on Applications of Computer Vision (WACV) 2009, Snowbird, Utah, December [4] E. Joseph and T. Pavlidis. Deblurring of bilevel waveforms. IEEE Transactions on Image Processing, 2, [5] W. Kongqiao. 1D barcode reading on camera phones. International Journal of Image and Graphics, 7(3): , [6] S. Kreˇsi c-juri c, D. Madej, and F. Santosa. Applications of hidden Markov models in bar code decoding. Pattern Recogn. Lett., 27(14): , [7] E. Ohbuchi, H. Hanaizumi, and L. A. Hock. Barcode readers using the camera device in mobile phones. In CW 04: Proceedings of the 2004 International Conference on Cyberworlds, pages , Washington, DC, USA, IEEE Computer Society. [8] E. Tekin and J. Coughlan. Barcode project. lab/barcode. [9] E. Tekin and J. M. Coughlan. A bayesian algorithm for reading 1-D barcodes. In Sixth Canadian Conference on Computer and Robot Vision (CRV 2009), Kelowna, BC, CA,May Pooja Gajul, Supriya Gawai, Shashank Chavan, Prof.Archita Dad