AN APPROACH TO EXTRACT LINE, WORD AND CHARACTER FROM SCENE TEXT IMAGE

Transcription

1 AN APPROACH TO EXTRACT LINE, WORD AND CHARACTER FROM SCENE TEXT IMAGE DANESHWARI A NOOLA 1, M M KODABAGI 2 daneshwari.noola@gmail.com malik123_mk@rediffmail.com Abstract:- Text translation from Scene Image plays a very important role considering any individual or in any work which needs interpretation of translation Scene Text of a Scene Image. Especially when a person travels from a place to other he/she wishes for a device which helps for interpretation of translation of text in scene images to their native language. Now the proposed work is the first step towards the design of such device which helps for interpretation, i.e. before the translation of text, we need to extract line, word and character from the text of scene image, so the system designed helps for line, word and character segmentation of a text in scene image. The algorithm is implemented in MATLAB. The data sets are collected through various shop names, name plates, city names and others through mobile phone or digital camera. The algorithm is tested with several scene images. The line segmentation accuracy of 100% for good quality images is observed, the word segmentation accuracy of 98% for good quality is observed, the character segmentation accuracy of 98% for good quality is observed. The limitation of this method is that it resulted in segmentation errors for touching and broken characters especially like any character prefixed or post fixed to V or T. Thus the system is tested for 100 scene images with atleast on an average of 200 lines in all and over 2000 words and characters are determined successfully. Keywords: Line segmentation, word segmentation and character segmentation I. INTRODUCTION As all the people in the world travel around, it becomes difficult for oneself to move around places which follow different languages and culture. Assume if each of us had a device or can call it an application in PDA s or Mobile phone, which indeed gave a translated version of all the sign boards or it can be name plates on buses or important matter in our native language then it would have been more beneficial. The work what is discussed in following sections is just a first step towards this achievement. The following work describes regarding separation of line, word and character from the scene text image captured through digital camera or mobile phone. Capturing, Detecting and then extracting line, word and character from scene image is main aim of this work presented. In the proposed work, a technique is derived to extract line, word and character from a scene text image so that the extracted line, word and character would help for furthr translation of words and characters wherever it is necessary. When a design of intelligent device is done for interpretation of translated characters, it would help any individual to translate sign/board. It is the first step towards the design of intelligent devices that can help in providing translated information to their native language needed which would indeed make our lives easier when we are familiar with them. submission@ijorcs.org II. LITERATURE SURVEY The literature survey revealed the following information about extraction with respect to document images and some with scene text images [S A Angadi and M M Kodabagi, 2009], presents a texture based methodology for Text Region Extraction from Low Resolution Natural Scene. The proposed work is texture based and uses DCT based high pass filter to remove constant background. The texture features are obtained on every 50x50 block of the processed image and potential text blocks are identified using newly defined discriminant functions. Further, the detected text blocks are merged and refined to extract text regions. The presented method is robust and achieves a detection rate of 96.6% on a variety of 100 low resolution natural scene images each of size 240x320. [M Swamy Das, 2010], proposes segmentation of overlapping text lines, characters in printed telugu text document images The proposed algorithm is based on projection profiles, connected components and spatial vertical relationships. It also uses nearest neighborhood method to cluster the connected components. With experimental results it is observed that 100% line segmentation and about 98% character segmentation accuracy can be achieved with overlapping lines and characters. [U Pal and B B Chaudhari, 1999], describes an automatic technique of separating text lines using script characteristics and shape based features is presented. The scheme presented is based for bangla and Devnagari. The scheme is based on the structural and statistical features of the machine-printed 916

2 and hand-written text lines. The classification scheme has accuracy about 98.3%. [M.C. Padma and P. A. Vijaya, 2010], develop a model to identify and separate text words of Kannada, Hindi and English scripts from a printed tri-lingual document. The proposed method is trained to learn thoroughly the distinct features of each script and uses the simple voting technique for classification. Experimentation conducted involved 1500 text words for learning and 1200 text words for testing. Extensive experimentation has been carried out on both manually created data set and scanned data set. The results are very encouraging and prove the efficacy of the proposed model. The average success rate is found to be 99% for manually created data set and 98.5% for data set constructed from scanned document images. [M C Padma and Vijaya, 2010], describes a model to identify the script type of a trilingual document printed in Kannada, Hindi and English scripts. The distinct characteristic features of Kannada, Hindi and English scripts are thoroughly studied from the nature of the top and bottom profiles. The proposed model is trained to learn thoroughly the distinct features of each script. Experimentation conducted involved 1500 text lines for learning and 1500 text lines for testing. The k- nearest neighbor classifier is used to classify the test sample. The results are encouraging and prove the efficacy of the proposed model. The average success rate is found to be 99.5% for data set constructed from scanned document images. III. DESIGN A novel algorithm for Text Extraction from Scene images is proposed. Here system would be using Horizontal and vertical profiles to perform the task. First the input image is taken then, preprocessing is applied which resizes and crops the scene image and later is scanned for horizontal gaps, for line segmentation; each horizontal gap corresponds to a single separated line. And then the preprocessed image is scanned for vertical gaps, for words and characters to be segmented. The system uses some image processing tools to perform the job for extraction of lines, words and characters from these points. The proposed methodology for Extraction of Scene text from Scene Image shown has following steps: Image acquisition, Image preprocessing, Binarization, Extract line from scene text image, Extract word and character from line extracted from scene text image, testing which examines the scene images for extractions. The extraction of text line wise, word wise and character wise is the output for the given sample input image. Image Acquisition The process begins with acquiring natural scene images containing text. The scene images are captured using high resolution digital cameras which are present in.jpeg form. About 100 samples of the images containing English, Kannada and Hindi text scene images are collected as requirement. Image Preprocessing The scene text images have issues like lighting effects, shadowing, blur images, color degradation, skew image, size etc. The purpose of this phase is to make the images to be of standard size, remove complex backgrounds, and makes them easier for further processing in extraction of line, word and character from scene image. Preprocessing is done manually, but we don t remove skew angle of the image, thus can just increase the resolution of the image in case of dullness, here for this implementation it involves mainly 2 steps namely resizing and cropping. Block Diagram Preprocessing Input Scene Text (Image Resizing & Image Cropping) Binarisation (Conversion of grayscale to binarised image) Horizontal Profile determination and line separation Vertical Profile determination and word, character extraction Extracted line Fig 1.1 Block Diagram For Proposed Methodology IV. METHODOLOGY Extracted Word & Character Following are the steps involved in proposed methodology Image Resize obtained from digital cameras are of high resolution. Hence they need to be resized. Either scene images from digital cameras or from mobile phone can be given as input to resize. This is done to remove unnecessary areas to reduce the number of pixels. So with help of resizing the resolution of image gets reduced. So, by this step preprocessing is done manually. Image Cropping: The required text image is cropped from the scene image, either we can crop the whole image containing text or can choose part of it from the scene image. Especially the images collected are simple one with background of dark colour and 917

3 the textual matter written on it specially we can consider any name boards, hospital names, Restaurant name, Building names. Image cropping is done as intial step of preprocessing and it is an automated step which is done by using imcrop function which crops the image to the desired length. Image Binarization The image binarization makes the image convert into grayscale image. This stage converts the image into binary image where each pixel is represented by either 0 or 1. It allows reducing image information by removing background so that the image is of black and white type. This type of image is easier for further processing. Extraction System The preprocessed and binarized image acts as an input to the extraction system, through which The extraction of line, word and character from a scene text image is done, which is based on horizontal and vertical This extraction system uses the scene image converted into grayscale image and then using horizontal profiles one can extract number of lines in a scene text image and display the same. Whereas one can extract words and characters from the line extracted through use of vertical profiles, which takes an approximation of space count between word and character spacing and then deduces the output image with words and characters from the cropped scene image. Testing The scene images are taken as samples and then cropped up with the scene text in a image, Testing is done based on whether the line segmentation is taken place properly or no else whether the word and character segmentation is done properly or no. and it is found that out of 100 images we find an up to 95% correct answers for different images it even works for kannada text of scene image but as far it doesn t work well for Hindi text scene images. V. ALGORITHMS PROPOSED The following are the algorithms proposed Overall Proposed Algorithm(Extract Image Parameters) Description: The overall proposed algorithm is as follows: Input: Captured Scene Image Output: The recognized scene text is deduced, line, word and character wise. Begin: Take the scene image as input which contains scene text Phase 1: Image is resized to a basic resolution Phase 2: Cropped image is converted to grayscale Phase 3: Extraction of the line from scene image using horizontal profiles by calling extract line routine Phase 4: Extraction of the word from line extracted of scene image using vertical profiles by calling extract words and characters routine 918 Phase 5: Extraction of the characters from line extracted of scene image using vertical profiles by calling extract words and characters routine Phase 6: Verification is done of the line, word and character extracted by displaying them one at a time. End//End of Proposed Algorithm Algorithm 1.1 Extraction of Line from Text of Scene Image Description: This describes how the line segmentation is carried from scene image Step 1:Take the preprocessed and grayscale converted cropped scene image Step 2: Image background is converted to black if it is other then black Step 3:The grayscale scale image is binarized that is converted to binary 0 s and 1 s. Step 4: Considering the horizontal profiles, the lines are segmented. Here it would first take the input line and then scan it for some low horizontal pixel value, if any low horizontal pixel value are found then they are marked as line in the scene image. All the lines are stored in an array. The start and end index of each of the lines in the input image is marked to find out the number of lines Algorithm 1.2 Extraction of Word and Character from extracted line of scene image Description: This describes the word and character segmentation Step 1: Among the extracted lines from cropped and binarized scene image choose one at time Step 2: Considering the extracted line from the algorithm of the scene image Step 3: To the binarized extracted line, apply with vertical profiles, the words and characters are segmented. Words would have more spacing between them than characters, so it is decided with a threshold value for word spacing dynamically by scanning the line and as per experimental results, then the gaps/spacing is used to extract the words, each of the word is then processed individually for getting the characters from them. Step 4: Using vertical profiles the extraction of word and character from one line each is considered. All the lines are stored in an array. The start and end index of each of the lines in the input image is marked to find out the number of lines, and the start and end location for the same is marked. The start and end index of each of the words in a line are marked in order to extract the words from each of the lines, The start and end index of each of the character in a word are marked in order to extract the characters from each of the word extracted.

4 VI. EXPERIMENTAL RESULTS AND DISCUSSIONS The proposed methodology for extraction system is discussed in detail. The proposed method uses horizontal and vertical profiles. The scene image is taken and the scene text is extracted from it. The system performs well for images captured from high resolution digital cameras as compared to low resolution images. It involves the various algorithms involved for line, word and character segmentation. This section the results of the proposed methodology have been discussed in detail considering several examples of the images. This section discusses regarding experimental results obtained and discusses regarding system performance. The algorithm is implemented in MATLAB. The data sets are collected through various shop names, name plates, city names and others through mobile phone or digital camera. The algorithm is tested with several scene images. Some of these images contained low resolution images and hindi scripts. Sample test results are shown in following Figures. From the experimentation it is understood that the proposed method is reliable to segment text from scene images for English and kannada script. The line segmentation accuracy of 100% for good quality images is observed, the word segmentation accuracy of 98% for good quality is observed, the character segmentation accuracy of 98% for good quality is observed. The limitation of this method is that it resulted in segmentation errors for touching and broken characters especially like any character prefixed or post fixed to V or T. Thus the system is tested for 100 scene images with atleast on an average of 200 lines in all and over 2000 words and characters are determined successfully. Just for curiosity, it was tested on hindi and kannada text of scene images, even though proposed system best works for English text of scene image. It even gave accuracy to a desired rate. Sample Input Image 1: to extraction system to obtain the line, word and character segmentation. Output Image Do you want to crop theimage(y/n): y Press Ctrl + C to Quit Fig 1.3 Cropped Image Enter line number to process (1-2):1 Fig 1.4 Grayscale converted image with line Segmentation Fig 1.5 Word Segmentation Fig 1.2 Input Image Following are the grayscale and binarized images after image preprocessing for each word. These words are given as input Fig 1.6 Character Segmentation Figure 1.3 shows the cropped image from scene image which is cropped from the original image which has two lines and various words. Figure 1.4 shows the converted grayscale image with line segmentation done using horizontal 919

5 Figure 1.5 shows the converted grayscale image with word segmentation done using line extracted with vertical Figure 1.6 shows the converted grayscale image with character segmentation done using line extracted with vertical Sample Input Image 2: Fig 1.10 Word Segmentation Fig 1.11 Character Segmentation Fig1.7 Sample Input Image 2 Above Figure 1.7 shows the selection of an input test sample. Following are the grayscale and binarized images after image preprocessing for each word. These words are given as input to extraction system to obtain the line, word and character segmentation. Output Image Do you want to crop the image(y/n): y Figure 1.8 shows the cropped image from scene image which is cropped from the original image which has two lines and various words. Figure 1.9 shows the converted grayscale image with line segmentation done using horizontal Figure 1.10 shows the converted grayscale image with word segmentation done using line extracted with vertical Figure 1.11 shows the converted grayscale image with character segmentation done using line extracted with vertical System Performance The following Table 1.1 describes the overall system performance of the implementation done, as mainly the extraction system is concentrating on English text of scene image so in the implementation we have considered around only 50 images of different text of a scene image. Fig 1.8 Cropped Image Press Ctrl+c to Quit Enter line number to process ( 1 1):1 Fig 1.9 Grayscale Image Line Segmentation 920

6 Table 1.1 System Performance Parameter s Scene English Text in Scene Kannada Text in Scene Hindi Text in Scene Actual Result in Numbers for 100 for English,50 others Number of Lines in Image Number of words in Image Number of Character s in Image Observed Rate of Segmentation for 100 Lines Extracte d Rate Words Extracted Rate Characters Extracted Rate / / / / / / /50 80/100 0/300 VII. CONCLUSION AND FUTURESCOPE In this paper, the proposed algorithm is tested with several scene text images. Some of the images contain dull images and even hindi scripts. Even though it could segment all the documents in a robust way and gave results. But, it couldn t segment the touching lines and characters. The broken characters have been over segmented. Segmentation of the touching lines and characters may require some heuristic approaches. The word and character segmentation fails in case of hindi scripts and it even fails for English text with touching characters like A, T, V. The system is tested for 100 sample images. Experimental results show that system performance is efficient. The method can be enhanced for same scene image is over coming of segmentation of word and character for touching words and characters and experimentation can be done on this method for efficient extraction and interpretation. The system 921 can be enhanced to extract more than 100 images and also for several different type text of scene images. The future work for the same scene image is over coming of segmentation of word and character for touching words and characters present in image. Even the future work of same can be done for hindi and kannada text with perfection for scene images. And see that surely system produces 100% results for any type of scene text image and for any type of dull, tilted and skewed scene image. VIII. REFERENCES [1] S A Angadi and M M Kodabagi, A Texture Based Methodology for Text Region Extraction From Low Resolution Natural Scene, International Journal of Image Processing (IJIP)Volume (3), Issue (5). [2] M Swamy Das, Segmentation of overlapping text lines, characters in printed Telugu text document, International Journal of Engineering Science and Technology Vol. 2(11), [3] U Pal and B B Chaudhari Automatic Separation of Machine-Printed and Hand-Written Text lines. Computer Vision and Pattern Recognition Unit Indian Statistical Institute 203 B. T. Road, Calcutta, INDIA. [4] Vijaya Kumar Koppula, Negi Atul and Utpal Garain, Robust Text Line, Word And Character Extraction From Telugu Document Image, Second International Conference on Emerging Trends in Engineering and Technology, ICETET-09. [5] Nallapareddy Priyanka, Srikanta Pal&Ranju Mandal, Line and Word Segmentation Approach for Printed Documents, IJCA Special Issue on Recent Trends in Image Processing and Pattern Recognition RTIPPR, [6] M Abdul Rahamin, M. S. Rajasree, A Detailed Study and Analysis of OCR Research in South Indian Scripts, 2009 International Conference on Advances in Recent Technologies in Communication and Computing. [7] Xiaoqing Liu and Jagath Samarabandu, Multiscale edge-based text extraction from complex images. ICME 2006.

7 [8] M C Padma and Vijaya, Script Identification of Text Words from a Tri Lingual Document Using Voting Technique, International Journal of Image Processing (IJIP), Volume (4): Issue 1 [9] U. Pal, S. Sinha and B. B. Chaudhuri, Multi-Script Line identification from Indian Documents, International Conference on Document Analysis and Recognition (ICDAR 2003). [10] M C Padma and Vijaya, script identification from trilingual documents using profile based Feature, International Journal of Computer Science and Applications, Techno mathematicsresearch Foundation Vol. 7 No. 4, pp , [11] Mona Sharma, Performance Evaluation of Image Segmentation and Texture Extraction Methods in Scene Analysis. [12] Farshideh Einsele-Aazami, Recognition of ultra low resolution, antialiased text with small font sizes, work submitted to the the Faculty of Science, University of Fribourg Switzerland. [13] Keechul Jung, Kwang In Kim, Anil K. Jain, Text Information Extraction in and Video: A Survey. [14] Jie Yang, Xilin Chen, Jing Zhang, Ying Zhang, Alex Waibel, Automatic detection and translation of text from natural scenes IEEE. [15] J. Fabrizio1, M. Cord1, B. Marcotegui, Text extraction from street level images, Stilla U, Rottensteiner F, Paparoditis N ( Eds) CMRT09. IAPRS, Vol. XXXVIII, Part 3/W4 --- Paris, France, 3-4 September, [16] K. C. Kim, H. R. Byun, Y. J. Song, Y. W. Choi, S. Y. Chi,K. K. Kim, Y. K. Chungin, Scene text extraction in natural scene images using hierarchical feature combining and Verification, 17th International Conference on Pattern Recognition (ICPR 04). [17] Celine mancas-thillou, Book Natural Scene Text Understanding. [18] Hwan-chul Park, Se-young Ok, Hwan-gue Cho, Word Extraction in Text/Graphic Mixed Image Using 3-Dimensional Graph Model,Proc of ICCPOL 99,pp ,March [19] Chin-Shyurng Fahn and Chia-Wei Liu, An AdaBoost Approach to Detecting and Extracting Texts from Natural Scene, [20] Eng. Yasser M. Abbass, Dr. Waleed Fakher, Dr. Mohsen Rashwan, Arabic / English Identification in a hybrid complex documents images. GVIP 05 Conference, December 2005, CICC, Cairo, Egypt. 922