Fast Restoration of Warped Document Image based on Text Rectangle Area Segmentation

Transcription

1 1162 JOURNAL OF SOFTWARE, VOL. 8, NO. 5, MAY 2013 Fast Restoration of Warped Document Image based on Tet Rectangle Area Segmentation Kuo-Hsien Hsia Department of Computer Science and Information Engineering, Far East University, Tainan, Taiwan(R.O.C.) Sao-Fan Lien Graduate Scool of Engineering Science & Tecnology (Doctoral Program), National Yunlin University of Science & Tecnology, Yunlin, Taiwan(R.O.C.) Jung-Perng Su Department of Electrical Engineering, National Yunlin University of Science & Tecnology, Yunlin, Taiwan(R.O.C.) Abstract Te warp problems usually make te documents being ardly recognized. Specifically, wen we copy a page of a tick book or bound document by digital potocopier, te resulted image is usually warped because of te tickness of te document. We focus on tis problem and propose a fast metod to restore te warped document image in tis paper. Te tet rectangle area of te document is one of te features of a document. Te morpological operation is utilized for tet rectangle area segmentation. Te DLT metod is used to compute te mapping relations between te warped document and te non-warped document. In eperimental results, te proposed metod works on ig resolution image very quickly. Te warping tet and te figures in documents ave been restored by te proposed metod successfully. Tis metod is efficiency and fast for implementing on te module of digital potocopier. Inde Terms warped document; image process; DLT metod; tet rectangle area segmentation. I. INTRODUCTION Te first potocopier (or copier) is invented by Carlson (American) in Te first commercial copier Xero 914 is developed by Xero in Te Xero 914 can copy 7 documents per second. Te potocopier can copy te large numbers of tets and images from documents to papers quickly. Even in tis era of igly computerized, te potocopier is still commonly used in offices, scools, or libraries. Te main process of potocopier are carge, eposure, development, transfer and fusing. In recent years, te traditional potocopier is replaced by te digital potocopier, gradually. Te process of digital potocopier is almost te same wit te traditional potocopier. Te process of Manuscript received Mar. 1, 2011; revised Apr. 1, 2011; accepted Apr. 12, Corresponding autor: Kuo-Hsien Hsia. digital potocopier just adds te digitalization after eposure. In oter word, te Organic Poto-Conductor Drum (OPC Drum) of digital potocopier can capture te digital signals of documents. Wen we copy a page of a tick book or a bound document, te resulted image is usually warped because of te tickness of te document. Terefore, our objective is to design a fast algoritm for restoring te warped document image for a digital potocopier. Te metod sould be developed on embedded systems of digital potocopier. We ope tat te copied documents are nonwarped and te copy speed is uncanged. Te warp makes te tet not only difficult to read but also being ardly recognized by Optical Caracter Recognition (OCR). Many researcers ave been worked on te warp problem. We are interested in tree major types of approac about tem. Te first type is te model based de-warping metod. Suc as [1], te plane transform model is establised in advance for restoring te warping from camera captured image. Zang and Tan [2][3] proposed te model of books from te sading and pysical analysis. Brown and Brent [4] presented a framework for acquiring and restoring images of warped documents. Te second type is segmentation based recovery metod. Tis kind of metods are usually segmenting te caracters and detecting te tet lines. Based on te tet lines, te documents can be restored. Gatos and Pratikakis [5] acieved binary image dewarping based on word rotation and translation according to upper and lower word baselines. Moreover, [6] [7] and [8] are using te similar approaces for warped document restoring. Te last is te 3D acquisition metod. Suc as [9] and [10], a lot of control points are acquired for establising te surface of document. However, tis metod is te slowest one of te tree approaces. For our applications, te 3D acquisition metod is out of our consideration because of its time-consuming. Te model based approac needs te parameters of digital doi: /jsw

2 JOURNAL OF SOFTWARE, VOL. 8, NO. 5, MAY potocopier or book. However, tese parameters are not easy to obtain from potocopier manufacturing companies practically. Tus we will propose a fast metod similar to segmentation based approac for warped documents restoring in tis paper. However, a lot of computation is inevitable for detecting te caracters and te base line of tet. In contrast to former metods, te tet area is muc easier to segment by our metod. Te DLT algoritm is used to compute te mapping relations between te warped document and non-warped document. Terefore, te process could restore te document quickly and effectively. Some cases of te warped document restored by te metod will be sown in tis paper. From te eperimental results, it is demonstrated tat te metod can work for tets, equations and figures. Moreover, te computing time of ig resolution document image is under 0.1 sec. In our survey, few literatures about tis topic consider te computing time. Terefore, we strongly believe tat te metod could be realized in embedded system of digital potocopier. II. WARPED DOCUMENT IMAGE RESTORATION METHOD Tere are tree parts in te process. Te proposed metod is sown in Figure 1. Te major steps are image pre-processing, tet rectangle area segmentation and document image restoration. More details will be discussed in te following. Image input Binarization Filtering Image pre-processing Morpological operation Edge and Corer Detection Tet rectangle area segmentation Projective Transformation Direct Linear Transformation Algoritm Document image restoration Revised image Figure 1. Warped document image restoration metod. A. Tet Rectangle Area Segmentation We found tat te tet areas in a general document are usually in rectangle. But te area will often be warped in te copied image. Tree steps are proposed for segmenting te tet rectangle area. In [11] and [12], some useful image segmentation algoritms ave been proposed. However, te processes of te metods are complicated for our application. Terefore, te fast algoritm, Niblack algoritm [13], is utilized for segmenting te document into binary image. Net, filtering and te morpology operation [14] are used for computing te tet rectangle area. 1) Niblack Algoritm Te Niblack algoritm is te metod for computing local tresold. ρ(, y) I(, y) m(, y) 1 k1 (1) L were m(, y) is te mean of te area image, ρ(, y) is te standard deviation. k and L are constants depending on practical applications. For te cases discussed in tis paper, k = 0.25 and L=136. Niblack algoritm can remove te sadow and salt-and-pepper noise preliminarily. Figure 2 sows a result of Niblack algoritm. Te sadow area in Figure 2 is almost removed in Figure 2 by Niblack algoritm. Te resulting image is clear enoug for te net process. 2) Tet Rectangle Area Segmentation In ere, te dilation and erosion is utilized to determine te tet area. Erosion and dilation are commonly used operations of morpological image processing. Te related introductions are described in te following. An image A eroded by a matri B, denoted by A B, is defined as: A B A( i, j) min( A( i, j) B( k, l)) (2) were B is a square mask, and te size of erosion range depends on B. Similarly, an image A via dilation operation by a matri B is denoted by A B. Te relation is described as: A B ( Bˆ) A (3) were is te empty set and is a point on te image. Figure 3 is te result from Figure 2 by inversion. Figure 3 is te result of 3 times erosion by a mask. Te tet parts become some rectangular areas. Net, we dilated Figure 3 4 times by a mask. In Figure 3(c), 5 tet areas are segmented. 3) Feature Points and Corresponding Control Points Detection From Figure 3(c), te tet areas are warped rectangles. Te features of te tet areas are te corners and edges. Harris corner detector [15] is used for detecting te corners. Te corner points and te mapping of te corresponding control points are illustrated in figure 4. Figure 4 is te default page and control points. Te areas in figure 4 are defined from tet rectangle area of original image.

3 1164 JOURNAL OF SOFTWARE, VOL. 8, NO. 5, MAY 2013 B. Document Image Restoration Te document image restoration is a 2D to 2D projective transformation. In tis paper, direct linear transformation (DLT) algoritm is used for recovering te documents. Figure 2. Te binarization of document image. Original image. Te result of Niblack algoritm. 1) Projective transformation Te projective transformation matri [16] from te warped document to te original document can be given by or X =HX. Let te inomogeneous coordinates X and in plane A and be (, y) and (, y ), respectively. Te projective transformation of equation (4) will be written as (4) (c) Figure 3. Te igligt of binarization document image. Te result of erosion operation. Te result of dilation operation.

4 JOURNAL OF SOFTWARE, VOL. 8, NO. 5, MAY P1 P3 P4 P6 P7 P8 P2 P5 Figure 4. Te control points mapping relation. Te feature points of tet areas. Te default page and control points. y p1 p3 p4 p6 p7 y y p8 13 y y Re-arrange equation (5) and we will ave y 13 y 23 y y 23 y y y 0 0 Equation (6) is an over-determined system, te DLT algoritm [14] is utilized for solving te system. 2) Direct Linear Transformation Algoritm Rewrite equation (6), te linear system can be epressed in matri form: A H ˆ ˆ (7) (5) (6) p2 p5 ˆ, were 1 y1 2 y2 T i y i ˆ T, and H y y y1 1 y1 1 y1 y1 y1 2 y y2 2 A y2 1 y2 2 y2 y2 y2 (8) i i i i i i 1 i Te algoritm is illustrated as follows: STEP 1 For eac corresponding point, compute te matri A. (i 4 ) STEP 2 Obtain te SVD (Singular value decomposition) of A. STEP 3 If A=UDV T. Te last column of vector V corresponding to te smallest singular value of A are te elements of te vector H. III. EXPERIMENTAL RESULTS AND ANALYSIS In tis section, we demonstrated te warped document image restoration by our metod. Te sizes of te left and rigt page images are bot piels. Tey are large image files. Te test images are scanned from a 750-page book. Te tickness of te book will cause te warp in copng or scanning. Figure 5 is te left (odd-numbered) page image. Figure 5 is te de-warped result by te proposed metod. Figure 6 is te rigt (even-numbered) page image. Te image is warped seriously. Tis image as rotation and warping. Moreover, tere are not only tets but also figures in tis image. In Figure 6, te tet and figures in te image are restored successfully by our metod. Figure 5. Case 1: Left page document image restoration. Original image. Te result of te restoration.

5 1166 JOURNAL OF SOFTWARE, VOL. 8, NO. 5, MAY 2013 embedded systems of digital copier. In some case, suc as figure 7, te restored tet area as a little trapezoidal deformation. Tat needs furter works to improve it in te future. Figure 6. Case 2: Rigt page document image restoration. Original image. Te result of te restoration. Te last case is sown in Figure 7. Tis is a common situation wen we copng or scanning a tick books. Figure 7 is also te rigt page image. Te warp in upper rigt corner is severer tan oter parts of te document. By our metod, te tet area in te image is restored. Te computation time of case 1, case 2, and case 3 are sec, sec and sec, respectively. IV. CONCLUSION In tis paper, te tet rectangle area segmentation and direct linear transformation (DLT) algoritm are used for restoring te warped document image. Te comple tet areas are segmented by epansion and erosion. Te simple rectangles and corners are used for computing te mapping relation between te warped document and nonwarped document by DLT algoritm. In te eperimental results, tree ig resolution images are restored quickly and successfully. Not only te warped tet but also te figures in documents can be restored by te proposed metod. Tis metod is easy to implement on te Figure 7. Case 3: Anoter rigt page document image restoration. Original image. Te result of te restoration. REFERENCES [1] M. Wu, R. Li, B. Fu, W. Li and Z. Xu, A Model-based Book Dewarping Metod to Handle 2D Images Captured by a Digital Camera, International Conference on Document Analysis and Recognition, vol. 1, pp , [2] L. Zang and C. L. Tan, Warped Document Image Restoration Using Sape-from-Sading and Pysically- Based Modeling, IEEE Worksop on Applications of Computer Vision, pp.29-29, [3] L. Zang and C. L. Tan, Restoring Warped Document Images using Sape-from-Sading and Surface Interpolation, International Conference on Pattern Recognition, vol.1, pp , 2006.

6 JOURNAL OF SOFTWARE, VOL. 8, NO. 5, MAY [4] M. S. Brown and W. B. Seales, Image Restoration of Arbitrarily Warped Documents, IEEE Transactions on Pattern Analysis and Macine Intelligence, vol. 26, No.10: pp , [5] B. Gatos, I. Pratikakis and K. Ntirogiannis, Segmentation Based Recovery of Arbitrarily Warped Document Images, International Conference on Document Analysis and Recognition, vol. 2, pp , [6] W. Zang, X. Li and X. Ma, Perspective Correction Metod for Cinese Document Images, International Symposium on Intelligent Information Tecnology Application Worksops, pp , [7] Y. Zang, C. Liu, X. Ding and Y. Zou, Arbitrary Warped Document Image Restoration Based on Segmentation and Tin-Plate Splines, International Conference on Pattern Recognition, pp. 1-4, [8] A. Ulges, C. H. Lampert and T. M. Breuel, Document Image Dewarping using Robust Estimation of Curled Tet Lines, International Conference on Document Analysis and Recognition, vol. 2, pp , [9] K. B. Cua, L. Zang, Y. Zang and C. L. Tan, A Fast and Stable Approac for Restoration of Warped Document Images, International Conference on Document Analysis and Recognition, vol. 1, pp , [10] Z. Zang, C. L. Tan and L. Fan, Estimation of 3D Sape of Warped Document Surface for Image Restoration, International Conference on Pattern Recognition, vol. 1, pp , [11] C. Lui, X. Zang, X Li, Y. Li, and J. Yang, Gaussian Kernelized Fuzzy c-means wit Spatial Information Algoritm for Image Segmentation, Journal of Computer, vol. 7, no. 6, pp , [12] J. Lin and J Wang, Researc on Contour Correction in Medical CT Image Segmentation, Journal of Computer, vol. 7, no. 3, pp , [13] W. Niblack, An Introduction to Digital Image Processing. Prentice Hall, Englewood Cliffs, [14] L. G. Sapiro and G. C. Stockman, Computer Vision, Prentice Hill [15] M. Nion and A. Aguado, Feature Etraction & Image Processing, Amsterdam Academic, Boston, [16] R. I. Hartley and A. Zisserman, Multiple View Geometry in computer vision, Cambridge, UK, Kuo-Hsien Hsia was born in Taiwan in He received te P.D. degree in electrical engineering from te National Sun Yat-Sen University, Taiwan, in He is currently an associate professor of te Department of Computer Science and Information Engineering, Far-East University, Taiwan. His researc interests are in te area of differential game teory, fuzzy teory, imageassistant measurement and control software design. Sao-Fan Lien was born in Taiwan in He received te B.S. degree from Department of Electronic Engineering of Huafan University in 2004 and received te M.S. degree from Tungnan University in He is P.D. student of Graduate Scool of Engineering Science and Tecnology, National Yunlin University of Science and Tecnology. His researc interests include computer vision and vision servoing system control. Jung-Perng Su was born in Taiwan in He received te M.S. degree in electrical engineering from te National Taiwan University of Science and Tecnology, Taipei, Taiwan, in 1986, and te P.D. degree in electrical engineering from Sun Yat-sen University, Kaosiung, Taiwan, in Since 1991, e as been wit te National Yunlin University of Science and Tecnology, Yunlin, Taiwan, were e is currently a Professor in te Department of Electrical Engineering. Since 2004, e as been conducting researc projects on autonomous unmanned elicopters. His current researc interests include te design of autonomous unmanned veicles, robust estimation and optimal control, intelligent systems, and nonlinear control. Dr. Su was a recipient of te distinguised researc award from Yunlin University of Science and Tecnology in 2005 and awards from various domestic competitions.