DIGITISATION: CONCEPT, NEED, METHODS AND EQUIPMENT

Transcription

1 UNIT 7 DIGITISATION: CONCEPT, NEED, METHODS AND EQUIPMENT Structure 7.0 Objectives 7.1 Introduction 7.2 Digitisation: Basics Definition Need for Digitisation 7.3 Seection of Materias for Digitisation 7.4 Steps in the Process of Digitisation Scanning Indexing Storing Retrieving 7.5 Digitisation: Input and Output Options Scanning as Image Ony Optica Character Recognition (OCR) and Retaining Page Layout Retaining Page Layout using Acrobat Capture Re-keying 7.6 Technoogy of Digitisation Bit Depth or Dynamic Range Resoution Threshod Image Enhancement 7. 7 Compression Lossess Compression Lossy Compression Compression Protocos 7.8 Fie Formats and Media Types Formats and Encoding used for Text 7.9 Toos of Digitisation Scanners Scanning Software 7.10 Digitisation of Audio and Video 7.11 Organising Digita Images 7.12 Digita Library Softwares Panning and Impementation Feasibiity Panning the Project Purchase of Hardware and Software Seection of Materia for Digitisation and Born Digita

2 Pacement and Training of Manpower Content Creation Execution of the Project 7.14 Summary 7.15 Answers to Sef Check Exercises 7.16 Keywords 7.17 References and Further Reading Digitisation: Concept, Need, Methods and Equipment 7.0 OBJECTIVES After reading this Unit you wi know about the foowing concepts reate to the technoogy of digitisation: digitisation: basics, concept and need; steps in the process of digitisation; technoogy of digitisation; compression; Optica character recognition (OCR); fie formats; toos of digitisation; and organising digita images. 7.1 INTRODUCTION A recorded information in a traditiona ibrary is anaogue in nature. The anaogue information can incude printed books, periodica artices, manuscripts, cards, photographs, viny disks, video and audiotapes. However, when anaogue information is fed into a computer, it is broken down into 0s and 1s changing its characteristics from anaogue to digita. These bits of data can be re-combined for manipuation and compressed for storage. Vouminous encycopaedias that take-up yards of shef-space in anaogue form can fit into a sma space on a computer drive or stored on to a CD ROM disc, which can be searched, retrieved manipuated and sent over the network. One of the most important traits of digita information is that it is not fixed in the way that texts printed on a paper are. Digita texts are neither fina nor finite, and are not fixed either in essence or in form except, when it is printed out as a hard copy. Fexibiity is one of the chief assets of digita information. An endess number of identica copies can be created from a digita fie, because a digita fie does not decay by copying. Moreover, digita information can be made accessibe from remote ocation simutaneousy by a arge number of users. Digitisation is the process of converting the content of physica media (e.g., periodica artices, books, manuscripts, cards, photographs, viny disks, etc.) into digita format. In most ibrary appications, digitisation normay resuts in documents that are accessibe from the web site of a ibrary and thus, on the Internet. Optica scanners and digita cameras are used to digitise images by transating them into bit maps. It is aso possibe to digitise sound, video, graphics and animations, etc. Digitisation is not an end in itsef. It is the process that creates a digita image from an anaogue image. Seection criteria, particuary those, which refect user needs, are of paramount importance. Therefore, the principes that are appicabe in traditiona coection 153

3 deveopment are appicabe when materias are being seected for digitisation. However, there are severa other considerations reated to technica, ega, poicy, and resources that become important in a digitisation project. Digitisation is one of the three important methods of buiding digitised coections. The other two methods incude providing access to eectronic resources (whether free or icensed) and creating ibrary portas for important Internet resources. 7.2 DIGITISATION: BASICS Definition The word digita describes any system based on discontinuous data or events. Computers are digita machines because at their most basic eve they can distinguish between just two vaues, 0 and 1, or off and on. A data that a computer processes must be encoded digitay as a series of zeroes and ones. The opposite of digita is anaogue. A typica anaogue device is a cock in which the hands move continuousy around the face. Such a cock is capabe of indicating every possibe time of the day. In contrast, a digita cock is capabe of representing ony a finite number of times (every tenth of a second, for exampe). As mentioned before, a printed book is anaogue form of information. The contents of a book need to be digitised to convert it into digita form. Digitisation is the process of converting the content of physica media (e.g., periodica artices, books, manuscripts, cards, photographs, viny disks, etc.) to digita formats. Digitisation refers to the process of transating a piece of information such as a book, journa artices, sound recordings, pictures, audio tapes or videos recordings, etc. into bits. Bits are the fundamenta units of information in a computer system. Converting information into these binary digits is caed digitisation, which can be achieved through a variety of existing technoogies. A digita image, in turn, is composed of a set of pixes (picture eements), arranged according to a pre-defined ratio of coumns and rows. An image fie can be managed as a reguar computer fie and can be retrieved, printed and modified using appropriate software. Further, textua images can be OCRed so as to make its contents searchabe. An image of the physica object is captured using a scanner or digita camera and converted into digita format that can be stored eectronicay and accessed via computers. The process of digitisation, however does not stop at scanning of physica objects, a considerabe amount of work is invoved in optimising usage of digitised documents. Sometimes, these post- scanning processes are often assumed in the meaning of digitisation. At other times the word digitisation is used in restricted sense to incude ony the process of scanning Need for Digitisation Digitising a document in print or other physica media (e.g., sound recordings) makes the document more usefu as we as more accessibe. It is possibe for a user to conduct a fu-text search on a document that is digitised and OCRed. It is possibe to create hyperinks to ead a reader to reated items within the text itsef as we as to externa resources. Utimatey, digitisation does not mean repacing the traditiona ibrary coections and services; rather, it serves to enhance them. 154 A document can be converted into digita format depending on the objective of digitisation, end user, avaiabiity of finances, etc. Whie the objectives of digitisation initiatives differ from organisation to organisation, the primary objective is to improve the access. Other objectives incude cost savings, preservation, keeping pace with technoogy and information sharing. The most significant chaenges in panning and execution of a digitisation project reate to technica imitations, budgetary constraints, copyright considerations, ack of poicy guideines and asty, the seection of materias for digitisation.

4 Whie new and emerging technoogies aow digita information to be presented in innovative ways, the majority of potentia users are unikey to have access to sophisticated hardware and software. Sharing of information among various institutions is often restricted by the use of incompatibe software. Digitisation: Concept, Need, Methods and Equipment One of the main benefits of digitisation is to preserve rare and fragie objects by enhancing their access to mutipe users simutaneousy. Very often, when an object is rare and precious, access is ony aowed for a certain category of peope. Going digita coud aow more users to enjoy the benefit of access. Athough, digitisation offers great advantages for access ike, aowing users to find, retrieve, study and manipuate materia, it cannot be considered as a good aternative for preservation because of ever changing formats, protocos and software used for creating digita objects. There are severa reasons for ibraries to go for digitisation and there are as many ways to create the digitised images, depending on the needs and uses. The prime reason for digitisation is the need of the user for convenient access to high quaity information. Other important considerations are: Quaity Preservation: The digita information has potentia for quaitative preservation of information. The preservation-quaity images can be scanned at high resoution and bit depth for best possibe quaity. The quaity remains the same inspite of mutipe usages by severa users. However, caution needs to be exercised whie choosing digitisation for preservation of information. Mutipe Referencing: Digita information can be used simutaneousy by severa users at a time. Wide Area Usage: Digita information can be made accessibe to distant users through the computer networks over the Internet. Archiva Storage: Digitisation is used for restoration of rare materia. The rare books, images or archiva materia are kept in digitised format as a common practice. Security Measure: Vauabe documents and records are scanned and kept in digita format for safety and security. Sef Check Exercise 1) Define digitisation. What are the major benefits of digitisation? Note: i) ii) Write your answer in the space given beow. Check your answer with the answers given at the end of the Unit SELECTION OF MATERIALS FOR DIGITISATION To begin the process of digitisation, first of a, we need to seect documents for digitisation. The proccess of seection of materia for digitisation invoves identification, seection and prioritisation of documents that are to be digitised. If an organisation generates contents, strategies may be adopted to capture data that is born digita. If documents are avaiabe in digita form, it can be easy converted into other formats. If the seected materia is 155

5 from the externa sources, IPR issues need to be resoved. If materia being digitised is not avaiabe in pubic-domain, then it is important to obtain permission from the pubishers and data suppiers for digitisation. The IPR issues must be addressed eary in the seection process. Getting permissions from the pubishers and individuas coud be time consuming, difficut and may invove negotiation and payment of copyright fees. Moreover, decision may be taken whether to OCR the digitised images. Documents seected for digitisation may aready be avaiabe in digita format. It is aways economica to buy e-media, if avaiabe, than their conversion. Moreover, over-sized materia, deteriorating coections, bound voumes of journas, manuscripts, etc. woud require highy speciaized equipment and highy skied manpower. The documents to be digitised may incude text, ine art, photographs, coour images, etc. The seection of documents needs to be reviewed very carefuy considering a the factors of utiity, quaity, security and cost. Rare and much in-demand documents and images are seected as first priority without considering the quaity. Factors that may be considered for seecting appropriate media for digitisation incude the foowing: Audio: The sound quaity has to be checked and required corrections made together by the subject expert and computer sound editor. Video: The video cippings are normay edited on Beta max tapes, which can be used for transferring on to digita format. Whie editing coour tone, resoution is checked and corrected. Photographs: The seection of photographs is very crucia process. High resoution is required for photographic images and sides. Aso, the quaity and future needs are to be checked and the copyright aspects are to be taken care of. Documents: Documents which are much in demand, too fragie to hande, and rare in avaiabiity are reviewed and seected for the process. If the correction of iterary vaue demands much input, then documents are considered for pubication rather than digitisation. Moreover, the purpose of a digitisation is reated to increased access to digitised materias and vaue addition. The first consideration for digitisation of documents shoud be inteectua significance of contents in terms of quaity, authority, uniqueness, timeiness, and demand. The inteectua contents, physica nature of the source materias, number of current and potentia users are therefore, major considerations. Sef Check Exercise 2) Describe criteria used for seection of materia for digitisation. Note: i) ii) Write your answer in the space given beow. Check your answer with the answers given at the end of the Unit. 156

6 7.4 STEPS IN THE PROCESS OF DIGITISATION The foowing four steps are invoved in the process of digitisation. Software, variaby caed Document Image Processing (DIP), Eectronic Fiing System (EFS) and Document Management System (DMS), provides a or most of these functions. Digitisation: Concept, Need, Methods and Equipment Scanning Eectronic scanners are used for acquisition of an eectronic image into a computer from its origina that may be a photograph, text, manuscript, etc. An image is read or scanned at a predefined resoution and dynamic range. The resuting fie, caed bit-map page image is formatted (image formats described esewhere) and tagged for storage and subsequent retrieva by the software package used for scanning. Acquisition of image through fax card, eectronic camera or other imaging devices is aso feasibe. However, image scanners are most important and most commony used component of an imaging system for the transfer of norma paper-based documents. Fig. 7.1: Scanning using a Fatbed Scanner Steps in the Process of Scanning using a Fatbed Scanner Step 1 Pace picture on the scanner s gass Step 2 Start scanner software Step 3 Seect the area to be scanned Step 4 Choose the image type Step 5 Sharpen the image Step 6 Set the image size Step 7 Save the scanned image using a desirabe format (GIF or JPEG) 157

7 Sef Check Exercise 3) Describe the steps invoved in the process of scanning a document using a fatbed scanner. Note: i) ii) Write your answer in the space given beow. Check your answer with the answers given at the end of the Unit Indexing If converting a document into an image or text fie is considered as the first step in the process of imaging, indexing these fies comprises the second step. The process of indexing scanned images invoves inking of the database of scanned images to a text database. Scanned images are just ike a set of pictures that need to be reated to a text database describing them and their contents. An imaging system typicay stores a arge amount of unstructured data in a two fie system for storing and retrieving scanned images. The first is traditiona fie that has a text description of the image (keywords or descriptors) aong with a key to a second fie. The second fie contains the document ocation. The user seects a record from the first fie using a search agorithm. Once the user seects a record, the appication program keys into the ocation index, finds the document and dispays it. Author Tite keywords keywords Image45 Image45 /image/new/smith.pdf Key to image Image ocation Fig. 7.2: Two Fie System in a Image Retrieva System 158 Most of the document imaging software packages through their menu driver or command driven interface, faciitate eaborate indexing of documents. Whie some document management systems faciitate seection of indexing terms from the image fie, others aow ony manua keying in of indexing terms. Further, many DMS packages provide OCRed capabiities for transforming the images into standard ASCII fies. The OCRed text then serves as a database for fu-text search of the stored images Storing The most tenacious probem of a document image reates to its fie size and, therefore, to its storage. Every part of an eectronic page image is saved regardess of the presence or absence of ink. The fie size varies directy with scanning resoution, the size of the area being digitised and the stye of graphic fie format used to save the image. The scanned images, therefore, need to be transferred from the hard disc of scanning workstation to an externa arge capacity storage devices such as an optica disc, CD ROM/DVD ROM disc, snap servers, etc. Whie the smaer document imaging system may use offine media, which need to be reoaded when required, or fixed hard disc drives aocated for image storage, arger document management systems use auto-changers such as optica jukeboxes and tape ibrary systems. The storage required by the scanned images varies

8 and depends upon factors such as scanning resoution, page size, compression ratio and page content. Further, the image storage device may be either remote or oca to the retrieva workstation depending upon the imaging system and document management system used. Digitisation: Concept, Need, Methods and Equipment Retrieving Once scanned images and OCRed text documents have been saved as a fie, a database is needed for seective retrieva of data contained in one or more fieds within each record in the database. Typicay, a document imaging system uses at east two fies to store and retrieve documents. The first is traditiona fie that has a text description of the image aong with a key to the second fie. The second fie contains the document ocation. The user seects a record from the first-fie using a search agorithm. Once the user seects a record, the appication program keys into the ocation index, finds the document and dispays it. Most of the document management systems provide eaborate search possibiities incuding use of Booean and proximity operators (AND, OR, NOT) and wid cards. Users are aso aowed to refine their search strategy. Once the required images have been identified their associated document image can quicky be retrieved from the image storage device for dispay or for getting printed output. 7.5 DIGITISATION: INPUT AND OUTPUT OPTIONS A document can be converted into digita format depending on the objective of digitisation, end users, avaiabiity of finances, etc. There are four basic approaches that can be adapted to transform from print to digita: Scanning as Image Ony OCRing and Retaining Page Layout Retaining Page Layout using Acrobat Capture; and Re-keying the Data Scanning as Image Ony Image ony is the owest cost option in which each page is an exact repica of the origina source document. Severa digita ibrary projects are concerned with providing digita access to materias that aready exists in printed media in traditiona ibraries. Scanned page images are practicay the ony reasonabe soution for institutions such as ibraries for converting existing paper coection (egacy documents) without having access to the origina data in computer processibe formats convertibe into HTML / SGML or in any other structured or unstructured text. Scanned page images are natura choice for arge-scae conversions for major digita ibrary initiatives. Printed text, pictures and figures are transformed into computer-accessibe forms using a digita scanner or a digita camera in a process caed document imaging or scanning. The digitay scanned images are stored in a fie as bit-mapped page images, irrespective of the fact that a scanned page contains a photograph, a ine drawing or text. A bit-mapped page image is a type of computer graphic, iteray an eectronic picture of the page which can most easiy be equated to a facsimie image of the page and as such they can be read by humans, but not by the computers, understaby text in a page image is not searchabe on a computer using the present-day technoogy. An image-based impementation requires a arge space for data storage and transmission. Capturing page image format is comparativey easy and inexpensive, therefore, it is a faithfu reproduction of its origina maintaining page integrity and originaity. The scanned textua images, however, are not searchabe uness it is OCRed, which in itsef, is highy error prone process speciay when it invoves scientific texts. Options of technoogy for converting print to digita are given separatey. 159

9 If OCR is not carried out, the document is not searchabe. Most scanning softwares generate TIFF format by defaut, which, can be converted into PDF using a number of software toos. Scanning to TIFF / PDF format is recommended ony when the requirement of project is to make documents portabe and accessibe from any computing patform. The image can be browsed through a tabe of contents fie composed in HTML that provides ink to scanned image objects Optica Character Recognition (OCR) and Retaining Page Layout The atest versions of both Xerox s TextBridge and Caere s Omnipage incorporate technoogy that aow the option of maintaining text and graphics in their origina ayout as we as pain ASCII and word-processing formats. Output can aso incude HTML with attributes ike bod, underine, and itaic which are retained. Retaining Layout after OCR A scanned document is nothing more than a picture of a printed page. It cannot be edited or manipuated or managed based on their contents. In other words, scanned documents have to be referred to by their abes rather than characters in the documents. OCR (Optica Character Recognition) programs are software toos used to transform scanned textua page images into word processing fie. OCR or text recognition is the process of eectronicay identifying text in a bit-mapped page image or set of images and generate a fie containing that text in ASCII code or in a specified word processing format eaving the image intact in the process Retaining Page Layout using Acrobat Capture The Acrobat Capture 2.0 provides severa options for retaining not ony the page ayout but aso the fonts, and to fit text into the exact space occupied in the origina, so that the scanned and OCRed copy never over- or under-shoots the page. Accordingy, it treats unrecognisabe text as images that are pasted in its pace. Such images are perfecty readabe by anyone by ooking at the PDF fie, but wi be absent from the editabe and searchabe text fie. In contrast, ordinary OCR programs treat unrecognised text as wid or some other specia character in the ASCII output. Acrobat Capture can be used to scan pages as images, image +text and as norma PDF, a the three options retain page ayout. i) Image Ony: Image ony option has aready been described in Section ii) iii) Image + Text: In image+text soutions, OCRed text is generated for each image where each page is an exact repica of the origina and eft untouched, however, the OCRed text sits behind the image and is used for searching. The OCRed text is generay not corrected for errors since; it is used ony for searching. The cost invoved is much ess than PDF Norma. However, the entire page is a bitmap and neither fonts nor ine drawings are vectorised, so the fie size of Image + Text PDFs is consideraby arger than the corresponding PDF Norma fies and pages wi not dispay as quicky or ceany on screen. PDF Norma: PDF norma gives the cear view on-screen dispay. It is searchabe, with significanty smaer fie size than Image+Text. The resut is not, however, an exact repica of the scanned page. Whie a graphics and formatting are preserved, substitute fonts may be used where direct matches are not possibe. It is a good choice when fies need to be posted on to the web or otherwise deivered onine. If during the Capture and OCR process, a word cannot be recognised to the specified confidence eve, Capture, by defaut, substitutes a sma portion of the origina bitmap image. Capture best guess of the suspect word ies behind the bitmap so that searching and indexing are sti possibe. However, one cannot guarantee that these

10 bitmapped words are correcty guessed. In addition, the bitmap is somewhat obtrusive and detracting from the ook of the page. Further, Capture provides option to correct suspected errors eft as bit-mapped image or eave them untouched Re-keying Digitisation: Concept, Need, Methods and Equipment A cassic soution of this kind woud comprise of keying-in the data and its verification. This invoves a compete keying of the text, foowed by a fu re-keying by a different operator, the two keying-in operations might take pace simutaneousy. The two keyed fies are compared and any errors or inconsistencies are corrected. This woud guarantee at east 99.9% accuracy, but to reach % accuracy eve, it woud normay require fu proof-reading of the keyed fies, pus tabe ookups and dictionary spe checks. Fig. 7.3: Re-keying-in as an Option for Digitisation 7.6 TECHNOLOGY OF DIGITISATION Digita images, aso caed bit-mapped page image are eectronic photographs composed or set of bits or pixes (picture eements) represented by 0 and 1. A bitmapped page image is a true representation of its origina in terms of typefaces, iustrations, ayout and presentation of scanned documents. As such, information or contents of bitmapped page image cannot be searched or manipuated unike text fie documents (or ASCII). However, an ASCII fie can be generated from a bit-mapped page image using an optica character recognition (OCR) software such as Xerox s TextBridge and Caere s OmniPage. The quaity of digita image can be monitored at the time of capture by the foowing factors: Bit depth / Dynamic Range Resoution Threshod Image Enhancement Terminoogy associated with technoogica aspects of digitisation described beow is given in the keywords. Students are advised to understand the terminoogy, speciay bit, byte and pixe before going through the Unit Bit Depth or Dynamic Range The number of bits used to define each pixe determines the bit depth. The greater the bit depth, the greater the number of gray scae or coour tones that can be represented. Dynamic range is the term used to express the fu range of tota variations, as measured by a densitometer between the ightest and the darkest of a document. Digita images can be captured at varied density or bits per pixe depending upon (i) the nature of source materia or document to be scanned; (ii) target audience or users; and (iii) capabiities of the dispay and print subsystem that are to be used. Bitona or back & white or binary 161

11 scanning is generay empoyed in ibraries to scan pages containing text or the drawings. Bitona or binary scanning represents one bit per pixe (either 0 (back) or 1 (white). Gray scae scanning is used for reiabe reproduction of intermediate or continuous tones found in back & white photographs to represent shades of grey. Mutipe numbers of bits ranging from 2-8 are assigned to each pixe to represent shades of grey in this process. Athough each bit is either back or white, as in the case of bitona images, but bits are combined to produce a eve of grey in the pixe that is, back, white or somewhere in between. Fig. 7.4: Setting Bit Depth in Precisionscan Pro Scanning Software Lasty, coour scanning can be empoyed to scan coour photographs. As in the case of grey-scae scanning, mutipe bits per pixes typicay 2 (owest quaity) to 8 (highest quaity) per primary coour are used for representing coour. Coour images are evidenty more compex than grey scae images, because it invoves encoding of shades of each of the three primary coours, i.e., red, green and bue (RGB). If a cooured image is captured at 2 bits per primary coour, each primary coour can have 2 2 or 4 shades and each pixe can have 4 3 shades for each of the three primary coours. Evidenty, increase in bit depth increases the quaity of image captured and the space required to store the resutant image. Generay speaking, 12 bits per pixe (4 bits per primary coour) is considered minimum pixe depth for good quaity coour image. Most of today s coour scanners can scan at 24-bit coour (8 bit per primary coour). Tabe 7.1: No. of Bits Used for Representing Shades in Coour and Gray-scae Scanning S. No. No. of Bits No. of bits/shades No. of shades No. of Shades/pixe = = = = = = = = = = = = = =

12 Sef Check Exercise 4) Why is Bit depth not important for bitona scanning? Digitisation: Concept, Need, Methods and Equipment Note: i) ii) Write your answer in the space given beow. Check your answer with the answers given at the end of the Unit Resoution The resoution of an image is defined in terms of number of pixes (picture eements) in a given area. It is measured in terms of dots per inch (dpi) in case of an image fie and as ratio of number of pixes on horizonta ine x number of pixe in vertica ines in case of dispay resoution on a monitor. The higher the dpi set on the scanner, the better the resoution and quaity of image and arger the image fie. Regardess of the resoution, the quaity of an image can be improved by capturing an image in grayscae. The additiona gray-scae data can be processed eectronicay to sharpen edges, fie-in characters, remove extraneous dirt, remove unwanted page strains or discooration, so as to create a much higher quaity image than possibe with binary scanning aone. A major drawback in gray scae is that there is arge amount of data capture. It may be noted that continuing increase in resoution wi not resut in any appreciabe gain in image quaity after some time, except for increase in fie size. It is thus important to determine the point where sufficient resoution has been used to capture a the significant detais present in the source document. The back and white or bitona images (textua) are scanned most commony at 300 dpi that preserve 99.9% of the information content of a page and can be considered as adequate access resoution. Some preservation projects scan at 600 dpi for better quaity. A standard SVGA/VGA monitor has a resoution of ines whie the utra-high monitors have a resoution of about (about 150 dpi). Fig. 7.5: Setting-up Resoution Manuay 163

13 Sef Check Exercise 5) What is resoution? Is print resoution different from scanning resoution? Note: i) ii) Write your answer in the space given beow. Check your answer with the answers given at the end of the Unit Threshod The threshod setting in bitona scanning defines the point on a scae, usuay ranging from 0-255, at which gray vaues wi be interpreted as back or white pixes. In bitona scanning, resoution and threshod are the key determinants of image quaity. Bitona scanning is best suited to high-contrast documents, such as text and ine drawings. Gray scae or coour scanning is required for continuous tone or ow contrast for documents such as photographs. In gray scae/coour scanning both resoution and bit depth combine to pay significant roe in image quaity. In ine art mode, every pixe has ony two possibe vaues. Every pixe wi either be back or white. The Line art Threshod contro determines the decision point about brightness determining if the samped vaue wi be a back dot or a white dot. The norma threshod defaut is 128 (the midrange of the 8-bit range). Image intensity vaues above the threshod are white pixes, and vaues beow the threshod are back pixes. Adjusting threshod is ike a brightness setting to determine what is back and what is white. Threshod for text printed on a cooured background or cheap-quaity paper ike newsprint has to be kept at ower range. Reducing threshod from 128 to about 85 woud greaty improve the quaity of scan. Such adjustments woud aso improve the performance of OCR software. Threshod Setting = 128 Threshod Setting = Fig. 7.6: Threshod Setting in Bitona Scanning Image Enhancement Image enhancement process can be used to improve scanned images at a cost of image authenticity and fideity. The process of image enhancement is, however, time consuming; it requires specia skis and woud invariaby increase the cost of conversion. Typica image enhancement features avaiabe in a scanning or image editing software incude fiters, tona reproduction, curves and coour management, touch, crop, image sharpening, contrast, transparent background, etc. In a page scanned in gray-scae, the text /ine art and haf tone areas can be decomposed and each area of the page can be fitered separatey

14 to maximise its quaity. The text area on page can be treated with edge sharpening fiters, so as to ceary define the character edges, a second fiter coud be used to remove the high-frequency noise and finay another fiter coud fi-in broken characters. Gray-scae area of the page coud be processed with different fiters to maximise the quaity of the haftone. Digitisation: Concept, Need, Methods and Equipment 7.7 COMPRESSION Fig. 7.7: Sharpening Image using HP Precisionscan Pro Image fies are evidenty arger than textua ASCII fies. It is thus necessary to compress image fies so as to achieve economic storage, processing and transmission over a network. A back & white image of a page of text scanned at 300 dpi is about 1 mb in size whereas a text fie containing the same information is about 2-3 kb. Image compression is the process of reducing size of an image by abbreviating the repetitive information such as one or more rows of white bits to a singe code. The compression agorithms may be grouped into the foowing two categories: Lossess Compression The conversion process converts repeated information as a mathematica agorithm that can be decompressed without oosing any detais into the origina image with absoute fideity. No information is ost or sacrificed in the process of compression. Lossess compression is primariy used in bitona images Lossy Compression Lossy compression process discards or minimises detais that are east significant or which may not make appreciabe effect on the quaity of image. This kind of compression is caed ossy because when the image that is compressed using ossy compression techniques is decompressed, it wi not be an exact repica of the origina image. Lossy compression is used with gray-scae/coour scanning. Compression is a necessity in digita imaging but more important is the abiity to output or produce the uncompressed true repica of images. This is especiay important when images are transferred from one patform to another or are handed by software packages under different operating systems. Uncompressed images often work better than compressed images for different reasons. It is thus suggested that scanned images shoud be either stored as uncompressed images or at the most as ossess compressed images. Further, it is optima to use one of the standard and widey supported compression protocos than a proprietary one, even if it offers efficient compression and better quaity. Attributes of origina documents may aso be considered whie seecting compression techniques. For exampe ITU G-4 is designed to compress text where as JPEG, GIF and ImagePAC are designed to compress pictures. It is important to ensure migration of images from one patform to another and from one hardware medium to another. It may be noted that highy compressed fies are more prone to corruption than uncompressed fies. 165

15 Sef Check Exercise 6) What is image compression? Describe the types of image compression. Note : i) ii) Write your answer in the space given beow. Check your answer with the answers given at the end of the Unit Compression Protocos The foowing protocos are commony used for bitona, gray scae or coour compression: TIFF-G4 Internationa Teecommunication Union (ITU Group 4) is considered as de facto standard compression scheme for back and white or bitona images. An image created as a TIFF and compressed using ITU-G4 compression technique is caed a Group-4 TIFF or TIFF- G4 and is considered as defacto standard for storing bitona images. TIFF- G-4 is a ossess compression scheme. Joint Bi-eve Image Group (JBIG) (ISO-11544) is another standard compression technique for bitona images. JPEG (Joint Photographic Expert Group) JPEG (Joint Photographic Expert Group) is an ISO I compression protoco that works by finding areas of the image that have same tone, shade, coour or other characteristics and represents this area by a code. Compression is achieved at oss of data. Preiminary testing indicates that a compression of about 10 or 15 to one can be achieved without visibe degradation of image quaity. LZW (Lenpe-Ziv Wech) LZW compression technique uses a tabe-based ookup agorithm invented by Abraham Lempe, Jacob Ziv, and Terry Wech. Two commony-used fie formats in which LZW compression is used are the Graphics Interchange Format (GIF) and Tag Image Fie Format (TIFF). LZW compression is aso suitabe for compressing text fies. A particuar LZW compression agorithm takes each input sequence of binary digit of a given ength (for exampe, 12 bits) and creates an entry in a tabe (sometimes caed a dictionary or codebook ) for that particuar bit pattern, consisting of the pattern itsef and a shorter code. As input is read, any pattern that has been read before the resuts in the substitution of the shorter code effectivey compresses the tota amount of input to something smaer. The decoding program that uncompresses the fie is abe to buid the tabe itsef by using the agorithm as it processes the encoded input. Sef Check Exercise 166 7) How does LZW compression protoco work? Which fie formats use LZW compression protoco?

16 Note : i) ii) Write your answer in the space given beow. Check your answer with the answers given at the end of the Unit. Digitisation: Concept, Need, Methods and Equipment OCR (Optica Character Recognition) OCR (Optica Character Recognition) programs are software toos used to transform scanned textua page images into word processing fie. OCR or text recognition is the process of eectronicay identifying text in a bit-mapped page image or set of images and generate a fie containing that text in ASCII code or in a specified word processing format eaving the image intact in the process. The OCR is performed in order to make every word in a scanned document readabe and fuy searchabe without having to keyin everything in the computer manuay. Once a bit-mapped page image has gone through the process of OCR, a document can be manipuated and managed by its contents, i.e., using the words avaiabe in the text. OCR does not actuay convert an image into text but rather creates a separate fie containing the text whie eaving the image intact. There are four types of OCR technoogy that are prevaiing in the market. These technoogies are: matrix matching, feature extraction, structura anaysis and neura network. i) Matrix / Tempate Matching: Compares each character with a tempate of the same character. Such a system is usuay imited to a specific number of fonts, or must be taught to recognise a particuar font. ii) iii) iv) Feature Extraction: Can recognise a character from its structure and shape (anges, points, breaks, etc.) based on a set of rues. The process caims to recognise a fonts. Structura anaysis: Determines characters on the basis of density gradations or character darkness. Neura Networking: Neura networking is a form of artificia inteigence that attempts to mimic processes of the human mind. Combined with traditiona OCR techniques pus pattern recognition, a neura network-based system can perform text recognition and earn from its success and faiure. Referred to as Inteigent Character Recognition, a neura network-based system is being used to recognize hand-written text as we as other traditionay difficut source materia. Neura network ICR can contempate characters in the context of an entire word. Newer ICR combines neura networking with fuzzy ogic. The image scanner opticay captures text images to be recognised. Text images are processed with OCR software and hardware. The process invoves three operations: document anaysis (extracting individua character images), recognising these images based on (i) their tempate stored in the OCR database; (ii) structure and shape (anges, points, breaks, etc.) (iii) density gradations or character darkness and (iv) contextua processing. The output interface is responsibe for communication of OCR system that resuts, to the outside word. 167

17 Fig. 7.8: OCR Technoogy Severa software packages now offer faciity of retaining the page ayout after it has been OCRed. The process for retaining the page ayout is software dependent. Caere s Omnipro offers two ways of retaining page ayout foowing OCR. It cas them True Page Cassic and True Page Easy. True Page Cassic paces each paragraph within a separate frame of a word processor into which the OCR output is saved. If one wishes to edit anything subsequenty, then the reevant paragraph box may need to be resized. However, Easy Edit faciitates editing of pages without the necessity of resizing the boxes athough there are greater chances of spiage over the page. Xerox Text Bridge offers simiar feature caed DocuRT which is broady equivaent to True Page Easy edit. The process of OCR dismantes the page, OCRs it, and then reassembes it in such a way that a the component parts such as tabs, coumns, tabe, graphics can be used in a text manipuation package such as word processor. There is a itte doubt about the fact that OCR is ess accurate than rekeying-in the data. At an accuracy ratio of 98%, a page having 1800 characters wi have 36 errors per page on an average. It is therefore, imperative to ceanup after OCR uness origina scanned image wi be viewed as a page and OCR is being used purey to create a searchabe index on the words that wi be searched via a fuzzy retrieva engine ike Excaibur, which is highy toerant to OCR errors. 168 Another possibiity for ceaned-up OCR is use of a speciaist OCR system such as, Prime Recognition. With production OCR in mind, Prime OCR icenses eading to recognising engine and passes the data through severa of them using voting technoogy aong with artificia inteigence agorithms. Athough it takes onger initiay, but saves time in the ong run and Prime contends that it improves the resut achieved by a singe engine by %. The technoogy is avaiabe at a price depending upon the number of search engines that one woud ike to incorporate. Michigan Digita Library production services used Prime OCR for pacing more than two miion pages of SGML encoded text and the same number of page images on the web.

18 Sef Check Exercise 8) What is OCR? Why is it important to OCR a digitised image? Digitisation: Concept, Need, Methods and Equipment Note : i) ii) Write your answer in the space given beow. Check your answer with the answers given at the end of the Unit FILE FORMATS AND MEDIA TYPES A defined arrangement for discrete sets of data that aow a computer and software to interpret the data is caed a fie format. Different fie formats are used to store different media types ike text, images, graphics, pictures, musica works, computer programs, databases, modes and designs, video programs and compound works combining many types of information. Athough, amost every type of information can be represented in digita form, a few important fie formats for text and images typicay appicabe to a ibrary-based digita coections are described here. However, every object in a digita ibrary needs to have a name or identifier which distincty identifies its type and format. This is achieved by assigning fie extensions to the digita objects. The fie extensions in a digita ibrary typicay denote formats, protocos and rights management that are appropriate for the type of materia. Names of fie formats appicabe in digita ibrary and their fie extensions are given in Tabe 7.2. Sef Check Exercise 9) What are fie formats? How is unstructured text different from structured text? Note : i) ii) Write your answer in the space given beow. Check your answer with the answers given at the end of the Unit Formats and Encoding Used for Text Text and image-based contents of a digita ibrary can be stored and presented as (i) simpe text or ASCII (American Standard Code for Information Interchange); (ii) unstructured text; (iii) Structured text (SGML or HTML or XML); (iv) page description anguage and (v) page image formats. Simpe Text or ASCII Simpe text or ASCII (American Standard Code for Information Exchange) is the most commony used encoding scheme used for faciitating exchange of data from one software 169

19 to another or from one patform to another. Fu-text of artices from many journas are avaiabe eectronicay through onine vendors ike DIALOG and STN in this format since more than two decades. Simpe text or ASCII is compact, economic to capture and store, searchabe, inter-operabe and is maeabe with other text-based services. On the other hand, the simpe text or ASCII cannot be used for dispaying compex tabes or mathematica formuas. Photographs, diagrams, graphics, specia characters cannot be dispayed in ASCII. ASCII format does not store text formatting information, i.e., itaics, bod, font type, font size or paragraph justification information. Simpe text or ASCII in many ways is inadequate to represent many journa artices because of the reasons mentioned above. Athough simpe text or ASCII is extremey usefu for searching and seection, its inabiity to capture the richness of the origina makes it an interim step to structured text formats. Structured Text Format Structured text format attempts to capture the essence of documents by marking-up the text so that the origina form coud be recreated or even produce other forms such as ASCII. Structured text formats have provision for imbed images, graphics and other mutimedia formats in the text. SGML (Standard Generaized Markup Language) is one of the most important and popuar structured text format. ODA (Office Document Architecture) is a simiar and competing standard. SGML is an internationa standard (ISO, 1986) around which severa reated standards are buit. SGML is a fexibe anguage that gave birth to HTML (Hyper-Text Markup Language), de facto markup anguage of the Word Wide Web, to contro the dispay format of documents and even the appearance of the user interface for interacting with the documents. Like simpe text or ASCII, structured text can be searched or manipuated. It is highy fexibe and suitabe both for eectronic and paper production. We-formated text increases visua presentation voume of textua, graphica and pictoria information. Structured formats can easiy dispay compex tabes and equations. Moreover, structured text is compact in comparison to the imagebased formats, even after incuding imbedded graphics and pictures. Creation of structured text, if rekeyed, is aways too expensive on a production basis. However, creation of structured text is generay integrated with the production of printed artifacts. SGML is in fact, a format generated as a by-product of printed artifacts generated eectronicay. Besides SGML and HTML, there are other formats used in digita ibrary impementation. TeX, used for formatting highy mathematica text is one such format which aows greater contro over the resuting dispay of document, incuding reviewing the formatting of errors. Page Description Language (PDL) Page Description Languages (PDLs), such as Adobe s PostScript and PDF (Portabe Document Format) are simiar to image but the formatted pages dispayed to the user are text-based rather than image-based. PostScript and PDF formats can easiy be captured during the typesetting process. PostScript is especiay easy to capture since most of the systems automaticay generate it and conversion program, caed Acrobat Distier, can be used to convert PostScript fie into PDF fies. The documents stored as PDF require Acrobat Reader at the user s end to read or print the document. The Acrobat Reader can be downoaded free of cost from the Adobe s Web Site. Acrobat s Portabe Document Format (PDF) is a by-product of PostScript. Adobe s pagedescription anguage had become the standard way to describe pages eectronicay in the graphics word. Whie PostScript is a programming anguage, PDF is a page-description format. 170 PDF can have two formats: (i) Text-based PDF that uses outine font technoogy of PostScript PDL (Page Description Language) from Adobe to describe format of a page; (ii) raster-scanned image PDF without the text output of OCR (Optica Character

20 Recognition). The image PDF is essentiay equivaent to TIFF or CCITT G4 formats or to a photograph where text characters cannot be manipuated by the computer. Besides, an image-based PDF may be converted into text-based PDF once it goes through the process of OCR. In this process, scanned image is repaced by the text with fonts and ayout matching with the scanned document. Digitisation: Concept, Need, Methods and Equipment Sef Check Exercise 10) How is Page Description Language (PDL) different from structured text? Note : i) ii) Write your answer in the space given beow. Check your answer with the answers given at the end of the Unit. Page Image Format The digitay scanned images are stored in a fie as a bit-mapped page image, irrespective of the fact that a scanned page contains a photograph, a ine drawing or text. The bitmapped page image can be created in dozens of different formats depending upon the scanner and its software. Nationa and internationa standards for image-fie formats and compression methods exist to ensure that data wi be interchangeabe amongst systems. An image fie stores discrete sets of data and information aowing a computing system to dispay, interpret and print the image in a pre-defined fashion. An image fie format consists of three distinct components, i.e., header which stores information on fie identifier and image specifications; Image data consisting of ook-up tabe and image raster and asty, footer that signas fie termination information. Whie bit-mapped portion of a raster image is standardised, it is the fie header that differentiates one format from another. TIFF (Tagged Image Fie Format) is the most commony used page image fie format and is considered to be the de facto standard for bitona images. Some image formats are proprieary deveoped by commercia vendors and require specific software or hardware for dispay and printing. Images can be cooured, grey-scae or back and white (caed bitona). They can be uncompressed (raw) or compressed using severa different compression agorithms. Tabe 7.2: Fie Formats Used in a Digita Library Abbreviation Format Fie Extension Fie Format for Unstructured Text ASCII American Standard Code for Information Interchange.txt Fie Format for Structured Text SGML Standard Generaized Markup Language.sgm HTML Hypertext Markup Language.htm XML Extended Markup Language.xm PDF Portabe Document Format (Adobe).pdf PostScript PostScript (Adobe).ps TEX Texture Format.txt 171

21 Abbreviation Format Fie Extension Fie Format for Images PDF Portabe Document Format.pdf BMP Bit Map Page (Windows).bmp IMG Ventura Pubisher.img JPEG Joint Photographic Expert Group.mpg JFIF JPEG Fie Format.jfif PCP PC Paint (B&W Mode).pcp PCX PC Paint Brush (Coor & B&W).pcx PSD Photoshop.psd TGA True Vision Targa.tga PNG Portabe Network Graphic.png TIFF Taged Image Fie Format.tif TIFF-G4 Taged Image Fie Format with Group 4 Fax Compression.tif SPIFF Sti Picture Interchange Fie Format.spf PCD Photo CD (Kodak).pcd Audio and Video Fie Format WAVE Waveform Audio (Microsoft).wav AIFF Audio Interchange Format.aif VoC Creative Voice.voc MIDI Musica Instrument Digita Interface.midi SND Sound.snd AU Audio (Sun Microsystems).au RAF Rea Audio Format (Progressive Networks).ra AVI Audio Visua Intereave.avi FLA Macromedia Fash Movie.fa FLC AutoDesk FLIC Animation.fc MOV Quicktime for Windows Movie.mov MPEG Motion Picture Expert Group.mpg MP2 MPEG Audio Layer 2.mp2 MP3 MPEG Audio Layer 3.mp3 7.9 TOOLS OF DIGITISATION 172 Digita imaging is an inter-inked system of hardware, software, image database and access sub-system with each having their own components. Toos used for digitisation incude severa core and periphera systems. An image scanning system may consist of a stand-aone workstation where most or a the work is done on the same workstation or as a part of a network of workstations with imaging work distributed and shared amongst various workstations. The network usuay incudes a scanning station, a server and one or more editing, retrieva stations. A typica scanning workstation for a sma, productioneve project coud consist of the foowing: Hardware (Scanners, computers, data storage and data output peripheras) Software (image capturing and image editing) Network (data transmission) Dispay and Printing technoogies This Unit concentrates on scanners and scanning software as important components of the scanning system.

22 7.9.1 Scanners Digita scanners are used to capture digita images from anaogue media such as printed pages or a microfiche / microfim at a predefined resoution and dynamic range (bit range). There are two types of image scanners: vector scanner and raster scanners. The vector scanners scan an image as a compex set of x,y coordinates. Vector images are generay used in Geographica Information Systems (GIS). The dispay software for the vector image interprets the image as function of coordinates and other incuded information to produce an eectronic repica of the origina drawing or photograph. Vector images can be zoomed in portion to dispay minute detais of a drawing or a map. Maps, engineering drawings, and architectura bueprints are often scanned as vector images. Raster images are captured by raster scanners by passing ights (aser in some cases) down the page and digitay encoding it row by row. Mutipe passes of ights may be required to capture basic (as a set of bits known as bit map) coours in a cooured image. Raster scanners are used in ibraries to convert printed pubications into eectronic forms. Majority of eectronic imaging systems generate raster images. The scanners used for digitizing anaogue images into digita images come in a variety of shapes and sizes. Digitisation: Concept, Need, Methods and Equipment How Scanner Works? Scanners are equipped with a amp that moves with the scanner head to ight-up the object being scanned. Most scanners use a cod cathode forescent amp or a xenon amp. The scan head is made up of the mirrors, ens, fiter, and charge-couped devices (CCD) array. A bet that is connected to the stepper motor makes the scan head move. A stabiizing bar prevents wobbing during the pass. The mirrors refect what is being scanned into the ens and the image is then focused through a fiter on the CCD array. Three smaer images of the origina are made by the ens. These images then go through a coor fiter and onto a section of the CCD array. The data is then combined into a singe image. Whie seecting a scanner, one shoud consider resoution, sharpness, and rate of image transfer. The resoution is measured in dots per inch (dpi). The average scanner has at east 300x300 dpi. The number of sensors in a row of the CCD array determines a scanner s dpi. Sharpness depends on how bright the amp is and the quaity of the ens. Image transfer depends on the connection used to connect the scanner to the computer. The sowest is the parae port. Universa Seria Bus or USB scanners are affordabe, easy to use, and have good speed. The hardware required for a scanner is a connector such as a USB. The software required is a driver. The driver is needed to communicate with the scanner. TWAIN is the anguage spoken by scanners. Any program that supports TWAIN can acquire a scanned image. There are foowing types of Scanners: Fatbed Scanners right ange, prism and overhead fatbed Sheet-Feed Scanners Drum Scanners Digita Cameras Side Scanners Microfim Scanners Video Frame Grabbers Hand-hed scanners The type of scanner seected for an imaging project woud be infuenced by the type, size and source of documents to be scanned. Many scanners can hande ony transparent materia, whereas others can hande ony refective materias. 173

23 Fatbed Scanners: Fatbed scanners are most common, and widey used scanners that ook ike a photocopier and are used in much the same way. Source materia in a fatbed scanner is paced face down for scanning. The ight source and charge-couped devices (CCDs) move beneath the paten, whie the document remains stationary as in the case of photocopying machine. Fatbed scanner comes in various modes ike right-ange, prism and panetary/overhead to hande bound voumes and books. Fatbed scanner can scan usuay a document at 600 dpi. Many fatbed scanners however, offer higher resoution. Fig. 7.9: Fatbed Scanner Sheet feed Scanners: In a sheet-feed scanner, as is indicated in the name, document is fed over a stationary CCD and ight source via roer, bet, drum, or vacuum transport. In contrast to a fat-bed scanner, sheet-feed scanner has optiona attachment of auto- feed uniform-sized stack of documents to be scanned. Fig. 7.10: Sheet feed Scanners Drum Scanners: Source materia in a drum scanner is wrapped on a drum, which is then rotated past a high-intensity ight source to capture the image. Drum scanners offer superior image quaity, but require fexibe source materia of imited size that can be wrapped around the photosensitive drum. Drum scanners are speciay targeted for graphic arts market. Drum scanners offer highest resoution for grey scae and coour scanning. Drum Scanner uses Photo-Mutipier (Vacuum) Tubes (PMTs) instead of CCDs, which offer a greater bit depth (12 to 16 bits). 174 Fig. 7.11: Drum Scanners

24 Digita Cameras: Digita cameras mounted on copy crade resembe microfiming stand. Source materia is paced on the stand and the camera is cranked up or down in order to focus the materia within the fied of view. Digita cameras are most promising scanner deveopment for ibrary and archiva appications. Digitisation: Concept, Need, Methods and Equipment Fig. 7.12: Digita Camera Side Scanner: Side scanners have a sot in the side to accommodate a 35mm side. Inside the box, the ight passes through the side to hit a CCD array behind the side. Side scanners can generay scan ony 35mm transparent source materias. Fig. 7.13: Drum Scanner Microfim Scanner: Speciay targeted to ibrary/archiva appication, microfim scanners have adapters to convert ro fim, fiche, and aperture cards into the same mode. Fig. 7.14: Microfim Scanner Video Frame Grabber or VideoDigitiser: Video digitisers are circuit boards paced inside a computer and attached to a standard video camera. Any thing that is fimed by the video camera is digitised by the video digitiser. 175

25 Fig. 7.15: Video Graber Hand-hed Scanners Hand-hed scanners are used for scanning seective sections of data. It may require mutipe pass to capture arge area. Moreover, a user shoud have a steady hand whie moving the scanner over the document to be scanned. These scanners are normay used for circuation work in a ibrary Scanning Software The scanning software is used for scanning the image and capturing it in the computer. This software is provided by the manufacturer of the product to the buyers. These drivers transate the instructions into commands, which the scanner understands. Image Editing Appications Image editing appications are used once the process of scanning the image is over and the image is avaiabe in the computer for further manipuation. Most image editing software offer features ike image editing, sharpening, fiter, cropping, coour adjustments, forms conversion, resising, etc. Most image editing software can aso be used for capturing the images DIGITISATION OF AUDIO AND VIDEO The songs or speeches that we generay isten from tape recorders or radio are in an anaogue form. The anaogue sound tracks can be digitized by attaching an audio payer to a system through an audio capture card so as to record the sound to the system. The audio fies can be saved as wav, mp3, midi, etc. MP3 format is highy compact and the sound quaity is better in comparison to other formats. Audio fies can be further processed using noise reduction software. Like audio, video capture aso requires a video capture card with input from video cassette payer (VCP / VCR), TV antenna, cabe or movie camera. The digitised fies can be saved as mov, avi, mpg fie formats ORGANISING DIGITAL IMAGES 176 A disc fu of digita images without any organisation, browse and search options may have no meaning except for one who created it. Scanned images need to be organised in order to be usefu. Moreover, images need to be inked to the associated metadata to