Category: Data/Information Keywords: Records Management, Digitization, Imaging, Image capture, Scanning and Indexing

Transcription

1 IMT Standards IMT Standards Oversight Committee Government of Alberta Effective Date: Scheduled Review: Last Reviewed: Type: Technical Standard number A Digitization Technical Requirements Category: Data/Information Keywords: Records Management, Digitization, Imaging, Image capture, Scanning and Indexing Description of Standard This standard describes the technical requirements that ministries should implement for all digitization projects and must implement when the goal of the digitization project is to create digital business records: Whose authenticity and reliability are demonstrable, That are legally admissible in place of the Source Records, That are in compliance with CGSB (Canadian General Standards Board) , Microfilm and Electronic Images as Documentary Evidence, and That are available to be produced at all times. Standard Specification Terms Batch: A grouping of records by a common factor such as a period of time or an alphabetic or numeric segment. Source: Government-Wide Records Glossary of Terms Bibliographic Information: Information regarding the content and context of a document. It is created by the organization (possibly obtained from the Source Record) and aids in the retrieval of an image. Source: CAN/CGSB Microfilm and Electronic Images as Documentary Evidence Biographic Information: Information regarding image capture that may include the date captured, the time, the operator identification, the capture device identification and location and details of modification, if any. Source: CAN/CGSB Microfilm and Electronic Images as Documentary Evidence Bitonal: Having picture elements (pixels) that are only one bit. A Bitonal image has two intensity values (0 and 1), corresponding to black and white. Source: ANSI/AIIM TR Glossary of Document Technologies INFORMATION SENSITIVITY: Unrestricted Page 1 of 8

2 Compression: A technique used to reduce the number of bits in an electronic image file. For digital images, Compression may be lossy or lossless. Lossless Compression is preferred for archival purposes, as the decompressed file is identical to the original uncompressed image. Lossy images may not be identical to the original uncompressed image and information may be lost during the Compression process. Source: ISO :2012 Electronic imaging - Vocabulary De-Speckle: The process of removing specks or marks from images, especially those created in the scanning process. Source: McGraw-Hill Dictionary of Scientific & Technical Terms, 6E, The McGraw-Hill Companies, Inc. (2003) De-Skew: The process of realigning tilted images. Skew is an artifact that can occur in scanned images because of the camera or document being misaligned, imperfections in the scanning surface, or because the paper was not completely flat when scanned. Source: McGraw-Hill Dictionary of Scientific & Technical Terms, 6E, The McGraw-Hill Companies, Inc. (2003) Digitization: The use of a scanner or similar equipment to convert documents to digitally coded electronic images suitable for electronic storage. Source: ANSI/AIIM TR Glossary of Document Technologies Dots Per Inch (DPI): A measure of output device resolution and quality. Measures the number of dots horizontally and vertically. Source: ANSI/AIIM TR Glossary of Document Technologies Greyscale: The capability to display varying levels of grey and usually represented as a number such as 16 levels of grey. The grey levels are created by varying the strength of the electron beam - the higher the level of Greyscale, the smoother the transition from light to dark. Source: ANSI/AIIM TR Glossary of Document Technologies Indexing: Identification of specific attributes of a document to facilitate retrieval. Source: ANSI/AIIM TR Glossary of Document Technologies Metadata: Structured information about data. Metadata describes, defines, explains, locates, and otherwise makes it easier to retrieve and use an information resource or data asset. Essentially, Metadata is anything that might influence the way in which the data or information is used. Source: Understanding Metadata National Information Standards Organization (NISO) Press (2004) Microform: A generic term for any form, usually film, which contains microimages. Microform images are commonly reduced about 25 times from the original document size. For special purposes, greater optical reductions may be used. Source: ANSI/AIIM TR Glossary of Document Technologies PDF or Portable Document Format: A file format which captures formatting information from a variety of applications and makes it possible to transmit and display documents in an identical way, independent of the platform. Source: ANSI/AIIM TR Glossary of Document Technologies INFORMATION SENSITIVITY: Unrestricted Page 2 of 8

3 Portable Document Format/Archival (PDF/A): A file format based on a subset of the Adobe PDF format that is optimized for the long-term archiving of electronic documents. Source: ISO :2005 Document management Electronic document file format for long-term preservation Part 1: Use of PDF 1.4 (PDF/A-1) Pixel Density or Pixels Per Inch (PPI): The measurement of how a monitor displays an image. Sometimes referred to as display resolution, it refers to the number of picture elements (pixels) per inch (Source: ISO :2012, Electronic Imaging Vocabulary). For example, a monitor that is 16 inches wide and displays 1600 pixels across its width would have a resolution of 100 PPI (1600 divided by 16). Quality Assurance (QA): Similar to Quality Control (QC), but with the program independent of the production organization (Source: ANSI/AIIM TR Glossary of Document Technologies). In the case of GoA Digitization projects, it refers to the process of reviewing images (the content) and associated data (Metadata, etc.). QA should be conducted by personnel not directly involved in the Digitization or Indexing of documents and may be conducted by an independent third party. This level of quality review is above the normal QC deployed to ensure that procedures met requirements. Quality Control (QC): The review of the processes being used, to ensure that they are meeting the project s requirements. It is defined as procedures, plans or actions used to ensure processes conform with established requirements, comply with standards and maintain confidence in integrity and accuracy. Source: ANSI/AIIM TR Glossary of Document Technologies Raster data: Data specifying the values of pixels in an image formed by a set of pixels arranged in a grid pattern, often called a bitmap. Source: ANSI/AIIM TR Glossary of Document Technologies Rasterization: 1) Process of converting text or images from an original source document into raster data via scanning. 2) Conversion of information stored as a series of vectors or other codes into raster data for display or printing. Source: ANSI/AIIM TR Glossary of Document Technologies Secure Storage: Storage repositories of quality assured digital records that satisfy all relevant control objectives. Source: CAN/CGSB Microfilm and Electronic Images as Documentary Evidence Source Record: The record itself or any facsimile intended by the author of the record to have the same effect. Source: CAN/CGSB Microfilm and Electronic Images as Documentary Evidence TIFF or Tagged Image File Format: Describes image data that typically comes from scanners, frame grabbers and paint- or photo-retouching programs. The purpose of TIFF is to describe and store raster image data. Source: TIFF 6.0 Specification. TIFF Revision 6.0: Final June 3, 1992 (Retrieved February 2015) INFORMATION SENSITIVITY: Unrestricted Page 3 of 8

4 Minimum Digitization Requirements The hardware chosen for a Digitization project must be maintained according to a regular maintenance program that is documented and followed. Scanner equipment must meet the specifications required by the project. The technical specifications of acceptable formats are found in the table below. All identified classes of Source Records must be scanned at a minimum of 300 PPI, and all classes must be output to TIFF, PDF or PDF/A format. JPEG format may only be used for black and white or colour photographs. Class Tonality Bits Compression Black and white Bitonal 1 Lossless documents or Microform Colour documents Bitonal or Colour 24 Lossy acceptable, Lossless preferred Black and white Greyscale 8 Lossless photographs Colour photographs Bitonal or Colour 24 Lossy acceptable, Lossless preferred The final format in terms of number of pages can be either single-page or multi-page. The determination of which pages are to be captured as a single record should be made based on business requirements. Note that for litigation purposes the Department of Justice and Solicitor General requires single-page TIFF with a resolution of 300 DPI (refer to Civil Practice Note 4 - Guidelines for the use of Technology in any Civil Litigation Matter for more details). Quality Control of Source Documents In order for the documents to proceed through an appropriate and auditable imaging procedure, the items should undergo Quality Control checks in the following areas: document selection, document preparation, and quality of the scanned/re-scanned product. Quality checking must include ensuring the readability of the output images and that the number of physical items scanned is equal to the number of images produced. This process may also include De-skewing and removing of blank pages. Automatic Despeckle and other enhancements should not be used. Ministries should consider PDF Optimization as part of their Digitization program due to the unnecessary file size overhead associated with unused layers in a PDF file. Layers can be set to 0 without any irreversible affect to the future usability of the file. In addition, white space optimization and various combinations of Rasterization and image formats can be utilized to greatly reduce the size of a PDF without altering the compliance of the image. INFORMATION SENSITIVITY: Unrestricted Page 4 of 8

5 Project specific standards for the images created must be clearly defined and documented prior to the commencement of the Digitization so that image quality can be evaluated. Minimum Indexing Requirements Indexing describes business objects or resources and must be retained for at least as long as the records to which they relate. The Digitization process includes four phases where Indexing must be applied. These phases are: Image capture (scanning) Image re-capture (re-scanning) Quality Assurance Data Transfer The re-capture stage is required if the images and associated Indexing do not pass the Quality Control or Quality Assurance checks and must be re-captured. There are two types of Indexing information that are pertinent to these specifications: Biographical and Bibliographical. Biographical information deals with the lifecycle of the image file, and relates to the context of the image and file properties that must be captured, logged, and certified during the Digitization process. The values surrounding obligation to include these fields are as follows: Mandatory: The property must be provided Mandatory if applicable: The property must be provided if appropriate for the business context and/or the resources (business objects) Recommended: The property should be provided if appropriate for the business context and/or the resources (business objects) Optional: The property may be provided to add Metadata of value based on business requirements Biographical Indexing Specifications: Obligation Digitization Process Digitization Indexing Properties Mandatory Capture (scan) Image reference Digitization date and time (Note: time is recommended, but not required) Number of pages digitized Digitization equipment operator and device name Cross referencing information about the image record Mandatory Re-capture (re-scan) (If Image reference process is required) Digitization date and time (Note: time is recommended, but not required) Number of pages digitized Digitization equipment operator and INFORMATION SENSITIVITY: Unrestricted Page 5 of 8

6 Obligation Digitization Process Digitization Indexing Properties device name Cross referencing information about Mandatory Mandatory If applicable the image record Quality Assurance Batch reference (Mandatory for Batch (When QA is done) input) Quality Assurance operator Quality Assurance check approval date Data Transfer Transfer date Transfer title Transfer description Transfer reason Transfer receiving Some or all of this information may be captured automatically by the Digitization software. In the case of transfer to Secure Storage, it is important to know the date of transfer to establish whether any delay was reasonable. Organizations should implement procedures to prevent any form of electronic modification after the image has been appropriately captured and indexed. Bibliographical Indexing Specifications: Bibliographical information relates to the content and context of the record. The decisions to capture information should be made at the beginning of a project. This information must be captured and then associated with the image, preferably by automated means or by manual data entry (double entry verification required, whereby items are entered twice to ensure accuracy) during the Digitization process. The following Bibliographical information terms are the Indexing requirements as per the GoA Metadata Core Standard: Obligation Digitization Indexing Properties Creator Date Created Mandatory Date Modified Title Alternative title Description Security Classification Mandatory if Format applicable Language Medium Identifier Subject Recommended Usage considerations Optional Extent INFORMATION SENSITIVITY: Unrestricted Page 6 of 8

7 The GoA Metadata Core Content Standard and Indexing Field Descriptions support document describes all the terms listed in this standard. Where differences exist between the Metadata Standard, support documents and this standard, the Metadata Standard shall supersede and prevail. Minimum Documentation Requirements A procedures manual must be written by each ministry describing the Digitization program. For further information on documentation, refer to the Digitization Process Standard (A000015) and supporting documentation. Where to Apply This Standard This applies to all Government of Alberta ministries, boards, agencies and commissions. Authority and Exceptions Verify individual Ministry Acts to determine any specific exclusions This standard applies to all records, as described in the Description of Standard section Records excluded from the Electronic Transactions Act, S.A. 2011, c. E-5.5, section 7(1) and (2) may be digitized, but legal recognition of the digital version is not guaranteed, as per the Act. These records include: o Wills and codicils o Trusts created by wills or codicils o Enduring powers of attorney under the Powers of Attorney Act o Personal directives under the Personal Directives Act o Records that create or transfer interests in land, including interests in mines and minerals o Guarantees under the Guarantees Acknowledgment Act o Negotiable instruments o Records that are prescribed, or that belong to a class that is prescribed, as records or a class of records to which the Act does not apply o Documents of title, with some exceptions (refer to the Act for details) Supporting Documentation Digitization Process Standard A Government of Alberta IMT Standards. Metadata - Core Content Standard A Government of Alberta IMT Standards. Under Review. Digitization Chain of Custody (RIM0001) Digitization Document Preparation (RIM0002) Digitization Program Required Documentation (RIM0003) Digitization Quality Assurance (RIM0004) Digitization Recommended Practice (RIM0005) Digitization Frequently Asked Questions (RIM0006) Digitization Indexing Field Descriptions (RIM0007) Digitization Program Business Analysis (RIM0008) Digitization Procedures Manual Guide References A-Z guide to personal technology, Houghton Mifflin (2006) INFORMATION SENSITIVITY: Unrestricted Page 7 of 8

8 ANSI/AIIM TR2-1998, Glossary of Document Technologies CAN/CGSB , Microfilm and Electronic Images as Documentary Evidence CAN/CGSB , Electronic Records as Documentary Evidence Civil Practice Note No. 4 - Guidelines for the use of Technology in any Civil Litigation Matter (Court of Queen s Bench). ISO 12639:2004/Amd 1:2007, Use of JBIG2-Amd2 compression in TIFF/IT ISO :2012, Electronic Imaging - Vocabulary ISO 14492, Information Technology Lossy/lossless coding of bi-level images AMENDMENT 1: Encoder-First Edition; Amendment 2: 12/15/2003; Amendment 1: 12/15/2004 ISO :2005, Document management Electronic document file format for long-term preservation Part 1: Use of PDF 1.4 (PDF/A-1) ISO :2008, Document management - Portable document format - Part 1: PDF 1.7 ISO/IEC :1994, Information technology - Digital compression and coding of continuous-tone still images: Requirements and guidelines Understanding Metadata, National Information Standards Organization (NISO) Press (2004) McGraw-Hill Dictionary of Scientific & Technical Terms, 6E, The McGraw-Hill Companies, Inc. (2003) TIFF Revision 6.0: Final June 3, 1992 WordNet 2.1, Princeton University (2005) Owner Service Alberta, Information Management Branch SA.InformationManagement@gov.ab.ca Contact IMT Standards at imt.standards@gov.ab.ca Additional Information Audience Source Sensitivity Government of Alberta Service Alberta, Information Management Branch Unrestricted Proposed Date Proposed By Service Alberta, Open Government Laurel Frank, Executive Director Information Management Branch (780) INFORMATION SENSITIVITY: Unrestricted Page 8 of 8