Successful Quality Management for Third-party Outsourced Mass Digitization. (the Revs Program)

Transcription

1 Successful Quality Management for Third-party Outsourced Mass Digitization (the Revs Program) Matt Pearson Stanford University Libraries Eric Philcox Pixel Acuity LLC Best Practices Exchange 2012 December 5, Annapolis, MD

2 About the Revs Program The Revs Institute Stanford The Collier Collection

3 EXAMPLE Revs film and slides

4 EXAMPLE Revs film and slides

5 EXAMPLE Revs film and slides

6 About the Revs Program The Revs Institute Stanford The Collier Collection

7 Collections: source materials More than 10 distinct collections covering more than 100 years on the history, production, sales and study of the automobile Well over 300,000 negatives and slides Glass plate negatives and unique formats 200,000 serials 17,000 monographs Books in and out of copyright Advertisement materials Motion picture film 400,000 items of ephemera Scrapbooks and clippings Posters, blueprints and schematics

8 EXAMPLE Revs film and slides

9 EXAMPLE Revs film and slides

10 EXAMPLE Revs film and slides

11 EXAMPLE Revs film and slides

12 EXAMPLE Revs film and slides

13 Cataloging/metadata Manifests created for each collection Metadata created from manifests Mapping and automation Objects usually pre-registered Large scale, batch accessioning Mapping and automation MODS MARC and Getty vocabularies

14 Proposed imaging workflow

15 Training Revs Institute staff Environment Hardware and software Guidelines and procedural documents Image defect page Training webcasts Webex and screen sharing conference calls

16 SUL QC reports Daily capture of calibration targets Weekly analysis and reporting Image auditing and QC review Remakes Communication

17 About Pixel Acuity Extensive history digitizing collections Seasoned perspective Superior technical knowledge Proven results

18 The Pilot Large scale project Working out details and expectations Proving approach and workflow were correct fit

19 Planning Tested lenses, digital backs Tested and customized copy stands Researched and tested dust control Tested multi able negative carriers

20 Focused Pilot Focus on 35mm and 120 film Demonstrate throughput capabilities

21 Set up Set up and align Testing and validation Demonstrating results

22 Brass Tacks First production run Demonstrated impressive though put Surpassed quality requirements

23 Cars in a Row Discussed images just shot Talked though the over all work flow with examples Able to address questions about workflow

24 Results Over 6,000 image digitized Very high quality requirements surpassed Understanding by all parties what it would take for this to be a successful project

25 Lessons learned Need for more application-specific carriers Adjustments in optical bench efficient handling Workflow adjustments for faster throughput

26 Lessons learned Collect data before evaluating and reporting on performance Under promise and over deliver does not always help Pacing the project Preparing enough content for a healthy lead (cataloging, rehousing) Mistakes happen catch them, own them, and have a plan Great qualities in a vendor: skill, professionalism, good communication, willingness to work with us, willingness to listen ( customer is always right, even when they re not) Make site visits Don t skip materials assessment! Pictures, pictures, pictures

27 Contact information Matt Pearson Quality Assurance Specialist Stanford University Libraries Stanford, CA Eric Philcox owner Pixel Acuity p c eric@pixelacuity.com

28 LINKS Image Science Associates Pixel Acuity Revs content in Stanford s catalog QC training webcasts Revs Institute Revs@Stanford Stanford University Libraries Image defects page

29 EXTRAS: 20 day snapshots detail

30 EXTRAS: daily reports (Bassi Verrati)

31 EXTRAS: QC report detail

32 EXTRAS: SUL accessioning workflow While not critical for understanding how JP2s are generated, the actual workflow for accessioning depends on the source up to a certain point: For external sources with non pre registered objects (e.g. Revs), we need to do steps 1 3 below first. For other sources with pre registered objects (e.g. legacy projects from the labs) or stuff coming from the DPG labs, we skip steps 1 3 below. 1. A hard drive is provided with a CSV manifest, one row per object the row must contain at least a source ID, and either a folder name (for an object with many files) or a filename (for an object with a single file, like Revs). The file and/or folder names are largely irrelevant as long as they are unique and don't clash with each other 2. The manifest is parsed and objects are registered in our repository, assigning each a unique ID (the druid). The druid is associated with the supplied source ID. 3. If additional columns of metadata were provided with the CSV, they can be parsed to create object level descriptive metadata. 4. If checksums were provided for the source files, checksums are recomputed and compared with those provided. Common to all projects 1. All files are staged into the accessioning workspace, organized into new object level folders named by druid. Filenames are irrelevant, as long as they are unique within each object. 2. Basic content metadata XML is generated and stored with the object. Checksums are added if provided. 3. Assembly workflow is initialized. Assembly workflow 1. A JP2 generation robot generates a JP2 for each master TIFF that is part of an "image" or "book" object which is deemed to be a "valid" image. Validity is defined as (a) the file has one of a specific set of image mimetypes; (b) the file has EXIF data, c) the EXIF data species a color profile. The robot uses the assembly image gem to both confirm image validity and generate the JP2. 2. A checksum computation robot computes and compares checksums with any provided checksums. 3. An exif robot computes and adds file and exif data to the content metadata XML file (including width and height for images, and mimetype, encoding and file size for all files). 4. An accessioning initiate robot starts up the accessioning workflow. Accessioning Workflow 1. This it the same as always, performing the long list of tasks needed to move the complete object in the repository, including publishing it, creating a PURL and shelving files as needed. All of the relevant gems that are used are briefly described on the consul page below, but the JP2 one is called "assembly image". A better description of a gem would be a self contained software module that provides specific functionality for use in any Ruby application. In this context, the assembly image gem can be used in the JP2 generation robot, but it can also be used by the DPG labs to generate JP2s as needed, or in any other Ruby based application that needed JP2s to be generated.