What is a "Good Image"?

What is a "Good Image"? Norman Koren, Imatest Founder and CTO, Imatest LLC, Boulder, Colorado Image quality is a term widely used by industries that put cameras in their products, but what is image quality? The process of defining metrics, determining measurement methods, and designing ways of reporting them is not simple. At Imatest, we attempt to provide measurement processes for all key image quality metrics under a wide range circumstances to assist customers in finding what makes a "good image" for their application. We will go through how we define metrics, design test charts, and present results in ways that are accessible to non-specialists. P. I-2 Imatest: Background Norman Koren grew up in Rochester near Eastman House. Got interested in photography at 12. Summer job at U of R Institute of Optics, summer 1961. Created to enable photographers to test lenses and cameras. Released v1.0 September 2004. Measured Sharpness, Tonal response & dynamic range, Color accuracy Consists of several modules that analyze images of standard and custom targets. GUI-based version for R&D; non-gui version for production line testing. Compiled Matlab. Frequent enhancements in Beta version. P. I-3 Imatest 2 day class III Norman Koren www.imatest.com 2011, Imatest LLC P. III-1

Beg to differ! If you have something to say and your equipment sabotages it, you are not a happy camper. P. I-4 Image Quality Factors Many factors contribute to image quality. The weighting factors depend on the use case. This image from Utah will be used to illustrate image quality degradations. http://www.imatest.com/docs/iqfactors/ Sharpness Lateral chromatic aberration Noise Color accuracy Tonal response and contrast Dynamic range Exposure accuracy Light falloff Lens distortion ISO sensitivity P. I-5 Imatest 2 day class III Norman Koren www.imatest.com 2011, Imatest LLC P. III-2

Sharpness (MTF / SFR) SFR results: MTF tends to decrease with spatial frequency faster for blurred images. Derived from slanted-edge or other target Arguably the most important IQ factor Determines how much detail can be conveyed Affected by the lens, sensor, and digital sharpening. Depends on target MTF50 (MtF50P), the spatial frequency f where MTF falls to 50% of low freq. (peak), is a good sharpness indicator. original blurred SFR & other sharpness modules P. I-6 Charts for testing sharpness Most sharpening Signal processing affects different patterns differently. Most noise reduction P. I-7 Imatest 2 day class III Norman Koren www.imatest.com 2011, Imatest LLC P. III-3

MTF measurement matrix Different patterns needed because of non-uniform signal processing: http://www.imatest.com/docs/sharpness/#matrix Test chart Properties Slanted-edge (SFR, ) Most efficient use of space; can map sharpness over image surface. Auto region detection with. Good noise immunity. Fast. Most sensitive to sharpening. Log Frequency Check on slanted-edge. Inefficient use of space. Log F-Contrast Measures loss of texture detail due to software noise reduction. Siemens star Random/Dead Leaves/Spilled Coins Wedge Directional MTF. Less sensitive to sharpening. Inefficient use of space. Measures texture response (affected by signal processing). DL/SC pattern has statistics similar to natural scene: in CPIQ standard. Random is most sensitive to noise reduction. Measures vanishing resolution (loss of closely-spaced lines). Not consistent for MTF. Oct. 2013 Imatest 2 day class Norman Koren www.imatest.com 2008-2013, Imatest LLC P. I-8 SFR Slanted-Edge analysis Most strongly sharpened pattern. This image has moderate sharpening. Several edges can be analyzed. Upper: Average edge, with statistics: 10-90% rise, etc. Lower: MTF with statistics: MTF50, MTF50P, secondary readouts, etc. P. I-9 Imatest 2 day class III Norman Koren www.imatest.com 2011, Imatest LLC P. III-4

Slanted-Edge analysis Shows summary metric (MTF50P in this case) over entire image. Surface & summary plots, summary results (weighted). Many options available. P. I-10 Star Chart: Shows directional MTF Moderately affected by sharpening. Several displays, including standard MTF plots, are available. On right: Directional MTFnn (or MTFnnP) 8 segments shown. Limiting resolution (MTF10) P. I-11 Imatest 2 day class III Norman Koren www.imatest.com 2011, Imatest LLC P. III-5

Batchview (Postprocessor) Display and compare SFR/ batch results Compare results of several (batch) runs using saved rootname_sfrbatch. csv files. Display MTF50, MTF50P, MTF20, MTF20P, R1090 (rise distance), Line spread PW50, Chromatic Aberration (various) Independent variables: Aperture, Camera, Lens, ISO Speed, Focal length, Shutter speed, Aug. 2012 Imatest 2 day class Norman Koren www.imatest.com 2008-2012, Imatest LLC P. I-12 Batchview Compare up to four SFR/ batch results Compare several sets of runs for different lenses, focal lengths: The Canon 24-70mm L lens at 24mm is compared to two 17-85mm IS lenses at various focal lengths. APS- C sensor. Aug. 2012 Imatest 2 day class Norman Koren www.imatest.com 2008-2012, Imatest LLC P. I-13 Imatest 2 day class III Norman Koren www.imatest.com 2011, Imatest LLC P. III-6

MTF Compare Compare (deconvolve) MTF results Compare MTFs of two SFR or runs using saved CSV detailed files. Display A/B transfer function B/A transfer function A emphasized B emphasized Edge plots Aug. 2012 Imatest 2 day class Norman Koren www.imatest.com 2008-2012, Imatest LLC P. I-14 Sharpness (oversharpening) Digital sharpening enhances edges; it boosts MTF and may cause peaking. Oversharpening causes severe halos at edges. Common in compact digital cameras & camera phones. SFR results: Top: Oversharpened edge Bottom: MTF original oversharpened SFR & other sharpness modules P. I-15 Imatest 2 day class III Norman Koren www.imatest.com 2011, Imatest LLC P. III-7

Texture (removed by software noise reduction) - Signal processing is nonuniform in most consumer cameras: Sharpening near edges & contrasty features; Noise reduction (blurring) in texture areas Can cause loss of texture detail. MTF from the charts below can measure texture loss. Random/ Scale-invariant Log F- Contrast Spilled Coins/ (Dead Leaves) original noise-reduced Random/Dead. Leaves, Log F-C P. I-16 Log F-Contrast Display: MTF (linear frequency scale); MTF Normalized Normalized MTF decreases as chart contrast decreases because software sharpening is reduced and noise reduction may be applied where contrast Is low. There are better displays. P. I-17 Imatest 2 day class III Norman Koren www.imatest.com 2011, Imatest LLC P. III-8

Log F-Contrast Display: MTF/contrast (2D pseudocolor contour); Normalized contrast level Same information as normalized MTF curves, but presented as a contour plot. Most useful display. Shows decrease in MTF with decreasing contrast: mild because it s a DSLR at ISO 100. P. I-18 Log F-Contrast MTF/contrast (2D pseudocolor contour); Normalized contrast level Panasonic TZ-1 compact digital camera. Signal processing varies with ISO speed. ISO 80 ISO 800 Same camera, framing! Nov. 2013 What is a good image? Norman Koren www.imatest.com 2013, Imatest LLC P. I-19 Imatest 2 day class III Norman Koren www.imatest.com 2011, Imatest LLC P. III-9

Spilled coins Shows very little sign of software sharpening; slight texture loss compared to slanted-edge (next). P. I-20 Slanted-edge (same image as Spilled coins) Shows moderate software sharpening. P. I-21 Imatest 2 day class III Norman Koren www.imatest.com 2011, Imatest LLC P. III-10

Lateral Chromatic Aberration (LCA) Seen as color fringing near corners. Can be digitally corrected: best before demosaicing. New Dot Pattern module does I3A CIPQ LCA measurement. SFR original with LCA Dot Pattern P. I-22 Noise Random perturbations of signal level. A serious degradation; corresponds to grain in film. Software noise reduction can remove fine detail. Stepchart results, showing noise in two different scales. Top: Noise in f-stops (referenced to signal). Bottom: Noise in pixels. original noisy Colorcheck, Multicharts Stepchart P. I-23 Imatest 2 day class III Norman Koren www.imatest.com 2011, Imatest LLC P. III-11

Color accuracy: Colorcheck, Colortest, Multicharts Uses image of X-Rite Colorchecker or other color chart. Many supported. Errors displayed in L*a*b* space. Many other displays. Several color difference metrics can be selected. Accurate colors may not be pleasing. You can enter custom (pleasing) reference colors. Color correction matrix can be calculated original color-shifted Colorcheck, Multicharts P. I-24 Colortest (Matrix correction) Reference/Input Corrected ΔE 2000 Input (top right); Corrected (bottom right) P. I-25 Imatest 2 day class III Norman Koren www.imatest.com 2011, Imatest LLC P. III-12

Tonal response, Contrast, and dynamic range Pixel level luminance γ (γ is gamma = contrast); S-curve often superimposed. Dynamic range is a function of quality level (max noise or min SNR). original clipped Stepchart, Multicharts Log exposure P. I-26 Exposure accuracy Important in cameras with autoexposure Affected by history: may change after exposure to very bright or dim light. Calculated from reference values for step chart or Colorchecker, which may be part of a scene. original overexposed Stepchart Colorcheck, Multicharts. P. I-27 Imatest 2 day class III Norman Koren www.imatest.com 2011, Imatest LLC P. III-13

ISO Sensitivity Measures a sensor s ability to respond to light Two measurements derived from the ISO standard Saturation-based Standard Output Sensitivity (specified pixel level for middle gray tone) Calculated in Stepchart, ColorCheck, Multicharts, or when incident light in lux is entered. original low sensitivity Stepchart Colorcheck, Multicharts. P. I-28 Light falloff Measures light falloff due to lens and sensor, color shifts due to sensor pixel shading, and other sensor nonuniformities (dust, local noise, etc.). original vignetted Uniformity, Uniformity-Interactive, Blemish detect P. I-29 Imatest 2 day class III Norman Koren www.imatest.com 2011, Imatest LLC P. III-14

Photograph a uniform white or gray region. Settings can be tuned so visible defects are flagged and invisible defects are ignored. Blemishes original blemishes Blemish detect Blemish statistics (size, locations, etc.) are used for determining Pass/Fail in production environments (in Imatest IT). Oct. 2013 Imatest 2 day class Norman Koren www.imatest.com 2008-2013, Imatest LLC P. I-30 Lens distortion Barrel or pincushion Can be measured using a grid or a single line near the image boundary. Several correction coefficients calculated. original distorted Distortion Dot Pattern P. I-31 Imatest 2 day class III Norman Koren www.imatest.com 2011, Imatest LLC P. III-15