Digitl Color Fundus Imge Qulity: the Impct of Tonl Resolution 15 ORIGINAL ARTICLE Lrry D Hurd, MAT Deprtment of Ophthlmology nd Visul Sciences University of Wisconsin 406 Science Dr., Suite 400 Mdison, WI 53711 608/263-2245 Hurd@rc.ophth.wisc.edu Digitl Color Fundus Imge Qulity: the Impct of Tonl Resolution T Introduction o explore how to chieve high qulity retinl imge, it is importnt first to sk wht the chrcteristics of tht imge should e. It is prolemtic to define imge qulity in terms of showing things s they ctully re. Delving into the imging process, one sees tht relity is ctully reltive stte ffected y severl importnt vriles, ll of which need to e considered: not only ttriutes of the oject eing imged, ut lso the light eing used to imge it, the imging device, nd ultimtely the oserver, who is conditioned y the strengths nd limittions of the humn visul system. How Do Humns See? Fundmentlly, humns process the visul phenomen of the world into fetures resolved ginst ckground. Much of this hs to do with humn evolution s species. Historiclly, the feture ginst the ckground my hve een n ttcking predtor or fleeing prey. Humns cn see fetures ginst ckground ecuse they re different from it. Tht is to sy, they re resolved in some mnner. There re two fundmentl imge ttriutes tht determine how successfully the fetures of interest cn e discerned ginst the ckground with which they might e confused. These re sptil nd tonl resolution. Briefly, y sptil resolution is ment the numer or density of the pixels (picture elements) devoted to depicting the feture of interest, nd y tonl resolution is ment the difference etween the ppernce of the feture nd the ckground in rightness nd/or color. Sptil resolution will e explored in future rticle. Two imge exmples will help to illustrte the effect of vrying sptil nd tonl resolution on the ility to see. Figure 1 depicts polr er ginst snowy ckground, nd Figure 1 documents retin with dietic retinopthy (DR), in this cse, microneurysms (M), retinl hemorrhges, nd intr-retinl microvsculr normlities (IRMA), ginst the retinl pigment epithelil (RPE) ckground. These two imges represent dimetriclly opposed interests: the polr er does not wnt to e seen (especilly y sel!), while the fundus photogrpher intends the ophthlmologist to e le to see the retinl disese. These two instnces my seem different in kind, ut they re merely t opposite ends of spectrum. The effect of vrying tonl resolution through rnge for oth polr er nd retin is displyed in Figures 2 nd 3. In Figure 2, the overll exposure is vried, ffecting oth rightness nd contrst, while in Figure 3 only the contrst is vried. Typiclly, t imge cpture time, rightness nd contrst re gnged together s in Figure 2. Figure 3 hs een included to show contrst hs n independent effect eyond tht of rightness. In oth series it should e noted tht t the left end of the scle, the fetures re poorly resolved ginst the ckgrounds. As one moves to the right, the fetures ecome etter defined ginst the ckground until oth polr er nd DR re esily detectile. At the fr right end of the scle, contrst (s well s rightness) of the Figure 1: () Polr er ginst snowy ckground. () Retinl vessels nd DR ginst n RPE ckground. Both imges chllenge the cpilities of the medium to depict feture ginst ckground, ecuse in oth the feture nd its ckground hve somewht similr rightness nd color.
16 The Journl of Ophthlmic Photogrphy Volume 31, Numer 1 Spring 2009 imge hs een pushed fr enough to see rtifcts polr ers should not hve rinow cots, nd the RPE should not e so vriegted ut rther uniform. These exmples were constructed for Figure 2 y using the Adoe Photoshop Exposure tool (on the Imge/ Adjust menu), nd for Figure 3 y moving the Contrst tool (lso on the Imge/Adjust menu s Brightness/Contrst) seprtely for ech of the three color chnnels, while holding the Brightness setting constnt. Tonl Resolution How Much is Enough? Becuse erly color fundus cmers did not hve dequte sptil resolution, mny reserchers were focused on improving sptil resolution s the key to otining etter fundus imges. However, s sptil resolution incresed, it ecme pprent tht tonl resolution ws n eqully importnt limiting fctor in digitl imge qulity. Oviously, hving greter sptil resolution (more pixels) will not help the oserver if the feture to e detected is nerly the sme rightness nd color s the ckground (i.e., hs low tonl contrst). In Figure 3, note tht in the vrint with the lest contrst (the polr er) is virtully indistinguishle from the snowy ckground, even though the sptil resolution is dequte, eing unltered from the originl imge. To understnd the tonl resolution requirements for good qulity color fundus imges, it is helpful to explore further how humns see tonl detil in imges. More on How Humns See There re two mjor differences etween humns nd digitl imging systems tht hve importnt consequences for imge qulity. Firstly, humns hve mximum discrimintion to resolve fetures in imges tht re modertely rightly illuminted. This sweet spot in the rnge of illumintion occurs ecuse of the following. 1. When imges re too drk the retin detects them chiefly with the rods (scotopic vision), which cnnot detect differences in color. 2. When imges re somewht righter, the rods egin to get help from the cones (mesopic vision, which dd color discrimintion. Figure 2: nd show wht the polr er nd retin with dietic retinopthy look like over rnge of exposure. Oviously, the opposite of hving too little tonl resolution for stisfctory rendering is hving exggerted tonl resolution. 3. When the imge is modertely right, the rin is getting optiml input from the cones (photopic vision), t which point humn vision hs its gretest discrimintion. 4. When the imge is too right (even if it is not oversturted), the ility to discriminte fetures egins to decline gin. To summrize, digitl imges need to e rought up into the sweet spot of illumintion for est pprecition. Secondly, humns re more limited in the numer of different tonl shdes they cn discriminte compred to digitl imging systems. Conventionl digitl systems keep trck of 256 shdes in ech color chnnel. However, humn s ility to keep trck of shdes is pproximtely n order of mgnitude lower. For monochrome imges, estimtes of norml humn discrimintion in the literture rnge from pproximtely 20 shdes on the lower end, to 32 shdes in the middle, to 50+ shdes under optiml conditions. Exmine Figure 4 s check of the ility to discriminte on 23-step gryscle. Becuse sptil detil in retinl imge is cptured primrily in one chnnel (green) the limited cpility of the humn with monochrome imges lso ffects how color retinl imges re perceived.
Digitl Color Fundus Imge Qulity: the Impct of Tonl Resolution 17 Figure 3: nd show wht the polr er nd retin with DR look like over rnge of contrst (with rightness unshifted): 90% reduced, 50% reduced, originl, 25% incresed, nd 50% incresed. This vrint of contrst mnipultion is included to show tht insufficient contrst is prolemtic for feture rendering even if imges re not drk. Figure 4: The full rnge etween lck nd white divided into 23-step gry scle. Tonl Resolution Required for Retinl Fetures nd Disese Anormlities As with sptil resolution, how much tonl resolution is needed depends upon wht the oserver is trying to see. Different retinl fetures demnd different tonl resolution. Retinl fetures nd disese normlities re seen ginst the ckground of the reddish-ornge RPE. The right fetures, such s the optic disc, firous tissue, cotton wool spots, hrd exudtes, nd drusen, re not demnding. The difference etween these often whitish fetures nd the reddish-ornge RPE ckground is usully detectle even in n imge with poor tonl resolution. On the other hnd, the drk fetures, such s norml lood vessels, Ms, IRMAs, nd new vessels, re much more demnding. Becuse they often hve the rightness nd color of venous lood, which is similr in ppernce to the RPE, they tend to lend into the ckground in n imge with poor tonl resolution, especilly if the imge is very drk or the focus/clrity is elow pr. To illustrte the impct of these limittions in humn vision, exmine some contrst vrints of the eye with DR, restricted to squre region of interest centered on lrge M. Figure 5 shows series in which the Topcon Imgenet Adjust Exposure tool hs een utilized to progressively decrese oth rightness nd contrst. In the vrint with lest contrst, using the Photoshop Informtion tool to mesure the luminnce levels inside nd outside of the M would llow one to find the feture. However, the humn eye could not ctully see it directly ecuse even in the sweet spot of humn vision difference of 8 luminnce levels (256 shdes for the digitl system divided y the 32 you cn see) is needed to perceive difference. Even with difference of 8 luminnce levels etween feture nd ckground, if the imge is too drk for utiliztion of photopic vision, or too right for the cones to function most sensitively, the difference will still not e perceived. Exmine the series of the sme DR imge in Figure 6, in which the Imgenet Adjust Contrst tool hs een used to djust only contrst nd not rightness. Agin, mesuring the luminnce vlues within nd outside of the M still identifies the lesion, ut humn vision is physiclly incple of seeing it. Anlysis of Tonl Resolution An informl wy of nlyzing the tonl resolution in n imge is simply to exmine it criticlly. Fortuntely, photogrphers hve ccess to very useful sttisticl tool, known s the luminnce histogrm, to determine how much of the dynmic rnge is ctully eing exercised. Typiclly, the luminnce histogrm is presented grphiclly s plot, with the x-xis divided into luminnce vlues from zero to 255, nd the y-xis showing the numer of pixels in the imge t ech luminnce vlue. In retinl photogrphs, the rightness vlues for ech color re typiclly clustered t different points of the dynmic rnge, yielding three distinct color curves.
18 The Journl of Ophthlmic Photogrphy Volume 31, Numer 1 Spring 2009 Figure 7 shows the 3-color luminnce histogrm otined from clssic film photogrph: Age Relted Eye Diseses Study (AREDS) Stndrd #12, crefully digitized for preservtion. Any photogrpher cn perform such n nlysis on their photogrphs y exporting them from the digitl cmer system nd importing them into Adoe Photoshop (or similr progrm). Simply click on the Histogrm Figure 5: In the inset squre portion of the originl imge on the left there is lrge M t the center. In the smll squres (right), oth rightness nd contrst hve een decresed {Imgenet Exposure tool) to settings of -1, -2 nd -4. commnd on the Windows menu, nd select the All Chnnels nd Show Chnnels in Color options. In order to eliminte noise from the imge periphery, which is often mrred y rtifcts, it is helpful to select region of interest (ROI) in the imge, s illustrted in Figure 7. This is ccomplished y selecting the ellipticl Mrquee tool, clicking nd holding the mouse utton with the cursor t the center of the imge, pressing nd holding the <Alt> nd <Shift> keys simultneously, then expnding the circle out from this centrl point y moving the mouse in ny direction, stopping short of the edge rtifcts, nd then relesing the mouse utton. Selecting the sweet spot of the imge s the ROI shpes up the luminnce curves nd mkes them esier to interpret. Figure 8 presents composite digitl fundus imge (2006 OPS Best of Show wrd winner) nd its corresponding luminnce histogrm. Note tht the overll configurtion of the luminnce curves is remrkly similr to those in Figure 7. An imge with exemplry B/C/CB genertes histogrm somewht resemling suspension ridge. The red nd green peks, locted in the upper nd lower hlves of the dynmic rnge, represent the towers, nd their curves represent the cles, overlpping slightly in the middle. Blue mkes minor ppernce ner the ottom. Color digitl fundus imges hve hd mixed reputtion for qulity. Seen from the perspective of centrl reding center (UW-Mdison), the est digitl imges mtch the qulity of the est film imges, nd there is hope tht s more photogrphers lern to use them, digitl imges cn surpss film. Unfortuntely, digitl imges on the lower end of the qulity spectrum of tonl resolution cn e very unstisfctory, s shown in Figure 9. Ctegories of Poor Tonl Resolution Post-processing (enhncement) of n imge cn sve n imge tht would e difficult to interpret s it cme out of the photogrphy session, nd while t other times Figure 6: The contrst of the inset squre portion of the originl imge on the left, which contins lrge M t its center, hs een decresed y 25%, 50%, nd 75% (the squres on the right, top to ottom), without sustntilly chnging rightness. Figure 7: AREDS Stndrd #12, with luminnce histogrm of its three composite color chnnels (red/green/lue, or RGB), nd with monochrome of the three chnnels to show their reltive rightness nd contrst. (1) The green nd red chnnels contriute most of the contrst informtion. (2) The green nd red chnnels re offset from ech other so tht they rcket the middle of the dynmic rnge. (3) The lue curve is t the lower end of the dynmic rnge nd hs nrrow spn. the imge my hve insufficient tonl resolution to e retrievle. To void the ltter prolem, the fundus photogrpher should shoot to otin imges tht hve resonly good tonl resolution s they re tken. Following re the min types of tonl resolution prolems, with exmples ccompnied y luminnce histogrms to help the reder nlyze the imges.
Digitl Color Fundus Imge Qulity: the Impct of Tonl Resolution 19 Figure 8: Ophthlmic Photogrphers Society Best of Show Awrd Winner, 2006. In the luminnce histogrm, this retinl photo montge shows similr curves for red nd green chnnels compred to Figure 7. Courtesy Richrd Hckel, CRA, FOPS. Figure 9: () The originl of very drk (under-exposed) digitl color retinl imge. Unfortuntely, the detil is so poorly documented tht the oserver could only guess which disese this eye hs. () Brightness nd contrst enhncement of the very under-exposed digitl color retinl imge shown in 9. Note the posterized ppernce of the enhnced imge, prticulrly in the upper left portion of the imge. Oviously, the originl imge hd too little tonl contrst to support this degree of enhncement. Improper Illumintion Assuming the cmer s sic settings re correctly djusted, getting the right exposure for ech eye is proly the most criticl dy-to-dy chllenge. Figure 10 is n exmple of n imge tht is tken with too little illumintion to e esily interpretle. This imge cn still e slvged y contrst enhncement s the fter version shows. However, the imge in Figure 9 is eyond sving. The fter version revels tht the tonl rnge of the originl imge is so compressed tht enhncing it into the rnge optimum for humn vision produces disstrous posterized rtifct tht would interfere with detecting sutle disese normlities. There is n upper limit to the mount of stretch tht cn e pplied to the color curves without introducing ovious tonl discontinuities rguly no more thn four-fold or less. Figure 11 shows n exmple of n imge tht is over-illuminted to the point of sturtion in the red color chnnel. Not only is the detil of the feture ginst ckground in the red chnnel oliterted in some res, overll the red chnnel overpowers the detil in the green chnnel so tht the entire picture looks rther flt. While there is no wy to recover the lost detil in the red chnnel, it is possile to ring out more of the pictoril informtion in the green chnnel, s the fter version of this imge shows. This ws ccomplished y sliding the red curve down in the dynmic rnge, nd if necessry enhncing (oth rightening nd stretching) the green chnnel. Of course, it is etter to void the need for such processing. If the red/green color lnce of the cmer is pproprite, the photogrpher should le to void over-sturting in red while getting dequte detil in green. With experience, the photogrpher cn lern to see, t glnce, whether n imge is over-sturted in red. The ffected res show RPE tht looks wshed out nd uniform. Normlly there should e some vriegted pttern. If there is some question, exmine the luminnce histogrm. The red chnnel will terminte in cliff, perhps s spike, t the top end of the dynmic rnge, rther thn terminting with tpered right end, s is norml for ell curve, efore reching the top of the rnge. Figure 10: A digitl color retinl imge of n eye with extensive soft drusen of ge-relted mculr degenertion (AMD). Note the improved visiility of retinl vessels nd drusen ginst the RPE fter stndrdized enhncement. Improper Color Blnce The retinl scene presents something of prdox to the digitl imging sensor. The scene ppers mostly red,
20 The Journl of Ophthlmic Photogrphy Volume 31, Numer 1 Spring 2009 ut much of the criticl detil is green. Becuse the silicon-sed sensor rects most strongly to red, this difficulty is compounded. Thus, digitl cmers re vulnerle to producing imges tht re too strong in the red chnnel nd too wek in the green chnnel. Figure 12 illustrtes n imge with improper green/red lnce (G/R = 0.35), ccompnied y corrected version (G/R = 0.50). Note tht the second imge ppers to hve lot more contrst or depth thn the first, mking it esier to detect retinl fetures nd disese normlities. A secondry considertion in color lnce is the lue/red rtio (gin defined s the rtio of the lue pek loction to the red pek loction). In film, this rtio is typiclly less thn 0.20, mening tht lue is de-emphsized, compred to red nd green. On the other hnd, mny digitl cmers re set up to produce higher lue component thn film. Photogrphers who re wre of this my even prefer the higher lue setting. It lends the imge purplish cst tht some find estheticlly plesing. However, the photogrpher should e wre tht there is very little ctul lue signl in norml retinl imge. Thus, if the lue chnnel is incresed you re effectively enhncing noise element. Figure 13 shows this luish effect pushed towrd the extreme, ccompnied y color-corrected version with lue minimized. The ltter my not e s pretty s the former, ut experienced grders see more detil in the imge with lue under control. Figure 11: A digitl color retinl imge tken with over-sturtion in the red chnnel, due to over-exposure. Figure 12: An eye with extensive lrge drusen from ge-relted mculr degenertion. The originl shows the extreme of improper green/red color lnce (G/R = 0.017), with very strong red chnnel nd very wek green chnnel. After the imge hs een djusted to correct this y demoting the red chnnel nd enhncing the green chnnel (G/R = 0.50), the oserver cn see the extent of the soft drusen much more clerly. Summry Tonl resolution (rightness, contrst, nd color lnce) is criticlly importnt fctor in the qulity of digitl color retinl imges. Photogrphers nd their ophthlmologists need to estlish criteri regrding how they wnt their photogrphs to look. Forml tools exist, e.g. the 3-color luminnce histogrm, which llows photogrpher to nlyze nd mnge the tonl prmeters of the retinl imges tken. Ultimtely, the fundus photogrpher nd ophthlmologist hve responsiility for the utility of their retinl imges for providing ptient cre nd for contriuting to reserch studies. Fortuntely, photogrphers lso hve the Figure 13: A digitl color retinl photogrph with normlly high lue content, in its originl form (left) nd enhnced form (right). To most experienced grders, the version with less lue is esier to exmine for disese normlities ecuse the pprent contrst is not muddied y the purplish cst of the originl. power to mke nd implement informed choices out the qulity of tonl resolution in their retinl imges. Reference 1 Hurd LD, Dnis RP, Neider MW, Thyer DW, Wers HD, White JK, Pugliese AJ, Pugliese MF nd the Age-Relted Eye Disese 2 Reserch Group. Brightness, contrst, nd color lnce of digitl versus film retinl imges in the Age-Relted Eye Disese Study 2. Invest Ophthlmol Vis Sci. 2008; 49:3269-82.