Comparison of 4 methods for quantifying posterior capsule opacification

Transcription

1 Comparison of 4 methods for quantifying posterior capsule opacification Oliver Findl, MD, Wolf Buehl, MD, Rupert Menapace, MD, Michael Georgopoulos, MD, Georg Rainer, MD, Hannes Siegl, MSc, Alexandra Kaider, MSc, Axel Pinz, PhD Purpose: To compare the results of posterior capsule opacification (PCO) quantification and the repeatability of a fully automated analysis system (Automated Quantification of After-Cataract [AQUA]) with that of 2 other quantification methods and subjective grading of PCO. A test set of digital retroillumination images of 100 eyes with PCO of varying degrees was used. Setting: Department of Ophthalmology, University of Vienna, Vienna, Austria. Methods: One hundred digital retroillumination images of eyes (100 patients) with PCO were selected to attain an even distribution from mild to severe cases. The images were evaluated by 4 methods: subjective grading by 4 experienced and 4 inexperienced examiners, the subjective Evaluation of Posterior Capsular Opacification (EPCO) system, posterior capsule opacification (POCO) software, and the AQUA system. Ten images were presented twice to assess the reproducibility of the analysis systems. Results: Subjective grading correlated best with the subjective EPCO system and the objective AQUA system (r 0.94 and r 0.93, respectively). The POCO system showed very early saturation and therefore a much weaker correlation (r 0.73). The POCO scores reached the maximum of 100% in several minimal to mild PCO cases. The reproducibility of the AQUA software was perfect and that of the other analysis systems, comparably satisfactory. Conclusion: The objective AQUA score correlated well with subjective methods including the EPCO system. The POCO system, which assesses PCO area, did not adequately describe PCO intensity and includes a subjective step in the analysis process. The AQUA system could become an important tool for randomized masked trials of PCO inhibition. J Cataract Refract Surg 2003; 29: ASCRS and ESCRS Posterior capsule opacification (PCO) remains a common complication of modern cataract surgery. Although its incidence has decreased slightly as a result of improved surgical technique and new intraocular lens (IOL) designs, the reported occurrence is still significant and varies largely among studies. 1 Many studies use different criteria to clinically judge and quantify PCO. Accepted for publication April 30, Reprint requests to Oliver Findl, MD, Department of Ophthalmology, University of Vienna, Waehringer Guertel 18-20, 1090 Vienna, Austria. To compare interventions for preventing PCO, an objective and standardized method of measurement is needed. This is especially important in cases of PCO, in which masking of examiners is not possible because the IOL type is always detectable. However, to perform studies compatible with the criteria of evidence-based medicine, a masked evaluation of the main outcome variable the intensity of PCO is mandatory. Therefore, the aim is to develop a reliable, time-sparing, easyto-use method of assessing PCO that does not include subjective steps. Our group recently introduced such a system, Automated Quantification of After-Cataract (AQUA), in 2003 ASCRS and ESCRS /03/$ see front matter Published by Elsevier Science Inc. doi: /s (02)

2 cooperation with the Institute of Electrical Measurement and Measurement Signal Processing at the Technical University of Graz (O. Findl, MD, et al., AQUA A Novel Approach to Automated Quantification of After-Cataract, presented at the XVIIIth Congress of the European Society of Cataract & Refractive Surgeons, Brussels, Belgium, September 2000). This system is to be used for prospective randomized masked clinical trials assessing the effectiveness of surgical techniques, drugs, and IOL design and material on preventing PCO. This study compared the results of PCO quantification and the repeatability of the fully automated AQUA analysis system with those of subjective grading of PCO and 2 other quantification methods based on a test set of digital retroillumination images of 100 eyes with PCO of varying degrees. Patients and Methods Digital Image Data Set The intention was to create a data set of digital retroillumination photographs of regeneratory PCO of varying degrees and to use this set to compare 4 scoring methods for PCO. The results of the subjective grading of this test set should serve as a gold standard for the further development of the AQUA software. The high-resolution digital coaxial retroillumination photographic system is similar to the system reported by Pande et al 2 and Buehl et al. 3 The optical system consists of a Zeiss 30 slitlamp for observation and imaging, a Zeiss retrolux illumination module with illumination provided by a Zeiss anterior segment flash pack through a fiber-optic cable, and beam splitters. A Kodak NC2000 digital camera with high light sensitivity resulting from a 16.0 mm 21.0 mm chargecoupled device (CCD) chip is used, resulting in a high signalto-noise ratio in the acquired images. The CCD has a geometric resolution of 1268 pixels 1012 pixels and a radiometric resolution of 36 bits (red, green, and blue [RGB]). The images are directly imported into Adobe Photoshop 5.5 and saved to a hard disk in tagged image file format (TIFF, 3.85 megabytes per image). One hundred digital images of eyes (100 patients) with PCO were subjectively selected by an experienced examiner to attain an even distribution of mild to severe cases; images with no visible PCO were included. Good image quality was important, whereas IOL design and time since surgery were not. To ensure that only the region of interest the posterior capsule was assessed, only the central 4.0 mm of the IOL not containing any part of the anterior capsule was made visible. This was done with Adobe Photoshop by cutting the central circular 4.0 mm area out of the image. Thus, exactly the same area was evaluated by all analysis methods. To assess reproducibility, 2 photographs were taken consecutively in 10 cases. Another 10 images were duplicated and the copies rotated using Adobe Photoshop. Thus, the complete data set consisted of 120 images of 100 eyes. None of the examiners scoring or operating the analysis systems was informed about the intent of the 20 additional images. The file names of the images were randomly assigned to produce a random sequence of TIFF images, and the images were stored consecutively on a CD-ROM. Using the image data set, 2 subjective PCO evaluation methods (subjective scoring and Evaluation of Posterior Capsular Opacification [EPCO] software) and 2 objective methods (posterior capsule opacification [POCO] software and AQUA software) were compared. The time needed for analysis was noted. Subjective Analysis Four experienced examiners from the Vienna University Department of Ophthalmology graded all images subjectively and independently using a score of 0 to 10 in 0.5 steps. The images were displayed on a high-resolution 19-inch computer screen in a dark room. Each examiner viewed 20 images before starting the grading process. All images were scored in 1 sequence at 1 session, and the results could not be changed later. The examiners assessed the average opacification over the entire area without taking localization, distribution, or type of PCO into consideration. A completely clear capsule received a score of 0. A capsule completely covered with severe, inhomogeneous, strongly light-attenuating PCO, classified as the most intense PCO possible, received a score of 10. A second round of scoring using identical methods was performed after a washout period of 1 week to determine intraobserver variability (repeatability). The images were also independently evaluated by 4 inexperienced student examiners in the same way as above to reveal possible differences between their results and those of the expert examiners. Four weeks later, 3 expert examiners graded the images together on 1 occasion to discuss them and then determine a common score. EPCO Analysis A computer-assisted system, EPCO was introduced in 1997 by Tetz et al. 4,5 and is commercially available. It is based exclusively on the morphological assessment of PCO. It incorporates planimetric and grading assessment. The density of the opacification behind the IOL is graded clinically as follows: 0 none; 1 minimal; 2 mild; 3 moderate; 4 severe. The individual PCO score is calculated by multiplying the opacification grade by the fraction of capsule area involved behind the IOL optic. The selection process and grading of areas are subjective. All images from the data set were evaluated with the EPCO system by 3 expert examiners on 1 occasion. Each J CATARACT REFRACT SURG VOL 29, JANUARY

3 examiner drew the outlines of the EPCO areas with a computer mouse in the same number of images after discussing the case with the other 2 examiners. POCO System With the computer-based POCO system, developed at St. Thomas Hospital and King s College in London, 6 the images are evaluated using pixel analysis based on texture differences. Analysis results in a percentage of opacification area from 0% to 100%. Different opacification densities are not directly accounted for but are incorporated through the texture analysis. The data set of images from this study was sent to London and evaluated there with the POCO system. The percentage of the opacification area and black-and-white illustrations were returned to Vienna. The amount of time needed for evaluation was not known. AQUA System The AQUA software is also based on texture analysis. The outcome measure, the inhomogeneity of the image, is based on statistical texture analysis. To calculate the inhomogeneity of the texture of an image, a gray level co-occurrence matrix (GLCM) of the bitmap of the image is built. The GLCM belongs to 2nd-order statistics, which deal with the probability that 2 pixels connected by a position operator (eg, the relative position of the 2 pixels) have 2 gray levels. A GLCM is a square matrix with a side length corresponding to the number of gray levels of a bitmap. The grading is performed using the GLCM entropy, which is a measure of inhomogeneity. The GLCM is computed for 1 of the 3 color channels (RGB), currently the red channel. The exact range of possible AQUA scores (the entropy values) for digital images of the posterior capsule was unknown. Thus, the upper and lower limits were determined empirically during the evaluation phase by evaluating images not included in the study s data. Twenty images of eyes without PCO and 10 images with extremely severe PCO were used. The mean results were taken as the lower limit and upper limit, respectively, for the entropy values calculated by the software. With these limits, it was possible to convert all AQUA results to a score of 0 to 100; this score was used as the final outcome measure. One person operated the program; the scoring process contained no interactive or subjective steps. Statistical Analysis For statistical evaluation, all results were converted to scores from 0 to 100 (ie, subjective score 10, EPCO result 25), which can therefore also be read as percentages. All data were statistically evaluated with the Excel 2000 (Microsoft), SAS 6.12 (SAS Institute), and Statistica 5.1 (Stat- Soft) software packages. Correlation coefficients (r) were calculated to compare the results of the 4 scoring methods and the subjective scoring results of experts and students. The reproducibility of the 3 analysis systems was assessed by the repeatability coefficient using the 10 duplicated images. 7 The repeatability coefficient describes the maximum difference that can be expected with a probability of 95% for repeated use of each system. In addition, the interobserver agreement and intraobserver agreement were determined for subjective scoring. The interobserver agreement was analyzed by means of the upper and lower limit of agreement and the mean difference for each comparison (eg, between 2 experts). The limit of agreement is the maximum difference in the results of 2 observers that can be expected with a probability of 95%. 6 The mean absolute pairwise difference was calculated for each image to obtain summary limits of agreement describing the maximum differences in the subjective grading of 2 observers that can be expected with a probability of 95%. The intraobserver agreement (reproducibility) was again analyzed by the repeatability coefficient, which was calculated for each observer separately and as a summary of the pooled data of all observers. The agreement between students and experts was calculated in the same way. Results Figures 1 and 2 show the correlations and the correlation coefficients (r), respectively, among the 4 scoring methods. Figure 3 shows the results as box plots including the minimum, maximum, 25th to 75th percentile (as a box), median, and outliers. The 100 cases are divided into 4 groups according to the subjective PCO grade (minimal, mild, moderate, or severe). For each group, 4 box plots represent the 4 analysis methods. Thus, the agreement between the scoring methods and the range of results can be judged for cases within a certain range of PCO intensity. Representative cases close to the median subjective score in each of the 4 groups were chosen as examples (Figure 4). Subjective grading revealed no significant differences in the scoring results between experienced observers and inexperienced observers. There was little difference between the intraobserver and interobserver agreement. For subjective scoring, the intraobserver agreement between the 4 experts was good (repeatability coefficient, 14.5), as was the interobserver agreement (upper and lower limits of agreement, 1.2 and 17.3, respectively; mean absolute difference, 9.3). The correlation between the mean scoring results of the experts and the students was The correlation between the mean subjective scoring results and the common score of the 3 experts was J CATARACT REFRACT SURG VOL 29, JANUARY 2003

4 Figure 1. (Findl) Plot of the subjective score for all cases to illustrate the distribution of the data set (top left). Scatter plots and linear regression lines of the subjective score with EPCO (top right), POCO (bottom left), and AQUA (bottom right) scores. Figure 2. systems. (Findl) Correlation coefficients for the 4 PCO analysis Figure 3. (Findl) Box plot of PCO grade with the subjective, EPCO, POCO, and AQUA methods. Four groups with a different severity of PCO according to subjective grading were formed and presented as box plots, showing the minimum, maximum, 25th to 75th percentile (as a box), median, and outliers. In contrast to the relatively balanced distribution of the results of subjective grading, the EPCO method, and the AQUA method, the POCO score was lower than 20% in 11 cases (ie, 11%) but higher than 80% in 54 cases and higher than 90% in 32 cases. The minimum score (0) was given in 1 case with subjective scoring, in 3 cases with the EPCO system, in 5 cases with the J CATARACT REFRACT SURG VOL 29, JANUARY

5 the first and the second result for each scoring method. The AQUA software produced almost identical results for the original images and the duplicates (differences essentially 0). However, all other scoring methods led to significant differences (maximum 12.8% with the POCO system). Assessing the reproducibility of the photographic technique and the analysis method, the 10 cases with 2 consecutive (not identical) photographs per eye led to similar differences with all scoring methods. The maximum difference was 9.7% with subjective scoring, 8.7% with the EPCO system, 14% with the POCO system, and 8.6% with the AQUA system. The mean time for analysis of 1 photograph was 25 seconds for subjective scoring 144 seconds for EPCO, and 78 seconds for AQUA. No data were available for the POCO system. Figure 4. (Findl) Representative cases close to the median subjective score in each group. The intensity of the PCO is minimal (top left), mild (top right), moderate (bottom left), and severe (bottom right). AQUA system, and in no case with the POCO method (minimum 5.8). The maximum score (100) was not reached in any case with subjective scoring (maximum 87) or the EPCO system (maximum 92) but was reached in 2 cases with the AQUA system and in 16 cases with the POCO system. Assessing the reproducibility of analysis methods, Figure 5 shows the results in the 10 cases with the duplicated images, illustrating the 10 differences between Figure 5. (Findl) Reproducibility of analysis methods. Plot of the differences between the first and second result for the 10 cases with the duplicated images by system. Discussion In this study, we compared the results of PCO quantification by 4 methods. Subjective grading correlated best with the EPCO system (r 0.94). This was not unexpected as the EPCO system is a subjective method and was operated by the same examiners who performed subjective scoring. The fully automated AQUA system showed a similarly good correlation with subjective scoring (r 0.93). The distribution of results was similar to that with the EPCO system but with a slightly higher mean variation. As expected, the reproducibility of this automated method was perfect. In contrast, the POCO system showed early saturation. It reached the maximum score of 100% in several minimal to mild cases. This could mean that it is more responsive and therefore perhaps more sensitive in mild cases but that the scores are not very meaningful in moderate and severe cases. This should be taken into consideration when reviewing the results of studies using this system. 8,9 The POCO analysis results in a percentage of opacification area. Therefore, the result does not indicate the overall PCO intensity. An eye with a thin monolayer that does not disturb vision may have the same percentage of opacification area as an eye with diffuse, dense Elschnig pearls. A severe case of PCO with the POCO system might be graded subjectively as minimal PCO intensity. Such patients often have excellent vision and would not be candidates for an intervention such as 110 J CATARACT REFRACT SURG VOL 29, JANUARY 2003

6 a capsulotomy. Therefore, the correlation between subjective scoring and the POCO results was only The evaluation of the 10 additional cases with duplicated images showed large, significant differences among the systems. As expected for an automated system, the AQUA results were virtually equal. However, the POCO results showed significant deviations similar to those of the subjective systems. This was unexpected because the underlying principle of the POCO system resembles that of the AQUA system. This indicates that a subjective step must be incorporated in the POCO method. This is mentioned indirectly in the discussion section of the article describing the POCO method. 6 In this study, subjective grading revealed no significant differences in the scoring results of experienced observers and inexperienced observers (students), and the results of all examiners showed a high correlation as well as the results of repeated scoring by the same person. In addition, the correlation between the mean scores and the common score (3 experts together) was high (0.98). Therefore, this score seems to be a reliable measure of PCO and thus can be taken as a gold standard for comparison with objective methods. Further development of the AQUA software will be based on this standard. Our results of the time needed for scoring revealed that the AQUA method is currently the fastest objective and automated scoring system, taking 1.3 minutes per image with the existing software version. Calculation and acquisition of an image should become faster in future versions. In conclusion, of the 4 methods for the quantification of PCO studies, the objective AQUA score correlated well with subjective methods, including the commercially available EPCO system. The POCO system, which assesses the PCO area, did not adequately describe PCO intensity and includes a subjective step in the analysis process. The AQUA system could become an important tool for prospective randomized masked clinical trials to assess the effectiveness of surgical techniques, drugs, and IOL design and material on preventing PCO. References 1. Schaumberg DA, Dana MR, Christen WG, Glynn RJ. A systematic overview of the incidence of posterior capsule opacification. Ophthalmology 1998; 105: Pande MV, Ursell PG, Spalton DJ, et al. High-resolution digital retroillumination imaging of the posterior lens capsule after cataract surgery. J Cataract Refract Surg 1997; 23: Buehl W, Findl O, Menapace R, et al. Reproducibility of standard retroillumination photography for quantification of posterior capsule opacification. J Cataract Refract Surg 2002; 28: Tetz MR, Auffarth GU, Sperker M, et al. Photographic image analysis system of posterior capsule opacification. J Cataract Refract Surg 1997; 23: Tetz MR, Nimsgern C. Posterior capsule opacification. Part 2: clinical findings. J Cataract Refract Surg 1999; 25: Barman SA, Hollick EJ, Boyce JF, et al. Quantification of posterior capsular opacification in digital images after cataract surgery. Invest Ophthalmol Vis Sci 2000; 41: Bland JM, Altman DG. Statistical methods for assessing agreement between two methods of clinical measurement. Lancet 1986; 1: Hollick EJ, Spalton DJ, Ursell PG, et al. The effect of polymethylmethacrylate, silicone, and polyacrylic intraocular lenses on posterior capsular opacification 3 years after cataract surgery. Ophthalmology 1999; 106:49 54; discussion by RC Drews, Ursell PG, Spalton DJ, Pande MV, et al. Relationship between intraocular lens biomaterials and posterior capsule opacification. J Cataract Refract Surg 1998; 24: From the Departments of Ophthalmology (Findl, Buehl, Menapace, Georgopoulos, Rainer) and Medical Computer Sciences, Section of Clinical Biometrics (Kaider), University of Vienna, Vienna, and the Institute of Electrical Measurement and Measurement Signal Processing, Graz Technical University, Graz (Siegl, Pinz), Austria. Presented in part at the XVIIIth Congress of the European Society of Cataract & Refractive Surgeons, Brussels, Belgium, September Funded in part by grant P14269-MED, Fonds zur Förderung der wissenschaftlichen Forschung, Vienna, Austria. None of the authors has a financial or proprietary interest in any material or method mentioned. Drs. Spalton and Boyce, St. Thomas Hospital & King s College, London, evaluated the digital images with the POCO software. J CATARACT REFRACT SURG VOL 29, JANUARY