Gastrointestinal Imaging Original Research

Transcription

1 Gastrointestinal Imaging Original Research Lee et al. Low-Dose CT in Cases of Suspected Appendicitis Gastrointestinal Imaging Original Research Yoon Jin Lee 1 Bohyoung Kim 1 Yousun Ko 1 Kyung Eun Cho 1 Seong Sook Hong 2 Dong Hwan Kim 3 Hyunjoo Song 4 Kyoung Ho Lee 5 Lee YJ, Kim B, Ko Y, et al. Keywords: abdomen, appendicitis, CT, low-dose CT, sliding-slab averaging DOI: /AJR Received October 26, 2014; accepted after revision May 3, Supported by grants from Korean Health Technology R&D Project funded by the Ministry of Health & Welfare, Republic of Korea (no. HI13C0004), Basic Science Research Program through the National Research Foundation of Korea (NRF) funded by the Ministry of Education, Science, and Technology (nos. NRF-2013R1A1A and NRF-2013R1A1A ), and Seoul National University Bundang Hospital Research Fund (no ). 1 Department of Radiology, Seoul National University Bundang Hospital, 82 Gumi-ro 173 Beon-gil, Bundang-gu, Seongnam-si, Gyeonggi-do, , Korea. Address correspondence to bhkim@snubh.org. 2 Department of Radiology, Soonchunhyang University Seoul Hospital, Seoul, Korea. 3 Department of Radiology, Daejin Medical Center, Bundang Jesaeng General Hospital, Gyeonggi-do, Korea. 4 Department of Computer Science and Engineering, Seoul National University, Seoul, Korea. 5 Department of Radiology, Seoul National University Bundang Hospital, Seoul National University College of Medicine, Gyeonggi-do, Korea. WEB This is a web exclusive article. Supplemental Data Available online at AJR 2015; 205:W485 W X/15/2055 W485 American Roentgen Ray Society Low-Dose (2-mSv) CT in Adolescents and Young Adults With Suspected Appendicitis: Advantages of Additional Review of Thin Sections Using Multiplanar Sliding-Slab Averaging Technique OBJECTIVE. The purpose of this study is to assess the advantages of additional multiplanar sliding-slab averaging review of 2-mm-thick (thin) sections over stack review of 5-mmthick (thick) sections in difficult cases of 2-mSv CT in adolescents and young adults with suspected appendicitis. MATERIALS AND METHODS. We included 149 patients (mean age, 28.0 years; 61 male patients and 88 female patients) for whom the initial CT reports were inconclusive for the diagnosis of appendicitis. Five independent radiologists retrospectively reviewed the thick sections in the stack mode and then the thin sections using sliding-slab averaging. In each review, they rated the likelihood of appendicitis and the appendix visualization using 5- and 3-point Likert scales, respectively. Diagnostic performance and confidence were compared between the two reviews using ROC analysis, McNemar tests, and Wilcoxon signed-rank tests. RESULTS. The pooled AUCs were 0.90 and 0.93 for the stack and sliding-slab averaging reviews, respectively (90% CI for the difference, ; p = 0.087). For the individual readers, the sliding-slab averaging review tended to increase the AUC (range, for stack vs for sliding-slab averaging review), improve the confidence in diagnosing (mean score, vs ) or ruling out ( vs ) appendicitis, reduce indeterminate interpretations (0 15% vs 0 11%), and enhance the normal appendix visualization ( vs ), although the differences were not always statistically significant. CONCLUSION. Sliding-slab averaging review of thin sections is helpful when the diagnosis of appendicitis is difficult at 2-mSv CT in adolescents and young adults. M ultiplanar sliding-slab averaging is a widely available real-time image postprocessing technique that enables efficient review of large thinsection CT image datasets [1 3]. This technique may be advantageous particularly for low-dose CT. The image noise in the source thin-section images is canceled by averaging the pixel values within a viewing slab; as a result, the final image has better contrast resolution. In addition, slab sliding creates the illusion of continuity from image to image, and the through-plane spatial resolution inherent to the source thin sections can be almost fully preserved in the final displayed images. With such contrast and spatial resolutions, as well as its multiplanar reformation capability, the slidingslab averaging technique can potentially compensate for the low image quality of the lowdose CT source images, particularly when tracing a small tortuous tubular low-contrast structure such as the appendix. Since Lee et al. [4] reported the advantage of the sliding-slab averaging technique compared with conventional stack review in improving the radiologists confidence in diagnosing acute appendicitis, the technique has been used to image the appendix, particularly with reduced-tube-current techniques [5 8]. However, the potential advantages of the sliding-slab averaging technique in low-dose CT have not been validated. In the study by Lee et al. [4], the effective dose of CT was approximately 8 msv, far exceeding the currently accepted dose of low-dose CT. Using conventional stack review, even without sliding-slab averaging, CT diagnosis of appendicitis is unequivocal and very accurate in many cases [9], even using a low-dose technique [6, 7]. Therefore, the patients who could truly benefit from sliding-slab averaging would be the subgroup whose CT diagnosis remains inconclusive with the conventional stack review. W485

2 Lee et al. TABLE 1: Grading Scheme for the Likelihood of Appendicitis and Appendix Visualization Analyzed Finding, Grading Criteria Likelihood of appendicitis Grade 1 Grade 2 Grade 3 Grade 4 Grade 5 Visualization of the appendix Grade 0 Grade 1 Grade 2 The purpose of our study was to assess the advantages of an additional sliding-slab averaging review of 2-mm-thick (thin) sections compared with the stack review of 5-mmthick (thick) sections in difficult cases of 2-mSv CT in adolescents and young adults with suspected appendicitis. Materials and Methods Study Overview The institutional review board approved this retrospective study and waived the requirement for informed consent. The study included 149 patients for whom the initial reports of 2-mSv CT were inconclusive for the diagnosis of appendicitis. Five independent radiologists retrospectively reviewed the 5-mm-thick CT (thick) images in the conventional stack mode and then the 2-mm-thick CT (thin) images using the sliding-slab averaging technique. The advantages of the additional sliding-slab averaging review of the thin sections over the stack review of the thick sections were evaluated regarding the diagnostic performance and confidence. The details of the imaging protocol, CT interpretation criteria, and reference standards for the final diagnosis are available elsewhere [6]. Patient Inclusion The study sample was designed to represent patients for whom the CT diagnosis is difficult to make in a tertiary center in metropolitan Seoul, Korea, where 2-mSv CT is used as the first-line imaging test for patients with suspected appendicitis. Subjects were selected from a previous randomized controlled trial [6] in which 891 adolescents and young adults were randomized to undergo either 2- or 8-mSv CT for suspected appendicitis from September 2009 through January The original trial compared the two radiation doses in terms of negative-appendectomy rate, whereas the Description Definitely absent. Clinical observation is recommended. Probably absent. Clinical observation is recommended. Indeterminate. Clinical observation or surgical exploration is recommended. Probably present. Surgical exploration is recommended. Definitely present. Surgical exploration is recommended. Not identified. Unsure or partly visualized. Clearly and entirely visualized. current study evaluated the advantages of an additional sliding-slab averaging review of the thin sections obtained using low-dose CT. The initial CT reports in the original trial were made by radiologists on service who primarily reviewed the 5-mmthick axial and coronal sections in the stack mode. If the radiologist was not completely confident in this initial interpretation, he or she was encouraged to also review the 2-mm-thick sections using the sliding-slab averaging technique. The likelihood of TABLE 2: Patient Characteristics appendicitis was rated in a 5-point Likert scale (Table 1) in the CT reports. Of the 444 patients assigned to the 2-mSv CT group, 295 were excluded for the following reasons: dropped out of the trial (n = 6), the Likert scale was not recorded in the CT report (n = 5), the initial CT report was conclusive for the likelihood of appendicitis (grade 1 or 5) (n = 281), or the CT image dataset was unavailable for the current study (n = 3). Finally, the remaining 149 patients for whom the radiologists (three attending abdominal radiologists and 31 less-experienced on-call radiologists) were not confident (grades 2 4) in diagnosing or ruling out appendicitis were included (Fig. 1). This sample was deemed to represent a subpopulation in whom the diagnosis of appendicitis is difficult to make at 2-mSv CT. The sample comprised 61 male and 88 female patients aged years (mean [± SD] age, 28.0 ± 8.4 years; male patients, 29.3 ± 8.0 years; female patients, 27.0 ± 8.5 years). Their baseline characteristics were similar to those of the total sample in the original trial (Table 2). CT Image Acquisition IV contrast-enhanced abdomen and pelvis CT scans were obtained using 16-MDCT or higher machines (Table S1, supplemental material, can Characteristic Study Sample (n = 149) Age (y), mean ± SD All patients 28.0 ± 8.4 Female patients 27.0 ± 8.5 Male patients 29.3 ± 8.0 Sex Female 88 (59) Male 61 (41) Body mass index a < 18.5 (underweight) 21 (14) (normal) 103 (69) 25.0 (overweight or obese) 25 (17) Radiologist making initial CT report b Attending abdominal radiologists 50 (34) Less-experienced on-call radiologists 99 (66) Likelihood of appendicitis in initial CT report Grade 2 71 (48) Grade 3 23 (15) Grade 4 55 (37) Note Unless otherwise indicated, data are number (%) of patients. a Weight in kilograms divided by the square of the height in meters. b Significantly more reports were made by on-call radiologists than in the total sample (n = 891) of the original trial. Otherwise, there was no statistically significant difference in the patient characteristics between the current study and the original trial. W486

3 Low-Dose CT in Cases of Suspected Appendicitis 891 Included in the previous trial 444 Underwent 2-mSv CT 152 Initial CT reports were inconclusive (grade 2 4) 149 In the final study sample 82 Surgery 67 No surgery be viewed in the AJR electronic supplement to this article, available at The effective radiation dose was estimated at 2 msv in an anthropomorphic phantom [6, 10]. The tube current was automatically modulated according to the patient s body size and shape. The median doselength product measured on the CT consoles in 447 Underwent 8-mSv CT 6 Dropouts 5 CT reports not mentioning 5-point Likert scale 281 Initial CT reports were conclusive (grade 1 or 5) 3 Image datasets not available 71 Appendicitis 11 Not appendicitis 0 Appendicitis 67 Not appendicitis Fig. 1 Patient flow diagram. Final sample (n = 149) was deemed to represent patients in whom diagnosis of appendicitis is difficult to make at 2-mSv CT. Fig. 2 Heat maps of likelihood of appendicitis in 71 appendicitis cases. For stack (left) and additional sliding-slab averaging reviews (middle), color of grid element at each intersection indicates likelihood of appendicitis rated by corresponding reader (column) for corresponding patient (row). Definitions of each grade are summarized in Table 1. In difference heat map (right), color of each grid element indicates difference between additional sliding-slab averaging review and stack review alone. Although most cases showed no difference (light gray), positive differences (orange) occurred more frequently than negative differences (green), indicating that additional sliding-slab averaging review tended to improve diagnostic confidence in diagnosing appendicitis. Orders of readers (readers 1 5 from left to right) and patients (from top to bottom) are identical in three heat maps. the 149 patients was mgy cm (interquartile range, mgy cm). Two transverse image datasets were reconstructed from each helical scan: 5-mm section thickness with 4-mm interval (thick sections) and 2-mm section thickness with 1-mm interval (thin sections). Other reconstruction parameters such as the FOV and reconstruction filter type were identical in the two image datasets. The technical advantages of this two-tier (thick and thin) image reconstruction have been previously discussed [11]. From the thin sections, coronal images were reformatted with 5-mm section thickness with 4-mm interval. Readers The images were retrospectively reviewed by five radiologists invited from other hospitals, including two radiology residents (readers 1 and 2), two general radiologists (readers 3 and 4 with 1 and 3 years of experience, respectively), and one abdominal radiologist (reader 5 with 11 years of experience). They had various experience levels with low-dose appendiceal CT interpretation: two of them had no experience, two of them had experience with less than 20 cases, and one of them had experience with more than 100 cases. They were aware of the study hypothesis on the potential advantages of the additional sliding-slab averaging review. They were blinded to the initial CT reports and final diagnoses of individual patients; however, they could not be blinded to the review mode (described in the next section) because of the obvious difference in the user interface in image interpretation. Image Review Each of the five independent readers retrospectively reviewed the images of the 149 patients. For each patient, they reviewed the 5-mm-thick transverse and coronal images in the stack mode, which was immediately followed by the review of the 2-mm-thick transverse images using the sliding-slab averaging technique ( AquariusNET, Fig. 3 Heat maps for likelihood of appendicitis in 78 nonappendicitis cases. For stack (left) and additional sliding-slab averaging reviews (middle), color of grid element at each intersection indicates likelihood of appendicitis rated by corresponding reader (column) for corresponding patient (row). Definitions of each grade are summarized in Table 1. In difference heat map (right), color of each grid element indicates difference between additional sliding-slab averaging review and stack review alone. Although most cases showed no difference (light gray), negative differences (green) occurred more frequently than positive differences (orange), indicating that additional sliding-slab averaging review tended to improve diagnostic confidence in ruling out appendicitis. Orders of readers (readers 1 5 from left to right) and patients (from top to bottom) are identical in three heat maps. W487

4 Lee et al. TeraRecon). We chose this reading scheme instead of separate reading sessions for the stack and sliding-slab averaging reviews for the following reasons: first, this reading scheme better reproduced our clinical practice wherein the sliding-slab averaging technique is used as an adjunct to the standard stack review for individual patients; and second, our study aimed to assess the additional value of the sliding-slab averaging, not to compare the stack review with the sliding-slab averaging review alone. The sliding-slab averaging review began with a 5-mm-thick transverse slab, and the readers were allowed to change the slab thickness and the viewing angle during the review, which is a theoretic advantage of the sliding-slab averaging review over the conventional stack review. The patient order was randomized for each reader. After each of the two reviews for each patient, the readers were asked to rate the likelihood of appendicitis and the appendix visualization using 5- and 3-point Likert scales, respectively (Table 1). The grading criteria were identical to those used in making the initial CT reports [6]. Final Diagnosis Appendicitis was pathologically confirmed in 71 patients. The remaining 78 patients were considered as not having appendicitis on the basis of negative pathologic findings from the appendectomy specimen (n = 5), gross surgical findings (n = 6), or clinical follow-up, including telephone interview (n = 67). Statistical Analysis A radiologist and a statistician planned and performed all analyses. The sample size was determined as the number of available cases. Considerations in retrospective sample size estimation are available in the Supplemental Materials. Heat maps [12, 13] were constructed to present the complete dataset of the readers responses (Supplemental Materials). Agreement among the five readers was measured using the Krippendorff alpha statistic [14]. An alpha value close to 1 indicated high agreement, a value close to 0 indicated that the agreement was due to chance, and a value close to 1 indicated that there was systematic disagreement. The overall diagnostic performance for the diagnosis of appendicitis was compared between the two reviews by using multireader multicase ROC analysis [15]. For each reader, the two reviews were compared using the McNemar tests and Wilcoxon signed-rank tests for the following parameters: diagnostic sensitivity and specificity, confidence in diagnosing or ruling out appendicitis, frequency of indeterminate response (grade 3), and visualization of the normal appendix. In calculating the sensitivity and specificity, we used a decision threshold of grade 3 or higher considered as positive. This threshold was based on the fact that appendicitis is truly present in a considerable number of patients whose CT images are interpreted as equivocal [6, 16, 17]. The results of the five readers were pooled by averaging. Data were analyzed using statistical software (imrmc, U.S. Food and Drug Administration Center for Devices and Radiologic Health; and Stata version 13.0, StataCorp). We set the significance level at α = 0.1 instead of the conventional α = 0.05, to minimize the risk of a type II error (discarding the sliding-slab averaging review that truly has an additional value) even at the expense of a higher chance of a type I error (adding useless sliding-slab averaging review). This was because the additional sliding-slab averaging review would never impose any additional risk or cost to patients in CT data acquisition, whereas it would only cast a slight burden on the radiology departmental infrastructure and workload (detailed in the Discussion). The 90% CIs were calculated for the differences [18]. Results Readers Responses and Interobserver Agreement Heat maps were generated for the complete dataset of the five readers responses. The heat maps showed that the additional sliding-slab averaging review, compared with Fig. 4 Heat maps for visualization of normal appendix in 78 nonappendicitis cases. In heat maps for stack (left) and additional sliding-slab averaging reviews (middle), color of grid element at each intersection indicates appendix visualization grade rated by corresponding reader (column) for corresponding patient (row). Definitions of each grade are summarized in Table 1. In difference heat map (right), color of each grid element indicates difference between additional sliding-slab averaging review and stack review alone. Although most cases showed no difference (light gray), positive differences (orange) occurred more frequently than negative differences (green), indicating that additional sliding-slab averaging review tended to improve appendix visualization. Orders of readers (readers 1 5 from left to right) and patients (from top to bottom) are identical in three heat maps. the stack review alone, tended to improve the overall diagnostic confidence in diagnosing (Fig. 2) and ruling out (Fig. 3) appendicitis, and tended to improve the visualization of the normal appendix (Fig. 4). The heat maps did not show any remarkable trend in the readers responses according to the readers experience level. The Krippendorff alpha statistics for the stack and additional sliding-slab averaging reviews were 0.68 and 0.73 for the likelihood of appendicitis, respectively, and 0.40 and 0.26 for the visualization of the normal appendix, respectively. Diagnostic Performance The pooled AUC was 0.90 for the stack review and 0.93 for the additional sliding-slab averaging review (90% CI for the difference, ; p = 0.087). The AUC for the individual readers ranged from 0.86 to 0.93 for the stack review and from 0.87 to 0.97 for the additional sliding-slab averaging review. For all individual readers, the AUC value tended to increase with the additional sliding-slab averaging review compared with the stack review, and the increase was statistically significant for readers 3 (p = 0.012) and 5 (p = 0.009). There was no remarkable trend in the AUC according to the readers experience level. The sensitivity for the individual readers ranged from 80% to 94% for the stack review and from 83% to 96% for the additional sliding-slab averaging review. The sensitivity tended to increase with the additional slidingslab averaging review for readers 2, 3, 4, and 5, but without statistical significance. The specificity for the individual readers ranged from 77% to 86% for the stack review and from 78% to 91% for the additional slidingslab averaging review. The specificity tended to increase with the additional sliding-slab averaging review for readers 1, 3, 4, and 5, but the increase was statistically significant for reader 5 only (p = 0.063; Table S2). W488

5 Low-Dose CT in Cases of Suspected Appendicitis TABLE 3: Diagnostic Confidence Diagnostic Confidence Stack Review TABLE 4: Visualization of the Normal Appendix (n = 78) Reader Additional Sliding-Slab Averaging Review Likelihood of appendicitis a Patients confirmed as having appendicitis (n = 71) Reader ± ± b Reader ± ± Reader ± ± b Reader ± ± b Reader ± ± b Pooled 4.2 ± ± 1.1 Patients confirmed as not having appendicitis (n = 78) Reader ± ± Reader ± ± Reader ± ± b Reader ± ± b Reader ± ± b Pooled 1.9 ± ± 1.1 Indeterminate interpretation c Reader 1 13 (19) 2 (3) < b Reader 2 1 (2) 0 (0) 0.50 Reader 3 15 (23) 11 (16) 0.25 Reader 4 5 (8) 3 (4) 0.34 Reader 5 0 (0) 1 (2) 0.50 Pooled 7 (52) 3 (25) a Data are mean ± SD scores for the likelihood of appendicitis. Statistically significant at the significance level of α = 0.1. c Data are percentage (no.) of patients rated as grade 3 among the 149 patients. p Stack Review Diagnostic Confidence In diagnosing appendicitis, the five readers pooled mean confidence score for the likelihood of appendicitis in the 71 appendicitis cases was 4.2 and 4.4 for the stack and additional sliding-slab averaging reviews, respectively. The individual readers mean confidence score ranged from 3.6 to 4.7 for the stack review and from 3.9 to 4.7 for the additional sliding-slab averaging review. The mean confidence score was 4.0 or greater for both reviews in readers 1, 2, 4, and 5. The confidence score statistically significantly increased with the additional sliding-slab averaging review for readers 1 (p = 0.080), 3 (p = 0.037), 4 (p = 0.054), and 5 (p = 0.083) (see Video S3 and Fig. S4). In ruling out appendicitis, the five readers pooled mean confidence score for the likelihood of appendicitis in the 78 nonappendicitis cases was 1.9 and 1.8 for the stack and additional sliding-slab averaging reviews, respectively. The individual readers mean confidence score ranged from 1.6 to 2.1 for the stack review and from 1.5 to 1.9 for the additional sliding-slab averaging review. The mean confidence score was 2.1 or less for both reviews for all readers. The confidence score statistically significantly decreased (i.e., the diagnostic confidence was statistically significantly improved) with the additional sliding-slab averaging review for readers 3 (p = 0.071), 4 (p = 0.001), and 5 (p = 0.011). The five readers pooled percentage of an indeterminate interpretation (grade 3) was 7% and 3% for the stack and additional sliding-slab averaging reviews, respectively. The percentage of indeterminate interpretation by the individual readers ranged from 0% to 15% for the stack review and from 0% to 11% for the additional sliding-slab averaging review. The percentage tended to decrease with the additional sliding-slab averaging review for readers 1, 2, 3, and 4, but the decrease was statistically significant for reader 1 only (p < 0.001; Table 3). Visualization of the Normal Appendix The five readers pooled mean score for the normal appendix visualization in the 78 nonappendicitis cases was 1.5 and 1.6 for the stack and additional sliding-slab averaging reviews, respectively. The individual readers mean score ranged from 1.1 to 1.7 for the stack review and from 1.1 to 1.9 for the additional sliding-slab averaging review. The visualization score statistically significantly increased with the additional sliding-slab averaging review for readers 1 (p = 0.084), 2 (p = 0.002), and 3 (p < 0.001; Table 4). Discussion We assessed the advantages of the additional sliding-slab averaging review of thinsection images over the conventional stack review of the thick-section images alone in 149 patients for whom the initial 2-mSv CT reports were inconclusive for the diagnosis of appendicitis. The heat maps showed the advantages of the additional sliding-slab averaging review in the diagnostic performance, diagnostic confidence, and normal appendix visualization. The use of the additional sliding-slab averaging review increased the Additional Sliding-Slab Averaging Review Reader ± ± a Reader ± ± a Reader ± ± 0.4 < a Reader ± ± Reader ± ± Pooled 1.5 ± ± 0.6 Note Data are mean ± SD of the visualization score. a Statistically significant at the significance level of α = 0.1. p W489

6 Lee et al. pooled AUC. For individual readers, the differences between the two reviews in the diagnostic performance or confidence were often small, and some readers showed statistical significance between the two reviews but the others did not. The small differences may be attributed to the already high diagnostic performance and confidence of the stack review, as shown in our results; therefore, there may have been only small room for improvement by the additional sliding-slab averaging review. With the additional sliding-slab averaging review, the interobserver agreement increased for the likelihood of appendicitis but decreased for the visualization of the normal appendix. This indicates that the extent of the advantage of the additional sliding-slab averaging review was not uniform across the five readers. Overall, our data suggest that the additional sliding-slab averaging review can marginally improve the diagnostic performance and confidence when the diagnosis of appendicitis is difficult to make at 2-mSv CT. We used α = 0.1 instead of the conventional α = 0.05, which could be justified as follows. As we discussed already, we intended to minimize the risk of a type II error (discarding the sliding-slab averaging review that truly has an additional value) even at the expense of a higher chance of a type I error (adding useless sliding-slab averaging review). Although the additional sliding-slab averaging review would never impose any additional patient risk or cost, the only practical prerequisite for the additional sliding-slab averaging review would be radiologists additional efforts in implementing the slidingslab averaging technique. Many radiologists have been already reconstructing additional thin sections for coronal reformation [19, 20], and many departments already have the infrastructure necessary to accommodate the thin sections [11]. Because the sliding-slab averaging is one of the most fundamental image postprocessing techniques, it is expected to be increasingly available in standard PACS in the near future. Once the infrastructure is established, the average reading time required for the additional sliding-slab averaging review for the diagnosis of appendicitis was measured as only 1 minute for attending radiologists participating in a training course of 2-mSv CT interpretation [21]. From our results, we are not claiming that the additional sliding-slab averaging review should be routinely used in all patients undergoing low-dose CT for suspected appendicitis. The current study included patients for whom the initial CT reports were inconclusive (grades 2 4) for the diagnosis of appendicitis. This was based on the premise that the additional sliding-slab averaging review would most benefit such a subgroup among the population undergoing preoperative CT under the suspicion of appendicitis. Interestingly, the readers in the current study conclusively diagnosed (grade 5) or ruled out (grade 1) appendicitis in many of the cases in the stack mode even without using the sliding-slab averaging review. Such shifts in diagnostic confidence can be explained by several factors besides interobserver variations. First, the readers in the current study likely experienced a learning curve during or even before the current study, which we did not measure. At a minimum, they had likely been exposed to the promising study results on low-dose CT [6, 7] or to illustrative lowdose CT cases in educational conferences. Three of the readers already had prior experience in interpreting low-dose appendiceal CT. Second, in contrast to the original trial [6], where many radiologists had prospectively made the initial CT reports as a part of their daily practice, each of the five readers in the current study retrospectively reviewed all 149 cases in a laboratory environment. Our study has several limitations. First, our reading scheme may have been biased toward overestimating the advantages of the sliding-slab averaging review. The readers reviewed each case in the order of stack mode and then the sliding-slab averaging mode, and they were aware of the study hypothesis. This may have biased the reviewers to assign higher diagnostic confidence in the additional sliding-slab averaging review than in the stack review. A more robust reading scheme would be to have one reading session of the stack review and the other session of sequential stack and sliding-slab averaging reviews with a sufficient interval between the two sessions, and to randomize the order of the two sessions for each case. Second, conversely, the study sample may have been biased toward underestimating the advantages of the sliding-slab averaging review. When making the initial reports in the original trial [6], the radiologists were encouraged to use the sliding-slab averaging technique when they were not completely confident in their stack review. Therefore, our study sample may have included some cases interpreted as inconclusive even after the additional sliding-slab averaging review, whereas other cases that could truly benefit from slidingslab averaging may have failed to enter the study sample. Third, our sample size was not large enough to show statistical significance for all readers. Again, this was mainly because the diagnostic performance and confidence of the stack review was already fairly high, leaving only small room for improvement by the additional sliding-slab averaging review. Fourth, few of our patients were obese. Further investigation is needed to verify the generalizability of our results to patients with larger body sizes. Weighing the observed potential advantages of the additional sliding-slab averaging review of thin sections against the expected associated burden on a radiology department, we conclude that it is worthwhile to add the sliding-slab averaging review of thin sections when the diagnosis of appendicitis is difficult at 2-mSv CT. Acknowledgments We thank Sung Min Lee (Department of Electrical Engineering and Computer Science, Seoul National University) for his help in data preparation and the Medical Research Collaborating Center at Seoul National University Bundang Hospital for statistical analyses. References 1. Kim B, Lee KH, Kim KJ, Mantiuk R, Kim HR, Kim YH. Artifacts in slab average-intensity-projection images reformatted from JPEG 2000 compressed thin-section abdominal CT data sets. AJR 2008; 190:[web]W342 W Lee KH, Hong H, Hahn S, Kim B, Kim KJ, Kim YH. Summation or axial slab average intensity projection of abdominal thin-section CT datasets: can they substitute for the primary reconstruction from raw projection data? J Digit Imaging 2008; 21: von Falck C, Galanski M, Shin HO. Informatics in radiology: sliding-thin-slab averaging for improved depiction of low-contrast lesions with radiation dose savings at thin-section CT. RadioGraphics 2010; 30: Lee KH, Kim YH, Hahn S, et al. Computed tomography diagnosis of acute appendicitis: advantages of reviewing thin-section datasets using sliding slab average intensity projection technique. Invest Radiol 2006; 41: Joo SM, Lee KH, Kim YH, et al. Detection of the normal appendix with low-dose unenhanced CT: use of the sliding slab averaging technique. Radiology 2009; 251: Kim K, Kim YH, Kim SY, et al. Low-dose abdominal CT for evaluating suspected appendici- W490

7 Low-Dose CT in Cases of Suspected Appendicitis tis. N Engl J Med 2012; 366: Kim SY, Lee KH, Kim K, et al. Acute appendicitis in young adults: low- versus standard-radiation-dose contrast-enhanced abdominal CT for diagnosis. Radiology 2011; 260: Seo H, Lee KH, Kim HJ, et al. Diagnosis of acute appendicitis with sliding slab ray-sum interpretation of low-dose unenhanced CT and standarddose intravenous contrast-enhanced CT scans. AJR 2009; 193: Paulson EK, Kalady MF, Pappas TN. Clinical practice: suspected appendicitis. N Engl J Med 2003; 348: Huda W, Mettler FA. Volume CT dose index and dose-length product displayed during CT: what good are they? Radiology 2011; 258: Lee KH, Lee HJ, Kim JH, et al. Managing the CT data explosion: initial experiences of archiving volumetric datasets in a mini-pacs. J Digit Imaging 2005; 18: Lee H, Kim B, Kim KJ, et al. Introduction of heat map to fidelity assessment of compressed CT images. Med Phys 2011; 38: Seong NJ, Kim B, Lee S, et al. Off-site smartphone reading of CT images for patients with inconclusive diagnoses of appendicitis from on-call radiologists. AJR 2014; 203: Krippendorff K. Computing Krippendorff s alpha-reliability. Annenberg School for Communication, University of Pennsylvania website. www. asc.upenn.edu/usr/krippendorff/mwebreliability5.pdf. Published January 25, Updated September 13, Accessed May 10, Dorfman DD, Berbaum KS, Metz CE. Receiver operating characteristic rating analysis: generalization to the population of readers and patients with the jackknife method. Invest Radiol 1992; 27: Ahn S; LOCAT Group. LOCAT (low-dose computed tomography for appendicitis trial) comparing clinical outcomes following low- vs standarddose computed tomography as the first-line FOR YOUR INFORMATION The data supplement accompanying this web exclusive article can be viewed by clicking Supplemental at the top of the article. imaging test in adolescents and young adults with suspected acute appendicitis: study protocol for a randomized controlled trial. Trials 2014; 15: Daly CP, Cohan RH, Francis IR, Caoili EM, Ellis JH, Nan B. Incidence of acute appendicitis in patients with equivocal CT findings. AJR 2005; 184: Newcombe RG. Improved confidence intervals for the difference between binomial proportions based on paired data. Stat Med 1998; 17: Paulson EK, Harris JP, Jaffe TA, Haugan PA, Nelson RC. Acute appendicitis: added diagnostic value of coronal reformations from isotropic voxels at multidetector row CT. Radiology 2005; 235: Park JH; LOCAT Group. Diagnostic imaging utilization in cases of acute appendicitis: multi-center experience. J Korean Med Sci 2014; 29: Yang HK, Ko Y, Lee MH, et al. Initial performance of radiologists and radiology residents in interpreting low-dose (2-mSv) appendiceal CT. AJR (in press) W491