UNDERSTANDING THE CHALLENGES IN THE TRANSITION FROM FILM TO DIGITAL RADIOGRAPHY IN THE NUCLEAR POWER INDUSTRY

Transcription

1 UNDERSTANDING THE CHALLENGES IN THE TRANSITION FROM FILM TO DIGITAL RADIOGRAPHY IN THE NUCLEAR POWER INDUSTRY RM Meyer *, P Ramuhalli *, TL Moran *, CA Nove, and AF Pardini * U.S. Nuclear Regulatory Commission, Rockville, MD, USA * Pacific Northwest National Laboratory, Richland, WA, USA ABSTRACT The use of film radiography for nondestructive examination (NDE) applications in the nuclear power industry is diminishing due to the advent of modern digital imaging technologies. Technologies that are used routinely in the medical industry for patient diagnosis are being adapted to industrial NDE applications, including the detection and characterization of defects in welds. From the user perspective, non-film inspection techniques provide several advantages over film techniques. It is anticipated that the shift away from the application of film radiography in the nuclear power industry represents an irreversible trend. The U.S. Nuclear Regulatory Commission (NRC) has noted this trend in the U.S. nuclear power industry and will be working to ensure that the effectiveness and reliability of component inspections is not compromised by this transition. Currently, specific concerns are associated with 1) obtaining a fundamental understanding of how inspection effectiveness and reliability may be impacted by this transition and 2) ensuring training standards and qualifications remain compatible with modern industrial radiographic practice. This paper discusses recent trends in industrial radiography and assesses their advantages and disadvantages from the perspective of nuclear power plant component inspections. INTRODUCTION The use of film radiography for nondestructive examination (NDE) applications in the nuclear power industry is diminishing due to the advent of modern digital imaging technologies. Technologies that are used routinely in the medical industry for patient diagnoses are being adapted to industrial NDE applications including the detection and characterization of defects in welds. From the user perspective, non-film inspection techniques provide several advantages over film techniques including the elimination of film costs (which are rising) and costs associated with film handling, processing, and long-term storage. Other advantages associated with non-film radiography techniques include increased image contrast dynamic range and better linearity, digital image enhancement and manipulation capabilities, and lower radiation exposure requirements for image generation. Based on the advantages just cited, it is anticipated that the shift away from the application of film radiography in the nuclear power industry will continue. The NRC is interested in obtaining a more fundamental understanding of how radiographic inspection effectiveness and reliability are impacted by factors that differentiate film and digital radiographic techniques, and to ensuring that training standards and qualifications of NDE personnel in the nuclear industry are compatible with modern industrial radiographic practice. RECENT TRENDS IN INDUSTRIAL RADIOGRAPHY An overview of industrial radiography trends is provided in [1]. Filmless radiographic testing (RT) techniques, such as fluoroscopes and direct imaging methods, have been investigated and used since at least the late 1980s [2]. Recent advances in solid-state technology have led to the development of highly sensitive digital detectors for radiography, leading to what is commonly called digital RT (DRT) [3]. Interest in DRT for industrial applications seems to be increasing world-wide, with organizations such as the International Atomic Energy Agency (IAEA) supporting Coordinated Research Programs (CRP) in this area [4]. In the United States, progress in DRT is being monitored by the Federal Working Group on Industrial Digital Radiography ( The key to DRT is to find a way of directly recording the RT image on a computer (or other digital media). DRT first made its appearance in 478

2 medical diagnostics. Until recently, limitations on detector size and image quality have hindered the adoption of these detectors in industrial NDE. DRT is a generic term encompassing several filmless detector technologies. DRT is to be distinguished from digitized radiography, which consists of the digitization of exposed film. DRT currently uses one of the following types of detectors [5]: Storage phosphor plates (so-called computed radiography or CR ). Indirect digital detectors (scintillation detectors or scintillators). Direct digital detector arrays (DDA flat panels). The remainder of this paper covers only standard radiographic techniques that acquire images from a single view (i.e., single angle of incidence of the x-radiation). Multiview approaches, such as computed tomography, are rarely used for nuclear power plant (NPP) inspections, although a multi-view device has recently been qualified by European standards for weld inspections in NPPs [6]. These techniques may be used with greater prevalence in the future; but regardless, a greater understanding of single-view techniques is needed initially, to assess current industry practice and to benefit future assessments of multi-view techniques. FILM-TO-DIGITAL TRANSITION UNDERSTANDING THE IMPACT ON INSPECTION RELIABILITY In NDE, performance is often specified in the form of probability of detection (POD), probability of false calls (PFC), and/or probability of rejection (POR). POD or POR curves may be used to evaluate individual systems. Additionally, plotting POD vs. PFC gives the receiver operating characteristic (ROC) curve that demonstrates the tradeoff between increased detection rates and increased false-call rates. The ROC method has been used for quantifiable training and evaluation of medical radiologists [7]. In. radiography, the ROC curve will depend in a holistic way on all the parameters that describe the radiographic system and the ways these parameters are related to and interact with each another. Performance Aspects Affected by Transition from Film to Digital Techniques The reliability of a radiographic inspection will depend on the quality of the radiographic image and on the quality of the analysis of that image. In turn, both the image quality and the quality of image analysis depend on the quality of the equipment, procedures, and personnel involved in the radiographic system. This relationship is illustrated in Figure 1. The quality of an image may be quantitatively described in terms of the image contrast, sensitivity, and signal-to-noise ratio (SNR). 479

3 Figure 1 Illustration of performance aspects impacted by the film-to-digital transition and potentially affecting the reliability of radiographic examinations Performance Parameter Comparison Film and Digital RT Systems The impact of the film to digital transition in radiography depends on several performance factors that can be ascertained from a review of the literature. A summary of key literature findings is provided in Table 1 [1]. There are several disadvantages to using digital detectors in terms of image quality, including inferior native contrast sensitivity, inferior modulation transfer function (MTF), and inferior limiting spatial resolution (LSR). Other disadvantages in current inherent digital detection equipment include poorer basic spatial resolution (SR b ) and lower dose tolerance than film media. However, the storage of radiographic images digitally allows for greater flexibility in terms of image processing, image enhancement, and image interpretation. Sophisticated filters can be applied to reduce noise and enhance radiographic images overall. The use of post-processing (such as high-pass filtering [8]) can further improve contrast sensitivity and detection reliability. Additionally, detector limitations with respect to exposure tolerance can potentially be compensated for by digitally combining several exposures. Further, digital storage of radiographic images allows for the application of image analysis algorithms to automate flaw detection and sizing. It is possible that these advantages of digital radiography can compensate for poorer native image quality. This is partly reflected in the final row of 480

4 Table 1, which indicates that some studies show that radiography performed with DDAs provides better crack-detection sensitivity than film. Table 1 Summary of factors impacting radiographic detection reliability [1] Issues Impacting Reliability Key Findings: CR Key Findings: Digital Detectors Relevant References Dynamic Range Higher than film Higher than film [9] SR b Lower resolution than Lower resolution than [5, 10-16] film CR MTF and LSR LSR less than that of film (MTF is narrower than that of film), implying lower contrast sensitivity. LSR less than that of film (MTF is narrower). This implies lower contrast [5, 10-13, 16, 17] Contrast Sensitivity Dose and Dose Rate, Exposure Setting Thickness of Specimens Generally lower than that of film. Among CR detectors, higher resolution (HD-CR) has better contrast sensitivity under similar exposure conditions. Smaller changes in contrast sensitivity for varying thickness, due to larger dynamic range. Lower dose rates required than film. Under identical conditions, film has higher sensitivity leading to higher quality (Class B). Limitations in dose rates and resolution may not permit applicability of CR to thick specimens. (a) sensitivity. Generally lower than that of film (under identical conditions). Higher resolution provides better contrast sensitivity under similar exposure conditions. Smaller changes in contrast sensitivity for varying thickness, due to larger dynamic range. Lower dose rates than film. For the same dose rates, DRT using digital detectors had better detectability than film. Limitations in dose rates and resolution may not permit applicability of DRT using digital detectors for thick specimens. (a) [16-18] [14-16] [9, 14-16] 481

5 Issues Impacting Reliability Increased Exposure Time Front Lead Screens Post Processing Geometric Unsharpness Projection Magnification (PM) Microfocus Tubes Manufacturing Variabilities Key Findings: CR Can saturate detector. Structure noise becomes apparent at longer exposures. Decrease contrast sensitivity and degrade image quality Limited by code to twice the basic spatial resolution (EN 444, EN 462-5, ASTM-E 2002). This will limit maximum projection magnification. Can improve detectability. However, too high a value of PM increases geometric unsharpness and reduces detectability. Can improve detection sensitivity to cracks, when used with projection magnification Differences in detector sensitivity, SNR and CNR Key Findings: Relevant Digital Detectors References Can saturate detector. [15] However, can improve SNR by resetting sensors periodically, and averaging over multiple images. [14-16] Improved sensitivity (by about 1 2 wire pair IQI) Limited by code to twice the basic spatial resolution (EN 444, EN 462-5, ASTM-E 2002). This will limit maximum projection magnification. Can improve detectability. However, too high a value of PM increases geometric unsharpness and reduces detectability. Can improve detection sensitivity to cracks, when used with projection magnification. [8] [9] [15, 19-21] [15, 16, 19-21] Calibration Necessary to correct for pixel-pixel variations. Detection Poorer than film (on Better than film (on [16] Sensitivity for limited set of flaws) limited set of flaws). Cracks Higher spatial resolution (smaller pixel pitch) gives better detection sensitivity. (a)maximum wall thicknesses described in the published literature are typically tens of millimeters, though there appears to be no restriction on the maximum wall thickness that may be radiographed with DRT detectors provided the right source with adequate energy is used to fully penetrate the part. [22] [15, 16] The information in the table should be interpreted with caution. Almost all of the studies cited above draw conclusions based on a limited set of flaws, and the conclusions reached in these studies may not be applicable to inspection scenarios in the nuclear industry. Moreover, the experimental conditions 482

6 (such as detectors, source strengths, specimen thicknesses, etc.) are not the same across all of these studies. With these caveats, some general trends may be observed. First, detection sensitivity in DRT is a complex function of several variables. Understanding how each of these variables impacts the sensitivity is important for obtaining high-quality radiographic images. Characterization of the detector response, either experimentally or through simulation [23], may be necessary to ensure that appropriate calibration (such as the multi-gain calibration procedure [16]) can be performed to address any pixel-to-pixel or detector-to-detector variability. On the other hand, current CR technology does not appear to match the detection sensitivity of film RT, and further developments in this technology may be necessary. The Electric Power Research Institute (EPRI) conducted a multi-year program called Evaluation of Filmless Radiography to better understand the capabilities of digital radiographic inspections using both CR and DR techniques. This program aimed to assess the capability of CR and DR for detection of cracks and SCC flaws in piping welds, considered examinations of water filled components using DRT techniques, and evaluated oriented slotted-bar reference blocks as a method to measure the tolerance of a DRT technique to crack orientation misalignment. These studies resulted in the accumulation of a significant set of data useful for assessment of inspection performance using DRT techniques [24-26]. CODES, STANDARDS AND GUIDELINES Standards and Regulatory Guides Relevant to Digital Radiography Equipment and Practice A significant need for standardization arises in the field of digital radiography because of the nature of technology progression and also because of the large number of vendors of digital radiography technology. The development of a methodology to transition from film to CR while meeting regulatory requirements has been the focus of a recent Edison Welding Institute (EWI) effort. As part of this effort, a review of CR codes and standards was performed [27]. ASTM International has published several standards related to digital radiography equipment and standard practices for usage of digital radiography equipment. Several ASTM standards related to both CR imaging plate technologies and DDAs are tabulated in Table 2. Table 2 List of ASTM standards relevant to digital radiography using CR and DDA technologies Standard ASTM E1475 ASTM E2339 ASTM E2699 ASTM E2738 ASTM E2597 ASTM E2698 ASTM E2736 ASTM E2737 Title Standard Guide for Data Fields for Computerized Transfer of Digital Radiological Examination Data Standard Practice for Digital Imaging and Communication in Nondestructive Evaluation (DICONDE) Standard Practice for Digital Imaging and Communication in Nondestructive Evaluation (DICONDE) for Digital Radiographic (DR) Test Methods Standard Practice for Digital Imaging and Communication Nondestructive Evaluation (DICONDE) for Computed Radiography (CR) Test Methods Standard Practice for Manufacturing Characterization of Digital Detector Arrays Standard Practice for Radiological Examination Using Digital Detector Arrays Standard Guide for Digital Detector Array Radiology Standard Practice for Digital Detector Array Performance Evaluation 483

7 ASTM E2007 ASTM E2033 ASTM E2445 ASTM E2446 & Long Term Stability Standard Guide for Computed Radiography Standard Practice for Computed Radiology (Photostimulable Luminescence Method) Standard Practice for Qualification and Long-Term Stability of Computed Radiology Systems Standard Practice for Classification of Computed Radiology Systems In addition to the standards in Table 2, codes have been developed in the United States and Europe (EN 14784, and Mandatory Appendix VIII of Section V, Article 2, ASME Boiler and Pressure Vessel Code [BPVC]) for weld inspection using CR. The European codes split the testing into two classes (A and B), with differing quality requirements. Requirements on contrast sensitivity, normalized signal-to-noise ratio (nsnr), and SR b are specified, based on such parameters as the source energy and specimen density/thickness. The ASME Code specifies contrast sensitivity only through the use of IQIs. Mandatory Appendix IX of Section V, Article 2, ASME BPVC includes requirements for performing examinations using DR. The specification for image quality assessment is through the use of IQIs. The International Organization for Standardization (ISO) has published the first international standard for digital radiographic inspection of pressurized welded piping in gas and oil industries ISO/DIS [28]. This code specifies the use of IQIs for assessing the image quality. The Federal Working Group on Industrial Digital Radiography (FWGIDR) has published white papers dealing with recommended guidelines for qualification of digital radiography systems and processes, based on currently available codes and standards, recommended guidelines for standardization and management of digital data, and guidelines for the conformance and verification of DICONDE data [29-31]. PERSONNEL - TRAINING AND QUALIFICATION Digital radiography procedures are sufficiently different from film radiography procedures that individuals qualified for film RT are not necessarily competent in digital RT procedures. Among the major differences are the digitization of images and the handling of digital imaging media. Individuals qualified for film RT can lack fundamental knowledge regarding digital representations of information and digital image processing. They may also lack an understanding of digital detectors and storage media. The FWGIDR has recognized the need to standardize the qualifications of NDT personnel who perform DRT examinations and has worked on the development of supplemental training requirements for individuals who perform DRT examinations. The FWGIDR has published white papers dealing with guidelines for qualifications of Level I, II, and III digital radiography personnel [32-34]. Performance-based Criteria for Film and Digital Radiography Traditionally, RT inspections have been performed to workmanship standards that have focused on detecting and identifying volumetric inclusions in construction and fabrication welds [1]. The detection performance of RT is due to a combination of the instrumentation settings (source, detector, exposure time, etc.) and operator performance. In the case of film RT, film processing also plays an important role in detection performance [35]. RT is not generally used for inservice inspections due to its sensitivity to the orientation of planar flaws, which limits its effectiveness and reliability for detection of cracks and its effectiveness for depth sizing of planar flaws. As the industry moves towards the adoption of fitness-forpurpose standards for construction and fabrication inspections, planar flaw detection and depth sizing is of greater importance. This raises questions with regard to the ability of RT to meet construction and fabrication inspection requirements of this alternative philosophy, independently of whether film or digital 484

8 radiography is used. Few round-robin studies exist that explicitly separate the effect of each of these contributors to the final result. There have been a number of studies on NDE reliability in general [36], and on human factors in NDE reliability in particular [37-39]. NDE reliability is a function of several variables, including the socalled application variables and human factors. All of these variables need to be controlled sufficiently if the reliability of the technique is to be improved [37-40]. A lack of adequate training for operators in DRT principles is likely to play a decisive role in the performance of individual inspectors, though there are few studies to address this. Training for NDT personnel has been identified as an important issue impacting the transition to DRT [32-34]. This indicates that digital RT is very skill-dependent and that there may be a need to develop and implement a performance demonstration process to ensure that digital RT inspections are effective at reliably detecting and accurately sizing the target flaw types and sizes. CONCLUSIONS Studies indicate that the performance of DRT techniques (using DDAs) may be able to equal or exceed that of film procedures with respect to sensitivity and POD for detection of cracks [16]. Despite generally poorer native image quality obtained with digital equipment versus film, the ability to digitally manipulate images may be able to compensate for this. However, such studies have been conducted on a limited number of specimens not necessarily relevant to inspection scenarios of the nuclear industry. Evidence also highlights that DRT techniques are very skill-dependent and that training is needed to ensure the competency of personnel performing DRT examinations. At the moment, no comprehensive studies have been performed to determine the effectiveness and reliability of DRT techniques for inspections in the nuclear power industry and to understand the separate impacts of equipment, procedure, and personnel on overall performance. As a consequence, the adequacy of current standards and training requirements for DRT inspections is difficult to judge. The implications of the film-to-digital RT transition on inspection performance for both volumetric and planar flaws in the nuclear industry needs to be individually assessed. The qualification of digital RT inspections in lieu of film RT will need to be performed for each type of flaw. Obtaining a more fundamental understanding of how the multiple factors of DRT examinations impact overall performance will benefit modeling of digital RT performance and planning of potential round-robin studies to enable demonstrating the need for the creation of compatible standards and qualification processes. ACKNOWLEDGEMENTS The work was sponsored by the U.S. Nuclear Regulatory Commission (NRC) through project number JCN V6097. REFERENCE 1) Moran, T L, Ramuhalli, P, Pardini, A F, Anderson, M T, and Doctor, S R, Replacement of Radiography with Ultrasonics for the Nondestructive Inspection of Welds - Evaluation of Technical Gaps - An Interim Report, PNNL-19086, Richland, Washington: Pacific Northwest National Laboratory; ) Fletcher, M J, "Fully Automatic Inspection of Welds Now a Reality," Welding and Metal Fabrication, (2) ) Klyuev, V V, and Sosnin, F R, "The Current State of Digital X-ray Technology," Russian Journal of Nondestructive Testing, (4) ) Ewert, U, Zscherpel, U, Fratzscher, D, and Alekseychuk, O, "Optimization of Digital Industrial Radiography (DIR) Techniques for Specific Applications: An IAEA Coordinated Research Project," 4th Pan American Conference for NDT (PANNDT 2007), Buenos Aires, ) Mishra, D, Venkatachalam, R, and Manoharan, V, 2007, "Digital Radiography for Non-destructive 485

9 Testing," Computerized Tomography for Scientists and Engineers, CRC Press, Boca Raton, Florida p ) Ewert, U, and Redmer, B, "Lecture: Radiographic In-Service Inspection in Nuclear Power Industry and Mobile Tomographic Cross Sectioning of Weldments," Conferência de Tecnologia (Conference on Equipment Technology), Salvador, Brazil, ) Swets, J A, and Pickett, R M, Evaluation of Diagnostic Systems: Methods from Signal Detection Theory, New York, Academic Press, ) Moreira, E V, and Simeies, H R, "Flat-Panel Detectors are Accepted for Digital Radiography in Place of Conventional Radiography in Pipeline Weld Inspection," 4th Pan American Conference for NDT (PANNDT 2007), Buenos Aires, Argentina, Available at 9) Ewert, U, Zscherpel, U, and Bavendiek, K, "Replacement of Film Radiography by Digital Techniques and Enhancement of Image Quality," Annual Conference of Indian NDT Society, Kalkutta, ) Mohr, G A, and Bueno, C, "GE a-si Flat-Panel Detector Performance in Industrial Digital Radiography," Insight (UK), (10) ) Spartiotis, K, Pyyhtia, J, Schulman, T, Puhakka, K, and Muukkonen, K, "A Directly Converting High- Resolution Intra-Oral X-ray Imaging Sensor," Nuclear Instruments and Methods in Physics Research A, ) Spartiotis, K, Havulinna, J, Leppanen, A, Pantsar, T, Puhakka, K, Pyyhtia, J, and Schulman, T, "A CdTe Real Time X-ray Imaging Sensor and System," Nuclear Instruments and Methods in Physics Research A, ) Spartiotis, K, Leppanen, A, Pantsar, T, Pyyhtia, J, Laukka, P, Muukkonen, K, Mannisto, O, Kinnari, J, and Schulman, T, "A Photon Counting CdTe Gamma- and X-ray Camera," Nuclear Instruments and Methods in Physics Research A, ) Beckmann, J, Rachny, S, Zscherpel, U, and Ewert, U, "Improved Procedure for Computed Radiography - A Comparative Study on Welded Tube Sections by Film Radiography and Computed Radiography," 17th World Conference on Nondestructive Testing, Shanghai, China, ) Ewert, U, Zscherpel, U, and Bavendiek, K, "Strategies for Film Replacement in Radiography - Films and Digital Detectors in Comparison," 17th World Conference on Nondestructive Testing, Shanghai, China, ) [16] Marinho, C A, Lopes, R T, and Rebello, J M A, "Film Replacement by Digital Techniques in Weld Testing," Mater Eval, (5) ) Howard, B D, "Lens-Coupled, Area-Detector Digital Radiography versus Conventional Radiography of Fusion Bonded Pinch Welds: Are They Equivalent?," Digital Imaging IV Topical Conference, Mashantucket, Connecticut, ) Lu, Z F, Nickoloff, E L, So, J C, and Dutta, A K, "Comparison of Computed Radiography and Film/Screen Combination Using a Contrast-Detail Phantom," Journal of Applied Clinical Medical Physics, (1) ) Ewert, U, "Upheaval In Industrial Radiology," Journal of Nondestructive Testing, (12) ) Blakeley, B, and Spartiotis, K, "Digital Radiography for the Inspection of Small Defects," Insight- Non-Destructive Testing and Condition Monitoring, (2) ) Jaenisch, G R, Bellon, C, and Ewert, U, "Reliability Assessment by Simulation for High Energy Radiography," 33rd Annual Review of Progress in Quantitative Nondestructive Evaluation, Volume 26, Portland, Oregon, American Institute of Physics, Melville, New York, ) Charnock, P, Connolly, P A, Hughes, D, and Moores, B M, "Evaluation and Testing of Computed Radiography Systems," Radiation Protection Dosimetry, (1-3) ) Jensen, T, and Xu, J, "Computer Modeling of Digital X-ray Detector Response," Review of Quantitative Nondestructive Evaluation, Volume 23, Green Bay, Wisconsin, American Institute of 486

10 Physics, Melville, New York, ) EPRI, Nondestructive Evaluation: Evaluation of Filmless Radiography, 2008 Update, Report No , Palo Alto, California: Electric Power Research Institute (EPRI); ) EPRI, Nondestructive Evaluation: Evaluation of Filmless Radiography, 2009 Update, Report No , Palo Alto, California: Electric Power Research Institute (EPRI); ) EPRI, Nondestructive Evaluation: Evaluation of Filmless Radiography, 2010 Update, Report No , Palo Alto, California: Electric Power Research Institute (EPRI); ) Clear, K, and Lozev, M, Developing and Validating a Methodology for Transitioning from Film to Digital Computed Radiography, EWI Cooperative Research Program Report, Report No. MR1011, Columbus, Ohio: Edison Welding Institute (EWI); ) Moreira, E V, Rabello, J M B, Pereira, M D S, Lopes, R T, and Zscherpel, U, "Digital Radiography Using Digital Detector Arrays Fulfills Critical Applications for Offshore Pipelines," Journal on Advances in Signal Processing, Article ) FWGIDR, Guide for the Qualification of Digital Radiography Systems and Processes, Rev. 1, Paper No : Federal Working Group on Industrial Digital Radiography (FWGIDR); ) FWGIDR, Guide for the Standardization and Management of Digital NDT Data, Paper No : Federal Working Group on Industrial Digital Radiography (FWGIDR); ) FWGIDR, Digital Imaging and Communication in Nondestructive Evaluation (DICONDE) Conformance and Verification, Paper No : Federal Working Group on Industrial Digital Radiography (FWGIDR); ) FWGIDR, Recommended Training Curriculum for Digital Radiography Personnel (Level I), Paper No : Federal Working Group on Industrial Digital Radiography (FWGIDR); ) FWGIDR, Recommended Training Curriculum for Digital Radiography Personnel (Level II), Paper No : Federal Working Group on Industrial Digital Radiography (FWGIDR); ) FWGIDR, Recommended Training Curriculum for Digital Radiography Personnel (Level III), Paper No : Federal Working Group on Industrial Digital Radiography (FWGIDR); ) Lapides, M E, Radiographic Detection of Intergranular Stress Corrosion Cracking: analysis, Qualification, and Field Testing, Report NP-3164-SR, Palo Alto, California: Electric Power Research Institute; ) Singh, R, Three Decades of NDE Reliability Assessment, Karta , San Antonio, Texas: Karta Technologies, Inc.; ) Fucsok, F, "Experiences and Problems of a Manual Ultrasonic Round Robin Test," 7th European Conference on Non-destructive Testing, Copenhagen, Denmark, ) Dymkin, G Y, and Konshina, V N, "Systematics of NDE Reliability: A Practical Point of View," 15th World Conference on Nondestructive Testing, Rome, Italy, Available at 39) Rummel, W D, "Qualification and Validation of the Performance Capability (POD) For Nondestructive Inspection Procedures," 16th WCNDT World Conference on NDT, Montreal, Canada, ) Rebello, J M A, Carvalho, A A, Sagrilo, L V S, and Soares, S D, "Reliability of the Ultrasonic Technique Applied to Detection of Pipe Weld Defects," 33rd Annual Review of Quantitative Nondestructive Evaluation, Volume 26, Portland, Oregon, American Institute of Physics, Melville, New York,