METHODOLOGY FOR PHYSCS & ENGNEERNG OF RELABLE PRODUCTS Dr. Steven L. Cornforcl and Mark Gibbel Jet Propulsion Laboratory California institute of Technology 48 ak Grove Drive Pasadena, Ca 9119 Abstract Physics of failure approaches have gained wide spread acceptance by ost within tile Electronic Rcdiability Counity. These ethodologies involve identifying root cause failure echaniss, developing associated odels, and utilizing these odels to iprove tie to arket, lower developent and build costs and higher reliability. The ethodology outlined herein sets forth a process, based on integration of both physics and engineering principles, for achieving the sae goals. 1 he proposed ethodology is consistent with a pure physics of failure ethodology, but it has ttle distinct advantage of not being dead-in-the-water if failure physics odels do not eist. t also goes a long way to overcoing the age old aio that typically the things that fail are not the things that were analyzed, evaluated, etc. but rather the things that were assued to not to be a proble. t outlines a ethodology for integrating all available data, at various data cluality levels, to ake the best possible decisions, l he key coponents are: 1 ) eisting physics and engineering odels, 2) utilizing a problen]/failure reporting syste and other data sources to identify tall poles and t}leir root cause for current designs and process and 3) new technology evaluations based on failure physics assessents. Objectives The objectives of the Physics (and Engineering) of Failure based Methodology (PEFM) are: 1 ) to cobine the principles of physics and engineering to arrive at a product developent and ipleentation processes that result in cost effective reliable products being developed and fielded, 2) easy to ipleent and 3) applicable to organizations independent of their size. Tklis ethodology should also provide clear guidance for utilization of eisting data at all levels, generate road-aps for new inforation requireents and identify specific areas where study efforts are needed. Background/Methodology Description The PEFM ethodology utilizes the root cause/root effect approach. n deterining the effectiveness of a particular test or other activity, it is necessary to clearly establish the requireents to be verified or validated and thus the objectives and etrics of each activity. This effectiveness data is needed to assess the value added by these activities as well as to provide a feedback loop to identify opportunities for process iproveents. This ethodology cobines systes engineering principals, product inforation (test/activity objectives, hardware perforance rec~uireents, etc.), behavior/anoaly data (at various levels of integration and in varying levels of data quality/detail), along with failure physics odels to assess the root cause and effect of possik)le defect detection and/or prevention options. 1 he systes approach (Defect detection and Prevention (DDP)) presented in Reference 1, utilizes a PACT (prevention, analysis, control or tf3st) effectiveness atri to systeatically identify and rate the ability of various PAC1- S to prevent and/or detect failure echaniss ttlat have the pc)tential to i pa ct product requireents. This PACT effectiveness atri is itself a product of two atrices. One atri rates the criticality and likelihood for the occurrence c)f individual failure echaniss on the product requireents to establish weighting factors, while the ottler atri ranks the effectiveness of each available PACT at detecting/participating defects on a failure ode by failure ode basis. The ratings (or etrics) involved in both atrices are based on a cobination of failure ptlysics, engineering principles and analysis of available data. This approach is an integral part of NASA s Test Strategy developent efforts (Reference
2 ) and is described in ore detail in Reference 3. This paper briefly suarizes the DDP ethodology as applied to the NASA/JPL Test Effectiveness Progra and provides a discussion of the approach for using data fro a proble/failure reporting syste to arrive at these rankings when failure odels do not eist. Standard systes engineering tools and approaches are applied to assessing the ipact of various failure odes on the products basic and derived requireents. The PACT Effectiveness Matri (EM) described above involves rating the effectiveness of tt]e things that can be done to prevent (usually best) or detect failure odes/echaniss versus the weighied failure odes/echaniss that were identified in the requireents atri. These ratings are arrived at by evaluating the effectiveness of the various knobs (paraeters which can be varied for each PACT (teperature level, electrical and/or echanical load, etc.), and the individual settings selected for each knob (-2C to 75C, etc. ) of a particular PACT. Concept llustrations Being liited to only using ratings that were derived cjnly fro failure physics odels would be an etree handicap because physics and engineering odels do not eist for any of the failure odes. Therefc)re, knowing how to coe up with rankings wt]en odels do not eist can significantly etend the usefulness of this approach. Work done as part of the physics of failure efforts at the University of Maryland Electronic Packaging Research Center (EPRC) is being integrated and suppleented by otlwr physics of failure data and the engineering data available across NASA and our industry partners through the NASA Test Effectiveness Working Group. The NASA Test Effectiveness Progra is integrating tt]e best of coercial and governent approaches witl] currently available data to significantly iprove the test and validation process for spacefligl]t applications, although ttle results have general applicability. The concept illustrations presented below, show how data fro a variety of sources (such as a proble/failure syste, SPC charts, research data, technology evaluations, etc. ) can be used in cobination with physics/engineering odels to arrive at effectiveness ratings. Related data and findings ay be found in References 4-8. n the illustrations presented, the following notations have been used: Capital X s are used to denote that a particular PACT (or knob) is thought to be effective, although no specific effectiveness deterination has been ade. n a siilar sense, sall s denote that this PACT is slightly effective but again no specific effectiveness deterination has been ade. Where the effectiveness is not known, the entry is represented by a question ark and where it is not effective at all, a O (zero) is entered, n the illustrations where nuerical ratings are shown, the sae interpretations are applied to the zeros and question arks. A 1 (one) is siilar in interpretation to the sall, in that it represents the case where data has indicated that the PACT can prevent and/or detect the failure ode/echanis being evaluated, but not very effectively. n a like anner, a 3 (three) is used to indicate that a PACT is effective at preventing/detecting a failure ode/echanis but it should not be considered as a priary PACT for this failure ode/echanis. An entry of a 9 (nine) is used to indicate that a significant nuber of these failure ode/echaniss are prevented/detected when this PACT is applied conipared to the total nuber of the present in the hardware/software. Phenoenoogical/En~ pirical Data Source Eaples Useful PACT effectiveness assessents can still be ade when the only inforation available is phenornenological in nature (i.e. cause and associated effect not deterined at the root level). Consider the following eaple. Progra XYZ was a successful Class A prograrrl and it ipleented the following design, assurance and test progra..... While this stateent iplies that the things that were done to prevent and/or detect defects were successful on progra XYZ, no cause and effect data was presented to differentiate between various PACTs and thus tailoring is essentially ipossible. n the absence of any other data, a pac1- effectiveness atri (based on the PACT s ipleented on progra XYZ) could still be developed for a new product. }{owever, these entries would have a greater uncertainty associated with the and ay not have as uch sub-categorization as entries based solely on PEFM evaluations. n this case, the entries in the F)ACT effectiveness atri could be represented by either X s or question arks. The copleted effectiveness atri based on these entries would still indicate potential areas of overlap, issing coverage and so on.
For eaple, consider Table 1 which is a hypothetical partial top level PACT effectiveness atri fro the above XYZ progra. Note that both powered and urr-powered rando vibration testing were used on this successful progra. f we are trying to apply progra XYZ S PACT effectiveness atri to an electronic card on a new progra, we would ost likely chose to do the powered vibration PACT because Table 1 indicates that it is ore effective on the failure odes ost likely to be present in electronic cards. On the other hand, if we are testing a piece of spacecraft structure, then the non-powered PACT would be the choice. The tradeoff illustrated in Table 1, was so siple that a atri was not really necessary. }owever, consider a coplete PAC1 effectiveness atri for this sae siple electronic card. t would not be unreasonable for there to be 25 PACT s listed (inspections, tests design rules, reviews, etc. and their respective knobs and setting ranges) and possibly an equal or greater nuber of failure odeshnechaniss. agine trying to do trade-offs for this set of variables without the aid of the atri tool. Now lets look ore closely at a particular PACT to see what else can be learned. Table 2 presents a partial high level assessent of a PACT called theral test. Note that for any of the failure odes it is not known if any of the knobs are effective in detecting the. Also note that the overall effectiveness of soe of the knobs is unclear. Table 2, can be read like a road-ap with respect to areas where new inforation requireents are needed or where specialized, detailed effectiveness studies are warranted. These studies could be a cobination of proble/failure data searches, siple engineering/failure physics odels and/or engineering judgernent. These studies would be like reoving layers fro an onion, one layer at a tie. Table 3 represents a hypothetical eaple of oving one layer deeper than Table 2 because we now have specific PACT levels and thus nuerical ratings. At the Table 3 level of cjetail, we can clearly see that one should not count on this PACT to be the priary ode of probje detection for: application specific integrated circuit (ASC) parts, solder balls (workanship) or echanical design probles/failure odes. Again, we can reove another layer and now look at the effectiveness of the individual knobs and rank the effectiveness of the individual settings. Table 4 illustrates these points. _fhe specific PACT illustrated is a theral test. The envirc~nental knobs for a theral test are considered to be teperature levels (hot and cold), dwell ties (at hot and cold), rate of teperature transitions, nuber of theral cycles and pressure level (i, e. vacuu, abient or other). By aking trade-offs between the various knobs and their settings, one can optiize the cost to benefit ratio of an individual PACT which can then be rolled up for coparisons with other PACTS. ~-o facilitate this illustration Table 4 has an added colun which sus the effectiveness of each knob or setting option. Note that, in this illustration, the ost effective knob is teperature level. n contrast, the two least effective knobs/settings are unpowered testing and perforing any theral cycles. One final point to keep in ind is that while a knob or setting ay be very effective at preventing or detecting a failure ode or echanis it is not cost effective to prevent or detect failures that could not occur in the ission life cycle. For eaple, setting test pressure knob to the vacuu setting is ore effective for failure odes/echaniss that are due to: 1) high teperature and/or gradients (circuit tiing, part paraeter drift over jife, transistor gain, etc.), 2) pure vacuu effects (such as corona) and 3) a variety of workanship failure odes (such as bondtines, etc.). However, it is less effective at detecting cold teperature related failure odes/echaniss (tiing, paraeter drift, hot carrier injection, etc. ) because, in general, testing in a vacuu biases device teperatures, board teperatures, etc. upward. Another consideration would be the probability of occurrence in the ission life cycle. Reeber that as a result of Recluireents atri the effectiveness rating for the PACTS discussed above would be ultiplied by the weights (probability of occurrence and ipact on the ission objectives) for each failure ode to arrive at the applicable ission effectiveness of a given PACT and allow tradeoffs and prioritizations within cost and schedule constraints. Therefore, ission life cycles which do not epose ttle hardware to vacuu conditions would have a zero probability of occurrence for a pure vacuu related failure odehnechanisrri. While the vacuu pressure setting could be used on hardware where the ission life cycle only involved abient office pressures, this choice probably would not be cost effective.
?-he above illustrations clearly deonstrate that useful PACT effectiveness atrices can be ade without necessarily having detailed failure physics odels for every entry. These illustrations also show that as one goes into ore and ore detail, the specific strengths and weaknesses of a given PACT are revealed, allowing cost benefit trade-offs to be ade within a given PACT and across collections of PACT S. Root Cause Analysis Eaples Net we will look at how an organization s proble/failure reporting syste can be used to coe up with effectiveness rating for various PACT s/knobs. At the board level and higher, it is routine to establish foral test plans and docuent hardware anoalies in a proble/failure reporting (P/FR) syste or its equivalent. These P/FRs docuent the sypto or failure ode(s) observed or easured and assess the significance of the P/F relative ission risk, should it occur in the ission.?he P/FR process also docuents the root cause and corrective action for the problc/failure. t is at tl~is point where deterination is ade of the required fi and the likelihood of tile fi eliinating siilar future probles and revisions (if required) of the significance of the original failure. n any cases these causes are not always at the physics of failure level. That is, it is not always necessary to establish a odel based on ptlysics principles which predicts the anoaly and establishes the point(s) in paraeter space where the fi lies. Eaples are presented below which illustrate the usefulness of the data containecj within typical P/FR s for identifying the types of failure c)cles detectec~ by various PACT S..As.scssn]er~t_ by_.lnsp_ection.erocesses A bonclline failure was detected during high teperature testing after rando vibration epc)sure. An inspection of the failure site indicated that only 1% of the bondline area adhered to one surface. A subsequent cl]eck of the build checklist indicated the prier application step was skipped. This eaple clearly illustrates that the root cause could be deterined without going down to the physics of failure level. Moreover, the root cause can be considered ore in the real of process control than failure physics. ~ssessl]]g.nt~.lspe~tiou _Q. _AD@hsis A bondline failure was cletected during high teperature testing after rando vibration eposure. Upon inspection it was noted that the bondline was only half as long as required by the design guidelines. Mechanical analysis confired that a one siga rando vibration test load (on this out-of-specification bondline area) would eceed the ultiate strength of this aterial by 25Y. n this case, a violation of one of the prevention easures (i.e. the design guideline for bondlines) can be considered as the root cause. Assessent_by Failure_Analysis A bondline failure was detected during high teperature testing after rando vibration eposure. Upon visual inspection, the bondline geoetry was found to be within specification. A sall saple of the bonding aterial was reoved and placed in a ass spectroeter for analysis. The analysis indicated that the two part adhesive was only 25% ied. Based on anufacturers data it was deterined that this iture would only have 1% of the strength of a fully ied aterial. Mechanical analysis confired that a one siga load during rando vibration testing would eceed the reduced ultiate strength of this aterial. This case also presents a process control proble as the root cause. Suary and Conclusions The key benefit of this ethodology to anageent is that it provides a tool for identifying the tall poles within the organization (based on accepted processes), their overall ipact on the organization anc~ possible solutions that coulci result in a better overall optiization of organizational resources. The key benefit tc] the practitioner is that it provides the a tool to estiate the consequences of the choices they are considering and then easure the ipact of the choices they actually ade. These etrics are critical to enabling decisions and institutional changes to be ade intelligently both within, and across, progras. l llis process also enables several key functions within an organization. For eaple, the PEFM ethodology: 1. Can be used as the engine for Reengineering activities, 2. s capable of identifying the tall pole activities within an organization s operations, 3. s capable of identifying appropriate etrics to easure the value-added by individual activities
. 4. 5. within an organization, Provides the feedback process necessary for achieving continuous process iproveent, Recognizes the effects and constraints of the technologies involved, resources available, anufacturing and use environents and eisting corporate culture. Using all of the eisting data allows an organization to close the loop with respect to the effectiveness of their PACTS (such as, design guidelines, process controls, analyses, etc.). h can also identify the need for new or updated standards, process controls, etc. The reason that the eaples cited above were effective in identifying the root cause was that the these PACTS were based originally on soe for of research/odeling activities. n fact, any were the result of new technology evaluations which identified the liits/sensitivity of a new technology and the associated screens as PACTS. Even the PACTS which were the result of tiger tea findings ( i.e. odels developed in quasi-real-tie to predict a previously unknown phenoena) for a particular failure ode ay now bc generalized and provide a data source for evaluating PACT effectiveness on future progras. nforation of the type presented above, can be used to ake useful, first-order predictions of the effectiveness and likelihood of escapes, or probles which were undetected. This historical data can be superseded by either engineering analysis data or data fro physics of failure odels to arrive at second through Nth order reliability epectations, as ay be appropriate. The typical physics of failure approaches have ignored these types of engineering probles, priarily because the approach fails if the effect cannot be odeled using physics principles. Technology, under contract with the National Aeronautics and Space Adinistration (NASA). This work was sponsored by Mr. Steven Wander of NASA Code OT, as part of Code CT s Test Effectiveness Progra, References: 1. Defect Detection and Prevention, Steven L. Cornford, Proceedings of the 16th Aerospace Testing Seinar, Los Angeles, March 1996.?. Risk as a Resource - A New Paradig, M. A. Grecnficld and T. E. Gindorf, Probabilistic Safety Assessent and Manageent 96, ESREL 96/PSAM- 111, Crete, Greece, June 24-?8, 1996, Eds, Carlo Cacciabue and Loannis A. Papazoglue, Vol 3, pp. 1597-165 (Springer, New York, 1996) 3. A Systeatic Approach to Clualification, Phillip ~. Barela and Steven L. Cornford, Proceeding of the nstitute of Environental Sciences, PP. 118-126, 41st Annual Technical Meeting, Anahei, California, April 3 to May 5, 1995. 4. Theral Testing Study Report, Steven Cornford, Paul Plub, JPL D-1 1958, October 1994, JPL nternal docuent 5. Electronic Assebly Theral Testing - Dwell/Duration/Cycling, Mark Gibbel ancj Jaes Clawson, Proceedings of the 1?th Aerospace Testing Seinar, Los Angeles, March 1989 6. Satellite Environental Testing Cost Elenefits, Otto Haberg, Willia Tosney, Charles A. Brackin, Proceedings of the 12th Aerospace Testing Seinar, Los Angeles, March 1989 7. Long-life Assurance Study for Manned Spacecraft Long-Life Hardware, Vols. 1-5, Martin Marietta Corporation, Denver, CO, Decekler 1972 8. Environrrlental Test Effectiveness Analysis Reports, JPL D-1 12 95, Deceber 1994, JPi- nternal docuent Tllcse eaples have illustrated the approach for getting the root cause of the proble without necessarily going down to the failure physics level. n the cases presented above, the root cause can be considered ore in real of engineering. This cobination of engineering and physics of failure provides the ost cost effective utilization of all available data. Acknowledgents This work described in this paper was carried out at tile Jet Propulsion Laboratory, California nstitute of
,
AtlOWW SNOSSNW1 S.31SV ntlohvstl lwd ONOHM. SNOHS S1lWH M31OS Xoo SNOr ~~CllOS 12 Xoo c- 6 c- SdW31 OWO1 c- + o SdW~l lnvc l-- Xo ~W4K!lNl $)1~ *... z F u u a. (n z =
, 1 1! Auowai Sllolslstrftl -. (9 -- t+ - T Oal>alaol MJwlol. XlMa (-/ -1- ----- Klw to *.3*.- lwd W4M lluou1 81 we wow *- $! 91 NWUXJ1OS 1: W3WW< swill Ouvol ---.84W1 lwd. - e5 w- -1--- ------ llwo-umos a M9v>v t*al Ou #o WZl lw, i13vjm21nl TW.- -NWMH33U t.> w3in13i11> r------ -- 3 ~ w n U1 *- -. U-1 J Q! :) $ ll ll A 1- U l-- 2 F- U. c) Q. -) J -1 w-- -J -