Lessons Learned in Integrating Risk Management and Process Validation

Transcription

1 Lessons Learned in Integrating Risk Management and Process Validation Medical Device Congress Harvard March 2007 Jim Handzo Senior Manager QA Innovative Spinal Technologies Fran Akelewicz Principal Practical Solutions

2 Agenda Optimizing the validation activities sequence using risk management Determining the statistical approach and acceptance criteria based on risk Maximizing test plans and resources

3 Approach For illustration purposes we will use an example of: Safety product (functional approach) New feature -a a collar with a hinged rectangular plastic guard which will clip over the needle after use to prevent the user from getting stuck with the needle

4 Optimizing the validation activities sequence

5 Key questions for process validation What are the important design characteristics? Where and how are they impacted in the process? What resources do I need? What do I do first? How should I conduct the validation? What do I do if I don t t like the answer?

6 What are the important design characteristics? Product inputs to process validation Key design features Residual risk profile (dfmea) Controls for the risk-prioritized design features

7 What are the important design characteristics? Product Inputs (User and Patient) Key Design Features Tools Customer Requirements Matrix Design Requirements Matrix

8 What are the important design characteristics? Product Inputs (User and Patient) Customer Requirements Matrix Focused on user needs Focused on business needs Output Quality Characteristic Description Importance To Customer Gap Seen by Customer Rank (C x G) Marketing Value Safe Syringe Prevent accidental sticks Enclose needle point upon withdrawal from patient

9 What are the important design characteristics? Product Inputs (User and Patient) Design Requirements Matrix Focused on risk analysis Product Description Design Requirement (Input) Critical to Quality? (Y/N) Market Value Design Output (dfmea) Design Verification (dfmea) Design Validation Safe Syringe Prevent accidental sticks Yes Retraction function Needle guard (drawings, dimensions, force, etc.) Measurements Testing & inspection In user setting: Retraction studies over time Ergonomics

10 What are the important design characteristics? Product residual risk profile Without safety feature PROBABILITY 5: Frequent 4: Probable 3: Occasional 2: Remote 1: Improbable SEVERITY 1: Negligible 2: Marginal 3: Critical 4: Catastrophic X Acceptable ALARP Intolerable

11 What are the important design characteristics? Product residual risk profile With safety feature PROBABILITY 5: Frequent 4: Probable 3: Occasional 2: Remote 1: Improbable SEVERITY 1: Negligible 2: Marginal 3: Critical 4: Catastrophic X Acceptable ALARP Intolerable

12 Where and how are key design features impacted in the process? Process Inputs Determine process control points that potentially impact the design features Are process control points One to one? One to many? Many to one? Determine residual risk impact (pfmea, FTA, etc.)

13 Where and how are key design features impacted in the process? Determine various process points that potentially impact the design feature Design Feature Design Controls Process Feature Process Controls (Note traceability to actual documents) Prevent accidental sticks Safety shield design Injection molding Shield to barrel assembly DOE (See File 1234) Resin specification Y8279 SPC Maintenance Procedures M38654, M88876

14 Where and how are key design features impacted in the process? Determine if the control points are One process point to one design feature One process point to many design features Many process points to one design feature Design Specification Process Process Control Points Type of Control Safety shield dimensions Injection molding Process Spec Temperature Pressure Cycle time Clamping pressure Resin feed rate Many to one Proper retraction Shield to barrel assembly Drawing 1863 Shield to barrel locator One to one

15 Where and how are key design features impacted in the process? Use tools to characterize process outputs vs. design features Design of Experiments (DOE) Edge of Failure (EOF) Studies Ranging Studies Mean Time between Failures (MTBF) Cpk vs. Ppk Determine process residual risk Process Map & pfmea FTA Waterfall with Feedback Loop EN PEMS V Diagram

16 Where and how are key design features impacted in the process? Process residual risk profile Changes probability, not severity PROBABILITY 5: Frequent 4: Probable 3: Occasional 2: Remote 1: Improbable SEVERITY 1: Negligible 2: Marginal 3: Critical 4: Catastrophic X Acceptable ALARP Intolerable

17 What resources do I need? Dependent on type of product and/or process being evaluated Electrical Engineer Software Engineer Reliability Engineer Quality Engineer Commensurate with the degree of residual risk

18 What do I do first? Develop Process Validation Priority Risk Chart Similar methodology to Product Risk Chart Process effects on product are rated based on their impact Safety Performance Acceptance Compliance Verification Can also be used to develop a company Validation Master Plan

19 Process Map Formation and attachment of needle guard with collar attachment

20 What do I do first? Process Validation Priority Risk Chart Category Definition Examples Yes Rating No Rating Safety Process output directly affects patient/user safety Needle formation parameters Sterile package heat sealing parameters (maintain sterile barrier) 5 1 Performance Process output directly affects product performance Injection molding parameters (stacking tolerances) Sterile package heat sealing parameters (peelability) 4 1 Acceptance Process output directly affects product accept/reject decision Vision system for needle dimensions and appearance 3 1 Compliance Process output is objective evidence of compliance Line SPC on dimensions ( requirement to inspect prior to release) 2 1 Verification Process output is fully verifiable Laser etch 1 2 Needle alignment to syringe barrel and guard placement 1 2

21 FA9 What do I do first Apply to Entire processes Sub-processes (then add) Category Sterile Package Heat Sealer Injection Molding Machine Automated Inspection Safety Yes 5 No 1 No 1 Performance Yes 4 Yes 4 No 1 Acceptance No 1 No 1 Yes 3 Compliance No 1 No 1 No 1 Verification No - 2 No - 2 No - 2 Total

22 Slide 21 FA9 doesn't this table just address sub processes? shouldn't we show this to entire process and then sub processes? Would it be ok if we changed the 2nd, 3rd and 4th columns to match the needle guard assembly process? Fran Akelewicz, 3/6/2007

23 Where should validation testing occur?

24 What do I do if I don t t like the answer? Have a Plan B B Identify assignable causes Justify discounting results with great care Revisit assumptions made during validation setup Revisit probabilities assigned to key design features But you cannot change severity unless you change the design Define contingencies in acceptance criteria

25 Determining the statistical approach and acceptance criteria based on risk

26 Determining acceptance criteria Document in company policy Similar to product risk chart Key elements derived from product risk chart Severity drives confidence levels Confidence levels and probability drive sampling plans Qualitative probabilities (Frequent, Probable, etc.) must be expressed quantitatively Sampling plans should be discriminating enough to detect defects that relate to high severity/frequently occurring hazards

27 Determining acceptance criteria Acceptance Criteria Risk Chart Severity Confidence Level Qualitative Probability Quantitative Probability (Percent Defective for Sampling Plan) Catastrophic 99% Frequent p = Probable p = 0.01 Occasional p = Remote p = 0.05 Improbable p = 0.10 Critical 95% As above As above Marginal 90% As above As above Negligible 85% As above As above

28 Sampling plan determination Determine the operating characteristic (OC) curve of selected sampling plan Incorporate quantitative probabilities Determine acceptable errors Type I (Producer s s Risk) we say it s s bad when it s s not Type II (Consumer Risk) we say it s s good when it s s not Does it match residual risk and probability?

29 Sampling plan determination Determine how much risk (error) you can tolerate in your estimates Attributes Percent defective error tolerance Variables Error tolerance of the average Must constantly weigh sample size vs. estimate risk (error) Economics vs. certainty

30 Sampling plan determination

31 Sampling plan determination Use appropriate probability distributions when determining sampling plans Binomial Large populations Sampling with replacement Hypergeometric Small populations Sampling without replacement

32 Sampling plan determination

33 Sampling plan determination

34 Sampling plan determination Sampling Calculators available for download at: (Once on the website, go to Track A, 4:10pm session, for links to presentation materials. Calculators are in the excel document.)

35 Maximizing test plans and resources

36 Maximizing Test Plans and Resources Identify resources based on risk Prioritize the process validation activities based on risk Choose the acceptance criteria Choose the appropriate sample methodology to minimize the business risk

37 Change Control Once process validation is completed you have established a state of control Any changes made should consider: Impact to product design risk (design verification testing; design validation testing) Impact to process design risk (equipment design decisions) Impact to control of risk within process

38 Conclusions Risk management in process validation: Provides the opportunity to: challenge existing assumptions think out of the box Acknowledges the risks that we have taken (product and business) Provides a common language Provides a framework for consistency Formalizes what we have done Contributes to safer products

39 Questions

40 Thank You! Jim Handzo Innovative Spinal Technologies Fran Akelewicz Practical Solutions