Risk assessments are an essential part of technology transfer, but it is surprising just how often they are misused – usually by being used to justify what is already being done rather than looking to see how process risks can be eliminated, or at best mitigated. Another common failing is to recognise a risk but to mitigate by saying “controlled by SOP” rather than actually doing anything about it.
However more often as not this is due to the people charged with performing the risk assessment not understanding its relevance – or not knowing how to carry it out properly.
There are many ways of course to perform a risk assessment – and perhaps I will list these in a later note. But one of the most often used methods is the good old Failure Modes & Effects Analysis (FMEA). This method looks at ways in which the examined process could fail, the effect of this failure and how these effects could be eliminated or mitigated.
Often the analysis points to how to minimise the effects of the risk rather than trying to eliminate the cause of the risk- and this is by far the most common failing (back to “controlled by SOP”).
Before I progress – a quick description of what an FMEA is:
Failure Mode and Effects Analysis (FMEA) is a structured approach to discovering potential failures that may exist within a process and the consequences of those failures. A risk score is derived by multiplying together individual scores assigned to the severity, frequency of occurrence and detectability of an identified potential failure in the process. A table such as the example below is often used to perform this analysis.
So – to look at some of the ways an FMEA is “mis-performed”:
- Performed by one person – no single person can assess all the potential failure modes, or effects of, from all manufacturing operations perspectives. Although I have seen many people try. A cross-functional team incorporating people from manufacturing, engineering, QA, MSAT and process development.
- Not allowing sufficient time – It will take longer than you think. If the time allocated is in “hours” rather than “days” you are probably not being thorough enough.
- The FMEA and its creation is not always the responsibility of the QA department. The QA department does not own the production process and to be honest don’t know the process as well as the production department. The FMEA should be created and owned by the person responsible for the production process.
- Performing the FMEA at the wrong time. Quite often performance of the FMEA is just seen as a box ticking operation that can be carried out at any time. To be effective the FMEA should be performed at the beginning of the manufacturing process design since its purpose is to assess the adequacy of the manufacturing process design. Any shortcomings after that could entail the redesign of the production process – costly and time consuming.
- Confusing Failure Mode with Failure Cause. For instance, in the example FMEA given, “feed volume too low” is not a failure cause – it is a failure mode. The cause is the incorrectly set pump speed.
- Mis-using / fixing PRN number. This is a biggy – it is assumed that a high PRN number (severity x occurrence x detectability) represents a big risk, but that is not so. Firstly, the PRN is only a relative number and only gives a relative value for that study. Unless very strict scoring methods are defined scores from one study cannot be compared. The PRN only gives an indication of which failure modes should be tackled first. I have seen many instances where scores have been “massaged” to result in PRN values under a certain “action” value and used as proof that the whole process is “low risk” and no action is required. FMEA’s are only useful if carried out in good faith. Personally I don’t believe a potential failure that could kill a patient (say severity score 10) could ever be an acceptable risk even if it would only very happen rarely (occasion score 1) and “almost” certainly be detected (detection score 1).
- Assuming all incoming components and excipients are within specification and error free thus not included in the FMEA. They are not. Indeed, in my experience this has been one of the most significant and costly process failures.
- Recommended Activities don’t address the root cause of the failure mode. Its all too easy to ignore identified risks by simply commenting “acceptable risk” or “controlled by SOP”.
- Not repeating the risk assessment after remedial actions have been taken. FMEA’s should not be a once off exercise – but should be repeated once remedial actions have been taken and can be used to demonstrate continual improvement.
About The Author:
Trefor Jones is a technology transfer specialist with Bluehatch Consultancy Ltd. After spending over 30 years in the pharmaceutical industry in engineering design, biopharmaceutical processes, and scale-up of new manufacturing processes, he now specializes in technology transfer especially of biotechnology and sterile products.
He can be reached at trefor ”at” bluehatchconsultancy.com.