Practical Risk Management - Webcast Follow Up

Thanks once again to those of you who joined us for yesterday's webcast - the second stop on our "virtual book tour" which looked at practical risk management. We had a good number of questions asked as part of the registration process which we handled in yesterday's webcast (you can watch a recording of the webcast and download the slides here) but unfortunately we didn't have time to answer all of your questions that were asked during the session.

As usual, we taken the time to answer the outstanding questions here on a life sciences blog.

Q. Can you say more about regulators who are worried about misused risk assessments?

A. During the webcast we mentioned that the number of inspectors from European and US regulatory agencies had commented that they have concerns about the quality of risk assessments and the resulting validation. This comments have been made during informal discussions and in one case at a conference.

Their concern is that the resulting validation is either not broad enough in terms of scope or not rigorous enough in terms of depth and that this has been uncovered during inspection of what they believe to be relatively critical systems. In a couple of cases inspectors have commented that they believe that this is a case of the companies involved using risk assessment as an excuse to reduce the level of effort and resources applied in validating such systems.

We know from their comments that in a number of cases this has led to inspection observations and enforcement actions and it appears that a number of regulatory inspectors are in their words "wise to the trick". As we said in a webcast yesterday is important that the scope and rigour of any validation is appropriate to the system and the risk assessment is used to determine which areas and functions in the system require greater focus. The objective of risk-based validation is not to simply produce a level of effort and expenditure but ensure that the efforts and resources are applied most appropriately.

Q. How much time and effort can be saved by using the right risk assessment approach?

A. Our experience is that by using a relative risk assessment process rather than a quantitative risk assessment process it is possible to reduce the time and effort spent on assessing risks by between 50 to 75%. We have also studied the outputs of both types of risk assessment process on very similar systems and it is encouraging to note that in many cases both processes have provided very similar outputs in terms of the distribution of high, medium and low risk priorities both in terms of the relative number of each risk priority grouping and the functions allocated to each group.

This means that for enterprise systems with lower risk it is possible to reduce the time spent assessing risks by half or three quarters and still come up with results which are sufficiently accurate to support appropriate risk-based validation. This is why it is so important that regulated companies have a variety of risk management processes and tools available to them so they can use the most appropriate and cost-effective approach.

Q. When would you use a quantitative risk assessment approach? For what type of systems?

A. You would typically use a quantitative risk assessment approach where it is necessary to distinguish low, medium and high risk impact amongst a variety of requirements or functions that are all or are mostly of high GxP significance. In this case a quantitative (numeric) approach allows you to take a more granular view and again focus your verification activities on the requirements or functions which are of the highest risk impact.

Typically these will be systems which are safety critical and while this approach could be very useful in terms of manufacturing systems, in terms of enterprise systems we see this approach being used to the most critical systems such as adverse event systems (AES), LIMS systems used for product release, MES etc. Even with these systems quantitative risk assessment can be used on a selective basis for those modules which the initial risk assessment determines to be most critical.

Q. Who should conduct the risk assessment of EDMS system supporting the whole Enterprise?

A. Risk assessments cannot be conducted alone. This was a key points bought out in this week's GAMP UK meeting where we ran a risk assessment exercise and it was clearly valuable to have a variety of opinions and experience feeding into the process. You need people who understand the requirements, the business processes and the resulting risks to give their expertise with respect to risk impact.

You also need technical subject matter experts from the engineering or IT group who are much more likely to understand the risk likelihood. Both groups can contribute to thinking about risk detectability, either in terms of detecting risks within the system or as part of the normal business process checks.

It is therefore very important to invite the right people with the right breadth and depth of knowledge to any risk assessment exercise and to allow sufficient time for the relevant risk scenarios to be identified and assessed.


Thank you as ever for your interesting questions - we hope we find the answers above useful. Remember that you can join us on 17th October when will be looking at the very thorny issue of validating enterprise systems in the Cloud  as Software-as-a-Service (registration is free and is open here)

Risk-Based Validation - How Are We Doing?

Yesterday, I posted a poll on LinkedIn asking the question

"How is Life Sciences industry embracing risk-based computer systems validation? Do we have the balance between risk assessment and risk mitigation right? Are we just rushing to reduce costs...?" 

People's responses are starting to come in and we're already starting to get some interesting comments on the question. If you're interesting in the poll and want to take part you can find it here (once you've answered the question you'll be able to see how everybody else has been voting).

Since I asked the question I guess it's only reasonable that I try and answer it and provide my own opinion. There is of course no single answer and even within a single Regulated Company there are different individuals and departments taking different approaches to risk-based validation. In many cases you have the business, and IT tried to reduce costs against a background of life sciences companies struggling to maintain profitability. At the other extreme you have the quality unit who are used to doing things the old-fashioned way and like to see every 'i' dotted and 't' crossed.

Hopefully, somewhere in the middle, you'll find some pragmatic validation practitioners genuinely trying to do their best to take a risk-based approach to validation and apply the appropriate resources to the highest areas of risk to patient safety, product quality and data integrity.

However, this is often thwarted by two obstacles:

1. The pragmatic practitioners in the middle are seldom the people with control of the budgets or with the power to effect real change. This often means that they'll pull from pole to pole at the behest of the business owners, the IT group and the quality unit who have different perceptions of how things should be done. In many cases these practical practitioners of the validation art are not recognised as true subject matter experts and they guidance, advice - and even wisdom - is ignored, often for the sake of point scoring and political expediency in large organisations.

2. Where practical and experienced practitioners are not available (often in small to medium sized organisations but also in organisations in emerging economies where computerised systems validation is a relatively new discipline) there is a genuine lack of understanding of how to take a practical approach to risk-based validation. in these cases and inability to invest in training and education is an obstacle to adopting risk-based validation.

I think therefore the what we will see is an industry which continues to adopt risk-based validation at a relatively slow pace. While some companies are using risk-based validation as an excuse to do less (and in some cases, too little) I think the majority of companies want to do the right thing but are hampered because of their own organisational structures or lack of experience, training and education.


From my experience as a consultant, and with my experience working with the small number of companies who are doing a good job with risk-based validation I genuinely do believe that taking a risk-based approach to validation has advantages both in terms of cost effectiveness but also in mitigating risk to patients. Regrettably, it's one of those things that most regulated companies will have to see for themselves before they believe, and they were set for themselves until they have gained the necessary experience.


As an industry I believe that we need to do a better job in publishing case studies demonstrating the effectiveness of risk-based validation, that we need to recognise the value of our internal and external subject matter expertsand he must be prepared to invest in training and education.