Food microbiological risk assessment (MRA)

Properties and Principles of Best-Practice Risk Assessments:

Microbiological risk assessment contains a list of general principles of microbiological risk assessment, including that:

• Risk assessment be objective and soundly based on the best available science and presented in a transparent manner;
• Constraints that affect the risk assessment, such as cost, resources or time, be identified and their possible consequences described;
• MRA should clearly state the purpose, including the form of risk estimate that will be the output;
• The dynamics of microbial growth, survival, and death in foods and the complexity of the interaction (including sequelae) between human and agent following consumption (as well as the potential for further spread) be specifically considered;
• Data should be such that uncertainty in the risk estimate can be determined;
• Data and data collection systems should, as far as possible, be of sufficient quality and precision that uncertainty in the risk estimate is minimized;
• The risk estimate should include a description of the uncertainty and where that uncertainty arose; and
• MRA should be conducted according to a structured approach that includes hazard identification, hazard characterization, exposure assessment and risk characterization.

The scope of the risk assessment in terms of content and timeframe should be appropriate to meet its objectives and fulfil the needs of the risk managers. As such, before embarking on a risk assessment, the purpose and scope should be clearly identified and articulated by those who commission it.

Risk assessments should be initiated in response to well-defined risk management questions; where possible these questions should target the evaluation of the specific risk management options under consideration. Discussions with risk managers are needed to define what information is required to support the decisions they have to make and the type of work that needs to be undertaken to provide that information. Depending on the question(s), this may, for example, include provision of surveillance data, or epidemiological data; a qualitative risk assessment; or a quantitative production-to-consumption exposure assessment. Even if a fully quantitative risk assessment is thought to be necessary, it may be useful to commence with a qualitative approach to better define the nature of the work, the feasibility and the time needed to meet the risk manager’s requirements. This approach highlights the likely iterative nature of risk assessments.

The risk assessment for microbiological hazards should provide risk managers with a “best estimate” of the risk. The basis of this best estimate, whether the average risk (mean), or the most likely risk (mode), or some other metric, should be clearly communicated and include a description of why that metric is the best measure of risk. The chosen risk estimate should be as free of bias as is possible. Bias describes forms of error that lead to consistent over- or underestimation of the true risk. If bias cannot be eliminated (e.g., the decision to use a worst-case estimate), that bias and the reasons for it should be clearly stated.

Risk assessments should represent the real world as closely as possible and reflect the full range of possible outcomes. For example, this may include probabilities and levels of exposure and consequent risk (e.g., through a distribution of risk per serving). A risk manager may also express the need for information on a particular subset of outcomes, such as “most likely” or “worst-case” scenarios, and the MRA should accommodate those. However, deliberately conservative estimates can reduce the usefulness of the estimate for cost–benefit and cost–effectiveness studies and decrease the ability to describe the uncertainty of the risk estimates. However, they may be useful in certain situations, e.g., to better understand the effect of risk mitigations.

Purpose and Scope of MRA:

Risk assessment is commonly undertaken to help risk managers understand which, if any, intervention strategies can best improve food safety outcomes, or if current risk management actions are adequate.

Before beginning a risk assessment, the purpose and scope should be clearly defined, either explicitly or implicitly through the risk management questions. This may involve a discussion between all relevant parties, including the risk managers, risk assessment team, risk communication specialist, and, when appropriate, relevant stakeholders. Definition of the purpose and scope usually specifically identifies the population that should be protected (e.g. general population, young children, pregnant women, immunologically compromised), the stages of the food supply chain that are to be included, as well as the metrics of risks best suited for decision-making. The scope may need to be revised during the preparation of the risk assessment if it becomes evident that the original scope cannot be achieved; any change in scope should be discussed and agreed with the risk manager.

If the risk assessment aims to find the option resulting in the greatest reduction in risk, then a statement of purpose should be prepared to identify all potential risk management options to be considered. The questions and the statement of purpose will, to a great extent, guide the choice of the approach to be taken to characterize the risk. Clearly, this should be done prior to commencing the risk assessment so that the relevant data are gathered, synthesized and analysed in a way that most effectively informs the risk manager. However, if the purpose of the risk assessment is not clear initially, inappropriate data and information may be collected and analysed. While the results may provide insight into some aspects of the risk, they do not provide clear answers to inform the risk manager appropriately.

Risk managers initially define the intended use of a risk assessment in their preliminary risk management activities (Figure 1). They may need to iteratively interact with risk assessors to refine the specific questions to be answered, the scope, focus or outputs of the risk assessment, possibly throughout the conduct of the risk assessment. Risk managers are expected to ask risk assessors to answer specific questions about potential risk management options, which when answered, provide the managers with the information and analysis they need to support their food safety decisions (FAO, 2017).

The purpose and scope of risk assessment can vary depending on the risk managers’ questions. The following sections contain a discussion of three possible approaches to risk assessment. No correct approach can be recommended or specified; the choice of approach depends on the risk assessment question, the data and resources available, etc. Three approaches, considered as examples, are:

• Estimating a baseline risk;
• Comparing risk intervention strategies; and
• Research-related study or model

Estimating baseline risk:

A common and practical starting point for a risk assessment is to estimate the existing level of risk, often termed the baseline risk, i.e., the level of food safety risk posed without any changes to the current system. This risk estimate is most frequently used as the baseline against which intervention strategies can be evaluated (Figure 3). Using the current level of risk as a baseline has the advantage that the magnitude of the risk after a change is relative to this baseline. This approach implies that the baseline risk is the starting point of any risk management actions. For some purposes, a baseline other than the existing level of risk might be used as a point of comparison. For example, the baseline risk could be set as that which would exist under some preferred (e.g., least costly) risk management approach, and the risk under an alternative approach compared with that.

Estimating a baseline risk may not be for the immediate purpose of managing the risk. It may be to estimate the magnitude of a food safety problem and hence decide whether the risk merits further management. Whilst in theory it may not be necessary to determine a baseline risk to evaluate intervention strategies, it is nonetheless almost always carried out in practice.

Comparing risk management strategies:

Ideally, agencies with responsibility for safety of foods would consider all possible risk management options along the food chain without regard to who has the authority to enact them. This objective has led to the creation of integrated food safety authorities in many nations and regions. For example, Berends et al. (1998) considered the likely effects on exposure (i.e. Salmonella contamination of pork retail cuts) under different intervention strategies, covering various steps from the farm to the retailer.

A farm-to-table model may be most appropriate for this purpose, though for some risk questions, analysis of epidemiological data or a model of part of the food chain may be adequate. In practice, however, the scope of the assessment may be limited to those sections of the food chain within the risk manager’s area of authority. Nevertheless, a more comprehensive risk assessment might identify areas where the risk manager needs to work with other stakeholders to achieve effective change in the food chain.

Evaluations of potential risk management actions are often based on comparisons of a baseline risk estimate with an estimate that could result from pursuing alternative strategies (FAO and WHO, 2009b; Perrin et al., 2015; USFDA, 2005) as shown in Figure 3. Such alternatives may be evaluated through “what-if ” scenarios. One includes a future with no new intervention, the other a future with a new intervention. Initially, a baseline model is constructed and run to give a baseline estimate of risk and what is expected to happen in the future if no intervention is implemented. Then the model or selected model parameters are changed to determine the probable effect of the putative intervention(s).

Research-related study or model:

Reliable data are needed to do good risk assessment. There are a number of large microbiological risk assessment models that have been initiated as academic exercises (Guo et al., 2015; Pang et al., 2017; Van Abel et al., 2017). These models have helped advance the field of microbiological risk assessment by identifying what techniques are necessary, developing new techniques, and stimulating research that has value within a risk assessment context. In some situations, those models have subsequently been used by risk managers to assist in making risk management decisions. Such models have also made apparent the changes needed in collection and reporting methods for microbiological, epidemiological, production, dietary and other data that would make the data more useful for risk assessment.

Risk assessment is also a very useful aid in identifying where gaps in knowledge exist and thus where additional information is needed. A risk assessment may be undertaken specifically or incidentally to identify research needs, to establish research priorities, and to help design commissioned studies. Experience with microbiological risk assessments has proven these assessments to be valuable in aiding understanding of complex systems. The very process of systematically investigating a food chain has contributed to the appreciation and understanding of the complexity of the systems that make up the food chain.

Role of Best- and Worst-Case Scenarios:

It may be useful to evaluate the best- or worst-case scenarios to get a sense of the most optimistic and pessimistic risk estimates. These scenarios may be used as a filtering technique or as part of a risk profile. For example, the worst-case scenario can be used to filter out whether a risk, or an exposure pathway, is worth worrying about. No further analysis is necessary if the most pessimistic estimate shows the risk level to be below some threshold of interest, such as a negligible-risk level or an acceptable level of risk as defined by a competent authority.

Best- and worst-case scenarios operate like extreme what-if scenarios. Where there is considerable but quantified uncertainty about a model parameter, a value is used that gives the required extreme. This will usually be an extreme value from the uncertainty distribution of the parameter, e.g. its 1st or 99th percentile. Where there is uncertainty about exposure pathways and risk attribution, the extreme risk estimate is achieved by picking the most pessimistic (or optimistic) pathway, for example, “imagine that all Salmonella came from chicken.”

Potential problems with worst-case analyses include focusing the analysis on the consequences of the worst case, without the context of the probability of that scenario occurring – absolute extremes may be limited only by imagination, no matter how unlikely. In addition, there may be difficulty in specifying the conditions that could lead to the worst (or best) case. Conversely, when parameter values or exposure pathways are known with considerable certainty, they should be used to avoid exaggerating the extreme scenario beyond what is likely.

Assessing the results of A Risk Assessment:

When undertaking a risk assessment, the risk assessor needs to consider two basic probability concepts that can affect the outcome. The first is the apparently random nature of the world. The second is the level of uncertainty about how the real world is operating. Together, they limit the ability to predict the future and the consequences of decisions made. Inevitably, a risk assessment will not have included all possible information about a risk issue because of limited data access (for example, time constraints for the collection of data, or unwillingness of data owners to share information) or because the data simply do not exist. Complying with all the requirements of transparency – describing model and parameter uncertainties, and all explicit and implicit assumptions – does not necessarily communicate to risk managers the degree of confidence that the risk assessor has in the results or limitations in its application. Thus, risk assessors should clearly explain how confident they are in the risk assessment results. The confidence in the results depends on the extent of variability and uncertainty in the model outcomes.

All assumptions should be acknowledged and made explicit in a manner that is meaningful to the risk manager. In particular, it should be explained what the assumption is, why it was made, why it is appropriate, and what the expected effect is if the assumption does not hold.

The process of microbiological food safety risk assessment is most affected by uncertainty, such as:

• uncertainty about what is happening in the exposure pathways resulting in human illness;
• uncertainty about processes that lead from ingestion through to infection and illness;
• uncertainty in the factors that dictate the severity of the illness in different people; and
• uncertainty about the parameter values that would describe those pathways and processes

In general, risk assessments should be as simple as possible whilst meeting the risk manager’s needs. The MRA should strive to balance greater detail and complexity (e.g., through addressing more questions or alternative scenarios) against having to include more assumptions that this would entail. That is because more assumptions increase the uncertainty in the results. A draft risk assessment, in which the data gaps and assumptions are clearly identified, may elicit new information, if distributed widely to important stakeholders.

In the process of performing a risk assessment one usually learns which gaps in knowledge are more, and which are less, critical. Some of those uncertainties are readily quantified with statistical techniques where data are available, which gives the risk manager the most objective description of uncertainty. If, however, a risk assessment assumes a particular set of pathways and causal relationships that are incorrect, then the assessment will be flawed. This is clearly different from variability and uncertainty and should be avoided as much as possible. 

Choosing the type of Risk Assessment to Perform:

Risk assessments methods span a continuum from qualitative through semi[1]quantitative to fully quantitative. These approaches may vary in their key attributes such as the quality of risk inference, timeliness, complexity, assessor training requirements, and data requirements. Regardless of the approach used, a scientifically sound risk assessment requires collection of suitable information/ data/assumptions which are documented and fully referenced and synthesized in a logical and transparent manner. All are valid approaches to food safety risk assessment, but the appropriateness of a particular method depends on the ability of the risk assessment to address the specific risk question, i.e. that it is fit-for[1]purpose to support the risk management decision process. A benefit of undertaking a risk assessment, irrespective of the approach, is that solutions to minimize risk often present themselves out of the process of assessing risk.

Qualitative risk assessments are descriptive or categorical treatments of information. A qualitative assessment may be undertaken as part of a first evaluation of a food safety issue, to determine if the risk is significant enough to warrant a more detailed analysis. This again highlights that risk assessments tend to be, and frequently are, iterative. Nevertheless, a qualitative exposure assessment alone may, in some circumstances, provide all the decision support needed by the risk manager. If a more detailed analysis is warranted, then a fully quantitative assessment is usually the preferred approach if data, time and resources are available to support it.

Semi-quantitative risk assessments evaluate risks with a score. They provide an intermediary level between the textual evaluation of risk of qualitative risk assessments and the numerical evaluation of quantitative risk assessments. They offer a more consistent and rigorous approach to assessing and comparing risks and risk management strategies than qualitative risk assessment. They also avoid some of the ambiguities that a qualitative risk assessment may produce. Semi-quantitative risk assessments do not require the same mathematical skills of quantitative risk assessments, nor do they require the same amount of data, which means they can be applied where precise data are missing.

Quantitative risk assessments provide numerical estimates of risk and most models use combinations of mathematics and logic statements. Quantitative risk assessments require the development of mathematical models. In these models the relationships between factors affecting exposure can be quantified and explained using logical tests and conditional statements. An exposure estimate may be combined with a mathematical function that quantifies the dose–response relationship to provide an estimate of risk.

It should be noted that there is a gradation of model types from qualitative to fully quantitative and while such classifications may be helpful, they are not strictly defined categories.

For more information, please Chat with us Ask The Expert.