Consultancy in elicitation

My expertise

I have been a leading proponent in the Bayesian statistics community of the use of elicitation to formulate expert knowledge about uncertain quantities.

I led a major research project, funded by the National Health Service, into methods for elicitation. One of the outputs of that research was the most comprehensive survey of the field to date, published by Wiley in 2006.
I am co-developer of the SHELF package of elicitation tools, and have pioneered methods to quantify the imprecision in elicited distributions.
I was an expert member of a Working Group of the European Food Safety Authority which developed guidance on expert elicitation. The guidance forms a part of EFSA's standard procedures in risk assessment.

I can offer various options for training in elicitation. I have established training courses in the theory and practice of elicitation, and can also arrange mock elicitation sessions to further develop facilitation skills. Following training given to key staff, the SHELF protocol is routinely used in several companies to quantify uncertainty about critical parameters, including Novartis and GlaxoSmithKline. I also developed an online e-learning course for people who will participate as experts in an elicitation exercise, teaching them how to make probabilistic judgements.

My clients

I have conducted expert elicitations for Her Majesty's Inspectorate of Pollution, Risk Management Solutions Ltd, AstraZeneca, Pfizer, the European Food Safety Authority, GlaxoSmithKline and the Centre for Workforce Intelligence. Methods that I developed have been coded into software which has been widely used in the water and rail industries, and I have advised a number of users of those methods.

My one-day general course on elicitation has been given to Evira (Finland), Pfizer (UK), Procter and Gamble (USA), the University of Texas at Austin (USA), the University of St Andrews (UK), the Netherlands Forensic Institute (Netherlands), Humboldt University (Germany), Chalmers University (Sweden), Grünenthal (Germany), L'Oreal (France) and also as an open-entry course at Warwick University.

My three-day intensive course has been given several times as an open-entry course to a total of more than thirty individuals, and also as an in-house course to GlaxoSmithKline (UK and USA), the University of St Andrews (UK), Chalmers University (Sweden), Novartis (Switzerland and USA).

Typical challenges

Expert judgement is a crucial part of decision-making under uncertainty. Eliciting the knowledge of experts in the form of probability distributions for uncertain parameters of a decision model is a complex task. If done badly, the resulting probability distributions do not represent the expert's knowledge accurately, may not bear any resemblance to reality and can be dominated by arbitrary choices that the expert might make entirely differently if asked another way. Getting accurate, reproducible and defensible judgements requires careful management of the elicitation by a skilled facilitator. Skill and understanding of group psychology are particularly important when working with several experts (which is often needed to capture the full range of expert opinion). I can offer a comprehensive service as an experienced facilitator in elicitations across a range of areas including nuclear waste disposal, environmental hazards, project costing, manpower planning and pharmaceutical development.

Risk assessment and management is an important field that has special challenges. Experts are often called upon to assess risks smaller than can be informed by real data. Risk typically depends on the occurrence of extreme events, and particularly on the concurrence of several extreme values. From the perspective of elicitation, this poses particular difficulties in assessing the tails of probability distributions and correlations between variables. I can offer some novel methods to address these challenges.

I am happy to consult on these or other problems in elicitation.

Updated: 4 April 2025

Maintained by: Tony O'Hagan