SHELF

About SHELF

The SHeffield ELicitation Framework (SHELF) is a package of documents, templates and software to carry out elicitation of probability distributions for uncertain quantities from a group of experts.

Elicitation is increasingly important for quantifying expert knowledge in situations where hard data are sparse. This is often the context in which difficult policy decisions are made. It is generally important to elicit from a group of experts, rather than a single expert, in order to synthesise the range of knowledge and opinions of the expert community. (However, SHELF may be used for a single expert with only trivial modification.)

Despite this growing role for elicitation, there is little in the way of training and support available to those who wish to conduct elicitations. SHELF is a response to this shortage. By reading and carefully following the SHELF documentation, it should be possible for an untrained facilitator to carry out competent elicitation.

Courses

Nevertheless, most people may benefit from formal training in the use of SHELF. Tony O'Hagan, Jeremy Oakley and colleagues offer comprehensive SHELF Training Events from time to time. The next such event will run in January 2020.

Tony O'Hagan has designed an e-learning course to teach people how to make the kind of probabilistic judgements that we ask of experts in a SHELF elicitation workshop. The intention is that experts will prepare themselves by taking this course, so making better use of the experts' time in the workshop itself.

Other resources

We held our first SHELF User Forum on Tuesday 22nd January, 2019, at the University of Sheffield. This was an opportunity for anyone using, or interested in using, the SHELF method for elicitation to meet other users and to hear the experiences and tips of several guest speakers from the SHELF user comunity. We plan to hold further User Forums in the future.

MATCH is an online tool that implements much of the SHELF methodology. It can be used as an alternative to the SHELF software for fitting distributions, and quite nicely allows the user to see how the fitted distribution changes as the elicited judgements (median, quartiles, etc.) change. As a product of the project that developed the e-learning course, a simple user guide for MATCH is now available.

Who are we?

SHELF has been developed by Tony O'Hagan and Jeremy Oakley, originally in the Department of Probability and Statistics in the University of Sheffield. It arose out of our long-standing commitment to research and practice in elicitation. The principal spur for developing SHELF was discussions in the project "Bayesian Analysis in Microbial Risk Assessment," led by Helen Clough at the University of Liverpool and Marc Kennedy at the DEFRA Central Science Laboratory (CSL), and in particular the encouragement of Andy Hart at CSL.

Downloading SHELF

Click here to download the current version (v4).

Using SHELF

All materials in the SHELF package are made freely available, but they are nevertheless covered by copyright. They may be copied for the purposes of conducting elicitations, for private study or personal use. They may not be reproduced on any website, offered for sale or otherwise distributed without the written permission of either Tony O’Hagan or Jeremy Oakley. Further information about using SHELF and using the SHELF name are given in the "SHELF Overview.pdf" document in the SHELF package.

Ongoing development

Version 1.0 of SHELF was released in 2008, and a minor upgrade, version 1.01, implemented some small improvements to the R software.

Version 2.0 was released in September 2010 and was a significant upgrade. New procedures and new templates were provided, there was additional advice for facilitators and the software was completely revised.

Version 3.0 was released in October 2016 and was an even more substantial upgrade. Enhancements included new procedures for multivariate elicitation, extended advice documents and PowerPoint slide sets to help guide experts in making their judgements.

Version 4 was released in June 2019. It includes new templates and advice on eliciting knowledge about many uncertain quantities.

Comments are welcomed by Tony O’Hagan (shelf@tonyohagan.co.uk) and Jeremy Oakley (j.oakley@sheffield.ac.uk). We would particularly welcome offers of additional materials or suggested amendments to components of SHELF. We intend SHELF to be a growing and open resource for anyone wishing to conduct effective elicitation.

Last updated: 26 October 2019

Maintained by: Tony O'Hagan