Tony O'Hagan - SHELF: the Sheffield Elicitation Framework
- Announcing a SHELF training course on 13-16 September 2022.
- Some users are experiencing problems with the e-learning course. We hope to fix this soon.
- A new paper on my publications page, published in Pharmaceutical Statistics,
describes a case study in the use of multivariate elicitation tools in SHELF.
- A recently published paper on my publications page presents a case study
in the use of SHELF to set a target response for treatments of Autism Spectrum Disorder (ASD) in children
and adolescents.
- The current version of SHELF is version 4.
SHELF
About SHELF
The SHeffield ELicitation Framework (SHELF) is a package of documents,
templates and software to carry out elicitation of probability
distributions for uncertain quantities from a group of experts.
Elicitation is increasingly important for quantifying expert knowledge in
situations where hard data are sparse. This is often the context in which
difficult policy decisions are made. It is generally important to elicit
from a group of experts, rather than a single expert, in order to synthesise
the range of knowledge and opinions of the expert community. (However, SHELF
may be used for a single expert with only trivial modification.)
Despite this growing role for elicitation, there is little in the way of training
and support available to those who wish to conduct elicitations. SHELF is a
response to this shortage. By reading and carefully following the SHELF documentation,
it should be possible for an untrained facilitator to carry out competent elicitation.
Courses
Nevertheless, most people may benefit from formal training in the use of SHELF.
Tony O'Hagan, Jeremy Oakley and colleagues will be running four days of tuition on Expert Knowledge Elicitation
with SHELF, on 13-16 September 2022.
It will be based on the very successful training we have given four times previously, most recently in January 2020,
but with a few improvements.
Tony O'Hagan has designed an e-learning course
to teach people how to make the kind of probabilistic judgements that we ask of experts
in a SHELF elicitation workshop. The intention is that experts will prepare themselves
by taking this course, so making better use of the experts' time in the workshop itself.
Other resources
We held our first SHELF User Forum on Tuesday 22nd January, 2019,
at the University of Sheffield.
This was an opportunity for anyone using, or interested in using, the SHELF method for elicitation to meet other
users and to hear the experiences and tips of several guest speakers from the SHELF user comunity.
We plan to hold further User Forums in the future.
MATCH is an online tool that implements much of the
SHELF methodology. It can be used as an alternative to the SHELF software for fitting distributions,
and quite nicely allows the user to see how the fitted distribution changes as the elicited judgements
(median, quartiles, etc.) change.
As a product of the project that developed the e-learning course, a simple
user guide for MATCH is now available.
Who are we?
SHELF has been developed by Tony O'Hagan and Jeremy Oakley, originally in the Department of
Probability and Statistics in the University of Sheffield. It arose out of our
long-standing commitment to research and practice in elicitation. The principal spur
for developing SHELF was discussions in the project "Bayesian Analysis in Microbial
Risk Assessment," led by Helen Clough at the University of Liverpool and Marc Kennedy
at the DEFRA Central Science Laboratory (CSL), and in particular the encouragement
of Andy Hart at CSL.
Downloading SHELF
Click here to download the current version (v4).
Using SHELF
All materials in the SHELF package are made freely available, but they are
nevertheless covered by copyright. They may be copied for the purposes of
conducting elicitations, for private study or personal use.
They may not be reproduced on any website, offered for sale or otherwise
distributed without the written permission of either Tony O’Hagan or Jeremy Oakley.
Further information about using SHELF and using the SHELF name are given in the
"SHELF Overview.pdf" document in the SHELF package.
Ongoing development
Version 1.0 of SHELF was released in 2008, and a minor upgrade, version 1.01,
implemented some small improvements to the R software.
Version 2.0 was released in September 2010 and was a significant upgrade.
New procedures and new templates were provided, there was
additional advice for facilitators and the software was completely revised.
Version 3.0 was released in October 2016 and was an even more substantial upgrade.
Enhancements included new procedures for multivariate elicitation,
extended advice documents and PowerPoint slide sets to help guide experts in making their judgements.
Version 4 was released in June 2019. It includes new templates and advice on eliciting knowledge about
many uncertain quantities.
Comments are welcomed by Tony O’Hagan
(shelf@tonyohagan.co.uk) and
Jeremy Oakley
(j.oakley@sheffield.ac.uk).
We would particularly welcome offers of additional materials or
suggested amendments to components of SHELF. We intend SHELF to be a growing
and open resource for anyone wishing to conduct effective elicitation.
Last updated: 13 April 2022
Maintained by: Tony O'Hagan
