Survey
* Your assessment is very important for improving the workof artificial intelligence, which forms the content of this project
* Your assessment is very important for improving the workof artificial intelligence, which forms the content of this project
The NPF & Scotland Performs: Analytical Underpinning and Challenges Mairi Spowage Office of the Chief Statistician 9th June 2009 An outcome based approach • This “New Approach” presented many opportunities for analysts • We could now bring to bear the whole package of evidence to say something meaningful about change • A way to go, particularly in evaluation Introduced since May 2007 National Performance Framework Summary of National Performance Framework How were the indicators chosen? There was a need to be able to measure progress against the 5 strategic objectives and the 15 national outcomes How were the indicators chosen? A selection of indicators were chosen to act as a representative set, so when taken all together they may be able to tell us something about progress on the outcome Longer, Healthier Lives Scotland Performs • Designed to show how the government is performing against its key indicators and targets • New and innovative approach • Big difference to previous administrations The Indicators • Mixture of many types of targets and indicators, from existing targets to those which were set down by legislation to some which were not currently measureable • Many are direction of travel indicators Existing Targets/ Those from other frameworks • HEAT – Health improvement, Efficiency, Access and Treatment • E.g. Achieve annual milestones for reducing inpatient or day case waiting times culminating in the delivery of an 18 week referral to treatment time from December 2011 Those contained in legislation • All unintentionally homeless households will be entitled to settled accommodation by 2012 • Reduce the number of Scottish public bodies by 25% by 2011 Those which could not be/were not measured • Improve knowledge transfer from research activity in universities • Increase the average score of adults on the Warwick-Edinburgh Mental Wellbeing Scale by 2011 • Reduce overall ecological footprint Challenges • Huge range of data sources – Survey vs. Administrative Data – Composite measures – Some SG sources, some UK Departments, some ONS, some from partner agencies • Many are National Statistics, but many are not, at least at the moment • Differing frequencies of publication and lags Recent Change Arrows • Each indicator has an arrow assigned to it • Arrows comment on the change between the last two data points – Simple, easy to understand, can be universally applied and is transparent Indicator Page Recent Change Arrow Performance Improving Performance Maintaining Performance Worsening Thresholds • Thresholds are used to ensure that small fluctuations are not claimed as worsening or improvement, e.g. for smoking, NI 22; any difference within +/- 0.5 percentage points of last year's figure suggests that the position is more likely to be maintaining than showing any change. A decrease of 0.5 percentage points or more suggests the position is improving; whereas an increase of 0.5 percentage points or more suggests the position is worsening. Thresholds Thresholds take into account: – Known Variance – Past trends – Significance of change – To some extent, the change required by the target Threshold Examples • GDP – 0.1 percentage points • Social Economy – £10M • Housing Supply – 1,000 Houses Who sets the thresholds? Scotland Performs Technical Assessment Group (SPTAG) – Chief Statistician (chair) – Chief Researcher – Head of OCEA – Chief Scientist • SPTAG make all analytical decisions to do with Scotland Performs When data need to be updated… • Each indicator has a Lead Analyst assigned to it from within Government • We have built in the SP updates to the standard publication process • The analyst submits a recommendation to the SPTAG • The group make comments on the presentation of the information, to improve accessibility • They approve or reject recommendation SPAN • Made up of all of the analysts who are involved across Government • Network meets regularly to debate technical issues which underpin Scotland Performs • Support to SPTAG • Acts as a “peer-review” function Future Challenges • Reflecting the contribution of the wider public sector – our partners • Evaluation of outcomes – how do we know we have achieved them? • What does success look like? Questions? www.scotlandperforms.com [email protected]