Survey

* Your assessment is very important for improving the work of artificial intelligence, which forms the content of this project

Document related concepts

no text concepts found

Transcript

RCPCH SPRAT Feedback May 06 – August 06 A Guide to Interpretation Interpreting the Feedback • As you click through the slides you will be guided through the feedback • This is designed to help both trainees and trainers in interpreting feedback • You may want to go through this first and then look at your own/your trainee’s feedback matching it to the sample slides in this presentation. Copies of the feedback • Trainees receive a copy of the feedback excluding the group mean scores for their peer group (i.e. core [year 2] or post-core [year 4]). • Educational supervisors receive a copy of the feedback including the group mean scores for their trainee’s peer group (i.e. core [year 2] or post-core [year 4]). • Trainees need to retain a copy including the comparator data for their portfolio after discussion with their educational supervisor. Presentation of graph In response to feedback on the graph format we have produced a black and white version with different symbols for the 3 data points. This is an interim measure until we make further improvements where we hope to have colour and black and white versions available. The Feedback Graph These numbers refer to the scale of the SPRAT assessment tool where 1 = below expectations for a trainee at the same stage of training and 6 = above expectations for a trainee at the same stage of training. These data points () are your/your trainee’s self rating score for each question. These data points (×) are the mean score per question from assessors who responded to that question. These data points (Φ) are the mean score +/- 2 SD for the relevant peer group (i.e. core [year 2] or post-core [year 4]) for each question. Number of assessors who completed and submitted a SPRAT form. The questions of SPRAT, shown here divided into the domains of GMP. Good clinical care The questions of SPRAT, shown here divided into the domains of GMP. Maintaining good medical practice The questions of SPRAT, shown here divided into the domains of GMP. Teaching and training, appraising and assessing The questions of SPRAT, shown here divided into the domains of GMP. Relationship with patients The questions of SPRAT, shown here divided into the domains of GMP. Working with colleagues Statistical Interpretation Standard Deviations (SDs) are a measure of spread of data from the mean and not a range (range meaning highest and lowest scores) • Use of SDs helps provide a feel for where a score is in relation to the whole comparative group by giving some idea of the distribution of ratings around the cohort mean. • By focussing less on the actual mean score (remembering that 50% of people will of course be below the mean) you/your trainee should be able to place performance better in context. • For individual questions where the score falls close to or outside +2SDs, this may be an area for discussion of strengths. • For individual questions where the score falls close to or outside -2SDs, this may be an area of weakness which requires developmental planning. Statistical Interpretation Contd. • If you/your trainee scores less than 4.0 overall then further evaluation is advisable. Why does the standard deviation plot appear to show a score greater than 6 when 6 (above expectations) was the highest score obtainable? As standard deviation is calculated using a mathematical formula, this formula does not know that the maximum score a doctor can achieve is 6 and may provide values in excess. The Comments Sheet This is the overall self rating mean score for the 25 questions of SPRAT. This is the overall assessor mean score for the 25 questions of SPRAT. This is the overall mean score for all trainees in the relevant peer group (i.e. core [year 2] or post-core [year 4]) for the 25 questions of SPRAT. These are strengths identified by the assessors who completed the form. They are typed verbatim where each paragraph represents a comment from a different assessor. These are suggestions for development identified by assessors completing the form. They are typed verbatim where each paragraph represents a comment from a different assessor. These are probity or health concerns raised by assessors. They are typed verbatim where each paragraph represents a comment from a different assessor. Thank you for your time - we hope this guidance has been helpful