Survey
* Your assessment is very important for improving the workof artificial intelligence, which forms the content of this project
* Your assessment is very important for improving the workof artificial intelligence, which forms the content of this project
USING STRUCTURED REPORTING OF FOCAL LESIONS IN THE ABDOMEN TO ASSESS RADIOLOGY TRAINEES’ PERFORMANCE JOSEPH WILDENBERG, MD, PHD PO-HAO CHEN, MD, MBA HANNA M. ZAFAR, MD CHARLES E. KAHN, JR., MD, MS TESSA COOK, MD, PHD UNIVERSITY OF PENNSYLVANIA PHILADELPHIA, PA The authors have no financial relationship to disclose Purpose ¨ ¨ ¨ ¨ ¨ Call situations are often the only environment in which radiology trainees must produce an interpretation without immediate attending radiologist feedback. Understanding the types of error made by trainees can be difficult with free-form dictation. Compiling quantitative summary statistics for an individual or group of trainees is also challenging without clear classification of error types. Data from these situations may provide an opportunity to give focused feedback to trainees based on performance. Understanding group performance can allow identification of any gaps in trainee education and modification of the curriculum. Purpose ¨ ¨ ¨ Smith (1967) and Renfrew et al. (1992) described a system for classifying common radiology interpretation errors. Error Category Explanation Overreading Falsepositive Reporting an abnormality that is not present (e.g. a normal anatomical structure). Underreading Falsenegative The finding is missed but clearly present in hindsight Misinterpretation Truepositive A finding was seen, but misinterpreted due to lack of knowledge or faulty reasoning Structured reporting can facilitate easy classification of discrepancies using this system. A structured reporting initiative at our institution for describing abdominal solid-organ lesions provides a platform for categorizing discrepancies and compiling summary statistics. Structured Reporting ¨ ¨ ¨ ¨ Structured reporting is increasingly utilized to standardize the language used to describe findings, make follow-up recommendations, and for the overall organization of radiology reports. Initiatives to encourage the use of structured reporting are being advanced by the ACR, RSNA, and other major radiology organizations. The consistent use of structured reporting facilitates largescale data mining for research or educational goals. BI-RADS for breast imaging is the most well known and widely used. Many current initiatives are modeled after the successes demonstrated by BI-RADS. Code Abdomen ¨ Coding scheme for suspected malignancy in abdominal organs ¤ University of Pennsylvania structured reporting initiative ¤ Designed using ACR Breast Imaging Reporting and Data System (BI-RADS™) as a model. ¤ Liver, Kidneys, Pancreas, Adrenal glands ¨ Nine codes (0 through 7, plus 99) ¤ Five categories: benign, indeterminate, suspicious, known cancer, and non-diagnostic. ¤ Code 99 = focal masses cannot be excluded due to technical factors (e.g. incomplete visualization of the organ) Code Abdomen Category Classification Descriptor 0 Indeterminate Incompletely evaluated. If indicated within the patient’s clinical context, follow-up is advised 1 No mass 2 Benign 3 Indeterminate 4 Suspicious Suspicious. May represent malignancy. 5 Suspicious Highly suspicious. Clear imaging evidence of malignancy. 6 Malignant Known cancer. 7 Benign 99 Cannot be classified No mass. Benign. No further follow-up needed Indeterminate. Future imaging follow-up may be needed. Completely treated cancer. Technically inadequate for evaluation of masses. Methods ¨ ¨ ¨ All CT and US of the abdomen performed in a call situation between 7/1/13 and 12/15/14 required the use of the Code Abdomen structured reporting initiative. The numerical code assigned by the trainee for all visualized organs was compared to the final attending assignment, and any discrepancy noted. Discrepancies were categorized as follows: Detection: The trainee did not identify the lesion (1è2-7) ¤ Upgrade: The trainee identified the lesion, but incorrectly classified the lesion as benign (2è3-7) ¤ Downgrade: The trainee identified the lesion, but incorrectly classified the lesion as suspicious for malignancy (3-7è2) ¤ ¨ Trainees were grouped as either residents or fellows, and a Student’s t-test used to compare performance. Results ¨ 9484 studies met inclusion criteria ¤ ¤ ¨ ¨ ¨ The overall discrepancy rate was 2.9% for residents and 1.8% for fellows. A significant difference was seen between residents and fellows in the ability to detect lesions (underreading). No difference was seen in the ability for trainees to correctly characterize lesions (misinterpretation). ¤ ¨ 4419 interpreted by residents 5065 interpreted by fellows Due to low numbers, downgrade and upgrade were combined for analysis No discrepancies were found attributable to overreading. Characterize (not significant) Detection (p < 0.001) Conclusions ¨ ¨ ¨ ¨ ¨ Structured reporting of focal lesions in a call situation provides a method for objectively measuring errors made by radiology trainees. Using data from a structured reporting initiative for abdominal solid-organ lesions, we were able to compile group statistics and compare residents to fellows. Residents made more under-reading errors than fellows. Residents and fellows made similar errors of misinterpretation. This data could be used to target education of trainees to identify and address any system weaknesses. References ¨ ¨ ¨ ¨ Smith MJ. Error and Variation in Diagnostic Radiology. Springfield, IL: Thomas, 1967. Renfrew DL, et al. Error in radiology: classification and lessons in 182 cases presented at a problem case conference. Radiology 1992; 183:145-150. Kahn CE Jr, et al. Towards best practices in radiology reporting. Radiology 2009; 252:852-856 Zafar HM, et al. Code Abdomen: an assessment coding scheme for abdominal imaging findings possibly representing cancer. Journal of the American College of Radiology 2015 (in press).