Download USING STRUCTURED REPORTING OF FOCAL LESIONS IN THE

USING STRUCTURED REPORTING OF
FOCAL LESIONS IN THE ABDOMEN
TO ASSESS RADIOLOGY TRAINEES’
PERFORMANCE
JOSEPH WILDENBERG, MD, PHD
PO-HAO CHEN, MD, MBA
HANNA M. ZAFAR, MD
CHARLES E. KAHN, JR., MD, MS
TESSA COOK, MD, PHD
UNIVERSITY OF PENNSYLVANIA
PHILADELPHIA, PA
The authors have no financial relationship to disclose
Purpose
¨ 
¨ 
¨ 
¨ 
¨ 
Call situations are often the only environment in which
radiology trainees must produce an interpretation without
immediate attending radiologist feedback.
Understanding the types of error made by trainees can be
difficult with free-form dictation.
Compiling quantitative summary statistics for an individual or
group of trainees is also challenging without clear
classification of error types.
Data from these situations may provide an opportunity to
give focused feedback to trainees based on performance.
Understanding group performance can allow identification of
any gaps in trainee education and modification of the
curriculum.
Purpose
¨ 
¨ 
¨ 
Smith (1967) and Renfrew
et al. (1992) described a
system for classifying
common radiology
interpretation errors.
Error
Category
Explanation
Overreading
Falsepositive
Reporting an abnormality that
is not present (e.g. a normal
anatomical structure).
Underreading
Falsenegative
The finding is missed but
clearly present in hindsight
Misinterpretation
Truepositive
A finding was seen, but
misinterpreted due to lack of
knowledge or faulty reasoning
Structured reporting can
facilitate easy classification
of discrepancies using this
system.
A structured reporting
initiative at our institution
for describing abdominal
solid-organ lesions provides
a platform for categorizing
discrepancies and compiling
summary statistics.
Structured Reporting
¨ 
¨ 
¨ 
¨ 
Structured reporting is increasingly utilized to standardize
the language used to describe findings, make follow-up
recommendations, and for the overall organization of
radiology reports.
Initiatives to encourage the use of structured reporting are
being advanced by the ACR, RSNA, and other major
radiology organizations.
The consistent use of structured reporting facilitates largescale data mining for research or educational goals.
BI-RADS for breast imaging is the most well known and
widely used. Many current initiatives are modeled after the
successes demonstrated by BI-RADS.
Code Abdomen
¨ 
Coding scheme for suspected malignancy in abdominal
organs
¤  University
of Pennsylvania structured reporting initiative
¤  Designed using ACR Breast Imaging Reporting and Data
System (BI-RADS™) as a model.
¤  Liver, Kidneys, Pancreas, Adrenal glands
¨ 
Nine codes (0 through 7, plus 99)
¤  Five
categories: benign, indeterminate, suspicious, known
cancer, and non-diagnostic.
¤  Code 99 = focal masses cannot be excluded due to
technical factors (e.g. incomplete visualization of the organ)
Code Abdomen
Category
Classification
Descriptor
0
Indeterminate
Incompletely evaluated. If indicated within the patient’s clinical context,
follow-up is advised
1
No mass
2
Benign
3
Indeterminate
4
Suspicious
Suspicious. May represent malignancy.
5
Suspicious
Highly suspicious. Clear imaging evidence of malignancy.
6
Malignant
Known cancer.
7
Benign
99
Cannot be
classified
No mass.
Benign. No further follow-up needed
Indeterminate. Future imaging follow-up may be needed.
Completely treated cancer.
Technically inadequate for evaluation of masses.
Methods
¨ 
¨ 
¨ 
All CT and US of the abdomen performed in a call situation
between 7/1/13 and 12/15/14 required the use of the
Code Abdomen structured reporting initiative.
The numerical code assigned by the trainee for all visualized
organs was compared to the final attending assignment, and
any discrepancy noted.
Discrepancies were categorized as follows:
Detection: The trainee did not identify the lesion (1è2-7)
¤  Upgrade: The trainee identified the lesion, but incorrectly
classified the lesion as benign (2è3-7)
¤  Downgrade: The trainee identified the lesion, but incorrectly
classified the lesion as suspicious for malignancy (3-7è2)
¤ 
¨ 
Trainees were grouped as either residents or fellows, and a
Student’s t-test used to compare performance.
Results
¨ 
9484 studies met inclusion criteria
¤ 
¤ 
¨ 
¨ 
¨ 
The overall discrepancy rate was
2.9% for residents and 1.8% for
fellows.
A significant difference was seen
between residents and fellows in the
ability to detect lesions
(underreading).
No difference was seen in the ability
for trainees to correctly characterize
lesions (misinterpretation).
¤ 
¨ 
4419 interpreted by residents
5065 interpreted by fellows
Due to low numbers, downgrade and
upgrade were combined for analysis
No discrepancies were found
attributable to overreading.
Characterize (not
significant)
Detection (p < 0.001)
Conclusions
¨ 
¨ 
¨ 
¨ 
¨ 
Structured reporting of focal lesions in a call situation
provides a method for objectively measuring errors
made by radiology trainees.
Using data from a structured reporting initiative for
abdominal solid-organ lesions, we were able to compile
group statistics and compare residents to fellows.
Residents made more under-reading errors than fellows.
Residents and fellows made similar errors of
misinterpretation.
This data could be used to target education of trainees
to identify and address any system weaknesses.
References
¨ 
¨ 
¨ 
¨ 
Smith MJ. Error and Variation in Diagnostic Radiology.
Springfield, IL: Thomas, 1967.
Renfrew DL, et al. Error in radiology: classification and lessons
in 182 cases presented at a problem case conference.
Radiology 1992; 183:145-150.
Kahn CE Jr, et al. Towards best practices in radiology
reporting. Radiology 2009; 252:852-856
Zafar HM, et al. Code Abdomen: an assessment coding
scheme for abdominal imaging findings possibly representing
cancer. Journal of the American College of Radiology 2015 (in
press).