Download Update on the Lung Image Database Consortium (LIDC)

Michael F. McNitt-Gray, PhD: Update on the
lung image database consortium (LIDC): current
and future status of public imaging databases
Update on the Lung Image Database
Consortium (LIDC): Current and Future
Status of Public Image Databases
Mike McNitt
McNitt--Gray
P ofesso
Professor
Radiological Sciences
David Geffen School of Medicine at UCLA
For the LIDC, IDRI and RIDER groups
May 15, 2008- 8:10 AM
Lung Image Database Consortium
„
„
LIDC consortium funded by NCI
Five Institutions
„
„
„
„
„
Cornell (Weill Medical College)
U. Chicago
U. Iowa
U. Michigan
UCLA
Image Database Resource Initiative
(IDRI)
LIDC
Extension of LIDC funded using a Public Private
Partnership through the FNIH
LIDC Institutions Plus 2 Cancer Centers
Mission:
„ (a) to develop a publicly available image database
as a research resource for the development,
training and evaluation of lung CAD
training,
„
„
„
„
„
„
„
„
„
Cornell (Weill Medical College)
U. Chicago
U. Iowa
U. Michigan
UCLA
MD Anderson Cancer Center
Memorial Sloan Kettering Cancer Center
„
(b) create this database to enable the correlation
of performance of CAD methods for detection and
classification of lung nodules with spatial, temporal
and pathological ground truth.
LIDC
Creation of this database requires:
„ collection of an appropriate set of image data,
„ data to describe “truth” for each case.
Background
Wanted to provide information about
„
„
the presence or absence of nodules
the spatial extent of nodules.
Had expert thoracic radiologists read, annotate CTs.
However, recent studies indicate significant variability
between expert readers in nodule detection.
In addition, we observed significant variability
between experts in identification of nodule
boundaries
Stanford Radiology 10th Annual Multidetector
CT Symposium
1
Michael F. McNitt-Gray, PhD: Update on the
lung image database consortium (LIDC): current
and future status of public imaging databases
May 15, 2008- 8:10 AM
Data Collection
Methods
„
LIDC designed a two phase data collection process to
allow multiple (4) radiologists to review and annotate
each CT series across LIDC.
In the first or “blinded” phase, each of
four radiologists reviews the CT series
independently
Design
i Goals:
G l
„ No forced consensus (no truth panel)
„
Capture reader variability
„
Provide best estimate of truth
„
Asynchronous collaboration
Methods
„
In the second or “unblinded” review phase,
„
„
„
„
„
Results from all four blinded reviews are compiled
Presented to each radiologist for a second review.
Each radiologist independently reviews their own
annotations along with those of the other radiologists;
Each radiologist can modify their own annotations or
can choose to leave them unchanged.
Results from each of the 4 radiologist’s
unblinded reviews are compiled to form final
unblinded review
Stanford Radiology 10th Annual Multidetector
CT Symposium
The Marking Task
1. For Nodules ≥ 3 mm diameter
• Radiologist draws boundaries
• Description of characteristics
2. For Nodules < 3 mm
• Radiologist
d l
marks
k only
l centroid
d
• No description characteristics
3. For Non
Non--Nodules ≥ 3 mm
• Radiologist marks only centroid
• No description characteristics
2
Michael F. McNitt-Gray, PhD: Update on the
lung image database consortium (LIDC): current
and future status of public imaging databases
May 15, 2008- 8:10 AM
Blinded Read
APICAL SCAR – Non-Nodule > 3mm
Nodule < 3mm
Stanford Radiology 10th Annual Multidetector
CT Symposium
3
Michael F. McNitt-Gray, PhD: Update on the
lung image database consortium (LIDC): current
and future status of public imaging databases
May 15, 2008- 8:10 AM
NODULES ≥ 3mm; Provide Contour
UnBlinded Reading Session
„
„
Read ALL nodule markings
Decide which to keep
„
„
Non-Nodules - Scars in the Lung Apices
(Don’t have to edit/correct these)
Can keep,
keep delete or revise their own
Can add markings (based on what they
observe)
„
„
Can copy other’s markings (only for Nod < 3mm)
Create their own contours based on others’
markings (for Nod > 3mm)
Stanford Radiology 10th Annual Multidetector
CT Symposium
4
Michael F. McNitt-Gray, PhD: Update on the
lung image database consortium (LIDC): current
and future status of public imaging databases
May 15, 2008- 8:10 AM
Nodule < 3mm
Markings of Several Readers
No Forced Consensus
However, Reader can
modifyy their own
boundaries on UBR, if
they wish
For Nodules ≥ 3 mm,
fill out nodule worksheet
Current Status
Approximately 100 CT image data sets now
publicly available with markups in xml
format at:
http://ncia.nci.nih.gov/
Documentation on xml markup results
- under LIDC collection information
Stanford Radiology 10th Annual Multidetector
CT Symposium
5
Michael F. McNitt-Gray, PhD: Update on the
lung image database consortium (LIDC): current
and future status of public imaging databases
May 15, 2008- 8:10 AM
Current Status
RIDER (Reference Image Database to
Evaluate Response to Therapy):
„Also publicly available at www.ncia.nih.gov
„Serial CT exams of same patient
„No annotations
„No outcome regarding response
„Primarily for evaluating algorithms being
used to assess change
Near Future
In the next few months (~ Aug/Sept 2008):
„
LIDC will grow to approximately 400 CT cases
Pathology information (whenever available) will
be made available in the next few months
„
„
Near Future
RIDER datasets will expand to:
„
Images from Anthropomorphic Phantom studies
being carried out
Scans from “Coffee
Coffee Break
Break” experiments
„
„
DCE-MRI image data
DCEPET--CT (a few data sets there now and more
PET
phantom studies coming as well).
„
Entire LIDC/ IDRI dataset will be made
publicly available
„
1000 CTs
300 CXRs with CT correlation
„
Same patient scanned twice within a few minutes
„
„
A Little Further into the Future
„
Further Extension of RIDER data
„
Conclusion
„
NCI is sponsoring several publicly available dBs
„
LIDC
„
IDRI extension
t
i
„
„
„
„
Increased number of CT cases
CXR cases
RIDER
„
„
CT scans w/ markup and annotations, pathology, etc.
Multiple CT scans of patients to evaluate response to
treatment
Available at http://ncia.nci.nih.gov
Stanford Radiology 10th Annual Multidetector
CT Symposium
Acknowledgements
„
„
„
NCI
FNIH
Colleagues at
„
„
„
„
„
„
U. Chicago
U. Michigan
U. Iowa
Cornell
MD Anderson Cancer Center
Memorial Sloan Kettering Cancer Center
6