Download Automatic Localization of Target Vertebrae in Spine Surgery using

Otake et al.
Automatic Localization of Target Vertebrae in Spine Surgery
using Fast CT-to-Fluoroscopy (3D-2D) Image Registration
Y. Otake,a,b S. Schafer,b J. W. Stayman,b W. Zbijewski,b G. Kleinszig,c R. Graumann,c
A. J. Khanna,d and J. H. Siewerdsena,b
a
Department of Computer Science, Johns Hopkins University, Baltimore MD
b
Department of Biomedical Engineering, Johns Hopkins University, Baltimore MD
c
Siemens AG, Healthcare Sector, Clinical Products Division, Erlangen, Germany
d
Department of Orthopaedic Surgery, Johns Hopkins Medical Institute, Baltimore MD
ABSTRACT
Localization of target vertebrae is an essential step in minimally invasive spine surgery, with conventional methods relying on “level counting” – i.e., manual counting of vertebrae under fluoroscopy starting from readily identifiable anatomy
(e.g., the sacrum). The approach requires an undesirable level of radiation, time, and is prone to counting errors due to
the similar appearance of vertebrae in projection images; wrong-level surgery occurs in 1 of every ~3000 cases. This
paper proposes a method to automatically localize target vertebrae in x-ray projections using 3D-2D registration between
preoperative CT (in which vertebrae are preoperatively labeled) and intraoperative fluoroscopy. The registration uses an
intensity-based approach with a gradient-based similarity metric and the CMA-ES algorithm for optimization. Digitally
reconstructed radiographs (DRRs) and a robust similarity metric are computed on GPU to accelerate the process. Evaluation in clinical CT data included 5,000 PA and LAT projections randomly perturbed to simulate human variability in
setup of mobile intraoperative C-arm. The method demonstrated 100% success for PA view (projection error: 0.42mm)
and 99.8% success for LAT view (projection error: 0.37mm). Initial implementation on GPU provided automatic target
localization within about 3 sec, with further improvement underway via multi-GPU. The ability to automatically label
vertebrae in fluoroscopy promises to streamline surgical workflow, improve patient safety, and reduce wrong-site surgeries, especially in large patients for whom manual methods are time consuming and error prone.
Keywords: 3D-2D registration, minimally invasive spine surgery, level localization, image-guided surgery, wrong-site
surgery, wrong-level surgery, patient safety, fluoroscopy
1. INTRODUCTION
Wrong site surgery results in failure to deliver proper therapy and carries profound medical, legal, and social implications. In spine surgery, the potential for wrong site surgery (viz., “wrong level” surgery, referring to the level
of vertebral body) is significant due to the difficulty of localization of target vertebrae based solely on visual impression, palpation, and fluoroscopy. Vertebrae in the thoracic region can be particularly challenging to localize,
since they have similar visual and radiographic appearance and are at a distance from unambiguous anatomical
landmarks. According to a study of 3,505 surgeons and 1,300,000 procedures, 418 (0.032%, 1 in 3,110) constituted wrong-level spine surgeries [1]. The conventional method to localize a vertebral level is to “count” vertebrae, typically beginning at the sacrum (or cervical spine) and then counting under fluoroscopic visualization up
to the targeted vertebral level. Such a method involves an undesirable amount of time and ionizing radiation. To
remedy the difficulty and reduce the risk of wrong-level surgery, this paper introduces a method to automatically
localize target vertebrae in intraoperative fluoroscopy based on target location which was preoperatively ident ified in preoperative CT. Using an intensity-based 3D-2D registration, the method searches the target anatomy
taking into account the tissue contrast of surrounding organs as well as the vertebrae structures themselves to improve accuracy and robustness of localization compared to manual counting of vertebral bodies. The method is
evaluated retrospectively using clinical CT data, and performance is assessed in terms of the success rate in vertebrae labeling in PA and LAT fluoroscopic views. The sensitivity of the proposed method to geometric calibration
accuracy is also evaluated.
2. METHODS
2.1. Method for 3D-2D Registration
Figure 1 illustrates the proposed method. Preoperative CT provides the basis for labeling of target vertebrae (by the
surgeon or a trained technologist) as either segmentations of individual vertebrae or, more simply, a nominal central
SPIE Medical Imaging 2012, Volume 8316
Contact: [email protected]
Otake et al.
marker location (x,y,z) defined in the CT coordinate system (FCT). Such preoperative labeling can be done in various offthe-shelf software utilities (e.g., 3D Slicer or ITK Snap) or, preferably, within the surgical planning software. The coorti
dinates of the ith target point ( PCT
) are transferred to the 3D-2D registration module (see inset detail of Figure 1) which
computes the transformation between FCT and the x-ray fluoroscopy coordinate system (Fx-ray), which we denote x-rayTCT,
representing a 4x4 homogeneous matrix (commonly referred to as the extrinsic parameter matrix). Then, using the calibration of the C-arm source-detector geometry (computed preoperatively as now common with 3D rotational C-arms) we
ti
have a 3x4 intrinsic parameter matrix C from which the projections of the target points ( Pimg
) were computed as follows.
ti
ti
ti
Pimg
 u v 1 ~ Cx  rayTCT  PCT
 PM  PCT
(i  1,2,, n)
T
n : number of target points
(1)
where the ~ symbol denotes that the left and right sides are equal to within scalar multiplication – i.e., (a b 1) ~ (A B C)
implying a = A/C and b = B/C. The projection matrix (PM) defines the projection geometry.
We employed an intensity-based rigid 3D-2D registration approach [2][3] in which an optimization algorithm searches
the 6DOF rigid transformation parameters to maximize similarity metric between the digitally reconstructed radiograph
(DRR) and the real 2D projection (fluoro image). We used Gradient Information (GI) [4] as a similarity metric and the
CMA-ES algorithm [5] as an optimizer. CMA-ES is a non-gradient stochastic search strategy known for robustness in a
rugged search space with the disadvantage of a larger number of required function evaluations in comparison to the
Nelder-Mead Downhill Simplex algorithm [6] or Powell’s method [7]. Initial studies employed CMA-ES implemented
in Matlab. To enhance computation time, computation of the DRR and GI similarity metric were implemented on GPU
using C++ and CUDA ver.4.0 as an external dynamic link library.
As an initialization for the optimizer, a 2 DOF brute-force type search was used. Two translational parameters, x and y
translation, were varied every 10 mm within the range covering entire CT volume, and a DRR was generated at each
pose. The pose that provided maximum similarity between the DRR and fluoroscopy images was used as an initialization
for the 6DOF optimization search detailed below.
Preoperative Process
Intraoperative Process
Preoperative CT
Fluoroscopy
Image
Vertebrae Definition
Geometric
Calibration
3D-2D
Registration
(Segmentation or Point identification
of individual vertebrae)
Auto-Labeled
Fluoroscopy image
detail
Preop CT + Labels
Geometric Calibration
Forward Projector
(GPU Accelerated)
(Grid Interpolation, Siddon)
Transformation
DRR
Fluoro
Similarity Metric
Optimizer
(e.g. Gradient information)
(CMA-ES)
No
Converged?
Apply
Labels
to
Fluoro
Yes
Figure 1. Overview of the proposed method for automatic labeling of vertebrae in intraoperative fluoroscopy. System
inputs are preoperative CT, CT labels (either sophisticated segmentations or simple point definitions at each vertebral
level), geometric calibration of the C-arm source-detector pose, and the (unlabeled) fluoroscopy. The 3D-2D registration
process is detailed in the callout flow diagram at the bottom. System output is a set of labels automatically applied to the
fluoroscopic images.
SPIE Medical Imaging 2012, Volume 8316
Contact: [email protected]
Otake et al.
2.2. Performance Evaluation
Performance of the proposed method was evaluated with a clinical CT dataset from The Cancer Imaging Archive (TCIA)
provided by National Cancer Institute (NCI) [8]. Figure 2a-c show example “preoperative” CT data in which the target
vertebrae were manually labeled using 3D Slicer (Brigham and Women’s Hospital, Boston MA). Table 2 summarizes
details of the CT volumes used in the experiment. The projection and volume data were down-sampled as listed in Table
2 to enhance computation speed. As discussed in Section 4, the particular down-sampling ratio is a trade-off between
computation time and robustness of the algorithm.
We defined the “default PA position” as a pose of the CT dataset with respect to the x-ray image coordinate system in
which: i.) the center of the CT volume was aligned at the isocenter of C-arm; ii.) the craniocaudal fluoro axis was aligned
with the z-axis of CT; and iii.) the horizontal axis in fluoroscopy was aligned with left-right axis in CT. Figure 2d shows
a DRR at the default PA position. We assumed a typical C-arm geometry: source-detector distance = 1200 mm, and
source-isocenter distance = 600 mm.
First, in order to evaluate the robustness of the method, we simulated x-ray images based on projection geometries that
were randomly perturbed from the default PA position. Assuming the spine to be reasonably well aligned at the center of
a PA fluoroscopy image acquired by a reasonably experienced fluoroscopist, we assumed gaussian distribution in perturbation from the default PA position. Six DOF random numbers based on a gaussian distribution (mean = 0, 3 = 50 mm
translation in (x), 100 mm translation in (y, z), and 10o rotation about the (x, y, z) axes) were generated, and the 4x4 homogeneous matrix representing random offset from the default PA position (Trand,) was computed based on these 6DOF
pose parameters. Since translation in the y direction (superior-anterior direction with respect to the patient) is unknown,
and the z direction is harder to align at the isocenter of the C-arm, we assumed larger variability in y and z translation. A
total of 5,000 random simulated x-ray images were generated from the CT dataset.
truei
The “true” vertebral positions in image coordinates ( Pimg
) were computed for each simulated x-ray projection image
using the random offset as follows:
truei
ti
Pimg
 u v 1 ~ PM  Trand  PCT
T
(2)
The proposed method was applied to each simulated x-ray image. Five vertebrae around the center of the CT volume
(T5-T9 in the example case; see Figure 2b) were chosen as target vertebrae, and the average distance between the true
esti
positions and the positions projected using the transformation estimated by the proposed method ( Pimg
) of these target
vertebrae at the detector plane were defined as the projection distance (PD) as follows:
PD 
1 5 truei
 Pimg  Pimgesti
5 i1
(3)
A successful trial was defined as any trial resulting in PD less than 5 mm – i.e., less than approximately half the width or
height of a vertebra, such that the projected point is expected to be within the projection of the vertebral body. The ratio
of the number of successful trials to the total number of trials was defined as the success rate (%). Table 2 summarizes
the overall hardware specifications used in the experiments.
Target
vertebrae for
evaluation
T7
(a)
T2
T3
T4
T5
T6
T7
T8
T9
T10
T11
T12 (b)
(c)
(d)
Figure 2. Example CT dataset used in system evaluation. (a-c) Axial, coronal, and sagittal slices showing the labeled
locations of vertebrae (T2-T12). (d) DRR at the default PA position. (See text for details.)
SPIE Medical Imaging 2012, Volume 8316
Contact: [email protected]
Otake et al.
Table 1. Details of the fluoroscopy and CT volume data formats. In
the initial implementation, both images were down-sampled to accelerate computation speed of DRR and similarity measure.
Fluoro image
# Pixels
Original
Downsampled
Pixel size
(mm)
Preoperative CT volume
# Voxels
Voxel size
(mm)
768x768 0.388x0.388 512x512x588 0.51x0.51x0.50
96x96
3.104x3.104 128x128x147
2.04x2.04x2.0
Table 2. Specifications of the hardware
used in the initial implementation.
Windows 7 64 bit
Intel® Xeon®
(2 processors)
2.00 GHz
NVIDIA® GeForce®
GTX 470
448
133.9 GB/s
1280 MB
OS
Processor Type
CPU Clock
Graphics Card
# CUDA Cores
Memory BW
Graphics Mem.
Lateral fluoroscopy (i.e., projection images acquired through the right-left axis of the patient) was similarly evaluated. In
this case, the “default LAT view” was defined as the pose in which the horizontal axis in fluoroscopy was aligned with
the anterior-posterior axis in the CT dataset. A randomly computed perturbed projection geometry was determined in the
same way as described above based on the default LAT view.
The sensitivity of the proposed method to geometric calibration accuracy was also evaluated. In a realistic clinical scenario, a precise geometric calibration of the C-arm geometry is sometimes unavailable (especially in case of conventional
image intensifier C-arm not intended for 3D imaging). We hypothesized that the proposed method would be insensitive
to the error in geometric calibration since it computes the projected target position based on the relative relationship with
the features of surrounding tissues. Although the estimation of absolute position of the patient is susceptible to geometric
calibration errors, the relative position may not be sensitive to it. The resulting experiment involved addition of an offset
(300mm) to the translational components of the geometric calibration parameter (relative position between detector and
source), and generated simulated x-ray images. A total of 150 trials were then conducted on these simulated x-ray images
using the proposed method with the original geometric calibration parameter and success rate was computed for each
offset value.
3. RESULTS
3.1 Evaluation using Random C-arm Poses
The results of 5,000 simulation trials for both PA and LAT views are summarized in Table 1. The median value of the
projection distance error was 0.42 mm and 0.37 mm, respectively. Computation time was 2.88 0.32 sec and 3.13
0.35 sec after approximately 1000 function evaluations. We did not find a failure in trials of the PA view, and we observed 8 failures (i.e., projection distance exceeding 5 mm) in the LAT view trials, resulting 100 % and 99.84 % success
rate respectively. Figure 3 shows example simulated x-ray images used in the experiment. As shown in the LAT view
images, the failure cases present a small amount of image features due to overlapping surrounding structures, which introduced local minima in the search space, resulting in less robust 3D-2D registration than observed for the PA view.
Table 3 Results of 5,000 simulation trials using randomly generated C-arm geometry
PA view
LAT view
Projection distance error, median
Computation time
Number of function evaluation
Number of failures
Success rate
0.42 mm
0.37 mm
2.88 0.32 sec
3.13 0.35 sec
1053 133
1060 160
0
8
100 %
99.84 %
SPIE Medical Imaging 2012, Volume 8316
Contact: [email protected]
Otake et al.
PA
LAT
Center
(FOV: 15 cm)
Rotation
(+10 deg)
Magnification
(FOV: 11.25 cm)
Figure 3 Simulated x-ray images generated in the system evaluation. The labels of vertebrae are overlaid in magenta.
Each row shows different poses of the C-arm. (left column) AP view, (right column) lateral view.
3.2 Evaluation of sensitivity to geometric calibration accuracy
Figure 4 summarizes results of the sensitivity analysis to geometric calibration error in x, y, and z translation components. The plots show 100% success rate for up to 200 mm translational offset in all directions. Daly et al [9] reported that variation in geometric calibration for various C-arm poses was well within 10 mm from its average value.
This suggests that the proposed method is sufficiently tolerant to geometric calibration errors that are likely to occur in
realistic C-arm devices due to mechanical flex of the gantry or irreproducibility. Furthermore, in the case of a conventional C-arm device for which the geometric calibration parameters are difficult to acquire (e.g., conventional fluoroscopic C-arms for which geometric calibration is not part of regular quality assurance), the proposed method could be
applied using approximated parameters that were either derived from one-time calibration using a specially designed
calibration phantom or a manual external measurement (e.g., approximate distance between the detector center and an
approximate position of the x-ray source).
SPIE Medical Imaging 2012, Volume 8316
Contact: [email protected]
X translation (mm)
Success rate (%)
Success rate (%)
Success rate (%)
Otake et al.
Y translation (mm)
Z translation (mm)
Figure 4 Analysis of sensitivity to geometric calibration error. A translational offset was added to the relative position
between the detector and source to simulate errors in geometric calibration, and the success rate was computed based on
150 trials for each case.
4. CONCLUSIONS
A method for automatic localization and labeling of vertebrae in intraoperative fluoroscopy has been developed as a new
tool for minimally invasive spine surgery, offering potentially streamlined surgical workflow, reduced radiation dose,
and more accurate localization of target vertebrae compared to conventional, manual methods. The method incorporates
intensity-based 3D-2D registration to estimate the transformation between preoperative CT and fluoroscopic projections.
An initial simulation study demonstrated excellent performance (>99% success) in localizing target vertebrae in both PA
and LAT views. Computation time in the initial implementation was ~3 sec, suggesting potential utility in realistic
workflow, with further improvement anticipated via multi-GPU implementation. Future work includes account of patient
deformation between preoperative CT and intraoperative fluoroscopy (e.g., due to difference in patient setup and movement of thoracic cage due to breathing), further improvement in robustness in LAT views by tuning optimization parameters or combining multiple similarity metrics, and translation to retrospective and/or prospective clinical pilot studies.
ACKNOWLEDGEMENT
This work was supported by the National Institutes of Health Grant No. R01-CA-127444 and academic-industry partnership with Siemens Healthcare (XP, Erlangen Germany). The authors thank Dr. S. C. Mears (Department of Orthopaedic
Surgery, Johns Hopkins University) and Dr. J. A. Carrino (Department of Radiology, Johns Hopkins Univerisity) for
valuable discussion.
REFERENCES
[1] M. G. Mody, A. Nourbakhsh, D. L. Stahl, M. Gibbs, M. Alfawareh and K. J. Garges, "The prevalence of wrong level surgery among spine surgeons," Spine (Phila Pa. 1976) 33, 194-198 (2008).
[2] J. Weese, G. P. Penney, P. Desmedt, T. M. Buzug, D. L. Hill and D. J. Hawkes, "Voxel-based 2-D/3-D registration of fluoroscopy images and CT
scans for image-guided surgery," IEEE Trans. Inf. Technol. Biomed. 1, 284-293 (1997).
[3] Y. Otake, M. Armand, R. Armiger, M. Kutzer, E. Basafa, P. Kazanzides and R. Taylor, "Intraoperative Image-based Multi-view 2D/3D Registration for Image-guided Orthopaedic Surgery: Incorporation of Fiducial-based C-arm Tracking and GPU-acceleration," IEEE Trans. Med. Imaging,
(in Press, epub date 18 Nov 2011).
[4] J. P. Pluim, J. B. Maintz and M. A. Viergever, "Image registration by maximization of combined mutual information and gradient information,"
IEEE Trans. Med. Imaging 19, 809-814 (2000).
[5] N. Hansen, "The CMA evolution strategy: A comparing review," in Towards a New Evolutionary Computation. Advances on Estimation of Distribution Algorithms, J. A. Lozano, P. Larranaga, I. Inza and E. Bengoetxea, Eds. Springer, 75-102 (2006).
[6] J. A. Nelder and R. Mead, "A Simplex Method for Function Minimization," The Computer Journa, 7, 308-313 (1965).
[7] M. J. D. Powell, "An efficient method for finding the minimum of a function of several variables without calculating derivatives," The Computer
Journal 7, 155-162 (1964).
[8] S. G. Armato, G. McLennan, L. Bidaut, et al., "The Lung Image Database Consortium (LIDC) and Image Database Resource Initiative (IDRI): A
Completed Reference Database of Lung Nodules on CT Scans," Med. Phys. 38, 915-17 (2011).
[9] M. J. Daly, J. H. Siewerdsen, Y. B. Cho, D. A. Jaffray and J. C. Irish, "Geometric calibration of a mobile C-arm for intraoperative cone-beam CT,"
Med. Phys. 35, 2124-2136 (2008).
SPIE Medical Imaging 2012, Volume 8316
Contact: [email protected]