Survey
* Your assessment is very important for improving the workof artificial intelligence, which forms the content of this project
* Your assessment is very important for improving the workof artificial intelligence, which forms the content of this project
Otake et al. Automatic Localization of Target Vertebrae in Spine Surgery using Fast CT-to-Fluoroscopy (3D-2D) Image Registration Y. Otake,a,b S. Schafer,b J. W. Stayman,b W. Zbijewski,b G. Kleinszig,c R. Graumann,c A. J. Khanna,d and J. H. Siewerdsena,b a Department of Computer Science, Johns Hopkins University, Baltimore MD b Department of Biomedical Engineering, Johns Hopkins University, Baltimore MD c Siemens AG, Healthcare Sector, Clinical Products Division, Erlangen, Germany d Department of Orthopaedic Surgery, Johns Hopkins Medical Institute, Baltimore MD ABSTRACT Localization of target vertebrae is an essential step in minimally invasive spine surgery, with conventional methods relying on “level counting” – i.e., manual counting of vertebrae under fluoroscopy starting from readily identifiable anatomy (e.g., the sacrum). The approach requires an undesirable level of radiation, time, and is prone to counting errors due to the similar appearance of vertebrae in projection images; wrong-level surgery occurs in 1 of every ~3000 cases. This paper proposes a method to automatically localize target vertebrae in x-ray projections using 3D-2D registration between preoperative CT (in which vertebrae are preoperatively labeled) and intraoperative fluoroscopy. The registration uses an intensity-based approach with a gradient-based similarity metric and the CMA-ES algorithm for optimization. Digitally reconstructed radiographs (DRRs) and a robust similarity metric are computed on GPU to accelerate the process. Evaluation in clinical CT data included 5,000 PA and LAT projections randomly perturbed to simulate human variability in setup of mobile intraoperative C-arm. The method demonstrated 100% success for PA view (projection error: 0.42mm) and 99.8% success for LAT view (projection error: 0.37mm). Initial implementation on GPU provided automatic target localization within about 3 sec, with further improvement underway via multi-GPU. The ability to automatically label vertebrae in fluoroscopy promises to streamline surgical workflow, improve patient safety, and reduce wrong-site surgeries, especially in large patients for whom manual methods are time consuming and error prone. Keywords: 3D-2D registration, minimally invasive spine surgery, level localization, image-guided surgery, wrong-site surgery, wrong-level surgery, patient safety, fluoroscopy 1. INTRODUCTION Wrong site surgery results in failure to deliver proper therapy and carries profound medical, legal, and social implications. In spine surgery, the potential for wrong site surgery (viz., “wrong level” surgery, referring to the level of vertebral body) is significant due to the difficulty of localization of target vertebrae based solely on visual impression, palpation, and fluoroscopy. Vertebrae in the thoracic region can be particularly challenging to localize, since they have similar visual and radiographic appearance and are at a distance from unambiguous anatomical landmarks. According to a study of 3,505 surgeons and 1,300,000 procedures, 418 (0.032%, 1 in 3,110) constituted wrong-level spine surgeries [1]. The conventional method to localize a vertebral level is to “count” vertebrae, typically beginning at the sacrum (or cervical spine) and then counting under fluoroscopic visualization up to the targeted vertebral level. Such a method involves an undesirable amount of time and ionizing radiation. To remedy the difficulty and reduce the risk of wrong-level surgery, this paper introduces a method to automatically localize target vertebrae in intraoperative fluoroscopy based on target location which was preoperatively ident ified in preoperative CT. Using an intensity-based 3D-2D registration, the method searches the target anatomy taking into account the tissue contrast of surrounding organs as well as the vertebrae structures themselves to improve accuracy and robustness of localization compared to manual counting of vertebral bodies. The method is evaluated retrospectively using clinical CT data, and performance is assessed in terms of the success rate in vertebrae labeling in PA and LAT fluoroscopic views. The sensitivity of the proposed method to geometric calibration accuracy is also evaluated. 2. METHODS 2.1. Method for 3D-2D Registration Figure 1 illustrates the proposed method. Preoperative CT provides the basis for labeling of target vertebrae (by the surgeon or a trained technologist) as either segmentations of individual vertebrae or, more simply, a nominal central SPIE Medical Imaging 2012, Volume 8316 Contact: [email protected] Otake et al. marker location (x,y,z) defined in the CT coordinate system (FCT). Such preoperative labeling can be done in various offthe-shelf software utilities (e.g., 3D Slicer or ITK Snap) or, preferably, within the surgical planning software. The coorti dinates of the ith target point ( PCT ) are transferred to the 3D-2D registration module (see inset detail of Figure 1) which computes the transformation between FCT and the x-ray fluoroscopy coordinate system (Fx-ray), which we denote x-rayTCT, representing a 4x4 homogeneous matrix (commonly referred to as the extrinsic parameter matrix). Then, using the calibration of the C-arm source-detector geometry (computed preoperatively as now common with 3D rotational C-arms) we ti have a 3x4 intrinsic parameter matrix C from which the projections of the target points ( Pimg ) were computed as follows. ti ti ti Pimg u v 1 ~ Cx rayTCT PCT PM PCT (i 1,2,, n) T n : number of target points (1) where the ~ symbol denotes that the left and right sides are equal to within scalar multiplication – i.e., (a b 1) ~ (A B C) implying a = A/C and b = B/C. The projection matrix (PM) defines the projection geometry. We employed an intensity-based rigid 3D-2D registration approach [2][3] in which an optimization algorithm searches the 6DOF rigid transformation parameters to maximize similarity metric between the digitally reconstructed radiograph (DRR) and the real 2D projection (fluoro image). We used Gradient Information (GI) [4] as a similarity metric and the CMA-ES algorithm [5] as an optimizer. CMA-ES is a non-gradient stochastic search strategy known for robustness in a rugged search space with the disadvantage of a larger number of required function evaluations in comparison to the Nelder-Mead Downhill Simplex algorithm [6] or Powell’s method [7]. Initial studies employed CMA-ES implemented in Matlab. To enhance computation time, computation of the DRR and GI similarity metric were implemented on GPU using C++ and CUDA ver.4.0 as an external dynamic link library. As an initialization for the optimizer, a 2 DOF brute-force type search was used. Two translational parameters, x and y translation, were varied every 10 mm within the range covering entire CT volume, and a DRR was generated at each pose. The pose that provided maximum similarity between the DRR and fluoroscopy images was used as an initialization for the 6DOF optimization search detailed below. Preoperative Process Intraoperative Process Preoperative CT Fluoroscopy Image Vertebrae Definition Geometric Calibration 3D-2D Registration (Segmentation or Point identification of individual vertebrae) Auto-Labeled Fluoroscopy image detail Preop CT + Labels Geometric Calibration Forward Projector (GPU Accelerated) (Grid Interpolation, Siddon) Transformation DRR Fluoro Similarity Metric Optimizer (e.g. Gradient information) (CMA-ES) No Converged? Apply Labels to Fluoro Yes Figure 1. Overview of the proposed method for automatic labeling of vertebrae in intraoperative fluoroscopy. System inputs are preoperative CT, CT labels (either sophisticated segmentations or simple point definitions at each vertebral level), geometric calibration of the C-arm source-detector pose, and the (unlabeled) fluoroscopy. The 3D-2D registration process is detailed in the callout flow diagram at the bottom. System output is a set of labels automatically applied to the fluoroscopic images. SPIE Medical Imaging 2012, Volume 8316 Contact: [email protected] Otake et al. 2.2. Performance Evaluation Performance of the proposed method was evaluated with a clinical CT dataset from The Cancer Imaging Archive (TCIA) provided by National Cancer Institute (NCI) [8]. Figure 2a-c show example “preoperative” CT data in which the target vertebrae were manually labeled using 3D Slicer (Brigham and Women’s Hospital, Boston MA). Table 2 summarizes details of the CT volumes used in the experiment. The projection and volume data were down-sampled as listed in Table 2 to enhance computation speed. As discussed in Section 4, the particular down-sampling ratio is a trade-off between computation time and robustness of the algorithm. We defined the “default PA position” as a pose of the CT dataset with respect to the x-ray image coordinate system in which: i.) the center of the CT volume was aligned at the isocenter of C-arm; ii.) the craniocaudal fluoro axis was aligned with the z-axis of CT; and iii.) the horizontal axis in fluoroscopy was aligned with left-right axis in CT. Figure 2d shows a DRR at the default PA position. We assumed a typical C-arm geometry: source-detector distance = 1200 mm, and source-isocenter distance = 600 mm. First, in order to evaluate the robustness of the method, we simulated x-ray images based on projection geometries that were randomly perturbed from the default PA position. Assuming the spine to be reasonably well aligned at the center of a PA fluoroscopy image acquired by a reasonably experienced fluoroscopist, we assumed gaussian distribution in perturbation from the default PA position. Six DOF random numbers based on a gaussian distribution (mean = 0, 3 = 50 mm translation in (x), 100 mm translation in (y, z), and 10o rotation about the (x, y, z) axes) were generated, and the 4x4 homogeneous matrix representing random offset from the default PA position (Trand,) was computed based on these 6DOF pose parameters. Since translation in the y direction (superior-anterior direction with respect to the patient) is unknown, and the z direction is harder to align at the isocenter of the C-arm, we assumed larger variability in y and z translation. A total of 5,000 random simulated x-ray images were generated from the CT dataset. truei The “true” vertebral positions in image coordinates ( Pimg ) were computed for each simulated x-ray projection image using the random offset as follows: truei ti Pimg u v 1 ~ PM Trand PCT T (2) The proposed method was applied to each simulated x-ray image. Five vertebrae around the center of the CT volume (T5-T9 in the example case; see Figure 2b) were chosen as target vertebrae, and the average distance between the true esti positions and the positions projected using the transformation estimated by the proposed method ( Pimg ) of these target vertebrae at the detector plane were defined as the projection distance (PD) as follows: PD 1 5 truei Pimg Pimgesti 5 i1 (3) A successful trial was defined as any trial resulting in PD less than 5 mm – i.e., less than approximately half the width or height of a vertebra, such that the projected point is expected to be within the projection of the vertebral body. The ratio of the number of successful trials to the total number of trials was defined as the success rate (%). Table 2 summarizes the overall hardware specifications used in the experiments. Target vertebrae for evaluation T7 (a) T2 T3 T4 T5 T6 T7 T8 T9 T10 T11 T12 (b) (c) (d) Figure 2. Example CT dataset used in system evaluation. (a-c) Axial, coronal, and sagittal slices showing the labeled locations of vertebrae (T2-T12). (d) DRR at the default PA position. (See text for details.) SPIE Medical Imaging 2012, Volume 8316 Contact: [email protected] Otake et al. Table 1. Details of the fluoroscopy and CT volume data formats. In the initial implementation, both images were down-sampled to accelerate computation speed of DRR and similarity measure. Fluoro image # Pixels Original Downsampled Pixel size (mm) Preoperative CT volume # Voxels Voxel size (mm) 768x768 0.388x0.388 512x512x588 0.51x0.51x0.50 96x96 3.104x3.104 128x128x147 2.04x2.04x2.0 Table 2. Specifications of the hardware used in the initial implementation. Windows 7 64 bit Intel® Xeon® (2 processors) 2.00 GHz NVIDIA® GeForce® GTX 470 448 133.9 GB/s 1280 MB OS Processor Type CPU Clock Graphics Card # CUDA Cores Memory BW Graphics Mem. Lateral fluoroscopy (i.e., projection images acquired through the right-left axis of the patient) was similarly evaluated. In this case, the “default LAT view” was defined as the pose in which the horizontal axis in fluoroscopy was aligned with the anterior-posterior axis in the CT dataset. A randomly computed perturbed projection geometry was determined in the same way as described above based on the default LAT view. The sensitivity of the proposed method to geometric calibration accuracy was also evaluated. In a realistic clinical scenario, a precise geometric calibration of the C-arm geometry is sometimes unavailable (especially in case of conventional image intensifier C-arm not intended for 3D imaging). We hypothesized that the proposed method would be insensitive to the error in geometric calibration since it computes the projected target position based on the relative relationship with the features of surrounding tissues. Although the estimation of absolute position of the patient is susceptible to geometric calibration errors, the relative position may not be sensitive to it. The resulting experiment involved addition of an offset (300mm) to the translational components of the geometric calibration parameter (relative position between detector and source), and generated simulated x-ray images. A total of 150 trials were then conducted on these simulated x-ray images using the proposed method with the original geometric calibration parameter and success rate was computed for each offset value. 3. RESULTS 3.1 Evaluation using Random C-arm Poses The results of 5,000 simulation trials for both PA and LAT views are summarized in Table 1. The median value of the projection distance error was 0.42 mm and 0.37 mm, respectively. Computation time was 2.88 0.32 sec and 3.13 0.35 sec after approximately 1000 function evaluations. We did not find a failure in trials of the PA view, and we observed 8 failures (i.e., projection distance exceeding 5 mm) in the LAT view trials, resulting 100 % and 99.84 % success rate respectively. Figure 3 shows example simulated x-ray images used in the experiment. As shown in the LAT view images, the failure cases present a small amount of image features due to overlapping surrounding structures, which introduced local minima in the search space, resulting in less robust 3D-2D registration than observed for the PA view. Table 3 Results of 5,000 simulation trials using randomly generated C-arm geometry PA view LAT view Projection distance error, median Computation time Number of function evaluation Number of failures Success rate 0.42 mm 0.37 mm 2.88 0.32 sec 3.13 0.35 sec 1053 133 1060 160 0 8 100 % 99.84 % SPIE Medical Imaging 2012, Volume 8316 Contact: [email protected] Otake et al. PA LAT Center (FOV: 15 cm) Rotation (+10 deg) Magnification (FOV: 11.25 cm) Figure 3 Simulated x-ray images generated in the system evaluation. The labels of vertebrae are overlaid in magenta. Each row shows different poses of the C-arm. (left column) AP view, (right column) lateral view. 3.2 Evaluation of sensitivity to geometric calibration accuracy Figure 4 summarizes results of the sensitivity analysis to geometric calibration error in x, y, and z translation components. The plots show 100% success rate for up to 200 mm translational offset in all directions. Daly et al [9] reported that variation in geometric calibration for various C-arm poses was well within 10 mm from its average value. This suggests that the proposed method is sufficiently tolerant to geometric calibration errors that are likely to occur in realistic C-arm devices due to mechanical flex of the gantry or irreproducibility. Furthermore, in the case of a conventional C-arm device for which the geometric calibration parameters are difficult to acquire (e.g., conventional fluoroscopic C-arms for which geometric calibration is not part of regular quality assurance), the proposed method could be applied using approximated parameters that were either derived from one-time calibration using a specially designed calibration phantom or a manual external measurement (e.g., approximate distance between the detector center and an approximate position of the x-ray source). SPIE Medical Imaging 2012, Volume 8316 Contact: [email protected] X translation (mm) Success rate (%) Success rate (%) Success rate (%) Otake et al. Y translation (mm) Z translation (mm) Figure 4 Analysis of sensitivity to geometric calibration error. A translational offset was added to the relative position between the detector and source to simulate errors in geometric calibration, and the success rate was computed based on 150 trials for each case. 4. CONCLUSIONS A method for automatic localization and labeling of vertebrae in intraoperative fluoroscopy has been developed as a new tool for minimally invasive spine surgery, offering potentially streamlined surgical workflow, reduced radiation dose, and more accurate localization of target vertebrae compared to conventional, manual methods. The method incorporates intensity-based 3D-2D registration to estimate the transformation between preoperative CT and fluoroscopic projections. An initial simulation study demonstrated excellent performance (>99% success) in localizing target vertebrae in both PA and LAT views. Computation time in the initial implementation was ~3 sec, suggesting potential utility in realistic workflow, with further improvement anticipated via multi-GPU implementation. Future work includes account of patient deformation between preoperative CT and intraoperative fluoroscopy (e.g., due to difference in patient setup and movement of thoracic cage due to breathing), further improvement in robustness in LAT views by tuning optimization parameters or combining multiple similarity metrics, and translation to retrospective and/or prospective clinical pilot studies. ACKNOWLEDGEMENT This work was supported by the National Institutes of Health Grant No. R01-CA-127444 and academic-industry partnership with Siemens Healthcare (XP, Erlangen Germany). The authors thank Dr. S. C. Mears (Department of Orthopaedic Surgery, Johns Hopkins University) and Dr. J. A. Carrino (Department of Radiology, Johns Hopkins Univerisity) for valuable discussion. REFERENCES [1] M. G. Mody, A. Nourbakhsh, D. L. Stahl, M. Gibbs, M. Alfawareh and K. J. Garges, "The prevalence of wrong level surgery among spine surgeons," Spine (Phila Pa. 1976) 33, 194-198 (2008). [2] J. Weese, G. P. Penney, P. Desmedt, T. M. Buzug, D. L. Hill and D. J. Hawkes, "Voxel-based 2-D/3-D registration of fluoroscopy images and CT scans for image-guided surgery," IEEE Trans. Inf. Technol. Biomed. 1, 284-293 (1997). [3] Y. Otake, M. Armand, R. Armiger, M. Kutzer, E. Basafa, P. Kazanzides and R. Taylor, "Intraoperative Image-based Multi-view 2D/3D Registration for Image-guided Orthopaedic Surgery: Incorporation of Fiducial-based C-arm Tracking and GPU-acceleration," IEEE Trans. Med. Imaging, (in Press, epub date 18 Nov 2011). [4] J. P. Pluim, J. B. Maintz and M. A. Viergever, "Image registration by maximization of combined mutual information and gradient information," IEEE Trans. Med. Imaging 19, 809-814 (2000). [5] N. Hansen, "The CMA evolution strategy: A comparing review," in Towards a New Evolutionary Computation. Advances on Estimation of Distribution Algorithms, J. A. Lozano, P. Larranaga, I. Inza and E. Bengoetxea, Eds. Springer, 75-102 (2006). [6] J. A. Nelder and R. Mead, "A Simplex Method for Function Minimization," The Computer Journa, 7, 308-313 (1965). [7] M. J. D. Powell, "An efficient method for finding the minimum of a function of several variables without calculating derivatives," The Computer Journal 7, 155-162 (1964). [8] S. G. Armato, G. McLennan, L. Bidaut, et al., "The Lung Image Database Consortium (LIDC) and Image Database Resource Initiative (IDRI): A Completed Reference Database of Lung Nodules on CT Scans," Med. Phys. 38, 915-17 (2011). [9] M. J. Daly, J. H. Siewerdsen, Y. B. Cho, D. A. Jaffray and J. C. Irish, "Geometric calibration of a mobile C-arm for intraoperative cone-beam CT," Med. Phys. 35, 2124-2136 (2008). SPIE Medical Imaging 2012, Volume 8316 Contact: [email protected]