Download Document

Planning in surgery and surgical
simulation
Comp 790-058 course presentation
Mert Sedef
Planning in robotic radiosurgery
R. Tombropoulos, J.R. Adler, and J.C. Latombe. CARABEAMER: A
Treatment Planner for a Robotic Radiosurgical System with
General Kinematics. Accepted for publication in Medical Image
Analysis, Oxford University Press, 1998.
Rhea Tombropoulos et al., Treatment Planning for Image-Guided
Robotic Radiosurgery, Computer Vision, Virtual Reality and
Robotics in Medicine, 1997
R.Z. Tombropoulos, J.C. Latombe, and J.R. Adler. A General
Algorithm for Beam Selection in Radiosurgery. In Preprints of
the IARP Workshop on Medical Robotics, 91--98, Vienna, Austria,
1996.
A. Schweikard, J.R. Adler, and J.C. Latombe. Motion Planning in
Stereotaxic Radiosurgery. IEEE Tr. on Robotics and Automation,
9(6):764--774, 1993
Radiosurgery
Non-invasive procedure
Moving beam of radiation to
ablate (destroy) brain tumors
The problem is delivering


Enough dose of radiation to
the tumor to destroy the
tumor
Minimum dose of radiation to
the healthy and dose-sensitive
tissue (e.g., brain stem and
optic nerves) not to destroy
them
The solution is

Crossfiring at the tumor:
several weaker beams from
different directions
Tombropoulos, Adler, and Latombe, 1998
Treatment planning in radiosurgery
Determination of a series of beam
configuration (position and orientation)
Constraints:



The beams should intersect to form a region
of high-dose on the tumor
The dose distribution should match the shape
of the tumor
Healthy or critical tissues should get minimum
or no radiation
A treatment planning system
6-dof robotic manipulator arm

Positions the radiation source
Real-time imaging system

Monitors patient’s motion
continuously
A treatment planning algorithm


Allows the surgeon to specify
particular region of interest
(e.g., tumors, dose-sensitive
tissue) and range of dose
Uses linear programming to
optimize the plans and satisfy
constraints
Tombropoulos, Adler, and Latombe, 1998
Steps of the treatment planning
system - 1
The surgeon specifies
regions of interest on the
CTs (e.g., the tumor and
critical structures)

the system makes a 3D
reconstruction of the
geometry
and imposes constraints
on the amount of
radiation that these
regions should receive.

Eg., Tumor should get
2000 rads min and brain
stem should get 500 rads
max
Tombropoulos, Adler, and Latombe, 1998
Steps of the treatment planning
system - 2
Beam selection



Target point selection: Evenly space targets on
the surface of the 3D tumor model coming from the
CT
Source point selection: Select source points
making use of pre-recorded robot configurations.
Record the target point and robot configuration.
Path generation: Connect all beam configurations
into a path such that the robot traverses in a
collision-free path in the environment.
Steps of the treatment planning
system - 3
Plan refinement


Problem! Beam selection does not consider
the location of critical tissues and does not
guarantee a highly homogeneous dose
distribution on the tumor
Given these constraints, Linear
Programming adjusts and finds the optimal
values of the dose and diameter of individual
beams.
Steps of the treatment planning
system - 4
Plan Evaluation

The surgeon is provided with the results of
planning
3D iso-dose surfaces, dose-volume histograms,
etc.

If the surgeon is not satisfied, planning is
restarted from the desired step
Motion planning in maxillofacial
robotic surgery
Burghart et al., On-line motion planning
for medical applications, Proceedings of
the 24th Annual Conference of the IEEE,
1998
Maxillofacial robotic surgery
Maxillofacial surgery:
Surgery in the maxilla
and face area
Motion of the surgical
robot should be planned
for

Bone cutting
Planned motion should


be safe and adequate
Have online capabilities
to react dynamical
changes (i.e. movements
of the patient and surgical
instruments)
Planner – Overall concept
A volume and surface model of the
patient data is constructed beforehand
Surgery setup:

6-dof surgical robot for
Bone cutting, hole creating in patient’s
skull

Infrared navigation system for
Detecting and monitoring the positions
of

Patient’s skull, robot’s tools, surgeons
instruments
Environment modeling

3D modeling of the whole environment
including patient data and surgical tools
and screws attached to the skull
Online collision-free motion planning
for the 6-dof robot

The planner reacts according to the
current state of the environment
Burghart et al., 1999
Planner – Environment modeling
Convex hull of the surgical wound, patient, and the
hooks are generated at different levels dynamically
Burghart et al., 1999
Planner – Robot motion planning
The planner searches a
solution in the implicit robot
joint-value space (c-space)
and checks for collisions in the
workspace (environment
space)
C-space: A* search algorithm


A*: a graph-tree search
algorithm. Example of bestsearch algorithm
Eg. Depth-first, breath-first,
djkstra
Collision-detection by distance
computation in the workspace
Burghart et al., 1999
Motion planning for performance
assessment in Minimally Invasive
Surgery
Haniffa et al., Motion Planning System
for Minimally Invasive Surgery, 14th
Annual IEEE International Conference and
Workshops on the Engineering of
Computer-Based Systems (ECBS'07), pp.
609-610, 2007
Minimally invasive surgery –
laparoscopic surgery
monitor
laparoscopic
instruments
surgeon
Basdogan, Ho, and Sirinivasan, 2001
A box-training system mimicking
MIS settings
An enclosure (box) with openings for surgical
instruments

Surgical tasks are performed within the box
Surgical instruments are mounted with motion
sensors

The maneuvers of the trainee are recorded during the
performance
Given the task, the optimal traverse of the
surgical tools calculated

The optimal traverse is compared with the maneuvers
of the trainee for performance assessment
Motion planning for replacement of
a rubber band across two hooks
Potential field method


Instrument tips = point
robots in c-space
Artificial potential field:
attractive towards the
goal state and
repulsive towards
forbidden regions
Haniffa et al., 2007
Performance Assessment
Steepest descent path


Movement towards
steepest descent earns
credit
Movement towards
obstacles imposes
penalties
Also



Completion time
# of collisions with
obstacles and
instruments
# of c-space violations
Haniffa et al., 2007
Planning for needle insertion
Alterovitz, R et al., “Sensorless planning for medical needle
insertion procedures,” IEEE/RSJ International Conference on
Intelligent Robots and Systems, volume 4, pp. 3337 – 3343, 2003.
R. Alterovitz, K. Goldberg, and A. Okamura, "Planning for
steerable bevel-tip needle insertion through 2D soft tissue with
obstacles," in Proc. IEEE Int. Conf. on Robotics and Automation,
Apr. 2005, pp. 1652--1657
Alterovitz, R et al., “Steering flexible needles under Markov
motion uncertainty,” IEEE/RSJ International Conference on
Intelligent Robots and Systems, pp. 1570- 1575, 2005.
R. Alterovitz, M. Branicky, and K. Goldberg, "Constant-Curvature
Motion Planning Under Uncertainty with Applications in ImageGuided Medical Needle Steering," in Proc. Workshop on the
Algorithmic Foundations of Robotics, July 2006.
Needle insertion
Medical applications:



Brachytheraphy (seed implantation):
radiotherapy in which the source of radiation
is placed (as by implantation) in or close to
the area being treated
Biopsies: the removal and examination of
tissue, cells, or fluids from the living body
Treatment injections: inserting a needle to a
specific target location inside the body to
inject a drug
Needle insertion
Aim

Ultrasound images of human prostate
needle tip should be as
close as possible to an
internal target when the
procedure is performed
Challenge



needle insertion causes
soft tissues to displace
and deform
difficult or impossible to
obtain precise imaging data
during insertion
Incorrect placement of a
radioactive seed cannot
treat tumor and can
damage healthy tissue
Alterovitz et al., 2003
Sensorless planning algorithm for
radioactive seed implantation – rigid needle
The system computes needle
insertion offsets that
compensate for tissue
deformations. It



uses 2D FEM model
(simulation) of the soft tissues
surrounding the target implant
location (hence sensorless!)
performs dynamic simulation
of needle insertion to compute
tissue deformations
iteratively tests different
insertion locations and depths
to compute the optimal needle
offset
Alterovitz et al., 2003
Problem definition
Deformable body in 2D plane,
attached target at Pt
Insert needle from a specified
height until a specified depth,
Pr
Release seed at that depth =
Pr
Retract needle
Final actual seed position = Pa
Note that Pa≠Pr due to tissue
defrmation
Error = ||Pa-Pt||
Alterovitz et al., 2003
Planning
For a target, define a set of
insertion heights and insertion
depths. A virtual 2D grid
consisting of different (height,
depth) couples
Inset needle with constant velocity
from a chosen height up to a
chosen depth
Deform 2D model with FEM
Release seed, retract needle, wait
for the model to stop its dynamic
deformations
Calculate error between the final
position of the seed and target for
that (height, depth) couple
(height, depth) couple with
minimum error is the solution
Alterovitz et al., 2003
Steering Flexible Needles Under
Markov Motion Uncertainty
Unlike rigid needles, flexible bevel tip
needles can be steered around
obstacles.
Flexible needles with bevel tips follow a
path of constant curvature in the
direction of the bevel.
Controlling 2 DOF at the needle base
(rotation or bevel direction and insertion
distance), the needle can be steered
around obstacles to reach targets
inaccessible to rigid needles.
Planning motion for such a needle is
difficult due to uncertainty constraints,
i.e. uncertainty in



tissue properties
needle mechanics
interaction forces.
Alterovitz, Goldberg, Okamura 2005
Alterovitz et al., 2005
Motion planning for flexible bevel
tip needles in uncertainty
Motion planning problem as a Markov Decision
Process (MDP) based on Dynamic Programming
The planner


Computes discrete control sequence of insertion & direction
changes in needle
Minimizes expected cost (tissue deformation & damage) due to
insertion distance
direction changes
obstacle collisions
Considers


Deterministic case (needle response to controls known)
Uncertain case (probability distribution of needle response
known)
Problem definition
Needle




Flexible & bevel tip
Stiff soft tissue relative to needle
Rotation, i.e. bevel direction (right or left, 180 degree turn)
Insertion only (no retraction)
2D rectangular workspace

Alterovitz et al., 2005
Specified by segmenting 2D cross-section of patient anatomy via MRI or
ultrasound
Two actions for needle:


Insert a distance (constant velocity)
Change direction (180 degree turn, no insertion) and insert a distance (constant
velocity)
Needle movement damages tissue


Cost for both insertion and rotation (occurs as long as needle moves)
Prohibitive cost for obstacle colliision
Planning goal: Find a set of discrete needle controls to reach a target from a
starting point with minimum cost
Problem formulation
Dynamic programming requires discrete state
Discrete state space:


2D workspace discretized as a grid
Needle movement is discretized based on
Needle tip position
Rotation
Control circle variables

These two states are merged
State transitions:


Deterministic: Next state calculated using current
state values. P = 1
Uncertain: Uncertainty due to tissue inhomogeneity
80% deterministic
10% needle tip deviates from input orientation by some
positive amount
10% needle tip deviates from input orientation by some
negative amount
Cost function based on



Amount of path traversed from current to next state
If next state is the target – C = 0
If next state collides with any obstacle – C = high,
terminate
Total cost: Expected value of sum of state
transition costs
Alterovitz et al., 2005
Motion planning optimization
Compute a sequence of controls that minimizes total expected cost of
needle insertion
Stochastic shortest path problem

Solved using infinite horizon dynamic programming
Path planning of catheters in Liver
Chemoembolization
Gayle et al., Path Planning for
Deformable Robots in Complex
Environments, Proceedings of Robotics:
Systems and Science, 2005
Liver Chemoembolization
Under x-ray guidance, catheter
(tube-like cylinder) is inserted
into femoral artery and
advanced through a set of
arteries to reach near the
tumor
When reached to the artery
supplying a tumor, it injects
chemotherapy drugs
Careful catheter manipulation
is critical:


Spasms in small vessels
Reflux of chemoembolization
into other arteries due to size
similarity
Gayle et al., 2005
A constrained-based planning
algorithm
Constrained dynamic simulation

Motion planning = solving a list of constraints
Geometric constraints

Obstacle avoidance, non-penetration
Physical constraints

Volume preservation, energy minimization
A constrained-based planning
algorithm
Catheter is designed as a deformable robot

Deformation model = mass-spring system
Planning problem: finding sequential robot configurations such that


No configuration intersects any obstacle
All configurations satisfy constraints and minimize the energy of the
system
Constraints

Hard:
non-penetration (collision detection and response)

Soft:
Goal-seeking (initial path = medial axis)
Volume preservation
Obstacle avoidance

Energy minimization
A constrained-based planning
algorithm
Update robot state given the constraints, Fc constraint
force, Fe external force
Check if constraints are satisfied subject to energy
minimization
If not:





Set the last valid milestone as the next destination
Back trace one step on the current roadmap
Find a new path from the last valid milestone to the goal
configuration
Compute new constraint forces and solve the ODE, using the
previous state of the robot R and Fe
Set the next robot state to be the new ODE solution
A constrained-based planning
algorithm - demo
Additional references
Basdogan, C., Ho, C., Srinivasan, M.A.,
2001, "Virtual Environments for Medical
Training: Graphical and Haptic Simulation
of Common Bile Duct Exploration",
accepted to the IEEE/ASME Transactions
on Mechatronics