Download PowerPoint format

Tutorial: Statistics of Object Geometry
Stephen Pizer
Medical Image Display & Analysis Group
University of North Carolina, USA
with credit to
T. Fletcher, A. Thall, S. Joshi, P. Yushkevich, G. Gerig
10 October 2002
Uses of Statistical
Geometric Characterization





Medical science: determine geometric ways in which
pathological and normal classes differ
Diagnostic: determine if particular patient’s geometry is in
pathological or normal class
Educational: communicate anatomic variability in atlases
Priors for segmentation
Monte Carlo generation of images
Object Representation
Objectives
 Relation
to other instances of the shape class
 Representing
the real world
 Deformation while staying in shape class
 Discrimination by shape class
 Locality
 Relation
to Euclidean space/projective
Euclidean space
 Matching
image data
Geometric aspects
Invariants and correspondence

Desire: An image space geometric representation
that
 is
at multiple levels of scale (locality)
 at one level of scale is based on the object
 and at lower levels based on object’s figures
 at each level recognizes invariances associated with
shape
 provides positional and orientational and metric
correspondence across various instances of the shape
class
Object Representations

Atlas voxels with a displacement at each voxel : Dx(x)

Set of distinguished points {xi} with a displacement at each


Landmarks
Boundary points in a mesh

With normal b = (x,n)
u


Loci of medial atoms: m = (x,F,r,q)
end atom (x,F,r,q,h)
or

v
t
Continuous M-reps: B-splines in
(x,y,z,r) [Yushkevich]
Building an Object Representation
from Atoms a

Sampled
 aij
 can

have inter-atom mesh (active surface)
Parametrized
 a(u,v)



e.g., spherical harmonics, where coefficients become
representation
e.g., quadric or superquadric surfaces
some atom components are derivatives of others
Object representation:
Parametrized Boundaries
 Parametrized
 n(u,v)
is normalized x/u  x/v
 Coefficients
 x(u,v)
boundaries x(u,v)
= Si
 Spherical
 Sampled
of decompositions
ci
i
f (u,v)
harmonics: (u,v) = latitude, longitude
point positions are linear in
coefficients: Ax=c
Object representation:
Parametrized Medial Loci
 Parametrized
medial loci m(u,v) = [x,r](u,v)
is normalized x/u  x/v
 xr(u,v) = -cos(q)b
 n(u,v)
 gradient
per distance on x(u,v)
b
q
x
n
Sampled medial shape representation:
Discrete M-rep slabs (bars)
x
Meshes of medial atoms



Objects connected as host,
subfigures
Multiple such objects,
interrelated


x
r
p
q
- q
r
n
r
s
o
o
o o
o
o
o
o
o
o
t=-1
b
t=0
u
v
r
n r
s
t=+1
r
t
o
q -q
hrb
r


r
p
Interpolating Medial Atoms in a Figure

Interpolate x, r via B-splines
[Yushkevich]

Trimming curve via r<0 at outside control
points





Avoids corner problems of quadmesh
Yields continuous boundary
Via modified subdivision surface [Thall]


Medial sheet
Approximate orthogonality at sail ends
Interpolated atoms via boundary and distance
At ends elongation h needs also to be
interpolated
Need to use synthetic medial geometry
[Damon]
Implied boundary
End Atoms: (x,F,r,q,h)
h=1
Extremely rounded
end atom
in cross-section
h=1.4
Rounded
end atom
in cross-section
h=1/cos(q)
Corner atom
in cross-section
Medial atom with one more parameter: elongation h
Sampled medial shape representation:
M-rep tube figures




Same atoms as for slabs
r is radius of tube
sails are rotated about b
Chain rather than mesh
x+rRb,n(-q)b
x+
rRb,n(q)b
b
q
x
n
For correspondence: Object-intrinsic coordinates
Geometric coordinates from m-reps

Single figure





Medial sheet: (u,v)
[(u) in 2D]
t: medial side
: signed r-prop’l
dist from implied
boundary
3-space: (u,v,t, )
Implied boundary:
(u,v,t)
u

t

v
x

r
p

x
q
- q
r
r
p
r
n
r
s
q -q
t=-1
r
n r
s
t=+1
r
hrb
b
t=0
Sampled medial shape representation:
Linked m-rep slabs


Linked figures
Hinge atoms known in
figural coordinates
(u,v,t) of parent figure
Other atoms known in
the medial coordinates
of their neighbors
x+rRb,n(-q)b
b
q
x
n
o
o
o
o
o
o o
o
o
o
o

x+
rRb,n(q)b
Figural Coordinates for Object
Made From Multiple Attached Figures
 Blend in hinge regions
 w=(d1/r1 - d2 /r2 )/T
 Blended d/r when |w| <1 and u-u0 < T
 Implicit boundary: (u,w, t)
 Or blend by subdivision surface
w
Figural Coordinates for
Multiple Objects

Inside objects or on
boundary
 Per object
 In neighbor
object’s
coordinates

Interobject space
 In
neighbor object’s
coordinates
 Far outside boundary:
(u[,v],t, ) via distance
(scale) related figural
convexification ??
 ??
Heuristic Medial Correspondence
Original (Spline Parameter)
Arclength
1
0.8
0.6
0.4
0.2
0.2
Radius
0.4
0.6
0.8
Coordinate Mapping
1
Continuous Analytical Features



Can be sampled
arbitrarily.
Allow functional
shape analysis
Possible at many
scales: medial, bdry,
other object
Medial Curvature
Boundary texture scale
Feature-Based Correspondence on Medial
Locus by Statistical Registration of Features
curvature
dr/ds
Also works in 3D
dr/ds
What is Statistical
Geometric Characterization

Given a population of instances of an object class
 e.g.,

Given a geometric representation z of a given
instance
 e.g.,

subcortical regions of normal males of age 30
a set of positions on the boundary of the object
and thus the description zi of the ith instance
A statistical characterization of the class is the
probability density p(z)
 which is estimated from the instances zi

Benefits of Probabilistically Describing
Anatomic Region Geometry

Discrimination among geometric classes, Ck
 Compare

probabilities p(z | Ck)
Comprehension of asymmetries or distinctions of
classes
 Differences between means
 Difference between variabilities
Segmentation by deformable models
 Probability of geometry p(z) provides prior
 Provides object-intrinsic coord’s in which multiscale
image probabilities p(I|z) can be described
 Educational atlas with variability
 Monte Carlo generation of shapes, of diffeomorphisms, to produce pseudo-patient test images

Necessary Analysis Provisions To
Achieve Locality & Training Feasibility

Multiple scales
 Allows
few random
variables per scale
 At
each scale, a level of
locality (spatial extent)
associated with its random
variable

Positional correspondence
 Across instances
 Between scales
Large scale
Smaller scale
Discussion of Scale

Spatial aspects of a
geometric feature
 Its
 Its

position
spatial extent
Region summarized
 Level

of detail captured
Residues from larger scales
 Distances
to neighbors
with which it has a
statistical relationship


Markov random field
Cf PDM, spherical
harmonics, dense Euclidean
positions, landmarks, m-reps
Large scale
Smaller scale
Scale Situations in Various Statistical
Geometric Analysis Approaches
Multidetail feature,
Detail residues
each level of detail,
E.g., spher. harm.
E.g., boundary pt.
E.g., object hierarchy
Level of Detail
Global coef for
Fine
Coarse
Location
Location
Location
Principles of Object-Intrinsic
Coordinates at a Scale Level
 Coordinates
at one scale must relate to parent
coordinates at next larger scale
 Coordinates at one scale must be writable in
neighbor’s coordinate system
 Statistically stable features at all scales must be
relatable at various scale levels







Figurally Relevant Spatial Scale Levels:
Primitives and Neighbors
Multi-object complex
Individual object



= multiple figures
in geom. rel’n to neighbors
in relation to complex
Individual figure



= mesh of medial atoms
subfigs in relation to neighbors
in relation to object
Figural section

= multiple figural sections



each centered at medial atom
medial atoms in relation to neigbhors
in relation to figure
Figural section residue, more finely spaced, ..

=> multiple boundary sections (vertices)
Boundary section


vertices in relation to vertex neighbors
in relation to figural section
Boundary section more finely spaced, ...
Multiscale Probability Leads to
Trainable Probabilities

If the total geometric representation z is at all scales
or smallest scale, it is not stably trainable with
attainable numbers of training cases, so multiscale
 Let zk be
the geometric representation at scale level k
 Let zki be the ith geometric primitive at scale level k
 Let N(zki) be the neighbors of zki (at level k)
 Let P(zki) be the parent of zki (at level k-1)

Probability via Markov random fields
 p(zki
| P(zki), N(zki) )
 Many trainable probabilities
If p(zki rel. to P(zki), zki rel. to N(zki) )
 Requires parametrized probabilities

Multi-Scale-Level Image Analysis
Geometry + Probability

Multiscale critical for effectiveness with efficiency
 O(number
of smallest scale primitives)
 Markov random field probabilistic basis
 Vs. methods working at small scale only or at
global scale + small scale only
Multi-Scale-Level Image Analysis
via M-reps

Thesis: multi-scale-level image analysis is
particularly well supported by representation built
around m-reps
 Intuitive,
medically relevant scale levels
 Object-based positional and orientational correspondence
 Geometrically well suited to deformation
Geometric Typicality
Metrics
Statistical Metrics
Statistics/Probability Aspects :
Principal component analysis
 Any
shape, x, can be written as
x = xmean + Pb + r
 log p(x) = f(b1, … bt,|r|2)
x2
p1
b1
xmean
xi
x1
Visualizing & Measuring
Global Deformation
 Shape
Measurement
 Modes of shape variation across patients
 Measurement = z amount of each mode
c = cmean + z1s1p1
c = cmean + z2s2p2
Statistics/Probability Aspects :
Markov random fields
(z1 … zn)
 p(zi | {zj, ji}) =
p(zi | {zk : k a neighbour of i})
 Suppose zT=
(i. e., assume sparse covariance matrix)
 Need
only evaluate O(n) terms to
optimize p(z) or p(z | image)
 Can only evaluate p(zi), i.e., locally
 Interscale; within scale by locality
Multiscale Geometry and Probability

If z is at all scales or smallest scale, it is not stably
trainable, so multiscale
 Let zk be
the geometric rep’n at scale k
 Let zki be the ith geometric primitive at scale k
 Let N(zki) be the neighbors of zki
 Let P(zki) be the parent of zki
 Let C(zki) be the children of zki

Probability via Markov random fields
 p(zki
| P(zki), N(zki), C(zki) )
 Many trainable probabilities
 Requires parametrized probabilities for training
Examples with m-reps components
p(zki | P(zki), N(zki), C(zki) )
 z1 (necessarily
global): similarity transform for body section
 z2i: similarity transform for the ith object

Neighbors are adjacent (perhaps abutting) objects
 z3i:
“similarity” transform for the ith figure of its object in its
parent’s figural coordinates

Neighbors are adjacent (perhaps abutting) figures
 z4i:

medial atom transform for the ith medial atom
Neighbors are adjacent medial atoms
 z5i:
medial atom transform for the ith medial atom residue at
finer scale (see next slide)
 z6i: boundary offset along medially implied normal for the ith
boundary vertex

Neighbors are adjacent vertices
Multiscale Geometry and Probability
for a Figure

Geometrically  smaller scale
 Interpolate
(1st order) finer spacing of
atoms
 Residual atom change, i.e., local

coarse, global
coarse resampled
Probability
 At
any scale, relates figurally
homologous points
 Markov random field relating medial
atom with
its immediate neighbors at that scale
 its parent atom at the next larger scale and
the corresponding position
 its children atoms

fine, local
Published Methods of Global Statistical
Geometric Characterization in Medicine

Global variability
 via
principal component analysis on
features globally, e.g., boundary points or
landmarks, or
 global features, e.g., spherical harmonic
coefficients for boundary


Global difference
 via


globally or on global features
Globally based diagnosis
 via


linear (or other) discriminant on features
linear (or other) discriminant on features
globally or on global features
Example authors: [Bookstein][Golland] [Gerig]
[Joshi] [Thompson & Toga][Taylor]
Published Methods of Local Statistical
Geometric Characterization
A

Local variability


Local difference



via principal component analysis on
features globally or on global features,
plus display of local properties of
principal component
via linear (or other) discriminant on
global geometric primitives, plus
display of local properties of
discriminant direction
On displacement vectors:
signed, unsigned re inside/outside
Outward, p < 0.05
p > 0.05
Inward, p < 0.05
R
Example authors: [Gerig] [Golland]
[Joshi] [Taylor] [Thompson & Toga]
L
Displacement significance:
Schizophrenic vs. control
hippocampus
Shortcomings of Published Methods of
Statistical Geometric Characterization

Unintuitive
 Would
like terms like bent, twisted, pimpled, constricted,
elongated, extra figure

Frequently nonlocal or local wrt global template
 Depends
on getting correspondence to template correct
 Need where the differences are in object coordinates


Which object, which figure, where in figure, where on boundary
surface
Requires infeasible number of training cases
 Due
to too many random variables (features)
Overcoming Shortcomings of Methods of
Statistical Geometric Characterization

Intuitive


Figural (medial) representation provides terms like bent,
twisted, pimpled, constricted, elongated, extra figure
Local


Hierarchy by scale level provides appropriate level of
locality
Object & figure based hierarchy yields intuitive locality
and good positional correspondences



Which object, which figure, where in figure, where on
boundary surface
Positional correspondences across training cases & scale levels
Trainable by feasible number of cases

Few features in residue between scale levels


Relative to description at next larger scale level
Relative to neighbors at same scale level
Conclusions re Object Based
Image Analysis
 Work
at multiple levels of scale
 At
each scale use representation appropriate for that
scale
 At
intermediate scales
 Represent
medially
 Sense at (implied) boundary
Papers at midag.cs.unc.edu/pubs/papers
Extensions
 Variable
topology
 jump
diffusion (local shape)
 level set?
 Active
 shape
Appearance Models
and intensity
 ‘explaining’ the image
 iterative matching algorithm
Recommended Readings




For deformable sampled boundary models: T Cootes, A Hill, CJ
Taylor (1994). Use of active shape models for locating structures
in medical images. Image & Vision Computing 12: 355-366.
For deformable parametrized boundary models: Kelemen, Gerig,
et al
For m-rep based shape: Pizer, Fritsch, et al, IEEE TMI, Oct.
1999
For 3D deformable m-reps: Joshi, Pizer, et al, IPMI 2001
(Springer LNCS 2082); Pizer, Joshi, et al, MICCAI 2001
(Springer LNCS 2208)
Recommended Readings

For Procrustes, landmark based deformation (Bookstein), shape
space (Kendall): especially understandable in Dryden & Mardia,
Statistical Shape Analysis

For iterative conditional posterior, pixel primitive based shape:
Grenander & Miller; Blake; Christensen et al