Download Lecture Note 08 EECS 4101/5101 Instructor: Andy Mirzaian All

Lecture Note 08
EECS 4101/5101
Instructor: Andy Mirzaian
All Nearest Neighbors: The Lifting Method
Introduction
Suppose we are given a set P = { p1 , p2 , ... , p n } of n points in the plane. The
given coordinates of the i-th point in P are pi = (x i , y i ), i = 1.. n. A nearest neighbor of
pi ∈ P is a point p j ∈ P − { pi } such that the Euclidean distance dist( pi , p j ) is minimum possible. We denote this nearest neighbor as p j = NN ( pi ). In case pi has more
than one nearest neighbor (with the same minimum distance from pi ), we pick one of
them arbitrarily.
The All Nearest Neighbors Problem (ANNP):
Input: A set P = { p1 , p2 , ... , p n } of n points in the plane.
Output: The nearest neighbor NN ( pi ), for each point pi ∈ P, i = 1.. n.
Figure 1(a) shows a set of points, for each point an arrow pointing to its nearest neighbor.
Figure 1(b) shows the property of nearest neighbor, namely, that p j = NN ( pi ) if none of
the points in P − { pi } are in the interior of the circle with center pi and radius
dist( pi , p j ).
p
p
j
p
i
(a) The Nearest Neighbor Graph
i
(b) p = NN ( p )
j
j
p
(c) Empty circle
i
Figure 1.
A related problem is the Closest Pair Problem (CPP), which asks to compute a distinct
pair of points in P so that the Euclidean distance between them is minimum among all
distinct pairs of points in P. Chapter 33 of [CLRS] describes an O(n lg n) time algorithm to compute the closest pair. This divide-and-conquer algorithm is due to Shamos
and Hoey [ShH75]. It should be obvious that a pair ( pi , p j ) in P is the closest pair only
if pi and p j are mutually nearest neighbors of each other. Note that the latter condition is
necessary, but not sufficient. However, this shows that if we already have the solution to
ANNP, we can solve CPP in O(n) additional time. (Just find the pair ( pi , NN ( pi )), for
i = 1. . n that minimizes dist( pi , NN ( pi )).)
In this lecture note we want to describe a method to solve ANNP in O(n lg n) time. The
main ingredients of the solution are:
(i) The Sparsification Method: We can view ANNP as a graph problem: consider the
complete graph with n vertices in one-to-one correspondance with the n points in P,
and an edge between each pair ( pi , p j ) of vertices with weight dist( pi , p j ). The
-2problem is now to compute, for each vertex pi in this graph, its nearest adjacent vertex p j . But since the graph is complete, it has Θ(n2) edges. The sparsification
method will eliminate many irrelevant edges to obtain a sparse subgraph that still
contains the edges we are seeking. The sparse graph in our solution turns out to be a
planar graph (known as the Delaunay triangulation), with n vertices and O(n)
edges. Once we have such a sparse graph, we can inspect all its edges in O(n) time
to complete the solution to ANNP.
(ii) The Linearization Method: This method, tries to turn non-linear equations involved
in the problem formulation to linear equations as much as possible.
(iii) The Lifting Method. Due to the linearization method, this method lifts the 2D points
of P into 3D.
(iv) The use of an O(n lg n) time algorithm to compute 3D convex hull (e.g., the divideand-conquer 3D convex hull algorithm).
Figure 1(b) shows the condition for p j = NN ( pi ). Figure 1(c) also shows the circle with
diameter ( pi , p j ) which is inside the circle as shown in Figure 1(b). The property of the
smaller circle is that none of the points of P are in its interior. Any such circle is called an
empty circle. Thus, we arrive at our first necessary condition for the nearest neighbor
property:
Fact 1: If p j is nearest neighbor of pi , then the circle with diameter ( pi , p j ) is an
empty circle.
As we shall see later, there are a total of O(n) pairs ( pi , p j ) that are diameter of an empty
circle. This would accomplish ingredient (i), sparsification.
Now, let us concentrate on the empty circle property. The equation of a circle C with
center coordinates (a , b) and radius r is (x − a)2 + (y − b)2 = r 2 . This is the quadratic
equation:
x 2 + y2 = 2ax + 2by + (r 2 − a2 − b2) .
(1)
Using ingredient (ii), the linearization method, we name the quadratic quantity x 2 + y2 on
the left hand side of eq. (1) as a new variable z. Now, eq. (1) can be expressed as a pair of
simultanious equations in 3D (with coordinate system (x , y , z)) as:
 z = 2ax + 2by + (r 2 − a2 − b2)
(2)

2
2
z
=
x
+
y

From eq. (2) we see that the circle parameters (a , b , r) all show up only in the linear
equation z = 2ax + 2by + (r 2 − a2 − b2), which is now the equation of a plane in 3D.
Let’s denote this plane Π(C). The second, and quadratic equation z = x 2 + y2 is fixed,
independent of the circle parameters. The latter is the equation of the so called
paraboloid of revolution, denoted Λ. See Figure 2.
Thus, we can interprete the circle equation (1) as follows: in 3D, take the intersection
λ (C) = Π(C) ∩ Λ of the plane Π(C) with the paraboloid of revolution Λ and project it
back down to the ground plane, i.e., the xy-plane, to get circle C.
Remark 1: Assuming coordinate z is the vertical direction in 3D, the intersection of any
non-vertical plane with Λ is an ellipse in 3D, but its projection on the xy-plane is always
-3-
z
2
2
Λ :z=x +y
λ(C )
Π (C )
y
x
C
Figure 2.
a circle. The intersection of a vertical plane with Λ in 3D is a parabola whose projection
on the xy-plane is a line.)
From the above discussion and the convexity of Λ, we obtain our next observation:
Fact 2: Let Π be any non-vertical plane in the xyz-space. The projection of the intersection of Π and the paraboloid of revolution Λ down on the xy-plane is some circle C.
Conversely, any circle C in the xy-plane is the projection of the intersection
λ (C) = Π(C) ∩ Λ of the non-vertical plane Π(C) with Λ in the xyz-space. Furthermore,
any point on Λ below the plane Π(C) projects down to a point in the interior of circle C,
and any point on Λ above the plane Π(C) projects to a point in the exterior of circle C.
Consider a point p = (x , y) on the ground plane, and vertically "lift" it to the point
λ ( p) = (x , y , x 2 + y2) on the paraboloid of revolution in the xyz-space. Similarly, let
λ (P) = { λ ( pi ) pi ∈ P } denote the n points in P lifted vertically on to Λ. Now, the
next observation:
Fact 3: Let C be a circle on the xy-plane, let Π(C) denote its corresponding plane in
xyz-space. Then, C is an empty circle (i.e., none of the points of P are in its interior) if
and only if none of the 3D points λ (P) are below the plane Π(C).
Remark 2: Thus, by this lifting method, we have converted the 2D non-linear
"inside/outside circle test" to the 3D linear "below/above plane test".
Now consider a pair of points ( pi , p j ) in P. According to Fact 3, the circle C with diameter ( pi , p j ) is an empty circle if and only if all lifted points λ (P) are on or above the
plane Π(C). The latter plane passes through λ ( pi ) and λ ( p j ). This, and Fact 1 imply that
if p j is nearest neighbor of pi , then in the xyz-space there is a plane (namely, Π(C)) that
passes through λ ( pi ) and λ ( p j ), and all n points λ (P) are on or above it. But such a plane
is a supporting plane of λ (P), one that touches the 3D convex hull of λ (P) along the edge
(λ ( pi ) , λ ( p j )) and the remaining points of λ (P) are above that supporting plane. Furthermore, since this supporting plane touches convex hull of λ (P) from below, then the line
segment (λ ( pi ) , λ ( p j )) is an edge of one of the "lower" faces of the convex hull of λ (P)
-4(i.e., a face whose normal outside the convex hull is pointing downward). Hence, we
arrive at our next observation:
Fact 4: If p j is a nearest neighbor of pi , then (λ ( pi ) , λ ( p j )) is one of the edges of the
"lower" convex hull of λ (P).
Let G = (P , T ) be the straight-line graph on the xy-plane with n vertices P, and the edge
set T , where each edge of G is the projection of a lower-convex-hull edge of λ (P).
Remark 3: Since the paraboloid Λ is convex, all points λ (P) are convexly positioned
and they will all be vertices of the lower-convex hull of λ (P). Thus the vertex set of G is
indeed the entire set P of the input points.
From Fact 4, we see that all the nearest neighbor edges appear in this graph G. Furthermore, G is sparse, since it is planar and hence it has at most 3n edges. If on the contrary,
G were nonplanar, i.e., had a pair of crossing edges, that pair of edges would correspond
(in the lifted version) to a pair of edges of the lower hull of λ (P) such that one is vertically "above" the other. This is impossible for a convex hull.
Remark 4: In fact, this straight line planar graph G = (P , T ) is the well known Delaunay Triangulation of P. A triangulation of P is a maximal set of non-crossing edges T
that partition the interior of the convex hull of P into triangles. A point set P has many
triangulations. The Delaunay triangulation is the unique triangulation with the property
that the circumcircle of each of its triangles is an empty circle. In other words, each triangle in T is the projection of a (triangular) face of the lower convex hull of λ (P). See Figure 3.
z
y
x
G
Figure 3.
From the above development we conclude the following algorithm:
-5Algorithm All-Nearest-Neighbors (P)
Step 1. Construct the 3D point set λ (P)
Step 2. Compute the 3D convex hull CH(λ (P))
Step 3. T′ ← edges of the lower hull of CH(λ (P)) computed in step 2.
Step 4. T ← projection of T′ down on the xy-plane
Step 5. Search the edges T to find all NN’s. That is, for each pi , i = 1.. n, look at all its
incident edges ( pi , p j ) ∈ T and pick the shortest one.
end
Complexity Analysis:
Step 2 takes O(n lg n) time, by using any worst-case efficient 3D convex hull algorithm
(e.g., the divide-and-conquer algorithm). The other steps take O(n) time. Space complexity is O(n).
Bibliography
The topic of this note is covered in many standard books on computational geometry such
as [BKOS97], [ORo94] or [Ede87] just to name a few. Delaunay Triangulation, and its
dual, the Voronoi Diagram are well known. There are literally hundreds of publications
on these geometric structures. You may also find more details about them in the books
mentioned above.
References:
[BKOS97] M. de Berg, M. van Kreveld, M. Overmars, O. Schwarzkopf, "Computational
Geometry: Algorithms and Applications," Springer, Second Edition, 2000.
[Ede87]
H. Edelsbrunner, "Algorithms in Combinatorial Geometry," Springer-Verlag,
1987.
[ORo94] J. O’Rourke, "Computational Geometry in C," Cambridge Univ. Press, 2nd
edition, 1998.
[ShH75]
M.I. Shamos and D. Hoey, "Closest-point problems," In Proceedings of the
16th Annual IEEE Symposium on Foundations of Computer Science
(FOCS), 1975, pp. 151-162.