Download lim f(n)/g(n) = 0

lim f(n)/g(n) = 0
< inf
0< lim
< inf
0< lim
= inf
-->
-->
-->
-->
-->
f(n) o(g(n)) f<g
O
<=
Θ
=
Ω
>=
ω
>
For a >= 0, b > 0, lim ( lga n / nb ) = 0, so lga n =
o(nb), and nb = ω (lga n )
lg(n!) = Θ(n lg n)
A
5n2 + 100n
log3(n2)
nlg4
lg2n
B
3n2 + 2
log2(n3)
3lg n
n1/2
A?B
Θ
Θ
Ω
O
Sum1 to n i = n(n+1)/2
Sumi a to b 1 = b-a+1
Sum1 to n i 2 = (2n3+3n2+n)/6
Sum1 to n i 3 = (n2(n+1)2)/4
Sum1 to n ici = [(n-1)c(n+1) - ncn + c] / (c-1)2
Sum1 to n xi = (xn+1-1)/(x-1)
As n -->
Sum1 to n
Sum1 to n
Sum1 to n
inf, and if abs(x) < 1,
xi = 1/(1-x)
1/k (k is int >0) = ln(n) + O(1)
ixi = x/(1-x)2
Merge: divide until 1 or no elements, merge in order
Insertion: take next input, insert in output at
appropriate place (possibly forcing the shift of many
values in array)
Shell: insertion sort on partitions of input
Heap: build a heap in Θ(n), getMax in Θ(lg(n)),
parent(i) = floor(i/2), left(i) = 2i Heapify costs O(h)
where h = lg(n)
*comparison-based sort Ω(nlg(n))*
Counting:
1. for i  1 to k
2.
do C[i]  0
3. for j  1 to length[A]
4.
do C[A[j]]  C[A[j]] + 1
5. for i  2 to k
6.
do C[i]  C[i] + C[i-1]
7. for j  length[A] downto 1
8.
do B[C[A[ j ]]]  A[j]
9.
C[A[j]]  C[A[j]] - 1
Sort
Time
in place
merge
Θ(nlg(n))
0
insertion
O(n2)
1
Shell
Θ(n3/2)
0
Heap
Θ(nlg(n))
1
Quicksort
Θ(n2)*
1
Counting
O(n)
0
Radix
O(n)
0**
*best and average are Θ(nlg(n))
** if it uses counting sort
stable
1
1
0
0
0
1
1
T(n) = T(n/2) + 1 is O(lg n)
T(n) = 2T(n/2 + 17) + n is O(n lg n)
Consider T(n) = 2T(n1/2) + lg n
Rename m = lg n and we have
T(2m) = 2T(2m/2) + m
Set S(m) = T(2m) and we have
S(m) = 2S(m/2) + m  S(m) = O(m lg m)
Changing back from S(m) to T(n), we have
T(n) = T(2m) = S(m) = O(m lg m) =
O(lg n lg lg n)
T(n) = 3T(n/3) + n
T(n)  3c n/3 lg n/3 + n
 c n lg (n/3) + n
= c n lg n - c n lg3 + n
= c n lg n - n (c lg 3 - 1)
 c n lg n
T(n) = 3T(n/4) + n
T(n) = n + 3T(n/4)
= n + 3[ n/4 + 3T(n/16) ]
= n + 3[n/4] + 9T(n/16)
= n + 3[n/4] + 9 [n/16] + 27T(n/64)
T(n)  n + 3n/4 + 9n/16 + … + 3log4 n(1)
 n  (3/4)i + (nlog43)
= 4n+ o(n)
= O(n)
Master Method
the recurrence T(n) = a T(n/b) + f(n), T(n) can be
bounded asymptotically as follows:
1. If f(n)=O(nlogba-) for some constant  > 0, then
T(n)= (nlogba).
2. If f(n) = (nlogba), then T(n) = (nlogba lg n).
3. If f(n) =  ( nlogba+ ) for some constant  > 0, and if
a f(n/b)  c f(n) for some constant c < 1 and all
sufficiently large n, then T(n)=(f(n)).
Simplified Master Method
Let a  1 and b > 1 be constants and let T(n) be the
recurrence
T(n) = a T(n/b) + c nk
defined for n  0.
1. If a > bk, then T(n) = ( nlogba ).
2. If a = bk, then T(n) = ( nk lg n ).
3. If a < bk, then T(n) = ( nk ).
Probability
• Sample Space: a set whose elements are elementary
events.
For example: flipping 2 coins, S = {HH,HT, TH, TT}.
• Events: a subset A of the sample space S, i.e. A
subset S.
• Certain event: S, null event: ø.
• Two events A and B are mutually exclusive, if A
intersects B = ø.
• Axioms of Probability: A probability distribution Pr{}
on a sample space
S is a mapping from events of S to real numbers such
that the following are
satisfied:
1. Pr{A} >= 0 for any event A.
2. Pr{S} = 1.
3. Pr{A union B} = Pr{A} + Pr{B} for any two mutually
exclusive events
A and B.
Pr{A} is the probability of event A. Axiom 2 is a
normalization requirement.
Axiom 3 can be generalized to the following:
•
Pr{U Ai} = Sum Pr{Ai}
• Pr{ ø } = 0
• If A subset B, then Pr{A}<= Pr{B}.
• Pr{ !A} = Pr{S − A} = 1 − Pr{A}.
• For any two events A and B,
Pr{A intr B} = Pr{A} + Pr{B} − Pr{A intr B}
<= Pr{A} + Pr{B}
Hash Tables
Search an element takes (n) time in the worst case
and O(1) time on average (direct addressing takes O(1)
time in the worst case)
Conditional Probability of an event A given another
event B is:
Pr{A|B} = Pr{A intr B}/Pr{B}
, provided Pr{B} != 0
For example, A = {TT} and B = {HT, TH, TT}. Pr{A|B}
= (1/4)/(3/4) = 1/3
• Independent Events: Events A and B are independent
if
Pr{AintrB}=Pr{A}*Pr{B}, provided Pr{B} != 0
Equivalently, Pr{A|B} = Pr{A}
• Bayes’s Theorem:
Pr{A intr B} = Pr{B}*Pr{A|B} =Pr{A}*Pr{B|A}
Therefore,
Pr{A|B} =(Pr{A}*Pr{B|A})/Pr{B}
Pr{B} = Pr{B intr A} + Pr{B intr ~A}
= Pr{A}*Pr{B|A} + Pr{~ A}*Pr{B|~A}
Therefore,
Pr{A|B}=(Pr{A}*Pr{B|A})/(Pr{A}*Pr{B|A}+
Pr{~A}*Pr{B|~A})
For example, given a fair coin and a biased coin that
always come up heads.
We choose one of the two and flip the coin twice. The
chosen coin comes up
with heads both times. What is the probability that it is
biased?
Let A be the event that the biased coin is chosen and B
be the event that
the coins comes up heads both times.
Pr{A} = 1/2, Pr{~A} = 1/2,
Pr{B|A} = 1, Pr{B|~A} = (1/2)* (1/2) = 1/4
Pr{A|B} =((1/2)* 1)/((1/2)*1 + (1/2)*(1/4))
= 4/5
The longest simple path from a node in a R-B tree to a
descendent leaf has length at most twice that of the
shortest simple path from this node to a descendent
leaf.
More on back(?)
Given n independent trials, each trial has two possible
outcomes. Such trials are called “Bernoulli trials”. If p
is the probability of getting a head, then the probability
of getting k heads in n tosses is given by (CLRS
p.1113)
P(X=k) = (n!/(k!(n-k)!))pk (1-p)n-k = b(k;n,p)
This probability distribution is called the “binomial
distribution”. pk is the probability of tossing k heads
and (1-p)n-k is the probability of tossing n-k tails.
(n!/(k!(n-k)!)) is the total number of different ways
that the k heads could be distributed among n tosses.
Selection
1 Divide the n elements of input array into n/5 groups
of 5 elements each and at most one group made up
of the remaining (n mod 5) elements.
2 Find the median of each group by insertion sort &
take its middle element (smaller of 2 if even number
input).
3 Use Select recursively to find the median x of the
n/5 medians found in step 2.
4 Partition the input array around the median-ofmedians x using a modified Partition. Let k be the
number of elements on the low side and n-k on the
high side.
5 Use Select recursively to find the ith smallest element
on the low side if i  k , or the (i-k)th smallest
element on the high side if i > k
Load factor =n/m = average keys per slot,
for m slots to store n elements
Worst case: (n) + time to compute h(k)
Average case depends on how well h distributes the
keys among m slots.
Assume simple uniform hashing and O(1) time to
compute h(k), time required to search an element with
key k depends linearly on length of T[h(k)].
Consider expected number of elements examined by
search algorithm, i.e. the number of elements in
T[h(k)] that are checked to see if their keys = k.
Division Method
Map a key k into one of m slots by taking the remainder
of k divided by m. That is,
h(k) = k mod m
Don’t pick certain values, such as m = 2 p
Or hash won’t depend on all bits of k.
Primes, not too close to power of 2 (or 10) are good.
Red-Black Trees (everything’s O(lg n))
Properties: Every node is either red or black.
Every leaf (NIL) is black
If a node is red, then both its children are black
Every simple path from a node to a descendant leaf
contains the same number of black nodes
A red-black tree with n internal nodes has height at
most 2lg(n+1).
Inserting
Case 1 (the child x, its parent and its uncle y are red):
Change parent and uncle to black
Change grandparent to red (preserves black height)
Make grandparent new x, grandparent’s uncle new y
Case 2 (x and its parent are red, y is black, and x is an
“inside child”):
Rotate outwards on x’s parent
Former parent becomes new x
Old x becomes parent
Case 3 (x and its parent are red, y is black, and x is an
“outside child”):
Rotate “away from x” around x’s grandparent
Color x’s parent black
Now x’s old grandparent is x’s sibling; color it red to
maintain the right number of black nodes down each
path
Graphs
Combinatorial Facts:
Graph: 0  e  C(n,2) = n (n-1) / 2  O(n2)
 vV deg(v) = 2e
Digraph: 0  e  n2
 vV in-deg(v) =  vV out-deg(v) = e
graph is said to be sparse if e  O(n)
-Eulerian cycle is a cycle (not necessarily simple) that
visits every edge of a graph exactly once.
-Hamiltonian cycle (path) is a cycle (path in a directed
graph) that visits every vertex exactly once.
Acyclic: if a graph contains no simple cycles
Connected: if every vertex of a graph can reach every
other vertex
Connected: every pair of vertices is connected by a
path
Connected Components: equivalence classes of
vertices under “is reachable from” relation
Strongly connected: for any 2 vertices, they can reach
each other in a digraph
G = (V, E) & G’ = (V’, E’) are isomorphic, if  a bijection
f : V V’ s.t. v, uE iff (f(v), f(u)) E’.
Shortest Path – BFS: O(V+E), linear in size of Adj. list.
- DFS is (V+E)
Vertex v is a proper descendant of vertex u in the
depth-first forest for a (direct or undirected) graph G if
and only if d[u]<d[v]<f[v]<f[u] d=discover f=finish
Tree edges: edges in the depth-first forest G . Edge
(u, v) is a tree edge if v was first discovered by
exploring edge (u, v).
Back edges: edges (u, v) connecting a vertex u to an
ancestor v in a depth-first tree. Self-loops are
considered to be back edges.
Forward edges: nontree edges (u, v) connecting a
vertex u to a descendant v in a depth-first tree.
Cross edges: all other edges.
Topological Sort: Call DFS(G) to compute finishing time
f[v] for each vertex, As each vertex is finished, insert it
onto the front of linked list, return linked list. (V+E)
Strong Comp: Run DFS(G), computng finish time f[u]
for each vertex u, Compute R = Reverse(G), reversing
all edges of G, Sort the vertices of R (by CountingSort)
in decreasing order of f[u], Run DFS(R) using this
order, Each DFS tree is a strong component; output
vertices of each tree in the DFS forest Total running
time is (n+e)
Articulation Point (or cut vertex): any vertex whose
removal (together with any incident edges) results in a
disconnected graph
Bridge: an edge whose removal results in a
disconnected graph
Biconnected: containing no articulation points. (In
general a graph is k -connected, if k vertices must be
removed to disconnect the graph.)
Biconnected Components: a maximal set of edges s.t.
any 2 edges in the set lie on a common simple cycle
Greedy algorithms are typically used to solve
optimization problems & normally consist of
-Set of candidates
-Set of candidates that have already been used
-Function that checks whether a particular set of
candidates provides a solution to the problem
-Function that checks if a set of candidates is feasible
-Selection function indicating at any time which is the
most promising candidate not yet used
-Objective function giving the value of a solution; this is
the function we are trying to optimize
Step by Step Approach
-Initially, the set of chosen candidates is empty
-At each step, add to this set the best remaining
candidate; this is guided by selection function.
-If enlarged set is no longer feasible, then remove the
candidate just added; else it stays.
-Each time the set of chosen candidates is enlarged,
check whether the current set now constitutes a
solution to the problem.
When a greedy algorithm works correctly, the first
solution found in this way is always optimal.
Dijkstra’s shortest path algorithm:
Start with the value for the shortest path to all vertices
being infinty, except for s, the source which is set as
being 0 (form s). Add the nearest vertex (the next
unexplored vertex with the lowest weight) to the
explored set. Do this until all vertices are explored.
Minimum Spanning Tree
A free tree composed of the minimum weighted edges
that connect all vertices (|E| = |V|-1)
Steiner Minimum Tree is a MST that doesn’t include all
vertices.
 There exists a unique path between any two vertices
of a tree
 Adding any edge to a tree creates a unique cycle;
breaking any edge on this cycle restores a tree
 A cut (S, V-S) is just a partition of the vertices into 2
disjoint subsets. An edge (u, v) crosses the cut if one
endpoint is in S and the other is in V-S. Given a
subset of edges A, we say that a cut respects A if no
edge in A crosses the cut.

 An edge of E is a light edge crossing a cut, if among
all edges crossing the cut, it has the minimum weight
(the light edge may not be unique if there are
duplicate edge weights).
Kruskal’s MST algorithm: O(E lg E) So, start off with
each vertex having a set with just that vertex in it.
Sort the edges by increasing weight. For each edge
(starting with the lightest), if the two vertices are not
in the same set, union their sets.
Prim’s MST algorithm: