Download Asymptotic Notation - Department of Computer Science

Asymptotic Notation
Asymptotic Notation
Joseph Spring
School of Computer Science
BSc - Quantum Computing / QIP
QC - Asymptotic Notation
1
Keeping Score on Algorithm Performance
QC - Asymptotic Notation
2
Random Access Machine
Algorithms can be studied in a language and
machine independent way
Hypothetical computer with following
assumptions:
– Each simple operation takes 1 time step
– using techniques/tools to compare without
implementation
• (+, *, -, =, if, call)
– Loops/subroutines are not considered simple
Techniques/Tools
• These are composition of many single-step operations
– RAM Random Access Machine
– Each memory access takes exactly one time step
• We assume we have as much memory as required
– Asymptotic Analysis of worst case complexity
• We ignore whether an item is in cache or on the disk
QC - Asymptotic Notation
3
Random Access Machine
4
Random Access Machine
Problem - Too Coarse a Model?
With the RAM Model
– We measure the run time of an algorithm
by counting the number of steps taken on a
given problem instance
– We assume the RAM executes a given
number of steps per second
• The assumptions make the analysis and conclusions too
coarse to believe in practice?
– Example: Multiplying two numbers takes longer than adding
two numbers on most processors, violating the first
assumption
– Memory access times differ greatly depending upon whether
data sits in Cache or on the disc, violating assumption 3
• Despite these and other criticisms it is felt to be an
excellent model for understanding how an algorithm
will perform on a real computer
• Difficult to find an algorithm such that the RAM model
will lead to substantially misleading results
– The count operation converts easily to the
actual run time
QC - Asymptotic Notation
QC - Asymptotic Notation
5
QC - Asymptotic Notation
6
1
Best, Worst and Average Case Complexity
Best, Worst and Average Case Complexity
• To understand how good or bad an algorithm
is we must look at how it performs over all
instances of input data
Example: Problem of Sorting
– The set of possible instances of input data =
the number of possible arrangements of the keys
involved = k! = k(k – 1)(k – 2) …. 3*2*1
– We can express this on a graph as follows:
QC - Asymptotic Notation
7
Best, Worst and Average Case Complexity
8
Best, Worst and Average Case Complexity
• In practice the most useful of these three
measures turns out to be worst-case
complexity
• Each of these time complexities defines a
numerical function representing time versus
problem size
• These may sometimes be ‘expressed’ using
polynomials otherwise they can be extremely
complicated functions
• We use the big Oh notation to simplify our
work with complicated functions
• Worst Case Complexity
– The function defined by the maximum number of steps
involved on any instance of input size n
– represented by curve passing through highest points
• Best Case Complexity
– The function defined by the minimum number of steps
involved on any instance of input size n
– represented by curve passing through lowest points
• Average Case Complexity
– The function defined by the average number of steps
involved on any instance of input size n
QC - Asymptotic Notation
QC - Asymptotic Notation
9
Best, Worst and Average Case Complexity
QC - Asymptotic Notation
10
Asymptotic Notation
• With complicated functions
– Difficult to work with exactly due to lots of
bumps, ‘tight’ polynomial approximations are
awkward to work with
– usually cleaner and easier to work with upper and
lower bounds of such functions (see following
graph)
– Smoothing effect sought in order to ignore levels
of detail that do not impact on comparison of
algorithms
QC - Asymptotic Notation
11
QC - Asymptotic Notation
12
2
Asymptotic Notation
Asymptotic Notations - O Notation
f(n) = O(g(n)) means k.g(n) is an upper bound on
f(n).
So there exists some constant k such
that for all n large enough f(n) ≤ k.g(n)
f(n) = Ω(g(n)) means k.g(n) is a lower bound on
f(n). So there exists some constant k such
that for all n large enough f(n) ≥ k.g(n)
f(n) = Θ(g(n)) means k 1.g(n) is an upper bound on
f(n) and k 2.g(n) is a lower bound on f(n).
So there exist constants k 1 and k 2 such that
for all n large enough f(n) ≤ k 1.g(n) and
f(n) ≥ k 2.g(n) So g(n) is a tight bound on f(n)
QC - Asymptotic Notation
13
Asymptotic Notations - Ω Notation
14
Asymptotic Notations - Θ Notation
QC - Asymptotic Notation
15
Which of the following are true, and why?
QC - Asymptotic Notation
16
NB. The Big Oh Notation …
3n
2
−
1 0 0n
+
6
=
O( n 2 )
2 .
3n
2
−
1 0 0n
+
6
=
O( n 3 )
3 .
3n
2
−
1 0 0n
+
6
≠
O( n )
3n
2
−
1 0 0n
+
6
=
Ω( n 2 )
1 .
QC - Asymptotic Notation
• Provides for a rough notion of equality when
comparing functions
5 .
3n
2
−
1 0 0n
+
6
≠
Ω( n )
• Read ‘=’ as meaning ‘is one of the functions
that are’
6 .
3n
2
−
1 0 0n
+
6
=
Ω( n )
• For example:
7 .
3n
2
−
1 0 0n
+
6
=
Θ( n 2 )
n = O(n3)
3n
2
−
1 0 0n
+
6
≠
Θ( n )
n is one of the functions that are O(n3)
3n
2
−
1 0 0n
+
6
≠
Θ( n )
4 .
8 .
9 .
QC - Asymptotic Notation
3
3
17
QC - Asymptotic Notation
18
3
References
•
•
•
•
•
•
•
•
•
Nielson and Chuang, Quantum Computation and Quantum Information,
Cambridge University Press, 2002 (ISBN 0-521-63503-9)
Steven S. Skiena, The Algorithm Design Manual, Telos, 1998 (ISBN 0-38794860-0)
Gennady Berman et al; Introduction to Quantum Computers, World
Scientific, 1998 (ISBN 981-02-3490-2)
Jozef Gruska, Quantum Computing, McGraw-Hill, 1999 (ISBN 0-07709503-0)
Mika Hirvensalo, Quantum Computing, Springer, 2001 (ISBN 3-54066783-0)
Dirk Bouwmeester, Artur Ekert, Anton Zeilinger (Eds.), The Physics of
Quantum Information, Springer, 2000 (ISBN 3-540-66778-4)
Byron and Fuller, Mathematics of Classical and Quantum Physics, Dover,
1992 (ISBN 0-486-67164-X)
Roger Penrose, Shadows of the Mind, Vintage, 1995 (ISBN 0-09-958211-2)
Julian Brown, Minds, Machines and the Multiverse – The Quest for the
Quantum Computer, Simon & Schuster, 2000 (ISBN 0-684-81481-1)
QC - Asymptotic Notation
19
4