Download Logic, Algorithms and Data Structures The Big O(h)

Logic, Algorithms
and Data Structures
The Big O(h)
M2
How do we measure
complexity?

There are four (more) interesting aspects to
complexity:


Accuracy
Speed


Space


Number of operations required in (best, average, worst)
case
Memory usage (in memory and on disk)
Code

Readability and portability of code
Algorithmic speed

The Big O(h) notation (“Order of magnitude”)




O(n), O(n^2), O(n log n), …
Refers to the performance of the algorithm in
the worst case
An approximation to make it easier to
discuss the relative performance of algorithms
Expresses the rate of growth in computational
resources needed
Some common expressions

O(1)

O(n)

O(log n)

O(n^2)
The best time for any algorithm;
regardless of data size, it takes
a fixed amount of time
Linear time, depends heavily on
the data size
Logarithmic increase of time in
relation to data size
Increases with the square of the
data size
Calculating the Big O



Calculate taking the worst case into
consideration!
Count the number of operations needed to
complete the algorithm.
The highest-order term is usually the
dominating rate of growth (log n, n, n^2, n^3)
Example algorithm
1: y := 0
2: x := 0
3: while x < N do begin
4: z := x * 10
5: y := y + z
6: x := x + 1
7: end
Constant time: L1, L2
Variable time: N * (L4, L5, L6)
Complexity: O(2 + 3n)
Constants in Big O Notation


We can usually ignore the constants when
reasoning around performance
By O(2+3n) we mean:




In the worst case, the algorithm needs to go
through the entire data set, consisting of n
elements, and for each perform 4 operations.
Compare against O(10+50n), O(log n), O(n^2)
n is the highest order term and determines
the rate of growth
Simplify to O(n)
Another example algorithm
i := 2
while i < N do
begin
A[i] := i
i := i * 2
end;
16
40
14
35
12
30
10
25
8
20
6
15
4
10
2
5
0
O(log n)
0
2
1
4
3
6
5
8 10 12 14 16 18 20 22 24 26 28 30 32
7 9 11 13 15 17 19 21 23 25 27 29 31
150
0
450
300
750
600
1050 1350 1650 1950
900
1200 1500 1800 2100
A variation of the same
for j := 1 to N do
begin
i := 2
while i < N do
begin
A[i] := i
i := i * 2
end;
end;
O(n log n)
Performance of insertion sort?

If the data is sorted?

If the data is reversed?
O(n)
O(n*n) = O(n^2)
Getting rid of m through a
different data structure


This is what we have:
l
8 7
3 5 9
m
7 8
? ? ?
Modifying both l and m takes time, but most
importantly, memory!
Our new data structure

Insert Sort Version 2:
int [] l; // An array
int s;
// How much of l is sorted
s
l
7 8
3 5 9
Measuring the performance
V1
V2
V3
V1,V2
V3
Effect on space usage
V1
V2
V3
V1
V2, V3
What happened with
accuracy?




Of course, you need to verify your algorithms!
Simple algorithms are easy to implement
Clever algorithms, not so much
Most clever algorithms have already been
invented and proven correct
Refer to your literature and
known sources!
Comparing our algorithms
Time complexity
V1,V2
Space complexity
V1
V3
V2, V3
Which is the better one?
Readability
V1 – Good
V2 – OK
V3 – Tricky
Choosing an algorithm

Implementations of algorithms vary with:





Speed
Space
Code
Choosing an algorithm is a tradeoff between
those qualities
Word of warning:


Implement clever algorithms
Don't invent clever algorithms
Relating ADTs to the Big O(h)
Final notes

Algorithms affect performance, through;





Accuracy
Speed
Space usage
Code readability
Choose wisely, prioritising between these
qualities!