Download section home

Algorithms
Anuj Dawar
University of Cambridge Computer Laboratory
The Vocabulary of Big Data, Cambridge, 26 January 2015
The Algorithmic Method
1
1 Wikimedia
Anuj Dawar
Commons/CC-SA-3.0
January 2015
The Algorithmic Method
Muhammad bin Musa Al-Khwarizmi, c.780–c.850
Bayt al-Hikma, Baghdad
1
1 Wikimedia
Anuj Dawar
Commons/CC-SA-3.0
January 2015
The Algorithmic Method
Muhammad bin Musa Al-Khwarizmi, c.780–c.850
Bayt al-Hikma, Baghdad
1
1 Wikimedia
Anuj Dawar
A Latin translation of his book popularised “Arabic numerals” in medieval Europe.
Commons/CC-SA-3.0
January 2015
The Algorithmic Method
Muhammad bin Musa Al-Khwarizmi, c.780–c.850
Bayt al-Hikma, Baghdad
1
A Latin translation of his book popularised “Arabic numerals” in medieval Europe.
The numerals offered much easier method for adding and multiplying
numbers than an abacus (or Roman numerals),
This came to be known as the algorithmic method.
1 Wikimedia
Anuj Dawar
Commons/CC-SA-3.0
January 2015
The Algorist vs. The Abacist
The picture shows a contest
between an Algorist and and
Abacist.
But how fast is the algorist?
Anuj Dawar
January 2015
Adding and Multiplying
Algorithms are finitely described, step-bu-step procedures for solving an
infinite class of problems, by the systematic application of rules.
Anuj Dawar
January 2015
Adding and Multiplying
Algorithms are finitely described, step-bu-step procedures for solving an
infinite class of problems, by the systematic application of rules.
+
···
Anuj Dawar
7
1
8
3
2
5
6
0
7
4
8
2
January 2015
Adding and Multiplying
Algorithms are finitely described, step-bu-step procedures for solving an
infinite class of problems, by the systematic application of rules.
+
···
Anuj Dawar
7
1
8
3
2
5
6
0
7
4
8
2
×
···
···
···
7
1
8
0
2
3
2
9
0
8
6
0
1
0
4
8
2
January 2015
Adding and Multiplying
Algorithms are finitely described, step-bu-step procedures for solving an
infinite class of problems, by the systematic application of rules.
+
···
7
1
8
3
2
5
6
0
7
4
8
2
×
···
···
···
7
1
8
0
2
3
2
9
0
8
6
0
1
0
4
8
2
Adding two 100-digit numbers takes 100 single-digit operations.
Anuj Dawar
January 2015
Adding and Multiplying
Algorithms are finitely described, step-bu-step procedures for solving an
infinite class of problems, by the systematic application of rules.
+
···
7
1
8
3
2
5
6
0
7
4
8
2
×
···
···
···
7
1
8
0
2
3
2
9
0
8
6
0
1
0
4
8
2
Adding two 100-digit numbers takes 100 single-digit operations.
Multiplying two 100-digit numbers takes 10,000 single-digit operations.
Anuj Dawar
January 2015
Complexity
We say that the addition algorithm is of linear complexity.
Anuj Dawar
January 2015
Complexity
We say that the addition algorithm is of linear complexity.
The number of operations grows in proportion to the size of the
problem.
Anuj Dawar
January 2015
Complexity
We say that the addition algorithm is of linear complexity.
The number of operations grows in proportion to the size of the
problem.
The multiplication algorithm is of quadratic complexity.
Anuj Dawar
January 2015
Complexity
We say that the addition algorithm is of linear complexity.
The number of operations grows in proportion to the size of the
problem.
The multiplication algorithm is of quadratic complexity.
The number of operations grows in proportion to square of the
size of the problem.
Anuj Dawar
January 2015
Complexity
We say that the addition algorithm is of linear complexity.
The number of operations grows in proportion to the size of the
problem.
The multiplication algorithm is of quadratic complexity.
The number of operations grows in proportion to square of the
size of the problem.
Is there any faster way to do multiplication?
Anuj Dawar
January 2015
Complexity
We say that the addition algorithm is of linear complexity.
The number of operations grows in proportion to the size of the
problem.
The multiplication algorithm is of quadratic complexity.
The number of operations grows in proportion to square of the
size of the problem.
Is there any faster way to do multiplication?
Yes, and breakthroughs have been reported even in recent years.
Anuj Dawar
January 2015
Complexity
We say that the addition algorithm is of linear complexity.
The number of operations grows in proportion to the size of the
problem.
The multiplication algorithm is of quadratic complexity.
The number of operations grows in proportion to square of the
size of the problem.
Is there any faster way to do multiplication?
Yes, and breakthroughs have been reported even in recent years.
Do we know that multiplication cannot be done by a linear algorithm?
Anuj Dawar
January 2015
Complexity
We say that the addition algorithm is of linear complexity.
The number of operations grows in proportion to the size of the
problem.
The multiplication algorithm is of quadratic complexity.
The number of operations grows in proportion to square of the
size of the problem.
Is there any faster way to do multiplication?
Yes, and breakthroughs have been reported even in recent years.
Do we know that multiplication cannot be done by a linear algorithm?
The question is still open.
Anuj Dawar
January 2015
Complexity
We say that the addition algorithm is of linear complexity.
The number of operations grows in proportion to the size of the
problem.
The multiplication algorithm is of quadratic complexity.
The number of operations grows in proportion to square of the
size of the problem.
Is there any faster way to do multiplication?
Yes, and breakthroughs have been reported even in recent years.
Do we know that multiplication cannot be done by a linear algorithm?
The question is still open. No such algorithms is known but it
has not been proved impossible.
Anuj Dawar
January 2015
Abstraction
Algorithmic problems are usually expressed at a level of abstraction that
is removed from the specifics of the data involved.
Anuj Dawar
January 2015
Abstraction
Algorithmic problems are usually expressed at a level of abstraction that
is removed from the specifics of the data involved.
For example, a graph is just a collection of elements and their pairwise connectivity.
Anuj Dawar
January 2015
Abstraction
Algorithmic problems are usually expressed at a level of abstraction that
is removed from the specifics of the data involved.
For example, a graph is just a collection of elements and their pairwise connectivity.
Anuj Dawar
January 2015
Abstraction
Algorithmic problems are usually expressed at a level of abstraction that
is removed from the specifics of the data involved.
For example, a graph is just a collection of elements and their pairwise connectivity.
The elements could be people on a
social network; or pixels in an image or neurons in the brain, etc.
Anuj Dawar
January 2015
Abstraction
Algorithmic problems are usually expressed at a level of abstraction that
is removed from the specifics of the data involved.
For example, a graph is just a collection of elements and their pairwise connectivity.
The elements could be people on a
social network; or pixels in an image or neurons in the brain, etc.
Assign the elements into six different bins so that two elements in the
same bin are not connected.
Anuj Dawar
January 2015
Abstraction
Algorithmic problems are usually expressed at a level of abstraction that
is removed from the specifics of the data involved.
For example, a graph is just a collection of elements and their pairwise connectivity.
The elements could be people on a
social network; or pixels in an image or neurons in the brain, etc.
Assign the elements into six different bins so that two elements in the
same bin are not connected.
Find a 100 distinct elements that are pairwise not connected.
Anuj Dawar
January 2015
Algorithms and Big Data
There is a rich theory analysing the complexity of algorithms from the
point of view of:
Anuj Dawar
January 2015
Algorithms and Big Data
There is a rich theory analysing the complexity of algorithms from the
point of view of:
• computational resources;
Anuj Dawar
January 2015
Algorithms and Big Data
There is a rich theory analysing the complexity of algorithms from the
point of view of:
• computational resources;
• statistical resources;
Anuj Dawar
January 2015
Algorithms and Big Data
There is a rich theory analysing the complexity of algorithms from the
point of view of:
• computational resources;
• statistical resources;
• physical resources
Anuj Dawar
January 2015
Algorithms and Big Data
There is a rich theory analysing the complexity of algorithms from the
point of view of:
• computational resources;
• statistical resources;
• physical resources
Along with developing new, more efficient algorithms, we also establish
the limitations of algorithmic methods through establishing complexity
lower bounds.
Anuj Dawar
January 2015
Algorithms and Big Data
There is a rich theory analysing the complexity of algorithms from the
point of view of:
• computational resources;
• statistical resources;
• physical resources
Along with developing new, more efficient algorithms, we also establish
the limitations of algorithmic methods through establishing complexity
lower bounds.
logarithmic; linear; quadratic; cubic; polynomial;
superpolynomial; subexponential; exponential
Anuj Dawar
January 2015