Download Algorithmic Problems Related To The Internet

The Algorithmic Lens
on the Sciences
Christos H. Papadimitriou
UC Berkeley
What is Computer Science?
“CS is the only scientific discipline that
cannot be defined in a single sentence”
(now seriously folks…)
What is Computer Science?
•
•
•
•
Applied science?
Engineering discipline?
Narrative of a transformative technology?
(in the era of the Internet) Also a natural
and social science?
Mathematics:
“The queen and servant of sciences”
• Queen: Power, authority,
pride, beauty
• Servant: Influences and
transforms by being useful,
powerful, and universal
• My point: CS is the new
math
Perturbing Physics
The algorithmic worldview provides
new insights into, and tests, some of
the most prestigious theories about
the universe
Statistical Physics and Algorithms
How does the
lake freeze?
The mystery
of phase
transitions
vs. the convergence of algorithms
Quantum computation:
reinventing the bit
many electrons
bit: a wire can have
few electrons
qubit: an electron
can be
close
to the nucleus
far
Big difference
• A bit is either 0 or 1
• A qubit is in both states Q =  |0 +  |1
complex numbers
“probabilities”
• An n-qubit system is in 2n states at the same
time!
Three possible reactions
1. How curious, Nature is extravagant!
2. Oh my God, how do you simulate such a
system on a computer?
3. But what if we built a computer
out of these things?
How to factor a 1000-bit integer
in ~1000 easy steps
(on a quantum computer)
input
1000 bits
the “probabilities”
of 21000 states
maintained throughout
output
(measurement)
1000 bits
But can we build these computers?
The three eventualities
1. Yes!
2. No, because of a thousand annoying little
problems and details (plus, eventually,
lack of funding…)
3. No, because Quantum Physics breaks
down for large numbers of particles!!!
“Quantum computation is as much
about testing Quantum Physics as it
is about building powerful
computers.”
Umesh Vazirani
Equilibria: Behavior predictions
in Economics
The Story of Equilibria
They exist in two-player
zero-sum games,1928
John Nash 1950:
all games have one!
In markets too!
Price equilibria (Arrow-Debreu 1954)
“The Nash equilibrium lies at the
foundations of modern economic
thought.”
Roger Myerson
Universality (Nash’s theorem)
is important
“Nobody
would take seriously a
solution concept that is empty for
some games.”
Roger Myerson
Surprise!
Finding
equilibria
is an
intractable
problem!
And intractability means
Nash equilibrium’s universality
is suspect
“If your laptop can’t find it, neither
can the market”
Kamal Jain
Next
• New, more “algorithmic” solution concept
based on player dynamics?
Evolution under the Lens
“What algorithm could have created
all this
in a mere 1012 steps?”
The Origin of Species
• Possibly the world’s most
masterfully compelling scientific
argument
• Natural selection
• Common ancestry
Evolution Theory since 1859
•
•
•
•
Genetics (Mendel, 1866 – really, 1901)
The crisis (1901 – 1930)
The synthesis through math (1930 – )
The genomics revolution (1980 – )
The Synthesis: The Fisher-WrightHaldane model
Entries: fitness
of the genotype
(exp. # offspring,
normalized)
1.05 1.00 0.96
1.01 1.03 1.02
0.94 1.02 0.99
Columns: alleles
of gene B
Rows: alleles
of gene A
The Fisher-Wright-Haldane model:
frequencies at current generation
0.33
0.33
0.33
0.33
0.33
0.33
1.05 1.00 0.96
1.01 1.03 1.02
0.94 1.02 0.99
The Fisher-Wright-Haldane model:
frequencies at next generation
0.33
0.34
0.34
0.31
0.35
0.32
1.05 1.00 0.96
1.01 1.03 1.02
0.94 1.02 0.99
Brilliant theory, a deluge of data
-- and yet most important questions
unanswered
• Why so much genetic diversity?
• What is the role of sex/recombination?
• Is Evolution optimizing something?
Btw: 100 generations later?
1.05 1.00 0.96
1.01 1.03 1.02
0.94 1.02 0.99
Btw: 100 generations later?
0.30
0.24
0.73
0.03
0.63 0.07
1.05 1.00 0.96
1.01 1.03 1.02
0.94 1.02 0.99
Mixability Theory of the Role of
Sex: [LPF 2007]:
good mixers win!
0.30
0.24
0.73
0.03
0.63 0.07
1.05 1.00 0.96
1.01 1.03 1.02
0.94 1.02 0.99
Contrast: In asexual species
the fittest genotype wins
1.05 1.00 0.96
1.01 1.03 1.02
0.94 1.02 0.99
Surprising connection with
algorithms [CLPV 2014]
• The Fisher-Wright-Haldane model is
mathematically equivalent to a repeated game
between genes:
• The strategies of each gene are its alleles
• The probabilities of play are the frequencies
• The common utility is the organism’s fitness
• The genes update their probabilities of play
through the multiplicative updates algorithm!
Multiplicative updates!
• (Also known as no-regret learning)
• A simple, common-sense algorithm known
in CS for its surprising aptness at solving
many sophisticated problems
• At each step, increase the weight of allele i
by a factor of (1 + ε fi)
• fi is the allele’s fitness in the current
environment created by the other genes
Multiplicative updates:
Dual interpretation
Convex optimization duality: Each gene “seeks
to optimize” the sum of two quantities:
allele frequencies
cumulative fitness
maxx Φ(x) = H(x) + s F x
entropy
selection strength
Recall the Big Questions
•
•
•
•
Why so much genetic diversity?
What is the role of sex/recombination?
Is Evolution optimizing something?
“What algorithm could have done this
in a mere 1012 steps?”
Finally: The frontier in our head
Brain and Computation:
The Great Disconnects
• Babies vs computers
• Clever algorithms vs what happens in cortex
• Understanding Brain anatomy and function
vs understanding the emergence of the Mind
• How does one think computationally about
the Brain?
A possible approach
• (Current work with Santosh Vempala, and
Wolfgang Maas and his group in Graz)
• What is the boundary between
“subsymbolic” and “symbolic” brain
processes?
• One candidate: Assemblies of excitatory
neurons in MTL
The experiment by
[Ison et al. 2016]
The experiment by
[Ison et al. 2016]
The experiment by
[Ison et al. 2016]
The experiment by
[Ison et al. 2016]
The experiment by
[Ison et al. 2016]
The experiment by
[Ison et al. 2016]
The experiment by
[Ison et al. 2016]
Interpretation
• Each “thing” is represented by an assembly
of many neurons (~0,5% of all)
• Every time we are thinking of the “thing”
all these neurons fire
• If two “things” become related, these
assemblies are “JOINed:” some neurons fire
on either “thing”
Questions
• Can this interpretattion be predicted by a
realistic mathematical model of neurons and
synapses? (It is predicted in simulations)
• How about operations besides JOIN such as
LINK or BIND (e.g., “kick” is a “verb”)?
• Can assemblies be the right conceptual
interface between CS and Brain Science?
• Connection with language?
Soooo…
• CS is worth teaching not only because it is
(a) a fascinating and open-ended subject;
(b) useful and in dmand;
(c) as central and present in the world today
as mass, energy, and life…
…but also because (just like math)
• (d) without a solid understanding of
computation you are handicapped as a
scientist
Danke!
Merci!
Grazie!
Engrazie!
Ευχαριστώ!