Download Random walks Mechanisms for Generating Power-Law Size Distributions I The essential random walk:

Power-Law
Mechanisms I
Mechanisms for Generating
Power-Law Size Distributions I
Power-Law
Mechanisms I
Random walks
Random Walks
Random Walks
The First Return Problem
The First Return Problem
Examples
Principles of Complex Systems
CSYS/MATH 300, Fall, 2011
Examples
Variable
transformation
Basics
The essential random walk:
Holtsmark’s Distribution
PLIPLO
Prof. Peter Dodds
References
Department of Mathematics & Statistics | Center for Complex Systems |
Vermont Advanced Computing Center | University of Vermont
Licensed under the Creative Commons Attribution-NonCommercial-ShareAlike 3.0 License.
Outline
Variable
transformation
Basics
I
One spatial dimension.
I
Time and space are discrete
I
Random walker (e.g., a drunk) starts at origin x = 0.
I
Step at time t is t :
+1 with probability 1/2
t =
−1 with probability 1/2
Holtsmark’s Distribution
PLIPLO
References
1 of 56
Power-Law
Mechanisms I
4 of 56
Power-Law
Mechanisms I
Random walks
Random Walks
Random Walks
The First Return Problem
The First Return Problem
Examples
Random Walks
The First Return Problem
Examples
Examples
Variable
transformation
Variable
transformation
Displacement after t steps:
Basics
Basics
Holtsmark’s Distribution
PLIPLO
xt =
References
t
X
Holtsmark’s Distribution
PLIPLO
i
References
i=1
Variable transformation
Basics
Holtsmark’s Distribution
PLIPLO
Expected displacement:
* t +
t
X
X
hi i = 0
hxt i =
i =
i=1
i=1
References
2 of 56
Mechanisms
Power-Law
Mechanisms I
5 of 56
Power-Law
Mechanisms I
Random walks
Random Walks
Random Walks
The First Return Problem
The First Return Problem
Examples
Examples
Variable
transformation
Variances sum: ()
Variable
transformation
∗
Basics
Basics
Holtsmark’s Distribution
PLIPLO
A powerful story in the rise of complexity:
I
I
Var(xt ) = Var
References
t
X
!
i
Holtsmark’s Distribution
PLIPLO
References
i=1
structure arises out of randomness.
Exhibit A: Random walks... ()
=
t
X
i=1
Var (i ) =
t
X
1=t
i=1
∗
Sum rule = a good reason for using the variance to measure
spread; only works for independent distributions.
3 of 56
6 of 56
Power-Law
Mechanisms I
Random walks
Power-Law
Mechanisms I
Random walks—some examples
Random Walks
Random Walks
The First Return Problem
The First Return Problem
Examples
Examples
50
Variable
transformation
0
x
Variable
transformation
−50
Basics
Basics
Holtsmark’s Distribution
So typical displacement from the origin scales as
Holtsmark’s Distribution
−100
0
PLIPLO
1000
2000
3000
4000
5000
6000
7000
8000
9000
10000
t
References
50
σ = t 1/2
PLIPLO
References
x
0
−50
⇒ A non-trivial power-law arises out of
additive aggregation or accumulation.
−100
0
1000
2000
3000
4000
5000
6000
7000
8000
9000
10000
6000
7000
8000
9000
10000
t
200
x
100
0
−100
0
1000
2000
3000
4000
5000
t
7 of 56
Random walks
Power-Law
Mechanisms I
10 of 56
Power-Law
Mechanisms I
Random walks—some examples
Random Walks
Random Walks
The First Return Problem
The First Return Problem
Examples
Variable
transformation
−50
Basics
For example:
Examples
Variable
transformation
0
x
Random walks are weirder than you might think...
50
Basics
Holtsmark’s Distribution
Holtsmark’s Distribution
−100
0
PLIPLO
ξr ,t = the probability that by time step t, a random
walk has crossed the origin r times.
I
Think of a coin flip game with ten thousand tosses.
2000
3000
4000
5000
6000
7000
8000
9000
10000
t
References
PLIPLO
References
200
100
x
I
1000
I
See
If you are behind early on, what are the chances you
will make a comeback?
2000
3000
4000
5000
6000
7000
8000
9000
10000
6000
7000
8000
9000
10000
t
100
The most likely number of lead changes is... 0.
Feller, [2]
1000
200
x
I
0
−100
0
0
−100
0
Intro to Probability Theory, Volume I
1000
2000
3000
4000
5000
t
8 of 56
Random walks
Power-Law
Mechanisms I
11 of 56
Random walks
Random Walks
Random Walks
The First Return Problem
The First Return Problem
Examples
Examples
Variable
transformation
Variable
transformation
Basics
Basics
Holtsmark’s Distribution
In fact:
PLIPLO
The problem of first return:
I
What is the probability that a random walker in one
dimension returns to the origin for the first time after t
steps?
Even crazier:
I
Will our drunkard always return to the origin?
The expected time between tied scores = ∞!
I
What about higher dimensions?
ξ0,t > ξ1,t > ξ2,t > · · ·
Power-Law
Mechanisms I
References
9 of 56
Holtsmark’s Distribution
PLIPLO
References
12 of 56
Power-Law
Mechanisms I
First returns
Power-Law
Mechanisms I
First Returns
Random Walks
The First Return Problem
Examples
Variable
transformation
Basics
Reasons for caring:
Random Walks
The First Return Problem
Examples
Holtsmark’s Distribution
For random walks in 1-d:
Variable
transformation
Basics
I
Return can only happen when t = 2n.
I
Call Pfirst return (2n) = Pfr (2n) probability of first return
at t = 2n.
I
Assume drunkard first lurches to x = 1.
I
The problem
Holtsmark’s Distribution
PLIPLO
1. We will find a power-law size distribution with an
interesting exponent
PLIPLO
References
2. Some physical structures may result from random
walks
3. We’ll start to see how different scalings relate to
each other
References
Pfr (2n) = 2Pr (xt ≥ 1, t = 1, . . . , 2n − 1, and x2n = 0)
13 of 56
Power-Law
Mechanisms I
Random Walks
17 of 56
Power-Law
Mechanisms I
First Returns
Random Walks
The First Return Problem
4
Walks
4Random
The First Return Problem
3
3Variable
Examples
Examples
Variable
transformation
x
Basics
Holtsmark’s Distribution
2
x
0
−50
transformation
x
50
PLIPLO
−100
0
1000
2000
3000
4000
5000
6000
7000
8000
9000
10000
Basics
2Holtsmark’s Distribution
PLIPLO
References
t
200
1
1References
0
0
x
100
2
4
6
8
10
12
14
16
0
0
−100
0
1000
2000
3000
4000
5000
6000
7000
8000
9000
10000
t
Again: expected time between ties = ∞...
Let’s find out why... [2]
I
A useful restatement: Pfr (2n) =
2 · 12 Pr (xt ≥ 1, t = 1, . . . , 2n − 1, and x1 = x2n−1 = 1)
I
Want walks that can return many times to x = 1.
I
(The 12 accounts for stepping to 2 instead of 0 at
t = 2n.)
15 of 56
Power-Law
Mechanisms I
First Returns
18 of 56
First Returns
Random Walks
Random Walks
The First Return Problem
The First Return Problem
Examples
Basics
I
Holtsmark’s Distribution
PLIPLO
2
References
0
5
10
15
Use a method of images
I
Define N(i, j, t) as the # of possible walks between
x = i and x = j taking t steps.
I
Consider all paths starting at x = 1 and ending at
x = 1 after t = 2n − 2 steps.
I
Subtract how many hit x = 0.
20
t
16 of 56
Counting problem (combinatorics/statistical
mechanics)
I
−2
−4
0
Power-Law
Mechanisms I
Examples
Variable
transformation
4
x
2
t
0
Variable
transformation
Basics
Holtsmark’s Distribution
PLIPLO
References
19 of 56
4
Power-Law
Mechanisms I
First Returns
Power-Law
Mechanisms I
First Returns
Random Walks
Random Walks
4
The First Return Problem
The First Return Problem
Examples
3
Variable
transformation
Variable
transformation
Basics
Basics
2
Holtsmark’s Distribution
Holtsmark’s Distribution
PLIPLO
PLIPLO
References
References
1
0
x
Key observation:
# of t-step paths starting and ending at x = 1
and hitting x = 0 at least once
= # of t-step paths starting at x = −1 and ending at x = 1
= N(−1, 1, t)
Examples
−1
So Nfirst return (2n) = N(1, 1, 2n − 2) − N(−1, 1, 2n − 2)
−2
See this 1-1 correspondence visually...
−3
−4
0
2
4
6
8
10
12
14
16
t
20 of 56
Power-Law
Mechanisms I
First Returns
23 of 56
Power-Law
Mechanisms I
First Returns
Random Walks
4
Random Walks
The First Return Problem
The First Return Problem
Examples
3
Examples
Variable
transformation
Basics
2
Holtsmark’s Distribution
I
Next problem: what is N(i, j, t)?
I
# positive steps + # negative steps = t.
I
Random walk must displace by j − i after t steps.
I
# positive steps - # negative steps = j − i.
I
# positive steps = (t + j − i)/2.
PLIPLO
References
1
x
0
−1
Basics
Holtsmark’s Distribution
PLIPLO
References
I
−2
N(i, j, t) =
−3
−4
0
Variable
transformation
2
4
6
8
10
12
14
t
t
=
# positive steps
(t + j − i)/2
16
t
21 of 56
First Returns
Power-Law
Mechanisms I
24 of 56
Power-Law
Mechanisms I
First Returns
Random Walks
Random Walks
The First Return Problem
The First Return Problem
Examples
Examples
Variable
transformation
Variable
transformation
Basics
Basics
Holtsmark’s Distribution
PLIPLO
I
I
For any path starting at x = 1 that hits 0,
there is a unique matching path starting at x = −1.
We now have
Holtsmark’s Distribution
PLIPLO
References
References
Nfirst return (2n) = N(1, 1, 2n − 2) − N(−1, 1, 2n − 2)
Matching path first mirrors and then tracks.
where
N(i, j, t) =
22 of 56
t
(t + j − i)/2
25 of 56
Power-Law
Mechanisms I
First Returns
Insert question from assignment 4 ()
Find Nfirst return (2n) ∼
22n−3/2
√
2πn3/2
.
Random walks
Random Walks
Random Walks
The First Return Problem
The First Return Problem
Examples
Examples
Variable
transformation
Basics
On finite spaces:
Holtsmark’s Distribution
I
I
I
Normalized Number of Paths gives Probability
Total number of possible paths =
22n
References
1 22n−3/2
√
22n 2πn3/2
Variable
transformation
Basics
Holtsmark’s Distribution
I
PLIPLO
1
Pfirst return (2n) = 2n Nfirst return (2n)
2
'
Power-Law
Mechanisms I
In any finite volume, a random walker will visit every
site with equal probability
I
Random walking ≡ Diffusion
I
Call this probability the Invariant Density of a
dynamical system
I
Non-trivial Invariant Densities arise in chaotic
systems.
PLIPLO
References
1
= √ (2n)−3/2
2π
26 of 56
Power-Law
Mechanisms I
First Returns
29 of 56
Random walks on
Random Walks
Random Walks
The First Return Problem
The First Return Problem
Examples
Examples
Variable
transformation
I
Same scaling holds for continuous space/time walks.
P(t) ∝ t
−3/2
, γ = 3/2
I
P(t) is normalizable
I
Recurrence: Random walker always returns to origin
I
Moral: Repeated gambling against an infinitely
wealthy opponent must lead to ruin.
Variable
transformation
Basics
Basics
Holtsmark’s Distribution
PLIPLO
I
Power-Law
Mechanisms I
References
Holtsmark’s Distribution
On networks:
I
On networks, a random walker visits each node with
frequency ∝ node degree
I
Equal probability still present:
walkers traverse edges with equal frequency.
PLIPLO
References
27 of 56
First Returns
Power-Law
Mechanisms I
30 of 56
Scheidegger Networks [4, 1]
Random Walks
Random Walks
The First Return Problem
The First Return Problem
Examples
Examples
Variable
transformation
Variable
transformation
Basics
Higher dimensions:
I
Walker in d = 2 dimensions must also return
I
Walker may not return in d ≥ 3 dimensions
I
For d = 1, γ = 3/2 → hti = ∞
I
Even though walker must return, expect a long wait...
Power-Law
Mechanisms I
Basics
Holtsmark’s Distribution
Holtsmark’s Distribution
PLIPLO
PLIPLO
References
28 of 56
References
I
Triangular lattice
I
‘Flow’ is southeast or southwest with equal
probability.
32 of 56
Power-Law
Mechanisms I
Scheidegger Networks
Connections between Exponents
Random Walks
Random Walks
The First Return Problem
The First Return Problem
Examples
I
Creates basins with random walk boundaries
I
Observe Subtracting one random walk from another
gives random walk with increments

 +1 with probability 1/4
0 with probability 1/2
t =

−1 with probability 1/4
Examples
Variable
transformation
Variable
transformation
Basics
Basics
Holtsmark’s Distribution
I
Power-Law
Mechanisms I
Holtsmark’s Distribution
I
PLIPLO
References
I
Generalize relationship between area and length
Hack’s
PLIPLO
References
law [3] :
` ∝ ah
where 0.5 . h . 0.7
I
Redo calc with γ, τ , and h.
Basin length ` distribution: P(`) ∝ `−3/2
33 of 56
Connections between Exponents
Power-Law
Mechanisms I
36 of 56
Connections between Exponents
Random Walks
Random Walks
The First Return Problem
I
Examples
I
For a basin of length `, width ∝
I
Basin area a ∝ ` · `1/2 = `3/2
`1/2
a2/3
I
Invert: ` ∝
I
d` ∝ d(a2/3 ) = 2/3a−1/3 da
I
Pr (basin area = a)da
= Pr (basin length = `)d`
∝ `−3/2 d`
∝ (a2/3 )−3/2 a−1/3 da
= a−4/3 da
= a−τ da
Power-Law
Mechanisms I
The First Return Problem
Given
Variable
transformation
Examples
` ∝ ah , P(a) ∝ a−τ , and P(`) ∝ `−γ
Basics
Holtsmark’s Distribution
Variable
transformation
Basics
Holtsmark’s Distribution
PLIPLO
PLIPLO
References
d(ah )
hah−1 da
I
d` ∝
I
Pr (basin area = a)da
= Pr (basin length = `)d`
∝ `−γ d`
∝ (ah )−γ ah−1 da
= a−(1+h (γ−1)) da
=
References
I
τ = 1 + h(γ − 1)
34 of 56
Connections between Exponents
Power-Law
Mechanisms I
37 of 56
Connections between Exponents
Random Walks
Random Walks
The First Return Problem
Examples
Variable
transformation
The First Return Problem
With more detailed description of network structure,
τ = 1 + h(γ − 1) simplifies:
Basics
I
Both basin area and length obey power law
distributions
I
Observed for real river networks
I
Typically: 1.3 < τ < 1.5 and 1.5 < γ < 2
I
Smaller basins more allometric (h > 1/2)
I
Power-Law
Mechanisms I
Examples
Variable
transformation
Basics
Holtsmark’s Distribution
Holtsmark’s Distribution
τ =2−h
PLIPLO
References
PLIPLO
References
γ = 1/h
Larger basins more isometric (h = 1/2)
35 of 56
I
Only one exponent is independent
I
Simplify system description
I
Expect scaling relations where power laws are found
I
Characterize universality class with independent
exponents
38 of 56
Power-Law
Mechanisms I
Other First Returns
Power-Law
Mechanisms I
Variable Transformation
Random Walks
Random Walks
The First Return Problem
The First Return Problem
Examples
Failure
A very simple model of failure/death:
Basics
I
xt = entity’s ‘health’ at time t
PLIPLO
I
x0 could be > 0.
I
Entity fails when x hits 0.
I
Examples
Variable
transformation
Holtsmark’s Distribution
References
Variable
transformation
I
Random variable X with known distribution Px
I
Second random variable Y with y = f (x).
I
Streams
I
I
Dispersion of suspended sediments in streams.
I
Long times for clearing.
Basics
Holtsmark’s Distribution
PLIPLO
References
Py (y )dy = Px (x)dx
=
P
−1 (y ))
y |f (x)=y Px (f
dy Figure...
|f 0 (f −1 (y ))|
Often easier to do by
hand...
39 of 56
More than randomness
Power-Law
Mechanisms I
43 of 56
Power-Law
Mechanisms I
General Example
Assume relationship between x and y is 1-1.
Random Walks
The First Return Problem
I
Examples
Variable
transformation
Can generalize to Fractional Random Walks
Basics
I
Levy flights, Fractional Brownian Motion
PLIPLO
I
In 1-d,
I
I
Power-law relationship between variables:
y = cx −α , α > 0
Look at y large and x small
The First Return Problem
Examples
Variable
transformation
Basics
Holtsmark’s Distribution
I
Holtsmark’s Distribution
dy = d cx
References
σ ∼tα
−α
PLIPLO
References
= c(−α)x −α−1 dx
α > 1/2 — superdiffusive
α < 1/2 — subdiffusive
I
Random Walks
−1 α+1
x
dy
cα
−1 y −(α+1)/α
dx =
dy
cα c
invert: dx =
Extensive memory of path now matters...
dx =
−c 1/α −1−1/α
y
dy
α
44 of 56
40 of 56
Variable Transformation
Power-Law
Mechanisms I
Power-Law
Mechanisms I
General Example
Now make transformation:
Random Walks
Random Walks
The First Return Problem
The First Return Problem
Py (y )dy = Px (x)dx
Examples
Examples
Variable
transformation
Variable
transformation
Basics
(x)
Holtsmark’s Distribution
Understand power laws as arising from
dx
}|
{
z
}|
{z
y −1/α c 1/α
−1−1/α
Py (y )dy = Px
y
dy
c
α
PLIPLO
References
1. elementary distributions (e.g., exponentials)
Basics
Holtsmark’s Distribution
PLIPLO
References
2. variables connected by power relationships
I
If Px (x) → non-zero constant as x → 0 then
Py (y ) ∝ y −1−1/α as y → ∞.
I
If Px (x) → x β as x → 0 then
Py (y ) ∝ y −1−1/α−β/α as y → ∞.
42 of 56
45 of 56
Power-Law
Mechanisms I
Example
Power-Law
Mechanisms I
Transformation
Random Walks
Random Walks
The First Return Problem
The First Return Problem
Examples
Examples
Variable
transformation
Exponential distribution
I
dF ∝ d(r −2 )
Basics
Holtsmark’s Distribution
Given Px (x) = λ1 e−x/λ and y = cx −α , then
P(y ) ∝ y −1−1/α + O y −1−2/α
PLIPLO
Variable
transformation
Basics
Holtsmark’s Distribution
PLIPLO
References
I
∝ r −3 dr
I
References
invert:
dr ∝ r 3 dF
I
Exponentials arise from randomness...
I
More later when we cover robustness.
I
∝ F −3/2 dF
46 of 56
Power-Law
Mechanisms I
Gravity
Random Walks
The First Return Problem
50 of 56
Power-Law
Mechanisms I
Transformation
Using r ∝ F −1/2 , dr ∝ F −3/2 dF and Pr (r ) ∝ r 2
Random Walks
The First Return Problem
Examples
Examples
Variable
transformation
I
PF (F )dF = Pr (r )dr
Basics
Basics
Holtsmark’s Distribution
Holtsmark’s Distribution
PLIPLO
Variable
transformation
PLIPLO
I
I
Select a random point in the
universe ~x
I
(possible all of space-time)
I
I
Measure the force of gravity
F (~x )
I
I
Observe that PF (F ) ∼ F −5/2 .
∝ Pr (F −1/2 )F −3/2 dF
References
References
2
∝ F −1/2 F −3/2 dF
= F −1−3/2 dF
I
= F −5/2 dF
51 of 56
48 of 56
Power-Law
Mechanisms I
Ingredients [5]
Matter is concentrated in stars:
I
I
F is distributed unevenly
Probability of being a distance r from a single star at
~x = ~0:
Pr (r )dr ∝ r 2 dr
Power-Law
Mechanisms I
Gravity
Random Walks
Random Walks
The First Return Problem
The First Return Problem
Examples
Examples
Variable
transformation
PF (F ) = F −5/2 dF
Basics
Holtsmark’s Distribution
PLIPLO
PLIPLO
References
I
References
γ = 5/2
I
Mean is finite
I
I
Variance = ∞
I
Law of gravity:
I
A wild distribution
I
Random sampling of space usually safe
but can end badly...
F ∝ r −2
I
Basics
Holtsmark’s Distribution
Assume stars are distributed randomly in space
(oops?)
Assume only one star has significant effect at ~x .
I
Variable
transformation
invert:
r ∝ F −1/2
49 of 56
52 of 56
Caution!
Power-Law
Mechanisms I
Random Walks
The First Return Problem
Examples
Variable
transformation
Basics
I
PLIPLO = Power law in, power law out
I
Explain a power law as resulting from another
unexplained power law.
I
Yet another homunculus argument ()...
I
Don’t do this!!! (slap, slap)
I
We need mechanisms!
Holtsmark’s Distribution
PLIPLO
References
54 of 56
References I
Power-Law
Mechanisms I
Random Walks
[1] P. S. Dodds and D. H. Rothman.
Scaling, universality, and geomorphology.
Annu. Rev. Earth Planet. Sci., 28:571–610, 2000.
pdf ()
[2] W. Feller.
An Introduction to Probability Theory and Its
Applications, volume I.
John Wiley & Sons, New York, third edition, 1968.
The First Return Problem
Examples
Variable
transformation
Basics
Holtsmark’s Distribution
PLIPLO
References
[3] J. T. Hack.
Studies of longitudinal stream profiles in Virginia and
Maryland.
United States Geological Survey Professional Paper,
294-B:45–97, 1957. pdf ()
55 of 56
References II
Power-Law
Mechanisms I
Random Walks
The First Return Problem
Examples
Variable
transformation
[4] A. E. Scheidegger.
The algebra of stream-order numbers.
United States Geological Survey Professional Paper,
525-B:B187–B189, 1967. pdf ()
Basics
Holtsmark’s Distribution
PLIPLO
References
[5] D. Sornette.
Critical Phenomena in Natural Sciences.
Springer-Verlag, Berlin, 2nd edition, 2003.
56 of 56