Download probability mass functions - Departamento de Ecologia - IB

Statistical Distributions
Why do we care about probability?
• Foundation of theory of statistics.
• Description of uncertainty associated with random
variables.
– Measurement error
– Process error
• Needed to understand:
 Estimating model parameters.
 Model validation
 Prediction
Types of random variables
• Discrete random variables take only integer
values (e.g. counts, memberships in
categories). They are represented by
probability mass functions.
• Continuous random variables are represented
by probability density functions.
Terminology
y ~ f (a, b)
[ y | a, b]
P ( y | a, b)
This notation is equivalent.
You will see it all in this course.
These are general stand-ins for distributions.
y ~ gamma(a, b)
gamma( y | a, b)
Probability Mass Functions
For a discrete random variable, X, the probability that x takes on
a value x is a discrete density function, f(x) also known as
probability mass or distribution function.
[ z]  1
zS
0.2
0.18
0.16
Probability
[ z ]  Pr[ z ]
f ( z )  f {Z  z}
0  [ z]  1
0.14
0.12
0.1
0.08
0.06
0.04
0.02
S=support is the domain
of function [z]
0
0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20
Event (z)
Probability Density Functions: Continuous variables
A probability density function [z] gives the probability that a
random variable Z takes on values within a range.
[z]≥0
Pr(a ≤ z ≤ b)
𝑏
𝑧 𝑑𝑧
𝑎
∞
𝑧
−∞
𝑑𝑧=1
Probability Mass Functions:
First Moment
• The expectation of a random variable z is the weighted
value of the possible values that z can take, each value
weighted by the probability that z assumes it. For discrete
variables:
N
zi

  E[ z ]  i 1   z[ z ]
N
zS
• Analogous to “center of gravity”. First moment.
-1
0
1
p(-1)=0.10 p(0)=0.25 p(1)=0.3
2
p(2)=0.35
Probability Mass Functions:
Second Central Moment
• The variance of a random variable reflects the spread of Z
values around the expected value. For discrete variables:
  Var[ z ]  E[( z  E ( z )) ] 
2
2
  ( z   ) [ z]
2
zS
• Second central moment of a distribution.
Continuous distributions

  E ( z )   z[ z ]dz


  E (( z   ) )   ( z   ) [ z ]dz
2
2

2
Cumulative distribution and
quantile function
F ( z )   [u ]
u z
z
F ( z )   [u ]du

Z takes on a value less than or equal to u
F 1 ( p )
1
F ( p)  z
Probability Distributions & Stochasticity
• To build stochastic models for ecological
data, we need a toolbox of [z]s. This toolbox
contains probability functions and probability
density functions that describe the way in
which different types of data arise.
• The [z]’s link our deterministic model with
the data in a way that reveals uncertainty.
A toolbox of [z]’s
1.
Discrete
 Bernoulli: binary outcome of one trial
 Binomial: Outcome of multiple trials.
 Poisson: Counts.
 Negative binomial: Overdispersed counts.
 Multinomial: Multiple categorical outcomes.
2. Continuous
 Normal.
 Lognormal.
 Exponential
 Gamma
 Beta
 Others (Uniform, Dirilecht, Wishart).
Binomial distribution: Number of successes in n
trials (Discrete events can take one of two values)
zi ~ Binomial (n, p )
E[z] = np
Variance =np(1-p)
Probability
n z
P( zi | n, p )    p (1  p ) n  z
z
0.2
n
0.18
n!
  
0.16
z
z
!
(
n

z
)!
 
n =20
p = 0.5
0.14
0.12
0.1
0.08
0.06
0.04
n = number of trials
p = prob. of success
0.02
0
0
1
2
3
4
5
6
7
8
9 10 11 12 13 14 15 16 17 18 19 20
Event (xz
Example: Probability of survival derived from population data
Binomial distribution: Number of successes in n
trials (Discrete events can take one of two values)
zi ~ Binomial (n, p )
n z
P( zi | n, p )    p (1  p ) n  z
z
n
n!
  
 z  z!(n  z )!
E[x] = np
Variance =np(1-p)
n = number of trials
p = prob. of success
0≤p≤1
What are the data?
What parameters do we
want to estimate?
What terms control the
uncertainty?
Binomial distribution
Bernoulli=Binomial with n=1 trial
zi ~ Bernoulli ( p )
P( z  1)  p
P( z  0)  1  p
Mean  p
Var  p (1  p )
Multinomial distribution: Number of successes in n
trials with > 2 possible outcomes for “success”
zi ~ Multinomial ( p, n)
 ( z )!
| p, n) 
p

z!
i
P ( zi
zi
i
i
i
i
i
Mean of category i  npi
Variance of category i  npi (1  pi )
Poisson Distribution: Counts
(or getting hit in the head by a horse)
e   zi
P( zi   )  Poisson( z |  ) 
zi !
λ=average number of units
per sampling effort
**Alt param= λ=rt
500
0.5
400
0.4
300
0.3
200
0.2
100
0.1
0
0
1
2
3
4
5
6
NumberPOISSON
of Seedlings/quadrat
0.0
7
Proportion per Bar
y= number of units
per sampling effort
Count
E[ Z ]  
Variance[ Z ]  
Poisson distribution
Exercise
What are the data?

e 
P ( zi   ) 
zi !
zi
What parameters do we
want to estimate?
What terms control the
uncertainty?
Clustering in space or time (overdispersion)
Poisson process
E[z]=Variance[z]
Negative binomial?
Poisson process
E[z]<Variance[z]
Overdispersed Clumped or patchy
Negative binomial:
Table 4.2 & 4.3 in H&M Bycatch Data
E[Z]=0.279
Variance[Z]=1.56
Suggests temporal or spatial aggregation
in the data
Negative Binomial: Counts
 zi  1 r
 p (1  p ) zi  r
P( zi | p, r )  

r

1


0.2
100
90
r
E[ z ] 
p
80
r (1  p )
p2
Count
Variance 
60
0.1
50
40
30
20
10
0
0
10
20
30
NEGBIN
Number
of Seeds
40
0.0
50
Proportion per Bar
70
Negative Binomial: Counts
k
 ( k  zi )  m   m 
Pr( zi | k , m) 

1   
k  mk 
 ( k ) zi ! 
zi
0.2
100
E[ z ]  m
90
80
Count
m
k
k related to variance :
k   Poisson;
k  0 large
Variance  m 
60
0.1
50
40
30
20
10
0
0
10
20
30
NEGBIN
Number
of Seeds
40
0.0
50
Proportion per Bar
70
2
Negative binomial
Normal Distribution
f ( zi |  ,  ) 
( zi   ) 2
exp(
)
2
2
2
2
1
E[x] = μ
Variance = σ2
Mean  
Variance   2
Normal PDF with mean = 0
1
Var = 0.25
Var = 0.5
Var = 1
Var = 2
Var = 5
Var = 10
Prob(x)
0.8
0.6
0.4
0.2
0
-5 -4 -3 -2 -1
0
y
1
2
3
4
5
Normal Distribution
How do the data arise?
Often, the process we model represents the sum of continuous variables
(ex., growth increment in trees increases on soil C content).
The fact that sums of things tend to be normally distributed is what
stands behind the central limit theorem, which shows that the mean of
random variables will be normally distributed even when the variables
themselves follow other distributions.
With large sample sizes, many other distributions (Poisson, Gamma
etc) can be approximated by a normal
Normal Distribution
f ( zi |  ,  ) 
( zi   ) 2
exp(
)
2
2
2
2
1
What are the data?
What parameters do we
want to estimate?
What terms control the
uncertainty?
A JAGS aside
• Note that in JAGS, the normal distribution is
parameterized in terms of precision, τ (tau).
• Precision is the inverse of variance, so a when
we use a small precision as a prior, it’s
equivalent to a large variance (i.e., the
uncertainty for that parameter is large).
Lognormal: One tail and no negative values
 ln( zi )   2 
1

f ( zi |  ,  ) 
exp 
2

2

2 2 x


1
E( z)  e
 
2
2   2
0.8
2
 e
(e  1)
  mean(ln( z ))
  StdDev(ln( z ))
2
0.7
0.6
0.5
y is always positive
0.4
0.3
0.2
0.1
x  mean distribution on log scale
s  Std Dev distribution on log scale
0
x0
10
20
30
40
50
60
Lognormal: Radial growth data
150
40
Hemlock
0
0
1
2
3
HEMLOCK
Growth (cm/yr)
0.0
4
Count
Count
0.1
50
30
20
0.1
10
0
0
1
2
REDCEDAR
Proportion per Bar
100
Red cedar
Proportion per Bar
0.2
0.2
0.0
3
Growth (cm/yr)
(Data from Date Creek, British Columbia)
Exponential
f ( zi |  )  e
80
70
1
60
1

2
Count

Variance 
0.4
0.3
50
40
0.2
30
20
0.1
10
0
0
1
2
3
4
Variable
5
0.0
6
Proportion per Bar
E[ zi ] 
 z i
Exponential: Growth data (negatives assumed 0)
1200
0.7
Beilschemedia pendula
0.6
800
0.5
600
0.4
0.3
400
0.2
200
0
0
Proportion per Bar
Count
1000
0.1
1
2
3 4 5 6 7
Growth (mm/yr)
0.0
8
(Data from BCI, Panama)
Gamma: One tail and flexibility
1
a 1
f ( zi | a, b)  b
zi e bzi
(a )
a
(n)  (n  1)!

( z )   e t t z 1dt
0
E[ z ]  a / b  shape * scale
Var[ z ]  a / b 2  shape * scale 2
a  shape parameter
b  rate parameter
scale  1/rate
Gamma: “raw” growth data
1000
200
Count
800
Alseis blackiana
150
Cordia bicolor
600
100
400
50
200
0
0
1
2
3 4 5 6
Growth (mm/yr)
7
8
9
0
0
10
20
Growth (mm/yr)
30
(Data from BCI, Panama)
Beta distribution: proportions
f ( zi | a, b) 
 ( a, b) 
E( z) 
a
a b
Var ( z ) 
1
 a 1
zi (1  zi ) b 1
 ( a, b)
(a )(b)
 ( a  b)
 expected value of z
ab
(a  b ) 2 (a  b  1)
for 0  zi  1; 0 otherwise.
Beta: Light interception by crown trees
600
0.6
500
Count
0.4
300
0.3
200
0.2
100
0.1
Proportion per Bar
400
0.5
0
0.0
0.0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 1.0
GLI
(Data from Luquillo, PR)
Others
• Dirilecht: multivariate version of beta
• Multivariate normal
Bolker 2007
Bolker 2007
The Method of Moments
• You can calculate the sample values of the moments of the
distributions and match them up with the theoretical
moments.
• Recall that:
E[ Z ] 
 zp( z )
z: p ( z )  0
Var[ z ]  E[ z 2 ]  ( E[ z ]) 2
• The MOM is a good way to get a first (but biased) estimate
of the parameters of a distribution.
Why is MOM important?
• We often need to account for uncertainty in our models
but we only have data on sample mean and variance.
• How can we make predictions when the mean and
variance are not the parameters of the distribution (i.e.,
the normal distribution)?
• The MOM can give us initial estimates of distribution
parameters by matching moments (shape parameters)
with sample mean and variance.
0.6
What distribution?
What are the parameters?
0.2
0.4
Frequency
400
0.0
200
0
#Hauls
600
0.8
800
Haul & capture data (ED)
0 1 2 3 4 5 6 7 8 9
#Captures
11
13
15
17
0 1 2 3 4 5 6 7 8 9
Captures
11
13
15
17
MOM: Negative binomial
E[ z ]   zp ( z )  mu
i
2
mu
Var[ Z ]  E[ Z 2 ]  ( E[ Z ]) 2  mu 
k
We can get these from the data…
MOM Gamma
Model biomass production (μ) as a function of rainfall (x).
Production cannot be negative, so you need a distribution
for data that are continuous and strictly positive. Moreover, a
plot of the data shows that the spread of the residuals
increases with increasing production, casting doubt on the
assumption that variance is constant
How do you represent uncertainty in production?
Why is normal not appropriate?
MOM: Gamma
• Match moments (mean and variance) to parameters of
gamma distribution, scale (α) and rate (β).
Mixture models
• What do you do when your data don’t fit
any known distribution?
– Add covariates
– Mixture models
• Discrete
• Continuous
Zero-inflated models
• Zero-inflated models are a common type of finite mixture
models.
• Combine a standard discrete probability distribution (e.g.
binomial, Poisson, or negative binomial), which typically
include some probability of sampling zero counts even
when some individuals are present with some additional
process that can also lead to a zero count (e.g. complete
absence of the species or trap failure).
• Extremely useful in ecology.
An example: Seed predation
y =no seeds taken out of N available
t1
t2 ( )
Pr ob Pr edVisit  V
Assume each seed has equal probability (p) of being taken.
Then:
prob( y seeds taken)  p y
prob(( N  y ) seeds not taken)  (1  p) N  y
prob( y )  (1  V )  V (1  p) N
if y  0

 y
N!
Ny
prob( y )  V 
p
(
1

p
)

y
!
(
N

y
)!


Normalization constant
if y  0
Zero-inflated binomial
200
100
0
Frequency
300
Histogram of rzibinom(n = 1000, prob = 0.6, size = 12, zprob = 0.3)
0
2
4
6
8
rzibinom(n = 1000, prob = 0.6, size = 12, zprob = 0.3)
10
Discrete mixtures
Discrete mixture:
Zero-inflated binomial
prob( x )  ( 1  V ) prob( 0 ) if x  0
prob( x )  V * prob( x ) if x  0
0.20
Continuous mixture
Seed retention times
Regurgitation
Probability
0.10
0.15
Turdus albicollis
0.00
0.05
Defecation
0
3
6
9 12
16
20
24
Retentiontime
times(sec)
Retention
(min)
28
32
36
Uriarte et al. 2011
Many other distributions…..
Zimmerman et al. 2007
Some intuition for likelihood and
stochasticity
0.14
0.12
0.10
0.08
P(z|θ)
0.06
0.04
0.02
0.000
-10
-5
0
5
10
Observations of z
What is the probability of obtaining the observations (z) conditional
on the value of θ?
Some intuition for likelihood and
stochasticity
i  g ( )
0.14
0.12
0.10
“Unlikely” values
for θ
0.08
P(z|θ)
0.06
0.04
0.02
0.000
-10
-5
0
5
Observations of z
10
zi ~ f (  i ,  )
Some intuition for likelihood and
stochasticity
i  g ( )
0.14
0.12
0.10
The most “likely”
Values for θ
0.08
P(z|θ)
0.06
0.04
0.02
0.000
-10
-5
0
5
Observations of z
10
zi ~ f (  i ,  )