Survey
* Your assessment is very important for improving the workof artificial intelligence, which forms the content of this project
* Your assessment is very important for improving the workof artificial intelligence, which forms the content of this project
Probability distributions and likelihood Readings • Ecological detective: – Chapter 3 Probability distributions • Wikipedia (seriously!) – e.g. Beta distribution, lognormal distribution, etc. Overview (three lectures) • Probability vs. likelihood • Probability distributions: binomial, poisson, normal, lognormal, negative binomial, beta, gamma, multinomial • Likelihood profile • The concept of support • Model selection, likelihood ratio, AIC • Robustness • Contradictory data Probability Likelihood If I flip a fair coin 10 times, what is the probability of it landing heads up every time? I flipped a coin 10 times and obtained 10 heads. What is the likelihood that the coin is fair? Given the fixed parameter (p = 0.5), what is the probability of different outcomes? Given the fixed outcomes (data), what is the likelihood of different parameter values? Probabilities add up to 1. Likelihoods do not add up to 1. Hypotheses (parameter values) are compared using likelihood values (higher = better). Probability Area under curve between 5 and 10 0.12 Height of curve at x = 10 Height of curve at x = 14 0.12 0.10 Probability density Probability density Likelihood 0.08 0.06 0.04 0.02 0.00 0.10 0.08 0.06 0.04 0.02 0.00 0 5 10 15 20 25 30 Values of x What is the probability that 5 ≤ x ≤ 10 given a normal distribution with µ = 13 and σ = 4? Answer: 0.204 What is the probability that –1000 ≤ x ≤ 1000 given a normal distribution with µ = 13 and σ = 4? Answer: 1.000 0 5 10 15 20 25 30 Values of x What is the likelihood that µ = 13 and σ = 4 if you observed a value of (a) x = 10 (answer: the likelihood is 0.075) (b) x = 14 (answer: the likelihood is 0.097) Conclusion: if the observed value was 14, it is more likely that the parameters are µ = 13 and σ = 4, because 0.097 is higher than 0.075. We use the same (in this case normal) probability distribution function for both probability and likelihood! Probability density 0.12 Likelihood is the height, probability is the area 0.10 0.08 0.06 0.04 0.02 0.00 0 5 10 15 Values of x 20 25 30 Common probability distributions • Discrete: binomial, Poisson, negative binomial, multinomial • Continuous: normal, lognormal, beta, gamma, (negative binomial) • For all of these the Excel sheet gives the likelihood at each value of x, given the parameters of the distribution 7 Statistical distributions.xlsx Many distributions defined here, including Excel functions and functions defined directly in the spreadsheet SD and CV (all distributions) standard deviation (SD) variance SD coefficient of variation (CV) mean Binomial probability distribution Number of trials N k N k Pr{ Z k} p 1 p k Number of successes mean(Z ) Np Probability of success [0,1] var(Z ) Np(p 1) N N! k k!(N k)! The “factorial term” How many ways are there of selecting k objects from among N objects Example: probability of getting k = 5 heads when flipping a coin N = 10 times, if the coin is fair (p = 0.5). Note: known number of trials. 7 Statistical distributions.xlsx, sheet Binomial Poisson probability distribution Pr{ Z k} e mean(Z ) var(Z ) k k! Expected number of events Number of events Example: On average there are λ = 9.4 fatal traffic accidents in Washington State every week. What is the probability that there would be k = 0 in a week? (Note: rare event out of large number of possible events.) 7 Statistical distributions.xlsx, sheet Poisson Limitations of Poisson • Has only one parameter, which is both the mean and the variance • We often have discrete count data, but in real-life data the variance is often larger than predicted by the Poisson Thus we often use the negative binomial • Closely related to the Poisson and binomial • One extra parameter related to the variance • VERY useful Standard negative binomial Number of failures (k r 1)! r k Pr{ Z k} 1 p p (r 1)! k! Probability of a success pr Number of successes mean(Z ) 1 p Squint a lot and this looks pr var(Z ) 2 kind of like a binomial (1 p) Example: a factory makes widgets successfully with probability p. How many successful widgets have been made when r = 3 failed widgets have been made? The distribution predicts the probability of k = 0, 1, 2, … successful widgets being made. 7 Statistical distributions.xlsx, sheet Neg binomial (std) Ecological usefulness? • Great for factories, but almost no ecological problems can be thought of as successes or failures in this way • Great for factory production problems • But we want a function with parameters for – Mean – Overdispersion (increased variance = increased chance of extreme events) • Integer events are rare in nature, we want to deal with real numbers Practitioner’s negative binomial Gamma function (factorial that accepts non-integers, see later) Overdispersion parameter 1 ( k) Pr{ Z k} 1 1 ( )(k 1) 1 mean(Z ) var(Z ) 1 1 k Predicted mean 2 As θ increases, variance increases, hence “overdispersion” As θ → , var(Z) → As θ → 0, var(Z) = λ, just like a Poisson! Example: our data contain observations k, with mean λ and variance greater than λ. Find the value of overdispersion θ that best accounts for this increased variance. 7 Statistical distributions.xlsx, sheet Neg binomial (practitioner) Weird facts about the practitioner’s negative binomial • When θ → 0 this doesn’t just smell like a Poisson, and act like a Poisson, it is the Poisson • By replacing the factorials with gamma functions, the r and k can be real numbers not just integers • What on earth is a gamma function??? Gamma function Γ() For background information only: this is a generalized factorial function that accepts real numbers not just integers (z) e t t z 1dt when z is a real number 0 (z 1) z(z) one of its properties (z) (z 1)! when z is an integer Excel: does not have a gamma function but has a ln of gamma function (GAMMALN) ( 1 k) ( 1 )(k 1) ( 1 k) exp ln 1 ( ) ( k 1) exp ln ( 1 k) ln ( 1 ) ln (k 1) Multinomial probability distribution Total number observed Observed number in category k n! x1 x2 xk Pr{ X i xi } p1 p2 ...pk x1 ! x2 !...xk ! Model-predicted proportion mean( X i ) npi in category k var( X i ) npi (1 pi ) Example: fitting a model to proportions at age (or proportions at length) data. Model produces predicted proportions pi and data gives observed numbers xi in each category. Total numbers sampled = n = x1 + x2+ … + xk 7 Statistical distributions.xlsx, sheet Multinomial Probability density 0.14 0.12 Predicted values 0.10 Data (n=100) 0.08 0.06 0.04 0.02 0.00 Probability density 0 2 4 6 8 10 12 14 16 18 20 22 24 26 Values of x 0.10 0.09 0.08 0.07 0.06 0.05 0.04 0.03 0.02 0.01 0.00 Predicted values Data (n=10000) 0 2 4 6 8 10 12 14 16 18 20 22 24 26 Values of x 7 Statistical distributions.xlsx, sheet Multinomial Unrealism of multinomial (and other distributions too!) • Assumes every sampling event is completely independent • But there is much correlation in reality – Same trawl, area, time of day, day of year, gender, etc. • Real data never ever fit a multinomial this well • Later lectures will introduce the concept of “effective sample size” neff, which will be smaller than reported sample size n. Normal distribution Probability density 0.12 0.10 0.08 0.06 0.04 0.02 0.00 0 5 10 15 20 25 30 Values of x 2 x 1 f ( x) exp 2 2 2 2 mean(x) var(x) 2 7 Statistical distributions.xlsx, sheet Normal Lognormal distribution Probability density 0.12 0.10 0.08 0.06 0.04 0.02 0.00 0 5 10 15 20 25 30 Values of x 2 ln x ln 1 1 f ( x) exp 2 2 x 2 2 2 mean(x) exp 2 var(x) 2 exp( 2 ) 1 exp( 2 ) 7 Statistical distributions.xlsx, sheet Lognormal Lognormal: key notes • 0<x< • Mean(x) is not µ • If we want the mean to be µ, then replace the model parameter with: * exp( 2 ) 2 • Used widely for abundance and biomass Probability density 3.5 Beta distribution 3.0 0.5,0.5 2.5 2.0 1.5 1.0 0.5 0.0 0 0.2 0.4 0.6 0.8 1 Values of x 1.2 Probability density ( ) 1 1 f (x) x 1 x ( )( ) mean(x) 1.0 0.8 1,1 0.6 0.4 0.2 0.0 0 0.2 0.4 0.6 0.8 1 Values of x 1.4 Probability density var(x) ( )2 ( 1) ( ) 1 Note: is often written as ( )( ) B( , ) 1.2 1.0 0.8 0.6 1.3,1.3 0.4 0.2 0.0 0 0.2 0.4 0.6 0.8 1 0.8 1 Values of x Probability density 2.5 2.0 1.5 1.0 4,4 0.5 0.0 0 0.2 3.0 0.5,2 5 4 3 2 1 2.5 0.6 8 Probability density 6 0.4 Values of x 9 7 Probability density Probability density 8 2,6 2.0 1.5 1.0 0.5 7 6 5 50,50 4 3 2 1 0 0 0.2 0.4 0.6 Values of x 0.8 1 0.0 0 0 0.2 0.4 0.6 Values of x 0.8 1 0 0.2 0.4 0.6 0.8 1 Values of x 7 Statistical distributions.xlsx, sheet Beta Beta: key notes • Values confined to be 0 < x < 1 • Can mimic almost any shape within those bounds • Although bounded, can change the bounds by multiplying / dividing x values • E.g. survival parameters Probability density 0.25 Gamma distribution 1 x f (x) x e ( ) mean(x) var(x) 2 0.20 0.15 0.10 4, 1 0.05 0.00 0 2 4 6 8 10 8 10 8 10 8 10 Values of x 0.50 Probability density 0.45 0.40 0.35 4, 2 0.30 0.25 0.20 0.15 0.10 0.05 0.00 0 2 Probability density 4 6 Values of x 0.40 0.35 0.30 1.1, 0.5 0.25 0.20 0.15 0.10 0.05 0.00 0 0.30 2 0.25 Probability density Probability density 0.0007 0.20 0.15 0.10 60, 5 0.05 4 6 Values of x 0.9, 0.0001 0.0006 0.0005 0.0004 0.0003 0.0002 0.0001 0.0000 0.00 0 5 10 15 Values of x 20 25 0 2 4 6 Values of x 7 Statistical distributions.xlsx, sheet Gamma Gamma: key notes • 0≤x< • Somewhat like an exponential, lognormal, or normal • Flexibility without being bounded like the beta distribution • E.g. salmon arrival numbers plotted over time • Excel function beta.dist() assumes parameters α* = α and β* =1/β