Survey
* Your assessment is very important for improving the workof artificial intelligence, which forms the content of this project
* Your assessment is very important for improving the workof artificial intelligence, which forms the content of this project
Special Probability Distributions Special Probability Densities Formal Modeling in Cognitive Science Lecture 23: Special Distributions and Densities Steve Renals (notes by Frank Keller) School of Informatics University of Edinburgh [email protected] 5 March 2007 Steve Renals (notes by Frank Keller) Formal Modeling in Cognitive Science 1 Special Probability Distributions Special Probability Densities 1 Special Probability Distributions Uniform Distribution Binominal Distribution 2 Special Probability Densities Uniform Distribution Exponential Distribution Normal Distribution Steve Renals (notes by Frank Keller) Formal Modeling in Cognitive Science 2 Special Probability Distributions Special Probability Densities Uniform Distribution Binominal Distribution Uniform Distribution Definition: Uniform Distribution A random variable X has a discrete uniform distribution iff its probability distribution is given by: f (x) = 1 for x = x1 , x2 , . . . , xk k where xi 6= xj when i 6= j. The mean and variance of the uniform distribution are: µ= k X i=1 xi · 1 k Steve Renals (notes by Frank Keller) σ2 = k X i=1 (xi − µ)2 1 k Formal Modeling in Cognitive Science 3 Special Probability Distributions Special Probability Densities Uniform Distribution Binominal Distribution Binominal Distribution Often we are interested in experiments with repeated trials: assume there is a fixed number of trials; each of the trial can have two outcomes (e.g., success and failure, head and tail); the probability of success and failure is the same for each trial: θ and 1 − θ; the trials are all independent. Then the probability of getting x successes in n trials in a given n order is θx (1 − θ)n−x . And there are x different orders, so the n x n−x overall probability is x θ (1 − θ) . Steve Renals (notes by Frank Keller) Formal Modeling in Cognitive Science 4 Special Probability Distributions Special Probability Densities Uniform Distribution Binominal Distribution Binominal Distribution Definition: Binomial Distribution A random variable X has a binominal distribution iff its probability distribution is given by: n x b(x; n, θ) = θ (1 − θ)n−x for x = 0, 1, 2, . . . , n x Example The probability of getting five heads and seven tail in 12 flips of a balanced coin is: 1 12 1 5 1 b(5; 12, ) = ( ) (1 − )12−5 2 5 2 2 Steve Renals (notes by Frank Keller) Formal Modeling in Cognitive Science 5 Special Probability Distributions Special Probability Densities Uniform Distribution Binominal Distribution Binominal Distribution 1 0.9 0.8 b(x; 12, 0.5) 0.7 0.6 0.5 0.4 0.3 0.2 0.1 0 1 1 2 3 4 5 Steve Renals (notes by Frank Keller) 6 x 7 8 9 10 11 12 Formal Modeling in Cognitive Science 6 Special Probability Distributions Special Probability Densities Uniform Distribution Binominal Distribution Binominal Distribution If we invert successes and failures (or heads and tails), then the probability stays the same. Therefore: Theorem: Binomial Distribution b(x; n, θ) = b(n − x; n, 1 − θ) Two other important properties of the binomial distribution are: Theorem: Binomial Distribution The mean and the variance of the binomial distribution are: µ = nθ and σ 2 = nθ(1 − θ) Steve Renals (notes by Frank Keller) Formal Modeling in Cognitive Science 7 Special Probability Distributions Special Probability Densities Uniform Distribution Exponential Distribution Normal Distribution Uniform Distribution Definition: Uniform Distribution A random variable X has a continuous uniform distribution iff its probability density is given by: 1 β−α for α < x < β u(x; α, β) = 0 elsewhere The mean and variance of the uniform distribution are: µ= α+β 2 Steve Renals (notes by Frank Keller) σ2 = 1 (α − β)2 12 Formal Modeling in Cognitive Science 8 Uniform Distribution Exponential Distribution Normal Distribution Special Probability Distributions Special Probability Densities Uniform Distribution 0.6 0.5 0.4 0.3 0.2 0.1 0 1 1.5 2 2.5 3 3.5 4 x Uniform distribution for α = 1 and β = 4. Steve Renals (notes by Frank Keller) Formal Modeling in Cognitive Science 9 Special Probability Distributions Special Probability Densities Uniform Distribution Exponential Distribution Normal Distribution Exponential Distribution Definition: Exponential Distribution A random variable X has an exponential distribution iff its probability density is given by: 1 −x/θ for x > 0 θe g (x; θ) = 0 elsewhere The mean and variance of the exponential distribution are: µ=θ Steve Renals (notes by Frank Keller) σ2 = θ2 Formal Modeling in Cognitive Science 10 Uniform Distribution Exponential Distribution Normal Distribution Special Probability Distributions Special Probability Densities Exponential Distribution 0.35 0.3 0.25 0.2 0.15 0.1 0.05 0 1 2 3 4 5 x Exponential distribution for θ = {0.5, 1, 2, 3}. Steve Renals (notes by Frank Keller) Formal Modeling in Cognitive Science 11 Special Probability Distributions Special Probability Densities Uniform Distribution Exponential Distribution Normal Distribution Normal Distribution Definition: Normal Distribution A random variable X has a normal distribution iff its probability density is given by: 1 x−µ 2 1 n(x; µ, σ) = √ e − 2 ( σ ) for − ∞ < x < ∞ σ 2π Normally distributed random variables are ubiquitous in probability theory; many measurements of physical, biological, or cognitive processes yield normally distributed data; such data can be modeled using a normal distributions (sometimes using mixtures of several normal distributions); also called the Gaussian distribution. Steve Renals (notes by Frank Keller) Formal Modeling in Cognitive Science 12 Uniform Distribution Exponential Distribution Normal Distribution Special Probability Distributions Special Probability Densities Standard Normal Distribution 0.4 0.3 0.2 0.1 0 -4 -2 0 2 4 x Steve Renals (notes by Frank Keller) Formal Modeling in Cognitive Science 13 Special Probability Distributions Special Probability Densities Uniform Distribution Exponential Distribution Normal Distribution Normal Distribution Definition: Standard Normal Distribution The normal distribution with µ = 0 and σ = 1 is referred to as the standard normal distribution: 1 2 1 n(x; 0, 1) = √ e − 2 x 2π Theorem: Standard Normal Distribution If a random variable X has a normal distribution, then: P(|x − µ| < σ) = 0.6826 P(|x − µ| < 2σ) = 0.9544 This follows from Chebyshev’s Theorem (see previous lecture). Steve Renals (notes by Frank Keller) Formal Modeling in Cognitive Science 14 Special Probability Distributions Special Probability Densities Uniform Distribution Exponential Distribution Normal Distribution Normal Distribution Theorem: Z -Scores If a random variable X has a normal distribution with the mean µ and the standard deviation σ then: Z= X −µ σ has the standard normal distribution. This conversion is often used to make results obtained by different experiments comparable: convert the distributions to Z -scores. Steve Renals (notes by Frank Keller) Formal Modeling in Cognitive Science 15 Special Probability Distributions Special Probability Densities Uniform Distribution Exponential Distribution Normal Distribution Summary The uniform distribution assigns each value the same probability; The binomial distributions models an experiment with a fixed number of independent binary trials, each with the same probability; The normal distribution models the data generated by measurements of physical, biological, or cognitive processes; Z -scores can be used to convert a normal distribution into the standard normal distribution. Steve Renals (notes by Frank Keller) Formal Modeling in Cognitive Science 16