Survey
* Your assessment is very important for improving the workof artificial intelligence, which forms the content of this project
* Your assessment is very important for improving the workof artificial intelligence, which forms the content of this project
Probability Models Weather example: Venn diagram for all combinations of 3 binary (true/false) events. Ø Ø Ø ● ● ● Raining or not Sunny or not Hot or not Sample spaces: When a random event happens, what is the set of all possible outcomes? May be discrete or continuous. Conditioning: Suppose I observe some data. How does my probability model change? Independence: Is there any relationship between pairs of variables in my model? Would data provide knowledge? Defining a Probabilistic Model flip a coin, roll a die, receive an email, take a picture, … The Axioms of Probability event A Ø Ø Ø Ø event B Valid probabilities defined by any function mapping subsets of to [0,1] that satisfies these axioms (assumptions) The nonnegativity and additivity axioms are fundamental to our intuitive understanding of probability and uncertainty Unit normalization is just a convention, and other options could also be used (e.g., probability between 0% and 100%) The additivity axiom guarantees that the probabilities of any finite set of disjoint events are additive (induction) Conditional Probability Bayes’ Rule Independence of Two Events Computing a PMF Several Random Variables y z May compute marginal of any subset of variables, possibly conditioned on values of any other variables. Expected Values of Functions Ø Consider a non-random (deterministic) function of a random variable: Ø What is the expected value of random variable Y? Ø Correct approach #1: Ø Correct approach #2: Ø Incorrect approach: (except in special cases) Linearity of Expectation Ø Consider a linear function: Ø Ø Example: Change of units (temperature, length, mass, currency, …) Ø Ø In this special case, mean of Y is the linear function applied to E[X]: Expectation of Multiple Variables Ø The expectation or expected value of a function of two discrete variables: Ø A similar formula applies to functions of 3 or more variables Ø Expectations of sums of functions are sums of expectations: Ø Ø This is always true, whether or not X and Y are independent Specializing to linear functions, this implies that: Variance and Standard Deviation Ø Ø The variance is the expected squared deviation of a random variable from its mean (these definitions are equivalent): By definition, the standard deviation is the square root of the variance: Variance and Moments Ø The variance is the expected squared deviation of a random variable from its mean (these definitions are equivalent): Terminology: Moments of random variables first moment or mean of X second moment of X pth moment of X Continuous Random Variables CDF: cumulative distribution function PDF: probability density function Model processes or data which are encoded as real numbers: temperature, commodity price, DNA expression level, light on camera sensor, … Continuous Random Variables Ø Ø Ø For any discrete random variable, the CDF is discontinuous and piecewise constant If the CDF is monotonically increasing and continuous, have a continuous random variable: The probability that continuous random variable X lies in the interval (x1,x2] is then 1 0 Probability Density Function (PDF) Ø Ø If the CDF is differentiable, its first derivative is called the probability density function (PDF): 1 By the fundamental theorem of calculus: 0 Ø For any valid PDF: 0 Expectations of Continuous Variables Ø The expectation or expected value of a continuous random variable is: Ø The expected value of a function of a continuous random variable: Ø The variance of a continuous random variable: Ø Intuition: Create a discrete variable by quantizing X, and compute discrete expectation. As number of discrete values grows, sum approaches integral. Joint Probability Distributions Marginal Distributions Example: Uniform Distributions Independence Distributions ● Bernoulli Distribution ● Gaussian Distribution ● Exponential Distribution ● Geometric Distribution ● Uniform Distribution ● Formulas will be provided, but you should be familiar with their properties Things to Prepare ● Lecture Notes ● Homework Questions ● (Textbook Problems) ● (Recitation Material)