Download Probability Models - Brown Computer Science

Probability Models
Weather example:
Venn diagram for all
combinations of 3
binary (true/false) events.
Ø
Ø
Ø
●
●
●
Raining or not
Sunny or not
Hot or not
Sample spaces: When a random event happens, what is the
set of all possible outcomes? May be discrete or continuous.
Conditioning: Suppose I observe some data. How does
my probability model change?
Independence: Is there any relationship between pairs of
variables in my model? Would data provide knowledge?
Defining a Probabilistic Model
flip a coin,
roll a die,
receive an email,
take a picture, …
The Axioms of Probability
event
A
Ø
Ø
Ø
Ø
event
B
Valid probabilities defined by any function mapping subsets of
to [0,1] that satisfies these axioms (assumptions)
The nonnegativity and additivity axioms are fundamental to
our intuitive understanding of probability and uncertainty
Unit normalization is just a convention, and other options
could also be used (e.g., probability between 0% and 100%)
The additivity axiom guarantees that the probabilities of any
finite set of disjoint events are additive (induction)
Conditional Probability
Bayes’ Rule
Independence of Two Events
Computing a PMF
Several Random Variables
y
z
May compute marginal of any subset of variables,
possibly conditioned on values of any other variables.
Expected Values of Functions
Ø
Consider a non-random (deterministic) function of a random variable:
Ø
What is the expected value of random variable Y?
Ø
Correct approach #1:
Ø
Correct approach #2:
Ø
Incorrect approach:
(except in
special cases)
Linearity of Expectation
Ø
Consider a linear function:
Ø
Ø
Example: Change of units (temperature, length, mass, currency, …)
Ø
Ø
In this special case, mean of Y is the linear function applied to E[X]:
Expectation of Multiple Variables
Ø
The expectation or expected value of a function of two discrete variables:
Ø
A similar formula applies to functions of 3 or more variables
Ø
Expectations of sums of functions are sums of expectations:
Ø
Ø
This is always true, whether or not X and Y are independent
Specializing to linear functions, this implies that:
Variance and Standard Deviation
Ø
Ø
The variance is the expected squared deviation of a random variable
from its mean (these definitions are equivalent):
By definition, the standard deviation is the square root of the variance:
Variance and Moments
Ø
The variance is the expected squared deviation of a random variable
from its mean (these definitions are equivalent):
Terminology: Moments of random variables
first moment or mean of X
second moment of X
pth moment of X
Continuous Random Variables
CDF: cumulative
distribution function
PDF: probability
density function
Model processes or
data which are encoded
as real numbers:
temperature,
commodity price,
DNA expression level,
light on camera sensor,
…
Continuous Random Variables
Ø
Ø
Ø
For any discrete random variable, the CDF is
discontinuous and piecewise constant
If the CDF is monotonically increasing and
continuous, have a continuous random variable:
The probability that continuous random variable
X lies in the interval (x1,x2] is then
1
0
Probability Density Function (PDF)
Ø
Ø
If the CDF is differentiable, its first derivative is
called the probability density function (PDF):
1
By the fundamental theorem of calculus:
0
Ø
For any valid PDF:
0
Expectations of Continuous Variables
Ø
The expectation or expected value of a continuous random variable is:
Ø
The expected value of a function of a continuous random variable:
Ø
The variance of a continuous random variable:
Ø
Intuition: Create a discrete variable by quantizing X, and compute
discrete expectation. As number of discrete values grows, sum approaches
integral.
Joint Probability Distributions
Marginal Distributions
Example: Uniform Distributions
Independence
Distributions
●
Bernoulli Distribution
●
Gaussian Distribution
●
Exponential Distribution
●
Geometric Distribution
●
Uniform Distribution
●
Formulas will be provided, but you should be familiar with their
properties
Things to Prepare
●
Lecture Notes
●
Homework Questions
●
(Textbook Problems)
●
(Recitation Material)