Download probability basics

CS6825: Probability
An Introduction
Definitions


An experiment is the process of observing
a phenomenon with multiple possible
outcomes
The sample space of an experiment is all
possible outcomes
• The sample space may be discrete or
continuous

An event is a set (collection) of one or
more outcomes in the sample space
Presenting data

Pie and bar charts
Body Pixels
Background
Face Pixels
Other

Frequency diagram

Scatter diagram

Taken from
“Multidimensional
Representation of Concepts
as Cognitive Engrams in the
Human Brain “
Pie chart

A Pie Chart is useful for presenting
nominal data.
• For each category we calculate the relative
frequency of its occurrence.
• Then we take a circle and divide (slice) it
proportionally to the relative frequency and
portions of the circle are allocated for the
different groups
Body Pixels
Background
Face Pixels
Other
Example

A manager of Athletics store has to
decide, which brands to keep in the new
season. 200 runners were asked to
indicate their favorite type of running
shoe.
Type of shoe
# of runners
% of total
Nike
92
46.0
Adidas
49
24.5
Reebok
37
18.5
Asics
13
6.5
Other
9
4.5
Example: Pie chart for running
shoes
18.50%
6.50%
4.50%
Nike
Adidas
Reebok
Asics
Other
24.50%
46%
We can express this in words by saying the probability of
Nike is 46% and the probability of Reebok is 18.5%
The probability of an
event is the proportion of
times the event is
expected to occur in
repeated experiments
Probability Properties



The probability of an event, say event A, is denoted P(A).
All probabilities are between 0 and 1.
(i.e. 0 < P(A) < 1)
Sample Space – set of all possible events. In previous
example Set = {Nike, Adidas, Reebok, Asic, Other}
The sum of the probabilities of all possible outcomes
(sample space) must be 1.

NOTE: it is possible to us a scale of 100% instead of 1 but,
in statistics we use the scale of 1.
What are the Probabilities
18.50%
6.50%
4.50%
Nike
Adidas
Reebok
Asics
Other
24.50%
46%
P(Nike) = 46/100 = .46
P(Adidas) = 24.5/100 = .245
P(Reebok) = 18.5/100 = .185
P(Asics) = 6.5/100 = .065
P(Other) = 4.5/100 = .045
Assigning Probabilities




Guess based on prior knowledge
alone
Guess based on knowledge of
probability distribution (to be
discussed later)
Assume equally likely outcomes
Use relative frequencies
Guess based on prior
knowledge alone
a priori Knowledge
Event B = {It rains
Tomorrow}
Weth R. Guy says “There is a
30% chance of rain
tomorrow.”
P(B) = .30
What do to when no prior
knowledge and no training
data …..Assume equally likely
outcomes
Use Relative Frequencies
Gather training data to estimate
probabilities….Flip a coin how
many times get head versus
tails.
i.e. Take a bunch of images of the
data and see what it means to
be yellow for a banana?
Additional material….

Beyond the very beginning
Complement*
The complement of an event A,
denoted by A, is the set of
outcomes that are not in A
A means A does not occur
* Some texts use Ac to denote the complement of A
Law of Complement
P(A) = Probability of any
event except A occurring
= P(all Events) - P(A)
= Sum(all events i P(i)) – P(A)
= 1 – P(A)
Union
The union of two events A and B,
denoted by A U B, is the set of
outcomes that are in A, or B, or
both
If A U B occurs, then either A or B
or both occur
Intersection
The intersection of two events A and B,
denoted by AB, is the set of outcomes that
are in both A and B.
If AB occurs, then both A and B occur
Addition Law
P(A U B) = P(A) + P(B) - P(AB)
(The probability of the union of A
and B is the probability of A plus
the probability of B minus the
probability of the intersection of A
and B)
Mutually Exclusive
Events*
Two events are mutually
exclusive if their
intersection is empty.
Two events, A and B, are
mutually exclusive if and
only if P(AB) = 0
Addition Law for
Mutually Exclusive
Events
P(A U B) = P(A) + P(B)
Conditional Probability
The probability of event A occurring,
given that event B has occurred, is
called the conditional probability of
event A given event B, denoted
P(A|B)
Conditional Probability
P(AB)
P(A|B) = -------P(B)
or
P(AB) = P(B)P(A|B)
Independence
If
P(A|B) = P(A)
or
P(B|A) = P(B)
or
P(AB) = P(A)P(B)
then A and B are
independent.
Independence
Two events A and B are
independent if
P(A|B) = P(A)
or
P(B|A) = P(B)
or
P(AB) = P(A)P(B)
NOTE: this is an assumption sometimes researchers make about their
systems when they have no a priori knowledge to tell them differently.
They do it as it makes math simpler. BE CAREFUL, it may be a WRONG
Assumption!!!
i.e. in motion tracking – person 1 leaves means nothing about person 2
leaving. They are independent….. But, is this true in practice?