Download Basic Concepts and Approaches

Systems Engineering Program
Department of Engineering Management, Information and Systems
EMIS 7370/5370 STAT 5340 :
PROBABILITY AND STATISTICS FOR SCIENTISTS AND ENGINEERS
ProbabilityBasic Concepts and Approaches
Dr. Jerrell T. Stracener, SAE Fellow
Leadership in Engineering
1
Probability-Basic Concepts and Approaches
•Basic Terminology & Notation
•Basic Concepts
•Approaches to Probability
•Axiomatic
•Classical (A Priori)
• Frequency or Empirical (A Posteriori)
• Subjective
2
Basic Terminology
• Definition – Experiment
Any well-defined action. It is any action or process
that generates observations.
• Definition - Outcome
The result of performing an experiment
3
Basic Terminology
• Definition - Sample Space
The set of all possible outcomes of a statistical experiment is
called the sample space and is represented by S.
Remark: Each outcome in a sample space is called an
element or a member of the sample space or simply a sample
point.
4
Example
An experiment consists of tossing a fair coin three times in
sequence.
How many outcomes are in the sample space?
List all of the outcomes in the sample space.
5
Example
An biased coin (likelihood of a head is 0.75) is tossed three
times in sequence.
How many outcomes are in the sample space?
List all of the outcomes in the sample space.
6
Basic Terminology
• Definition - Event
An event is the set of outcomes of the sample space each having
a given characteristic or attribute
• Remark: An event, A, is a subset of a sample space, S, i.e.,
A  S.
7
Basic Terminology
• Definition - Types of Events
If an event is a set containing only one element or outcome
of the sample space, then it is called a simple event. A
compound event is one that can be expressed as the union
of simple events.
• Definition - Null Event
The null event or empty space is a subset of the sample space
that contains no elements. We denote the event by the symbol
.
8
Operations With Events
Certain operations with events will result in the formation of new
events. These new events will be subsets of the same sample
space as the given events.
• Definition - The intersection of two events A and B, denoted by
the symbol A  B, or by AB is the event containing all elements
that are common to A and B.
• Definition - Two events A and B are mutually exclusive if
A  B = .
• Definition - The union of two events A and B, denoted by the
symbol A  B, is the event containing all the elements that
belong to A or to B or to both.
9
Operations With Events
• Definition - The complement of an event A with respect to S
is the set of all elements of S that are not in A. We denote the
complement of A by the symbol A´.
Results that follow from the above definitions:
• A   = 0.
• A   = A.
Venn Diagram
• A  A´ = 
• A  A´ = S.
• S´ = .
• ´ = S.
A
• (A´) ´ = A.
A´
S
10
Basic Concept
For any event A in S, the probability of A occurring is
a number between 0 and 1, inclusive, i.e.,
0  P( A)  1
where
P( )  0
and
P(S)  1
where Ø is the null event
11
Probability-Basic Questions
(1) First, there is a question of what we mean when we say
that a probability is 0.82, or 0.25.
- What is probability?
(2) Then, there is the question of how to obtain numerical
values of probabilities, i.e., how do we determine that
a certain probability is 0.82, or 0.25.
- How is probability determined?
(3) Finally, there is the question of how probabilities can
be combined to obtain other probabilities.
- What are the rules of probability?
12
Approaches to Probability
• Axiomatic
• Classical (A Priori)
• Frequency or Empirical (A Posteriori)
• Subjective
13
Axiomatic Approach
Given a finite sample space S and an event A in S, we
define P(A), the probability of A, to be a value of an additive set
function P, which must satisfy the following three conditions:
AXIOM 1.
P(A)  0
for any event A in S.
AXIOM 2.
P(S) = 1
14
Axiomatic Approach
AXIOM 3.
If A1, A2 …, Ak is a finite collection of mutually
exclusive events in S, then
 k
 k
P  A    PA i 
 i 1 i  i 1
15
Probability - Classical Approach
If an experiment can result in n equally likely and mutually
exclusive ways, and if nA of these outcomes have the
characteristic A, then the probability of the occurrence of A,
denoted by P(A) is defined to be the fraction
nA
P(A) 
n
16
Frequency of Empirical Approach
If an experiment is repeated or conducted n times, and
if a particular attribute A occurred fA times, then an estimate of
the probability of the event A is defined as:
fA
P(A) 
n
fA
P( A) 
, as n  
n
^
Note that
Remark: Probability can be interpreted as relative frequency in
the long run.
17
Example
An experiment consists of tossing a fair coin three times in
sequence.
What is the probability that 2 heads will occur?
18
Example
An biased coin (likelihood of a head is 0.75) is tossed three times
in sequence.
What is the probability that 2 heads will occur?
19
Relative Frequency vs. n
1
fA
n
0
1
2
3
...
n
n = number of experiments performed
20
Probability - Subjective Approach
•Definition
The probability P(A) is a measure of the degree of belief
one holds in a specified proposition A.
Note: Under this interpretation, probability may be directly related
to the betting odds one would wager on the stated proposition.
•Odds
The relative chances for the event A and the event that A
does not occur, i.e.,
odds in favor of A
P( A)

P( A)
21