Download Lecture 3: Variances and Binomial distribution

Survey
yes no Was this document useful for you?
   Thank you for your participation!

* Your assessment is very important for improving the workof artificial intelligence, which forms the content of this project

Document related concepts

Statistics wikipedia , lookup

History of statistics wikipedia , lookup

Randomness wikipedia , lookup

Probability interpretations wikipedia , lookup

Ars Conjectandi wikipedia , lookup

Probability wikipedia , lookup

Transcript
Stats for Engineers: Lecture 3
Conditional probability
Suppose there are three cards:
A red card that is red on both sides,
A white card that is white on both sides, and
A mixed card that is red on one side and white on the other.
All the cards are placed into a hat and one is pulled at random and placed on a table.
If the side facing up is red, what is the probability that the other side is also red?
44%
1.
2.
3.
4.
5.
1/6
1/3
1/2
2/3
5/6
29%
14%
11%
3%
1
2
3
4
5
Conditional probability
Suppose there are three cards:
A red card that is red on both sides,
A white card that is white on both sides, and
A mixed card that is red on one side and white on the other.
All the cards are placed into a hat and one is pulled at random and placed on a table.
If the side facing up is red, what is the probability that the other side is also red?
Probability tree
Let R=red card, TR = top red.
1
1
3
1
3
1
3
Top Red
1
3
Top White
1
3
1
2
Top Red
1
6
1
2
Top White
Red card
1
White card
Mixed card
1
6
𝑃 𝑅 ∩ 𝑇𝑅
𝑃 𝑅 𝑇𝑅 =
𝑃 𝑇𝑅
1
= 3
1 1
3+6
=
2
3
Conditional probability
Suppose there are three cards:
A red card that is red on both sides,
A white card that is white on both sides, and
A mixed card that is red on one side and white on the other.
All the cards are placed into a hat and one is pulled at random and placed on a table.
If the side facing up is red, what is the probability that the other side is also red?
Let R=red card, W = white card, M = mixed card. Let TR = top is a red face.
For a random draw P(R)=P(W)=P(M)=1/3.
Total probability rule:
𝑃 𝑇𝑅 = 𝑃 𝑇𝑅 𝑅 𝑃 𝑅 + 𝑃 𝑇𝑅 𝑀 𝑃 𝑀
1 1 1
1
=1× + × =
3 2 3
2
The probability we want is P(R|TR) since having the red card is the only way for the other
side also to be red.
This is
𝑃 𝑇𝑅 𝑅 𝑃 𝑅
𝑃 𝑅 𝑇𝑅 =
𝑃 𝑇𝑅
=
1
1×3
2
=
3
1
2
Intuition: 2/3 of the three red faces are on the red card.
Summary From Last Time
Bayes’ Theorem
𝑃 𝐡𝐴 𝑃 𝐴
𝑃 𝐴𝐡 =
𝑃 𝐡
Total Probability Rule:
𝑃 𝐡 =
e.g. from
𝑃 𝐴𝐡 =
𝑃 𝐴∩𝐡
𝑃 𝐡
𝑃 𝐡 π΄π‘˜ 𝑃(π΄π‘˜ )
π‘˜
Permutations - ways of ordering k items: k!
Ways of choosing k things from n, irrespective of ordering:
πΆπ‘˜π‘›
𝑛!
𝑛
=
=
π‘˜
π‘˜! 𝑛 βˆ’ π‘˜ !
Random Variables: Discreet and Continuous
Mean
πœ‡=𝐸 𝑓 𝑋
≑ 𝑓 𝑋
=
𝑓 π‘˜ 𝑃(𝑋 = π‘˜)
π‘˜
Means add:
π‘Žπ‘‹ + π‘π‘Œ = π‘Žπ‘‹ + π‘π‘Œ = π‘Ž 𝑋 + 𝑏 π‘Œ = π‘Žπœ‡π‘‹ + π‘πœ‡π‘Œ
Mean of a product of independent random variables
If 𝑋 and π‘Œ are independent random variables, then 𝑃 𝑋 ∩ π‘Œ = 𝑃 𝑋 𝑃(π‘Œ)
π‘‹π‘Œ =
𝑃 π‘₯ ∩ 𝑦 π‘₯𝑦 =
π‘₯
𝑦
𝑃 π‘₯ 𝑃 𝑦 π‘₯𝑦
π‘₯
=
𝑦
𝑃 π‘₯ π‘₯
π‘₯
𝑃 𝑦 𝑦
𝑦
= 𝑋 π‘Œ = πœ‡π‘‹ πœ‡π‘Œ
Note: in general this is not true if the variables are not independent
Example: If I throw two dice, what is the mean value of the product of the throws?
The mean of one throw is πœ‡ =
6
π‘˜=1 π‘˜π‘ƒ
𝑋=π‘˜
1
1
1
1
1
1
=1× +2× +3× +4× +5× +6×
6
6
6
6
6
6
1 21
= 1+2+3+4+5+6 × =
= 3.5
6
6
Two throws are independent, so 𝑋1 𝑋2 = πœ‡π‘‹1 πœ‡π‘‹2 = 3.52 = 12.25
Variance and standard deviation of a distribution
For a random variable X taking values 0, 1, 2 the mean πœ‡ is a measure of the average
value of a distribution, πœ‡ = βŒ©π‘‹βŒͺ.
The standard deviation, 𝜎 , is a measure of how spread out the distribution is
𝑃(𝑋 = π‘˜)
π‘˜
𝜎
𝜎
πœ‡
Definition of the variance (=𝜎 2 )
2
𝜎𝜎22 ≑
≑ var(𝑋)
var(𝑋) =
= 𝑋𝑋 βˆ’
βˆ’πœ‡πœ‡ 2 =
=
π‘˜ βˆ’ πœ‡ 2 𝑃(𝑋 = π‘˜)
π‘˜
πœ‡ 𝑋 = πœ‡2
Note that
π‘‹βˆ’πœ‡
2
= 𝑋 2 βˆ’ 2 𝑋 πœ‡ + πœ‡2
= 𝑋 2 βˆ’ 2 π‘‹πœ‡ + πœ‡2
= 𝑋 2 βˆ’ 2πœ‡2 + πœ‡2
= 𝑋 2 βˆ’ πœ‡2
So the variance can also be written
𝜎 2 = var 𝑋 = βŒ©π‘‹ 2 βŒͺ βˆ’ πœ‡2 =
π‘˜ 2 𝑃 𝑋 = π‘˜ βˆ’ πœ‡2
π‘˜
This equivalent form is often easier to evaluate in practice, though can be
less numerically stable (e.g. when subtracting two large numbers).
Example: what is the mean and standard deviation of the result of a dice throw?
Answer: Let 𝑋 be the random variable that is the number on the dice
The mean is πœ‡ = 3.5 as shown previously.
The variance is 𝜎 2 = 6π‘˜=1 π‘˜ 2 𝑃 𝑋 = π‘˜ βˆ’ πœ‡2
1
= (12 + 22 + 32 + 42 + 52 + 62 ) × 6 βˆ’ 3.52
=
91
6
βˆ’ 3.52 β‰ˆ 2.917
πœ‡ = 3.5
Hence the standard deviation is 𝜎 = 2.917 β‰ˆ 1.71
𝜎
𝜎
Sums of variances
For two independent (or just uncorrelated) random variables X and Y the variance of
X+Y is given by the sums of the separate variances.
Why? If 𝑋 has 𝑋 = πœ‡π‘‹ , and π‘Œ has π‘Œ = πœ‡π‘Œ , then
𝑋 + π‘Œ = 𝑋 + π‘Œ = πœ‡π‘‹ + πœ‡π‘Œ .
Hence since var 𝑍 =
var 𝑋 + π‘Œ =
𝑍 βˆ’ πœ‡π‘
2
𝑋 + π‘Œ βˆ’ πœ‡π‘‹ βˆ’ πœ‡π‘Œ
, if 𝑍 = 𝑋 + π‘Œ then
2
= 〈 𝑋 βˆ’ πœ‡π‘‹ + π‘Œ βˆ’ πœ‡π‘Œ
= 〈 𝑋 βˆ’ πœ‡π‘‹
2
+ π‘Œ βˆ’ πœ‡π‘Œ
2
2βŒͺ
+ 2 𝑋 βˆ’ πœ‡π‘‹ π‘Œ βˆ’ πœ‡π‘¦ βŒͺ
= 〈 𝑋 βˆ’ πœ‡π‘‹ 2 βŒͺ + 〈 π‘Œ βˆ’ πœ‡π‘Œ 2 βŒͺ + 2〈 𝑋 βˆ’ πœ‡π‘‹ π‘Œ βˆ’ πœ‡π‘¦ βŒͺ
If X and Y are independent (or just uncorrelated) then
𝑋 βˆ’ πœ‡π‘‹ π‘Œ βˆ’ πœ‡π‘Œ
=
𝑋 βˆ’ πœ‡π‘‹
π‘Œ βˆ’ πœ‡π‘Œ
= (πœ‡π‘‹ βˆ’ πœ‡π‘‹ )(πœ‡π‘Œ βˆ’ πœ‡π‘Œ ) = 0
Hence
var 𝑋 + π‘Œ =
𝑋 βˆ’ πœ‡π‘‹
2
+
= var 𝑋 + var π‘Œ
π‘Œ βˆ’ πœ‡π‘Œ
2
[β€œVariances add”]
In general, for both discrete and continuous independent (or uncorrelated) random variables
var 𝑋 + π‘Œ + 𝑍 + β‹― = var 𝑋 + var π‘Œ + var 𝑍 + β‹―
Example:
The mean weight of people in England is ΞΌ=72.4kg,
with standard deviation 𝜎 =15kg.
What is the mean and standard deviation of the weight of
the passengers on a plane carrying 200 people?
In reality be careful - assumption
of independence unlikely to be accurate
Answer:
The total weight 𝑀 =
Since means add πœ‡π‘€ =
200
𝑖=1 π‘šπ‘–
200
𝑖=1 βŒ©π‘šπ‘– βŒͺ
= 200 × 72.4Kg = 14480Kg
Assuming weights independent, variances also add, with 𝜎 2 = 152 Kg 2 = 225 Kg 2
200
2
πœŽπ‘€
=
225Kg 2 = 200 × 225 Kg 2 = 45000Kg 2
𝑖=1
𝜎=
45000Kg 2 β‰ˆ 212 Kg
Error bars
A bridge uses 100 concrete slabs, each
weighing (10 ± 0.1) tonnes
[i.e. the standard deviation of each is
0.1 tonnes]
What is the total weight in tonnes of
the concrete slabs?
1.
2.
3.
4.
5.
43%
1000 ± 0.01
1000 ± 0.1
1000 ± 1
1000 ± 10
1000 ± 100
30%
13%
7%
1
2
6%
3
4
5
Error bars
A bridge uses 100 concrete slabs,
each weighing (10 ± 0.1) tonnes
[i.e. the standard deviation of each is
0.1 tonnes]
What is the total weight in tonnes of
the concrete slabs?
Means add, so πœ‡π‘‘π‘œπ‘‘ = 100 × 10 = 1000 π‘‘π‘œπ‘›π‘›π‘’π‘ 
2
Variances add, with 𝜎 2 = 0.12 , so πœŽπ‘‘π‘œπ‘‘
= 100 × 0.12 = 1
Hence π‘€π‘‘π‘œπ‘‘ = 1000 ± 1 π‘‘π‘œπ‘›π‘›π‘’π‘  = 1000 ± 1 π‘‘π‘œπ‘›π‘›π‘’π‘ 
Note: Error grows with the square root of the number: ∝
But the mean of the total is ∝ 𝑁
β‡’ fractional error decreases ∝ 1/ 𝑁
𝑁
Reminder:
Discrete Random Variables
=
πΆπ‘˜π‘›
Binomial distribution
𝑛!
=
π‘˜! 𝑛 βˆ’ π‘˜ !
A process with two possible outcomes, "success" and "failure" (or yes/no, etc.) is
called a Bernoulli trial.
e.g.
coin tossing:
quality control:
Polling:
Heads or Tails
Satisfactory or Unsatisfactory
Agree or disagree
An experiment consists of n independent Bernoulli trials and p = probability
of success for each trial. Let X = total number of successes in the n trials.
Then 𝑃 𝑋 = π‘˜ =
𝑛 π‘˜
𝑝 1βˆ’π‘
π‘˜
π‘›βˆ’π‘˜
for k = 0, 1, 2, ... , n.
This is called the Binomial distribution with parameters n and p, or B(n, p) for short.
X ~ B(n, p) stands for "X has the Binomial distribution with parameters n and p."
Situations where a Binomial might occur
1) Quality control: select n items at random; X = number found to be
satisfactory.
2) Survey of n people about products A and B; X = number preferring A.
3) Telecommunications: n messages; X = number with an invalid address.
4) Number of items with some property above a threshold; e.g. X = number
with height > A
Justification
"X = k" means k successes (each with probability p) and n-k failures (each with
probability 1-p).
Suppose for the moment all the successes come first. Assuming independence
probability = 𝑝 × π‘ × π‘ … × π‘ × 1 βˆ’ 𝑝 × 1 βˆ’ 𝑝 × β‹― × (1 βˆ’ 𝑝)
π‘˜ successes: π‘π‘˜
= π‘π‘˜ 1 βˆ’ 𝑝
𝑛 βˆ’ π‘˜ failures: 1 βˆ’ 𝑝
π‘›βˆ’π‘˜
π‘›βˆ’π‘˜
Every possible different ordering also has this same probability. The total number of
𝑛
𝑛
ways of choosing k out of the n trails to be successes is
, so there are
,
π‘˜
π‘˜
possible orderings.
Since each ordering is an exclusive possibility, by the special addition rule the
𝑛
overall probability is π‘π‘˜ 1 βˆ’ 𝑝 π‘›βˆ’π‘˜ added
times:
π‘˜
𝑃 𝑋=π‘˜ =
𝑛 π‘˜
𝑝 1βˆ’π‘
π‘˜
π‘›βˆ’π‘˜
𝑝 = 0.5
π‘›βˆ’π‘˜
𝑃 𝑋=π‘˜
𝑃 𝑋=π‘˜ =
𝑛 π‘˜
𝑝 1βˆ’π‘
π‘˜
Example: If I toss a coin 100 times, what is the probability of getting exactly 50 tails?
Answer:
Let X = number tails in 100 tosses
Bernoulli trial: tail or head, 𝑋 ∼ 𝐡 𝑛, 𝑝 = 𝐡(100,0.5)
100
𝑃 𝑋 = 50 = πΆπ‘˜π‘› π‘π‘˜ (1 βˆ’ 𝑝)π‘›βˆ’π‘˜ = 𝐢50
0.550 1 βˆ’ 0.5
β‰ˆ 0.0796
50
Example: A component has a 20% chance of being a dud. If five are selected from a
large batch, what is the probability that more than one is a dud?
Answer:
Let X = number of duds in selection of 5
Bernoulli trial: dud or not dud, 𝑋 ∼ 𝐡(5,0.2)
P(More than one dud)
= 𝑃 𝑋 > 1 = 1 βˆ’ 𝑃 𝑋 ≀ 1 = 1 βˆ’ P X = 0 βˆ’ P(X = 1)
= 1 βˆ’ 𝐢05 0.20 1 βˆ’ 0.2
5
βˆ’ 𝐢15 0.21 1 βˆ’ 0.2
= 1 βˆ’ 1 × 1 × 0.85 βˆ’ 5 × 0.2 × 0.84
= 1 - 0.32768 - 0.4096 β‰ˆ 0.263.
4