Download Power and Multiple Regression

Power and Multiple
Regression
1
Relationship between Power and
hypothesis testing
Accept Null
Hypothesis
Null Hypothesis is true Correct decision
Null Hypothesis is
False
Reject Null
Hypothesis
Type I error( alpha
typically set to 5%)
Type II error (aka Beta) Correct decision:
Probability of making
this decision correctly
is defined as Power
2
Why is Power important?

Insure sample is large enough to detect effect of interest
3
Requirements to estimate Power

Alpha

Effect size of interest

Sample size
4
Requirements to estimate Power
 Effect
size of interest
 Determined

by theory or intuition
Are men heavier than women? What is an
“important” difference?
 Two kilograms?
 Twenty kilograms?

5
Requirements to estimate Power
 Alpha

Risk of committing Type I error (rejecting
null hypothesis when it is true) vs Type II
error (accepting null hypothesis when false)

6
Power in Stata

Powerreg command: gives power estimates for changes in R2

Example: Predicting student’s weight based on height

Requirements:

Alpha: We will use conventional .05

Effect size: What impact do we expect the use height to have on R2?


This is based on prior research, experience, intuition or theory

Let’s use .20 or a 20% increase in r-square
Sample size: 22 students in class
7
Power in Stata
. powerreg, r2f(.05) r2r(.0) nvar(1) ntest(1) n(22)
Linear regression power analysis
alpha=.05 nvar=1 ntest=1
R2-full=.05 R2-reduced=0 R2-change=.05
n = 22
power = 0.1763
r2f R-squared for the full model (required).
r2r R-squared for the reduced model (required).
nvar total number of predictor variables (default = 1).
ntest number of predictors being tested (default = 1).
n total number of observations (optional).
8
Power in Stata

When choosing effect size of interest you are choosing the change in Rsquare of interest

In most instances you will not have strong a priori reasons for a specific
effect size or change in R-square

For the assignment 5% is reasonable
9
Power for Logistic Regression

Use powerlog in Stata

Gives power estimates for changes in predicted probabilities

Requirements:

Alpha:

Effect size:

Sample size:
10
Power for Logistic Regression

Use powerlog in Stata

Gives power estimates for changes in predicted probabilities

Requirements:

Alpha: Conventional .05

Effect size:


Predicted probability at mean of independent variable (P1)

Predicted probability at mean + 1 SD of independent variable (P2)
Sample size: To be determined by program
11
Power for Logistic Regression

Example: Power for baseball team making playoffs based on ERA

Gives power estimates for changes in predicted probabilities

Requirements:

Alpha: Conventional .05

Effect size:


Predicted probability at mean of ERA = . 25 (P1)

Predicted probability at mean + 1 SD of independent variable = .13 (P2)
Sample size: To be determined by program
12
Obtaining predicted probabilities for
independent variables

If you have the data

Use summarize command to obtain standard deviation

Use margins command to produce predicted probabilities


E.g. margins, at independent_variable=(1, 2,3…))
If you don’t have data

Use published research

Make educated guess
Power for Logistic Regression
. powerlog, p1(.25) p2(.38) alpha(.05) help
Logistic regression power analysis
One-tailed test: alpha=.05 p1=.25
power
0.60
0.65
0.70
0.75
0.80
0.85
0.90
p2=.38
rsq=0
odds ratio=1.838709677419355
n
69
79
89
101
115
132
156
Explanation of terms
p1 -- the probability that the response variable equals 1
when the predictor is at the mean
p2 -- the probability that the response variable equals 1
when the predictor is one standard deviation above the mean
rsq -- the squared mulitple correlation between the predictor
variable and all other variables in the model
14
Power for Logistic Regression using .025
to assume two tailed test
. powerlog, p1(.25) p2(.38) alpha(.025) help
Logistic regression power analysis
One-tailed test: alpha=.025 p1=.25
power
0.60
0.65
0.70
0.75
0.80
0.85
0.90
p2=.38
rsq=0
odds ratio=1.838709677419355
n
95
106
117
131
147
167
193
15
Power for Logistic Regression using
different predicted probabilities
. powerlog, p1(.51) p2(.47) alpha(.025) help
Logistic regression power analysis
One-tailed test: alpha=.025 p1=.51
power
0.60
0.65
0.70
0.75
0.80
0.85
0.90
p2=.47
rsq=0
odds ratio=.8520162782093971
n
768
862
966
1086
1227
1403
1641
16
For Assignment


If using OLS regression

Estimate power for bivariate model (you can add additional variables for the rest
of assignment)

Estimate power for a r-square and an increase in r-square

Use an r-square consistent with prior research, or

Use a change of five percentage points
If using Logistic Regression

Use power analysis for bivariate model (you can add additional variables for the
rest of assignment)

Use two predicted probabilities to estimate power

Determine if sample is sufficient for desired power (e.g. .8)
17