Download EDUC5504 Class 6 agenda and notes

Class #6 Agenda and Notes
Various Details
•
Reminders:
– Class # 7 on Tuesday, April 20
– Electronic tutoring on Thursday, April 22 (as long as I can get online
at CEC; I am anticipating having no trouble
– Electronic tutoring on Tuesday April 27
– If requested, F2F tutoring on Monday, April 26 (I will have to be sure I
have a classroom for this, so let me know early)
– Final—NEW DATE!!!!!—Wednesday, April 23
•
If you can’t take it then, arrange a different date/time with
me
– Wednesday, May 5—Celebration and Feedback
•
We will have dinner and a chance for you to give written and
verbal feedback on how the program is going
Article Analysis
•
Is there anyone who has not gotten very far in the article analysis and
would still like to have a group to work with? Send me a private chat.
•
Null Hypotheses:
– Remember that the null hypothesis will simply be the negative
statement of whatever the researchers are looking for
•
There will be no difference…
•
There will be no relationship…
•
There will be no ______
Article Analysis
•
Research Hypotheses
•
The research hypothesis is the opposite of the null hypothesis.
It will state what the authors are trying to demonstrate or
discover.
– There will be a difference between…
– There will be a relationship between…
– There will be _________
•
Hint:
– If you are trying to establish reliability, you will probably
be looking for a specific strength and direction in a
correlation coefficient.
– If you are trying to establish validity, you may also be
looking for a specific strength and direction in a
correlation coefficient (unless using content validity)
Article Analysis
•
Dependent and independent variables
– The authors actually tell you the independent and dependent
variables; you just have to be able to recognize the terms that the
authors use (predictor and criterion variables); don’t over think this!!!
– Admittedly, it becomes a little confusing because the researchers
are trying to establish reliability and validity of an assessment
measure, not trying to figure out which of two treatments work
better. Just remember that:
•
An independent variable may predict or produce variation in
the dependent variable and is sometimes manipulated by
the researcher
•
A dependent variable, when statistically related to the
independent variable, “depends on” or is predicted by the
value of the independent variable.
Population and Sample
•
Sample indicates the actual participants in the study
•
Population indicates the larger group of individuals from which the sample
is drawn.
– Many times authors do not clearly state who they believe their
population to be. This is partly because they would like to be able
to apply it as widely as possible (makes it more publishable!!!)
– A good reader of research needs to try to conclude what
population the authors are trying to infer to, then decide whether
s/he thinks the authors are on target or are trying to stretch the
inference beyond what is really appropriate considering the scope
of the actual study.
– These authors seem to suggest a fairly broad population. If you
apply that population and explain why you chose that population,
that will be fine.
– Some of you may disagree with the authors, thinking they have
spread the net too far in their establishment of the wider population.
If you narrow the population and explain why you think it should be
applied to this narrower population (and make a good argument
for your case), I will accept the answer.
•
Hint about reading research: Sometimes it can be like reading literature.
Opinions can go into how much you like whether you read and whether
you think the writers have given an authentic view of the real world.
Article Analysis
•
Question #11
– I have not given you the part of the article where the authors
discuss their findings (partly because it would give you all the
answers and take away any thinking on your part!!)
– What I would like you to do in question 11 is to pretend to be the
authors and explain how the data you have looked at answers the
first two questions that they asked at the beginning of their
research. The questions were:
•
What is the alternate form reliability of student- and
administrator-read vocabulary matching measures?
– You need to look at the data and find out what the
reliability is for these two measures. Does the data
support that there is reliability for each of these two
measures? What is the strength of the data?
•
Does the alternate-form reliability differ for the two types of
measures?
– Is one more reliable than the other? What does the
evidence tell us?
Chapter Eight
•
The normal distribution is based on the study of probability.
•
Many characteristics in people and in the world are distributed according
to a normal distribution.
•
A normal distribution has three characteristics
– Mean, median, mode the same
– Perfectly symmetrical
– Asymptotic (tails of the distribution come close to but never
intersect the x-axis
Chapter 8-Normal Distribution
•
The normal distribution can actually go on infinitesimally, with surprising
findings at far ends of the distribution (e.g., geniuses and savants), but the
majority (nearly 100%) of individuals (or things) will fall between -3
standard deviations and +3 standard deviations
•
When measured over and over again (many random samples, the
distribution will fall into the same pattern:
– Between the mean and the first sd, there is about 34% of the
population
– Between the 1st and 2nd sd, there is a little less than 14% of the
population
– Between the 2nd and 3rd sd, there is about 2% of the population
•
For our purposes, rounding to whole numbers is sufficient
Chapter 8—Z scores
•
Z scores allow you to convert each raw score in a distribution into a score
that is based on how many standard deviations the score is above the
mean of the distribution.
•
Example to help you understand:
– In biology you get a 65 out 100 on your final exam. In statistics, you
get 42 out of 200. On which score did you get a “better” score?
What does “better” mean? If better means % of correct answers,
then the answer would be the biology test. Another alternative is to
determine how well you did compared to other students in the
classes. To make this comparison, you need to know the mean and
standard deviation of each distribution. With these statistics, we
can generate a z score and make a more accurate comparison.
– Suppose the mean on the biology exam was 60 with a standard
deviation of 10. That means you scores 5 points above the mean,
which is half of a standard deviation (17% or at the 67th percentile)
above the mean. Suppose further that the mean for the statistics
class was 37 with a standard deviation of 5. Again you scored 5
points above the mean, but this represents a full standard deviation
(34% or the 84 percentile) above the mean.
– Now, which test would you say you performed better on?
Chapter 8 – Z scores and Hypothesis Testing
•
In Hypothesis Testing, we typically deem a research hypothesis to be
significant, if the odds of two means actually being equal are no greater
than 1 in 20 or .05 (5%) or less. (Look at Figure 9.2 on page 169)
•
Why do we set such a high standard? Because for every mean value
obtained as part of a research project, there is a chance that the
difference found between two scores might just be because of the error
that can occur in our measurement process. If we could measure over
and over again, we could eventually come very close to what the true
score of a group of participants would actually be, but because we
can’t, we have to take into consideration that there can be error on
either side of an obtained mean. If the error for each of the means being
compared are not far enough apart, we cannot say for sure that the true
scores are not going to be the same
•
I will demonstrate.
Chapter 8--Calculate a Z score
•
What is the probability of a z-score falling above a z score of 2?
•
What is the probability of a score falling below a z score of -1.5?
•
What is the probability of a score falling between a -1.5 and 2.0?
Chapter 9—Significance
•
Significant difference (based on the statistical significance you set for your
study)
– An obtained difference in scores that can thought to be a result of
some systematic difference rather than chance
•
Significance level
– The level at which you set the amount of risk you are willing to
accept that your findings are not due to chance (or some other
reason)
•
Statistical significance
– Results when the results you obtain matches or betters your
established significance level (usually .05 in social science research)
Chapter 9 – Significance
•
How would you interpret a p-value (significance level) of p<.05?
– The probability is less than 1 in 20 of observing the obtained
outcome; or, there is a less than a 5% chance that the outcome
you are observing is due to chance (or some other reason)
•
How would you interpret a p-value (significance level) of p<.01?
– The probability is less than 1 in 100 of observing the obtained
outcome; or, there is a less than a 1% chance that the outcome
you are observing is due to chance (or some other reason)
Significance-Relationship to Hypotheses
•
A p-value of <.05 indicates that the research hypothesis can be accepted
(as long as that was the significance level that you set)
– Actual results can fall between 0.0 and .4999999…
•
A p-value of >.05 indicates that the null hypothesis should be accepted
– This is also referred to as non-significant (n.s.)
– Actual results fall between .050001 and 1.0
Significance—other terms
•
Inferential statistics—the statistical procedures employed to allow the
researcher to infer something about the population based on the sample
•
The obtained value or test statistic – the value that results from a statistical
test
•
The critical value of the test statistic –the value required for rejection (or
non-acceptance of the null hypothesis)
•
Method for decreasing the likelihood of Type II errors (accepting a false
null hypothesis
– Increase the sample size so that, hopefully, you it is more similar to
the population
Type I Error—Rejecting the null hypothesis when there really is no difference
between groups
•
Scenario:
– The average shoe size for men in the population is known to be 9. I
theorize that the average shoe size in Farmville will be larger
because of the years and years of growing big men to work in the
fields.
– I take a sample of men in the local café on a rainy day in May and
compare their average size and find that it is significantly different
(p=.0445) than the average size of 9, so I reject the null hypothesis.
•
What Type 1 error may have occurred?
Type II Error—Accepting a false null hypothesis
•
Scenario:
– I decide that I need a more random sample of men from Farmville
to test my hypothesis, so I find the phone book and choose every
100th male name in the book to send a survey. There are 1,000 male
names in the book.
– When I get my results, I find no significant difference (p=.0505)
between the mean shoe size and the population shoe size.
– What Type II error might have occurred?