Download t-test

Instructor: Da-Fu Huang
Students:
M98C0209 李威德 Eden
M98C0104 陳泓銚 William
M98C0101 汪怡菁 Dara
Quasi-Experimental
Research:
Vocabulary Learning
Techniques
All life is an
experiment
Ralph Waldo Emerson, 1802-1882
Introducing experimental research

The most appropriate way to resolve a
question about language learning or
teaching is to conduct an experiment.
Introducing experimental research

The experience in this chapter involves an
experiment comparing two ways that have
been proposed to help learners study and
remember second language vocabulary.
Exercise 7.1
The vocabulary recall experiment
There are 2 lists of 25 words each.
(List A & List B) P.196
 It takes 2 minutes and then write down all
the words you can remember.


You better have timer, watch or a partner.
List A
1.Bath
2.Write
3.Bed
4.Crisp
5.Dream
6.Fresh
7.Glass
8.Shiny
9.Jar
10.Lie
11.Cup
12.Listen 13.Window
14.New
15.Pail
16.Read 17.Bright 18.Think
19.Rest
20.Bucket
21.Room 22.Sleep 23.Speak
24.Relax 25.Table
List B
1.Light
2.Lamp
3.Candle
4.Bulb
5.Lantern
6.Sit
7.Stand
8.Kneel
9.Bend
10.Jump
11.Old
12.Ancient 13.Tired
14.Weak 15.Bent
16.Tire
17.Wheel
19.Gear
18.Engine
20.Door
21.Desk 22.Student 23.Teacher 24.Pencil 25.eraser
List B (version of tip)
Exercise 7.2

This experiment deals with learning
vocabulary pairs in two languages.

You will have ten seconds to practice
saying the Spanish word and its English
translation to yourself as many times as
you can.
Test E
no. / Spanish











/Translation
1. Caracol
2. Barro
3. Toalla
4. Vientre
5. Carpa
____________________
6. Comedor
7. Tallarin
8. Corazon
9. Prado
10. Parabriasa
Score ______











11. Sabor
12. Pollo
13. Sabio
14. Herida
15. Lata
__________________
16. Jefe
17. Polvo
18. Espaldas
19. Helado
20. Viajero
Test F
no. / Spanish word / Translation











1. Tarde
2. Cama
3. Fondo
4. Ciudad
5. Payaso
________________________
6. Postre
7. Mneca
8. Huevo
9. Bombero
10. Piso
Score ______











11. Saltamontes
12. Ayuda
13. Pulgado
14. Bolsillo
15. Arroz
________________________
16. Arena
17. Sabana
18. Guerra
19. Mujer
20. Gusano
no. / Spanish











Test E
/ Translation
1. Caracol Snail
2. Barro
Mud
3. Toalla
Towel
4. Vientre Belly
5. Carpa
Tent
____________________
6. Comedor Dining room
7. Tallarin Noodle
8. Corazon Heart
9. Prado
Meadow
10. Parabriasa Windshield











11. Sabor Taste
12. Pollo
Chicken
13. Sabio Scholar
14. Herida Wound
15. Lata
Tin can
__________________
16. Jefe
Boss
17. Polvo Dust
18. Espaldas Back
19. Helado Ice cream
20. Viajero Traveler
Test F
no. / Spanish word / Translation











1. Tarde
Afternoon
2. Cama
Bed
3. Fondo
Bottom
4. Ciudad
City
5. Payaso
Clown
________________________
6. Postre
Dessert
7. Mneca
Doll
8. Huevo
Egg
9. Bombero
Fireman
10. Piso
Floor











11. Saltamontes Grasshopper
12. Ayuda
Help
13. Pulgado
Inch
14. Bolsillo
Pocket
15. Arroz
Rice
________________________
16. Arena
Sand
17. Sabana
Sheet
18. Guerra
War
19. Mujer
Woman
20. Gusano
Worm
Compiling experimental data

Compile the in any way that makes sense
to you as a group, in a format similar to
that shown in table 7.1.
Analyzing experimental data
 One
of the women wondered if
the women remember things or
learn words need make a list than
men.
Analyzing experimental data
 Some
of men objected, claiming
they were dedicated list maker
and user.
Table 7.3 Example recall data (www)
List A
M (SD)
List B
M (SD)
Women N=17
18.5(2.5)
22.5(2.0)
Men N=17
14.5(3.5)
21.5(2.5)
Computing the t-test in data
analysis

The t-test is the most frequently used measure
in second language research when comparing
mean scores for two groups.
  sum.of (or.add.up)
Computing the t-test in data
analysis







SD = standard deviation
X = values
M = mean of the values
N = number values
M= mean
X= values
N= number of values

Once you have computed these data, you will
want to know if the difference between the
two means is significant or not?

Significance is a statistical way of talking
about whether the difference between groups’
means is biggish or smallish.

When comparing pairs of means the statistical
measure typically used is the t-test.
T test (T檢定)
The t-test is the most frequently used
measure in second language research
when comparing mean scores for two
groups.
 It can be used with very large or very small
groups.

Degrees of Freedom

Group size is (roughly) adjusted
for by degree of freedom.
df. = N1+N2-2 =(N-1)+(n-1)
 Example: (17-1)+(17-1)=32

Degrees of Freedom

Adjustments for differences in types of
decision is made by considering one-tailed
and two tailed decisions separately.
T test (T檢定)



w= women
m= men
M= mean
T test (T檢定)
Women (N=17)
Mean M=18.5
Standard Deviation S=2.5
Men (N=17)
Mean M =14.5
18.5  14.5
t
Standard Deviation
S=3.5
2.5
3.5
2
w
17
→

2
m
17

t= 3.83

df= 32
How should we do if the exact df is not
shown in the table?

The value is greater than the critical value
found in the table at .01 or .05.

The result means the women’s mean is
significantly higher than the men’s mean
score.

Abstract

Gender differences on list recall of 25 words

Women had a mean word recall of 18.5 (SD=
2.5)

Men had a mean word recall of 14.5 (SD= 3.5)

A t-test analysis of the differences between
means yielded a t of 3.83

This was significant at the p< .01 level (df= 32)

Women were significantly better than men at
remember words
If the group sizes are different:
The Same Result
This formula could also be used in situations where
the group sizes are the same.
Designing Your Own
Experimental Studies
a)
Experimental approaches to language
teaching research
b)
The big method-comparison studies
c)
Creating research questions and the
accompanying hypotheses
d)
Designing experiments to test hypotheses
Experimental Approaches to
Language Teaching
What is experiment?
“a situation in which one observes the
relationship between two variables by
deliberately producing a change in one
and looking to see whether this alteration
produces a change in the other”
(Anderson 1969).
True Experimental Study
a)
Students are randomly selected and
assigned to two groups.
b)
Two experimental conditions or
treatments are provided.
c)
For two groups, a pre- and post- test are
given.

Two groups of participants are assumed to
be equivalent.

Control group- it will receive a control
treatment.

Treatment group- it will receive the
experimental treatment in the form of some
new approach.
XO
X is some kind of educational treatment
 O is an observation following the treatment
 Campbell and Stanley note: Research
today conforms to a design in which a
single group only once, subsequent to
some agent or treatment presumed to
cause change.

Limitation

Second language researchers can’t obtain
randomly selected groups of participants.

The experiment already exists intact groups.

Such variation from the formal requirements
for a true experiment causes such a study to
be called Quasi-Experimental.
The Big Method-Comparison
Studies

“The age of methods” in second language
instruction.

There are many methods shown in Table 7.7.

Many method-comparison studies were
undertaken with designs hoping to
approximate pure experimental studies.

The research methodology of the studies was
very heavily criticized.

However, these research becomes the
focusing of greater qualitative studies.
Creating Research Questions
and Hypotheses

Research Questions- Questions that you
will try to answer in the study

Research Hypotheses- claims that you
believe the experimental study might
support
 List A:
25 randomly organized
common English words.
 List
B: 25 common English words
organized into sets of five words.
 The
researcher posed a research and
hypothesis.
Research Question
 Will
List B be easier to recall than
List A?
Hypothesis 1a
Participants will better recall words in List
B than List A.
------------------------------------------------If this research is hypothesis rather than
questions, experimenters must also
anticipate competing hypotheses.

Competing hypotheses = Alternative
hypotheses
Hypothesis 1b
List B will promote less successful recall
than allowing subjects to apply their own
strategies.
------------------------------------------------1. This hypothesis is alternative hypothesis
to 1a.

Hypothesis 1c
List B no better than List A
-----------------------------------------------------1. Competing hypothesis = Null hypothesis
2. It may be that neither List A nor List B will
prove superior to the other.
3. They will be equally effective.

Setting up Experiments to Test
Hypotheses
New Experimental Research
Areas in Vocabulary Learning
Significance of Experimental Research
in Teaching Vocabulary

Research and pedagogical neglect during the
1950s and 1960s.

New methodologies came in the 1970s.
a)
Communicative Language Learning
b)
Silent Way
c)
Suggestopedia
d)
Natural Approach, etc.
Significance of Experimental Research
in Teaching Vocabulary

Krashen (1977): Commonsensical view
 Sound
and meaning are two main elements of
language.
 Words
hold sound and meaning together.
 Words
are combined by sounds.
 When
you teach how sentences convey meaning,
you teach how words go together.
Significance of Experimental Research
in Teaching Vocabulary
 Laufer
and Hulstijn (2001):
 Stimulate
theoretical thinking & Empirical
research
 L2
learning = learning of large numbers of
words
 The
association of new vocabulary words with
existing knowledge.
Summary
t

Experimental subject vs. Control subject

t-test: to compare mean scores for two groups
Mw - Mm
 N w - 1Sw2  N m - 1Sm2   1
1 

 N  N 
N

N
2
w
m
m

 w
Summary
 df
(degree of freedom): to adjust group size
(N1-1) + (N2-1) = df
 Table
7.5 (P. 207)
 Research
questions or hypotheses
Example Paper #1
 English
department students conducted
remedial teaching to non-English
department students
 Questionnaires
 Pre-test
and Post-test
 XO
study (1 group / 1 treatment)
 use
of t-test
T-Test Results of the Listening
Comprehension Tests
Groups
N
**p=0.05
SD
t
df
0.61
158
85.20 21.57
Pre-test
Post-test
Mean
80
83.21 19.50
T-Test Results of the Reading
Comprehension Tests
Groups
N
**p=0.05
t
df
57.23 16.33
Pre-test
Post-test
Mean SD
80
70.99 19.95
-4.78 158
Example Paper #2
Purpose
To investigate whether or not statistically
significant differences exist between using
the task-based approach VS the Grammar
Translation Method, both integrated with
computer-assisted instruction, to improve
Taiwanese EFL college students’ speaking
performance.
Research Question

Is there a significant difference between
integrating the task-based approach and
the Grammar Translation Method, both
with computer-assisted instruction, in
terms of improving Taiwanese EFL college
students’ speaking performance?
TBA GTM
Pretest
N=50
M=54.00
SD=16.78
N=43
M=52.09
SD=15.82
df= 91
t= 0.16
p>.05
Posttest
N= 50
M= 58.00
SD= 16.78
N= 43
M= 45.58
SD= 21.08
df= 91
t= 5.022
p<.05
The End