Download EVAL 6000: Foundations of Evaluation

EVAL 6000: Foundations
of Evaluation
Dr. Chris L. S. Coryn
Nick Saxton
Fall 2014
Agenda
•
•
•
•
Course overview
Introductions
Questions and discussion
Activity 1 & 2
Course description
• This course is designed to provide an
overview of the theory, method, and
practice of evaluation
– Comparative study of theory, research,
and practice perspectives
– Analysis of core concepts and
definitions, rationale and uses, the
field’s history and standards, alternative
models and approaches, and emerging
and enduring issues
Course website
• The website for this course is located at
http://www.wmich.edu/evalphd/courses/
eval-6000-foundations-of-evaluation/
• From this site you can access
–
–
–
–
The course syllabus
Required and supplementary readings
Weekly lecture notes
Other materials related to the course
Required textbooks
Learning objectives
1. A deep understanding of a wide array of
evaluation theory and practice perspectives
2. An in-depth understanding of the origins
and history of evaluation as well as its
evolution toward an independent discipline
3. A clear understanding of key evaluation
concepts/vocabulary/terminology
4. A clear understanding of the nature and
purpose of evaluation, and the distinctions
between evaluation, basic and applied
research, and related terms such as
assessment and diagnosis
Learning objectives
5. An ability to describe, distinguish among,
and critically evaluate the usefulness and
validity of selected models and approaches
to evaluation, and to identify the
conditions under which each should be
used
6. A firm grasp of the fundamental logic and
methodology of evaluation
7. A basic understanding of how to integrate
traditional methodologies with evaluationspecific methodologies
8. An evaluative and critical thinking mindset,
in general
By the end of this course you
should have a clear
understanding of evaluation’s…
history, standards, theories,
methods, & practices
Secondary learning objectives
1. Conveying constructive criticism in a
professional, balanced, and tactful
manner
2. Facilitating discussion to engage
others in dialogue about evaluation
theory, method, and practice
3. Writing clearly and concisely for both
academic and non-academic audiences
4. Giving high quality, professional oral
presentations for both academic and
non-academic audiences
Course components
•
•
•
•
•
Attendance & class participation
Critical readings papers
Application paper
Thought paper & presentation
Final examination
10%
30%
20%
20%
20%
A note on course assignments
• All course assignments are due by
5:00 PM on the date indicated in the
course schedule
• All assignments should be e-mailed
to the instructor and teaching
assistant with “EVAL 6000” in the
subject line
• NO late assignments will be
accepted, will not be graded, and will
be assigned a grade of “0” (“F”)
Schedule of topics
• Fundamentals of evaluation (two
parts)
• Evaluation approaches & models
(three parts)
• Evaluation tasks, procedures, & tools
(three parts)
• Metaevaluation & institutionalizing
evaluation
Course structure
• If necessary, a question-and-answer
session for prior week’s material
• Lecture
– At the end of each lecture is a list of
entries from the Encyclopedia of
Evaluation that you are expected to
study (many will appear in the final
examination)
• Discussion
• Activity
Who are you?
• Why are you here?
• What do you expect to learn?
• What prior experiences do you have
with evaluation? Research? Design?
Measurement? Statistics (or analysis
in general)?
Who are we?
An abbreviated life history…
that has significantly influenced my world view and
evaluation practice
Major influence on my
thinking about evaluation
“…bad is bad and
good is good and it
is the job of
evaluators to decide
which is which”
— Scriven
Other influences
Davidson
Cronbach
Hattie
Stufflebeam
Cook
Chelimsky
Shadish
Patton
Most profound influence on
my evaluation practice
Who are we?
An abbreviated life history…
that has significantly influenced my world view
and evaluation practice
Major Influence on My Thinking
About Evaluation
“A system of
morality […] based
on relative emotional
values is […] a
thoroughly vulgar
conception, which
has nothing sound in
it and nothing true”
— Socrates
Activity 1
• Draw the first image that comes to
mind when you hear the word
‘evaluation’
• In small groups share your images
and identify similar and dissimilar
themes
• Share your group’s thematic analysis
Activity 2
• Carefully review the ‘definitions’ of
evaluation enumerated by major
evaluation theorists on the following
slides
• In small groups, discuss what you
see as potential factors influencing
these theorists’ definitions of
evaluation and how their definitions
might influence evaluation practice
• Share your group’s analysis
“…the act or process
of determining the
merit, worth, or
significance of
something or the
product of that
process”
— Scriven
“…the use of
evaluation concepts
and techniques to
foster selfdetermination”
— Fetterman
“…the use of social
science research
procedures to
systematically
investigate the
effectiveness of
social intervention
programs”
— Rossi, Lipsey, &
Freeman
“…[to] describe and
assess what was
intended (goals and
objectives), what
happened that was
unintended, what was
actually implemented,
and what outcomes and
results were achieved”
— Patton
“…[to explain] how
and why programs
work, for whom, and
under what
conditions”
— Chen
Encyclopedia entries
•
•
•
•
•
•
•
•
•
•
•
•
Assessment
Accountability
Auditing
Campbell, Donald T.
Cook, Thomas D.
Criteria
Evaluand
Evaluation
Evaluation Theory
External Evaluation
Formative Evaluation
History of Evaluation
•
•
•
•
•
•
•
•
•
•
Independence
Logic of Evaluation
Objectivity
Scriven, Michael
Shadish, William R.
Standards
Summative Evaluation
Value-free Inquiry
Value Judgment
Values
“…just
because you
have a library
card doesn’t
make you
Yoda”
— Pitt