Download Evaluation

Outline
•
•
•
•
•
Definitions of Research/Evaluation
Purposes of Evaluation / Research
Science and Scientific Management
Research/Evaluation as Process
Types of Research / Evaluation
Definitions
• Evaluation = Process of judging the merit or
worth of something
• Research
– application of scientific methods to answer
questions
– controlled inquiry directed at increasing
knowledge/establishing truth
• Evaluation Research - combine the two
Research
Evaluation
Evaluation Research
Why Evaluate - the Academic list
• To assess merits of alternative programs
• To discover whether & how well objectives
are being fulfilled
• To determine the reasons for successes &
failures.
• To uncover the principles underlying a
successful program.
• To refine, revise, update or track a program
Why Evaluate- "The Real List"
1. Because we are required to
2. To make better decisions.
3. To learn from experience
4. To justify programs
5. To kill programs
Purposes of Research
•
•
•
•
Answer management questions - applied research
For sake of knowing - pure or basic research
Answer research questions - methodological
Develop/test management alternatives developmental research
• Assess worth or merit of programs - evaluation
research
Research Purposes - II
•
•
•
•
Exploratory
Descriptive
Explanatory
Predictive
Ways of Knowing
or Establishing belief or truth
•
•
•
•
Tradition
Authority
Repetition/Tenacity
Science
Science
• Body of Knowledge
• Method of Inquiry
•
•
•
•
• logical
systematic
abstract
general
parsimonious
– induction
– deduction
• self-corrective
• empirical
Scientific Management
• Application of scientific principles to
management and decision making
– systematic information gathering
– empirical, objective, self-corrective
Process -- Steps
•
•
•
•
•
•
Research
define problem
objectives/hypotheses
literature review
research methods
gather data/analysis
conclusions
•
•
•
•
•
•
Evaluation
describe program
evaluation criteria
program scoping
evaluation methods
gather data/analysis
conclusions
Types of Evaluation
by Program Stage
• formative (conceptualization/design)]
• process (implementation)
• summative (outcomes, impacts, efficiency)
Types - By Approach
• Standards
– norm-based
– criterion-referenced
• Goals and objectives
• Impacts or effects
Types- by method
•
•
•
•
•
Qualitative - quantitative
Secondary data - Primary
Survey - Experiment
Internal vs external
Who - Peers, participants, superiors,
subordinates, scientists
Evaluation Criteria
•
•
•
•
•
Effort - qnty and qlty of inputs
Performance - qnty and qlty of outputs
Adequacy - meet needs?
Efficiency - benefits/costs
Equity - distributional issues, fairness
Process Evaluation
• Identifies how and why program works
–
–
–
–
attributes
recipients
conditions
effects
•
•
•
•
single or multiple
intended or side effects
timing & duration, long/short term
cognitive, affective or behavioral
Types by program area/subarea
• Emphasis: Parks, Tourism, Interp, Program
Mgmt, Commercial Rec, ...
• Program : lands, facilities, personnel,
budget, marketing, PR, maintenance,
design, policies, plans
Types of Research
• Basic - Applied
• Exploratory, Descriptive, Explanatory,
Predictive
• In-house / out- house
• by discipline - sociological, psych, economics
• by topic - boating, wilderness, legal, tourism, ...
• by method - survey, expmt, ...
Examples of Evaluation Studies
•
•
•
•
•
•
•
•
Community needs assessment
Feasibility study, SWOT analysis
Program Audits, Market audit
Cost/benefit anaysis
Impact assessment
PPBS, MBO, GPRA, CAPRA, LAC, ...
Peer review
Importance-performance analysis