Survey
* Your assessment is very important for improving the workof artificial intelligence, which forms the content of this project
* Your assessment is very important for improving the workof artificial intelligence, which forms the content of this project
Rest of Course Proposals & Research Design Measurement Sampling Survey methods Basic Statistics for Survey Analysis Experiments Other Approaches- Observation, qualitative methods Topics for Today 1. Evaluation Research Proposal Format 2. Introduction to Research Design Primary & Secondary Data Qualitative & Quantitative Approaches Survey & Experiments On-Site & Household designs 3. Common Problems/Approaches in PRTR Purposes of Proposal • Communicate with Client • Demonstrate your grasp of problem • Plan the study in advance, so others can evaluate the study approach – – – – will it work? have you overlooked something? will results be useful to client? Can we afford it? Proposal Format 1. Problem Statement - define program to be evaluated/problem to be studied, users & uses of results. Justify importance of the problem/study. 2. Objectives : Concise listing . In evaluation studies, the objectives usually focus on the key elements of program to be evaluated & the evaluation criteria. These are the study objectives NOT the program objectives. 3. Background/Literature Review - place for more extensive history/structure of program. Focus on aspects most relevant to proposed evaluation. Discuss previous studies or the relevant methods. 4. Methods - details on procedures for achieving objectives - data gathering and analysis, population, sampling, measures, etc. Who will do what to whom, when, where, how and why? 5. Attachments - budget, timeline, measurement instruments, etc. NOTE: Most “programs” must be narrowed to specific components to be evaluated. Think of a “Program of studies” rather than a single evaluation study. The proposal should define this specific study & how it fits into a broader program of studies. Sample Objectives 1. Estimate benefits and costs of program 2. Estimate economic impacts of program on local community (social, environmental, fiscal). 3. Determine effects of program on target population. 4. Describe users and non-users of program 5. Assess community recreation needs, preferences 6. Determine market/financial feasibility of program 7. Evaluate adequacy or performance of program Typical Research Objectives Describe a sample or population Average income of MI Snowmobilers in 1998 is $45K Identify/test relationships between variables in a population: statistical cause-effect Quantify the relationship SB with higher incomes spend more money After safety program, SB’s have fewer accidents SB spend per day = $25 + .4 * Income Variable Terminology Variables : any characteristic that varies across individuals in a population (i.e. takes on different values for different individuals). Dependent variable is the one you are trying to predict or explain, usually the focus of your study Independent variables are the ones that help explain the dependent variable. In Program evaluation, the outcomes are generally the dependent variables and characteristics of program or target populations are independent. In a cause effect relationship, cause is independent & effect dependent. Methods Choices • Overall Approach/Design – Qualitative or Quantitative – Primary or secondary data – Survey, experiment, case study, etc. • Who to study - population, sample – individuals, market segments, populations • What to study - concepts, measures – behavior, knowledge, attitudes • Cost vs Benefit of Study Primary or Secondary Data • Secondary data are data that were collected for some purpose other than your study, e.g. government records, internal documents, previous surveys • Choice between Primary /Secondary Data – Costs (time, money, personnel) – Relevance, accuracy, adequacy of data Qualitative vs Quantitative Approaches Qualitative Focus Group In-Depth Interview Case Study Participant observation Secondary data analysis Quantitative Surveys Experiments Structured observation Secondary data analysis Qualitative vs Quantitative Quantitative Gen’l Laws Qualitative Unique/Individual case Test Hypotheses Predict behavior Understanding Meanings/Intentions Perspective Outsider-Objective Insider-Subjective Procedures Structured formal measures Unstructured open ended measures Purpose probability samples statistical analysis judgement samples interpretation of data Survey vs Experiment Survey - measure things as they are, snapshot of population at one point in time, generally refers to questionnaires (telephone, self-administered, personal interview) Experiment - manipulate at least one variable (treatment) to evaluate response, to study cause-effect relationships (field and lab experiments) Research Designs/Data Collection Approaches How ....Where Gathered Household On-Site Laboratory Personal Interview Surveys Surveys, Field Expmts Focus Groups Telephone/ Computer Self-Admin. Quest. Surveys Computer Interviews Surveys, Field Expmts Computer Interviews Experiments Observable Observable Characteristics Characteristics Observation & Traces NA Secondary Sources NA Internal Records NA General Guidelines on when to use different approaches 1. Describing a population - surveys 2. Describing users/visitors - on-site survey 3. Describing non-users, potential users or general population - household survey 4. Describing observable characteristics of visitors - on-site observation 5. Measuring impacts, cause-effect relationships experiments Guidelines (cont) 6. Anytime suitable secondary data exists secondary data 7. Short, simple household studies - phone 8. Captive audience or very interested population - self-administered survey 9. Testing new ideas - experimentation or focus groups 10. In-depth study - in-depth personal interviews, focus groups, case studies