Download Chapter 2

Chapter 10
Standards of Validation
and Evaluation
Creswell Qualitative Inquiry 2e
10.1
Key Questions
• What are some of the qualitative
perspectives on validation?
• What are some alternative procedures
useful in establishing validation?
• How is reliability used in qualitative
research?
• What are some alternative stances on
evaluating the quality of qualitative
research?
• How do these stances differ by types of
approaches to qualitative inquiry?
Creswell Qualitative Inquiry 2e
10.2
Perspectives on Validity
• Qualitative parallel equivalents to
quantitative (LeCompte & Goetz, 1982)
– Perspective: Parallel qualitative equivalents to
counterparts in experimental and survey
research
– Terms: Internal Validity, External Validity,
Reliability, Objectivity
• New terms should be used (Lincoln & Guba,
1985)
– Perspective: Alternative terms that apply more to
naturalistic axioms
– Terms: Credibility, Transferability, Dependability,
Confirmability
Creswell Qualitative Inquiry 2e
10.3
Perspectives and Terms About
Verification (cont.)
• New terms focused on credibility (Eisner,
1991)
– Perspective: Alternative terms that focus on
reasonable standards for judging the credibility
of qualitative research
– Terms: Structural Corroboration, Consensual
Validation, Referential Adequacy
• Feminist perspective (Lather, 1993)
– Perspective: Reconceptualized: Four frames of
validity
– Terms: Iconic Validity, Paralogic Validity,
Rhizomatic Validity, Situated/Embedded
Voluptuous Validity
Creswell Qualitative Inquiry 2e
10.4
Perspectives and Terms About
Verification (cont.)
• Metaphor for (Richardson, 1990)
– Perspective: Reconceptualized: Metaphor of a
crystal
– Terms: Crystals: Grow, change, alter, reflect
externalities, refract within themselves
• Distraction (Wolcott, 1994)
– Perspective: Distraction: Neither guides nor
informs
– Terms: Understanding (better term to use than
validity)
Creswell Qualitative Inquiry 2e
10.5
Types of Validation
• Interpretivist approach (Angen, 2000)
– Ethical Validation: All research agendas must
question their underlying moral assumptions,
their political and ethical implications, and the
equitable treatment of diverse voices
• Provides practical answers to questions
• Stimulates new dialogue
– Substantive Validation: Researchers
understand their own understanding of the topic
that is derived from other sources and document
this in the study
• Self-reflection process
• Interaction
with the subject matter to co-create
interpretations
• Written accounts must resonate with audiences
Creswell Qualitative Inquiry 2e
10.6
Types of Validation (cont.)
• Synthesis of approaches (Whittemore,
Chase, & Mendele, 2001). Found four
primary criteria:
– Credibility: the results are an accurate
interpretation of the participant’s meaning
– Authenticity: different voices are heard
– Criticality: critical appraisal of all aspects of the
research
– Integrity: the investigators are self-critical
Creswell Qualitative Inquiry 2e
10.7
Validation Strategies (cont.)
• Prolonged engagement in the field
– Builds trust with the participant
– The researcher learns from the culture
– The researcher can check for misinformation
• Triangulation
–
–
–
–
Using multiple sources
Using multiple research methods
Using multiple investigators
Using multiple theories to provide
corroborating evidence
Creswell Qualitative Inquiry 2e
10.8
Validation Strategies (cont.)
• Peer Review
– Provides an external check of the research
process
– Asks questions about methods, meanings,
and interpretations
– Provides a record of notes by both peer and
researcher
• Negative Case Analysis
– Uses disconfirming evidence to get alternative
points of view
– Enables the researcher to revise the initial
hypothesis until all the cases fit
Creswell Qualitative Inquiry 2e
10.9
Validation Strategies (cont.)
• Rich, Thick Description
– Involves descriptive detail of the participants
and the setting
– Allows the reader to make decisions regarding
any transferability of findings
• External Audits
– Uses an external consultant who examines
both the product, process, and account to
assess accuracy
– Examines whether or not the findings and
interpretation are supported by evidence
– Uses an auditor not connected to the study or
the researcher
Creswell Qualitative Inquiry 2e
10.10
Validation Strategies
• Clarifying researcher bias: The researcher
comments on past experiences that have
likely shaped the interpretation and
approach to the study
• In member checking
– Solicits participants’ views on the credibility of
the findings and interpretations
– A critical technique for establishing credibility
– A focus group could be used
– The preliminary analysis of descriptions and
themes is given to participants NOT
transcripts or raw data
Creswell Qualitative Inquiry 2e
10.11
Reliability Perspectives
in Qualitative Research
• Observations will enhance reliability
through:
– Use of multiple data analysts
– Use of fieldnote conventions such as double
quotation marks to indicate verbatim quotes
• Transcript analysis can enhance reliability
through:
– Use of multiple data analysts
– Use of fieldnote conventions such as double
quotation marks to indicate verbatim quotes
Creswell Qualitative Inquiry 2e
10.12
Reliability Perspectives in
Qualitative Research (cont.)
• Textual analysis can enhance reliability
through:
– Use multiple coders and calculate intercoder
reliability
– Use agreed upon standardized categories
• Interview procedures can enhance reliability
through:
–
–
–
–
Pretest interview protocols
Train interviewers
Use of fixed-choice answers
Reliability checks of open-ended questions
Creswell Qualitative Inquiry 2e
10.13
Evaluation Criteria:
A Methods Approach
Howe & Eisenhardt 1990
• The data collection and analysis are driven
by the research questions
• Data collection and analysis techniques
are completely applied in a technical
sense
• The researcher’s assumptions are made
explicit including the researcher’s own
subjectivity
Creswell Qualitative Inquiry 2e
10.14
Evaluation Criteria:
A Methods Approach
Howe & Eisenhardt 1990 (cont.)
• The study has overall warrant including
being robust and using respected
theoretical explanations
• The study has value both in informing and
improving practice
Creswell Qualitative Inquiry 2e
10.15
Evaluation Criteria:
A Postmodern, Interpretive Approach
Lincoln 1995
• Standards set in inquiry community
(guidelines for publication)
• Positionality (“text” honest and authentic)
• Community (serves community
purposes)
• Voice (participants heard)
• Critical subjectivity (researcher
heightened self-awareness/creates social
transformation)
Creswell Qualitative Inquiry 2e
10.16
Evaluation Criteria:
A Postmodern, Interpretive Approach
From Lincoln 1995 (cont.)
• Reciprocity (between researcher and
participants)
• Sacredness of relationships (respect for
participants)
• Sharing privileges (sharing of rewards
with participants)
Creswell Qualitative Inquiry 2e
10.17
Evaluation Criteria:
General, Interpretive
Richardson & St. Pierre 2005
• Substantive contribution (significant
understanding of social life)
• Aesthetic merit (practices open up text,
artistically shaped, not boring)
• Reflexivity (adequate self-awareness, selfexposure to reader)
• Impact (affects the reader emotionally,
intellectually, moved to action)
Creswell Qualitative Inquiry 2e
10.18
Evaluation Criteria:
Methodological Approach
Creswell 2007
• Rigorous data collection (multiple forms,
extensive data)
• Consistent with philosophical
assumptions of qualitative research
(evolving design, multiple perspectives)
• Employs tradition of inquiry (e.g., Case
Study, Grounded Theory, Narrative)
• Starts with focus on central phenomenon
Creswell Qualitative Inquiry 2e
10.19
Evaluation Criteria:
Methodological Approach
Creswell 2007 (cont.)
•
•
•
•
Written persuasively
Multiple levels of analysis
Narrative engages the reader
Includes strategies to confirm accuracy
Creswell Qualitative Inquiry 2e
10.20
Evaluation Guidelines for
Narrative Research
• The researcher focuses on single
individual (or 2 - 3)
• The researcher collects stories about a
significant issue related to this individual’s
life
• The researcher develops a chronology that
connects different phrases or aspects of a
story
• The researcher tells a story that restories
the story of the study participants
Creswell Qualitative Inquiry 2e
10.21
Evaluation Guidelines for
Narrative Research (cont.)
• The researcher tells a persuasive story
told in a literary way
• The researcher reports themes that
build from the story to tell a broader
analysis
• The researcher brings reflexivity into
the study
Creswell Qualitative Inquiry 2e
10.22
Evaluation Guidelines for
Phenomenological Research
• The researcher conveys an understanding
of the philosophical tenants of
phenomenology
• The researcher has a phenomenon to
study that is concisely articulated
• The researcher uses procedures of data
analysis in phenomenology
• The researcher conveys the overall
essence of the experience of the
participants, which includes a description
of the experience and the context in which
it occurred
Creswell Qualitative Inquiry 2e
10.23
Evaluation Guidelines for
Grounded Theory Research
• The researcher studies a process, action,
or interaction as the key element of the
study
• The researcher uses a coding process
that contains the three central areas of
coding found in grounded theory, open
coding, axial coding, and selective coding
• The researcher presents a theoretical
model in the form of a visual diagram
Creswell Qualitative Inquiry 2e
10.24
Evaluation Guidelines for
Grounded Theory Research (cont.)
• The researcher develops a storyline or
propositions that connect the
categories in the theoretical model
and that present further questions to
be answered
• The researcher uses reflexivity or selfdisclosure about his or her stance in
the study
Creswell Qualitative Inquiry 2e
10.25
Evaluation Guidelines for
Ethnographic Research
• The researcher identifies a culturesharing group
• The researcher provides a detailed
description of the culture-sharing
group
• The researcher examines themes
related to the culture-sharing group
Creswell Qualitative Inquiry 2e
10.26
Evaluation Guidelines for
Ethnographic Research
• The researcher addresses field issues
that have surfaced
• The researcher explains how the
culture-sharing group works
• The researcher self-discloses and is
reflexive about his/her position
Creswell Qualitative Inquiry 2e
10.27
Evaluation Guidelines
for Case Study Research
• The researcher provides a clear
identification of the case or cases
• The researcher uses multiple data
sources
• The researcher identifies the type of
case study procedures (e.g., intrinsic,
instrumental)
• The researcher provides a clear
description of the case
Creswell Qualitative Inquiry 2e
10.28
Evaluation Guidelines
for Case Study Research
• The researcher identifies themes in
the case
• The researcher makes assertions
or generalizations from the case
analysis
• The researcher is reflexive and
self-disclosing
Creswell Qualitative Inquiry 2e
10.29
Chapter 10
Standards of Validation
and Evaluation
Creswell Qualitative Inquiry 2e
10.30