Download Concept Mapping to Reveal Prior Knowledge

Concept Mapping to Reveal Prior Knowledge and Conceptual
Change in a Mock Summit Course on Global Climate Change
Stacy Rebich
Department of Geography, Institute for Computational Earth Systems Science,
University of California, Santa Barbara, CA 93106 [email protected]
Catherine Gautier
Department of Geography, Institute for Computational Earth Systems Science,
University of California, Santa Barbara, CA 93106 [email protected]
ABSTRACT
The complex nature of climate change science poses special challenges for educators. Learners come to the classroom with prior knowledge on the topic, which serves as
a foundation for further knowledge building, but can
also pose barriers to conceptual change. Learners have
existing mental models that may limit their perception
and processing of conflicting information and prevent
adoption of scientific conceptions. Instructional strategies that attempt conceptual change by simply provoking cognitive conflict have had limited success due to the
importance of epistemological beliefs and motivation to
the conceptual change process. The Mock Environment
Summit course uses role-playing, argumentation and
discussion to heighten epistemological awareness and
motivation and thereby facilitate conceptual change. The
pre/post-course concept map evaluation of students'
knowledge about the science of global climate change reported here shows evidence of significant learning and
conceptual change. Our study also provides useful information about gaps in knowledge and the types of misconceptions students are likely to have about this topic.
Insight gained from this assessment study can be used to
tailor the curriculum and enhance student progress towards more scientific conceptions of the problem.
CLIMATE CHANGE SCIENCE - THE
NATURE OF THE DISCIPLINE
Interdisciplinary Nature - Climate change science
embodies an effort to understand an inherently
interdisciplinary problem that has interwoven human
and natural causes. Since global climate change has
connections to society that are mediated by a complex
range of political, social, technological and economic
factors, the study of the problem in the context of each of
these fields is equally relevant and important. Only
through consideration of the forces and processes
operating in not only the natural sphere, but also in each
of these social spheres will society be able to formulate a
reasonable path for future human-environment
interactions.
Uncertainty and Ill-defined Problem Solving - Climate
change science necessitates the ability to deal with
uncertainty on several levels-not only uncertainty about
the workings of the complex physical climate system, but
also uncertainty with respect to social and cultural
processes that mediate human response to changes
within the system. The interdisciplinary and complex
nature of climate change science results in an abundance
of ill-defined problems, and finding solutions to such
problems requires skills that go beyond the relatively
constrained problems generally presented in science
textbooks (Schraw, Dunkle and Bendixen, 1995).
Ill-defined or ill-structured problems are those that 1)
begin with a lack of all information necessary to develop
a solution or even to precisely define the problem, 2)
have no single right approach for solution, 3) have
problem definitions that change as new information is
gathered, and 4) have no identifiable 'correct' solution
(Gallagher, et al., 1995).
Research suggests that problem-based learning
approaches that use ill-defined problems facilitate
learning and conceptual change and the ability to
transfer that learning to other domains (Kitchener and
King, 1981; Kuhn, 1991). In a problem-based learning
environment, instructors function as "metacognitive
coaches" (Barrows, 1988) rather than simply information
presenters or discussion leaders. In their well-known
report, Science for All Americans, Rutherford and
Ahlgren (1990) discuss the potentials of ill-defined
problem solving for enhancing not only subject matter
learning, but also the metacognitive skills that are
integral to scientific literacy: "Students should be given
problems-at levels appropriate to their maturity-that
require them to decide what evidence is relevant and to
offer their own interpretations of what the evidence
means. This puts a premium, just as science does, on
careful observations and thoughtful analysis. Students
need guidance, encouragement, and practice in
collecting, sorting and analyzing evidence, and in
building arguments based on it." Instructional
approaches that utilize ill-structured problem solving
not only result in increased learning (Gallagher et al.,
1995) and information retention (Boud and Feletti, 1991),
but also encourage epistemological understanding of the
discipline (Wilkinson and Maxwell, 1991) and enhance
motivation (Carter, 1988; Tobias, 1990). In this paper we
discuss an instructional approach that makes use of
collaborative problem-based learning to accrue these
benefits and encourage conceptual change.
LEARNING AND CONCEPTUAL CHANGE
Factors that Determine Conceptual Change - The
basis for many current conceptual change theories is an
idea that learning is a process of integration that involves
both individual and social processes and consists of
revising or fitting new information into existing mental
models (Driver et al., 1994; Mayer, 2002). Not all learning
is considered conceptual change; this term is generally
reserved for learning that results from deep processing
of knowledge (Strike and Posner, 1992). Since conceptual
change involves modifications in core knowledge and
beliefs, it is generally not easily achieved (Hynd, 1998)
and occurs as a continuous gradual process (Smith,
diSessa and Roschelle, 1993). In spite of the fact that
theorists often differ with regards to the exact nature of
the underlying cognitive processes, there is general
agreement about a group of cognitive and social factors
that seem to be vital to this type of learning. Prior
Rebich and Gautier - Concept Mapping to Reveal Prior Knowledge
355
knowledge is generally considered of high importance as
it is deemed to serve both as the foundation for
integration of new concepts and as a potential obstacle to
conceptual change (Mason, 2002; Vosniadou, 2002; Chi
and Roscoe, 2002). Metacogntive awareness is
knowledge and self-awareness of one's own learning,
and is considered a necessary but not sufficient condition
for conceptual change (Carey, 1985; Limon, 2002).
Epistemology, or knowledge and beliefs about the
nature of the discipline and the nature of knowledge and
learning, is linked to both the learner's ability to
recognize incongruous beliefs and to motivation to
change them (Leach and Lewis, 2002; Mason, 2002).
Motivation as a factor was initially ignored in many 'cold
cognition' models (Pintrich et al., 1993) but now plays a
role in many models. Motivation is intrinsically linked to
goals, and there is evidence that learners with mastery
goals who focus on learning and understanding more
readily achieve conceptual change than those with
performance goals who focus on demonstrating ability
(Linnenbrink and Pintrich, 1999). Awareness of
meta-concepts or second-order concepts refers to having
appropriate conceptions of higher-order concepts such
as evidence, cause, explanation, time, space, change
source, fact, and description (Limon, 2002) that influence
reasoning across the subject matter. If conceptual change
can only occur in the space where these factors intersect,
it is easy to appreciate that achieving conceptual change
should present quite a challenge. In the next section, we
focus on three of these factors  prior knowledge,
epistemology and motivation  which are of particular
interest in our discussion of the Mock Environment
Summit course.
Focus on Prior Knowledge - Researchers in cognitive
science have consistently found that the knowledge
learners possess is a very strong determinant in what
information they attend to, how that information is
perceived, what learners judge to be important or
relevant, and what they are able to understand and
remember (Alexander, 1996). With this in mind, a
learner's knowledge base can be thought of as a scaffold
for all of his or her future learning. Results of the
overwhelming number of studies on prior knowledge
and conceptual change illustrate the inadequacy and
inaccuracy of a conception of the learner as an 'empty
vessel' or 'blank slate' that needs to be 'filled' with
knowledge. Awareness of the critical role of prior
knowledge in the acquisition of new knowledge
naturally leads to attempts to elicit and evaluate a
learner's relevant knowledge prior to instruction. Many
studies of science learning have utilized an approach that
involves identification of prior knowledge and
application of specific instructional strategies intended
to build upon and/or modify or replace that knowledge.
The study reported here is based on a similar framework;
our primary purpose here, however, is limited to the
discussion of a strategy for eliciting and evaluating prior
knowledge and suggested ways in which this evaluation
could be used for instructional enhancement.
MISCONCEPTIONS
While prior knowledge can be seen as the foundation for
integration of new concepts, it is also commonly viewed
as an obstacle to conceptual change (Chi and Roscoe,
2002; Mason, 2002; Vosniadou, 2002). In research on
science education, much of the attention given to prior
356
knowledge has been focused on identifying and
eliminating misconceptions. (Some find the term
"misconceptions" pejorative or reserve it for a specific
type of non-scientific conception; Guzzetti et al. (1993)
discuss a number of alternate, and possibly more
appropriate, terms that have been proposed. However,
for the sake of simplicity, "misconceptions" will suffice
for the purposes of our discussion.) Misconceptions are
common features of learners' prior knowledge
throughout the sciences and have proven resistant to
instruction (Champagne, Gunstone and Klopfer, 1983;
Limon, 2001). A significant portion of the misconceptions
research has been devoted to identifying causes for cause
the persistence of misconceptions, and a variety of
factors seems to be at play. The difficulty associated with
overcoming an inadequate conception can be related to
whether it is derived from interactions with the physical
environment, the social environment or a formal
instructional environment (Guzzetti et al., 1993). Prior
knowledge may be characterized by varying levels
affective entrenchment related to social values, ideology
and identity (Limon, 2002; Pintrich et al., 1993), and
presumably higher levels of affective entrenchment
would correspond with greater difficulty in achieving
conceptual change. Revision of misconceptions may also
prove costly at the level of cognitive processing if
revision of a particular mental model will require
revision of a number of related models.
Focus on Epistemological Beliefs - Beliefs about the
nature of knowledge, knowing and learning and about
the nature of science have a strong impact on a learner's
capacity for conceptual change (Gregoire, 2003; Leach
and Lewis, 2002; Limon, 2001; Mason, 2002).
Development of epistemological beliefs is generally
viewed as a progression through stages (King and
Kitchener, 1994). Kuhn (1999) classifies these stages as
absolutist and overly-objective (characterized by beliefs
that knowledge is absolute, certain, non-problematic,
right or wrong, and does not require justification since it
originates from observations of reality or the authorities),
multiplist and overly-subjective (characterized by beliefs
that knowledge is ambiguous, idiosyncratic, and thinks
each individual has his or her own views and own
truths), evaluativist and having an appropriate balance
of objectivity and subjectivity (characterized by beliefs
that there are shared norms of inquiry and knowing, and
some positions are reasonably more justified and
sustainable than others) (see also Schommer, 1993).
Learners' epistemological beliefs are not likely to be
equally developed or applicable across a variety of
subject contexts (White, 2002), which implies a need for a
specific focus on beliefs about the nature of science.
Specifically, science learning environments should focus
on the development of understanding that "the objects of
science are not the phenomena of nature but constructs
that are advanced by the scientific community to
interpret nature" (Driver et al., 1994).
Research has shown that learners who have more
sophisticated epistemological beliefs are more likely to
accept evidence that conflicts with their prior knowledge
and achieve conceptual change (Mason, 2002; 2003).
Schraw, Dunkle and Bendixen (1995) found that less
advanced epistemological beliefs were associated with
lower performance in ill-defined problem solving, but
found no correlation between epistemological beliefs
and success at solving well-defined problems.
Interpretation of and learning about controversial issues
Journal of Geoscience Education, v. 53, n. 4, September, 2005, p. 355-365
seems to be particularly affected by levels of
epistemological belief development (Kardash and
Scholes, 1996; Sinatra et al., 2003) and only individuals
with advanced epistemological understanding seem to
be able to contemplate, reason and judge evidence,
arguments and alternative perspectives (Mason and
Boscolo, 2004). In a reciprocal fashion, including
controversial
issues
in
curricula
promotes
epistemological development (Schommer-Aikins and
Hutter, 2002).
Focus on Motivation - Pintrich, Marx and Boyle (1993)
speculate on the reasons why "cold and isolated
cognition models" are not able to explain students'
inability to activate prior knowledge, and assert that a
learner's goals, intentions, purposes, expectations, and
needs must be taken into account. They propose that
motivational
beliefs
(including
goals,
values,
self-efficacy, and control beliefs) play a major role in
conceptual change, and caution against assumption that
students will find it easy to 'act like scientists' when
motivational issues for students and scientists are likely
different and students’ motivations are constrained by
the larger context of the educational system. In
examining the goal dimension of motivation, learners are
often distinguished based on the possession of mastery
goals (focus on learning and understanding) or
performance goals (focus on good grades, approval of
peers, or viewing the classroom as a competition)
(Linnenbrink and Pintrich, 2002). Pintrich (1999)
proposes that adoption of a mastery goal orientation will
facilitate conceptual change, and cites evidence that
supports this proposition. He goes on to suggest that the
structure of the learning environment plays a major role
in determining whether students will adopt performance
or mastery goals. Constructivist approaches to teaching
and learning and problem-based learning environments
in which learners are given challenging, meaningful and
authentic tasks, especially those that involve ill-defined
problems, promote mastery goals and influence learners'
motivation and cognition (Carter, 1988; Gallagher et al.,
1995; Pintrich, Marx and Boyle, 1993). This is especially
true when students are encouraged to engage in
discussions and develop scientific arguments as they
collaborate in their problem solving efforts (Nussbaum
and Sinatra, 2003).
The Cognitive Conflict Approach to Conceptual
Change - The most popular instructional approach
intended to facilitate conceptual change focuses on
identifying and removing misconceptions and involves
the presentation of anomalous data or information that
conflicts with inadequate prior knowledge. The
cognitive conflict approach generally involves
evaluating a learner's current knowledge, presenting
contradictory information, and then re-evaluating to
identify changes in the learner's conceptions. While this
approach appears logical enough and there is evidence
that the stimulation of cognitive conflict is a necessary
condition for conceptual change, the results of numerous
studies suggest that this approach in and of itself is not
sufficient. While a handful of studies report success with
this approach (e.g., Jensen and Finely, 1995), many more
have reported their unsuccessful attempts to support
conceptual change through the presentation of
conflicting information or anomalous data to create
cognitive conflict (e.g., Champagne, Gunstone and
Klopfer, 1985; Dreyfus, Jungwirth and Eliovitch, 1990;
Guzzetti et al., 1993). The observation that cognitive
conflict in the absence of knowledge-building activity
did not result in conceptual change (Chan, Burtis and
Bereiter, 1997) suggests that multiple methods should be
used to encourage meaningful learning. Based on an
analysis of the difficulties of achieving conceptual
change through the cognitive conflict strategy, Limon
(2001) proposes that presentation of anomalous data
could be augmented with cooperative and shared
learning to promote collective discussion of ideas.
The Mock Environment Summit course discussed
here utilizes the complementary strategy of cooperative
learning through discussion and argumentation as the
basis for encouraging conceptual change. Argumentation involves "constructing a rationale for a particular
outcome, refuting opposing arguments, and weighing
competing considerations" (Nussbaum and Sinatra,
2003). As learning tools, discussion and argumentation
have proven to enhance motivation, encourage development of more sophisticated epistemological beliefs,
heighten awareness of the social nature of scientific
knowledge construction, and facilitate conceptual
change (Mason, 1998; Mason and Santi, 1998; Mortimer
and Machado, 2000; Soja and Huerta 2001).
MOCK ENVIRONMENT SUMMIT
Course Overview - The "Mock Environment Summit"
(Geography 135) upper-division undergraduate course
has been offered at The University of California Santa
Barbara for the past several years as a means to
encourage students to gain a deeper understanding of
the scientific evidence of global climate change, consider
problem-solving approaches that arise from within a
variety of disciplines, and utilize a variety of skills and
knowledge of different topics to negotiate an
'international agreement' that represents a collaborative
effort to deal with the complex problem of climate
change. (see more detailed description in Gautier and
Rebich, 2005) The design of the Mock Environment
Summit curriculum is firmly rooted in constructivist
pedagogy, which departs from the traditional paradigm
of the student as an "empty vessel" and embraces the
notion that each individual comes into a class with a
unique existing knowledge framework. The "Mock
Environment Summit" course has been designed to
create a learning environment where individuals use
specially designed tools to build upon their own
knowledge in an effort to experience enriched
understanding and conceptual change. Integration of
new knowledge with prior knowledge is accomplished
through research, role playing, and the presentation of
arguments to support the position appropriate to a
chosen role. As students apply newly-gained knowledge
to problem-solving tasks, meaningful learning is
enhanced and the likelihood of the transfer of this
knowledge to other relevant problem-solving situations
increases (Ausubel, 1963).
The "Mock Environment Summit" is an
interdisciplinary, student-directed, inquiry-based Earth
Science course during which students attend several
lectures on the science of global change and then engage
in a series of role-playing activities for which they act as
policymaker-representatives of different countries. The
course focuses on the drafting and negotiation of an
international agreement similar to the Kyoto protocol. In
their roles as policymakers, students conduct (primarily)
web research to investigate the present and potential
Rebich and Gautier - Concept Mapping to Reveal Prior Knowledge
357
physical, social and economic impacts that global change
may have on their countries. On a regular basis, the
students present their research findings to the class and
begin to build the foundation for the position they plan to
take in the global change debate. Each country
representative also joins with representatives from other
countries to research a topic related to global climate
change (e.g., fresh water availability, technology
transfer, carbon trading, and population control) and as
they present information about these topics to the class,
they lobby to have these issues included as elements of
the final agreement. In addition to feedback and
coaching throughout the in-class discussion and
argumentation, the instructor and teaching assistant
provide individualized written feedback to the students
to help them improve their research and related writing
and presentation assignments. Grades for the course are
based on participation in class discussions,
presentations, writing assignments and contributions to
the final negotiated agreement.
CONCEPT MAPPING STUDY
Overview and Rationale - We recognized concept
mapping as a potentially useful strategy for assessing the
conceptual change experienced by students enrolled in
the Mock Environment Summit class. Concept mapping
presents itself as a particularly adapted assessment tool
for this class since it allows an exploration of student
knowledge at a sufficient level of complexity, does not
presuppose that all students have mastered exactly the
same material, and has also been shown to create a more
equitable assessment situation for those who have
difficulty coping with test anxiety (Okebukola and
Jegede, 1989). The usefulness of concept mapping for
assessment is partially due to its level of complexity,
which distinguishes it from more conventional
evaluation techniques such as multiple-choice tests.
Markham, Mintzes and Jones (1994) suggest that these
traditional unidimensional assessment measures
represent a failure to recognize that much disciplinary
knowledge is based on an understanding of
relationships among concepts. Other researchers have
found concept map-based evaluations to yield equally
comprehensive and accurate overviews of knowledge as
compared to well-planned structured personal
interviews (Edwards and Fraser, 1983) and assessment
through writing (Osmundson, et al., 1999). However,
concept mapping allows for more efficient data
collection than interviews do, and presents an advantage
over writing-based assessments in that it is inherently
non-linear and facilitates self-monitoring. Students faced
with an essay-writing assessment task will often
complete it in a linear fashion, starting at the beginning
and writing straight through. On the other hand,
students constructing concept maps can easily put down
concepts and/or visual symbols and add details and
connections between concepts in any order, which offers
a chance for metacognitive reflection. Concept maps may
be useful in revealing thought processes that generally
remain private to the learner (Cohen, 1987), and it has
been suggested that they may be more sensitive to
developmental changes than traditional testing in which
questions often focus on isolated ideas (Kinchin, Hay
and Adams, 2000).
Before going into more detail about the specifics of
the proposed research, a brief overview of the nature of
concept maps and how they can be used for assessment
in the classroom may be useful. The basic element of a
concept map is a proposition (see Figure 1). A
proposition is a pair of concepts whose relationship is
specified in the form of a link. Concepts are things
usually referred to by nouns or noun phrases, while links
are usually verbs. Concepts from a map on global climate
Course Evaluation and Learning Assessment - Since
we are aware of the role of prior knowledge in the
conceptual change process, we also appreciate the value
of assessing learners' existing conceptions of topics
central to a course before instruction begins. Information
about learners' initial beliefs and conceptions can then be
used to guide the design and refinement of the learning
environment. With this transition from more traditional
teaching methods, there comes a set of challenges for
assessing learning. When applied to constructivist
learning environments, the assessment methods that
have been considered appropriate under traditional
pedagogy seem increasingly inappropriate. In response
to this need for new evaluation tools, new methods that
involve portfolio and project-based assessment have
been proposed and utilized successfully (Mintzes,
Wandersee and Novak, 2001). The current assessment
methods utilized in Geography 135 are of this sort, and
students are evaluated on the basis of performance on
presentation, writing and negotiation activities. No tests
are given as part of the course, and the final means for
evaluation is the quality of the final agreement
negotiated by the class.
While these evaluation methods have proven very
useful for evaluating individual student performance, it
is relatively more difficult to extract from them detailed
information about how well the course is achieving its
stated goals and intended learning outcomes in terms of
content knowledge. In particular, we have been
interested in obtaining a measure of how much scientific
knowledge is being gained during the course as the
students listen to and participate in discussions of the
various positions presented and the evidence that
supports these positions. While the course is focused on
achieving an interdisciplinary perspective on the climate
change issue rather than on acquisition of specific
climate science content, we are interested in evaluating
students' level of conceptual change for the physical
science content. Throughout the history of the course,
carefully designed multiple-choice tests have been
administered prior to and immediately following the
course. The insight into learning gained from these
measures, however, has seemed to be both incomplete
and lacking in the complexity necessary for a more
Figure 1. The basic element of a concept map is a
in-depth analysis.
proposition, consisting of two concepts connected by
a link that shows the relationship between them.
358
Journal of Geoscience Education, v. 53, n. 4, September, 2005, p. 355-365
change might include things like "aerosol emissions",
"industrial activities", "longwave radiation trapping",
and "greenhouse effect." These individual concepts
could then be linked to form propositions such as
"aerosol emissions [are generated by] industrial
activities," or "longwave radiation trapping [is associated
with] greenhouse effect." Basically, a concept map is a
(sometimes hierarchically) structured network of
propositions.
Cognitive theories that emphasize the structure of
knowledge underlie instructional approaches and
assessments that involve concept mapping. Anderson
(1984) asserts that structure is the essence of knowledge,
and the process of constructing a concept map focuses
the learner's attention on the structure of knowledge and
the importance of knowledge integration. Concept maps
can be used to elucidate a learner's knowledge
representation and organization of ideas - characteristics
of understanding are also related to a learner's ability to
engage in higher-order thinking (Gobbo and Chi, 1989;
Jonassen et al., 1997; Kinchin, 2000).
The objectives of the study reported here were to 1)
elicit information about what students had learned in the
class at a level of knowledge complexity that had not
been observable through the multiple-choice tests used
for previous course assessments, and 2) to gain an
understanding of both the types of learning occurring
and the specific content knowledge gained by the
students in the Mock Environmental Summit class. This
enhanced evaluation of learning outcomes provides
valuable information that can be used to tailor and refine
the curriculum; at the same time, it helps to verify the
usefulness and suitability of concept map assessment for
evaluating the learning taking place in open-ended,
student-directed science courses.
Research Design and Data Collection - The participants in the study were 17 undergraduates (ages 19 to 25;
7 female, 10 male) who were enrolled in an upper division undergraduate geography course titled "Mock Environment Summit" (Geography 135). Geography 135 was
offered as an intensive course (4 class hours per day) and
met for a three-week period during the 2003 Summer
quarter. The majority of the students in the class were
Geography and Environmental Studies majors, but sev eral were from the English, Philosophy and Global
Studies departments. In this paper we examine a set of
pre- and post-course concept maps on the topic of global
climate change that were constructed by the participants.
Prior to the construction of the pre-course concept
map, students were given a one-hour training session on
concept map construction. During the training session,
the researcher presented a brief introduction to concept
mapping, and provided some background information
on how concept mapping can be useful for learning, described the purpose of the concept maps the students
would be constructing. A hands-on demonstration/tutorial gave students the opportunity to practice using the
CmapTools™ software (http://cmap.ihmc.us/) they
would be using to construct their concept maps. Students
were also given time to browse a collection of concept
maps on topics unrelated to climate change, and these
maps, as well as the others used during the training session, were available for browsing throughout the series
of concept mapping tasks. Following the presentation
and demonstration, students were asked to brainstorm
in groups on the topic of 'collaborative work.' (This topic
was chosen because it also offered a forum for discussion
of the benefits and challenges of the group work that they
would be doing throughout the course.) When they had
finished brainstorming, each student constructed a concept map that reflected his or her group's thoughts about
collaborative work. As students constructed maps, the
researcher monitored their progress and provided feedback on the construction process. Aside from this initial
training session, the students received no additional
training or practice in concept map construction during
the course.
The day following the training session, each student
constructed a concept map on the topic of global climate
change. The following focus questions were given to
direct their map construction:
•
•
•
•
•
What is global climate change?
What is the evidence?
What are the causes?
What are the mechanisms?
What are the consequences?
Students were given 45 minutes to complete the
concept-mapping activity.
The concept-mapping activity completed on the
second day of the course was repeated at the end of the
course. Students were given equal amounts of time to
construct their pre-course and post-course concept
maps.
ANALYSIS AND RESULTS
The pre- and post-course student maps were analyzed
for the presence or absence of key concepts and for
proposition accuracy and usefulness. In addition to the
calculation and evaluation of summary statistics, our
analysis used a visualization method for examining the
structural nature of the concept maps, their levels of
interconnectedness, and the content areas where most
learning occurred. Although we were not interested in
'grading' the concept maps, we based our evaluation on a
combination of scoring methods that have proven
effective in other studies (Kinchin, 2000; McClure, Sonak
and Suen, 1999; Nicoll, Francisco and Nakhleh, 2001;
Novak and Gowin, 1984). The specifics of our evaluation
procedure are described below.
Before beginning the concept map evaluation, we
reviewed the student maps for syntax, and propositions
that were not in a suitable form for our analyses were
modified to a syntactically appropriate form. These
modifications were generally limited to ensuring that
nouns or noun phrases were in nodes and verbs were on
linking lines. In addition, in cases when it appeared that a
student had chained together several concepts intending
to have them to all refer to a particular concept, we
re-linked these concepts in a manner that was suitable for
our analysis. Once the concept maps were in proper
form, we extracted all propositions (a feature offered by
the CmapTools™ software) and displayed them in a
three-column format (concept-link-concept) for analysis.
We began our analysis by classifying concepts from
the student maps into categories based on a concept map
created by the course instructor (second author). This 'expert' map may be viewed at http://www.icess.ucsb.
edu/esrg/135_instrucor_map.html. Student concepts
were classified by the first author as either exact uses of
the concepts used in the expert map, or as 'near' concepts
that were similar to or examples of the expert concepts.
Some student concepts did not fit well within the expert
Rebich and Gautier - Concept Mapping to Reveal Prior Knowledge
359
Figure 2. This figure shows each student’s gains in number of concepts (light gray), number of propositions
(dark gray bars), ratio of concepts to propositions (white bars) between the pre-course and post-course
concept maps. Gains in concepts and links represent more elaborated knowledge; gains in the ratio of
propositions to concepts represent increased interconnection of knowledge.
concept categories, and for these we created additional
categories. We evaluated changes in the average number
of concepts, useful links and the ratio of links to concepts
on student's maps using paired Student's t-tests. On av erage, students included 45% (significant at p=.01 level
for all) more concepts on their post-course maps than on
their pre-course maps. While an increase in the number
of concepts alone is not conclusive evidence of conceptual change, these results indicate that at the very least
the students' knowledge of global climate change was
significantly enriched.
Propositions were classified into four categories:
useful propositions, examples (useful propositions that
simply correspond to examples of members within a
category, e.g. "greenhouse gases include carbon
dioxide", "greenhouse gases include methane"), 'weak'
propositions that indicate the student is likely to have an
incomplete understanding of the relationship between
the concepts in question, and misconceptions. The
observed increase in number of concepts reported above
was accompanied by a 77% average increase in the
number of useful links and a 22% average increase in the
ratio of links to concepts (see Figure 2). This indicates
that the new knowledge gained by the students is on
average characterized by a greater degree of
interconnection, which is associated with the facilitation
of knowledge retrieval and enhanced problem-solving
ability. By the end of the course, students' incidence of
misconceptions and weak conceptions had fallen from
17% of the total propositions on their maps to 9%
(difference significant at p=.01 level) (see Figure 3). This
360
significant decline in the appearance of misconceptions
suggests that some conceptual change had taken place.
At the very least, a significant number of misconceptions
had been weakened to the point that students no longer
strongly associated them with global climate change and
so did not include them on their concept maps.
DISCUSSION
Identification of Misconceptions - Identification of
common misconceptions that students were likely to
possess when entering the Mock Environment Summit
class was one of the primary purposes of our concept
map assessment. Our efforts to modify the course
curriculum to facilitate meaningful learning will be
based in part on knowledge of these misconceptions, and
modifications to our instructional approach will depend
on our interpretation of their likely causes.
Students' inappropriate mental models of shortwave
and longwave radiative processes was evidenced by a
variety of inadequate propositions. On some of the
student maps, we found evidence of a flawed mental
model that attributes increased global temperatures to
increased solar input through the ozone hole. Some
students with this mental model also understood the
greenhouse effect as the trapping of this extra (reflected)
solar energy by greenhouse gases or clouds. Other
students thought it was the greenhouse gases themselves
being trapped. This misunderstanding of the greenhouse
effect may result in part from the direct analogy to a
Journal of Geoscience Education, v. 53, n. 4, September, 2005, p. 355-365
Figure 3. This figure illustrates the observed changes in the average quality of student’s propositions between
the pre-course and post-course concept maps. Frequencies of weak and misconceptions decreased
significantly, from 17% to 9%.
greenhouse maintaining heat by trapping warm air
inside. In many cases, it seemed that longwave radiative
processes did not play any part in students' models of the
greenhouse effect, which indicates that they probably do
not conceive of the earth (let alone greenhouse gases and
aerosol particles) as radiating bodies.
There was also a great deal of confusion about what
greenhouse gases are and the nature of their role in
climate and climate change. The term 'aerosol' was very
often used to describe a type of greenhouse gas, which
we considered to be evidence that many students were
making colloquial use of the word aerosol (to mean CFC)
and hadn't learned the scientific meaning of the term.
Appearance of this misconception on several post-course
concept maps indicates that some students were not
likely to have understood the discussions of aerosols that
occurred throughout the course, and this continuing
misconception was somewhat surprising in light of the
fact that the instructor and teaching assistant made
efforts on several occasions to point out the distinction
between the scientific and colloquial uses of the term
aerosol.
Many students also seemed to use greenhouse gas,
greenhouse gas emission and pollution indiscriminately.
The lack of distinction between greenhouse gas and
emission seemed to indicate that some students thought
of greenhouse gases as "bad", while in reality they are
essential to maintaining a habitable temperature on
Earth. Students who thought of greenhouse gases as
synonymous with pollution were likely to attribute all
types of pollution-related health damages to greenhouse
gas emissions, associate ozone depletion with
greenhouse gas emissions in general, and think that all
types of pollution enhance the greenhouse effect. These
students demonstrated a lack of appreciation for the role
of aerosol pollution (both natural and anthropogenic) in
determining albedo and influencing the processes of
cloud seeding and precipitation.
In fact, several of the students seemed to consider
low albedo as something 'bad', most likely because of
their knowledge of the urban heat island effect. This
limited understanding of albedo mechanisms and their
effects on climate led them to associate all 'undesirable'
land use changes with decreases in albedo, even though
processes such as deforestation usually result in
increased albedo.
Gaps in Knowledge - In addition to information about
students' misconceptions before and after the course, our
concept map study allowed us to identify gaps in
students' knowledge of global climate change. Figure 4
shows a small set of topics that were not present on the
students' post-course concept maps, and these are areas
that we would like to address in future offerings of the
course. Although most students were aware that global
climate change is associated with sea level rise, nearly all
of them attributed the rise to melting snow, glaciers and
ice caps and neglected to mention the effect of thermal
expansion. There was a general lack of appreciation for
feedbacks that occur within the climate system, and
nearly no one mentioned the connection between the
greenhouse effect and the hydrological cycle via water
vapor. In fact, water vapor was seldom mentioned as a
greenhouse gas, and there was evidence that many
students considered water vapor and clouds to be the
same thing, even though they are part of very different
climate processes. Students also made very few
references to climate change in the long-term historical
context (for example, the effects of solar variability on
climate), and provided very little information about the
Rebich and Gautier - Concept Mapping to Reveal Prior Knowledge
361
Figure 4. This figure shows the frequencies of concepts appearing on the students’ post-course maps that did
not appear on their pre-course maps. The concepts are arranged to reflect the locations of the concepts on
the instructor’s map, and they fit roughly into categories of: evidence, causes, mechanisms, predicted
consequences, mitigation & adaptation. Dark circles represent instances in which students used the concept
term nearly exactly, while lighter circles represent examples of the concept (e.g. nitrogen oxide for GHG
emission) or similar concepts. Concept names shown in bold type are those that appear on the instructor’s
map. Concepts in regular type are those that appeared on student maps but not on the instructor map.
Italicized concepts were on the instructor’s map but didn’t appear on any student maps.
362
Journal of Geoscience Education, v. 53, n. 4, September, 2005, p. 355-365
Figure 5. These two diagrams show the incidence of weak conceptions and misconceptions on the pre-course
(left) and post-course (right) concept maps. Circle sizes are proportional to the number of weak conceptions
or misconceptions that involve each concept; dark dotted lines represent misconceptions and light solid
lines represent weak conceptions.
mechanisms of global climate change. The role of
computational models as a source for data that underlie
climate change research was not mentioned by any of the
students.
Primary Areas of Learning and Conceptual Change Our concept mapping assessment strategy provided us
with evidence that students experienced significant
learning in the Mock Environment Summit class through
knowledge enrichment (increases in the number of
concepts and useful links used) and conceptual change
(decrease in weak and misconceptions). A closer
examination of the data gives some insight into the
content areas where most of the learning was taking
place. Figure 4 shows the frequencies of concepts
appearing on the students' post-course maps that did not
appear on their pre-course maps. To identify these
concepts, each student's map pair was analyzed for new
concept appearances on the post-course map, and then
these new concept appearances were aggregated across
all students. This illustration shows large increases in
students' knowledge regarding both the causes of global
climate change and mitigation and adaptation
approaches to the problem. There was also noticeable
change in the students understanding of the predicted
consequences of climate change, especially in the areas of
human health and agriculture. Our concept map
assessment also showed that students had achieved a
greater level of interconnectedness and cohesion in their
knowledge about global climate change (Figure 2),
which is related to greater ability and creativity in
problem solving processes.
Conceptual Change - On the level of conceptual change
involving prior misconceptions, we were also able to
observe significant progress. Figure 5 shows observed
decreases in the overall number of weak conceptions and
misconceptions, and also illustrates which concepts were
involved in these conceptions. Although the total
number of propositions on the post-course maps
increased, the total numbers of weak and misconceptions
decreased. In addition to illustrating this generally
positive trend, Figure 5 also shows how certain concepts
(aerosol emissions, shortwave and longwave radiative
processes, changes in temperature, greenhouse effect,
greenhouse gas emissions, trapping and pollution) are
associated with misconceptions that seem to have been
resistant to instruction. Future modifications to the Mock
Environment Summit curriculum will seek to address
these misconceptions in multiple ways, taking
advantage of the unique opportunities for meaningful
learning provided by the role-playing, argumentation
and discussion activities that already form the basis for
the learning environment.
This study did not did not focus on conceptual
change in areas outside of the physical science realm,
which we believe make up a large portion of the learning
occurring throughout the class. In the future, we may
conduct further studies that seek to examine the
developments in knowledge regarding the economic,
political and social aspects of global climate change. We
are also particularly interested in how epistemological
beliefs about the nature of science and the role of science
in society affect learning about the topic. The Mock
Environment Summit course may prove a fruitful
ground for exploring these questions.
Rebich and Gautier - Concept Mapping to Reveal Prior Knowledge
363
SUMMARY AND CONCLUSIONS
Our evaluation of student learning in the Mock
Environment Summit course through the use of pre- and
post-course concept mapping provided valuable insight
into the science learning taking place in the class.
Concept mapping proved to be a valuable assessment
tool that allowed us to observe significant increases in
the breadth and interconnectedness of student
knowledge of climate change. This multi-dimensional
and open-ended approach to assessment also offered
information about student knowledge structures that
was detailed enough to be suitable for identification of a
set of commonly held misconceptions. This evaluation of
the effectiveness of the Mock Summit course in
facilitating conceptual change has provided a starting
point for further development and improvement of the
course, and the insight we gained may also be used to
inform development of instructional materials about
climate change for a variety of audiences in both formal
and informal educational contexts.
ACKNOWLEDGMENTS
This work was partially supported through the
Curricular Assessment program of the University of
California Santa Barbara Office of Instructional
Development. Special thanks to Joao Hespanha for
assistance with data analysis and visualization, and to
Julie Dillemuth, Dan Montello and anonymous
reviewers for useful comments on a draft of this
manuscript.
REFERENCES
Alexander, P.A., 1996, The past, the present and future of
knowledge research: A reexamination of the role of
knowledge in learning and instruction, Educational
Psychologist, v. 31, p. 89-92.
Anderson, R.C., 1984, Some reflections on the acquisition
of knowledge, Educational Researcher, v. 13, p. 5-10.
Ausubel, D.P., 1963, The psychology of meaningful
verbal learning: an introduction to school learning,
New York, Grune and Stratton.
Barrows, H.S., 1988, The tutorial process: IL, Southern
Illinois University School of Medicine.
Boud, D. and Feletti, G. (editors), 1991, The challenge of
problem-based learning, New York, St. Martin's
Press.
Carey, S., 1985, Conceptual change in childhood,
Cambridge, MA, MIT Press.
Carter, M., 1988, Problem solving reconsidered: a
pluralistic theory of problems, College Engligh, v. 50
p. 551-565.
Champagne, A., Gunstone, R., and Klopfer, L., 1985,
Effecting changes in cognitive structures among
physics students, in West, L. and Pines, L.
(editors), Cognitive structure and conceptual
change, Orlando, FL, Academic Press.
Chan, C., Burtis, J., and Bereiter, C., 1997, Knowledge
building as a mediator of conflict in conceptual
change, Cognition and Instruction, v. 15, p. 1-40.
Chi, M.T.H. and Roscoe, R.D, 2002, The processes and
challenges of conceptual change, in M. Limón and L.
Mason (editors), Conceptual change reconsidered,
issues in theory and practice (pp. 3-28), Dordrecht,
NL, Kluwer Academic Publishers.
364
Cohen, D., 1987, The use of concept maps to represent
unique thought processes: toward more meaningful
learning, Journal of Curriculum and Supervision, v.
2, p. 285-289.
Dreyfus, A., Jungwirth, E. , and Eliovitch, R., 1990,
Applying the "cognitive conflict" strategy for
conceptual change--some inplications, difficulties
and problems, Science Education, v. 74, p. 105-225.
Driver, R., Asoko, H., Leach, J., Mortimer, E., and Scott,
P., 1994, Constructing scientific knowledge in the
classroom, Educational Researcher, v. 23, p. 5-12.
Edwards, J. and Fraser, K., 1983, Concept maps as
reflectors of conceptual understanding, Research in
Science Education, v. 13, p. 19-26.
Gallagher, S.A., Sher, B.T., Stepien, W.J., and Workman,
D., 1995, Implementing problem-based learning in
science classrooms, School Science and Mathematics,
v. 95, p. 136-140.
Gautier, C. and Rebich, S., 2005, The use of a Mock
Environment Summit to support learning about
global climate change, Journal of Geoscience
Education, v. 53, p. 5-16.
Gobbo, C. and Chi, M., 1989, How knowledge is
structured and used by expert and novice children,
Cognitive Development, v. 1, p. 221-237.
Gregoire, M., 2003, Is it a challenge or a threat? A
dual-process model of teachers' cognition and
appraisal processes during conceptual change,
Educational Psychology Review, v. 15, p. 147-179.
Guzzetti, B.J., Snyder, T.E., Glass, G.V., and Gamas, W.S.,
1993, Promoting conceptual change in science: a
comparative
meta-analysis
of
instructional
interventions from reading education and science
education, Reading Research Quarterly, v. 28, p.
116-159.
Hynd, C., 1998, Conceptual change in a high school
physics class: In B. Guzzetti and C. Hynd (editors),
Perspectives on conceptual change: multiple ways to
understand knowing and learning in a complex
world (p. 27-36), Mahwah, NJ, Lawrence Earlbaum
Associates.
Jensen, M.S. and Finely, F.N., 1995, Teaching evolution
using historical arguments in a conceptual change
strategy,
Science Education, v. 79, p. 147-166.
Jonassen, D.H., Reeves, T.; Hong, N., Harvey, D., and
Peters, K., 1997, Concept mapping as cognitive
learning and assessment tools, Journal of Interactive
Learning Research, v. 8, p. 289-308.
Kardash, C.M. and Scholes, R.J., 1996, Effects of
preexisting beliefs, epistemological beliefs, and need
for cognition on interpretation of controversial
issues, Journal of Educational Psychology, v. 88, p.
260-271.
Kinchin, I.M., 2000, Using concept maps to reveal
understanding: a two tier analysis, School Science
Review, v. 8, p. 41-46
Kinchin, I.M., Hay, D.B., and Adams, A., 2000, How a
qualitative approach to concept map analysis can be
used to aid learning by illustrating patterns of
conceptual development,
Educational Research,
v. 42, p. 43-57.
King, P.M. and Kitchener, K.S., 1994, Developing
reflective judgment: understanding and promoting
intellectual growth and ciritical thinking in
adolescents and adults (1st ed.), San Francisco:
Jossey-Bass.
Kitchener, K.S. and King, P.A., 1981, Reflective
judgment: concepts of justification and their
relationship to age and education, Journal of
Applied Developmental Psychology, v. 2, p. 89-116.
Journal of Geoscience Education, v. 53, n. 4, September, 2005, p. 355-365
Kuhn, D., 1991, The skills of argument, New York,
Cambridge University Press.
Kuhn, D., 1999, A developmental model of critical
thinking, Educational Researcher, v.28, p. 16-26.
Leach, J. and Lewis, J., 2002, The role of students'
epistemological knowledge in the process of
conceptual change in science, in M. Limón and L.
Mason (editors), Conceptual change reconsidered,
issues in theory and practice (pp. 201-216),
Dordrecht, NL: Kluwer Academic Publishers.
Limón, M., 2001, On cognitive conflict as an instructional
strategy for conceptual change: a critical appraisal,
Learning and Instruction, v. 11, p, 357-380.
Limón, M., 2002, Conceptual change in history: In M.
Limón and L. Mason (editors), Conceptual change
reconsidered: issues in theory and practice (p.
301-336), Dordrecht, NL, Kluwer Academic
Publishers.
Linnenbrink, E.A. and Pintrich, P.R., 2002, The role of
motivational beliefs in conceptual change: In M.
Limón and L. Mason (editors), Conceptual change
reconsidered: issues in theory and practice (p.
115-135), Dordrecht, NL, Kluwer Academic
Publishers.
Markham, K.M., Mintzes, J.J., and Jones, G.M., 1994, The
concept map as a research and evaluation tool:
further evidence of validity, Journal of Research in
Science Teaching, v. 31, p. 91-101.
Mason, L., 1998, Sharing cognition to construct scientific
knowledge in school context: the role of oral and
written discourse, Instructional Science, v. 26, p.
359-389.
Mason, L., 2002, Developing epistemological thinking to
foster conceputal changes in different domains: In
M. Limón and L. Mason (editors), Conceptual
change reconsidered: issues in theory and practice
(pp. 301-336), Dordrecht, NL, Kluwer Academic
Publishers.
Mason, L., 2003, Personal epistemologies and intentional
conceptual change: In G.M. Sinatra and P.R. Pintrich
(editors), Intentional conceptual change (p. 199-236),
Mahwah, NJ, Lawrence Erlbaum Associates.
Mason, L. and Boscolo, P., 2004, Role of epistemological
understanding and interest in interpreting a
controversy and in topic-specific belief change,
Contemporary Educational Psychology, v. 29, p.
103-128.
Mason, L. and Santi, M., 1998, Discussing the greenhouse
effect: children's collaborative discourse reasoning
and conceptual change, Environmental Education
Research, v. 4, p. 67-85.
Mayer, R.E., 2002, Understanding conceptual change: a
commentary: In M. Limón and L. Mason (editors),
Conceptual change reconsidered: issues in theory
and practice (p. 101-114), Dordrecht, NL, Kluwer
Academic Publishers
McClure, J.R., Sonak, B., and Suen, H.K., 1999, Concept
map assessment of classroom learning: reliability,
validity and logistical practicality, Journal of
Research in Science Teaching, v. 36, p. 475-492.
Mortimer, E.F. and Machado, A.H., 2000, Anomalies and
conflicts in classroom discourse, Science Education,
v. 84, p. 429-444.
Nicoll, G., Francisco, J., and Nakhleh, M., 2001, A
three-tier system for assessing concept map links: a
methodological study, International Journal of
Science Education, v. 23, p. 863-875.
Novak, J.D. and Gowin, D.B., 1984, Learning how to
learn: Cambridge, UK, Cambridge University Press.
Nussbaum, E.M. and Sinatra, G.M., 2003, Argument and
conceptual engagement, Contemporary Educational
Psychology, v. 28, p. 384-395.
Okebukola, P.A. and Jegede, O.J., 1989, Students' anxiety
towards and perception of difficulty of some
biological concepts under the concept-mapping
heuristic, Research in Science and Technological
Education, v. 7, p. 85-92.
Osmundson, E., Chung, G.K.W.K., Herl, H.E., and Klein,
D.C.D., 1999, Knowledge mapping in the classroom:
a tool for examining the development of students'
conceptual understandings, Report: CSE-TR-507,
August 1999.
Pintrich, P.R., 1999, Motivational beliefs as resources for
and constraints on conceptual change: In W.
Schnotz, S. Vosniadou, and M. Carretero (editors),
New perspectives on conceptual change (p,. 33-50),
Amsterdam, Pergamon.
Pintrich, P.R., Marx, R.W., and Boyle, R.A., 1993, Beyond
cold conceptual change: the role of motivational
beliefs and classroom contextual factors in the
process of conceptual change, Review of Educational
Research, v. 63, p. 167-200.
Posner, G.J., Strike, K.A., Hewson, P.W., and Gertzon,
W.A., 1982, Accommodation of a scientific
conception: toward a theory of conceputal change,
Science Education, v. 66m o, 211-227.
Rutherford, F.J. and Ahlgren, A., 1990, Science for all
Americans: New York, Oxford University Press.
Schommer, M., 1993, Comparisons of beliefs about the
nature of knowledge and learning among
postsecondary students, Research in Higher
Education, v. 34, p. 355-369.
Schommer-Aikins, M. and Hutter, R., 2002,
Epistemological beliefs and thinking about everyday
controversial issues, The Journal of Psychology, v.
136, p. 5-20.
Schraw, G., Dunkle, M.E., and Bendixen, L.D., 1995,
Cognitive processes in well-defined and ill-defined
problem solving, Applied Cognitive Psychology, v.
9, p. 523-538.
Sinatra, G.M., Southerland, S.A., McConaughy, F., and
Demastes, J., 2003, Intentions and beliefs in students'
understanding and acceptance of biological
evolution, Journal of Research in Science Teaching,
v. 40, p. 510-528.
Smith, J., diSessa, A., and Roschelle, J., 1993,
Misconceptions reconceived: a constructivist
analysis of knowledge in transition, Journal of the
Learning Sciences, v. 3, p. 115-163.
Soja, C.M. and Huerta, D., 2001, Debating whether
dinosaurs should be "cloned" from ancient DNA to
promote cooperative learning in an introductory
evolution course, Journal of Geoscience Education,
v. 49, p. 150-157.
Strike, K.A. and Posner, G.J., 1992, A revisionist theory of
conceptual change: In Philosophy of science,
cognitive psychology, and educational theory and
practice, R.A. Duschl and R.J. Hamilton (editors),
Albany, State University of New York Press.
White, R.T., 2002, Content and conceptual change: a
commentary, in M. Limón and L. Mason (editors),
Conceptual change reconsidered: issues in theory
and practice (p. 301-336), Dordrecht, NL, Kluwer
Academic Publishers.
Wilkinson, W.K. and Maxwell, S., 1991, The influence of
college students' epistemological style on selected
problem-solving processes, Research in Higher
Education, v. 32, p. 333-350.
Rebich and Gautier - Concept Mapping to Reveal Prior Knowledge
365
Membership Application or Renewal Form
Name: __________________________
Mailing Address: __________________________________
________________________________________________
City: ______________________________ State: ________
___College/University Professor @ ______________
___Precollege Teacher @ _____________________
___Other @ ________________________________
Checks, MasterCard, or VISA (US funds
only) are payable to:National Association of
Geoscience Teachers. Mail to: NAGT, PO
Box 5443, Bellingham, WA 98227-5443
Phone:
__________________________
Fax: ____________________
Email: ___________________
Zip: _____________________
Membership
Regular USA
Outside USA
Student–USA
Student–outside USA
Retired NAGT member
Library Subscriptions
Regular USA
Outside USA
_______New
Rates (US funds)
$35 _____
$47 _____
$20 _____
$32 _____
$30 _____
$55 _____
$67 _____
_______Renewal
Check
q Credit card:
MC/VISA (circle one)
Number: ________________________
Signature: __________________________ Exp. Date
__________________________
q
The Journal and membership year runs from January to December. Subscriptions received after
June 1 will begin receiving the Journal in January of the following year. Back issues are
available for $15 (foreign $18) each.
*To qualify for student rate, indicate and obtain verification from a NAGT member:
___Undergraduate
___Graduate
____________________________________
Signature of NAGT member
School