Survey
* Your assessment is very important for improving the workof artificial intelligence, which forms the content of this project
* Your assessment is very important for improving the workof artificial intelligence, which forms the content of this project
Effects of global warming on humans wikipedia , lookup
Media coverage of global warming wikipedia , lookup
Climate change and poverty wikipedia , lookup
Scientific opinion on climate change wikipedia , lookup
Public opinion on global warming wikipedia , lookup
Climate change, industry and society wikipedia , lookup
Years of Living Dangerously wikipedia , lookup
Surveys of scientists' views on climate change wikipedia , lookup
Concept Mapping to Reveal Prior Knowledge and Conceptual Change in a Mock Summit Course on Global Climate Change Stacy Rebich Department of Geography, Institute for Computational Earth Systems Science, University of California, Santa Barbara, CA 93106 [email protected] Catherine Gautier Department of Geography, Institute for Computational Earth Systems Science, University of California, Santa Barbara, CA 93106 [email protected] ABSTRACT The complex nature of climate change science poses special challenges for educators. Learners come to the classroom with prior knowledge on the topic, which serves as a foundation for further knowledge building, but can also pose barriers to conceptual change. Learners have existing mental models that may limit their perception and processing of conflicting information and prevent adoption of scientific conceptions. Instructional strategies that attempt conceptual change by simply provoking cognitive conflict have had limited success due to the importance of epistemological beliefs and motivation to the conceptual change process. The Mock Environment Summit course uses role-playing, argumentation and discussion to heighten epistemological awareness and motivation and thereby facilitate conceptual change. The pre/post-course concept map evaluation of students' knowledge about the science of global climate change reported here shows evidence of significant learning and conceptual change. Our study also provides useful information about gaps in knowledge and the types of misconceptions students are likely to have about this topic. Insight gained from this assessment study can be used to tailor the curriculum and enhance student progress towards more scientific conceptions of the problem. CLIMATE CHANGE SCIENCE - THE NATURE OF THE DISCIPLINE Interdisciplinary Nature - Climate change science embodies an effort to understand an inherently interdisciplinary problem that has interwoven human and natural causes. Since global climate change has connections to society that are mediated by a complex range of political, social, technological and economic factors, the study of the problem in the context of each of these fields is equally relevant and important. Only through consideration of the forces and processes operating in not only the natural sphere, but also in each of these social spheres will society be able to formulate a reasonable path for future human-environment interactions. Uncertainty and Ill-defined Problem Solving - Climate change science necessitates the ability to deal with uncertainty on several levels-not only uncertainty about the workings of the complex physical climate system, but also uncertainty with respect to social and cultural processes that mediate human response to changes within the system. The interdisciplinary and complex nature of climate change science results in an abundance of ill-defined problems, and finding solutions to such problems requires skills that go beyond the relatively constrained problems generally presented in science textbooks (Schraw, Dunkle and Bendixen, 1995). Ill-defined or ill-structured problems are those that 1) begin with a lack of all information necessary to develop a solution or even to precisely define the problem, 2) have no single right approach for solution, 3) have problem definitions that change as new information is gathered, and 4) have no identifiable 'correct' solution (Gallagher, et al., 1995). Research suggests that problem-based learning approaches that use ill-defined problems facilitate learning and conceptual change and the ability to transfer that learning to other domains (Kitchener and King, 1981; Kuhn, 1991). In a problem-based learning environment, instructors function as "metacognitive coaches" (Barrows, 1988) rather than simply information presenters or discussion leaders. In their well-known report, Science for All Americans, Rutherford and Ahlgren (1990) discuss the potentials of ill-defined problem solving for enhancing not only subject matter learning, but also the metacognitive skills that are integral to scientific literacy: "Students should be given problems-at levels appropriate to their maturity-that require them to decide what evidence is relevant and to offer their own interpretations of what the evidence means. This puts a premium, just as science does, on careful observations and thoughtful analysis. Students need guidance, encouragement, and practice in collecting, sorting and analyzing evidence, and in building arguments based on it." Instructional approaches that utilize ill-structured problem solving not only result in increased learning (Gallagher et al., 1995) and information retention (Boud and Feletti, 1991), but also encourage epistemological understanding of the discipline (Wilkinson and Maxwell, 1991) and enhance motivation (Carter, 1988; Tobias, 1990). In this paper we discuss an instructional approach that makes use of collaborative problem-based learning to accrue these benefits and encourage conceptual change. LEARNING AND CONCEPTUAL CHANGE Factors that Determine Conceptual Change - The basis for many current conceptual change theories is an idea that learning is a process of integration that involves both individual and social processes and consists of revising or fitting new information into existing mental models (Driver et al., 1994; Mayer, 2002). Not all learning is considered conceptual change; this term is generally reserved for learning that results from deep processing of knowledge (Strike and Posner, 1992). Since conceptual change involves modifications in core knowledge and beliefs, it is generally not easily achieved (Hynd, 1998) and occurs as a continuous gradual process (Smith, diSessa and Roschelle, 1993). In spite of the fact that theorists often differ with regards to the exact nature of the underlying cognitive processes, there is general agreement about a group of cognitive and social factors that seem to be vital to this type of learning. Prior Rebich and Gautier - Concept Mapping to Reveal Prior Knowledge 355 knowledge is generally considered of high importance as it is deemed to serve both as the foundation for integration of new concepts and as a potential obstacle to conceptual change (Mason, 2002; Vosniadou, 2002; Chi and Roscoe, 2002). Metacogntive awareness is knowledge and self-awareness of one's own learning, and is considered a necessary but not sufficient condition for conceptual change (Carey, 1985; Limon, 2002). Epistemology, or knowledge and beliefs about the nature of the discipline and the nature of knowledge and learning, is linked to both the learner's ability to recognize incongruous beliefs and to motivation to change them (Leach and Lewis, 2002; Mason, 2002). Motivation as a factor was initially ignored in many 'cold cognition' models (Pintrich et al., 1993) but now plays a role in many models. Motivation is intrinsically linked to goals, and there is evidence that learners with mastery goals who focus on learning and understanding more readily achieve conceptual change than those with performance goals who focus on demonstrating ability (Linnenbrink and Pintrich, 1999). Awareness of meta-concepts or second-order concepts refers to having appropriate conceptions of higher-order concepts such as evidence, cause, explanation, time, space, change source, fact, and description (Limon, 2002) that influence reasoning across the subject matter. If conceptual change can only occur in the space where these factors intersect, it is easy to appreciate that achieving conceptual change should present quite a challenge. In the next section, we focus on three of these factors prior knowledge, epistemology and motivation which are of particular interest in our discussion of the Mock Environment Summit course. Focus on Prior Knowledge - Researchers in cognitive science have consistently found that the knowledge learners possess is a very strong determinant in what information they attend to, how that information is perceived, what learners judge to be important or relevant, and what they are able to understand and remember (Alexander, 1996). With this in mind, a learner's knowledge base can be thought of as a scaffold for all of his or her future learning. Results of the overwhelming number of studies on prior knowledge and conceptual change illustrate the inadequacy and inaccuracy of a conception of the learner as an 'empty vessel' or 'blank slate' that needs to be 'filled' with knowledge. Awareness of the critical role of prior knowledge in the acquisition of new knowledge naturally leads to attempts to elicit and evaluate a learner's relevant knowledge prior to instruction. Many studies of science learning have utilized an approach that involves identification of prior knowledge and application of specific instructional strategies intended to build upon and/or modify or replace that knowledge. The study reported here is based on a similar framework; our primary purpose here, however, is limited to the discussion of a strategy for eliciting and evaluating prior knowledge and suggested ways in which this evaluation could be used for instructional enhancement. MISCONCEPTIONS While prior knowledge can be seen as the foundation for integration of new concepts, it is also commonly viewed as an obstacle to conceptual change (Chi and Roscoe, 2002; Mason, 2002; Vosniadou, 2002). In research on science education, much of the attention given to prior 356 knowledge has been focused on identifying and eliminating misconceptions. (Some find the term "misconceptions" pejorative or reserve it for a specific type of non-scientific conception; Guzzetti et al. (1993) discuss a number of alternate, and possibly more appropriate, terms that have been proposed. However, for the sake of simplicity, "misconceptions" will suffice for the purposes of our discussion.) Misconceptions are common features of learners' prior knowledge throughout the sciences and have proven resistant to instruction (Champagne, Gunstone and Klopfer, 1983; Limon, 2001). A significant portion of the misconceptions research has been devoted to identifying causes for cause the persistence of misconceptions, and a variety of factors seems to be at play. The difficulty associated with overcoming an inadequate conception can be related to whether it is derived from interactions with the physical environment, the social environment or a formal instructional environment (Guzzetti et al., 1993). Prior knowledge may be characterized by varying levels affective entrenchment related to social values, ideology and identity (Limon, 2002; Pintrich et al., 1993), and presumably higher levels of affective entrenchment would correspond with greater difficulty in achieving conceptual change. Revision of misconceptions may also prove costly at the level of cognitive processing if revision of a particular mental model will require revision of a number of related models. Focus on Epistemological Beliefs - Beliefs about the nature of knowledge, knowing and learning and about the nature of science have a strong impact on a learner's capacity for conceptual change (Gregoire, 2003; Leach and Lewis, 2002; Limon, 2001; Mason, 2002). Development of epistemological beliefs is generally viewed as a progression through stages (King and Kitchener, 1994). Kuhn (1999) classifies these stages as absolutist and overly-objective (characterized by beliefs that knowledge is absolute, certain, non-problematic, right or wrong, and does not require justification since it originates from observations of reality or the authorities), multiplist and overly-subjective (characterized by beliefs that knowledge is ambiguous, idiosyncratic, and thinks each individual has his or her own views and own truths), evaluativist and having an appropriate balance of objectivity and subjectivity (characterized by beliefs that there are shared norms of inquiry and knowing, and some positions are reasonably more justified and sustainable than others) (see also Schommer, 1993). Learners' epistemological beliefs are not likely to be equally developed or applicable across a variety of subject contexts (White, 2002), which implies a need for a specific focus on beliefs about the nature of science. Specifically, science learning environments should focus on the development of understanding that "the objects of science are not the phenomena of nature but constructs that are advanced by the scientific community to interpret nature" (Driver et al., 1994). Research has shown that learners who have more sophisticated epistemological beliefs are more likely to accept evidence that conflicts with their prior knowledge and achieve conceptual change (Mason, 2002; 2003). Schraw, Dunkle and Bendixen (1995) found that less advanced epistemological beliefs were associated with lower performance in ill-defined problem solving, but found no correlation between epistemological beliefs and success at solving well-defined problems. Interpretation of and learning about controversial issues Journal of Geoscience Education, v. 53, n. 4, September, 2005, p. 355-365 seems to be particularly affected by levels of epistemological belief development (Kardash and Scholes, 1996; Sinatra et al., 2003) and only individuals with advanced epistemological understanding seem to be able to contemplate, reason and judge evidence, arguments and alternative perspectives (Mason and Boscolo, 2004). In a reciprocal fashion, including controversial issues in curricula promotes epistemological development (Schommer-Aikins and Hutter, 2002). Focus on Motivation - Pintrich, Marx and Boyle (1993) speculate on the reasons why "cold and isolated cognition models" are not able to explain students' inability to activate prior knowledge, and assert that a learner's goals, intentions, purposes, expectations, and needs must be taken into account. They propose that motivational beliefs (including goals, values, self-efficacy, and control beliefs) play a major role in conceptual change, and caution against assumption that students will find it easy to 'act like scientists' when motivational issues for students and scientists are likely different and students’ motivations are constrained by the larger context of the educational system. In examining the goal dimension of motivation, learners are often distinguished based on the possession of mastery goals (focus on learning and understanding) or performance goals (focus on good grades, approval of peers, or viewing the classroom as a competition) (Linnenbrink and Pintrich, 2002). Pintrich (1999) proposes that adoption of a mastery goal orientation will facilitate conceptual change, and cites evidence that supports this proposition. He goes on to suggest that the structure of the learning environment plays a major role in determining whether students will adopt performance or mastery goals. Constructivist approaches to teaching and learning and problem-based learning environments in which learners are given challenging, meaningful and authentic tasks, especially those that involve ill-defined problems, promote mastery goals and influence learners' motivation and cognition (Carter, 1988; Gallagher et al., 1995; Pintrich, Marx and Boyle, 1993). This is especially true when students are encouraged to engage in discussions and develop scientific arguments as they collaborate in their problem solving efforts (Nussbaum and Sinatra, 2003). The Cognitive Conflict Approach to Conceptual Change - The most popular instructional approach intended to facilitate conceptual change focuses on identifying and removing misconceptions and involves the presentation of anomalous data or information that conflicts with inadequate prior knowledge. The cognitive conflict approach generally involves evaluating a learner's current knowledge, presenting contradictory information, and then re-evaluating to identify changes in the learner's conceptions. While this approach appears logical enough and there is evidence that the stimulation of cognitive conflict is a necessary condition for conceptual change, the results of numerous studies suggest that this approach in and of itself is not sufficient. While a handful of studies report success with this approach (e.g., Jensen and Finely, 1995), many more have reported their unsuccessful attempts to support conceptual change through the presentation of conflicting information or anomalous data to create cognitive conflict (e.g., Champagne, Gunstone and Klopfer, 1985; Dreyfus, Jungwirth and Eliovitch, 1990; Guzzetti et al., 1993). The observation that cognitive conflict in the absence of knowledge-building activity did not result in conceptual change (Chan, Burtis and Bereiter, 1997) suggests that multiple methods should be used to encourage meaningful learning. Based on an analysis of the difficulties of achieving conceptual change through the cognitive conflict strategy, Limon (2001) proposes that presentation of anomalous data could be augmented with cooperative and shared learning to promote collective discussion of ideas. The Mock Environment Summit course discussed here utilizes the complementary strategy of cooperative learning through discussion and argumentation as the basis for encouraging conceptual change. Argumentation involves "constructing a rationale for a particular outcome, refuting opposing arguments, and weighing competing considerations" (Nussbaum and Sinatra, 2003). As learning tools, discussion and argumentation have proven to enhance motivation, encourage development of more sophisticated epistemological beliefs, heighten awareness of the social nature of scientific knowledge construction, and facilitate conceptual change (Mason, 1998; Mason and Santi, 1998; Mortimer and Machado, 2000; Soja and Huerta 2001). MOCK ENVIRONMENT SUMMIT Course Overview - The "Mock Environment Summit" (Geography 135) upper-division undergraduate course has been offered at The University of California Santa Barbara for the past several years as a means to encourage students to gain a deeper understanding of the scientific evidence of global climate change, consider problem-solving approaches that arise from within a variety of disciplines, and utilize a variety of skills and knowledge of different topics to negotiate an 'international agreement' that represents a collaborative effort to deal with the complex problem of climate change. (see more detailed description in Gautier and Rebich, 2005) The design of the Mock Environment Summit curriculum is firmly rooted in constructivist pedagogy, which departs from the traditional paradigm of the student as an "empty vessel" and embraces the notion that each individual comes into a class with a unique existing knowledge framework. The "Mock Environment Summit" course has been designed to create a learning environment where individuals use specially designed tools to build upon their own knowledge in an effort to experience enriched understanding and conceptual change. Integration of new knowledge with prior knowledge is accomplished through research, role playing, and the presentation of arguments to support the position appropriate to a chosen role. As students apply newly-gained knowledge to problem-solving tasks, meaningful learning is enhanced and the likelihood of the transfer of this knowledge to other relevant problem-solving situations increases (Ausubel, 1963). The "Mock Environment Summit" is an interdisciplinary, student-directed, inquiry-based Earth Science course during which students attend several lectures on the science of global change and then engage in a series of role-playing activities for which they act as policymaker-representatives of different countries. The course focuses on the drafting and negotiation of an international agreement similar to the Kyoto protocol. In their roles as policymakers, students conduct (primarily) web research to investigate the present and potential Rebich and Gautier - Concept Mapping to Reveal Prior Knowledge 357 physical, social and economic impacts that global change may have on their countries. On a regular basis, the students present their research findings to the class and begin to build the foundation for the position they plan to take in the global change debate. Each country representative also joins with representatives from other countries to research a topic related to global climate change (e.g., fresh water availability, technology transfer, carbon trading, and population control) and as they present information about these topics to the class, they lobby to have these issues included as elements of the final agreement. In addition to feedback and coaching throughout the in-class discussion and argumentation, the instructor and teaching assistant provide individualized written feedback to the students to help them improve their research and related writing and presentation assignments. Grades for the course are based on participation in class discussions, presentations, writing assignments and contributions to the final negotiated agreement. CONCEPT MAPPING STUDY Overview and Rationale - We recognized concept mapping as a potentially useful strategy for assessing the conceptual change experienced by students enrolled in the Mock Environment Summit class. Concept mapping presents itself as a particularly adapted assessment tool for this class since it allows an exploration of student knowledge at a sufficient level of complexity, does not presuppose that all students have mastered exactly the same material, and has also been shown to create a more equitable assessment situation for those who have difficulty coping with test anxiety (Okebukola and Jegede, 1989). The usefulness of concept mapping for assessment is partially due to its level of complexity, which distinguishes it from more conventional evaluation techniques such as multiple-choice tests. Markham, Mintzes and Jones (1994) suggest that these traditional unidimensional assessment measures represent a failure to recognize that much disciplinary knowledge is based on an understanding of relationships among concepts. Other researchers have found concept map-based evaluations to yield equally comprehensive and accurate overviews of knowledge as compared to well-planned structured personal interviews (Edwards and Fraser, 1983) and assessment through writing (Osmundson, et al., 1999). However, concept mapping allows for more efficient data collection than interviews do, and presents an advantage over writing-based assessments in that it is inherently non-linear and facilitates self-monitoring. Students faced with an essay-writing assessment task will often complete it in a linear fashion, starting at the beginning and writing straight through. On the other hand, students constructing concept maps can easily put down concepts and/or visual symbols and add details and connections between concepts in any order, which offers a chance for metacognitive reflection. Concept maps may be useful in revealing thought processes that generally remain private to the learner (Cohen, 1987), and it has been suggested that they may be more sensitive to developmental changes than traditional testing in which questions often focus on isolated ideas (Kinchin, Hay and Adams, 2000). Before going into more detail about the specifics of the proposed research, a brief overview of the nature of concept maps and how they can be used for assessment in the classroom may be useful. The basic element of a concept map is a proposition (see Figure 1). A proposition is a pair of concepts whose relationship is specified in the form of a link. Concepts are things usually referred to by nouns or noun phrases, while links are usually verbs. Concepts from a map on global climate Course Evaluation and Learning Assessment - Since we are aware of the role of prior knowledge in the conceptual change process, we also appreciate the value of assessing learners' existing conceptions of topics central to a course before instruction begins. Information about learners' initial beliefs and conceptions can then be used to guide the design and refinement of the learning environment. With this transition from more traditional teaching methods, there comes a set of challenges for assessing learning. When applied to constructivist learning environments, the assessment methods that have been considered appropriate under traditional pedagogy seem increasingly inappropriate. In response to this need for new evaluation tools, new methods that involve portfolio and project-based assessment have been proposed and utilized successfully (Mintzes, Wandersee and Novak, 2001). The current assessment methods utilized in Geography 135 are of this sort, and students are evaluated on the basis of performance on presentation, writing and negotiation activities. No tests are given as part of the course, and the final means for evaluation is the quality of the final agreement negotiated by the class. While these evaluation methods have proven very useful for evaluating individual student performance, it is relatively more difficult to extract from them detailed information about how well the course is achieving its stated goals and intended learning outcomes in terms of content knowledge. In particular, we have been interested in obtaining a measure of how much scientific knowledge is being gained during the course as the students listen to and participate in discussions of the various positions presented and the evidence that supports these positions. While the course is focused on achieving an interdisciplinary perspective on the climate change issue rather than on acquisition of specific climate science content, we are interested in evaluating students' level of conceptual change for the physical science content. Throughout the history of the course, carefully designed multiple-choice tests have been administered prior to and immediately following the course. The insight into learning gained from these measures, however, has seemed to be both incomplete and lacking in the complexity necessary for a more Figure 1. The basic element of a concept map is a in-depth analysis. proposition, consisting of two concepts connected by a link that shows the relationship between them. 358 Journal of Geoscience Education, v. 53, n. 4, September, 2005, p. 355-365 change might include things like "aerosol emissions", "industrial activities", "longwave radiation trapping", and "greenhouse effect." These individual concepts could then be linked to form propositions such as "aerosol emissions [are generated by] industrial activities," or "longwave radiation trapping [is associated with] greenhouse effect." Basically, a concept map is a (sometimes hierarchically) structured network of propositions. Cognitive theories that emphasize the structure of knowledge underlie instructional approaches and assessments that involve concept mapping. Anderson (1984) asserts that structure is the essence of knowledge, and the process of constructing a concept map focuses the learner's attention on the structure of knowledge and the importance of knowledge integration. Concept maps can be used to elucidate a learner's knowledge representation and organization of ideas - characteristics of understanding are also related to a learner's ability to engage in higher-order thinking (Gobbo and Chi, 1989; Jonassen et al., 1997; Kinchin, 2000). The objectives of the study reported here were to 1) elicit information about what students had learned in the class at a level of knowledge complexity that had not been observable through the multiple-choice tests used for previous course assessments, and 2) to gain an understanding of both the types of learning occurring and the specific content knowledge gained by the students in the Mock Environmental Summit class. This enhanced evaluation of learning outcomes provides valuable information that can be used to tailor and refine the curriculum; at the same time, it helps to verify the usefulness and suitability of concept map assessment for evaluating the learning taking place in open-ended, student-directed science courses. Research Design and Data Collection - The participants in the study were 17 undergraduates (ages 19 to 25; 7 female, 10 male) who were enrolled in an upper division undergraduate geography course titled "Mock Environment Summit" (Geography 135). Geography 135 was offered as an intensive course (4 class hours per day) and met for a three-week period during the 2003 Summer quarter. The majority of the students in the class were Geography and Environmental Studies majors, but sev eral were from the English, Philosophy and Global Studies departments. In this paper we examine a set of pre- and post-course concept maps on the topic of global climate change that were constructed by the participants. Prior to the construction of the pre-course concept map, students were given a one-hour training session on concept map construction. During the training session, the researcher presented a brief introduction to concept mapping, and provided some background information on how concept mapping can be useful for learning, described the purpose of the concept maps the students would be constructing. A hands-on demonstration/tutorial gave students the opportunity to practice using the CmapTools™ software (http://cmap.ihmc.us/) they would be using to construct their concept maps. Students were also given time to browse a collection of concept maps on topics unrelated to climate change, and these maps, as well as the others used during the training session, were available for browsing throughout the series of concept mapping tasks. Following the presentation and demonstration, students were asked to brainstorm in groups on the topic of 'collaborative work.' (This topic was chosen because it also offered a forum for discussion of the benefits and challenges of the group work that they would be doing throughout the course.) When they had finished brainstorming, each student constructed a concept map that reflected his or her group's thoughts about collaborative work. As students constructed maps, the researcher monitored their progress and provided feedback on the construction process. Aside from this initial training session, the students received no additional training or practice in concept map construction during the course. The day following the training session, each student constructed a concept map on the topic of global climate change. The following focus questions were given to direct their map construction: • • • • • What is global climate change? What is the evidence? What are the causes? What are the mechanisms? What are the consequences? Students were given 45 minutes to complete the concept-mapping activity. The concept-mapping activity completed on the second day of the course was repeated at the end of the course. Students were given equal amounts of time to construct their pre-course and post-course concept maps. ANALYSIS AND RESULTS The pre- and post-course student maps were analyzed for the presence or absence of key concepts and for proposition accuracy and usefulness. In addition to the calculation and evaluation of summary statistics, our analysis used a visualization method for examining the structural nature of the concept maps, their levels of interconnectedness, and the content areas where most learning occurred. Although we were not interested in 'grading' the concept maps, we based our evaluation on a combination of scoring methods that have proven effective in other studies (Kinchin, 2000; McClure, Sonak and Suen, 1999; Nicoll, Francisco and Nakhleh, 2001; Novak and Gowin, 1984). The specifics of our evaluation procedure are described below. Before beginning the concept map evaluation, we reviewed the student maps for syntax, and propositions that were not in a suitable form for our analyses were modified to a syntactically appropriate form. These modifications were generally limited to ensuring that nouns or noun phrases were in nodes and verbs were on linking lines. In addition, in cases when it appeared that a student had chained together several concepts intending to have them to all refer to a particular concept, we re-linked these concepts in a manner that was suitable for our analysis. Once the concept maps were in proper form, we extracted all propositions (a feature offered by the CmapTools™ software) and displayed them in a three-column format (concept-link-concept) for analysis. We began our analysis by classifying concepts from the student maps into categories based on a concept map created by the course instructor (second author). This 'expert' map may be viewed at http://www.icess.ucsb. edu/esrg/135_instrucor_map.html. Student concepts were classified by the first author as either exact uses of the concepts used in the expert map, or as 'near' concepts that were similar to or examples of the expert concepts. Some student concepts did not fit well within the expert Rebich and Gautier - Concept Mapping to Reveal Prior Knowledge 359 Figure 2. This figure shows each student’s gains in number of concepts (light gray), number of propositions (dark gray bars), ratio of concepts to propositions (white bars) between the pre-course and post-course concept maps. Gains in concepts and links represent more elaborated knowledge; gains in the ratio of propositions to concepts represent increased interconnection of knowledge. concept categories, and for these we created additional categories. We evaluated changes in the average number of concepts, useful links and the ratio of links to concepts on student's maps using paired Student's t-tests. On av erage, students included 45% (significant at p=.01 level for all) more concepts on their post-course maps than on their pre-course maps. While an increase in the number of concepts alone is not conclusive evidence of conceptual change, these results indicate that at the very least the students' knowledge of global climate change was significantly enriched. Propositions were classified into four categories: useful propositions, examples (useful propositions that simply correspond to examples of members within a category, e.g. "greenhouse gases include carbon dioxide", "greenhouse gases include methane"), 'weak' propositions that indicate the student is likely to have an incomplete understanding of the relationship between the concepts in question, and misconceptions. The observed increase in number of concepts reported above was accompanied by a 77% average increase in the number of useful links and a 22% average increase in the ratio of links to concepts (see Figure 2). This indicates that the new knowledge gained by the students is on average characterized by a greater degree of interconnection, which is associated with the facilitation of knowledge retrieval and enhanced problem-solving ability. By the end of the course, students' incidence of misconceptions and weak conceptions had fallen from 17% of the total propositions on their maps to 9% (difference significant at p=.01 level) (see Figure 3). This 360 significant decline in the appearance of misconceptions suggests that some conceptual change had taken place. At the very least, a significant number of misconceptions had been weakened to the point that students no longer strongly associated them with global climate change and so did not include them on their concept maps. DISCUSSION Identification of Misconceptions - Identification of common misconceptions that students were likely to possess when entering the Mock Environment Summit class was one of the primary purposes of our concept map assessment. Our efforts to modify the course curriculum to facilitate meaningful learning will be based in part on knowledge of these misconceptions, and modifications to our instructional approach will depend on our interpretation of their likely causes. Students' inappropriate mental models of shortwave and longwave radiative processes was evidenced by a variety of inadequate propositions. On some of the student maps, we found evidence of a flawed mental model that attributes increased global temperatures to increased solar input through the ozone hole. Some students with this mental model also understood the greenhouse effect as the trapping of this extra (reflected) solar energy by greenhouse gases or clouds. Other students thought it was the greenhouse gases themselves being trapped. This misunderstanding of the greenhouse effect may result in part from the direct analogy to a Journal of Geoscience Education, v. 53, n. 4, September, 2005, p. 355-365 Figure 3. This figure illustrates the observed changes in the average quality of student’s propositions between the pre-course and post-course concept maps. Frequencies of weak and misconceptions decreased significantly, from 17% to 9%. greenhouse maintaining heat by trapping warm air inside. In many cases, it seemed that longwave radiative processes did not play any part in students' models of the greenhouse effect, which indicates that they probably do not conceive of the earth (let alone greenhouse gases and aerosol particles) as radiating bodies. There was also a great deal of confusion about what greenhouse gases are and the nature of their role in climate and climate change. The term 'aerosol' was very often used to describe a type of greenhouse gas, which we considered to be evidence that many students were making colloquial use of the word aerosol (to mean CFC) and hadn't learned the scientific meaning of the term. Appearance of this misconception on several post-course concept maps indicates that some students were not likely to have understood the discussions of aerosols that occurred throughout the course, and this continuing misconception was somewhat surprising in light of the fact that the instructor and teaching assistant made efforts on several occasions to point out the distinction between the scientific and colloquial uses of the term aerosol. Many students also seemed to use greenhouse gas, greenhouse gas emission and pollution indiscriminately. The lack of distinction between greenhouse gas and emission seemed to indicate that some students thought of greenhouse gases as "bad", while in reality they are essential to maintaining a habitable temperature on Earth. Students who thought of greenhouse gases as synonymous with pollution were likely to attribute all types of pollution-related health damages to greenhouse gas emissions, associate ozone depletion with greenhouse gas emissions in general, and think that all types of pollution enhance the greenhouse effect. These students demonstrated a lack of appreciation for the role of aerosol pollution (both natural and anthropogenic) in determining albedo and influencing the processes of cloud seeding and precipitation. In fact, several of the students seemed to consider low albedo as something 'bad', most likely because of their knowledge of the urban heat island effect. This limited understanding of albedo mechanisms and their effects on climate led them to associate all 'undesirable' land use changes with decreases in albedo, even though processes such as deforestation usually result in increased albedo. Gaps in Knowledge - In addition to information about students' misconceptions before and after the course, our concept map study allowed us to identify gaps in students' knowledge of global climate change. Figure 4 shows a small set of topics that were not present on the students' post-course concept maps, and these are areas that we would like to address in future offerings of the course. Although most students were aware that global climate change is associated with sea level rise, nearly all of them attributed the rise to melting snow, glaciers and ice caps and neglected to mention the effect of thermal expansion. There was a general lack of appreciation for feedbacks that occur within the climate system, and nearly no one mentioned the connection between the greenhouse effect and the hydrological cycle via water vapor. In fact, water vapor was seldom mentioned as a greenhouse gas, and there was evidence that many students considered water vapor and clouds to be the same thing, even though they are part of very different climate processes. Students also made very few references to climate change in the long-term historical context (for example, the effects of solar variability on climate), and provided very little information about the Rebich and Gautier - Concept Mapping to Reveal Prior Knowledge 361 Figure 4. This figure shows the frequencies of concepts appearing on the students’ post-course maps that did not appear on their pre-course maps. The concepts are arranged to reflect the locations of the concepts on the instructor’s map, and they fit roughly into categories of: evidence, causes, mechanisms, predicted consequences, mitigation & adaptation. Dark circles represent instances in which students used the concept term nearly exactly, while lighter circles represent examples of the concept (e.g. nitrogen oxide for GHG emission) or similar concepts. Concept names shown in bold type are those that appear on the instructor’s map. Concepts in regular type are those that appeared on student maps but not on the instructor map. Italicized concepts were on the instructor’s map but didn’t appear on any student maps. 362 Journal of Geoscience Education, v. 53, n. 4, September, 2005, p. 355-365 Figure 5. These two diagrams show the incidence of weak conceptions and misconceptions on the pre-course (left) and post-course (right) concept maps. Circle sizes are proportional to the number of weak conceptions or misconceptions that involve each concept; dark dotted lines represent misconceptions and light solid lines represent weak conceptions. mechanisms of global climate change. The role of computational models as a source for data that underlie climate change research was not mentioned by any of the students. Primary Areas of Learning and Conceptual Change Our concept mapping assessment strategy provided us with evidence that students experienced significant learning in the Mock Environment Summit class through knowledge enrichment (increases in the number of concepts and useful links used) and conceptual change (decrease in weak and misconceptions). A closer examination of the data gives some insight into the content areas where most of the learning was taking place. Figure 4 shows the frequencies of concepts appearing on the students' post-course maps that did not appear on their pre-course maps. To identify these concepts, each student's map pair was analyzed for new concept appearances on the post-course map, and then these new concept appearances were aggregated across all students. This illustration shows large increases in students' knowledge regarding both the causes of global climate change and mitigation and adaptation approaches to the problem. There was also noticeable change in the students understanding of the predicted consequences of climate change, especially in the areas of human health and agriculture. Our concept map assessment also showed that students had achieved a greater level of interconnectedness and cohesion in their knowledge about global climate change (Figure 2), which is related to greater ability and creativity in problem solving processes. Conceptual Change - On the level of conceptual change involving prior misconceptions, we were also able to observe significant progress. Figure 5 shows observed decreases in the overall number of weak conceptions and misconceptions, and also illustrates which concepts were involved in these conceptions. Although the total number of propositions on the post-course maps increased, the total numbers of weak and misconceptions decreased. In addition to illustrating this generally positive trend, Figure 5 also shows how certain concepts (aerosol emissions, shortwave and longwave radiative processes, changes in temperature, greenhouse effect, greenhouse gas emissions, trapping and pollution) are associated with misconceptions that seem to have been resistant to instruction. Future modifications to the Mock Environment Summit curriculum will seek to address these misconceptions in multiple ways, taking advantage of the unique opportunities for meaningful learning provided by the role-playing, argumentation and discussion activities that already form the basis for the learning environment. This study did not did not focus on conceptual change in areas outside of the physical science realm, which we believe make up a large portion of the learning occurring throughout the class. In the future, we may conduct further studies that seek to examine the developments in knowledge regarding the economic, political and social aspects of global climate change. We are also particularly interested in how epistemological beliefs about the nature of science and the role of science in society affect learning about the topic. The Mock Environment Summit course may prove a fruitful ground for exploring these questions. Rebich and Gautier - Concept Mapping to Reveal Prior Knowledge 363 SUMMARY AND CONCLUSIONS Our evaluation of student learning in the Mock Environment Summit course through the use of pre- and post-course concept mapping provided valuable insight into the science learning taking place in the class. Concept mapping proved to be a valuable assessment tool that allowed us to observe significant increases in the breadth and interconnectedness of student knowledge of climate change. This multi-dimensional and open-ended approach to assessment also offered information about student knowledge structures that was detailed enough to be suitable for identification of a set of commonly held misconceptions. This evaluation of the effectiveness of the Mock Summit course in facilitating conceptual change has provided a starting point for further development and improvement of the course, and the insight we gained may also be used to inform development of instructional materials about climate change for a variety of audiences in both formal and informal educational contexts. ACKNOWLEDGMENTS This work was partially supported through the Curricular Assessment program of the University of California Santa Barbara Office of Instructional Development. Special thanks to Joao Hespanha for assistance with data analysis and visualization, and to Julie Dillemuth, Dan Montello and anonymous reviewers for useful comments on a draft of this manuscript. REFERENCES Alexander, P.A., 1996, The past, the present and future of knowledge research: A reexamination of the role of knowledge in learning and instruction, Educational Psychologist, v. 31, p. 89-92. Anderson, R.C., 1984, Some reflections on the acquisition of knowledge, Educational Researcher, v. 13, p. 5-10. Ausubel, D.P., 1963, The psychology of meaningful verbal learning: an introduction to school learning, New York, Grune and Stratton. Barrows, H.S., 1988, The tutorial process: IL, Southern Illinois University School of Medicine. Boud, D. and Feletti, G. (editors), 1991, The challenge of problem-based learning, New York, St. Martin's Press. Carey, S., 1985, Conceptual change in childhood, Cambridge, MA, MIT Press. Carter, M., 1988, Problem solving reconsidered: a pluralistic theory of problems, College Engligh, v. 50 p. 551-565. Champagne, A., Gunstone, R., and Klopfer, L., 1985, Effecting changes in cognitive structures among physics students, in West, L. and Pines, L. (editors), Cognitive structure and conceptual change, Orlando, FL, Academic Press. Chan, C., Burtis, J., and Bereiter, C., 1997, Knowledge building as a mediator of conflict in conceptual change, Cognition and Instruction, v. 15, p. 1-40. Chi, M.T.H. and Roscoe, R.D, 2002, The processes and challenges of conceptual change, in M. Limón and L. Mason (editors), Conceptual change reconsidered, issues in theory and practice (pp. 3-28), Dordrecht, NL, Kluwer Academic Publishers. 364 Cohen, D., 1987, The use of concept maps to represent unique thought processes: toward more meaningful learning, Journal of Curriculum and Supervision, v. 2, p. 285-289. Dreyfus, A., Jungwirth, E. , and Eliovitch, R., 1990, Applying the "cognitive conflict" strategy for conceptual change--some inplications, difficulties and problems, Science Education, v. 74, p. 105-225. Driver, R., Asoko, H., Leach, J., Mortimer, E., and Scott, P., 1994, Constructing scientific knowledge in the classroom, Educational Researcher, v. 23, p. 5-12. Edwards, J. and Fraser, K., 1983, Concept maps as reflectors of conceptual understanding, Research in Science Education, v. 13, p. 19-26. Gallagher, S.A., Sher, B.T., Stepien, W.J., and Workman, D., 1995, Implementing problem-based learning in science classrooms, School Science and Mathematics, v. 95, p. 136-140. Gautier, C. and Rebich, S., 2005, The use of a Mock Environment Summit to support learning about global climate change, Journal of Geoscience Education, v. 53, p. 5-16. Gobbo, C. and Chi, M., 1989, How knowledge is structured and used by expert and novice children, Cognitive Development, v. 1, p. 221-237. Gregoire, M., 2003, Is it a challenge or a threat? A dual-process model of teachers' cognition and appraisal processes during conceptual change, Educational Psychology Review, v. 15, p. 147-179. Guzzetti, B.J., Snyder, T.E., Glass, G.V., and Gamas, W.S., 1993, Promoting conceptual change in science: a comparative meta-analysis of instructional interventions from reading education and science education, Reading Research Quarterly, v. 28, p. 116-159. Hynd, C., 1998, Conceptual change in a high school physics class: In B. Guzzetti and C. Hynd (editors), Perspectives on conceptual change: multiple ways to understand knowing and learning in a complex world (p. 27-36), Mahwah, NJ, Lawrence Earlbaum Associates. Jensen, M.S. and Finely, F.N., 1995, Teaching evolution using historical arguments in a conceptual change strategy, Science Education, v. 79, p. 147-166. Jonassen, D.H., Reeves, T.; Hong, N., Harvey, D., and Peters, K., 1997, Concept mapping as cognitive learning and assessment tools, Journal of Interactive Learning Research, v. 8, p. 289-308. Kardash, C.M. and Scholes, R.J., 1996, Effects of preexisting beliefs, epistemological beliefs, and need for cognition on interpretation of controversial issues, Journal of Educational Psychology, v. 88, p. 260-271. Kinchin, I.M., 2000, Using concept maps to reveal understanding: a two tier analysis, School Science Review, v. 8, p. 41-46 Kinchin, I.M., Hay, D.B., and Adams, A., 2000, How a qualitative approach to concept map analysis can be used to aid learning by illustrating patterns of conceptual development, Educational Research, v. 42, p. 43-57. King, P.M. and Kitchener, K.S., 1994, Developing reflective judgment: understanding and promoting intellectual growth and ciritical thinking in adolescents and adults (1st ed.), San Francisco: Jossey-Bass. Kitchener, K.S. and King, P.A., 1981, Reflective judgment: concepts of justification and their relationship to age and education, Journal of Applied Developmental Psychology, v. 2, p. 89-116. Journal of Geoscience Education, v. 53, n. 4, September, 2005, p. 355-365 Kuhn, D., 1991, The skills of argument, New York, Cambridge University Press. Kuhn, D., 1999, A developmental model of critical thinking, Educational Researcher, v.28, p. 16-26. Leach, J. and Lewis, J., 2002, The role of students' epistemological knowledge in the process of conceptual change in science, in M. Limón and L. Mason (editors), Conceptual change reconsidered, issues in theory and practice (pp. 201-216), Dordrecht, NL: Kluwer Academic Publishers. Limón, M., 2001, On cognitive conflict as an instructional strategy for conceptual change: a critical appraisal, Learning and Instruction, v. 11, p, 357-380. Limón, M., 2002, Conceptual change in history: In M. Limón and L. Mason (editors), Conceptual change reconsidered: issues in theory and practice (p. 301-336), Dordrecht, NL, Kluwer Academic Publishers. Linnenbrink, E.A. and Pintrich, P.R., 2002, The role of motivational beliefs in conceptual change: In M. Limón and L. Mason (editors), Conceptual change reconsidered: issues in theory and practice (p. 115-135), Dordrecht, NL, Kluwer Academic Publishers. Markham, K.M., Mintzes, J.J., and Jones, G.M., 1994, The concept map as a research and evaluation tool: further evidence of validity, Journal of Research in Science Teaching, v. 31, p. 91-101. Mason, L., 1998, Sharing cognition to construct scientific knowledge in school context: the role of oral and written discourse, Instructional Science, v. 26, p. 359-389. Mason, L., 2002, Developing epistemological thinking to foster conceputal changes in different domains: In M. Limón and L. Mason (editors), Conceptual change reconsidered: issues in theory and practice (pp. 301-336), Dordrecht, NL, Kluwer Academic Publishers. Mason, L., 2003, Personal epistemologies and intentional conceptual change: In G.M. Sinatra and P.R. Pintrich (editors), Intentional conceptual change (p. 199-236), Mahwah, NJ, Lawrence Erlbaum Associates. Mason, L. and Boscolo, P., 2004, Role of epistemological understanding and interest in interpreting a controversy and in topic-specific belief change, Contemporary Educational Psychology, v. 29, p. 103-128. Mason, L. and Santi, M., 1998, Discussing the greenhouse effect: children's collaborative discourse reasoning and conceptual change, Environmental Education Research, v. 4, p. 67-85. Mayer, R.E., 2002, Understanding conceptual change: a commentary: In M. Limón and L. Mason (editors), Conceptual change reconsidered: issues in theory and practice (p. 101-114), Dordrecht, NL, Kluwer Academic Publishers McClure, J.R., Sonak, B., and Suen, H.K., 1999, Concept map assessment of classroom learning: reliability, validity and logistical practicality, Journal of Research in Science Teaching, v. 36, p. 475-492. Mortimer, E.F. and Machado, A.H., 2000, Anomalies and conflicts in classroom discourse, Science Education, v. 84, p. 429-444. Nicoll, G., Francisco, J., and Nakhleh, M., 2001, A three-tier system for assessing concept map links: a methodological study, International Journal of Science Education, v. 23, p. 863-875. Novak, J.D. and Gowin, D.B., 1984, Learning how to learn: Cambridge, UK, Cambridge University Press. Nussbaum, E.M. and Sinatra, G.M., 2003, Argument and conceptual engagement, Contemporary Educational Psychology, v. 28, p. 384-395. Okebukola, P.A. and Jegede, O.J., 1989, Students' anxiety towards and perception of difficulty of some biological concepts under the concept-mapping heuristic, Research in Science and Technological Education, v. 7, p. 85-92. Osmundson, E., Chung, G.K.W.K., Herl, H.E., and Klein, D.C.D., 1999, Knowledge mapping in the classroom: a tool for examining the development of students' conceptual understandings, Report: CSE-TR-507, August 1999. Pintrich, P.R., 1999, Motivational beliefs as resources for and constraints on conceptual change: In W. Schnotz, S. Vosniadou, and M. Carretero (editors), New perspectives on conceptual change (p,. 33-50), Amsterdam, Pergamon. Pintrich, P.R., Marx, R.W., and Boyle, R.A., 1993, Beyond cold conceptual change: the role of motivational beliefs and classroom contextual factors in the process of conceptual change, Review of Educational Research, v. 63, p. 167-200. Posner, G.J., Strike, K.A., Hewson, P.W., and Gertzon, W.A., 1982, Accommodation of a scientific conception: toward a theory of conceputal change, Science Education, v. 66m o, 211-227. Rutherford, F.J. and Ahlgren, A., 1990, Science for all Americans: New York, Oxford University Press. Schommer, M., 1993, Comparisons of beliefs about the nature of knowledge and learning among postsecondary students, Research in Higher Education, v. 34, p. 355-369. Schommer-Aikins, M. and Hutter, R., 2002, Epistemological beliefs and thinking about everyday controversial issues, The Journal of Psychology, v. 136, p. 5-20. Schraw, G., Dunkle, M.E., and Bendixen, L.D., 1995, Cognitive processes in well-defined and ill-defined problem solving, Applied Cognitive Psychology, v. 9, p. 523-538. Sinatra, G.M., Southerland, S.A., McConaughy, F., and Demastes, J., 2003, Intentions and beliefs in students' understanding and acceptance of biological evolution, Journal of Research in Science Teaching, v. 40, p. 510-528. Smith, J., diSessa, A., and Roschelle, J., 1993, Misconceptions reconceived: a constructivist analysis of knowledge in transition, Journal of the Learning Sciences, v. 3, p. 115-163. Soja, C.M. and Huerta, D., 2001, Debating whether dinosaurs should be "cloned" from ancient DNA to promote cooperative learning in an introductory evolution course, Journal of Geoscience Education, v. 49, p. 150-157. Strike, K.A. and Posner, G.J., 1992, A revisionist theory of conceptual change: In Philosophy of science, cognitive psychology, and educational theory and practice, R.A. Duschl and R.J. Hamilton (editors), Albany, State University of New York Press. White, R.T., 2002, Content and conceptual change: a commentary, in M. Limón and L. Mason (editors), Conceptual change reconsidered: issues in theory and practice (p. 301-336), Dordrecht, NL, Kluwer Academic Publishers. Wilkinson, W.K. and Maxwell, S., 1991, The influence of college students' epistemological style on selected problem-solving processes, Research in Higher Education, v. 32, p. 333-350. Rebich and Gautier - Concept Mapping to Reveal Prior Knowledge 365 Membership Application or Renewal Form Name: __________________________ Mailing Address: __________________________________ ________________________________________________ City: ______________________________ State: ________ ___College/University Professor @ ______________ ___Precollege Teacher @ _____________________ ___Other @ ________________________________ Checks, MasterCard, or VISA (US funds only) are payable to:National Association of Geoscience Teachers. Mail to: NAGT, PO Box 5443, Bellingham, WA 98227-5443 Phone: __________________________ Fax: ____________________ Email: ___________________ Zip: _____________________ Membership Regular USA Outside USA Student–USA Student–outside USA Retired NAGT member Library Subscriptions Regular USA Outside USA _______New Rates (US funds) $35 _____ $47 _____ $20 _____ $32 _____ $30 _____ $55 _____ $67 _____ _______Renewal Check q Credit card: MC/VISA (circle one) Number: ________________________ Signature: __________________________ Exp. Date __________________________ q The Journal and membership year runs from January to December. Subscriptions received after June 1 will begin receiving the Journal in January of the following year. Back issues are available for $15 (foreign $18) each. *To qualify for student rate, indicate and obtain verification from a NAGT member: ___Undergraduate ___Graduate ____________________________________ Signature of NAGT member School