Survey
* Your assessment is very important for improving the workof artificial intelligence, which forms the content of this project
* Your assessment is very important for improving the workof artificial intelligence, which forms the content of this project
Epistemology wikipedia , lookup
Plato's Problem wikipedia , lookup
Social learning in animals wikipedia , lookup
DIKW pyramid wikipedia , lookup
Situated cognition wikipedia , lookup
Machine learning wikipedia , lookup
Personal knowledge base wikipedia , lookup
Rationalism wikipedia , lookup
Traditional knowledge wikipedia , lookup
Agile Technologies for Personalizing Instruction Faisal Ahmad, Sebastian de la Chica, Qianyi Gu, Shaw Ketels, Ifi Okoye Tammy Sumner, Jim Martin, Alice Healy, Kirsten Butcher, Michael Wright Digital Learning Sciences University of Colorado at Boulder University Corporation for Atmospheric Research This work is supported in part by an ICS Generalization Grant, and NSF awards #0537194 and #0734875 Central Challenge Enable personalized learning, while still supporting recognized learning goals Do it at scale How People Learn (NRC) Extreme Diversity (KnowledgeWorks) Disrupting Class (Christiansen) N=1, R=G (Prahalad) 2 www.DLESE.org Strandmaps.NSDL.org Curriculum Customization CLICK Personalization Service CLICK Personalization Service Automatically identify potential learner misconceptions by analyzing student work Customize the selection and presentation of learning resources based on identified misconceptions High school plate tectonics 4 Guiding Principles Personal and intentional Build on learner understanding Learner control Learning goals organize and guide Agile technologies Domain independent: knowledge maps for human cognition and machine reasoning Automatic: NLP and ML Embeddable: web services, not applications Open: leverage existing web content 5 6 DEMO 7 Major CLICK Components What should students know? What do they already understand? Compare student and domain maps What learning activities would be useful? Domain knowledge map Select resources to address misconceptions and gaps How to embed in learning environments? Provide web service to application and portal developers 8 Detecting Potential Knowledge Gaps Student Essay Digital Library Resources (1) Student Knowledge Model (2) Domain Competency Model (3) Knowledge Trace Alignment and Comparison 9 Human-Centered Methodology Expert studies to inform algorithms (Ahmad et al 2007) Domain knowledge map creation Student essay to student knowledge map Knowledge gap diagnosis Personal instruction plan generation Expert scoring of intermediate results Mixed-method learning study 10 Algorithms Concept extraction (de la Chica 2008) MEAD: multi-document summarization toolkit (Radev et al 2004) Custom sentence scoring features: standards, gazetteer, hypertext, content word density Eliminate redundancy, rank and choose top 5% Student essays – lexical chains (de la Chica 2008) Knowledge gaps – NLP and graph structure comparisons (Ahmad 2008) Personalized information retrieval – concept matrix (Gu 2008) 11 CLICK Personalization Web Service Misconception diagnoses and knowledge map generation exposed via request types (Ahmad 2008) Submit or remove a concept map Construct student map from essay Construct domain map from URLs Get student misconceptions Get important concepts Get related concepts 12 Mixed-Method Learning Study 32 undergraduates 16 – CLICK to revise essays on Earthquakes and Plate Tectonics 16 – control Digital Library environment Data collected original essays, revised essays, detailed screen capture “movies”, reflective questions, factual knowledge tests 13 14 Essay Content Revisions Deep vs. Shallow Essay Revisions (% of Total) 80 Shallow revisions Digital Library (Control) 70 60 CLICK (Experimental) 50 40 Deep revisions 30 20 10 0 % Shallow Revisions Copying out of resource, Paraphrasing, Integrated copying, Integrated paraphrasing, Concept deletion Integrated sentence paraphrasing to create new sentence, Integrated resource paraphrasing to create new sentence, Inferencing, Generation % Deep Revisions Shallow. CLICK< Control: F (1, 27) = 3.602, p = .068 (TREND) Deep. CLICK > Control: F (1, 27) = 5.222, p = .030 (SIG EFFECT) Codes based on Wiley and Voss 1999, Constructing arguments from multiple sources 15 Types of Content Revisions Percent Omissions Corrected 50 Omissions 45 40 35 Gaps in student content knowledge such as missing details and missing concepts Incorrect Statements 30 25 Coding still underway 20 15 10 5 0 Digital Library (Control) CLICK (Experimental) CLICK > Control: F (1, 27) = 6.490. P = 0.17 (SIG EFFECT) 16 Process Data 70 Exploration Episodes Digital Library (Control) 60 CLICK (Experimental) 50 Exploring learning resources and personalized feedback Essay Episodes 40 30 Revising or working with essay Switches 20 10 0 Exploration Episodes Essay Episodes Switches Exploration. CLICK>Control: F (1, 27) = 6.076, p = .02 (SIG EFFECT) Essay. CLICK>Control: F (1, 27) = 6.815, p = .015 (SIG EFFECT) Switches. CLICK>Control: F (1, 27) = 6.447, p = .017 (SIG EFFECT) Moving between essay and exploration Integration of content resources and developing essay Recognizing need for outside knowledge source 17 Conclusions Learning - Initial CLICK results promising Encourages deep content revisions Promotes integration between information seeking and knowledge transformation Students more likely to recognize that they need new knowledge, a critical element of selfdirected learning Algorithm Generalization: Promising results for “near” domain Misconception prioritization and link generation need further work 18 Further Reading Ahmad, F., S. de la Chica, K. Butcher, T. Sumner, and J. Martin. (2007). Towards automatic conceptual personalization tools. In Proceedings of the 7th ACM/IEEE-CS Joint Conference on Digital Libraries (JCDL 2007): Vancouver, Canada (June 18-23), pp. 452-461. Butcher, K. and S. de la Chica. (in press). Supporting student learning with adaptive technology: Personalized conceptual assessment and remediation. In M. Banich and D. Caccamise (Eds.), Generalization of Knowledge: Multidisciplinary Perspectives. London, England: Taylor and Francis. de la Chica, S., F. Ahmad, J. Martin, and T Sumner. (2008). Pedagogically useful extractive summaries for science education. 22nd Meeting of the International Committee for Computational Linguistics (COLING 2008). de la Chica, S., F. Ahmad, T. Sumner, J. Martin, and K. Butcher. (2008). Computational foundations for personalizing instruction with digital libraries. International Journal of Digital Libraries. To appear in the Special Issue on Digital Libraries and Education. Gu, Q., de la Chica, S., Ahmad, F., Khan, H., Sumner, T., Martin, J., Butcher, K. (2008). Personalizing the Selection of Digital Library Resources to Support Intentional Learning. Research and Advanced Technology for Digital Libraries, 12th European Conference, ECDL 2008, Aarhus, Denmark, September 14-19. Lecture Notes in Computer Science, pp. 244-255. 19 Examples of “Good” Concepts Plate Tectonics Weather and Climate Good standalone concept A gradual build-up of mechanical stress in the crust, primarily the result of tectonic forces, provides the source of energy for earthquakes; sudden motion along a fault releases it in the form of seismic waves. The shape and position of waves in the polar jet stream determine the location and the intensity of the mid-latitude cyclones. Good concept in context Many places near this plate boundary are at high risk for earthquakes, including the San Francisco area, the Pacific Northwest, and Alaska, yet fully half the nation's earthquake hazard is in Southern California. This energy is used to heat the Earth's surface and lower atmosphere, melt and evaporate water, and run photosynthesis in plants. 20 21 Detecting Potential Knowledge Gaps Student Essay Digital Library Resources (1) Student Knowledge Model (2) Domain Competency Model (3) Knowledge Trace Alignment and Comparison 22