Survey
* Your assessment is very important for improving the workof artificial intelligence, which forms the content of this project
* Your assessment is very important for improving the workof artificial intelligence, which forms the content of this project
Mobile Data Mining: A look at real Usage Data I. Abstract Initial mobile computing research at Abilene Christian University focused on self-report questionnaires to answer questions about technology adoption and device usage. This project brought various computer log systems together in one place to enable researchers to study how students actually use computing devices and access course technologies. II. Introduction o Statement of the problem Although ACU's mobile computing initiative has been exceptionally wellreceived, we have reached the limits of mobile computing research based on selfreporting surveys. New insights into mobile computing require access to real usage data. This project captures data and enables analysis on a scale previously impossible to researchers at Abilene Christian University. III. Hypothesis or research question(s) For the last school year we have been manually collecting computer log data by user (ixs03a), device type (desk top, lap top, mobile device), campus location (COBA atrium), Blackboard and Blog application use (from Blackboard and Blog logs), content accessed (Quiz) , date / time (4/7/11 at 5:22 pm), and Banner demographic information (GPA). This data is allowing us to research mobile data use patterns at ACU. We believe that this data will better inform us as to how users take their devices and learn within the educational environment. We are gathering this data in order to answer questions like “How does the mobile device usage pattern of an "A" student differ from that of a "D" student? What does an iPad user profile look like? Does this user prefer the iPad over the laptop based on user profiles? Where do students prefer to learn and do device types vary by locations chosen to study? IV. Literature review -Citations and References • ACU Connected 2011 - ACU Research Sheds Light on Mobility in Teaching, Learning - http://www.acu.edu/news/2011/110919-mobility-research.html • Apple. (2012b). Apple – education – iBooks textbooks for iPad. Retrieved from http://www.apple.com/education/ibooks-textbooks Shepherd / Reeves Page 1 V. • California Department of Education. (2012). Digital textbook initiative. Retrieved from http://www.clrn.org/fdti • Connexions. (2012). Connexions – sharing knowledge and building communities. Retrieved from http://www.cnx.org • Inkling (2012). Inkling – interactive textbooks for iPad. Retrieved from http://inkling.com • Inkling, "Textbooks now featuring features!" http://www.inkling.com/product/. Inkling, n.d. Web. 14 Feb 2011. • Kay, A. (1972). A personal computer for children of all ages. Retrieved from http://www.mprove.de/diplom/gui/Kay72a.pdf • Kahn Academy (2012). Kahn academy. Retrieved from http://www.khanacademy.org • Kno. (2012). Kno. Retrieved from http://www.kno.com • Massachusetts Institute of Technology. (2012). Free online course materials | MIT open courseware. Retrieved from http://ocw.mit.edu/index.htm • Mayer, R. E. (2003). The promise of multimedia learning: Using the same instructional design methods across different media. Learning and Instruction, 13 (2003), pp. 125–139. • Pearson. (2012). Openclass. Retrieved from http://www.pearsonlearningsolutions.com/openclass/ • Schiller, Bradley. The Micro Economy Today. 12th. New York: McGraw Hill, 2011. eBook. • Shepherd, I. & Reeves, B. “How to Structure and Evaluate Information Technology Assignments.” Association of Business Information Systems, Federation of Business Disciplines, 1 (1), 53 - 56. (2006). Methods o Research design This project developed two systems: I. A data capture process from existing ACU system logs that stores “device use data” for analysis. This process also filters, joins, and categorizes transactions for analysis. II. A data reporting and query system that allows mobile fellows to query the data using web based analysis tools. Shepherd / Reeves Page 2 o Data collection procedures The usage data includes demographics data, wireless logs and two application sources, Blackboard and Wordpress. Blackboard data is provided via Oracle database tables: • Log: every time a Blackboard user accesses a resource, we keep track of the time and page or resource they requested • Users: all identified Blackboard users • Session: details of the http session such as user agent and ip address • Course: course numbers and names Wordpress data is accessible to us via MySQL tables and http logs. The tables include: • Users: all identified ACU blog users • Blog: creation data and name • Content: author, timestamp and content of a post • Comment: author, timestamp, comment The Wordpress extracts contain no ip addresses so in order to determine whether an access was wireless, we use http logs to match access by time and course id. These two applications, Blackboard, and Wordpress do not contain enough information in the logs to be able to determine device usage and location. That required another set of log files: • Wireless AP logs: logs access point usage • DHCP logs: logs IP numbers which are assigned dynamically In order to associate demographic data with these logs, we use Excel worksheets published on myACU. None of the various subsystems have all the data required to answer a question like: “When Brent accessed the Powerpoint slides of his Intro do Database course, where was he and what device was he using?” We produced software to match the various datasets and merge them so that the data is all in one place and amenable to queries and data mining. Shepherd / Reeves Page 3 VI. Analyzing the Data The typical ACU user profile for data usage appears in heat map format in Figure 1. Here we took a snapshot of a typical week for any device and display the “data use” by hour of day. As you can see, the patters of use (green- “lower use” to red – “intense use”) follow what we would typically expect of a students work week. We see intensive use during the latter part of the weekend as students begin to prepare for Monday classes. This is then followed by intense use of devices to work school assignments throughout the school week until the noon hour on Friday when activity dramatically tapers off. Figure 1 – All Device User Profile for Typical School Week. If we break this pattern out by device we can begin to see the differing use patterns for each device at ACU. We found three distinct patters of use. Figure 2 shows the device user patterns for laptop devices at ACU. Shepherd / Reeves Page 4 Figure 2 – Laptop User Profile for Typical School Week. Removing mobile devices from the data shows a more intense clustering of use around class days and class times as students work on assignments or activities closer to class periods or actually within the classes themselves. Figure 3 shows a typical user week for tablet devices. Once again, the nature of the tablet shows a more diffused use pattern over the week as students are more likely to have the devices with them outside of normal school work times. The blandness of color shows a fairly consistent use of the device throughout the week. Intensity of use during any one period might indicate that the device is only available to me at this time (while I am in my dorm or in my class) therefor I must work on it now. Diffuse use indicates a more mobile and relaxed response to the device use and capabilities. Figure 3 – Tablet User Profile for Typical School Week. Shepherd / Reeves Page 5 There are two significantly intense periods of use found in the mobile tablet data. These are centered on the intense in-class use of the Tuesday / Thursday 1:30 classes being researched with regard to use of the iPad. Mobile phone devices have a different use pattern than suspected. Figure 4 shows a very intense user pattern around the typical Monday through Friday class hours. It appears that during these times students are accessing significant amounts of data on their mobile phone devices from the Blackboard system. Figure 4 – Mobile Phone User Profile for Typical School Week. Further analysis shows that there is quite the eclectic combination of devices being use at ACU. Table 1 shows the number and distribution of devices used by the students and faculty at ACU. Shepherd / Reeves Page 6 Table 1 – Device Count by Type The iPad count at ACU shows that we currently have an approximate penetration of 25% into the ACU community. Mobile devices such as a combination of iPhone or iTouch show an almost 100% penetration on the campus. ACU students were given a choice of either an iPhone or iTouch during course registration. Knowing who our power users are at ACU helps us see which classes and instructors are using the LMS system to design, develop, and distribute educational content using these new delivery technologies. Table 2 lists the top 20 power users for ACU using the Blackboard LMS system. “Count” is the number of transactions measured during a typical month for each of the listed classes. Shepherd / Reeves Page 7 Table 2 – LMS System Power Users – Top 20 Within Blackboard the students are accessing data that is important to them. Table 3 is list in descending order of importance the data that students access in Blackboard. Shepherd / Reeves Page 8 Table 3 – Blackboard Content in order of Preference Seeing what areas are accessed by device might also help us understand user preferences for functionality. We took the above data, removed non-wireless activity and found that based on device, there are different access patterns. Shepherd / Reeves Page 9 Table 4 – Blackboard Content in order of Preference by Device Examining the data in Table 3 we can see that a typical student prefers to take a quiz on the laptop first, then the iPad, and then the iPhone. This may inform us as to what methods of testing are best suited to each device. Given that we can track data use by device and location we wanted to generate a profile showing which devices were preferred in each possible location. Table 5 shows a comparison of laptops, phones, and tablets by location on campus. Table 5 – Device Preference by Campus Location Shepherd / Reeves Page 10 VII. Findings/ Implications Our data analysis has allowed us to begin to ask questions that we had never thought to ask. Table 6 shows a comprehensive list of data fields by which we can analyze any variation with all demographics. This next year we will spend running trend analysis and statistical inferences on our findings. It is our hope that this extensive data analysis will further our understanding of how our students prefer to learn. Table 6 – Data fields for analysis. VIII. Recommendations for further study Having just completed 12 months of data capture we believe the next steps in this project to be: Shepherd / Reeves Page 11 a. Process - Automation We propose to automate the process of gathering all relevant data under one database. This last year’s attempts to automate all data collection and discussions with Information Technology personnel revealed the need for several sources: 1. 2. 3. 4. 5. 6. 7. 8. 9. Blackboard logs delivered via Oracle Database server. Wordpress logs delivered via MySQL Database server. blogs.acu.edu http logs logs delivered via SFTP. m.acu.edu logs delivered via SFTP. xythos course files http logs delivered via SFTP. wireless access point logs via csv file posted in xythos. wireless dhcp logs via csv file posted in xythos. registration data via csv files on my.acu.edu/campus reporting. demographic data via csv files on my.acu.edu/campus reporting. ACU Information Technology has worked hard to enable us to obtain this data. The process for obtaining and merging the data is not fully automated. Much manual work is still involved in both producing and analyzing the data. The next step ACU should take is to complete the process of automating data input and reporting. We propose to put in place a process whereby data inputs are automatically obtained and merged so that reports are always up to date. b. Patterns - Data Mining The database we have built is able to answer questions like; • • • • • Do users with iPads access Blackboard data on Sunday evenings more than users with laptops? Do students with GPA’s higher than 3.5 access Blackboard data from more locations than other students? What characteristics of device use do we find in better students? How are the devices helping students study? Can we link improved outcomes in the classroom with the types of device being used? Divide and Conquer The next step ACU should take is to analyze this data in order to understand patterns of student habits with mobile technologies. We propose to do data mining to find patterns of more and less-successful strategies. The goal is to see whether there are identifiable patterns of better learning strategies. Perhaps different colleges or majors Shepherd / Reeves Page 12 within the ACU campus have different strategies for mobile device use. Our plan is to divide and conquer the untracked data this next year. Dr. Brent Reeves will continue to run and automate the data collection process expanding our collection with the help of our systems group to include a small data log that tracks access to non-Blackboard data. The addition of this tracking methodology is critical to our ability to offer research capabilities to interested parties such as Bell Labs or others who want to monitor the application and device usage patterns of users with relation to their education products. The expansion of the logs to manage this information will allow ACU the opportunity to seek considerable research funding through either Government or Private Research funding agencies. Dr. Ian Shepherd will continue to expand on the pattern analysis of the data using query tools (such as Tableau and Excel) to support other research fellows in their search for information and provide unique views for ACU to support the mobile learning initiative. In addition to data pattern analysis, Dr. Ian Shepherd will again teach 1 class of Microeconomics on 50 ACU sponsored iPads for two semesters, and two Macroeconomics classes of 100 students using laptops allowing us to model more closely a purely mobile class versus two traditional laptop based classes. Expanding the analysis from just Blackboard logs will allow us to closely model and understand the use of other mobile applications such as eBooks, Responseware and other supporting classroom applications. The iPad class could also be used by outside research organizations to test application use by students using ACU supported data from this data mining project. IX. Research Outlets Research output from this fellowship by Dr. Reeves and Dr. Shepherd is as follows: Presentations (6 x 2 Faculty = 12) o o o o o o 2011 ACU Mobility Conference (International) 2011 ACU Connected Fall 2011 Open House (International) 2011 Blackboard World Las Vegas 7-12-11 (International) 2012 The National Business and Economics Society (NBES) in Maui, Hawaii in March (International) 2012 Spring ACU Connected Open House (International) 2012 ACU COBA Faculty Presentation (Local) Shepherd / Reeves Page 13 Targeted Presentations for Future Output from Data Analysis o Educause http://net.educause.edu/content.asp?page_id=1352&bhcp=1 o SACS Annual Meeting http://sacscoc.org/aamain.asp o New Media Consortium http://www.nmc.org/news/2012-‐summer-‐ conference-‐hosted-‐mit-‐registration-‐now-‐open o Educause LEarning Initiative (ELI) http://www.educause.edu/eli/events o Educause Center for Applied Research (ECAR) http://www.educause.edu/ecar Proceedings (1 x 2 Faculty = 2) 2012 The National Business and Economics Society (NBES) in Maui, Hawaii in March. o Journals (1 x 2 Faculty = 2) o Our journal goal for this fellowship was the Journal of Higher Education Theory and Practice. The Journal of Higher Education Theory and Practice (JHETP) is dedicated to the advancement and dissemination of academic and intellectual knowledge by publishing, through a blind, refereed process, ongoing results of research in accordance with international scientific or scholarly standards. Articles combine disciplinary methods with key insight to contemporary issues central to faculty, administrators, and industry specialists. o On Friday June 1, 2012 we received notice that our paper entitled “iPad or iFad – The Mobile Classroom” was accepted for publication in the Journal of Higher Education Theory and Practice (ISSN# 2158-3595). Publication will be in volume 12(5), 2012. Refer to Appendix A for the acceptance letter confirmation. Targeted Journals for Future Output From Data Analysis o o o o o o o o Journal of Interactive Learning Research Journal of Educational Computing Research Journal of Research on Technology in Education EDUCAUSE Quarterly Journal of Instructional Science and Technology (Australia) Educational Technology Review Journal of Computing in Teacher Education Journal of Computing in Higher Education Shepherd / Reeves Page 14 Appendix A – JHEDTP Acceptance Letter North American Business Press Hello Ian J. Shepherd, In reference to your paper recently submitted at the 2012 National Business and Economics Society Annual Conference, I am happy to inform you that your paper entitled “iPad or iFad – The Mobile Classroom” has been accepted for publication in the Journal of Higher Education Theory and Practice (ISSN# 2158-3595). The ultimate decision to place your paper in this journal was based on the scope and topic of your paper. If you accept and are able to make specific formatting changes to your paper so that it conforms to the journal’s standards, it will be published in volume 12(5), 2012. We will email the format requirements to you shortly if you accept. In an effort to promote the journal and increase author citation rates, at this time, we are requiring a one-year subscription for each accepted manuscript. Only one subscription is required regardless of the number of authors. With your subscription, all authors will receive two hard copies of their published manuscript issue, along with electronic access to the journal for one year. The subscription price for the Journal of Higher Education Theory and Practice is $320. Further instructions are provided in the Publication Format Guidelines upon your acceptance. Please let us know as soon as possible whether or not you would like your paper included. Again, congratulations. Our journals are indexed by UMI-Proquest-ABI Inform, EBSCOhost, GoogleScholar, and listed with Cabell's Directory, Ulrich's Listing of Periodicals, Bowkers Publishing Resources, the Library of Congress, the National Library of Canada, and Australia's Department of Education Science and Training. Furthermore, our journals have been affirmed as scholarly research outlets by the following business school accrediting bodies: AACSB, ACBSP, & IACBE. Best Regards, Donna Mitchell Dr. Donna Mitchell, Editor Journal of Higher Education Theory and Practice North American Business Press, Inc. [email protected] Shepherd / Reeves Page 15