Download Full Report

Mobile Data Mining: A look at real Usage Data
I.
Abstract
Initial mobile computing research at Abilene Christian University focused on self-report
questionnaires to answer questions about technology adoption and device usage. This
project brought various computer log systems together in one place to enable researchers
to study how students actually use computing devices and access course technologies.
II.
Introduction
o
Statement of the problem
Although ACU's mobile computing initiative has been exceptionally wellreceived, we have reached the limits of mobile computing research based on selfreporting surveys. New insights into mobile computing require access to real
usage data. This project captures data and enables analysis on a scale previously
impossible to researchers at Abilene Christian University.
III.
Hypothesis or research question(s)
For the last school year we have been manually collecting computer log data by
user (ixs03a), device type (desk top, lap top, mobile device), campus location
(COBA atrium), Blackboard and Blog application use (from Blackboard and
Blog logs), content accessed (Quiz) , date / time (4/7/11 at 5:22 pm), and Banner
demographic information (GPA). This data is allowing us to research mobile data
use patterns at ACU. We believe that this data will better inform us as to how
users take their devices and learn within the educational environment.
We are gathering this data in order to answer questions like “How does the mobile
device usage pattern of an "A" student differ from that of a "D" student? What
does an iPad user profile look like? Does this user prefer the iPad over the laptop
based on user profiles? Where do students prefer to learn and do device types vary
by locations chosen to study?
IV.
Literature review -Citations and References • ACU Connected 2011 - ACU Research Sheds Light on Mobility in Teaching,
Learning - http://www.acu.edu/news/2011/110919-mobility-research.html
•
Apple. (2012b). Apple – education – iBooks textbooks for iPad. Retrieved from
http://www.apple.com/education/ibooks-textbooks
Shepherd / Reeves Page 1 V.
•
California Department of Education. (2012). Digital textbook initiative. Retrieved
from http://www.clrn.org/fdti
•
Connexions. (2012). Connexions – sharing knowledge and building communities.
Retrieved from http://www.cnx.org
•
Inkling (2012). Inkling – interactive textbooks for iPad. Retrieved from
http://inkling.com
•
Inkling, "Textbooks now featuring features!" http://www.inkling.com/product/.
Inkling, n.d. Web. 14 Feb 2011.
•
Kay, A. (1972). A personal computer for children of all ages. Retrieved from
http://www.mprove.de/diplom/gui/Kay72a.pdf
•
Kahn Academy (2012). Kahn academy. Retrieved from http://www.khanacademy.org
•
Kno. (2012). Kno. Retrieved from http://www.kno.com
•
Massachusetts Institute of Technology. (2012). Free online course materials | MIT
open courseware. Retrieved from http://ocw.mit.edu/index.htm
•
Mayer, R. E. (2003). The promise of multimedia learning: Using the same
instructional design methods across different media. Learning and Instruction, 13
(2003), pp. 125–139.
•
Pearson. (2012). Openclass. Retrieved from
http://www.pearsonlearningsolutions.com/openclass/
•
Schiller, Bradley. The Micro Economy Today. 12th. New York: McGraw Hill, 2011.
eBook.
•
Shepherd, I. & Reeves, B. “How to Structure and Evaluate Information Technology
Assignments.” Association of Business Information Systems, Federation of Business
Disciplines, 1 (1), 53 - 56. (2006).
Methods
o
Research design
This project developed two systems:
I.
A data capture process from existing ACU system logs that stores “device
use data” for analysis. This process also filters, joins, and categorizes
transactions for analysis.
II.
A data reporting and query system that allows mobile fellows to query the
data using web based analysis tools.
Shepherd / Reeves Page 2 o
Data collection procedures
The usage data includes demographics data, wireless logs and two application sources, Blackboard and Wordpress. Blackboard data is provided via Oracle database tables: • Log: every time a Blackboard user accesses a resource, we keep track of the time and page or resource they requested • Users: all identified Blackboard users • Session: details of the http session such as user agent and ip address • Course: course numbers and names Wordpress data is accessible to us via MySQL tables and http logs. The tables include: • Users: all identified ACU blog users • Blog: creation data and name • Content: author, timestamp and content of a post • Comment: author, timestamp, comment The Wordpress extracts contain no ip addresses so in order to determine whether an access was wireless, we use http logs to match access by time and course id. These two applications, Blackboard, and Wordpress do not contain enough information in the logs to be able to determine device usage and location. That required another set of log files: • Wireless AP logs: logs access point usage • DHCP logs: logs IP numbers which are assigned dynamically In order to associate demographic data with these logs, we use Excel worksheets published on myACU. None of the various subsystems have all the data required to answer a question like: “When Brent accessed the Powerpoint slides of his Intro do Database course, where was he and what device was he using?” We produced software to match the various datasets and merge them so that the data is all in one place and amenable to queries and data mining.
Shepherd / Reeves Page 3 VI.
Analyzing the Data
The typical ACU user profile for data usage appears in heat map format in Figure 1. Here
we took a snapshot of a typical week for any device and display the “data use” by hour of
day. As you can see, the patters of use (green- “lower use” to red – “intense use”) follow
what we would typically expect of a students work week. We see intensive use during the
latter part of the weekend as students begin to prepare for Monday classes. This is then
followed by intense use of devices to work school assignments throughout the school
week until the noon hour on Friday when activity dramatically tapers off.
Figure 1 – All Device User Profile for Typical School Week.
If we break this pattern out by device we can begin to see the differing use patterns for
each device at ACU. We found three distinct patters of use. Figure 2 shows the device
user patterns for laptop devices at ACU.
Shepherd / Reeves Page 4 Figure 2 – Laptop User Profile for Typical School Week.
Removing mobile devices from the data shows a more intense clustering of use around
class days and class times as students work on assignments or activities closer to class
periods or actually within the classes themselves.
Figure 3 shows a typical user week for tablet devices. Once again, the nature of the tablet
shows a more diffused use pattern over the week as students are more likely to have the
devices with them outside of normal school work times. The blandness of color shows a
fairly consistent use of the device throughout the week. Intensity of use during any one
period might indicate that the device is only available to me at this time (while I am in
my dorm or in my class) therefor I must work on it now. Diffuse use indicates a more
mobile and relaxed response to the device use and capabilities.
Figure 3 – Tablet User Profile for Typical School Week.
Shepherd / Reeves Page 5 There are two significantly intense periods of use found in the mobile tablet data. These
are centered on the intense in-class use of the Tuesday / Thursday 1:30 classes being
researched with regard to use of the iPad.
Mobile phone devices have a different use pattern than suspected. Figure 4 shows a very
intense user pattern around the typical Monday through Friday class hours. It appears that
during these times students are accessing significant amounts of data on their mobile
phone devices from the Blackboard system.
Figure 4 – Mobile Phone User Profile for Typical School Week.
Further analysis shows that there is quite the eclectic combination of devices being use at
ACU. Table 1 shows the number and distribution of devices used by the students and
faculty at ACU.
Shepherd / Reeves Page 6 Table 1 – Device Count by Type
The iPad count at ACU shows that we currently have an approximate penetration of 25%
into the ACU community. Mobile devices such as a combination of iPhone or iTouch
show an almost 100% penetration on the campus. ACU students were given a choice of
either an iPhone or iTouch during course registration.
Knowing who our power users are at ACU helps us see which classes and instructors are
using the LMS system to design, develop, and distribute educational content using these
new delivery technologies. Table 2 lists the top 20 power users for ACU using the
Blackboard LMS system. “Count” is the number of transactions measured during a
typical month for each of the listed classes.
Shepherd / Reeves Page 7 Table 2 – LMS System Power Users – Top 20
Within Blackboard the students are accessing data that is important to them. Table 3 is
list in descending order of importance the data that students access in Blackboard.
Shepherd / Reeves Page 8 Table 3 – Blackboard Content in order of Preference
Seeing what areas are accessed by device might also help us understand user preferences
for functionality. We took the above data, removed non-wireless activity and found that
based on device, there are different access patterns.
Shepherd / Reeves Page 9 Table 4 – Blackboard Content in order of Preference by Device
Examining the data in Table 3 we can see that a typical student prefers to take a quiz on
the laptop first, then the iPad, and then the iPhone. This may inform us as to what
methods of testing are best suited to each device.
Given that we can track data use by device and location we wanted to generate a profile
showing which devices were preferred in each possible location. Table 5 shows a
comparison of laptops, phones, and tablets by location on campus.
Table 5 – Device Preference by Campus Location
Shepherd / Reeves Page 10 VII.
Findings/ Implications
Our data analysis has allowed us to begin to ask questions that we had never thought to
ask. Table 6 shows a comprehensive list of data fields by which we can analyze any
variation with all demographics. This next year we will spend running trend analysis and
statistical inferences on our findings. It is our hope that this extensive data analysis will
further our understanding of how our students prefer to learn.
Table 6 – Data fields for analysis.
VIII.
Recommendations for further study
Having just completed 12 months of data capture we believe the next steps in this project
to be:
Shepherd / Reeves Page 11 a. Process - Automation
We propose to automate the process of gathering all relevant data under one database.
This last year’s attempts to automate all data collection and discussions with
Information Technology personnel revealed the need for several sources:
1.
2.
3.
4.
5.
6.
7.
8.
9.
Blackboard logs delivered via Oracle Database server.
Wordpress logs delivered via MySQL Database server.
blogs.acu.edu http logs logs delivered via SFTP.
m.acu.edu logs delivered via SFTP.
xythos course files http logs delivered via SFTP.
wireless access point logs via csv file posted in xythos.
wireless dhcp logs via csv file posted in xythos.
registration data via csv files on my.acu.edu/campus reporting.
demographic data via csv files on my.acu.edu/campus reporting.
ACU Information Technology has worked hard to enable us to obtain this data. The
process for obtaining and merging the data is not fully automated. Much manual
work is still involved in both producing and analyzing the data.
The next step ACU should take is to complete the process of automating data input
and reporting. We propose to put in place a process whereby data inputs are
automatically obtained and merged so that reports are always up to date.
b. Patterns - Data Mining
The database we have built is able to answer questions like;
•
•
•
•
•
Do users with iPads access Blackboard data on Sunday evenings more than
users with laptops?
Do students with GPA’s higher than 3.5 access Blackboard data from more
locations than other students?
What characteristics of device use do we find in better students?
How are the devices helping students study?
Can we link improved outcomes in the classroom with the types of device
being used?
Divide and Conquer
The next step ACU should take is to analyze this data in order to understand patterns
of student habits with mobile technologies. We propose to do data mining to find
patterns of more and less-successful strategies. The goal is to see whether there are
identifiable patterns of better learning strategies. Perhaps different colleges or majors
Shepherd / Reeves Page 12 within the ACU campus have different strategies for mobile device use.
Our plan is to divide and conquer the untracked data this next year. Dr. Brent Reeves
will continue to run and automate the data collection process expanding our collection
with the help of our systems group to include a small data log that tracks access to
non-Blackboard data. The addition of this tracking methodology is critical to our
ability to offer research capabilities to interested parties such as Bell Labs or others
who want to monitor the application and device usage patterns of users with relation
to their education products. The expansion of the logs to manage this information will
allow ACU the opportunity to seek considerable research funding through either
Government or Private Research funding agencies. Dr. Ian Shepherd will continue to
expand on the pattern analysis of the data using query tools (such as Tableau and
Excel) to support other research fellows in their search for information and provide
unique views for ACU to support the mobile learning initiative.
In addition to data pattern analysis, Dr. Ian Shepherd will again teach 1 class of
Microeconomics on 50 ACU sponsored iPads for two semesters, and two
Macroeconomics classes of 100 students using laptops allowing us to model more
closely a purely mobile class versus two traditional laptop based classes. Expanding
the analysis from just Blackboard logs will allow us to closely model and understand
the use of other mobile applications such as eBooks, Responseware and other
supporting classroom applications.
The iPad class could also be used by outside research organizations to test application
use by students using ACU supported data from this data mining project.
IX.
Research Outlets
Research output from this fellowship by Dr. Reeves and Dr. Shepherd is as follows:
Presentations (6 x 2 Faculty = 12)
o
o
o
o
o
o
2011 ACU Mobility Conference (International)
2011 ACU Connected Fall 2011 Open House (International)
2011 Blackboard World Las Vegas 7-12-11 (International)
2012 The National Business and Economics Society (NBES) in Maui, Hawaii in
March (International)
2012 Spring ACU Connected Open House (International)
2012 ACU COBA Faculty Presentation (Local)
Shepherd / Reeves Page 13 Targeted Presentations for Future Output from Data Analysis
o Educause http://net.educause.edu/content.asp?page_id=1352&bhcp=1 o SACS Annual Meeting http://sacscoc.org/aamain.asp o New Media Consortium http://www.nmc.org/news/2012-­‐summer-­‐
conference-­‐hosted-­‐mit-­‐registration-­‐now-­‐open o Educause LEarning Initiative (ELI) http://www.educause.edu/eli/events o Educause Center for Applied Research (ECAR) http://www.educause.edu/ecar Proceedings (1 x 2 Faculty = 2)
2012 The National Business and Economics Society (NBES) in Maui, Hawaii in
March.
o
Journals (1 x 2 Faculty = 2)
o
Our journal goal for this fellowship was the Journal of Higher Education Theory
and Practice. The Journal of Higher Education Theory and Practice (JHETP) is
dedicated to the advancement and dissemination of academic and intellectual
knowledge by publishing, through a blind, refereed process, ongoing results of
research in accordance with international scientific or scholarly standards.
Articles combine disciplinary methods with key insight to contemporary issues
central to faculty, administrators, and industry specialists.
o
On Friday June 1, 2012 we received notice that our paper entitled “iPad or iFad –
The Mobile Classroom” was accepted for publication in the Journal of Higher
Education Theory and Practice (ISSN# 2158-3595). Publication will be in
volume 12(5), 2012. Refer to Appendix A for the acceptance letter confirmation.
Targeted Journals for Future Output From Data Analysis
o
o
o
o
o
o
o
o
Journal of Interactive Learning Research
Journal of Educational Computing Research
Journal of Research on Technology in Education
EDUCAUSE Quarterly
Journal of Instructional Science and Technology (Australia)
Educational Technology Review
Journal of Computing in Teacher Education
Journal of Computing in Higher Education
Shepherd / Reeves Page 14 Appendix A – JHEDTP Acceptance Letter
North American
Business Press
Hello Ian J. Shepherd,
In reference to your paper recently submitted at the 2012 National Business and Economics
Society Annual Conference, I am happy to inform you that your paper entitled “iPad or iFad –
The Mobile Classroom” has been accepted for publication in the Journal of Higher Education
Theory and Practice (ISSN# 2158-3595). The ultimate decision to place your paper in this
journal was based on the scope and topic of your paper.
If you accept and are able to make specific formatting changes to your paper so that it conforms
to the journal’s standards, it will be published in volume 12(5), 2012. We will email the format
requirements to you shortly if you accept.
In an effort to promote the journal and increase author citation rates, at this time, we are
requiring a one-year subscription for each accepted manuscript. Only one subscription is
required regardless of the number of authors. With your subscription, all authors will receive
two hard copies of their published manuscript issue, along with electronic access to the journal
for one year. The subscription price for the Journal of Higher Education Theory and Practice is
$320.
Further instructions are provided in the Publication Format Guidelines upon your acceptance.
Please let us know as soon as possible whether or not you would like your paper included.
Again, congratulations.
Our journals are indexed by UMI-Proquest-ABI Inform, EBSCOhost, GoogleScholar, and listed
with Cabell's Directory, Ulrich's Listing of Periodicals, Bowkers Publishing Resources, the
Library of Congress, the National Library of Canada, and Australia's Department of Education
Science and Training. Furthermore, our journals have been affirmed as scholarly research outlets
by the following business school accrediting bodies: AACSB, ACBSP, & IACBE.
Best Regards,
Donna Mitchell
Dr. Donna Mitchell, Editor
Journal of Higher Education Theory and Practice
North American Business Press, Inc.
[email protected]
Shepherd / Reeves Page 15