Download Data Mining and Analysis Task - Florida APTS Program

CUTR
LCTR
FDOT
RAPTS
Resource for Advanced Public Transportation Systems (RAPTS)
APTS Data Archiving and Mining System
(ADAMS)
http://technology.rapts.org
http://www.rapts.org
Data Mining and Analysis Tasks
• Literature search and review
• Determine APTS technologies and candidate
Florida transit properties
• Develop the APTS Data Archiving and Mining
System (ADAMS)
• Acquisition of hardware and software tools
required to conduct analysis
FDOT/CUTR/LCTR
Data Mining and Analysis Task
Overall Goal
 Research and analyze the use of
various APTS technology data outputs
for prototypical information systems that
enhance the management and
performance of public transportation
services.
FDOT/CUTR/LCTR
Previous Work Sponsored by FDOT
• Visited Tri-Met and AMTRAK
• Collected APC Data from JTA
• Developed a Virtual Test Bed
• Created GIS Web Maps
• Developed Customized Reports
• Started Exploring Data Mining
FDOT/CUTR/LCTR
http://technology.rapts.org
Technology Web Site to Deploy Adams
FDOT/CUTR/LCTR
Research Area
FDOT/CUTR/LCTR
Data Mining Flow Chart
AVL Data
Automatic Vehicle
Location System
Users
APC Data
Automatic Passenger
Counters
Farebox
Data
Electronic Fare
Collection System
Data
Transformation
and
Internet
SQL Server
Quality
Assurance
Users
ADAMS
BSI Data
Bus Stop Inventory
Users
GIS Data
Geographic Information
System
FDOT/CUTR/LCTR
Business Intelligence
Be more efficient
Data Analysis to help make decisions
Data extract, clean, and transformation
BI: Put questions first, then get the answers from the dataset.
FDOT/CUTR/LCTR
Data Process
•
Decision Data
•Data Clean and
Transformation
•Analysis
•Data Mining
•Reports
Raw Data
FDOT/CUTR/LCTR
Sample
• Raw Data:
FDOT/CUTR/LCTR
Sample
• Analysis:
FDOT/CUTR/LCTR
Sample
• Analysis:
FDOT/CUTR/LCTR
Sample
• Mining:
FDOT/CUTR/LCTR
Sample
• Report:
FDOT/CUTR/LCTR
Data Mining Algorithms
• Microsoft Association Rules
• Microsoft Neural Network
• Microsoft Naive Bayes
• Microsoft Decision Trees
• Microsoft Time Series
• Microsoft Clustering
• Microsoft Sequence Clustering
FDOT/CUTR/LCTR
Microsoft Naive Bayes
Bayes’ Rule states that if you have a hypothesis H and evidence
about that hypothesis E, then you can calculate the probability of
H using the following formula:
FDOT/CUTR/LCTR
Microsoft Decision Trees
Entropy (p1, p2,...,pn) = -p1log2p1 –p2log2p2... -pnlog2pn
Where p1, p2,...,Pn are the probability of each state on the
predictable attribute, p1+ p2...+Pn= 1.
FDOT/CUTR/LCTR
Microsoft Time Series
Xt=f Xt 1,Xt 2,Xt 3,Xt n+εt _- - - -i
where xt is the time series under investigation, and n is the
order of auto regression, which is generally much less than the
length of the series. The last term, epsilon, represents the
noise.
FDOT/CUTR/LCTR
Prediction
FDOT/CUTR/LCTR
Prediction
FDOT/CUTR/LCTR
Charts
FDOT/CUTR/LCTR
SQL Server Reporting Services
FDOT/CUTR/LCTR
APC Reports
FDOT/CUTR/LCTR
APC Report in pdf format
FDOT/CUTR/LCTR
AVL Reports
FDOT/CUTR/LCTR
AVL Report in pdf format
FDOT/CUTR/LCTR
Farebox Reports
FDOT/CUTR/LCTR
Farebox Report in pdf format
FDOT/CUTR/LCTR
Performance Measures
FDOT/CUTR/LCTR
Web GIS
FDOT/CUTR/LCTR
List of Performed Activities
• Visited Model Transit Properties
• Collected Data from BCT, MDT, SCAT
• Converted Data to suitable formats
• Stored Data into SQL Server
• Analyzed Data
• Developed http://technology.RAPTS.org
• Created ADAMS
FDOT/CUTR/LCTR
Software Used
 Windows Server 2003
 Visual Studio 2005
 ArcGIS 9
 Image Mapper 10
 SQL Server 2005
FDOT/CUTR/LCTR
So, What does it all mean?
• Ability to organize, integrate, analyze, and
report APTS data
• Utilize off the shelf software
• Ability to Standardize or Customize Reports
• Ability to Access and Share Data via Web
• Experiment with software and data
• Help monitor the transit system and improve
efficiencies
FDOT/CUTR/LCTR
Next Steps
• More of the same…
• Test other software applications
• Explore practical applications of using SQL
Server in mining transit data
• Improve ADAMS
FDOT/CUTR/LCTR
RAPTS Technology Goals
• Provide Data Warehouse and Web Reporting
capabilities to transit agencies throughout
Florida
• Expand the Data Mining project
• Promote State of the Art Technologies
FDOT/CUTR/LCTR
Discussion Slide
• Sharing Experiencies.
• How is the data collected, analyzed and
used at your agency?
• Is your agency actually using data from
AVL, APCs, and Electronic Fareboxes for
Decision Making?
• Innovative Approaches?
FDOT/CUTR/LCTR
CUTR
LCTR
FDOT
RAPTS
Resource for Advanced Public Transportation Systems (RAPTS)
http://www.rapts.org
http://technology.rapts.org
APTS Data Archiving and Mining System
(ADAMS)
For additional information, contact:
Fabian Cevallos, Ph.D.
Transit Program Director
Lehman Center for Transportation Research
Florida International University
Email: [email protected]
LCTR: http://lctr.eng.fiu.edu