Download On-Line Business Data Mining - University of Nebraska–Lincoln

On-Line Business Data
Mining
David L. Olson
University of Nebraska-Lincoln
Current demand
Our programs
New World Order
Innovation
11th INFORMS Workshop on Data Mining & Decision Analytics
2016 Nashville
1
Demand for Business Analytics
• Capgemini [2012]
• 9 of 10 business leaders believe data is now the fourth factor of production
• As fundamental to business as land, labor, capital
• Lee & Stewart [2012] (Deloitte)
• Over 90% of Fortune 500 companies will start Big Data projects by the end of 2014
• By 2015 4.4 million IT jobs (globally) will be created to support big data
• 1.9 million of these in the US (Gartner [2012])
• Gartner’s Peter Sondergaard
• “There is not enough talent in the industry. Our public and private education
systems are failing us.”
• Data Scientist – IT plus statistics
INFORMS Historical Evolution
• Applied Mathematics to business decisions
• At least the TIMS side
• 1980s – DSS
• INTERFACES articles all seemed to claim DSS
• MIS might have quibbled
• I agreed with INTERFACES
• 2000s – Business Analytics
• Seems to be THE marketing theme
• I think appropriately
Sathi (2012)
• Internal Corporate data
• Generated by e-mails, logs, blogs, documents
• Business process events
• ERP
• External to firm
• Social media
• Competitor literature
• Customer Web data
• Complaints
BIG DATA (Davenport, 2014)
• Data too big to fit on single server
• Too unstructured to fit in row-and-column database
• Too continuously flowing to fit into static data warehouse
• THE MOST IMPORTANT ASPECT IS LACK OF STRUCTURE, NOT SIZE
• The point is to ANALYZE
• Convert data into insights, innovation, business value
• Waller & Fawcett (2013)
• Shed obsession for causality in exchange for simple correlations
• Not knowing why, but only what
The Rise of Big Data:
How It’s Changing the Way We Think About the World
Cukier & Mayer-Schoenberger, April 2013
• Massive data scale
• After 2000, digitalization explosion
• Over 98% of data now digital
• Big Data – not just massive scale
• Also includes ability to quantify almost everything
• Datafication
• Location datafied by GPS satellites
Governmental & Non-Profit Examples
• European & US food safety regulations
• Need to monitor, gather data
• Need to analyze
• Hospitals
• Biological data
• Operational data
• Insurance data
• Schools
• Government
• Monitor Web site use
• Monitor use of apps
Contemporary Big Data Examples
• Baseball
• Moneyball
• Flu detection
• Google searches
• Wal-Mart disaster relief
• Hurricane Katrina
• Pop-tarts & water
Knowledge Management
Performance management How things are done (tacit knowledge,
resources
BPR)
Elaboration
Process control
Six Sigma
Information systems
Database, reports, decision support
Cloud computing
Data sources
ERP & related systems
External sources
Big data
Descriptive analysis
Data mining
RFID
Government publications
Social media
Analytics
Operations Research
Classification
Prediction
Clustering
Link analysis
Text mining
Mathematical programming
Stochastic modeling
Monte Carlo Simulation
Business Analytics (MIS view)
Hugh Watson, BizEd May/June 2013
• DESCRIPTIVE – what has happened
•
•
•
•
•
•
•
OLAP
Dashboards
Scorecards
Data visualization
Association rules
Cluster analysis
Link analysis
• (DIAGNOSTIC) – computer control
•
•
Sensors provide automatic data collection
Preprogrammed automatic control
• PREDICTIVE – what will occur
•
•
•
•
Regression, factor analysis, neural networks
Demand forecasting
Customer segmentation
Fraud detection
• PRESCRIPTIVE – what should occur
•
Revenue optimization
Watson’s view of Business Analytic Jobs
• Business Users
• Graduating students apply tools
• Do their jobs more intelligently
• Business Analysts
• Specialize in data analysis
• Support others within their organizations
• Operations research groups / MIS
• Data Scientists
• Specialize in data mining
• R, Enterprise Miner, Intelligent Miner
• Mathematical programming / simulation
Recent Growth – Big 10 (as of summer 2014)
Summer 2013 some, mostly retread OR classes
Masters
Certificate
Undergraduate
Major
Courses
Business/IT
Maryland –
marketing
analysis
Northwestern
Michigan State
(Acct & IS)
Minnesota
Purdue
Wisconsin (6
courses)
Indiana (4 courses
– Executive)
Rutgers
Purdue
Iowa (MIS)
Maryland (minor)
Ohio State (2
courses)
Wisconsin
Statistics
Illinois –
concentration in
MS
Penn State – IT
Library
Science
Michigan
MBA Certificate
• GRADUATE
• Any course can be used as an MBA elective
• All four make a Business Analytics Specialization
• Four course sequence (in order) for Certificate
1. Quantitative methods (renamed Business Analytics)
• Some revision to refocus
• Descriptive/Predictive/Prescriptive
2. Econometrics
• Statistical tools (SAS)
3. Marketing Analytics
• CRM (SAS, R)
4. Business Data Mining
• Typical business applications (Prescriptive)
• Standard tools (R, WEKA)
Future Potential Paths
• INFORMS Evolution
• Extension of applied math/decision support/context of big data
• MIS View
• Database focus
• Vendor view
• Turban
• Descriptive/Diagnostic/Predictive/Prescriptive paradigm
• Statistics Perspective
• Focus on econometrics
• I remember when data mining was pejorative
• Then statisticians got consulting money
• Systems View
• Decision making focus
US
• Great economic changes
• Wages too high
• Outsourcing
• Computer programming (service) to India
• Manufacturing to China
• Technology
• Robotics – no health benefits, no vacations, no complaints
• Computers
• ERP systems replacing multiple legacy systems
• Layoff most human IT people
• Business Analytics
• BIG DATA
JOBS
• Tradable sectors have not been net job generators for 2 decades
• Job creation exclusively within the nontradable sector
• Wages held down by increasing displaced workers
• As work stops chasing cheap labor, it will gravitate toward the final market
• To shorten delivery time, reduce inventory, etc.
The Coming Robot Dystopia:
All too inhuman
Illah Reza Nourbakhsh June 2015
• Transhumanism
• Post-evolutionary transformation replacing humans with hybrid of man &
machine
• The Age of Big Data
• Greater access to all kinds of information
• Robotic technology
• Increasingly more efficient than human labor
• Can collect & interpret unprecedented amounts of data about human behavior
• Threatens access to information
• Threatens freedom of choice
Will Humans Go the Way of Horses?
Labor in the Second Machine Age
Brynjolfsson & McAfee, June 2014
• Society in a Labor-Tight Economy
• Horses – by 1900 21 million horses & mules in the US
• Internal combustion engine – by 1960 3 million horses
• We face a similar tipping point
• How to share gains
• Control capitalist tendency toward greater inequality
• While keeping ability to efficiently allocate resources
The Fourth Industrial Revolution:
What It Means and How to Respond
Klaus Schwab – Executive Chairman – World Economic Forum
• First Industrial Revolution
• Water & Steam Power – mechanize production
• Second
• Electric Power – mass production
• Third
• Digital – fusion of technologies
• Fourth
• Billions connected by mobile devices
• Velocity, scope, systems impact
•
•
•
•
•
•
Internet of Things
Autonomous vehicles
3-D printing
Nanotechnology
Energy storage
Quantum computing
Impact of Fourth Revolution
• Might lead to greater inequality
• Particularly in disruption of labor markets
• Automation substitutes for labor
• Increase gap between returns to capital, labor
• Might also result in more safe & rewarding jobs
• Demand for highly skilled increasing
Main Effects of Fourth Industrial Revolution
1. Customer expectations
•
More focus on pleasing customers
2. Product enhancement
•
•
•
Enhanced by digital capabilities
More durable & resilient products
Data & analytics transform their maintenance
3. Collaborative innovation
4. Organizational forms
•
More flexible
Erik Brynjolfsson and Andrew McAfee 2011
Digital Frontier Press
Race Against The Machine:
How the Digital Revolution is Accelerating Innovation, Driving
Productivity, and Irreversibly Transforming Employment and the
Economy
• Computer progress advancing exponentially
• AFFECT ON
•
•
•
•
Jobs
Skills
Wages
The Economy
Big Data Changes
1 Collect a lot of data
•
Instead of sampling, use population
2 Don’t worry about data purity
•
Mass of data means errors offset
3 Correlation over Causation
•
•
Associations
Patterns
•
WHAT, not WHY
New World Order:
Labor, Capital, and Ideas in the Power Law Economy
Brynjolfsson, McAfee & Spence June 2014
• Technology has sped globalization
• Leading to single large global market
• Supply chains can move to labor’s location
• About 1/3 of goods & services in advanced economies are tradable
• This figure is rising
• Spilling over to nontradable part of the economy
Transformation of Goods & Services
1 LOW COST
Free
2 RAPID UBIQUITY
Now
3 PERFECT FIDELITY
Perfect
• Codification of processes
• Leads to digitization
• That allows replication at nearly zero cost
• IMPACT
• Abundance of consumer goods, labor, capital
• Returns follow power law (a few make a lot)
The Power of Market Creation:
How innovation can spur development
Mezue, Christensen & van Bever Dec 2014
• INNOVATION
1 Sustaining innovation
•
Replace old products with new & better
•
•
•
•
Samsung – improved smartphone replaces older
Toyota - Prius replaces Camry
Keep markets vibrant & competitive
NO NEW JOBS
INNOVATION
2 Efficiency innovation
•
Produce more for less
•
•
•
Wal-Mart
Lower prices
ELIMINATE JOBS
INNOVATION
3 Market-creating innovation
• Transform products & services cheap enough to REACH NEW POPULATION OF
CUSTOMERS
•
•
•
•
Model T Ford
Personal computer
Smartphone
Online equity trading
• Need to hire more people to make, distribute, service
• Need new supply chain networks & distribution channels
• NEW GROWTH – NEW JOBS
INNOVATION INVESTMENT
• Investment in resource industries
• Efficiency innovations
• Produce more with less
• REDUCE EMPLOYMENT
• Infrastructure investment
• Efficiency
• Limited to existing customers
• Foreign direct investment
• Usually to set up low-cost factory
• Migratory
MARKET-CREATING INNOVATIONS
• Greater increase in jobs from growth than loss from efficiency gains
• Wind energy?
• But displaces coal employment
• Solar energy?
• Sustainable economically?
• Moving people from current shorelines to higher ground?
• RIGHT NOW WE DON’T KNOW WHAT WILL WORK
• That’s what makes it innovative