Download Solutions Big Data IBM

Isabelle Claverie-Berge, IM Architect
13 03 2012
Solutions Big Data IBM
Information Management
© 2012 IBM Corporation
The BIG Data Challenge
•
•
•
•
Manage and benefit from massive and growing amounts of data
Handle uncertainty around format variability and velocity of data
Handle unstructured data
Exploit BIG Data in a timely and cost effective fashion
COLLECT
Collect
INTEGRATE
Integrat
e
MANAGE
Manage
ANALYZE
Analyze
Information Management
By 2015 the number of networked devices will
be double the entire global population. All
sensor data has uncertainty.
9000
8000
100
7000
6000
The total number of social media
accounts exceeds the entire global
population. This data is highly uncertain
in both its expression and content.
90
80
2000
1000
40
30
20
0 10
ing
Th
of
et
or
s
rn
ns
Data quality solutions exist for
enterprise data like customer,
product, and address data, but
this is only a fraction of the
total enterprise data.
te
50
(I n
3000
Se
4000 60
s)
5000 70
Aggregate Uncertainty %
Global Data Volume in Exabytes
By 2015, 80% of all available data will be uncertain
dia nd te
e
M a
xt)
l
io
cia , aud
o
S de o
P
VoI
(vi
Enterprise Data
2005
Multiple sources: IDC,Cisco
2010
2015
3
© 2012 IBM Corporation
Information Management
Uncertainty arises from many sources
Process Uncertainty
Data Uncertainty
Model Uncertainty
Processes contain
“randomness”
Data input is uncertain
All modeling is approximate
Actual
Spelling
Intended
Spelling Text Entry
? ?
?
Uncertain travel times
Fitting a curve to data
GPS Uncertainty
?
Testimony
?
?
{Paris Airport}
Ambiguity
Semiconductor yield
Contaminated?
Rumors
{John Smith, Dallas}
{John Smith, Kansas}
Conflicting Data
Forecasting a hurricane
(www.noaa.gov)
4
© 2012 IBM Corporation
Information Management
The fourth dimension of Big Data: Veracity – handling data in doubt
Volume
Velocity
Data at Rest
Data in Motion
Scale from terabytes
to petabytes (1K TBs)
to zettabytes (1B
TBs)
Streaming data,
milliseconds to
seconds to respond
Often time-sensitive,
streaming data and
large volume data
movement
Variety
Data in Many
Forms
Structured,
unstructured, text,
multimedia
Veracity*
Data in Doubt
Uncertainty due to
data inconsistency
& incompleteness,
ambiguities, latency,
deception, model
approximations
* Truthfulness, accuracy or precision, correctness
5
© 2012 IBM Corporation
Information Management
Big Data Technologies
Variety
InfoSphere BigInsights to
perform exploratory analytics
InfoSphere Streams to score
incoming data against analytic
models
Nontraditional/
internet
data
Volume
InfoSphere
Warehouse
Traditional
Data
Netezza
Reuse Warehouse
Analytic models
Velocity
At-Rest Data
In-Motion Data
© 2012 IBM Corporation
Platform Capability
3
Improved Relevance
4
Higher campaign profitability
D I G I T A L M E D I A | Page 7
All in one
place
Segmentation
Increased Targeting Precision
Clickstream
Transactions
Events
CRM
Support calls
Clustering
Scoring
Feature Selection
Associations
Matching
2
Single view of customer
Personalized message
Matching algorithms
Matrix computations
Single Value Decomp.
Optimization
1
Consolidation
Marketeer’s objectives
Forecasting
Predictive algorithms
Decision trees
Linear Regression
Information Management
Platform Capability
2
Single view of customer
Increased Targeting Precision
Improved Relevance
4
Higher campaign profitability
Confidential
D I G I T A L MIBM
ED
I A | Page 8
Clickstream
Transactions
Events
CRM
Support calls
All in one
place
Big Data
Clustering
Scoring
Feature Selection
Associations
Personalized message
Matching algorithms
Matrix computations
Single Value Decomp.
Matching
3
Segmentation
1
Consolidation
Marketeer’s objectives
Optimization
Complex Analytics
© 2011 IBM Corporation
Forecasting
Predictive algorithms
Decision trees
Linear Regression
IBM Offers a Comprehensive Set of
Solutions for BIG Data
Non-Traditional
Data
InfoSphere
Big Insights
Traditional
Data
Streams filters
incoming data
Streams reuses
Warehouse Analytic
models
Persistent Data
In-Motion Data
Information Management
Un nouveau mode d’exploration des données
Traditional Approach
Big Data Approach
Structured & Repeatable Analysis
Iterative & Exploratory Analysis
IT
Business Users
Delivers a platform to
enable creative
discovery
Determine what
question to ask
IT
Business
Structures the
data to answer
that question
Explores what
questions could be
asked
Monthly sales reports
Profitability analysis
Customer surveys
IBM Confidential
Brand sentiment
Product strategy
Maximum asset utilization
© 2011 IBM Corporation
Information Management – Big Data
Big Data : integration points
Rules / BPM
Client and Partner Solutions
IBM Big Data Solutions
iLog & Lombardi
Data Warehouse
InfoSphere
Warehouse
Statistics
Image/Video
Financial
Mining
Geospatial
Times Series
Acoustic
Mathematical
InfoSphere BigInsights
Productivity Tools & Optimization
Management
Provisioning
Workflow
Job
Scheduling
Job
Tracking
Data
Ingestion
Admin
Tools
Configuration
Manager
Activity
Monitor
Identity &
Access Mgmt
Data
Protection
Connectors
Applications
InfoSphere MDM
Database
DB2 & non-IBM
Content Analytics
ECM
Business Analytics
Blue Prints
Manageability
Workload
Management &
Optimization
Processes
InfoSphere Streams
Master Data Mgmt
INTEGRATION
Data
Big Data Enterprise Engines
IBM & non-IBM
Information Server
Text
Warehouse Appliances
Applications
Big Data Analytics
Cognos & SPSS
Marketing
Unica
Data Growth
Management
InfoSphere Optim
© 2012 IBM Corporation
Information Management
Typical IBM Big Data Solution Use Cases
Create context
(classification, text mining)
Unstructured
data
Analyze
Semi-structured
data
Parse, aggregate
Structured data
Analyze, report
IBM Confidential
© 2011 IBM Corporation
Company Confidential
Analyze, report
Active archival
Long running queries
Page 12
Information Management

Big Data Use Case
Customer Experience
(Call Center)
IBM Confidential
© 2011 IBM Corporation
Information Management
Business Value Hypotheses - Summary
Analyze Customer Survey Data to
Advocates vs. Antagonist
 Cost Efficiencies
Analyze Customer Complaint Data to
Understand Key Customer Issues
Understand Impact of Social Networks
on Customer Behavior & Influence
Analyze Agent-to-Agent Internal
Memos
 Performance
 Revenue Lift
 Regulatory Compliance
Analyze Customer Interaction
Notes (i.e. emails, etc.)
Analyze Policy & Procedures
IBM Confidential
© 2011 IBM Corporation
Information Management
Addressing the Customer Experience Issues
1
2
CSR Agent asks
customer questions
to understand
situation and
customer need
4
3
CSR Agent researches
question by consulting
policies & procedures,
looking up data,
consulting other agents,
or transfers the call to
another agent
CSR Agent spends
time clarifying
question based on
research and
providing best
possible answer.
CSR Agent ensures
customer is satisfied
& probes for other
needs before closing
inquiry
Search Highly Organized Knowledge, Better Predict
Customer Needs & Drive Differentiated Experience
Future State Knowledge Management Solution
Phase 1
Phase 2
Phase 3
Establish a searchable
knowledge base database
organized by need with
structured actions and
procedures for CSR to
follow
Watson: Incorporate
transactions and
interactions into generating
a list of potential reasons
for call and prioritize
potential next best actions
Watson 2: Automatically provide
suggestions for specific
customer inquiries, as well as
suggestion for customer
treatments based on transaction
& interaction history
IBM Confidential
© 2011 IBM Corporation
Information Management
Phase 1: Establish Knowledge Base
Systems of
Record
Content
Loaders
Analytics
Knowledge
Interface
Agent Assist
2
Call Center Logs
• Caller Identified
Voice
Transcriptions
Search
Engine
ETL
3
1
Other Customer
Interactions
MDM Interface
Watson
Big Insights
Analytics
Data &
Text
Analytics
Documented
Policies & Rules
Transactions &
Statements
1
2
3
Knowledge Base
Search & Retrieve
• Issue(s)
Researched
• Issue Selected
• Apply Treatment
• Cross or Up Sell
• Closure
Call Center
Knowledge Base
Rules
Engine
Call Center
IVR
Account / Product specifications, Customer Transactions and Interactions, Policies and Procedures are loaded into Big Insights and
relevant information context linkages are established in an indexed Knowledge Base (KB). The KB has natural language capabilities to link
issues with treatments.
A customer transfers from IVR to an Agent, and gets identified
A customized UI helps the agent access the Content search engine to research the customer’s issue, locate appropriate treatment and
apply it.
IBM Confidential
© 2011 IBM Corporation
Information Management
Phase 2: Automate Issue Detection & Treatment
Analytics
Watson / Big
Insights Analytics
Knowledge
Interface
Search
Engine
Knowledge Base
Search & Retrieve
Predictive
Engine
Active
Profiling
• Predict Issues
• Prioritize
Treatments
Rules
Engine
Call Center
Knowledge Base
Agent
Assist
• Caller Identified
3
2
• Prioritized Potential Issue(s) Presented
to Agent
• Issue Selected, or where necessary,
researched
• Treatments for Issue are applied through
automation, or where necessary,
manually
• Cross or Up Sell
• Closure
1
Active Profile
Watson / Streaming
Analytics
Call Center
IVR
1
The Knowledge Base is compiled into an Active Profile
2
Customer transfers out of IVR to the call center and gets identified
3
Agent is provided with prioritized potential issues by the Rules Engine, together with treatments. Where needed the agent
can research the Knowledge Base. Treatment application is automated where possible, reducing manual time required
IBM Confidential
© 2011 IBM Corporation