Download Integrated Customer Call Information

Integrated Customer Call
Information
John Nysse
Senior Consultant, SAS Institute Australia
Overview
The Customer (Optus)
Business Needs
SAS Applications @ Optus
ICR, CRDS – Integration of data from both
CRDS in more detail
Questions
Optus
Optus is the second carrier and has about 25%
mobile market share.
They have about 3% churn per month (36%
annually).
~2.6 million customers
Optus Business Division
Optus Mobile
Business & Data Services (Sub Divison Enterprise)
Consumer & Multimedia (includes residential and
customer marketing)
used in all of them
Optus (Key Goals)
A leading position in the data and IP based
services
Putting customers first
Leading edge network (capacity and reach)
Strategic Project possibilites
Churn Prediction
Segmentation analysis
Monitor traffic flows and Switch performance
Campaign Analysis
Measurement of Service to Customers
Calculate the true value of the customer (by
recognising the value generated from incoming calls).
@ Optus
Transactions for Wholesale Customer Billing
Lost & Stolen Handset tracking
Call Record Data Store (CRDS)
Integrated Customer Relationship facility (ICR)
Network Message Handling
Traffic Forecasting System
Traffic Reporting
Capacity Reporting
User Applications
Business Need (ICR)
Customer segmentation
Customer Value
Cross Selling
Churn Prediction and Modelling
Campaign Analysis
Target Marketing and Bundling
Business Need (CRDS)
Establish a database that stores the unrated
CDRs for all switch types.
Require inbound and outbound Call Detail
Records (CDRs) for analysis at the
cellular/service level.
The inbound and outbound CDRs can be used to
calculate the true value of the customer (by
recognising the value generated from incoming
calls).
Business Need (CRDS)
Projects that will use CRDS include…
Mobile Churn Prediction Model
Wireline Churn Prediction Model
Mobile Segmentation analysis
Monitor traffic flows and ensure switch performance
Campaign Analysis – evaluating success of
campaigns
ICR
Source
Data
Extraction
&
Validation
Publish
Business
Metadata
Staging Tables #1
Transformation
& Staging
Oracle DB & Load
(load ready data)
Data Extracts
Staging Tables #2
Datamart
Downstream Apps.
CRDS
CRDS
DCS
Downstream Apps.
Ref.
Data
CRDS
Time
Job Monitoring (every 15 min)
Read Input Directory
User
Window
(8:30am –
7:00pm)
Read Spawner
Hourly
(8:00am-7:00pm)
Read Jobs
Load Spawner
8:00pm (until finished)
Load Jobs
Rollback Spawner
Rollback Jobs
Create DataMarts
FTP DataMarts to Users
Housekeeping Jobs
When load completed
and only if load problems
2:00am
Once DM’s created
6:00am
Putting ICR & CRDS together
ICR
•Customer
•Product
•Billing
& IBM
CRDS
•CDR
•Call
Only
DM’s, Data Warehouse
•Query
•Campaign Management
•Data Mining
Prod&Consult.
“Who was my largest customer in
terms of billing and what Cell Site
carries their greatest amount of
traffic, but has the highest number
of call drop-outs?”.
CRDS
In more detail
What is a CDR?
Call Detail Record (CDR)
Contains information about a call
Calling Party, Called Party, Start Time, Duration
Written out by a Switch once the call terminates
A call can be routed through many switches, so there
many CDR’s per call.
CRDS
Store CDR’s for required Switch Types
Store CDR’s for many days (up to 32 days required)
Can handle the data volumes (Bytes, # of records)
Perform Call Matching
High Quality – Low Maintenance Application
Read/Load time to be minimal
Tough Constraints – H/W Budget, Project Timeframe
CDRS Data Collection
DMS100 (Local & National calls)
MSC
(Mobile calls)
ESN
(Extended NW Services – 13*, 1800*, 1900*)
TOPS
(Operator assisted calls)
DMS300 (International calls)
What is Call Matching?
Matching all the CDR’s together belonging to a
single call
A Unique Call Key is allocated to each call
Business Rule – Match on…
A Party
B Party
| Time of Call | < 10 seconds
| Call Duration | < 5 seconds
Handling Data Volumes
FILE_KEY
= jjjnnnn
(10K / Day)
In Unix the largest number represented exactly is
253 = 9,007,199,254,740,992.
CDR_KEY
= jjjnnnnnnnnnnnnn
(1B / Year)
jjj
= Current System Julian Day (0-366)
nnnn
= File Number (0-9,999)
nnnnnnnnn = CDR Number (0-999,999,999)
Handling Data Volumes
CALL_KEY = yymmddnnnnnnnnnn (10B/Day)
yymmdd
= Aus. Eastern Standard Time
nnnnnnnnnn = Sequentially allocated number
(0-9,999,999,999)
Handling Data Volumes
20 GB Input Data / day
50-55 Million CDR’s / day
Read Time: ~1.5 hours
Load Time: ~6.5 hours (majority of time doing Call
Matching)
Without Call Matching can Read & Load < 2.5 hours
Read Process
Switch
data from
DCS2000
(Hourly)
Input File
Directories
Read
Spawner
(Hourly)
Read
Input File
Directory
(Hourly)
O/S
Job
Monitoring
Read
File 1...
Reference
Data
Read
File N
Control
Tables
Intermediate Area
Day 1 (S1)
Day 1 (S2)
Day N (S1)
Read Process
File #1, Date 03/11/2000
Structure 11
Module 1
Module 2
000
File #2, Date 03/11/2000
Structure 12
Module 1
Module 3
IN_20001103
Structure 11
Structure 12
Module 1
Module 1
Store in Directory based
Store
Directory
oninAEST
Date based
on AEST Date
Module 2
Module 3
CDR
CDR
Intermediate Area
READ Job
READ Job
Create a Unique
CDR Key
Create a Unique CDR Key
000
Reading Input Files
Input File Area
F1/S2/D1
F1/M1/D1 F1/M3/D1
F1/S1/D2
F1/M1/D2
F1/S1/D1
F1/M1/D1
F1/M2/D2
F2/S1/D3
F2/M1/D3 F2/M3/D3
F2/S1/D2
F2/M1/D2
F2/S2/D1
F2/M2/D2
F2/M1/D1 F2/M3/D1
Intermediate Area
Day 1
F2/M3
F1/M3
F1/S1
F1/M1
F1/M1
F2/M1
Day 2
F1/S2
F1/S1
F2/S1
F2/S2
F1/M1
Day 3
F2/M1
F2/M2
F1/M2
F2/S1
F2/M3
F2/M1
Reading Input Files
Input File Area
F1/S2/D1
F1/M1/D1 F1/M3/D1
F1/S1/D2
F1/M1/D2
F1/S1/D1
F1/M1/D1
F1/M2/D2
F2/S1/D3
F2/M1/D3 F2/M3/D3
F2/S1/D2
F2/M1/D2
F2/S2/D1
F2/M2/D2
F2/M1/D1 F2/M3/D1
Intermediate Area
Day 1
F2/M3
F1/M3
F1/S1
F1/M1
F1/M1
F2/M1
Day 2
F1/S2
F1/S1
F2/S1
F2/S2
F1/M1
Day 3
F2/M1
F2/M2
F1/M2
F2/S1
F2/M3
F2/M1
Load Process
Load
Spawner
(Nightly)
O/S
Job
Monitoring
Control
Tables
Load Job
Day 1…
Intermediate Area
Day 1 (S1)
Rollback Spawner
(Nightly)
Day 1 (S2)
Load Job
Day N
Rollback Job
File 1…
Rollback Job
File N
Staging Area
Day N (S1)
Day 1
Day 2
Day 3
Day N
Load into Staging Area
Intermediate Area
Day 1
F2/M3
F1/M3
F1/S1
F1/M1
F1/M1
F2/M1
Day 2
F1/S2
F2/S2
F2/M1
F1/S1
F2/S1
Day 3
F2/M2
F1/M1
F2/S1
F2/M3
F2/M1
F1/M2
Staging Area
Day 1
S1
M3
M3
M1
M1
M1
Day 2
S2
S2
S1
S1
Day 3
M2
M1
M1
S1
M1
M3
Load into Staging Area
Intermediate Area
Day 1
F2/M3
F1/M3
F1/S1
F1/M1
F1/M1
F2/M1
Day 2
F1/S2
F2/S2
F2/M1
F1/S1
F2/S1
Day 3
F2/M3
F2/M2
F1/M1
F2/S1
F2/M1
F1/M2
Staging Area
Day 1
S1
M3
M3
M1
M1
M1
Day 2
S2
S2
S1
S1
M1
M1
Day 3
M2
M2
S1
M1
M3
Number of Input Files (per Day)
F1/S2/D1
F1/M1/D1
F1/S1/D2
F1/M1/D2
F1/S1/D1
F1/M1/D1
~ 270
F1/M3/D1
F1/M2/D2
F2/M1/D3
F2/S1/D2
F2/M1/D2
F2/S2/D1
Day 1
F2/M3
F2/S1/D3
F2/M1/D1
F2/M3/D3
F2/M2/D2
F2/M3/D1
Day 2
F1/S2
Day 3
F2/M1
F1/S1
F2/S1
F1/M1
F1/M3
F1/S1
F2/M3
F1/M1
F2/M1
F2/S1
F2/S2
F1/M1
Day 1
M1
M3
F1/M2
S1
M1
M1
F2/M1
Day 2
S1
M3
F2/M2
S2
S2
S1
Day 3
M2
M1
M1
S1
M1
M3
Max. # of Intermediate datasets (per Day)
F1/S2/D1
F1/M1/D1
F1/M1/D2
F1/S1/D1
F1/M1/D1
Day 1
F2/M3
F1/S1
F1/S2
F1/M1
F2/M1
F1/M2/D2
F1/S1
F2/S2
F2/M1/D1
F2/M3/D3
F2/M2/D2
F2/M3/D1
Day 3
F2/S1
F2/M3
F1/M1
F2/M1
F1/M2
Day 2
M1
M3
F2/M1/D2
F2/M2
S1
M1
F2/S1/D2
F2/M1
F2/S1
S1
M3
F2/M1/D3
F2/S2/D1
Day 1
M1
F2/S1/D3
Day 2
F1/M1
F1/M3
~ 270
~ 8,640
F1/S1/D2
F1/M3/D1
S2
S2
S1
Day 3
M2
M1
M1
S1
M1
M3
Number of Staging Area datasets (per Day)
F1/S2/D1
F1/M1/D1
F1/M1/D2
F1/S1/D1
F1/M1/D1
Day 1
F2/M3
F1/S1
F1/S2
F1/M1
F2/M1
F1/M2/D2
F1/S1
F2/S2
F2/M1/D1
F2/M3/D3
F2/M2/D2
F2/M3/D1
Day 3
F2/S1
F2/M3
F2/M1
F1/M2
Day 2
M1
M3
F2/M1/D2
F2/M2
F1/M1
S1
M1
F2/S1/D2
F2/M1
F2/S1
S1
M3
F2/M1/D3
F2/S2/D1
Day 1
M1
F2/S1/D3
Day 2
F1/M1
F1/M3
~ 270
~ 8,640
F1/S1/D2
F1/M3/D1
S2
S2
S1
32
M1
M1
Day 3
M2
S1
M1
M3
Reporting Process (DataMarts)
O/S
Staging Area
Day 1..
Job
Monitoring
Selected DM Day
Control
Tables
Reference
Data
DataMart Job
DataMarts
Mobile
Originating
Mobile
Terminating
Wireline
Originating
Wireline
Terminating
Control Tables
Gives a snapshot and history of the application
processing
Allows subsequent processes to know how previous
processes have completed
Analysis of Job Run stats (plus other externally
gathered stats) allow performance analysis to be
carried out.
Reference Data
Contains Business Rules used to Mediate the input data.
Convert Local Time of call to Australia Eastern Standard
Time
Convert a Wireline Phone Number or Cell Site into Latitude
& Longitude Coordinates and therefore calculate the
distance between the two parties
Determine the Provider (e.g. Optus, Telstra, etc)
Determine the Service (e.g. GSM, AMPS, International)
SAS Software
Base (Macro), Standard SAS Datasets
Warehouse Admin – Manage Data Marts / Extracts
Enterprise Miner – Mining
SAS Access – Input Reference Data, User Queries
SPD Server – Speed up load & user queries
Machine Sizing
Alpha Digital GS140 6/700
8 CPUs (expandable to 14)
6 GB RAM (expandable to 28)
2 RAID Controllers
~1 TB Disk Space (63 * 18.2GB disk)
Time to Deliver
Duration (Jan 2000 – Feb 2001)
H/W delays (3 months)
Person Hours (4,800) ~ 3 Person Years
FRS/Design (22%)
Coding / Testing (44%)
Implementation / User Doc (7%)
Project Management / Meetings (26%)
Handover (1%)
What took the time
Reference Data (2 person months extra)
Difference between User App VS Corporate App
Loose connection to suppliers (No Interface Contracts)
Needed to…Find stable sources, Verify data, Cope with
Manually maintained data, Cope with Obsolete upstream
Apps.
More up-front Business Analysis would have helped.
What took the time
Call Matching Rules (2 person months extra)
Knowledge of the Data & Busines Rules spread out over
the client organisation
Little or no documentation available
Huge data volumes, Slow turn-around in testing runs
Relied on users words
Better up-front Business Analysis would have helped.
Data Mining to find relationships.
Lessons Learned
Re-quote after each phase…
+/- 60% after SOW completed
+/- 40% after FRS completed
+/- 15% after Design completed
Modularise more (plug & play)
Expandability (CDR_Key > 1 billion CDR’s / Day)
Better Scoping/Bus. Analysis. (Lack of Interface
Contracts/Automation a warning sign).
CRDS I/O Bound
I/O Bound
The Future
More Switch Types, Structure & Modules
e.g. SMSC
Structure 000013 – MSC Incoming Gateway
Modules 18 & 20 – Mobile Number Portability
Reference data (where possible) supplied by SRDM.
Improve Call Matching Algorithm
The Future
Performance Improvements (boost capacity and speed to
meet user requirements).
Additional Disk, Disk Controllers, Memory & Cabling.
Add Indexing and SPD Server (SPD Server: Tight
Security, Partitioned Tables, Multi-threads per user,
Advanced Indexing)
New downstream Apps coming along.
Any Questions