Survey
* Your assessment is very important for improving the workof artificial intelligence, which forms the content of this project
* Your assessment is very important for improving the workof artificial intelligence, which forms the content of this project
Integrated Customer Call Information John Nysse Senior Consultant, SAS Institute Australia Overview The Customer (Optus) Business Needs SAS Applications @ Optus ICR, CRDS – Integration of data from both CRDS in more detail Questions Optus Optus is the second carrier and has about 25% mobile market share. They have about 3% churn per month (36% annually). ~2.6 million customers Optus Business Division Optus Mobile Business & Data Services (Sub Divison Enterprise) Consumer & Multimedia (includes residential and customer marketing) used in all of them Optus (Key Goals) A leading position in the data and IP based services Putting customers first Leading edge network (capacity and reach) Strategic Project possibilites Churn Prediction Segmentation analysis Monitor traffic flows and Switch performance Campaign Analysis Measurement of Service to Customers Calculate the true value of the customer (by recognising the value generated from incoming calls). @ Optus Transactions for Wholesale Customer Billing Lost & Stolen Handset tracking Call Record Data Store (CRDS) Integrated Customer Relationship facility (ICR) Network Message Handling Traffic Forecasting System Traffic Reporting Capacity Reporting User Applications Business Need (ICR) Customer segmentation Customer Value Cross Selling Churn Prediction and Modelling Campaign Analysis Target Marketing and Bundling Business Need (CRDS) Establish a database that stores the unrated CDRs for all switch types. Require inbound and outbound Call Detail Records (CDRs) for analysis at the cellular/service level. The inbound and outbound CDRs can be used to calculate the true value of the customer (by recognising the value generated from incoming calls). Business Need (CRDS) Projects that will use CRDS include… Mobile Churn Prediction Model Wireline Churn Prediction Model Mobile Segmentation analysis Monitor traffic flows and ensure switch performance Campaign Analysis – evaluating success of campaigns ICR Source Data Extraction & Validation Publish Business Metadata Staging Tables #1 Transformation & Staging Oracle DB & Load (load ready data) Data Extracts Staging Tables #2 Datamart Downstream Apps. CRDS CRDS DCS Downstream Apps. Ref. Data CRDS Time Job Monitoring (every 15 min) Read Input Directory User Window (8:30am – 7:00pm) Read Spawner Hourly (8:00am-7:00pm) Read Jobs Load Spawner 8:00pm (until finished) Load Jobs Rollback Spawner Rollback Jobs Create DataMarts FTP DataMarts to Users Housekeeping Jobs When load completed and only if load problems 2:00am Once DM’s created 6:00am Putting ICR & CRDS together ICR •Customer •Product •Billing & IBM CRDS •CDR •Call Only DM’s, Data Warehouse •Query •Campaign Management •Data Mining Prod&Consult. “Who was my largest customer in terms of billing and what Cell Site carries their greatest amount of traffic, but has the highest number of call drop-outs?”. CRDS In more detail What is a CDR? Call Detail Record (CDR) Contains information about a call Calling Party, Called Party, Start Time, Duration Written out by a Switch once the call terminates A call can be routed through many switches, so there many CDR’s per call. CRDS Store CDR’s for required Switch Types Store CDR’s for many days (up to 32 days required) Can handle the data volumes (Bytes, # of records) Perform Call Matching High Quality – Low Maintenance Application Read/Load time to be minimal Tough Constraints – H/W Budget, Project Timeframe CDRS Data Collection DMS100 (Local & National calls) MSC (Mobile calls) ESN (Extended NW Services – 13*, 1800*, 1900*) TOPS (Operator assisted calls) DMS300 (International calls) What is Call Matching? Matching all the CDR’s together belonging to a single call A Unique Call Key is allocated to each call Business Rule – Match on… A Party B Party | Time of Call | < 10 seconds | Call Duration | < 5 seconds Handling Data Volumes FILE_KEY = jjjnnnn (10K / Day) In Unix the largest number represented exactly is 253 = 9,007,199,254,740,992. CDR_KEY = jjjnnnnnnnnnnnnn (1B / Year) jjj = Current System Julian Day (0-366) nnnn = File Number (0-9,999) nnnnnnnnn = CDR Number (0-999,999,999) Handling Data Volumes CALL_KEY = yymmddnnnnnnnnnn (10B/Day) yymmdd = Aus. Eastern Standard Time nnnnnnnnnn = Sequentially allocated number (0-9,999,999,999) Handling Data Volumes 20 GB Input Data / day 50-55 Million CDR’s / day Read Time: ~1.5 hours Load Time: ~6.5 hours (majority of time doing Call Matching) Without Call Matching can Read & Load < 2.5 hours Read Process Switch data from DCS2000 (Hourly) Input File Directories Read Spawner (Hourly) Read Input File Directory (Hourly) O/S Job Monitoring Read File 1... Reference Data Read File N Control Tables Intermediate Area Day 1 (S1) Day 1 (S2) Day N (S1) Read Process File #1, Date 03/11/2000 Structure 11 Module 1 Module 2 000 File #2, Date 03/11/2000 Structure 12 Module 1 Module 3 IN_20001103 Structure 11 Structure 12 Module 1 Module 1 Store in Directory based Store Directory oninAEST Date based on AEST Date Module 2 Module 3 CDR CDR Intermediate Area READ Job READ Job Create a Unique CDR Key Create a Unique CDR Key 000 Reading Input Files Input File Area F1/S2/D1 F1/M1/D1 F1/M3/D1 F1/S1/D2 F1/M1/D2 F1/S1/D1 F1/M1/D1 F1/M2/D2 F2/S1/D3 F2/M1/D3 F2/M3/D3 F2/S1/D2 F2/M1/D2 F2/S2/D1 F2/M2/D2 F2/M1/D1 F2/M3/D1 Intermediate Area Day 1 F2/M3 F1/M3 F1/S1 F1/M1 F1/M1 F2/M1 Day 2 F1/S2 F1/S1 F2/S1 F2/S2 F1/M1 Day 3 F2/M1 F2/M2 F1/M2 F2/S1 F2/M3 F2/M1 Reading Input Files Input File Area F1/S2/D1 F1/M1/D1 F1/M3/D1 F1/S1/D2 F1/M1/D2 F1/S1/D1 F1/M1/D1 F1/M2/D2 F2/S1/D3 F2/M1/D3 F2/M3/D3 F2/S1/D2 F2/M1/D2 F2/S2/D1 F2/M2/D2 F2/M1/D1 F2/M3/D1 Intermediate Area Day 1 F2/M3 F1/M3 F1/S1 F1/M1 F1/M1 F2/M1 Day 2 F1/S2 F1/S1 F2/S1 F2/S2 F1/M1 Day 3 F2/M1 F2/M2 F1/M2 F2/S1 F2/M3 F2/M1 Load Process Load Spawner (Nightly) O/S Job Monitoring Control Tables Load Job Day 1… Intermediate Area Day 1 (S1) Rollback Spawner (Nightly) Day 1 (S2) Load Job Day N Rollback Job File 1… Rollback Job File N Staging Area Day N (S1) Day 1 Day 2 Day 3 Day N Load into Staging Area Intermediate Area Day 1 F2/M3 F1/M3 F1/S1 F1/M1 F1/M1 F2/M1 Day 2 F1/S2 F2/S2 F2/M1 F1/S1 F2/S1 Day 3 F2/M2 F1/M1 F2/S1 F2/M3 F2/M1 F1/M2 Staging Area Day 1 S1 M3 M3 M1 M1 M1 Day 2 S2 S2 S1 S1 Day 3 M2 M1 M1 S1 M1 M3 Load into Staging Area Intermediate Area Day 1 F2/M3 F1/M3 F1/S1 F1/M1 F1/M1 F2/M1 Day 2 F1/S2 F2/S2 F2/M1 F1/S1 F2/S1 Day 3 F2/M3 F2/M2 F1/M1 F2/S1 F2/M1 F1/M2 Staging Area Day 1 S1 M3 M3 M1 M1 M1 Day 2 S2 S2 S1 S1 M1 M1 Day 3 M2 M2 S1 M1 M3 Number of Input Files (per Day) F1/S2/D1 F1/M1/D1 F1/S1/D2 F1/M1/D2 F1/S1/D1 F1/M1/D1 ~ 270 F1/M3/D1 F1/M2/D2 F2/M1/D3 F2/S1/D2 F2/M1/D2 F2/S2/D1 Day 1 F2/M3 F2/S1/D3 F2/M1/D1 F2/M3/D3 F2/M2/D2 F2/M3/D1 Day 2 F1/S2 Day 3 F2/M1 F1/S1 F2/S1 F1/M1 F1/M3 F1/S1 F2/M3 F1/M1 F2/M1 F2/S1 F2/S2 F1/M1 Day 1 M1 M3 F1/M2 S1 M1 M1 F2/M1 Day 2 S1 M3 F2/M2 S2 S2 S1 Day 3 M2 M1 M1 S1 M1 M3 Max. # of Intermediate datasets (per Day) F1/S2/D1 F1/M1/D1 F1/M1/D2 F1/S1/D1 F1/M1/D1 Day 1 F2/M3 F1/S1 F1/S2 F1/M1 F2/M1 F1/M2/D2 F1/S1 F2/S2 F2/M1/D1 F2/M3/D3 F2/M2/D2 F2/M3/D1 Day 3 F2/S1 F2/M3 F1/M1 F2/M1 F1/M2 Day 2 M1 M3 F2/M1/D2 F2/M2 S1 M1 F2/S1/D2 F2/M1 F2/S1 S1 M3 F2/M1/D3 F2/S2/D1 Day 1 M1 F2/S1/D3 Day 2 F1/M1 F1/M3 ~ 270 ~ 8,640 F1/S1/D2 F1/M3/D1 S2 S2 S1 Day 3 M2 M1 M1 S1 M1 M3 Number of Staging Area datasets (per Day) F1/S2/D1 F1/M1/D1 F1/M1/D2 F1/S1/D1 F1/M1/D1 Day 1 F2/M3 F1/S1 F1/S2 F1/M1 F2/M1 F1/M2/D2 F1/S1 F2/S2 F2/M1/D1 F2/M3/D3 F2/M2/D2 F2/M3/D1 Day 3 F2/S1 F2/M3 F2/M1 F1/M2 Day 2 M1 M3 F2/M1/D2 F2/M2 F1/M1 S1 M1 F2/S1/D2 F2/M1 F2/S1 S1 M3 F2/M1/D3 F2/S2/D1 Day 1 M1 F2/S1/D3 Day 2 F1/M1 F1/M3 ~ 270 ~ 8,640 F1/S1/D2 F1/M3/D1 S2 S2 S1 32 M1 M1 Day 3 M2 S1 M1 M3 Reporting Process (DataMarts) O/S Staging Area Day 1.. Job Monitoring Selected DM Day Control Tables Reference Data DataMart Job DataMarts Mobile Originating Mobile Terminating Wireline Originating Wireline Terminating Control Tables Gives a snapshot and history of the application processing Allows subsequent processes to know how previous processes have completed Analysis of Job Run stats (plus other externally gathered stats) allow performance analysis to be carried out. Reference Data Contains Business Rules used to Mediate the input data. Convert Local Time of call to Australia Eastern Standard Time Convert a Wireline Phone Number or Cell Site into Latitude & Longitude Coordinates and therefore calculate the distance between the two parties Determine the Provider (e.g. Optus, Telstra, etc) Determine the Service (e.g. GSM, AMPS, International) SAS Software Base (Macro), Standard SAS Datasets Warehouse Admin – Manage Data Marts / Extracts Enterprise Miner – Mining SAS Access – Input Reference Data, User Queries SPD Server – Speed up load & user queries Machine Sizing Alpha Digital GS140 6/700 8 CPUs (expandable to 14) 6 GB RAM (expandable to 28) 2 RAID Controllers ~1 TB Disk Space (63 * 18.2GB disk) Time to Deliver Duration (Jan 2000 – Feb 2001) H/W delays (3 months) Person Hours (4,800) ~ 3 Person Years FRS/Design (22%) Coding / Testing (44%) Implementation / User Doc (7%) Project Management / Meetings (26%) Handover (1%) What took the time Reference Data (2 person months extra) Difference between User App VS Corporate App Loose connection to suppliers (No Interface Contracts) Needed to…Find stable sources, Verify data, Cope with Manually maintained data, Cope with Obsolete upstream Apps. More up-front Business Analysis would have helped. What took the time Call Matching Rules (2 person months extra) Knowledge of the Data & Busines Rules spread out over the client organisation Little or no documentation available Huge data volumes, Slow turn-around in testing runs Relied on users words Better up-front Business Analysis would have helped. Data Mining to find relationships. Lessons Learned Re-quote after each phase… +/- 60% after SOW completed +/- 40% after FRS completed +/- 15% after Design completed Modularise more (plug & play) Expandability (CDR_Key > 1 billion CDR’s / Day) Better Scoping/Bus. Analysis. (Lack of Interface Contracts/Automation a warning sign). CRDS I/O Bound I/O Bound The Future More Switch Types, Structure & Modules e.g. SMSC Structure 000013 – MSC Incoming Gateway Modules 18 & 20 – Mobile Number Portability Reference data (where possible) supplied by SRDM. Improve Call Matching Algorithm The Future Performance Improvements (boost capacity and speed to meet user requirements). Additional Disk, Disk Controllers, Memory & Cabling. Add Indexing and SPD Server (SPD Server: Tight Security, Partitioned Tables, Multi-threads per user, Advanced Indexing) New downstream Apps coming along. Any Questions