Download Good, Fast, and Cost Effective Priceline Tries to Do All Three

Good, Fast, and Cost Effective
Priceline Tries to Do All Three
Ron Rose
Chief Information Officer
SIM CT Chapter
April 10, 2003
Slide Number CONFIDENTIAL INFORMATION • Property of priceline.com incorporated. All Right Reserved.
7/7/2017
Agenda
•
Priceline’s Speed and Execution
•
The Data Center Operating System
•
Web Services (Good News, Bad News)
•
Technological Excellence
•
Portfolio Management
•
Expense Reduction Steps
•
Offshore Lessons
•
IT Goals Framework
Slide Number CONFIDENTIAL INFORMATION • Property of priceline.com incorporated. All Right Reserved.
7/7/2017
Slide 2
The N-Tier Ecosystem Challenge
•
•
•
•
•
•
•
•
•
•
•
•
Hundreds of Boxes
Separate Operating Systems
Multiple Tiers
Multiple Security Architectures
No control of objects among the tiers
No synchronization of the object/database interface
Very difficult diagnostics between tiers
Usually different monitoring tools among tiers
Intra-tier failures and the correlation among them is problematic
No synchronized software distribution among tiers
No synchronized rollback and recovery among tiers
Configuration management among architectures is problematic
In other words, if this was sold as one product, no CIO would buy it!
Slide Number CONFIDENTIAL INFORMATION • Property of priceline.com incorporated. All Right Reserved.
7/7/2017
Slide 3
Slide Number CONFIDENTIAL INFORMATION • Property of priceline.com incorporated. All Right Reserved.
7/7/2017
Configuration Enforcement
(Bladelogic, Altiris, etc.)
Software Distribution (Bladelogic, etc.)
Diagnostic and Alerting
(Wily, Xtremesoft, BMC, Relicore, etc.)
Management Tool Integration
(HyperNOC, Netuitive, etc.)
Transaction Integrity (Bristol, etc.)
Security (Approva.net, etc.)
Work Distribution & Control
The Data Center Operating System
Slide 4
Web Services Lessons
•
Just because it’s called “SOAP”, doesn’t necessarily mean it’s clean
—Just enough holes to keep it from inter-operating cleanly
•
The “X” in XML is for Xtensible, not Xtremely fast
—Not designed for high velocity interfaces
—Bandwidth-intensive to pass around
—CPU-intensive to parse
—Memory-intensive to parse
•
Loosely Coupled – guaranteed messages are a problem – this is not MQ
Series
—If timed-out, did you get message, was the data received, should you retry?
—Credit card Authorizations - need to build duplication detection at the target site to
prevent duplicate authorizations
—Reliable web services will address this, but specifications are still evolving
•
Tightly Connecting to Suppliers means your uptime is the aggregate of
theirs
•
Continuous Reporting/Alerting/Diagnostics become very important
Slide Number CONFIDENTIAL INFORMATION • Property of priceline.com incorporated. All Right Reserved.
7/7/2017
Slide 5
Web Services Lessons, The Good News
•
Security – Very Mature
—Supported and Maintainable without programming; common tools
• SSL – privacy for data
• Authorization – use digital certificates
• Network – TCP segments
• Basic password – username/password
•
Fully leverage Web Infrastructures
—SSL accelerators
—HTTP load balancers
—Proxy servers
Slide Number CONFIDENTIAL INFORMATION • Property of priceline.com incorporated. All Right Reserved.
7/7/2017
Slide 6
Technological Excellence – Web and Database Architecture
•
•
•
•
•
Web/DB transactional speed and high levels of availability (>99.99%) that are
fully competitive with Airline/Mainframe levels
Web Architecture which masks failures, Session State Servers
Web Architecture which finds surviving database, DPM
Proven Scalability - 20 million customers+
One of the most scalable and reliable database architectures in the Fortune
1000
— Primary Secondary Tertiary Quaternary, 2 forms of replication
— Horizontally Scalable Replication
— SUN link is
http://dcb.sun.com/practices/casestudies/priceline_part1.jsp
— 3 6800’s on line and usable by middle tier at any time
— ETL pulls and data warehouse queries don’t compromise the speed
or integrity of the primary or secondary DB servers
— Rolling upgrades supported without downtime
Slide Number CONFIDENTIAL INFORMATION • Property of priceline.com incorporated. All Right Reserved.
7/7/2017
Slide 7
Technological Excellence - Disaster Recovery Architecture
• Two Hot sites, not merely a theoretical failover site
• Load Balancing across both sites, using BGP, to insure both sites are
truly operational
• Each site is on a separate power grid & separate communications
grid
• Sites are connected with redundant 8 GB dark fiber connectivity
across diverse paths
•
•
•
•
Tertiary Quaternary customer databases at 2nd Site
Redundant forms of Redundancy in replication to databases
ATT to one building, MCI to the other, high speed network between
Disk SAN, Tape backup, operations monitoring all operate as though
local to one site
• Remote connectivity, with security, allow remote support if NOC
building is compromised
Slide Number CONFIDENTIAL INFORMATION • Property of priceline.com incorporated. All Right Reserved.
7/7/2017
Slide 8
Portfolio Management – Capital
•
•
•
•
•
•
•
•
•
•
Most significant problem for Internet CIO’s
Candidate is brought forward under review process, PIF
Passes ROI examination
Revenue
Costs (Development + Maintenance + Customer Service +
Marketing)
Man-Days (as function of cost And complexity)
Estimated Margin
Applies to all projects greater than 50 man-days or $100k
Ask the Business folks to comply Every project
Ask the Business folks to comply Audit Every result
Slide Number CONFIDENTIAL INFORMATION • Property of priceline.com incorporated. All Right Reserved.
7/7/2017
Slide 9
The ROI of MTTR Reduction
•
•
•
•
•
Determine your Cost/Hour for each major component failure
Determine how many incidents per month and average duration
Understand your MTTR
– Alerting Lag
– Diagnostic Time for NOC Level One Support
– Escalation Lag for Level Two Engineering Call
– Engineering Time to Respond
– Diagnostic Time for Level Two Engineering Support
– Action Time for Level Two Engineer to recover the major
component
Analyze the ROI for the toolsets
– Predict which tools can help which step above to reduce MTTR
– Predict the reduction in incidents, and thus manpower
Keep an eye out for automation to recover from the outage
automatically
Slide Number CONFIDENTIAL INFORMATION • Property of priceline.com incorporated. All Right Reserved.
7/7/2017
Slide 10
Top Ten List of Expense Reduction Steps
•
•
•
•
•
•
•
•
•
•
Prioritize by highest ROI
Treat expense reduction like a project/process
– PM, Accountant, Negotiator, Vendor Coordinator, etc.
– Centralize contracts around your best negotiator
Good requirements (eliminate confusion on number of CPUs, etc.)
Competitive Knowledge (Gartner)
Competitive Bidding (DB2)
Competitive Threats (Gartner  DB2 & Redbrick)
Get ready early (Last week before end of Quarter/Year
– Take 3 weeks to 3 months to prepare for that week
Subtract any irrelevant intrinsic product features/services
Develop a training methodology
Always “Name Your Own Price”
Slide Number CONFIDENTIAL INFORMATION • Property of priceline.com incorporated. All Right Reserved.
7/7/2017
Slide 11
Offshore Lessons Learned – Best Practices
•
Vendor Selection is Very Important
— Large enough for adequate bench strength, (>1000)
— Multiple capabilities (customer service)
— Hiring and recurrent training methodology
•
Use technology to help you
— Workflow software
— Web-enabled Project Management software
— Secure Network topology
— VPN’s that access Source Control, (SecurID can help)
— VOIP phones, Cheap Videoconference tools
Slide Number CONFIDENTIAL INFORMATION • Property of priceline.com incorporated. All Right Reserved.
7/7/2017
Slide 12
Offshore Lessons Learned – Best Practices (con’t)
Anticipate the Inefficiencies (and adjust for them)
•
Combination of Onsite and Offshore initially for Knowledge Transfer
•
Cross train India resources in America
•
Team reverts to India and develops
•
Offshore Program Management Office at Priceline.com
•
Priceline Lead at HCL’s Offshore Center
•
Weekly Status Reports & Reviews
•
Send American managers to India
•
Train American managers to build relationships
Slide Number CONFIDENTIAL INFORMATION • Property of priceline.com incorporated. All Right Reserved.
7/7/2017
Slide 13
Offshore Lessons Learned – Common Myths
•
•
•
•
Traditional Perceptions
— The savings are illusory due to inefficiencies
Priceline Experience
— Over $25 million saved thus far among all 5 companies
Traditional Perception
— “Offshore partners can only can do maintenance, simple projects,
legacy projects well. Not strategic partners.”
Priceline Experience
— Rapid Development and Deployment is possible
— Offshore extended priceline development capacity and helped
strengthen the company
— Offshore has increased sharing of priceline IP among affiliate
companies
— Our offshore strategy/partner has been fundamental to our
continued success
Slide Number CONFIDENTIAL INFORMATION • Property of priceline.com incorporated. All Right Reserved.
7/7/2017
Slide 14
IT Goals Framework (for enhancing Shareholder Value)
1. Build and Launch New Products
A. Help with the Business Prioritization Process
(Do the right things)
B. Increase the effectiveness of product development
(Do the right thing, the right way)
B1. Increase Development Speed
B2. Increase QA Speed
B3. Increase teamwork within IT
B4. Increase teamwork between IT and Business
C. Project Management Archaeology
(Estimate and control effectively)
Slide Number CONFIDENTIAL INFORMATION • Property of priceline.com incorporated. All Right Reserved.
7/7/2017
Slide 15
IT Goals Framework (for enhancing Shareholder Value)
2. Exceed Financial Targets
A. Prioritize contracts and renegotiate each downward
B. Examine supply chain for meaningful optimizations
3. Maintain Stability and Scalability Needed
A. Anticipate business and initiatives
B. Progress DR initiatives
4. Enhance Professional Development
5. Increase Security
A. Enhance security from attacks from the external networks
B. Enhance security form attacks from the internal networks
C. Progress customer data security
Slide Number CONFIDENTIAL INFORMATION • Property of priceline.com incorporated. All Right Reserved.
7/7/2017
Slide 16
Q&A
Slide Number CONFIDENTIAL INFORMATION • Property of priceline.com incorporated. All Right Reserved.
7/7/2017
Slide 17