Download DevOps_Cloud_Future_DBA

The Future of The DBA
DevOps, the Cloud Paradigm, & the Microsoft Data Platform
Stuart R Ainsworth
http://codegumbo.com
Agenda
Describe the typical state of database administration
Define & describe DevOps and the cloud computing paradigm
Explore (high-level) the Microsoft Data Platform
Discuss the implications for data professionals
Assumptions About You
SQL Server experience
Exposure to database admin & architecture
Learning-centered
Desire to build modern skills
About Me
Manage IT for financial services company
Former Data Architect, DBA, developer
AtlantaMDF Chapter Leader
Infrequent blogger: http://codegumbo.com
My career
trajectory
You have brains in your head
And SQL Skills to boot
You'll soar to great heights
On the Data Platform too
You're on your own, and you know what you know,
And YOU are the one who'll decide where to go.
http://bit.ly/2j8Wi5z
Where Are We?
Typical SQL Server Person
Typical SQL Server Person
• Specialized knowledge
• May have exposure to other infrastructure or development tools
• Usually reports to Operations (not Development)
• Often one of the “last stops” in deployment chain
Typical SQL Server Person
Development Skills
Administration Skills
SQL (DDL & DML)
Server Configuration
Performance Tuning
(Code)
Performance Tuning
(Server)
Index Analysis
Data Warehousing
Reporting
Index Maintenance
Backups and Restores
Security
MCSA SQL 2012\2014
• 70-461: Querying Microsoft SQL Server 2012/2014
• 70-462: Administering Microsoft SQL Server 2012/2014 Databases
• 70-463: Implementing a Data Warehouse with Microsoft SQL Server
2012/2014
What’s coming…
• Data production is accelerating
• Est .79ZB in 2009
• Est 7.9ZB in 2015
• Est 35ZB in 2020 (44 times greater than 2009)
• Data is diversifying
•
•
•
•
•
•
Relational Data
Big (Size) Data
Fast Data
Dark Data
Lost Data
New Data
DevOps
Philosophy, not Methodology
A (Very) Brief Overview
• DevOps is focused on delivering quality, faster.
• Philosophical approach, not methodological
• Automation, infrastructure as code, continuous deployment
• Emphasis on communication; silo reduction
• Born out of Agile, several innovators contributing
• Patrick Debois & Andrew Clay Shafer – Agile Infrastructure (Agile 08)
• John Allspaw & Paul Hammond – 10+ Deploys Per Day (Velocity 09)
• Gene Kim, Kevin Behr, & George Spafford – The Phoenix Project
The Phoenix Project
The Phoenix Project
The Phoenix Project
Unicorns, Horses, and Mules
• Unicorns are sparkly, magical
companies that do amazing
things with DevOps
• Horses are the typical enterprise;
strong in some areas, always
looking to improve.
• Mules are conservative; slow and
steady, reluctant to change.
Key Takeaways
• DevOps is rooted in a sense of continuous improvement
• People over tools
• Reduce silos by focusing on shared goals, not technology
• Technology spans function
• Goals fulfill function; method matters less
The Cloud Paradigm
Infrastructure, Platform, Software
What do we mean by “The Cloud”?
• Trendy marketing term?
• Network hosting?
• Internet connected services?
Distributed, scalable, shared computing resources
Product-Focused Paradigm
Cloud Paradigm
Applications
Data
Runtime
Middleware
O/S
Virtualization
Servers
Storage
Networking
Cloud Paradigm
Cloud Paradigm
Cloud Paradigm
Limoncelli, Chalup, Hogan
http://the-cloud-book.com/
Ideal System Architecture
• Scalable
• Resilient to Failure (Redundant)
• Service-Oriented Architecture
• Automated Monitoring, Configuration and Build
• “Infrastructure As Code”
Ideal Release Process
• Completely Automatic
•
•
•
•
•
Code checked in -> new build
Unit & regression testing
User acceptance testing
Continuous Integration
Dependent on “infrastructure as code”
• Micro-releases (100 deployments per day)
• No rollbacks
Ideal Operations
• Automatic instrumentation (Logging)
• Long Term Storage
• Predictive Analytics
• Automatic Error Logging & Alerting
• Respond to Every Error
• On-call Rotation includes Developers
• Automatic Scaling
• Scale Up
• Scale Down
• “Zero Maintenance”
Ideal Data Architecture
• CAP Principle (Gilbert & Lynch)
• Consistency - all nodes are guaranteed to see same data
• Availability – every request receives feedback for success/failure
• Partition Tolerant – system operates despite loss of part of system
• At any one time, any two attributes are achievable in combination, but
not all three at the same time.
CAP Principle
Consistent
SQL Server
Relational
Engine
Available
Hadoop
Cassandra
Partition Tolerant
Microsoft Data Platform
Diversity in Data
Microsoft Data Platform
Microsoft Data Platform
Microsoft Data Platform
Impact on Careers
Future Prognostications
Current Trends
• Companies are recognizing the value of different kinds of data, and
an increased need for analytics.
• Adoption of Big Data technologies is on the rise
• Data Science jobs are increasing
• The Internet (or Fog) of Things is coming.
• Operational methodologies like Agile and DevOps are pushing
companies toward the Cloud Paradigm.
Current Trends
• The Cloud Paradigm with its separation of duties is causing
companies to realign resources.
• Infrastructure Teams
• Platform Teams
• Software Teams
• Operational technologies (virtualization, scripting) are allowing
organizations to scale out computing resources with fewer human
resources.
Cloud Paradigm
Applications
SQL
SERVER
SOFTWARE
Data
Runtime
Middleware
PLATFORM
O/S
Virtualization
Servers
Storage
Networking
INFRA
STRUCTURE
My Predictions
• Increased segregation between Dev and Admin roles.
• Number of development jobs will increase
• Diversity of data platform.
• Need for integration.
• Data mining and analysis skills.
• Number of administrative jobs will decrease
• Infrastructure as code, scripting, virtualization
• Product specific specialists for initial configuration
What does this mean for YOU?
• Choose your path: Development or Administration
• Developers have opportunities for breadth:
• Big Data (Hadoop, HDInsight)
• Data Science (Statistics, R)
• Visualizations (Reporting, Power BI)
• Administrators have opportunities for depth:
• Always On
• Infrastructure & Platform impacts
• Scripting & configurations
MCSA SQL 2016
Querying Data with
Transact-SQL
Developing SQL
Databases
Administering a SQL
Database
Infrastructure
Provisioning SQL
Databases
Implementing a SQL
Data Warehouse
Developing SQL
Models
Contact Me
Stuart R. Ainsworth
Twitter: @codegumbo
Email: [email protected]
Blog: http://codegumbo.com
Slides available: http://bit.ly/2j7lFVb