Download Tlog backups will clear VLFs for reuse but probably will not shrink tlog

Get Into the Goldilocks Zone
Managing SQL Transaction Log VLF Growth
Michelle Poolet, MCIS
Zachman-Certified Enterprise Architect™
[email protected]
www.MountVernonDataSystems.com
https://www.linkedin.com/in/michellepoolet
4/2/2016
What We’ll Do Today
 Talk about VLFs, their value, impact and
relationship to the SQL transaction log.
 What causes the VLF count to grow?
 How to rein in VLF and subsequent SQL
Server transaction log growth.
 How to get into the Goldilocks Zone, that is,
get a handle on stabilizing VLFs and the SQL
Server transaction log size.
2 |
4/2/2016
What’s a VLF & Why Do I Care?
 Some background
 All changes to a database (data & structure) are
preserved
 First write to the transaction log
 Then write to physical database
 One transaction log per database
 SQL Server writes changes to available space in the
log file
 SQL Server must have enough tlog space so that
active (uncommitted) transactions can be rolled back
3 |
4/2/2016
What’s a VLF & Why Do I Care?
 VLF structure
 Tlog is comprised of blocks of space -- VLF
 Space management inside the transaction log file
 No single fixed size for a VLF
 No fixed number of VLFs per physical log file
4 |
4/2/2016
Why Would a VLF Not Be Reusable?
 Long-running transactions that don’t commit…
 No commit = VLFs remain “active”, cannot reuse
 Lack of transaction log backup…
 a backup issues a CHECKPOINT
 CHECKPOINT confirms that committed transactions
on the tlog have been written to the database
 committed/checkpointed transactions on the tlog are
marked for overwrite
 It all boils down to...
 A VLF is reusable ONLY when ALL the transactions stored in
that VLF have been committed and CHECKPOINTED
5 |
4/2/2016
What Causes the VLFs (& TLog) to Grow?
 If no VLFs available for re-use
 SQL Server must auto-grow
 Allows SQL Server to allocate additional space to the tlog
 must pause processing (no instant initialization)
 Increases the number of VLFs based on auto-grow size
 SQL Server must stop processing
 If auto-grow is not enabled and
 “out of space in the log”
6 |
4/2/2016
VLF Trivia 2012
 TRIVIA FACT for <= SQL Server 2012
 Tlog auto-grows between 1-64 MB, +4 VLFs
 Tlog auto-grows > 64-9999 MB, +8 VLFs
 Tlog auto-grows >= 1GB, +16 VLFs
 TRIVIA QUESTION
 SQL Server created with tlog file = 2 MB & autogrowth by 10%
 By the time the tlog reaches 500 MB, how many
VLFs will it have?
7 |
4/2/2016
VLF Trivia 2012
 TRIVIA ANSWER
 416 VLFs…






8 |
4/2/2016
264 VLFs < 1 MB in size (0.25 to 0.93 MB)
56 VLFs < 2 MB in size (1.03 to 1.82 MB)
40 VLFs < 3 MB in size (2.0 to 2.93 MB)
24 VLFs < 4 MB in size (3.23 to 3.9 MB)
16 VLFs < 5 MB in size (4.3 to 4.72 MB)
16 VLFs < 6 MB in size (5.2 to 5.72 MB)
VLF Trivia 2014
 More Trivia for >= SQL Server 2014
 If the impending growth unit is < 1/8 size of the
current log:
 A single VLF of size = auto-grow unit will be created
 Else
 Use old algorithm
9 |
4/2/2016
VLF Algorithm Comparison
 Comparing old & new auto-grow algorithms
The old formula: auto-grow using a 10% increment causes a fast build-up of VLF count,
even with very small transaction logs. The new formula controls VLF count much more
efficiently, but the VLF segment size increases more quickly.
10 |
4/2/2016
VLF Algorithm Comparison
 Comparing old & new auto-grow algorithms
As the transaction log size approaches 1GB, VLF counts skyrocket under the old formula,
but the VLF segment size stays very stable. Under the new formula, VLF count is still low,
but the VLF segment sizes are getting quite large – this may cause problems with VLF
segment reuse.
11 |
4/2/2016
VLF Algorithm Comparison
 Comparing old & new auto-grow algorithms
The transaction log is in the 10-20GB range. The old formula has very high VLF count but
small VLF segment size, good for VLF reuse. The new formula has a small VLF count but
very large VLF segment sizes. VLF reuse is at risk, the transaction log could uncontrollably
balloon to an "out of space on disk" condition.
12 |
4/2/2016
Why Is This Such A Big Deal?
 Too many VLFs adversely impact database
operations…
 Database startup takes longer;
 Database restores take longer;
 Inserts, updates and deletes can take longer;
 Backups are larger because active VLFs (uncommitted,
not cleared for reuse) must be included;
 Tlog file fragmentation - SQL Server has to search for
the appropriate VLF whenever writing to the log file or
clearing a VLF for reuse
13 |
4/2/2016
Can VLFs Get Too Large?
 Overly large VLFs (4 GB?) = performance
problems…
 File I/O
 Buffer cache crunch
 Clearing large VLFs for reuse takes time…
 SQL Server search for correct VLF to commit
 Tlog backups take longer
 Inability to clear VLFs rapidly = more VLFs
created
 New VLFs will be large = more disk space required
14 |
4/2/2016
Can There Be Too Many VLFs?
 There is no hard and fast rule as to how
many VLFs any database should have…
 Too many VLFs = tlog fragmentation, negative
impact on performance, slower database startup
& restore.
 Too few and very large VLFs = negative impact
on performance, larger backups, ballooning
transaction logs.
15 |
4/2/2016
Help! What Do I Do With A Ballooning TLog?
1. Check for open transactions
2. Take a tlog backup
3. Shrink the tlog file, reduce the VLF count (no
maintenance window required)
16 |
4/2/2016
Help! What Do I Do With A Ballooning TLog?
 CHECK FOR OPEN TRANSACTIONS
Dbcc opentran;
 What is a tlog file waiting on?
SELECT name,
log_reuse_wait,
log_reuse_wait_desc
FROM sys.databases;
 is there a way to make open transactions
commit more often?
17 |
4/2/2016
Help! What Do I Do With A Ballooning TLog?
 TAKE A TLOG BACKUP
 You think that tlog backups have been running
regularly?
 Try running a SQL tlog backup…
 VM or Hyper-V snapshots will not shrink the tlog
 Rethink your backup strategy
 Tlog backups will clear VLFs for reuse but probably will
not shrink tlog
18 |
4/2/2016
Help! What Do I Do With A Ballooning TLog?
 SHRINK THE TLOG FILE (no maintenance
window required)
 TEMPORARY SOLUTION ONLY, reduces VLF count
and releases log space to the operating system
 Root cause has not been addressed
 Tlog files will re-grow over time, will have to repeat process
 No instant file initialization as SQL tlog grows, performance hit
USE MyDatabase
GO
dbcc loginfo
-- check VLF count before shrink
DBCC SHRINKFILE (N‘MyDatabase_log' , 100)
GO
dbcc loginfo
-- check VLF count after shrink
19 |
4/2/2016
What’s the Ultimate Goal?
 GET INTO THE GOLDILOCKS ZONE





Study & monitor tlog growth and behaviour
Schedule a Maintenance Window
Resize and pre-grow the tlog file
Monitor tlog growth and behaviour after-the-fact
Repeat if necessary
 If you understand well how each database uses its tlog
file, you may not have to do this step. Ever. Until things
change.
20 |
4/2/2016
Goldilocks Zone: Pre-Grow A Tlog File
Schedule a Maintenance Window
1.
2.
3.
4.
21 |
4/2/2016
determine a new size for the log file
shrink the transaction log
re-grow the transaction log
change tlog auto-grow settings
Goldilocks Zone: Pre-Grow A Tlog File
1. How large should the tlog be?
 How much data is being added to the database in one
year’s time?
 How often are index rebuilds being done?
 Are there excessively-long transactions that are
running against the database? Can they be
shortened?
 Is data being bulk-loaded into the database on a
regular basis? How much and when?
 What’s the disk usage history? (applies only to mature
databases…)
 Does this database take part in mirroring, replication,
or AlwaysOn Availability Groups?
22 |
4/2/2016
Goldilocks Zone: Pre-Grow A Tlog File
2. Shrink the tlog to smallest possible size
 Use dbcc shrinkfile to shrink the log file
 Shrink slowly, in small increments
 Allow SQL Server to transfer “active” transactions
to the newer portion of the tlog
23 |
4/2/2016
Goldilocks Zone: Pre-Grow A Tlog File
3. Re-grow the tlog file
 Use alter database modify file command
 Incrementally grow the log file to an appropriate size
 Grow it slowly
 Grow too fast results in too-large VLFs
 IDEAL CASE: size the tlog so it will not have to autogrow in future operations
24 |
4/2/2016
Goldilocks Zone: Pre-Grow A Tlog File
4. Change auto-grow settings
 Adjust auto-grow to new growth increment
 If dynamically-created databases (Sharepoint)
 Adjust model database to minimize tlog growth
ALTER DATABASE PROD
MODIFY FILE (NAME=PRODLOG, FILEGROWTH = 100 MB)
GO
25 |
4/2/2016
DEMO
26 |
4/2/2016
Goldilocks Database
27 |
3/25/2017
Goldilocks – what do I know?
 3rd-party package used to make reservations.
 Normal processing behaviour is bursty, based on upcoming
events.
 Transactions are short (ticket purchase, integrate with
PayPal, send email to purchaser).
 Business & activity reports are run against the same
database.
 Tlog size is <2% of the database file (seems very small, out
of proportion, especially given the large auto-grow size).
 This is a copy sent for review.
 DBA may have shrunk transaction log file prior to backup?
28 |
3/25/2017
Goldilocks Assumptions…
 IF...this transaction log is indeed a good size for
this database...
 the VLF count is high, higher than optimal for the
type of transactions (short, succinct),
 auto-grow value is disproportionately large,
 larger VLF will take longer to clear. During high
activity periods, SQL may need to acquire
additional log space...
 ...having to pause SQL while the tlog initializes
during high-activity periods is not good.
29 |
3/25/2017
Goldilocks Recommendations
 Let's restructure the tlog file, reduce the VLF
count, to better align with the type of
processing and the type of data.
30 |
3/25/2017
Goldilocks: Shrink the tlog file slowly…
 REPEAT as necessary until shrinking stops
31 |
3/25/2017
Goldilocks: REGROW THE Tlog file
 Use growth increments
until target size is
reached
 grow to 500 MB, growth
unit = 100 MB
32 |
3/25/2017
Goldilocks: Change the tlog auto-grow value
 to match the value used to re-grow the file
 and make a final check
33 |
3/25/2017
Real-world Case: Personify
 ERP solution, SQL Server 2012
 PPROD data file = 22.9 GB, FULL recovery, growth unit = 20 MB
 log file = 16.25 GB, VLFs growing over time, 30 MB growth unit,
4.55% of log space used
 log file should be regrown to 4 GB, growth unit = 500 MB
34 |
4/2/2016
Real-world Case: Sharepoint 2013
 SQL Server 2012
 msdb data file = 22.9 GB, FULL recovery, growth unit = 20 MB
 log file = 16.25 GB, VLFs growing over time, 30 MB growth unit,
4.55% of log space used
 log file should be regrown to 4 GB, growth unit = 500 MB
35 |
4/2/2016
Is There a Methodology? Sort of…
 Track changes over time to determine
transaction activity and log growth needs.
 Set up a maintenance window
 Reduce VLF count
 Regrow tlog size while controlling number and
size of VLFs
 Reset database > file properties > auto-grow size
to align with changes made
 Continue to monitor; if necessary, fine-tune
assumptions and repeat process.
36 |
4/2/2016
What We’ve Covered…
 VLFs, what they are, what impact they have
on your SQL Server
 Why VLFs to grow?
 How to shut down a ballooning transaction log
 How to get into the Goldilocks Zone - get a
handle on stabilizing VLFs and the SQL
Server transaction log size.
37 |
4/2/2016
Questions?
Michelle Poolet, MCIS
Zachman-Certified Enterprise Architect™
[email protected]
www.MountVernonDataSystems.com
https://www.linkedin.com/in/michellepoolet
38 |
3/25/2017
Good References
 Ivan Stankovic
http://www.sqlshack.com/virtual-log-files-sqlserver-transaction-log/
 Kim Tripp,SQLSkills.com
http://www.sqlskills.com/blogs/kimberly/transa
ction-log-vlfs-too-many-or-too-few/
 Eric Darling
https://www.brentozar.com/archive/2015/12/w
hat-happens-to-transaction-log-backupsduring-full-backups/
39 |
3/25/2017
Thank You Sponsors
Platinum
Gold
Silver
Bronze
Swag
40 |
3/25/2017