Download Corporate Title * 36pt, Three Lines Max. Anchor: Bottom Left

Trends in Functional
Verification:
A 2016 Industry Study
Invited Talk
Harry D Foster
Chief Scientist Verification
Mentor Graphics Corp
February 28, 2017
Design Size by Projects
20%
~31% of designs over 80M gates
~20% of designs over 500M gates
Design Projects
15%
10%
5%
0%
Source: Wilson Research Group and Mentor Graphics, 2016 Functional Verification Study
3
Harry Foster, DVCon February 2017
2014
2016
Study Background

Worldwide study
— North America, India/China, Japan, Rest of World

Sample frame consisted of 1738 participants
— 8% smaller than our 2014 study (1886)
— 3.3x larger than our 2012 study
— 9.4x larger than the 2004 Collett International study

Confidence interval 95%
— ±2.36% Margin of Error
4
Harry Foster, DVCon February 2017
Design Productivity Grew 5 Orders
of Magnitude Since 1985
1,000,000,000,000,000
100,000,000,000,000
10,000,000,000,000
1,000,000,000,000
Quantity
100,000,000,000
10,000,000,000
1,000,000,000
100,000,000
10,000,000
1,000,000
100,000
10,000
Transistors Produced
Total Electronic Engineers
Source: Technology Research Group – EDA Database, 1986, EDA TAM, 1989 & Gartner/Dataquest 2005 Seat Count Report,
Gary Smith EDA, 2013 Seat Count Analysis, VLSI Research, 2013 - Transistors Produced Analysis
5
Harry Foster, DVCon February 2017
Demand for Design Engineers Grows Slowly
12
Mean Peak Number of Engineers
10
8
6
4
2
0
2007
2010
Source: Wilson Research Group and Mentor Graphics, 2016 Functional Verification Study
6
Harry Foster, DVCon February 2017
2012
2014
2016
Demand for Verification Engineers 3X Designers
12
Mean Peak Number of Engineers
10
8
6
4
2
0
2007
2010
Source: Wilson Research Group and Mentor Graphics, 2016 Functional Verification Study
7
Harry Foster, DVCon February 2017
2012
2014
2016
Emergence of New Verification Requirements
Requirements
Software
Security & Safety
Power
Clocking
Functionality
9
Harry Foster, DVCon February 2017
It’s an SoC World
72% of designs contain embedded processors
49% of designs contain 2 or more processors
16% of designs contain 8 or more processors
40%
Design Projects
30%
20%
10%
2012
2014
2016
0%
0
1
2
3
4
5 or more
Number of Embedded Processor Cores in ASIC-IC Designs
Source: Wilson Research Group and Mentor Graphics, 2016 Functional Verification Study
10
Harry Foster, DVCon February 2017
Number of Clock Domains Increases with
Design Size
Number of Clock Domains
14
16
12
10
10
8
6
4
5
2
0
< 5M
5 - 80M
Gates
Source: Wilson Research Group and Mentor Graphics, 2016 Functional Verification Study
11
Harry Foster, DVCon February 2017
> 80M
Designs Projects Implementing Security Features
42%
54%
58%
ASIC/IC Projects
46%
FPGA Projects
Security Features
No Security Features
Source: Wilson Research Group and Mentor Graphics, 2016 Functional Verification Study
12
Harry Foster, DVCon February 2017
Projects Working on Safety Critical Design
56%
54%
46%
ASIC/IC Projects
44%
FPGA Projects
Safety Critical Design
DO-254, ISO26262, IEC60601, IEC61508, etc.
Source: Wilson Research Group and Mentor Graphics, 2016 Functional Verification Study
13
Harry Foster, DVCon February 2017
Not Safety Critical
Verification Project Time
30%
ASIC/IC Design Projects
25%
2012: Average 56%
2014: Average 56%
2016: Average 55%
World 2012
World 2014
World 2016
20%
15%
10%
5%
0%
1%-20% 21%-30% 31%-40% 41%-50% 51%-60% 61%-70% 71%-80%
>80%
Percentage of Project Time Spent in Verification
Source: Wilson Research Group and Mentor Graphics, 2016 Functional Verification Study
Source: Wilson Research Group and Mentor Graphics, 2016 Functional Verification Study
15
Harry Foster, DVCon February 2017
Project Completion to Original Schedule
ASIC/IC Design Projects
35%
2012: 67% projects behind schedule
2014: 61% projects behind schedule
2016: 69% projects behind schedule
30%
25%
World 2012
20%
World 2014
World 2016
15%
10%
5%
0%
More than
10% EARLY
10% EARLY ON-SCHEDULE 10% BEHIND
SCHEDULE
20%
30%
40%
50%
>50%
BEHIND
SCHEDULE
Actual ASIC-IC design completion compared to project's original schedule
Ahead of Schedule
Source: Wilson Research Group and Mentor Graphics, 2014 Functional Verification Study
16
Harry Foster, DVCon February 2017
Behind Schedule
Number of Required Spins Before Production
50%
ASSIC/IC Design Projects
World 2012
40%
World 2014
World 2016
30%
20%
10%
0%
1 (FIRST
SILICON
SUCCESS)
2
3
4
Number of Required Spins
Source: Wilson Research Group and Mentor Graphics, 2016 Functional Verification Study
Source: Wilson Research Group and Mentor Graphics, 2016 Functional Verification Study
17
Harry Foster, DVCon February 2017
5
6
7 SPINS or
MORE
Number of FPGA Bug Escapes to Production
35%
78% of FPGA design projects
have non-trivial bugs that
escape into productions
30%
Design Projects
25%
20%
15%
10%
5%
0%
0
1
2
3
4
5
2016 FPGA Non-Trivial Bug Escapes to Production
Source: Wilson Research Group and Mentor Graphics, 2016 Functional Verification Study
Source: Wilson Research Group and Mentor Graphics, 2016 Functional Verification Study
18
Harry Foster, DVCon February 2017
6 or More
Flaws Contributing to Respins
ASIC/IC Design Projects
50%
40%
2012
2014
2016
30%
20%
10%
0%
Trends in Types of Flaws Resulting in Respins
Source: Wilson Research Group and Mentor Graphics, 2016 Functional Verification Study
Source: Wilson Research Group and Mentor Graphics, 2016 Functional Verification Study
19
Harry Foster, DVCon February 2017
* Multiple answers possible
Root Cause of Functional Flaws
80%
2012
ASIC/IC Design Projects
70%
2014
2016
60%
50%
40%
30%
20%
10%
0%
DESIGN ERROR
CHANGES IN
SPECIFICATION
INCORRECT or
INCOMPLETE
SPECIFICATION
FLAW IN INTERNAL FLAW IN EXTERNAL
REUSED BLOCK, CELL,
IP BLOCK or
MEGACELL or IP
TESTBENCH
Root Cause of Functional Flaws
Source: Wilson Research Group and Mentor Graphics, 2016 Functional Verification Study
Source: Wilson Research Group and Mentor Graphics, 2016 Functional Verification Study
20
Harry Foster, DVCon February 2017
* Multiple answers possible
Projects Working on Safety Critical Design
54%
46%
ASIC/IC Projects
DO-254, ISO26262, IEC60601, IEC61508, etc.
Source: Wilson Research Group and Mentor Graphics, 2016 Functional Verification Study
21
Harry Foster, DVCon February 2017
Safety Critical Design
Not Safety Critical
Required Spins for Safety Critical Designs
40%
72% of Safety Critical ASIC/IC
designs require a respins!
ASIC/IC Design Projects
30%
20%
Safety Critical Design
Non-Safety Critical Design
10%
0%
1 (FIRST
SILICON
SUCCESS)
2
3
Number of Required Spins
DO-254, ISO26262, IEC60601, IEC61508, etc.
Source: Wilson Research Group and Mentor Graphics, 2016 Functional Verification Study
22
Harry Foster, DVCon February 2017
4
5
6
7 SPINS or
MORE
Verification Language Adoption Trends
ASIC/IC Design Projects
80%
World 2012
World 2014
World 2016
70%
60%
50%
40%
30%
20%
10%
0%
Languages Used for Verification (Testbenches)
Source: Wilson Research Group and Mentor Graphics, 2016 Functional Verification Study
Source: Wilson Research Group and Mentor Graphics, 2016 Functional Verification Study
24
Harry Foster, DVCon February 2017
* Multiple answers possible
Testbench Methodology Adoption Trends
ASIC/IC Design Projects
70%
World 2012
World 2014
60%
World 2016
50%
40%
30%
20%
10%
0%
Accellera
UVM
OVM
Mentor AVM Synopsys
VMM
Synopsys
RVM
Cadence
eRM
Cadence
URM
None/Other
* Multiple answers possible
Methodologies and Testbench Base-Class Libraries
Source: Wilson Research Group and Mentor Graphics, 2016 Functional Verification Study
Source: Wilson Research Group and Mentor Graphics, 2016 Functional Verification Study
25
Harry Foster, DVCon February 2017
Assertion Language Adoption
80%
World 2007
World 2012
World 2014
World 2016
ASIC/IC Design Projects
70%
60%
50%
40%
30%
20%
10%
0%
Accellera Open
Verification Library
(OVL)
SystemVerilog
Assertions (SVA)
PSL
Assertion Languages and Libraries
Source: Wilson Research Group and Mentor Graphics, 2016 Functional Verification Study
Source: Wilson Research Group and Mentor Graphics, 2016 Functional Verification Study
26
Harry Foster, DVCon February 2017
Other
* Multiple answers possible
Power Intent Standards Trends
40%
2012
2014
35%
2016
Design Projects
30%
25%
20%
15%
10%
5%
0%
CPF 1.0
CPF 1.1
CPF 2.0
UPF 1.0
UPF 2.x
Notation Used to Describe Power Intent
Source: Wilson Research Group and Mentor Graphics, 2016 Functional Verification Study
Source: Wilson Research Group and Mentor Graphics, 2016 Functional Verification Study
27
Harry Foster, DVCon February 2017
Internal /
Proprietary
* Multiple answers possible
Formal Technology Adoption Trends
40%
World 2012
ASIC/IC Design Projects
35%
World 2014
World 2016
30%
25%
20%
15%
10%
5%
0%
Formal property checking
Source: Wilson Research Group and Mentor Graphics, 2016 Functional Verification Study
Source: Wilson Research Group and Mentor Graphics, 2016 Functional Verification Study
29
Harry Foster, DVCon February 2017
Automatic formal applications
Dynamic Verification Adoption Trends
2007
2012
2014
2016
Code coverage
Assertions
Functional coverage
Constrained-Random Simulation
0% 10% 20% 30% 40% 50% 60% 70% 80%
ASIC/IC Design Projects
Source: Wilson Research Group and Mentor Graphics, 2016 Functional Verification Study
Source: Wilson Research Group and Mentor Graphics, 2016 Functional Verification Study
30
Harry Foster, DVCon February 2017
Comparing required spins, and project maturity
in adopting various verification techniques.
Code coverage
Assertions
Functional coverage
Constrained-Random Simulation
Multiple Spins
0% 10% 20% 30% 40% 50% 60% 70% 80%
First Silicon Success
Source: Wilson Research Group and Mentor Graphics, 2016 Functional Verification Study
31
Harry Foster, DVCon February 2017
ASIC/IC Design Projects
Closing the Verification Productivity Gap
Common
Methodology
Acceleration
Automation
Reuse
33
Harry Foster, DVCon February 2017
Abstraction
Summary

The design productivity gap has been closed through the adoption of
advanced automation and reuse methodology
— What about the verification gap?

The industry has matured its verification processes rapidly due to
rising complexity

Closing the verification productivity gap will require…
— The adoption of a common methodology, more reuse (VIP), acceleration, raising
the abstraction level when possible, and automating verification to ease adoption
and broaden usage
Source: Wilson Research Group and Mentor Graphics, 2016 Functional Verification Study
34
Harry Foster, DVCon February 2017
Additional Resources

Paper accompany this talk in the DVCon proceedings

See my blog for additional details related to this study
— http://go.mentor.com/4Qa1S
— Includes both ASIC/IC and FPGA findings
Source: Wilson Research Group and Mentor Graphics, 2016 Functional Verification Study
35
Harry Foster, DVCon February 2017
www.mentor.com