1. Welcome to RTP
DVClub!
Topics in Verification
Pete LaFauci
October 18, 2006
2. Agenda
Brief Discussion/Overview about “the Club”
Historical Perspective of Languages & Simulators
Topics in Verification
Reuse
Coverage
Regression Engineering
Planning
Qualification
Summary
3. DVClub Overview
Voluntary, Community Based Verification Group
Discuss Present Day Challenges
EDA neutral
Presentation of “sharable” ideas, experiences, and results
Advisory Board
Currently about 10 members, spanning 6 companies (looking for
additional participants)
Planning, Logistics, Speakers, Topics
Topics should be Chip Verification related, but can span
adjacent areas:
Design for Verification, IT, Project Management
Please Contact Pete LaFauci or Justin Sprague regarding interest
in the Advisory Board or Presenting at DVClub!
4. Historical Perspective: What’s done
Produce some test vector stimulus, and simulate
Look at the output and waveforms on workstation, debug the rest in the lab Then
Produce a test plan and lots of directed, procedural test cases
Run each test case in the test bucket
Completion: each individual test case passes
Produce a verification plan, which includes model
requirements, functions & coverage goals
Combine generation capabilities, checkers, randomness, and coverage monitors to
gauge the simulations
Run tests with multiple “seeds”
Completion: all tests pass, and coverage output is “analyzed”
Produce a verification methodology, “architect” a
verification environment & plan, and leverage powerful tools
Automate both efficiency and thoroughness, through prediction, steering, and proofs
VIP, Environment, Library, and Test “Reuse” Now
“Rank” the test suite efficiency, directed tests for coverage closure
Completion: all tests pass, coverage goals are met using measurable metrics
5. Historical Prevalence:
Verification Languages
HVDL – SystemVerilog
RTL/HVL Integration/Unification
Today / Tomorrow / Future?
SystemC & TLM Interfaces
Assertion Based Verification
HVL – ‘e’ & Vera DPI
Constraint Solvers
Modeling – SystemC
Late ’90s – Easy RTL Access
Higher Abstraction, TLM, HW/SW
Early 00’s Functional Coverage
Algorithmic Modeling & Prototype
Built-in libraries
C, C++. Perl
VHDL Complex Data Structs
Mid ’90s Records / Dynamic Superior String Functions
Variable Indices & Slicing
2-d ports
Verilog
Concurrency
Early ’90s Tasks & Functions
PLI
7. Reuse: Going Beyond Traditional VIP
“Components”: traditional VIP
Common bus and data protocols
Monitors, Predictors, Drivers “instance & program”
Environments: new(er) with OOP/HVL
Testbenches & “Harnesses”
Constraints & Test Cases
Configuration Components
Addresses Scalability “inherit, derive, build, configure”
8. Reuse: Inheritance and Abstraction
Application User Test Case & Constraints
Specific Environment Test Methods and User Interfaces
Verification Test Benches/Harnesses Protocol VIP
Environment Interface VIP
DUT VIP
Inheritance
And
Instantiations
Harness Interface & Protocol
Globals
templates
Company Scoreboard Test Harness DUT template
Library Data Objects Drivers Registers Memories
Base Class Packages
Base
Library Tool Packages: any_unit, any_env, any_sequence
Note : “Horizontal (Interface/Protocol) and Vertical (Base Classes and Methods) Reuse Strategies are
Deployable using OO Design Techniques”
11. Coverage: Achieving
Thoroughness
Identify which Design Functions have not been exercised
Identify What Test Code has not been simulated
Complements the Checking Code (traffic types, latency,
performance, etc)
Code Functional
Coverage Coverage
Structural
Coverage
Assertions
12. Coverage: Collection, Merging,
and Reporting
Test Cases,
Test Cases, Coverage Data
Regression 1 Procedural
Test Cases,
Procedural
Code Cases,
Test
Simulation
Procedural
Merging
Code
Procedural
Coverage “Process”
Code
Code
Output
(Cover DB)
Test Cases,
Test Cases,
Test Cases,
Test Cases,
Procedural
Test Cases,
Procedural
Test Cases,
Procedural
Regression n Code Cases,
Test
Procedural
Code Cases,
Test
Simulation
Procedural
Simulation
Procedural “Total Coverage”
Code
Procedural
Code
Procedural
Coverage
Code
Coverage
Code Reports & HTML views
Code
Code
Output
Output
(Cover DB)
Project Level Coverage
Progress Reports
13. Coverage: Challenges in Coverage
Driven Methodology
Goals are manually created from the engineering
Specs -> Labor Intensive
Random generation can be wasteful if overall
coverage is not increasing
Simulation cycles can be slow and/or expensive,
especially if they are being wasted
Execution Predictability with Schedules &
Resources
14. Coverage: Execution Predictability
Coverage Progress Chart: “S”age Commi t Phenomena
Ti gr i s 2.0 Test Cover Curve
2900
2700
2500
Total Pl anned
2300 Wr i tten Actual
Wr i tten Commi t
2100 Cover ed Actual
Cover ed Commi t
1900
1700
1500
16. Regression “Engineering”:
Challenges
Manage thousands of simulation jobs & results
Multiple environments
Many Servers, Lots of output!
Team environment, Individual Owners
Analysis
Reproducing test fails easily & accurately -> test/bench, machine, seed, tools,
versions
More debug output necessary?
Correctly assigned or delegated to appropriate owner(s)
Efficiency
Dynamically Allocating Right Machines for the Right Job Types
Maximize Utilization of both Hardware and Software
Coverage Closure: Hole Analysis, Ranking
24/7 rebalancing, including interactive session priority management
18. Planning: Improvements in Plan
Creation and Change Management
More Tightly Couple the Design Specification with the
Verification Plan
Change Management System
More Automatic Synchronization and Notification
Improve the Efficiency between Coverage Plan and
Implementation
Reduce or Remove the Disconnect between the Functional
Coverage Code and the Coverage Plan
Automate Functional Coverage Code “Generation”
20. Qualification: What’s this?
What is Qualification?
Who’s checking the verification, and when?
Error insertion to prove the accuracy and effectiveness of your
verification systems
Error insertion is placed in the DUT (not verification code)
Challenges in adoption: “how, what, when”, effort levels
Benefits for Simulation
Qualify the Test Bench
Qualify the Models & VIP
Qualify the Test Cases & Constraints
Benefits for Formal Proofs
Qualify the Assertions
Qualify the Constraints
*Qualify the Answer the Tool is Giving you*
21. Summary
Combining innovative techniques, tools, processes can
yield significant results!
Not “uncommon” to achieve 10x productivity and
thoroughness improvements through methodology and
tuning
Verification processes will continue to change
considerably to keep up with managing the exponential
growth in design complexity
