The document discusses efficient verification methodology. It recommends defining a conceptual framework or methodology to standardize some aspects while allowing diversity. The methodology should define interfaces and transactions upfront using an interface definition language to generate verification components and reusable assertions. It also recommends modeling systems at the transaction level using executable specifications to frontload the verification schedule.

1. Stop Writing Assertions!
Efficient Verification Methodology
Dave Whipp – DVClub SV, August 2008

2. Todays Talk
An Overview of Methodology Creation
Methodology as User Interface
Making ESL work as a Hardware Design Flow
Evolution of a Interface Definition Language
Transaction Level Assertions
Transaction Level Debug
NVIDIA Confidential

3. A HW Development Flow
Big Paper Spec
ISS Model
Verification Coverage
Debug
Design
Testbench Checkers Tests
RTL Synthesis
Clocks, Resets C Model Directed
Assertions
BFMs, Assertions Random
TLMs Scoreboard Formal
NVIDIA Confidential

4. User Model for Running Tests
Build Run
Testbench Predictor
* Generate Compare
Test Behaviors
Run
DUT
NVIDIA Confidential

5. Part 1
Methodology Design

6. Purpose of a Flow
Conceptual Framework
Where do things live?
How do things work?
Keep out of the way
Don’t make life difficult for people
Define Metaphors
Subtly influence future directions
Anchor for Variation
Give people something to complain about
NVIDIA Confidential

7. Two Philosophies
Conformity
Emphasize Standardization
Diversity
Emphasize Innovation
Synergy or Conflict?
NVIDIA Confidential

8. Understanding Variation
To understand what to standardize:
you need to understand what not to standardize
Personal Preferences
Technical Aspects of the Designs
Supporting Legacy
Seeking the Next Big Thing
NVIDIA Confidential

9. Personal Preferences
Emacs!
vi
NVIDIA Confidential

10. Personal Preferences
Choice of editor doesn’t affect others
At least, not much
Choice of scripting language has greater impact
But is encapsulated
A script’s users don’t see the implementation language
Choice of HVL affects whole team
Can’t write “E” tests for a “Vera” testbench!
But a unit testbench isn’t seen by other units
A good flow will allow encapsulation of preferences
I can go to any unit and build & run its tests
Enables rapid localization of infrastructure issues
NVIDIA Confidential

11. Technological Differences
NVIDIA Confidential

12. Technical Characteristics
P Graphics Pipe
C
I Frame
E Video
Buffer
Off-chip Memory
NVIDIA Confidential

13. Reuse Vs Stagnation
Reuse considered Good
Avoid reinventing the wheel
Build on the shoulders of giants
Reuse invites Inertia
Reuse can propagate dependencies
Dependencies make things harder to change
Resistance to change is known as inertia
Inertia can lead to Stagnation
Improper reuse accumulates dependencies
Reused code that is not understood will bit-rot
To avoid stagnation, inject agitation
NVIDIA Confidential

14. Are Single Paradigm Projects Possible?
Paradigm 1 Unit A
Unit B Unit B Unit B
Paradigm 2
Unit C Unit C
Paradigm 3 Unit D
time
Project 1 Project 2 Project 3
NVIDIA Confidential

15. Watch some Real Users
NVIDIA Confidential Image courtesy of DAZ Productions

16. How to “Watch” Users
Meetings and Discussions
Coffee-Break Grousing
Bug Reports
Keep Track of Support Requests
create FAQs
VNC (Remote Desktop)
Instrumentation
NVIDIA Confidential

17. Build Time Distribution: 10,000 per sample
100%
90%
80% > 1 hour
< 1 hour
70%
< 30 min
60% < 15 min
< 10 min
50%
< 5 min
40% < 2 min
< 1 min
30%
< 30 sec
20% < 20 sec
10%
0%
NVIDIA Confidential

18. Build Time Distribution: 1000 per sample
100%
90%
80% > 1 hour
< 1 hour
70%
< 30 min
60% < 15 min
< 10 min
50%
< 5 min
40% < 2 min
< 1 min
30%
< 30 sec
20% < 20 sec
10%
0%
NVIDIA Confidential

19. Part 2
ESL for Transaction Design

20. A HW Development Flow (BAD)
Big Paper Spec
ISS Model
Verification Coverage
Debug
Design
Testbench Checkers Tests
RTL Synthesis
Clocks, Resets C Model
Assertions Directed
BFMs, Assertions
Randoms
TLMs Scoreboard
NVIDIA Confidential

21. A HW Development Flow (BAD)
Big Paper Spec
ISS Model
Verification Coverage
Debug
Design
Testbench Checkers Tests
RTL Synthesis
Clocks, Resets C Model
Assertions Directed
BFMs, Assertions
Randoms
TLMs Scoreboard
NVIDIA Confidential

22. A HW Development Flow (Better)
Small Paper Spec
Coverage
Triage
ISS Model ESL
Verification
C Model Debug
Interfaces Testbench Randoms
Validation Design
Assertions
TLMs RTL Scoreboards
Directed Tests BFMs Assertions
Clocks, Resets
Synthesis
NVIDIA Confidential

23. Who Writes Assertions?
Designers
Bottom Up Assumptions
Verification Engineers
Top-down Intent
NVIDIA Confidential

24. Who Writes Assertions?
Designers
Bottom Up Assumptions
Verification Engineers
Top-down Intent
Architects
The Specification
Top Down Assumptions
Bottom Up Intent
NVIDIA Confidential

25. Where to Write Assertions
The RTL
Inline
Bound
The Testbench
Scoreboard
Environment
E.g. Post Process Log file
NVIDIA Confidential

26. Where to Write Assertions
The RTL
Inline
Bound
The Testbench
Scoreboard
Environment
Post Process Log files
The Specification
C Models (?)
NVIDIA Confidential

27. Where To Write Specification Assertions
Functionality
Model
RTL
Design
Performance
Model
NVIDIA Confidential

28. Where To Write Specification Assertions
ISS Model
Transaction
Model
Correlation RTL
Model Design
Performance
Model
Debug/Triage
Model
NVIDIA Confidential

29. Where To Write Specification Assertions
Transaction
Model
Structural RTL
Model Design
Performance
Model
NVIDIA Confidential

30. Interfaces Vs State
Two approaches to comparing models:
Compare “Architectural State”
Registers/flops within the design whose existence is
required by the specification
Compare externally visible behavior
Compare interface traffic
B. F. Skinner?
NVIDIA Confidential

31. Birth of an IDL
Interface Description Language
Initially, a language just to define signals
Interface a2b
clock clk
down U valid 1
up U busy 1
down U cmd 24
down U data 32
NVIDIA Confidential

32. Evolution of an IDL
Quickly added flow-control protocol abstraction
Interface a2b
clock clk
flow valid_busy
down U cmd 24
down U data 32
From this we can generate:
Testbench components (BFMs: producers, consumers)
Protocol Assertions
…
NVIDIA Confidential

33. Continued Evolution of an IDL
Separation of packet structure from interface
group SOP
down U cmd 24
group MOP
down U data 32
group EOP
down U checksum 32
Interface a2b
clock clk
flow valid_busy
packet SOP, MOP, EOP
NVIDIA Confidential

34. Transaction Assertions in YACC
valid_interface_traffic:
| valid_interface_traffic transaction;
transaction: begin middle end;
begin: SOP;
middle:
| middle MOP;
end: EOP
NVIDIA Confidential

35. Cycle Level Assertions in SVA
clk
cmd SOP MOP MOP EOP
sequence valid_trans;
(cmd==SOP)
(##1 cmd != SOP && cmd != EOP) [*0:$]
##1 cmd == EOP
endsequence
a_well_formed_transaction: assert @(posedge clk)
cmd == SOP |-> sequence (valid_trans)
NVIDIA Confidential

36. Transaction Level Assertions in SVA
clk
valid
cmd SOP MOP MOP EOP
sequence valid_transaction;
(cmd==SOP)
(##1 cmd != SOP && cmd != EOP) [*0:$]
##1 cmd == EOP
endsequence
NVIDIA Confidential

37. Transaction Level Assertions in SVA
clk
valid
busy
cmd SOP MOP MOP EOP
event sample;
always @(posedge clk)
if (valid && ! busy) -> sample
assert @(sample)
cmd == SOP |-> sequence (valid_transaction)
NVIDIA Confidential

38. Transaction Level Assertions in SVA
clk
valid
busy
cmd SOP MOP MOP EOP
event sample;
always @(posedge clk)
if (valid && ! busy) -> sample
assert @(sample)
cmd == BEGIN |-> sequence (valid_transaction)
NVIDIA Confidential

39. Grammer using IDL
assert “temporal representation”
data SOP => past( EOP )
data MOP => past( SOP | MOP )
data EOP => past( SOP | MOP )
assert “bnf representation”
bnf SOP
-> MOP ** past( SOP.length-1 :sample( SOP ) )
-> EOP
NVIDIA Confidential

40. Multi-Unit Assemblies
A B C D E F G
a2b b2c c2d d2e e2f f2g
A simple pipeline
NVIDIA Confidential

41. Multi-Unit Assemblies
a2b b2c c2d
A B C D
d2e E F G
e2f f2g
Simple rearrangement
NVIDIA Confidential

42. Multi-Unit Assemblies
a2b b2c c2d
A B C D
d2e E F G
e2f f2g
Identify units with similar behaviors
NVIDIA Confidential

43. Multi-Unit Assemblies
d2be cf2d
D
B be2cf C
a2be E F
A cf2g
G
Extract common behavior into unified components
be2cf === b2c + e2f
NVIDIA Confidential

44. Reusing Interface Definitions
A B C D E F G
D
B C
E F
A G
How to maximize reuse between these two architectures?
NVIDIA Confidential

45. Packets as Traffic Streams
group b2c
down U data 32
group e2f
down U data 32
Interface be2cf
clock clk
flow valid_credit
packet b2c, e2f
NVIDIA Confidential

46. Time Units of Temporal Expressions
Group b2c
down U value 4
assert value != past( value )
Group e2f
down U value 4
assert ( value == 0 ) => ( past( value ) != 0 )
Interface be2cf
packet b2c, e2f
assert b2c => ( b2c.value != past( b2c.value :sample(b2c) ) )
assert past( e2f && e2f.value == 0 ) => (b2c && b2c.value != 0)
NVIDIA Confidential

47. Example
Renderer Memory
NVIDIA Confidential

48. The Traffic
group mem_write
down U address 16
down U data 1
group sync
down U shape 2
enum SQUARE, CIRCLE, TRIANGE, BLANK
down U radius 3
NVIDIA Confidential

49. Accumulate Memory State
group mem_write
down U address 16
down U data 1
assign mem[ x = 0 .. 15 ][ y = 0 .. 15 ]
= past( data :sample( address == {x,y} ))
NVIDIA Confidential

50. Add Predicates
group_more mem_write
assign is_circle[ r = 0..7 ] = “&&”(
[ x = -8 .. 7 ]
[ y = -8 .. 7 ]
mem[ x+8 ][ y+8 ] == ( x**2 + y**2 <= r**2 )
)
NVIDIA Confidential

51. Define the interface
interface render2memory
clock mem_clk
flow valid_busy
packet sync, mem_write
assert “correct sync shape”
sync && sync.shape == CIRCLE
=> past( mem_write.is_circle[ sync.radius ] )
NVIDIA Confidential

52. Example
Renderer Memory
assert “correct sync shape”
sync && sync.shape == CIRCLE
=> past( mem_write.is_circle[ sync.radius ] )
NVIDIA Confidential

53. Summary
Efficient Verification
Understand Variation
Encourage Diversity
avoid anarchy
Encourage Standardization
avoid stagnation
Frontload The Schedule
Provide Verification with Executable Spec (ESL)
C Model, Correlation Model (pre validated)
Interfaces, Transactions (definitions for code generators)
Assertions, Testpoints (neutral language, pre validated)
NVIDIA Confidential

54. Questions
http://dave.whipp.name/dv

55. Summary
Architects should write assertions
Validated assertions are input to Verification
Assertions must be directly reusable across models
Manual recoding invites errors
Explicitly model the structure that is common to
architectural models and to design
Tie assertions to these common points
NVIDIA Confidential

56. Threading Models
Thread Follows Resource
Traditional SystemC approach
Thread Follows Transaction
Think “Multithreaded ISS”
Cycle-based Model
The “Old” way of doing things
A non-threaded version of “thread follows resource”
Different models are convenient at various times
But SystemC forces you to commit early to one approach
Therefore need extra code for transaction level debug
NVIDIA Confidential

57. 70%
70% of schedule is Verification
What is correct figure
0%
synthesize architectural models, constraints
Correct by construction!
100%
synthesize verification models, constraints
no design needed!
NVIDIA Confidential

58. Transactions Vs Cycles
Data min_val (Addr a1, Addr a2)
Pipelined Bus
{
Data d1 = mem_read(a1);
Data d2 = mem_read(a2); t2 t3 t4
t1 t5 t6
if (d1 < d2) Address a1 a2
return d1;
else Data d1 d2
return d2;
}
NVIDIA Confidential

59. Unit Level System Testing
Graphics Pipeline (Transaction Model)
Stage Stage Stage
N-1 N N+1
RTL
DIFF
Unit
NVIDIA Confidential

60. Avoiding Stagnation
New Challenges
New Tools
New Platforms
New People
New Ideas
Refactoring
Testability
D.R.Y.
NVIDIA Confidential