Stop Writing Assertions!Efficient Verification Methodology  Dave Whipp – DVClub SV, August 2008
Todays Talk         An Overview of Methodology Creation                      Methodology as User Interface         Making ...
A HW Development Flow                                  Big Paper Spec         ISS Model                                   ...
User Model for Running Tests           Build                       Run         Testbench                   Predictor      ...
Part 1Methodology Design
Purpose of a Flow         Conceptual Framework                      Where do things live?                      How do thin...
Two Philosophies         Conformity                      Emphasize Standardization         Diversity                      ...
Understanding Variation         To understand what to standardize:                      you need to understand what not to...
Personal Preferences                      Emacs!                      viNVIDIA Confidential
Personal Preferences         Choice of editor doesn’t affect others                      At least, not much         Choice...
Technological DifferencesNVIDIA Confidential
Technical Characteristics               P              Graphics Pipe               C               I                      ...
Reuse Vs Stagnation         Reuse considered Good                      Avoid reinventing the wheel                      Bu...
Are Single Paradigm Projects Possible?Paradigm 1             Unit A                       Unit B      Unit B      Unit BPa...
Watch some Real UsersNVIDIA Confidential     Image courtesy of DAZ Productions
How to “Watch” Users         Meetings and Discussions         Coffee-Break Grousing         Bug Reports         Keep Track...
Build Time Distribution: 10,000 per sample       100%         90%         80%                         > 1 hour            ...
Build Time Distribution: 1000 per sample       100%         90%         80%                         > 1 hour              ...
Part 2ESL for Transaction Design
A HW Development Flow (BAD)                                  Big Paper Spec         ISS Model                             ...
A HW Development Flow (BAD)                                  Big Paper Spec         ISS Model                             ...
A HW Development Flow (Better)           Small Paper Spec                                                      Coverage   ...
Who Writes Assertions?         Designers                      Bottom Up Assumptions         Verification Engineers        ...
Who Writes Assertions?         Designers                      Bottom Up Assumptions         Verification Engineers        ...
Where to Write Assertions         The RTL                      Inline                      Bound         The Testbench    ...
Where to Write Assertions         The RTL                      Inline                      Bound         The Testbench    ...
Where To Write Specification Assertions           Functionality              Model                             RTL        ...
Where To Write Specification Assertions                           ISS Model             Transaction                Model  ...
Where To Write Specification Assertions             Transaction                Model                           Structural ...
Interfaces Vs State         Two approaches to comparing models:                      Compare “Architectural State”        ...
Birth of an IDL         Interface Description Language         Initially, a language just to define signals           Inte...
Evolution of an IDL         Quickly added flow-control protocol abstraction           Interface a2b               clock cl...
Continued Evolution of an IDL         Separation of packet structure from interface           group SOP              down ...
Transaction Assertions in YACC                 valid_interface_traffic:                        | valid_interface_traffic t...
Cycle Level Assertions in SVA clk cmd                  SOP   MOP   MOP EOP         sequence valid_trans;            (cmd==...
Transaction Level Assertions in SVA clk valid cmd                  SOP   MOP   MOP        EOP         sequence valid_trans...
Transaction Level Assertions in SVA clk valid busy cmd                  SOP   MOP   MOP         EOP         event sample; ...
Transaction Level Assertions in SVA clk valid busy cmd                  SOP   MOP   MOP        EOP         event sample;  ...
Grammer using IDLassert “temporal representation”           data SOP => past( EOP )           data MOP => past( SOP | MOP ...
Multi-Unit Assemblies            A               B         C         D         E         F         G                      ...
Multi-Unit Assemblies                            a2b       b2c       c2d                        A         B         C     ...
Multi-Unit Assemblies                            a2b          b2c            c2d                        A         B       ...
Multi-Unit Assemblies                                  d2be                    cf2d                                       ...
Reusing Interface Definitions                      A    B     C    D       E       F   G                                  ...
Packets as Traffic Streams           group b2c              down U data 32           group e2f              down U data 32...
Time Units of Temporal ExpressionsGroup b2c           down U value 4           assert value != past( value )Group e2f     ...
Example                      Renderer   MemoryNVIDIA Confidential
The Traffic         group mem_write            down U address 16            down U data 1         group sync            do...
Accumulate Memory Stategroup mem_write         down U address 16         down U data 1         assign mem[ x = 0 .. 15 ][ ...
Add Predicatesgroup_more mem_write         assign is_circle[ r = 0..7 ] = “&&”(            [ x = -8 .. 7 ]            [ y ...
Define the interfaceinterface render2memory   clock mem_clk   flow valid_busy   packet sync, mem_write         assert “cor...
ExampleRenderer                                           Memory    assert “correct sync shape”          sync && sync.shap...
Summary         Efficient Verification                      Understand Variation                      Encourage Diversity ...
Questionshttp://dave.whipp.name/dv
Summary         Architects should write assertions                      Validated assertions are input to Verification    ...
Threading Models         Thread Follows Resource                      Traditional SystemC approach         Thread Follows ...
70%         70% of schedule is Verification         What is correct figure         0%                      synthesize arch...
Transactions Vs Cycles                      Data min_val (Addr a1, Addr a2)                                               ...
Unit Level System Testing  Graphics Pipeline (Transaction Model)   Stage                      Stage       Stage    N-1    ...
Avoiding Stagnation      New Challenges      New Tools      New Platforms      New People      New Ideas      Refactoring ...
Upcoming SlideShare
Loading in …5
×

Stop Writing Assertions! Efficient Verification Methodology

931 views

Published on

Published in: Technology, Design
0 Comments
0 Likes
Statistics
Notes
  • Be the first to comment

  • Be the first to like this

No Downloads
Views
Total views
931
On SlideShare
0
From Embeds
0
Number of Embeds
101
Actions
Shares
0
Downloads
22
Comments
0
Likes
0
Embeds 0
No embeds

No notes for slide

Stop Writing Assertions! Efficient Verification Methodology

  1. 1. Stop Writing Assertions!Efficient Verification Methodology Dave Whipp – DVClub SV, August 2008
  2. 2. Todays Talk An Overview of Methodology Creation Methodology as User Interface Making ESL work as a Hardware Design Flow Evolution of a Interface Definition Language Transaction Level Assertions Transaction Level DebugNVIDIA Confidential
  3. 3. A HW Development Flow Big Paper Spec ISS Model Verification Coverage Debug Design Testbench Checkers Tests RTL Synthesis Clocks, Resets C Model Directed Assertions BFMs, Assertions Random TLMs Scoreboard FormalNVIDIA Confidential
  4. 4. User Model for Running Tests Build Run Testbench Predictor * Generate Compare Test Behaviors Run DUTNVIDIA Confidential
  5. 5. Part 1Methodology Design
  6. 6. Purpose of a Flow Conceptual Framework Where do things live? How do things work? Keep out of the way Don’t make life difficult for people Define Metaphors Subtly influence future directions Anchor for Variation Give people something to complain aboutNVIDIA Confidential
  7. 7. Two Philosophies Conformity Emphasize Standardization Diversity Emphasize Innovation Synergy or Conflict?NVIDIA Confidential
  8. 8. Understanding Variation To understand what to standardize: you need to understand what not to standardize Personal Preferences Technical Aspects of the Designs Supporting Legacy Seeking the Next Big ThingNVIDIA Confidential
  9. 9. Personal Preferences Emacs! viNVIDIA Confidential
  10. 10. Personal Preferences Choice of editor doesn’t affect others At least, not much Choice of scripting language has greater impact But is encapsulated A script’s users don’t see the implementation language Choice of HVL affects whole team Can’t write “E” tests for a “Vera” testbench! But a unit testbench isn’t seen by other units A good flow will allow encapsulation of preferences I can go to any unit and build & run its tests Enables rapid localization of infrastructure issuesNVIDIA Confidential
  11. 11. Technological DifferencesNVIDIA Confidential
  12. 12. Technical Characteristics P Graphics Pipe C I Frame E Video Buffer Off-chip MemoryNVIDIA Confidential
  13. 13. Reuse Vs Stagnation Reuse considered Good Avoid reinventing the wheel Build on the shoulders of giants Reuse invites Inertia Reuse can propagate dependencies Dependencies make things harder to change Resistance to change is known as inertia Inertia can lead to Stagnation Improper reuse accumulates dependencies Reused code that is not understood will bit-rot To avoid stagnation, inject agitationNVIDIA Confidential
  14. 14. Are Single Paradigm Projects Possible?Paradigm 1 Unit A Unit B Unit B Unit BParadigm 2 Unit C Unit CParadigm 3 Unit D time Project 1 Project 2 Project 3NVIDIA Confidential
  15. 15. Watch some Real UsersNVIDIA Confidential Image courtesy of DAZ Productions
  16. 16. How to “Watch” Users Meetings and Discussions Coffee-Break Grousing Bug Reports Keep Track of Support Requests create FAQs VNC (Remote Desktop) InstrumentationNVIDIA Confidential
  17. 17. Build Time Distribution: 10,000 per sample 100% 90% 80% > 1 hour < 1 hour 70% < 30 min 60% < 15 min < 10 min 50% < 5 min 40% < 2 min < 1 min 30% < 30 sec 20% < 20 sec 10% 0%NVIDIA Confidential
  18. 18. Build Time Distribution: 1000 per sample 100% 90% 80% > 1 hour < 1 hour 70% < 30 min 60% < 15 min < 10 min 50% < 5 min 40% < 2 min < 1 min 30% < 30 sec 20% < 20 sec 10% 0%NVIDIA Confidential
  19. 19. Part 2ESL for Transaction Design
  20. 20. A HW Development Flow (BAD) Big Paper Spec ISS Model Verification Coverage Debug Design Testbench Checkers Tests RTL Synthesis Clocks, Resets C Model Assertions Directed BFMs, Assertions Randoms TLMs ScoreboardNVIDIA Confidential
  21. 21. A HW Development Flow (BAD) Big Paper Spec ISS Model Verification Coverage Debug Design Testbench Checkers Tests RTL Synthesis Clocks, Resets C Model Assertions Directed BFMs, Assertions Randoms TLMs ScoreboardNVIDIA Confidential
  22. 22. A HW Development Flow (Better) Small Paper Spec Coverage Triage ISS Model ESL Verification C Model Debug Interfaces Testbench Randoms Validation Design AssertionsTLMs RTL ScoreboardsDirected Tests BFMs Assertions Clocks, Resets Synthesis NVIDIA Confidential
  23. 23. Who Writes Assertions? Designers Bottom Up Assumptions Verification Engineers Top-down IntentNVIDIA Confidential
  24. 24. Who Writes Assertions? Designers Bottom Up Assumptions Verification Engineers Top-down Intent Architects The Specification Top Down Assumptions Bottom Up IntentNVIDIA Confidential
  25. 25. Where to Write Assertions The RTL Inline Bound The Testbench Scoreboard Environment E.g. Post Process Log fileNVIDIA Confidential
  26. 26. Where to Write Assertions The RTL Inline Bound The Testbench Scoreboard Environment Post Process Log files The Specification C Models (?)NVIDIA Confidential
  27. 27. Where To Write Specification Assertions Functionality Model RTL Design Performance ModelNVIDIA Confidential
  28. 28. Where To Write Specification Assertions ISS Model Transaction Model Correlation RTL Model Design Performance Model Debug/Triage ModelNVIDIA Confidential
  29. 29. Where To Write Specification Assertions Transaction Model Structural RTL Model Design Performance ModelNVIDIA Confidential
  30. 30. Interfaces Vs State Two approaches to comparing models: Compare “Architectural State” Registers/flops within the design whose existence is required by the specification Compare externally visible behavior Compare interface traffic B. F. Skinner?NVIDIA Confidential
  31. 31. Birth of an IDL Interface Description Language Initially, a language just to define signals Interface a2b clock clk down U valid 1 up U busy 1 down U cmd 24 down U data 32NVIDIA Confidential
  32. 32. Evolution of an IDL Quickly added flow-control protocol abstraction Interface a2b clock clk flow valid_busy down U cmd 24 down U data 32 From this we can generate: Testbench components (BFMs: producers, consumers) Protocol Assertions …NVIDIA Confidential
  33. 33. Continued Evolution of an IDL Separation of packet structure from interface group SOP down U cmd 24 group MOP down U data 32 group EOP down U checksum 32 Interface a2b clock clk flow valid_busy packet SOP, MOP, EOPNVIDIA Confidential
  34. 34. Transaction Assertions in YACC valid_interface_traffic: | valid_interface_traffic transaction; transaction: begin middle end; begin: SOP; middle: | middle MOP; end: EOPNVIDIA Confidential
  35. 35. Cycle Level Assertions in SVA clk cmd SOP MOP MOP EOP sequence valid_trans; (cmd==SOP) (##1 cmd != SOP && cmd != EOP) [*0:$] ##1 cmd == EOP endsequence a_well_formed_transaction: assert @(posedge clk) cmd == SOP |-> sequence (valid_trans)NVIDIA Confidential
  36. 36. Transaction Level Assertions in SVA clk valid cmd SOP MOP MOP EOP sequence valid_transaction; (cmd==SOP) (##1 cmd != SOP && cmd != EOP) [*0:$] ##1 cmd == EOP endsequenceNVIDIA Confidential
  37. 37. Transaction Level Assertions in SVA clk valid busy cmd SOP MOP MOP EOP event sample; always @(posedge clk) if (valid && ! busy) -> sample assert @(sample) cmd == SOP |-> sequence (valid_transaction)NVIDIA Confidential
  38. 38. Transaction Level Assertions in SVA clk valid busy cmd SOP MOP MOP EOP event sample; always @(posedge clk) if (valid && ! busy) -> sample assert @(sample) cmd == BEGIN |-> sequence (valid_transaction)NVIDIA Confidential
  39. 39. Grammer using IDLassert “temporal representation” data SOP => past( EOP ) data MOP => past( SOP | MOP ) data EOP => past( SOP | MOP )assert “bnf representation” bnf SOP -> MOP ** past( SOP.length-1 :sample( SOP ) ) -> EOPNVIDIA Confidential
  40. 40. Multi-Unit Assemblies A B C D E F G a2b b2c c2d d2e e2f f2g A simple pipelineNVIDIA Confidential
  41. 41. Multi-Unit Assemblies a2b b2c c2d A B C D d2e E F G e2f f2g Simple rearrangementNVIDIA Confidential
  42. 42. Multi-Unit Assemblies a2b b2c c2d A B C D d2e E F G e2f f2g Identify units with similar behaviorsNVIDIA Confidential
  43. 43. Multi-Unit Assemblies d2be cf2d D B be2cf C a2be E F A cf2g G Extract common behavior into unified components be2cf === b2c + e2fNVIDIA Confidential
  44. 44. Reusing Interface Definitions A B C D E F G D B C E F A G How to maximize reuse between these two architectures?NVIDIA Confidential
  45. 45. Packets as Traffic Streams group b2c down U data 32 group e2f down U data 32 Interface be2cf clock clk flow valid_credit packet b2c, e2fNVIDIA Confidential
  46. 46. Time Units of Temporal ExpressionsGroup b2c down U value 4 assert value != past( value )Group e2f down U value 4 assert ( value == 0 ) => ( past( value ) != 0 )Interface be2cf packet b2c, e2f assert b2c => ( b2c.value != past( b2c.value :sample(b2c) ) ) assert past( e2f && e2f.value == 0 ) => (b2c && b2c.value != 0)NVIDIA Confidential
  47. 47. Example Renderer MemoryNVIDIA Confidential
  48. 48. The Traffic group mem_write down U address 16 down U data 1 group sync down U shape 2 enum SQUARE, CIRCLE, TRIANGE, BLANK down U radius 3NVIDIA Confidential
  49. 49. Accumulate Memory Stategroup mem_write down U address 16 down U data 1 assign mem[ x = 0 .. 15 ][ y = 0 .. 15 ] = past( data :sample( address == {x,y} ))NVIDIA Confidential
  50. 50. Add Predicatesgroup_more mem_write assign is_circle[ r = 0..7 ] = “&&”( [ x = -8 .. 7 ] [ y = -8 .. 7 ] mem[ x+8 ][ y+8 ] == ( x**2 + y**2 <= r**2 ) )NVIDIA Confidential
  51. 51. Define the interfaceinterface render2memory clock mem_clk flow valid_busy packet sync, mem_write assert “correct sync shape” sync && sync.shape == CIRCLE => past( mem_write.is_circle[ sync.radius ] )NVIDIA Confidential
  52. 52. ExampleRenderer Memory assert “correct sync shape” sync && sync.shape == CIRCLE => past( mem_write.is_circle[ sync.radius ] )NVIDIA Confidential
  53. 53. Summary Efficient Verification Understand Variation Encourage Diversity avoid anarchy Encourage Standardization avoid stagnation Frontload The Schedule Provide Verification with Executable Spec (ESL) C Model, Correlation Model (pre validated) Interfaces, Transactions (definitions for code generators) Assertions, Testpoints (neutral language, pre validated)NVIDIA Confidential
  54. 54. Questionshttp://dave.whipp.name/dv
  55. 55. Summary Architects should write assertions Validated assertions are input to Verification Assertions must be directly reusable across models Manual recoding invites errors Explicitly model the structure that is common to architectural models and to design Tie assertions to these common pointsNVIDIA Confidential
  56. 56. Threading Models Thread Follows Resource Traditional SystemC approach Thread Follows Transaction Think “Multithreaded ISS” Cycle-based Model The “Old” way of doing things A non-threaded version of “thread follows resource” Different models are convenient at various times But SystemC forces you to commit early to one approach Therefore need extra code for transaction level debugNVIDIA Confidential
  57. 57. 70% 70% of schedule is Verification What is correct figure 0% synthesize architectural models, constraints Correct by construction! 100% synthesize verification models, constraints no design needed!NVIDIA Confidential
  58. 58. Transactions Vs Cycles Data min_val (Addr a1, Addr a2) Pipelined Bus { Data d1 = mem_read(a1); Data d2 = mem_read(a2); t2 t3 t4 t1 t5 t6 if (d1 < d2) Address a1 a2 return d1; else Data d1 d2 return d2; }NVIDIA Confidential
  59. 59. Unit Level System Testing Graphics Pipeline (Transaction Model) Stage Stage Stage N-1 N N+1 RTL DIFF UnitNVIDIA Confidential
  60. 60. Avoiding Stagnation New Challenges New Tools New Platforms New People New Ideas Refactoring Testability D.R.Y.NVIDIA Confidential

×