Stop Writing Assertions! Efficient Verification Methodology
Upcoming SlideShare
Loading in...5
×
 

Like this? Share it with your network

Share

Stop Writing Assertions! Efficient Verification Methodology

on

  • 655 views

 

Statistics

Views

Total Views
655
Views on SlideShare
592
Embed Views
63

Actions

Likes
0
Downloads
7
Comments
0

3 Embeds 63

http://www.dvclub.org 58
http://dvclub.org 4
http://dubai.directrouter.com 1

Accessibility

Categories

Upload Details

Uploaded via as Adobe PDF

Usage Rights

© All Rights Reserved

Report content

Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

Cancel
  • Full Name Full Name Comment goes here.
    Are you sure you want to
    Your message goes here
    Processing…
Post Comment
Edit your comment

Stop Writing Assertions! Efficient Verification Methodology Presentation Transcript

  • 1. Stop Writing Assertions!Efficient Verification Methodology Dave Whipp – DVClub SV, August 2008
  • 2. Todays Talk An Overview of Methodology Creation Methodology as User Interface Making ESL work as a Hardware Design Flow Evolution of a Interface Definition Language Transaction Level Assertions Transaction Level DebugNVIDIA Confidential
  • 3. A HW Development Flow Big Paper Spec ISS Model Verification Coverage Debug Design Testbench Checkers Tests RTL Synthesis Clocks, Resets C Model Directed Assertions BFMs, Assertions Random TLMs Scoreboard FormalNVIDIA Confidential
  • 4. User Model for Running Tests Build Run Testbench Predictor * Generate Compare Test Behaviors Run DUTNVIDIA Confidential
  • 5. Part 1Methodology Design
  • 6. Purpose of a Flow Conceptual Framework Where do things live? How do things work? Keep out of the way Don’t make life difficult for people Define Metaphors Subtly influence future directions Anchor for Variation Give people something to complain aboutNVIDIA Confidential
  • 7. Two Philosophies Conformity Emphasize Standardization Diversity Emphasize Innovation Synergy or Conflict?NVIDIA Confidential
  • 8. Understanding Variation To understand what to standardize: you need to understand what not to standardize Personal Preferences Technical Aspects of the Designs Supporting Legacy Seeking the Next Big ThingNVIDIA Confidential
  • 9. Personal Preferences Emacs! viNVIDIA Confidential
  • 10. Personal Preferences Choice of editor doesn’t affect others At least, not much Choice of scripting language has greater impact But is encapsulated A script’s users don’t see the implementation language Choice of HVL affects whole team Can’t write “E” tests for a “Vera” testbench! But a unit testbench isn’t seen by other units A good flow will allow encapsulation of preferences I can go to any unit and build & run its tests Enables rapid localization of infrastructure issuesNVIDIA Confidential
  • 11. Technological DifferencesNVIDIA Confidential
  • 12. Technical Characteristics P Graphics Pipe C I Frame E Video Buffer Off-chip MemoryNVIDIA Confidential
  • 13. Reuse Vs Stagnation Reuse considered Good Avoid reinventing the wheel Build on the shoulders of giants Reuse invites Inertia Reuse can propagate dependencies Dependencies make things harder to change Resistance to change is known as inertia Inertia can lead to Stagnation Improper reuse accumulates dependencies Reused code that is not understood will bit-rot To avoid stagnation, inject agitationNVIDIA Confidential
  • 14. Are Single Paradigm Projects Possible?Paradigm 1 Unit A Unit B Unit B Unit BParadigm 2 Unit C Unit CParadigm 3 Unit D time Project 1 Project 2 Project 3NVIDIA Confidential
  • 15. Watch some Real UsersNVIDIA Confidential Image courtesy of DAZ Productions
  • 16. How to “Watch” Users Meetings and Discussions Coffee-Break Grousing Bug Reports Keep Track of Support Requests create FAQs VNC (Remote Desktop) InstrumentationNVIDIA Confidential
  • 17. Build Time Distribution: 10,000 per sample 100% 90% 80% > 1 hour < 1 hour 70% < 30 min 60% < 15 min < 10 min 50% < 5 min 40% < 2 min < 1 min 30% < 30 sec 20% < 20 sec 10% 0%NVIDIA Confidential
  • 18. Build Time Distribution: 1000 per sample 100% 90% 80% > 1 hour < 1 hour 70% < 30 min 60% < 15 min < 10 min 50% < 5 min 40% < 2 min < 1 min 30% < 30 sec 20% < 20 sec 10% 0%NVIDIA Confidential
  • 19. Part 2ESL for Transaction Design
  • 20. A HW Development Flow (BAD) Big Paper Spec ISS Model Verification Coverage Debug Design Testbench Checkers Tests RTL Synthesis Clocks, Resets C Model Assertions Directed BFMs, Assertions Randoms TLMs ScoreboardNVIDIA Confidential
  • 21. A HW Development Flow (BAD) Big Paper Spec ISS Model Verification Coverage Debug Design Testbench Checkers Tests RTL Synthesis Clocks, Resets C Model Assertions Directed BFMs, Assertions Randoms TLMs ScoreboardNVIDIA Confidential
  • 22. A HW Development Flow (Better) Small Paper Spec Coverage Triage ISS Model ESL Verification C Model Debug Interfaces Testbench Randoms Validation Design AssertionsTLMs RTL ScoreboardsDirected Tests BFMs Assertions Clocks, Resets Synthesis NVIDIA Confidential
  • 23. Who Writes Assertions? Designers Bottom Up Assumptions Verification Engineers Top-down IntentNVIDIA Confidential
  • 24. Who Writes Assertions? Designers Bottom Up Assumptions Verification Engineers Top-down Intent Architects The Specification Top Down Assumptions Bottom Up IntentNVIDIA Confidential
  • 25. Where to Write Assertions The RTL Inline Bound The Testbench Scoreboard Environment E.g. Post Process Log fileNVIDIA Confidential
  • 26. Where to Write Assertions The RTL Inline Bound The Testbench Scoreboard Environment Post Process Log files The Specification C Models (?)NVIDIA Confidential
  • 27. Where To Write Specification Assertions Functionality Model RTL Design Performance ModelNVIDIA Confidential
  • 28. Where To Write Specification Assertions ISS Model Transaction Model Correlation RTL Model Design Performance Model Debug/Triage ModelNVIDIA Confidential
  • 29. Where To Write Specification Assertions Transaction Model Structural RTL Model Design Performance ModelNVIDIA Confidential
  • 30. Interfaces Vs State Two approaches to comparing models: Compare “Architectural State” Registers/flops within the design whose existence is required by the specification Compare externally visible behavior Compare interface traffic B. F. Skinner?NVIDIA Confidential
  • 31. Birth of an IDL Interface Description Language Initially, a language just to define signals Interface a2b clock clk down U valid 1 up U busy 1 down U cmd 24 down U data 32NVIDIA Confidential
  • 32. Evolution of an IDL Quickly added flow-control protocol abstraction Interface a2b clock clk flow valid_busy down U cmd 24 down U data 32 From this we can generate: Testbench components (BFMs: producers, consumers) Protocol Assertions …NVIDIA Confidential
  • 33. Continued Evolution of an IDL Separation of packet structure from interface group SOP down U cmd 24 group MOP down U data 32 group EOP down U checksum 32 Interface a2b clock clk flow valid_busy packet SOP, MOP, EOPNVIDIA Confidential
  • 34. Transaction Assertions in YACC valid_interface_traffic: | valid_interface_traffic transaction; transaction: begin middle end; begin: SOP; middle: | middle MOP; end: EOPNVIDIA Confidential
  • 35. Cycle Level Assertions in SVA clk cmd SOP MOP MOP EOP sequence valid_trans; (cmd==SOP) (##1 cmd != SOP && cmd != EOP) [*0:$] ##1 cmd == EOP endsequence a_well_formed_transaction: assert @(posedge clk) cmd == SOP |-> sequence (valid_trans)NVIDIA Confidential
  • 36. Transaction Level Assertions in SVA clk valid cmd SOP MOP MOP EOP sequence valid_transaction; (cmd==SOP) (##1 cmd != SOP && cmd != EOP) [*0:$] ##1 cmd == EOP endsequenceNVIDIA Confidential
  • 37. Transaction Level Assertions in SVA clk valid busy cmd SOP MOP MOP EOP event sample; always @(posedge clk) if (valid && ! busy) -> sample assert @(sample) cmd == SOP |-> sequence (valid_transaction)NVIDIA Confidential
  • 38. Transaction Level Assertions in SVA clk valid busy cmd SOP MOP MOP EOP event sample; always @(posedge clk) if (valid && ! busy) -> sample assert @(sample) cmd == BEGIN |-> sequence (valid_transaction)NVIDIA Confidential
  • 39. Grammer using IDLassert “temporal representation” data SOP => past( EOP ) data MOP => past( SOP | MOP ) data EOP => past( SOP | MOP )assert “bnf representation” bnf SOP -> MOP ** past( SOP.length-1 :sample( SOP ) ) -> EOPNVIDIA Confidential
  • 40. Multi-Unit Assemblies A B C D E F G a2b b2c c2d d2e e2f f2g A simple pipelineNVIDIA Confidential
  • 41. Multi-Unit Assemblies a2b b2c c2d A B C D d2e E F G e2f f2g Simple rearrangementNVIDIA Confidential
  • 42. Multi-Unit Assemblies a2b b2c c2d A B C D d2e E F G e2f f2g Identify units with similar behaviorsNVIDIA Confidential
  • 43. Multi-Unit Assemblies d2be cf2d D B be2cf C a2be E F A cf2g G Extract common behavior into unified components be2cf === b2c + e2fNVIDIA Confidential
  • 44. Reusing Interface Definitions A B C D E F G D B C E F A G How to maximize reuse between these two architectures?NVIDIA Confidential
  • 45. Packets as Traffic Streams group b2c down U data 32 group e2f down U data 32 Interface be2cf clock clk flow valid_credit packet b2c, e2fNVIDIA Confidential
  • 46. Time Units of Temporal ExpressionsGroup b2c down U value 4 assert value != past( value )Group e2f down U value 4 assert ( value == 0 ) => ( past( value ) != 0 )Interface be2cf packet b2c, e2f assert b2c => ( b2c.value != past( b2c.value :sample(b2c) ) ) assert past( e2f && e2f.value == 0 ) => (b2c && b2c.value != 0)NVIDIA Confidential
  • 47. Example Renderer MemoryNVIDIA Confidential
  • 48. The Traffic group mem_write down U address 16 down U data 1 group sync down U shape 2 enum SQUARE, CIRCLE, TRIANGE, BLANK down U radius 3NVIDIA Confidential
  • 49. Accumulate Memory Stategroup mem_write down U address 16 down U data 1 assign mem[ x = 0 .. 15 ][ y = 0 .. 15 ] = past( data :sample( address == {x,y} ))NVIDIA Confidential
  • 50. Add Predicatesgroup_more mem_write assign is_circle[ r = 0..7 ] = “&&”( [ x = -8 .. 7 ] [ y = -8 .. 7 ] mem[ x+8 ][ y+8 ] == ( x**2 + y**2 <= r**2 ) )NVIDIA Confidential
  • 51. Define the interfaceinterface render2memory clock mem_clk flow valid_busy packet sync, mem_write assert “correct sync shape” sync && sync.shape == CIRCLE => past( mem_write.is_circle[ sync.radius ] )NVIDIA Confidential
  • 52. ExampleRenderer Memory assert “correct sync shape” sync && sync.shape == CIRCLE => past( mem_write.is_circle[ sync.radius ] )NVIDIA Confidential
  • 53. Summary Efficient Verification Understand Variation Encourage Diversity avoid anarchy Encourage Standardization avoid stagnation Frontload The Schedule Provide Verification with Executable Spec (ESL) C Model, Correlation Model (pre validated) Interfaces, Transactions (definitions for code generators) Assertions, Testpoints (neutral language, pre validated)NVIDIA Confidential
  • 54. Questionshttp://dave.whipp.name/dv
  • 55. Summary Architects should write assertions Validated assertions are input to Verification Assertions must be directly reusable across models Manual recoding invites errors Explicitly model the structure that is common to architectural models and to design Tie assertions to these common pointsNVIDIA Confidential
  • 56. Threading Models Thread Follows Resource Traditional SystemC approach Thread Follows Transaction Think “Multithreaded ISS” Cycle-based Model The “Old” way of doing things A non-threaded version of “thread follows resource” Different models are convenient at various times But SystemC forces you to commit early to one approach Therefore need extra code for transaction level debugNVIDIA Confidential
  • 57. 70% 70% of schedule is Verification What is correct figure 0% synthesize architectural models, constraints Correct by construction! 100% synthesize verification models, constraints no design needed!NVIDIA Confidential
  • 58. Transactions Vs Cycles Data min_val (Addr a1, Addr a2) Pipelined Bus { Data d1 = mem_read(a1); Data d2 = mem_read(a2); t2 t3 t4 t1 t5 t6 if (d1 < d2) Address a1 a2 return d1; else Data d1 d2 return d2; }NVIDIA Confidential
  • 59. Unit Level System Testing Graphics Pipeline (Transaction Model) Stage Stage Stage N-1 N N+1 RTL DIFF UnitNVIDIA Confidential
  • 60. Avoiding Stagnation New Challenges New Tools New Platforms New People New Ideas Refactoring Testability D.R.Y.NVIDIA Confidential