0
Dealing with the Three Horrible Problems in Verification Prof. David L. Dill Department of Computer Science Stanford Unive...
An excursion out of the ivory tower <ul><li>0-In, July 1996, initial product design discussions:  There are  three horribl...
Topics <ul><li>Mutation coverage (Certess) </li></ul><ul><li>System-Level Equivalence Checking (Calypto) </li></ul><ul><li...
Typical verification experience Weeks Bugs per week (Based on fabricated data) Functional testing Tapeout Purgatory
Coverage Analysis: Why? <ul><li>What aspects of design haven’t been exercised? </li></ul><ul><ul><li>Guides test improveme...
Coverage Metrics <ul><li>A metric identifies important  </li></ul><ul><ul><li>structures in a design representation </li><...
Code-Based Coverage Metrics <ul><li>On the HDL description </li></ul><ul><ul><li>Line/code block coverage </li></ul></ul><...
Circuit Structure-Based Metrics <ul><li>Toggle coverage : Is each  node in the circuit toggled? </li></ul><ul><li>Register...
Observability problem <ul><li>A buggy assignment may be stimulated, but still missed </li></ul><ul><li>Examples: </li></ul...
Detection terminology <ul><li>To detect a bug </li></ul><ul><ul><li>Stimuli must  activate  buggy logic </li></ul></ul>Ver...
Detection terminology <ul><li>To detect a bug </li></ul><ul><ul><li>Stimuli must  activate  buggy logic </li></ul></ul><ul...
Detection terminology <ul><li>To detect a bug </li></ul><ul><ul><li>Stimuli must  activate  buggy logic </li></ul></ul><ul...
Detection terminology <ul><li>Traditional verification metrics do not account for non-propagated, or non-detected bugs </l...
Mutation testing <ul><li>To evaluate testbench’s bug  detection  ability </li></ul><ul><ul><li>Inject fake bugs into desig...
Certess approach to Mutation Analysis Fault  Model  Analysis Fault  Activation  Analysis Qualify the Verif. Env. Static an...
Avoiding the horrible problems <ul><li>Qualify test framework, not design </li></ul><ul><ul><li>Environment/properties are...
SEC Advantages <ul><li>SEC vs. Simulation </li></ul><ul><ul><li>Simulation is resource intensive, with lengthy run times -...
<ul><li>Enabling ESL™ </li></ul>SLEC™ <ul><li>SLEC comprehensively proves functional equivalence </li></ul><ul><li>Identif...
SLEC Finds Functional Differences in C-C Verification  <ul><li>Customer example  </li></ul><ul><li>Verify HLS model is the...
Design Bugs Caught by SLEC System Bugs Found Application High- Level Synthesis bug in “wait_until()” interpretation DCT Fu...
System-level Formal Verification <ul><li>Sequential Logic Equivalence Checking (SLEC) </li></ul><ul><ul><li>Leverages syst...
High-level Synthesis Bugs found by SLEC Bugs Found Application Shift by an integer in the C-code could be a shift by a neg...
RTL to RTL Verification with  Sequential Differences <ul><li>RTL Pipelining </li></ul><ul><ul><li>Latency & throughput cha...
RTL to RTL Verification with  Sequential Differences <ul><li>RTL Resource Sharing </li></ul><ul><ul><li>State and latency ...
RTL to RTL Verification with  Sequential Differences <ul><li>RTL Re-Timing </li></ul><ul><ul><li>State changes </li></ul><...
Designer’s Dilemma– Efficient Design for Power <ul><li>At 90nm and below, power is becoming the most critical design const...
Addressing Power in the Design Flow <ul><li>Power management  schemes are considered globally as part of the system model ...
Combinational clock gating Sequential clock gating Sequential clock gating Combinational Equivalence Checking Sequential E...
Research <ul><li>Verification currently based on finding and removing bugs. </li></ul><ul><li>Finding bugs earlier in the ...
An Experiment <ul><li>DARPA-sponsored “Smart Memories” project starting up </li></ul><ul><li>Have verification PhD student...
Initial Results: Dismal <ul><li>Used a variety of formal verification tools </li></ul><ul><ul><li>SRI’s PVS system </li></...
Desperate measures required <ul><li>We discarded tools, used pen and paper </li></ul><ul><li>This actually helped! </li></...
What did we learn? <ul><li>Early verification methods need to be nimble </li></ul><ul><ul><li>Must be able to keep up with...
Approach: Perspective-based Verification <ul><li>Need to minimize the number of issues that we tackle at one time. </li></...
Example: Resource dependencies <ul><li>Verify Cache Coherence message system is deadlock free </li></ul><ul><li>Model </li...
Dependency graph (cache & memory)
Resource Dependency Perspective <ul><li>Partial formalization of design </li></ul><ul><ul><li>Relevant details </li></ul><...
Bug found <ul><li>Dependency Cycle found </li></ul><ul><ul><li>Taken into account dependency behavior of </li></ul></ul><u...
Parallel Transaction Perspective <ul><li>Many systems process a set of transactions </li></ul><ul><ul><li>Memory reads/wri...
Parallel Transaction Perspective <ul><li>Partial formalization of design </li></ul><ul><ul><li>Relevant details </li></ul>...
Transaction Diagram Verifier <ul><li>Tool developed for verification of the parallel transaction perspective </li></ul><ul...
TDV <ul><li>User supplies </li></ul><ul><ul><li>Blocks (transaction steps) </li></ul></ul><ul><ul><ul><li>Pre-conditions, ...
Tradeoffs <ul><li>Sacrifices must be made </li></ul><ul><ul><li>Perspectives are necessarily partial </li></ul></ul><ul><u...
The horrible problems <ul><li>Perspectives omit irrelevant details. </li></ul><ul><ul><li>including irrelevant environment...
Conclusions <ul><li>Practical verification technology must take account of the three horrible problems </li></ul><ul><li>P...
Upcoming SlideShare
Loading in...5
×

Dill may-2008

269

Published on

0 Comments
0 Likes
Statistics
Notes
  • Be the first to comment

  • Be the first to like this

No Downloads
Views
Total Views
269
On Slideshare
0
From Embeds
0
Number of Embeds
0
Actions
Shares
0
Downloads
1
Comments
0
Likes
0
Embeds 0
No embeds

No notes for slide
  • When the initial block of design does not meet timing engineers must transform the RTL into a faster implementation . A common technique is re-timing. By moving logic around, long paths can be reduced . However, by moving logic around the value/meaning of state elements is completely changed . Combinatorial formal techniques do not support these changes. SLEC handles this type of design easily Outside of changes, two designs should have identical functionality No requirement for internal statepoints to map/match
  • When the initial block of design does not meet timing engineers must transform the RTL into a faster implementation . A common technique is re-timing. By moving logic around, long paths can be reduced . However, by moving logic around the value/meaning of state elements is completely changed . Combinatorial formal techniques do not support these changes. SLEC handles this type of design easily Outside of changes, two designs should have identical functionality No requirement for internal statepoints to map/match
  • When the initial block of design does not meet timing engineers must transform the RTL into a faster implementation . A common technique is re-timing. By moving logic around, long paths can be reduced . However, by moving logic around the value/meaning of state elements is completely changed . Combinatorial formal techniques do not support these changes. SLEC handles this type of design easily Outside of changes, two designs should have identical functionality No requirement for internal statepoints to map/match
  • Transcript of "Dill may-2008"

    1. 1. Dealing with the Three Horrible Problems in Verification Prof. David L. Dill Department of Computer Science Stanford University
    2. 2. An excursion out of the ivory tower <ul><li>0-In, July 1996, initial product design discussions: There are three horrible problems in verification: </li></ul><ul><li>Specifying the properties to be checked </li></ul><ul><li>Specifying the environment </li></ul><ul><li>Computational complexity of attaining high coverage </li></ul><ul><li>Up to then, I had assumed that the first two were someone else’s problem, and focused on the last. </li></ul><ul><li>I still think this is a reasonable framework for thinking about verification. </li></ul>
    3. 3. Topics <ul><li>Mutation coverage (Certess) </li></ul><ul><li>System-Level Equivalence Checking (Calypto) </li></ul><ul><li>Integrating verification into early system design (research). </li></ul><ul><li>Conclusions </li></ul>
    4. 4. Typical verification experience Weeks Bugs per week (Based on fabricated data) Functional testing Tapeout Purgatory
    5. 5. Coverage Analysis: Why? <ul><li>What aspects of design haven’t been exercised? </li></ul><ul><ul><li>Guides test improvement </li></ul></ul><ul><li>How comprehensive is the verification so far? </li></ul><ul><ul><li>Stopping criterion </li></ul></ul><ul><li>Which aspects of the design have not been well-tested? </li></ul><ul><ul><li>Helps allocate verification resources. </li></ul></ul>
    6. 6. Coverage Metrics <ul><li>A metric identifies important </li></ul><ul><ul><li>structures in a design representation </li></ul></ul><ul><ul><ul><li>HDL lines, FSM states, paths in netlist </li></ul></ul></ul><ul><ul><li>classes of behavior </li></ul></ul><ul><ul><ul><li>Transactions, event sequences </li></ul></ul></ul><ul><li>Metric classification based on level of representation. </li></ul><ul><ul><li>Code-based metrics (HDL code) </li></ul></ul><ul><ul><li>Circuit structure-based metrics (Netlist) </li></ul></ul><ul><ul><li>State-space based metrics (State transition graph) </li></ul></ul><ul><ul><li>Functionality-based metrics (User defined tasks) </li></ul></ul><ul><ul><li>Spec-based metrics (Formal or executable spec) </li></ul></ul>
    7. 7. Code-Based Coverage Metrics <ul><li>On the HDL description </li></ul><ul><ul><li>Line/code block coverage </li></ul></ul><ul><ul><li>Branch/conditional coverage </li></ul></ul><ul><ul><li>Expression coverage </li></ul></ul><ul><ul><li>Path coverage </li></ul></ul><ul><li>Useful guide for writing test cases </li></ul><ul><li>Little overhead </li></ul><ul><li>Inadequate in practice </li></ul><ul><li>always @ (a or b or s) // mux </li></ul><ul><li>begin </li></ul><ul><li>if ( ~s && p ) </li></ul><ul><li>d = a; </li></ul><ul><li>r = x </li></ul><ul><li>else if( s ) </li></ul><ul><li>d = b; </li></ul><ul><li>else </li></ul><ul><li>d = 'bx; </li></ul><ul><li>if( sel == 1 ) </li></ul><ul><li>q = d; </li></ul><ul><li>else if ( sel == 0 ) </li></ul><ul><li>q = z </li></ul>
    8. 8. Circuit Structure-Based Metrics <ul><li>Toggle coverage : Is each node in the circuit toggled? </li></ul><ul><li>Register activity : Is each register initialized? Loaded? Read? </li></ul><ul><li>Counters : Are they reset? Do they reach the max/min value? </li></ul><ul><li>Register-to-register interactions : Are all feasible paths exercised? </li></ul><ul><li>Datapath-control interface : Are all possible combinations of control and status signals exercised? </li></ul>(0-In checkers have these kinds of measures.) s init s 3 s 4 s 2 s 5 s 6 Control Datapath
    9. 9. Observability problem <ul><li>A buggy assignment may be stimulated, but still missed </li></ul><ul><li>Examples: </li></ul><ul><li>Wrong value generated speculatively, but never used. </li></ul><ul><li>Wrong value is computed and stored in register </li></ul><ul><ul><li>Read 1M cycles later, but simulation doesn’t run that long. </li></ul></ul>
    10. 10. Detection terminology <ul><li>To detect a bug </li></ul><ul><ul><li>Stimuli must activate buggy logic </li></ul></ul>Verification Environment Compare Reference Model Stimuli Activation Bug Design under Verification
    11. 11. Detection terminology <ul><li>To detect a bug </li></ul><ul><ul><li>Stimuli must activate buggy logic </li></ul></ul><ul><ul><li>The bug must propagate to a checker </li></ul></ul>Verification Environment Compare Reference Model Stimuli Propagation Activation Bug Design under Verification
    12. 12. Detection terminology <ul><li>To detect a bug </li></ul><ul><ul><li>Stimuli must activate buggy logic </li></ul></ul><ul><ul><li>The bug must propagate to a checker </li></ul></ul><ul><ul><li>The checker must detect the bug </li></ul></ul>Verification Environment Compare Reference Model Stimuli Propagation Detection Activation Bug Design under Verification
    13. 13. Detection terminology <ul><li>Traditional verification metrics do not account for non-propagated, or non-detected bugs </li></ul>Traditional verification metrics No visibility with traditional metrics Verification Environment Compare Reference Model Stimuli Propagation Detection Activation Bug Design under Verification
    14. 14. Mutation testing <ul><li>To evaluate testbench’s bug detection ability </li></ul><ul><ul><li>Inject fake bugs into design (“mutations”). </li></ul></ul><ul><ul><li>Simulate and see whether they are detected. </li></ul></ul><ul><ul><li>If not, there is a potential gap in the testbench. </li></ul></ul><ul><li>There can be many kinds of mutations </li></ul><ul><ul><li>“ Stuck at” faults </li></ul></ul><ul><ul><li>Wrong logical or other operators </li></ul></ul><ul><li>Idea originates in software testing </li></ul><ul><ul><li>But is obviously related to testability. </li></ul></ul><ul><li>Efficient implementation is a challenge. </li></ul>
    15. 15. Certess approach to Mutation Analysis Fault Model Analysis Fault Activation Analysis Qualify the Verif. Env. Static analysis of the design Analysis of the verification environment behavior Measure the ability of the verification environment to detect mutations Iterate if needed Report Report Report
    16. 16. Avoiding the horrible problems <ul><li>Qualify test framework, not design </li></ul><ul><ul><li>Environment/properties are in existing test bench. </li></ul></ul><ul><li>High-quality coverage metric targets resources at maximizing useful coverage. </li></ul>
    17. 17. SEC Advantages <ul><li>SEC vs. Simulation </li></ul><ul><ul><li>Simulation is resource intensive, with lengthy run times - SEC runs orders of magnitude faster than simulation </li></ul></ul><ul><ul><li>Vector generation effort-laden, may be source of errors – SEC requires minimal setup, no test vectors </li></ul></ul><ul><ul><li>Simulation output often requires further processing for answers – SEC is exhaustive (all sequences over all time) </li></ul></ul><ul><li>SEC vs. Property Checkers </li></ul><ul><ul><li>Properties are created to convey specification requirements – SEC uses the golden model as the specification </li></ul></ul><ul><ul><li>Properties are often incomplete, and not independently verifiable </li></ul></ul><ul><ul><li>Properties are time consuming to construct </li></ul></ul>
    18. 18. <ul><li>Enabling ESL™ </li></ul>SLEC™ <ul><li>SLEC comprehensively proves functional equivalence </li></ul><ul><li>Identifies design differences (bugs) </li></ul><ul><li>Supports sequential design changes </li></ul><ul><ul><li>State Changes </li></ul></ul><ul><ul><li>Temporal differences </li></ul></ul><ul><ul><li>I/O differences </li></ul></ul>= ? Reference Model Implementation Model
    19. 19. SLEC Finds Functional Differences in C-C Verification <ul><li>Customer example </li></ul><ul><li>Verify HLS model is the functionally equivalent to the reference model </li></ul><ul><li>Simulation uncovered no differences - for the given test-bench </li></ul><ul><li>SLEC System found differences between the two models </li></ul><ul><ul><li>Reference model was incorrect </li></ul></ul><ul><ul><ul><li>Probable corner case not easily detectable by simulation </li></ul></ul></ul><ul><li>SLEC System finds all possible errors or inconsistencies. Simulation is not exhaustive, and therefore cannot fully prove equivalence. </li></ul><ul><li>Typical functional differences introduced during refinement </li></ul><ul><ul><li>Code optimization for HLS </li></ul></ul><ul><ul><li>Datapath word size optimization </li></ul></ul><ul><ul><li>Ambiguous ESL code </li></ul></ul><ul><ul><ul><li>Ex: Out of array bounds </li></ul></ul></ul>Behavioral C/C++ HLS C/C++ wrapper HLS Model wrapper Ref Model C to C Verification Failed! DIFFERENCE FOUND! User Defined Input Constraints Reference Model HLS Input Code
    20. 20. Design Bugs Caught by SLEC System Bugs Found Application High- Level Synthesis bug in “wait_until()” interpretation DCT Function High-Level Synthesis bug in array’s access range Wireless baseband Design bug in logic on asynchronous reset line Video processing Design bug in normalization of operands Custom DSP Block Design bug at proof depth = 1. Image Resizer High-Level Synthesis bug in sign extension Video processing
    21. 21. System-level Formal Verification <ul><li>Sequential Logic Equivalence Checking (SLEC) </li></ul><ul><ul><li>Leverages system-level verification </li></ul></ul><ul><ul><li>Comprehensive verification – 100% coverage </li></ul></ul><ul><ul><li>Quick setup - no testbenches required </li></ul></ul><ul><ul><li>Rapid results – eliminates long regressions </li></ul></ul><ul><ul><li>Focused debug – short counter examples </li></ul></ul><ul><li>Why is it needed </li></ul><ul><ul><li>Independent verification </li></ul></ul><ul><ul><li>Find bugs caused by language ambiguities or incorrect synthesis constraints </li></ul></ul><ul><ul><ul><li>Shift left by -1 </li></ul></ul></ul><ul><ul><ul><li>Divide by zero </li></ul></ul></ul><ul><ul><li>Verify RTL ECO’s </li></ul></ul><ul><ul><li>Parallels current RTL synthesis methodology </li></ul></ul>
    22. 22. High-level Synthesis Bugs found by SLEC Bugs Found Application Shift by an integer in the C-code could be a shift by a negative number which is undefined in C Multi-media processor Dead end states created Multi-media Processor Combinational loops created FFT Shift left or right by N bits, when the value being shifted is less than N bits Ultra wideband Filter Divide by zero defined in RTL, but undefined in C code Quantize
    23. 23. RTL to RTL Verification with Sequential Differences <ul><li>RTL Pipelining </li></ul><ul><ul><li>Latency & throughput changes </li></ul></ul><ul><ul><li>Clock speed enhancement </li></ul></ul>cmd1 data1 calcA1 out1 calcB1 cmd2 data2 calcA2 out2 calcB2 cmd3 data3 calcA3 out3 calcB3 cmd4 data4 calcA4 out4 calcB4 cmd1 data1 calc1 out1 cmd2 data2 calc2 out2 cmd3 calc3 out3 cmd4 data4 calc4 out4 data3 Verified Equivalent Or Counter-Example = ?
    24. 24. RTL to RTL Verification with Sequential Differences <ul><li>RTL Resource Sharing </li></ul><ul><ul><li>State and latency changes </li></ul></ul><ul><ul><li>Size optimization </li></ul></ul>Verified Equivalent Or Counter-Example A B C Sum + + clk reset B C Sum + A = ?
    25. 25. RTL to RTL Verification with Sequential Differences <ul><li>RTL Re-Timing </li></ul><ul><ul><li>State changes </li></ul></ul><ul><ul><li>Slack adjustment </li></ul></ul><ul><li>Allows micro-architecture modifications without breaking testbenches </li></ul>Verified Equivalent Or Counter-Example C2 C1 C2 C1 D Q Comb. Logic C2 C1 C2 C1 Comb. Logic D Q Reduced Comb. Logic Comb. Logic = ?
    26. 26. Designer’s Dilemma– Efficient Design for Power <ul><li>At 90nm and below, power is becoming the most critical design constraint </li></ul><ul><ul><li>Exponential increase in leakage power consumption </li></ul></ul><ul><ul><li>Quadratic increase in power density </li></ul></ul><ul><li>Clock Gating is the most common design technique used for reducing power </li></ul><ul><ul><li>Designers manually add clock gating to control dynamic power </li></ul></ul><ul><li>Clock gating is most efficiently done at the RTL level, but is error prone </li></ul><ul><ul><li>Mistakes in implementation cause delays and re-spins </li></ul></ul><ul><ul><li>Difficult to verify with simulation regressions </li></ul></ul><ul><ul><ul><li>Requires modifications to testbenches </li></ul></ul></ul><ul><ul><ul><li>Insufficient coverage of clock gating dependencies </li></ul></ul></ul><ul><ul><ul><li>Aggressive clock gating approaches sometimes rejected due to verification complexity </li></ul></ul></ul>
    27. 27. Addressing Power in the Design Flow <ul><li>Power management schemes are considered globally as part of the system model and initial RTL functionality </li></ul><ul><ul><li>Sleep modes </li></ul></ul><ul><ul><li>Power down </li></ul></ul><ul><li>Power optimizations are local changes made to RTL that do not effect the design functionality </li></ul><ul><ul><li>Disabling previous pipeline stages when the data is not used </li></ul></ul><ul><ul><li>Data dependent computation, like multiple by zero </li></ul></ul>Physical Implementation Manual RTL Optimization RTL High level Synthesis or Manual Creation Optimized RTL System Model
    28. 28. Combinational clock gating Sequential clock gating Sequential clock gating Combinational Equivalence Checking Sequential Equivalence Checking CG CG CG en clk en clk CG CG CG CG
    29. 29. Research <ul><li>Verification currently based on finding and removing bugs. </li></ul><ul><li>Finding bugs earlier in the design process would be beneficial </li></ul><ul><ul><li>Early descriptions (protocol, microarchitecture) are smaller, more tractable </li></ul></ul><ul><ul><li>Early bugs are likely to be serious, possibly lethal </li></ul></ul><ul><ul><li>Bug cost goes up by >10x at each stage of design. </li></ul></ul><ul><li>People have been saying this for years. </li></ul><ul><li>Why can’t we start verifying at the beginning of the design? </li></ul>
    30. 30. An Experiment <ul><li>DARPA-sponsored “Smart Memories” project starting up </li></ul><ul><li>Have verification PhD student (Jacob Chang) work with system designers </li></ul><ul><ul><li>Try to verify subsystems as soon as possible. </li></ul></ul><ul><ul><li>Understand what “keeps designers awake at night.” </li></ul></ul><ul><ul><li>Try to understand “Design for verification” (willing to trade off some system efficiency for verification efficiency). </li></ul></ul>
    31. 31. Initial Results: Dismal <ul><li>Used a variety of formal verification tools </li></ul><ul><ul><li>SRI’s PVS system </li></ul></ul><ul><ul><li>Cadence SMV </li></ul></ul><ul><li>Did some impressive verification work </li></ul><ul><li>Didn’t help the design much </li></ul><ul><ul><li>By the time something was verified, design had changed. </li></ul></ul><ul><ul><li>We know this would be a problem, but our solutions weren’t good enough </li></ul></ul>
    32. 32. Desperate measures required <ul><li>We discarded tools, used pen and paper </li></ul><ul><li>This actually helped! </li></ul><ul><li>Real bugs were found </li></ul><ul><li>Design principles were clarified </li></ul><ul><li>Designers started listening </li></ul>
    33. 33. What did we learn? <ul><li>Early verification methods need to be nimble </li></ul><ul><ul><li>Must be able to keep up with design changes. </li></ul></ul><ul><li>Existing formal methods are not nimble. </li></ul><ul><ul><li>Require comprehensive descriptions </li></ul></ul><ul><ul><li>High level of abstraction helps… </li></ul></ul><ul><ul><li>But one description still takes on too many issues </li></ul></ul><ul><ul><li>So design changes necessitate major changes in descriptions – too slow! </li></ul></ul>
    34. 34. Approach: Perspective-based Verification <ul><li>Need to minimize the number of issues that we tackle at one time. </li></ul><ul><li>Perspective : Minimal high-level formalization of a design to analyze a particular class of properties. </li></ul><ul><ul><li>Perspectives should be based on designer’s abstractions </li></ul></ul><ul><ul><ul><li>What does he/she draw on the whiteboard? </li></ul></ul></ul><ul><ul><li>Should capture designer’s reasoning about correctness </li></ul></ul>
    35. 35. Example: Resource dependencies <ul><li>Verify Cache Coherence message system is deadlock free </li></ul><ul><li>Model </li></ul><ul><ul><li>Dependency graph and check for cycles </li></ul></ul><ul><li>Analysis method </li></ul><ul><ul><li>Search for cycles </li></ul></ul><ul><ul><li>In this case: by hand! </li></ul></ul><ul><li>System-level deadlocks are notoriously hard to find using conventional formal verification tools. </li></ul>
    36. 36. Dependency graph (cache & memory)
    37. 37. Resource Dependency Perspective <ul><li>Partial formalization of design </li></ul><ul><ul><li>Relevant details </li></ul></ul><ul><ul><ul><li>Request buffers dependency </li></ul></ul></ul><ul><ul><ul><li>Protocol dependencies </li></ul></ul></ul><ul><ul><ul><ul><li>e.g. cancel must follow all other SyncOp commands </li></ul></ul></ul></ul><ul><ul><ul><li>Virtual channel in networks </li></ul></ul></ul><ul><ul><li>Irrelevant details </li></ul></ul><ul><ul><ul><li>Network ordering requirements </li></ul></ul></ul><ul><ul><ul><li>Cache controller ordering requirements </li></ul></ul></ul><ul><ul><ul><li>Buffer implementation </li></ul></ul></ul><ul><li>One class of verification properties </li></ul><ul><ul><li>Deadlock free </li></ul></ul><ul><li>Captures why the property is correct </li></ul><ul><ul><li>Ensure no cycle in resource dependency </li></ul></ul>
    38. 38. Bug found <ul><li>Dependency Cycle found </li></ul><ul><ul><li>Taken into account dependency behavior of </li></ul></ul><ul><ul><ul><li>Virtual channel </li></ul></ul></ul><ul><ul><ul><li>Memory controller </li></ul></ul></ul><ul><ul><ul><li>Cache controller </li></ul></ul></ul><ul><li>Easy to find once formal model is constructed </li></ul><ul><ul><li>Hard to find using simulation </li></ul></ul><ul><ul><ul><li>All channels must be congested </li></ul></ul></ul><ul><ul><ul><li>Bug found before implementation </li></ul></ul></ul>Cache Controller Memory Controller Cache Controller SyncMiss Sync Op Unsuccessful SyncMiss Sync Op Successful Wake Up Replay Replay
    39. 39. Parallel Transaction Perspective <ul><li>Many systems process a set of transactions </li></ul><ul><ul><li>Memory reads/writes/updates </li></ul></ul><ul><ul><li>Packet processing/routing </li></ul></ul><ul><li>User thinks of transactions as non-interfering processes </li></ul><ul><li>Hardware needs to maintain this illusion. </li></ul><ul><li>Model: State transaction diagram </li></ul><ul><li>Analysis: Systematically check whether one transaction can interfere with another. </li></ul><ul><li>Several important bugs were found by manually applying this method. </li></ul>
    40. 40. Parallel Transaction Perspective <ul><li>Partial formalization of design </li></ul><ul><ul><li>Relevant details </li></ul></ul><ul><ul><ul><li>Effect of transition on self and others </li></ul></ul></ul><ul><ul><ul><li>Locking mechanism </li></ul></ul></ul><ul><ul><li>Irrelevant details </li></ul></ul><ul><ul><ul><li>Timing and ordering information </li></ul></ul></ul><ul><ul><ul><li>Buffering issues </li></ul></ul></ul><ul><ul><ul><li>Deadlock issues </li></ul></ul></ul><ul><li>Targets on one verification property </li></ul><ul><ul><li>Same behavior of single process in a multi-process environment </li></ul></ul><ul><li>Captures why the property is correct </li></ul><ul><ul><li>Interrupts are conflict free </li></ul></ul>
    41. 41. Transaction Diagram Verifier <ul><li>Tool developed for verification of the parallel transaction perspective </li></ul><ul><li>User input </li></ul><ul><ul><li>Invariants </li></ul></ul><ul><ul><li>Transition guards </li></ul></ul><ul><ul><li>Transition state changes </li></ul></ul><ul><li>Invariants easy to see it’s true for single process </li></ul><ul><li>TDV verifies invariant for single process, plus </li></ul><ul><ul><li>Invariants are true even if other processes execute at the same time </li></ul></ul>
    42. 42. TDV <ul><li>User supplies </li></ul><ul><ul><li>Blocks (transaction steps) </li></ul></ul><ul><ul><ul><li>Pre-conditions, post-conditions, guards, assignments </li></ul></ul></ul><ul><ul><li>Links between blocks (control flow) </li></ul></ul><ul><li>Tool loops through all pairs of block </li></ul><ul><ul><li>Construct the verification tasks </li></ul></ul><ul><ul><li>Verify the tasks through another tool </li></ul></ul><ul><ul><ul><li>STP decision procedure </li></ul></ul></ul><ul><li>Not a model checker </li></ul><ul><ul><li>Verifies unbounded number of transactions </li></ul></ul><ul><ul><li>Uses theorem-proving technology. </li></ul></ul>
    43. 43. Tradeoffs <ul><li>Sacrifices must be made </li></ul><ul><ul><li>Perspectives are necessarily partial </li></ul></ul><ul><ul><li>Not easy to link perspectives to RTL </li></ul></ul><ul><ul><li>Not easy to link perspectives to each other </li></ul></ul><ul><li>…but, at least, you can verify or find bugs while their still relevant to the design! </li></ul>
    44. 44. The horrible problems <ul><li>Perspectives omit irrelevant details. </li></ul><ul><ul><li>including irrelevant environmental constraints. </li></ul></ul><ul><li>Properties are at the level the designer thinks, so they are easier to extract </li></ul><ul><li>Computational complexity reduced as well </li></ul>
    45. 45. Conclusions <ul><li>Practical verification technology must take account of the three horrible problems </li></ul><ul><li>Products currently on the market do this in innovative ways </li></ul><ul><ul><li>Coverage analysis that is a closer match to actual bug-finding ability </li></ul></ul><ul><ul><ul><li>Evaluates existing verification environment </li></ul></ul></ul><ul><ul><li>System-level equivalence checking avoids need to add assertions </li></ul></ul><ul><ul><ul><li>Environmental constraint problem reduced. </li></ul></ul></ul><ul><li>We need a new perspective on system-level verification  </li></ul>
    1. A particular slide catching your eye?

      Clipping is a handy way to collect important slides you want to go back to later.

    ×