Basics of Functional Verification - Arrow Devices

2,548
-1

Published on

Are you new to functional verification? Or do you need a refresher? This presentation takes you through the basics of functional verification - overall scope and process with examples. Also included are some tips on do's and don'ts!

Published in: Technology, Design

Basics of Functional Verification - Arrow Devices

  1. 1. June 2014 Basics of Functional Verification Anand Shirahatti @ashirahatti
  2. 2. | 2 Contents ▪  Verification Philosophy ▪  Familiarity ▪  Scope of Verification ▪  Functional Verification as a Career ▪  Overview of Process ▪  Some Examples: The 3 Musketeers ▪  Do’s and Don’t’s ▪  State of the Art
  3. 3. | 3 The Verification Philosophy KNOW EVOLUTION to understand why things are the way they are! GET THE DNA to evolve further GO BEYOND basics to get productive
  4. 4. | 4 Contents ▪  Verification Philosophy ▪  Familiarity ▪  Scope of Verification ▪  Functional Verification as a Career ▪  Overview of Process ▪  Some Examples: The 3 Musketeers ▪  Do’s and Don’t’s ▪  State of the Art
  5. 5. | 5 Basic Lab Verification How did you Verify ICs used during the Digital Design Lab ? -  Covered using the Truth table by giving various inputs and checking outputs -  Downsides are -  Time consuming -  Error prone -  Repeatability is not automatic
  6. 6. | 6 Have I seen a test bench? Yes!
  7. 7. | 7 Contents ▪  Verification Philosophy ▪  Familiarity ▪  Scope of Verification ▪  Functional Verification as a Career ▪  Overview of Process ▪  Some Examples: The 3 Musketeers ▪  Do’s and Don’t’s ▪  State of the Art
  8. 8. | 8 •  Four major types of verification : 1.  Functional 2.  Timing 3.  Test 4.  Equivalence •  Functional verification ensures correct logic performance - under all conditions •  70 % of chip cycle is verification and more than 70 % re-spins are functional bugs •  Functional verification is very time consuming and thus is always on the critical path Scope of Verification (1/2)
  9. 9. | 9 •  Design technologies are evolving in Moore’s Law friendly. •  Functional verification technologies are yet to catch up with the advancement happening in the are of Design •  Functional Verification Tools and methodologies are attempting to reduce the overall verification time by enabling parallelism of effort, higher abstraction levels and automation •  There is no single formula of success Scope of Verification (2/2)
  10. 10. | 10 Contents ▪  Verification Philosophy ▪  Familiarity ▪  Scope of Verification ▪  Functional Verification as a Career ▪  Overview of Process ▪  Some Examples: The 3 Musketeers ▪  Do’s and Don’t’s ▪  State of the Art
  11. 11. | 11 DESIGN Functional Verification as a Career Before Verification After Verification Makes the world a better place!
  12. 12. | 12 •  Knows the design •  Contributes to architecture •  Gives feedback as first user •  Is Creative, Logical and Lateral •  Programmer •  “Quality cop” •  Bridges specification, architecture & design Hats worn by a Verification Engineer
  13. 13. | 13 Contents ▪  Verification Philosophy ▪  Familiarity ▪  Scope of Verification ▪  Functional Verification as a Career ▪  Overview of Process ▪  Some Examples: The 3 Musketeers ▪  Do’s and Don’t’s ▪  State of the Art
  14. 14. | 14 1.  Verification plan 2.  Test bench 3.  Bring up 4.  Debug 5.  Regressions 6.  Sign off Process of Functional Verification 1 2 3 4 5 6
  15. 15. | 15 •  Create varied stimuli and check if the response is as expected •  Ensure design is not doing anything unexpected •  Look at both Functional specifications and Micro- architecture specifications (Black box & White box) Stimuli Response Verification Plan (1/2)1
  16. 16. | 16 •  Test plan: Create the list of programmable parameters and features 1.  Normal operation : Basic & Concurrent 2.  Corner cases 3.  Error cases 4.  Negative cases 5.  Stress design cases •  Functional coverage plan: Capture important features and parameters •  Strategy to execute the test plan Verification Plan (2/2)1
  17. 17. | 17 Every Test Bench Should have the following characteristics •  Ability to exercise all features and parameters •  Ability achieve the desired coverage •  Necessary abstraction •  Scalability •  Balance ease of verification vs. reality •  Ease of understanding and maintenance •  Ease of debugging – Developer vs User •  Developer is interested in Internal architecture •  User is interested in only black box view •  Easily controllable and observable Test Bench - Architecture2
  18. 18. | 18 External interfaces of DUT •  Reset and Clock generators •  Bus functional models/ drivers/monitors DUT Ports BFM Driver Test Bench - Components (1/3)2
  19. 19. | 19 Functionality •  API or Transactors •  Scoreboard •  Golden models •  End of test •  Assertions •  Functional coverage DUT Ports BFM Driver SB Reg Xfer Test Bench - Components (2/3)2
  20. 20. | 20 Stimuli : •  Randomized parameters •  Constrained random gens •  Self checking directed test DUT Ports BFM Driver SB Reg Xfer CFG GEN TES T Test Bench - Components (3/3)2
  21. 21. | 21 •  Get the clocks and resets up •  No ‘X’ & ‘Z’ on critical inputs, Tie them off if needed •  Get all critical interfaces of DUT up •  Start enabling features in its basic form and introduce variables one at a time Design and Test Bench Bring Up3
  22. 22. | 22 •  Done when things not going as expected •  Takes up 30% of functional verification activity •  First step is to isolate between the DUT/TB •  Isolation is done with help of logging •  Use regular expression friendly logging •  Extract stimuli and response from log •  If its not isolatable from log look at waves •  Benchmark debug is to be able to suggest the design fix •  Verification engineer is owner of bug till it is fixed Debug4
  23. 23. | 23 •  Intersection among features •  Introduction of new features can cause a domino effect of failures on previously passing ones •  Protect what has been working by running a set of critical tests before every update •  Periodically complete run of all tests to maintain the health •  Random tests run with seeds Regressions5
  24. 24. | 24 •  Verification never really completes •  Minimize risk using metrics •  100 % code coverage •  100 % functional coverage •  All directed tests passing •  Randoms having zero failures or acceptable failure rate over few weeks •  No critical bugs found over last few weeks Sign Off6
  25. 25. | 25 Contents ▪  Verification Philosophy ▪  Familiarity ▪  Scope of Verification ▪  Functional Verification as a Career ▪  Overview of Process ▪  Some Examples: The 3 Musketeers ▪  Do’s and Don’t’s ▪  State of the Art
  26. 26. | 26 •  Action begins… •  Lets demonstrate what we have learnt by attacking The Three Musketeers described below •  Each of these will take you through complete flow and also focus on clarifying one aspect we have learnt in previous slides 1.  Counters – 2-bit and 256-bit •  Verification strategy 2.  FIFO Verification •  Test cases and assertions 3.  UART Verification •  Test bench architecture The 3 Musketeers
  27. 27. | 27 Test plan ▪ Make sure on reset assertion count can reset to 0 ▪ Counter can roll back to 0 after max ▪ Counter can count through all counts Verification strategy ▪ Do we need random? ▪ Use the directed tests Test bench ▪ Clock and reset generator ▪ 2 directed tests Counters: 2-Bit Counter Verification Strategy
  28. 28. | 28 Test plan: ▪ Make sure on reset assertion count can reset to 0 ▪ Counter can roll back to 0 after max ▪ Counter can count through all counts Verification strategy: ▪ Do we need random? ▪ Sequentially counting all counts 2 ** 256 = 1e77 would take almost forever Counters: 256-Bit Counter Verification Strategy (1/3)
  29. 29. | 29 Bring up ▪ Start off with the pre load count as 0 ▪ Disable the intermediate reset generation ▪ Make sure counter can count up to random count after initial reset Enabling random ▪ Enable the random pre-load count ▪ Enable intermediate reset generation ▪ Complete the directed negative test Closure ▪ Code and functional coverage defined above are met ▪ 50 seeded random regressions are clean over couple of weeks Counters: 256-Bit Counter Verification Strategy (2/3)
  30. 30. | 30 •  Start from random value and count up to another random value •  Divide count range into 4 and make sure each range covers some counts •  All the bits should be toggling •  Count from Max to 0 •  Reset during random count •  Bias the design change for pre load •  How do you handle condition of reset and enable asserted together ? Counters: 256-Bit Counter Verification Strategy (3/3)
  31. 31. | 31 •  Identify variables that need randomization –  Random pre-load count –  Random count up value to end the test –  Control to enable/disable intermediate reset generation –  Random count at which reset needs to be generated ▫  All random stimulus should have control to make it directed if needed •  Clock generator •  Counter model with load and reset control - This will act as a score board. •  The model’s output will be used for check - Comparing at every clock will ensure that the bug’s can be caught immediately Counters: 256-Bit Counter Test Bench
  32. 32. | 32 count <= count + 1 •  Models are behavioral code •  Simpler to write •  Smaller than DUT •  Quicker to write •  Lesser code means lesser bugs! Counters: 256-Bit Counter DUT vs Model
  33. 33. | 33 FIFO FULL PUSH DATA_IN POP EMPT Y DATA_OU T CLK RST_ •  Both read and write side use same clock •  Ordering •  Full & empty •  Ask : What happens on push@full and pop@empty? Synchronous FIFO
  34. 34. | 34 Normal operation ▪  Order of data pushed should match POP ▪  After N PUSH and 0 POP, FIFO FULL assertion ▪  Num of PUSH == Num of POP EMPTY assertion ▪  Stress : PUSH and POP at same time ▪  Reset intermediately Negative case ▪  PUSH on FULL and POP on EMPTY Implementation specific ▪  Internal counter alignments for FULL/EMPTY generation Always true conditions [Assertion] ▪  Both FULL and EMPTY should never get asserted together ▪  Number of clocks between the assertion of Write on input to de-assertion of empty on output - This is also indicative of performance of the FIFO. Synchronous FIFO - Test Plan
  35. 35. | 35 Normal FIFO operation should be covered with the random approach ▪ Random delay on the POP side will lead to FULL conditions on PUSH side ▪ Random delay on PUSH and POP side together will lead to correct checking of FULL/EMPTY conditions on various read and write counter alignments ▪ Data pushed should be randomized - Cover for all toggling bits Negative and intermediate reset cases can be covered as directed case ▪ PUSH when full and POP when empty ▪ Reset during the various combinations of full/empty Synchronous FIFO – Verification Strategy
  36. 36. | 36 DUT DRV PUSH DRV POPSCOREBOARD GEN PUSH GEN POP RND TESTS DIR TESTS Synchronous FIFO – Test Bench (1/4)
  37. 37. | 37 DRV PUSH ▪ Handles the push side interface of FIFO ▪ push_data(data)/is_full() DRV POP ▪ Handles the pop side interface of FIFO ▪ pop_data(data)/is_empty() GEN PUSH ▪ Uses the PUSH DRV and generates specified number of PUSH ▪ Injects random delay between the PUSH GEN POP ▪ Uses the PUSH DRV and generates specified number of PUSH ▪ Injects random delay between the PUSH Synchronous FIFO – Test Bench (2/4)
  38. 38. | 38 Scoreboard ▪ Model of FIFO - Implemented in the behavioral code ▪ Connected to test bench environment using the call backs ▪ Drivers indicate the scoreboard when PUSH and POP happens ▪ On PUSH, scoreboard will store the data pushed - This is “golden data” ▪ On POP, scoreboard will compare the data popped from design with the data stored internally and indicate if there are mismatches - This will verify both ordering and data integrity ▪ Scoreboard should have capability to get disabled - This will be used during some of the directed tests for negative cases and resets Synchronous FIFO – Test Bench (3/4)
  39. 39. | 39 Random tests ▪ Simplest level just sets up counts of FIFO operations ▪ Set the delays between PUSH/POP ▪ With and without random resets enabled Directed tests ▪ Negative cases directly using the API’s of DRV PUSH and DRV POP End of test ▪ Random tests check specified count completed ▪ Make sure at the end FIFO is its default state Synchronous FIFO – Test Bench (4/4)
  40. 40. | 40 •  Confirm clocks and resets are running and no ‘x’ on any input •  Start off with a simple directed test that disables the scoreboard and does couple of pushes and pops •  In the directed test enable the scoreboard and get the basic case fully working Synchronous FIFO – Bring up
  41. 41. | 41 Full feature enabling ▪ Start off the normal operation random test with smaller counts of operations ▪ Gradually increase the count of operations to reasonable number ▪ Do multiple seeded regressions of the random operation command line ▪ Write up the various negative condition and any corner case directed test cases Closure ▪ Code and functional coverage met ▪ 50 seeded random regressions are clean over couple of weeks Synchronous FIFO – Closure
  42. 42. | 42 DUT HOST BUS BFM UART BFMSCOREBOARD REG API INTR API TX GEN RX GEN BFM API TESTS UART Test Bench (1/2)
  43. 43. | 43 UART BFM ▪ Behavioral model of UART only modeling the Bus operation and Tx and Rx pins ▪ Does not need to implement all the internal registers of the BFM but it does need to provide the necessary controls ▪ Does not need to implement the interrupts set_configuration()/tx_bfm_data()/rx_bfm_data() ▪ Provides support for the tx error injections ▪ Does framing checks on rx UART Test Bench (2/2)
  44. 44. | 44 REGAPI ▪ All the register access details can be implemented here ▪ DUT initialization will be done here ▪ Data transmitted/received from UART will have series of steps. That sequence will be implemented here. Interrupt API ▪ Will implement the interrupt servicing of the UART interrupts ▪ Can be for popping received data or can be for the error conditions UART Functional Layer (1/2)
  45. 45. | 45 BFM API ▪ Sets up the BFM configuration matching the DUT configuration ▪ Pops the data received from the BFM ▪ Provides hooks for TB to inject the errors TX GEN/RX GEN ▪ Generates the specified number of the data packets ▪ Injects the random delay between the packets UART Functional Layer (2/2)
  46. 46. | 46 •  Extracts DUT Tx data transmitted from HOST BFM and compares in order with the data received from UART BFM on it’s Rx •  Extracts UART BFM Tx data transmitted from UART BFM and compares in order with the data received from HOST BFM on DUT Rx UART Scoreboard
  47. 47. | 47 Contents ▪  Verification Philosophy ▪  Familiarity ▪  Scope of Verification ▪  Functional Verification as a Career ▪  Overview of Process ▪  Some Examples: The 3 Musketeers ▪  Do’s and Don’t’s ▪  State of the Art
  48. 48. | 48 •  Focus on optimum verification to achieve quality •  Design evolves - make sure verification plan evolves around that •  Get plugged into various sources of information to track changes •  Use emails or better bug reports •  Document and archive important discussions •  Avoid bypass and short cuts. If you really need to take one always clearly comment/document Do’s and Don’t’s
  49. 49. | 49 Contents ▪  Verification Philosophy ▪  Familiarity ▪  Scope of Verification ▪  Functional Verification as a Career ▪  Overview of Process ▪  Some Examples: The 3 Musketeers ▪  Do’s and Don’t’s ▪  State of the Art
  50. 50. | 50 •  Constrained random approach has been found to be very effective in functional verification •  System Verilog has OOPs support to enhance the re- usability, special constructs for randomization and coverage State of the Art – SystemVerilog
  51. 51. | 51 •  Common good practices of constrained random verification environment development have been standardized as methodologies •  There are multiple such industry level standard verification methodologies such as VMM, OVM and UVM •  These methodologies also provide set of base class libraries and •  utilities that are very useful across the entire spectrum of verification activities State of the Art – Verification Methodologies
  52. 52. | 52 End of Document Visit us on: http://www.arrowdevices.com

×