Heuristic Stimuli Generation for Coverage Closure Exploiting Simulation Feedback

Uploaded on


More in: Technology , Design
  • Full Name Full Name Comment goes here.
    Are you sure you want to
    Your message goes here
    Be the first to comment
    Be the first to like this
No Downloads


Total Views
On Slideshare
From Embeds
Number of Embeds



Embeds 0

No embeds

Report content

Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

    No notes for slide


  • 1. Heuristic Stimuli GenerationHeuristic Stimuli GenerationFor Coverage ClosureFor Coverage ClosureExploiting Simulation FeedbackExploiting Simulation FeedbackGiovanni SquilleroPolitecnico di Torino - ItalyCAD Group ( )giovanni.squillero@polito.it
  • 2. GOALGOAL• To propose a methodology for coverage-directed stimuli generation based onsimulation feedback• Such stimuli could be added as new content toimprove existing validation suitesDVClub 2010-01-18 giovanni.squillero@polito.it 2
  • 3. AcknowledgementsAcknowledgements• Danilo Ravotto• Ernesto Sanchez• Matteo Sonza Reorda• Alberto Tonda• + many othersDVClub 2010-01-18 giovanni.squillero@polito.it 3
  • 4. OutlineOutline• Proposed methodology• Case studies• ConclusionsDVClub 2010-01-18 giovanni.squillero@polito.it 4
  • 5. Design ChoicesDesign Choices• Being able to tackle real problems• Develop a versatile and broadly applicablemethodology– Compatible with different environment– Compatible with any coverage metric• Minimize effort to change goal/target– Exploit common aspectsDVClub 2010-01-18 giovanni.squillero@polito.it 5
  • 6. Feedback-Based ApproachFeedback-Based Approach• Simulation-based approach• Exploits feedback from simulation• Incremental improvement/refinement of thesolution (trial-and-error)• Trade-off between computational resourcesand confidence• May exploit heuristics or problem-specificknowledgeDVClub 2010-01-18 giovanni.squillero@polito.it 6
  • 7. Proposed MethodologyProposed MethodologyDVClub 2010-01-18 giovanni.squillero@polito.it 7StimuliGeneratorSystemStimuliFeedback
  • 8. StimuliStimuliDVClub 2010-01-18 giovanni.squillero@polito.it 8StimuliGeneratorSystemStimuliFeedback
  • 9. StimuliStimuli• Sequences of bits• Sequences of keys• Full fledged assembly language programs• VHDL test case• External world• …DVClub 2010-01-18 giovanni.squillero@polito.it 9
  • 10. Stimuli GeneratorStimuli GeneratorDVClub 2010-01-18 giovanni.squillero@polito.it 10StimuliGeneratorSystemStimuliFeedback
  • 11. Stimuli GeneratorStimuli Generator• Exploit an Evolutionary Algorithm to generatestimuli to maximize a given functionDVClub 2010-01-18 giovanni.squillero@polito.it 11
  • 12. Evolutionary AlgorithmsEvolutionary Algorithms• Meta-heuristic optimization algorithm basedon the concept of population and exploitingsome principles of natural evolutionDVClub 2010-01-18 giovanni.squillero@polito.it 12
  • 13. Evolutionary AlgorithmsEvolutionary Algorithms• Succession of random and deterministic steps– A systematic way of throwing dices– Better than pure random“The great effect produced by theaccumulation in one direction,during successive generations,of differences absolutely inappreciableby an uneducated eye”DVClub 2010-01-18 giovanni.squillero@polito.it 13
  • 14. Evolutionary AlgorithmsEvolutionary Algorithms• Population– Multiple solutions considered in each step– More resistant than pure hill-climbing– Different solutions may interbreedDVClub 2010-01-18 giovanni.squillero@polito.it 14
  • 15. Evolutionary AlgorithmsEvolutionary Algorithms• Why using an evolutionary algorithm?– Adaptative– Able to find unexpected solutions– Better than randomBetter than randomDVClub 2010-01-18 giovanni.squillero@polito.it 15
  • 16. Evolutionary AlgorithmsEvolutionary Algorithms• Problem: Fitness function– GOAL: Optimize the wheel– FITNESS: Minimize the number of “bumps”DVClub 2010-01-18 giovanni.squillero@polito.it 16
  • 17. Evolutionary AlgorithmsEvolutionary Algorithms• Problem: Black magicDVClub 2010-01-18 giovanni.squillero@polito.it 17
  • 18. µGP (MicroGP)µGP (MicroGP)• CAD Group general-purpose evolver– 3 versions (only 2 released under GPL)– Project started in 2002– 11 developers + contractors, students, …• Current version– ≈ 300 file, > 40,000 lines in C++DVClub 2010-01-18 giovanni.squillero@polito.it 18
  • 19. µGP (MicroGP)µGP (MicroGP)Evolutionary Optimization: the µGPtoolkitE. Sanchez, M. Schillaci, G. SquilleroSpringer, 2010ISBN: 978-0-387-09425-0DVClub 2010-01-18 giovanni.squillero@polito.it 19
  • 20. µGP (MicroGP)µGP (MicroGP)DVClub 2010-01-18 giovanni.squillero@polito.it 20http://ugp3.sourceforge.net/MicroGP++ (aka. ugp3, µGP3)• Information• Download• Credits
  • 21. System & FeedbackSystem & FeedbackDVClub 2010-01-18 giovanni.squillero@polito.it 21StimuliGeneratorSystemStimuliFeedback
  • 22. SystemSystem• Strongly problem dependant• Model via simulation/emulation– HDL (netlist to high-level)– HW accelerated (e.g., exploiting FPGA)– Architectural simulator– ISA simulator• Real deviceDVClub 2010-01-18 giovanni.squillero@polito.it 22
  • 23. Feedback (examples)Feedback (examples)• From simulation– Code coverage metrics (e.g., instruction coverage)– HW specific metrics (e.g., toggle coverage)– High-level information (e.g., FSM coverage)• From the real system– Performance counters– Physical measures (e.g., temperature, time, powerconsumption)DVClub 2010-01-18 giovanni.squillero@polito.it 23
  • 24. OutlineOutline• Proposed methodology• Case studies• ConclusionsDVClub 2010-01-18 giovanni.squillero@polito.it 24
  • 25. Design VerificationDesign Verification• Devise a set of programs maximizing differentcoverage metricsDVClub 2010-01-18 giovanni.squillero@polito.it 25
  • 26. FeedbackFeedback• Code coverage metrics– Statement coverage (SC)– Branch coverage (BC)– Condition coverage CC)– Expression coverage (EC)• HW specific metric– Toggle coverage (TC)DVClub 2010-01-18 giovanni.squillero@polito.it 26
  • 27. SystemSystem• DLX/pII– Cleaned and simplified MIPS intended primarilyfor teaching purposes– 32-bit load/store architecture, 5-stage pipeline– VHDL RTL description• 4,558 statements• 3,695 branches• 193 conditional statements (1,764 expressions)• 8,283 logic bitsDVClub 2010-01-18 giovanni.squillero@polito.it 27
  • 28. Experimental ResultsExperimental ResultsDVClub 2010-01-18 giovanni.squillero@polito.it 28
  • 29. Experimental ResultsExperimental ResultsDVClub 2010-01-18 giovanni.squillero@polito.it 29230Kinstructions2,978instructions
  • 30. Experimental ResultsExperimental ResultsDVClub 2010-01-18 giovanni.squillero@polito.it 3020h 33h27h37h95h≈1 week
  • 31. Post-silicon VerificationPost-silicon Verification• Generate functional test programs for post-silicon verification• Activity performed in collaborationwith the ETM Group, Intel (Phoenix)• Intel Pentium 4– 42-55 millions of transistors– 2GHz clock– NetBurst architectureDVClub 2010-01-18 giovanni.squillero@polito.it 31
  • 32. System & FeedbackSystem & Feedback• Real Pentium 4 (no simulation)• Performance counters as metric– Introduced in 1993 in IA32 architecture– 48 event detectors + 18 programmable counters– Instruction-tagging (for discriminating non-speculative performance events)DVClub 2010-01-18 giovanni.squillero@polito.it 32
  • 33. Experimental ResultsExperimental Results• Two sets of experiments:– Mispredicted branches over the total branches– Clock cycles in which the trace cache is deliveringµops to the execution unit instead of decoding orbuilding traces (hard metric!)• Intrinsic features of the µarchitectural design• Time (both): 12h/programHaifa 2008 G. Squillero 33
  • 34. Mobile PhoneMobile Phone• 12m-activity performed in collaboration withMotorola Research Center in Turin(“Automation of Current Drain Measures”)DVClub 2010-01-18 giovanni.squillero@polito.it 34
  • 35. SystemSystem• Real prototypical mobile phone– P2K platform– GSM/3G networks– Complex applications• Radio Communication Analyzer, anechoicchamber• Controllable power supply• Custom hardwareDVClub 2010-01-18 giovanni.squillero@polito.it 35
  • 36. StimuliStimuli• Sequence of keys• Power supply• Network statusDVClub 2010-01-18 giovanni.squillero@polito.it 36
  • 37. FeedbackFeedback• Physical measures– Power consumption (+ autocorrelation)– Network status• FSM transition coverageDVClub 2010-01-18 giovanni.squillero@polito.it 37
  • 38. Experimental ResultsExperimental Results• Major bugs discovered in the prototype– Test infrastructure– LCD display– Fake deep-sleep stateDVClub 2010-01-18 giovanni.squillero@polito.it 38
  • 39. OutlineOutline• Proposed methodology• Case studies• ConclusionsDVClub 2010-01-18 giovanni.squillero@polito.it 39
  • 40. ConclusionsConclusions• Simulation-based & feedback-based• Evolutionary algorithm• Broadly applicable• Human resources– Limited (set-up )• Computational resources– Easily parallelizable (generation level)– Trade-off quality vs. effortDVClub 2010-01-18 giovanni.squillero@polito.it 40
  • 41. ConclusionsConclusions• May exploit other methodologies– Useful starting point– Effective completion• May be coupled with other methodologies– Rule-based instruction randomizers– Simulation-based approachesDVClub 2010-01-18 giovanni.squillero@polito.it 41