2010 bristol q1_heuristic-stimuli-generation
Upcoming SlideShare
Loading in...5
×
 

2010 bristol q1_heuristic-stimuli-generation

on

  • 320 views

 

Statistics

Views

Total Views
320
Views on SlideShare
320
Embed Views
0

Actions

Likes
0
Downloads
6
Comments
0

0 Embeds 0

No embeds

Accessibility

Categories

Upload Details

Uploaded via as Adobe PDF

Usage Rights

© All Rights Reserved

Report content

Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

Cancel
  • Full Name Full Name Comment goes here.
    Are you sure you want to
    Your message goes here
    Processing…
Post Comment
Edit your comment

2010 bristol q1_heuristic-stimuli-generation 2010 bristol q1_heuristic-stimuli-generation Presentation Transcript

  • Heuristic Stimuli Generation For Coverage Closure Exploiting Simulation Feedback Giovanni Squillero Politecnico di Torino - Italy CAD Group ( ) giovanni.squillero@polito.it
  • GOAL • To propose a methodology for coverage- directed stimuli generation based on simulation feedback • Such stimuli could be added as new content to improve existing validation suites DVClub 2010-01-18 giovanni.squillero@polito.it 2
  • Acknowledgements • Danilo Ravotto • Ernesto Sanchez • Matteo Sonza Reorda • Alberto Tonda • + many others DVClub 2010-01-18 giovanni.squillero@polito.it 3
  • Outline • Proposed methodology • Case studies • Conclusions DVClub 2010-01-18 giovanni.squillero@polito.it 4
  • Design Choices • Being able to tackle real problems • Develop a versatile and broadly applicable methodology – Compatible with different environment – Compatible with any coverage metric • Minimize effort to change goal/target – Exploit common aspects DVClub 2010-01-18 giovanni.squillero@polito.it 5
  • Feedback-Based Approach • Simulation-based approach • Exploits feedback from simulation • Incremental improvement/refinement of the solution (trial-and-error) • Trade-off between computational resources and confidence • May exploit heuristics or problem-specific knowledge DVClub 2010-01-18 giovanni.squillero@polito.it 6
  • Proposed Methodology Stimuli Stimuli System Generator Feedback DVClub 2010-01-18 giovanni.squillero@polito.it 7
  • Stimuli Stimuli Stimuli System Generator Feedback DVClub 2010-01-18 giovanni.squillero@polito.it 8
  • Stimuli • Sequences of bits • Sequences of keys • Full fledged assembly language programs • VHDL test case • External world • … DVClub 2010-01-18 giovanni.squillero@polito.it 9
  • Stimuli Generator Stimuli Stimuli System Generator Feedback DVClub 2010-01-18 giovanni.squillero@polito.it 10
  • Stimuli Generator • Exploit an Evolutionary Algorithm to generate stimuli to maximize a given function DVClub 2010-01-18 giovanni.squillero@polito.it 11
  • Evolutionary Algorithms • Meta-heuristic optimization algorithm based on the concept of population and exploiting some principles of natural evolution DVClub 2010-01-18 giovanni.squillero@polito.it 12
  • Evolutionary Algorithms • Succession of random and deterministic steps – A systematic way of throwing dices – Better than pure random “The great effect produced by the accumulation in one direction, during successive generations, of differences absolutely inappreciable by an uneducated eye” DVClub 2010-01-18 giovanni.squillero@polito.it 13
  • Evolutionary Algorithms • Population – Multiple solutions considered in each step – More resistant than pure hill-climbing – Different solutions may interbreed DVClub 2010-01-18 giovanni.squillero@polito.it 14
  • Evolutionary Algorithms • Why using an evolutionary algorithm? – Adaptative – Able to find unexpected solutions – Better than random DVClub 2010-01-18 giovanni.squillero@polito.it 15
  • Evolutionary Algorithms • Problem: Fitness function – GOAL: Optimize the wheel – FITNESS: Minimize the number of “bumps” DVClub 2010-01-18 giovanni.squillero@polito.it 16
  • Evolutionary Algorithms • Problem: Black magic DVClub 2010-01-18 giovanni.squillero@polito.it 17
  • µGP (MicroGP) • CAD Group general-purpose evolver – 3 versions (only 2 released under GPL) – Project started in 2002 – 11 developers + contractors, students, … • Current version – ≈ 300 file, > 40,000 lines in C++ DVClub 2010-01-18 giovanni.squillero@polito.it 18
  • µGP (MicroGP) Evolutionary Optimization: the µGP toolkit E. Sanchez, M. Schillaci, G. Squillero Springer, 2010 ISBN: 978-0-387-09425-0 DVClub 2010-01-18 giovanni.squillero@polito.it 19
  • µGP (MicroGP) http://ugp3.sourceforge.net/ MicroGP++ (aka. ugp3, µGP3) • Information • Download • Credits DVClub 2010-01-18 giovanni.squillero@polito.it 20
  • System & Feedback Stimuli Stimuli System Generator Feedback DVClub 2010-01-18 giovanni.squillero@polito.it 21
  • System • Strongly problem dependant • Model via simulation/emulation – HDL (netlist to high-level) – HW accelerated (e.g., exploiting FPGA) – Architectural simulator – ISA simulator • Real device DVClub 2010-01-18 giovanni.squillero@polito.it 22
  • Feedback (examples) • From simulation – Code coverage metrics (e.g., instruction coverage) – HW specific metrics (e.g., toggle coverage) – High-level information (e.g., FSM coverage) • From the real system – Performance counters – Physical measures (e.g., temperature, time, power consumption) DVClub 2010-01-18 giovanni.squillero@polito.it 23
  • Outline • Proposed methodology • Case studies • Conclusions DVClub 2010-01-18 giovanni.squillero@polito.it 24
  • Design Verification • Devise a set of programs maximizing different coverage metrics DVClub 2010-01-18 giovanni.squillero@polito.it 25
  • Feedback • Code coverage metrics – Statement coverage (SC) – Branch coverage (BC) – Condition coverage CC) – Expression coverage (EC) • HW specific metric – Toggle coverage (TC) DVClub 2010-01-18 giovanni.squillero@polito.it 26
  • System • DLX/pII – Cleaned and simplified MIPS intended primarily for teaching purposes – 32-bit load/store architecture, 5-stage pipeline – VHDL RTL description • 4,558 statements • 3,695 branches • 193 conditional statements (1,764 expressions) • 8,283 logic bits DVClub 2010-01-18 giovanni.squillero@polito.it 27
  • Experimental Results DVClub 2010-01-18 giovanni.squillero@polito.it 28
  • Experimental Results 2,978 instructions 230K instructions DVClub 2010-01-18 giovanni.squillero@polito.it 29
  • Experimental Results 20h 33h 27h 95h 37h ≈1 week DVClub 2010-01-18 giovanni.squillero@polito.it 30
  • Post-silicon Verification • Generate functional test programs for post- silicon verification • Activity performed in collaboration with the ETM Group, Intel (Phoenix) • Intel Pentium 4 – 42-55 millions of transistors – 2GHz clock – NetBurst architecture DVClub 2010-01-18 giovanni.squillero@polito.it 31
  • System & Feedback • Real Pentium 4 (no simulation) • Performance counters as metric – Introduced in 1993 in IA32 architecture – 48 event detectors + 18 programmable counters – Instruction-tagging (for discriminating non- speculative performance events) DVClub 2010-01-18 giovanni.squillero@polito.it 32
  • Experimental Results • Two sets of experiments: – Mispredicted branches over the total branches – Clock cycles in which the trace cache is delivering µops to the execution unit instead of decoding or building traces (hard metric!) • Intrinsic features of the µarchitectural design • Time (both): 12h/program Haifa 2008 G. Squillero 33
  • Mobile Phone • 12m-activity performed in collaboration with Motorola Research Center in Turin (“Automation of Current Drain Measures”) DVClub 2010-01-18 giovanni.squillero@polito.it 34
  • System • Real prototypical mobile phone – P2K platform – GSM/3G networks – Complex applications • Radio Communication Analyzer, anechoic chamber • Controllable power supply • Custom hardware DVClub 2010-01-18 giovanni.squillero@polito.it 35
  • Stimuli • Sequence of keys • Power supply • Network status DVClub 2010-01-18 giovanni.squillero@polito.it 36
  • Feedback • Physical measures – Power consumption (+ autocorrelation) – Network status • FSM transition coverage DVClub 2010-01-18 giovanni.squillero@polito.it 37
  • Experimental Results • Major bugs discovered in the prototype – Test infrastructure – LCD display – Fake deep-sleep state DVClub 2010-01-18 giovanni.squillero@polito.it 38
  • Outline • Proposed methodology • Case studies • Conclusions DVClub 2010-01-18 giovanni.squillero@polito.it 39
  • Conclusions • Simulation-based & feedback-based • Evolutionary algorithm • Broadly applicable • Human resources – Limited (set-up ) • Computational resources – Easily parallelizable (generation level) – Trade-off quality vs. effort DVClub 2010-01-18 giovanni.squillero@polito.it 40
  • Conclusions • May exploit other methodologies – Useful starting point – Effective completion • May be coupled with other methodologies – Rule-based instruction randomizers – Simulation-based approaches DVClub 2010-01-18 giovanni.squillero@polito.it 41