CPU Verification Metrics

                     Shahram Salamian
                CPU Verification Manager
    Mobility Grou...
CPU Verification
    - Architectural Verification (AV)
        - Implementation meets Intel architecture definition
      ...
Architecture Verification
    - Directed set of directed & semi-random templates generating instruction
    set level (ass...
UArchitecture Verification



                             Coverage
                              Reset




    -   Functi...
Bug Rate
    160


    140


    120


    100


        80


        60


        40


        20


        0
           ...
RTL Lines Of Change

      80000
                                                          # of changed lines
            ...
Health Of the Model (HOM)
                         He a lth of the Mode l (HOM)

          100
          90
          80
 ...
2nd Tier metrics

    - Cycles run each week, licenses, etc

    - Bugs caught at full chip vs cluster.
        -Used to i...
Upcoming SlideShare
Loading in …5
×

Salamian dv club_foils_intel_austin

252
-1

Published on

0 Comments
0 Likes
Statistics
Notes
  • Be the first to comment

  • Be the first to like this

No Downloads
Views
Total Views
252
On Slideshare
0
From Embeds
0
Number of Embeds
1
Actions
Shares
0
Downloads
3
Comments
0
Likes
0
Embeds 0
No embeds

No notes for slide

Salamian dv club_foils_intel_austin

  1. 1. CPU Verification Metrics Shahram Salamian CPU Verification Manager Mobility Group- Texas Design Center 6/27/2006 1
  2. 2. CPU Verification - Architectural Verification (AV) - Implementation meets Intel architecture definition - Arch state compared against Architectural simulator - Split across a few categories - Done at full chip (FC). - uArchitecture Verification (uAV) - Cluster level (Fetch, LD/ST, etc) - Full chip - Cluster level checkers, templates, etc - Power management features Verification - Formal Verification - System level Verification - DFD/DFT feature Verification 2
  3. 3. Architecture Verification - Directed set of directed & semi-random templates generating instruction set level (assembly) tests - Accumulated over years. Assumed to have high coverage - Large test base. Smaller subset with good sampling used first - Highly scrutinized by mgmt. Needs to be almost perfect by tape out Legacy Tests Passrate 100 90 80 70 as g 60 %p s in pass_rate 50 goal_line 40 30 20 10 0 w ork w eek 3
  4. 4. UArchitecture Verification Coverage Reset - Functional coverage conditions jointly specified by verification & design - Internal tools to specify and measure - Use random and/or directed-random templates to cover - Conditions are typically prioritized based on complexity, bugs, etc - Tape out targets varies by cluster. - Focusing on raw coverage be misleading - A few “Easy” to cover set of monitors can skew covered % - Can be misinterpreted by mgmt as having great or bad coverage - Have to be looked at in conjunction with bug count, pass rate, etc ` 4
  5. 5. Bug Rate 160 140 120 100 80 60 40 20 0 3_1 4_1 4_1 4_1 _2 _2 _2 1_2 1_2 2_2 2_2 2_2 3_2 3_2 4_2 4_2 4_2 _3 70 10 50 90 10 50 90 30 70 10 50 90 30 70 10 50 90 10 New Bugs (14) Open Bugs (7) Open LT Bugs (0) "Smoke Alarm" - Many different views of bug data base (Total bugs, open bugs, etc) - Smoke alarm set based on previous projects bug history - Exceeding smoke alarm causes scrutiny by design & validation - Design reviews of areas where bug count jumps up - At times, it is a sign of better checkers, new tests going on line 5
  6. 6. RTL Lines Of Change 80000 # of changed lines # of RTL checkins 60000 40000 20000 0 ay-01 ay-02 S -00 N v-00 -01 ar-01 l-01 S -01 N v-01 -02 ar-02 l-02 S -02 N v-02 ep Jan ep Jan ep Ju Ju o o o M M M M - RTL Change rate to measure stability, allowing verification team to make progress in exercising RTL - Also measure RTL change request rate & type of request 6
  7. 7. Health Of the Model (HOM) He a lth of the Mode l (HOM) 100 90 80 70 60 Soe cr 50 40 30 20 10 0 0 Qt r 5 Qt r 4 Qt r 3 Qt r 2 Qt r 1 Qt r 0 Tim e to Tape out - Measures functional convergence trend. Informs project on RTL health is affecting verification team’s progress - Uses empirical formula using past projects data - Incorporates new bugs, bugs unresolved, and verification team’s ability to make progress. Subjective & quantative components - Low HOM relative to goal drives actions on fixing bugs and issues affecting verification 7
  8. 8. 2nd Tier metrics - Cycles run each week, licenses, etc - Bugs caught at full chip vs cluster. -Used to improve test bench quality - Bug cause - New condition hit or as result of timing, other bug fix, etc - Test bench & other validation collateral bugs 8

×