Coverage Solutions on
Emulators

Ravi P Gupta
Agenda                                                2



• Functional Verification Overview         (~5 Mins)
   • Traditional Verification Challenges
   • Future requirements and Solutions

• Code coverage with PXP (~5 Mins)

• Performance analysis with coverage enabled (~5 Mins)
   • PXP Code coverage Result analysis

• Support of Functional coverage




                                                         Presentation Title   18/04/13
Functional Verification                                                           3



• Definition of a test plan.

• Implementation using random test generators that produce a large number of
  test cases.

• Comparing the result to the expected results in order to say if the test passed.

• Analyze how much functionality of the design has been verified.

• coverage tools – measures the percentage of design functionality covered.
  • Detect the occurrence of events in the test plan.
  • Provide information related to the progress of the test plan.

• Analysis of the coverage reports allows the verification team to modify the
  directives for the test generators

in Verification


                                                                    Presentation Title   04/18/13
Challenges in Functional Verification                                                          4




• Around 70% of product development cycle time is consumed in verification
  activity.

• Simulation based verification is very slow and no longer sufficient to meet the
  demands of a complex IPs and SOC.

• Dedicated hardware solutions are too expensive to develop.

• Considerable effort is invested in finding ways to close the loop of coverage
  analysis and test generation.




                                                                    Presentation Title   04/18/13
Requirement & Solution                                             5



Requirements
• Accelerating functional verification

• Closing verification process        Can verification closure be accelerated??

Solution
• Hardware accelerated simulation


                                                       Hardware
               Testbench                               Accelerator

              Module 0   Module 1                     Module 2 is
                                                     synthesized &
                                                     compiled into
                    Module 2
                                                   Custom processors



                                                                       Presentation Title   04/18/13
Hardware Accelerated Simulation                                                     6




• Pros
  • Much faster than simulation.
  • Provides simulation like verification flow.
  • Debugging is easier than customize hardware.


• Cons (Obstacles to overcome)
  • Mapping RTL design into the hardware can be substantial longer.
  • SW-HW communication speed can degrade the performance.
  • Tool cost is much more than simulators.




                                                             Presentation Title   04/18/13
Idea!!                         7




• The addition of coverage to emulators can accelerates the detection of

 inadequate functional verification and augments the efficiency of the

 verification engineer in writing test cases by focusing on uncovered areas

 in the design.




                                                             Presentation Title   04/18/13
Metric driven verification flow                         8




                        Presentation Title   04/18/13
Simulation vs. Emulation                                   9



• Design has two main sub-module
   • Downstream
   • Upstream




                                       Presentation Title   04/18/13
Steps to generate Toggle database                                                                10




• Instrumentation
     add +tcov and +tcovType+ to elaboration command


• Run the design
      irun –R +ixccWorkDir+<> +ixccTest+<>


• Report generation (Equivalent to .res file of internal solution)
     ixcc –inputDir <> -input <> -summary | tee <>.log



NOTE: tcovType can be only ports or ports, Verilog & VHDL variables,
 Verilog nets & VHDL signals



                                                                 Presentation Title   04/18/13
Result analysis (1/3)                                          11




Testcase             IES Coverage   PXP Coverage


     Sx_ipbypass     17.4%                 17.2%


  Sx_ipreseqbypass   19.1%                 18.9%


   Sx_macbypass      17.6%                 17.5%


   Sx_macipbypass    19.3%                 19.1%




                                            Presentation Title   04/18/13
Result analysis (2/3)                                             12

        • Coverage results come in txt file as below


: Instance Name: mem_tb.DUT

File Name: mem.v

Hit(Full)    Hit(Rise)   Hit(Fall)   Signal

1            1           1           clk

0            1           0           rst

<><><><><><><><><><><><><><><><><><><><><><><><><><><><>

: Instance Name: mem_tb.DUT.addr_inst

File Name: mem.v

Hit(Full)    Hit(Rise)   Hit(Fall)   Signal

    1        1           1           a[7]

    0        1           0           out[7]




                                                           Presentation Title   04/18/13
Result analysis (3/3)                                                                                  13




LOG = Local Overall Grade

LOC = Local Overall Covered

OG = Overall Grade

OC = Overall Covered



Instance                                                LOG           LOC                OG           OC

---------------------------------------------------------------------------------------------------------------

mem_tb.DUT                                                   81.5       44 / 54         87.5       77 / 88

mem_tb.DUT.addr_inst                                         97.1       33 / 34          97.1       33 / 34



       • Merging of different test coverage database is possible.


       • iccr support likely to come in next release.



                                                                                                                  Presentation Title   04/18/13
Performance impact                                                            14




Config. Type     Resource used   Resource used   Frequency       Frequency after
                 before toggle   after toggle    before toggle   toggle enabled
                 enabled         enabled         enabled

Only ports       62%             63%             1.33MHz         1.33MHz

Ports+internal   62%             88%             1.33MHz         0.97MHz




                                                                 Presentation Title   04/18/13
Steps to generate functional cov.
                                                                                                 15

             database
• Instrumentation
     add +sv and –functional coverage options to elaboration command


• Run the design
      irun –R <irun options>


• Report generation (Equivalent to .res file of internal solution)
     iccr –gui –cov.work/scope/test




                                                                 Presentation Title   04/18/13
Conclusion                                                           16



• Palladium-XP is able to support most of the SV functional coverage
  constructs and toggle coverage

• Coverage on Emulation without much penalty on performance & area
  utilization.

• Traditional flow for result analysis.




 PXP, not only just accelerate the
 verification but also verification closure.


                                                                 Presentation Title   04/18/13
17




Presentation Title   04/18/13
18




Presentation Title   04/18/13

Coverage Solutions on Emulators

  • 1.
  • 2.
    Agenda 2 • Functional Verification Overview (~5 Mins) • Traditional Verification Challenges • Future requirements and Solutions • Code coverage with PXP (~5 Mins) • Performance analysis with coverage enabled (~5 Mins) • PXP Code coverage Result analysis • Support of Functional coverage Presentation Title 18/04/13
  • 3.
    Functional Verification 3 • Definition of a test plan. • Implementation using random test generators that produce a large number of test cases. • Comparing the result to the expected results in order to say if the test passed. • Analyze how much functionality of the design has been verified. • coverage tools – measures the percentage of design functionality covered. • Detect the occurrence of events in the test plan. • Provide information related to the progress of the test plan. • Analysis of the coverage reports allows the verification team to modify the directives for the test generators in Verification Presentation Title 04/18/13
  • 4.
    Challenges in FunctionalVerification 4 • Around 70% of product development cycle time is consumed in verification activity. • Simulation based verification is very slow and no longer sufficient to meet the demands of a complex IPs and SOC. • Dedicated hardware solutions are too expensive to develop. • Considerable effort is invested in finding ways to close the loop of coverage analysis and test generation. Presentation Title 04/18/13
  • 5.
    Requirement & Solution 5 Requirements • Accelerating functional verification • Closing verification process Can verification closure be accelerated?? Solution • Hardware accelerated simulation Hardware Testbench Accelerator Module 0 Module 1 Module 2 is synthesized & compiled into Module 2 Custom processors Presentation Title 04/18/13
  • 6.
    Hardware Accelerated Simulation 6 • Pros • Much faster than simulation. • Provides simulation like verification flow. • Debugging is easier than customize hardware. • Cons (Obstacles to overcome) • Mapping RTL design into the hardware can be substantial longer. • SW-HW communication speed can degrade the performance. • Tool cost is much more than simulators. Presentation Title 04/18/13
  • 7.
    Idea!! 7 • The addition of coverage to emulators can accelerates the detection of inadequate functional verification and augments the efficiency of the verification engineer in writing test cases by focusing on uncovered areas in the design. Presentation Title 04/18/13
  • 8.
    Metric driven verificationflow 8 Presentation Title 04/18/13
  • 9.
    Simulation vs. Emulation 9 • Design has two main sub-module • Downstream • Upstream Presentation Title 04/18/13
  • 10.
    Steps to generateToggle database 10 • Instrumentation add +tcov and +tcovType+ to elaboration command • Run the design irun –R +ixccWorkDir+<> +ixccTest+<> • Report generation (Equivalent to .res file of internal solution) ixcc –inputDir <> -input <> -summary | tee <>.log NOTE: tcovType can be only ports or ports, Verilog & VHDL variables, Verilog nets & VHDL signals Presentation Title 04/18/13
  • 11.
    Result analysis (1/3) 11 Testcase IES Coverage PXP Coverage Sx_ipbypass 17.4% 17.2% Sx_ipreseqbypass 19.1% 18.9% Sx_macbypass 17.6% 17.5% Sx_macipbypass 19.3% 19.1% Presentation Title 04/18/13
  • 12.
    Result analysis (2/3) 12 • Coverage results come in txt file as below : Instance Name: mem_tb.DUT File Name: mem.v Hit(Full) Hit(Rise) Hit(Fall) Signal 1 1 1 clk 0 1 0 rst <><><><><><><><><><><><><><><><><><><><><><><><><><><><> : Instance Name: mem_tb.DUT.addr_inst File Name: mem.v Hit(Full) Hit(Rise) Hit(Fall) Signal 1 1 1 a[7] 0 1 0 out[7] Presentation Title 04/18/13
  • 13.
    Result analysis (3/3) 13 LOG = Local Overall Grade LOC = Local Overall Covered OG = Overall Grade OC = Overall Covered Instance LOG LOC OG OC --------------------------------------------------------------------------------------------------------------- mem_tb.DUT 81.5 44 / 54 87.5 77 / 88 mem_tb.DUT.addr_inst 97.1 33 / 34 97.1 33 / 34 • Merging of different test coverage database is possible. • iccr support likely to come in next release. Presentation Title 04/18/13
  • 14.
    Performance impact 14 Config. Type Resource used Resource used Frequency Frequency after before toggle after toggle before toggle toggle enabled enabled enabled enabled Only ports 62% 63% 1.33MHz 1.33MHz Ports+internal 62% 88% 1.33MHz 0.97MHz Presentation Title 04/18/13
  • 15.
    Steps to generatefunctional cov. 15 database • Instrumentation add +sv and –functional coverage options to elaboration command • Run the design irun –R <irun options> • Report generation (Equivalent to .res file of internal solution) iccr –gui –cov.work/scope/test Presentation Title 04/18/13
  • 16.
    Conclusion 16 • Palladium-XP is able to support most of the SV functional coverage constructs and toggle coverage • Coverage on Emulation without much penalty on performance & area utilization. • Traditional flow for result analysis. PXP, not only just accelerate the verification but also verification closure. Presentation Title 04/18/13
  • 17.
  • 18.