TRACK H: Using Formal Tools to Improve the Productivity of Verification at STMicroelectronics/ James Pascoe

Uploaded on


More in: Technology , Design
  • Full Name Full Name Comment goes here.
    Are you sure you want to
    Your message goes here
    Be the first to comment
    Be the first to like this
No Downloads


Total Views
On Slideshare
From Embeds
Number of Embeds



Embeds 0

No embeds

Report content

Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

    No notes for slide


  • 1. May 4, 2011Using Formal Tools to Improve theProductivity of Verification atSTMicroelectronicsJames Pascoe1000 Aztec West, Almondsbury, Bristol, UKChipEx 2013 – Tel Aviv, IsraelMay 1, 2013
  • 2. May 1, 2013Overview• Who are we:– The ‘CPU’ part of ‘CPU/GPU’ in TR&D (ST Bristol)– We develop ARM based CPU sub-systems for a range of SoCs• Organisation:– System-level functional verification (Noida)– Block-level activities (Bristol)– Low-power and DFT verification (Grenoble)• Formal evaluation:– Sensor Control Block (SCB) – block-level verification– Clock and Reset Manager (CRM) – block-level verification– Documentation driven point-to-point connectivity checking
  • 3. May 1, 2013Subsystem
  • 4. May 1, 2013Verification Strategy• Unit testing:– Performed by Designers– Is this block ready to enter verification?– Designers typically implement HDL based test-benches• Block-level:– Performed by Verification engineers– Does this block conform to its specification in isolation?– Typically verified using a constrained random approach• System-level testing:– Point-to-point checks– Functional testing– Performance verification
  • 5. May 1, 2013Evaluating Formal• We evaluated a formal tool (Jasper) to determine its potential toenhance our productivity. Our study had three aims:1. To close verification projects with appreciably less effort thanconstrained random;2. To promote greater use of assertions by encouraging designersto develop formal properties for their blocks;3. To augment or replace legacy in-house flows with matureindustry tools that reduce maintenance and other overheads.
  • 6. May 1, 2013Approach• Projects increase in complexity:– Sensor Control Block• Digital memory mapped sensor block• Developed in Bristol• Constrained Random test-bench developed in parallel– Clock and Reset Manager• Provides clock and reset sequencing in subsystem (complex)• Developed in Grenoble• Very good micro-architectural documentation but no functional specification– Point-to-point connectivity checking• Flow developed to extract assertions from project specifications• Use Jasper to verify connectivity at the subsystem level• Also very useful in the context of low-power (e.g. UPF aware proofs)
  • 7. May 1, 2013OverviewSensorControl BlockClock andResetManagerPoint-to-pointconnectivitycheckingConclusionsSensor Control Block
  • 8. May 1, 2013Sensor Control Block• Provides a digital front-end to thermal sensors:– Monitors chip temperature– Selected for its simplicity:• Connects to thermal sensors• Samples sensors periodically• Generates warning interrupts– Well specified and understood• Used Jasper to check:– Registers– APB interface and protocol– Temperature functionality– Sampling periods– Interrupt properties
  • 9. May 1, 2013Results• Found 3 bugs:– Found subtle problem with PREADY signal on APB interface• Not detected by Specman– 1 RTL problem:• Inverted ‘or’ concatenation for sensor overflow / underflow bits– 1 specification problem:• Registers listed as ‘Write Clear 1’ can be cleared with other values• Validity:– Didn’t expect to find much wrong • Designer had performed extensive unit testing prior to verification• Some previous Specman verification performed• Tried and tested components used in the development
  • 10. May 1, 2013Project Management• We need to answer the following questions:– How complete is the verification?• Q: are there enough assertions to verify all features?• A: measure ‘out-of-coi’ coverage to find coverage holes– Is the environment over constrained?• Q: are we masking bugs by constraining the environment too much?• A: measure stimuli coverage and check that all legal scenarios are covered– How good are the bounded proofs for my block?• Q: do we require more depth in the search space?• A: use bounded coverage analysis– How does the combination of formal and dynamic cover the design?• Q: how do I interpret Jasper coverage with constrained random coverage?• A: Jasper is working on merging results into other UCDB files
  • 11. May 1, 2013Coverage• Dead code analysis:– Dead code due to RTL – 64 cover points• Branches that will not be taken during reset• 1 feature that was later deprecated (but masked with a tie-off in the RTL)• Register implementation uses generic code to minimise flop generation at synthesis• Justified exceptions with designer review• Out of Cone of Influence:– 219 of 1000 coverage points were uncovered– > 150 uncovered points were due to the register model– Remainder due to P1500 signals:• Feature added when Jasper work was close to sign-off• Later covered in Specman TB– Exceptions justified by designer• Bounded proof coverage – 100% covered
  • 12. May 1, 2013OverviewSensorControl BlockClock andResetManagerPoint-to-pointconnectivitycheckingConclusionsClock and Reset Manager
  • 13. May 1, 2013Clock and Reset Manager• Generates clocks and resets to blocks within subsystem:– Sequences actions to control transitions between operating points– Enables subsystem to be fully asynchronous– Provides an abstract interface to the SoC– Good micro-architectural documentation but difficult to verify against– Critical block and complex• Verification approach:– Use Jasper to perform feature extraction• Not possible with constrained random test-bench– Prove critical properties using Jasper• Event sequences, specific timings etc.– Develop Specman test bench in parallel• Compare approaches
  • 14. May 1, 2013Feature Extraction• Key problem: CRM functionality not well specified:– Once ‘cmd_in’ rises ‘cmd_ack’ should follow after ‘N’ cycles– Once ‘cmd_in’ falls, ‘cmd_ack’ will follow after ‘M’ cycles– where ‘N’ and ‘M’ are not known …• Solution– Use Visualise to evaluate the range of cycles the ‘ack’ follows– Write number of properties with different values for ‘N’– Find the lowest ‘N’ for which the property proves– Keep the assertion of the lowest proven ‘N’assert: $rose(cmd_in) |-> ##[1:80] cmd_ack– Define cover item for the Min ‘N-1’ for regressioncover: $rose(cmd_in) ##1 !cmd_ack [*80-1]
  • 15. May 1, 2013Visualize
  • 16. May 1, 2013Results• No RTL bugs were found. But:– CRM was verified previously. Jasper was used to boost confidence – Specification is now well understood – solved a big problem– Used to bound overlap conditions on functional stimuli• Exhaustive proofs on all functional features– 73 assertions – all fully proven– 88 covers – all covered– Not including test-mode• Used coverage for:– Checking for over-constraints – none found– Ensuring that formal touched all functionality– More details about coverage to follow
  • 17. May 1, 2013Coverage• Dead code analysis:– Dead code due to RTL – 4 cover points• Branches that will not be taken during reset or test mode• Due to synchronisers that have reset signals tied low• Designer could have replaced them with synchronisers without reset– Dead code due to no test mode constraint: 54 cover points (58-4)• Test code that is not active in functional mode – designer reviewed and approved• Wanted to make sure that no functional branches were made dead by mistake– Dead code due to reset: 63 cover points (121-58)• These are the (!resetn) branches• Reviewed and designer approved– Dead code due to functional assumptions: 4 cases• Do we constrain away any valid branches?• 4 cases were discovered and approved as part of test mode
  • 18. May 1, 2013Coverage• Bounded Proof Coverage:– Initially, not all of the assertions converged• Out of 55 assertions, 28 did not converge• Lowest bound for non-converging assertions is: 774 cycles– Used bounded proof coverage• Measured coverage within the bounds reached by non-converging properties– Bounded proof coverage tells us that bound is acceptable because all coveritems are covered in less than 268 cycles– Subsequently, more advanced engines were used and all properties converged• Out of Cone of Influence:– 46 branches were detected as out of cone of influence– All branches relate to DFT
  • 19. May 1, 2013OverviewSensorControl BlockClock andResetManagerPoint-to-pointconnectivitycheckingConclusionsPoint-to-Point Connectivity Checking
  • 20. May 1, 2013Point-to-Point Connectivity• Point-to-point connectivity checking:– Once everything is verified at the unit and block-level …– Point-to-point connectivity checking provides a first check that blockshave been assembled into the subsystem correctly– Eliminates wiring errors - useful before functional system testing• Developed a documentation driven point-to-point flow:– 2564 reference connections generated from key project document– Point-to-point checking flow setup in 1 day (with help of Jasper) – All 2564 properties have been proven• Useful for low-power:– Can check connectivity rules for changing power states– Rule validity can be tied to power states
  • 21. May 1, 2013OverviewSensorControl BlockClock andResetManagerPoint-to-pointconnectivitycheckingConclusionsConclusions
  • 22. May 1, 2013Conclusions• Formal delivers!• Found bugs quickly:– SCB required 6 m/w of effort to build a Constrained Random test-bench– The formal approach found almost all the bugs within the first week• Potential for designer involvement is high:– Designers found Jasper easier to learn than other formal tools– Jasper was used to develop assertions and to perform unit-testing• Unit-testing was then reused in the block-level verification• Enabled us to verify blocks with incomplete specifications:– Used formal to test implicit assumptions on the CRM– Provided results that could be quickly verified by designers
  • 23. May 1, 2013Quality Improvements• Insight can be captured as properties when:– Interpreting specifications– Making assumptions• Formal provides a good way of stimulating early designs:– No need for HDL test benches that are discarded once verification starts– Formal is a great way of performing unit-testing– Assertions are reused throughout the life-cycle of the IP• Certain aspects can only be verified formally:– Absence of deadlock– Liveness properties• Properties automatically validate system level behaviour:– Detects system-level inconsistencies in specifications and assumptions
  • 24. May 1, 2013The Value of Formal• Unit / block-level:– Significant time savings– No need to develop complicated Constrained Random test-benches– Potential for property reuse is high– Provides feedback early in the design process– Allows designers to stimulate designs without having to write HDL TBs– Properties add value to subsequent design phases– Feature extraction can be performed• System-level:– Point-to-point checking is easy to setup and provides good insights– Low level properties assist in verifying system-level assumptions– Absence of dead-lock / liveness
  • 25. May 1, 2013OverviewSensorControl BlockClock andResetManagerPoint-to-pointconnectivitycheckingConclusionsQuestions