Welcome to RTPWelcome to RTPDVClubDVClub!!Topics in VerificationTopics in VerificationPete LaFauciPete LaFauciOctober 18, ...
AgendaAgendaBrief Discussion/Overview aboutBrief Discussion/Overview about ““the Clubthe Club””Historical Perspective of L...
DVClubDVClub OverviewOverviewVoluntary, Community Based Verification GroupVoluntary, Community Based Verification GroupDis...
Historical Perspective: WhatHistorical Perspective: What’’s dones doneProduce some test vector stimulus, and simulateProdu...
Historical Prevalence:Historical Prevalence:Verification LanguagesVerification LanguagesVHDLRecords / DynamicVariable Indi...
ReuseReuse
Reuse: Going Beyond Traditional VIPReuse: Going Beyond Traditional VIP““ComponentsComponents””: traditional VIP: tradition...
Reuse: Inheritance and AbstractionReuse: Inheritance and AbstractionBase Class PackagesDUT templateTest HarnessGlobalsMemo...
CoverageCoverage
Coverage: TypesCoverage: TypesCode CoverageCode CoverageAssertion & Formal CoverageAssertion & Formal CoverageFunctional C...
Coverage: AchievingCoverage: AchievingThoroughnessThoroughnessIdentify which Design Functions have not been exercisedIdent...
Coverage: Collection, Merging,Coverage: Collection, Merging,and Reportingand ReportingProject Level CoverageProgress Repor...
Coverage: Challenges in CoverageCoverage: Challenges in CoverageDriven MethodologyDriven MethodologyGoals are manually cre...
Coverage: Execution PredictabilityCoverage: Execution PredictabilityTigr is 2.0Test Cover ageCommit15001700190021002300250...
RegressionRegression““EngineeringEngineering””
RegressionRegression ““EngineeringEngineering””::ChallengesChallengesManage thousands of simulation jobs & resultsManage t...
PlanningPlanning
Planning: Improvements in PlanPlanning: Improvements in PlanCreation and Change ManagementCreation and Change ManagementMo...
QualificationQualification
Qualification: WhatQualification: What’’s this?s this?What is Qualification?What is Qualification?WhoWho’’s checking the v...
SummarySummaryCombining innovative techniques, tools, processes canCombining innovative techniques, tools, processes canyi...
Interesting Materials,Good Food,Catch up with other Engineers in the IndustryInvite your Co-Workers!www.dvclub.org
Upcoming SlideShare
Loading in …5
×

Topics in Verification: Reuse, Coverage, Regression Engineering, Planning, Qualification

270 views

Published on

Published in: Technology, Design
0 Comments
0 Likes
Statistics
Notes
  • Be the first to comment

  • Be the first to like this

No Downloads
Views
Total views
270
On SlideShare
0
From Embeds
0
Number of Embeds
6
Actions
Shares
0
Downloads
16
Comments
0
Likes
0
Embeds 0
No embeds

No notes for slide

Topics in Verification: Reuse, Coverage, Regression Engineering, Planning, Qualification

  1. 1. Welcome to RTPWelcome to RTPDVClubDVClub!!Topics in VerificationTopics in VerificationPete LaFauciPete LaFauciOctober 18, 2006October 18, 2006
  2. 2. AgendaAgendaBrief Discussion/Overview aboutBrief Discussion/Overview about ““the Clubthe Club””Historical Perspective of Languages & SimulatorsHistorical Perspective of Languages & SimulatorsTopics in VerificationTopics in VerificationReuseReuseCoverageCoverageRegression EngineeringRegression EngineeringPlanningPlanningQualificationQualificationSummarySummary
  3. 3. DVClubDVClub OverviewOverviewVoluntary, Community Based Verification GroupVoluntary, Community Based Verification GroupDiscuss Present Day ChallengesDiscuss Present Day ChallengesEDA neutralEDA neutralPresentation ofPresentation of ““sharablesharable”” ideas, experiences, and resultsideas, experiences, and resultsAdvisory BoardAdvisory BoardCurrently about 10 members, spanning 6 companies (looking forCurrently about 10 members, spanning 6 companies (looking foradditional participants)additional participants)Planning, Logistics, Speakers, TopicsPlanning, Logistics, Speakers, TopicsTopics should be Chip Verification related, but can spanTopics should be Chip Verification related, but can spanadjacent areas:adjacent areas:Design for Verification, IT, Project ManagementDesign for Verification, IT, Project ManagementPlease Contact Pete LaFauci or Justin Sprague regarding interestPlease Contact Pete LaFauci or Justin Sprague regarding interestin the Advisory Board or Presenting atin the Advisory Board or Presenting at DVClubDVClub!!
  4. 4. Historical Perspective: WhatHistorical Perspective: What’’s dones doneProduce some test vector stimulus, and simulateProduce some test vector stimulus, and simulateLook at the output and waveforms on workstation, debug the restLook at the output and waveforms on workstation, debug the rest in the labin the labProduce a test plan and lots of directed, procedural test casesProduce a test plan and lots of directed, procedural test casesRun each test case in the test bucketRun each test case in the test bucketCompletion: each individual test case passesCompletion: each individual test case passesProduce a verification plan, which includes modelProduce a verification plan, which includes modelrequirements, functions & coverage goalsrequirements, functions & coverage goalsCombine generation capabilities, checkers, randomness, and coverCombine generation capabilities, checkers, randomness, and coverage monitors toage monitors togauge the simulationsgauge the simulationsRun tests with multipleRun tests with multiple ““seedsseeds””Completion: all tests pass, and coverage output isCompletion: all tests pass, and coverage output is ““analyzedanalyzed””Produce a verification methodology,Produce a verification methodology, ““architectarchitect”” aaverification environment & plan, and leverage powerful toolsverification environment & plan, and leverage powerful toolsAutomate both efficiency and thoroughness, through prediction, sAutomate both efficiency and thoroughness, through prediction, steering, and proofsteering, and proofsVIP, Environment, Library, and TestVIP, Environment, Library, and Test ““ReuseReuse””““RankRank”” the test suite efficiency, directed tests for coverage closurethe test suite efficiency, directed tests for coverage closureCompletion: all tests pass, coverage goals are met using measuraCompletion: all tests pass, coverage goals are met using measurable metricsble metricsThenNow
  5. 5. Historical Prevalence:Historical Prevalence:Verification LanguagesVerification LanguagesVHDLRecords / DynamicVariable Indices & Slicing2-d portsHVL – ‘e’ & VeraConstraint SolversEasy RTL AccessFunctional CoverageBuilt-in librariesC, C++. PerlComplex Data StructsSuperior String FunctionsVerilogConcurrencyTasks & FunctionsPLIEarly ’90sLate ’90s –Early 00’sMid ’90sToday / Tomorrow / Future?Modeling – SystemCHigher Abstraction, TLM, HW/SWAlgorithmic Modeling & PrototypeHVDL – SystemVerilogRTL/HVL Integration/UnificationSystemC & TLM InterfacesAssertion Based VerificationDPI
  6. 6. ReuseReuse
  7. 7. Reuse: Going Beyond Traditional VIPReuse: Going Beyond Traditional VIP““ComponentsComponents””: traditional VIP: traditional VIPCommon bus and data protocolsCommon bus and data protocolsMonitors, Predictors, DriversMonitors, Predictors, DriversEnvironments:Environments: new(ernew(er) with OOP/HVL) with OOP/HVLTestbenchesTestbenches && ““HarnessesHarnesses””Constraints & Test CasesConstraints & Test CasesConfiguration ComponentsConfiguration ComponentsAddresses ScalabilityAddresses Scalability“instance & program”“inherit, derive, build, configure”
  8. 8. Reuse: Inheritance and AbstractionReuse: Inheritance and AbstractionBase Class PackagesDUT templateTest HarnessGlobalsMemoriesRegistersData ObjectsInterface & ProtocoltemplatesHarnessDriversScoreboardUser Test Case & ConstraintsEnvironment Test Methods and User InterfacesTest Benches/HarnessesDUT VIPTool Packages: any_unit, any_env, any_sequenceApplicationSpecificVerificationEnvironmentCompanyLibraryInheritanceAndInstantiationsBaseLibraryProtocol VIPInterface VIPNote : “Horizontal (Interface/Protocol) and Vertical (Base Classes and Methods) Reuse Strategies areDeployable using OO Design Techniques”
  9. 9. CoverageCoverage
  10. 10. Coverage: TypesCoverage: TypesCode CoverageCode CoverageAssertion & Formal CoverageAssertion & Formal CoverageFunctional CoverageFunctional CoverageIntegration CoverageIntegration CoverageSystem & Validation CoverageSystem & Validation Coverage
  11. 11. Coverage: AchievingCoverage: AchievingThoroughnessThoroughnessIdentify which Design Functions have not been exercisedIdentify which Design Functions have not been exercisedIdentify What Test Code has not been simulatedIdentify What Test Code has not been simulatedComplements the Checking Code (traffic types, latency,Complements the Checking Code (traffic types, latency,performance, etc)performance, etc)StructuralCoverageAssertionsCodeCoverageFunctionalCoverage
  12. 12. Coverage: Collection, Merging,Coverage: Collection, Merging,and Reportingand ReportingProject Level CoverageProgress ReportsRegression 1Test Cases,ProceduralCodeTest Cases,ProceduralCodeTest Cases,ProceduralCodeTest Cases,ProceduralCodeSimulationCoverageOutput(Cover DB)Coverage DataMerging“Process”Test Cases,ProceduralCodeTest Cases,ProceduralCodeTest Cases,ProceduralCodeTest Cases,ProceduralCodeSimulationCoverageOutputRegression n “Total Coverage”Reports & HTML viewsTest Cases,ProceduralCodeTest Cases,ProceduralCodeTest Cases,ProceduralCodeTest Cases,ProceduralCodeSimulationCoverageOutput(Cover DB)
  13. 13. Coverage: Challenges in CoverageCoverage: Challenges in CoverageDriven MethodologyDriven MethodologyGoals are manually created from the engineeringGoals are manually created from the engineeringSpecsSpecs --> Labor Intensive> Labor IntensiveRandom generation can be wasteful if overallRandom generation can be wasteful if overallcoverage is not increasingcoverage is not increasingSimulation cycles can be slow and/or expensive,Simulation cycles can be slow and/or expensive,especially if they are being wastedespecially if they are being wastedExecution Predictability with Schedules &Execution Predictability with Schedules &ResourcesResources
  14. 14. Coverage: Execution PredictabilityCoverage: Execution PredictabilityTigr is 2.0Test Cover ageCommit15001700190021002300250027002900Total PlannedWr ittenActualWr ittenCommitCover edActualCover edCommitCoverage Progress Chart: “S” Curve Phenomena
  15. 15. RegressionRegression““EngineeringEngineering””
  16. 16. RegressionRegression ““EngineeringEngineering””::ChallengesChallengesManage thousands of simulation jobs & resultsManage thousands of simulation jobs & resultsMultiple environmentsMultiple environmentsMany Servers, Lots of output!Many Servers, Lots of output!Team environment, Individual OwnersTeam environment, Individual OwnersAnalysisAnalysisReproducing test fails easily & accuratelyReproducing test fails easily & accurately --> test/bench, machine, seed, tools,> test/bench, machine, seed, tools,versionsversionsMore debug output necessary?More debug output necessary?Correctly assigned or delegated to appropriateCorrectly assigned or delegated to appropriate owner(sowner(s))EfficiencyEfficiencyDynamically Allocating Right Machines for the Right Job TypesDynamically Allocating Right Machines for the Right Job TypesMaximize Utilization of both Hardware and SoftwareMaximize Utilization of both Hardware and SoftwareCoverage Closure: Hole Analysis, RankingCoverage Closure: Hole Analysis, Ranking24/7 rebalancing, including interactive session priority managem24/7 rebalancing, including interactive session priority managementent
  17. 17. PlanningPlanning
  18. 18. Planning: Improvements in PlanPlanning: Improvements in PlanCreation and Change ManagementCreation and Change ManagementMore Tightly Couple the Design Specification with theMore Tightly Couple the Design Specification with theVerification PlanVerification PlanChange Management SystemChange Management SystemMore Automatic Synchronization and NotificationMore Automatic Synchronization and NotificationImprove the Efficiency between Coverage Plan andImprove the Efficiency between Coverage Plan andImplementationImplementationReduce or Remove the Disconnect between the FunctionalReduce or Remove the Disconnect between the FunctionalCoverage Code and the Coverage PlanCoverage Code and the Coverage PlanAutomate Functional Coverage CodeAutomate Functional Coverage Code ““GenerationGeneration””
  19. 19. QualificationQualification
  20. 20. Qualification: WhatQualification: What’’s this?s this?What is Qualification?What is Qualification?WhoWho’’s checking the verification, and when?s checking the verification, and when?Error insertion to prove the accuracy and effectiveness of yourError insertion to prove the accuracy and effectiveness of yourverification systemsverification systemsError insertion is placed in the DUT (not verification code)Error insertion is placed in the DUT (not verification code)Challenges in adoption:Challenges in adoption: ““how, what, whenhow, what, when””, effort levels, effort levelsBenefits for SimulationBenefits for SimulationQualify the Test BenchQualify the Test BenchQualify the Models & VIPQualify the Models & VIPQualify the Test Cases & ConstraintsQualify the Test Cases & ConstraintsBenefits for Formal ProofsBenefits for Formal ProofsQualify the AssertionsQualify the AssertionsQualify the ConstraintsQualify the Constraints*Qualify the Answer the Tool is Giving you**Qualify the Answer the Tool is Giving you*
  21. 21. SummarySummaryCombining innovative techniques, tools, processes canCombining innovative techniques, tools, processes canyield significant results!yield significant results!NotNot ““uncommonuncommon”” to achieve 10x productivity andto achieve 10x productivity andthoroughness improvements through methodology andthoroughness improvements through methodology andtuningtuningVerification processes will continue to changeVerification processes will continue to changeconsiderably to keep up with managing the exponentialconsiderably to keep up with managing the exponentialgrowth in design complexitygrowth in design complexity
  22. 22. Interesting Materials,Good Food,Catch up with other Engineers in the IndustryInvite your Co-Workers!www.dvclub.org

×