SlideShare a Scribd company logo
1 of 10
Download to read offline
Validation and Design
 in a Small Team Environment:
The Development of the High
Performance Embedded Computing
Process at Northrop Grumman

                  Matthew Clark, Ph.D
                  February 19th, 2009
Motivation


Validation is:                   What if your project is:

• Complex                        • Complex

• Long                           • Lasts 3-6 months,

• Requires expensive tools       • Has a limited budget,
                                     – And new grads.
• Requires experienced
  employees.                     • Is done in a group with 30
                                   years of SW experience,
                                   and you represent the sum
                                   total of HW development
It is both an Art and Science.
                                   experience.
First Question

What makes verification so expensive and time consuming?
                                   Coverage Analysis
Functional Verification Issues
• Code with Branches
                                   Assertions
• Multiple Clock Rates
• Packet and Clock Rate
  adaptation.                      1 TC per config + Random.

• Configurations
• SAR                              Corner Case Analysis

• Encapsulation
                                   Special SW modeling (e/Vera)

      Identifying and reducing the green arrows is the Art.
Problem Statement


How can you reduce the number of green arrows so that
designs can be completed quickly, reliably, without having
the “normal” 2.5:1 Verification : Designer ratio?
A: Do what Capt. Kirk did, change the problem.
We can’t change the verification restrictions, perhaps we
can change the design issues so that we don’t have some
verification problems… by design.
Problem Space


• Offload CPU-intensive portions of numerical algorithms.
• Rapid deployment. Standard project is 3-6 months.
• Must be modular:
    – Single module stand-alone.
    – Combined into algorithm.
    – Multiple algorithms in single container.

• Exact duplicates of existing SW applications.
    – Goal is to reduce execution time, or improve performance (ie replace 64K pt
      FFT with 512K pt FFT).
    – To SW, there must be no difference between CPU, FPGA, FPGA+DSP, DSP
      implementation.
Design and Verification constraints.


• Single Clock Rate for every block.
• No Packet and Clock Rate Adaptation
    – Force it to occur at a single ingress/egress point.

• Heavily branched code
    – Algorithms (fortunately) tend to be datapath centric.

• Configurations
    – Separate algorithm implementation verification from IO
      verification.
    – Build bit/cycle accurate model of algorithm, use IO controls to vary data
      tx/rx.
Design constraints reduce verification space

 Design Constraints
 •   All blocks use small input fifo’s, and programmable full/empty
     indicators.
 •   All blocks execute at 210 MHz. (Max Register Access speed).
 •   Top Level register interface can be pipelined for timing closure.
 •   Coregen modules only (Ref. Designs Need Not Apply)
 •   Each block uses (1+) async resets, that are externally synch’d.
 •   Each block will have a testbench that:
      – Uses golden Matlab code to create all stimulus/checker files.
            • Scenarios are generated from configure files.
      –   Test bench is nothing but interface BFM comparing values against file.
      –   Can slew input/output rates independently.
      –   Can randomly assert the full/empty flags on interface pins.
      –   Interrupt conditions may be checked by hand.
      –   Pass fail conditions checked in VHDL at run time.
      –   Test bench can be run as regression.
Design constraints reduce verification space

Functional Verification Issues
• Code with Branches
• Multiple Clock Rates           IO_clock and funct_clock only.

• Packet and Clock Rate          All IO is FIFO based. IO ring handles.
  adaptation.
• Configurations
• SAR                            Not an issue.
• Encapsulation                  Not an issue
Results


• MegaCore Function (FFT, IFFT, Matched Filter, Correlation,
  Polyphase/WOLA ), plus Shim for PCI bridge.
• Designer Tasks: Architect, Document (MAS), Block Test, Synthesize.
    – Forward FFT: BSEE (1 mo Verilog)
    – Inverse FFT : MSEE (6 mo Verilog)
    – Packet Aggregation: Intern (2 mo VHDL).

• Verification:
    – MSEE (no HDL).
        • Create Matlab based Testbench, stimulus checker files.
        • Created Stimulus files for Block Level Test Benches
        • Reused FIFO BFM’s from previous project.

• Code + Block Test: 4 Months
• Verification + Synthesis: 2 Months.
Summary


Q: How can you reduce the number of green arrows so that designs can be completed
quickly, reliably, without having the “normal” Verification : Designer ratio?


• Use standard, non-optimal, fifo interfaces for every block.
• Separate algorithm verification from implementation verification.
• Repeatable test cases at the cost of text file storage.
• Design each module like it is going into a reuse container:
    –   Standard IF
    –   Standard Clock and reset.
    –   Standard IO and register interfaces.
    –   All data widths are 64 bit + Vld @ 210 MHz.

 Summary: Trade improved verification schedule and TTM with reduced performance
and complexity.

More Related Content

What's hot

SANER 2015 ERA track: Differential Flame Graphs
SANER 2015 ERA track: Differential Flame GraphsSANER 2015 ERA track: Differential Flame Graphs
SANER 2015 ERA track: Differential Flame Graphscorpaulbezemer
 
Trends in Mixed Signal Validation
Trends in Mixed Signal ValidationTrends in Mixed Signal Validation
Trends in Mixed Signal ValidationDVClub
 
Stinson post si and verification
Stinson post si and verificationStinson post si and verification
Stinson post si and verificationObsidian Software
 
WebLogic Stability; Detect and Analyse Stuck Threads
WebLogic Stability; Detect and Analyse Stuck ThreadsWebLogic Stability; Detect and Analyse Stuck Threads
WebLogic Stability; Detect and Analyse Stuck ThreadsMaarten Smeets
 
Continuous Performance Testing
Continuous Performance TestingContinuous Performance Testing
Continuous Performance TestingC4Media
 
Bristol 2009 q1_blackmore_tim
Bristol 2009 q1_blackmore_timBristol 2009 q1_blackmore_tim
Bristol 2009 q1_blackmore_timObsidian Software
 
Code Management Workshop
Code Management WorkshopCode Management Workshop
Code Management WorkshopSameh El-Ashry
 
Fpga Verification Methodology and case studies - Semisrael Expo2014
Fpga Verification Methodology and case studies - Semisrael Expo2014Fpga Verification Methodology and case studies - Semisrael Expo2014
Fpga Verification Methodology and case studies - Semisrael Expo2014Avi Caspi
 
9 d57105 hardware software co design
9 d57105  hardware software co design9 d57105  hardware software co design
9 d57105 hardware software co designVinod Kumar Gorrepati
 
Performance Testing Java Applications
Performance Testing Java ApplicationsPerformance Testing Java Applications
Performance Testing Java ApplicationsC4Media
 
Reverse engineering
Reverse engineeringReverse engineering
Reverse engineeringSaswat Padhi
 
Ghoshal_resume_LinkedIn_20160705
Ghoshal_resume_LinkedIn_20160705Ghoshal_resume_LinkedIn_20160705
Ghoshal_resume_LinkedIn_20160705Siddhartha Ghoshal
 
Inside Dolphin Test Infrastructure
Inside Dolphin Test InfrastructureInside Dolphin Test Infrastructure
Inside Dolphin Test InfrastructureTao Jiang
 

What's hot (18)

SANER 2015 ERA track: Differential Flame Graphs
SANER 2015 ERA track: Differential Flame GraphsSANER 2015 ERA track: Differential Flame Graphs
SANER 2015 ERA track: Differential Flame Graphs
 
Trends in Mixed Signal Validation
Trends in Mixed Signal ValidationTrends in Mixed Signal Validation
Trends in Mixed Signal Validation
 
Stinson post si and verification
Stinson post si and verificationStinson post si and verification
Stinson post si and verification
 
Chris brown ti
Chris brown tiChris brown ti
Chris brown ti
 
WebLogic Stability; Detect and Analyse Stuck Threads
WebLogic Stability; Detect and Analyse Stuck ThreadsWebLogic Stability; Detect and Analyse Stuck Threads
WebLogic Stability; Detect and Analyse Stuck Threads
 
Verification strategies
Verification strategiesVerification strategies
Verification strategies
 
Continuous Performance Testing
Continuous Performance TestingContinuous Performance Testing
Continuous Performance Testing
 
ASIC design verification
ASIC design verificationASIC design verification
ASIC design verification
 
Bristol 2009 q1_blackmore_tim
Bristol 2009 q1_blackmore_timBristol 2009 q1_blackmore_tim
Bristol 2009 q1_blackmore_tim
 
Java Performance Tuning
Java Performance TuningJava Performance Tuning
Java Performance Tuning
 
Code Management Workshop
Code Management WorkshopCode Management Workshop
Code Management Workshop
 
Fpga Verification Methodology and case studies - Semisrael Expo2014
Fpga Verification Methodology and case studies - Semisrael Expo2014Fpga Verification Methodology and case studies - Semisrael Expo2014
Fpga Verification Methodology and case studies - Semisrael Expo2014
 
9 d57105 hardware software co design
9 d57105  hardware software co design9 d57105  hardware software co design
9 d57105 hardware software co design
 
Performance Testing Java Applications
Performance Testing Java ApplicationsPerformance Testing Java Applications
Performance Testing Java Applications
 
Burst clock controller
Burst clock controllerBurst clock controller
Burst clock controller
 
Reverse engineering
Reverse engineeringReverse engineering
Reverse engineering
 
Ghoshal_resume_LinkedIn_20160705
Ghoshal_resume_LinkedIn_20160705Ghoshal_resume_LinkedIn_20160705
Ghoshal_resume_LinkedIn_20160705
 
Inside Dolphin Test Infrastructure
Inside Dolphin Test InfrastructureInside Dolphin Test Infrastructure
Inside Dolphin Test Infrastructure
 

Viewers also liked

Lear design con_east_2004_simplifing_mixed_signal_simulation
Lear design con_east_2004_simplifing_mixed_signal_simulationLear design con_east_2004_simplifing_mixed_signal_simulation
Lear design con_east_2004_simplifing_mixed_signal_simulationObsidian Software
 
The Verification Methodology Landscape
The Verification Methodology LandscapeThe Verification Methodology Landscape
The Verification Methodology LandscapeDVClub
 

Viewers also liked (8)

Lear design con_east_2004_simplifing_mixed_signal_simulation
Lear design con_east_2004_simplifing_mixed_signal_simulationLear design con_east_2004_simplifing_mixed_signal_simulation
Lear design con_east_2004_simplifing_mixed_signal_simulation
 
Schulz sv q2_2009
Schulz sv q2_2009Schulz sv q2_2009
Schulz sv q2_2009
 
Robert page-abstract
Robert page-abstractRobert page-abstract
Robert page-abstract
 
Mcdermot bio
Mcdermot bioMcdermot bio
Mcdermot bio
 
09 10 parent sign up email
09 10 parent sign up email09 10 parent sign up email
09 10 parent sign up email
 
Thaker q3 2008
Thaker q3 2008Thaker q3 2008
Thaker q3 2008
 
Tobin verification isglobal
Tobin verification isglobalTobin verification isglobal
Tobin verification isglobal
 
The Verification Methodology Landscape
The Verification Methodology LandscapeThe Verification Methodology Landscape
The Verification Methodology Landscape
 

Similar to Validation and-design-in-a-small-team-environment

How to become a testing expert
How to become a testing expertHow to become a testing expert
How to become a testing expertgaoliang641
 
Case Study of End to End Formal Verification Methodology
Case Study of End to End Formal Verification MethodologyCase Study of End to End Formal Verification Methodology
Case Study of End to End Formal Verification MethodologyJacob Ryan Maas
 
Hardware Software Codesign
Hardware Software CodesignHardware Software Codesign
Hardware Software Codesigndestruck
 
Ramprasad-CV_3+yrs
Ramprasad-CV_3+yrsRamprasad-CV_3+yrs
Ramprasad-CV_3+yrsRamprasad B
 
VIKAS _SENIOR HARDWARE
VIKAS _SENIOR HARDWAREVIKAS _SENIOR HARDWARE
VIKAS _SENIOR HARDWAREVIKAS G
 
Intel Atom Processor Pre-Silicon Verification Experience
Intel Atom Processor Pre-Silicon Verification ExperienceIntel Atom Processor Pre-Silicon Verification Experience
Intel Atom Processor Pre-Silicon Verification ExperienceDVClub
 
FPGA Design Challenges
FPGA Design ChallengesFPGA Design Challenges
FPGA Design ChallengesKrishna Gaihre
 
Constraint Programming in Compiler Optimization: Lessons Learned
Constraint Programming in Compiler Optimization: Lessons LearnedConstraint Programming in Compiler Optimization: Lessons Learned
Constraint Programming in Compiler Optimization: Lessons LearnedPeter van Beek
 
TDD and Related Techniques for Non Developers (2012)
TDD and Related Techniques for Non Developers (2012)TDD and Related Techniques for Non Developers (2012)
TDD and Related Techniques for Non Developers (2012)Peter Kofler
 
Mixing d ps building architecture on the cross cutting example
Mixing d ps building architecture on the cross cutting exampleMixing d ps building architecture on the cross cutting example
Mixing d ps building architecture on the cross cutting examplecorehard_by
 
Digital VLSI Design : Introduction
Digital VLSI Design : IntroductionDigital VLSI Design : Introduction
Digital VLSI Design : IntroductionUsha Mehta
 
Syste O CHip Concepts for Students.ppt
Syste O CHip Concepts for Students.pptSyste O CHip Concepts for Students.ppt
Syste O CHip Concepts for Students.pptmonzhalabs
 
10 Reasons You MUST Consider Pattern-Aware Programming
10 Reasons You MUST Consider Pattern-Aware Programming10 Reasons You MUST Consider Pattern-Aware Programming
10 Reasons You MUST Consider Pattern-Aware ProgrammingPostSharp Technologies
 
Improving Batch-Process Testing Techniques with a Domain-Specific Language
Improving Batch-Process Testing Techniques with a Domain-Specific LanguageImproving Batch-Process Testing Techniques with a Domain-Specific Language
Improving Batch-Process Testing Techniques with a Domain-Specific LanguageDr. Spock
 
System On Chip (SOC)
System On Chip (SOC)System On Chip (SOC)
System On Chip (SOC)Shivam Gupta
 

Similar to Validation and-design-in-a-small-team-environment (20)

How to become a testing expert
How to become a testing expertHow to become a testing expert
How to become a testing expert
 
Rashmi_Palakkal_CV
Rashmi_Palakkal_CVRashmi_Palakkal_CV
Rashmi_Palakkal_CV
 
Case Study of End to End Formal Verification Methodology
Case Study of End to End Formal Verification MethodologyCase Study of End to End Formal Verification Methodology
Case Study of End to End Formal Verification Methodology
 
Hardware Software Codesign
Hardware Software CodesignHardware Software Codesign
Hardware Software Codesign
 
Ramprasad-CV_3+yrs
Ramprasad-CV_3+yrsRamprasad-CV_3+yrs
Ramprasad-CV_3+yrs
 
VIKAS _SENIOR HARDWARE
VIKAS _SENIOR HARDWAREVIKAS _SENIOR HARDWARE
VIKAS _SENIOR HARDWARE
 
Intel Atom Processor Pre-Silicon Verification Experience
Intel Atom Processor Pre-Silicon Verification ExperienceIntel Atom Processor Pre-Silicon Verification Experience
Intel Atom Processor Pre-Silicon Verification Experience
 
FPGA Design Challenges
FPGA Design ChallengesFPGA Design Challenges
FPGA Design Challenges
 
Christopher_Reder_2016
Christopher_Reder_2016Christopher_Reder_2016
Christopher_Reder_2016
 
Constraint Programming in Compiler Optimization: Lessons Learned
Constraint Programming in Compiler Optimization: Lessons LearnedConstraint Programming in Compiler Optimization: Lessons Learned
Constraint Programming in Compiler Optimization: Lessons Learned
 
TDD and Related Techniques for Non Developers (2012)
TDD and Related Techniques for Non Developers (2012)TDD and Related Techniques for Non Developers (2012)
TDD and Related Techniques for Non Developers (2012)
 
Mixing d ps building architecture on the cross cutting example
Mixing d ps building architecture on the cross cutting exampleMixing d ps building architecture on the cross cutting example
Mixing d ps building architecture on the cross cutting example
 
Spyglass dft
Spyglass dftSpyglass dft
Spyglass dft
 
Digital VLSI Design : Introduction
Digital VLSI Design : IntroductionDigital VLSI Design : Introduction
Digital VLSI Design : Introduction
 
Unit2 soc
Unit2 socUnit2 soc
Unit2 soc
 
Syste O CHip Concepts for Students.ppt
Syste O CHip Concepts for Students.pptSyste O CHip Concepts for Students.ppt
Syste O CHip Concepts for Students.ppt
 
10 Reasons You MUST Consider Pattern-Aware Programming
10 Reasons You MUST Consider Pattern-Aware Programming10 Reasons You MUST Consider Pattern-Aware Programming
10 Reasons You MUST Consider Pattern-Aware Programming
 
Improving Batch-Process Testing Techniques with a Domain-Specific Language
Improving Batch-Process Testing Techniques with a Domain-Specific LanguageImproving Batch-Process Testing Techniques with a Domain-Specific Language
Improving Batch-Process Testing Techniques with a Domain-Specific Language
 
System On Chip (SOC)
System On Chip (SOC)System On Chip (SOC)
System On Chip (SOC)
 
Agile Testing Days
Agile Testing DaysAgile Testing Days
Agile Testing Days
 

More from Obsidian Software (20)

Yang greenstein part_2
Yang greenstein part_2Yang greenstein part_2
Yang greenstein part_2
 
Yang greenstein part_1
Yang greenstein part_1Yang greenstein part_1
Yang greenstein part_1
 
Williamson arm validation metrics
Williamson arm validation metricsWilliamson arm validation metrics
Williamson arm validation metrics
 
Whipp q3 2008_sv
Whipp q3 2008_svWhipp q3 2008_sv
Whipp q3 2008_sv
 
Vishakantaiah validating
Vishakantaiah validatingVishakantaiah validating
Vishakantaiah validating
 
Tierney bq207
Tierney bq207Tierney bq207
Tierney bq207
 
The validation attitude
The validation attitudeThe validation attitude
The validation attitude
 
Thaker q3 2008
Thaker q3 2008Thaker q3 2008
Thaker q3 2008
 
Shultz dallas q108
Shultz dallas q108Shultz dallas q108
Shultz dallas q108
 
Shreeve dv club_ams
Shreeve dv club_amsShreeve dv club_ams
Shreeve dv club_ams
 
Schulz dallas q1_2008
Schulz dallas q1_2008Schulz dallas q1_2008
Schulz dallas q1_2008
 
Salamian dv club_foils_intel_austin
Salamian dv club_foils_intel_austinSalamian dv club_foils_intel_austin
Salamian dv club_foils_intel_austin
 
Sakar jain
Sakar jainSakar jain
Sakar jain
 
Runner sv q307
Runner sv q307Runner sv q307
Runner sv q307
 
Roy omap validation_dvc_lub_092106
Roy omap validation_dvc_lub_092106Roy omap validation_dvc_lub_092106
Roy omap validation_dvc_lub_092106
 
Roy aerofone power_verif
Roy aerofone power_verifRoy aerofone power_verif
Roy aerofone power_verif
 
Rash
RashRash
Rash
 
Pedneau dv employment_081705
Pedneau dv employment_081705Pedneau dv employment_081705
Pedneau dv employment_081705
 
Oop dv club_rev2b-4
Oop dv club_rev2b-4Oop dv club_rev2b-4
Oop dv club_rev2b-4
 
Oop dv club_rev2b-3
Oop dv club_rev2b-3Oop dv club_rev2b-3
Oop dv club_rev2b-3
 

Validation and-design-in-a-small-team-environment

  • 1. Validation and Design in a Small Team Environment: The Development of the High Performance Embedded Computing Process at Northrop Grumman Matthew Clark, Ph.D February 19th, 2009
  • 2. Motivation Validation is: What if your project is: • Complex • Complex • Long • Lasts 3-6 months, • Requires expensive tools • Has a limited budget, – And new grads. • Requires experienced employees. • Is done in a group with 30 years of SW experience, and you represent the sum total of HW development It is both an Art and Science. experience.
  • 3. First Question What makes verification so expensive and time consuming? Coverage Analysis Functional Verification Issues • Code with Branches Assertions • Multiple Clock Rates • Packet and Clock Rate adaptation. 1 TC per config + Random. • Configurations • SAR Corner Case Analysis • Encapsulation Special SW modeling (e/Vera) Identifying and reducing the green arrows is the Art.
  • 4. Problem Statement How can you reduce the number of green arrows so that designs can be completed quickly, reliably, without having the “normal” 2.5:1 Verification : Designer ratio? A: Do what Capt. Kirk did, change the problem. We can’t change the verification restrictions, perhaps we can change the design issues so that we don’t have some verification problems… by design.
  • 5. Problem Space • Offload CPU-intensive portions of numerical algorithms. • Rapid deployment. Standard project is 3-6 months. • Must be modular: – Single module stand-alone. – Combined into algorithm. – Multiple algorithms in single container. • Exact duplicates of existing SW applications. – Goal is to reduce execution time, or improve performance (ie replace 64K pt FFT with 512K pt FFT). – To SW, there must be no difference between CPU, FPGA, FPGA+DSP, DSP implementation.
  • 6. Design and Verification constraints. • Single Clock Rate for every block. • No Packet and Clock Rate Adaptation – Force it to occur at a single ingress/egress point. • Heavily branched code – Algorithms (fortunately) tend to be datapath centric. • Configurations – Separate algorithm implementation verification from IO verification. – Build bit/cycle accurate model of algorithm, use IO controls to vary data tx/rx.
  • 7. Design constraints reduce verification space Design Constraints • All blocks use small input fifo’s, and programmable full/empty indicators. • All blocks execute at 210 MHz. (Max Register Access speed). • Top Level register interface can be pipelined for timing closure. • Coregen modules only (Ref. Designs Need Not Apply) • Each block uses (1+) async resets, that are externally synch’d. • Each block will have a testbench that: – Uses golden Matlab code to create all stimulus/checker files. • Scenarios are generated from configure files. – Test bench is nothing but interface BFM comparing values against file. – Can slew input/output rates independently. – Can randomly assert the full/empty flags on interface pins. – Interrupt conditions may be checked by hand. – Pass fail conditions checked in VHDL at run time. – Test bench can be run as regression.
  • 8. Design constraints reduce verification space Functional Verification Issues • Code with Branches • Multiple Clock Rates IO_clock and funct_clock only. • Packet and Clock Rate All IO is FIFO based. IO ring handles. adaptation. • Configurations • SAR Not an issue. • Encapsulation Not an issue
  • 9. Results • MegaCore Function (FFT, IFFT, Matched Filter, Correlation, Polyphase/WOLA ), plus Shim for PCI bridge. • Designer Tasks: Architect, Document (MAS), Block Test, Synthesize. – Forward FFT: BSEE (1 mo Verilog) – Inverse FFT : MSEE (6 mo Verilog) – Packet Aggregation: Intern (2 mo VHDL). • Verification: – MSEE (no HDL). • Create Matlab based Testbench, stimulus checker files. • Created Stimulus files for Block Level Test Benches • Reused FIFO BFM’s from previous project. • Code + Block Test: 4 Months • Verification + Synthesis: 2 Months.
  • 10. Summary Q: How can you reduce the number of green arrows so that designs can be completed quickly, reliably, without having the “normal” Verification : Designer ratio? • Use standard, non-optimal, fifo interfaces for every block. • Separate algorithm verification from implementation verification. • Repeatable test cases at the cost of text file storage. • Design each module like it is going into a reuse container: – Standard IF – Standard Clock and reset. – Standard IO and register interfaces. – All data widths are 64 bit + Vld @ 210 MHz. Summary: Trade improved verification schedule and TTM with reduced performance and complexity.