Real time image processing in fpga

7,047 views

Published on

Real time visualization of in vivo processes to predict the biological processes in future is the next challenge in biomedical applications. To cope with such high speed acquisition rates in real time obtained from high resolution microscopes requires a custom solution. In this work we have set a foundation for a dedicated embedded platform for preprocessing of images to calculate the processing time before the images are sent to the computer.
The objective of the designed system is to read high definition real time digital video from an input such as a microscope or a camera and implement image processing algorithms of smoothing and filtering before sending the output. The processor used in the system allows run time control. The system functionality is divided into software part which provides control and integration of processor with the image processing blocks and hardware part (implementation of the system on FPGA and meeting timing requirements). The final aim is to design a system to dynamically change its configuration during run time and reduce the computational time for real time image processing.

Published in: Education

Real time image processing in fpga

  1. 1. Universidad Politecnica de Madrid Master Thesis PresentationReal Time Image Processing in FPGA Using Altera VIP Suite By Sneha Nidhi Supervised byPedro Guerra Gutiérrez, Ph.D. Prof. Andres Santos y Lleo May 2012
  2. 2. Overview of the Presentation Universidad Politecnica de Madrid  Problem Statement  Theoretical Background  Design Models and Implementation  Existing Reference Designs and Problems  Video Design with Cyclone III Board  System Synthesis  Future Work  Conclusion
  3. 3. Universidad Politecnica de MadridProblem Statement Challenge ?? Framework: Real time analysis of biological samples : Anticipate and explain complex biological processes Approach  High resolution 3D microscopes controlled via feedback  Problem faced??  High quality and increasing image size  Extensive computation on acquired data for quick feedback  Limited visibility window for real time response Proposed Solution  Explore Altera VIP Suite for real time pre processing of images
  4. 4. Universidad Politecnica de Madrid Problem Statement Feedback Processor Digital Input Digital OutputMicroscope Video Design PC
  5. 5. Universidad Politecnica de MadridTheoretical Framework Why FPGAs?  Flexibility of rapid prototyping and verification  Multiple hardware units create concurrency  Increased concurrent transfers between the blocks Why Altera Cyclone III FPGA?  Supports high-speed external memory interfaces (DDR2)  Allows integration of x32 bit soft processor cores  Inbuilt IP cores for video design development  Evaluation kit available in the lab
  6. 6. Universidad Politecnica de MadridTheoretical Framework Nios II Soft core Processor  Reduced Instruction Set Computer (RISC) processor core  Allows creation of custom components with 256 custom instructions.  Configured into 3 different core types: economy, standard and fast core. Digital Video Interface DVI  Based on Panel Link, a high speed serial format  Transmits data in a rasterized format  Plug and Play transmission and interconnect solution.
  7. 7. Universidad Politecnica de MadridTheoretical Framework Altera Video and Image Processing Suite  Avalon-ST interface and Avalon Streaming Protocol to communicate  Avalon MM for run time control and connect to external memory Avalon ST Protocol  Packet oriented way to send and receive data  Packets are made of symbols  Video data packet contains color plane values  Control packets consist of packet identifier followed by data
  8. 8. Universidad Politecnica de MadridTheoretical FrameworkStructure of a Video Data PacketStructure of a Control Data Packet
  9. 9. Theoretical Framework Universidad Politecnica de Madrid Power Supply External DDR2 DVI Input Memory Programmer DDR2 Memory Controller Image Input I2C Bus Cyclone III FPGA Image Output LEDsDVI Output Digital Video Daughter Card
  10. 10. Universidad Politecnica de MadridDesign Models and Implementation Existing Reference Designs  Video Design from Altera with NTSC/PAL input Mixing of a real time images with test pattern.  Problems faced: Ref designs do not work. Moreover:  Negative set up and hold timing problems  Lower resolution 1024x768 input image.  Designed with Quartus v9.1 lower DDR2 memory controller clock 75 Mhz.
  11. 11. Universidad Politecnica de MadridDesign Models and Implementation Existing Reference Designs and Problems  Video Design from Bitec UK with NTSC/PAL input
  12. 12. Universidad Politecnica de MadridDesign Models and Implementation Video Design with Altera Cyclone III Video Bridge path External Memory DVI Output DDR2JTAG On chip I2C Open MemoryUART Memory Core Controller Frame Buffer Clock Crossing Instruction Data Bridge Master Master Nios II Processor DVI Input
  13. 13. Universidad Politecnica de MadridDesign Models and Implementation Video Design with Cyclone III Board FPGA interface of the design Video PLL DVI Clock Output Clock Source CPU Reset Nios II Processor External DDR2 Memory Video IP Suite DVI Output DVI Input SOPC System Top
  14. 14. Universidad Politecnica de MadridDesign Models and ImplementationFinal Video Design Cyclone III Development Board Nios II ProcessorPC DVI Input VIP Suite Video Display Design DVI Output
  15. 15. Universidad Politecnica de MadridDesign Models and Implementation Final Video Design with FIR Filtering Color Plane Sequencer DVI Output 2D FIR Filter External MemoryJTAG On chip I2C OpenUART Memory Core Color Plane Sequencer DDR2 Memory Controller Frame Instruction Data Buffer Clock Crossing Master Master Bridge Nios II Processor DVI Input
  16. 16. Universidad Politecnica de MadridDesign Models and Implementation Final Video Design on SOPC Builder
  17. 17. Universidad Politecnica de MadridDesign Models and Implementation Output on the Screen
  18. 18. Universidad Politecnica de MadridSystem Synthesis  Violation of setup or hold time leads to metastable state.  Writing synopsis design constraint files (.sdc) Positive  constrained all clock (ports and pins), Timing Slack  input I/O paths,  output I/O paths
  19. 19. Universidad Politecnica de MadridFuture Work Solve the FIFO underflow problems in CVO output to obtain a stable image. Developing custom components to extend the design
  20. 20. Universidad Politecnica de MadridFuture Work Migrate to new xilinx plaform: Zynq  Hard ARM core + FPGA  HDMI Input/Output FMC Module
  21. 21. Universidad Politecnica de MadridConclusion Designed the embedded platform to process high definition video in real time. Analyzed Altera VIP blocks functionality despite scattered documentation. Calculated video processing time with 3x3 Sobel and Prewitt filtering masks. Interface and control the external memory and other VIP suite components with Nios II processor.
  22. 22. Universidad Politecnica de MadridBibliography Altera Corporation, "Cyclone III Device Handbook, Chapter 6: I/O Features in the Cyclone III Device Family," 2009 December. [Online]. Available: http://www.altera.com/literature/hb/cyc3/cyc3_ciii51007.pdf Altera Corporation, Altera, [Online]. Available: http://www.altera.com/devices/fpga/cyclone3/overview/architecture/cy3architect ure.html. Altera Corporation, "Nios II Software Developers Handbook," May 2011. [Online]. Available: http://www.altera.com/literature/hb/nios2/n2sw_nii52004.pdf. Altera Corporation, "Developing Programs Using Hardware Abstraction Layer," May 2011. [Online]. Available: http://www.altera.com/literature/hb/nios2/n2sw_nii52003.pdf. Altera Corporation, "Nios II Processor Reference Handbook," 2010 July. [Online]. Available: http://www.altera.com/literature/hb/nios2/n2cpu_nii5v1.pdf Digital Display Working Group, "Digital Visual Interface DVI," 02 April 1999. [Online]. Available: http://www.ddwg.org/lib/dvi_10.pdf Altera Corporation, "Video and Image Processing Suite User Guide," July 2010. [Online]. Available: http://www.altera.com/literature/an/an427.pdf Altera Corporation, "Cyclone III 3C120 Development Board Reference Manual,“ http://www.altera.com/literature/manual/rm_cycloneiii_dev_kit_host_board.pdf
  23. 23. Universidad Politecnica de Madrid

×