The document summarizes a master's thesis presentation on real-time image processing using an Altera FPGA. It discusses using the FPGA to process high-resolution microscope images in real-time for feedback control. It presents the problem statement, theoretical background on FPGAs and image processing, and design and implementation of a system using the Altera Cyclone III FPGA board. The design implements a Nios II soft processor, video processing IP cores, and interfaces to DDR memory and DVI input/output. Future work focuses on improving system stability and migrating to the Zynq platform.

1. Universidad Politecnica de Madrid
Master Thesis Presentation
Real Time Image Processing in FPGA
Using Altera VIP Suite
By Sneha Nidhi
Supervised by
Pedro Guerra Gutiérrez, Ph.D.
Prof. Andres Santos y Lleo
May 2012

2. Overview of the Presentation Universidad Politecnica de Madrid
 Problem Statement
 Theoretical Background
 Design Models and Implementation
 Existing Reference Designs and Problems
 Video Design with Cyclone III Board
 System Synthesis
 Future Work
 Conclusion

3. Universidad Politecnica de Madrid
Problem Statement
 Challenge ??
Framework: Real time analysis of biological samples :
Anticipate and explain complex biological processes
 Approach
 High resolution 3D microscopes controlled via feedback
 Problem faced??
 High quality and increasing image size
 Extensive computation on acquired data for quick feedback
 Limited visibility window for real time response
 Proposed Solution
 Explore Altera VIP Suite for real time pre processing of images

4. Universidad Politecnica de Madrid
Problem Statement
Feedback
Processor
Digital Input
Digital
Output
Microscope
Video Design
PC

5. Universidad Politecnica de Madrid
Theoretical Framework
Why FPGAs?
 Flexibility of rapid prototyping and verification
 Multiple hardware units create concurrency
 Increased concurrent transfers between the blocks
Why Altera Cyclone III FPGA?
 Supports high-speed external memory interfaces (DDR2)
 Allows integration of x32 bit soft processor cores
 Inbuilt IP cores for video design development
 Evaluation kit available in the lab

6. Universidad Politecnica de Madrid
Theoretical Framework
 Nios II Soft core Processor
 Reduced Instruction Set Computer (RISC) processor core
 Allows creation of custom components with 256 custom
instructions.
 Configured into 3 different core types: economy, standard and
fast core.
 Digital Video Interface DVI
 Based on Panel Link, a high speed serial format
 Transmits data in a rasterized format
 Plug and Play transmission and interconnect solution.

7. Universidad Politecnica de Madrid
Theoretical Framework
 Altera Video and Image Processing Suite
 Avalon-ST interface and Avalon Streaming Protocol to
communicate
 Avalon MM for run time control and connect to external
memory
 Avalon ST Protocol
 Packet oriented way to send and receive data
 Packets are made of symbols
 Video data packet contains color plane values
 Control packets consist of packet identifier followed by data

8. Universidad Politecnica de Madrid
Theoretical Framework
Structure of a Video Data Packet
Structure of a Control Data Packet

9. Theoretical Framework Universidad Politecnica de Madrid
Power Supply External DDR2
DVI Input Memory Programmer
DDR2 Memory Controller
Image Input
I2C Bus Cyclone III FPGA
Image Output
LEDs
DVI Output Digital Video
Daughter Card

10. Universidad Politecnica de Madrid
Design Models and Implementation
Existing Reference Designs
 Video Design from Altera with NTSC/PAL input
Mixing of a real time images with test pattern.
 Problems faced: Ref designs do not work. Moreover:
 Negative set up and hold timing problems
 Lower resolution 1024x768 input image.
 Designed with Quartus v9.1 lower DDR2 memory
controller clock 75 Mhz.

11. Universidad Politecnica de Madrid
Design Models and Implementation
Existing Reference Designs and Problems
 Video Design from Bitec UK with NTSC/PAL input

12. Universidad Politecnica de Madrid
Design Models and Implementation
Video Design with Altera Cyclone III
Video Bridge path External
Memory
DVI Output
DDR2
JTAG On chip I2C Open Memory
UART Memory Core Controller
Frame
Buffer Clock
Crossing
Instruction Data Bridge
Master Master
Nios II Processor DVI Input

13. Universidad Politecnica de Madrid
Design Models and Implementation
Video Design with Cyclone III Board
FPGA interface of the design
Video PLL DVI Clock
Output
Clock Source
CPU Reset Nios II Processor
External DDR2
Memory
Video IP Suite
DVI Output
DVI Input SOPC System Top

14. Universidad Politecnica de Madrid
Design Models and Implementation
Final Video Design
Cyclone III Development
Board
Nios II Processor
PC
DVI Input VIP Suite Video Display
Design DVI Output

15. Universidad Politecnica de Madrid
Design Models and Implementation
Final Video Design with
FIR Filtering Color Plane Sequencer
DVI Output
2D FIR
Filter External Memory
JTAG On chip I2C Open
UART Memory Core
Color Plane
Sequencer DDR2 Memory
Controller
Frame
Instruction Data Buffer
Clock Crossing
Master Master Bridge
Nios II Processor DVI Input

16. Universidad Politecnica de Madrid
Design Models and Implementation
Final Video Design on SOPC Builder

17. Universidad Politecnica de Madrid
Design Models and Implementation
Output on the Screen

18. Universidad Politecnica de Madrid
System Synthesis
 Violation of setup or hold time leads to metastable state.
 Writing synopsis design constraint files (.sdc) Positive
 constrained all clock (ports and pins), Timing Slack
 input I/O paths,
 output I/O paths

19. Universidad Politecnica de Madrid
Future Work
 Solve the FIFO underflow problems in CVO output to
obtain a stable image.
 Developing custom components to extend the design

20. Universidad Politecnica de Madrid
Future Work
 Migrate to new xilinx plaform: Zynq
 Hard ARM core + FPGA
 HDMI Input/Output FMC Module

21. Universidad Politecnica de Madrid
Conclusion
 Designed the embedded platform to process high
definition video in real time.
 Analyzed Altera VIP blocks functionality despite
scattered documentation.
 Calculated video processing time with 3x3 Sobel and
Prewitt filtering masks.
 Interface and control the external memory and other
VIP suite components with Nios II processor.

22. Universidad Politecnica de Madrid
Bibliography
 Altera Corporation, "Cyclone III Device Handbook, Chapter 6: I/O Features in
the Cyclone III Device Family," 2009 December. [Online]. Available:
http://www.altera.com/literature/hb/cyc3/cyc3_ciii51007.pdf
 Altera Corporation, Altera, [Online]. Available:
http://www.altera.com/devices/fpga/cyclone3/overview/architecture/cy3architect
ure.html.
 Altera Corporation, "Nios II Software Developer's Handbook," May 2011.
[Online]. Available:
http://www.altera.com/literature/hb/nios2/n2sw_nii52004.pdf.
 Altera Corporation, "Developing Programs Using Hardware Abstraction Layer,"
May 2011. [Online]. Available:
http://www.altera.com/literature/hb/nios2/n2sw_nii52003.pdf.
 Altera Corporation, "Nios II Processor Reference Handbook," 2010 July.
[Online]. Available: http://www.altera.com/literature/hb/nios2/n2cpu_nii5v1.pdf
 Digital Display Working Group, "Digital Visual Interface DVI," 02 April 1999.
[Online]. Available: http://www.ddwg.org/lib/dvi_10.pdf
 Altera Corporation, "Video and Image Processing Suite User Guide," July 2010.
[Online]. Available: http://www.altera.com/literature/an/an427.pdf
 Altera Corporation, "Cyclone III 3C120 Development Board Reference Manual,“
http://www.altera.com/literature/manual/rm_cycloneiii_dev_kit_host_board.pdf

23. Universidad Politecnica de Madrid