The document discusses the design of a soft core multiprocessor architecture on an FPGA to implement an adaptive background mixture model algorithm for motion segmentation in images. The goals are to learn FPGA design, leverage parallelism for real-time processing, and use a multiprocessor approach to process different image regions simultaneously. Each processor will perform the algorithm on a sub-region of the image in parallel. They will communicate via shared external memory and FIFO-based links. The proposed architecture includes multiple MicroBlaze processors connected in an array topology to process images from a video camera in real-time.

1. Multiprocessor Architecture for Image processing Mayank Kumar – 2006EE10331 Pushpendre Rastogi – 2006EE50412 Under the guidance of Dr.Anshul Kumar

2. Objectives To learn to work on FPGA platform. Learning the design philosophy of a soft core multiprocessor architecture. Using multiprocessor architecture to implement adaptive background mixture model for motion segmentation. i.e Background modeling and change detection algorithm. (Cris stauffer)

3. Motivation Signal processing, especially Image processing involves repetitive computation which can easily be divided into parallel computations There are many algorithm that follows “locally sequential globally parallel” computation. Surveillance camera related processing need economic real time solutions. FPGA provides an easy way to alter design depending on algorithm requirements – making soft multicore processing feasible. Architecture of the Processing Elements (PE) could be optimized for image processing.

4. Adaptive background mixture model The algorithm helps to generate an adaptive model for the background and helps in motion detection. This can be applied for surveillance. The algorithm is spatially parallel, thus computations for different part of the image can be done simultaneously. This will help in real time operation of this algorithm on embedded platform.

5. Nature of parallelism Data Partitioning: The task is partitioned so that each processor performs exactly the same function, but on different sub-blocks of data. For different image regions, our chosen algorithm is sequential in nature, which can be efficiently implemented through a processor.

6. Basic Idea Camera Video ADC` Virtex II Pro RGB Conversion Power PC M1 M1 M1 M1 M1 M1 M1 M1 M1 M E M O R Y Video DAC MPMC Monitor Array Topology

7. Basic Idea Camera Video ADC` Virtex II Pro RGB Conversion Power PC M1 M1 M1 M1 M1 M1 M1 M1 M1 M E M O R Y Video DAC MPMC Monitor Array Topology

8. Inter-processor Communication [12] For transferring large chunks of image data, we will be using shared external DDR Ram as It provides large memory space for storing multiple frames. Multiple processors can access the RAM simultaneously using MPMC. For sharing intermediate computation results we will use FSL Links between neighboring processors. Unidirectional point-to-point communication. Unshared non-arbitrated communication mechanism. FIFO based communication.

9. Network Topology [1] Completely meshed: Each node is connected to all other nodes. Adv: Reduce inter processor communication time. Disadvantage: Max 9 processors possible. Ring Network : Star Network Array Network: (our choice)

10. Overall Plan Analysis of the algorithm. Configuring Video input and output for XUPV2P FPGA kit. Finalizing the architecture For one processor For two processors For multiprocessors Implementing Basic Test algorithm The algorithm

11. Work Done Studied Background mixture Model for foreground subtraction algorithm [2], [3] , as a case study. Analysis of the algorithm for: Parallelism exploitation Length of code for implementation Memory requirements to store data. Feasibility

12. Work Done Camera Video ADC` Virtex II Pro RGB Conversion Power PC Video DAC Monitor Top Down Approach

13. Work Done Studied Microblaze architecture. [4] Studied FSL Link [5] PLB, LMB, OPB Buses [6] XPS Design Flow [7] Literary survey on related works [8],[9], [10], [11] Configuration Video input and output for XUPV2P FPGA kit. Bottom Up Approach

14. Work Ahead – Step 1 Camera Video ADC` Virtex II Pro RGB Conversion Power PC M E M O R Y Video DAC MPMC Monitor Memory Read And Write 23 rd Feb – 7 th March

15. Work Ahead – Step 2 Camera Video ADC` Virtex II Pro RGB Conversion Power PC M1 M E M O R Y Video DAC MPMC Monitor Some Simple Processing 8 th March – 15 th March

16. Work Ahead – Step 3 Camera Video ADC` Virtex II Pro RGB Conversion Power PC M1 M1 M1 M1 M1 M1 M1 M1 M1 M E M O R Y Video DAC MPMC Monitor Simple processing 24 th March – 5 th April

17. Work Ahead – Step 4 Camera Video ADC` Virtex II Pro RGB Conversion Power PC M1 M1 M1 M1 M1 M1 M1 M1 M1 M E M O R Y Video DAC MPMC Monitor Complex Processing 6 th April – 19 th April

18. Time Line Step 1 – 23 rd Feb – 7 th March Step 2 – 8 th March – 15 th March Step 3 – 24 th March – 5 th April Step 4 – 6 th April – 19 th April

19. Related Works Design Development and performance evaluation of multiprocessor system on FPGA. Somen Barma, CSE IITD. [8]. A Microblaze based Multiprocessor SoC[1] An FPGA based soft multiprocessor system for IPv4 packet forwarding. [9] An automated framework for FPGA based soft Multiprocessor System. [10] Multiprocessor interconnection based on DMA for FPGA.[11]

20. REFERENCES [1] A Microblaze based Multiprocessor SOC – 2003 [2] Adaptive background mixture model for real-time tracking – 1999 [3] Understanding background mixture model for foreground segmentation – 2002 [4] Microblaze processor reference guide [5] Xilinx FSL datasheet [6] Xilinx Microblaze bus interface (ppt) [7] Virtex II Pro design flow – getting started [8] Design Development and performance evaluation of multiprocessor system on FPGA. Somen Barma, CSE IITD. Under Prof Kolin Paul [9] An FPGA based soft multiprocessor system for IPv4 packet forwarding. [10] An automated framework for FPGA based soft Multiprocessor System. [11] Multiprocessor interconnection based on DMA for FPGA. [12] XPS White paper – Designing multiprocessor System on Platform Stdio. Visit www.cse.iiitd.ernet.in/~ee5060412