Vchat : Audio based public chat terminal with automated login using FPGA Spartan 6 boards Key Features: Hands-free 2-way Audio communication Automatic login with face detection Simple and user friendly interface

1. Vchat
EE4214 Project Presentation

2. OUTLINE
•
•
•
•
About the Project
Design Process
Demo
Technical Details
• Future Improvements
• Learning Points
• Team and Task
Distribution
• Q&A

3. About the Project
Vchat :
Audio based public chat terminal
with automated login
Key Features
Applications
• Hands-free
• 2-way Audio communication
• Automatic login with face
detection
• Simple and user friendly
interface
•
•
•
•
•
Public Terminals
Elderly care
Hospital Use
Company Network
Virtual Window

4. System Design – Initial Concept
Initial Design Concept
•
•
•
•
•
•
Interface two Spartan 3E
2-way Video
USB Webcams
Keyboard based chatting
Automatic login
Interface with USB webcam
Challenges Faced:
• USB Drivers
• Spartan 3 too weak
•
•
8-bit colour
Slow

5. System Design - Final
Final Design
•
•
•
•
•
•
Interface two Spartan 6
2-way Audio
HDMI-in Camera
Automatic login
High resolution video
Hardware Accelerated
• Data manipulation, Logic
separation

6. Demo

7. Technical Details

8. System Architecture
Audio
In/Out
- Audio Processing
- Communication
AC97 Audio
Codec
Microblaze 0
Face Detected
Ethernet
Lite
Data
In/Out
Audio Received
Video
In/Out
HDMI
VDMA
Microblaze 1
- Face Detection/Tracking
- Graphic User Interface
VMDA = Video Direct Memory Access

9. System Architecture
AXI = Advanced eXtensible Interface

10. HDMI & GUI
Data Corruption:
• Microblaze has to sample
a frame for face detection.
• HDMI Read/Write speed
>> Microblaze.
• HDMI In refreshes frame
at constant rate (60fps),
and should not be
stopped.
Video In/Out (60fps)
HDMI
VDMA
Microblaze 1
(Face Detection)
(5-10fps)
Frame
RAM
VMDA = Video Direct Memory Access

11. HDMI & GUI
Frame Sampling with Data Corruption Prevention
Video In/Out (Fast)
Video In/Out (Fast)
Microblaze 1
(Face Detection)
HDMI
HDMI
(Slow)
Frame
Frame
Frame
Frame
Frame
Frame
Frame
Frame
RAM
RAM
Normal
Sampling

12. HDMI & GUI
bmp
Pixel Extraction
Load at Initialization
HDMI Frame
RAM
128MB
Microblaze 1

13. Face Detection
Original
Skin Color
Erode Noise
Result
Face Detected
Fill Holes
Nguyen, T. T. (2012). Real-time Face Detection and Tracking. Master of Engineering Thesis, School of Electrical and Computer Engineering,
Cornell University.

14. Audio & Ethernet
Spartan-6
Mic
Line Out
LM4550
AC-97
Audio
Codec
Hardware
AC97
Controller
Microblaze 0
Software
Spartan-6
Connector
Marvell
M88E1111
Ethernet
lite
Microblaze 0

15. Ethernet
Ethernet Communication Scheme:
• Send: Implicit naming, non-blocking (broadcast message)
• Receive: Implicit naming, non-blocking (receive message from everyone), but
check for “valid” address and message.
2
1
ACK: Node 1 Online
3
4
ACK: Node 1 Online
Ethernet frame, Destination MAC Address = 0xFF 0xFF 0xFF 0xFF 0xFF 0xFF

16. Audio & Ethernet
N samples
Audio In
44 kHz
Buffer
Full
Tx Data
Ethernet
1 packet = N samples
Microblaze 0
Buffer
Audio Out
𝐻 𝑧 =
Filter
1
(1 + 2𝑧 −1 + 𝑧 −2 )
4
At least
1 empty
Ethernet
Rx Data
…
44 kHz
Buffer
Transmitting
Buffer
Microblaze 0
Receiving

17. Audio & Ethernet
e.g. Buffer N = 2
Transmitting every N Audio Cycles
Deterministic
Time
Audio Cycle
(44kHz)
Transmitting
Nondeterministic
Time
Receiving
Audio
Transmitting
Receiving

18. Audio & Ethernet
• Audio In/Out: Fixed rate at 44kHz.
• It is more efficient to send multiple samples in 1 Ethernet packet.
• Transmitting: deterministic, the processor know when the buffer is full to
broadcast data away → fixed rate and synchronous with audio.
• Receiving: nondeterministic, incoming data can arrive at any time, not
synchronous with audio → multiple buffers. Moreover, data can come from
multiple senders.
Schedule
Audio In
Audio Out
Data Tx
Data Rx
Time
Audio Cycle
Data Arrive (Interrupt)
Time

19. Real Time Requirements
• Audio & Ethernet
• Audio in, Audio out, Ethernet Tx,
and Ethernet Rx scheduling.
• Data buffering.
• Video
• Data corruption prevention.
• DDR2 RAM
• Shared memory access
(hardware).
• Dual Microblaze
• Producer/Consumer.

20. Future Improvements
• Implementation of TCP/IP protocol
for scalable network.
• Face and voice recognition for
automated login with personal
information
• Machine learning software.
• More powerful hardware.
• 2-way video capabilities
• Faster microprocessor (ARM + FPGA).
• HDL implementation of video codec.

21. Learning Outcomes
• Complete system implementation
• Application of concepts
• Data and logic separation for
hardware optimized processes
• Parallel processing
• Dealing with time-critical IO and
real-time deadlines
• Team work is essential

22. Team and Task Distribution
Anh Tuan Nguyen
A0074465Y
System Design & Integration.
Dao Duc Huan
A0030591N
Face Detection Algorithm.
Huynh Van Vinh
A0055979H
Audio to Ethernet Communication.
Rahul Rajeev
A0077497J
Shambavi
Krishnamurthi
A0077496L
Ziyi Wang
A0091881X
Interfacing with peripherals:
• HDMI (PLB Bus).
• Audio Interfacing.
• Keyboard Controller (Spartan3)
User Interface Design:
• Image to Pixel-Array

23. Thank You
Prof. Akash and Chin Hau

24. Q&A