Vchat
EE4214 Project Presentation
OUTLINE
•
•
•
•

About the Project
Design Process
Demo
Technical Details

• Future Improvements
• Learning Points
• Team a...
About the Project
Vchat :
Audio based public chat terminal
with automated login

Key Features

Applications

• Hands-free
...
System Design – Initial Concept
Initial Design Concept
•
•
•
•
•
•

Interface two Spartan 3E
2-way Video
USB Webcams
Keybo...
System Design - Final
Final Design
•
•
•
•
•
•

Interface two Spartan 6
2-way Audio
HDMI-in Camera
Automatic login
High re...
Demo
Technical Details
System Architecture
Audio
In/Out

- Audio Processing
- Communication
AC97 Audio
Codec

Microblaze 0

Face Detected

Ethern...
System Architecture

AXI = Advanced eXtensible Interface
HDMI & GUI
Data Corruption:
• Microblaze has to sample
a frame for face detection.
• HDMI Read/Write speed
>> Microblaze.
...
HDMI & GUI
Frame Sampling with Data Corruption Prevention

Video In/Out (Fast)

Video In/Out (Fast)

Microblaze 1
(Face De...
HDMI & GUI
bmp
Pixel Extraction
Load at Initialization
HDMI Frame
RAM
128MB

Microblaze 1
Face Detection
Original

Skin Color

Erode Noise

Result
Face Detected

Fill Holes

Nguyen, T. T. (2012). Real-time Face D...
Audio & Ethernet
Spartan-6
Mic

Line Out

LM4550
AC-97
Audio
Codec

Hardware

AC97
Controller

Microblaze 0

Software
Spar...
Ethernet
Ethernet Communication Scheme:
• Send: Implicit naming, non-blocking (broadcast message)
• Receive: Implicit nami...
Audio & Ethernet
N samples

Audio In
44 kHz

Buffer

Full

Tx Data

Ethernet

1 packet = N samples

Microblaze 0

Buffer
A...
Audio & Ethernet
e.g. Buffer N = 2

Transmitting every N Audio Cycles
Deterministic
Time
Audio Cycle
(44kHz)

Transmitting...
Audio & Ethernet
• Audio In/Out: Fixed rate at 44kHz.
• It is more efficient to send multiple samples in 1 Ethernet packet...
Real Time Requirements
• Audio & Ethernet
• Audio in, Audio out, Ethernet Tx,
and Ethernet Rx scheduling.
• Data buffering...
Future Improvements
• Implementation of TCP/IP protocol
for scalable network.
• Face and voice recognition for
automated l...
Learning Outcomes
• Complete system implementation
• Application of concepts

• Data and logic separation for
hardware opt...
Team and Task Distribution
Anh Tuan Nguyen

A0074465Y

System Design & Integration.

Dao Duc Huan

A0030591N

Face Detecti...
Thank You
Prof. Akash and Chin Hau
Q&A
Upcoming SlideShare
Loading in …5
×

VChat | Audio based public chat terminal with automated login

597 views

Published on

Vchat : Audio based public chat terminal with automated login using FPGA Spartan 6 boards

Key Features:
Hands-free
2-way Audio communication
Automatic login with face detection
Simple and user friendly interface

Published in: Technology
0 Comments
0 Likes
Statistics
Notes
  • Be the first to comment

  • Be the first to like this

No Downloads
Views
Total views
597
On SlideShare
0
From Embeds
0
Number of Embeds
63
Actions
Shares
0
Downloads
7
Comments
0
Likes
0
Embeds 0
No embeds

No notes for slide

VChat | Audio based public chat terminal with automated login

  1. 1. Vchat EE4214 Project Presentation
  2. 2. OUTLINE • • • • About the Project Design Process Demo Technical Details • Future Improvements • Learning Points • Team and Task Distribution • Q&A
  3. 3. About the Project Vchat : Audio based public chat terminal with automated login Key Features Applications • Hands-free • 2-way Audio communication • Automatic login with face detection • Simple and user friendly interface • • • • • Public Terminals Elderly care Hospital Use Company Network Virtual Window
  4. 4. System Design – Initial Concept Initial Design Concept • • • • • • Interface two Spartan 3E 2-way Video USB Webcams Keyboard based chatting Automatic login Interface with USB webcam Challenges Faced: • USB Drivers • Spartan 3 too weak • • 8-bit colour Slow
  5. 5. System Design - Final Final Design • • • • • • Interface two Spartan 6 2-way Audio HDMI-in Camera Automatic login High resolution video Hardware Accelerated • Data manipulation, Logic separation
  6. 6. Demo
  7. 7. Technical Details
  8. 8. System Architecture Audio In/Out - Audio Processing - Communication AC97 Audio Codec Microblaze 0 Face Detected Ethernet Lite Data In/Out Audio Received Video In/Out HDMI VDMA Microblaze 1 - Face Detection/Tracking - Graphic User Interface VMDA = Video Direct Memory Access
  9. 9. System Architecture AXI = Advanced eXtensible Interface
  10. 10. HDMI & GUI Data Corruption: • Microblaze has to sample a frame for face detection. • HDMI Read/Write speed >> Microblaze. • HDMI In refreshes frame at constant rate (60fps), and should not be stopped. Video In/Out (60fps) HDMI VDMA Microblaze 1 (Face Detection) (5-10fps) Frame RAM VMDA = Video Direct Memory Access
  11. 11. HDMI & GUI Frame Sampling with Data Corruption Prevention Video In/Out (Fast) Video In/Out (Fast) Microblaze 1 (Face Detection) HDMI HDMI (Slow) Frame Frame Frame Frame Frame Frame Frame Frame RAM RAM Normal Sampling
  12. 12. HDMI & GUI bmp Pixel Extraction Load at Initialization HDMI Frame RAM 128MB Microblaze 1
  13. 13. Face Detection Original Skin Color Erode Noise Result Face Detected Fill Holes Nguyen, T. T. (2012). Real-time Face Detection and Tracking. Master of Engineering Thesis, School of Electrical and Computer Engineering, Cornell University.
  14. 14. Audio & Ethernet Spartan-6 Mic Line Out LM4550 AC-97 Audio Codec Hardware AC97 Controller Microblaze 0 Software Spartan-6 Connector Marvell M88E1111 Ethernet lite Microblaze 0
  15. 15. Ethernet Ethernet Communication Scheme: • Send: Implicit naming, non-blocking (broadcast message) • Receive: Implicit naming, non-blocking (receive message from everyone), but check for “valid” address and message. 2 1 ACK: Node 1 Online 3 4 ACK: Node 1 Online Ethernet frame, Destination MAC Address = 0xFF 0xFF 0xFF 0xFF 0xFF 0xFF
  16. 16. Audio & Ethernet N samples Audio In 44 kHz Buffer Full Tx Data Ethernet 1 packet = N samples Microblaze 0 Buffer Audio Out 𝐻 𝑧 = Filter 1 (1 + 2𝑧 −1 + 𝑧 −2 ) 4 At least 1 empty Ethernet Rx Data … 44 kHz Buffer Transmitting Buffer Microblaze 0 Receiving
  17. 17. Audio & Ethernet e.g. Buffer N = 2 Transmitting every N Audio Cycles Deterministic Time Audio Cycle (44kHz) Transmitting Nondeterministic Time Receiving Audio Transmitting Receiving
  18. 18. Audio & Ethernet • Audio In/Out: Fixed rate at 44kHz. • It is more efficient to send multiple samples in 1 Ethernet packet. • Transmitting: deterministic, the processor know when the buffer is full to broadcast data away → fixed rate and synchronous with audio. • Receiving: nondeterministic, incoming data can arrive at any time, not synchronous with audio → multiple buffers. Moreover, data can come from multiple senders. Schedule Audio In Audio Out Data Tx Data Rx Time Audio Cycle Data Arrive (Interrupt) Time
  19. 19. Real Time Requirements • Audio & Ethernet • Audio in, Audio out, Ethernet Tx, and Ethernet Rx scheduling. • Data buffering. • Video • Data corruption prevention. • DDR2 RAM • Shared memory access (hardware). • Dual Microblaze • Producer/Consumer.
  20. 20. Future Improvements • Implementation of TCP/IP protocol for scalable network. • Face and voice recognition for automated login with personal information • Machine learning software. • More powerful hardware. • 2-way video capabilities • Faster microprocessor (ARM + FPGA). • HDL implementation of video codec.
  21. 21. Learning Outcomes • Complete system implementation • Application of concepts • Data and logic separation for hardware optimized processes • Parallel processing • Dealing with time-critical IO and real-time deadlines • Team work is essential
  22. 22. Team and Task Distribution Anh Tuan Nguyen A0074465Y System Design & Integration. Dao Duc Huan A0030591N Face Detection Algorithm. Huynh Van Vinh A0055979H Audio to Ethernet Communication. Rahul Rajeev A0077497J Shambavi Krishnamurthi A0077496L Ziyi Wang A0091881X Interfacing with peripherals: • HDMI (PLB Bus). • Audio Interfacing. • Keyboard Controller (Spartan3) User Interface Design: • Image to Pixel-Array
  23. 23. Thank You Prof. Akash and Chin Hau
  24. 24. Q&A

×